-- Logs begin at Sat 2022-09-03 20:01:17 UTC, end at Sat 2022-09-03 22:07:51 UTC. -- Sep 03 20:31:37.100352 ncdv-master-1 systemd[1]: Starting Devstack devstack@keystone.service... Sep 03 20:31:37.108558 ncdv-master-1 devstack@keystone.service[99201]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 03 20:31:37.110655 ncdv-master-1 devstack@keystone.service[99201]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 3 20:31:37 2022] *** Sep 03 20:31:37.110655 ncdv-master-1 devstack@keystone.service[99201]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 03 20:31:37.110655 ncdv-master-1 devstack@keystone.service[99201]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 03 20:31:37.110655 ncdv-master-1 devstack@keystone.service[99201]: nodename: ncdv-master-1 Sep 03 20:31:37.110655 ncdv-master-1 devstack@keystone.service[99201]: machine: x86_64 Sep 03 20:31:37.110655 ncdv-master-1 devstack@keystone.service[99201]: clock source: unix Sep 03 20:31:37.110655 ncdv-master-1 devstack@keystone.service[99201]: pcre jit disabled Sep 03 20:31:37.110655 ncdv-master-1 devstack@keystone.service[99201]: detected number of CPU cores: 4 Sep 03 20:31:37.110655 ncdv-master-1 devstack@keystone.service[99201]: current working directory: / Sep 03 20:31:37.110655 ncdv-master-1 devstack@keystone.service[99201]: detected binary path: /usr/bin/uwsgi-core Sep 03 20:31:37.111171 ncdv-master-1 devstack@keystone.service[99201]: your processes number limit is 47680 Sep 03 20:31:37.111171 ncdv-master-1 devstack@keystone.service[99201]: your memory page size is 4096 bytes Sep 03 20:31:37.111171 ncdv-master-1 devstack@keystone.service[99201]: detected max file descriptor number: 2048 Sep 03 20:31:37.111171 ncdv-master-1 devstack@keystone.service[99201]: lock engine: pthread robust mutexes Sep 03 20:31:37.111171 ncdv-master-1 devstack@keystone.service[99201]: thunder lock: enabled Sep 03 20:31:37.111171 ncdv-master-1 devstack@keystone.service[99201]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 03 20:31:37.111171 ncdv-master-1 devstack@keystone.service[99201]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 03 20:31:37.155317 ncdv-master-1 devstack@keystone.service[99201]: Python main interpreter initialized at 0x5567fb18c9c0 Sep 03 20:31:37.155317 ncdv-master-1 devstack@keystone.service[99201]: python threads support enabled Sep 03 20:31:37.155317 ncdv-master-1 devstack@keystone.service[99201]: your server socket listen backlog is limited to 100 connections Sep 03 20:31:37.155317 ncdv-master-1 devstack@keystone.service[99201]: your mercy for graceful operations on workers is 90 seconds Sep 03 20:31:37.155662 ncdv-master-1 devstack@keystone.service[99201]: mapped 403077 bytes (393 KB) for 2 cores Sep 03 20:31:37.155769 ncdv-master-1 devstack@keystone.service[99201]: *** Operational MODE: preforking *** Sep 03 20:31:37.156056 ncdv-master-1 devstack@keystone.service[99201]: *** uWSGI is running in multiple interpreter mode *** Sep 03 20:31:37.156056 ncdv-master-1 devstack@keystone.service[99201]: spawned uWSGI master process (pid: 99201) Sep 03 20:31:37.159511 ncdv-master-1 devstack@keystone.service[99201]: spawned uWSGI worker 1 (pid: 99210, cores: 1) Sep 03 20:31:37.159511 ncdv-master-1 devstack@keystone.service[99201]: spawned uWSGI worker 2 (pid: 99211, cores: 1) Sep 03 20:31:37.159511 ncdv-master-1 devstack@keystone.service[99201]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 03 20:31:37.156256 ncdv-master-1 systemd[1]: Started Devstack devstack@keystone.service. Sep 03 20:31:41.789653 ncdv-master-1 devstack@keystone.service[99211]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 03 20:31:41.789653 ncdv-master-1 devstack@keystone.service[99211]: from cryptography.utils import int_from_bytes Sep 03 20:31:41.790438 ncdv-master-1 devstack@keystone.service[99211]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 03 20:31:41.790438 ncdv-master-1 devstack@keystone.service[99211]: from cryptography.utils import int_from_bytes Sep 03 20:31:41.790731 ncdv-master-1 devstack@keystone.service[99210]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 03 20:31:41.790731 ncdv-master-1 devstack@keystone.service[99210]: from cryptography.utils import int_from_bytes Sep 03 20:31:41.791176 ncdv-master-1 devstack@keystone.service[99210]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 03 20:31:41.791176 ncdv-master-1 devstack@keystone.service[99210]: from cryptography.utils import int_from_bytes Sep 03 20:31:41.840037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 03 20:31:41.840259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 03 20:31:41.840422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] command line args: [] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 03 20:31:41.840689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 03 20:31:41.840865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ================================================================================ {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 03 20:31:41.840977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 03 20:31:41.841176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 03 20:31:41.841265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] admin_token = **** {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.841375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] command line args: [] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 03 20:31:41.841453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] config_dir = [] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.841612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.841699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 03 20:31:41.841781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] config_source = [] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.841856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ================================================================================ {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 03 20:31:41.841969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] debug = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.842053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] admin_token = **** {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.842176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.842399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] config_dir = [] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.842469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.842530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.842650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] config_source = [] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.842752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] insecure_debug = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.842848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] debug = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.842952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.843047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.843181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.843282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.843381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] list_limit = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.843484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] insecure_debug = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.843579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] log_config_append = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.843684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.843779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.843873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] log_dir = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.843969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.844080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] log_file = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.844176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] list_limit = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.844269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.844374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] log_config_append = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.844453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.844567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.844648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] log_dir = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.844720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.844788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] log_file = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.844934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.845024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.845102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.845270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.845394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.845515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.845599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.845759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.845887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.845966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.846079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.846159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.846239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.846353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.846438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.847103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.847232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.847346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.847422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.847582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.847669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] notification_format = cadf {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.847818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.847896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.848059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.848310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.848385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.848537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] publish_errors = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.848652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] notification_format = cadf {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.848764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.848875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.849026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.849180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.849430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.849554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.849664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] publish_errors = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.849783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.849890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.850001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] standard_threads = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.850131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.850257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] strict_password_check = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.850370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.850480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.850591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.850713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] use_eventlog = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.850713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] use_journal = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.850932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.851049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] use_json = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.851166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] standard_threads = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.851285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] use_stderr = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.851285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] use_syslog = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.851548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] strict_password_check = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.851664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] watch_log_file = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.851776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.851776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] use_eventlog = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.851988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.852106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] use_journal = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.852233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.852347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] use_json = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.852347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] use_stderr = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.852554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.852665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] use_syslog = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.852780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.852928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] watch_log_file = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 20:31:41.853054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.853168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.853286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.853405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.853518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.853637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.853750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.853867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.853979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.854090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.854201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.854320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.854439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.854552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.854669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.854787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.854899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.855018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.855132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.855243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.855361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.855475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.855596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.855721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.855840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.855958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.856129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.856244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.856355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.856477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.856590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.856707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.856821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.856957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.857072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.857258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.857374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.857487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.857589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.857589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.857809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.857922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.858043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.858162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.858309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.858423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.858582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.858696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.858808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.858919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.859020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.859121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.859223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.859324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.859419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.859512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.859605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.859699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.859800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.859902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.860019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.860130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.860241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.860343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.860467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.860581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.860683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.860793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.860921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.861035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.861162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.861267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.861378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.861489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.861600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.861712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.861835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.861954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.862072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.862207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.862861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.862979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.863170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.863286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.863399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.863510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.863627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.863741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.863862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.863966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.864084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.864200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.864303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.864414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.864533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.864646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.864756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.864868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.865007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.865007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.865230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.865345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.865456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.865566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.865566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.865780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.865902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.866016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.866127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.866262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.866389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.866515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.866629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.866784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.866900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.867013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.867125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.867248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.867363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.867483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.867597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.867715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.867828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.867948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.868064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.868188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.868310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.868425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.868536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.868649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.868761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.868880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.869061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.869174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.869286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.869398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.869509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.869634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.869749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.869861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.869973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.870120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.870236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.870357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.enabled = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.870471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.870583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.870694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.870818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.870932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.871052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.871167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.enabled = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.871279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.871391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.871503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.871615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.871779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.871894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.872006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.872118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.872231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.872231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.872448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.872563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.872675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.872675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.872890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.873021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.873145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.873267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.873386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.873499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.873611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.873611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.873829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.873944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.874056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.874168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.874281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.874392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.874575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.874691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.874803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.874914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.875038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.875160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.875274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.875396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.875515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.875636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.875750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.875861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.875973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.876092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.876207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.876324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.876440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.876635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.876753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.876874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.877041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.877157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.877270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.877381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.877495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.877615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.877730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.877852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] auth.external = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.877967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.878079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] auth.mapped = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.878224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.878347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.878466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.878590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.878706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.878828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] auth.password = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.878943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] auth.external = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.879056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] auth.token = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.879169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] auth.mapped = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.879290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.879405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.879521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] catalog.caching = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.879634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.879747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.879859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] auth.password = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.879980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.880096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] auth.token = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.880209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.880328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.880443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.880557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] catalog.caching = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.880678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.880794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.880921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] credential.caching = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.881036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.881149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] credential.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.881262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.881387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.881505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.881619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.881733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.881845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.881958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] credential.caching = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.882078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.882193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] credential.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.882307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.882420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.882533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.882650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.882772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.882893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.883008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.883121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.883233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.883346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.883467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.883582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.883696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.883809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.883921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.884034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.884158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.884279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.884393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.884506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.884618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] federation.caching = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.884618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.884839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.884988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] federation.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.885102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.885215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.885328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.885431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.885556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.885671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.885793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] federation.caching = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.885908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.886022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.886134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.886256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] federation.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.886370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.886483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.886596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.886710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.886831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.886947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.887061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.887173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.887286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.caching = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.887399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.887521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.887636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.887749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.887875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.887990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.888103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.888289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.888407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.888519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.888639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.caching = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.888753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.888873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.889026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.889143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.889262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.889377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.889490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.889602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.889723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.889845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.889960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.890074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.890187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.890300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.890421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.890556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.890671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.890784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.890897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.891009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.891131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.891246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.891359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.891472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.891585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.891698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.891819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.891934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.892047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.892161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.892274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.892386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.892508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.892626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.892748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.892748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.893009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.893124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.893251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.893367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.893490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.893606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.893726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.893841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.893963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.894078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.894193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.894305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.894418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.894531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.894653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.894768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.894882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.894995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.895108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.895222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.895344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.895459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.895572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.895685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.password = **** {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.895804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.895919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.896040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.896155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.896274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.896390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.896503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.896617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.896738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.896853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.896983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.897096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.897209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.password = **** {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.897322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.897448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.897564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.897678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.897791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.897893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.898013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.898128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.898241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.898354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.898468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.898580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.898705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.898820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.898933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.899046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.899158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.899272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.899393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.899509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.899627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.899742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.899860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.899974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.900094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.900209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.900327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.900441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.900555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.900668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.900790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.900923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.901038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.901150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.901263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.901375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.901500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.901617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.901730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.901730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.901942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.902054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.902175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.902290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.902402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.902518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.902633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.902745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.902866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.902982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.903095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.903207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.903320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.903433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.903553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.903674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.903795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.903910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.904023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.904135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.904257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.904372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.904484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.904597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.904710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.904822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.904996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.905114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.905226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.905339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] policy.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.905452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.905565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.905686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.905803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.905915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.906028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.906191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.906309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] receipt.caching = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.906432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.906547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.906660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.906774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.906886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.906999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.907120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] policy.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.907235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.907348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.907461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.907573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.907686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] resource.caching = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.907806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.907922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.908035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] receipt.caching = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.908148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] resource.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.908260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.908374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.908498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.908613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.908726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.908839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.908967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.909574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] revoke.caching = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.909706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.909823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.909936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] resource.caching = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.910049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.910162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.910275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] role.cache_time = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.910389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] resource.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.910503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] role.caching = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.910615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.910728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] role.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.910841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.910961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] role.list_limit = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.910961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.911249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.911249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] revoke.caching = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.911462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.911575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.911700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.911815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.911928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.912042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] role.cache_time = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.912155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.912268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] role.caching = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.912392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.912508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] role.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.912621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.912734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] role.list_limit = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.912847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.912990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.913118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.913234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.913348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.913348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.913561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.913673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.913796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.913912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.914025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.914025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.914238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.914351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.914472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.914587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.914700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.914813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.914926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.915038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.915163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.915280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.915393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.915506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.915928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.916044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.916167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.916167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.916384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.916497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.916610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.916723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.916845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.916845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.916845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.917182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.917298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.917424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.917540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.917654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.917767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.917881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.917994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.918114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.918229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.918344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.918457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.918570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.918682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.918804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] token.cache_time = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.918919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.919032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] token.caching = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.919151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.919266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.919381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.919502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] token.provider = fernet {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.919502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.919716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.919716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.919928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.920042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.920164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.920280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] token.cache_time = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.920394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.920508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] token.caching = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.920621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.920734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.920855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.921026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] token.provider = fernet {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.921141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] trust.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.921254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.921367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.921367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.921591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.921709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.921822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.921936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.922049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.922049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.922271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.922271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.922499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.922642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.922758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] trust.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.922758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.922970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.923091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.923206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.connection = **** {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.923320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.923432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.923546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.923659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.923659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.923881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.923994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.924107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.924228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.924351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.924466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.924580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.924702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.925394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.connection = **** {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.925509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.925623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.925737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.925850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.925977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.926094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.926208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.926321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.926434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.926547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.926669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.926786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.926899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.927013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.927126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.927240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.927362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.927477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.927590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.927703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.927817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.927817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 03 20:31:41.928040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.928040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.928040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.928040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.928040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.928040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.928040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.928761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 20:31:41.928761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 03 20:31:41.956208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:41.956415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:41.956638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:41.957139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:41.957381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:41.957614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:41.965262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:41.966338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:41.995610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:41.995837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:41.996584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:41.996823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.048010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.048679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.097934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.098166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.098362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.098606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.165697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.165929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.166155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.166321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.166407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.166563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.166645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.166788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.166879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.167011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.167236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.167459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.190977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.191150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.191238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.191343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.191425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.191573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.191664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.191801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.191883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.192026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.192107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.192247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.192338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.192467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.203488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.203786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.216267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.216504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.216754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.216836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.216998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.217082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.217218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.217300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.217434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.217515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.217649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.217730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.217884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.218108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 20:31:42.234671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.234882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 03 20:31:42.234965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.235094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 03 20:31:42.235225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.235427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 03 20:31:42.235506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.235629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 03 20:31:42.235708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.235913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.235995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.236184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.236264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.236446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.236538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.236708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.246459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.246738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.246819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.247009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.247105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.247293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.247380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.247646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.254888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.255228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.257860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.258145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.258227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.258427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.258521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.258712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.258794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.258996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.259077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.259257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.259340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.259527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.259607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.259803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.259882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.260077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.260159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.260352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.260432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.260621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.260718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.260915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.284982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.285435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.286441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.286723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.286916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.287210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.291266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.291557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.291932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.292027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.292223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.292510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.298413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.298654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.299067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.299361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.303237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.303884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.304072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.304185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.304374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.304578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.304679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.304868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.305011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.305184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.305301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.305459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.305584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.305731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.305871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.306155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.321967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.322243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.322487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.322777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.326118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.326680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.329194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.329599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.329971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.330287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.334124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.334871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.337297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.338122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.340390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.340690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.341006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.341404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.341490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.341702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.341784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.342020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.342121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.342281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.342568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.342855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.357781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.358068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.358338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.358603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.358822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.359118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.359394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.359666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.365949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.367157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.367607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.368705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.369169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.369449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.370291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.370583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.372161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.372453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.372736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.373058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.373272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.373353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.373564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.373852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.374152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.374546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.385138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.385424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.385704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.385982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.386277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.386431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.386558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.386720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.386851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.387008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.387298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.387690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.387980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.388285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.402252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.403837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.405594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.406820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.408531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.409791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.410026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.411259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.411497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.411786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.412074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.412743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.413066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.413472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.419413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.419879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.420881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.421394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.423764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.424057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.424271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.424370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.424600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.426044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.426346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.426647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.433093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.433382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.433663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.434614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.434913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.435205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.441094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=99211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.441378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.441667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.441961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.442220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.442501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.442746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=99210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 20:31:42.442828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.443036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.443116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.443304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.443391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.443599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.443688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.443879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.444006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.444168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.444248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.444441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.444520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.444718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=99211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.444800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.445074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.445353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.445631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.445913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.446196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.446637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=99210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 20:31:42.477862 ncdv-master-1 devstack@keystone.service[99211]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5567fb18c9c0 pid: 99211 (default app) Sep 03 20:31:42.479953 ncdv-master-1 devstack@keystone.service[99210]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5567fb18c9c0 pid: 99210 (default app) Sep 03 20:31:42.481242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b5af0-ef3f-49af-ad34-610e3094dce9 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:42.481531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b5af0-ef3f-49af-ad34-610e3094dce9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:42.481808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b5af0-ef3f-49af-ad34-610e3094dce9 None None] PATH_INFO: `/v3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:42.485936 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Sat Sep 3 20:31:42 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:31:49.505030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62229e0b-32ba-4e31-8ec5-4cf9e8140d08 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:49.505582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62229e0b-32ba-4e31-8ec5-4cf9e8140d08 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:49.505582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62229e0b-32ba-4e31-8ec5-4cf9e8140d08 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:49.509792 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:31:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:31:49.515029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96cfb0-1294-48e8-8b3c-bf6420fe95dc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:49.515551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96cfb0-1294-48e8-8b3c-bf6420fe95dc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:49.515712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96cfb0-1294-48e8-8b3c-bf6420fe95dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:49.593251 ncdv-master-1 devstack@keystone.service[99211]: INFO dbcounter [None req-ea96cfb0-1294-48e8-8b3c-bf6420fe95dc None None] Registered counter for database keystone Sep 03 20:31:49.613203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG oslo_db.sqlalchemy.engines [None req-ea96cfb0-1294-48e8-8b3c-bf6420fe95dc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=99211) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 03 20:31:49.619336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writer thread running {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 03 20:31:49.796997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG passlib.handlers.bcrypt [None req-ea96cfb0-1294-48e8-8b3c-bf6420fe95dc None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=99211) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 03 20:31:49.797337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG passlib.handlers.bcrypt [None req-ea96cfb0-1294-48e8-8b3c-bf6420fe95dc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=99211) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 03 20:31:49.855776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ea96cfb0-1294-48e8-8b3c-bf6420fe95dc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:31:49.857431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ea96cfb0-1294-48e8-8b3c-bf6420fe95dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:31:49.985982 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:31:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 03 20:31:50.495389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62229e0b-32ba-4e31-8ec5-4cf9e8140d08 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:31:50.571739 ncdv-master-1 devstack@keystone.service[99210]: INFO dbcounter [None req-62229e0b-32ba-4e31-8ec5-4cf9e8140d08 None None] Registered counter for database keystone Sep 03 20:31:50.593520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG oslo_db.sqlalchemy.engines [None req-62229e0b-32ba-4e31-8ec5-4cf9e8140d08 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=99210) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 03 20:31:50.600715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writer thread running {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 03 20:31:50.720153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62229e0b-32ba-4e31-8ec5-4cf9e8140d08 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:31:50.725052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1b6ba-cf45-46b1-b71e-38968ce23a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:31:50.725672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1b6ba-cf45-46b1-b71e-38968ce23a96 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:50.725956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1b6ba-cf45-46b1-b71e-38968ce23a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:50.726230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1b6ba-cf45-46b1-b71e-38968ce23a96 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:50.734202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1b6ba-cf45-46b1-b71e-38968ce23a96 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:31:50.785255 ncdv-master-1 devstack@keystone.service[99210]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8ae1b6ba-cf45-46b1-b71e-38968ce23a96 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 03 20:31:50.785635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1b6ba-cf45-46b1-b71e-38968ce23a96 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:31:50.794921 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Sat Sep 3 20:31:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 20:31:50.800873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96cfb0-1294-48e8-8b3c-bf6420fe95dc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:31:50.815702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96cfb0-1294-48e8-8b3c-bf6420fe95dc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:31:50.820199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7287c-c022-48fd-ac69-5eb691f645fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:31:50.821347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7287c-c022-48fd-ac69-5eb691f645fb admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:50.821634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7287c-c022-48fd-ac69-5eb691f645fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:50.821913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7287c-c022-48fd-ac69-5eb691f645fb admin admin] PATH_INFO: `/v3/projects/53641d46504c444d9350b7e021f6a4a4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:50.823392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7287c-c022-48fd-ac69-5eb691f645fb admin admin] RBAC: Authorizing `identity:get_project(project_id=53641d46504c444d9350b7e021f6a4a4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:31:50.873567 ncdv-master-1 devstack@keystone.service[99211]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0fb7287c-c022-48fd-ac69-5eb691f645fb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 03 20:31:50.873908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7287c-c022-48fd-ac69-5eb691f645fb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:31:50.879097 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Sat Sep 3 20:31:50 2022] GET /identity/v3/projects/53641d46504c444d9350b7e021f6a4a4 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:31:53.010817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4e745-2151-4ee2-95f6-ca753b3774b6 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:53.011353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4e745-2151-4ee2-95f6-ca753b3774b6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:53.011353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4e745-2151-4ee2-95f6-ca753b3774b6 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:53.015465 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:31:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:31:53.020704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a3753-7655-4eb7-9b98-0c90a3451c53 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:53.021106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a3753-7655-4eb7-9b98-0c90a3451c53 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:53.021295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a3753-7655-4eb7-9b98-0c90a3451c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:53.040888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e19a3753-7655-4eb7-9b98-0c90a3451c53 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:31:53.042427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e19a3753-7655-4eb7-9b98-0c90a3451c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:31:53.050733 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:31:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 03 20:31:53.373962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4e745-2151-4ee2-95f6-ca753b3774b6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:31:53.386075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4e745-2151-4ee2-95f6-ca753b3774b6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:31:53.390284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fc788-f4fe-4f1a-9ee6-9a53b636e5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:31:53.390891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fc788-f4fe-4f1a-9ee6-9a53b636e5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:53.391179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fc788-f4fe-4f1a-9ee6-9a53b636e5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:53.391521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fc788-f4fe-4f1a-9ee6-9a53b636e5f1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:53.399163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fc788-f4fe-4f1a-9ee6-9a53b636e5f1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:31:53.400998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fc788-f4fe-4f1a-9ee6-9a53b636e5f1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:31:53.410323 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Sat Sep 3 20:31:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:31:53.415521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a3753-7655-4eb7-9b98-0c90a3451c53 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:31:53.424471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a3753-7655-4eb7-9b98-0c90a3451c53 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:31:53.429351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d224b-6ab7-477e-9888-c2d5039167cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:31:53.429971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d224b-6ab7-477e-9888-c2d5039167cd admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:53.430254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d224b-6ab7-477e-9888-c2d5039167cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:53.430531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d224b-6ab7-477e-9888-c2d5039167cd admin admin] PATH_INFO: `/v3/users/efa1ecf6d7f2438dad00007510b3e878` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:53.431835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d224b-6ab7-477e-9888-c2d5039167cd admin admin] RBAC: Authorizing `identity:get_user(user_id=efa1ecf6d7f2438dad00007510b3e878)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:31:53.433681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d224b-6ab7-477e-9888-c2d5039167cd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:31:53.435175 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Sat Sep 3 20:31:53 2022] GET /identity/v3/users/efa1ecf6d7f2438dad00007510b3e878 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:31:55.660527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7488f-7813-4be9-a0d4-d1c8d280165f None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:55.661109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7488f-7813-4be9-a0d4-d1c8d280165f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:55.661109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7488f-7813-4be9-a0d4-d1c8d280165f None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:55.662644 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:31:55.667543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566e1e6-b397-4730-8f9d-e1d5ad270199 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:55.667931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566e1e6-b397-4730-8f9d-e1d5ad270199 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:55.668244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566e1e6-b397-4730-8f9d-e1d5ad270199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:55.687865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1566e1e6-b397-4730-8f9d-e1d5ad270199 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:31:55.689085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1566e1e6-b397-4730-8f9d-e1d5ad270199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:31:55.713094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba05fb-dc61-4ed2-aaeb-35f0ac12194e None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:55.713388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba05fb-dc61-4ed2-aaeb-35f0ac12194e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:55.713672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba05fb-dc61-4ed2-aaeb-35f0ac12194e None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:55.715017 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:31:55.720867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b6cc2-288c-41de-9781-c8a8c96ce7b2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:55.721180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b6cc2-288c-41de-9781-c8a8c96ce7b2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:55.722027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b6cc2-288c-41de-9781-c8a8c96ce7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:55.766375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG passlib.handlers.bcrypt [None req-834b6cc2-288c-41de-9781-c8a8c96ce7b2 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=99210) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 03 20:31:55.766774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG passlib.handlers.bcrypt [None req-834b6cc2-288c-41de-9781-c8a8c96ce7b2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=99210) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 03 20:31:55.817728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-834b6cc2-288c-41de-9781-c8a8c96ce7b2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:31:55.819386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-834b6cc2-288c-41de-9781-c8a8c96ce7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:31:55.879695 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:31:55.918855 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:31:56.236599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566e1e6-b397-4730-8f9d-e1d5ad270199 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:31:56.242828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b6cc2-288c-41de-9781-c8a8c96ce7b2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:31:56.261477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:31:56.342777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566e1e6-b397-4730-8f9d-e1d5ad270199 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:31:56.343091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b6cc2-288c-41de-9781-c8a8c96ce7b2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:31:56.345455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:31:56.376822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a658e16-9633-42f7-b04c-468271bab500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:31:56.377482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a658e16-9633-42f7-b04c-468271bab500 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:56.378256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a658e16-9633-42f7-b04c-468271bab500 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:56.378535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a658e16-9633-42f7-b04c-468271bab500 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:56.381015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dff17-12dd-43cc-8a05-04ba61fc59a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:31:56.381868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dff17-12dd-43cc-8a05-04ba61fc59a8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:56.382159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dff17-12dd-43cc-8a05-04ba61fc59a8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:56.382441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dff17-12dd-43cc-8a05-04ba61fc59a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:56.389677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a658e16-9633-42f7-b04c-468271bab500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-4a658e16-9633-42f7-b04c-468271bab500 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:31:56.399155 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:31:56.400211 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:31:56.401220 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:31:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:31:56.405994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a658e16-9633-42f7-b04c-468271bab500 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:31:56.449886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dff17-12dd-43cc-8a05-04ba61fc59a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:31:56.451425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dff17-12dd-43cc-8a05-04ba61fc59a8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:31:56.488393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:31:56.506690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a658e16-9633-42f7-b04c-468271bab500 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:31:56.541873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e04245-ddc3-43d8-99e6-b96794d1ac47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:31:56.542572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e04245-ddc3-43d8-99e6-b96794d1ac47 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:56.542853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e04245-ddc3-43d8-99e6-b96794d1ac47 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:56.543126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e04245-ddc3-43d8-99e6-b96794d1ac47 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:56.543798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e04245-ddc3-43d8-99e6-b96794d1ac47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:31:56.544103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e04245-ddc3-43d8-99e6-b96794d1ac47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:31:56.545521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e04245-ddc3-43d8-99e6-b96794d1ac47 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:31:56.558443 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:31:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:31:56.564070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e04245-ddc3-43d8-99e6-b96794d1ac47 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:31:56.564197 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:31:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 20:31:56.569562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dff17-12dd-43cc-8a05-04ba61fc59a8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:31:56.572214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:31:56.663219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e04245-ddc3-43d8-99e6-b96794d1ac47 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:31:56.666580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dff17-12dd-43cc-8a05-04ba61fc59a8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:31:56.696043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537237d1-6a03-4cf6-b55f-fc9a82c23757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:31:56.696677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537237d1-6a03-4cf6-b55f-fc9a82c23757 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:56.696987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537237d1-6a03-4cf6-b55f-fc9a82c23757 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:56.697264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537237d1-6a03-4cf6-b55f-fc9a82c23757 None admin] PATH_INFO: `/v3/users/efa1ecf6d7f2438dad00007510b3e878` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:56.698398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537237d1-6a03-4cf6-b55f-fc9a82c23757 None admin] RBAC: Authorizing `identity:get_user(user_id=efa1ecf6d7f2438dad00007510b3e878)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:31:56.698931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06940803-d531-4b15-b674-803dcefbeed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:31:56.699781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06940803-d531-4b15-b674-803dcefbeed5 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:56.699870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537237d1-6a03-4cf6-b55f-fc9a82c23757 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:31:56.700070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06940803-d531-4b15-b674-803dcefbeed5 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:56.700351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06940803-d531-4b15-b674-803dcefbeed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:56.701406 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:31:56 2022] GET /identity/v3/users/efa1ecf6d7f2438dad00007510b3e878 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:31:56.707696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537237d1-6a03-4cf6-b55f-fc9a82c23757 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:31:56.720611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06940803-d531-4b15-b674-803dcefbeed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:31:56.722753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06940803-d531-4b15-b674-803dcefbeed5 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:31:56.723507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:31:56.818771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537237d1-6a03-4cf6-b55f-fc9a82c23757 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-06940803-d531-4b15-b674-803dcefbeed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:31:56.848531 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:31:56.849461 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:31:56.850476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:31:56.850476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:31:56.850476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:31:56.850476 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 9/18] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:31:56.855569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06940803-d531-4b15-b674-803dcefbeed5 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:31:56.856318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d7bd3-9d08-4df6-91b5-4412a2c3b197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:31:56.857456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d7bd3-9d08-4df6-91b5-4412a2c3b197 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:56.857980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d7bd3-9d08-4df6-91b5-4412a2c3b197 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:56.858489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d7bd3-9d08-4df6-91b5-4412a2c3b197 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:56.860465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d7bd3-9d08-4df6-91b5-4412a2c3b197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:31:56.863662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d7bd3-9d08-4df6-91b5-4412a2c3b197 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:31:56.865580 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 10/19] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:31:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:31:56.871163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d7bd3-9d08-4df6-91b5-4412a2c3b197 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:31:56.912510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:31:56.953102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:31:56.962324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06940803-d531-4b15-b674-803dcefbeed5 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:31:56.974363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d7bd3-9d08-4df6-91b5-4412a2c3b197 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:31:56.995936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7ec17-7708-4814-a724-27de57e7514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:31:56.996541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7ec17-7708-4814-a724-27de57e7514d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:56.996828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7ec17-7708-4814-a724-27de57e7514d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:56.997134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7ec17-7708-4814-a724-27de57e7514d None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:56.997777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7ec17-7708-4814-a724-27de57e7514d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:31:56.998087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7ec17-7708-4814-a724-27de57e7514d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:31:56.999469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7ec17-7708-4814-a724-27de57e7514d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:31:57.008171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f7da6-b81e-41b5-9d44-bcd5e68a6e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:31:57.008809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f7da6-b81e-41b5-9d44-bcd5e68a6e31 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:57.009125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f7da6-b81e-41b5-9d44-bcd5e68a6e31 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:57.009410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f7da6-b81e-41b5-9d44-bcd5e68a6e31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:57.010132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f7da6-b81e-41b5-9d44-bcd5e68a6e31 None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.domain.id=default, user.id=efa1ecf6d7f2438dad00007510b3e878) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:31:57.010446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f7da6-b81e-41b5-9d44-bcd5e68a6e31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:31:57.011967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f7da6-b81e-41b5-9d44-bcd5e68a6e31 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:31:57.016230 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 10/20] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:31:57.018726 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 11/21] 10.222.0.16 () {56 vars in 1322 bytes} [Sat Sep 3 20:31:56 2022] GET /identity/v3/role_assignments?user.id=efa1ecf6d7f2438dad00007510b3e878&scope.domain.id=default&role.id=bc768522dc984239be4a1b39a45ab437 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:31:59.639230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6a9b3-6268-4eef-a07a-e9a9ca590416 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:59.639752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6a9b3-6268-4eef-a07a-e9a9ca590416 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:59.640015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6a9b3-6268-4eef-a07a-e9a9ca590416 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:59.641213 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 11/22] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:31:59.647488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5243493-4170-4d39-9005-b365347098c8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:59.647938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5243493-4170-4d39-9005-b365347098c8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:59.648134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5243493-4170-4d39-9005-b365347098c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:59.672304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c5243493-4170-4d39-9005-b365347098c8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:31:59.673809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c5243493-4170-4d39-9005-b365347098c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:31:59.732493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd75f3d-04cb-44c8-b7db-1b3bb1bdeb52 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:59.733060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd75f3d-04cb-44c8-b7db-1b3bb1bdeb52 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:59.733477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd75f3d-04cb-44c8-b7db-1b3bb1bdeb52 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:59.734945 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 12/23] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:31:59.740000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32851875-ecce-46e0-92d2-8b5d2bdb1181 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:31:59.740444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32851875-ecce-46e0-92d2-8b5d2bdb1181 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:31:59.740883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32851875-ecce-46e0-92d2-8b5d2bdb1181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:31:59.765135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-32851875-ecce-46e0-92d2-8b5d2bdb1181 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:31:59.767331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-32851875-ecce-46e0-92d2-8b5d2bdb1181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:31:59.811416 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 12/24] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:31:59.886038 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 13/25] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:00.014757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025c51c-ef86-4f6c-b7b1-33416c3741bb None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:00.015395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025c51c-ef86-4f6c-b7b1-33416c3741bb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:00.015871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025c51c-ef86-4f6c-b7b1-33416c3741bb None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:00.017443 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 13/26] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:32:00.023607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ed9af-14d6-40f8-97a6-eecad5e38858 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:00.023903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ed9af-14d6-40f8-97a6-eecad5e38858 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:00.024185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ed9af-14d6-40f8-97a6-eecad5e38858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:00.047474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7d5ed9af-14d6-40f8-97a6-eecad5e38858 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:00.048694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7d5ed9af-14d6-40f8-97a6-eecad5e38858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:00.103428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:00.138461 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 14/27] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:00.283317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025c51c-ef86-4f6c-b7b1-33416c3741bb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:00.374731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:00.386397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ed9af-14d6-40f8-97a6-eecad5e38858 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:00.414691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025c51c-ef86-4f6c-b7b1-33416c3741bb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:00.466473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfa3d7-20ca-4a3b-99a4-8da879bf4abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:00.467124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfa3d7-20ca-4a3b-99a4-8da879bf4abb None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:00.467411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfa3d7-20ca-4a3b-99a4-8da879bf4abb None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:00.467687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfa3d7-20ca-4a3b-99a4-8da879bf4abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:00.480604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfa3d7-20ca-4a3b-99a4-8da879bf4abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:00.482851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfa3d7-20ca-4a3b-99a4-8da879bf4abb None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-e8bfa3d7-20ca-4a3b-99a4-8da879bf4abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:32:00.515788 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:00.516682 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:32:00.517506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:32:00.517506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:32:00.517506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:00.529877 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 14/28] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:00.531245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7358b-72bd-4e32-aa4d-871114abaa14 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:00.531541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7358b-72bd-4e32-aa4d-871114abaa14 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:00.531815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7358b-72bd-4e32-aa4d-871114abaa14 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:00.533094 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 15/29] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:00.540519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7358b-72bd-4e32-aa4d-871114abaa14 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:00.584644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ed9af-14d6-40f8-97a6-eecad5e38858 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:00.620299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dead9ba-5498-4941-8cf0-85f5b57fc6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:00.620993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dead9ba-5498-4941-8cf0-85f5b57fc6a6 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:00.621287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dead9ba-5498-4941-8cf0-85f5b57fc6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:00.621616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dead9ba-5498-4941-8cf0-85f5b57fc6a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:00.633990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dead9ba-5498-4941-8cf0-85f5b57fc6a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-3dead9ba-5498-4941-8cf0-85f5b57fc6a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:00.640584 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:00.641821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:00.642558 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 15/30] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:32:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:32:00.644392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d624be3-cfd4-4604-ad00-dbe67ce91256 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:00.644392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d624be3-cfd4-4604-ad00-dbe67ce91256 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:00.644392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d624be3-cfd4-4604-ad00-dbe67ce91256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:00.669818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1d624be3-cfd4-4604-ad00-dbe67ce91256 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:00.671167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1d624be3-cfd4-4604-ad00-dbe67ce91256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:00.702994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:00.745284 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 16/31] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:00.752843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d624be3-cfd4-4604-ad00-dbe67ce91256 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:00.755544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7358b-72bd-4e32-aa4d-871114abaa14 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:00.794468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:00.806584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12ee0a-ef1f-491d-b7b9-4ea4cf8febce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:00.807577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12ee0a-ef1f-491d-b7b9-4ea4cf8febce None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:00.808073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12ee0a-ef1f-491d-b7b9-4ea4cf8febce None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:00.808800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12ee0a-ef1f-491d-b7b9-4ea4cf8febce None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:00.809667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12ee0a-ef1f-491d-b7b9-4ea4cf8febce None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:00.810299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12ee0a-ef1f-491d-b7b9-4ea4cf8febce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:00.812150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12ee0a-ef1f-491d-b7b9-4ea4cf8febce None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:00.833724 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 16/32] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:00.835023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12ee0a-ef1f-491d-b7b9-4ea4cf8febce None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:00.943743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12ee0a-ef1f-491d-b7b9-4ea4cf8febce None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:00.951553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d624be3-cfd4-4604-ad00-dbe67ce91256 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:00.982178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476555b4-9fb1-447a-9e67-4538928f4233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:00.983126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476555b4-9fb1-447a-9e67-4538928f4233 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:00.983634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476555b4-9fb1-447a-9e67-4538928f4233 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:00.984086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476555b4-9fb1-447a-9e67-4538928f4233 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:00.984965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476555b4-9fb1-447a-9e67-4538928f4233 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:00.987528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476555b4-9fb1-447a-9e67-4538928f4233 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:00.989077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1694426-d626-4ef6-93cc-6a945f90cbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:00.989800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1694426-d626-4ef6-93cc-6a945f90cbbd None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:00.990182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1694426-d626-4ef6-93cc-6a945f90cbbd None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:00.990553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1694426-d626-4ef6-93cc-6a945f90cbbd None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:00.991320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1694426-d626-4ef6-93cc-6a945f90cbbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:00.991727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1694426-d626-4ef6-93cc-6a945f90cbbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:00.993331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1694426-d626-4ef6-93cc-6a945f90cbbd None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:01.009365 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 17/33] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:01.010409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1694426-d626-4ef6-93cc-6a945f90cbbd None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:01.040224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:01.116869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1694426-d626-4ef6-93cc-6a945f90cbbd None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:01.138935 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 17/34] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Sep 3 20:32:00 2022] POST /identity/v3/roles => generated 225 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:01.142701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7d67c-255b-4b04-9e68-46eeb55ac47c None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:01.143906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7d67c-255b-4b04-9e68-46eeb55ac47c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:01.144501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7d67c-255b-4b04-9e68-46eeb55ac47c None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:01.146544 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 18/35] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:01.147654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7d67c-255b-4b04-9e68-46eeb55ac47c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:01.158640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afa022-f934-4814-930f-8715c940ab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:01.160018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afa022-f934-4814-930f-8715c940ab3f None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:01.160303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afa022-f934-4814-930f-8715c940ab3f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:01.160578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afa022-f934-4814-930f-8715c940ab3f None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:01.161543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afa022-f934-4814-930f-8715c940ab3f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:01.163193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afa022-f934-4814-930f-8715c940ab3f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:01.208302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:01.276774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7d67c-255b-4b04-9e68-46eeb55ac47c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:01.318092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6f64f-51ab-460c-b8b1-62efa290bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:01.318762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6f64f-51ab-460c-b8b1-62efa290bb05 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:01.319055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6f64f-51ab-460c-b8b1-62efa290bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:01.319395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6f64f-51ab-460c-b8b1-62efa290bb05 None admin] PATH_INFO: `/v3/users/efa1ecf6d7f2438dad00007510b3e878` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:01.320908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6f64f-51ab-460c-b8b1-62efa290bb05 None admin] RBAC: Authorizing `identity:get_user(user_id=efa1ecf6d7f2438dad00007510b3e878)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:01.322586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6f64f-51ab-460c-b8b1-62efa290bb05 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:01.324643 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 19/36] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:01 2022] GET /identity/v3/users/efa1ecf6d7f2438dad00007510b3e878 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:01.325473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6f64f-51ab-460c-b8b1-62efa290bb05 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:01.329261 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 18/37] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:32:01 2022] POST /identity/v3/projects => generated 301 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:01.336829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da009c60-fe72-46ca-9806-9067d54d58fa None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:01.337165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da009c60-fe72-46ca-9806-9067d54d58fa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:01.337557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da009c60-fe72-46ca-9806-9067d54d58fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:01.369703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-da009c60-fe72-46ca-9806-9067d54d58fa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:01.371212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-da009c60-fe72-46ca-9806-9067d54d58fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:01.458493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6f64f-51ab-460c-b8b1-62efa290bb05 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:01.466782 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 19/38] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:01.467985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da009c60-fe72-46ca-9806-9067d54d58fa None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:01.481563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:01.496626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:01.523038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c745c91-2173-4f46-a51d-79a6c3225edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:01.523038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c745c91-2173-4f46-a51d-79a6c3225edf None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:01.523038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c745c91-2173-4f46-a51d-79a6c3225edf None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:01.523038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c745c91-2173-4f46-a51d-79a6c3225edf None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:01.523445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c745c91-2173-4f46-a51d-79a6c3225edf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:01.524755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c745c91-2173-4f46-a51d-79a6c3225edf None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:01.574582 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 20/39] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Sep 3 20:32:01 2022] POST /identity/v3/roles => generated 219 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:01.622150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da009c60-fe72-46ca-9806-9067d54d58fa None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:01.633339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea510368-c5ee-4c41-bd79-ce6b41649d2d None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:01.633661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea510368-c5ee-4c41-bd79-ce6b41649d2d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:01.633910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea510368-c5ee-4c41-bd79-ce6b41649d2d None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:01.635568 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 21/40] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:01.641129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8371c-1778-4f37-9595-dbc2597bcb6b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:01.641415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8371c-1778-4f37-9595-dbc2597bcb6b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:01.641703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8371c-1778-4f37-9595-dbc2597bcb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:01.670729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2ad17-cb8e-49a4-a4d2-aa7dc9ac7614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:01.672156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2ad17-cb8e-49a4-a4d2-aa7dc9ac7614 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:01.672666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2ad17-cb8e-49a4-a4d2-aa7dc9ac7614 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:01.673213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2ad17-cb8e-49a4-a4d2-aa7dc9ac7614 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:01.675378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2ad17-cb8e-49a4-a4d2-aa7dc9ac7614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:01.677430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2ad17-cb8e-49a4-a4d2-aa7dc9ac7614 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:01.679467 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 20/41] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:01.682089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-21b8371c-1778-4f37-9595-dbc2597bcb6b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:01.683651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-21b8371c-1778-4f37-9595-dbc2597bcb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:01.685447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2ad17-cb8e-49a4-a4d2-aa7dc9ac7614 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:01.780254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:01.788687 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 22/42] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:01.806433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2ad17-cb8e-49a4-a4d2-aa7dc9ac7614 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:01.843731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1786b2-7c93-4c0b-a284-62c5a76e07e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:01.844731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1786b2-7c93-4c0b-a284-62c5a76e07e7 None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:01.845244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1786b2-7c93-4c0b-a284-62c5a76e07e7 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:01.845797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1786b2-7c93-4c0b-a284-62c5a76e07e7 None admin] PATH_INFO: `/v3/domains/default/users/efa1ecf6d7f2438dad00007510b3e878/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:01.849030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1786b2-7c93-4c0b-a284-62c5a76e07e7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=efa1ecf6d7f2438dad00007510b3e878, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:01.850793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1786b2-7c93-4c0b-a284-62c5a76e07e7 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:01.867268 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 21/43] 10.222.0.16 () {58 vars in 1361 bytes} [Sat Sep 3 20:32:01 2022] PUT /identity/v3/domains/default/users/efa1ecf6d7f2438dad00007510b3e878/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:32:01.919288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8371c-1778-4f37-9595-dbc2597bcb6b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:02.029660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8371c-1778-4f37-9595-dbc2597bcb6b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:02.032168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:02.071614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a2723-e9c9-4eeb-b04a-b0027e781f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:02.072234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a2723-e9c9-4eeb-b04a-b0027e781f35 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:02.072518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a2723-e9c9-4eeb-b04a-b0027e781f35 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:02.072792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a2723-e9c9-4eeb-b04a-b0027e781f35 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:02.073511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a2723-e9c9-4eeb-b04a-b0027e781f35 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:02.074889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a2723-e9c9-4eeb-b04a-b0027e781f35 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:02.089720 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 23/44] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Sep 3 20:32:01 2022] POST /identity/v3/roles => generated 223 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:02.128408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1786b2-7c93-4c0b-a284-62c5a76e07e7 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:02.243008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1786b2-7c93-4c0b-a284-62c5a76e07e7 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:02.278336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d371ff-8343-410c-a4a7-c136fc022fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:02.278970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d371ff-8343-410c-a4a7-c136fc022fc4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:02.279258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d371ff-8343-410c-a4a7-c136fc022fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:02.279540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d371ff-8343-410c-a4a7-c136fc022fc4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:02.280646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d371ff-8343-410c-a4a7-c136fc022fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:02.282150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d371ff-8343-410c-a4a7-c136fc022fc4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:02.284320 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 22/45] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:02.289886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a2723-e9c9-4eeb-b04a-b0027e781f35 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:02.396638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a2723-e9c9-4eeb-b04a-b0027e781f35 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:02.429322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e608b-1560-495a-8dd6-c419e44cae7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:02.429932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e608b-1560-495a-8dd6-c419e44cae7e None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:02.430220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e608b-1560-495a-8dd6-c419e44cae7e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:02.430505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e608b-1560-495a-8dd6-c419e44cae7e None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:02.431193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e608b-1560-495a-8dd6-c419e44cae7e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:02.432651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e608b-1560-495a-8dd6-c419e44cae7e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:02.461974 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 24/46] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:32:02 2022] POST /identity/v3/projects => generated 312 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:04.064275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799cd01-91ca-4da4-88bb-0b01f6595d08 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:04.064793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799cd01-91ca-4da4-88bb-0b01f6595d08 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:04.064933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799cd01-91ca-4da4-88bb-0b01f6595d08 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:04.066378 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 23/47] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:04.071501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd962008-11df-4ba3-85a3-3dcdacef9dee None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:04.071909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd962008-11df-4ba3-85a3-3dcdacef9dee None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:04.072072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd962008-11df-4ba3-85a3-3dcdacef9dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:04.090554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bd962008-11df-4ba3-85a3-3dcdacef9dee None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:04.091767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bd962008-11df-4ba3-85a3-3dcdacef9dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:04.103096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:04.161460 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 25/48] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:04.499566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799cd01-91ca-4da4-88bb-0b01f6595d08 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:04.509797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:04.601794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799cd01-91ca-4da4-88bb-0b01f6595d08 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:04.635116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af68c00-8c73-41d0-8421-90a3fa796a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:04.635758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af68c00-8c73-41d0-8421-90a3fa796a40 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:04.636044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af68c00-8c73-41d0-8421-90a3fa796a40 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:04.636324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af68c00-8c73-41d0-8421-90a3fa796a40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:04.648072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af68c00-8c73-41d0-8421-90a3fa796a40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:04.652486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6d220-2a8f-4daf-b4af-98d4c2406867 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:04.652779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6d220-2a8f-4daf-b4af-98d4c2406867 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-3af68c00-8c73-41d0-8421-90a3fa796a40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:04.652869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:04.653863 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:04.654927 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:04.654927 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:04.654927 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:04.654927 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:04.654927 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:04.654927 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:04.654927 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:04.654927 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:04.654927 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:04.654927 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:04.654927 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:04.654927 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:04.655784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6d220-2a8f-4daf-b4af-98d4c2406867 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:04.655863 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 24/49] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:32:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:04.655937 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 26/50] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:32:04.659563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6d220-2a8f-4daf-b4af-98d4c2406867 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:04.660110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f84a40-05a0-4c25-ae4b-c0abfcdabbbc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:04.660426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f84a40-05a0-4c25-ae4b-c0abfcdabbbc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:04.660805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f84a40-05a0-4c25-ae4b-c0abfcdabbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:04.682833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d5f84a40-05a0-4c25-ae4b-c0abfcdabbbc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:04.684041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d5f84a40-05a0-4c25-ae4b-c0abfcdabbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:04.742876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:04.751152 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 25/51] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:04.765560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6d220-2a8f-4daf-b4af-98d4c2406867 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:04.800114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3f1af-91ae-4ed8-a633-0d465948f05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:04.800728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3f1af-91ae-4ed8-a633-0d465948f05c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:04.801054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3f1af-91ae-4ed8-a633-0d465948f05c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:04.801342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3f1af-91ae-4ed8-a633-0d465948f05c None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:04.802009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3f1af-91ae-4ed8-a633-0d465948f05c None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:04.802317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3f1af-91ae-4ed8-a633-0d465948f05c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:04.803714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3f1af-91ae-4ed8-a633-0d465948f05c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:04.811661 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 27/52] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:04.817890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f84a40-05a0-4c25-ae4b-c0abfcdabbbc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:04.915821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f84a40-05a0-4c25-ae4b-c0abfcdabbbc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:04.948532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63731c74-8273-462e-b8e9-7ce0b3f6b952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:04.949202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63731c74-8273-462e-b8e9-7ce0b3f6b952 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:04.949494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63731c74-8273-462e-b8e9-7ce0b3f6b952 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:04.949784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63731c74-8273-462e-b8e9-7ce0b3f6b952 None admin] PATH_INFO: `/v3/users/efa1ecf6d7f2438dad00007510b3e878` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:04.950930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63731c74-8273-462e-b8e9-7ce0b3f6b952 None admin] RBAC: Authorizing `identity:get_user(user_id=efa1ecf6d7f2438dad00007510b3e878)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:04.952382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63731c74-8273-462e-b8e9-7ce0b3f6b952 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:04.954009 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 26/53] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:04 2022] GET /identity/v3/users/efa1ecf6d7f2438dad00007510b3e878 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:04.960119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3f1af-91ae-4ed8-a633-0d465948f05c None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:04.971778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:05.058969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3f1af-91ae-4ed8-a633-0d465948f05c None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:05.074262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63731c74-8273-462e-b8e9-7ce0b3f6b952 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:05.093212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52273ca7-8d2a-4495-88d9-be1db2af8b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:05.093855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52273ca7-8d2a-4495-88d9-be1db2af8b62 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:05.094138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52273ca7-8d2a-4495-88d9-be1db2af8b62 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:05.094414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52273ca7-8d2a-4495-88d9-be1db2af8b62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:05.095917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52273ca7-8d2a-4495-88d9-be1db2af8b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:05.097469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52273ca7-8d2a-4495-88d9-be1db2af8b62 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:05.099021 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 28/54] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:05.104567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52273ca7-8d2a-4495-88d9-be1db2af8b62 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:05.138413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:05.176012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63731c74-8273-462e-b8e9-7ce0b3f6b952 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:05.182253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:05.205553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52273ca7-8d2a-4495-88d9-be1db2af8b62 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:05.210002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5bc04-b737-4eac-b02e-afa3014a5f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:05.210635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5bc04-b737-4eac-b02e-afa3014a5f50 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:05.210921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5bc04-b737-4eac-b02e-afa3014a5f50 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:05.211201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5bc04-b737-4eac-b02e-afa3014a5f50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:05.212307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5bc04-b737-4eac-b02e-afa3014a5f50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:05.213787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5bc04-b737-4eac-b02e-afa3014a5f50 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:05.215339 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 27/55] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:05.220800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5bc04-b737-4eac-b02e-afa3014a5f50 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:05.238779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0e034-d06b-4f56-8f49-fe1535186bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:05.239398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0e034-d06b-4f56-8f49-fe1535186bbc None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:05.239686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0e034-d06b-4f56-8f49-fe1535186bbc None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:05.239965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0e034-d06b-4f56-8f49-fe1535186bbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:05.240692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0e034-d06b-4f56-8f49-fe1535186bbc None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.domain.id=default, user.id=efa1ecf6d7f2438dad00007510b3e878) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:05.241037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0e034-d06b-4f56-8f49-fe1535186bbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:05.242492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0e034-d06b-4f56-8f49-fe1535186bbc None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:05.248103 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 29/56] 10.222.0.16 () {56 vars in 1322 bytes} [Sat Sep 3 20:32:05 2022] GET /identity/v3/role_assignments?user.id=efa1ecf6d7f2438dad00007510b3e878&scope.domain.id=default&role.id=bc768522dc984239be4a1b39a45ab437 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:05.327148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5bc04-b737-4eac-b02e-afa3014a5f50 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:05.357747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:05.362991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47998e92-40d4-4a87-a2cd-a1291f6a1022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:05.363812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47998e92-40d4-4a87-a2cd-a1291f6a1022 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:05.364273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47998e92-40d4-4a87-a2cd-a1291f6a1022 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:05.364762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47998e92-40d4-4a87-a2cd-a1291f6a1022 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:05.365460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47998e92-40d4-4a87-a2cd-a1291f6a1022 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:05.366953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47998e92-40d4-4a87-a2cd-a1291f6a1022 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:05.396613 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 28/57] 10.222.0.16 () {60 vars in 1140 bytes} [Sat Sep 3 20:32:05 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:32:07.558984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e6a24-e9a6-4516-8e1b-a0923e8d5134 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:07.559505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e6a24-e9a6-4516-8e1b-a0923e8d5134 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:07.559608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e6a24-e9a6-4516-8e1b-a0923e8d5134 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:07.560888 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 30/58] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:07.566597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8a235-bbc5-4383-86ea-cd140ca7c958 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:07.566989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8a235-bbc5-4383-86ea-cd140ca7c958 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:07.567171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8a235-bbc5-4383-86ea-cd140ca7c958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:07.585599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-75c8a235-bbc5-4383-86ea-cd140ca7c958 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:07.586807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-75c8a235-bbc5-4383-86ea-cd140ca7c958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:07.650898 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 29/59] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:07.980280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e6a24-e9a6-4516-8e1b-a0923e8d5134 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:08.081210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e6a24-e9a6-4516-8e1b-a0923e8d5134 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:08.114271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0241a2-c28e-4b4c-97f0-e255899ed1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:08.114898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0241a2-c28e-4b4c-97f0-e255899ed1cc None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:08.115188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0241a2-c28e-4b4c-97f0-e255899ed1cc None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:08.115470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0241a2-c28e-4b4c-97f0-e255899ed1cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:08.116592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0241a2-c28e-4b4c-97f0-e255899ed1cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:08.118128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0241a2-c28e-4b4c-97f0-e255899ed1cc None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:08.119856 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 31/60] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:08.125103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8a235-bbc5-4383-86ea-cd140ca7c958 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:08.220915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8a235-bbc5-4383-86ea-cd140ca7c958 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:08.235188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:08.255508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5ad3c-5e32-4f3b-9351-afe66252c4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:08.256114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5ad3c-5e32-4f3b-9351-afe66252c4e3 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:08.256403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5ad3c-5e32-4f3b-9351-afe66252c4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:08.256686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5ad3c-5e32-4f3b-9351-afe66252c4e3 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:08.257375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5ad3c-5e32-4f3b-9351-afe66252c4e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:08.258852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5ad3c-5e32-4f3b-9351-afe66252c4e3 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:08.285976 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 30/61] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:32:08 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:11.479246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba92077-be49-4ae8-b0e5-f0a4b6fe3d8e None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:11.480259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba92077-be49-4ae8-b0e5-f0a4b6fe3d8e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:11.480791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba92077-be49-4ae8-b0e5-f0a4b6fe3d8e None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:11.482371 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 32/62] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:11.487652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad11b42-0cd1-4855-98ac-b6c9d7932e99 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:11.488454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad11b42-0cd1-4855-98ac-b6c9d7932e99 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:11.488972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad11b42-0cd1-4855-98ac-b6c9d7932e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:11.518422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-aad11b42-0cd1-4855-98ac-b6c9d7932e99 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:11.519971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-aad11b42-0cd1-4855-98ac-b6c9d7932e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:11.603567 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 31/63] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:11.795309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d8317-050f-4bb2-b3b7-4029aa3ac630 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:11.795960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d8317-050f-4bb2-b3b7-4029aa3ac630 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:11.796471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d8317-050f-4bb2-b3b7-4029aa3ac630 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:11.798079 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 33/64] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:11.803768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202535ce-af13-4f55-b2bf-f601326554af None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:11.804307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202535ce-af13-4f55-b2bf-f601326554af None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:11.804792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202535ce-af13-4f55-b2bf-f601326554af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:11.835848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-202535ce-af13-4f55-b2bf-f601326554af None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:11.837448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-202535ce-af13-4f55-b2bf-f601326554af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:11.858402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0355a-2bd7-48f2-9d10-150dddd7ac74 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:11.859008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0355a-2bd7-48f2-9d10-150dddd7ac74 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:11.859489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0355a-2bd7-48f2-9d10-150dddd7ac74 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:11.861040 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 34/65] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:11.874148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6cd69-d9c4-4dcc-bba1-3183a875bdbd None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:11.874708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6cd69-d9c4-4dcc-bba1-3183a875bdbd None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:11.875194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6cd69-d9c4-4dcc-bba1-3183a875bdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:11.902714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7cc6cd69-d9c4-4dcc-bba1-3183a875bdbd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:11.904949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7cc6cd69-d9c4-4dcc-bba1-3183a875bdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:11.909325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:11.931893 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 32/66] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:12.006552 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 35/67] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:12.037094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202535ce-af13-4f55-b2bf-f601326554af None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:12.082079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:12.194592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202535ce-af13-4f55-b2bf-f601326554af None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:12.242736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679b43f-47c5-4e1d-8744-83569d56856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:12.243386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679b43f-47c5-4e1d-8744-83569d56856e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:12.243679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679b43f-47c5-4e1d-8744-83569d56856e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:12.243964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679b43f-47c5-4e1d-8744-83569d56856e None admin] PATH_INFO: `/v3/roles/member` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:12.263657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679b43f-47c5-4e1d-8744-83569d56856e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-f679b43f-47c5-4e1d-8744-83569d56856e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:12.269131 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:12.269943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:12.270670 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 33/68] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:12.276055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6cd69-d9c4-4dcc-bba1-3183a875bdbd None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:12.294438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679b43f-47c5-4e1d-8744-83569d56856e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:12.409697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:12.412379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6cd69-d9c4-4dcc-bba1-3183a875bdbd None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:12.420835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679b43f-47c5-4e1d-8744-83569d56856e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:12.450343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f99e7-9e24-4528-a806-527d533e70fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:12.450965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f99e7-9e24-4528-a806-527d533e70fd None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:12.451251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f99e7-9e24-4528-a806-527d533e70fd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:12.451535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f99e7-9e24-4528-a806-527d533e70fd None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:12.452207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f99e7-9e24-4528-a806-527d533e70fd None admin] RBAC: Adding query filter params (name=member) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:12.452537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f99e7-9e24-4528-a806-527d533e70fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:12.453943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f99e7-9e24-4528-a806-527d533e70fd None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:12.460563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:12.465749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568ac20-2d8e-4ee7-ba03-614e39e8cd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:12.466450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568ac20-2d8e-4ee7-ba03-614e39e8cd25 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:12.466747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568ac20-2d8e-4ee7-ba03-614e39e8cd25 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:12.467027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568ac20-2d8e-4ee7-ba03-614e39e8cd25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:12.468711 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 36/69] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:12.469930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74756-ae4f-4ccc-b555-1ab02b09d6ce None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:12.470488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74756-ae4f-4ccc-b555-1ab02b09d6ce None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:12.470488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74756-ae4f-4ccc-b555-1ab02b09d6ce None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:12.472369 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 37/70] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:12.476888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a74756-ae4f-4ccc-b555-1ab02b09d6ce None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:12.479325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568ac20-2d8e-4ee7-ba03-614e39e8cd25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-d568ac20-2d8e-4ee7-ba03-614e39e8cd25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:12.485194 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:12.486059 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 34/71] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:32:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:32:12.488227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568ac20-2d8e-4ee7-ba03-614e39e8cd25 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:12.587559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a74756-ae4f-4ccc-b555-1ab02b09d6ce None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:12.591830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568ac20-2d8e-4ee7-ba03-614e39e8cd25 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:12.620981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74249ddf-c2b4-4010-88a6-fb09d5a517c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:12.621628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74249ddf-c2b4-4010-88a6-fb09d5a517c5 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:12.621919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74249ddf-c2b4-4010-88a6-fb09d5a517c5 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:12.622207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74249ddf-c2b4-4010-88a6-fb09d5a517c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:12.626531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe995297-b70f-419b-91cc-0d7fa8f162dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:12.627167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe995297-b70f-419b-91cc-0d7fa8f162dc None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:12.627987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe995297-b70f-419b-91cc-0d7fa8f162dc None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:12.628268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe995297-b70f-419b-91cc-0d7fa8f162dc None admin] PATH_INFO: `/v3/users/c2d9267d86294911a8b40405544b5a2f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:12.632771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74249ddf-c2b4-4010-88a6-fb09d5a517c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-74249ddf-c2b4-4010-88a6-fb09d5a517c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:12.637633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:12.638451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:12.639221 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 38/72] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:32:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:12.641006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9f5e2-50a2-4418-9394-81803e97cfcc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:12.641302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9f5e2-50a2-4418-9394-81803e97cfcc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:12.641589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9f5e2-50a2-4418-9394-81803e97cfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:12.645410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe995297-b70f-419b-91cc-0d7fa8f162dc None admin] RBAC: Authorizing `identity:get_user(user_id=c2d9267d86294911a8b40405544b5a2f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:12.646917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe995297-b70f-419b-91cc-0d7fa8f162dc None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:12.648711 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 35/73] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:12 2022] GET /identity/v3/users/c2d9267d86294911a8b40405544b5a2f => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:12.649367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe995297-b70f-419b-91cc-0d7fa8f162dc None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:12.659430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2ec9f5e2-50a2-4418-9394-81803e97cfcc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:12.660376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:12.660646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2ec9f5e2-50a2-4418-9394-81803e97cfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:12.717756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:12.729302 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 39/74] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:12.730436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e137bc5-56e3-443f-8a8e-3616e627fd90 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:12.730731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e137bc5-56e3-443f-8a8e-3616e627fd90 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:12.731016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e137bc5-56e3-443f-8a8e-3616e627fd90 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:12.732248 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 40/75] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:12.736295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e137bc5-56e3-443f-8a8e-3616e627fd90 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:12.759717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe995297-b70f-419b-91cc-0d7fa8f162dc None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:12.798118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb133c-4b7f-47dc-b673-ab644c0a7562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:12.798737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb133c-4b7f-47dc-b673-ab644c0a7562 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:12.799026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb133c-4b7f-47dc-b673-ab644c0a7562 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:12.799330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb133c-4b7f-47dc-b673-ab644c0a7562 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:12.800002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb133c-4b7f-47dc-b673-ab644c0a7562 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:12.800312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb133c-4b7f-47dc-b673-ab644c0a7562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:12.801739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb133c-4b7f-47dc-b673-ab644c0a7562 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:12.809952 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 36/76] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:32:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:12.810734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb133c-4b7f-47dc-b673-ab644c0a7562 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:12.852798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e137bc5-56e3-443f-8a8e-3616e627fd90 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:12.882253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:12.892125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d80e4a-1d00-401a-a725-554298c56fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:12.892726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d80e4a-1d00-401a-a725-554298c56fbe None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:12.893631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d80e4a-1d00-401a-a725-554298c56fbe None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:12.893918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d80e4a-1d00-401a-a725-554298c56fbe None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:12.894592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d80e4a-1d00-401a-a725-554298c56fbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:12.894900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d80e4a-1d00-401a-a725-554298c56fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:12.896725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d80e4a-1d00-401a-a725-554298c56fbe None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:12.905627 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 41/77] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:12.906723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8abdf-2116-4b62-b3de-ff4238fc8dc2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:12.907020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8abdf-2116-4b62-b3de-ff4238fc8dc2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:12.907300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8abdf-2116-4b62-b3de-ff4238fc8dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:12.919071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb133c-4b7f-47dc-b673-ab644c0a7562 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:12.926004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f8f8abdf-2116-4b62-b3de-ff4238fc8dc2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:12.927189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f8f8abdf-2116-4b62-b3de-ff4238fc8dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:12.956733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cd583-8338-4cf6-90cc-3e9e1f288ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:12.957373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cd583-8338-4cf6-90cc-3e9e1f288ad3 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:12.957664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cd583-8338-4cf6-90cc-3e9e1f288ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:12.957944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cd583-8338-4cf6-90cc-3e9e1f288ad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:12.973723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:13.001251 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 42/78] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:13.001251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8abdf-2116-4b62-b3de-ff4238fc8dc2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:13.051268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cd583-8338-4cf6-90cc-3e9e1f288ad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:13.052767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cd583-8338-4cf6-90cc-3e9e1f288ad3 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:13.115725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8abdf-2116-4b62-b3de-ff4238fc8dc2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:13.125955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:13.158137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ac1d5-e244-4970-993a-8378da3751ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:13.159390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ac1d5-e244-4970-993a-8378da3751ca None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:13.159703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ac1d5-e244-4970-993a-8378da3751ca None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:13.159980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ac1d5-e244-4970-993a-8378da3751ca None admin] PATH_INFO: `/v3/users/c2d9267d86294911a8b40405544b5a2f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:13.161169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ac1d5-e244-4970-993a-8378da3751ca None admin] RBAC: Authorizing `identity:get_user(user_id=c2d9267d86294911a8b40405544b5a2f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:13.161707 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 37/79] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:13.162373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cd583-8338-4cf6-90cc-3e9e1f288ad3 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:13.167301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ac1d5-e244-4970-993a-8378da3751ca None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:13.169182 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 43/80] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:12 2022] GET /identity/v3/users/c2d9267d86294911a8b40405544b5a2f => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:13.169759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ac1d5-e244-4970-993a-8378da3751ca None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:13.228382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:13.268905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cd583-8338-4cf6-90cc-3e9e1f288ad3 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:13.284511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ac1d5-e244-4970-993a-8378da3751ca None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:13.304424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612ff32-7d75-4ee5-91e2-55c6372c8a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:13.305091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612ff32-7d75-4ee5-91e2-55c6372c8a86 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:13.305379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612ff32-7d75-4ee5-91e2-55c6372c8a86 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:13.306001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612ff32-7d75-4ee5-91e2-55c6372c8a86 None admin] PATH_INFO: `/v3/users/efa1ecf6d7f2438dad00007510b3e878` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:13.306788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612ff32-7d75-4ee5-91e2-55c6372c8a86 None admin] RBAC: Authorizing `identity:get_user(user_id=efa1ecf6d7f2438dad00007510b3e878)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:13.308241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612ff32-7d75-4ee5-91e2-55c6372c8a86 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:13.310295 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 38/81] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:13 2022] GET /identity/v3/users/efa1ecf6d7f2438dad00007510b3e878 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:13.314040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612ff32-7d75-4ee5-91e2-55c6372c8a86 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:13.320906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316f9da-29fb-4c4f-9517-acaefc98337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:13.321553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316f9da-29fb-4c4f-9517-acaefc98337f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:13.321839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316f9da-29fb-4c4f-9517-acaefc98337f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:13.322115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316f9da-29fb-4c4f-9517-acaefc98337f None admin] PATH_INFO: `/v3/roles/member` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:13.332657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316f9da-29fb-4c4f-9517-acaefc98337f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-2316f9da-29fb-4c4f-9517-acaefc98337f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:13.337908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:13.338773 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:13.340337 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 44/82] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:13.341798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316f9da-29fb-4c4f-9517-acaefc98337f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:13.358275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:13.417750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612ff32-7d75-4ee5-91e2-55c6372c8a86 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:13.441409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316f9da-29fb-4c4f-9517-acaefc98337f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:13.451548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:13.451811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37696f-a012-4460-8764-b87d06766c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:13.452475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37696f-a012-4460-8764-b87d06766c90 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:13.452835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37696f-a012-4460-8764-b87d06766c90 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:13.453201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37696f-a012-4460-8764-b87d06766c90 None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:13.454445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37696f-a012-4460-8764-b87d06766c90 None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:13.456419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37696f-a012-4460-8764-b87d06766c90 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:13.458082 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 39/83] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:13 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:13.458848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37696f-a012-4460-8764-b87d06766c90 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:13.481514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c36d46-0b2d-40c7-b8f4-27143a7ee804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:13.482127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c36d46-0b2d-40c7-b8f4-27143a7ee804 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:13.482441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c36d46-0b2d-40c7-b8f4-27143a7ee804 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:13.482730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c36d46-0b2d-40c7-b8f4-27143a7ee804 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:13.548327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c36d46-0b2d-40c7-b8f4-27143a7ee804 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:13.550418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c36d46-0b2d-40c7-b8f4-27143a7ee804 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:13.555713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37696f-a012-4460-8764-b87d06766c90 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:13.562726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:13.589730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11431f0a-6282-4ff7-aafe-2f7c3a71667d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:13.590346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11431f0a-6282-4ff7-aafe-2f7c3a71667d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:13.590640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11431f0a-6282-4ff7-aafe-2f7c3a71667d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:13.590926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11431f0a-6282-4ff7-aafe-2f7c3a71667d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:13.647300 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 45/84] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:13.648136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c36d46-0b2d-40c7-b8f4-27143a7ee804 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:13.659367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11431f0a-6282-4ff7-aafe-2f7c3a71667d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:13.659821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:13.661326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11431f0a-6282-4ff7-aafe-2f7c3a71667d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:13.747081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c36d46-0b2d-40c7-b8f4-27143a7ee804 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:13.758394 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 40/85] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:13.760446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11431f0a-6282-4ff7-aafe-2f7c3a71667d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:13.768794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:13.781067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21c16c-370a-4789-80d2-23dd3f88df8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:13.781709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21c16c-370a-4789-80d2-23dd3f88df8a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:13.782001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21c16c-370a-4789-80d2-23dd3f88df8a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:13.782283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21c16c-370a-4789-80d2-23dd3f88df8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:13.783403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21c16c-370a-4789-80d2-23dd3f88df8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:13.785460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21c16c-370a-4789-80d2-23dd3f88df8a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:13.787025 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 46/86] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:13.789720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21c16c-370a-4789-80d2-23dd3f88df8a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:13.860709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11431f0a-6282-4ff7-aafe-2f7c3a71667d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:13.873118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:13.895214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d1c9c-ea5e-4fd7-a325-4eccb928df43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:13.895810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21c16c-370a-4789-80d2-23dd3f88df8a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:13.895909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d1c9c-ea5e-4fd7-a325-4eccb928df43 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:13.896102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d1c9c-ea5e-4fd7-a325-4eccb928df43 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:13.896383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d1c9c-ea5e-4fd7-a325-4eccb928df43 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:13.897266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d1c9c-ea5e-4fd7-a325-4eccb928df43 None admin] RBAC: Adding query filter params (name=member) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:13.897578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d1c9c-ea5e-4fd7-a325-4eccb928df43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:13.898962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d1c9c-ea5e-4fd7-a325-4eccb928df43 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:13.906867 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 41/87] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:13.907570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d1c9c-ea5e-4fd7-a325-4eccb928df43 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:13.932295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142246f-b0de-47a1-a504-74c4dfcd1877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:13.932928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142246f-b0de-47a1-a504-74c4dfcd1877 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:13.933225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142246f-b0de-47a1-a504-74c4dfcd1877 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:13.933508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142246f-b0de-47a1-a504-74c4dfcd1877 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:13.934232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142246f-b0de-47a1-a504-74c4dfcd1877 None admin] RBAC: Adding query filter params (role.id=d167c236cee64ede8d62bce075f83fff, scope.project.id=bb6a16a02b534f94be33eea4c77eea11, user.id=c2d9267d86294911a8b40405544b5a2f) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:13.934547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142246f-b0de-47a1-a504-74c4dfcd1877 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:13.935983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142246f-b0de-47a1-a504-74c4dfcd1877 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:13.942286 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 47/88] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:13 2022] GET /identity/v3/role_assignments?user.id=c2d9267d86294911a8b40405544b5a2f&scope.project.id=bb6a16a02b534f94be33eea4c77eea11&role.id=d167c236cee64ede8d62bce075f83fff => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:13.943079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142246f-b0de-47a1-a504-74c4dfcd1877 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:13.978931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:14.023851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d1c9c-ea5e-4fd7-a325-4eccb928df43 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:14.063249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404ff2f-a5ca-4cd1-b995-bea9803022d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:14.063911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404ff2f-a5ca-4cd1-b995-bea9803022d5 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:14.064210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404ff2f-a5ca-4cd1-b995-bea9803022d5 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:14.064497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404ff2f-a5ca-4cd1-b995-bea9803022d5 None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:14.066412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404ff2f-a5ca-4cd1-b995-bea9803022d5 None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:14.067968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404ff2f-a5ca-4cd1-b995-bea9803022d5 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:14.069731 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 42/89] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:13 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:14.074913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404ff2f-a5ca-4cd1-b995-bea9803022d5 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:14.082469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142246f-b0de-47a1-a504-74c4dfcd1877 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:14.129936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0b80f-ac5e-43cf-b991-643f24011694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:14.130581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0b80f-ac5e-43cf-b991-643f24011694 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:14.130866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0b80f-ac5e-43cf-b991-643f24011694 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:14.131145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0b80f-ac5e-43cf-b991-643f24011694 None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:14.132313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0b80f-ac5e-43cf-b991-643f24011694 None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:14.133800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0b80f-ac5e-43cf-b991-643f24011694 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:14.135600 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 48/90] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:13 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:14.136205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0b80f-ac5e-43cf-b991-643f24011694 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:14.153718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:14.193742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404ff2f-a5ca-4cd1-b995-bea9803022d5 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:14.229841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf7797-f520-4771-8563-a0ee63f87a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:14.230463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf7797-f520-4771-8563-a0ee63f87a42 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:14.230755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf7797-f520-4771-8563-a0ee63f87a42 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:14.231041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf7797-f520-4771-8563-a0ee63f87a42 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:14.231700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf7797-f520-4771-8563-a0ee63f87a42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:14.233200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf7797-f520-4771-8563-a0ee63f87a42 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:14.243199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:14.258934 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 43/91] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:32:14 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:14.259772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf7797-f520-4771-8563-a0ee63f87a42 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:14.261688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0b80f-ac5e-43cf-b991-643f24011694 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:14.324702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855d839-2277-4185-b39b-08a66a1d5b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:14.326141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d839-2277-4185-b39b-08a66a1d5b59 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:14.326709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d839-2277-4185-b39b-08a66a1d5b59 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:14.327160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d839-2277-4185-b39b-08a66a1d5b59 None admin] PATH_INFO: `/v3/users/c2d9267d86294911a8b40405544b5a2f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:14.329344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855d839-2277-4185-b39b-08a66a1d5b59 None admin] RBAC: Authorizing `identity:get_user(user_id=c2d9267d86294911a8b40405544b5a2f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:14.332220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855d839-2277-4185-b39b-08a66a1d5b59 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:14.335259 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 49/92] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:14 2022] GET /identity/v3/users/c2d9267d86294911a8b40405544b5a2f => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:14.336364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855d839-2277-4185-b39b-08a66a1d5b59 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:14.401514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf7797-f520-4771-8563-a0ee63f87a42 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:14.448926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938f382-c0e3-4b8e-a594-f31ec48779d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:14.449618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938f382-c0e3-4b8e-a594-f31ec48779d8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:14.449914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938f382-c0e3-4b8e-a594-f31ec48779d8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:14.450197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938f382-c0e3-4b8e-a594-f31ec48779d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:14.450940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938f382-c0e3-4b8e-a594-f31ec48779d8 None admin] RBAC: Adding query filter params (role.id=542f58433b2747d2abd5d405ebabdea1, scope.project.id=bb6a16a02b534f94be33eea4c77eea11, user.id=c2d9267d86294911a8b40405544b5a2f) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:14.451261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938f382-c0e3-4b8e-a594-f31ec48779d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:14.453294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938f382-c0e3-4b8e-a594-f31ec48779d8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:14.458475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:14.461709 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 44/93] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:14 2022] GET /identity/v3/role_assignments?user.id=c2d9267d86294911a8b40405544b5a2f&scope.project.id=bb6a16a02b534f94be33eea4c77eea11&role.id=542f58433b2747d2abd5d405ebabdea1 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:14.468924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938f382-c0e3-4b8e-a594-f31ec48779d8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:14.486156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855d839-2277-4185-b39b-08a66a1d5b59 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:14.538399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cba48c-b3d2-4071-aa22-12570f8634db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:14.539171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cba48c-b3d2-4071-aa22-12570f8634db None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:14.539929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cba48c-b3d2-4071-aa22-12570f8634db None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:14.540214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cba48c-b3d2-4071-aa22-12570f8634db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:14.541353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cba48c-b3d2-4071-aa22-12570f8634db None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=bb6a16a02b534f94be33eea4c77eea11, user.id=efa1ecf6d7f2438dad00007510b3e878) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:14.541630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cba48c-b3d2-4071-aa22-12570f8634db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:14.543435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cba48c-b3d2-4071-aa22-12570f8634db None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:14.552658 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 50/94] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:14 2022] GET /identity/v3/role_assignments?user.id=efa1ecf6d7f2438dad00007510b3e878&scope.project.id=bb6a16a02b534f94be33eea4c77eea11&role.id=bc768522dc984239be4a1b39a45ab437 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:14.560259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:14.615990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938f382-c0e3-4b8e-a594-f31ec48779d8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:14.673568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e353014-03a1-4aef-95f1-d85eee2c1dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:14.674771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e353014-03a1-4aef-95f1-d85eee2c1dfc None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:14.675137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e353014-03a1-4aef-95f1-d85eee2c1dfc None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:14.675466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e353014-03a1-4aef-95f1-d85eee2c1dfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:14.806574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e353014-03a1-4aef-95f1-d85eee2c1dfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:14.808336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e353014-03a1-4aef-95f1-d85eee2c1dfc None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:14.883108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:14.956624 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 45/95] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:14.963130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cba48c-b3d2-4071-aa22-12570f8634db None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:15.087657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cba48c-b3d2-4071-aa22-12570f8634db None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:15.124356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c63ad-01ce-498e-9057-c558fd68133f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:15.125327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c63ad-01ce-498e-9057-c558fd68133f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:15.125856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c63ad-01ce-498e-9057-c558fd68133f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:15.126315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c63ad-01ce-498e-9057-c558fd68133f None admin] PATH_INFO: `/v3/projects/c41abc73140f4536b5e4bed1c8ae6fb1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:15.128635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c63ad-01ce-498e-9057-c558fd68133f None admin] RBAC: Authorizing `identity:get_project(project_id=c41abc73140f4536b5e4bed1c8ae6fb1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:15.130336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c63ad-01ce-498e-9057-c558fd68133f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:15.132183 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 51/96] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:14 2022] GET /identity/v3/projects/c41abc73140f4536b5e4bed1c8ae6fb1 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:15.139570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e353014-03a1-4aef-95f1-d85eee2c1dfc None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:15.276727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e353014-03a1-4aef-95f1-d85eee2c1dfc None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:15.317090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f5ad2-7724-4a8a-bb4a-23a7e5db1be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:15.317720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f5ad2-7724-4a8a-bb4a-23a7e5db1be9 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:15.318013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f5ad2-7724-4a8a-bb4a-23a7e5db1be9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:15.318312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f5ad2-7724-4a8a-bb4a-23a7e5db1be9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:15.319041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f5ad2-7724-4a8a-bb4a-23a7e5db1be9 None admin] RBAC: Adding query filter params (role.id=d167c236cee64ede8d62bce075f83fff, scope.project.id=c41abc73140f4536b5e4bed1c8ae6fb1, user.id=c2d9267d86294911a8b40405544b5a2f) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:15.319361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f5ad2-7724-4a8a-bb4a-23a7e5db1be9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:15.320840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f5ad2-7724-4a8a-bb4a-23a7e5db1be9 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:15.331604 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 46/97] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:15 2022] GET /identity/v3/role_assignments?user.id=c2d9267d86294911a8b40405544b5a2f&scope.project.id=c41abc73140f4536b5e4bed1c8ae6fb1&role.id=d167c236cee64ede8d62bce075f83fff => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:17.075043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1b517-c9ef-432e-b5f4-3cc6f24bab7d None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:17.075580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1b517-c9ef-432e-b5f4-3cc6f24bab7d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:17.075580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1b517-c9ef-432e-b5f4-3cc6f24bab7d None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:17.076963 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 52/98] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:17.083381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b1595-e8a1-45fc-85ec-c1f18ca526bd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:17.083847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b1595-e8a1-45fc-85ec-c1f18ca526bd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:17.084028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b1595-e8a1-45fc-85ec-c1f18ca526bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:17.106214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ec2b1595-e8a1-45fc-85ec-c1f18ca526bd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:17.107408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ec2b1595-e8a1-45fc-85ec-c1f18ca526bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:17.123756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:17.182385 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 47/99] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:17.426911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f599f-75df-42ab-a4a4-cf9d386ca251 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:17.427568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f599f-75df-42ab-a4a4-cf9d386ca251 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:17.428059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f599f-75df-42ab-a4a4-cf9d386ca251 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:17.429612 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 53/100] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:17.434883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39885847-3dfa-4608-97de-0d2b91fb82d8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:17.435390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39885847-3dfa-4608-97de-0d2b91fb82d8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:17.435937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39885847-3dfa-4608-97de-0d2b91fb82d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:17.463922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-39885847-3dfa-4608-97de-0d2b91fb82d8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:17.465257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-39885847-3dfa-4608-97de-0d2b91fb82d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:17.546963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f599f-75df-42ab-a4a4-cf9d386ca251 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:17.558852 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 48/101] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:17.562053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:17.692710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f599f-75df-42ab-a4a4-cf9d386ca251 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:17.744566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d269c06-08c4-40db-b7e3-8e0fed47011a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:17.745591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d269c06-08c4-40db-b7e3-8e0fed47011a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:17.746108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d269c06-08c4-40db-b7e3-8e0fed47011a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:17.746578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d269c06-08c4-40db-b7e3-8e0fed47011a None admin] PATH_INFO: `/v3/roles/member` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:17.763621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d269c06-08c4-40db-b7e3-8e0fed47011a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-5d269c06-08c4-40db-b7e3-8e0fed47011a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:17.768854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:17.769798 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:17.770526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:17.770526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:17.770526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:17.770526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:17.770526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:17.770526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:17.770526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:17.770526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:17.770526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:17.770526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:17.770526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:17.770526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:17.772810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 54/102] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:17.778751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39885847-3dfa-4608-97de-0d2b91fb82d8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:17.834771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd7401-c2d7-428e-8110-b4d0c022894e None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:17.835399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd7401-c2d7-428e-8110-b4d0c022894e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:17.835871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd7401-c2d7-428e-8110-b4d0c022894e None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:17.837762 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 55/103] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:32:17.843022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8208cfb-6fc0-4a46-a070-2e4bda5dd1be None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:17.843648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8208cfb-6fc0-4a46-a070-2e4bda5dd1be None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:17.844131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8208cfb-6fc0-4a46-a070-2e4bda5dd1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:17.874488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a8208cfb-6fc0-4a46-a070-2e4bda5dd1be None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:17.877131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a8208cfb-6fc0-4a46-a070-2e4bda5dd1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:17.901274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:17.921905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39885847-3dfa-4608-97de-0d2b91fb82d8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:17.965005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccb504-ba10-4453-8137-600192735b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:17.965976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccb504-ba10-4453-8137-600192735b14 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:17.966724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccb504-ba10-4453-8137-600192735b14 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:17.967196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccb504-ba10-4453-8137-600192735b14 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:17.968071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccb504-ba10-4453-8137-600192735b14 None admin] RBAC: Adding query filter params (name=member) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:17.968636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccb504-ba10-4453-8137-600192735b14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:17.970317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccb504-ba10-4453-8137-600192735b14 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:17.980259 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 49/104] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:17.986232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccb504-ba10-4453-8137-600192735b14 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:18.039357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:18.050856 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 56/105] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:18.052188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8208cfb-6fc0-4a46-a070-2e4bda5dd1be None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:18.099929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccb504-ba10-4453-8137-600192735b14 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:18.145483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a0377-54ae-4fa8-84dd-1329a2ed01bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:18.146549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a0377-54ae-4fa8-84dd-1329a2ed01bd None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.147037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a0377-54ae-4fa8-84dd-1329a2ed01bd None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.147511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a0377-54ae-4fa8-84dd-1329a2ed01bd None admin] PATH_INFO: `/v3/users/c2d9267d86294911a8b40405544b5a2f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:18.151411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a0377-54ae-4fa8-84dd-1329a2ed01bd None admin] RBAC: Authorizing `identity:get_user(user_id=c2d9267d86294911a8b40405544b5a2f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:18.153279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a0377-54ae-4fa8-84dd-1329a2ed01bd None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:18.155616 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 50/106] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:17 2022] GET /identity/v3/users/c2d9267d86294911a8b40405544b5a2f => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:18.161687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a0377-54ae-4fa8-84dd-1329a2ed01bd None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:18.190825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:18.304038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8208cfb-6fc0-4a46-a070-2e4bda5dd1be None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:18.320089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a0377-54ae-4fa8-84dd-1329a2ed01bd None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:18.372372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110a80f-4221-4f82-a9f0-5809d850ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:18.374829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110a80f-4221-4f82-a9f0-5809d850ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.376123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110a80f-4221-4f82-a9f0-5809d850ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.376742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110a80f-4221-4f82-a9f0-5809d850ecb4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:18.395680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110a80f-4221-4f82-a9f0-5809d850ecb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:18.401834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5bc5b-ca75-499d-a484-9b8a44a3741c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:18.402704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5bc5b-ca75-499d-a484-9b8a44a3741c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.403259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5bc5b-ca75-499d-a484-9b8a44a3741c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.403764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5bc5b-ca75-499d-a484-9b8a44a3741c None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-6110a80f-4221-4f82-a9f0-5809d850ecb4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:18.405551 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:18.406385 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:18.407202 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:18.407202 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:18.407202 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:18.407202 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:18.407202 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:18.407202 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:18.407202 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:18.407202 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:18.407202 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:18.407202 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:18.407202 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:18.407202 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:18.409020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5bc5b-ca75-499d-a484-9b8a44a3741c None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:18.410708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5bc5b-ca75-499d-a484-9b8a44a3741c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:18.412596 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 51/107] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:18 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:18.414902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffca635-2bdb-4ade-bcce-d2c4b05a7550 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.415503 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 57/108] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:18.421178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110a80f-4221-4f82-a9f0-5809d850ecb4 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:18.421697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffca635-2bdb-4ade-bcce-d2c4b05a7550 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.422206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffca635-2bdb-4ade-bcce-d2c4b05a7550 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:18.424621 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 52/109] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:18.426057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffca635-2bdb-4ade-bcce-d2c4b05a7550 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:18.529327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:18.541635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:18.567713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffca635-2bdb-4ade-bcce-d2c4b05a7550 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:18.573902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110a80f-4221-4f82-a9f0-5809d850ecb4 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:18.604282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df785970-3b6e-48e1-a86b-c68609106514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:18.604918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df785970-3b6e-48e1-a86b-c68609106514 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.605226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df785970-3b6e-48e1-a86b-c68609106514 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.605511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df785970-3b6e-48e1-a86b-c68609106514 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:18.606175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df785970-3b6e-48e1-a86b-c68609106514 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:18.606486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df785970-3b6e-48e1-a86b-c68609106514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:18.607908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df785970-3b6e-48e1-a86b-c68609106514 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:18.612253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78895c14-2860-4dd4-847e-53ea80c6302b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:18.613081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78895c14-2860-4dd4-847e-53ea80c6302b None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.613445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78895c14-2860-4dd4-847e-53ea80c6302b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.613801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78895c14-2860-4dd4-847e-53ea80c6302b None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11/users/c2d9267d86294911a8b40405544b5a2f/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:18.615990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78895c14-2860-4dd4-847e-53ea80c6302b None admin] RBAC: Authorizing `identity:create_grant(project_id=bb6a16a02b534f94be33eea4c77eea11, user_id=c2d9267d86294911a8b40405544b5a2f, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:18.617655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78895c14-2860-4dd4-847e-53ea80c6302b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:18.620979 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 53/110] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:18.622918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b426d8-3614-4d19-8c6b-db6e583bfa91 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.623361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b426d8-3614-4d19-8c6b-db6e583bfa91 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.623647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b426d8-3614-4d19-8c6b-db6e583bfa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:18.634969 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 58/111] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:32:18 2022] PUT /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11/users/c2d9267d86294911a8b40405544b5a2f/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:32:18.636290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90eaaee-3fc3-4ec2-9b0b-09b866279f78 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.636595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90eaaee-3fc3-4ec2-9b0b-09b866279f78 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.636954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90eaaee-3fc3-4ec2-9b0b-09b866279f78 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:18.638395 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 59/112] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:18.639285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90eaaee-3fc3-4ec2-9b0b-09b866279f78 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:18.648075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f1b426d8-3614-4d19-8c6b-db6e583bfa91 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:18.650319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f1b426d8-3614-4d19-8c6b-db6e583bfa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:18.739179 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 54/113] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:18.740325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b426d8-3614-4d19-8c6b-db6e583bfa91 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:18.778527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90eaaee-3fc3-4ec2-9b0b-09b866279f78 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:18.802408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:18.818927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91f475-a3bb-4065-9f51-257c34325ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:18.819539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91f475-a3bb-4065-9f51-257c34325ee1 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.819831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91f475-a3bb-4065-9f51-257c34325ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.820105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91f475-a3bb-4065-9f51-257c34325ee1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:18.833353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91f475-a3bb-4065-9f51-257c34325ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-5a91f475-a3bb-4065-9f51-257c34325ee1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:18.838633 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:18.839504 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:18.840408 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 60/114] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:32:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:18.846011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c319088-62a3-4818-8734-6d027c17780f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.846539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c319088-62a3-4818-8734-6d027c17780f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.847277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c319088-62a3-4818-8734-6d027c17780f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:18.859311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:18.870993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4c319088-62a3-4818-8734-6d027c17780f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:18.872473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4c319088-62a3-4818-8734-6d027c17780f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:18.899265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b426d8-3614-4d19-8c6b-db6e583bfa91 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:18.948745 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 61/115] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:18.950216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf446fe2-04ca-4bc8-a239-d50f23607e94 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.950583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf446fe2-04ca-4bc8-a239-d50f23607e94 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.950920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf446fe2-04ca-4bc8-a239-d50f23607e94 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:18.952245 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 62/116] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:18.953107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf446fe2-04ca-4bc8-a239-d50f23607e94 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:18.953212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b93fb-1ea2-46df-a56f-6b056bad057a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:18.954095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b93fb-1ea2-46df-a56f-6b056bad057a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.954554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b93fb-1ea2-46df-a56f-6b056bad057a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.955000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b93fb-1ea2-46df-a56f-6b056bad057a None admin] PATH_INFO: `/v3/users/f5be54078c4f4fe88902ba63c6eca5ea` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:18.974554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b93fb-1ea2-46df-a56f-6b056bad057a None admin] RBAC: Authorizing `identity:get_user(user_id=f5be54078c4f4fe88902ba63c6eca5ea)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:18.976320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b93fb-1ea2-46df-a56f-6b056bad057a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:18.978329 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 55/117] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:18 2022] GET /identity/v3/users/f5be54078c4f4fe88902ba63c6eca5ea => generated 289 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:18.979708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb0509-e7f5-4218-9841-f605988c1133 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:18.980191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb0509-e7f5-4218-9841-f605988c1133 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:18.980661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb0509-e7f5-4218-9841-f605988c1133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:19.005855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-41bb0509-e7f5-4218-9841-f605988c1133 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:19.007402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-41bb0509-e7f5-4218-9841-f605988c1133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:19.095808 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 56/118] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:19.097522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb0509-e7f5-4218-9841-f605988c1133 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:19.100782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf446fe2-04ca-4bc8-a239-d50f23607e94 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:19.117733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:19.170237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:19.176299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8fbdb3-68ee-454c-8208-e97ce83fd294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:19.177015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fbdb3-68ee-454c-8208-e97ce83fd294 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:19.177366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fbdb3-68ee-454c-8208-e97ce83fd294 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:19.177633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fbdb3-68ee-454c-8208-e97ce83fd294 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:19.178376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8fbdb3-68ee-454c-8208-e97ce83fd294 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:19.178685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8fbdb3-68ee-454c-8208-e97ce83fd294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:19.180146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8fbdb3-68ee-454c-8208-e97ce83fd294 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:19.199070 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 63/119] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:19.199979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8fbdb3-68ee-454c-8208-e97ce83fd294 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:19.227430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb0509-e7f5-4218-9841-f605988c1133 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:19.266703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2112b-3c52-4f89-954e-49a105aa2a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:19.267754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2112b-3c52-4f89-954e-49a105aa2a61 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:19.268747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2112b-3c52-4f89-954e-49a105aa2a61 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:19.269257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2112b-3c52-4f89-954e-49a105aa2a61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:19.344037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2112b-3c52-4f89-954e-49a105aa2a61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:19.346030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2112b-3c52-4f89-954e-49a105aa2a61 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:19.402375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8fbdb3-68ee-454c-8208-e97ce83fd294 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:19.409329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:19.443085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a1e8d-a0fa-4d4b-b7ec-3692f7409070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:19.443738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a1e8d-a0fa-4d4b-b7ec-3692f7409070 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:19.444083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a1e8d-a0fa-4d4b-b7ec-3692f7409070 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:19.444372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a1e8d-a0fa-4d4b-b7ec-3692f7409070 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:19.460320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a1e8d-a0fa-4d4b-b7ec-3692f7409070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-166a1e8d-a0fa-4d4b-b7ec-3692f7409070 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:19.465733 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:19.466563 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:19.467333 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 64/120] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:32:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:19.467957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a1e8d-a0fa-4d4b-b7ec-3692f7409070 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:19.472012 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 57/121] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:19.473247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2112b-3c52-4f89-954e-49a105aa2a61 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:19.504245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:19.578542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2112b-3c52-4f89-954e-49a105aa2a61 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:19.581916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a1e8d-a0fa-4d4b-b7ec-3692f7409070 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:19.616696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e30118-9085-449e-b07f-5e78a9656f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:19.617297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30118-9085-449e-b07f-5e78a9656f4d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:19.617590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30118-9085-449e-b07f-5e78a9656f4d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:19.617692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87256bd3-6904-4b57-a427-4eeb99cc10f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:19.617869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30118-9085-449e-b07f-5e78a9656f4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:19.618284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87256bd3-6904-4b57-a427-4eeb99cc10f2 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:19.618571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87256bd3-6904-4b57-a427-4eeb99cc10f2 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:19.618850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87256bd3-6904-4b57-a427-4eeb99cc10f2 None admin] PATH_INFO: `/v3/users/efa1ecf6d7f2438dad00007510b3e878` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:19.620084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87256bd3-6904-4b57-a427-4eeb99cc10f2 None admin] RBAC: Authorizing `identity:get_user(user_id=efa1ecf6d7f2438dad00007510b3e878)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:19.621683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87256bd3-6904-4b57-a427-4eeb99cc10f2 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:19.623450 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 65/122] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:19 2022] GET /identity/v3/users/efa1ecf6d7f2438dad00007510b3e878 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:19.629012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87256bd3-6904-4b57-a427-4eeb99cc10f2 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:19.629140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e30118-9085-449e-b07f-5e78a9656f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-f7e30118-9085-449e-b07f-5e78a9656f4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:19.633940 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:19.634850 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:19.635634 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 58/123] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:19.636284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e30118-9085-449e-b07f-5e78a9656f4d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:19.657515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:19.742794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:19.746685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87256bd3-6904-4b57-a427-4eeb99cc10f2 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:19.750962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e30118-9085-449e-b07f-5e78a9656f4d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:19.784496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e93614-fdec-4781-9e7b-ff405b137d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:19.785179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e93614-fdec-4781-9e7b-ff405b137d6d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:19.785471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e93614-fdec-4781-9e7b-ff405b137d6d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:19.785757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e93614-fdec-4781-9e7b-ff405b137d6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:19.786960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e93614-fdec-4781-9e7b-ff405b137d6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:19.788427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e93614-fdec-4781-9e7b-ff405b137d6d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:19.789552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224233c7-09a5-4866-a9f6-71c74a89a554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:19.790173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224233c7-09a5-4866-a9f6-71c74a89a554 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:19.790463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224233c7-09a5-4866-a9f6-71c74a89a554 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:19.790836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224233c7-09a5-4866-a9f6-71c74a89a554 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:19.791597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224233c7-09a5-4866-a9f6-71c74a89a554 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:19.791916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224233c7-09a5-4866-a9f6-71c74a89a554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:19.793345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224233c7-09a5-4866-a9f6-71c74a89a554 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:19.794886 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 66/124] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:19.795484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e93614-fdec-4781-9e7b-ff405b137d6d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:19.808248 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 59/125] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:32:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:19.808248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224233c7-09a5-4866-a9f6-71c74a89a554 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:19.894986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:19.903735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e93614-fdec-4781-9e7b-ff405b137d6d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:19.916539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224233c7-09a5-4866-a9f6-71c74a89a554 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:19.943365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3916af5-5831-4c95-a8c5-fa3e36e0781d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:19.944316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3916af5-5831-4c95-a8c5-fa3e36e0781d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:19.944862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3916af5-5831-4c95-a8c5-fa3e36e0781d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:19.945172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3916af5-5831-4c95-a8c5-fa3e36e0781d None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:19.946411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3916af5-5831-4c95-a8c5-fa3e36e0781d None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:19.948660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3916af5-5831-4c95-a8c5-fa3e36e0781d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:19.956443 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 67/126] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:19 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:19.956590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16468f09-f5f5-42df-b385-5cec77e09348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:19.956687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3916af5-5831-4c95-a8c5-fa3e36e0781d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:19.956769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16468f09-f5f5-42df-b385-5cec77e09348 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:19.956769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16468f09-f5f5-42df-b385-5cec77e09348 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:19.956769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16468f09-f5f5-42df-b385-5cec77e09348 None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:19.956769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16468f09-f5f5-42df-b385-5cec77e09348 None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:19.956769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16468f09-f5f5-42df-b385-5cec77e09348 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:19.958145 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 60/127] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:19 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:19.958523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16468f09-f5f5-42df-b385-5cec77e09348 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:20.000576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:20.070662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16468f09-f5f5-42df-b385-5cec77e09348 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:20.073948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3916af5-5831-4c95-a8c5-fa3e36e0781d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:20.108614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216da84f-4601-484c-b62c-72e79d403c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:20.109277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216da84f-4601-484c-b62c-72e79d403c61 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:20.109488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c7a5c-4b77-4152-95c9-9428d68315ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:20.109599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216da84f-4601-484c-b62c-72e79d403c61 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:20.109835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216da84f-4601-484c-b62c-72e79d403c61 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:20.110506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216da84f-4601-484c-b62c-72e79d403c61 None admin] RBAC: Adding query filter params (name=member) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:20.110821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216da84f-4601-484c-b62c-72e79d403c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:20.111039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c7a5c-4b77-4152-95c9-9428d68315ae None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:20.111580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c7a5c-4b77-4152-95c9-9428d68315ae None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:20.112552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c7a5c-4b77-4152-95c9-9428d68315ae None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:20.112756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216da84f-4601-484c-b62c-72e79d403c61 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:20.113621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c7a5c-4b77-4152-95c9-9428d68315ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:20.115571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c7a5c-4b77-4152-95c9-9428d68315ae None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:20.123477 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 68/128] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:20.124311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216da84f-4601-484c-b62c-72e79d403c61 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:20.142915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:20.157018 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 61/129] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:32:19 2022] POST /identity/v3/projects => generated 302 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:20.157018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c7a5c-4b77-4152-95c9-9428d68315ae None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:20.262266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:20.266504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216da84f-4601-484c-b62c-72e79d403c61 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:20.282604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c7a5c-4b77-4152-95c9-9428d68315ae None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:20.307413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1227f-3b3d-44b3-b32b-870af0b45142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:20.308068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1227f-3b3d-44b3-b32b-870af0b45142 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:20.308356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1227f-3b3d-44b3-b32b-870af0b45142 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:20.308637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1227f-3b3d-44b3-b32b-870af0b45142 None admin] PATH_INFO: `/v3/users/c2d9267d86294911a8b40405544b5a2f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:20.309847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1227f-3b3d-44b3-b32b-870af0b45142 None admin] RBAC: Authorizing `identity:get_user(user_id=c2d9267d86294911a8b40405544b5a2f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:20.311391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1227f-3b3d-44b3-b32b-870af0b45142 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:20.313419 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 69/130] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:20 2022] GET /identity/v3/users/c2d9267d86294911a8b40405544b5a2f => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:20.317919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1227f-3b3d-44b3-b32b-870af0b45142 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:20.330919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ed6d-034a-45a2-bbad-1421fa3ad835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:20.331818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ed6d-034a-45a2-bbad-1421fa3ad835 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:20.332110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ed6d-034a-45a2-bbad-1421fa3ad835 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:20.337029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ed6d-034a-45a2-bbad-1421fa3ad835 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:20.337029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ed6d-034a-45a2-bbad-1421fa3ad835 None admin] RBAC: Adding query filter params (role.id=2eaf93a02e4a40a0b74e2a9fcbb182d0, scope.project.id=bb6a16a02b534f94be33eea4c77eea11, user.id=f5be54078c4f4fe88902ba63c6eca5ea) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:20.337029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ed6d-034a-45a2-bbad-1421fa3ad835 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:20.337029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ed6d-034a-45a2-bbad-1421fa3ad835 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:20.344205 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 62/131] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:20 2022] GET /identity/v3/role_assignments?user.id=f5be54078c4f4fe88902ba63c6eca5ea&scope.project.id=bb6a16a02b534f94be33eea4c77eea11&role.id=2eaf93a02e4a40a0b74e2a9fcbb182d0 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:20.350549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ed6d-034a-45a2-bbad-1421fa3ad835 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:20.425403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:20.444984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1227f-3b3d-44b3-b32b-870af0b45142 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:20.472589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ed6d-034a-45a2-bbad-1421fa3ad835 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:20.493832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed5e6a-218c-4cea-a041-25135df1b39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:20.495321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed5e6a-218c-4cea-a041-25135df1b39f None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:20.495832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed5e6a-218c-4cea-a041-25135df1b39f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:20.496330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed5e6a-218c-4cea-a041-25135df1b39f None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11/users/efa1ecf6d7f2438dad00007510b3e878/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:20.499400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed5e6a-218c-4cea-a041-25135df1b39f None admin] RBAC: Authorizing `identity:create_grant(project_id=bb6a16a02b534f94be33eea4c77eea11, user_id=efa1ecf6d7f2438dad00007510b3e878, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:20.501542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed5e6a-218c-4cea-a041-25135df1b39f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:20.519671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd54b1-0e2e-4ad0-9749-0755497eba2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:20.520440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd54b1-0e2e-4ad0-9749-0755497eba2e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:20.520739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd54b1-0e2e-4ad0-9749-0755497eba2e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:20.521074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd54b1-0e2e-4ad0-9749-0755497eba2e None admin] PATH_INFO: `/v3/users/c2d9267d86294911a8b40405544b5a2f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:20.522436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd54b1-0e2e-4ad0-9749-0755497eba2e None admin] RBAC: Authorizing `identity:get_user(user_id=c2d9267d86294911a8b40405544b5a2f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:20.524001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd54b1-0e2e-4ad0-9749-0755497eba2e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:20.524209 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 70/132] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:32:20 2022] PUT /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11/users/efa1ecf6d7f2438dad00007510b3e878/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:32:20.525313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed5e6a-218c-4cea-a041-25135df1b39f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:20.525977 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 63/133] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:20 2022] GET /identity/v3/users/c2d9267d86294911a8b40405544b5a2f => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:20.541704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd54b1-0e2e-4ad0-9749-0755497eba2e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:20.571819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:20.709849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed5e6a-218c-4cea-a041-25135df1b39f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:20.716411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd54b1-0e2e-4ad0-9749-0755497eba2e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:20.777675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20603557-5fbc-4f01-ac4e-022179b1e3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:20.778352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20603557-5fbc-4f01-ac4e-022179b1e3bc None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:20.778643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20603557-5fbc-4f01-ac4e-022179b1e3bc None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:20.778929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20603557-5fbc-4f01-ac4e-022179b1e3bc None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:20.780165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20603557-5fbc-4f01-ac4e-022179b1e3bc None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:20.781651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20603557-5fbc-4f01-ac4e-022179b1e3bc None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:20.783306 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 71/134] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:20 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:20.784534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:20.790959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2037735-c2f2-4fce-9c9a-563a5ff0e37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:20.791625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2037735-c2f2-4fce-9c9a-563a5ff0e37f None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:20.791918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2037735-c2f2-4fce-9c9a-563a5ff0e37f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:20.792203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2037735-c2f2-4fce-9c9a-563a5ff0e37f None admin] PATH_INFO: `/v3/projects/c41abc73140f4536b5e4bed1c8ae6fb1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:20.792791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20603557-5fbc-4f01-ac4e-022179b1e3bc None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:20.793578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2037735-c2f2-4fce-9c9a-563a5ff0e37f None admin] RBAC: Authorizing `identity:get_project(project_id=c41abc73140f4536b5e4bed1c8ae6fb1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:20.795141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2037735-c2f2-4fce-9c9a-563a5ff0e37f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:20.796792 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 64/135] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:20 2022] GET /identity/v3/projects/c41abc73140f4536b5e4bed1c8ae6fb1 => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:20.805347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2037735-c2f2-4fce-9c9a-563a5ff0e37f None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:20.902445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:20.906142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20603557-5fbc-4f01-ac4e-022179b1e3bc None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:20.944029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e26b33-33a7-4d91-adc5-8172291ea0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:20.944712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e26b33-33a7-4d91-adc5-8172291ea0bb None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:20.945039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e26b33-33a7-4d91-adc5-8172291ea0bb None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:20.945323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e26b33-33a7-4d91-adc5-8172291ea0bb None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11/users/c2d9267d86294911a8b40405544b5a2f/roles/542f58433b2747d2abd5d405ebabdea1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:20.947558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e26b33-33a7-4d91-adc5-8172291ea0bb None admin] RBAC: Authorizing `identity:create_grant(project_id=bb6a16a02b534f94be33eea4c77eea11, user_id=c2d9267d86294911a8b40405544b5a2f, role_id=542f58433b2747d2abd5d405ebabdea1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:20.949045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e26b33-33a7-4d91-adc5-8172291ea0bb None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:20.958718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2037735-c2f2-4fce-9c9a-563a5ff0e37f None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:20.979211 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 72/136] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:32:20 2022] PUT /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11/users/c2d9267d86294911a8b40405544b5a2f/roles/542f58433b2747d2abd5d405ebabdea1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:32:21.004654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d199a8c-ae87-4f5f-ab2f-2693db1899eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:21.005509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d199a8c-ae87-4f5f-ab2f-2693db1899eb None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:21.005798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d199a8c-ae87-4f5f-ab2f-2693db1899eb None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:21.006086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d199a8c-ae87-4f5f-ab2f-2693db1899eb None admin] PATH_INFO: `/v3/projects/c41abc73140f4536b5e4bed1c8ae6fb1/users/c2d9267d86294911a8b40405544b5a2f/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:21.014653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d199a8c-ae87-4f5f-ab2f-2693db1899eb None admin] RBAC: Authorizing `identity:create_grant(project_id=c41abc73140f4536b5e4bed1c8ae6fb1, user_id=c2d9267d86294911a8b40405544b5a2f, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:21.016137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d199a8c-ae87-4f5f-ab2f-2693db1899eb None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:21.043525 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 65/137] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:32:20 2022] PUT /identity/v3/projects/c41abc73140f4536b5e4bed1c8ae6fb1/users/c2d9267d86294911a8b40405544b5a2f/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:32:21.296890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad138809-1608-42d1-a754-452cbfdf586b None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:21.297641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad138809-1608-42d1-a754-452cbfdf586b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:21.298158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad138809-1608-42d1-a754-452cbfdf586b None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:21.300469 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 73/138] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:21.307142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89c6d5-c79f-4831-8588-2fcf2fe6226f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:21.308058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89c6d5-c79f-4831-8588-2fcf2fe6226f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:21.308595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89c6d5-c79f-4831-8588-2fcf2fe6226f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:21.328241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cb89c6d5-c79f-4831-8588-2fcf2fe6226f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:21.329817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cb89c6d5-c79f-4831-8588-2fcf2fe6226f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:21.400449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:21.414365 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 66/139] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:22.092629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad138809-1608-42d1-a754-452cbfdf586b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:22.234687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad138809-1608-42d1-a754-452cbfdf586b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:22.281586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d6e05-0389-48d5-837c-9af3c78d5b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:22.282456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d6e05-0389-48d5-837c-9af3c78d5b3a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:22.282920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d6e05-0389-48d5-837c-9af3c78d5b3a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:22.283376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d6e05-0389-48d5-837c-9af3c78d5b3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:22.297725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d6e05-0389-48d5-837c-9af3c78d5b3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-d85d6e05-0389-48d5-837c-9af3c78d5b3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:22.303297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:22.304610 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:22.305544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:22.305544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:22.305544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:22.305544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:22.305544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:22.305544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:22.305544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:22.305544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:22.305544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:22.305544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:22.305544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:22.305544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:22.307818 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 74/140] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:22.317997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89c6d5-c79f-4831-8588-2fcf2fe6226f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:22.461930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89c6d5-c79f-4831-8588-2fcf2fe6226f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:22.506301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457da885-e659-4df4-a24c-04bfdfdabdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:22.506945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457da885-e659-4df4-a24c-04bfdfdabdb6 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:22.507228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457da885-e659-4df4-a24c-04bfdfdabdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:22.507517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457da885-e659-4df4-a24c-04bfdfdabdb6 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:22.508191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457da885-e659-4df4-a24c-04bfdfdabdb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:22.508508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457da885-e659-4df4-a24c-04bfdfdabdb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:22.509996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457da885-e659-4df4-a24c-04bfdfdabdb6 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:22.524535 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 67/141] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:22.530366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d6e05-0389-48d5-837c-9af3c78d5b3a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:22.558037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:22.660506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d6e05-0389-48d5-837c-9af3c78d5b3a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:22.706856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee08b8-4cec-4536-9ab4-c9ba055cf66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:22.707501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee08b8-4cec-4536-9ab4-c9ba055cf66d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:22.707787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee08b8-4cec-4536-9ab4-c9ba055cf66d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:22.708112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee08b8-4cec-4536-9ab4-c9ba055cf66d None admin] PATH_INFO: `/v3/users/c2d9267d86294911a8b40405544b5a2f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:22.709300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee08b8-4cec-4536-9ab4-c9ba055cf66d None admin] RBAC: Authorizing `identity:get_user(user_id=c2d9267d86294911a8b40405544b5a2f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:22.710839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee08b8-4cec-4536-9ab4-c9ba055cf66d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:22.712782 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 75/142] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:22 2022] GET /identity/v3/users/c2d9267d86294911a8b40405544b5a2f => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:22.718101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457da885-e659-4df4-a24c-04bfdfdabdb6 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:22.784509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:22.834238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457da885-e659-4df4-a24c-04bfdfdabdb6 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:22.868605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc0aeb-fc11-4556-990c-0d4d44272e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:22.869258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc0aeb-fc11-4556-990c-0d4d44272e57 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:22.869535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc0aeb-fc11-4556-990c-0d4d44272e57 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:22.869815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc0aeb-fc11-4556-990c-0d4d44272e57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:22.941260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc0aeb-fc11-4556-990c-0d4d44272e57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:22.942743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc0aeb-fc11-4556-990c-0d4d44272e57 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:22.995519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:23.060763 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 68/143] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:23.067028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee08b8-4cec-4536-9ab4-c9ba055cf66d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:23.185717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:23.204462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee08b8-4cec-4536-9ab4-c9ba055cf66d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:23.240072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07a246-2990-422c-9b4b-6c548ea84b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:23.240716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07a246-2990-422c-9b4b-6c548ea84b8e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:23.241038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07a246-2990-422c-9b4b-6c548ea84b8e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:23.241320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07a246-2990-422c-9b4b-6c548ea84b8e None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:23.242480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07a246-2990-422c-9b4b-6c548ea84b8e None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:23.243949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07a246-2990-422c-9b4b-6c548ea84b8e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:23.245843 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 76/144] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:23 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:23.251650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc0aeb-fc11-4556-990c-0d4d44272e57 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:23.425284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc0aeb-fc11-4556-990c-0d4d44272e57 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:23.474542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2b279-3c5b-4941-91b2-9b94d2351131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:23.475166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2b279-3c5b-4941-91b2-9b94d2351131 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:23.475456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2b279-3c5b-4941-91b2-9b94d2351131 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:23.475737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2b279-3c5b-4941-91b2-9b94d2351131 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:23.476469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2b279-3c5b-4941-91b2-9b94d2351131 None admin] RBAC: Adding query filter params (role.id=d167c236cee64ede8d62bce075f83fff, scope.project.id=bb6a16a02b534f94be33eea4c77eea11, user.id=c2d9267d86294911a8b40405544b5a2f) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:23.476786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2b279-3c5b-4941-91b2-9b94d2351131 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:23.478623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2b279-3c5b-4941-91b2-9b94d2351131 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:23.489967 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 69/145] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:23 2022] GET /identity/v3/role_assignments?user.id=c2d9267d86294911a8b40405544b5a2f&scope.project.id=bb6a16a02b534f94be33eea4c77eea11&role.id=d167c236cee64ede8d62bce075f83fff => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:23.719445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbee3d-5208-4c4b-91bd-31fd66390a7b None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:23.719811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbee3d-5208-4c4b-91bd-31fd66390a7b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:23.720012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbee3d-5208-4c4b-91bd-31fd66390a7b None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:23.721840 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 77/146] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:23.728231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058449b-e875-4e83-820f-7b89152ab03e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:23.728517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058449b-e875-4e83-820f-7b89152ab03e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:23.728795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058449b-e875-4e83-820f-7b89152ab03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:23.754113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b058449b-e875-4e83-820f-7b89152ab03e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:23.754841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb92f11-bad4-486b-92d8-f39a5a10e524 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:23.755141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb92f11-bad4-486b-92d8-f39a5a10e524 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:23.755343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b058449b-e875-4e83-820f-7b89152ab03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:23.755440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb92f11-bad4-486b-92d8-f39a5a10e524 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:23.756995 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 78/147] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:32:23.762486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11462952-b7c2-47df-8127-a29a40ff7599 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:23.762780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11462952-b7c2-47df-8127-a29a40ff7599 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:23.763067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11462952-b7c2-47df-8127-a29a40ff7599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:23.768033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:23.797006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-11462952-b7c2-47df-8127-a29a40ff7599 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:23.798254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-11462952-b7c2-47df-8127-a29a40ff7599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:23.845428 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 70/148] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:23.859134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df4478-97a8-46ca-83c6-2b14b8139741 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:23.859425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df4478-97a8-46ca-83c6-2b14b8139741 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:23.859703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df4478-97a8-46ca-83c6-2b14b8139741 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:23.861051 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 71/149] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:23.868330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdb7a5-b3c0-4e37-9190-9625fffbb33e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:23.868608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdb7a5-b3c0-4e37-9190-9625fffbb33e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:23.868889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdb7a5-b3c0-4e37-9190-9625fffbb33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:23.897845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-59cdb7a5-b3c0-4e37-9190-9625fffbb33e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:23.899576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-59cdb7a5-b3c0-4e37-9190-9625fffbb33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:23.959229 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 79/150] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:23.985497 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 72/151] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:24.261906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11462952-b7c2-47df-8127-a29a40ff7599 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:24.353791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:24.379379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11462952-b7c2-47df-8127-a29a40ff7599 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:24.425269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229f4c9-0d92-452e-bf84-9f8dcc3db45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:24.425911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229f4c9-0d92-452e-bf84-9f8dcc3db45e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:24.426194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229f4c9-0d92-452e-bf84-9f8dcc3db45e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:24.426470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229f4c9-0d92-452e-bf84-9f8dcc3db45e None admin] PATH_INFO: `/v3/domains/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:24.427601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229f4c9-0d92-452e-bf84-9f8dcc3db45e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:24.429143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229f4c9-0d92-452e-bf84-9f8dcc3db45e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:24.430696 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 80/152] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:24.436371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdb7a5-b3c0-4e37-9190-9625fffbb33e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:24.497633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229f4c9-0d92-452e-bf84-9f8dcc3db45e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:24.540647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:24.572932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdb7a5-b3c0-4e37-9190-9625fffbb33e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:24.620011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ceaa1a-07d2-4586-b317-c10fd4df51eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:24.621135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ceaa1a-07d2-4586-b317-c10fd4df51eb None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:24.621486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ceaa1a-07d2-4586-b317-c10fd4df51eb None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:24.622094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ceaa1a-07d2-4586-b317-c10fd4df51eb None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:24.622828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ceaa1a-07d2-4586-b317-c10fd4df51eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:24.624243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229f4c9-0d92-452e-bf84-9f8dcc3db45e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:24.624403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ceaa1a-07d2-4586-b317-c10fd4df51eb None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:24.653318 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 73/153] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:32:24 2022] POST /identity/v3/users => generated 283 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:24.654680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ceaa1a-07d2-4586-b317-c10fd4df51eb None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:24.667860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb97035-936b-483b-a08c-bd68dc0365dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:24.668992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb97035-936b-483b-a08c-bd68dc0365dd None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:24.669583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb97035-936b-483b-a08c-bd68dc0365dd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:24.670130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb97035-936b-483b-a08c-bd68dc0365dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:24.684168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb97035-936b-483b-a08c-bd68dc0365dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-7fb97035-936b-483b-a08c-bd68dc0365dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:24.691291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:24.692210 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:24.693493 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:24.693493 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:24.693493 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:24.693493 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:24.693493 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:24.693493 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:24.693493 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:24.693493 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:24.693493 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:24.693493 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:24.693493 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:24.693493 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:24.695494 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 81/154] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:24.702925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65742206-331d-4d9d-b349-0776c68e7507 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:24.703264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65742206-331d-4d9d-b349-0776c68e7507 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:24.703710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65742206-331d-4d9d-b349-0776c68e7507 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:24.706319 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 82/155] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:24.711573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65742206-331d-4d9d-b349-0776c68e7507 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:24.728935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:24.784343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ceaa1a-07d2-4586-b317-c10fd4df51eb None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:24.817629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:24.825744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8690460-e854-46a4-9e91-7299df705380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:24.827071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8690460-e854-46a4-9e91-7299df705380 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:24.827427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8690460-e854-46a4-9e91-7299df705380 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:24.827803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8690460-e854-46a4-9e91-7299df705380 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:24.837973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65742206-331d-4d9d-b349-0776c68e7507 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:24.841683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8690460-e854-46a4-9e91-7299df705380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-b8690460-e854-46a4-9e91-7299df705380 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:24.848101 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:24.849268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:24.850155 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 74/156] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:32:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:24.850776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5022f-ee42-446a-b471-44943fac01f2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:24.850776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5022f-ee42-446a-b471-44943fac01f2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:24.852130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5022f-ee42-446a-b471-44943fac01f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:24.871392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8ca5022f-ee42-446a-b471-44943fac01f2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:24.872611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8ca5022f-ee42-446a-b471-44943fac01f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:24.887686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da25182-9135-48a3-9d72-6f1a4645fff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:24.888407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da25182-9135-48a3-9d72-6f1a4645fff0 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:24.888697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da25182-9135-48a3-9d72-6f1a4645fff0 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:24.889001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da25182-9135-48a3-9d72-6f1a4645fff0 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:24.889671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da25182-9135-48a3-9d72-6f1a4645fff0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:24.889976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da25182-9135-48a3-9d72-6f1a4645fff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:24.892079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da25182-9135-48a3-9d72-6f1a4645fff0 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:24.901366 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 83/157] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:24.905622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f56b6b-9b09-4371-835f-1d26ed076556 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:24.905622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f56b6b-9b09-4371-835f-1d26ed076556 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:24.905622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f56b6b-9b09-4371-835f-1d26ed076556 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:24.905622 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 84/158] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:24.905622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f56b6b-9b09-4371-835f-1d26ed076556 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:24.953159 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 75/159] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:24.954375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5022f-ee42-446a-b471-44943fac01f2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:24.993792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:25.043747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f56b6b-9b09-4371-835f-1d26ed076556 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:25.082962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5022f-ee42-446a-b471-44943fac01f2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:25.092464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e438b-1918-40a4-a869-e078ffa68265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:25.093264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e438b-1918-40a4-a869-e078ffa68265 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:25.093677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e438b-1918-40a4-a869-e078ffa68265 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:25.094247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e438b-1918-40a4-a869-e078ffa68265 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:25.095461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e438b-1918-40a4-a869-e078ffa68265 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:25.095922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e438b-1918-40a4-a869-e078ffa68265 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:25.097836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e438b-1918-40a4-a869-e078ffa68265 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:25.109162 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 85/160] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:25.112926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35742cae-e5bf-4d46-8f1b-aad91a6a9c57 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:25.112926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35742cae-e5bf-4d46-8f1b-aad91a6a9c57 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:25.112926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35742cae-e5bf-4d46-8f1b-aad91a6a9c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:25.117444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:25.136026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-35742cae-e5bf-4d46-8f1b-aad91a6a9c57 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:25.137782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-35742cae-e5bf-4d46-8f1b-aad91a6a9c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:25.138621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265180c-1132-4f0d-8bb9-ecd576b7c8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:25.139801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265180c-1132-4f0d-8bb9-ecd576b7c8f1 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:25.140341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265180c-1132-4f0d-8bb9-ecd576b7c8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:25.141386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265180c-1132-4f0d-8bb9-ecd576b7c8f1 None admin] PATH_INFO: `/v3/users/f5be54078c4f4fe88902ba63c6eca5ea` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:25.145584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265180c-1132-4f0d-8bb9-ecd576b7c8f1 None admin] RBAC: Authorizing `identity:get_user(user_id=f5be54078c4f4fe88902ba63c6eca5ea)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:25.147857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265180c-1132-4f0d-8bb9-ecd576b7c8f1 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:25.150997 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 76/161] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:24 2022] GET /identity/v3/users/f5be54078c4f4fe88902ba63c6eca5ea => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:25.152031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265180c-1132-4f0d-8bb9-ecd576b7c8f1 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:25.234101 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 86/162] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:25.235349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35742cae-e5bf-4d46-8f1b-aad91a6a9c57 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:25.298747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265180c-1132-4f0d-8bb9-ecd576b7c8f1 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:25.323916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:25.350647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75109c-f20e-46da-be9f-1c10d19805a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:25.352139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75109c-f20e-46da-be9f-1c10d19805a4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:25.352708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75109c-f20e-46da-be9f-1c10d19805a4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:25.353182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75109c-f20e-46da-be9f-1c10d19805a4 None admin] PATH_INFO: `/v3/users/efa1ecf6d7f2438dad00007510b3e878` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:25.355190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75109c-f20e-46da-be9f-1c10d19805a4 None admin] RBAC: Authorizing `identity:get_user(user_id=efa1ecf6d7f2438dad00007510b3e878)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:25.356842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75109c-f20e-46da-be9f-1c10d19805a4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:25.359199 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 77/163] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:25 2022] GET /identity/v3/users/efa1ecf6d7f2438dad00007510b3e878 => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:25.365654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75109c-f20e-46da-be9f-1c10d19805a4 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:25.373418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35742cae-e5bf-4d46-8f1b-aad91a6a9c57 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:25.411470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f6ff6-5ff3-481f-81fd-d45db26f384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:25.412208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f6ff6-5ff3-481f-81fd-d45db26f384f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:25.412571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f6ff6-5ff3-481f-81fd-d45db26f384f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:25.413015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f6ff6-5ff3-481f-81fd-d45db26f384f None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:25.414486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f6ff6-5ff3-481f-81fd-d45db26f384f None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:25.415892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f6ff6-5ff3-481f-81fd-d45db26f384f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:25.417691 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 87/164] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:25 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:25.423628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f6ff6-5ff3-481f-81fd-d45db26f384f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:25.435985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:25.483583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75109c-f20e-46da-be9f-1c10d19805a4 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:25.524828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d37d-81a2-4940-a6e3-35acde1c46a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:25.525893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d37d-81a2-4940-a6e3-35acde1c46a0 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:25.526365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d37d-81a2-4940-a6e3-35acde1c46a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:25.526920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d37d-81a2-4940-a6e3-35acde1c46a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:25.537672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f6ff6-5ff3-481f-81fd-d45db26f384f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:25.568667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:25.575346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253e998-7ac2-4aa9-af7f-08eb10e0f02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:25.577537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253e998-7ac2-4aa9-af7f-08eb10e0f02b None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:25.577964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253e998-7ac2-4aa9-af7f-08eb10e0f02b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:25.578419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253e998-7ac2-4aa9-af7f-08eb10e0f02b None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11/users/f5be54078c4f4fe88902ba63c6eca5ea/roles/2eaf93a02e4a40a0b74e2a9fcbb182d0` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:25.580645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253e998-7ac2-4aa9-af7f-08eb10e0f02b None admin] RBAC: Authorizing `identity:create_grant(project_id=bb6a16a02b534f94be33eea4c77eea11, user_id=f5be54078c4f4fe88902ba63c6eca5ea, role_id=2eaf93a02e4a40a0b74e2a9fcbb182d0)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:25.582820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253e998-7ac2-4aa9-af7f-08eb10e0f02b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:25.603306 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 88/165] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:32:25 2022] PUT /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11/users/f5be54078c4f4fe88902ba63c6eca5ea/roles/2eaf93a02e4a40a0b74e2a9fcbb182d0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:32:25.608833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7d37d-81a2-4940-a6e3-35acde1c46a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:25.610251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253e998-7ac2-4aa9-af7f-08eb10e0f02b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:25.611067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7d37d-81a2-4940-a6e3-35acde1c46a0 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:25.679645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:25.730617 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 78/166] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 20:32:25.737443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d37d-81a2-4940-a6e3-35acde1c46a0 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:25.768232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253e998-7ac2-4aa9-af7f-08eb10e0f02b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:25.818949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39c00d-51e2-423a-8807-ce07cc623eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:25.819753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39c00d-51e2-423a-8807-ce07cc623eed None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:25.820152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39c00d-51e2-423a-8807-ce07cc623eed None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:25.820532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39c00d-51e2-423a-8807-ce07cc623eed None admin] PATH_INFO: `/v3/roles/member` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:25.838732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39c00d-51e2-423a-8807-ce07cc623eed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-cc39c00d-51e2-423a-8807-ce07cc623eed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:25.844859 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:25.846473 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:25.847661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:25.847661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:25.847661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:25.847661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:25.847661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:25.847661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:25.847661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:25.847661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:25.847661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:25.847661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:25.847661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:25.847661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:25.853567 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 89/167] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:25.854534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39c00d-51e2-423a-8807-ce07cc623eed None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:25.855180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d37d-81a2-4940-a6e3-35acde1c46a0 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:25.896878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63da0c-1680-4f6d-995d-21a54c0a048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:25.897785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63da0c-1680-4f6d-995d-21a54c0a048e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:25.898314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63da0c-1680-4f6d-995d-21a54c0a048e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:25.898734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63da0c-1680-4f6d-995d-21a54c0a048e None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:25.900144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63da0c-1680-4f6d-995d-21a54c0a048e None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:25.901862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63da0c-1680-4f6d-995d-21a54c0a048e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:25.903927 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 79/168] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:25 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:25.904856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63da0c-1680-4f6d-995d-21a54c0a048e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:25.932869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:25.941962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:26.006755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39c00d-51e2-423a-8807-ce07cc623eed None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:26.022149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63da0c-1680-4f6d-995d-21a54c0a048e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:26.046359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935f29c-fbf8-4aec-a19e-8fa7e933f83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:26.047482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935f29c-fbf8-4aec-a19e-8fa7e933f83f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:26.048073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935f29c-fbf8-4aec-a19e-8fa7e933f83f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:26.048548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935f29c-fbf8-4aec-a19e-8fa7e933f83f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:26.060873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c866a-0cce-4c6e-8603-b0684b52c221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:26.061834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c866a-0cce-4c6e-8603-b0684b52c221 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:26.062295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c866a-0cce-4c6e-8603-b0684b52c221 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:26.062774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c866a-0cce-4c6e-8603-b0684b52c221 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:26.063620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c866a-0cce-4c6e-8603-b0684b52c221 None admin] RBAC: Adding query filter params (name=member) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:26.064099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c866a-0cce-4c6e-8603-b0684b52c221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:26.064200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935f29c-fbf8-4aec-a19e-8fa7e933f83f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:26.066431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c866a-0cce-4c6e-8603-b0684b52c221 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-9935f29c-fbf8-4aec-a19e-8fa7e933f83f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:26.073001 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:26.073925 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:26.074718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:26.074718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:26.074718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:26.074718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:26.074718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:26.074718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:26.074718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:26.074718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:26.074718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:26.074718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:26.074718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:26.074718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:26.075317 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 90/169] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:32:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:26.077402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935f29c-fbf8-4aec-a19e-8fa7e933f83f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:26.079986 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 80/170] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:26.081331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c866a-0cce-4c6e-8603-b0684b52c221 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:26.180334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:26.196371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c866a-0cce-4c6e-8603-b0684b52c221 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:26.201533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935f29c-fbf8-4aec-a19e-8fa7e933f83f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:26.233129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7be63-70a1-403c-a030-6765f21c6c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:26.233358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:26.233439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7be63-70a1-403c-a030-6765f21c6c70 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:26.233647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7be63-70a1-403c-a030-6765f21c6c70 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:26.233841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7be63-70a1-403c-a030-6765f21c6c70 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:26.234516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7be63-70a1-403c-a030-6765f21c6c70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:26.234825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7be63-70a1-403c-a030-6765f21c6c70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:26.236212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7be63-70a1-403c-a030-6765f21c6c70 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:26.246155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e925d-bec9-4064-b44e-5736c999c380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:26.247162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e925d-bec9-4064-b44e-5736c999c380 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:26.248073 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 81/171] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:32:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:26.248252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7be63-70a1-403c-a030-6765f21c6c70 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:26.248735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e925d-bec9-4064-b44e-5736c999c380 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:26.249268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e925d-bec9-4064-b44e-5736c999c380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:26.250232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e925d-bec9-4064-b44e-5736c999c380 None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=bb6a16a02b534f94be33eea4c77eea11, user.id=efa1ecf6d7f2438dad00007510b3e878) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:26.250759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e925d-bec9-4064-b44e-5736c999c380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:26.253578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e925d-bec9-4064-b44e-5736c999c380 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:26.260521 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 91/172] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:26 2022] GET /identity/v3/role_assignments?user.id=efa1ecf6d7f2438dad00007510b3e878&scope.project.id=bb6a16a02b534f94be33eea4c77eea11&role.id=bc768522dc984239be4a1b39a45ab437 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:26.261687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e925d-bec9-4064-b44e-5736c999c380 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:26.450355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7be63-70a1-403c-a030-6765f21c6c70 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:26.523778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d820f7-fc61-4f87-be75-fdcd2b03bb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:26.525278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d820f7-fc61-4f87-be75-fdcd2b03bb2d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:26.525771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d820f7-fc61-4f87-be75-fdcd2b03bb2d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:26.526249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d820f7-fc61-4f87-be75-fdcd2b03bb2d None admin] PATH_INFO: `/v3/users/c2d9267d86294911a8b40405544b5a2f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:26.527646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d820f7-fc61-4f87-be75-fdcd2b03bb2d None admin] RBAC: Authorizing `identity:get_user(user_id=c2d9267d86294911a8b40405544b5a2f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:26.529368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d820f7-fc61-4f87-be75-fdcd2b03bb2d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:26.532766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e925d-bec9-4064-b44e-5736c999c380 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:26.533422 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 82/173] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:26 2022] GET /identity/v3/users/c2d9267d86294911a8b40405544b5a2f => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:26.544947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d820f7-fc61-4f87-be75-fdcd2b03bb2d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:26.574031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:26.587765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f394a0-d84f-437c-8d65-9d5e949d82b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:26.589615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f394a0-d84f-437c-8d65-9d5e949d82b3 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:26.590169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f394a0-d84f-437c-8d65-9d5e949d82b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:26.590874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f394a0-d84f-437c-8d65-9d5e949d82b3 None admin] PATH_INFO: `/v3/users/c2d9267d86294911a8b40405544b5a2f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:26.592944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f394a0-d84f-437c-8d65-9d5e949d82b3 None admin] RBAC: Authorizing `identity:get_user(user_id=c2d9267d86294911a8b40405544b5a2f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:26.594952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f394a0-d84f-437c-8d65-9d5e949d82b3 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:26.598162 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 92/174] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:26 2022] GET /identity/v3/users/c2d9267d86294911a8b40405544b5a2f => generated 275 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:26.607729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f394a0-d84f-437c-8d65-9d5e949d82b3 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:26.698578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:26.707276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d820f7-fc61-4f87-be75-fdcd2b03bb2d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:26.775041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a5c8d-e508-4cac-978b-397bf1a7050e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:26.775663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a5c8d-e508-4cac-978b-397bf1a7050e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:26.775954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a5c8d-e508-4cac-978b-397bf1a7050e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:26.776232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a5c8d-e508-4cac-978b-397bf1a7050e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:26.815615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f394a0-d84f-437c-8d65-9d5e949d82b3 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:26.912441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a5c8d-e508-4cac-978b-397bf1a7050e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:26.913951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a5c8d-e508-4cac-978b-397bf1a7050e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:26.914288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb29de-977c-4b07-afa2-e83ac67fe1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:26.915168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb29de-977c-4b07-afa2-e83ac67fe1db None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:26.915658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb29de-977c-4b07-afa2-e83ac67fe1db None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:26.916157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb29de-977c-4b07-afa2-e83ac67fe1db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:26.950526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:27.070420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb29de-977c-4b07-afa2-e83ac67fe1db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:27.072288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb29de-977c-4b07-afa2-e83ac67fe1db None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:27.087176 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 83/175] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 20:32:27.093985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a5c8d-e508-4cac-978b-397bf1a7050e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:27.170985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:27.277493 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 93/176] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:27.286455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb29de-977c-4b07-afa2-e83ac67fe1db None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:27.288790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a5c8d-e508-4cac-978b-397bf1a7050e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:27.319393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:27.356070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aba52-f15a-4c5a-8905-8861da8ee3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:27.356639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aba52-f15a-4c5a-8905-8861da8ee3a1 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:27.356940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aba52-f15a-4c5a-8905-8861da8ee3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:27.357227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aba52-f15a-4c5a-8905-8861da8ee3a1 None admin] PATH_INFO: `/v3/projects/c41abc73140f4536b5e4bed1c8ae6fb1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:27.362102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932aba52-f15a-4c5a-8905-8861da8ee3a1 None admin] RBAC: Authorizing `identity:get_project(project_id=c41abc73140f4536b5e4bed1c8ae6fb1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:27.363723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932aba52-f15a-4c5a-8905-8861da8ee3a1 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:27.365521 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 84/177] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:27 2022] GET /identity/v3/projects/c41abc73140f4536b5e4bed1c8ae6fb1 => generated 312 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:27.371575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aba52-f15a-4c5a-8905-8861da8ee3a1 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:27.457986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb29de-977c-4b07-afa2-e83ac67fe1db None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:27.507365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f93d53-0e7b-4a98-836d-5c71d27bc1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:27.508263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f93d53-0e7b-4a98-836d-5c71d27bc1e8 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:27.508547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f93d53-0e7b-4a98-836d-5c71d27bc1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:27.509752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f93d53-0e7b-4a98-836d-5c71d27bc1e8 None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:27.510981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f93d53-0e7b-4a98-836d-5c71d27bc1e8 None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:27.512429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f93d53-0e7b-4a98-836d-5c71d27bc1e8 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:27.515042 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 94/178] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:27 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:27.522291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f93d53-0e7b-4a98-836d-5c71d27bc1e8 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:27.546839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:27.561994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aba52-f15a-4c5a-8905-8861da8ee3a1 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:27.625985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e84c3-e3c3-4a8a-b6cb-afb870dd3bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:27.626682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e84c3-e3c3-4a8a-b6cb-afb870dd3bdd None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:27.626969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e84c3-e3c3-4a8a-b6cb-afb870dd3bdd None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:27.627277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e84c3-e3c3-4a8a-b6cb-afb870dd3bdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:27.628014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e84c3-e3c3-4a8a-b6cb-afb870dd3bdd None admin] RBAC: Adding query filter params (role.id=d167c236cee64ede8d62bce075f83fff, scope.project.id=c41abc73140f4536b5e4bed1c8ae6fb1, user.id=c2d9267d86294911a8b40405544b5a2f) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:27.628325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e84c3-e3c3-4a8a-b6cb-afb870dd3bdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:27.629801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e84c3-e3c3-4a8a-b6cb-afb870dd3bdd None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:27.646421 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 85/179] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:27 2022] GET /identity/v3/role_assignments?user.id=c2d9267d86294911a8b40405544b5a2f&scope.project.id=c41abc73140f4536b5e4bed1c8ae6fb1&role.id=d167c236cee64ede8d62bce075f83fff => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:27.706720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f93d53-0e7b-4a98-836d-5c71d27bc1e8 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:27.742883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150955fd-f702-46db-889c-de18c8bcc602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:27.743555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150955fd-f702-46db-889c-de18c8bcc602 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:27.743839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150955fd-f702-46db-889c-de18c8bcc602 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:27.744161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150955fd-f702-46db-889c-de18c8bcc602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:27.744911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150955fd-f702-46db-889c-de18c8bcc602 None admin] RBAC: Adding query filter params (role.id=542f58433b2747d2abd5d405ebabdea1, scope.project.id=bb6a16a02b534f94be33eea4c77eea11, user.id=c2d9267d86294911a8b40405544b5a2f) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:27.745232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150955fd-f702-46db-889c-de18c8bcc602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:27.746742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150955fd-f702-46db-889c-de18c8bcc602 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:27.753958 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 95/180] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:27 2022] GET /identity/v3/role_assignments?user.id=c2d9267d86294911a8b40405544b5a2f&scope.project.id=bb6a16a02b534f94be33eea4c77eea11&role.id=542f58433b2747d2abd5d405ebabdea1 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:28.053373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2eee42-ecc8-4caa-8cab-90c93cbfdf8a None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:28.053750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2eee42-ecc8-4caa-8cab-90c93cbfdf8a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:28.053944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2eee42-ecc8-4caa-8cab-90c93cbfdf8a None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:28.055323 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 86/181] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:28.061488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62afae9-013f-4685-bd8e-00dc2fbe581f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:28.062106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62afae9-013f-4685-bd8e-00dc2fbe581f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:28.062589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62afae9-013f-4685-bd8e-00dc2fbe581f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:28.083330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f62afae9-013f-4685-bd8e-00dc2fbe581f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:28.084861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f62afae9-013f-4685-bd8e-00dc2fbe581f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:28.172589 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 96/182] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:28.562400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2eee42-ecc8-4caa-8cab-90c93cbfdf8a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:28.617598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:28.677594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2eee42-ecc8-4caa-8cab-90c93cbfdf8a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:28.684665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bc985-c313-444d-b3c3-62b4a82a6f7c None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:28.685627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bc985-c313-444d-b3c3-62b4a82a6f7c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:28.686107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bc985-c313-444d-b3c3-62b4a82a6f7c None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:28.687907 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 97/183] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:28.693842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94c8a8-3ff8-400a-8e00-d860adcee09c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:28.694390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94c8a8-3ff8-400a-8e00-d860adcee09c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:28.694896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94c8a8-3ff8-400a-8e00-d860adcee09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:28.702880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:28.715134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cd305-54d3-4de4-b495-b15ae1baf4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:28.715768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cd305-54d3-4de4-b495-b15ae1baf4d4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:28.716050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cd305-54d3-4de4-b495-b15ae1baf4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:28.716323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cd305-54d3-4de4-b495-b15ae1baf4d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:28.717109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4e94c8a8-3ff8-400a-8e00-d860adcee09c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:28.718504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4e94c8a8-3ff8-400a-8e00-d860adcee09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:28.726887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cd305-54d3-4de4-b495-b15ae1baf4d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-134cd305-54d3-4de4-b495-b15ae1baf4d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:28.732216 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:28.733121 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:28.733923 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 87/184] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:32:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:28.735047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee5fbd-9396-4dc1-bad8-18e5dedf4085 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:28.735217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee5fbd-9396-4dc1-bad8-18e5dedf4085 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:28.735491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee5fbd-9396-4dc1-bad8-18e5dedf4085 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:28.737265 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 88/185] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:28.740043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee5fbd-9396-4dc1-bad8-18e5dedf4085 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:28.797062 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 98/186] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:28.798793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebfd36-f68f-46e9-98b3-f71861357aa1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:28.799943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebfd36-f68f-46e9-98b3-f71861357aa1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:28.800298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebfd36-f68f-46e9-98b3-f71861357aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:28.826684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7eebfd36-f68f-46e9-98b3-f71861357aa1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:28.828342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7eebfd36-f68f-46e9-98b3-f71861357aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:28.855109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee5fbd-9396-4dc1-bad8-18e5dedf4085 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:28.866990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:28.893128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2f4b3-0f0a-44a4-bd9c-c1671ae9b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:28.894206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2f4b3-0f0a-44a4-bd9c-c1671ae9b25c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:28.894554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2f4b3-0f0a-44a4-bd9c-c1671ae9b25c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:28.895011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2f4b3-0f0a-44a4-bd9c-c1671ae9b25c None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:28.895850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2f4b3-0f0a-44a4-bd9c-c1671ae9b25c None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:28.896165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2f4b3-0f0a-44a4-bd9c-c1671ae9b25c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:28.897852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2f4b3-0f0a-44a4-bd9c-c1671ae9b25c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:28.907666 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 89/187] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:28.910161 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 99/188] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:28.918864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2f4b3-0f0a-44a4-bd9c-c1671ae9b25c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:29.029423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2f4b3-0f0a-44a4-bd9c-c1671ae9b25c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:29.072347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fde065-439f-4f73-911e-73fab2cf4007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:29.073020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fde065-439f-4f73-911e-73fab2cf4007 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:29.073311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fde065-439f-4f73-911e-73fab2cf4007 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:29.073590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fde065-439f-4f73-911e-73fab2cf4007 None admin] PATH_INFO: `/v3/users/496773170381441b9067354f430d9aba` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:29.089242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fde065-439f-4f73-911e-73fab2cf4007 None admin] RBAC: Authorizing `identity:get_user(user_id=496773170381441b9067354f430d9aba)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:29.090638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fde065-439f-4f73-911e-73fab2cf4007 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:29.092381 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 90/189] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:28 2022] GET /identity/v3/users/496773170381441b9067354f430d9aba => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:29.098027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebfd36-f68f-46e9-98b3-f71861357aa1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:29.172993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:29.191433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fde065-439f-4f73-911e-73fab2cf4007 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:29.220009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebfd36-f68f-46e9-98b3-f71861357aa1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:29.223390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:29.258329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa4e82-60a6-446d-9160-f4551faccb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:29.259225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa4e82-60a6-446d-9160-f4551faccb2f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:29.259518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa4e82-60a6-446d-9160-f4551faccb2f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:29.259802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa4e82-60a6-446d-9160-f4551faccb2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:29.295496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fde065-439f-4f73-911e-73fab2cf4007 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:29.327841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa4e82-60a6-446d-9160-f4551faccb2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:29.329395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa4e82-60a6-446d-9160-f4551faccb2f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:29.329921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccccc2-1b74-47c5-b58d-8a0cecc35c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:29.331360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccccc2-1b74-47c5-b58d-8a0cecc35c51 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:29.331641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccccc2-1b74-47c5-b58d-8a0cecc35c51 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:29.331913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccccc2-1b74-47c5-b58d-8a0cecc35c51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:29.341920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bccccc2-1b74-47c5-b58d-8a0cecc35c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-9bccccc2-1b74-47c5-b58d-8a0cecc35c51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:29.346712 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:29.347528 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:29.348466 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 91/190] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:32:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:32:29.350154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68991399-5b47-426e-9181-f3d95e00fd8e None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:29.350948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68991399-5b47-426e-9181-f3d95e00fd8e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:29.351260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68991399-5b47-426e-9181-f3d95e00fd8e None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:29.352920 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 92/191] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:29.357134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68991399-5b47-426e-9181-f3d95e00fd8e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:29.391572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:29.433693 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 100/192] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:29.434360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa4e82-60a6-446d-9160-f4551faccb2f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:29.448474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:29.459955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68991399-5b47-426e-9181-f3d95e00fd8e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:29.494291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1ce97-34a6-4b0f-a6bc-ba6d3039131a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:29.495163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1ce97-34a6-4b0f-a6bc-ba6d3039131a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:29.495445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1ce97-34a6-4b0f-a6bc-ba6d3039131a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:29.495726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1ce97-34a6-4b0f-a6bc-ba6d3039131a None admin] PATH_INFO: `/v3/domains/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:29.496846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1ce97-34a6-4b0f-a6bc-ba6d3039131a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:29.498335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1ce97-34a6-4b0f-a6bc-ba6d3039131a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:29.499911 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 93/193] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:29.501052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d8a08-018e-4d2e-80b5-e741108b88bf None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:29.501343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d8a08-018e-4d2e-80b5-e741108b88bf None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:29.501631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d8a08-018e-4d2e-80b5-e741108b88bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:29.520321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c88d8a08-018e-4d2e-80b5-e741108b88bf None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:29.521632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c88d8a08-018e-4d2e-80b5-e741108b88bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:29.547312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa4e82-60a6-446d-9160-f4551faccb2f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:29.582415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4b6d2-b6f0-446c-a857-a064727174f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:29.583078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4b6d2-b6f0-446c-a857-a064727174f3 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:29.583360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4b6d2-b6f0-446c-a857-a064727174f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:29.583632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4b6d2-b6f0-446c-a857-a064727174f3 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:29.584329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4b6d2-b6f0-446c-a857-a064727174f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:29.584637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4b6d2-b6f0-446c-a857-a064727174f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:29.586064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4b6d2-b6f0-446c-a857-a064727174f3 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:29.587773 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 94/194] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:29.588597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d8a08-018e-4d2e-80b5-e741108b88bf None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:29.594276 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 101/195] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:32:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:29.596189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4b6d2-b6f0-446c-a857-a064727174f3 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:29.628252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:29.680968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:29.696126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d8a08-018e-4d2e-80b5-e741108b88bf None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:29.718394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4b6d2-b6f0-446c-a857-a064727174f3 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:29.732007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6352b44-f5f9-4d4f-a26a-cd592fed401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:29.732635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6352b44-f5f9-4d4f-a26a-cd592fed401d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:29.732990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6352b44-f5f9-4d4f-a26a-cd592fed401d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:29.733274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6352b44-f5f9-4d4f-a26a-cd592fed401d None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:29.734469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6352b44-f5f9-4d4f-a26a-cd592fed401d None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:29.735912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6352b44-f5f9-4d4f-a26a-cd592fed401d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:29.738044 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 95/196] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:29 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:29.738804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6352b44-f5f9-4d4f-a26a-cd592fed401d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:29.765591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9f0b9-1116-41aa-ac85-9d4152915a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:29.766199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9f0b9-1116-41aa-ac85-9d4152915a09 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:29.766482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9f0b9-1116-41aa-ac85-9d4152915a09 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:29.766759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9f0b9-1116-41aa-ac85-9d4152915a09 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:29.767400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9f0b9-1116-41aa-ac85-9d4152915a09 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:29.768854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9f0b9-1116-41aa-ac85-9d4152915a09 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:29.808464 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 102/197] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:32:29 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:29.817054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9f0b9-1116-41aa-ac85-9d4152915a09 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:29.856000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6352b44-f5f9-4d4f-a26a-cd592fed401d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:29.893585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ce5a6-822d-4fbd-92d3-7e3f6c742a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:29.894241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ce5a6-822d-4fbd-92d3-7e3f6c742a83 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:29.894534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ce5a6-822d-4fbd-92d3-7e3f6c742a83 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:29.894815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ce5a6-822d-4fbd-92d3-7e3f6c742a83 None admin] PATH_INFO: `/v3/users/496773170381441b9067354f430d9aba` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:29.896037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ce5a6-822d-4fbd-92d3-7e3f6c742a83 None admin] RBAC: Authorizing `identity:get_user(user_id=496773170381441b9067354f430d9aba)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:29.901219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ce5a6-822d-4fbd-92d3-7e3f6c742a83 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:29.901219 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 96/198] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:29 2022] GET /identity/v3/users/496773170381441b9067354f430d9aba => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:29.906477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ce5a6-822d-4fbd-92d3-7e3f6c742a83 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:29.918495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:29.943128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9f0b9-1116-41aa-ac85-9d4152915a09 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:29.944173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:29.983504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad1e88-514f-4cd2-a678-47df2ab24099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:29.984259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad1e88-514f-4cd2-a678-47df2ab24099 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:29.984557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad1e88-514f-4cd2-a678-47df2ab24099 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:29.984838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad1e88-514f-4cd2-a678-47df2ab24099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:29.985630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad1e88-514f-4cd2-a678-47df2ab24099 None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=0a64615765ab4a01a30fc0023bb081c4, user.id=496773170381441b9067354f430d9aba) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:29.985946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad1e88-514f-4cd2-a678-47df2ab24099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:29.988248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad1e88-514f-4cd2-a678-47df2ab24099 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:29.994586 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 103/199] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:29 2022] GET /identity/v3/role_assignments?user.id=496773170381441b9067354f430d9aba&scope.project.id=0a64615765ab4a01a30fc0023bb081c4&role.id=bc768522dc984239be4a1b39a45ab437 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:29.995455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad1e88-514f-4cd2-a678-47df2ab24099 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:30.046186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ce5a6-822d-4fbd-92d3-7e3f6c742a83 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:30.105335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba59cfd-d152-4acb-8c72-53cf418d4f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:30.106152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba59cfd-d152-4acb-8c72-53cf418d4f90 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:30.107524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba59cfd-d152-4acb-8c72-53cf418d4f90 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:30.107802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba59cfd-d152-4acb-8c72-53cf418d4f90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:30.155063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad1e88-514f-4cd2-a678-47df2ab24099 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:30.200561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd58f-d3e9-48d5-bfad-754f40838d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:30.201541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd58f-d3e9-48d5-bfad-754f40838d26 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:30.201824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd58f-d3e9-48d5-bfad-754f40838d26 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:30.202110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd58f-d3e9-48d5-bfad-754f40838d26 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:30.217089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54abd58f-d3e9-48d5-bfad-754f40838d26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:30.222869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba59cfd-d152-4acb-8c72-53cf418d4f90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:30.224343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba59cfd-d152-4acb-8c72-53cf418d4f90 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-54abd58f-d3e9-48d5-bfad-754f40838d26 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:30.224819 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:30.225718 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:30.226498 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 104/200] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:30.232069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd58f-d3e9-48d5-bfad-754f40838d26 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:30.244331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:30.257591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:30.353764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd58f-d3e9-48d5-bfad-754f40838d26 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:30.357631 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 97/201] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 20:32:30.364212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba59cfd-d152-4acb-8c72-53cf418d4f90 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:30.397225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dbab9-0d04-423e-817c-ac9894da63e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:30.398000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dbab9-0d04-423e-817c-ac9894da63e0 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:30.398627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dbab9-0d04-423e-817c-ac9894da63e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:30.398909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dbab9-0d04-423e-817c-ac9894da63e0 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:30.399867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dbab9-0d04-423e-817c-ac9894da63e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:30.400178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dbab9-0d04-423e-817c-ac9894da63e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:30.402254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dbab9-0d04-423e-817c-ac9894da63e0 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:30.413995 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 105/202] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:30.427054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dbab9-0d04-423e-817c-ac9894da63e0 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:30.491941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba59cfd-d152-4acb-8c72-53cf418d4f90 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:30.519689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:30.532533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c44c0-7222-47c6-ac08-248facdef07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:30.533246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c44c0-7222-47c6-ac08-248facdef07a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:30.533552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c44c0-7222-47c6-ac08-248facdef07a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:30.533866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c44c0-7222-47c6-ac08-248facdef07a None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:30.535080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c44c0-7222-47c6-ac08-248facdef07a None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:30.537902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c44c0-7222-47c6-ac08-248facdef07a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:30.545078 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 98/203] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:30 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:30.546417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c44c0-7222-47c6-ac08-248facdef07a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:30.617067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:30.678576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dbab9-0d04-423e-817c-ac9894da63e0 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:30.707163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c44c0-7222-47c6-ac08-248facdef07a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:30.734805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bd923-616f-40c0-95f6-0e4daaaca26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:30.735445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bd923-616f-40c0-95f6-0e4daaaca26d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:30.735740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bd923-616f-40c0-95f6-0e4daaaca26d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:30.736021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bd923-616f-40c0-95f6-0e4daaaca26d None admin] PATH_INFO: `/v3/users/f5be54078c4f4fe88902ba63c6eca5ea` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:30.738315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bd923-616f-40c0-95f6-0e4daaaca26d None admin] RBAC: Authorizing `identity:get_user(user_id=f5be54078c4f4fe88902ba63c6eca5ea)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:30.739800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bd923-616f-40c0-95f6-0e4daaaca26d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:30.741760 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 106/204] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:30 2022] GET /identity/v3/users/f5be54078c4f4fe88902ba63c6eca5ea => generated 289 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:30.746921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bd923-616f-40c0-95f6-0e4daaaca26d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:30.767892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169afbf-ab0b-45f7-afab-a690f66bfa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:30.768513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169afbf-ab0b-45f7-afab-a690f66bfa49 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:30.768801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169afbf-ab0b-45f7-afab-a690f66bfa49 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:30.769100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169afbf-ab0b-45f7-afab-a690f66bfa49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:30.769839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169afbf-ab0b-45f7-afab-a690f66bfa49 None admin] RBAC: Adding query filter params (role.id=542f58433b2747d2abd5d405ebabdea1, scope.project.id=0a64615765ab4a01a30fc0023bb081c4, user.id=496773170381441b9067354f430d9aba) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:30.770153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169afbf-ab0b-45f7-afab-a690f66bfa49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:30.771611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169afbf-ab0b-45f7-afab-a690f66bfa49 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:30.784535 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 99/205] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:30 2022] GET /identity/v3/role_assignments?user.id=496773170381441b9067354f430d9aba&scope.project.id=0a64615765ab4a01a30fc0023bb081c4&role.id=542f58433b2747d2abd5d405ebabdea1 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:30.879396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bd923-616f-40c0-95f6-0e4daaaca26d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:30.935039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347208a-b3d4-4738-8aea-2d300aeaf7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:30.935678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347208a-b3d4-4738-8aea-2d300aeaf7b3 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:30.935971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347208a-b3d4-4738-8aea-2d300aeaf7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:30.936269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347208a-b3d4-4738-8aea-2d300aeaf7b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:30.946170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:31.044584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347208a-b3d4-4738-8aea-2d300aeaf7b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:31.046123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347208a-b3d4-4738-8aea-2d300aeaf7b3 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:31.179019 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 107/206] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:31.184408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169afbf-ab0b-45f7-afab-a690f66bfa49 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:31.257631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:31.317065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169afbf-ab0b-45f7-afab-a690f66bfa49 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:31.354442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a8875-554e-4d04-8ee4-052758e76b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:31.355101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a8875-554e-4d04-8ee4-052758e76b58 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:31.355388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a8875-554e-4d04-8ee4-052758e76b58 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:31.355671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a8875-554e-4d04-8ee4-052758e76b58 None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:31.356929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a8875-554e-4d04-8ee4-052758e76b58 None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:31.358490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a8875-554e-4d04-8ee4-052758e76b58 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:31.360517 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 100/207] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:31 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:31.366756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347208a-b3d4-4738-8aea-2d300aeaf7b3 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:31.413403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:31.488742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347208a-b3d4-4738-8aea-2d300aeaf7b3 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:31.525001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a6e15-f547-406f-b6db-c582e49c9e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:31.525673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a6e15-f547-406f-b6db-c582e49c9e0f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:31.525962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a6e15-f547-406f-b6db-c582e49c9e0f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:31.526246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a6e15-f547-406f-b6db-c582e49c9e0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:31.527036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a6e15-f547-406f-b6db-c582e49c9e0f None admin] RBAC: Adding query filter params (role.id=2eaf93a02e4a40a0b74e2a9fcbb182d0, scope.project.id=bb6a16a02b534f94be33eea4c77eea11, user.id=f5be54078c4f4fe88902ba63c6eca5ea) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:31.527351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a6e15-f547-406f-b6db-c582e49c9e0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:31.528813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a6e15-f547-406f-b6db-c582e49c9e0f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:31.535410 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 108/208] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:31 2022] GET /identity/v3/role_assignments?user.id=f5be54078c4f4fe88902ba63c6eca5ea&scope.project.id=bb6a16a02b534f94be33eea4c77eea11&role.id=2eaf93a02e4a40a0b74e2a9fcbb182d0 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:32.623074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d13831-da86-4659-806d-d89518fab96d None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:32.623593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d13831-da86-4659-806d-d89518fab96d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:32.623593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d13831-da86-4659-806d-d89518fab96d None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:32.624994 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 101/209] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:32.627846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4d097-2db9-476c-bd88-7778e703b37d None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:32.628260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4d097-2db9-476c-bd88-7778e703b37d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:32.628426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4d097-2db9-476c-bd88-7778e703b37d None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:32.630028 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 109/210] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:32.630758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba0967-b501-4f5d-b477-e8a63c16dbf7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:32.631186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba0967-b501-4f5d-b477-e8a63c16dbf7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:32.631476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba0967-b501-4f5d-b477-e8a63c16dbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:32.634912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71ffe8-a3a1-4636-9385-9f862195b58e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:32.635128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71ffe8-a3a1-4636-9385-9f862195b58e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:32.635461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71ffe8-a3a1-4636-9385-9f862195b58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:32.650513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0eba0967-b501-4f5d-b477-e8a63c16dbf7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:32.651712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0eba0967-b501-4f5d-b477-e8a63c16dbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:32.654583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3c71ffe8-a3a1-4636-9385-9f862195b58e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:32.655761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3c71ffe8-a3a1-4636-9385-9f862195b58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:32.735300 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 102/211] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:32.738889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22502a45-0ac0-4f7b-8c0a-1c0cbd04f2a3 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:32.738889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22502a45-0ac0-4f7b-8c0a-1c0cbd04f2a3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:32.738889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22502a45-0ac0-4f7b-8c0a-1c0cbd04f2a3 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:32.739143 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 103/212] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:32.741066 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 110/213] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:32.745322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6153f3-b985-40da-af4c-dcc39d952507 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:32.745614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6153f3-b985-40da-af4c-dcc39d952507 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:32.745895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6153f3-b985-40da-af4c-dcc39d952507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:32.765572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8c6153f3-b985-40da-af4c-dcc39d952507 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:32.766777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8c6153f3-b985-40da-af4c-dcc39d952507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:32.819259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:32.838282 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 104/214] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:33.079180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71ffe8-a3a1-4636-9385-9f862195b58e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:33.098715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6153f3-b985-40da-af4c-dcc39d952507 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:33.113060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:33.203082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71ffe8-a3a1-4636-9385-9f862195b58e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:33.227739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6153f3-b985-40da-af4c-dcc39d952507 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:33.239633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047d372-7e5c-497c-a4e6-f51ab3749e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:33.240267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047d372-7e5c-497c-a4e6-f51ab3749e26 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:33.240558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047d372-7e5c-497c-a4e6-f51ab3749e26 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:33.240836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047d372-7e5c-497c-a4e6-f51ab3749e26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:33.242189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047d372-7e5c-497c-a4e6-f51ab3749e26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:33.244823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047d372-7e5c-497c-a4e6-f51ab3749e26 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:33.248958 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 111/215] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:33.249935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047d372-7e5c-497c-a4e6-f51ab3749e26 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:33.269397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635740a-276f-4058-a972-83dd4fab381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:33.270047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635740a-276f-4058-a972-83dd4fab381f None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:33.270338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635740a-276f-4058-a972-83dd4fab381f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:33.270615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635740a-276f-4058-a972-83dd4fab381f None admin] PATH_INFO: `/v3/roles/member` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:33.281266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635740a-276f-4058-a972-83dd4fab381f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-3635740a-276f-4058-a972-83dd4fab381f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:33.286721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:33.287597 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:33.289954 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 105/216] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:32:33.293209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635740a-276f-4058-a972-83dd4fab381f None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:33.339703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:33.351196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:33.355335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047d372-7e5c-497c-a4e6-f51ab3749e26 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:33.387982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea15da-3c44-45cd-88d7-f9e3528cbd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:33.388641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea15da-3c44-45cd-88d7-f9e3528cbd50 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:33.388957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea15da-3c44-45cd-88d7-f9e3528cbd50 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:33.389239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea15da-3c44-45cd-88d7-f9e3528cbd50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:33.394086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635740a-276f-4058-a972-83dd4fab381f None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:33.399719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea15da-3c44-45cd-88d7-f9e3528cbd50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-b3ea15da-3c44-45cd-88d7-f9e3528cbd50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:33.404950 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:33.405796 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:33.406637 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 112/217] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:32:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:33.407262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea15da-3c44-45cd-88d7-f9e3528cbd50 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:33.428978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888797e0-fe56-4619-89d2-44040a23b95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:33.429593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888797e0-fe56-4619-89d2-44040a23b95b None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:33.429877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888797e0-fe56-4619-89d2-44040a23b95b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:33.430152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888797e0-fe56-4619-89d2-44040a23b95b None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:33.430857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888797e0-fe56-4619-89d2-44040a23b95b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:33.432345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888797e0-fe56-4619-89d2-44040a23b95b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:33.456737 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 106/218] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:32:33 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:33.458021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6c1fb-8282-4a9a-a29c-7281e1905abc None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:33.458286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6c1fb-8282-4a9a-a29c-7281e1905abc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:33.458560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6c1fb-8282-4a9a-a29c-7281e1905abc None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:33.459791 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 107/219] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:33.460540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6c1fb-8282-4a9a-a29c-7281e1905abc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:33.537784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea15da-3c44-45cd-88d7-f9e3528cbd50 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:33.577171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:33.577990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9bf07-cd9c-40de-a186-8ca0e0617a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:33.578707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9bf07-cd9c-40de-a186-8ca0e0617a90 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:33.579066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9bf07-cd9c-40de-a186-8ca0e0617a90 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:33.579322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9bf07-cd9c-40de-a186-8ca0e0617a90 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:33.580117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9bf07-cd9c-40de-a186-8ca0e0617a90 None admin] RBAC: Adding query filter params (name=member) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:33.580511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9bf07-cd9c-40de-a186-8ca0e0617a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:33.582268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9bf07-cd9c-40de-a186-8ca0e0617a90 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:33.583047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6c1fb-8282-4a9a-a29c-7281e1905abc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:33.594882 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 113/220] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:33.601590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae286c2-b50b-43bc-8fb5-c75cd29e4148 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:33.601941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae286c2-b50b-43bc-8fb5-c75cd29e4148 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:33.602228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae286c2-b50b-43bc-8fb5-c75cd29e4148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:33.621896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1ae286c2-b50b-43bc-8fb5-c75cd29e4148 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:33.623355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1ae286c2-b50b-43bc-8fb5-c75cd29e4148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:33.625545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5679538-56fa-4ffd-acda-432ae0330c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:33.626399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5679538-56fa-4ffd-acda-432ae0330c9c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:33.627565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5679538-56fa-4ffd-acda-432ae0330c9c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:33.627847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5679538-56fa-4ffd-acda-432ae0330c9c None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:33.628524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5679538-56fa-4ffd-acda-432ae0330c9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:33.628834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5679538-56fa-4ffd-acda-432ae0330c9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:33.630241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5679538-56fa-4ffd-acda-432ae0330c9c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:33.638649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:33.641037 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 108/221] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:33.642183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5679538-56fa-4ffd-acda-432ae0330c9c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:33.703076 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 114/222] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:33.704536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae286c2-b50b-43bc-8fb5-c75cd29e4148 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:33.779055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5679538-56fa-4ffd-acda-432ae0330c9c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:33.827997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b68de-e338-4a61-98ae-70023244c078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:33.828962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b68de-e338-4a61-98ae-70023244c078 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:33.829387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b68de-e338-4a61-98ae-70023244c078 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:33.829810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b68de-e338-4a61-98ae-70023244c078 None admin] PATH_INFO: `/v3/users/d1f0272a3bd84524872bb6491d518fdf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:33.837473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae286c2-b50b-43bc-8fb5-c75cd29e4148 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:33.854200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b68de-e338-4a61-98ae-70023244c078 None admin] RBAC: Authorizing `identity:get_user(user_id=d1f0272a3bd84524872bb6491d518fdf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:33.855773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b68de-e338-4a61-98ae-70023244c078 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:33.858243 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 109/223] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:33 2022] GET /identity/v3/users/d1f0272a3bd84524872bb6491d518fdf => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:33.863568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b68de-e338-4a61-98ae-70023244c078 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:33.884108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e47a5d-c6c9-4d42-86e8-de0d55323455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:33.884746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e47a5d-c6c9-4d42-86e8-de0d55323455 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:33.885112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e47a5d-c6c9-4d42-86e8-de0d55323455 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:33.885383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e47a5d-c6c9-4d42-86e8-de0d55323455 None admin] PATH_INFO: `/v3/users/496773170381441b9067354f430d9aba` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:33.886522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e47a5d-c6c9-4d42-86e8-de0d55323455 None admin] RBAC: Authorizing `identity:get_user(user_id=496773170381441b9067354f430d9aba)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:33.887954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e47a5d-c6c9-4d42-86e8-de0d55323455 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:33.889713 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 115/224] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:33 2022] GET /identity/v3/users/496773170381441b9067354f430d9aba => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:33.895144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:33.895588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e47a5d-c6c9-4d42-86e8-de0d55323455 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:33.916773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:34.025793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b68de-e338-4a61-98ae-70023244c078 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:34.052250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e47a5d-c6c9-4d42-86e8-de0d55323455 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:34.093998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5623aa8-62da-41f0-98d3-57d9b874c427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:34.094670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5623aa8-62da-41f0-98d3-57d9b874c427 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:34.094957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5623aa8-62da-41f0-98d3-57d9b874c427 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:34.095229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5623aa8-62da-41f0-98d3-57d9b874c427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:34.115746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680074e-730d-49c0-a107-e7e6582d934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:34.116309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680074e-730d-49c0-a107-e7e6582d934a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:34.116593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680074e-730d-49c0-a107-e7e6582d934a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:34.116868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680074e-730d-49c0-a107-e7e6582d934a None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:34.118073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680074e-730d-49c0-a107-e7e6582d934a None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:34.119828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680074e-730d-49c0-a107-e7e6582d934a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:34.121881 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 116/225] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:33 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:34.122993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680074e-730d-49c0-a107-e7e6582d934a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:34.193579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5623aa8-62da-41f0-98d3-57d9b874c427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:34.195108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5623aa8-62da-41f0-98d3-57d9b874c427 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:34.217729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:34.223189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:34.239485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680074e-730d-49c0-a107-e7e6582d934a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:34.288353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea9836-8621-438b-ac2e-4f88e552b8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:34.289026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea9836-8621-438b-ac2e-4f88e552b8f6 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:34.289313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea9836-8621-438b-ac2e-4f88e552b8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:34.289594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea9836-8621-438b-ac2e-4f88e552b8f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:34.303913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea9836-8621-438b-ac2e-4f88e552b8f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:34.309073 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 110/226] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-50ea9836-8621-438b-ac2e-4f88e552b8f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:34.309507 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:34.310353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:34.311104 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 117/227] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:32:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:34.311743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea9836-8621-438b-ac2e-4f88e552b8f6 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:34.313001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5623aa8-62da-41f0-98d3-57d9b874c427 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:34.430646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea9836-8621-438b-ac2e-4f88e552b8f6 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:34.436978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5623aa8-62da-41f0-98d3-57d9b874c427 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:34.469274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:34.469687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676320d8-15ab-4fd2-ac4b-5d4ec07e64a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:34.470538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676320d8-15ab-4fd2-ac4b-5d4ec07e64a8 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:34.471578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676320d8-15ab-4fd2-ac4b-5d4ec07e64a8 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:34.472082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676320d8-15ab-4fd2-ac4b-5d4ec07e64a8 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:34.473555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676320d8-15ab-4fd2-ac4b-5d4ec07e64a8 None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:34.475212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676320d8-15ab-4fd2-ac4b-5d4ec07e64a8 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:34.481242 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 118/228] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:34 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:34.482389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676320d8-15ab-4fd2-ac4b-5d4ec07e64a8 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:34.487120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec45a8-4799-4a73-a2c3-3647fae18629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:34.487832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec45a8-4799-4a73-a2c3-3647fae18629 None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:34.489055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec45a8-4799-4a73-a2c3-3647fae18629 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:34.491235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec45a8-4799-4a73-a2c3-3647fae18629 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4/users/496773170381441b9067354f430d9aba/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:34.494187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec45a8-4799-4a73-a2c3-3647fae18629 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a64615765ab4a01a30fc0023bb081c4, user_id=496773170381441b9067354f430d9aba, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:34.496172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec45a8-4799-4a73-a2c3-3647fae18629 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:34.504847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:34.520972 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 111/229] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:32:34 2022] PUT /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4/users/496773170381441b9067354f430d9aba/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:32:34.520972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec45a8-4799-4a73-a2c3-3647fae18629 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:34.653931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676320d8-15ab-4fd2-ac4b-5d4ec07e64a8 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:34.693134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f8ed4-80e2-4b70-a435-e3df74fbae26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:34.693745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f8ed4-80e2-4b70-a435-e3df74fbae26 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:34.693952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f8ed4-80e2-4b70-a435-e3df74fbae26 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:34.694232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f8ed4-80e2-4b70-a435-e3df74fbae26 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:34.694910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f8ed4-80e2-4b70-a435-e3df74fbae26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:34.695220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f8ed4-80e2-4b70-a435-e3df74fbae26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:34.696739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f8ed4-80e2-4b70-a435-e3df74fbae26 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:34.708225 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 119/230] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:32:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:34.718258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f8ed4-80e2-4b70-a435-e3df74fbae26 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:34.725184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec45a8-4799-4a73-a2c3-3647fae18629 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:34.775908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538ce80-f4e9-4df8-813d-db193ce30e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:34.777336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538ce80-f4e9-4df8-813d-db193ce30e8b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:34.778982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538ce80-f4e9-4df8-813d-db193ce30e8b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:34.779896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538ce80-f4e9-4df8-813d-db193ce30e8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:34.782157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538ce80-f4e9-4df8-813d-db193ce30e8b None admin] RBAC: Adding query filter params (role.id=d167c236cee64ede8d62bce075f83fff, scope.project.id=0a64615765ab4a01a30fc0023bb081c4, user.id=d1f0272a3bd84524872bb6491d518fdf) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:34.784346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538ce80-f4e9-4df8-813d-db193ce30e8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:34.787917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538ce80-f4e9-4df8-813d-db193ce30e8b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:34.800840 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 112/231] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:34 2022] GET /identity/v3/role_assignments?user.id=d1f0272a3bd84524872bb6491d518fdf&scope.project.id=0a64615765ab4a01a30fc0023bb081c4&role.id=d167c236cee64ede8d62bce075f83fff => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:34.857274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:34.878919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f8ed4-80e2-4b70-a435-e3df74fbae26 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:34.921017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3d86d-922d-4f08-b5ce-80a8b21a8212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:34.921925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3d86d-922d-4f08-b5ce-80a8b21a8212 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:34.922307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3d86d-922d-4f08-b5ce-80a8b21a8212 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:34.922679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3d86d-922d-4f08-b5ce-80a8b21a8212 None admin] PATH_INFO: `/v3/users/496773170381441b9067354f430d9aba` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:34.924660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3d86d-922d-4f08-b5ce-80a8b21a8212 None admin] RBAC: Authorizing `identity:get_user(user_id=496773170381441b9067354f430d9aba)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:34.926266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3d86d-922d-4f08-b5ce-80a8b21a8212 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:34.928916 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 120/232] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:34 2022] GET /identity/v3/users/496773170381441b9067354f430d9aba => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:34.937015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538ce80-f4e9-4df8-813d-db193ce30e8b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:34.990332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:35.051302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538ce80-f4e9-4df8-813d-db193ce30e8b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:35.088601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b157fb-e9d0-4094-ad1a-619875c00813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:35.089471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b157fb-e9d0-4094-ad1a-619875c00813 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:35.089858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b157fb-e9d0-4094-ad1a-619875c00813 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:35.090821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b157fb-e9d0-4094-ad1a-619875c00813 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:35.092142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b157fb-e9d0-4094-ad1a-619875c00813 None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:35.093983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b157fb-e9d0-4094-ad1a-619875c00813 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:35.096528 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 113/233] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:34 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:35.107336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3d86d-922d-4f08-b5ce-80a8b21a8212 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:35.223863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3d86d-922d-4f08-b5ce-80a8b21a8212 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:35.266544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e48711-834e-40dd-84d1-b20242455cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:35.267599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e48711-834e-40dd-84d1-b20242455cca None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:35.268581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e48711-834e-40dd-84d1-b20242455cca None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:35.269340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e48711-834e-40dd-84d1-b20242455cca None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4/users/496773170381441b9067354f430d9aba/roles/542f58433b2747d2abd5d405ebabdea1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:35.274702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e48711-834e-40dd-84d1-b20242455cca None admin] RBAC: Authorizing `identity:create_grant(project_id=0a64615765ab4a01a30fc0023bb081c4, user_id=496773170381441b9067354f430d9aba, role_id=542f58433b2747d2abd5d405ebabdea1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:35.276468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e48711-834e-40dd-84d1-b20242455cca None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:35.312010 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 121/234] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:32:35 2022] PUT /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4/users/496773170381441b9067354f430d9aba/roles/542f58433b2747d2abd5d405ebabdea1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:32:36.204089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb781dc-bba1-48bd-b087-a97e4b002c2e None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:36.204592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb781dc-bba1-48bd-b087-a97e4b002c2e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:36.204834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb781dc-bba1-48bd-b087-a97e4b002c2e None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:36.206761 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 114/235] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:36.212284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140d184-2ef9-4ea6-a128-01525f248c7e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:36.213144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140d184-2ef9-4ea6-a128-01525f248c7e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:36.213646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140d184-2ef9-4ea6-a128-01525f248c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:36.221146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:36.233440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b140d184-2ef9-4ea6-a128-01525f248c7e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:36.234810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b140d184-2ef9-4ea6-a128-01525f248c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:36.322393 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 122/236] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:36.474266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46031a4-b633-4d28-903b-843b4c58763c None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:36.474623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46031a4-b633-4d28-903b-843b4c58763c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:36.474828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46031a4-b633-4d28-903b-843b4c58763c None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:36.476405 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 115/237] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:36.486393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81657502-069d-46d8-9bc4-480628677879 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:36.486904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81657502-069d-46d8-9bc4-480628677879 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:36.487381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81657502-069d-46d8-9bc4-480628677879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:36.518353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-81657502-069d-46d8-9bc4-480628677879 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:36.519843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-81657502-069d-46d8-9bc4-480628677879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:36.603209 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 123/238] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:36.902895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46031a4-b633-4d28-903b-843b4c58763c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:37.049230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46031a4-b633-4d28-903b-843b4c58763c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:37.062653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:37.068730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81657502-069d-46d8-9bc4-480628677879 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:37.093417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c3575-3556-4165-9e7b-251a0782a43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:37.094209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c3575-3556-4165-9e7b-251a0782a43c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:37.094502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c3575-3556-4165-9e7b-251a0782a43c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:37.094784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c3575-3556-4165-9e7b-251a0782a43c None admin] PATH_INFO: `/v3/domains/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:37.096083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c3575-3556-4165-9e7b-251a0782a43c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:37.097750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c3575-3556-4165-9e7b-251a0782a43c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:37.099645 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 116/239] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:37.104822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c3575-3556-4165-9e7b-251a0782a43c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:37.181371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:37.217405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81657502-069d-46d8-9bc4-480628677879 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:37.281258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d6982-1a0c-4989-a7f1-e8ee5f9c8a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:37.281925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6982-1a0c-4989-a7f1-e8ee5f9c8a5e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:37.282213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6982-1a0c-4989-a7f1-e8ee5f9c8a5e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:37.282492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6982-1a0c-4989-a7f1-e8ee5f9c8a5e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:37.304397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d6982-1a0c-4989-a7f1-e8ee5f9c8a5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-b88d6982-1a0c-4989-a7f1-e8ee5f9c8a5e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:37.309802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:37.310659 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:37.311435 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 124/240] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:37.318324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c3575-3556-4165-9e7b-251a0782a43c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:37.320569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d6982-1a0c-4989-a7f1-e8ee5f9c8a5e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:37.383598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01b26d-0e40-428a-b54b-9b9104ac7742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:37.384222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01b26d-0e40-428a-b54b-9b9104ac7742 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:37.384513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01b26d-0e40-428a-b54b-9b9104ac7742 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:37.384796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01b26d-0e40-428a-b54b-9b9104ac7742 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:37.385498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01b26d-0e40-428a-b54b-9b9104ac7742 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:37.386948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01b26d-0e40-428a-b54b-9b9104ac7742 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:37.430537 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 117/241] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:32:37 2022] POST /identity/v3/users => generated 293 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:37.432752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5b3a2-e52f-44b4-a732-056cdb05cb1e None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:37.433075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5b3a2-e52f-44b4-a732-056cdb05cb1e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:37.433653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5b3a2-e52f-44b4-a732-056cdb05cb1e None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:37.436117 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 118/242] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:37.441940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b745f-3419-45f5-aa3e-96930a37e433 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:37.442262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b745f-3419-45f5-aa3e-96930a37e433 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:37.442516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b745f-3419-45f5-aa3e-96930a37e433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:37.466400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ff2b745f-3419-45f5-aa3e-96930a37e433 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:37.468033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ff2b745f-3419-45f5-aa3e-96930a37e433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:37.494379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d6982-1a0c-4989-a7f1-e8ee5f9c8a5e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:37.509289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:37.533578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:37.550350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7a1f8-9476-48a7-b73f-0408d486d7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:37.551083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7a1f8-9476-48a7-b73f-0408d486d7f1 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:37.551391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7a1f8-9476-48a7-b73f-0408d486d7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:37.551672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7a1f8-9476-48a7-b73f-0408d486d7f1 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:37.552490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac7a1f8-9476-48a7-b73f-0408d486d7f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:37.552821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac7a1f8-9476-48a7-b73f-0408d486d7f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:37.554278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac7a1f8-9476-48a7-b73f-0408d486d7f1 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:37.554868 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 119/243] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:37.562942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf1449-550f-41d8-94d9-f7c4b435389b None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:37.563466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf1449-550f-41d8-94d9-f7c4b435389b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:37.563947 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 125/244] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:37.564195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf1449-550f-41d8-94d9-f7c4b435389b None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:37.566337 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 120/245] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:37.569980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7a1f8-9476-48a7-b73f-0408d486d7f1 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:37.574914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3a330-b86f-4f16-9655-892b842b2a3e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:37.575079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3a330-b86f-4f16-9655-892b842b2a3e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:37.575291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3a330-b86f-4f16-9655-892b842b2a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:37.613134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-79a3a330-b86f-4f16-9655-892b842b2a3e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:37.614368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-79a3a330-b86f-4f16-9655-892b842b2a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:37.698467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7a1f8-9476-48a7-b73f-0408d486d7f1 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:37.703089 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 121/246] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:37.747659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622451a-f94e-47eb-9fe1-427c802e16ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:37.748313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622451a-f94e-47eb-9fe1-427c802e16ba None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:37.748594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622451a-f94e-47eb-9fe1-427c802e16ba None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:37.748867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622451a-f94e-47eb-9fe1-427c802e16ba None admin] PATH_INFO: `/v3/users/bbc73583f3944c2f9b1dd82ee2552dc6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:37.765903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622451a-f94e-47eb-9fe1-427c802e16ba None admin] RBAC: Authorizing `identity:get_user(user_id=bbc73583f3944c2f9b1dd82ee2552dc6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:37.767735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622451a-f94e-47eb-9fe1-427c802e16ba None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:37.769978 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 126/247] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:37 2022] GET /identity/v3/users/bbc73583f3944c2f9b1dd82ee2552dc6 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:37.775504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3a330-b86f-4f16-9655-892b842b2a3e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:37.878538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:37.882903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3a330-b86f-4f16-9655-892b842b2a3e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:37.918333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fdd833-ec85-44fe-807d-2822ec2ced61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:37.919320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdd833-ec85-44fe-807d-2822ec2ced61 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:37.919811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdd833-ec85-44fe-807d-2822ec2ced61 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:37.920291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdd833-ec85-44fe-807d-2822ec2ced61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:38.002649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fdd833-ec85-44fe-807d-2822ec2ced61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:38.004059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fdd833-ec85-44fe-807d-2822ec2ced61 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:38.015375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622451a-f94e-47eb-9fe1-427c802e16ba None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:38.080024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:38.136635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:38.144469 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 122/248] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 20:32:38.145812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fdd833-ec85-44fe-807d-2822ec2ced61 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:38.162037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622451a-f94e-47eb-9fe1-427c802e16ba None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:38.208888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd5a20-f381-4b68-856e-7760aa3c722f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:38.209567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd5a20-f381-4b68-856e-7760aa3c722f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:38.209855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd5a20-f381-4b68-856e-7760aa3c722f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:38.210136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd5a20-f381-4b68-856e-7760aa3c722f None admin] PATH_INFO: `/v3/roles/member` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:38.223453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd5a20-f381-4b68-856e-7760aa3c722f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-e0dd5a20-f381-4b68-856e-7760aa3c722f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:38.228671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:38.229675 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:38.230731 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 127/249] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:38.231513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd5a20-f381-4b68-856e-7760aa3c722f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:38.267576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fdd833-ec85-44fe-807d-2822ec2ced61 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:38.309333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387eadc-4883-4ff9-bafe-04d4c5b52936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:38.310290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387eadc-4883-4ff9-bafe-04d4c5b52936 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:38.310846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387eadc-4883-4ff9-bafe-04d4c5b52936 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:38.311137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387eadc-4883-4ff9-bafe-04d4c5b52936 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:38.325301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0387eadc-4883-4ff9-bafe-04d4c5b52936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-0387eadc-4883-4ff9-bafe-04d4c5b52936 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:38.332557 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:38.333456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:38.334485 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 123/250] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:32:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:38.336442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdb127-787e-4802-a7f0-156621f1bf17 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:38.336878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdb127-787e-4802-a7f0-156621f1bf17 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:38.337401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdb127-787e-4802-a7f0-156621f1bf17 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:38.339673 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 124/251] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:38.340575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdb127-787e-4802-a7f0-156621f1bf17 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:38.362126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd5a20-f381-4b68-856e-7760aa3c722f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:38.365123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:38.407125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e929c45-c7cc-4ceb-b3f8-6a8815b72570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:38.407772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e929c45-c7cc-4ceb-b3f8-6a8815b72570 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:38.408580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e929c45-c7cc-4ceb-b3f8-6a8815b72570 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:38.408865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e929c45-c7cc-4ceb-b3f8-6a8815b72570 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:38.410114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e929c45-c7cc-4ceb-b3f8-6a8815b72570 None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:38.411585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e929c45-c7cc-4ceb-b3f8-6a8815b72570 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:38.413576 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 128/252] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:38 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:38.414447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e929c45-c7cc-4ceb-b3f8-6a8815b72570 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:38.449287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:38.464118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdb127-787e-4802-a7f0-156621f1bf17 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:38.504064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2767b40-c0b1-4b27-8726-fe7360cf77d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:38.505239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2767b40-c0b1-4b27-8726-fe7360cf77d2 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:38.505722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2767b40-c0b1-4b27-8726-fe7360cf77d2 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:38.506301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2767b40-c0b1-4b27-8726-fe7360cf77d2 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:38.507156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2767b40-c0b1-4b27-8726-fe7360cf77d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:38.507656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2767b40-c0b1-4b27-8726-fe7360cf77d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:38.509428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2767b40-c0b1-4b27-8726-fe7360cf77d2 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:38.518961 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 125/253] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:38.520590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474c7ab-0cea-4595-8aa1-fc7fb2dfff39 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:38.521107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474c7ab-0cea-4595-8aa1-fc7fb2dfff39 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:38.521617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474c7ab-0cea-4595-8aa1-fc7fb2dfff39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:38.526682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e929c45-c7cc-4ceb-b3f8-6a8815b72570 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:38.540800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3474c7ab-0cea-4595-8aa1-fc7fb2dfff39 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:38.542549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3474c7ab-0cea-4595-8aa1-fc7fb2dfff39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:38.563159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74569cff-5bed-411e-b94e-bcd8d5729d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:38.563682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74569cff-5bed-411e-b94e-bcd8d5729d79 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:38.563970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74569cff-5bed-411e-b94e-bcd8d5729d79 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:38.564253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74569cff-5bed-411e-b94e-bcd8d5729d79 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:38.565666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74569cff-5bed-411e-b94e-bcd8d5729d79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:38.565976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74569cff-5bed-411e-b94e-bcd8d5729d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:38.567506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74569cff-5bed-411e-b94e-bcd8d5729d79 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:38.576384 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 129/254] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:38.580918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74569cff-5bed-411e-b94e-bcd8d5729d79 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:38.613080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:38.624235 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 126/255] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:38.625704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474c7ab-0cea-4595-8aa1-fc7fb2dfff39 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:38.684092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74569cff-5bed-411e-b94e-bcd8d5729d79 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:38.685251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:38.722024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9eb4c-c858-4f49-b8b4-58e76c046101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:38.722897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9eb4c-c858-4f49-b8b4-58e76c046101 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:38.723191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9eb4c-c858-4f49-b8b4-58e76c046101 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:38.723472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9eb4c-c858-4f49-b8b4-58e76c046101 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:38.724210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9eb4c-c858-4f49-b8b4-58e76c046101 None admin] RBAC: Adding query filter params (role.id=2eaf93a02e4a40a0b74e2a9fcbb182d0, scope.project.id=0a64615765ab4a01a30fc0023bb081c4, user.id=bbc73583f3944c2f9b1dd82ee2552dc6) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:38.724526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9eb4c-c858-4f49-b8b4-58e76c046101 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:38.726517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9eb4c-c858-4f49-b8b4-58e76c046101 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:38.736966 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 130/256] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:38 2022] GET /identity/v3/role_assignments?user.id=bbc73583f3944c2f9b1dd82ee2552dc6&scope.project.id=0a64615765ab4a01a30fc0023bb081c4&role.id=2eaf93a02e4a40a0b74e2a9fcbb182d0 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:38.736966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9eb4c-c858-4f49-b8b4-58e76c046101 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:38.749340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474c7ab-0cea-4595-8aa1-fc7fb2dfff39 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:38.822378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b72ab6-8c80-4de4-903d-baedb22f0e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:38.823380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b72ab6-8c80-4de4-903d-baedb22f0e52 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:38.823999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b72ab6-8c80-4de4-903d-baedb22f0e52 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:38.825548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b72ab6-8c80-4de4-903d-baedb22f0e52 None admin] PATH_INFO: `/v3/users/d1f0272a3bd84524872bb6491d518fdf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:38.827056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b72ab6-8c80-4de4-903d-baedb22f0e52 None admin] RBAC: Authorizing `identity:get_user(user_id=d1f0272a3bd84524872bb6491d518fdf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:38.829751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b72ab6-8c80-4de4-903d-baedb22f0e52 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:38.832952 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 127/257] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:38 2022] GET /identity/v3/users/d1f0272a3bd84524872bb6491d518fdf => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:38.842073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b72ab6-8c80-4de4-903d-baedb22f0e52 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:38.894422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:38.901750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9eb4c-c858-4f49-b8b4-58e76c046101 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:38.967120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff571434-a649-458c-8503-745ed6bde7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:38.967120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff571434-a649-458c-8503-745ed6bde7c3 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:38.967120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff571434-a649-458c-8503-745ed6bde7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:38.967120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff571434-a649-458c-8503-745ed6bde7c3 None admin] PATH_INFO: `/v3/users/496773170381441b9067354f430d9aba` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:38.969906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff571434-a649-458c-8503-745ed6bde7c3 None admin] RBAC: Authorizing `identity:get_user(user_id=496773170381441b9067354f430d9aba)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:38.972062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff571434-a649-458c-8503-745ed6bde7c3 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:38.974123 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 131/258] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:38 2022] GET /identity/v3/users/496773170381441b9067354f430d9aba => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:38.979962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff571434-a649-458c-8503-745ed6bde7c3 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:39.020489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b72ab6-8c80-4de4-903d-baedb22f0e52 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:39.055885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:39.091746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1c56b-4185-45f0-bb2b-df0fe4a577a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:39.092576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1c56b-4185-45f0-bb2b-df0fe4a577a4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:39.092866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1c56b-4185-45f0-bb2b-df0fe4a577a4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:39.093295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1c56b-4185-45f0-bb2b-df0fe4a577a4 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:39.094601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1c56b-4185-45f0-bb2b-df0fe4a577a4 None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:39.096597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1c56b-4185-45f0-bb2b-df0fe4a577a4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:39.103823 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 128/259] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:38 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:39.104891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1c56b-4185-45f0-bb2b-df0fe4a577a4 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:39.219221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff571434-a649-458c-8503-745ed6bde7c3 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:39.233538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1c56b-4185-45f0-bb2b-df0fe4a577a4 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:39.270749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc74d0-0a85-4a41-a46d-9c30c6cd3aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:39.271533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc74d0-0a85-4a41-a46d-9c30c6cd3aab None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:39.271821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc74d0-0a85-4a41-a46d-9c30c6cd3aab None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:39.272096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc74d0-0a85-4a41-a46d-9c30c6cd3aab None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4/users/d1f0272a3bd84524872bb6491d518fdf/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:39.274595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc74d0-0a85-4a41-a46d-9c30c6cd3aab None admin] RBAC: Authorizing `identity:create_grant(project_id=0a64615765ab4a01a30fc0023bb081c4, user_id=d1f0272a3bd84524872bb6491d518fdf, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:39.276101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc74d0-0a85-4a41-a46d-9c30c6cd3aab None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:39.292574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31061abf-1204-415c-bf47-24f7f8d904a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:39.293629 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 129/260] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:32:39 2022] PUT /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4/users/d1f0272a3bd84524872bb6491d518fdf/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:32:39.294192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31061abf-1204-415c-bf47-24f7f8d904a9 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:39.294652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc74d0-0a85-4a41-a46d-9c30c6cd3aab None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:39.294868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31061abf-1204-415c-bf47-24f7f8d904a9 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:39.295331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31061abf-1204-415c-bf47-24f7f8d904a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:39.354589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:39.387844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:39.421858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc74d0-0a85-4a41-a46d-9c30c6cd3aab None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:39.476064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf95dae-7070-4414-b516-0fa073d2fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:39.477154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf95dae-7070-4414-b516-0fa073d2fc2e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:39.477654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf95dae-7070-4414-b516-0fa073d2fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:39.479443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf95dae-7070-4414-b516-0fa073d2fc2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:39.484832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31061abf-1204-415c-bf47-24f7f8d904a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:39.488362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31061abf-1204-415c-bf47-24f7f8d904a9 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:39.507591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf95dae-7070-4414-b516-0fa073d2fc2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-bbf95dae-7070-4414-b516-0fa073d2fc2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:39.518142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:39.519183 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:39.519964 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:39.519964 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:39.519964 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:39.519964 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:39.519964 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:39.519964 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:39.519964 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:39.519964 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:39.519964 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:39.519964 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:39.519964 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:39.519964 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:39.522373 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 130/261] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:32:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:39.528144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf95dae-7070-4414-b516-0fa073d2fc2e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:39.661191 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 132/262] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 20:32:39.664143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31061abf-1204-415c-bf47-24f7f8d904a9 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:39.746529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:39.791110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31061abf-1204-415c-bf47-24f7f8d904a9 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:39.793130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf95dae-7070-4414-b516-0fa073d2fc2e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:39.797856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:39.833509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0940a-a44a-4e2c-ba83-b5ea88b06a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:39.834643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0940a-a44a-4e2c-ba83-b5ea88b06a7a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:39.835401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0940a-a44a-4e2c-ba83-b5ea88b06a7a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:39.835911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0940a-a44a-4e2c-ba83-b5ea88b06a7a None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:39.837278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0940a-a44a-4e2c-ba83-b5ea88b06a7a None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:39.839039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0940a-a44a-4e2c-ba83-b5ea88b06a7a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:39.841393 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 133/263] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:39 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:39.848011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0940a-a44a-4e2c-ba83-b5ea88b06a7a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:39.877061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca15eb-8328-4ca2-a102-7773d72d6e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:39.877682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca15eb-8328-4ca2-a102-7773d72d6e00 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:39.877969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca15eb-8328-4ca2-a102-7773d72d6e00 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:39.878249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca15eb-8328-4ca2-a102-7773d72d6e00 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:39.878924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca15eb-8328-4ca2-a102-7773d72d6e00 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:39.879239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca15eb-8328-4ca2-a102-7773d72d6e00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:39.880616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca15eb-8328-4ca2-a102-7773d72d6e00 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:39.910987 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 131/264] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:32:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:39.916466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca15eb-8328-4ca2-a102-7773d72d6e00 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:39.963931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0940a-a44a-4e2c-ba83-b5ea88b06a7a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:40.009389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071f9f9-f35e-4b8e-91cc-85ea01a59f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:40.010010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071f9f9-f35e-4b8e-91cc-85ea01a59f23 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:40.010293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071f9f9-f35e-4b8e-91cc-85ea01a59f23 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:40.010571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071f9f9-f35e-4b8e-91cc-85ea01a59f23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:40.011304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071f9f9-f35e-4b8e-91cc-85ea01a59f23 None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=0a64615765ab4a01a30fc0023bb081c4, user.id=496773170381441b9067354f430d9aba) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:40.011649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071f9f9-f35e-4b8e-91cc-85ea01a59f23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:40.013099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071f9f9-f35e-4b8e-91cc-85ea01a59f23 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:40.015265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:40.025679 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 134/265] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:39 2022] GET /identity/v3/role_assignments?user.id=496773170381441b9067354f430d9aba&scope.project.id=0a64615765ab4a01a30fc0023bb081c4&role.id=bc768522dc984239be4a1b39a45ab437 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:40.087331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca15eb-8328-4ca2-a102-7773d72d6e00 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:40.145896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d249f5-0778-4179-a910-92728222d0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:40.146549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d249f5-0778-4179-a910-92728222d0c9 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:40.146841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d249f5-0778-4179-a910-92728222d0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:40.147118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d249f5-0778-4179-a910-92728222d0c9 None admin] PATH_INFO: `/v3/users/496773170381441b9067354f430d9aba` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:40.148312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d249f5-0778-4179-a910-92728222d0c9 None admin] RBAC: Authorizing `identity:get_user(user_id=496773170381441b9067354f430d9aba)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:40.149847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d249f5-0778-4179-a910-92728222d0c9 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:40.151540 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 132/266] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:39 2022] GET /identity/v3/users/496773170381441b9067354f430d9aba => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:40.158768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071f9f9-f35e-4b8e-91cc-85ea01a59f23 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:40.321181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071f9f9-f35e-4b8e-91cc-85ea01a59f23 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:40.364548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee8274-3956-4e5f-bcfd-97ccc35665f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:40.365308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee8274-3956-4e5f-bcfd-97ccc35665f6 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:40.365597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee8274-3956-4e5f-bcfd-97ccc35665f6 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:40.365875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee8274-3956-4e5f-bcfd-97ccc35665f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:40.436446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee8274-3956-4e5f-bcfd-97ccc35665f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:40.437965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee8274-3956-4e5f-bcfd-97ccc35665f6 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:40.441342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:40.544429 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 135/267] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:40.554158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d249f5-0778-4179-a910-92728222d0c9 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:40.602402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:40.699825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d249f5-0778-4179-a910-92728222d0c9 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:40.746552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976150a0-0279-47fb-a52d-c208e8d2a1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:40.747462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976150a0-0279-47fb-a52d-c208e8d2a1a6 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:40.747893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976150a0-0279-47fb-a52d-c208e8d2a1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:40.748315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976150a0-0279-47fb-a52d-c208e8d2a1a6 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:40.752515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976150a0-0279-47fb-a52d-c208e8d2a1a6 None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:40.754348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976150a0-0279-47fb-a52d-c208e8d2a1a6 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:40.756304 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 133/268] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:40 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:40.762529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee8274-3956-4e5f-bcfd-97ccc35665f6 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:40.885730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:40.889837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee8274-3956-4e5f-bcfd-97ccc35665f6 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:40.927087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075b8c3-ed3d-4e8c-8858-de55e832a78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:40.927708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075b8c3-ed3d-4e8c-8858-de55e832a78e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:40.927996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075b8c3-ed3d-4e8c-8858-de55e832a78e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:40.928277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075b8c3-ed3d-4e8c-8858-de55e832a78e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:40.929041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075b8c3-ed3d-4e8c-8858-de55e832a78e None admin] RBAC: Adding query filter params (role.id=542f58433b2747d2abd5d405ebabdea1, scope.project.id=0a64615765ab4a01a30fc0023bb081c4, user.id=496773170381441b9067354f430d9aba) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:40.929415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075b8c3-ed3d-4e8c-8858-de55e832a78e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:40.930865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075b8c3-ed3d-4e8c-8858-de55e832a78e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:40.937218 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 136/269] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:40 2022] GET /identity/v3/role_assignments?user.id=496773170381441b9067354f430d9aba&scope.project.id=0a64615765ab4a01a30fc0023bb081c4&role.id=542f58433b2747d2abd5d405ebabdea1 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:41.004240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49194dd1-8473-4f37-9299-b01febe137a2 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:41.004530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49194dd1-8473-4f37-9299-b01febe137a2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:41.004802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49194dd1-8473-4f37-9299-b01febe137a2 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:41.006196 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 134/270] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:41.011993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d7832-7653-4cb9-adee-421aad217646 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:41.012320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d7832-7653-4cb9-adee-421aad217646 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:41.012571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d7832-7653-4cb9-adee-421aad217646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:41.032706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a02d7832-7653-4cb9-adee-421aad217646 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:41.034622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a02d7832-7653-4cb9-adee-421aad217646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:41.112043 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 137/271] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:41.138589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3edac-74ec-4957-b1d0-360aa9d6258d None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:41.139192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3edac-74ec-4957-b1d0-360aa9d6258d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:41.139668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3edac-74ec-4957-b1d0-360aa9d6258d None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:41.141284 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 135/272] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:41.147805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe6f9d-e10a-4b92-8b73-597ed5d32d31 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:41.148162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe6f9d-e10a-4b92-8b73-597ed5d32d31 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:41.148446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe6f9d-e10a-4b92-8b73-597ed5d32d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:41.172232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a3fe6f9d-e10a-4b92-8b73-597ed5d32d31 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:41.173466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a3fe6f9d-e10a-4b92-8b73-597ed5d32d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:41.272523 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 138/273] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:41.507299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3edac-74ec-4957-b1d0-360aa9d6258d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:41.620929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3edac-74ec-4957-b1d0-360aa9d6258d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:41.622493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:41.660279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a449af7-087b-43b5-ade3-e18f2f755ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:41.660972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a449af7-087b-43b5-ade3-e18f2f755ce2 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:41.661564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a449af7-087b-43b5-ade3-e18f2f755ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:41.661564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a449af7-087b-43b5-ade3-e18f2f755ce2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:41.662828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a449af7-087b-43b5-ade3-e18f2f755ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:41.664368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a449af7-087b-43b5-ade3-e18f2f755ce2 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:41.666090 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 136/274] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:41.672239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe6f9d-e10a-4b92-8b73-597ed5d32d31 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:41.674516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a449af7-087b-43b5-ade3-e18f2f755ce2 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:41.712540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:41.799259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe6f9d-e10a-4b92-8b73-597ed5d32d31 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:41.807716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a449af7-087b-43b5-ade3-e18f2f755ce2 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:41.846044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08027203-6112-45bc-a47b-2d5d9129915b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:41.847058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08027203-6112-45bc-a47b-2d5d9129915b None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:41.847459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08027203-6112-45bc-a47b-2d5d9129915b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:41.847741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08027203-6112-45bc-a47b-2d5d9129915b None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:41.848409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08027203-6112-45bc-a47b-2d5d9129915b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:41.849909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08027203-6112-45bc-a47b-2d5d9129915b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:41.850548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2696dad-4b6a-4cc1-afe8-58edc763341c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:41.851191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2696dad-4b6a-4cc1-afe8-58edc763341c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:41.851475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2696dad-4b6a-4cc1-afe8-58edc763341c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:41.851751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2696dad-4b6a-4cc1-afe8-58edc763341c None admin] PATH_INFO: `/v3/roles/member` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:41.864115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2696dad-4b6a-4cc1-afe8-58edc763341c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-b2696dad-4b6a-4cc1-afe8-58edc763341c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:41.869003 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:41.869876 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:41.870692 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 137/275] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:41.874130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229370b-d46b-4d1d-94db-b8b580fe07bb None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:41.874677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229370b-d46b-4d1d-94db-b8b580fe07bb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:41.875194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229370b-d46b-4d1d-94db-b8b580fe07bb None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:41.875449 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 139/276] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:32:41 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:41.877483 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 138/277] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:41.882387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08027203-6112-45bc-a47b-2d5d9129915b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:41.883373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b378c9-6446-472c-931a-830b7ff4a7d2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:41.883899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b378c9-6446-472c-931a-830b7ff4a7d2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:41.884394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b378c9-6446-472c-931a-830b7ff4a7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:41.916622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-95b378c9-6446-472c-931a-830b7ff4a7d2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:41.918821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-95b378c9-6446-472c-931a-830b7ff4a7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:41.958382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:41.980385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:42.012306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08027203-6112-45bc-a47b-2d5d9129915b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:42.030286 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 139/278] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:42.060187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f26017-4159-4a50-8425-4658024bb240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:42.060719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f26017-4159-4a50-8425-4658024bb240 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:42.061052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f26017-4159-4a50-8425-4658024bb240 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:42.061331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f26017-4159-4a50-8425-4658024bb240 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:42.062011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f26017-4159-4a50-8425-4658024bb240 None admin] RBAC: Adding query filter params (name=member) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:42.062320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f26017-4159-4a50-8425-4658024bb240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:42.063721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f26017-4159-4a50-8425-4658024bb240 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:42.074467 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 140/279] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:42.082272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b378c9-6446-472c-931a-830b7ff4a7d2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:42.178506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e4347-67e0-4140-9806-3fd17963a799 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:42.178811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e4347-67e0-4140-9806-3fd17963a799 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:42.179084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e4347-67e0-4140-9806-3fd17963a799 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:42.180425 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 141/280] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:32:42.185650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9818d-b815-42fb-b8ff-938bcf51fe33 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:42.186071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9818d-b815-42fb-b8ff-938bcf51fe33 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:42.186457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9818d-b815-42fb-b8ff-938bcf51fe33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:42.206835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5ae9818d-b815-42fb-b8ff-938bcf51fe33 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:42.208054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5ae9818d-b815-42fb-b8ff-938bcf51fe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:42.218821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b378c9-6446-472c-931a-830b7ff4a7d2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:42.263036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fcd37-4b59-4e52-9ae5-6cdb0c5d8830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:42.264233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fcd37-4b59-4e52-9ae5-6cdb0c5d8830 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:42.264522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fcd37-4b59-4e52-9ae5-6cdb0c5d8830 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:42.264802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fcd37-4b59-4e52-9ae5-6cdb0c5d8830 None admin] PATH_INFO: `/v3/users/169c6a3afe7446a48929b78e658db391` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:42.282629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fcd37-4b59-4e52-9ae5-6cdb0c5d8830 None admin] RBAC: Authorizing `identity:get_user(user_id=169c6a3afe7446a48929b78e658db391)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:42.284822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fcd37-4b59-4e52-9ae5-6cdb0c5d8830 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:42.287563 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 140/281] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:42 2022] GET /identity/v3/users/169c6a3afe7446a48929b78e658db391 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:42.295166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fcd37-4b59-4e52-9ae5-6cdb0c5d8830 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:42.304864 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 142/282] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:42.316297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:42.401191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fcd37-4b59-4e52-9ae5-6cdb0c5d8830 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:42.414038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9818d-b815-42fb-b8ff-938bcf51fe33 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:42.447787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d24c3f-8177-4686-af6f-4a93c90c035e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:42.448971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d24c3f-8177-4686-af6f-4a93c90c035e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:42.449461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d24c3f-8177-4686-af6f-4a93c90c035e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:42.449962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d24c3f-8177-4686-af6f-4a93c90c035e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:42.450886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d24c3f-8177-4686-af6f-4a93c90c035e None admin] RBAC: Adding query filter params (role.id=d167c236cee64ede8d62bce075f83fff, user.id=169c6a3afe7446a48929b78e658db391, scope.system=all) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:42.451414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d24c3f-8177-4686-af6f-4a93c90c035e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:42.453216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d24c3f-8177-4686-af6f-4a93c90c035e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:42.460158 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 141/283] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Sep 3 20:32:42 2022] GET /identity/v3/role_assignments?user.id=169c6a3afe7446a48929b78e658db391&scope.system=all&role.id=d167c236cee64ede8d62bce075f83fff => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:42.502972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:42.538005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9818d-b815-42fb-b8ff-938bcf51fe33 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:42.576055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048dd2b-8010-49e8-bf97-c2683425671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:42.576634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048dd2b-8010-49e8-bf97-c2683425671b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:42.576984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048dd2b-8010-49e8-bf97-c2683425671b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:42.577240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048dd2b-8010-49e8-bf97-c2683425671b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:42.590504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048dd2b-8010-49e8-bf97-c2683425671b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-6048dd2b-8010-49e8-bf97-c2683425671b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:42.595618 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:42.596519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:42.597346 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 143/284] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:32:42.603962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d24c3f-8177-4686-af6f-4a93c90c035e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:42.722084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048dd2b-8010-49e8-bf97-c2683425671b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:42.733075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:42.753132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d24c3f-8177-4686-af6f-4a93c90c035e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:42.790565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c00eb-94ec-4418-9035-4206ebed0f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:42.791190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c00eb-94ec-4418-9035-4206ebed0f07 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:42.791480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c00eb-94ec-4418-9035-4206ebed0f07 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:42.791758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c00eb-94ec-4418-9035-4206ebed0f07 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:42.792451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c00eb-94ec-4418-9035-4206ebed0f07 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:42.792910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c00eb-94ec-4418-9035-4206ebed0f07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:42.794375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c00eb-94ec-4418-9035-4206ebed0f07 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:42.803818 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 142/285] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:42.809682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c00eb-94ec-4418-9035-4206ebed0f07 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:42.917555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c00eb-94ec-4418-9035-4206ebed0f07 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:42.953861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b4e4f-4150-4272-8eb5-dc4787774b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:42.954567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b4e4f-4150-4272-8eb5-dc4787774b4e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:42.954687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048dd2b-8010-49e8-bf97-c2683425671b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:42.954884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b4e4f-4150-4272-8eb5-dc4787774b4e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:42.955165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b4e4f-4150-4272-8eb5-dc4787774b4e None admin] PATH_INFO: `/v3/users/bbc73583f3944c2f9b1dd82ee2552dc6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:42.956699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b4e4f-4150-4272-8eb5-dc4787774b4e None admin] RBAC: Authorizing `identity:get_user(user_id=bbc73583f3944c2f9b1dd82ee2552dc6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:42.958216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b4e4f-4150-4272-8eb5-dc4787774b4e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:42.960143 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 143/286] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:42 2022] GET /identity/v3/users/bbc73583f3944c2f9b1dd82ee2552dc6 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:42.966068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b4e4f-4150-4272-8eb5-dc4787774b4e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:43.001258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:43.020807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:43.029532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c673322-0aab-4420-8d4b-9c27ca92a683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:43.030400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c673322-0aab-4420-8d4b-9c27ca92a683 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:43.030689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c673322-0aab-4420-8d4b-9c27ca92a683 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:43.030967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c673322-0aab-4420-8d4b-9c27ca92a683 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:43.053908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c673322-0aab-4420-8d4b-9c27ca92a683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-9c673322-0aab-4420-8d4b-9c27ca92a683 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:43.061736 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:43.062607 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:43.063375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:43.063375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:43.063375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:43.063375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:43.063375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:43.063375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:43.063375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:43.063375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:43.063375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:43.063375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:43.063375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:43.063375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:43.066412 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 144/287] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:43.072737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c673322-0aab-4420-8d4b-9c27ca92a683 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:43.152587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b4e4f-4150-4272-8eb5-dc4787774b4e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:43.199532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a45995-50fa-4756-b8e2-e0c10edf04e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:43.200175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a45995-50fa-4756-b8e2-e0c10edf04e5 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:43.200462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a45995-50fa-4756-b8e2-e0c10edf04e5 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:43.200741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a45995-50fa-4756-b8e2-e0c10edf04e5 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:43.202037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a45995-50fa-4756-b8e2-e0c10edf04e5 None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:43.203655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a45995-50fa-4756-b8e2-e0c10edf04e5 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:43.206685 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 144/288] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:42 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:43.212728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a45995-50fa-4756-b8e2-e0c10edf04e5 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:43.264925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c673322-0aab-4420-8d4b-9c27ca92a683 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:43.332199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7e2f4-8cb2-4f04-a4c2-6c8624d5e9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:43.333976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e2f4-8cb2-4f04-a4c2-6c8624d5e9c2 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:43.334461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e2f4-8cb2-4f04-a4c2-6c8624d5e9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:43.335086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e2f4-8cb2-4f04-a4c2-6c8624d5e9c2 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:43.337205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7e2f4-8cb2-4f04-a4c2-6c8624d5e9c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:43.337860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7e2f4-8cb2-4f04-a4c2-6c8624d5e9c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:43.338328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:43.340595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7e2f4-8cb2-4f04-a4c2-6c8624d5e9c2 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:43.364183 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 145/289] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:43.368005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7e2f4-8cb2-4f04-a4c2-6c8624d5e9c2 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:43.384918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a45995-50fa-4756-b8e2-e0c10edf04e5 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:43.450551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fb5d3-0541-405e-9b07-7dd29b20fbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:43.451639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fb5d3-0541-405e-9b07-7dd29b20fbf4 None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:43.452120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fb5d3-0541-405e-9b07-7dd29b20fbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:43.452590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fb5d3-0541-405e-9b07-7dd29b20fbf4 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4/users/bbc73583f3944c2f9b1dd82ee2552dc6/roles/2eaf93a02e4a40a0b74e2a9fcbb182d0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:43.459268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fb5d3-0541-405e-9b07-7dd29b20fbf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a64615765ab4a01a30fc0023bb081c4, user_id=bbc73583f3944c2f9b1dd82ee2552dc6, role_id=2eaf93a02e4a40a0b74e2a9fcbb182d0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:43.462065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fb5d3-0541-405e-9b07-7dd29b20fbf4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:43.474705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:43.495166 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 145/290] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:32:43 2022] PUT /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4/users/bbc73583f3944c2f9b1dd82ee2552dc6/roles/2eaf93a02e4a40a0b74e2a9fcbb182d0 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:32:43.543950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7e2f4-8cb2-4f04-a4c2-6c8624d5e9c2 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:43.604349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b85d26-9384-42f0-bdf1-fc192838feed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:43.605550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b85d26-9384-42f0-bdf1-fc192838feed None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:43.606066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b85d26-9384-42f0-bdf1-fc192838feed None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:43.606618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b85d26-9384-42f0-bdf1-fc192838feed None admin] PATH_INFO: `/v3/users/d1f0272a3bd84524872bb6491d518fdf` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:43.613955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b85d26-9384-42f0-bdf1-fc192838feed None admin] RBAC: Authorizing `identity:get_user(user_id=d1f0272a3bd84524872bb6491d518fdf)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:43.616029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b85d26-9384-42f0-bdf1-fc192838feed None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:43.618510 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 146/291] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:43 2022] GET /identity/v3/users/d1f0272a3bd84524872bb6491d518fdf => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:43.624804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fb5d3-0541-405e-9b07-7dd29b20fbf4 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:43.769320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fb5d3-0541-405e-9b07-7dd29b20fbf4 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:43.806173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ebb61-175c-4d4a-9d65-de3588699473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:43.806802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ebb61-175c-4d4a-9d65-de3588699473 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:43.807086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ebb61-175c-4d4a-9d65-de3588699473 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:43.807368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ebb61-175c-4d4a-9d65-de3588699473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:43.818760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:43.879801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ebb61-175c-4d4a-9d65-de3588699473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:43.881490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ebb61-175c-4d4a-9d65-de3588699473 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:44.018537 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 146/292] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:44.023896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b85d26-9384-42f0-bdf1-fc192838feed None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:44.163957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b85d26-9384-42f0-bdf1-fc192838feed None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:44.197328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:44.204374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f5350-f08b-43dd-b97b-d6e742d1b552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:44.205073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f5350-f08b-43dd-b97b-d6e742d1b552 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:44.205365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f5350-f08b-43dd-b97b-d6e742d1b552 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:44.206058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f5350-f08b-43dd-b97b-d6e742d1b552 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:44.207577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f5350-f08b-43dd-b97b-d6e742d1b552 None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:44.209458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f5350-f08b-43dd-b97b-d6e742d1b552 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:44.214974 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 147/293] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:44 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:44.221337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ebb61-175c-4d4a-9d65-de3588699473 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:44.280121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:44.363651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ebb61-175c-4d4a-9d65-de3588699473 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:44.402076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f85572-747f-4e7a-b0ec-9221f77dcaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:44.402692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f85572-747f-4e7a-b0ec-9221f77dcaeb None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:44.402976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f85572-747f-4e7a-b0ec-9221f77dcaeb None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:44.403252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f85572-747f-4e7a-b0ec-9221f77dcaeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:44.403976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f85572-747f-4e7a-b0ec-9221f77dcaeb None admin] RBAC: Adding query filter params (role.id=d167c236cee64ede8d62bce075f83fff, scope.project.id=0a64615765ab4a01a30fc0023bb081c4, user.id=d1f0272a3bd84524872bb6491d518fdf) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:44.404288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f85572-747f-4e7a-b0ec-9221f77dcaeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:44.405761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f85572-747f-4e7a-b0ec-9221f77dcaeb None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:44.412546 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 147/294] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:44 2022] GET /identity/v3/role_assignments?user.id=d1f0272a3bd84524872bb6491d518fdf&scope.project.id=0a64615765ab4a01a30fc0023bb081c4&role.id=d167c236cee64ede8d62bce075f83fff => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:44.702148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dfbb7-e75e-4314-a6d8-05dbdedee9a9 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:44.702527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dfbb7-e75e-4314-a6d8-05dbdedee9a9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:44.702726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dfbb7-e75e-4314-a6d8-05dbdedee9a9 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:44.704225 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 148/295] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:44.714648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270fdaf-f28f-4ef2-8689-232d7823eff7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:44.714648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270fdaf-f28f-4ef2-8689-232d7823eff7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:44.714648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270fdaf-f28f-4ef2-8689-232d7823eff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:44.732945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b270fdaf-f28f-4ef2-8689-232d7823eff7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:44.734192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b270fdaf-f28f-4ef2-8689-232d7823eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:44.817229 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 148/296] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:44.909695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ed57c-694a-4588-ba52-9d989a9f5b9a None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:44.910539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ed57c-694a-4588-ba52-9d989a9f5b9a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:44.911242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ed57c-694a-4588-ba52-9d989a9f5b9a None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:44.913846 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 149/297] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:44.921642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d21b0c-fc0c-42cc-8a17-7c1574ef823a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:44.922355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d21b0c-fc0c-42cc-8a17-7c1574ef823a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:44.923024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d21b0c-fc0c-42cc-8a17-7c1574ef823a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:44.947473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-21d21b0c-fc0c-42cc-8a17-7c1574ef823a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:44.949036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-21d21b0c-fc0c-42cc-8a17-7c1574ef823a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:44.987103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:45.032982 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 149/298] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:45.207441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ed57c-694a-4588-ba52-9d989a9f5b9a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:45.255359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03754e4a-4667-45db-a164-456d2dd8f46f None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:45.255701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03754e4a-4667-45db-a164-456d2dd8f46f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:45.256094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03754e4a-4667-45db-a164-456d2dd8f46f None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:45.258265 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 150/299] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:32:45.265346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fa735-9c27-4a62-a288-3954996d9cf3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:45.265905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fa735-9c27-4a62-a288-3954996d9cf3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:45.266298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fa735-9c27-4a62-a288-3954996d9cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:45.285347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c66fa735-9c27-4a62-a288-3954996d9cf3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:45.286715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c66fa735-9c27-4a62-a288-3954996d9cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:45.338563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ed57c-694a-4588-ba52-9d989a9f5b9a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:45.380155 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 151/300] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:45.389233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff990b61-0724-4a4f-8587-128a486401a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:45.390094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff990b61-0724-4a4f-8587-128a486401a3 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:45.390558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff990b61-0724-4a4f-8587-128a486401a3 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:45.391010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff990b61-0724-4a4f-8587-128a486401a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:45.402735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff990b61-0724-4a4f-8587-128a486401a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-ff990b61-0724-4a4f-8587-128a486401a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:45.408831 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:45.409741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:45.410515 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:45.410515 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:45.410515 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:45.410515 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:45.410515 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:45.410515 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:45.410515 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:45.410515 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:45.410515 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:45.410515 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:45.410515 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:45.410515 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:45.412662 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 150/301] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:45.418270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fa735-9c27-4a62-a288-3954996d9cf3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:45.419806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff990b61-0724-4a4f-8587-128a486401a3 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:45.484455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:45.532301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:45.539810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fa735-9c27-4a62-a288-3954996d9cf3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:45.547605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff990b61-0724-4a4f-8587-128a486401a3 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:45.577479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02571a55-fde9-455c-be62-6b463b07273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:45.578232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02571a55-fde9-455c-be62-6b463b07273e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:45.578620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02571a55-fde9-455c-be62-6b463b07273e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:45.578995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02571a55-fde9-455c-be62-6b463b07273e None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:45.579769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02571a55-fde9-455c-be62-6b463b07273e None admin] RBAC: Adding query filter params (name=reader) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:45.580350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02571a55-fde9-455c-be62-6b463b07273e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:45.582719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02571a55-fde9-455c-be62-6b463b07273e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:45.595929 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 152/302] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:45.601884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02571a55-fde9-455c-be62-6b463b07273e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:45.603613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc76cd-9f10-41cb-ba42-bbe52dd92e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:45.604378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc76cd-9f10-41cb-ba42-bbe52dd92e7c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:45.604663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc76cd-9f10-41cb-ba42-bbe52dd92e7c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:45.604962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc76cd-9f10-41cb-ba42-bbe52dd92e7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:45.606088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc76cd-9f10-41cb-ba42-bbe52dd92e7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:45.607898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc76cd-9f10-41cb-ba42-bbe52dd92e7c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:45.612939 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 151/303] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:45.617622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc76cd-9f10-41cb-ba42-bbe52dd92e7c None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:45.728748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc76cd-9f10-41cb-ba42-bbe52dd92e7c None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:45.744271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02571a55-fde9-455c-be62-6b463b07273e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:45.745180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:45.765648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dbe41-cc29-4e62-825e-32de2da297b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:45.766544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dbe41-cc29-4e62-825e-32de2da297b2 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:45.766832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dbe41-cc29-4e62-825e-32de2da297b2 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:45.767109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dbe41-cc29-4e62-825e-32de2da297b2 None admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:45.768448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928dbe41-cc29-4e62-825e-32de2da297b2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:45.770690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928dbe41-cc29-4e62-825e-32de2da297b2 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:45.780147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8330b5-28a6-448b-bc06-72706c85ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:45.780808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8330b5-28a6-448b-bc06-72706c85ac50 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:45.781123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8330b5-28a6-448b-bc06-72706c85ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:45.781414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8330b5-28a6-448b-bc06-72706c85ac50 None admin] PATH_INFO: `/v3/users/1c7083f894da4ba08b07c7538cd6e21e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:45.797012 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 152/304] 10.222.0.16 () {60 vars in 1134 bytes} [Sat Sep 3 20:32:45 2022] POST /identity/v3/groups => generated 229 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:45.797012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dbe41-cc29-4e62-825e-32de2da297b2 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:45.805457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8330b5-28a6-448b-bc06-72706c85ac50 None admin] RBAC: Authorizing `identity:get_user(user_id=1c7083f894da4ba08b07c7538cd6e21e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:45.807063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8330b5-28a6-448b-bc06-72706c85ac50 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:45.809254 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 153/305] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:45 2022] GET /identity/v3/users/1c7083f894da4ba08b07c7538cd6e21e => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:45.816508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8330b5-28a6-448b-bc06-72706c85ac50 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:45.836631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:45.950728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8330b5-28a6-448b-bc06-72706c85ac50 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:45.952333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dbe41-cc29-4e62-825e-32de2da297b2 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:45.999385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfd8e1-6fbf-4b1f-9c68-f5a749cede32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:46.000354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfd8e1-6fbf-4b1f-9c68-f5a749cede32 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:46.000354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfd8e1-6fbf-4b1f-9c68-f5a749cede32 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:46.000668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfd8e1-6fbf-4b1f-9c68-f5a749cede32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:46.002916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfd8e1-6fbf-4b1f-9c68-f5a749cede32 None admin] RBAC: Adding query filter params (role.id=2eaf93a02e4a40a0b74e2a9fcbb182d0, user.id=1c7083f894da4ba08b07c7538cd6e21e, scope.system=all) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:46.002916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfd8e1-6fbf-4b1f-9c68-f5a749cede32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:46.003119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ebc80-85d0-4e80-a353-ad8c1bfc211b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:46.003398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ebc80-85d0-4e80-a353-ad8c1bfc211b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:46.003681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ebc80-85d0-4e80-a353-ad8c1bfc211b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:46.003862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfd8e1-6fbf-4b1f-9c68-f5a749cede32 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:46.003936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ebc80-85d0-4e80-a353-ad8c1bfc211b None admin] PATH_INFO: `/v3/roles/member` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:46.010931 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 154/306] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Sep 3 20:32:45 2022] GET /identity/v3/role_assignments?user.id=1c7083f894da4ba08b07c7538cd6e21e&scope.system=all&role.id=2eaf93a02e4a40a0b74e2a9fcbb182d0 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:46.019824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ebc80-85d0-4e80-a353-ad8c1bfc211b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-768ebc80-85d0-4e80-a353-ad8c1bfc211b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:46.025854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:46.026829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:46.027602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:46.027602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:46.027602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:46.027602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:46.027602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:46.027602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:46.027602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:46.027602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:46.027602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:46.027602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:46.027602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:46.027602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:46.029648 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 153/307] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:46.035682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfd8e1-6fbf-4b1f-9c68-f5a749cede32 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:46.127871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:46.152987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfd8e1-6fbf-4b1f-9c68-f5a749cede32 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:46.157968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6993c3-e4fd-4908-9e4e-c1c6e52f627d None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:46.158259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6993c3-e4fd-4908-9e4e-c1c6e52f627d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:46.158540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6993c3-e4fd-4908-9e4e-c1c6e52f627d None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:46.160226 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 154/308] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:32:46.166060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaab678-f437-4722-b644-31b817dff3d5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:46.166395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaab678-f437-4722-b644-31b817dff3d5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:46.166632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaab678-f437-4722-b644-31b817dff3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:46.186515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ccaab678-f437-4722-b644-31b817dff3d5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:46.187772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ccaab678-f437-4722-b644-31b817dff3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:46.202043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980aca6-586c-44fb-807c-b38024a47bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:46.203212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980aca6-586c-44fb-807c-b38024a47bef None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:46.203641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980aca6-586c-44fb-807c-b38024a47bef None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:46.204057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980aca6-586c-44fb-807c-b38024a47bef None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:46.205205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980aca6-586c-44fb-807c-b38024a47bef None admin] RBAC: Adding query filter params (name=member) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:46.205677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980aca6-586c-44fb-807c-b38024a47bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:46.208270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980aca6-586c-44fb-807c-b38024a47bef None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:46.220017 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 155/309] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:46.224748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980aca6-586c-44fb-807c-b38024a47bef None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:46.228265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:46.281006 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 155/310] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:46.333122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980aca6-586c-44fb-807c-b38024a47bef None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:46.369482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744c7a5-8337-48ff-a4ad-e0c967c2580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:46.370418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744c7a5-8337-48ff-a4ad-e0c967c2580c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:46.370936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744c7a5-8337-48ff-a4ad-e0c967c2580c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:46.371389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744c7a5-8337-48ff-a4ad-e0c967c2580c None admin] PATH_INFO: `/v3/users/169c6a3afe7446a48929b78e658db391` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:46.372751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744c7a5-8337-48ff-a4ad-e0c967c2580c None admin] RBAC: Authorizing `identity:get_user(user_id=169c6a3afe7446a48929b78e658db391)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:46.374487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744c7a5-8337-48ff-a4ad-e0c967c2580c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:46.376268 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 156/311] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:46 2022] GET /identity/v3/users/169c6a3afe7446a48929b78e658db391 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:46.382744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaab678-f437-4722-b644-31b817dff3d5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:46.488641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaab678-f437-4722-b644-31b817dff3d5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:46.524086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe031715-392f-46ed-b01c-c411d85b9fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:46.524884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe031715-392f-46ed-b01c-c411d85b9fa3 None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:46.525287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe031715-392f-46ed-b01c-c411d85b9fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:46.525658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe031715-392f-46ed-b01c-c411d85b9fa3 None admin] PATH_INFO: `/v3/system/users/169c6a3afe7446a48929b78e658db391/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:46.527223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe031715-392f-46ed-b01c-c411d85b9fa3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=169c6a3afe7446a48929b78e658db391, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:46.528710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe031715-392f-46ed-b01c-c411d85b9fa3 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:46.541067 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 156/312] 10.222.0.16 () {58 vars in 1334 bytes} [Sat Sep 3 20:32:46 2022] PUT /identity/v3/system/users/169c6a3afe7446a48929b78e658db391/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:32:46.625672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744c7a5-8337-48ff-a4ad-e0c967c2580c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:46.639492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:46.748577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744c7a5-8337-48ff-a4ad-e0c967c2580c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:46.784142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2934078-79e6-466a-8ca5-179b6672eba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:46.784790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2934078-79e6-466a-8ca5-179b6672eba1 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:46.785103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2934078-79e6-466a-8ca5-179b6672eba1 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:46.785388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2934078-79e6-466a-8ca5-179b6672eba1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:46.795953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2934078-79e6-466a-8ca5-179b6672eba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a2934078-79e6-466a-8ca5-179b6672eba1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:46.802108 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:46.802982 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:46.803777 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 157/313] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:46.810259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe031715-392f-46ed-b01c-c411d85b9fa3 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:46.842119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:46.914785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe031715-392f-46ed-b01c-c411d85b9fa3 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:46.950226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c05a6-62ea-4e2c-a05b-dd2226de3364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:46.950842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c05a6-62ea-4e2c-a05b-dd2226de3364 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:46.951122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c05a6-62ea-4e2c-a05b-dd2226de3364 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:46.951394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c05a6-62ea-4e2c-a05b-dd2226de3364 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:46.952066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c05a6-62ea-4e2c-a05b-dd2226de3364 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:46.952368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c05a6-62ea-4e2c-a05b-dd2226de3364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:46.953799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c05a6-62ea-4e2c-a05b-dd2226de3364 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:46.967257 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 157/314] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:46.968619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2934078-79e6-466a-8ca5-179b6672eba1 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:47.041449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:47.078154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2934078-79e6-466a-8ca5-179b6672eba1 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:47.117059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774372f4-f480-4e57-970d-3dd846d640af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:47.117706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774372f4-f480-4e57-970d-3dd846d640af None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:47.117991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774372f4-f480-4e57-970d-3dd846d640af None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:47.118265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774372f4-f480-4e57-970d-3dd846d640af None admin] PATH_INFO: `/v3/users/bbc73583f3944c2f9b1dd82ee2552dc6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:47.119414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774372f4-f480-4e57-970d-3dd846d640af None admin] RBAC: Authorizing `identity:get_user(user_id=bbc73583f3944c2f9b1dd82ee2552dc6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:47.121153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774372f4-f480-4e57-970d-3dd846d640af None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:47.127313 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 158/315] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:46 2022] GET /identity/v3/users/bbc73583f3944c2f9b1dd82ee2552dc6 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:47.128665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c05a6-62ea-4e2c-a05b-dd2226de3364 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:47.228770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:47.238748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c05a6-62ea-4e2c-a05b-dd2226de3364 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:47.276980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6009e9d-8536-4784-8b06-77d4b2c3a6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:47.277608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6009e9d-8536-4784-8b06-77d4b2c3a6b3 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:47.277895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6009e9d-8536-4784-8b06-77d4b2c3a6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:47.278179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6009e9d-8536-4784-8b06-77d4b2c3a6b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:47.401937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6009e9d-8536-4784-8b06-77d4b2c3a6b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:47.403427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6009e9d-8536-4784-8b06-77d4b2c3a6b3 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:47.498764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:47.510131 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 158/316] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:47.515573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774372f4-f480-4e57-970d-3dd846d640af None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:47.622875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774372f4-f480-4e57-970d-3dd846d640af None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:47.656259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3810f-f5a1-484a-9962-f5e176aec64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:47.656926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3810f-f5a1-484a-9962-f5e176aec64f None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:47.657213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3810f-f5a1-484a-9962-f5e176aec64f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:47.657491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3810f-f5a1-484a-9962-f5e176aec64f None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:47.659165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3810f-f5a1-484a-9962-f5e176aec64f None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:47.660597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3810f-f5a1-484a-9962-f5e176aec64f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:47.667548 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 159/317] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:47 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:47.668976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6009e9d-8536-4784-8b06-77d4b2c3a6b3 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:47.779021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6009e9d-8536-4784-8b06-77d4b2c3a6b3 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:47.816434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca46510-a9a2-4d1a-b041-c5f0b26beac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:47.817088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca46510-a9a2-4d1a-b041-c5f0b26beac0 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:47.817376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca46510-a9a2-4d1a-b041-c5f0b26beac0 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:47.817654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca46510-a9a2-4d1a-b041-c5f0b26beac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:47.818391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca46510-a9a2-4d1a-b041-c5f0b26beac0 None admin] RBAC: Adding query filter params (role.id=2eaf93a02e4a40a0b74e2a9fcbb182d0, scope.project.id=0a64615765ab4a01a30fc0023bb081c4, user.id=bbc73583f3944c2f9b1dd82ee2552dc6) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:47.818704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca46510-a9a2-4d1a-b041-c5f0b26beac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:47.820146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca46510-a9a2-4d1a-b041-c5f0b26beac0 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:47.826573 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 159/318] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:32:47 2022] GET /identity/v3/role_assignments?user.id=bbc73583f3944c2f9b1dd82ee2552dc6&scope.project.id=0a64615765ab4a01a30fc0023bb081c4&role.id=2eaf93a02e4a40a0b74e2a9fcbb182d0 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:48.182746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d7dba-0f9c-4238-8100-db4fa3bffc64 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:48.183280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d7dba-0f9c-4238-8100-db4fa3bffc64 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:48.183280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d7dba-0f9c-4238-8100-db4fa3bffc64 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:48.184638 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 160/319] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:48.190107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994364f2-6cf1-4b5f-ab2d-889c362885e7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:48.190401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994364f2-6cf1-4b5f-ab2d-889c362885e7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:48.190682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994364f2-6cf1-4b5f-ab2d-889c362885e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:48.209407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-994364f2-6cf1-4b5f-ab2d-889c362885e7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:48.210604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-994364f2-6cf1-4b5f-ab2d-889c362885e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:48.262364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:48.280617 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 160/320] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:48.458766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88779cf1-8efa-4a7c-8e65-51394d2d2154 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:48.459054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88779cf1-8efa-4a7c-8e65-51394d2d2154 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:48.459344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88779cf1-8efa-4a7c-8e65-51394d2d2154 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:48.460682 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 161/321] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:48.466024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910fbb5-201d-40fa-8c6b-d2ec676677da None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:48.466322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910fbb5-201d-40fa-8c6b-d2ec676677da None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:48.466605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910fbb5-201d-40fa-8c6b-d2ec676677da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:48.485239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d910fbb5-201d-40fa-8c6b-d2ec676677da None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:48.487044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d910fbb5-201d-40fa-8c6b-d2ec676677da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:48.556944 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 161/322] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:48.608996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88779cf1-8efa-4a7c-8e65-51394d2d2154 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:48.613417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:48.712359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88779cf1-8efa-4a7c-8e65-51394d2d2154 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:48.749821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a4618-7b2f-4033-bf90-37c7b436ae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:48.750467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a4618-7b2f-4033-bf90-37c7b436ae59 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:48.750754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a4618-7b2f-4033-bf90-37c7b436ae59 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:48.751031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a4618-7b2f-4033-bf90-37c7b436ae59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:48.752207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a4618-7b2f-4033-bf90-37c7b436ae59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:48.753682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a4618-7b2f-4033-bf90-37c7b436ae59 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:48.755245 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 162/323] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:32:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:48.760804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910fbb5-201d-40fa-8c6b-d2ec676677da None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:48.858994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910fbb5-201d-40fa-8c6b-d2ec676677da None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:48.875334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:48.887933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf3c64-5fc9-421b-adb1-5e89f339c207 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:48.888226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf3c64-5fc9-421b-adb1-5e89f339c207 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:48.888511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf3c64-5fc9-421b-adb1-5e89f339c207 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:48.889878 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 163/324] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:32:48.890581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf3c64-5fc9-421b-adb1-5e89f339c207 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:48.894709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e4822-63f8-4522-aa07-cf9dd469d268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:48.895520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4822-63f8-4522-aa07-cf9dd469d268 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:48.895812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4822-63f8-4522-aa07-cf9dd469d268 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:48.896091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4822-63f8-4522-aa07-cf9dd469d268 None admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:48.896740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e4822-63f8-4522-aa07-cf9dd469d268 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:48.898242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e4822-63f8-4522-aa07-cf9dd469d268 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:48.913711 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 162/325] 10.222.0.16 () {60 vars in 1134 bytes} [Sat Sep 3 20:32:48 2022] POST /identity/v3/groups => generated 226 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:32:48.915086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9484c63-bb72-44df-ad80-4c52ded72a69 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:48.915376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9484c63-bb72-44df-ad80-4c52ded72a69 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:48.915653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9484c63-bb72-44df-ad80-4c52ded72a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:48.934395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a9484c63-bb72-44df-ad80-4c52ded72a69 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:48.935649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a9484c63-bb72-44df-ad80-4c52ded72a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:48.982385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:49.014921 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 163/326] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:49.019697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf3c64-5fc9-421b-adb1-5e89f339c207 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:49.055254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36c77d-918a-4263-aa34-445c4a275cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:49.055907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36c77d-918a-4263-aa34-445c4a275cd1 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:49.056194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36c77d-918a-4263-aa34-445c4a275cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:49.056469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36c77d-918a-4263-aa34-445c4a275cd1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:49.068712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36c77d-918a-4263-aa34-445c4a275cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-fd36c77d-918a-4263-aa34-445c4a275cd1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:49.074347 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:49.075284 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:49.076088 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 164/327] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:49.083655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9484c63-bb72-44df-ad80-4c52ded72a69 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:49.194503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9484c63-bb72-44df-ad80-4c52ded72a69 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:49.202522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:49.233982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaa55c-9f1f-40d6-afc8-cff7db0a686b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:49.234529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaa55c-9f1f-40d6-afc8-cff7db0a686b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:49.234812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaa55c-9f1f-40d6-afc8-cff7db0a686b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:49.235088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaa55c-9f1f-40d6-afc8-cff7db0a686b None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:49.235768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaaa55c-9f1f-40d6-afc8-cff7db0a686b None admin] RBAC: Adding query filter params (name=reader) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:49.236112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaaa55c-9f1f-40d6-afc8-cff7db0a686b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:49.237552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaaa55c-9f1f-40d6-afc8-cff7db0a686b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:49.246257 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 164/328] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:49.252650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36c77d-918a-4263-aa34-445c4a275cd1 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:49.346448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaa55c-9f1f-40d6-afc8-cff7db0a686b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:49.369145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36c77d-918a-4263-aa34-445c4a275cd1 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:49.403790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:49.410369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf131cb-8363-4402-9859-005a02573bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:49.411269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf131cb-8363-4402-9859-005a02573bc8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:49.411565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf131cb-8363-4402-9859-005a02573bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:49.411844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf131cb-8363-4402-9859-005a02573bc8 None admin] PATH_INFO: `/v3/users/1c7083f894da4ba08b07c7538cd6e21e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:49.413064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf131cb-8363-4402-9859-005a02573bc8 None admin] RBAC: Authorizing `identity:get_user(user_id=1c7083f894da4ba08b07c7538cd6e21e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:49.414529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf131cb-8363-4402-9859-005a02573bc8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:49.416501 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 165/329] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:49 2022] GET /identity/v3/users/1c7083f894da4ba08b07c7538cd6e21e => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:49.422430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf131cb-8363-4402-9859-005a02573bc8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:49.483591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaa55c-9f1f-40d6-afc8-cff7db0a686b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:49.537958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730ba12-4e29-4991-8396-c739b2ec629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:49.538794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730ba12-4e29-4991-8396-c739b2ec629a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:49.539086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730ba12-4e29-4991-8396-c739b2ec629a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:49.539632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730ba12-4e29-4991-8396-c739b2ec629a None admin] PATH_INFO: `/v3/roles/member` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:49.543394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf131cb-8363-4402-9859-005a02573bc8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:49.556281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730ba12-4e29-4991-8396-c739b2ec629a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-6730ba12-4e29-4991-8396-c739b2ec629a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:49.563353 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:49.564829 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:49.565650 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 165/330] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:32:49.571247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730ba12-4e29-4991-8396-c739b2ec629a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:49.588595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7da7c-6c7d-4b8c-9f18-c59e2bc1563b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:49.589392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7da7c-6c7d-4b8c-9f18-c59e2bc1563b None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:49.589679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7da7c-6c7d-4b8c-9f18-c59e2bc1563b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:49.589952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7da7c-6c7d-4b8c-9f18-c59e2bc1563b None admin] PATH_INFO: `/v3/system/users/1c7083f894da4ba08b07c7538cd6e21e/roles/2eaf93a02e4a40a0b74e2a9fcbb182d0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:49.591376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7da7c-6c7d-4b8c-9f18-c59e2bc1563b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1c7083f894da4ba08b07c7538cd6e21e, role_id=2eaf93a02e4a40a0b74e2a9fcbb182d0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:49.593004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7da7c-6c7d-4b8c-9f18-c59e2bc1563b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:49.603862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:49.609110 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 166/331] 10.222.0.16 () {58 vars in 1334 bytes} [Sat Sep 3 20:32:49 2022] PUT /identity/v3/system/users/1c7083f894da4ba08b07c7538cd6e21e/roles/2eaf93a02e4a40a0b74e2a9fcbb182d0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:32:49.691056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730ba12-4e29-4991-8396-c739b2ec629a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:49.733324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603d1a4-6413-4ac2-b08b-88c3b39dd8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:49.733956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603d1a4-6413-4ac2-b08b-88c3b39dd8d7 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:49.734240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603d1a4-6413-4ac2-b08b-88c3b39dd8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:49.734517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603d1a4-6413-4ac2-b08b-88c3b39dd8d7 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:49.735202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603d1a4-6413-4ac2-b08b-88c3b39dd8d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:49.735517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603d1a4-6413-4ac2-b08b-88c3b39dd8d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:49.736995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603d1a4-6413-4ac2-b08b-88c3b39dd8d7 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:49.746662 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 166/332] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:49.753241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7da7c-6c7d-4b8c-9f18-c59e2bc1563b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:49.845182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:49.889667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7da7c-6c7d-4b8c-9f18-c59e2bc1563b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:49.926939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fbb2c-3944-48e0-8a11-b85ff7ccb4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:49.927923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fbb2c-3944-48e0-8a11-b85ff7ccb4ee None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:49.928399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fbb2c-3944-48e0-8a11-b85ff7ccb4ee None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:49.928916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fbb2c-3944-48e0-8a11-b85ff7ccb4ee None admin] PATH_INFO: `/v3/users/169c6a3afe7446a48929b78e658db391` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:49.931328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fbb2c-3944-48e0-8a11-b85ff7ccb4ee None admin] RBAC: Authorizing `identity:get_user(user_id=169c6a3afe7446a48929b78e658db391)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:49.933138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fbb2c-3944-48e0-8a11-b85ff7ccb4ee None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:49.935323 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 167/333] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:49 2022] GET /identity/v3/users/169c6a3afe7446a48929b78e658db391 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:49.940925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603d1a4-6413-4ac2-b08b-88c3b39dd8d7 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:50.077307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:50.088879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603d1a4-6413-4ac2-b08b-88c3b39dd8d7 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:50.132572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce819f1-01db-4783-897f-47950aa29cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:50.133519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce819f1-01db-4783-897f-47950aa29cb6 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:50.133519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce819f1-01db-4783-897f-47950aa29cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:50.133679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce819f1-01db-4783-897f-47950aa29cb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:50.137015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce819f1-01db-4783-897f-47950aa29cb6 None admin] RBAC: Adding query filter params (role.id=d167c236cee64ede8d62bce075f83fff, user.id=169c6a3afe7446a48929b78e658db391, scope.system=all) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:50.137015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce819f1-01db-4783-897f-47950aa29cb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:50.137015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce819f1-01db-4783-897f-47950aa29cb6 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:50.149073 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 167/334] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Sep 3 20:32:49 2022] GET /identity/v3/role_assignments?user.id=169c6a3afe7446a48929b78e658db391&scope.system=all&role.id=d167c236cee64ede8d62bce075f83fff => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:51.960769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125cf91-bdbd-4c8f-9bdc-ec02bd6e3b93 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:51.961363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125cf91-bdbd-4c8f-9bdc-ec02bd6e3b93 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:51.961363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125cf91-bdbd-4c8f-9bdc-ec02bd6e3b93 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:51.962940 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 168/335] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:51.968422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460bf66-2027-4511-a21d-f4d4de595b46 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:51.968871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460bf66-2027-4511-a21d-f4d4de595b46 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:51.969078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460bf66-2027-4511-a21d-f4d4de595b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:51.998416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5460bf66-2027-4511-a21d-f4d4de595b46 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:51.999629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5460bf66-2027-4511-a21d-f4d4de595b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:52.073975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be90a20-c475-4ff3-9cc6-e1cc11d3443c None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:52.074318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be90a20-c475-4ff3-9cc6-e1cc11d3443c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:52.074539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be90a20-c475-4ff3-9cc6-e1cc11d3443c None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:52.076122 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 169/336] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:52.089831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab6083-288f-4626-a7aa-67d7eb03f49e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:52.090121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab6083-288f-4626-a7aa-67d7eb03f49e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:52.090391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab6083-288f-4626-a7aa-67d7eb03f49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:52.090784 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 168/337] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:52.137957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-92ab6083-288f-4626-a7aa-67d7eb03f49e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:52.139196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-92ab6083-288f-4626-a7aa-67d7eb03f49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:52.228393 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 170/338] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:52.474434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244475c-2069-4136-beb1-b3c14e276090 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:52.474825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244475c-2069-4136-beb1-b3c14e276090 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:52.475015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244475c-2069-4136-beb1-b3c14e276090 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:52.476476 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 169/339] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:52.482640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a48ac9-cc20-43ab-b895-baacc0f86002 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:52.483172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a48ac9-cc20-43ab-b895-baacc0f86002 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:52.483664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a48ac9-cc20-43ab-b895-baacc0f86002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:52.503727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c8a48ac9-cc20-43ab-b895-baacc0f86002 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:52.505376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c8a48ac9-cc20-43ab-b895-baacc0f86002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:52.575303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:52.591399 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 171/340] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:52.767445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744fc88-4792-4832-959e-c74dd8d40cf9 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:52.768576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744fc88-4792-4832-959e-c74dd8d40cf9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:52.769095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744fc88-4792-4832-959e-c74dd8d40cf9 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:52.771121 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 170/341] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:52.772077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a48ac9-cc20-43ab-b895-baacc0f86002 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:52.806716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49e5fc-c0db-42c7-a190-d475366b82f5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:52.807426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49e5fc-c0db-42c7-a190-d475366b82f5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:52.807939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49e5fc-c0db-42c7-a190-d475366b82f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:52.839093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8b49e5fc-c0db-42c7-a190-d475366b82f5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:52.840744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8b49e5fc-c0db-42c7-a190-d475366b82f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:52.932675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a48ac9-cc20-43ab-b895-baacc0f86002 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:52.943472 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 171/342] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:32:52.944346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49e5fc-c0db-42c7-a190-d475366b82f5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:52.964377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:52.982754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eadc1-0300-4230-9616-cf9abd80fb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:52.984171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eadc1-0300-4230-9616-cf9abd80fb95 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:52.984684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eadc1-0300-4230-9616-cf9abd80fb95 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:52.985194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eadc1-0300-4230-9616-cf9abd80fb95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:52.997371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3eadc1-0300-4230-9616-cf9abd80fb95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-0f3eadc1-0300-4230-9616-cf9abd80fb95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:53.002922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:53.003831 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:53.004670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:53.004670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:53.004670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:53.004670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:53.004670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:53.004670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:53.004670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:53.004670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:53.004670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:53.004670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:53.004670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:53.004670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:53.007582 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 172/343] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:32:53.014013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eadc1-0300-4230-9616-cf9abd80fb95 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:53.103991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49e5fc-c0db-42c7-a190-d475366b82f5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:53.125102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:53.178254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79556262-d056-4625-ac47-708322da6ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:53.179199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79556262-d056-4625-ac47-708322da6ffc None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:53.179646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79556262-d056-4625-ac47-708322da6ffc None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:53.180079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79556262-d056-4625-ac47-708322da6ffc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:53.198492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eadc1-0300-4230-9616-cf9abd80fb95 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:53.202152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79556262-d056-4625-ac47-708322da6ffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-79556262-d056-4625-ac47-708322da6ffc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:53.207461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:53.208391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:53.209192 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:53.209192 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:53.209192 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:53.209192 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:53.209192 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:53.209192 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:53.209192 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:53.209192 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:53.209192 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:53.209192 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:53.209192 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:53.209192 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:53.217254 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 172/344] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:32:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:53.218178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79556262-d056-4625-ac47-708322da6ffc None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:53.240477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62037a2-a7a4-478f-a143-03ff51af189f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:53.241384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62037a2-a7a4-478f-a143-03ff51af189f None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:53.241841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62037a2-a7a4-478f-a143-03ff51af189f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:53.242276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62037a2-a7a4-478f-a143-03ff51af189f None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:53.243113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62037a2-a7a4-478f-a143-03ff51af189f None admin] RBAC: Adding query filter params (name=member) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:53.243569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62037a2-a7a4-478f-a143-03ff51af189f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:53.245131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62037a2-a7a4-478f-a143-03ff51af189f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:53.255704 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 173/345] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:53.260968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62037a2-a7a4-478f-a143-03ff51af189f None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:53.318968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:53.382776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79556262-d056-4625-ac47-708322da6ffc None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:53.388438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62037a2-a7a4-478f-a143-03ff51af189f None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:53.403253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:53.428036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ab47d-edd6-4c04-99ce-998dd2cdd162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:53.428036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ab47d-edd6-4c04-99ce-998dd2cdd162 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:53.428036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ab47d-edd6-4c04-99ce-998dd2cdd162 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:53.428036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ab47d-edd6-4c04-99ce-998dd2cdd162 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:53.431215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e6596-f8db-4c63-ab9f-bf15d1fd8555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:53.432110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e6596-f8db-4c63-ab9f-bf15d1fd8555 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:53.432635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e6596-f8db-4c63-ab9f-bf15d1fd8555 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:53.433165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e6596-f8db-4c63-ab9f-bf15d1fd8555 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:53.434062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e6596-f8db-4c63-ab9f-bf15d1fd8555 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:53.434582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e6596-f8db-4c63-ab9f-bf15d1fd8555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:53.436163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e6596-f8db-4c63-ab9f-bf15d1fd8555 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:53.448580 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 173/346] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:32:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:53.449730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e6596-f8db-4c63-ab9f-bf15d1fd8555 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:53.504520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ab47d-edd6-4c04-99ce-998dd2cdd162 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:53.506599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ab47d-edd6-4c04-99ce-998dd2cdd162 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:53.573659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e6596-f8db-4c63-ab9f-bf15d1fd8555 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:53.628589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:53.634169 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 174/347] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:53.637949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ab47d-edd6-4c04-99ce-998dd2cdd162 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:53.640153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a141cc-a933-4095-8926-d4055865dacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:53.641021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a141cc-a933-4095-8926-d4055865dacd None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:53.641453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a141cc-a933-4095-8926-d4055865dacd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:53.641899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a141cc-a933-4095-8926-d4055865dacd None admin] PATH_INFO: `/v3/roles/member` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:53.653069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:53.656580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a141cc-a933-4095-8926-d4055865dacd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-79a141cc-a933-4095-8926-d4055865dacd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:53.665098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:53.666138 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:53.667039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:53.667039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:53.667039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:53.667039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:53.667039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:53.667039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:53.667039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:53.667039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:53.667039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:53.667039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:53.667039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:53.667039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:53.671660 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 174/348] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:53.672801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a141cc-a933-4095-8926-d4055865dacd None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:53.769920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ab47d-edd6-4c04-99ce-998dd2cdd162 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:53.816809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d548e-0096-4ea0-8f10-b90732e089d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:53.817464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d548e-0096-4ea0-8f10-b90732e089d9 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:53.817749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d548e-0096-4ea0-8f10-b90732e089d9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:53.818025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d548e-0096-4ea0-8f10-b90732e089d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:53.828708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a141cc-a933-4095-8926-d4055865dacd None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:53.872045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9755b-30cb-4364-b21b-4dfe77f15593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:53.873226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9755b-30cb-4364-b21b-4dfe77f15593 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:53.873738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9755b-30cb-4364-b21b-4dfe77f15593 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:53.874227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9755b-30cb-4364-b21b-4dfe77f15593 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:53.887424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9755b-30cb-4364-b21b-4dfe77f15593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-34c9755b-30cb-4364-b21b-4dfe77f15593 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:53.894177 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:53.895053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:53.895861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:53.895861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:53.895861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:53.895861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:53.895861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:53.895861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:53.895861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:53.895861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:53.895861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:53.895861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:53.895861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 20:32:53.895861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:53.899824 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 175/349] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:53.900940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9755b-30cb-4364-b21b-4dfe77f15593 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:53.906510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d548e-0096-4ea0-8f10-b90732e089d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:53.908055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d548e-0096-4ea0-8f10-b90732e089d9 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:53.927437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:53.962891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:54.021821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9755b-30cb-4364-b21b-4dfe77f15593 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:54.050036 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 175/350] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:54.050901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d548e-0096-4ea0-8f10-b90732e089d9 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:54.069076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd34302-9451-4ef2-a624-db957b665d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:54.070133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd34302-9451-4ef2-a624-db957b665d9d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:54.070623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd34302-9451-4ef2-a624-db957b665d9d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:54.071081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd34302-9451-4ef2-a624-db957b665d9d None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:54.072435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd34302-9451-4ef2-a624-db957b665d9d None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:54.074153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd34302-9451-4ef2-a624-db957b665d9d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:54.077185 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 176/351] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:53 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:54.078293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd34302-9451-4ef2-a624-db957b665d9d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:54.183671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d548e-0096-4ea0-8f10-b90732e089d9 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:54.200831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:54.203670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd34302-9451-4ef2-a624-db957b665d9d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:54.228666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:54.228846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84131e00-6876-4894-9d66-8c01d52364ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:54.229454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131e00-6876-4894-9d66-8c01d52364ea None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:54.229817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131e00-6876-4894-9d66-8c01d52364ea None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:54.230065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131e00-6876-4894-9d66-8c01d52364ea None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:54.230735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84131e00-6876-4894-9d66-8c01d52364ea None admin] RBAC: Adding query filter params (name=member) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:54.231115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84131e00-6876-4894-9d66-8c01d52364ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:54.232601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84131e00-6876-4894-9d66-8c01d52364ea None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:54.246252 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 176/352] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:54.248976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae0a0e-0ddb-41d7-9043-2abe441a9bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:54.249137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f8f63-da75-43a7-8444-b909ee868122 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:54.249137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f8f63-da75-43a7-8444-b909ee868122 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:54.249137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f8f63-da75-43a7-8444-b909ee868122 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:54.249322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae0a0e-0ddb-41d7-9043-2abe441a9bf6 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:54.249322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae0a0e-0ddb-41d7-9043-2abe441a9bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:54.249322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae0a0e-0ddb-41d7-9043-2abe441a9bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:54.249322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae0a0e-0ddb-41d7-9043-2abe441a9bf6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:54.249567 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 177/353] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:54.250146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f8f63-da75-43a7-8444-b909ee868122 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:54.253216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae0a0e-0ddb-41d7-9043-2abe441a9bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:54.254880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae0a0e-0ddb-41d7-9043-2abe441a9bf6 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:54.265442 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 177/354] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:54.266409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae0a0e-0ddb-41d7-9043-2abe441a9bf6 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:54.387716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f8f63-da75-43a7-8444-b909ee868122 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:54.400566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae0a0e-0ddb-41d7-9043-2abe441a9bf6 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:54.432033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49599aa6-3c79-4b24-ac77-9e8730cf7b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:54.432745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49599aa6-3c79-4b24-ac77-9e8730cf7b9e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:54.433076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49599aa6-3c79-4b24-ac77-9e8730cf7b9e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:54.433358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49599aa6-3c79-4b24-ac77-9e8730cf7b9e None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:54.434809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49599aa6-3c79-4b24-ac77-9e8730cf7b9e None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:54.435964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49599aa6-3c79-4b24-ac77-9e8730cf7b9e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:54.438979 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 178/355] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:54 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:54.438979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49599aa6-3c79-4b24-ac77-9e8730cf7b9e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:54.451432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301f5c6-1c60-4d9f-8a06-81314c03fcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:54.452086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301f5c6-1c60-4d9f-8a06-81314c03fcac None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:54.452755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301f5c6-1c60-4d9f-8a06-81314c03fcac None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:54.453114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301f5c6-1c60-4d9f-8a06-81314c03fcac None admin] PATH_INFO: `/v3/groups/a62b4bf100a4409b823a863854490ce7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:54.460874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301f5c6-1c60-4d9f-8a06-81314c03fcac None admin] RBAC: Authorizing `identity:get_group(group_id=a62b4bf100a4409b823a863854490ce7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:54.462228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301f5c6-1c60-4d9f-8a06-81314c03fcac None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:54.464830 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 178/356] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:32:54 2022] GET /identity/v3/groups/a62b4bf100a4409b823a863854490ce7 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:54.465043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7e964-c615-4fc8-be9e-4124c4ad9c71 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:54.465332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7e964-c615-4fc8-be9e-4124c4ad9c71 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:54.465606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7e964-c615-4fc8-be9e-4124c4ad9c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:54.484749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-11a7e964-c615-4fc8-be9e-4124c4ad9c71 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:54.486258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-11a7e964-c615-4fc8-be9e-4124c4ad9c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:54.499386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:54.535191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:54.565988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49599aa6-3c79-4b24-ac77-9e8730cf7b9e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:54.568114 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 179/357] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:54.568999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7e964-c615-4fc8-be9e-4124c4ad9c71 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:54.620247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1b0ed-4ef6-4fc7-8b5f-ff35b30a782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:54.620926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1b0ed-4ef6-4fc7-8b5f-ff35b30a782a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:54.621233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1b0ed-4ef6-4fc7-8b5f-ff35b30a782a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:54.621515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1b0ed-4ef6-4fc7-8b5f-ff35b30a782a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:54.695949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7e964-c615-4fc8-be9e-4124c4ad9c71 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:54.718370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1b0ed-4ef6-4fc7-8b5f-ff35b30a782a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:54.720312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1b0ed-4ef6-4fc7-8b5f-ff35b30a782a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:54.738617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8609b-d48a-4a5e-bfe8-2862f34d824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:54.739263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8609b-d48a-4a5e-bfe8-2862f34d824c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:54.739553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8609b-d48a-4a5e-bfe8-2862f34d824c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:54.739999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8609b-d48a-4a5e-bfe8-2862f34d824c None admin] PATH_INFO: `/v3/users/1c7083f894da4ba08b07c7538cd6e21e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:54.741512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8609b-d48a-4a5e-bfe8-2862f34d824c None admin] RBAC: Authorizing `identity:get_user(user_id=1c7083f894da4ba08b07c7538cd6e21e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:54.742969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8609b-d48a-4a5e-bfe8-2862f34d824c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:54.745478 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 180/358] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:32:54 2022] GET /identity/v3/users/1c7083f894da4ba08b07c7538cd6e21e => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:54.746595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8326f-609b-424a-8c62-8864681bc18c None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:54.746889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8326f-609b-424a-8c62-8864681bc18c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:54.747174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8326f-609b-424a-8c62-8864681bc18c None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:54.750290 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 181/359] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:54.751040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8326f-609b-424a-8c62-8864681bc18c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:54.778942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:54.813437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:54.852359 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 179/360] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:54.853993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1b0ed-4ef6-4fc7-8b5f-ff35b30a782a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:54.901674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8326f-609b-424a-8c62-8864681bc18c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:54.944715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2204ab-ac68-4a3f-9bed-fd256e6158d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:54.944959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2204ab-ac68-4a3f-9bed-fd256e6158d1 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:54.945063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2204ab-ac68-4a3f-9bed-fd256e6158d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:54.945352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2204ab-ac68-4a3f-9bed-fd256e6158d1 None admin] PATH_INFO: `/v3/groups/a62b4bf100a4409b823a863854490ce7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:54.948267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2204ab-ac68-4a3f-9bed-fd256e6158d1 None admin] RBAC: Authorizing `identity:get_group(group_id=a62b4bf100a4409b823a863854490ce7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:54.949721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2204ab-ac68-4a3f-9bed-fd256e6158d1 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:54.951471 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 182/361] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:32:54 2022] GET /identity/v3/groups/a62b4bf100a4409b823a863854490ce7 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:54.952283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2204ab-ac68-4a3f-9bed-fd256e6158d1 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:54.986565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1b0ed-4ef6-4fc7-8b5f-ff35b30a782a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:55.025795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a2718-d1c4-492b-8df7-295976c7333e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:55.026412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a2718-d1c4-492b-8df7-295976c7333e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:55.026698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a2718-d1c4-492b-8df7-295976c7333e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:55.026979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a2718-d1c4-492b-8df7-295976c7333e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:55.027715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a2718-d1c4-492b-8df7-295976c7333e None admin] RBAC: Adding query filter params (group.id=a62b4bf100a4409b823a863854490ce7, role.id=d167c236cee64ede8d62bce075f83fff, scope.project.id=bb6a16a02b534f94be33eea4c77eea11) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:55.028028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a2718-d1c4-492b-8df7-295976c7333e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:55.030705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a2718-d1c4-492b-8df7-295976c7333e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:55.037294 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 180/362] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 3 20:32:54 2022] GET /identity/v3/role_assignments?group.id=a62b4bf100a4409b823a863854490ce7&scope.project.id=bb6a16a02b534f94be33eea4c77eea11&role.id=d167c236cee64ede8d62bce075f83fff => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:55.038485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e7b2b-1985-4609-87b8-d5304aaafba4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:55.038777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e7b2b-1985-4609-87b8-d5304aaafba4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:55.039093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e7b2b-1985-4609-87b8-d5304aaafba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:55.064979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c29e7b2b-1985-4609-87b8-d5304aaafba4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:55.067065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c29e7b2b-1985-4609-87b8-d5304aaafba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:55.069918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:55.095231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2204ab-ac68-4a3f-9bed-fd256e6158d1 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:55.107238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:55.154523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d776f53-22a2-4363-a445-388938703282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:55.154944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d776f53-22a2-4363-a445-388938703282 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:55.155231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d776f53-22a2-4363-a445-388938703282 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:55.155509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d776f53-22a2-4363-a445-388938703282 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:55.156279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d776f53-22a2-4363-a445-388938703282 None admin] RBAC: Adding query filter params (role.id=2eaf93a02e4a40a0b74e2a9fcbb182d0, user.id=1c7083f894da4ba08b07c7538cd6e21e, scope.system=all) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:55.156597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d776f53-22a2-4363-a445-388938703282 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:55.158633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d776f53-22a2-4363-a445-388938703282 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:55.163675 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 181/363] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:55.165190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e7b2b-1985-4609-87b8-d5304aaafba4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:55.174436 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 183/364] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Sep 3 20:32:54 2022] GET /identity/v3/role_assignments?user.id=1c7083f894da4ba08b07c7538cd6e21e&scope.system=all&role.id=2eaf93a02e4a40a0b74e2a9fcbb182d0 => generated 510 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:55.175317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d776f53-22a2-4363-a445-388938703282 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:55.338473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e7b2b-1985-4609-87b8-d5304aaafba4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:55.368348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d776f53-22a2-4363-a445-388938703282 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:55.386904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52eb24-ac40-4e7c-ba27-79ea9c106b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:55.386904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52eb24-ac40-4e7c-ba27-79ea9c106b71 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:55.386904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52eb24-ac40-4e7c-ba27-79ea9c106b71 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:55.386904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52eb24-ac40-4e7c-ba27-79ea9c106b71 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:55.388909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52eb24-ac40-4e7c-ba27-79ea9c106b71 None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:55.391464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52eb24-ac40-4e7c-ba27-79ea9c106b71 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:55.393905 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 182/365] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:55 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:55.394889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52eb24-ac40-4e7c-ba27-79ea9c106b71 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:55.405674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:55.430594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae16ba2-73b0-459b-89fd-0020354848bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:55.431249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae16ba2-73b0-459b-89fd-0020354848bf None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:55.431534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae16ba2-73b0-459b-89fd-0020354848bf None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:55.431810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae16ba2-73b0-459b-89fd-0020354848bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:55.454615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae16ba2-73b0-459b-89fd-0020354848bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-4ae16ba2-73b0-459b-89fd-0020354848bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:55.460262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:55.461260 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:55.462065 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:55.462065 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:55.462065 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:55.462065 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:55.462065 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:55.462065 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:55.462065 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:55.462065 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:55.462065 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:55.462065 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:55.462065 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:55.462065 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:55.463183 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 184/366] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:32:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:55.463909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae16ba2-73b0-459b-89fd-0020354848bf None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:55.502600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:55.532561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52eb24-ac40-4e7c-ba27-79ea9c106b71 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:55.579090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a1df8-0193-4778-86cb-464c7f2e0590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:55.580134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a1df8-0193-4778-86cb-464c7f2e0590 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:55.580267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a1df8-0193-4778-86cb-464c7f2e0590 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:55.580650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a1df8-0193-4778-86cb-464c7f2e0590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:55.582316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a1df8-0193-4778-86cb-464c7f2e0590 None admin] RBAC: Adding query filter params (group.id=a62b4bf100a4409b823a863854490ce7, role.id=542f58433b2747d2abd5d405ebabdea1, scope.project.id=bb6a16a02b534f94be33eea4c77eea11) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:55.582665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a1df8-0193-4778-86cb-464c7f2e0590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:55.584351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a1df8-0193-4778-86cb-464c7f2e0590 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:55.590974 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 183/367] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 3 20:32:55 2022] GET /identity/v3/role_assignments?group.id=a62b4bf100a4409b823a863854490ce7&scope.project.id=bb6a16a02b534f94be33eea4c77eea11&role.id=542f58433b2747d2abd5d405ebabdea1 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:55.591996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a1df8-0193-4778-86cb-464c7f2e0590 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:55.594889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae16ba2-73b0-459b-89fd-0020354848bf None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:55.641793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be7f02-c9d6-4cac-a0a4-356a089f2ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:55.643018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be7f02-c9d6-4cac-a0a4-356a089f2ad2 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:55.643471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be7f02-c9d6-4cac-a0a4-356a089f2ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:55.643894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be7f02-c9d6-4cac-a0a4-356a089f2ad2 None admin] PATH_INFO: `/v3/groups/a62b4bf100a4409b823a863854490ce7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:55.645333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be7f02-c9d6-4cac-a0a4-356a089f2ad2 None admin] RBAC: Authorizing `identity:get_group(group_id=a62b4bf100a4409b823a863854490ce7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:55.647054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be7f02-c9d6-4cac-a0a4-356a089f2ad2 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:55.649354 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 185/368] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:32:55 2022] GET /identity/v3/groups/a62b4bf100a4409b823a863854490ce7 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:55.655861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be7f02-c9d6-4cac-a0a4-356a089f2ad2 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:55.687438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:55.737045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a1df8-0193-4778-86cb-464c7f2e0590 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:55.783358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e26e8f-9db4-4918-9718-96b099fcfaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:55.784407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e26e8f-9db4-4918-9718-96b099fcfaf2 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:55.785014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e26e8f-9db4-4918-9718-96b099fcfaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:55.785518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e26e8f-9db4-4918-9718-96b099fcfaf2 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:55.786425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e26e8f-9db4-4918-9718-96b099fcfaf2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:55.786951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e26e8f-9db4-4918-9718-96b099fcfaf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:55.788545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e26e8f-9db4-4918-9718-96b099fcfaf2 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:55.798956 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 184/369] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:32:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:55.800345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e26e8f-9db4-4918-9718-96b099fcfaf2 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:55.802455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:55.811420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be7f02-c9d6-4cac-a0a4-356a089f2ad2 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:55.855955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33b918-cb03-4493-9659-8c0ecc13293d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:55.856842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33b918-cb03-4493-9659-8c0ecc13293d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:55.857164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33b918-cb03-4493-9659-8c0ecc13293d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:55.857441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33b918-cb03-4493-9659-8c0ecc13293d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:55.858166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33b918-cb03-4493-9659-8c0ecc13293d None admin] RBAC: Adding query filter params (group.id=a62b4bf100a4409b823a863854490ce7, role.id=d167c236cee64ede8d62bce075f83fff, scope.project.id=0a64615765ab4a01a30fc0023bb081c4) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:55.858479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33b918-cb03-4493-9659-8c0ecc13293d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:55.859898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33b918-cb03-4493-9659-8c0ecc13293d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:55.866612 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 186/370] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 3 20:32:55 2022] GET /identity/v3/role_assignments?group.id=a62b4bf100a4409b823a863854490ce7&scope.project.id=0a64615765ab4a01a30fc0023bb081c4&role.id=d167c236cee64ede8d62bce075f83fff => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:55.867293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33b918-cb03-4493-9659-8c0ecc13293d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:55.999929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e26e8f-9db4-4918-9718-96b099fcfaf2 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:56.072432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:56.073807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33b918-cb03-4493-9659-8c0ecc13293d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:56.081841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fbfdc-87a3-4370-9627-39f77fb63736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:56.082495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fbfdc-87a3-4370-9627-39f77fb63736 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:56.082780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fbfdc-87a3-4370-9627-39f77fb63736 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:56.083059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fbfdc-87a3-4370-9627-39f77fb63736 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:56.099147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4fbfdc-87a3-4370-9627-39f77fb63736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-dd4fbfdc-87a3-4370-9627-39f77fb63736 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:56.104744 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:56.105662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:56.106521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:56.106521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:56.106521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:56.106521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:56.106521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:56.106521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:56.106521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:56.106521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:56.106521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:56.106521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:56.106521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:32:56.106521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:56.107617 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 185/371] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:32:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:56.115034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fbfdc-87a3-4370-9627-39f77fb63736 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:56.156990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d490-6428-4908-8381-61ab93880c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:56.158323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d490-6428-4908-8381-61ab93880c10 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:56.158531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d490-6428-4908-8381-61ab93880c10 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:56.158785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d490-6428-4908-8381-61ab93880c10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:56.226108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:56.235766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fbfdc-87a3-4370-9627-39f77fb63736 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:56.277794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916418a-c571-4238-9bbf-799669e7dbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:56.280068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916418a-c571-4238-9bbf-799669e7dbf8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:56.280682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916418a-c571-4238-9bbf-799669e7dbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:56.281796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916418a-c571-4238-9bbf-799669e7dbf8 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:56.283227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916418a-c571-4238-9bbf-799669e7dbf8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:56.284262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916418a-c571-4238-9bbf-799669e7dbf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:56.286173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916418a-c571-4238-9bbf-799669e7dbf8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:56.303931 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 186/372] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:56.305125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6d490-6428-4908-8381-61ab93880c10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:56.306782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6d490-6428-4908-8381-61ab93880c10 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:56.312472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916418a-c571-4238-9bbf-799669e7dbf8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:56.372849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:56.463099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916418a-c571-4238-9bbf-799669e7dbf8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:56.495973 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 187/373] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:32:56.502652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d490-6428-4908-8381-61ab93880c10 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:56.530385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce4e77-ef79-47bd-8852-0d682d0f7935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:56.532519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce4e77-ef79-47bd-8852-0d682d0f7935 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:56.533725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce4e77-ef79-47bd-8852-0d682d0f7935 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:56.534847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce4e77-ef79-47bd-8852-0d682d0f7935 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:56.610035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:56.671431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce4e77-ef79-47bd-8852-0d682d0f7935 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:56.673457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce4e77-ef79-47bd-8852-0d682d0f7935 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:56.681946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d490-6428-4908-8381-61ab93880c10 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:56.755604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb3df2-9370-4ed1-9ada-fbb083dc35cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:56.757605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb3df2-9370-4ed1-9ada-fbb083dc35cd None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:56.758353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb3df2-9370-4ed1-9ada-fbb083dc35cd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:56.759792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb3df2-9370-4ed1-9ada-fbb083dc35cd None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:56.761183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb3df2-9370-4ed1-9ada-fbb083dc35cd None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:56.761676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:56.764010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb3df2-9370-4ed1-9ada-fbb083dc35cd None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:56.767983 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 188/374] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:56 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:56.773856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb3df2-9370-4ed1-9ada-fbb083dc35cd None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:56.851156 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 187/375] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:32:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 20:32:56.857280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce4e77-ef79-47bd-8852-0d682d0f7935 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:56.955326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb3df2-9370-4ed1-9ada-fbb083dc35cd None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:57.005843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:57.021547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60ebc4-9ed6-4afd-9e7a-7112feea6c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:57.022567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60ebc4-9ed6-4afd-9e7a-7112feea6c59 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:57.023064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60ebc4-9ed6-4afd-9e7a-7112feea6c59 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:57.023595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60ebc4-9ed6-4afd-9e7a-7112feea6c59 None admin] PATH_INFO: `/v3/groups/a62b4bf100a4409b823a863854490ce7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:57.025198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60ebc4-9ed6-4afd-9e7a-7112feea6c59 None admin] RBAC: Authorizing `identity:get_group(group_id=a62b4bf100a4409b823a863854490ce7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:57.026902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60ebc4-9ed6-4afd-9e7a-7112feea6c59 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:57.029146 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 189/376] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:32:56 2022] GET /identity/v3/groups/a62b4bf100a4409b823a863854490ce7 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:57.035149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60ebc4-9ed6-4afd-9e7a-7112feea6c59 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:57.042670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce4e77-ef79-47bd-8852-0d682d0f7935 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:57.091671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272d0bf-8222-4b51-905e-49ec1bcc07e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:57.092420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272d0bf-8222-4b51-905e-49ec1bcc07e2 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:57.092599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272d0bf-8222-4b51-905e-49ec1bcc07e2 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:57.092880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272d0bf-8222-4b51-905e-49ec1bcc07e2 None admin] PATH_INFO: `/v3/projects/53641d46504c444d9350b7e021f6a4a4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:57.094856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272d0bf-8222-4b51-905e-49ec1bcc07e2 None admin] RBAC: Authorizing `identity:get_project(project_id=53641d46504c444d9350b7e021f6a4a4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:57.096969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272d0bf-8222-4b51-905e-49ec1bcc07e2 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:57.098596 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 188/377] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:56 2022] GET /identity/v3/projects/53641d46504c444d9350b7e021f6a4a4 => generated 344 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:57.107856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272d0bf-8222-4b51-905e-49ec1bcc07e2 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:57.161259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:57.209941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60ebc4-9ed6-4afd-9e7a-7112feea6c59 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:57.257668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8454c-60d4-4c9b-944d-0c14642dca69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:57.258303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8454c-60d4-4c9b-944d-0c14642dca69 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:57.258588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8454c-60d4-4c9b-944d-0c14642dca69 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:57.258861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8454c-60d4-4c9b-944d-0c14642dca69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:57.259607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8454c-60d4-4c9b-944d-0c14642dca69 None admin] RBAC: Adding query filter params (group.id=a62b4bf100a4409b823a863854490ce7, role.id=542f58433b2747d2abd5d405ebabdea1, scope.project.id=0a64615765ab4a01a30fc0023bb081c4) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:57.259923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8454c-60d4-4c9b-944d-0c14642dca69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:57.261423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8454c-60d4-4c9b-944d-0c14642dca69 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:57.278873 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 190/378] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 3 20:32:57 2022] GET /identity/v3/role_assignments?group.id=a62b4bf100a4409b823a863854490ce7&scope.project.id=0a64615765ab4a01a30fc0023bb081c4&role.id=542f58433b2747d2abd5d405ebabdea1 => generated 260 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:57.305061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272d0bf-8222-4b51-905e-49ec1bcc07e2 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:57.352667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09c20d-ab64-4f6f-b194-bdd860df38c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:57.353373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09c20d-ab64-4f6f-b194-bdd860df38c8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:57.353665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09c20d-ab64-4f6f-b194-bdd860df38c8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:57.353938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09c20d-ab64-4f6f-b194-bdd860df38c8 None admin] PATH_INFO: `/v3/groups/836551814bbd43b994dffb2b4449a309` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:57.364098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09c20d-ab64-4f6f-b194-bdd860df38c8 None admin] RBAC: Authorizing `identity:get_group(group_id=836551814bbd43b994dffb2b4449a309)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:57.365615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09c20d-ab64-4f6f-b194-bdd860df38c8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:57.367603 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 189/379] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:32:57 2022] GET /identity/v3/groups/836551814bbd43b994dffb2b4449a309 => generated 229 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:57.373429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8454c-60d4-4c9b-944d-0c14642dca69 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:57.461605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:57.520341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8454c-60d4-4c9b-944d-0c14642dca69 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:57.556242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c9873-1b08-4546-b62a-32a2cc53ef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:57.556856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c9873-1b08-4546-b62a-32a2cc53ef60 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:57.557189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c9873-1b08-4546-b62a-32a2cc53ef60 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:57.557614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c9873-1b08-4546-b62a-32a2cc53ef60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:57.558358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c9873-1b08-4546-b62a-32a2cc53ef60 None admin] RBAC: Adding query filter params (group.id=836551814bbd43b994dffb2b4449a309, role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=53641d46504c444d9350b7e021f6a4a4) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:57.558629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c9873-1b08-4546-b62a-32a2cc53ef60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:57.560080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c9873-1b08-4546-b62a-32a2cc53ef60 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:57.566760 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 191/380] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 3 20:32:57 2022] GET /identity/v3/role_assignments?group.id=836551814bbd43b994dffb2b4449a309&scope.project.id=53641d46504c444d9350b7e021f6a4a4&role.id=bc768522dc984239be4a1b39a45ab437 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:57.919366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f9ebd-0d4b-4e70-b43b-721881a1869e None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:57.920035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f9ebd-0d4b-4e70-b43b-721881a1869e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:57.920517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f9ebd-0d4b-4e70-b43b-721881a1869e None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:57.922384 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 190/381] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:57.928475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36357077-546d-4825-aa6f-de7f2711502e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:57.928772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36357077-546d-4825-aa6f-de7f2711502e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:57.929083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36357077-546d-4825-aa6f-de7f2711502e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:57.950855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-36357077-546d-4825-aa6f-de7f2711502e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:57.952081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-36357077-546d-4825-aa6f-de7f2711502e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:58.039956 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 192/382] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:58.502723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f9ebd-0d4b-4e70-b43b-721881a1869e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:58.578481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a5e7a-ddf8-4e92-8057-f121963d9a24 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:58.578974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a5e7a-ddf8-4e92-8057-f121963d9a24 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:58.579048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a5e7a-ddf8-4e92-8057-f121963d9a24 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:58.581217 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 193/383] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:32:58.586650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada88d4-863c-4c4c-9a60-a29cedf8d0f9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:58.586940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada88d4-863c-4c4c-9a60-a29cedf8d0f9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:58.587217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada88d4-863c-4c4c-9a60-a29cedf8d0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:58.642985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:58.648436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f9ebd-0d4b-4e70-b43b-721881a1869e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:58.650087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1ada88d4-863c-4c4c-9a60-a29cedf8d0f9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:58.651377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1ada88d4-863c-4c4c-9a60-a29cedf8d0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:58.687613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ee8e3-4913-4f26-86e3-c9ecfa576b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:58.688282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ee8e3-4913-4f26-86e3-c9ecfa576b98 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:58.688570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ee8e3-4913-4f26-86e3-c9ecfa576b98 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:58.688851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ee8e3-4913-4f26-86e3-c9ecfa576b98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:58.702071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ee8e3-4913-4f26-86e3-c9ecfa576b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-5d4ee8e3-4913-4f26-86e3-c9ecfa576b98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:58.707521 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:58.709670 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:58.712142 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 191/384] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:58.721583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ee8e3-4913-4f26-86e3-c9ecfa576b98 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:58.766468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:58.798986 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 194/385] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:58.846783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa630b-eff1-45d9-bb53-950a16ea299b None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:58.847575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa630b-eff1-45d9-bb53-950a16ea299b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:58.848445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa630b-eff1-45d9-bb53-950a16ea299b None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:58.849993 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 195/386] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:32:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:32:58.856768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ee8e3-4913-4f26-86e3-c9ecfa576b98 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:58.857939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb454e3e-e5ad-4535-ba33-09c465e2d487 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:58.858432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb454e3e-e5ad-4535-ba33-09c465e2d487 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:58.858913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb454e3e-e5ad-4535-ba33-09c465e2d487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:58.896355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fb454e3e-e5ad-4535-ba33-09c465e2d487 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:32:58.898864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fb454e3e-e5ad-4535-ba33-09c465e2d487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:32:58.908711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d59102-37cf-4e79-a8dc-bc967a06150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:58.909891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d59102-37cf-4e79-a8dc-bc967a06150c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:58.910179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d59102-37cf-4e79-a8dc-bc967a06150c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:58.910461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d59102-37cf-4e79-a8dc-bc967a06150c None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:58.911249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d59102-37cf-4e79-a8dc-bc967a06150c None admin] RBAC: Adding query filter params (name=member) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:58.911556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d59102-37cf-4e79-a8dc-bc967a06150c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:58.914112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d59102-37cf-4e79-a8dc-bc967a06150c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:58.932125 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 192/387] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:58.940137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d59102-37cf-4e79-a8dc-bc967a06150c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:58.962897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:58.994417 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 196/388] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:32:59.120160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d59102-37cf-4e79-a8dc-bc967a06150c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:59.165676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fa15b-fd6b-49bb-86fc-ce79a3cc7216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:59.166330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fa15b-fd6b-49bb-86fc-ce79a3cc7216 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:59.166972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fa15b-fd6b-49bb-86fc-ce79a3cc7216 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:59.167258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fa15b-fd6b-49bb-86fc-ce79a3cc7216 None admin] PATH_INFO: `/v3/groups/a62b4bf100a4409b823a863854490ce7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:59.168394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fa15b-fd6b-49bb-86fc-ce79a3cc7216 None admin] RBAC: Authorizing `identity:get_group(group_id=a62b4bf100a4409b823a863854490ce7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:59.169883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fa15b-fd6b-49bb-86fc-ce79a3cc7216 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:59.171460 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 193/389] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:32:58 2022] GET /identity/v3/groups/a62b4bf100a4409b823a863854490ce7 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:59.177993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb454e3e-e5ad-4535-ba33-09c465e2d487 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:59.235150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fa15b-fd6b-49bb-86fc-ce79a3cc7216 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:59.288564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb454e3e-e5ad-4535-ba33-09c465e2d487 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:59.304572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:59.327608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8848f-0d77-40ee-83ec-98a4d9687a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:59.328248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8848f-0d77-40ee-83ec-98a4d9687a9b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:59.328529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8848f-0d77-40ee-83ec-98a4d9687a9b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:59.328802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8848f-0d77-40ee-83ec-98a4d9687a9b None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:59.330580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8848f-0d77-40ee-83ec-98a4d9687a9b None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:59.332445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8848f-0d77-40ee-83ec-98a4d9687a9b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:59.334379 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 197/390] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:59 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:59.340034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8848f-0d77-40ee-83ec-98a4d9687a9b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:59.347558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:59.389972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fa15b-fd6b-49bb-86fc-ce79a3cc7216 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:59.444777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f761f7-23a6-455c-af6a-f97d2c2b81b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:59.445547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f761f7-23a6-455c-af6a-f97d2c2b81b0 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:59.445840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f761f7-23a6-455c-af6a-f97d2c2b81b0 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:59.446211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f761f7-23a6-455c-af6a-f97d2c2b81b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:59.449358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8848f-0d77-40ee-83ec-98a4d9687a9b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:59.457772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f761f7-23a6-455c-af6a-f97d2c2b81b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-d8f761f7-23a6-455c-af6a-f97d2c2b81b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:59.463693 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:59.465868 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:32:59.467721 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 194/391] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:32:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:59.477188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f761f7-23a6-455c-af6a-f97d2c2b81b0 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:59.489151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17222c65-dc28-4551-bfa9-6a284268c667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:59.489825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17222c65-dc28-4551-bfa9-6a284268c667 None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:59.490112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17222c65-dc28-4551-bfa9-6a284268c667 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:59.490395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17222c65-dc28-4551-bfa9-6a284268c667 None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11/groups/a62b4bf100a4409b823a863854490ce7/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:59.492457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17222c65-dc28-4551-bfa9-6a284268c667 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb6a16a02b534f94be33eea4c77eea11, group_id=a62b4bf100a4409b823a863854490ce7, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:59.493931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17222c65-dc28-4551-bfa9-6a284268c667 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:59.515109 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 198/392] 10.222.0.16 () {58 vars in 1442 bytes} [Sat Sep 3 20:32:59 2022] PUT /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11/groups/a62b4bf100a4409b823a863854490ce7/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:32:59.515984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17222c65-dc28-4551-bfa9-6a284268c667 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:59.581379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:59.601621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f761f7-23a6-455c-af6a-f97d2c2b81b0 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:59.655089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75766190-11f0-418b-9fa0-7201c7b970eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:59.655651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75766190-11f0-418b-9fa0-7201c7b970eb None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:59.655941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75766190-11f0-418b-9fa0-7201c7b970eb None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:59.656228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75766190-11f0-418b-9fa0-7201c7b970eb None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:59.656941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75766190-11f0-418b-9fa0-7201c7b970eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:59.657259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75766190-11f0-418b-9fa0-7201c7b970eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:59.659261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75766190-11f0-418b-9fa0-7201c7b970eb None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:59.668026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:59.682361 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 195/393] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:32:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:59.684509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17222c65-dc28-4551-bfa9-6a284268c667 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:59.691970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75766190-11f0-418b-9fa0-7201c7b970eb None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:59.747602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dbd1d-9d92-436f-b955-4b78b7e98e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:59.748277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dbd1d-9d92-436f-b955-4b78b7e98e9f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:59.748566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dbd1d-9d92-436f-b955-4b78b7e98e9f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:59.748850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dbd1d-9d92-436f-b955-4b78b7e98e9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:59.762372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dbd1d-9d92-436f-b955-4b78b7e98e9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-ec0dbd1d-9d92-436f-b955-4b78b7e98e9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:32:59.770636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:32:59.771584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:32:59.772448 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 199/394] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:32:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:32:59.781696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dbd1d-9d92-436f-b955-4b78b7e98e9f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:59.861526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75766190-11f0-418b-9fa0-7201c7b970eb None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:59.904499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08322215-4474-4ac0-93cf-d44c10c47e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:59.905231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08322215-4474-4ac0-93cf-d44c10c47e64 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:59.905464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08322215-4474-4ac0-93cf-d44c10c47e64 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:59.905747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08322215-4474-4ac0-93cf-d44c10c47e64 None admin] PATH_INFO: `/v3/groups/a62b4bf100a4409b823a863854490ce7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:59.906883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08322215-4474-4ac0-93cf-d44c10c47e64 None admin] RBAC: Authorizing `identity:get_group(group_id=a62b4bf100a4409b823a863854490ce7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:59.909057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08322215-4474-4ac0-93cf-d44c10c47e64 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:59.915259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dbd1d-9d92-436f-b955-4b78b7e98e9f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:32:59.915490 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 196/395] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:32:59 2022] GET /identity/v3/groups/a62b4bf100a4409b823a863854490ce7 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:32:59.916675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08322215-4474-4ac0-93cf-d44c10c47e64 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:59.917663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:32:59.957357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e1000-1bca-4af2-ac64-943a92aba96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:32:59.958409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e1000-1bca-4af2-ac64-943a92aba96d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:32:59.958902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e1000-1bca-4af2-ac64-943a92aba96d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:32:59.959385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e1000-1bca-4af2-ac64-943a92aba96d None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:32:59.960298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e1000-1bca-4af2-ac64-943a92aba96d None admin] RBAC: Adding query filter params (name=member) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:32:59.960823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e1000-1bca-4af2-ac64-943a92aba96d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:32:59.962505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e1000-1bca-4af2-ac64-943a92aba96d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:32:59.972813 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 200/396] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:32:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:32:59.981926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e1000-1bca-4af2-ac64-943a92aba96d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:32:59.996938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:00.036922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08322215-4474-4ac0-93cf-d44c10c47e64 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:00.070994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d07e9-ee63-4387-9575-285dc6fc0a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:00.071636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d07e9-ee63-4387-9575-285dc6fc0a3b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:00.071921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d07e9-ee63-4387-9575-285dc6fc0a3b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:00.072198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d07e9-ee63-4387-9575-285dc6fc0a3b None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:00.073462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d07e9-ee63-4387-9575-285dc6fc0a3b None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:00.074929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d07e9-ee63-4387-9575-285dc6fc0a3b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:00.084952 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 197/397] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:32:59 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:00.084952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d07e9-ee63-4387-9575-285dc6fc0a3b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:00.153882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e1000-1bca-4af2-ac64-943a92aba96d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:00.201437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0dc95-0656-473a-a2d7-e4b3df846f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:00.202099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0dc95-0656-473a-a2d7-e4b3df846f6e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:00.202383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0dc95-0656-473a-a2d7-e4b3df846f6e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:00.202656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0dc95-0656-473a-a2d7-e4b3df846f6e None admin] PATH_INFO: `/v3/groups/a62b4bf100a4409b823a863854490ce7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:00.205829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0dc95-0656-473a-a2d7-e4b3df846f6e None admin] RBAC: Authorizing `identity:get_group(group_id=a62b4bf100a4409b823a863854490ce7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:00.207624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0dc95-0656-473a-a2d7-e4b3df846f6e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:00.214170 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 201/398] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:32:59 2022] GET /identity/v3/groups/a62b4bf100a4409b823a863854490ce7 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:00.214170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8c834-eb40-46f3-aa93-6a4db382dd3c None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:00.214170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8c834-eb40-46f3-aa93-6a4db382dd3c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:00.214170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8c834-eb40-46f3-aa93-6a4db382dd3c None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:00.214170 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 202/399] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:00.216992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8c834-eb40-46f3-aa93-6a4db382dd3c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:00.230451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d07e9-ee63-4387-9575-285dc6fc0a3b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:00.261783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:00.266651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7929f-24e4-49cf-aa9c-6b2ed648c855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:00.268009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7929f-24e4-49cf-aa9c-6b2ed648c855 None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:00.268309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7929f-24e4-49cf-aa9c-6b2ed648c855 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:00.268601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7929f-24e4-49cf-aa9c-6b2ed648c855 None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11/groups/a62b4bf100a4409b823a863854490ce7/roles/542f58433b2747d2abd5d405ebabdea1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:00.271135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7929f-24e4-49cf-aa9c-6b2ed648c855 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb6a16a02b534f94be33eea4c77eea11, group_id=a62b4bf100a4409b823a863854490ce7, role_id=542f58433b2747d2abd5d405ebabdea1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:00.272601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7929f-24e4-49cf-aa9c-6b2ed648c855 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:00.275095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:00.295994 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 198/400] 10.222.0.16 () {58 vars in 1442 bytes} [Sat Sep 3 20:33:00 2022] PUT /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11/groups/a62b4bf100a4409b823a863854490ce7/roles/542f58433b2747d2abd5d405ebabdea1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:33:00.297672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50025644-1b8c-4ed6-be6c-1e69ca3f6308 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:00.298201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50025644-1b8c-4ed6-be6c-1e69ca3f6308 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:00.299237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50025644-1b8c-4ed6-be6c-1e69ca3f6308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:00.340769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-50025644-1b8c-4ed6-be6c-1e69ca3f6308 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:00.343539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-50025644-1b8c-4ed6-be6c-1e69ca3f6308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:00.402621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8c834-eb40-46f3-aa93-6a4db382dd3c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:00.456310 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 199/401] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:00.456310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c4652-d850-4a3f-9fed-de073c448f6d None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:00.456310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c4652-d850-4a3f-9fed-de073c448f6d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:00.456310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c4652-d850-4a3f-9fed-de073c448f6d None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:00.456957 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 200/402] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:00.459387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4e0f3-498d-4d46-a99d-b798edf85fce None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:00.459678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4e0f3-498d-4d46-a99d-b798edf85fce None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:00.459961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4e0f3-498d-4d46-a99d-b798edf85fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:00.465274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658fd93-6f59-4bd1-82f9-33dcd0e2fb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:00.465934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658fd93-6f59-4bd1-82f9-33dcd0e2fb20 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:00.466224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658fd93-6f59-4bd1-82f9-33dcd0e2fb20 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:00.466506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658fd93-6f59-4bd1-82f9-33dcd0e2fb20 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:00.467829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658fd93-6f59-4bd1-82f9-33dcd0e2fb20 None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:00.469504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658fd93-6f59-4bd1-82f9-33dcd0e2fb20 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:00.471883 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 203/403] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:33:00 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:00.480098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658fd93-6f59-4bd1-82f9-33dcd0e2fb20 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:00.500434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bed4e0f3-498d-4d46-a99d-b798edf85fce None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:00.501718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bed4e0f3-498d-4d46-a99d-b798edf85fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:00.580641 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 201/404] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:00.606786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:00.610604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658fd93-6f59-4bd1-82f9-33dcd0e2fb20 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:00.662616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab7374-beef-4f8f-9058-f61c3086e155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:00.663294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab7374-beef-4f8f-9058-f61c3086e155 None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:00.663581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab7374-beef-4f8f-9058-f61c3086e155 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:00.663865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab7374-beef-4f8f-9058-f61c3086e155 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4/groups/a62b4bf100a4409b823a863854490ce7/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:00.667200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab7374-beef-4f8f-9058-f61c3086e155 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a64615765ab4a01a30fc0023bb081c4, group_id=a62b4bf100a4409b823a863854490ce7, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:00.668710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab7374-beef-4f8f-9058-f61c3086e155 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:00.697718 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 204/405] 10.222.0.16 () {58 vars in 1442 bytes} [Sat Sep 3 20:33:00 2022] PUT /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4/groups/a62b4bf100a4409b823a863854490ce7/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:33:00.818367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4e0f3-498d-4d46-a99d-b798edf85fce None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:00.891907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:00.953683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4e0f3-498d-4d46-a99d-b798edf85fce None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:00.994404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc3e84-3ff8-45e8-b13b-5e365f412c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:00.995062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc3e84-3ff8-45e8-b13b-5e365f412c79 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:00.995349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc3e84-3ff8-45e8-b13b-5e365f412c79 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:00.995632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc3e84-3ff8-45e8-b13b-5e365f412c79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:01.007358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc3e84-3ff8-45e8-b13b-5e365f412c79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-fdbc3e84-3ff8-45e8-b13b-5e365f412c79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:01.012474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:01.013450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:01.014236 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 202/406] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:33:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:01.020351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab7374-beef-4f8f-9058-f61c3086e155 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:01.066535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc3e84-3ff8-45e8-b13b-5e365f412c79 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:01.152818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab7374-beef-4f8f-9058-f61c3086e155 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:01.188580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc3e84-3ff8-45e8-b13b-5e365f412c79 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:01.190024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20021c-9d63-48c0-ad37-af2091cc4841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:01.190823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20021c-9d63-48c0-ad37-af2091cc4841 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:01.191277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20021c-9d63-48c0-ad37-af2091cc4841 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:01.191717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20021c-9d63-48c0-ad37-af2091cc4841 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:01.192551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20021c-9d63-48c0-ad37-af2091cc4841 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:01.193333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20021c-9d63-48c0-ad37-af2091cc4841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:01.194881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20021c-9d63-48c0-ad37-af2091cc4841 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:01.212151 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 205/407] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:01.217311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:01.222322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20021c-9d63-48c0-ad37-af2091cc4841 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:01.250707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f0cc2-f4c9-4301-89de-e0920d9e2625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:01.251352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f0cc2-f4c9-4301-89de-e0920d9e2625 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:01.251670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f0cc2-f4c9-4301-89de-e0920d9e2625 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:01.251951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f0cc2-f4c9-4301-89de-e0920d9e2625 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:01.268309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:01.274264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f0cc2-f4c9-4301-89de-e0920d9e2625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-3a1f0cc2-f4c9-4301-89de-e0920d9e2625 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:01.279174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:01.280089 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:01.280875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:01.280875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:01.280875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:01.280875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:01.280875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:01.280875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:01.280875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:01.280875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:01.280875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:01.280875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:01.280875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:01.280875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:01.281500 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 203/408] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:01.293900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f0cc2-f4c9-4301-89de-e0920d9e2625 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:01.359548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20021c-9d63-48c0-ad37-af2091cc4841 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:01.395536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74bc33-3535-475d-a474-8aa9c2202a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:01.396523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74bc33-3535-475d-a474-8aa9c2202a95 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:01.397069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74bc33-3535-475d-a474-8aa9c2202a95 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:01.397555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74bc33-3535-475d-a474-8aa9c2202a95 None admin] PATH_INFO: `/v3/groups/a62b4bf100a4409b823a863854490ce7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:01.398949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74bc33-3535-475d-a474-8aa9c2202a95 None admin] RBAC: Authorizing `identity:get_group(group_id=a62b4bf100a4409b823a863854490ce7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:01.400633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74bc33-3535-475d-a474-8aa9c2202a95 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:01.402811 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 206/409] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:33:01 2022] GET /identity/v3/groups/a62b4bf100a4409b823a863854490ce7 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:01.412968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74bc33-3535-475d-a474-8aa9c2202a95 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:01.513420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:01.521446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74bc33-3535-475d-a474-8aa9c2202a95 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:01.526855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f0cc2-f4c9-4301-89de-e0920d9e2625 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:01.558408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aefb27-9972-4e90-8de7-def512786955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:01.559397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aefb27-9972-4e90-8de7-def512786955 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:01.559889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aefb27-9972-4e90-8de7-def512786955 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:01.560371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aefb27-9972-4e90-8de7-def512786955 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:01.561965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aefb27-9972-4e90-8de7-def512786955 None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:01.563693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aefb27-9972-4e90-8de7-def512786955 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:01.566591 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 207/410] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:33:01 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:01.574346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aefb27-9972-4e90-8de7-def512786955 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:01.607691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf04ab-8a42-4965-aae0-67cf1a1417a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:01.608312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf04ab-8a42-4965-aae0-67cf1a1417a0 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:01.608594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf04ab-8a42-4965-aae0-67cf1a1417a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:01.608867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf04ab-8a42-4965-aae0-67cf1a1417a0 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:01.609576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf04ab-8a42-4965-aae0-67cf1a1417a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:01.609885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf04ab-8a42-4965-aae0-67cf1a1417a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:01.611810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf04ab-8a42-4965-aae0-67cf1a1417a0 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:01.629632 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 204/411] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:01.637340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf04ab-8a42-4965-aae0-67cf1a1417a0 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:01.713269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:01.767165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aefb27-9972-4e90-8de7-def512786955 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:01.823447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf04ab-8a42-4965-aae0-67cf1a1417a0 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:01.833824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58375e62-262f-4980-84f0-37ea2e862d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:01.834500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58375e62-262f-4980-84f0-37ea2e862d34 None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:01.834786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58375e62-262f-4980-84f0-37ea2e862d34 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:01.835064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58375e62-262f-4980-84f0-37ea2e862d34 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4/groups/a62b4bf100a4409b823a863854490ce7/roles/542f58433b2747d2abd5d405ebabdea1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:01.837300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58375e62-262f-4980-84f0-37ea2e862d34 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a64615765ab4a01a30fc0023bb081c4, group_id=a62b4bf100a4409b823a863854490ce7, role_id=542f58433b2747d2abd5d405ebabdea1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:01.838751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58375e62-262f-4980-84f0-37ea2e862d34 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:01.860484 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 208/412] 10.222.0.16 () {58 vars in 1442 bytes} [Sat Sep 3 20:33:01 2022] PUT /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4/groups/a62b4bf100a4409b823a863854490ce7/roles/542f58433b2747d2abd5d405ebabdea1 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:33:01.889190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9009372-5f6a-4308-875e-0dda88075d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:01.890008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9009372-5f6a-4308-875e-0dda88075d87 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:01.890302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9009372-5f6a-4308-875e-0dda88075d87 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:01.890585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9009372-5f6a-4308-875e-0dda88075d87 None admin] PATH_INFO: `/v3/groups/836551814bbd43b994dffb2b4449a309` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:01.893949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9009372-5f6a-4308-875e-0dda88075d87 None admin] RBAC: Authorizing `identity:get_group(group_id=836551814bbd43b994dffb2b4449a309)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:01.895462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9009372-5f6a-4308-875e-0dda88075d87 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:01.897405 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 205/413] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:33:01 2022] GET /identity/v3/groups/836551814bbd43b994dffb2b4449a309 => generated 229 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:01.903433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58375e62-262f-4980-84f0-37ea2e862d34 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:01.926244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:02.037938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0221da-bda3-4ca2-8ee8-e30489849c2d None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:02.038240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0221da-bda3-4ca2-8ee8-e30489849c2d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:02.038420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0221da-bda3-4ca2-8ee8-e30489849c2d None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:02.040057 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 206/414] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:02.045054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58375e62-262f-4980-84f0-37ea2e862d34 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:02.045628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe978d8d-3d2d-43d0-a41b-72a5ef016827 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:02.045920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe978d8d-3d2d-43d0-a41b-72a5ef016827 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:02.046192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe978d8d-3d2d-43d0-a41b-72a5ef016827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:02.066615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fe978d8d-3d2d-43d0-a41b-72a5ef016827 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:02.066976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fe978d8d-3d2d-43d0-a41b-72a5ef016827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:02.100334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e456d-2072-4fb2-bcf6-c976da420d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:02.101162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e456d-2072-4fb2-bcf6-c976da420d73 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:02.101583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e456d-2072-4fb2-bcf6-c976da420d73 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:02.101869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e456d-2072-4fb2-bcf6-c976da420d73 None admin] PATH_INFO: `/v3/projects/53641d46504c444d9350b7e021f6a4a4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:02.103021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e456d-2072-4fb2-bcf6-c976da420d73 None admin] RBAC: Authorizing `identity:get_project(project_id=53641d46504c444d9350b7e021f6a4a4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:02.105001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e456d-2072-4fb2-bcf6-c976da420d73 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:02.112262 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 209/415] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:33:01 2022] GET /identity/v3/projects/53641d46504c444d9350b7e021f6a4a4 => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:02.114205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e456d-2072-4fb2-bcf6-c976da420d73 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:02.163449 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 207/416] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:33:02.227845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:02.243992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e456d-2072-4fb2-bcf6-c976da420d73 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:02.279286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7919451-9691-435c-8595-01a8175ed84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:02.279955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7919451-9691-435c-8595-01a8175ed84a None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:02.280254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7919451-9691-435c-8595-01a8175ed84a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:02.280520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7919451-9691-435c-8595-01a8175ed84a None admin] PATH_INFO: `/v3/projects/53641d46504c444d9350b7e021f6a4a4/groups/836551814bbd43b994dffb2b4449a309/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:02.282512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7919451-9691-435c-8595-01a8175ed84a None admin] RBAC: Authorizing `identity:create_grant(project_id=53641d46504c444d9350b7e021f6a4a4, group_id=836551814bbd43b994dffb2b4449a309, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:02.284530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7919451-9691-435c-8595-01a8175ed84a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:02.305623 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 210/417] 10.222.0.16 () {58 vars in 1442 bytes} [Sat Sep 3 20:33:02 2022] PUT /identity/v3/projects/53641d46504c444d9350b7e021f6a4a4/groups/836551814bbd43b994dffb2b4449a309/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:33:02.588280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe978d8d-3d2d-43d0-a41b-72a5ef016827 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:02.622334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:02.698268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe978d8d-3d2d-43d0-a41b-72a5ef016827 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:02.733125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8e7b1-3ac9-4fb4-bc97-3cad4d81a2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:02.733759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8e7b1-3ac9-4fb4-bc97-3cad4d81a2f5 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:02.734046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8e7b1-3ac9-4fb4-bc97-3cad4d81a2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:02.734322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8e7b1-3ac9-4fb4-bc97-3cad4d81a2f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:02.744614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8e7b1-3ac9-4fb4-bc97-3cad4d81a2f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-f0e8e7b1-3ac9-4fb4-bc97-3cad4d81a2f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:02.749412 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:02.750267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:02.751068 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 208/418] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:02.759894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7919451-9691-435c-8595-01a8175ed84a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:02.899417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7919451-9691-435c-8595-01a8175ed84a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:02.937772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce341e8b-06ba-462b-b73e-c4ecad6bcc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:02.938396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce341e8b-06ba-462b-b73e-c4ecad6bcc06 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:02.938684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce341e8b-06ba-462b-b73e-c4ecad6bcc06 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:02.939036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce341e8b-06ba-462b-b73e-c4ecad6bcc06 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:02.940216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce341e8b-06ba-462b-b73e-c4ecad6bcc06 None admin] RBAC: Adding query filter params (name=member) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:02.940528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce341e8b-06ba-462b-b73e-c4ecad6bcc06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:02.941976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce341e8b-06ba-462b-b73e-c4ecad6bcc06 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:02.952185 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 211/419] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:33:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:02.957842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8e7b1-3ac9-4fb4-bc97-3cad4d81a2f5 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:03.080097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:03.091363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8e7b1-3ac9-4fb4-bc97-3cad4d81a2f5 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:03.134070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16744c0-29f7-48d4-b12a-d27d8844e392 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:03.134361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16744c0-29f7-48d4-b12a-d27d8844e392 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:03.134646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16744c0-29f7-48d4-b12a-d27d8844e392 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:03.136119 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 212/420] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:03.143823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13820b-6578-4bc8-aa23-fc47b0fe2287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:03.144450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13820b-6578-4bc8-aa23-fc47b0fe2287 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:03.144737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13820b-6578-4bc8-aa23-fc47b0fe2287 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:03.145053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13820b-6578-4bc8-aa23-fc47b0fe2287 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:03.153433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cd59b-ac5f-47e1-a865-70d41f18d7dc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:03.153674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cd59b-ac5f-47e1-a865-70d41f18d7dc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:03.153932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cd59b-ac5f-47e1-a865-70d41f18d7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:03.179262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-661cd59b-ac5f-47e1-a865-70d41f18d7dc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:03.180496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-661cd59b-ac5f-47e1-a865-70d41f18d7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:03.183189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:03.258814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13820b-6578-4bc8-aa23-fc47b0fe2287 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:03.260653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13820b-6578-4bc8-aa23-fc47b0fe2287 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:03.288795 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 213/421] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:33:03.401325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:03.411866 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 209/422] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:33:03.422020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cd59b-ac5f-47e1-a865-70d41f18d7dc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:03.546626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cd59b-ac5f-47e1-a865-70d41f18d7dc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:03.588726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c90926-820e-4fc2-98e7-30dd77b1e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:03.589416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c90926-820e-4fc2-98e7-30dd77b1e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:03.589708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c90926-820e-4fc2-98e7-30dd77b1e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:03.589990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c90926-820e-4fc2-98e7-30dd77b1e0f2 None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:03.591206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c90926-820e-4fc2-98e7-30dd77b1e0f2 None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:03.592702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c90926-820e-4fc2-98e7-30dd77b1e0f2 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:03.594616 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 214/423] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:33:03 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:03.600184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13820b-6578-4bc8-aa23-fc47b0fe2287 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:03.708367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c90926-820e-4fc2-98e7-30dd77b1e0f2 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:03.737265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:03.761192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13820b-6578-4bc8-aa23-fc47b0fe2287 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:03.798149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d31f2-391d-4636-a47a-462ed791a27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:03.799315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d31f2-391d-4636-a47a-462ed791a27a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:03.799607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d31f2-391d-4636-a47a-462ed791a27a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:03.799886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d31f2-391d-4636-a47a-462ed791a27a None admin] PATH_INFO: `/v3/groups/a62b4bf100a4409b823a863854490ce7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:03.801155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d31f2-391d-4636-a47a-462ed791a27a None admin] RBAC: Authorizing `identity:get_group(group_id=a62b4bf100a4409b823a863854490ce7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:03.802774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d31f2-391d-4636-a47a-462ed791a27a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:03.804725 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 210/424] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:33:03 2022] GET /identity/v3/groups/a62b4bf100a4409b823a863854490ce7 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:03.806259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cb713-c982-454b-9fe5-c4802241c7f9 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:03.806552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cb713-c982-454b-9fe5-c4802241c7f9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:03.806833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cb713-c982-454b-9fe5-c4802241c7f9 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:03.812071 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 211/425] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:03.817085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed5552-bdd7-4503-830b-a592cf8404b1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:03.817085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed5552-bdd7-4503-830b-a592cf8404b1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:03.817085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed5552-bdd7-4503-830b-a592cf8404b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:03.833027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-62ed5552-bdd7-4503-830b-a592cf8404b1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:03.834237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-62ed5552-bdd7-4503-830b-a592cf8404b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:03.871188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c90926-820e-4fc2-98e7-30dd77b1e0f2 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:03.908951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:03.916629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1465d8-f84e-4160-8d1a-31145b927644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:03.917311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1465d8-f84e-4160-8d1a-31145b927644 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:03.917622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1465d8-f84e-4160-8d1a-31145b927644 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:03.917880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1465d8-f84e-4160-8d1a-31145b927644 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:03.924986 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 212/426] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:03.925942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed5552-bdd7-4503-830b-a592cf8404b1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:03.941123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1465d8-f84e-4160-8d1a-31145b927644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-0d1465d8-f84e-4160-8d1a-31145b927644 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:03.946827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:03.947748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:03.948536 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:03.948536 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:03.948536 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:03.948536 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:03.948536 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:03.948536 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:03.948536 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:03.948536 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:03.948536 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:03.948536 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:03.948536 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:33:03.948536 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:03.955957 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 215/427] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:33:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:03.958425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1465d8-f84e-4160-8d1a-31145b927644 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:04.088283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:04.133258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed5552-bdd7-4503-830b-a592cf8404b1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:04.162550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1465d8-f84e-4160-8d1a-31145b927644 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:04.207483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e28470-5f06-4f9d-ad15-0441a0ac6fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:04.208110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e28470-5f06-4f9d-ad15-0441a0ac6fe8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:04.208498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e28470-5f06-4f9d-ad15-0441a0ac6fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:04.208730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e28470-5f06-4f9d-ad15-0441a0ac6fe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:04.209606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e28470-5f06-4f9d-ad15-0441a0ac6fe8 None admin] RBAC: Adding query filter params (group.id=a62b4bf100a4409b823a863854490ce7, role.id=d167c236cee64ede8d62bce075f83fff, scope.project.id=bb6a16a02b534f94be33eea4c77eea11) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:04.209926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e28470-5f06-4f9d-ad15-0441a0ac6fe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:04.211376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e28470-5f06-4f9d-ad15-0441a0ac6fe8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:04.222811 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 213/428] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 3 20:33:03 2022] GET /identity/v3/role_assignments?group.id=a62b4bf100a4409b823a863854490ce7&scope.project.id=bb6a16a02b534f94be33eea4c77eea11&role.id=d167c236cee64ede8d62bce075f83fff => generated 611 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:04.237353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf577c-cddc-4cef-a9cf-c8405c2fb4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:04.237990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf577c-cddc-4cef-a9cf-c8405c2fb4ec None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:04.238280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf577c-cddc-4cef-a9cf-c8405c2fb4ec None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:04.238563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf577c-cddc-4cef-a9cf-c8405c2fb4ec None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:04.239246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf577c-cddc-4cef-a9cf-c8405c2fb4ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:04.239557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf577c-cddc-4cef-a9cf-c8405c2fb4ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:04.240963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf577c-cddc-4cef-a9cf-c8405c2fb4ec None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:04.256349 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 216/429] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:04.262508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e28470-5f06-4f9d-ad15-0441a0ac6fe8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:04.326878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:04.333654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf577c-cddc-4cef-a9cf-c8405c2fb4ec None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:04.397647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e28470-5f06-4f9d-ad15-0441a0ac6fe8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:04.445662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f8930-c374-4197-9b58-653128d69cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:04.446303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f8930-c374-4197-9b58-653128d69cb2 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:04.446593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f8930-c374-4197-9b58-653128d69cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:04.446874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f8930-c374-4197-9b58-653128d69cb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:04.460825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf577c-cddc-4cef-a9cf-c8405c2fb4ec None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:04.489896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:04.498438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc52eaf-a29b-42d9-93bd-1f1d22846f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:04.499085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc52eaf-a29b-42d9-93bd-1f1d22846f42 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:04.499369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc52eaf-a29b-42d9-93bd-1f1d22846f42 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:04.499668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc52eaf-a29b-42d9-93bd-1f1d22846f42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:04.511496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc52eaf-a29b-42d9-93bd-1f1d22846f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-fbc52eaf-a29b-42d9-93bd-1f1d22846f42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:04.516888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:04.518389 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:04.519195 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 217/430] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:04.524479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc52eaf-a29b-42d9-93bd-1f1d22846f42 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:04.568214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f8930-c374-4197-9b58-653128d69cb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:04.569751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f8930-c374-4197-9b58-653128d69cb2 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:04.634141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:04.635354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc52eaf-a29b-42d9-93bd-1f1d22846f42 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:04.674156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b9f95-f77f-4f6c-b546-a542c6795b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:04.675156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b9f95-f77f-4f6c-b546-a542c6795b2c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:04.675645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b9f95-f77f-4f6c-b546-a542c6795b2c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:04.676125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b9f95-f77f-4f6c-b546-a542c6795b2c None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:04.677051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b9f95-f77f-4f6c-b546-a542c6795b2c None admin] RBAC: Adding query filter params (name=member) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:04.677566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b9f95-f77f-4f6c-b546-a542c6795b2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:04.679269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b9f95-f77f-4f6c-b546-a542c6795b2c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:04.684527 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 214/431] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:33:04.687371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4d6cf-cd5a-431d-ab1a-120d9b6d355e None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:04.687669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4d6cf-cd5a-431d-ab1a-120d9b6d355e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:04.687953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4d6cf-cd5a-431d-ab1a-120d9b6d355e None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:04.694214 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 218/432] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:33:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:04.694379 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 215/433] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:04.694471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b9f95-f77f-4f6c-b546-a542c6795b2c None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:04.695906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4d6cf-cd5a-431d-ab1a-120d9b6d355e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:04.749683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:04.825822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b9f95-f77f-4f6c-b546-a542c6795b2c None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:04.845561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4d6cf-cd5a-431d-ab1a-120d9b6d355e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:04.873338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab8bbe-b541-4364-8c33-5cab15e799e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:04.873681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab8bbe-b541-4364-8c33-5cab15e799e2 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:04.873989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab8bbe-b541-4364-8c33-5cab15e799e2 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:04.874291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab8bbe-b541-4364-8c33-5cab15e799e2 None admin] PATH_INFO: `/v3/projects/bb6a16a02b534f94be33eea4c77eea11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:04.875480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab8bbe-b541-4364-8c33-5cab15e799e2 None admin] RBAC: Authorizing `identity:get_project(project_id=bb6a16a02b534f94be33eea4c77eea11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:04.877533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab8bbe-b541-4364-8c33-5cab15e799e2 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:04.879345 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 219/434] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:33:04 2022] GET /identity/v3/projects/bb6a16a02b534f94be33eea4c77eea11 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:04.880365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c912a0-f294-4574-b330-304f6472a8ca None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:04.880659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c912a0-f294-4574-b330-304f6472a8ca None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:04.881528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c912a0-f294-4574-b330-304f6472a8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:04.892671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37b7a6-50ba-4e14-8fd8-45e7ed8af331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:04.893598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37b7a6-50ba-4e14-8fd8-45e7ed8af331 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:04.894041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37b7a6-50ba-4e14-8fd8-45e7ed8af331 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:04.894328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37b7a6-50ba-4e14-8fd8-45e7ed8af331 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:04.901631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-43c912a0-f294-4574-b330-304f6472a8ca None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:04.902824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-43c912a0-f294-4574-b330-304f6472a8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:04.922385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:04.971385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37b7a6-50ba-4e14-8fd8-45e7ed8af331 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:04.972865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37b7a6-50ba-4e14-8fd8-45e7ed8af331 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:04.977603 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 220/435] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:04.979005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c912a0-f294-4574-b330-304f6472a8ca None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:05.051301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:05.085265 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 216/436] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 20:33:05.090692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37b7a6-50ba-4e14-8fd8-45e7ed8af331 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:05.122849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c912a0-f294-4574-b330-304f6472a8ca None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:05.169834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:05.171490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12ecb3-5d78-4a67-8148-4ff63f47612b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:05.172095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12ecb3-5d78-4a67-8148-4ff63f47612b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:05.172380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12ecb3-5d78-4a67-8148-4ff63f47612b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:05.172657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12ecb3-5d78-4a67-8148-4ff63f47612b None admin] PATH_INFO: `/v3/groups/a62b4bf100a4409b823a863854490ce7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:05.173874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12ecb3-5d78-4a67-8148-4ff63f47612b None admin] RBAC: Authorizing `identity:get_group(group_id=a62b4bf100a4409b823a863854490ce7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:05.175336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12ecb3-5d78-4a67-8148-4ff63f47612b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:05.181264 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 221/437] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:33:04 2022] GET /identity/v3/groups/a62b4bf100a4409b823a863854490ce7 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:05.182932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12ecb3-5d78-4a67-8148-4ff63f47612b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:05.229474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37b7a6-50ba-4e14-8fd8-45e7ed8af331 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:05.273873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55167a-81ca-44bd-a310-8abab70492e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:05.274937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55167a-81ca-44bd-a310-8abab70492e3 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:05.275429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55167a-81ca-44bd-a310-8abab70492e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:05.275912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55167a-81ca-44bd-a310-8abab70492e3 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:05.277915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55167a-81ca-44bd-a310-8abab70492e3 None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:05.280091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55167a-81ca-44bd-a310-8abab70492e3 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:05.283684 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 217/438] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:33:05 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:05.286256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693a5b4-854c-41b3-9a85-49b81f6cbca3 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:05.286373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693a5b4-854c-41b3-9a85-49b81f6cbca3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:05.286593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693a5b4-854c-41b3-9a85-49b81f6cbca3 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:05.289137 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 218/439] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:33:05.289905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693a5b4-854c-41b3-9a85-49b81f6cbca3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:05.307313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12ecb3-5d78-4a67-8148-4ff63f47612b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:05.323528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:05.348504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e3e4f-4027-4fbb-a640-b7479459910a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:05.349151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e3e4f-4027-4fbb-a640-b7479459910a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:05.349443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e3e4f-4027-4fbb-a640-b7479459910a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:05.349721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e3e4f-4027-4fbb-a640-b7479459910a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:05.350454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e3e4f-4027-4fbb-a640-b7479459910a None admin] RBAC: Adding query filter params (group.id=a62b4bf100a4409b823a863854490ce7, role.id=542f58433b2747d2abd5d405ebabdea1, scope.project.id=bb6a16a02b534f94be33eea4c77eea11) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:05.350768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e3e4f-4027-4fbb-a640-b7479459910a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:05.352205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e3e4f-4027-4fbb-a640-b7479459910a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:05.357917 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 222/440] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 3 20:33:05 2022] GET /identity/v3/role_assignments?group.id=a62b4bf100a4409b823a863854490ce7&scope.project.id=bb6a16a02b534f94be33eea4c77eea11&role.id=542f58433b2747d2abd5d405ebabdea1 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:05.359048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992a0c5-13d6-4a60-9222-8406126d36a4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:05.359335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992a0c5-13d6-4a60-9222-8406126d36a4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:05.359608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992a0c5-13d6-4a60-9222-8406126d36a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:05.385823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b992a0c5-13d6-4a60-9222-8406126d36a4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:05.387185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b992a0c5-13d6-4a60-9222-8406126d36a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:05.405875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693a5b4-854c-41b3-9a85-49b81f6cbca3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:05.438339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:05.451444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013c12f-8b98-4e81-8c4e-0760ec301aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:05.452199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013c12f-8b98-4e81-8c4e-0760ec301aab None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:05.452471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013c12f-8b98-4e81-8c4e-0760ec301aab None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:05.452889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013c12f-8b98-4e81-8c4e-0760ec301aab None admin] PATH_INFO: `/v3/groups/a62b4bf100a4409b823a863854490ce7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:05.454020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013c12f-8b98-4e81-8c4e-0760ec301aab None admin] RBAC: Authorizing `identity:get_group(group_id=a62b4bf100a4409b823a863854490ce7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:05.456315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013c12f-8b98-4e81-8c4e-0760ec301aab None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:05.458529 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 219/441] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:33:05 2022] GET /identity/v3/groups/a62b4bf100a4409b823a863854490ce7 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:05.459344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013c12f-8b98-4e81-8c4e-0760ec301aab None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:05.472922 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 223/442] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:05.473156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992a0c5-13d6-4a60-9222-8406126d36a4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:05.583407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013c12f-8b98-4e81-8c4e-0760ec301aab None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:05.602831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992a0c5-13d6-4a60-9222-8406126d36a4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:05.616101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:05.628604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb5a3f-3566-4c91-9ef0-32d7779790ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:05.629315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb5a3f-3566-4c91-9ef0-32d7779790ef None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:05.629604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb5a3f-3566-4c91-9ef0-32d7779790ef None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:05.629884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb5a3f-3566-4c91-9ef0-32d7779790ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:05.641156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb5a3f-3566-4c91-9ef0-32d7779790ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-84cb5a3f-3566-4c91-9ef0-32d7779790ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:05.648279 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:05.649550 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:05.650395 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 220/443] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:33:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:05.652590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae947f3-4dc8-4bed-86d1-af27279e5e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:05.653280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae947f3-4dc8-4bed-86d1-af27279e5e6d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:05.653538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae947f3-4dc8-4bed-86d1-af27279e5e6d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:05.653819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae947f3-4dc8-4bed-86d1-af27279e5e6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:05.655016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae947f3-4dc8-4bed-86d1-af27279e5e6d None admin] RBAC: Adding query filter params (group.id=a62b4bf100a4409b823a863854490ce7, role.id=d167c236cee64ede8d62bce075f83fff, scope.project.id=0a64615765ab4a01a30fc0023bb081c4) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:05.655295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb5a3f-3566-4c91-9ef0-32d7779790ef None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:05.655398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae947f3-4dc8-4bed-86d1-af27279e5e6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:05.657451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae947f3-4dc8-4bed-86d1-af27279e5e6d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:05.670293 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 224/444] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 3 20:33:05 2022] GET /identity/v3/role_assignments?group.id=a62b4bf100a4409b823a863854490ce7&scope.project.id=0a64615765ab4a01a30fc0023bb081c4&role.id=d167c236cee64ede8d62bce075f83fff => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:05.720991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:05.779138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb5a3f-3566-4c91-9ef0-32d7779790ef None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:05.819640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc1024-c065-4c1e-b2ee-bf58bc1b106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:05.820342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc1024-c065-4c1e-b2ee-bf58bc1b106c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:05.820859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc1024-c065-4c1e-b2ee-bf58bc1b106c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:05.820993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc1024-c065-4c1e-b2ee-bf58bc1b106c None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:05.821628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc1024-c065-4c1e-b2ee-bf58bc1b106c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:05.821936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc1024-c065-4c1e-b2ee-bf58bc1b106c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:05.822631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae947f3-4dc8-4bed-86d1-af27279e5e6d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:05.823346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc1024-c065-4c1e-b2ee-bf58bc1b106c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:05.835016 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 221/445] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:05.840876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc1024-c065-4c1e-b2ee-bf58bc1b106c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:05.949203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae947f3-4dc8-4bed-86d1-af27279e5e6d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:05.958964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc1024-c065-4c1e-b2ee-bf58bc1b106c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:05.974959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:05.997934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28aa51-8016-4cf3-80af-9d4c91421e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:05.998804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28aa51-8016-4cf3-80af-9d4c91421e91 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:05.999094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28aa51-8016-4cf3-80af-9d4c91421e91 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:05.999376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28aa51-8016-4cf3-80af-9d4c91421e91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:05.999921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68854d54-46d6-4054-9b88-fea1d5b135ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:06.000497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68854d54-46d6-4054-9b88-fea1d5b135ac None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:06.000783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68854d54-46d6-4054-9b88-fea1d5b135ac None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:06.001214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68854d54-46d6-4054-9b88-fea1d5b135ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:06.014430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68854d54-46d6-4054-9b88-fea1d5b135ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-68854d54-46d6-4054-9b88-fea1d5b135ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:06.020926 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:06.021817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:06.022593 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:06.022593 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:06.022593 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:06.022593 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:06.022593 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:06.022593 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:06.022593 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:06.022593 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:06.022593 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:06.022593 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:06.022593 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:06.022593 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:06.032800 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 225/446] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:06.033709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68854d54-46d6-4054-9b88-fea1d5b135ac None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:06.073402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28aa51-8016-4cf3-80af-9d4c91421e91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:06.075652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28aa51-8016-4cf3-80af-9d4c91421e91 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:06.078906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:06.149025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68854d54-46d6-4054-9b88-fea1d5b135ac None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:06.180647 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 222/447] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 20:33:06.181568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a0714-e514-45d5-8c88-413260f6f769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:06.182261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a0714-e514-45d5-8c88-413260f6f769 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:06.182609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a0714-e514-45d5-8c88-413260f6f769 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:06.182947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a0714-e514-45d5-8c88-413260f6f769 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:06.183671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a0714-e514-45d5-8c88-413260f6f769 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:06.184039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a0714-e514-45d5-8c88-413260f6f769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:06.185523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a0714-e514-45d5-8c88-413260f6f769 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:06.186330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28aa51-8016-4cf3-80af-9d4c91421e91 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:06.193752 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 226/448] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:06.199475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a0714-e514-45d5-8c88-413260f6f769 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:06.202380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:06.286768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28aa51-8016-4cf3-80af-9d4c91421e91 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:06.292395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:06.298500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a0714-e514-45d5-8c88-413260f6f769 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:06.320520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c48068-1b31-4554-b682-57c258e67e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:06.321187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c48068-1b31-4554-b682-57c258e67e11 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:06.321472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c48068-1b31-4554-b682-57c258e67e11 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:06.321752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c48068-1b31-4554-b682-57c258e67e11 None admin] PATH_INFO: `/v3/projects/0a64615765ab4a01a30fc0023bb081c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:06.322872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c48068-1b31-4554-b682-57c258e67e11 None admin] RBAC: Authorizing `identity:get_project(project_id=0a64615765ab4a01a30fc0023bb081c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:06.324334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c48068-1b31-4554-b682-57c258e67e11 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:06.325980 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 223/449] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:33:06 2022] GET /identity/v3/projects/0a64615765ab4a01a30fc0023bb081c4 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:06.332312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c48068-1b31-4554-b682-57c258e67e11 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:06.333750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952e2f3-7258-4f8c-adc6-2572a3261206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:06.334349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952e2f3-7258-4f8c-adc6-2572a3261206 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:06.334644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952e2f3-7258-4f8c-adc6-2572a3261206 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:06.334906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952e2f3-7258-4f8c-adc6-2572a3261206 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:06.399799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952e2f3-7258-4f8c-adc6-2572a3261206 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:06.401192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952e2f3-7258-4f8c-adc6-2572a3261206 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:06.410612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:06.429166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c48068-1b31-4554-b682-57c258e67e11 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:06.462345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69767458-bd1a-413e-907e-823f666d96e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:06.462973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69767458-bd1a-413e-907e-823f666d96e5 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:06.463259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69767458-bd1a-413e-907e-823f666d96e5 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:06.463538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69767458-bd1a-413e-907e-823f666d96e5 None admin] PATH_INFO: `/v3/groups/a62b4bf100a4409b823a863854490ce7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:06.464659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69767458-bd1a-413e-907e-823f666d96e5 None admin] RBAC: Authorizing `identity:get_group(group_id=a62b4bf100a4409b823a863854490ce7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:06.466137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69767458-bd1a-413e-907e-823f666d96e5 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:06.467701 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 224/450] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:33:06 2022] GET /identity/v3/groups/a62b4bf100a4409b823a863854490ce7 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:06.473907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69767458-bd1a-413e-907e-823f666d96e5 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:06.491148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:06.498065 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 227/451] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 20:33:06.503694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952e2f3-7258-4f8c-adc6-2572a3261206 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:06.569866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69767458-bd1a-413e-907e-823f666d96e5 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:06.600801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952e2f3-7258-4f8c-adc6-2572a3261206 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:06.604080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e1e00-8dd7-415f-b0e4-a1243ed60354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:06.604674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e1e00-8dd7-415f-b0e4-a1243ed60354 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:06.604993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e1e00-8dd7-415f-b0e4-a1243ed60354 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:06.605272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e1e00-8dd7-415f-b0e4-a1243ed60354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:06.605988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e1e00-8dd7-415f-b0e4-a1243ed60354 None admin] RBAC: Adding query filter params (group.id=a62b4bf100a4409b823a863854490ce7, role.id=542f58433b2747d2abd5d405ebabdea1, scope.project.id=0a64615765ab4a01a30fc0023bb081c4) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:06.606296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e1e00-8dd7-415f-b0e4-a1243ed60354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:06.607733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e1e00-8dd7-415f-b0e4-a1243ed60354 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:06.613886 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 225/452] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 3 20:33:06 2022] GET /identity/v3/role_assignments?group.id=a62b4bf100a4409b823a863854490ce7&scope.project.id=0a64615765ab4a01a30fc0023bb081c4&role.id=542f58433b2747d2abd5d405ebabdea1 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:06.635498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c6119-ccf6-4cc7-8411-6bea9b37717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:06.636151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c6119-ccf6-4cc7-8411-6bea9b37717b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:06.636431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c6119-ccf6-4cc7-8411-6bea9b37717b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:06.636704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c6119-ccf6-4cc7-8411-6bea9b37717b None admin] PATH_INFO: `/v3/projects/53641d46504c444d9350b7e021f6a4a4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:06.637917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c6119-ccf6-4cc7-8411-6bea9b37717b None admin] RBAC: Authorizing `identity:get_project(project_id=53641d46504c444d9350b7e021f6a4a4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:06.639405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c6119-ccf6-4cc7-8411-6bea9b37717b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:06.641114 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 228/453] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:33:06 2022] GET /identity/v3/projects/53641d46504c444d9350b7e021f6a4a4 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:06.650132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e1e00-8dd7-415f-b0e4-a1243ed60354 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:06.667482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:06.759212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e1e00-8dd7-415f-b0e4-a1243ed60354 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:06.802924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ce19b-89b7-43a8-b58c-c629056852cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:06.803574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ce19b-89b7-43a8-b58c-c629056852cc None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:06.803863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ce19b-89b7-43a8-b58c-c629056852cc None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:06.804144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ce19b-89b7-43a8-b58c-c629056852cc None admin] PATH_INFO: `/v3/groups/836551814bbd43b994dffb2b4449a309` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:06.805667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ce19b-89b7-43a8-b58c-c629056852cc None admin] RBAC: Authorizing `identity:get_group(group_id=836551814bbd43b994dffb2b4449a309)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:06.808169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ce19b-89b7-43a8-b58c-c629056852cc None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:06.808845 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 226/454] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 3 20:33:06 2022] GET /identity/v3/groups/836551814bbd43b994dffb2b4449a309 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:06.814547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c6119-ccf6-4cc7-8411-6bea9b37717b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:06.896652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:06.939587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c6119-ccf6-4cc7-8411-6bea9b37717b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:06.976819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589bf97-2e3a-4d0d-be65-49012c453e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:06.977198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589bf97-2e3a-4d0d-be65-49012c453e99 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:06.977478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589bf97-2e3a-4d0d-be65-49012c453e99 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:06.977756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589bf97-2e3a-4d0d-be65-49012c453e99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:06.978489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589bf97-2e3a-4d0d-be65-49012c453e99 None admin] RBAC: Adding query filter params (group.id=836551814bbd43b994dffb2b4449a309, role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=53641d46504c444d9350b7e021f6a4a4) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:06.978860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589bf97-2e3a-4d0d-be65-49012c453e99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:06.980310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589bf97-2e3a-4d0d-be65-49012c453e99 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:06.987655 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 229/455] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 3 20:33:06 2022] GET /identity/v3/role_assignments?group.id=836551814bbd43b994dffb2b4449a309&scope.project.id=53641d46504c444d9350b7e021f6a4a4&role.id=bc768522dc984239be4a1b39a45ab437 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:09.643579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f31a5-c715-4190-80fe-2486ea0f5517 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:09.644551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f31a5-c715-4190-80fe-2486ea0f5517 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:09.645074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f31a5-c715-4190-80fe-2486ea0f5517 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:09.647106 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 227/456] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:09.652041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6421a9a-b663-4752-8ff2-e12898ac4765 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:09.652852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6421a9a-b663-4752-8ff2-e12898ac4765 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:09.653362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6421a9a-b663-4752-8ff2-e12898ac4765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:09.679426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a6421a9a-b663-4752-8ff2-e12898ac4765 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:09.681064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a6421a9a-b663-4752-8ff2-e12898ac4765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:09.770193 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 230/457] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:09.973604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c021e-c1e4-40bd-b173-2bc7f02f8483 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:09.974296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c021e-c1e4-40bd-b173-2bc7f02f8483 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:09.974792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c021e-c1e4-40bd-b173-2bc7f02f8483 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:09.976594 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 228/458] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:09.981649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e1dc2-ce8b-4550-a85b-180832b9f000 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:09.982193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e1dc2-ce8b-4550-a85b-180832b9f000 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:09.982676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e1dc2-ce8b-4550-a85b-180832b9f000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:10.005005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-801e1dc2-ce8b-4550-a85b-180832b9f000 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:10.006539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-801e1dc2-ce8b-4550-a85b-180832b9f000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:10.081995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:10.105962 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 231/459] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:10.241193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b25a4c-3a23-46b0-82d0-ab45a1a4a68c None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:10.241806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b25a4c-3a23-46b0-82d0-ab45a1a4a68c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:10.242407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b25a4c-3a23-46b0-82d0-ab45a1a4a68c None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:10.244237 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 229/460] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:10.249879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af9d81-bdc7-4f09-92ea-0f2418bcf87f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:10.250181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af9d81-bdc7-4f09-92ea-0f2418bcf87f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:10.250464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af9d81-bdc7-4f09-92ea-0f2418bcf87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:10.268834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-20af9d81-bdc7-4f09-92ea-0f2418bcf87f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:10.270063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-20af9d81-bdc7-4f09-92ea-0f2418bcf87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:10.276459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b25a4c-3a23-46b0-82d0-ab45a1a4a68c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:10.354200 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 232/461] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:10.415247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:10.436276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b25a4c-3a23-46b0-82d0-ab45a1a4a68c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:10.472407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d773cdf-db00-49cc-a533-56ce8a06f696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:10.473081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d773cdf-db00-49cc-a533-56ce8a06f696 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:10.473371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d773cdf-db00-49cc-a533-56ce8a06f696 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:10.473651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d773cdf-db00-49cc-a533-56ce8a06f696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:10.484747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d773cdf-db00-49cc-a533-56ce8a06f696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:10.486822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d773cdf-db00-49cc-a533-56ce8a06f696 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-5d773cdf-db00-49cc-a533-56ce8a06f696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:10.502701 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:10.503611 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:10.504463 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:10.504463 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:10.504463 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:10.504463 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 230/462] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:10.510003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af9d81-bdc7-4f09-92ea-0f2418bcf87f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:10.531962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb07d-de45-4444-b503-fd0f5aa042ba None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:10.532249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb07d-de45-4444-b503-fd0f5aa042ba None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:10.532557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb07d-de45-4444-b503-fd0f5aa042ba None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:10.535122 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 231/463] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:33:10.550547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d379f-9111-4bd9-8007-186f084c8e0f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:10.550828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d379f-9111-4bd9-8007-186f084c8e0f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:10.551100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d379f-9111-4bd9-8007-186f084c8e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:10.593929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-646d379f-9111-4bd9-8007-186f084c8e0f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:10.595693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-646d379f-9111-4bd9-8007-186f084c8e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:10.622961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af9d81-bdc7-4f09-92ea-0f2418bcf87f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:10.645740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:10.665168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933afa8c-e871-42fc-a984-76d084471e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:10.666944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933afa8c-e871-42fc-a984-76d084471e97 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:10.667405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933afa8c-e871-42fc-a984-76d084471e97 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:10.668137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933afa8c-e871-42fc-a984-76d084471e97 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:10.668996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933afa8c-e871-42fc-a984-76d084471e97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:10.669569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933afa8c-e871-42fc-a984-76d084471e97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:10.671744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933afa8c-e871-42fc-a984-76d084471e97 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:10.689390 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 232/464] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:33:10.689390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d379f-9111-4bd9-8007-186f084c8e0f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:10.694067 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 233/465] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:10.700665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933afa8c-e871-42fc-a984-76d084471e97 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:10.808392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:10.849583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d379f-9111-4bd9-8007-186f084c8e0f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:10.864870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933afa8c-e871-42fc-a984-76d084471e97 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:10.894738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127168c2-2f08-47cd-a8b0-64fa5a44927b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:10.895654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127168c2-2f08-47cd-a8b0-64fa5a44927b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:10.896111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127168c2-2f08-47cd-a8b0-64fa5a44927b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:10.896801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127168c2-2f08-47cd-a8b0-64fa5a44927b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:10.902532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5973d74-2750-47bf-915d-73354057bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:10.903370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5973d74-2750-47bf-915d-73354057bbf2 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:10.903804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5973d74-2750-47bf-915d-73354057bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:10.904278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5973d74-2750-47bf-915d-73354057bbf2 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:10.905266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5973d74-2750-47bf-915d-73354057bbf2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:10.906708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5973d74-2750-47bf-915d-73354057bbf2 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:10.912491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127168c2-2f08-47cd-a8b0-64fa5a44927b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:10.914096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127168c2-2f08-47cd-a8b0-64fa5a44927b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-127168c2-2f08-47cd-a8b0-64fa5a44927b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:10.931261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:10.932235 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:10.933134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:10.933134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:10.933134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:10.933134 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 233/466] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:10.934200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127168c2-2f08-47cd-a8b0-64fa5a44927b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:10.947674 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 234/467] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:33:10 2022] POST /identity/v3/users => generated 246 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:33:10.948636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5973d74-2750-47bf-915d-73354057bbf2 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:10.993804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:11.083636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5973d74-2750-47bf-915d-73354057bbf2 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:11.129531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127168c2-2f08-47cd-a8b0-64fa5a44927b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:11.145435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e77f7-20c6-4783-842e-8b9fed6bd4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:11.146068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e77f7-20c6-4783-842e-8b9fed6bd4e1 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:11.146373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e77f7-20c6-4783-842e-8b9fed6bd4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:11.146658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e77f7-20c6-4783-842e-8b9fed6bd4e1 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:11.147463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e77f7-20c6-4783-842e-8b9fed6bd4e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:11.147800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e77f7-20c6-4783-842e-8b9fed6bd4e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:11.149975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e77f7-20c6-4783-842e-8b9fed6bd4e1 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:11.150880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:11.167336 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 235/468] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:11.169492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e77f7-20c6-4783-842e-8b9fed6bd4e1 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:11.178089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a89c8-4095-4d42-a3ef-b49672471565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:11.179094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a89c8-4095-4d42-a3ef-b49672471565 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:11.179623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a89c8-4095-4d42-a3ef-b49672471565 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:11.180120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a89c8-4095-4d42-a3ef-b49672471565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:11.192588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a89c8-4095-4d42-a3ef-b49672471565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:11.194535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a89c8-4095-4d42-a3ef-b49672471565 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-e27a89c8-4095-4d42-a3ef-b49672471565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:11.212075 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:11.213050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:11.213875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:11.213875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:11.213875 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:11.214502 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 234/469] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:11.215105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a89c8-4095-4d42-a3ef-b49672471565 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:11.291625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:11.300288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e77f7-20c6-4783-842e-8b9fed6bd4e1 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:11.336255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d0891-665c-4027-9f04-26e52d6f1957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:11.336814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d0891-665c-4027-9f04-26e52d6f1957 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:11.337173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d0891-665c-4027-9f04-26e52d6f1957 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:11.337460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d0891-665c-4027-9f04-26e52d6f1957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:11.341008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a89c8-4095-4d42-a3ef-b49672471565 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:11.353702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d0891-665c-4027-9f04-26e52d6f1957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:11.355457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d0891-665c-4027-9f04-26e52d6f1957 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-070d0891-665c-4027-9f04-26e52d6f1957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:11.373456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:11.374636 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:11.375469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:11.375469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:11.375469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:11.375469 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 236/470] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:11.375691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d0891-665c-4027-9f04-26e52d6f1957 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:11.379810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b742efd-7fae-429b-a1a9-76fc91eebc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:11.380702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b742efd-7fae-429b-a1a9-76fc91eebc2b None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:11.381022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b742efd-7fae-429b-a1a9-76fc91eebc2b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:11.381300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b742efd-7fae-429b-a1a9-76fc91eebc2b None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:11.381959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b742efd-7fae-429b-a1a9-76fc91eebc2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:11.383411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b742efd-7fae-429b-a1a9-76fc91eebc2b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:11.406583 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 235/471] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:33:11 2022] POST /identity/v3/users => generated 248 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:33:11.407891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55906ed-6c1a-4e56-ae54-127fa4182897 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:11.408185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55906ed-6c1a-4e56-ae54-127fa4182897 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:11.408512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55906ed-6c1a-4e56-ae54-127fa4182897 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:11.413069 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 236/472] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:11.413069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55906ed-6c1a-4e56-ae54-127fa4182897 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:11.457513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:11.506872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d0891-665c-4027-9f04-26e52d6f1957 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:11.560161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8b261-8880-491c-89bc-4aa66c1273cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:11.561887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8b261-8880-491c-89bc-4aa66c1273cc None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:11.562708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8b261-8880-491c-89bc-4aa66c1273cc None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:11.563235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8b261-8880-491c-89bc-4aa66c1273cc None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:11.564936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8b261-8880-491c-89bc-4aa66c1273cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:11.565459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8b261-8880-491c-89bc-4aa66c1273cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:11.565907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55906ed-6c1a-4e56-ae54-127fa4182897 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:11.567392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8b261-8880-491c-89bc-4aa66c1273cc None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:11.587660 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 237/473] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:11.589444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078e146-4486-4b72-ad55-d905e2f45240 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:11.589980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078e146-4486-4b72-ad55-d905e2f45240 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:11.591098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078e146-4486-4b72-ad55-d905e2f45240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:11.608694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:11.609964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f173d-ca8b-41aa-a202-d466ad73b013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:11.611548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f173d-ca8b-41aa-a202-d466ad73b013 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:11.612087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f173d-ca8b-41aa-a202-d466ad73b013 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:11.612440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f173d-ca8b-41aa-a202-d466ad73b013 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:11.613127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f173d-ca8b-41aa-a202-d466ad73b013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:11.613433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f173d-ca8b-41aa-a202-d466ad73b013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:11.614811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f173d-ca8b-41aa-a202-d466ad73b013 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:11.616139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b078e146-4486-4b72-ad55-d905e2f45240 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:11.617809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b078e146-4486-4b72-ad55-d905e2f45240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:11.631093 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 237/474] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:11.632287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f173d-ca8b-41aa-a202-d466ad73b013 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:11.702530 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 238/475] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:11.704070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078e146-4486-4b72-ad55-d905e2f45240 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:11.754674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:11.773832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f173d-ca8b-41aa-a202-d466ad73b013 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:11.813650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad787da-0964-4e2b-aee2-2edd4a496af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:11.814819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad787da-0964-4e2b-aee2-2edd4a496af5 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:11.815103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad787da-0964-4e2b-aee2-2edd4a496af5 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:11.815763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad787da-0964-4e2b-aee2-2edd4a496af5 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:11.816767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad787da-0964-4e2b-aee2-2edd4a496af5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:11.818493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad787da-0964-4e2b-aee2-2edd4a496af5 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:11.838703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078e146-4486-4b72-ad55-d905e2f45240 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:11.867787 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 238/476] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:33:11 2022] POST /identity/v3/users => generated 248 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:33:11.884018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f879d4-dc9b-4cd3-9b9d-3178859847b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:11.884654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f879d4-dc9b-4cd3-9b9d-3178859847b0 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:11.884970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f879d4-dc9b-4cd3-9b9d-3178859847b0 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:11.885255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f879d4-dc9b-4cd3-9b9d-3178859847b0 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:11.885917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f879d4-dc9b-4cd3-9b9d-3178859847b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:11.887613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f879d4-dc9b-4cd3-9b9d-3178859847b0 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:11.898900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:11.927469 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 239/477] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:33:11 2022] POST /identity/v3/users => generated 249 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:33:12.175991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad787da-0964-4e2b-aee2-2edd4a496af5 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:12.297403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad787da-0964-4e2b-aee2-2edd4a496af5 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:12.336353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a666f51-a625-4688-8e2e-18d210bca7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:12.336995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a666f51-a625-4688-8e2e-18d210bca7d3 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:12.337280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a666f51-a625-4688-8e2e-18d210bca7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:12.337554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a666f51-a625-4688-8e2e-18d210bca7d3 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:12.338238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a666f51-a625-4688-8e2e-18d210bca7d3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:12.339744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a666f51-a625-4688-8e2e-18d210bca7d3 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:12.349752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.sql.core [None req-6a666f51-a625-4688-8e2e-18d210bca7d3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 03 20:33:12.349752 ncdv-master-1 devstack@keystone.service[99210]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 03 20:33:12.349752 ncdv-master-1 devstack@keystone.service[99210]: [parameters: {'id': 'cc912c222d404bc4ac6a76676dba60f9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 03 20:33:12.349752 ncdv-master-1 devstack@keystone.service[99210]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99210) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-6a666f51-a625-4688-8e2e-18d210bca7d3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application conn.query(q) Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application result.read() Sep 03 20:33:12.366620 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return ref.to_dict() Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 20:33:12.367539 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application next(self.gen) Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application yield resource Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application next(self.gen) Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.session.rollback() Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.force_reraise() Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise self.value Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application session.commit() Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 03 20:33:12.368424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._prepare_impl() Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.session.flush() Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._flush(objects) Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application compat.raise_( Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application flush_context.execute() Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rec.execute(self) Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 03 20:33:12.369308 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 20:33:12.370156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application conn.query(q) Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application result.read() Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 20:33:12.370969 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [parameters: {'id': 'cc912c222d404bc4ac6a76676dba60f9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:12.371756 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 03 20:33:12.372627 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:12.378257 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 239/478] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Sep 3 20:33:12 2022] POST /identity/v3/roles => generated 144 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:12.378397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f879d4-dc9b-4cd3-9b9d-3178859847b0 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:12.488681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f879d4-dc9b-4cd3-9b9d-3178859847b0 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:12.527564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1cf4f-b677-4586-801d-94b9226dd17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:12.528202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1cf4f-b677-4586-801d-94b9226dd17c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:12.528484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1cf4f-b677-4586-801d-94b9226dd17c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:12.528761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1cf4f-b677-4586-801d-94b9226dd17c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:12.539432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1cf4f-b677-4586-801d-94b9226dd17c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-abc1cf4f-b677-4586-801d-94b9226dd17c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:12.544432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:12.546555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:12.548567 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:12.548567 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:12.548567 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:12.548567 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:12.548567 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:12.548567 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:12.548567 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:12.548567 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:12.548567 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:12.548567 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:12.548567 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 20:33:12.548567 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:12.553689 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 240/479] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:33:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:12.559650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a666f51-a625-4688-8e2e-18d210bca7d3 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:12.585316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:12.670900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a666f51-a625-4688-8e2e-18d210bca7d3 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:12.707290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0f2f4-9023-4860-a859-4749e56c0296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:12.707903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0f2f4-9023-4860-a859-4749e56c0296 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:12.708184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0f2f4-9023-4860-a859-4749e56c0296 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:12.708457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0f2f4-9023-4860-a859-4749e56c0296 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:12.709149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0f2f4-9023-4860-a859-4749e56c0296 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:12.709509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0f2f4-9023-4860-a859-4749e56c0296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:12.710896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0f2f4-9023-4860-a859-4749e56c0296 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:12.725158 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 240/480] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:13.536572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8387eb-1857-4b81-944d-469ddadc83e1 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:13.537295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8387eb-1857-4b81-944d-469ddadc83e1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:13.537295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8387eb-1857-4b81-944d-469ddadc83e1 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:13.538705 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 241/481] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:13.547500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83788e-004d-45f9-8bff-2e2a12edf6ff None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:13.548540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83788e-004d-45f9-8bff-2e2a12edf6ff None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:13.548703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83788e-004d-45f9-8bff-2e2a12edf6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:13.570290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5a83788e-004d-45f9-8bff-2e2a12edf6ff None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:13.571493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5a83788e-004d-45f9-8bff-2e2a12edf6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:13.655381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:13.666746 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 241/482] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:14.119271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8387eb-1857-4b81-944d-469ddadc83e1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:14.194753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:14.229286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3e862-6dee-46e5-b3b4-cbcfbcd56612 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:14.229606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3e862-6dee-46e5-b3b4-cbcfbcd56612 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:14.229915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3e862-6dee-46e5-b3b4-cbcfbcd56612 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:14.231379 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 242/483] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:14.238258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ba92f-950b-469f-9640-8b4231829e21 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:14.238549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ba92f-950b-469f-9640-8b4231829e21 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:14.238834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ba92f-950b-469f-9640-8b4231829e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:14.259075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-963ba92f-950b-469f-9640-8b4231829e21 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:14.260593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-963ba92f-950b-469f-9640-8b4231829e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:14.263946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8387eb-1857-4b81-944d-469ddadc83e1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:14.330938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470dbce-b910-450e-817b-dc8d46e7bc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:14.332122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470dbce-b910-450e-817b-dc8d46e7bc3f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:14.332584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470dbce-b910-450e-817b-dc8d46e7bc3f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:14.333066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470dbce-b910-450e-817b-dc8d46e7bc3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:14.353974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470dbce-b910-450e-817b-dc8d46e7bc3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-9470dbce-b910-450e-817b-dc8d46e7bc3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:14.359268 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:14.360201 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:14.361110 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:14.361110 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:14.361110 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:14.361110 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:14.361110 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:14.361110 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:14.361110 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:14.361110 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:14.361110 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:14.361110 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:14.361110 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:14.361110 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:14.366826 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 242/484] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:14.375020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470dbce-b910-450e-817b-dc8d46e7bc3f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:14.390754 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 243/485] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:33:14.522895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470dbce-b910-450e-817b-dc8d46e7bc3f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:14.541953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:14.566454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d60b9-f14a-4972-800a-4e62e071ea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:14.567110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d60b9-f14a-4972-800a-4e62e071ea4f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:14.567397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d60b9-f14a-4972-800a-4e62e071ea4f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:14.567672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d60b9-f14a-4972-800a-4e62e071ea4f None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:14.568377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d60b9-f14a-4972-800a-4e62e071ea4f None admin] RBAC: Adding query filter params (name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:14.573400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d60b9-f14a-4972-800a-4e62e071ea4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:14.574840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d60b9-f14a-4972-800a-4e62e071ea4f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:14.584376 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 243/486] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:14.589687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ba92f-950b-469f-9640-8b4231829e21 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:14.741774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ba92f-950b-469f-9640-8b4231829e21 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:14.774224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:14.783664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100709f-ccc8-4c69-a457-cb2e473a8b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:14.784314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100709f-ccc8-4c69-a457-cb2e473a8b56 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:14.784603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100709f-ccc8-4c69-a457-cb2e473a8b56 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:14.784881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100709f-ccc8-4c69-a457-cb2e473a8b56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:14.795748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100709f-ccc8-4c69-a457-cb2e473a8b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:14.797257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100709f-ccc8-4c69-a457-cb2e473a8b56 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:14.810823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e42d0-13d7-4645-b547-5eb91f6832cf None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:14.811115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e42d0-13d7-4645-b547-5eb91f6832cf None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:14.811395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e42d0-13d7-4645-b547-5eb91f6832cf None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a100709f-ccc8-4c69-a457-cb2e473a8b56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:14.815725 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:14.816582 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:14.823377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:14.823377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:14.823377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:14.823377 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 244/488] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:14.823605 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 244/488] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:14.823676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdf402-4d53-4e95-907f-b5724b669121 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:14.823676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdf402-4d53-4e95-907f-b5724b669121 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:14.823676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdf402-4d53-4e95-907f-b5724b669121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:14.823841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e42d0-13d7-4645-b547-5eb91f6832cf None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:14.842806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-75cdf402-4d53-4e95-907f-b5724b669121 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:14.844142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-75cdf402-4d53-4e95-907f-b5724b669121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:14.943771 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 245/489] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:14.944969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdf402-4d53-4e95-907f-b5724b669121 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:14.975481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e42d0-13d7-4645-b547-5eb91f6832cf None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:15.014200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169b3ee-b634-4c03-835b-4a237f5d407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:15.014732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169b3ee-b634-4c03-835b-4a237f5d407f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:15.015085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169b3ee-b634-4c03-835b-4a237f5d407f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:15.015460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169b3ee-b634-4c03-835b-4a237f5d407f None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:15.016164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169b3ee-b634-4c03-835b-4a237f5d407f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:15.017269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169b3ee-b634-4c03-835b-4a237f5d407f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:15.018572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169b3ee-b634-4c03-835b-4a237f5d407f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:15.031894 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 245/490] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:15.037450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169b3ee-b634-4c03-835b-4a237f5d407f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:15.058426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdf402-4d53-4e95-907f-b5724b669121 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:15.069537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:15.079972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:15.097703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95185-7fd7-43c2-8b84-2f15d183a113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:15.098348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95185-7fd7-43c2-8b84-2f15d183a113 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:15.098639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95185-7fd7-43c2-8b84-2f15d183a113 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:15.098921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95185-7fd7-43c2-8b84-2f15d183a113 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:15.109458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b95185-7fd7-43c2-8b84-2f15d183a113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a6b95185-7fd7-43c2-8b84-2f15d183a113 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:15.114451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:15.115353 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:15.116145 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 246/491] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:15.123730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95185-7fd7-43c2-8b84-2f15d183a113 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:15.216746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169b3ee-b634-4c03-835b-4a237f5d407f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:15.257055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5920d39-8a8c-4d0d-b6f7-6d7935d9214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:15.257869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5920d39-8a8c-4d0d-b6f7-6d7935d9214d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:15.258152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5920d39-8a8c-4d0d-b6f7-6d7935d9214d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:15.258423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5920d39-8a8c-4d0d-b6f7-6d7935d9214d None admin] PATH_INFO: `/v3/users/nova` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:15.266367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5920d39-8a8c-4d0d-b6f7-6d7935d9214d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:15.268699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5920d39-8a8c-4d0d-b6f7-6d7935d9214d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:15.275157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95185-7fd7-43c2-8b84-2f15d183a113 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-d5920d39-8a8c-4d0d-b6f7-6d7935d9214d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:15.286141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:15.287078 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:15.287988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:15.287988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:15.287988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:15.287988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:15.287988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:15.287988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:15.287988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:15.287988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:15.287988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:33:15.287988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:15.287988 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 246/492] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 3 20:33:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:15.289072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c21271-f24e-44f8-9779-0b39ddfcf1f8 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:15.293127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c21271-f24e-44f8-9779-0b39ddfcf1f8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:15.294179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c21271-f24e-44f8-9779-0b39ddfcf1f8 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:15.295628 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 247/493] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:15.297796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c21271-f24e-44f8-9779-0b39ddfcf1f8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:15.327257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b06030-9dd4-400e-95c0-f730437c616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:15.328253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b06030-9dd4-400e-95c0-f730437c616c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:15.328754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b06030-9dd4-400e-95c0-f730437c616c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:15.329352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b06030-9dd4-400e-95c0-f730437c616c None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:15.330704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b06030-9dd4-400e-95c0-f730437c616c None admin] RBAC: Adding query filter params (name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:15.331227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b06030-9dd4-400e-95c0-f730437c616c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:15.333136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b06030-9dd4-400e-95c0-f730437c616c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:15.342115 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 247/494] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:15.343866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90321b85-b050-4a15-9624-b0846a04456b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:15.344369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90321b85-b050-4a15-9624-b0846a04456b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:15.344934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90321b85-b050-4a15-9624-b0846a04456b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:15.363222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-90321b85-b050-4a15-9624-b0846a04456b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:15.364431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-90321b85-b050-4a15-9624-b0846a04456b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:15.390243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:15.406749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:15.415188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c21271-f24e-44f8-9779-0b39ddfcf1f8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:15.437295 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 248/495] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:15.438015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90321b85-b050-4a15-9624-b0846a04456b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:15.465399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8cde2-fcb6-4d54-a28e-d5e66dc2259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:15.466796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8cde2-fcb6-4d54-a28e-d5e66dc2259e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:15.467532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8cde2-fcb6-4d54-a28e-d5e66dc2259e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:15.468079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8cde2-fcb6-4d54-a28e-d5e66dc2259e None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:15.469612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8cde2-fcb6-4d54-a28e-d5e66dc2259e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:15.470113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8cde2-fcb6-4d54-a28e-d5e66dc2259e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:15.472324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8cde2-fcb6-4d54-a28e-d5e66dc2259e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:15.521711 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 248/496] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 3 20:33:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:15.526863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8cde2-fcb6-4d54-a28e-d5e66dc2259e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:15.560385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90321b85-b050-4a15-9624-b0846a04456b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:15.598133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc475b9-6532-4ce1-a751-684764b2c1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:15.598931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc475b9-6532-4ce1-a751-684764b2c1de None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:15.599340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc475b9-6532-4ce1-a751-684764b2c1de None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:15.599701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc475b9-6532-4ce1-a751-684764b2c1de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:15.610354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc475b9-6532-4ce1-a751-684764b2c1de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:15.612586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc475b9-6532-4ce1-a751-684764b2c1de None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-abc475b9-6532-4ce1-a751-684764b2c1de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:15.628354 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:15.629519 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:15.630547 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:15.630547 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:15.630547 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:15.632198 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 249/497] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:15.633799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc475b9-6532-4ce1-a751-684764b2c1de None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:15.653782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:15.658252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8cde2-fcb6-4d54-a28e-d5e66dc2259e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:15.693232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca4e4a-336d-470c-89ed-dc164267d67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:15.693865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca4e4a-336d-470c-89ed-dc164267d67f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:15.694147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca4e4a-336d-470c-89ed-dc164267d67f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:15.694420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca4e4a-336d-470c-89ed-dc164267d67f None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:15.705713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca4e4a-336d-470c-89ed-dc164267d67f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-67ca4e4a-336d-470c-89ed-dc164267d67f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:15.711220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:15.712164 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:15.712981 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 249/498] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:15.713628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca4e4a-336d-470c-89ed-dc164267d67f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:15.734912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:15.750937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc475b9-6532-4ce1-a751-684764b2c1de None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:15.786660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbd534-01a5-457f-9a39-d4e9cc91520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:15.787198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbd534-01a5-457f-9a39-d4e9cc91520e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:15.787490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbd534-01a5-457f-9a39-d4e9cc91520e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:15.787771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbd534-01a5-457f-9a39-d4e9cc91520e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:15.820669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca4e4a-336d-470c-89ed-dc164267d67f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:15.855579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495125ec-c8e3-4e99-8739-47caaa6c0b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:15.856101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbd534-01a5-457f-9a39-d4e9cc91520e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:15.856207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495125ec-c8e3-4e99-8739-47caaa6c0b40 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:15.856437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495125ec-c8e3-4e99-8739-47caaa6c0b40 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:15.856707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495125ec-c8e3-4e99-8739-47caaa6c0b40 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:15.857394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495125ec-c8e3-4e99-8739-47caaa6c0b40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:15.857708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495125ec-c8e3-4e99-8739-47caaa6c0b40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:15.858592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbd534-01a5-457f-9a39-d4e9cc91520e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:15.859097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495125ec-c8e3-4e99-8739-47caaa6c0b40 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:15.870646 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 250/499] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:15.871755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251f13c-1d1e-4871-bf57-4de1139b317e None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:15.872043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251f13c-1d1e-4871-bf57-4de1139b317e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:15.872295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:15.872401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251f13c-1d1e-4871-bf57-4de1139b317e None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:15.873590 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 251/500] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:15.874387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251f13c-1d1e-4871-bf57-4de1139b317e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:15.947987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:15.966079 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 250/501] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:33:15.966921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbd534-01a5-457f-9a39-d4e9cc91520e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:15.985025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251f13c-1d1e-4871-bf57-4de1139b317e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:16.019078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235d8f2-6227-4088-a551-029d689c7733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:16.019690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235d8f2-6227-4088-a551-029d689c7733 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:16.019977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235d8f2-6227-4088-a551-029d689c7733 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:16.020254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235d8f2-6227-4088-a551-029d689c7733 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:16.020963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235d8f2-6227-4088-a551-029d689c7733 None admin] RBAC: Adding query filter params (name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:16.021387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235d8f2-6227-4088-a551-029d689c7733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:16.022756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235d8f2-6227-4088-a551-029d689c7733 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:16.030653 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 252/502] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:16.031579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235d8f2-6227-4088-a551-029d689c7733 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:16.064617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbd534-01a5-457f-9a39-d4e9cc91520e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:16.077566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:16.099732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfd781-bee5-4c8d-a72f-d45b94c5b976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:16.100369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfd781-bee5-4c8d-a72f-d45b94c5b976 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:16.100657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfd781-bee5-4c8d-a72f-d45b94c5b976 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:16.100965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfd781-bee5-4c8d-a72f-d45b94c5b976 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:16.111264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfd781-bee5-4c8d-a72f-d45b94c5b976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-5ddfd781-bee5-4c8d-a72f-d45b94c5b976 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:16.116118 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:16.117187 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:16.118153 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:16.118153 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:16.118153 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:16.118153 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:16.118153 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:16.118153 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:16.118153 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:16.118153 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:16.118153 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:16.118153 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:16.118153 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:16.118153 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:16.122115 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 251/503] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:16.123253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc476238-6e09-44e4-be91-53d68fc619fb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:16.123544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc476238-6e09-44e4-be91-53d68fc619fb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:16.123827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc476238-6e09-44e4-be91-53d68fc619fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:16.137492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235d8f2-6227-4088-a551-029d689c7733 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:16.144803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fc476238-6e09-44e4-be91-53d68fc619fb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:16.146043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fc476238-6e09-44e4-be91-53d68fc619fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:16.171795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4e3d5-9395-433f-8a1e-3b5ea9b0705f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:16.172423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4e3d5-9395-433f-8a1e-3b5ea9b0705f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:16.172711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4e3d5-9395-433f-8a1e-3b5ea9b0705f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:16.173017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4e3d5-9395-433f-8a1e-3b5ea9b0705f None admin] PATH_INFO: `/v3/users/glance` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:16.175537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:16.182335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4e3d5-9395-433f-8a1e-3b5ea9b0705f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:16.183809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4e3d5-9395-433f-8a1e-3b5ea9b0705f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-75b4e3d5-9395-433f-8a1e-3b5ea9b0705f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:16.194822 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:16.195712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:16.196576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:16.196576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:16.196576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:16.196576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:16.196576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:16.196576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:16.196576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:16.196576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:16.196576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 03 20:33:16.196576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:16.196576 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 253/504] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:16.197150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4e3d5-9395-433f-8a1e-3b5ea9b0705f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:16.216851 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 252/505] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:16.217736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc476238-6e09-44e4-be91-53d68fc619fb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:16.302739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4e3d5-9395-433f-8a1e-3b5ea9b0705f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:16.330928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc476238-6e09-44e4-be91-53d68fc619fb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:16.332845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:16.340269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461dfd8-797d-4f35-836c-c45b842855ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:16.340917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461dfd8-797d-4f35-836c-c45b842855ec None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:16.341240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461dfd8-797d-4f35-836c-c45b842855ec None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:16.341516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461dfd8-797d-4f35-836c-c45b842855ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:16.352458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461dfd8-797d-4f35-836c-c45b842855ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:16.354001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461dfd8-797d-4f35-836c-c45b842855ec None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:16.368413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56973c98-bae7-4db6-a62f-37d38d4ade05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-2461dfd8-797d-4f35-836c-c45b842855ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:16.368570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:16.369432 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:16.370265 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:16.370265 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:16.370265 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:16.370265 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 254/506] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:16.370482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461dfd8-797d-4f35-836c-c45b842855ec None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:16.371808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56973c98-bae7-4db6-a62f-37d38d4ade05 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:16.372100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56973c98-bae7-4db6-a62f-37d38d4ade05 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:16.372408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56973c98-bae7-4db6-a62f-37d38d4ade05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:16.385959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56973c98-bae7-4db6-a62f-37d38d4ade05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:16.387550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56973c98-bae7-4db6-a62f-37d38d4ade05 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-56973c98-bae7-4db6-a62f-37d38d4ade05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:16.402937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:16.403822 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:16.404664 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:16.404664 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:16.404664 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:16.404664 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 253/507] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:16.404870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56973c98-bae7-4db6-a62f-37d38d4ade05 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:16.428698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:16.492324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461dfd8-797d-4f35-836c-c45b842855ec None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:16.514951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56973c98-bae7-4db6-a62f-37d38d4ade05 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:16.529770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe39773-72fb-44b2-98f6-a59f7faa53d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:16.530608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe39773-72fb-44b2-98f6-a59f7faa53d3 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:16.531043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe39773-72fb-44b2-98f6-a59f7faa53d3 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:16.531296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe39773-72fb-44b2-98f6-a59f7faa53d3 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:16.531932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe39773-72fb-44b2-98f6-a59f7faa53d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:16.532238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe39773-72fb-44b2-98f6-a59f7faa53d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:16.534264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe39773-72fb-44b2-98f6-a59f7faa53d3 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:16.542488 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 255/508] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:16.543275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe39773-72fb-44b2-98f6-a59f7faa53d3 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:16.556046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf79f6-3a5f-45f4-8611-b689b4c8898a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:16.556655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf79f6-3a5f-45f4-8611-b689b4c8898a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:16.556979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf79f6-3a5f-45f4-8611-b689b4c8898a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:16.557262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf79f6-3a5f-45f4-8611-b689b4c8898a None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:16.557995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf79f6-3a5f-45f4-8611-b689b4c8898a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:16.558312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf79f6-3a5f-45f4-8611-b689b4c8898a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:16.559781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf79f6-3a5f-45f4-8611-b689b4c8898a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:16.596975 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 254/509] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Sep 3 20:33:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:16.597730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf79f6-3a5f-45f4-8611-b689b4c8898a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:16.611550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:16.649060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe39773-72fb-44b2-98f6-a59f7faa53d3 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:16.658422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:16.684060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acb508-0c6c-43d5-b42c-efee7400d630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:16.684667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acb508-0c6c-43d5-b42c-efee7400d630 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:16.684984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acb508-0c6c-43d5-b42c-efee7400d630 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:16.685266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acb508-0c6c-43d5-b42c-efee7400d630 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:16.685908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acb508-0c6c-43d5-b42c-efee7400d630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:16.686218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acb508-0c6c-43d5-b42c-efee7400d630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:16.687585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acb508-0c6c-43d5-b42c-efee7400d630 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:16.697228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf79f6-3a5f-45f4-8611-b689b4c8898a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:16.699547 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 256/510] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:16.700185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acb508-0c6c-43d5-b42c-efee7400d630 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:16.732072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0c993-9e50-46be-a68f-5735dbd19f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:16.732705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0c993-9e50-46be-a68f-5735dbd19f40 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:16.733018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0c993-9e50-46be-a68f-5735dbd19f40 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:16.733292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0c993-9e50-46be-a68f-5735dbd19f40 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:16.733929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0c993-9e50-46be-a68f-5735dbd19f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:16.734232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0c993-9e50-46be-a68f-5735dbd19f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:16.735649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0c993-9e50-46be-a68f-5735dbd19f40 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:16.746485 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 255/511] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:16.747259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0c993-9e50-46be-a68f-5735dbd19f40 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:16.800561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acb508-0c6c-43d5-b42c-efee7400d630 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:16.818014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:16.834951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5196150-1979-476b-87c8-c50575e08494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:16.835585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5196150-1979-476b-87c8-c50575e08494 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:16.835874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5196150-1979-476b-87c8-c50575e08494 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:16.836153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5196150-1979-476b-87c8-c50575e08494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:16.846069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5196150-1979-476b-87c8-c50575e08494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:16.847553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5196150-1979-476b-87c8-c50575e08494 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:16.848289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0c993-9e50-46be-a68f-5735dbd19f40 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-d5196150-1979-476b-87c8-c50575e08494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:16.861602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:16.863217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:16.865150 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:16.865150 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:16.865150 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:16.865150 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 257/512] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:16.865150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5196150-1979-476b-87c8-c50575e08494 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:16.885567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c538a-31f2-4b74-abfc-8d4529234d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:16.885774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:16.886189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c538a-31f2-4b74-abfc-8d4529234d1b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:16.886482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c538a-31f2-4b74-abfc-8d4529234d1b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:16.886762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c538a-31f2-4b74-abfc-8d4529234d1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:16.897854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c538a-31f2-4b74-abfc-8d4529234d1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:16.899306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c538a-31f2-4b74-abfc-8d4529234d1b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-699c538a-31f2-4b74-abfc-8d4529234d1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:16.913146 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:16.914028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:16.915124 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:16.915124 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:16.915124 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:16.915124 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 256/513] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:16.915124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c538a-31f2-4b74-abfc-8d4529234d1b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:16.967145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5196150-1979-476b-87c8-c50575e08494 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:16.999866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b236d-47c2-4e50-8836-53c4b0b40c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:17.000465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b236d-47c2-4e50-8836-53c4b0b40c22 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:17.000745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b236d-47c2-4e50-8836-53c4b0b40c22 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:17.001056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b236d-47c2-4e50-8836-53c4b0b40c22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:17.012074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c538a-31f2-4b74-abfc-8d4529234d1b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:17.034686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:17.045440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3924d-47b7-4acd-89ec-7efc2eb43736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:17.046056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3924d-47b7-4acd-89ec-7efc2eb43736 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:17.046338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3924d-47b7-4acd-89ec-7efc2eb43736 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:17.046616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3924d-47b7-4acd-89ec-7efc2eb43736 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:17.053053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3924d-47b7-4acd-89ec-7efc2eb43736 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:17.054524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3924d-47b7-4acd-89ec-7efc2eb43736 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-4ce3924d-47b7-4acd-89ec-7efc2eb43736 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:17.066500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:17.067310 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:17.068144 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:17.068144 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:17.068144 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:17.068144 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:17.068144 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:17.068144 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:17.068144 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:17.068144 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:17.068144 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 257/514] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:17.068601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b236d-47c2-4e50-8836-53c4b0b40c22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:17.068672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3924d-47b7-4acd-89ec-7efc2eb43736 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:17.069360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b236d-47c2-4e50-8836-53c4b0b40c22 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:17.093217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:17.169662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3924d-47b7-4acd-89ec-7efc2eb43736 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:17.172308 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 258/515] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:33:17.173700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b236d-47c2-4e50-8836-53c4b0b40c22 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:17.203773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df27eb-caa8-4f37-8a6a-b018a9f8d406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:17.204426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df27eb-caa8-4f37-8a6a-b018a9f8d406 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:17.204710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df27eb-caa8-4f37-8a6a-b018a9f8d406 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:17.205019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df27eb-caa8-4f37-8a6a-b018a9f8d406 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:17.212494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df27eb-caa8-4f37-8a6a-b018a9f8d406 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:17.213988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df27eb-caa8-4f37-8a6a-b018a9f8d406 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-62df27eb-caa8-4f37-8a6a-b018a9f8d406 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:17.226243 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:17.227164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:17.228024 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:17.228024 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:17.228024 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:17.228024 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:17.228024 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:17.228024 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:17.228024 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:17.228024 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:17.228024 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 20:33:17.228024 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:17.228024 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 258/516] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:17.228608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df27eb-caa8-4f37-8a6a-b018a9f8d406 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:17.272283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:17.278146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b236d-47c2-4e50-8836-53c4b0b40c22 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:17.297380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:17.312106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd55f5b-0082-4840-bb23-83ef79e8a794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:17.312709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd55f5b-0082-4840-bb23-83ef79e8a794 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:17.313022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd55f5b-0082-4840-bb23-83ef79e8a794 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:17.313296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd55f5b-0082-4840-bb23-83ef79e8a794 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:17.314479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd55f5b-0082-4840-bb23-83ef79e8a794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:17.314780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd55f5b-0082-4840-bb23-83ef79e8a794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:17.316153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd55f5b-0082-4840-bb23-83ef79e8a794 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:17.327090 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 259/517] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:17.327353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df27eb-caa8-4f37-8a6a-b018a9f8d406 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:17.327867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd55f5b-0082-4840-bb23-83ef79e8a794 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:17.363791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ebfee-d533-47ac-9fe7-8d38b4db1f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:17.364388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ebfee-d533-47ac-9fe7-8d38b4db1f0e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:17.364674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ebfee-d533-47ac-9fe7-8d38b4db1f0e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:17.364980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ebfee-d533-47ac-9fe7-8d38b4db1f0e None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:17.365621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ebfee-d533-47ac-9fe7-8d38b4db1f0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:17.365928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ebfee-d533-47ac-9fe7-8d38b4db1f0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:17.367307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ebfee-d533-47ac-9fe7-8d38b4db1f0e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:17.379307 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 259/518] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:17.379942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ebfee-d533-47ac-9fe7-8d38b4db1f0e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:17.429832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd55f5b-0082-4840-bb23-83ef79e8a794 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:17.464306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77d02a-9948-483d-a65d-bd010071f660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:17.464938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77d02a-9948-483d-a65d-bd010071f660 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:17.465230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77d02a-9948-483d-a65d-bd010071f660 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:17.465512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77d02a-9948-483d-a65d-bd010071f660 None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:17.466246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77d02a-9948-483d-a65d-bd010071f660 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:17.466600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77d02a-9948-483d-a65d-bd010071f660 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:17.468233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77d02a-9948-483d-a65d-bd010071f660 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:17.477911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ebfee-d533-47ac-9fe7-8d38b4db1f0e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:17.480283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:17.489923 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 260/519] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:17.490747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77d02a-9948-483d-a65d-bd010071f660 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:17.512934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14e361-fdb4-4478-9dd3-6476361e60ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:17.513552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14e361-fdb4-4478-9dd3-6476361e60ab None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:17.513838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14e361-fdb4-4478-9dd3-6476361e60ab None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:17.514110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14e361-fdb4-4478-9dd3-6476361e60ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:17.524404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14e361-fdb4-4478-9dd3-6476361e60ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:17.525877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14e361-fdb4-4478-9dd3-6476361e60ab None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:17.532625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-ae14e361-fdb4-4478-9dd3-6476361e60ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:17.540127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:17.541254 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:17.542104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:17.542104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:17.542104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:17.542104 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 260/520] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:17.542104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14e361-fdb4-4478-9dd3-6476361e60ab None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:17.596497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77d02a-9948-483d-a65d-bd010071f660 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:17.630827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e60d7-e64a-4b66-afc1-c8fcfb058a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:17.631429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e60d7-e64a-4b66-afc1-c8fcfb058a9b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:17.631714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e60d7-e64a-4b66-afc1-c8fcfb058a9b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:17.631987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e60d7-e64a-4b66-afc1-c8fcfb058a9b None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:17.632707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e60d7-e64a-4b66-afc1-c8fcfb058a9b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:17.633041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e60d7-e64a-4b66-afc1-c8fcfb058a9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:17.634466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e60d7-e64a-4b66-afc1-c8fcfb058a9b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:17.639254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14e361-fdb4-4478-9dd3-6476361e60ab None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:17.654997 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 261/521] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Sep 3 20:33:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:17.655618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e60d7-e64a-4b66-afc1-c8fcfb058a9b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:17.680304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acb865-5f40-4db6-a86a-580f62640479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:17.680845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb865-5f40-4db6-a86a-580f62640479 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:17.681150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb865-5f40-4db6-a86a-580f62640479 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:17.681432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb865-5f40-4db6-a86a-580f62640479 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:17.689302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acb865-5f40-4db6-a86a-580f62640479 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:17.690770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acb865-5f40-4db6-a86a-580f62640479 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-87acb865-5f40-4db6-a86a-580f62640479 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:17.701608 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:17.702598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:17.704156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:17.704156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:17.704156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:17.704156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:17.704156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:17.704156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:17.704156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:17.704156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:17.704156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 20:33:17.704156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:17.704156 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 261/522] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:17.704156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acb865-5f40-4db6-a86a-580f62640479 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:17.723148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:17.756405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:17.760265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e60d7-e64a-4b66-afc1-c8fcfb058a9b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:17.793854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfd3c1-5721-47e6-88b3-0303db0fb309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:17.794455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfd3c1-5721-47e6-88b3-0303db0fb309 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:17.794737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfd3c1-5721-47e6-88b3-0303db0fb309 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:17.795007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfd3c1-5721-47e6-88b3-0303db0fb309 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:17.795650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfd3c1-5721-47e6-88b3-0303db0fb309 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:17.797107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfd3c1-5721-47e6-88b3-0303db0fb309 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:17.803675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acb865-5f40-4db6-a86a-580f62640479 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:17.818700 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 262/523] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:33:17 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:33:17.819520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfd3c1-5721-47e6-88b3-0303db0fb309 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:17.855274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce334851-d993-4735-b269-45c4f07d24b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:17.855902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce334851-d993-4735-b269-45c4f07d24b4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:17.856186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce334851-d993-4735-b269-45c4f07d24b4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:17.856460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce334851-d993-4735-b269-45c4f07d24b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:17.857386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce334851-d993-4735-b269-45c4f07d24b4 None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=a56702329eae48819121dcb58f7e3070) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:17.857838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce334851-d993-4735-b269-45c4f07d24b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:17.859351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce334851-d993-4735-b269-45c4f07d24b4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:17.876936 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 262/524] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:17 2022] GET /identity/v3/role_assignments?user.id=a56702329eae48819121dcb58f7e3070&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:17.876936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce334851-d993-4735-b269-45c4f07d24b4 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:17.958912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfd3c1-5721-47e6-88b3-0303db0fb309 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:17.993838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:18.011168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8053e-0a41-4876-a39f-55d0d675d807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:18.011806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8053e-0a41-4876-a39f-55d0d675d807 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:18.012401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8053e-0a41-4876-a39f-55d0d675d807 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:18.012401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8053e-0a41-4876-a39f-55d0d675d807 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:18.013042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8053e-0a41-4876-a39f-55d0d675d807 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:18.013353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8053e-0a41-4876-a39f-55d0d675d807 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:18.014764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8053e-0a41-4876-a39f-55d0d675d807 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:18.028521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce334851-d993-4735-b269-45c4f07d24b4 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:18.030740 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 263/525] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:18.031823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8053e-0a41-4876-a39f-55d0d675d807 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:18.050269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:18.086099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaf2d3-e723-4261-88b7-719026a544c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:18.086720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaf2d3-e723-4261-88b7-719026a544c4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:18.087002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaf2d3-e723-4261-88b7-719026a544c4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:18.087277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaf2d3-e723-4261-88b7-719026a544c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:18.166579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aaf2d3-e723-4261-88b7-719026a544c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:18.168431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aaf2d3-e723-4261-88b7-719026a544c4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:18.178988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8053e-0a41-4876-a39f-55d0d675d807 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:18.224349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac3182-cf85-42bc-9623-1af2ab45235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:18.225477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac3182-cf85-42bc-9623-1af2ab45235a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:18.225970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac3182-cf85-42bc-9623-1af2ab45235a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:18.226474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac3182-cf85-42bc-9623-1af2ab45235a None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:18.227509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac3182-cf85-42bc-9623-1af2ab45235a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:18.228078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac3182-cf85-42bc-9623-1af2ab45235a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:18.229826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac3182-cf85-42bc-9623-1af2ab45235a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:18.250043 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 264/526] 10.222.0.16 () {56 vars in 1139 bytes} [Sat Sep 3 20:33:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:18.251310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac3182-cf85-42bc-9623-1af2ab45235a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:18.273379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:18.304866 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 263/527] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:33:18.304866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaf2d3-e723-4261-88b7-719026a544c4 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:18.339073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:18.375241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac3182-cf85-42bc-9623-1af2ab45235a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:18.413012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279c695-4d8b-4f1f-801b-50944adb0b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:18.414030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279c695-4d8b-4f1f-801b-50944adb0b47 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:18.414525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279c695-4d8b-4f1f-801b-50944adb0b47 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:18.415008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279c695-4d8b-4f1f-801b-50944adb0b47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:18.415113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaf2d3-e723-4261-88b7-719026a544c4 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:18.422799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279c695-4d8b-4f1f-801b-50944adb0b47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:18.425226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279c695-4d8b-4f1f-801b-50944adb0b47 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-e279c695-4d8b-4f1f-801b-50944adb0b47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:18.437702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:18.438630 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:18.439463 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:18.439463 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:18.439463 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:18.439463 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:18.439463 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:18.439463 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:18.439463 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:18.439463 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:18.439463 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 265/528] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:18.440470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279c695-4d8b-4f1f-801b-50944adb0b47 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:18.455075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d304b4-055d-4829-ac9d-cbc12a7f9826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:18.456155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d304b4-055d-4829-ac9d-cbc12a7f9826 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:18.456982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d304b4-055d-4829-ac9d-cbc12a7f9826 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:18.457530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d304b4-055d-4829-ac9d-cbc12a7f9826 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:18.510896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:18.536487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d304b4-055d-4829-ac9d-cbc12a7f9826 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:18.538013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d304b4-055d-4829-ac9d-cbc12a7f9826 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:18.560162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279c695-4d8b-4f1f-801b-50944adb0b47 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:18.597615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:18.604576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf45e1-25e9-4645-bdae-f8d6cb8d3c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:18.605559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf45e1-25e9-4645-bdae-f8d6cb8d3c4f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:18.605976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf45e1-25e9-4645-bdae-f8d6cb8d3c4f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:18.606389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf45e1-25e9-4645-bdae-f8d6cb8d3c4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:18.622101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf45e1-25e9-4645-bdae-f8d6cb8d3c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:18.623570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf45e1-25e9-4645-bdae-f8d6cb8d3c4f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-4bdf45e1-25e9-4645-bdae-f8d6cb8d3c4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:18.640025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:18.640973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:18.642357 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:18.642357 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:18.642357 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:18.646312 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 266/529] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:18.647172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf45e1-25e9-4645-bdae-f8d6cb8d3c4f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:18.656583 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 264/530] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:33:18.662817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d304b4-055d-4829-ac9d-cbc12a7f9826 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:18.748641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:18.753733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf45e1-25e9-4645-bdae-f8d6cb8d3c4f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:18.769838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d304b4-055d-4829-ac9d-cbc12a7f9826 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:18.792241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31987d-5743-48b2-9228-26b393bfbd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:18.792753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31987d-5743-48b2-9228-26b393bfbd0a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:18.792939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31987d-5743-48b2-9228-26b393bfbd0a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:18.793213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31987d-5743-48b2-9228-26b393bfbd0a None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:18.793909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31987d-5743-48b2-9228-26b393bfbd0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:18.794226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31987d-5743-48b2-9228-26b393bfbd0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:18.795725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31987d-5743-48b2-9228-26b393bfbd0a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:18.807637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae37f6d-002f-4cf4-8346-47aa8c507949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:18.808459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae37f6d-002f-4cf4-8346-47aa8c507949 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:18.812926 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 267/531] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:18.812926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31987d-5743-48b2-9228-26b393bfbd0a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:18.813490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae37f6d-002f-4cf4-8346-47aa8c507949 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:18.813961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae37f6d-002f-4cf4-8346-47aa8c507949 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:18.814759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae37f6d-002f-4cf4-8346-47aa8c507949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:18.815225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae37f6d-002f-4cf4-8346-47aa8c507949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:18.816735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae37f6d-002f-4cf4-8346-47aa8c507949 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:18.830350 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 265/532] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:18.831311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae37f6d-002f-4cf4-8346-47aa8c507949 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:18.864485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:18.925289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31987d-5743-48b2-9228-26b393bfbd0a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:18.938357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae37f6d-002f-4cf4-8346-47aa8c507949 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:18.964074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91265699-fc77-4320-bd2e-2824d3a37d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:18.964535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91265699-fc77-4320-bd2e-2824d3a37d63 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:18.964830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91265699-fc77-4320-bd2e-2824d3a37d63 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:18.965286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91265699-fc77-4320-bd2e-2824d3a37d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:18.976434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92e7f6-2b59-4dc8-bf98-3eb601703a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:18.977101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91265699-fc77-4320-bd2e-2824d3a37d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:18.977431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92e7f6-2b59-4dc8-bf98-3eb601703a6d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:18.977855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92e7f6-2b59-4dc8-bf98-3eb601703a6d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:18.978269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92e7f6-2b59-4dc8-bf98-3eb601703a6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:18.978619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91265699-fc77-4320-bd2e-2824d3a37d63 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:18.979499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92e7f6-2b59-4dc8-bf98-3eb601703a6d None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=7a36c3c09b2d425fbf6ccbe051c293d4) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:18.979952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92e7f6-2b59-4dc8-bf98-3eb601703a6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:18.981752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92e7f6-2b59-4dc8-bf98-3eb601703a6d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:18.992927 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 266/533] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:18 2022] GET /identity/v3/role_assignments?user.id=7a36c3c09b2d425fbf6ccbe051c293d4&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:18.994271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92e7f6-2b59-4dc8-bf98-3eb601703a6d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-91265699-fc77-4320-bd2e-2824d3a37d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:18.994816 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:18.995858 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:18.996747 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:18.996747 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:18.996747 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:19.001672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:19.002716 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 268/534] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:19.003993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91265699-fc77-4320-bd2e-2824d3a37d63 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:19.116392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:19.132141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91265699-fc77-4320-bd2e-2824d3a37d63 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:19.167452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92e7f6-2b59-4dc8-bf98-3eb601703a6d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:19.184080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed4d23-3400-4b52-a7c8-b6d0c1eeb075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:19.185102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed4d23-3400-4b52-a7c8-b6d0c1eeb075 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:19.185597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed4d23-3400-4b52-a7c8-b6d0c1eeb075 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:19.186176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed4d23-3400-4b52-a7c8-b6d0c1eeb075 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:19.187148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed4d23-3400-4b52-a7c8-b6d0c1eeb075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:19.187701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed4d23-3400-4b52-a7c8-b6d0c1eeb075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:19.189910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed4d23-3400-4b52-a7c8-b6d0c1eeb075 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:19.205337 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 269/535] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:19.214965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2361fc-adf8-4243-91b2-f163d57e465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:19.215201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed4d23-3400-4b52-a7c8-b6d0c1eeb075 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:19.215284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2361fc-adf8-4243-91b2-f163d57e465d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:19.215284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2361fc-adf8-4243-91b2-f163d57e465d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:19.215284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2361fc-adf8-4243-91b2-f163d57e465d None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:19.220928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2361fc-adf8-4243-91b2-f163d57e465d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:19.222825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2361fc-adf8-4243-91b2-f163d57e465d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-4f2361fc-adf8-4243-91b2-f163d57e465d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:19.235666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:19.236598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:19.237473 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:19.237473 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:19.237473 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:19.237473 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:19.237473 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:19.237473 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:19.237473 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:19.237473 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:19.237473 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 267/536] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:19.245156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2361fc-adf8-4243-91b2-f163d57e465d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:19.318928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:19.351375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2361fc-adf8-4243-91b2-f163d57e465d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:19.388134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a5988-c886-4349-8342-7a8214a7e561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:19.389078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a5988-c886-4349-8342-7a8214a7e561 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:19.389373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a5988-c886-4349-8342-7a8214a7e561 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:19.389657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a5988-c886-4349-8342-7a8214a7e561 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:19.390358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a5988-c886-4349-8342-7a8214a7e561 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:19.390671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a5988-c886-4349-8342-7a8214a7e561 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:19.392174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a5988-c886-4349-8342-7a8214a7e561 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:19.411545 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 268/537] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:19.417939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a5988-c886-4349-8342-7a8214a7e561 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:19.483880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed4d23-3400-4b52-a7c8-b6d0c1eeb075 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:19.554795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb763ab-0008-42cb-8294-d4b8dad3e7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:19.555452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb763ab-0008-42cb-8294-d4b8dad3e7cc None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:19.555740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb763ab-0008-42cb-8294-d4b8dad3e7cc None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:19.556022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb763ab-0008-42cb-8294-d4b8dad3e7cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:19.571481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb763ab-0008-42cb-8294-d4b8dad3e7cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:19.572999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb763ab-0008-42cb-8294-d4b8dad3e7cc None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-bdb763ab-0008-42cb-8294-d4b8dad3e7cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:19.598944 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:19.599843 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:19.601341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:19.601341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:19.601341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:19.601341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:19.601341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:19.601341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:19.601341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:19.601341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:19.601341 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 270/538] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:19.606456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb763ab-0008-42cb-8294-d4b8dad3e7cc None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:19.616858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a5988-c886-4349-8342-7a8214a7e561 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:19.644580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:19.659733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:19.681097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e9ffb-821f-4a69-bbe3-3617c8d5bbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:19.685823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e9ffb-821f-4a69-bbe3-3617c8d5bbb4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:19.686314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e9ffb-821f-4a69-bbe3-3617c8d5bbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:19.686787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e9ffb-821f-4a69-bbe3-3617c8d5bbb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:19.687718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e9ffb-821f-4a69-bbe3-3617c8d5bbb4 None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=133a1ff64e224593be467eb7c27cbb09) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:19.688225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e9ffb-821f-4a69-bbe3-3617c8d5bbb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:19.689935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e9ffb-821f-4a69-bbe3-3617c8d5bbb4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:19.702699 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 269/539] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:19 2022] GET /identity/v3/role_assignments?user.id=133a1ff64e224593be467eb7c27cbb09&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:19.774014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb763ab-0008-42cb-8294-d4b8dad3e7cc None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:19.814446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b4582-2ffe-418d-bf1b-25d9225b05b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:19.815913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b4582-2ffe-418d-bf1b-25d9225b05b0 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:19.816399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b4582-2ffe-418d-bf1b-25d9225b05b0 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:19.816872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b4582-2ffe-418d-bf1b-25d9225b05b0 None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:19.817843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b4582-2ffe-418d-bf1b-25d9225b05b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:19.818373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b4582-2ffe-418d-bf1b-25d9225b05b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:19.820071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b4582-2ffe-418d-bf1b-25d9225b05b0 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:19.843358 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 271/540] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:19.849438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e9ffb-821f-4a69-bbe3-3617c8d5bbb4 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:19.982295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e9ffb-821f-4a69-bbe3-3617c8d5bbb4 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:20.028419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b52f1b-4c90-4fa2-ac7f-c94683028ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:20.029470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b52f1b-4c90-4fa2-ac7f-c94683028ad8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:20.029950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b52f1b-4c90-4fa2-ac7f-c94683028ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:20.030435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b52f1b-4c90-4fa2-ac7f-c94683028ad8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:20.031375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b52f1b-4c90-4fa2-ac7f-c94683028ad8 None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=c0fb67db342b477aaf547543c498434f) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:20.032043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b52f1b-4c90-4fa2-ac7f-c94683028ad8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:20.034242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b52f1b-4c90-4fa2-ac7f-c94683028ad8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:20.042840 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 270/541] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:19 2022] GET /identity/v3/role_assignments?user.id=c0fb67db342b477aaf547543c498434f&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:20.433068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434a386-2a85-407d-b7dd-6702d29e8eab None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:20.433238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434a386-2a85-407d-b7dd-6702d29e8eab None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:20.433520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434a386-2a85-407d-b7dd-6702d29e8eab None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:20.435136 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 272/542] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:20.440839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e26fb-2dc4-4395-a7fc-8e2437d3e498 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:20.441297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e26fb-2dc4-4395-a7fc-8e2437d3e498 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:20.441722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e26fb-2dc4-4395-a7fc-8e2437d3e498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:20.470852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-310e26fb-2dc4-4395-a7fc-8e2437d3e498 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:20.472327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-310e26fb-2dc4-4395-a7fc-8e2437d3e498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:20.534447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:20.564514 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 271/543] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:33:20.661579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fe985-ed19-423e-9297-783816663943 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:20.661940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fe985-ed19-423e-9297-783816663943 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:20.662212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fe985-ed19-423e-9297-783816663943 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:20.663653 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 273/544] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:20.669058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19f3c9-e9e1-427b-a547-a86a9b4728e0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:20.669839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19f3c9-e9e1-427b-a547-a86a9b4728e0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:20.670284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19f3c9-e9e1-427b-a547-a86a9b4728e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:20.695794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6c19f3c9-e9e1-427b-a547-a86a9b4728e0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:20.697273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6c19f3c9-e9e1-427b-a547-a86a9b4728e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:20.790629 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 272/545] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:21.071014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fe985-ed19-423e-9297-783816663943 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:21.177666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:21.209905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fe985-ed19-423e-9297-783816663943 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:21.256175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be80fb-7101-41da-a7ab-01ab3d196210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:21.256814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be80fb-7101-41da-a7ab-01ab3d196210 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:21.257129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be80fb-7101-41da-a7ab-01ab3d196210 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:21.257465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be80fb-7101-41da-a7ab-01ab3d196210 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:21.274011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be80fb-7101-41da-a7ab-01ab3d196210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-22be80fb-7101-41da-a7ab-01ab3d196210 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:21.278821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:21.279715 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:21.281119 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 274/546] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:21.286374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19f3c9-e9e1-427b-a547-a86a9b4728e0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:21.316280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be80fb-7101-41da-a7ab-01ab3d196210 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:21.475628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19f3c9-e9e1-427b-a547-a86a9b4728e0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:21.498298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:21.515343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be80fb-7101-41da-a7ab-01ab3d196210 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:21.532027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81df23-f2f9-4d29-8299-17f46d503632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:21.533569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81df23-f2f9-4d29-8299-17f46d503632 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:21.534065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81df23-f2f9-4d29-8299-17f46d503632 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:21.534532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81df23-f2f9-4d29-8299-17f46d503632 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:21.535415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81df23-f2f9-4d29-8299-17f46d503632 None admin] RBAC: Adding query filter params (name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:21.535925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81df23-f2f9-4d29-8299-17f46d503632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:21.537567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81df23-f2f9-4d29-8299-17f46d503632 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:21.547341 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 273/547] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:21.549717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4a1b8-70ed-47f6-92f2-f262b3dd8cc3 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:21.550213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4a1b8-70ed-47f6-92f2-f262b3dd8cc3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:21.550746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4a1b8-70ed-47f6-92f2-f262b3dd8cc3 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:21.556668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c460eb1-7779-401b-b3f6-9c6e7c48b934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:21.556668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460eb1-7779-401b-b3f6-9c6e7c48b934 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:21.556668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460eb1-7779-401b-b3f6-9c6e7c48b934 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:21.556668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460eb1-7779-401b-b3f6-9c6e7c48b934 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:21.557236 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 274/548] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:21.558249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4a1b8-70ed-47f6-92f2-f262b3dd8cc3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:21.558365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:21.567371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c460eb1-7779-401b-b3f6-9c6e7c48b934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-5c460eb1-7779-401b-b3f6-9c6e7c48b934 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:21.572295 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:21.573221 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:21.574261 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 275/549] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:21.575547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777511fb-d14b-4113-872e-2312d509f3ba None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:21.575899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777511fb-d14b-4113-872e-2312d509f3ba None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:21.576244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777511fb-d14b-4113-872e-2312d509f3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:21.604505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-777511fb-d14b-4113-872e-2312d509f3ba None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:21.606862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-777511fb-d14b-4113-872e-2312d509f3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:21.678253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4a1b8-70ed-47f6-92f2-f262b3dd8cc3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:21.689459 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 276/550] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:21.690442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777511fb-d14b-4113-872e-2312d509f3ba None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:21.714975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0362d-1724-4278-beb7-6aa6e00c29d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:21.715612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0362d-1724-4278-beb7-6aa6e00c29d6 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:21.715900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0362d-1724-4278-beb7-6aa6e00c29d6 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:21.716181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0362d-1724-4278-beb7-6aa6e00c29d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:21.728813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0362d-1724-4278-beb7-6aa6e00c29d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:21.730256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0362d-1724-4278-beb7-6aa6e00c29d6 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-0fd0362d-1724-4278-beb7-6aa6e00c29d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:21.745275 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:21.746360 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:21.747655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:21.747655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:21.747655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:21.747655 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 275/551] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:21.753955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0362d-1724-4278-beb7-6aa6e00c29d6 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:21.814398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:21.852697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777511fb-d14b-4113-872e-2312d509f3ba None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:21.905242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:21.921282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af243fd-7eb6-4102-b0d0-360f0a97c53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:21.921902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af243fd-7eb6-4102-b0d0-360f0a97c53b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:21.922196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af243fd-7eb6-4102-b0d0-360f0a97c53b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:21.922478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af243fd-7eb6-4102-b0d0-360f0a97c53b None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:21.923193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af243fd-7eb6-4102-b0d0-360f0a97c53b None admin] RBAC: Adding query filter params (name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:21.923509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af243fd-7eb6-4102-b0d0-360f0a97c53b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:21.926075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af243fd-7eb6-4102-b0d0-360f0a97c53b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:21.946089 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 277/552] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:21.951995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af243fd-7eb6-4102-b0d0-360f0a97c53b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:21.953966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0362d-1724-4278-beb7-6aa6e00c29d6 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:22.026251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db7acf-2aa5-4fde-8303-6cc9a6b2866c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:22.027198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db7acf-2aa5-4fde-8303-6cc9a6b2866c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.027675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db7acf-2aa5-4fde-8303-6cc9a6b2866c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.028145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db7acf-2aa5-4fde-8303-6cc9a6b2866c None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.028998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db7acf-2aa5-4fde-8303-6cc9a6b2866c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:22.029589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db7acf-2aa5-4fde-8303-6cc9a6b2866c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:22.031264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db7acf-2aa5-4fde-8303-6cc9a6b2866c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:22.055345 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 276/553] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:22.060451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db7acf-2aa5-4fde-8303-6cc9a6b2866c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:22.061593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af243fd-7eb6-4102-b0d0-360f0a97c53b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:22.098638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54256e39-30a6-47be-bab8-ebebdf9e1359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:22.099324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54256e39-30a6-47be-bab8-ebebdf9e1359 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.099614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54256e39-30a6-47be-bab8-ebebdf9e1359 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.099889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54256e39-30a6-47be-bab8-ebebdf9e1359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.111461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54256e39-30a6-47be-bab8-ebebdf9e1359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:22.112959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54256e39-30a6-47be-bab8-ebebdf9e1359 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-54256e39-30a6-47be-bab8-ebebdf9e1359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:22.140081 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:22.141021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:22.142872 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:22.142872 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:22.142872 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:22.142872 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 278/554] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:22.143301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54256e39-30a6-47be-bab8-ebebdf9e1359 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:22.188656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:22.190370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:22.209337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db7acf-2aa5-4fde-8303-6cc9a6b2866c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:22.250919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1873b4b-28a8-4fd0-a4d4-bc69dce5d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:22.251906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1873b4b-28a8-4fd0-a4d4-bc69dce5d4b3 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.252357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1873b4b-28a8-4fd0-a4d4-bc69dce5d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.252783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1873b4b-28a8-4fd0-a4d4-bc69dce5d4b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.257177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54256e39-30a6-47be-bab8-ebebdf9e1359 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:22.261671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1873b4b-28a8-4fd0-a4d4-bc69dce5d4b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:22.263405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1873b4b-28a8-4fd0-a4d4-bc69dce5d4b3 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-c1873b4b-28a8-4fd0-a4d4-bc69dce5d4b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:22.277357 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:22.278572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:22.279669 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:22.279669 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:22.279669 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:22.279669 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:22.279669 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:22.279669 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:22.279669 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:22.279669 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:22.279669 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:33:22.279669 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:22.282078 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 277/555] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 3 20:33:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:22.283599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1873b4b-28a8-4fd0-a4d4-bc69dce5d4b3 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:22.293661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f17c38-f850-4f31-a518-5d68fe5aaeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:22.294148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f17c38-f850-4f31-a518-5d68fe5aaeb5 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.294439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f17c38-f850-4f31-a518-5d68fe5aaeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.294718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f17c38-f850-4f31-a518-5d68fe5aaeb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.306736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f17c38-f850-4f31-a518-5d68fe5aaeb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-69f17c38-f850-4f31-a518-5d68fe5aaeb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:22.312421 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:22.314089 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 279/556] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:22.315469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f17c38-f850-4f31-a518-5d68fe5aaeb5 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:22.404294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1873b4b-28a8-4fd0-a4d4-bc69dce5d4b3 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:22.420876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:22.424396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f17c38-f850-4f31-a518-5d68fe5aaeb5 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:22.440037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1aa3b-4b20-41fe-a38e-17bcd2862fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:22.440656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1aa3b-4b20-41fe-a38e-17bcd2862fa0 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.440969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1aa3b-4b20-41fe-a38e-17bcd2862fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.441259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1aa3b-4b20-41fe-a38e-17bcd2862fa0 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.441905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1aa3b-4b20-41fe-a38e-17bcd2862fa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:22.442339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1aa3b-4b20-41fe-a38e-17bcd2862fa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:22.443719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1aa3b-4b20-41fe-a38e-17bcd2862fa0 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:22.455258 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 278/557] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:22.456052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1aa3b-4b20-41fe-a38e-17bcd2862fa0 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:22.466064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:22.468656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9712ec2-f837-44ca-909e-01466292ec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:22.469515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9712ec2-f837-44ca-909e-01466292ec72 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.470013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9712ec2-f837-44ca-909e-01466292ec72 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.470294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9712ec2-f837-44ca-909e-01466292ec72 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.471030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9712ec2-f837-44ca-909e-01466292ec72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:22.471341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9712ec2-f837-44ca-909e-01466292ec72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:22.472772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9712ec2-f837-44ca-909e-01466292ec72 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:22.491875 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 280/558] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 3 20:33:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:22.492980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abea78-7179-4514-96a8-0e374bc230e8 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.493275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abea78-7179-4514-96a8-0e374bc230e8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.493553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abea78-7179-4514-96a8-0e374bc230e8 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.494881 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 281/559] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:22.495440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abea78-7179-4514-96a8-0e374bc230e8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:22.569465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1aa3b-4b20-41fe-a38e-17bcd2862fa0 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:22.603007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abea78-7179-4514-96a8-0e374bc230e8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:22.604525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff9137-b549-4e8b-af0b-f6aee73ff9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:22.605203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff9137-b549-4e8b-af0b-f6aee73ff9ec None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.605495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff9137-b549-4e8b-af0b-f6aee73ff9ec None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.605779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff9137-b549-4e8b-af0b-f6aee73ff9ec None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.606846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff9137-b549-4e8b-af0b-f6aee73ff9ec None admin] RBAC: Adding query filter params (name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:22.606846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff9137-b549-4e8b-af0b-f6aee73ff9ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:22.608196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff9137-b549-4e8b-af0b-f6aee73ff9ec None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:22.623116 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 279/560] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:22.623116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff9137-b549-4e8b-af0b-f6aee73ff9ec None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:22.647757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558e943-3ee2-44c0-a0da-8eb39b4802cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:22.648389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558e943-3ee2-44c0-a0da-8eb39b4802cf None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.648672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558e943-3ee2-44c0-a0da-8eb39b4802cf None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.648973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558e943-3ee2-44c0-a0da-8eb39b4802cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.656525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558e943-3ee2-44c0-a0da-8eb39b4802cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:22.658022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558e943-3ee2-44c0-a0da-8eb39b4802cf None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-6558e943-3ee2-44c0-a0da-8eb39b4802cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:22.670417 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:22.671355 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:22.672269 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:22.672269 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:22.672269 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:22.672269 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:22.672269 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:22.672269 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:22.672269 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:22.672269 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:22.672269 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 20:33:22.672269 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:22.672269 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 282/561] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:22.673912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c484303-ef03-491a-b33c-87a5d936bdcb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.673912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c484303-ef03-491a-b33c-87a5d936bdcb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.673912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c484303-ef03-491a-b33c-87a5d936bdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.691970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7c484303-ef03-491a-b33c-87a5d936bdcb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:22.693179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7c484303-ef03-491a-b33c-87a5d936bdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:22.702603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:22.708045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:22.740449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff9137-b549-4e8b-af0b-f6aee73ff9ec None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:22.777572 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 283/562] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:22.777572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c484303-ef03-491a-b33c-87a5d936bdcb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:22.781638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d0b71-889e-4983-aec1-adf46e70962c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:22.782845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d0b71-889e-4983-aec1-adf46e70962c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.783350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d0b71-889e-4983-aec1-adf46e70962c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.783533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d0b71-889e-4983-aec1-adf46e70962c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.795096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d0b71-889e-4983-aec1-adf46e70962c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:22.796599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d0b71-889e-4983-aec1-adf46e70962c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-fe3d0b71-889e-4983-aec1-adf46e70962c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:22.812382 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:22.817515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:22.818447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:22.818447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:22.818447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:22.818447 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 280/563] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:22.818447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d0b71-889e-4983-aec1-adf46e70962c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:22.882774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c484303-ef03-491a-b33c-87a5d936bdcb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:22.920084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61df80-c763-49a5-9e03-46b9e5243a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:22.920729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61df80-c763-49a5-9e03-46b9e5243a94 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.920842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d0b71-889e-4983-aec1-adf46e70962c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:22.921050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61df80-c763-49a5-9e03-46b9e5243a94 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.921328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61df80-c763-49a5-9e03-46b9e5243a94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.931946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61df80-c763-49a5-9e03-46b9e5243a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:22.933434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61df80-c763-49a5-9e03-46b9e5243a94 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:22.942638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:22.945476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-8f61df80-c763-49a5-9e03-46b9e5243a94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:22.949956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:22.950889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:22.951760 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:22.951760 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:22.951760 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:22.951760 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 284/564] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:22.952816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c342234-2ac2-43c2-98f6-2b463b1e5084 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.953070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c342234-2ac2-43c2-98f6-2b463b1e5084 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.953357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c342234-2ac2-43c2-98f6-2b463b1e5084 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.954755 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 285/565] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:22.959855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c342234-2ac2-43c2-98f6-2b463b1e5084 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:22.964656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79974d8-ab9a-4e17-94ac-8916d419cd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:22.966294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79974d8-ab9a-4e17-94ac-8916d419cd93 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:22.966294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79974d8-ab9a-4e17-94ac-8916d419cd93 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:22.966294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79974d8-ab9a-4e17-94ac-8916d419cd93 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:22.966562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79974d8-ab9a-4e17-94ac-8916d419cd93 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:22.967141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79974d8-ab9a-4e17-94ac-8916d419cd93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:22.968543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79974d8-ab9a-4e17-94ac-8916d419cd93 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:22.991936 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 281/566] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Sep 3 20:33:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:22.991936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79974d8-ab9a-4e17-94ac-8916d419cd93 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:23.074890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c342234-2ac2-43c2-98f6-2b463b1e5084 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:23.097798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79974d8-ab9a-4e17-94ac-8916d419cd93 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:23.110158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7caeb-13f6-4dcc-aece-20dd35fcc751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:23.111320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7caeb-13f6-4dcc-aece-20dd35fcc751 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:23.111770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7caeb-13f6-4dcc-aece-20dd35fcc751 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:23.112051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7caeb-13f6-4dcc-aece-20dd35fcc751 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:23.113117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7caeb-13f6-4dcc-aece-20dd35fcc751 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:23.113583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7caeb-13f6-4dcc-aece-20dd35fcc751 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:23.114977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7caeb-13f6-4dcc-aece-20dd35fcc751 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:23.126214 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 286/567] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:23.127578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6a657-99f7-440c-aaf8-85dcb58f3cf7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:23.127873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6a657-99f7-440c-aaf8-85dcb58f3cf7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:23.128148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6a657-99f7-440c-aaf8-85dcb58f3cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:23.136291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b3722-0f2b-4ee5-b1f5-ef3d9612429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:23.136921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b3722-0f2b-4ee5-b1f5-ef3d9612429d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:23.137210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b3722-0f2b-4ee5-b1f5-ef3d9612429d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:23.137483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b3722-0f2b-4ee5-b1f5-ef3d9612429d None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:23.138126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b3722-0f2b-4ee5-b1f5-ef3d9612429d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:23.138430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b3722-0f2b-4ee5-b1f5-ef3d9612429d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:23.139791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b3722-0f2b-4ee5-b1f5-ef3d9612429d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:23.146643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c4a6a657-99f7-440c-aaf8-85dcb58f3cf7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:23.147855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c4a6a657-99f7-440c-aaf8-85dcb58f3cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:23.151716 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 282/568] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:23.152515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b3722-0f2b-4ee5-b1f5-ef3d9612429d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:23.193555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:23.208786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:23.220715 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 287/569] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:23.221548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6a657-99f7-440c-aaf8-85dcb58f3cf7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:23.260007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b3722-0f2b-4ee5-b1f5-ef3d9612429d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:23.296444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ceeb2b-0e35-4dbc-a77c-a354e3bde23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:23.297356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ceeb2b-0e35-4dbc-a77c-a354e3bde23a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:23.297649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ceeb2b-0e35-4dbc-a77c-a354e3bde23a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:23.297957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ceeb2b-0e35-4dbc-a77c-a354e3bde23a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:23.330474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6a657-99f7-440c-aaf8-85dcb58f3cf7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:23.368827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9864d-42c4-4811-a9e6-0fee3688671a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:23.369507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9864d-42c4-4811-a9e6-0fee3688671a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:23.369772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9864d-42c4-4811-a9e6-0fee3688671a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:23.370053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9864d-42c4-4811-a9e6-0fee3688671a None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:23.370408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ceeb2b-0e35-4dbc-a77c-a354e3bde23a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:23.371956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ceeb2b-0e35-4dbc-a77c-a354e3bde23a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:23.380278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9864d-42c4-4811-a9e6-0fee3688671a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-c9c9864d-42c4-4811-a9e6-0fee3688671a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:23.385111 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:23.385988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:23.386844 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 288/570] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:23.387515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9864d-42c4-4811-a9e6-0fee3688671a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:23.417864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:23.441162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:23.481404 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 283/571] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:33:23.482402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ceeb2b-0e35-4dbc-a77c-a354e3bde23a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:23.500434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9864d-42c4-4811-a9e6-0fee3688671a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:23.536869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300e724-cecd-4f58-9772-24915c03777e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:23.537587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300e724-cecd-4f58-9772-24915c03777e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:23.537798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300e724-cecd-4f58-9772-24915c03777e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:23.538070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300e724-cecd-4f58-9772-24915c03777e None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:23.544627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300e724-cecd-4f58-9772-24915c03777e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:23.546161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300e724-cecd-4f58-9772-24915c03777e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-0300e724-cecd-4f58-9772-24915c03777e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:23.558609 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:23.559530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:23.560482 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:23.560482 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:23.560482 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:23.560482 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:23.560482 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:23.560482 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:23.560482 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:23.560482 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:23.560482 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 289/572] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:23.561017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300e724-cecd-4f58-9772-24915c03777e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:23.586557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ceeb2b-0e35-4dbc-a77c-a354e3bde23a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:23.620451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ed800-b17e-4ad5-a700-d6456d1a34b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:23.621103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ed800-b17e-4ad5-a700-d6456d1a34b4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:23.621387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ed800-b17e-4ad5-a700-d6456d1a34b4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:23.621659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ed800-b17e-4ad5-a700-d6456d1a34b4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:23.628413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ed800-b17e-4ad5-a700-d6456d1a34b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:23.629870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ed800-b17e-4ad5-a700-d6456d1a34b4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:23.631559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a16ed800-b17e-4ad5-a700-d6456d1a34b4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:23.642479 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:23.643437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:23.644377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:23.644377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:23.644377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:23.644377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:23.644377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:23.644377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:23.644377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:23.644377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:23.644377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 03 20:33:23.644377 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:23.644377 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 284/573] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:23.644377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ed800-b17e-4ad5-a700-d6456d1a34b4 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:23.669708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300e724-cecd-4f58-9772-24915c03777e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:23.680271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:23.704029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdad17-d886-4c8e-827b-eff996a05692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:23.705259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdad17-d886-4c8e-827b-eff996a05692 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:23.705546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdad17-d886-4c8e-827b-eff996a05692 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:23.705825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdad17-d886-4c8e-827b-eff996a05692 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:23.706491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdad17-d886-4c8e-827b-eff996a05692 None admin] RBAC: Adding query filter params (name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:23.707057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdad17-d886-4c8e-827b-eff996a05692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:23.708422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdad17-d886-4c8e-827b-eff996a05692 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:23.721145 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 290/574] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:23.722288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdad17-d886-4c8e-827b-eff996a05692 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:23.743943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ed800-b17e-4ad5-a700-d6456d1a34b4 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:23.777666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7dcf6-ed9c-46c4-bd60-9c86d3ca08f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:23.778283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7dcf6-ed9c-46c4-bd60-9c86d3ca08f6 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:23.778567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7dcf6-ed9c-46c4-bd60-9c86d3ca08f6 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:23.778839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7dcf6-ed9c-46c4-bd60-9c86d3ca08f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:23.790356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7dcf6-ed9c-46c4-bd60-9c86d3ca08f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:23.791804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7dcf6-ed9c-46c4-bd60-9c86d3ca08f6 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-c0d7dcf6-ed9c-46c4-bd60-9c86d3ca08f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:23.805979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:23.807067 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:23.807994 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:23.807994 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:23.807994 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:23.807994 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 285/575] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:23.808223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7dcf6-ed9c-46c4-bd60-9c86d3ca08f6 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:23.825521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdad17-d886-4c8e-827b-eff996a05692 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:23.858262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:23.859106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9965f-d0ad-4aef-96e7-d83a1bd366e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:23.859917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9965f-d0ad-4aef-96e7-d83a1bd366e9 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:23.860259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9965f-d0ad-4aef-96e7-d83a1bd366e9 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:23.861158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9965f-d0ad-4aef-96e7-d83a1bd366e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:23.871341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9965f-d0ad-4aef-96e7-d83a1bd366e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-afb9965f-d0ad-4aef-96e7-d83a1bd366e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:23.877148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:23.878091 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:23.878900 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 291/576] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:23.879580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9965f-d0ad-4aef-96e7-d83a1bd366e9 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:23.909497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:23.912828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7dcf6-ed9c-46c4-bd60-9c86d3ca08f6 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:23.945769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a37f4-7c4b-46b1-b20f-60b71e6c181e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:23.946372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a37f4-7c4b-46b1-b20f-60b71e6c181e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:23.946657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a37f4-7c4b-46b1-b20f-60b71e6c181e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:23.946935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a37f4-7c4b-46b1-b20f-60b71e6c181e None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:23.947626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a37f4-7c4b-46b1-b20f-60b71e6c181e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:23.947935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a37f4-7c4b-46b1-b20f-60b71e6c181e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:23.949424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a37f4-7c4b-46b1-b20f-60b71e6c181e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:23.961670 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 286/577] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:23.962774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a37f4-7c4b-46b1-b20f-60b71e6c181e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:23.980702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9965f-d0ad-4aef-96e7-d83a1bd366e9 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:24.017119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde20861-de2e-4c4e-8051-f689d69d6ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:24.017720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde20861-de2e-4c4e-8051-f689d69d6ae8 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:24.018002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde20861-de2e-4c4e-8051-f689d69d6ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:24.018274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde20861-de2e-4c4e-8051-f689d69d6ae8 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:24.018980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde20861-de2e-4c4e-8051-f689d69d6ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:24.019287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde20861-de2e-4c4e-8051-f689d69d6ae8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:24.020713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde20861-de2e-4c4e-8051-f689d69d6ae8 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:24.037495 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 292/578] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Sep 3 20:33:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:24.038284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde20861-de2e-4c4e-8051-f689d69d6ae8 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:24.067185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a37f4-7c4b-46b1-b20f-60b71e6c181e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:24.077732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:24.101517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c2b96-f56a-4d1c-9fd6-2fa43dfa04a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:24.102148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c2b96-f56a-4d1c-9fd6-2fa43dfa04a7 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:24.102433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c2b96-f56a-4d1c-9fd6-2fa43dfa04a7 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:24.102710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c2b96-f56a-4d1c-9fd6-2fa43dfa04a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:24.113310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c2b96-f56a-4d1c-9fd6-2fa43dfa04a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:24.114756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c2b96-f56a-4d1c-9fd6-2fa43dfa04a7 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:24.128123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-be3c2b96-f56a-4d1c-9fd6-2fa43dfa04a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:24.130655 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:24.131645 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:24.133251 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:24.133251 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:24.133251 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:24.133251 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 287/579] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:24.133251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c2b96-f56a-4d1c-9fd6-2fa43dfa04a7 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:24.145118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde20861-de2e-4c4e-8051-f689d69d6ae8 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:24.179234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efafff0-c2e6-499f-ab52-9f16a689cdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:24.179783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efafff0-c2e6-499f-ab52-9f16a689cdd2 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:24.180071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efafff0-c2e6-499f-ab52-9f16a689cdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:24.180349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efafff0-c2e6-499f-ab52-9f16a689cdd2 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:24.181061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efafff0-c2e6-499f-ab52-9f16a689cdd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:24.181370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efafff0-c2e6-499f-ab52-9f16a689cdd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:24.182786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efafff0-c2e6-499f-ab52-9f16a689cdd2 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:24.194394 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 293/580] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:24.195161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efafff0-c2e6-499f-ab52-9f16a689cdd2 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:24.234368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c2b96-f56a-4d1c-9fd6-2fa43dfa04a7 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:24.266886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95261836-3ce8-4dd9-ba94-e5b2327a4024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:24.267480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95261836-3ce8-4dd9-ba94-e5b2327a4024 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:24.267769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95261836-3ce8-4dd9-ba94-e5b2327a4024 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:24.268044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95261836-3ce8-4dd9-ba94-e5b2327a4024 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:24.268697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95261836-3ce8-4dd9-ba94-e5b2327a4024 None admin] RBAC: Adding query filter params (name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:24.269034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95261836-3ce8-4dd9-ba94-e5b2327a4024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:24.270394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95261836-3ce8-4dd9-ba94-e5b2327a4024 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:24.278148 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 288/581] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:24.278756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95261836-3ce8-4dd9-ba94-e5b2327a4024 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:24.302691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efafff0-c2e6-499f-ab52-9f16a689cdd2 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:24.304107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:24.336771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbdff6-2abe-4c7e-8691-21ee2f805a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:24.337456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbdff6-2abe-4c7e-8691-21ee2f805a36 None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:24.337796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbdff6-2abe-4c7e-8691-21ee2f805a36 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:24.338082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbdff6-2abe-4c7e-8691-21ee2f805a36 None admin] PATH_INFO: `/v3/projects/1e040b2fca724c58a2660441a72cd350/users/a56702329eae48819121dcb58f7e3070/roles/a211c6f9ebbc4889b636c26cd511e0e3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:24.352120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbdff6-2abe-4c7e-8691-21ee2f805a36 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e040b2fca724c58a2660441a72cd350, user_id=a56702329eae48819121dcb58f7e3070, role_id=a211c6f9ebbc4889b636c26cd511e0e3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:24.353577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbdff6-2abe-4c7e-8691-21ee2f805a36 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:24.362450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:24.366610 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 294/582] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:33:24 2022] PUT /identity/v3/projects/1e040b2fca724c58a2660441a72cd350/users/a56702329eae48819121dcb58f7e3070/roles/a211c6f9ebbc4889b636c26cd511e0e3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:33:24.367647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbdff6-2abe-4c7e-8691-21ee2f805a36 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:24.393434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95261836-3ce8-4dd9-ba94-e5b2327a4024 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:24.432008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad846f45-373d-4d6d-b8a8-600d3f622ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:24.432580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad846f45-373d-4d6d-b8a8-600d3f622ff7 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:24.432870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad846f45-373d-4d6d-b8a8-600d3f622ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:24.433247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad846f45-373d-4d6d-b8a8-600d3f622ff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:24.446400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad846f45-373d-4d6d-b8a8-600d3f622ff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:24.448151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad846f45-373d-4d6d-b8a8-600d3f622ff7 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-ad846f45-373d-4d6d-b8a8-600d3f622ff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:24.465604 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:24.466677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:24.468267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:24.468267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:24.468267 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:24.468267 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 289/583] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:24.468267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad846f45-373d-4d6d-b8a8-600d3f622ff7 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:24.484171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbdff6-2abe-4c7e-8691-21ee2f805a36 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:24.525098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4aede-c87b-43cc-83ec-24ec796fe4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:24.525733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4aede-c87b-43cc-83ec-24ec796fe4ec None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:24.526082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4aede-c87b-43cc-83ec-24ec796fe4ec None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:24.526370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4aede-c87b-43cc-83ec-24ec796fe4ec None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:24.527085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4aede-c87b-43cc-83ec-24ec796fe4ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:24.527401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4aede-c87b-43cc-83ec-24ec796fe4ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:24.528916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4aede-c87b-43cc-83ec-24ec796fe4ec None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:24.542220 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 295/584] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:24.543380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4aede-c87b-43cc-83ec-24ec796fe4ec None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:24.565828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:24.600463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad846f45-373d-4d6d-b8a8-600d3f622ff7 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:24.617005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:24.638359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeec33d-171a-4eef-bdbb-3c3888f72d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:24.639151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeec33d-171a-4eef-bdbb-3c3888f72d6a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:24.639564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeec33d-171a-4eef-bdbb-3c3888f72d6a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:24.639945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeec33d-171a-4eef-bdbb-3c3888f72d6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:24.648000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeec33d-171a-4eef-bdbb-3c3888f72d6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:24.649823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeec33d-171a-4eef-bdbb-3c3888f72d6a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-adeec33d-171a-4eef-bdbb-3c3888f72d6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:24.662050 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:24.663048 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:24.671035 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:24.671035 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:24.671035 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:24.671035 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:24.671035 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:24.671035 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:24.671035 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:24.671035 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:24.671035 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 290/585] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:24.671703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4aede-c87b-43cc-83ec-24ec796fe4ec None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:24.671792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeec33d-171a-4eef-bdbb-3c3888f72d6a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:24.708263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafd6ea-cb99-405c-8adc-d0d8a9b749aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:24.708918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafd6ea-cb99-405c-8adc-d0d8a9b749aa None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:24.709219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafd6ea-cb99-405c-8adc-d0d8a9b749aa None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:24.709502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafd6ea-cb99-405c-8adc-d0d8a9b749aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:24.721291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafd6ea-cb99-405c-8adc-d0d8a9b749aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:24.722757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafd6ea-cb99-405c-8adc-d0d8a9b749aa None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-0aafd6ea-cb99-405c-8adc-d0d8a9b749aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:24.739135 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:24.740288 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:24.741400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:24.741400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:24.741400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:24.745072 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 296/586] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:24.745936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafd6ea-cb99-405c-8adc-d0d8a9b749aa None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:24.782634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeec33d-171a-4eef-bdbb-3c3888f72d6a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:24.821629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40981a8-d966-4a20-84b4-68a02ffca580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:24.822451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40981a8-d966-4a20-84b4-68a02ffca580 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:24.822805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40981a8-d966-4a20-84b4-68a02ffca580 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:24.823300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40981a8-d966-4a20-84b4-68a02ffca580 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:24.824820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40981a8-d966-4a20-84b4-68a02ffca580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:24.825234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40981a8-d966-4a20-84b4-68a02ffca580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:24.826720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40981a8-d966-4a20-84b4-68a02ffca580 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:24.829798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:24.848208 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 291/587] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:24.848208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40981a8-d966-4a20-84b4-68a02ffca580 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:24.873546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafd6ea-cb99-405c-8adc-d0d8a9b749aa None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:24.889999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:24.918137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008b4ec-12ad-456c-8363-4fe6641a7a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:24.918777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008b4ec-12ad-456c-8363-4fe6641a7a33 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:24.919059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008b4ec-12ad-456c-8363-4fe6641a7a33 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:24.919336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008b4ec-12ad-456c-8363-4fe6641a7a33 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:24.926849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008b4ec-12ad-456c-8363-4fe6641a7a33 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:24.928302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008b4ec-12ad-456c-8363-4fe6641a7a33 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-5008b4ec-12ad-456c-8363-4fe6641a7a33 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:24.939591 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:24.940731 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:24.941748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:24.941748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:24.941748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:24.941748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:24.941748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:24.941748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:24.941748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:24.941748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:24.941748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 20:33:24.941748 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:24.941748 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 297/588] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:24.942335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008b4ec-12ad-456c-8363-4fe6641a7a33 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:24.981301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40981a8-d966-4a20-84b4-68a02ffca580 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:25.018792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a960c-0b16-4f20-8645-ed43c162b372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:25.019605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a960c-0b16-4f20-8645-ed43c162b372 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:25.019980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a960c-0b16-4f20-8645-ed43c162b372 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:25.020258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a960c-0b16-4f20-8645-ed43c162b372 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:25.020971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a960c-0b16-4f20-8645-ed43c162b372 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:25.021284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a960c-0b16-4f20-8645-ed43c162b372 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:25.022716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a960c-0b16-4f20-8645-ed43c162b372 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:25.034408 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 292/589] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:25.035422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a960c-0b16-4f20-8645-ed43c162b372 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:25.061605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008b4ec-12ad-456c-8363-4fe6641a7a33 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:25.097561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:25.100231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd25e9-2dfe-45f3-93d3-0aab4d4a835f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:25.101805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd25e9-2dfe-45f3-93d3-0aab4d4a835f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:25.102221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd25e9-2dfe-45f3-93d3-0aab4d4a835f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:25.102582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd25e9-2dfe-45f3-93d3-0aab4d4a835f None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:25.103447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd25e9-2dfe-45f3-93d3-0aab4d4a835f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:25.103916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd25e9-2dfe-45f3-93d3-0aab4d4a835f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:25.105727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd25e9-2dfe-45f3-93d3-0aab4d4a835f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:25.119912 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 298/590] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:25.119912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd25e9-2dfe-45f3-93d3-0aab4d4a835f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:25.154339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a960c-0b16-4f20-8645-ed43c162b372 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:25.155485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:25.194366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ef827-b3e9-48d6-a3f8-a27c2a96d028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:25.195006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef827-b3e9-48d6-a3f8-a27c2a96d028 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:25.195295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef827-b3e9-48d6-a3f8-a27c2a96d028 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:25.195578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef827-b3e9-48d6-a3f8-a27c2a96d028 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:25.202576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ef827-b3e9-48d6-a3f8-a27c2a96d028 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:25.204053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ef827-b3e9-48d6-a3f8-a27c2a96d028 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-8e9ef827-b3e9-48d6-a3f8-a27c2a96d028 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:25.215481 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:25.216512 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:25.217447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:25.217447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:25.217447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:25.217447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:25.217447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:25.217447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:25.217447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:25.217447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:25.217447 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 293/591] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:25.217447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ef827-b3e9-48d6-a3f8-a27c2a96d028 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:25.239742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd25e9-2dfe-45f3-93d3-0aab4d4a835f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:25.275181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cfa5d-c7dc-40ba-bcf0-dd6bea3d8931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:25.275942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cfa5d-c7dc-40ba-bcf0-dd6bea3d8931 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:25.276285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cfa5d-c7dc-40ba-bcf0-dd6bea3d8931 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:25.276631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cfa5d-c7dc-40ba-bcf0-dd6bea3d8931 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:25.277449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312cfa5d-c7dc-40ba-bcf0-dd6bea3d8931 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:25.277820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312cfa5d-c7dc-40ba-bcf0-dd6bea3d8931 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:25.279317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312cfa5d-c7dc-40ba-bcf0-dd6bea3d8931 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:25.298667 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 299/592] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Sep 3 20:33:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:25.299487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cfa5d-c7dc-40ba-bcf0-dd6bea3d8931 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:25.329840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ef827-b3e9-48d6-a3f8-a27c2a96d028 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:25.346693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:25.368000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42aa275-aacf-430e-81c7-88eaea41bc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:25.368696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42aa275-aacf-430e-81c7-88eaea41bc70 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:25.369087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42aa275-aacf-430e-81c7-88eaea41bc70 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:25.369431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42aa275-aacf-430e-81c7-88eaea41bc70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:25.370227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42aa275-aacf-430e-81c7-88eaea41bc70 None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=83865463b2a84f99a136b73dffeeb378) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:25.370614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42aa275-aacf-430e-81c7-88eaea41bc70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:25.372131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42aa275-aacf-430e-81c7-88eaea41bc70 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:25.388935 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 294/593] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:25 2022] GET /identity/v3/role_assignments?user.id=83865463b2a84f99a136b73dffeeb378&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:25.388935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42aa275-aacf-430e-81c7-88eaea41bc70 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:25.410045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:25.422578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cfa5d-c7dc-40ba-bcf0-dd6bea3d8931 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:25.463989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b542956-2be5-4ff8-8902-5800126f0067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:25.464849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b542956-2be5-4ff8-8902-5800126f0067 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:25.465267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b542956-2be5-4ff8-8902-5800126f0067 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:25.465656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b542956-2be5-4ff8-8902-5800126f0067 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:25.475289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b542956-2be5-4ff8-8902-5800126f0067 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:25.477114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b542956-2be5-4ff8-8902-5800126f0067 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-3b542956-2be5-4ff8-8902-5800126f0067 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:25.491319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:25.492477 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:25.493652 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:25.493652 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:25.493652 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:25.493652 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:25.493652 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:25.493652 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:25.493652 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:25.493652 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:25.493652 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 20:33:25.493652 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:25.495589 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 300/594] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:25.496736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b542956-2be5-4ff8-8902-5800126f0067 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:25.533855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42aa275-aacf-430e-81c7-88eaea41bc70 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:25.588288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb276c83-f084-4398-9a65-be69b40c0bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:25.589211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb276c83-f084-4398-9a65-be69b40c0bca None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:25.589937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb276c83-f084-4398-9a65-be69b40c0bca None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:25.590389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb276c83-f084-4398-9a65-be69b40c0bca None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:25.591805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb276c83-f084-4398-9a65-be69b40c0bca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:25.592315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb276c83-f084-4398-9a65-be69b40c0bca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:25.594257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb276c83-f084-4398-9a65-be69b40c0bca None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:25.611013 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 295/595] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:25.612458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb276c83-f084-4398-9a65-be69b40c0bca None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:25.630208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b542956-2be5-4ff8-8902-5800126f0067 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:25.667367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a653e-2af9-4696-99a6-739da9fa3719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:25.668163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a653e-2af9-4696-99a6-739da9fa3719 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:25.668552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a653e-2af9-4696-99a6-739da9fa3719 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:25.669050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a653e-2af9-4696-99a6-739da9fa3719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:25.672074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:25.682585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a653e-2af9-4696-99a6-739da9fa3719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:25.684639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a653e-2af9-4696-99a6-739da9fa3719 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-a92a653e-2af9-4696-99a6-739da9fa3719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:25.700735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:25.701911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:25.702992 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:25.702992 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:25.702992 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:25.705668 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 301/596] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:25.706226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a653e-2af9-4696-99a6-739da9fa3719 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:25.709429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:25.752383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb276c83-f084-4398-9a65-be69b40c0bca None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:25.789844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd353c1-f733-412f-b2a6-b0aea9d387f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:25.791134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd353c1-f733-412f-b2a6-b0aea9d387f4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:25.791646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd353c1-f733-412f-b2a6-b0aea9d387f4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:25.792072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd353c1-f733-412f-b2a6-b0aea9d387f4 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:25.793041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd353c1-f733-412f-b2a6-b0aea9d387f4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:25.793496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd353c1-f733-412f-b2a6-b0aea9d387f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:25.795141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd353c1-f733-412f-b2a6-b0aea9d387f4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:25.814480 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 296/597] 10.222.0.16 () {56 vars in 1139 bytes} [Sat Sep 3 20:33:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:25.815535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd353c1-f733-412f-b2a6-b0aea9d387f4 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:25.826565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a653e-2af9-4696-99a6-739da9fa3719 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:25.862750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe290c7-f3f1-4b67-b970-3463988be41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:25.863561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe290c7-f3f1-4b67-b970-3463988be41b None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:25.863957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe290c7-f3f1-4b67-b970-3463988be41b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:25.864363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe290c7-f3f1-4b67-b970-3463988be41b None admin] PATH_INFO: `/v3/projects/1e040b2fca724c58a2660441a72cd350/users/7a36c3c09b2d425fbf6ccbe051c293d4/roles/a211c6f9ebbc4889b636c26cd511e0e3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:25.880476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe290c7-f3f1-4b67-b970-3463988be41b None admin] RBAC: Authorizing `identity:create_grant(project_id=1e040b2fca724c58a2660441a72cd350, user_id=7a36c3c09b2d425fbf6ccbe051c293d4, role_id=a211c6f9ebbc4889b636c26cd511e0e3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:25.882143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe290c7-f3f1-4b67-b970-3463988be41b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:25.898725 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 302/598] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:33:25 2022] PUT /identity/v3/projects/1e040b2fca724c58a2660441a72cd350/users/7a36c3c09b2d425fbf6ccbe051c293d4/roles/a211c6f9ebbc4889b636c26cd511e0e3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:33:25.900013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe290c7-f3f1-4b67-b970-3463988be41b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:25.925220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:25.929596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd353c1-f733-412f-b2a6-b0aea9d387f4 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:25.966444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:25.970386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02d953-3670-40b1-a487-2380de3a8f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:25.971264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02d953-3670-40b1-a487-2380de3a8f5d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:25.971700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02d953-3670-40b1-a487-2380de3a8f5d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:25.972149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02d953-3670-40b1-a487-2380de3a8f5d None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:25.973040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02d953-3670-40b1-a487-2380de3a8f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:25.973513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02d953-3670-40b1-a487-2380de3a8f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:25.975745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02d953-3670-40b1-a487-2380de3a8f5d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:25.989518 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 297/599] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:25.997906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02d953-3670-40b1-a487-2380de3a8f5d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:26.029430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe290c7-f3f1-4b67-b970-3463988be41b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:26.067111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cf9af-5949-494b-8b79-81b62ec86348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:26.067901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cf9af-5949-494b-8b79-81b62ec86348 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:26.068295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cf9af-5949-494b-8b79-81b62ec86348 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:26.068677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cf9af-5949-494b-8b79-81b62ec86348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:26.081378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cf9af-5949-494b-8b79-81b62ec86348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:26.083273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cf9af-5949-494b-8b79-81b62ec86348 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-327cf9af-5949-494b-8b79-81b62ec86348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:26.102466 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:26.103660 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:26.104760 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:26.104760 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:26.104760 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:26.105906 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 303/600] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:26.112411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cf9af-5949-494b-8b79-81b62ec86348 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:26.146455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02d953-3670-40b1-a487-2380de3a8f5d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:26.185594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6b0cb-7d3a-4872-b80e-c40273560020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:26.186403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6b0cb-7d3a-4872-b80e-c40273560020 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:26.186821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6b0cb-7d3a-4872-b80e-c40273560020 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:26.187418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6b0cb-7d3a-4872-b80e-c40273560020 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:26.195070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6b0cb-7d3a-4872-b80e-c40273560020 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:26.196754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6b0cb-7d3a-4872-b80e-c40273560020 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-f4a6b0cb-7d3a-4872-b80e-c40273560020 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:26.209914 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:26.211028 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:26.212145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:26.212145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:26.212145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:26.212145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:26.212145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:26.212145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:26.212145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:26.212145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:26.214173 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 298/601] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:26.219617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6b0cb-7d3a-4872-b80e-c40273560020 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:26.262083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:26.270908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cf9af-5949-494b-8b79-81b62ec86348 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:26.274616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:26.322596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39245979-3675-45eb-8796-8cc8faacdafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:26.323434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39245979-3675-45eb-8796-8cc8faacdafc None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:26.323875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39245979-3675-45eb-8796-8cc8faacdafc None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:26.324611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39245979-3675-45eb-8796-8cc8faacdafc None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:26.326073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39245979-3675-45eb-8796-8cc8faacdafc None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:26.326573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39245979-3675-45eb-8796-8cc8faacdafc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:26.328637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39245979-3675-45eb-8796-8cc8faacdafc None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:26.345615 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 304/602] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:26.353958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39245979-3675-45eb-8796-8cc8faacdafc None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:26.375262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6b0cb-7d3a-4872-b80e-c40273560020 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:26.413920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94475510-0550-4f59-b44b-ad2256342358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:26.414585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94475510-0550-4f59-b44b-ad2256342358 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:26.414872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94475510-0550-4f59-b44b-ad2256342358 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:26.415598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94475510-0550-4f59-b44b-ad2256342358 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:26.416298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94475510-0550-4f59-b44b-ad2256342358 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:26.416864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94475510-0550-4f59-b44b-ad2256342358 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:26.418593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94475510-0550-4f59-b44b-ad2256342358 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:26.433969 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 299/603] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:26.445968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94475510-0550-4f59-b44b-ad2256342358 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:26.503776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39245979-3675-45eb-8796-8cc8faacdafc None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:26.549009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf45acf-ace3-4312-b386-754424141887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:26.549653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf45acf-ace3-4312-b386-754424141887 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:26.549942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf45acf-ace3-4312-b386-754424141887 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:26.550223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf45acf-ace3-4312-b386-754424141887 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:26.557579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf45acf-ace3-4312-b386-754424141887 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:26.559057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf45acf-ace3-4312-b386-754424141887 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-6cf45acf-ace3-4312-b386-754424141887 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:26.571074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:26.572113 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:26.573070 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:26.573070 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:26.573070 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:26.573070 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:26.573070 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:26.573070 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:26.573070 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:26.573070 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:26.576835 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 305/604] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:26.578055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf45acf-ace3-4312-b386-754424141887 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:26.594026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:26.607712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:26.633150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94475510-0550-4f59-b44b-ad2256342358 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:26.703260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329afdd-f32e-4130-8f78-9b5ed6f5cddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:26.703929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329afdd-f32e-4130-8f78-9b5ed6f5cddc None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:26.704769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329afdd-f32e-4130-8f78-9b5ed6f5cddc None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:26.705074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329afdd-f32e-4130-8f78-9b5ed6f5cddc None admin] PATH_INFO: `/v3/projects/1e040b2fca724c58a2660441a72cd350/users/133a1ff64e224593be467eb7c27cbb09/roles/a211c6f9ebbc4889b636c26cd511e0e3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:26.738247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329afdd-f32e-4130-8f78-9b5ed6f5cddc None admin] RBAC: Authorizing `identity:create_grant(project_id=1e040b2fca724c58a2660441a72cd350, user_id=133a1ff64e224593be467eb7c27cbb09, role_id=a211c6f9ebbc4889b636c26cd511e0e3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:26.740050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329afdd-f32e-4130-8f78-9b5ed6f5cddc None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:26.749106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf45acf-ace3-4312-b386-754424141887 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:26.771564 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 300/605] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:33:26 2022] PUT /identity/v3/projects/1e040b2fca724c58a2660441a72cd350/users/133a1ff64e224593be467eb7c27cbb09/roles/a211c6f9ebbc4889b636c26cd511e0e3 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:33:26.808803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa811d97-040c-402b-a81e-8a407df1245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:26.809578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa811d97-040c-402b-a81e-8a407df1245f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:26.809864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa811d97-040c-402b-a81e-8a407df1245f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:26.810140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa811d97-040c-402b-a81e-8a407df1245f None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:26.810851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa811d97-040c-402b-a81e-8a407df1245f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:26.811160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa811d97-040c-402b-a81e-8a407df1245f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:26.812691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa811d97-040c-402b-a81e-8a407df1245f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:26.834607 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 306/606] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:26.840369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329afdd-f32e-4130-8f78-9b5ed6f5cddc None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:26.845105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5459ab2-5489-4597-8d6b-f2eeb1da01da None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:26.845394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5459ab2-5489-4597-8d6b-f2eeb1da01da None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:26.845676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5459ab2-5489-4597-8d6b-f2eeb1da01da None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:26.847249 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 307/607] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:33:26.852251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d5cc5-ad5f-4772-9c11-864c8c6b72c1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:26.852606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d5cc5-ad5f-4772-9c11-864c8c6b72c1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:26.852977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d5cc5-ad5f-4772-9c11-864c8c6b72c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:26.884558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f95d5cc5-ad5f-4772-9c11-864c8c6b72c1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:26.886047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f95d5cc5-ad5f-4772-9c11-864c8c6b72c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:27.011751 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 308/608] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:33:27.027033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329afdd-f32e-4130-8f78-9b5ed6f5cddc None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:27.055226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:27.064757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c10d6c-de88-430f-856c-f6757e46bc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:27.065635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c10d6c-de88-430f-856c-f6757e46bc15 None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:27.066076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c10d6c-de88-430f-856c-f6757e46bc15 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:27.066510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c10d6c-de88-430f-856c-f6757e46bc15 None admin] PATH_INFO: `/v3/projects/1e040b2fca724c58a2660441a72cd350/users/c0fb67db342b477aaf547543c498434f/roles/a211c6f9ebbc4889b636c26cd511e0e3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:27.082749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c10d6c-de88-430f-856c-f6757e46bc15 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e040b2fca724c58a2660441a72cd350, user_id=c0fb67db342b477aaf547543c498434f, role_id=a211c6f9ebbc4889b636c26cd511e0e3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:27.084403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c10d6c-de88-430f-856c-f6757e46bc15 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:27.099215 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 301/609] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:33:26 2022] PUT /identity/v3/projects/1e040b2fca724c58a2660441a72cd350/users/c0fb67db342b477aaf547543c498434f/roles/a211c6f9ebbc4889b636c26cd511e0e3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:33:27.443209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d5cc5-ad5f-4772-9c11-864c8c6b72c1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:27.460263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:27.565122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d5cc5-ad5f-4772-9c11-864c8c6b72c1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:27.614968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1bb89-8ecf-4e3b-b743-a7d1e7af556b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:27.615614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1bb89-8ecf-4e3b-b743-a7d1e7af556b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:27.615903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1bb89-8ecf-4e3b-b743-a7d1e7af556b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:27.616182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1bb89-8ecf-4e3b-b743-a7d1e7af556b None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:27.626769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1bb89-8ecf-4e3b-b743-a7d1e7af556b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-7ab1bb89-8ecf-4e3b-b743-a7d1e7af556b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:27.631544 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:27.632956 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:27.633781 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 309/610] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:27.643514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c10d6c-de88-430f-856c-f6757e46bc15 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:27.785864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c10d6c-de88-430f-856c-f6757e46bc15 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:27.823246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bace09-8525-4b91-964b-cd046dc7f38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:27.823859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bace09-8525-4b91-964b-cd046dc7f38a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:27.824144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bace09-8525-4b91-964b-cd046dc7f38a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:27.824423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bace09-8525-4b91-964b-cd046dc7f38a None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:27.825196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bace09-8525-4b91-964b-cd046dc7f38a None admin] RBAC: Adding query filter params (name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:27.825506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bace09-8525-4b91-964b-cd046dc7f38a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:27.826885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bace09-8525-4b91-964b-cd046dc7f38a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:27.836395 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 302/611] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:27.841813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1bb89-8ecf-4e3b-b743-a7d1e7af556b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:27.939218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:27.975682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1bb89-8ecf-4e3b-b743-a7d1e7af556b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:27.980556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d042f97-fd5e-484d-98c6-b416c8052214 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:27.981045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d042f97-fd5e-484d-98c6-b416c8052214 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:27.981120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d042f97-fd5e-484d-98c6-b416c8052214 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:27.982546 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 303/612] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:33:27.988742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dfcbb-95be-462c-b847-4c355be0cc28 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:27.989213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dfcbb-95be-462c-b847-4c355be0cc28 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:27.989501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dfcbb-95be-462c-b847-4c355be0cc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:28.024713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-679dfcbb-95be-462c-b847-4c355be0cc28 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:28.026247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-679dfcbb-95be-462c-b847-4c355be0cc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:28.032327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62528b5-b82d-4ef0-bed4-cc0a81094013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:28.033187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62528b5-b82d-4ef0-bed4-cc0a81094013 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:28.033586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62528b5-b82d-4ef0-bed4-cc0a81094013 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:28.033998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62528b5-b82d-4ef0-bed4-cc0a81094013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:28.053203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62528b5-b82d-4ef0-bed4-cc0a81094013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:28.054623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62528b5-b82d-4ef0-bed4-cc0a81094013 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-d62528b5-b82d-4ef0-bed4-cc0a81094013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:28.083366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:28.084347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:28.085258 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:28.085258 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:28.085258 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:28.085258 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 310/613] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:28.090220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62528b5-b82d-4ef0-bed4-cc0a81094013 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:28.113259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:28.171392 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 304/614] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:28.256889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62528b5-b82d-4ef0-bed4-cc0a81094013 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:28.292083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:28.307242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc91ecb-c009-487d-89a1-227e105c7c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:28.307793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc91ecb-c009-487d-89a1-227e105c7c7c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:28.308075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc91ecb-c009-487d-89a1-227e105c7c7c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:28.308353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc91ecb-c009-487d-89a1-227e105c7c7c None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:28.309020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc91ecb-c009-487d-89a1-227e105c7c7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:28.309327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc91ecb-c009-487d-89a1-227e105c7c7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:28.310703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc91ecb-c009-487d-89a1-227e105c7c7c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:28.327314 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 311/615] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:28.338124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dfcbb-95be-462c-b847-4c355be0cc28 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:28.452080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dfcbb-95be-462c-b847-4c355be0cc28 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:28.487129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657e5d4-72f1-4559-a116-af0889c3e5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:28.487768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657e5d4-72f1-4559-a116-af0889c3e5f3 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:28.488058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657e5d4-72f1-4559-a116-af0889c3e5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:28.488343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657e5d4-72f1-4559-a116-af0889c3e5f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:28.495843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657e5d4-72f1-4559-a116-af0889c3e5f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:28.497342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657e5d4-72f1-4559-a116-af0889c3e5f3 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-b657e5d4-72f1-4559-a116-af0889c3e5f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:28.509133 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:28.510092 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:28.511327 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:28.511327 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:28.511327 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:28.511327 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:28.511327 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:28.511327 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:28.511327 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:28.511327 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:28.511327 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:33:28.511327 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:28.511327 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 305/616] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 3 20:33:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:28.515915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc91ecb-c009-487d-89a1-227e105c7c7c None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:28.624173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc91ecb-c009-487d-89a1-227e105c7c7c None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:28.660046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2b0d5-c5fb-4c5e-8b18-e6667802c20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:28.660663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2b0d5-c5fb-4c5e-8b18-e6667802c20f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:28.660980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2b0d5-c5fb-4c5e-8b18-e6667802c20f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:28.661264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2b0d5-c5fb-4c5e-8b18-e6667802c20f None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:28.661990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2b0d5-c5fb-4c5e-8b18-e6667802c20f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:28.662302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2b0d5-c5fb-4c5e-8b18-e6667802c20f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:28.664060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2b0d5-c5fb-4c5e-8b18-e6667802c20f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:28.683419 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 312/617] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 3 20:33:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:28.690058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657e5d4-72f1-4559-a116-af0889c3e5f3 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:28.701032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2b0d5-c5fb-4c5e-8b18-e6667802c20f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:28.754963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:28.767739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:28.819348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2b0d5-c5fb-4c5e-8b18-e6667802c20f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:28.872021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def832c2-4e8f-46dd-b5de-3997d163b472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:28.872664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def832c2-4e8f-46dd-b5de-3997d163b472 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:28.872861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657e5d4-72f1-4559-a116-af0889c3e5f3 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:28.872984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def832c2-4e8f-46dd-b5de-3997d163b472 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:28.873246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def832c2-4e8f-46dd-b5de-3997d163b472 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:28.885000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def832c2-4e8f-46dd-b5de-3997d163b472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-def832c2-4e8f-46dd-b5de-3997d163b472 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:28.890640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:28.892274 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:28.893315 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:28.893315 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:28.893315 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:28.893315 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:28.893315 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:28.893315 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:28.893315 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:28.893315 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:28.893315 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:28.893315 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:28.893315 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:28.893315 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:28.895893 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 313/618] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:28.901909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def832c2-4e8f-46dd-b5de-3997d163b472 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:28.927225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20095f-7800-43ea-a966-ac51092e513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:28.927835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20095f-7800-43ea-a966-ac51092e513f None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:28.928118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20095f-7800-43ea-a966-ac51092e513f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:28.928390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20095f-7800-43ea-a966-ac51092e513f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.007259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20095f-7800-43ea-a966-ac51092e513f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:29.008728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20095f-7800-43ea-a966-ac51092e513f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:29.038478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:29.039300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def832c2-4e8f-46dd-b5de-3997d163b472 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:29.041777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:29.088748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41e16a-ec0e-4cba-b09f-1a19b721aa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:29.089795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41e16a-ec0e-4cba-b09f-1a19b721aa63 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:29.090165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41e16a-ec0e-4cba-b09f-1a19b721aa63 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:29.090521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41e16a-ec0e-4cba-b09f-1a19b721aa63 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.091462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41e16a-ec0e-4cba-b09f-1a19b721aa63 None admin] RBAC: Adding query filter params (name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:29.091772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41e16a-ec0e-4cba-b09f-1a19b721aa63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:29.093402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41e16a-ec0e-4cba-b09f-1a19b721aa63 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:29.105015 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 314/619] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:29.106337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e429b-a224-4149-9bfd-af4af4622d2b None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:29.106633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e429b-a224-4149-9bfd-af4af4622d2b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:29.106916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e429b-a224-4149-9bfd-af4af4622d2b None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.108977 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 315/620] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:29.109745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e429b-a224-4149-9bfd-af4af4622d2b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:29.132764 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 306/621] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:33:29.134164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf8563-9ee2-48c4-a3e9-78a16432b8cd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:29.134464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf8563-9ee2-48c4-a3e9-78a16432b8cd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:29.134747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf8563-9ee2-48c4-a3e9-78a16432b8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.162772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3cdf8563-9ee2-48c4-a3e9-78a16432b8cd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:29.165246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3cdf8563-9ee2-48c4-a3e9-78a16432b8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:29.241976 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 307/622] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:29.241976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf8563-9ee2-48c4-a3e9-78a16432b8cd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:29.250392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e429b-a224-4149-9bfd-af4af4622d2b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:29.301302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:29.318200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cb9d1-575d-48e9-b784-c20054205d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:29.319282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cb9d1-575d-48e9-b784-c20054205d9a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:29.319752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cb9d1-575d-48e9-b784-c20054205d9a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:29.320205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cb9d1-575d-48e9-b784-c20054205d9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.339462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001cb9d1-575d-48e9-b784-c20054205d9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:29.341271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001cb9d1-575d-48e9-b784-c20054205d9a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:29.341466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf8563-9ee2-48c4-a3e9-78a16432b8cd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-001cb9d1-575d-48e9-b784-c20054205d9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:29.368011 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:29.369046 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:29.369915 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:29.369915 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:29.369915 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:29.371903 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 316/623] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:29.379073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cb9d1-575d-48e9-b784-c20054205d9a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:29.382538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240467b-6cd9-446b-9153-44fe599be795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:29.383932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240467b-6cd9-446b-9153-44fe599be795 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:29.383932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240467b-6cd9-446b-9153-44fe599be795 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:29.383932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240467b-6cd9-446b-9153-44fe599be795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.394563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240467b-6cd9-446b-9153-44fe599be795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:29.396343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240467b-6cd9-446b-9153-44fe599be795 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:29.406793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a240467b-6cd9-446b-9153-44fe599be795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:29.411783 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:29.413113 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:29.414286 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:29.414286 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:29.414286 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:29.414286 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 308/624] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:29.425902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240467b-6cd9-446b-9153-44fe599be795 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:29.506066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cb9d1-575d-48e9-b784-c20054205d9a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:29.542832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0742b9c-4f88-489f-8b96-78db7d11263e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:29.544274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0742b9c-4f88-489f-8b96-78db7d11263e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:29.544858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0742b9c-4f88-489f-8b96-78db7d11263e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:29.545371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0742b9c-4f88-489f-8b96-78db7d11263e None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.546379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0742b9c-4f88-489f-8b96-78db7d11263e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:29.546916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0742b9c-4f88-489f-8b96-78db7d11263e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:29.548530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0742b9c-4f88-489f-8b96-78db7d11263e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:29.561766 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 317/625] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:29.568435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0742b9c-4f88-489f-8b96-78db7d11263e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:29.642952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240467b-6cd9-446b-9153-44fe599be795 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:29.656486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:29.681248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8773e01-ba2d-4d7f-b0ee-38f62e930472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:29.681874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8773e01-ba2d-4d7f-b0ee-38f62e930472 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:29.682168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8773e01-ba2d-4d7f-b0ee-38f62e930472 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:29.682442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8773e01-ba2d-4d7f-b0ee-38f62e930472 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.683619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8773e01-ba2d-4d7f-b0ee-38f62e930472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:29.684130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8773e01-ba2d-4d7f-b0ee-38f62e930472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:29.685551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8773e01-ba2d-4d7f-b0ee-38f62e930472 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:29.704514 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 309/626] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:29.704514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9600f2-ac83-4c3d-b671-e83ba5e6f309 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:29.704514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9600f2-ac83-4c3d-b671-e83ba5e6f309 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:29.704514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9600f2-ac83-4c3d-b671-e83ba5e6f309 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.706607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:29.708761 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 310/627] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:29.709652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9600f2-ac83-4c3d-b671-e83ba5e6f309 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:29.736999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0742b9c-4f88-489f-8b96-78db7d11263e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:29.774002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69470205-ee26-4fa7-9350-1ff665e6ac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:29.774924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69470205-ee26-4fa7-9350-1ff665e6ac87 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:29.775354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69470205-ee26-4fa7-9350-1ff665e6ac87 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:29.776212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69470205-ee26-4fa7-9350-1ff665e6ac87 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.785942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69470205-ee26-4fa7-9350-1ff665e6ac87 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:29.787394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69470205-ee26-4fa7-9350-1ff665e6ac87 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-69470205-ee26-4fa7-9350-1ff665e6ac87 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:29.799888 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:29.801226 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 318/628] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975af2-f6d1-4d00-b53f-3ba52ba744db None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:29.803245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975af2-f6d1-4d00-b53f-3ba52ba744db None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:29.805632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975af2-f6d1-4d00-b53f-3ba52ba744db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.820621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f2975af2-f6d1-4d00-b53f-3ba52ba744db None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:29.821874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f2975af2-f6d1-4d00-b53f-3ba52ba744db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:29.826511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9600f2-ac83-4c3d-b671-e83ba5e6f309 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:29.862435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339fab5-eb7a-4dc2-b028-a553551e798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:29.863075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339fab5-eb7a-4dc2-b028-a553551e798a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:29.863364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339fab5-eb7a-4dc2-b028-a553551e798a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:29.863646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339fab5-eb7a-4dc2-b028-a553551e798a None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.870371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339fab5-eb7a-4dc2-b028-a553551e798a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:29.871817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339fab5-eb7a-4dc2-b028-a553551e798a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-0339fab5-eb7a-4dc2-b028-a553551e798a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:29.882874 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:29.883729 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:29.884595 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:29.884595 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:29.884595 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:29.884595 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:29.884595 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:29.884595 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:29.884595 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:29.884595 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:29.884595 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 311/629] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:29.885118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339fab5-eb7a-4dc2-b028-a553551e798a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:29.894501 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 319/630] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:29.895861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61417886-e0ac-4022-8953-8508d9d443bf None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:29.896103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61417886-e0ac-4022-8953-8508d9d443bf None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:29.896385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61417886-e0ac-4022-8953-8508d9d443bf None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:29.897632 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 320/631] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:29.898329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61417886-e0ac-4022-8953-8508d9d443bf None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:29.928527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:29.974033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:30.002856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339fab5-eb7a-4dc2-b028-a553551e798a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:30.010554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61417886-e0ac-4022-8953-8508d9d443bf None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:30.039511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510436ea-6c24-46e4-a0a2-f7999a7dcd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:30.040139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510436ea-6c24-46e4-a0a2-f7999a7dcd06 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:30.040426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510436ea-6c24-46e4-a0a2-f7999a7dcd06 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:30.040703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510436ea-6c24-46e4-a0a2-f7999a7dcd06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:30.050794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb87a1-1bb5-4d0d-a825-9b8e3c63e9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:30.051402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb87a1-1bb5-4d0d-a825-9b8e3c63e9d4 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:30.051690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb87a1-1bb5-4d0d-a825-9b8e3c63e9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:30.051970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb87a1-1bb5-4d0d-a825-9b8e3c63e9d4 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:30.052131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510436ea-6c24-46e4-a0a2-f7999a7dcd06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:30.052698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb87a1-1bb5-4d0d-a825-9b8e3c63e9d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:30.053041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb87a1-1bb5-4d0d-a825-9b8e3c63e9d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:30.054498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb87a1-1bb5-4d0d-a825-9b8e3c63e9d4 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-510436ea-6c24-46e4-a0a2-f7999a7dcd06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:30.057579 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:30.058470 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:30.059287 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 312/632] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:30.059965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510436ea-6c24-46e4-a0a2-f7999a7dcd06 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:30.077111 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 321/633] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Sep 3 20:33:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:30.077111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd8af-b720-4013-bc25-a75ffbe44599 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:30.077111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd8af-b720-4013-bc25-a75ffbe44599 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:30.077111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd8af-b720-4013-bc25-a75ffbe44599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:30.097122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b4bbd8af-b720-4013-bc25-a75ffbe44599 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:30.098394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b4bbd8af-b720-4013-bc25-a75ffbe44599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:30.178872 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 322/634] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:30.179651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbd8af-b720-4013-bc25-a75ffbe44599 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:30.193613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:30.203525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510436ea-6c24-46e4-a0a2-f7999a7dcd06 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:30.245051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464d38e-f995-414d-a9d0-b0c29111bb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:30.245660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464d38e-f995-414d-a9d0-b0c29111bb97 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:30.245947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464d38e-f995-414d-a9d0-b0c29111bb97 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:30.246221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464d38e-f995-414d-a9d0-b0c29111bb97 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:30.246903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464d38e-f995-414d-a9d0-b0c29111bb97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:30.247205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464d38e-f995-414d-a9d0-b0c29111bb97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:30.248740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464d38e-f995-414d-a9d0-b0c29111bb97 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:30.250015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:30.265034 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 313/635] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:30.265034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464d38e-f995-414d-a9d0-b0c29111bb97 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:30.302221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbd8af-b720-4013-bc25-a75ffbe44599 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:30.339316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24094ba-bbf4-42ef-a9d9-0e31b9c7cdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:30.339935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24094ba-bbf4-42ef-a9d9-0e31b9c7cdfe None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:30.340221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24094ba-bbf4-42ef-a9d9-0e31b9c7cdfe None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:30.340502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24094ba-bbf4-42ef-a9d9-0e31b9c7cdfe None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:30.341213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24094ba-bbf4-42ef-a9d9-0e31b9c7cdfe None admin] RBAC: Adding query filter params (name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:30.341523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24094ba-bbf4-42ef-a9d9-0e31b9c7cdfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:30.342916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24094ba-bbf4-42ef-a9d9-0e31b9c7cdfe None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:30.356563 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 323/636] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:30.356563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24094ba-bbf4-42ef-a9d9-0e31b9c7cdfe None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:30.372693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464d38e-f995-414d-a9d0-b0c29111bb97 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:30.410608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d7bd2-f02d-4a22-9f06-60f9badcadd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:30.411390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d7bd2-f02d-4a22-9f06-60f9badcadd7 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:30.411674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d7bd2-f02d-4a22-9f06-60f9badcadd7 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:30.411947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d7bd2-f02d-4a22-9f06-60f9badcadd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:30.421881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d7bd2-f02d-4a22-9f06-60f9badcadd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:30.423358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d7bd2-f02d-4a22-9f06-60f9badcadd7 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-228d7bd2-f02d-4a22-9f06-60f9badcadd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:30.439784 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:30.440921 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:30.441879 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:30.441879 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:30.441879 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:30.441879 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 314/637] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:30.441879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d7bd2-f02d-4a22-9f06-60f9badcadd7 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:30.447669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:30.469406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24094ba-bbf4-42ef-a9d9-0e31b9c7cdfe None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:30.487739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:30.506934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d35f11-e669-4248-94d5-afb715b0ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:30.507759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d35f11-e669-4248-94d5-afb715b0ca1f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:30.508051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d35f11-e669-4248-94d5-afb715b0ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:30.508330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d35f11-e669-4248-94d5-afb715b0ca1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:30.519783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d35f11-e669-4248-94d5-afb715b0ca1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-64d35f11-e669-4248-94d5-afb715b0ca1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:30.530024 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:30.531167 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:30.532917 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 324/638] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:30.533915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d35f11-e669-4248-94d5-afb715b0ca1f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:30.551911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d7bd2-f02d-4a22-9f06-60f9badcadd7 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:30.584396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac4a5c-2032-4dfc-9df9-2b4ec1c7fa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:30.585027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac4a5c-2032-4dfc-9df9-2b4ec1c7fa3c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:30.585308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac4a5c-2032-4dfc-9df9-2b4ec1c7fa3c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:30.585579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac4a5c-2032-4dfc-9df9-2b4ec1c7fa3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:30.586293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac4a5c-2032-4dfc-9df9-2b4ec1c7fa3c None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=a56702329eae48819121dcb58f7e3070) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:30.586601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac4a5c-2032-4dfc-9df9-2b4ec1c7fa3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:30.588059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac4a5c-2032-4dfc-9df9-2b4ec1c7fa3c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:30.593855 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 315/639] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:30 2022] GET /identity/v3/role_assignments?user.id=a56702329eae48819121dcb58f7e3070&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:30.595156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac4a5c-2032-4dfc-9df9-2b4ec1c7fa3c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:30.640575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d35f11-e669-4248-94d5-afb715b0ca1f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:30.667738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:30.682352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c9f4d-a296-4c61-a9e9-eadcf7b93e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:30.683003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c9f4d-a296-4c61-a9e9-eadcf7b93e7f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:30.683287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c9f4d-a296-4c61-a9e9-eadcf7b93e7f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:30.683561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c9f4d-a296-4c61-a9e9-eadcf7b93e7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:30.695933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c9f4d-a296-4c61-a9e9-eadcf7b93e7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:30.697773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c9f4d-a296-4c61-a9e9-eadcf7b93e7f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:30.706776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac4a5c-2032-4dfc-9df9-2b4ec1c7fa3c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-a73c9f4d-a296-4c61-a9e9-eadcf7b93e7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:30.714770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:30.715802 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:30.716671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:30.716671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:30.716671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:30.721259 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 325/640] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:30.722224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c9f4d-a296-4c61-a9e9-eadcf7b93e7f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:30.749917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d879802-7d19-4c5e-8a39-b7559e3ff44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:30.750545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879802-7d19-4c5e-8a39-b7559e3ff44a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:30.750870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879802-7d19-4c5e-8a39-b7559e3ff44a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:30.751719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879802-7d19-4c5e-8a39-b7559e3ff44a None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:30.752330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d879802-7d19-4c5e-8a39-b7559e3ff44a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:30.752639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d879802-7d19-4c5e-8a39-b7559e3ff44a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:30.754207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d879802-7d19-4c5e-8a39-b7559e3ff44a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:30.757800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:30.771576 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 316/641] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:30.771576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d879802-7d19-4c5e-8a39-b7559e3ff44a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:30.840125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c9f4d-a296-4c61-a9e9-eadcf7b93e7f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:30.878348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaaa90-ed86-4c2d-9e39-143d4e97ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:30.878964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaaa90-ed86-4c2d-9e39-143d4e97ef35 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:30.879249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaaa90-ed86-4c2d-9e39-143d4e97ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:30.879528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaaa90-ed86-4c2d-9e39-143d4e97ef35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:30.890436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facaaa90-ed86-4c2d-9e39-143d4e97ef35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:30.890983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d879802-7d19-4c5e-8a39-b7559e3ff44a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-facaaa90-ed86-4c2d-9e39-143d4e97ef35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:30.895825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:30.896789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:30.897662 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 326/642] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:30.898347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaaa90-ed86-4c2d-9e39-143d4e97ef35 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:30.919121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:30.928915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ca771-1e25-4ee4-b8ea-c75d45568d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:30.929530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ca771-1e25-4ee4-b8ea-c75d45568d65 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:30.929820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ca771-1e25-4ee4-b8ea-c75d45568d65 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:30.930100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ca771-1e25-4ee4-b8ea-c75d45568d65 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:30.930772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ca771-1e25-4ee4-b8ea-c75d45568d65 None admin] RBAC: Adding query filter params (name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:30.931083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ca771-1e25-4ee4-b8ea-c75d45568d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:30.932710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ca771-1e25-4ee4-b8ea-c75d45568d65 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:30.941507 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 317/643] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:30.942198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ca771-1e25-4ee4-b8ea-c75d45568d65 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:31.014485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:31.026794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaaa90-ed86-4c2d-9e39-143d4e97ef35 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:31.050063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ca771-1e25-4ee4-b8ea-c75d45568d65 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:31.062595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8964491-4744-4f80-9b26-4e66bafb3d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:31.063206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8964491-4744-4f80-9b26-4e66bafb3d81 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:31.063521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8964491-4744-4f80-9b26-4e66bafb3d81 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:31.063800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8964491-4744-4f80-9b26-4e66bafb3d81 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:31.064447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8964491-4744-4f80-9b26-4e66bafb3d81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:31.064747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8964491-4744-4f80-9b26-4e66bafb3d81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:31.066145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8964491-4744-4f80-9b26-4e66bafb3d81 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:31.078542 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 327/644] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:31.079381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8964491-4744-4f80-9b26-4e66bafb3d81 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:31.093434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c1a81-c484-4606-800b-ccdea02eff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:31.094346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c1a81-c484-4606-800b-ccdea02eff56 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:31.095179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c1a81-c484-4606-800b-ccdea02eff56 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:31.095794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c1a81-c484-4606-800b-ccdea02eff56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:31.102737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c1a81-c484-4606-800b-ccdea02eff56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:31.104198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c1a81-c484-4606-800b-ccdea02eff56 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-e61c1a81-c484-4606-800b-ccdea02eff56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:31.115937 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:31.116828 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:31.117922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:31.117922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:31.117922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:31.117922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:31.117922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:31.117922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:31.117922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:31.117922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:31.121773 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 318/645] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:31.122680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c1a81-c484-4606-800b-ccdea02eff56 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:31.178191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:31.203144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8964491-4744-4f80-9b26-4e66bafb3d81 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:31.231728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c1a81-c484-4606-800b-ccdea02eff56 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:31.241084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04035540-bc3f-42e3-ac05-f2d8d5d6d91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:31.241698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04035540-bc3f-42e3-ac05-f2d8d5d6d91b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:31.241992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04035540-bc3f-42e3-ac05-f2d8d5d6d91b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:31.242277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04035540-bc3f-42e3-ac05-f2d8d5d6d91b None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:31.242948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04035540-bc3f-42e3-ac05-f2d8d5d6d91b None admin] RBAC: Adding query filter params (name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:31.243259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04035540-bc3f-42e3-ac05-f2d8d5d6d91b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:31.244623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04035540-bc3f-42e3-ac05-f2d8d5d6d91b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:31.257274 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 328/646] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:31.257274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04035540-bc3f-42e3-ac05-f2d8d5d6d91b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:31.272412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:31.274734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0563ad1-3e92-496f-8a7b-d858ba9f72b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:31.275379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0563ad1-3e92-496f-8a7b-d858ba9f72b8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:31.275695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0563ad1-3e92-496f-8a7b-d858ba9f72b8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:31.276038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0563ad1-3e92-496f-8a7b-d858ba9f72b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:31.287838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0563ad1-3e92-496f-8a7b-d858ba9f72b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:31.289321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0563ad1-3e92-496f-8a7b-d858ba9f72b8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a0563ad1-3e92-496f-8a7b-d858ba9f72b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:31.304322 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:31.305373 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:31.306451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:31.306451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:31.306451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:31.309844 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 319/647] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:31.310676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0563ad1-3e92-496f-8a7b-d858ba9f72b8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:31.369727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04035540-bc3f-42e3-ac05-f2d8d5d6d91b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:31.407803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bb977-2ef8-421f-abd8-abacf5fea793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:31.408952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bb977-2ef8-421f-abd8-abacf5fea793 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:31.409474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bb977-2ef8-421f-abd8-abacf5fea793 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:31.409993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bb977-2ef8-421f-abd8-abacf5fea793 None admin] PATH_INFO: `/v3/users/glance` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:31.412307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0563ad1-3e92-496f-8a7b-d858ba9f72b8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:31.413868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:31.420051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bb977-2ef8-421f-abd8-abacf5fea793 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:31.421558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bb977-2ef8-421f-abd8-abacf5fea793 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-eb2bb977-2ef8-421f-abd8-abacf5fea793 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:31.433411 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:31.434323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:31.435251 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:31.435251 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:31.435251 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:31.435251 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:31.435251 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:31.435251 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:31.435251 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:31.435251 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:31.435251 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 03 20:33:31.435251 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:31.435251 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 329/648] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:31.435251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bb977-2ef8-421f-abd8-abacf5fea793 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:31.453494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a1fdd-f56b-48ec-9085-9ff05990667b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:31.454197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a1fdd-f56b-48ec-9085-9ff05990667b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:31.454517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a1fdd-f56b-48ec-9085-9ff05990667b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:31.454809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a1fdd-f56b-48ec-9085-9ff05990667b None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:31.455511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a1fdd-f56b-48ec-9085-9ff05990667b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:31.455824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a1fdd-f56b-48ec-9085-9ff05990667b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:31.457564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a1fdd-f56b-48ec-9085-9ff05990667b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:31.469018 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 320/649] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:31.469854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a1fdd-f56b-48ec-9085-9ff05990667b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:31.513832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:31.547539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bb977-2ef8-421f-abd8-abacf5fea793 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:31.574106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a1fdd-f56b-48ec-9085-9ff05990667b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:31.584469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168e259-dd2a-4a3d-8ba9-ca99596c554e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:31.585139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168e259-dd2a-4a3d-8ba9-ca99596c554e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:31.585429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168e259-dd2a-4a3d-8ba9-ca99596c554e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:31.585709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168e259-dd2a-4a3d-8ba9-ca99596c554e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:31.596588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168e259-dd2a-4a3d-8ba9-ca99596c554e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:31.598077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168e259-dd2a-4a3d-8ba9-ca99596c554e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:31.609040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bbb0d-1291-427d-aca1-d90b4bbef399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:31.609642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bbb0d-1291-427d-aca1-d90b4bbef399 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:31.609924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bbb0d-1291-427d-aca1-d90b4bbef399 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:31.610693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bbb0d-1291-427d-aca1-d90b4bbef399 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:31.611341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bbb0d-1291-427d-aca1-d90b4bbef399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:31.611644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bbb0d-1291-427d-aca1-d90b4bbef399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-2168e259-dd2a-4a3d-8ba9-ca99596c554e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:31.612729 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:31.613691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:31.619585 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:31.619585 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:31.619585 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:31.619796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bbb0d-1291-427d-aca1-d90b4bbef399 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:31.619891 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 330/650] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:31.619891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168e259-dd2a-4a3d-8ba9-ca99596c554e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:31.632098 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 321/651] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:31.632912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bbb0d-1291-427d-aca1-d90b4bbef399 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:31.658139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:31.735133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168e259-dd2a-4a3d-8ba9-ca99596c554e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:31.757727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bbb0d-1291-427d-aca1-d90b4bbef399 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:31.760703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:31.775135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288db53-3599-4beb-836a-d56c5bfd67fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:31.776235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288db53-3599-4beb-836a-d56c5bfd67fd None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:31.776858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288db53-3599-4beb-836a-d56c5bfd67fd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:31.777300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288db53-3599-4beb-836a-d56c5bfd67fd None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:31.778701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288db53-3599-4beb-836a-d56c5bfd67fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:31.779286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288db53-3599-4beb-836a-d56c5bfd67fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:31.782582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288db53-3599-4beb-836a-d56c5bfd67fd None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:31.796242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b29dc9-04a7-4d63-b6f3-a2a643103fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:31.797100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b29dc9-04a7-4d63-b6f3-a2a643103fa2 None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:31.797393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b29dc9-04a7-4d63-b6f3-a2a643103fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:31.797789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b29dc9-04a7-4d63-b6f3-a2a643103fa2 None admin] PATH_INFO: `/v3/projects/1e040b2fca724c58a2660441a72cd350/users/83865463b2a84f99a136b73dffeeb378/roles/a211c6f9ebbc4889b636c26cd511e0e3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:31.802568 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 331/652] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Sep 3 20:33:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:31.802959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288db53-3599-4beb-836a-d56c5bfd67fd None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:31.818310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b29dc9-04a7-4d63-b6f3-a2a643103fa2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e040b2fca724c58a2660441a72cd350, user_id=83865463b2a84f99a136b73dffeeb378, role_id=a211c6f9ebbc4889b636c26cd511e0e3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:31.819781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b29dc9-04a7-4d63-b6f3-a2a643103fa2 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:31.841108 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 322/653] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:33:31 2022] PUT /identity/v3/projects/1e040b2fca724c58a2660441a72cd350/users/83865463b2a84f99a136b73dffeeb378/roles/a211c6f9ebbc4889b636c26cd511e0e3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:33:31.841974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b29dc9-04a7-4d63-b6f3-a2a643103fa2 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:31.939222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288db53-3599-4beb-836a-d56c5bfd67fd None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:31.951837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:31.984255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482d346-c451-4858-9f78-99b007067d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:31.985301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482d346-c451-4858-9f78-99b007067d8f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:31.986215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482d346-c451-4858-9f78-99b007067d8f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:31.986651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482d346-c451-4858-9f78-99b007067d8f None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:31.987367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482d346-c451-4858-9f78-99b007067d8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:31.987783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482d346-c451-4858-9f78-99b007067d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:31.989653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482d346-c451-4858-9f78-99b007067d8f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:32.000969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b29dc9-04a7-4d63-b6f3-a2a643103fa2 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:32.008760 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 332/654] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:32.009971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482d346-c451-4858-9f78-99b007067d8f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:32.051971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b526e70-1cf6-4383-86a2-3bf3a2fb687b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:32.053371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b526e70-1cf6-4383-86a2-3bf3a2fb687b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:32.053663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b526e70-1cf6-4383-86a2-3bf3a2fb687b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:32.053950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b526e70-1cf6-4383-86a2-3bf3a2fb687b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:32.063126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:32.064456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b526e70-1cf6-4383-86a2-3bf3a2fb687b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:32.066294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b526e70-1cf6-4383-86a2-3bf3a2fb687b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-6b526e70-1cf6-4383-86a2-3bf3a2fb687b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:32.081277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:32.082506 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:32.083453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:32.083453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:32.083453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:32.083453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:32.083453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:32.083453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:32.083453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:32.083453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:32.083453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 20:33:32.083453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:32.085502 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 323/655] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:32.086520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b526e70-1cf6-4383-86a2-3bf3a2fb687b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:32.150033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482d346-c451-4858-9f78-99b007067d8f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:32.192618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403363a3-d059-4d42-88ee-afa2733cc33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:32.193383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403363a3-d059-4d42-88ee-afa2733cc33a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:32.193747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403363a3-d059-4d42-88ee-afa2733cc33a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:32.194098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403363a3-d059-4d42-88ee-afa2733cc33a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:32.217991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b526e70-1cf6-4383-86a2-3bf3a2fb687b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:32.257167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:32.264136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4071962-5109-43ff-a370-136c3ddb6824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:32.265133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4071962-5109-43ff-a370-136c3ddb6824 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:32.265522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4071962-5109-43ff-a370-136c3ddb6824 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:32.265901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4071962-5109-43ff-a370-136c3ddb6824 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:32.274968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4071962-5109-43ff-a370-136c3ddb6824 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:32.275478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403363a3-d059-4d42-88ee-afa2733cc33a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:32.276699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4071962-5109-43ff-a370-136c3ddb6824 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:32.277279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403363a3-d059-4d42-88ee-afa2733cc33a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-d4071962-5109-43ff-a370-136c3ddb6824 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:32.289851 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:32.290812 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:32.291739 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:32.291739 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:32.291739 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:32.291739 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:32.291739 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:32.291739 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:32.291739 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:32.291739 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:32.291739 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 20:33:32.291739 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:32.299321 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 324/656] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:32.299321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4071962-5109-43ff-a370-136c3ddb6824 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:32.330652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:32.410884 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 333/657] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:33:32.412379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403363a3-d059-4d42-88ee-afa2733cc33a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:32.425286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4071962-5109-43ff-a370-136c3ddb6824 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:32.467594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eac91a-aa81-479e-8e7a-2232a093829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:32.468219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eac91a-aa81-479e-8e7a-2232a093829b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:32.468502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eac91a-aa81-479e-8e7a-2232a093829b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:32.468773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eac91a-aa81-479e-8e7a-2232a093829b None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:32.469537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eac91a-aa81-479e-8e7a-2232a093829b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:32.470368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eac91a-aa81-479e-8e7a-2232a093829b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:32.472207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eac91a-aa81-479e-8e7a-2232a093829b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:32.495694 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 325/658] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Sep 3 20:33:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:32.497278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eac91a-aa81-479e-8e7a-2232a093829b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:32.553499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403363a3-d059-4d42-88ee-afa2733cc33a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:32.562279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:32.611842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645f985-37a6-4091-b56e-1ca41f2018b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:32.613092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645f985-37a6-4091-b56e-1ca41f2018b2 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:32.613992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645f985-37a6-4091-b56e-1ca41f2018b2 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:32.614743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645f985-37a6-4091-b56e-1ca41f2018b2 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:32.615977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645f985-37a6-4091-b56e-1ca41f2018b2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:32.616867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645f985-37a6-4091-b56e-1ca41f2018b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:32.619070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645f985-37a6-4091-b56e-1ca41f2018b2 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:32.623472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eac91a-aa81-479e-8e7a-2232a093829b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:32.638008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:32.648938 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 334/659] 10.222.0.16 () {56 vars in 1139 bytes} [Sat Sep 3 20:33:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:32.648938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645f985-37a6-4091-b56e-1ca41f2018b2 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:32.672691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b9b6f-4d0f-4f44-94fb-1e42c51edd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:32.674717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b9b6f-4d0f-4f44-94fb-1e42c51edd6b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:32.675410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b9b6f-4d0f-4f44-94fb-1e42c51edd6b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:32.676119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b9b6f-4d0f-4f44-94fb-1e42c51edd6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:32.688363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b9b6f-4d0f-4f44-94fb-1e42c51edd6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:32.690163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b9b6f-4d0f-4f44-94fb-1e42c51edd6b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-bf8b9b6f-4d0f-4f44-94fb-1e42c51edd6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:32.706732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:32.707949 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:32.708869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:32.708869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:32.708869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:32.710551 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 326/660] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:32.711553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b9b6f-4d0f-4f44-94fb-1e42c51edd6b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:32.754359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645f985-37a6-4091-b56e-1ca41f2018b2 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:32.790397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3660e9-6f5a-41c2-bdc4-db97d01bc5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:32.790999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3660e9-6f5a-41c2-bdc4-db97d01bc5d8 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:32.791283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3660e9-6f5a-41c2-bdc4-db97d01bc5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:32.791558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3660e9-6f5a-41c2-bdc4-db97d01bc5d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:32.827490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b9b6f-4d0f-4f44-94fb-1e42c51edd6b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:32.829445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:32.860064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:32.864728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3660e9-6f5a-41c2-bdc4-db97d01bc5d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:32.866162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3660e9-6f5a-41c2-bdc4-db97d01bc5d8 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:32.878082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec7a0e-aa06-42f1-a4bb-8f83aac62378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:32.879735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec7a0e-aa06-42f1-a4bb-8f83aac62378 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:32.880366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec7a0e-aa06-42f1-a4bb-8f83aac62378 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:32.880857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec7a0e-aa06-42f1-a4bb-8f83aac62378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:32.960361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec7a0e-aa06-42f1-a4bb-8f83aac62378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:32.961856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec7a0e-aa06-42f1-a4bb-8f83aac62378 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:32.985055 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 335/661] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:33:32.986234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3660e9-6f5a-41c2-bdc4-db97d01bc5d8 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:33.075361 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 327/662] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:33:33.075361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812de32-8f5f-4756-9f00-8bfbbf97cc73 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:33.075361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812de32-8f5f-4756-9f00-8bfbbf97cc73 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:33.075361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812de32-8f5f-4756-9f00-8bfbbf97cc73 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:33.078152 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 328/663] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:33.079449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812de32-8f5f-4756-9f00-8bfbbf97cc73 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:33.082394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:33.115818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:33.124130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3660e9-6f5a-41c2-bdc4-db97d01bc5d8 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:33.162061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aeee02-1450-4a1a-a1f9-9f9247cf37c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:33.162757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aeee02-1450-4a1a-a1f9-9f9247cf37c9 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:33.163116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aeee02-1450-4a1a-a1f9-9f9247cf37c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:33.163469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aeee02-1450-4a1a-a1f9-9f9247cf37c9 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:33.164183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aeee02-1450-4a1a-a1f9-9f9247cf37c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:33.164502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aeee02-1450-4a1a-a1f9-9f9247cf37c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:33.165919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aeee02-1450-4a1a-a1f9-9f9247cf37c9 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:33.181732 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 336/664] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:33.181732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aeee02-1450-4a1a-a1f9-9f9247cf37c9 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:33.195854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812de32-8f5f-4756-9f00-8bfbbf97cc73 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:33.233226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66461c8-3d6f-49ca-a145-0acd0e0e670a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:33.234175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66461c8-3d6f-49ca-a145-0acd0e0e670a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:33.234542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66461c8-3d6f-49ca-a145-0acd0e0e670a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:33.234922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66461c8-3d6f-49ca-a145-0acd0e0e670a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:33.246515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66461c8-3d6f-49ca-a145-0acd0e0e670a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:33.248055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66461c8-3d6f-49ca-a145-0acd0e0e670a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a66461c8-3d6f-49ca-a145-0acd0e0e670a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:33.264269 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:33.265593 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:33.271987 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:33.271987 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:33.271987 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:33.271987 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 329/665] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:33.271987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcbe1a-9139-4e2b-83c9-26eed925e423 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:33.271987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcbe1a-9139-4e2b-83c9-26eed925e423 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:33.271987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcbe1a-9139-4e2b-83c9-26eed925e423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:33.288200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e4bcbe1a-9139-4e2b-83c9-26eed925e423 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:33.289434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e4bcbe1a-9139-4e2b-83c9-26eed925e423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:33.293299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aeee02-1450-4a1a-a1f9-9f9247cf37c9 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:33.326320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d68909-e913-4f94-a658-1d844c07d2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:33.326953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d68909-e913-4f94-a658-1d844c07d2d5 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:33.327245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d68909-e913-4f94-a658-1d844c07d2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:33.327525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d68909-e913-4f94-a658-1d844c07d2d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:33.339482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d68909-e913-4f94-a658-1d844c07d2d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:33.340960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d68909-e913-4f94-a658-1d844c07d2d5 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:33.344578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-d3d68909-e913-4f94-a658-1d844c07d2d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:33.355644 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:33.356605 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:33.357766 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:33.357766 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:33.357766 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:33.357766 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 337/666] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:33.357766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d68909-e913-4f94-a658-1d844c07d2d5 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:33.368803 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 330/667] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:33.370024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcbe1a-9139-4e2b-83c9-26eed925e423 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:33.383703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:33.480141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d68909-e913-4f94-a658-1d844c07d2d5 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:33.489200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcbe1a-9139-4e2b-83c9-26eed925e423 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:33.532268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4383f8a-a0ad-44fe-9856-fffda752660d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:33.533049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4383f8a-a0ad-44fe-9856-fffda752660d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:33.533411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4383f8a-a0ad-44fe-9856-fffda752660d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:33.533821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4383f8a-a0ad-44fe-9856-fffda752660d None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:33.534463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cefe9-933c-4175-ac70-2e37d189fcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:33.535284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cefe9-933c-4175-ac70-2e37d189fcc7 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:33.535754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cefe9-933c-4175-ac70-2e37d189fcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:33.536328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cefe9-933c-4175-ac70-2e37d189fcc7 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:33.537508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cefe9-933c-4175-ac70-2e37d189fcc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:33.537979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cefe9-933c-4175-ac70-2e37d189fcc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:33.539520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cefe9-933c-4175-ac70-2e37d189fcc7 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:33.541867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4383f8a-a0ad-44fe-9856-fffda752660d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:33.544248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4383f8a-a0ad-44fe-9856-fffda752660d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:33.554243 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 331/668] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:33.555338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cefe9-933c-4175-ac70-2e37d189fcc7 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-e4383f8a-a0ad-44fe-9856-fffda752660d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:33.558345 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:33.559241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:33.560142 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:33.560142 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:33.560142 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:33.560142 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:33.560142 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:33.560142 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:33.560142 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:33.560142 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:33.564964 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 338/669] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:33.566008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4383f8a-a0ad-44fe-9856-fffda752660d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:33.619056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:33.662729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:33.684073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cefe9-933c-4175-ac70-2e37d189fcc7 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:33.699055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4383f8a-a0ad-44fe-9856-fffda752660d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:33.722533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c3e5e-a3ab-48fd-814c-351ca556f596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:33.723478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c3e5e-a3ab-48fd-814c-351ca556f596 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:33.724051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c3e5e-a3ab-48fd-814c-351ca556f596 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:33.724651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c3e5e-a3ab-48fd-814c-351ca556f596 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:33.725715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c3e5e-a3ab-48fd-814c-351ca556f596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:33.726234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c3e5e-a3ab-48fd-814c-351ca556f596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:33.728042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c3e5e-a3ab-48fd-814c-351ca556f596 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:33.736870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be790d2b-75c3-429a-9f7c-3f25828bb43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:33.737323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be790d2b-75c3-429a-9f7c-3f25828bb43b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:33.737660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be790d2b-75c3-429a-9f7c-3f25828bb43b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:33.737994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be790d2b-75c3-429a-9f7c-3f25828bb43b None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:33.743422 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 332/670] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:33.744586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c3e5e-a3ab-48fd-814c-351ca556f596 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:33.745828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be790d2b-75c3-429a-9f7c-3f25828bb43b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:33.747563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be790d2b-75c3-429a-9f7c-3f25828bb43b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-be790d2b-75c3-429a-9f7c-3f25828bb43b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:33.763286 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:33.764694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:33.765814 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:33.765814 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:33.765814 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:33.765814 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:33.765814 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:33.765814 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:33.765814 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:33.765814 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:33.765814 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 339/671] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:33.770082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be790d2b-75c3-429a-9f7c-3f25828bb43b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:33.855033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c3e5e-a3ab-48fd-814c-351ca556f596 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:33.858189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:33.892642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be790d2b-75c3-429a-9f7c-3f25828bb43b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:33.895566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f8263-1256-435c-a068-0510abd5e7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:33.895566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f8263-1256-435c-a068-0510abd5e7c1 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:33.895846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f8263-1256-435c-a068-0510abd5e7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:33.896125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f8263-1256-435c-a068-0510abd5e7c1 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:33.896821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f8263-1256-435c-a068-0510abd5e7c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:33.897167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f8263-1256-435c-a068-0510abd5e7c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:33.898611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f8263-1256-435c-a068-0510abd5e7c1 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:33.916442 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 333/672] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:33.916442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f8263-1256-435c-a068-0510abd5e7c1 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:33.924722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:33.931007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bfe22-6665-4b79-8929-94771975c738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:33.931738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bfe22-6665-4b79-8929-94771975c738 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:33.932033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bfe22-6665-4b79-8929-94771975c738 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:33.932593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bfe22-6665-4b79-8929-94771975c738 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:33.945920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bfe22-6665-4b79-8929-94771975c738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-e37bfe22-6665-4b79-8929-94771975c738 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:33.951485 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:33.952710 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:33.953584 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 340/673] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:33.955111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bfe22-6665-4b79-8929-94771975c738 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:34.047203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f8263-1256-435c-a068-0510abd5e7c1 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:34.087367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc10de7-c0e0-4f59-bb63-d9e7ee7f8155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:34.088033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc10de7-c0e0-4f59-bb63-d9e7ee7f8155 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:34.088311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc10de7-c0e0-4f59-bb63-d9e7ee7f8155 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:34.088581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc10de7-c0e0-4f59-bb63-d9e7ee7f8155 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:34.096424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc10de7-c0e0-4f59-bb63-d9e7ee7f8155 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:34.097965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc10de7-c0e0-4f59-bb63-d9e7ee7f8155 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-5dc10de7-c0e0-4f59-bb63-d9e7ee7f8155 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:34.112861 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:34.113932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:34.120102 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:34.120102 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:34.120102 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:34.120102 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:34.120102 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:34.120102 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:34.120102 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:34.120102 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:34.120102 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 334/674] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:34.120102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc10de7-c0e0-4f59-bb63-d9e7ee7f8155 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:34.150209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:34.197333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bfe22-6665-4b79-8929-94771975c738 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:34.226470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc10de7-c0e0-4f59-bb63-d9e7ee7f8155 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:34.230776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bdbe5-8a24-47c8-9c57-060e627604b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:34.231379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bdbe5-8a24-47c8-9c57-060e627604b1 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:34.231663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bdbe5-8a24-47c8-9c57-060e627604b1 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:34.231937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bdbe5-8a24-47c8-9c57-060e627604b1 None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:34.232626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bdbe5-8a24-47c8-9c57-060e627604b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:34.232963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bdbe5-8a24-47c8-9c57-060e627604b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:34.234396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bdbe5-8a24-47c8-9c57-060e627604b1 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:34.245581 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 341/675] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:34.246366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bdbe5-8a24-47c8-9c57-060e627604b1 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:34.259125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d0873-b2e1-44c9-b9ed-2ce5223b5db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:34.259728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0873-b2e1-44c9-b9ed-2ce5223b5db4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:34.260012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0873-b2e1-44c9-b9ed-2ce5223b5db4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:34.260290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0873-b2e1-44c9-b9ed-2ce5223b5db4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:34.261042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d0873-b2e1-44c9-b9ed-2ce5223b5db4 None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=7a36c3c09b2d425fbf6ccbe051c293d4) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:34.261355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d0873-b2e1-44c9-b9ed-2ce5223b5db4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:34.262893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d0873-b2e1-44c9-b9ed-2ce5223b5db4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:34.279465 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 335/676] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:34 2022] GET /identity/v3/role_assignments?user.id=7a36c3c09b2d425fbf6ccbe051c293d4&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:34.279465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dd101-e9a6-4c72-bcaf-35ed8ef9653e None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:34.279465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dd101-e9a6-4c72-bcaf-35ed8ef9653e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:34.279465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dd101-e9a6-4c72-bcaf-35ed8ef9653e None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:34.279465 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 336/677] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:34.279465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dd101-e9a6-4c72-bcaf-35ed8ef9653e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:34.306879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:34.361124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bdbe5-8a24-47c8-9c57-060e627604b1 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:34.383246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:34.399651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dd101-e9a6-4c72-bcaf-35ed8ef9653e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:34.401602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9841d12-5a9e-4c1b-a2c8-320620a0ad5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:34.402168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9841d12-5a9e-4c1b-a2c8-320620a0ad5d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:34.402453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9841d12-5a9e-4c1b-a2c8-320620a0ad5d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:34.402730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9841d12-5a9e-4c1b-a2c8-320620a0ad5d None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:34.403427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9841d12-5a9e-4c1b-a2c8-320620a0ad5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:34.403739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9841d12-5a9e-4c1b-a2c8-320620a0ad5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:34.405182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9841d12-5a9e-4c1b-a2c8-320620a0ad5d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:34.414790 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 342/678] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:34.415681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9841d12-5a9e-4c1b-a2c8-320620a0ad5d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:34.443914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebf74a-0233-484e-ab64-b60f054178e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:34.444539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebf74a-0233-484e-ab64-b60f054178e7 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:34.444823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebf74a-0233-484e-ab64-b60f054178e7 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:34.445139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebf74a-0233-484e-ab64-b60f054178e7 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:34.445840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebf74a-0233-484e-ab64-b60f054178e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:34.446152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebf74a-0233-484e-ab64-b60f054178e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:34.447606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebf74a-0233-484e-ab64-b60f054178e7 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:34.461631 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 337/679] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:34.462810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b624a-a4b4-4e58-bce5-1e6ea85fda78 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:34.463106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b624a-a4b4-4e58-bce5-1e6ea85fda78 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:34.463388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b624a-a4b4-4e58-bce5-1e6ea85fda78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:34.481512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-903b624a-a4b4-4e58-bce5-1e6ea85fda78 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:34.482710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-903b624a-a4b4-4e58-bce5-1e6ea85fda78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:34.538395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9841d12-5a9e-4c1b-a2c8-320620a0ad5d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:34.554157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:34.557606 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 338/680] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:34.558436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b624a-a4b4-4e58-bce5-1e6ea85fda78 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:34.584661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b3a44-1ed1-4d79-a77c-ba19e3211a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:34.585402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b3a44-1ed1-4d79-a77c-ba19e3211a54 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:34.585694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b3a44-1ed1-4d79-a77c-ba19e3211a54 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:34.585971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b3a44-1ed1-4d79-a77c-ba19e3211a54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:34.586797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b3a44-1ed1-4d79-a77c-ba19e3211a54 None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=133a1ff64e224593be467eb7c27cbb09) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:34.587111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b3a44-1ed1-4d79-a77c-ba19e3211a54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:34.588716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b3a44-1ed1-4d79-a77c-ba19e3211a54 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:34.596099 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 343/681] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:34 2022] GET /identity/v3/role_assignments?user.id=133a1ff64e224593be467eb7c27cbb09&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:34.597046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b3a44-1ed1-4d79-a77c-ba19e3211a54 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:34.644279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:34.669278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b624a-a4b4-4e58-bce5-1e6ea85fda78 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:34.707079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96015e-9279-456f-97f5-962badfb9e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:34.707824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96015e-9279-456f-97f5-962badfb9e9b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:34.708115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96015e-9279-456f-97f5-962badfb9e9b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:34.708396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96015e-9279-456f-97f5-962badfb9e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:34.713380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b3a44-1ed1-4d79-a77c-ba19e3211a54 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:34.720779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96015e-9279-456f-97f5-962badfb9e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:34.722418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96015e-9279-456f-97f5-962badfb9e9b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-ab96015e-9279-456f-97f5-962badfb9e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:34.738437 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:34.739396 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:34.741578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:34.741578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:34.741578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:34.741578 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 339/682] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:34.752450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96015e-9279-456f-97f5-962badfb9e9b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:34.761846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50577a-83a5-4060-9bc5-687f710fc15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:34.762472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50577a-83a5-4060-9bc5-687f710fc15e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:34.762754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50577a-83a5-4060-9bc5-687f710fc15e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:34.763024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50577a-83a5-4060-9bc5-687f710fc15e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:34.763770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50577a-83a5-4060-9bc5-687f710fc15e None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=c0fb67db342b477aaf547543c498434f) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:34.764380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50577a-83a5-4060-9bc5-687f710fc15e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:34.765868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50577a-83a5-4060-9bc5-687f710fc15e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:34.782085 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 344/683] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:34 2022] GET /identity/v3/role_assignments?user.id=c0fb67db342b477aaf547543c498434f&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:34.925091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96015e-9279-456f-97f5-962badfb9e9b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:34.960420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:34.968601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f134a-9175-4dc4-8584-820afb508b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:34.969462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f134a-9175-4dc4-8584-820afb508b2e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:34.970630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f134a-9175-4dc4-8584-820afb508b2e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:34.971317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f134a-9175-4dc4-8584-820afb508b2e None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:34.972341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f134a-9175-4dc4-8584-820afb508b2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:34.972755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f134a-9175-4dc4-8584-820afb508b2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:34.974371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f134a-9175-4dc4-8584-820afb508b2e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:34.976482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50577a-83a5-4060-9bc5-687f710fc15e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:34.996713 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 340/684] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:35.000271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f134a-9175-4dc4-8584-820afb508b2e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:35.048493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:35.137032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50577a-83a5-4060-9bc5-687f710fc15e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:35.144989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f134a-9175-4dc4-8584-820afb508b2e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:35.193200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958cd55-0ecd-4a0c-9d6c-6535cf5aaf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:35.193853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958cd55-0ecd-4a0c-9d6c-6535cf5aaf14 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:35.194136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958cd55-0ecd-4a0c-9d6c-6535cf5aaf14 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:35.194416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958cd55-0ecd-4a0c-9d6c-6535cf5aaf14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:35.195493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030757e-9092-4ee7-9dd5-e986fd8cca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:35.196126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030757e-9092-4ee7-9dd5-e986fd8cca20 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:35.196409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030757e-9092-4ee7-9dd5-e986fd8cca20 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:35.196684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030757e-9092-4ee7-9dd5-e986fd8cca20 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:35.210972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030757e-9092-4ee7-9dd5-e986fd8cca20 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:35.212440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030757e-9092-4ee7-9dd5-e986fd8cca20 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:35.219145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958cd55-0ecd-4a0c-9d6c-6535cf5aaf14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-c958cd55-0ecd-4a0c-9d6c-6535cf5aaf14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:35.229982 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:35.230923 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:35.231737 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 345/685] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:35.237376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958cd55-0ecd-4a0c-9d6c-6535cf5aaf14 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a030757e-9092-4ee7-9dd5-e986fd8cca20 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:35.238515 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:35.239417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:35.241304 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:35.241304 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:35.241304 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:35.241304 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:35.241304 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:35.241304 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:35.241304 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:35.241304 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:35.241304 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:33:35.241304 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:35.248833 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 341/686] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 3 20:33:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:35.249753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030757e-9092-4ee7-9dd5-e986fd8cca20 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:35.311458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:35.366609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030757e-9092-4ee7-9dd5-e986fd8cca20 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:35.414707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949523b1-8dd7-4508-952a-29213a44204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:35.415339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949523b1-8dd7-4508-952a-29213a44204c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:35.415629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949523b1-8dd7-4508-952a-29213a44204c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:35.415910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949523b1-8dd7-4508-952a-29213a44204c None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:35.416757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949523b1-8dd7-4508-952a-29213a44204c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:35.417100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949523b1-8dd7-4508-952a-29213a44204c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:35.418672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949523b1-8dd7-4508-952a-29213a44204c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:35.423418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958cd55-0ecd-4a0c-9d6c-6535cf5aaf14 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:35.434035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:35.452187 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 342/687] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 3 20:33:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:35.453501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949523b1-8dd7-4508-952a-29213a44204c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:35.476813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057ba72-0566-41f2-a049-7e7d7fea4b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:35.477463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057ba72-0566-41f2-a049-7e7d7fea4b9a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:35.477747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057ba72-0566-41f2-a049-7e7d7fea4b9a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:35.478026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057ba72-0566-41f2-a049-7e7d7fea4b9a None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:35.478699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057ba72-0566-41f2-a049-7e7d7fea4b9a None admin] RBAC: Adding query filter params (name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:35.479006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057ba72-0566-41f2-a049-7e7d7fea4b9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:35.480438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057ba72-0566-41f2-a049-7e7d7fea4b9a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:35.489445 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 346/688] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:35.494984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057ba72-0566-41f2-a049-7e7d7fea4b9a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:35.563906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949523b1-8dd7-4508-952a-29213a44204c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:35.573755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:35.614178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057ba72-0566-41f2-a049-7e7d7fea4b9a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:35.616108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492207ba-b301-490d-ac09-43bcdd2e65e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:35.617036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492207ba-b301-490d-ac09-43bcdd2e65e8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:35.617563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492207ba-b301-490d-ac09-43bcdd2e65e8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:35.618042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492207ba-b301-490d-ac09-43bcdd2e65e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:35.649429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b48043-0443-414d-b3cd-36cd7d2a1913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:35.650152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b48043-0443-414d-b3cd-36cd7d2a1913 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:35.650439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b48043-0443-414d-b3cd-36cd7d2a1913 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:35.650717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b48043-0443-414d-b3cd-36cd7d2a1913 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:35.670667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b48043-0443-414d-b3cd-36cd7d2a1913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:35.672139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b48043-0443-414d-b3cd-36cd7d2a1913 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-09b48043-0443-414d-b3cd-36cd7d2a1913 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:35.688412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:35.689645 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:35.690570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:35.690570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:35.690570 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:35.690570 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 347/689] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:35.695227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b48043-0443-414d-b3cd-36cd7d2a1913 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:35.722336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:35.791536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492207ba-b301-490d-ac09-43bcdd2e65e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:35.793039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492207ba-b301-490d-ac09-43bcdd2e65e8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:35.871119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b48043-0443-414d-b3cd-36cd7d2a1913 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:35.935084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea85519-51a6-4559-9fda-26d843e3cc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:35.936884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea85519-51a6-4559-9fda-26d843e3cc6b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:35.938158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea85519-51a6-4559-9fda-26d843e3cc6b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:35.938678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea85519-51a6-4559-9fda-26d843e3cc6b None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:35.939514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea85519-51a6-4559-9fda-26d843e3cc6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:35.941352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea85519-51a6-4559-9fda-26d843e3cc6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:35.942940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea85519-51a6-4559-9fda-26d843e3cc6b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:35.959145 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 348/690] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:35.964717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea85519-51a6-4559-9fda-26d843e3cc6b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:35.985887 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 343/691] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 20:33:35.991251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492207ba-b301-490d-ac09-43bcdd2e65e8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:36.012764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:36.048648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:36.072466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea85519-51a6-4559-9fda-26d843e3cc6b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:36.111093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed149b5f-b717-4603-9fc6-2de358802580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:36.112080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed149b5f-b717-4603-9fc6-2de358802580 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:36.112592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed149b5f-b717-4603-9fc6-2de358802580 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:36.113113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed149b5f-b717-4603-9fc6-2de358802580 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:36.121218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed149b5f-b717-4603-9fc6-2de358802580 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:36.123082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed149b5f-b717-4603-9fc6-2de358802580 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-ed149b5f-b717-4603-9fc6-2de358802580 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:36.137538 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:36.138464 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:36.139387 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:36.139387 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:36.139387 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:36.139387 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:36.139387 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:36.139387 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:36.139387 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:36.139387 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:36.139387 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 20:33:36.139387 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:36.142501 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 349/692] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:36.148384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed149b5f-b717-4603-9fc6-2de358802580 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:36.152755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492207ba-b301-490d-ac09-43bcdd2e65e8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:36.240003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03185e7d-81bc-418d-8cf2-d311e06ad063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:36.240885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03185e7d-81bc-418d-8cf2-d311e06ad063 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:36.241036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03185e7d-81bc-418d-8cf2-d311e06ad063 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:36.241261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03185e7d-81bc-418d-8cf2-d311e06ad063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:36.260322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed149b5f-b717-4603-9fc6-2de358802580 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:36.263641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03185e7d-81bc-418d-8cf2-d311e06ad063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:36.264602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03185e7d-81bc-418d-8cf2-d311e06ad063 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-03185e7d-81bc-418d-8cf2-d311e06ad063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:36.295935 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:36.296944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:36.298417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:36.298417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:36.298417 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:36.301247 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 344/693] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:36.303691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfa911-f9a5-4318-957f-193eabf69934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:36.304551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfa911-f9a5-4318-957f-193eabf69934 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:36.305153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfa911-f9a5-4318-957f-193eabf69934 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:36.305633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfa911-f9a5-4318-957f-193eabf69934 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:36.306007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03185e7d-81bc-418d-8cf2-d311e06ad063 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:36.306930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfa911-f9a5-4318-957f-193eabf69934 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:36.307509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfa911-f9a5-4318-957f-193eabf69934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:36.309265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfa911-f9a5-4318-957f-193eabf69934 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:36.312961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:36.334544 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 350/694] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Sep 3 20:33:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:36.340439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfa911-f9a5-4318-957f-193eabf69934 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:36.402793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:36.475810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03185e7d-81bc-418d-8cf2-d311e06ad063 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:36.497904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfa911-f9a5-4318-957f-193eabf69934 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:36.549540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e0873-6427-4b46-8b58-5d066c186f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:36.550183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e0873-6427-4b46-8b58-5d066c186f42 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:36.550469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e0873-6427-4b46-8b58-5d066c186f42 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:36.550751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e0873-6427-4b46-8b58-5d066c186f42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:36.551578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f992eb-8a46-4f46-bd96-746ccbf7da5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:36.552416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f992eb-8a46-4f46-bd96-746ccbf7da5e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:36.552916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f992eb-8a46-4f46-bd96-746ccbf7da5e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:36.553417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f992eb-8a46-4f46-bd96-746ccbf7da5e None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:36.554267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f992eb-8a46-4f46-bd96-746ccbf7da5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:36.554786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f992eb-8a46-4f46-bd96-746ccbf7da5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:36.557348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f992eb-8a46-4f46-bd96-746ccbf7da5e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:36.584846 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 345/695] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:36.586074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f992eb-8a46-4f46-bd96-746ccbf7da5e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:36.623859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e0873-6427-4b46-8b58-5d066c186f42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:36.625561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e0873-6427-4b46-8b58-5d066c186f42 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:36.629004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:36.742381 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 351/696] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 20:33:36.747498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e0873-6427-4b46-8b58-5d066c186f42 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:36.769344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f992eb-8a46-4f46-bd96-746ccbf7da5e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:36.810434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:36.843719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685d9b1-8836-431e-9689-41104248b8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:36.844369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685d9b1-8836-431e-9689-41104248b8fc None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:36.845251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685d9b1-8836-431e-9689-41104248b8fc None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:36.845537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685d9b1-8836-431e-9689-41104248b8fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:36.859415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685d9b1-8836-431e-9689-41104248b8fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:36.860937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685d9b1-8836-431e-9689-41104248b8fc None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-4685d9b1-8836-431e-9689-41104248b8fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:36.883450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:36.884424 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:36.885351 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:36.885351 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:36.885351 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:36.885351 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:36.885351 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:36.885351 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:36.885351 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:36.885351 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:36.885351 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 346/697] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:36.891167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685d9b1-8836-431e-9689-41104248b8fc None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:36.937304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:36.945813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e0873-6427-4b46-8b58-5d066c186f42 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:36.987368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c5a9a-73c1-4fd2-891b-00d8faf8bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:36.988028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c5a9a-73c1-4fd2-891b-00d8faf8bbf2 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:36.988313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c5a9a-73c1-4fd2-891b-00d8faf8bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:36.988589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c5a9a-73c1-4fd2-891b-00d8faf8bbf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:36.999414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c5a9a-73c1-4fd2-891b-00d8faf8bbf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:37.001579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c5a9a-73c1-4fd2-891b-00d8faf8bbf2 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-ea8c5a9a-73c1-4fd2-891b-00d8faf8bbf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:37.017216 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:37.018425 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:37.019602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:37.019602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:37.019602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:37.019602 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 352/698] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:37.019849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa904075-c6c3-49ce-b706-67189bc20a5a None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:37.020079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa904075-c6c3-49ce-b706-67189bc20a5a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:37.020356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa904075-c6c3-49ce-b706-67189bc20a5a None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:37.026699 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 353/699] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:37.027245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa904075-c6c3-49ce-b706-67189bc20a5a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:37.029990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685d9b1-8836-431e-9689-41104248b8fc None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:37.066910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2397555-8b6f-4002-9d7c-8bc4572444ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:37.067947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2397555-8b6f-4002-9d7c-8bc4572444ef None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:37.068448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2397555-8b6f-4002-9d7c-8bc4572444ef None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:37.069036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2397555-8b6f-4002-9d7c-8bc4572444ef None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:37.070163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2397555-8b6f-4002-9d7c-8bc4572444ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:37.070720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2397555-8b6f-4002-9d7c-8bc4572444ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:37.072381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2397555-8b6f-4002-9d7c-8bc4572444ef None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:37.086413 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 347/700] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:37.088299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1665065-de88-4008-995a-6d4f4b8c404a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:37.088774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1665065-de88-4008-995a-6d4f4b8c404a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:37.089276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1665065-de88-4008-995a-6d4f4b8c404a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:37.109414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a1665065-de88-4008-995a-6d4f4b8c404a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:37.111607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a1665065-de88-4008-995a-6d4f4b8c404a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:37.148505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa904075-c6c3-49ce-b706-67189bc20a5a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:37.149711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:37.196923 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 348/701] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:37.196923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1665065-de88-4008-995a-6d4f4b8c404a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:37.198465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228de05-c2f8-49c1-b2e9-fe725831f4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:37.199294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228de05-c2f8-49c1-b2e9-fe725831f4ba None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:37.199838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228de05-c2f8-49c1-b2e9-fe725831f4ba None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:37.200299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228de05-c2f8-49c1-b2e9-fe725831f4ba None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:37.201208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228de05-c2f8-49c1-b2e9-fe725831f4ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:37.201754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228de05-c2f8-49c1-b2e9-fe725831f4ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:37.203357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228de05-c2f8-49c1-b2e9-fe725831f4ba None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:37.226095 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 354/702] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:37.231094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228de05-c2f8-49c1-b2e9-fe725831f4ba None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:37.242671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:37.300965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1665065-de88-4008-995a-6d4f4b8c404a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:37.348301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1625f3-1b53-48da-9118-11cf7f01f3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:37.349411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1625f3-1b53-48da-9118-11cf7f01f3cf None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:37.350498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1625f3-1b53-48da-9118-11cf7f01f3cf None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:37.351082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1625f3-1b53-48da-9118-11cf7f01f3cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:37.352987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1625f3-1b53-48da-9118-11cf7f01f3cf None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=a56702329eae48819121dcb58f7e3070) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:37.353753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1625f3-1b53-48da-9118-11cf7f01f3cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:37.356128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1625f3-1b53-48da-9118-11cf7f01f3cf None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:37.370881 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 349/703] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:37 2022] GET /identity/v3/role_assignments?user.id=a56702329eae48819121dcb58f7e3070&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=bc768522dc984239be4a1b39a45ab437 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:37.421092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228de05-c2f8-49c1-b2e9-fe725831f4ba None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:37.469273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8706e-3c38-4c89-8230-ecbfc2553927 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:37.469877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8706e-3c38-4c89-8230-ecbfc2553927 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:37.470312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8706e-3c38-4c89-8230-ecbfc2553927 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:37.471874 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 350/704] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:33:37.477491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a6746-0be9-418d-a983-9e79df1afe06 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:37.477788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a6746-0be9-418d-a983-9e79df1afe06 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:37.478062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a6746-0be9-418d-a983-9e79df1afe06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:37.480832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261429c1-d0ae-4ed7-8267-7d95ec925838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:37.481571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261429c1-d0ae-4ed7-8267-7d95ec925838 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:37.481861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261429c1-d0ae-4ed7-8267-7d95ec925838 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:37.482141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261429c1-d0ae-4ed7-8267-7d95ec925838 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:37.496965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261429c1-d0ae-4ed7-8267-7d95ec925838 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:37.498534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261429c1-d0ae-4ed7-8267-7d95ec925838 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:37.499990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-091a6746-0be9-418d-a983-9e79df1afe06 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:37.502084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-091a6746-0be9-418d-a983-9e79df1afe06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-261429c1-d0ae-4ed7-8267-7d95ec925838 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:37.523495 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:37.524454 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:37.525383 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:37.525383 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:37.525383 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:37.525383 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:37.525383 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:37.525383 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:37.525383 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:37.525383 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:37.530612 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 355/705] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:37.534263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261429c1-d0ae-4ed7-8267-7d95ec925838 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:37.557512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:37.623810 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 351/706] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:37.657832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:37.707495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261429c1-d0ae-4ed7-8267-7d95ec925838 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:37.713135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3f273-444d-4100-81dd-6f0076a83a8e None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:37.713437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3f273-444d-4100-81dd-6f0076a83a8e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:37.713720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3f273-444d-4100-81dd-6f0076a83a8e None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:37.715315 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 352/707] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:33:37.721039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b304bbf-ac58-4c9b-b3bc-35e87fbc3398 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:37.721337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b304bbf-ac58-4c9b-b3bc-35e87fbc3398 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:37.721624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b304bbf-ac58-4c9b-b3bc-35e87fbc3398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:37.749553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1b304bbf-ac58-4c9b-b3bc-35e87fbc3398 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:37.751287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1b304bbf-ac58-4c9b-b3bc-35e87fbc3398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:37.764153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e86c6-1f9f-482e-a931-503c01e52e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:37.765082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e86c6-1f9f-482e-a931-503c01e52e2d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:37.765378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e86c6-1f9f-482e-a931-503c01e52e2d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:37.765657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e86c6-1f9f-482e-a931-503c01e52e2d None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:37.767055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e86c6-1f9f-482e-a931-503c01e52e2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:37.767373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e86c6-1f9f-482e-a931-503c01e52e2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:37.769539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e86c6-1f9f-482e-a931-503c01e52e2d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:37.785846 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 356/708] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:37.787058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e86c6-1f9f-482e-a931-503c01e52e2d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:37.835407 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 353/709] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 20:33:37.836697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b304bbf-ac58-4c9b-b3bc-35e87fbc3398 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:37.939054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e86c6-1f9f-482e-a931-503c01e52e2d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:37.969120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:37.984198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:37.988071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cd5a4-b77a-45d0-9803-333bcc02194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:37.988722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cd5a4-b77a-45d0-9803-333bcc02194f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:37.989031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cd5a4-b77a-45d0-9803-333bcc02194f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:37.989309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cd5a4-b77a-45d0-9803-333bcc02194f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:37.991982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b304bbf-ac58-4c9b-b3bc-35e87fbc3398 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:38.000044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cd5a4-b77a-45d0-9803-333bcc02194f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:38.001708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cd5a4-b77a-45d0-9803-333bcc02194f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-9a6cd5a4-b77a-45d0-9803-333bcc02194f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:38.017842 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:38.018821 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:38.019712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:38.019712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:38.019712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:38.019712 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 357/710] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:38.020390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cd5a4-b77a-45d0-9803-333bcc02194f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:38.034613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd2fca-073e-49c4-8b91-9ff260b32c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:38.035404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd2fca-073e-49c4-8b91-9ff260b32c9f None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:38.035700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd2fca-073e-49c4-8b91-9ff260b32c9f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:38.035984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd2fca-073e-49c4-8b91-9ff260b32c9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:38.036890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd2fca-073e-49c4-8b91-9ff260b32c9f None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=83865463b2a84f99a136b73dffeeb378) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:38.037233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd2fca-073e-49c4-8b91-9ff260b32c9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:38.038683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd2fca-073e-49c4-8b91-9ff260b32c9f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:38.045232 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 354/711] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:37 2022] GET /identity/v3/role_assignments?user.id=83865463b2a84f99a136b73dffeeb378&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:38.046048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd2fca-073e-49c4-8b91-9ff260b32c9f None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:38.211608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd2fca-073e-49c4-8b91-9ff260b32c9f None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:38.237882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cd5a4-b77a-45d0-9803-333bcc02194f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:38.255289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f951f8b-d7b7-4a18-99e3-8a75e4abb372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:38.256811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f951f8b-d7b7-4a18-99e3-8a75e4abb372 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:38.257620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f951f8b-d7b7-4a18-99e3-8a75e4abb372 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:38.258129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f951f8b-d7b7-4a18-99e3-8a75e4abb372 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:38.259363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f951f8b-d7b7-4a18-99e3-8a75e4abb372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:38.260181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f951f8b-d7b7-4a18-99e3-8a75e4abb372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:38.261860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f951f8b-d7b7-4a18-99e3-8a75e4abb372 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:38.277947 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 355/712] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:38.279000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f951f8b-d7b7-4a18-99e3-8a75e4abb372 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:38.280309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad41432-aab9-4ee0-ad48-3d6eaaf33c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:38.280976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad41432-aab9-4ee0-ad48-3d6eaaf33c7b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:38.281266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad41432-aab9-4ee0-ad48-3d6eaaf33c7b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:38.281538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad41432-aab9-4ee0-ad48-3d6eaaf33c7b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:38.282691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad41432-aab9-4ee0-ad48-3d6eaaf33c7b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:38.284848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad41432-aab9-4ee0-ad48-3d6eaaf33c7b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-9ad41432-aab9-4ee0-ad48-3d6eaaf33c7b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:38.297291 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:38.298681 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 20:33:38.299742 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:38.301827 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 358/713] 10.222.0.16 () {56 vars in 1129 bytes} [Sat Sep 3 20:33:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:38.302813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad41432-aab9-4ee0-ad48-3d6eaaf33c7b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:38.304992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:38.361478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:38.417315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f951f8b-d7b7-4a18-99e3-8a75e4abb372 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:38.440594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad41432-aab9-4ee0-ad48-3d6eaaf33c7b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:38.454774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a33ab-8a25-4b19-bed4-08d9ad48e500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:38.455665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a33ab-8a25-4b19-bed4-08d9ad48e500 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:38.456133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a33ab-8a25-4b19-bed4-08d9ad48e500 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:38.456629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a33ab-8a25-4b19-bed4-08d9ad48e500 None admin] PATH_INFO: `/v3/services/network` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:38.457683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a33ab-8a25-4b19-bed4-08d9ad48e500 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:38.459292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a33ab-8a25-4b19-bed4-08d9ad48e500 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a70a33ab-8a25-4b19-bed4-08d9ad48e500 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:38.471242 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:38.472244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 20:33:38.473114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:38.480327 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 356/714] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:38 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:38.481130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655b4d4-4fc2-4639-bead-b242bebebfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:38.481726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a33ab-8a25-4b19-bed4-08d9ad48e500 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:38.481836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655b4d4-4fc2-4639-bead-b242bebebfab None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:38.482027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655b4d4-4fc2-4639-bead-b242bebebfab None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:38.482310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655b4d4-4fc2-4639-bead-b242bebebfab None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:38.483118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655b4d4-4fc2-4639-bead-b242bebebfab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:38.484554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655b4d4-4fc2-4639-bead-b242bebebfab None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:38.513833 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 359/715] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:33:38 2022] POST /identity/v3/users => generated 254 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:33:38.517459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655b4d4-4fc2-4639-bead-b242bebebfab None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:38.638493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:38.641392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655b4d4-4fc2-4639-bead-b242bebebfab None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:38.678319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:38.685011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec4e17-8db8-4f60-901a-a213b05b4dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:38.686109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec4e17-8db8-4f60-901a-a213b05b4dba None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:38.686718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec4e17-8db8-4f60-901a-a213b05b4dba None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:38.687215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec4e17-8db8-4f60-901a-a213b05b4dba None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:38.688187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec4e17-8db8-4f60-901a-a213b05b4dba None admin] RBAC: Adding query filter params (name=network) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:38.689290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec4e17-8db8-4f60-901a-a213b05b4dba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:38.691104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec4e17-8db8-4f60-901a-a213b05b4dba None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:38.696999 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 360/716] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:33:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:38.704519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec4e17-8db8-4f60-901a-a213b05b4dba None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:38.729134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a33ab-8a25-4b19-bed4-08d9ad48e500 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:38.769767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85780118-15b5-4fe7-bdd7-f6f8f3655415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:38.770747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85780118-15b5-4fe7-bdd7-f6f8f3655415 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:38.771340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85780118-15b5-4fe7-bdd7-f6f8f3655415 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:38.771822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85780118-15b5-4fe7-bdd7-f6f8f3655415 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:38.772725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85780118-15b5-4fe7-bdd7-f6f8f3655415 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:38.773305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85780118-15b5-4fe7-bdd7-f6f8f3655415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:38.774886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85780118-15b5-4fe7-bdd7-f6f8f3655415 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:38.784176 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 357/717] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Sep 3 20:33:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:38.789620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85780118-15b5-4fe7-bdd7-f6f8f3655415 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:38.857560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec4e17-8db8-4f60-901a-a213b05b4dba None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:38.901230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27b311-cae4-433d-96b0-41dbbd6d319b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:38.902188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27b311-cae4-433d-96b0-41dbbd6d319b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:38.902677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27b311-cae4-433d-96b0-41dbbd6d319b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:38.903173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27b311-cae4-433d-96b0-41dbbd6d319b None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:38.904010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27b311-cae4-433d-96b0-41dbbd6d319b None admin] RBAC: Adding query filter params (type=network) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:38.904538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27b311-cae4-433d-96b0-41dbbd6d319b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:38.906198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27b311-cae4-433d-96b0-41dbbd6d319b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:38.914425 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 361/718] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:33:38 2022] GET /identity/v3/services?type=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:38.921791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85780118-15b5-4fe7-bdd7-f6f8f3655415 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:38.969104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8a3a1-dbfb-4cc6-bbaa-3f56d48a701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:38.970302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8a3a1-dbfb-4cc6-bbaa-3f56d48a701a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:38.970585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8a3a1-dbfb-4cc6-bbaa-3f56d48a701a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:38.970866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8a3a1-dbfb-4cc6-bbaa-3f56d48a701a None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:38.971506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8a3a1-dbfb-4cc6-bbaa-3f56d48a701a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:38.971814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8a3a1-dbfb-4cc6-bbaa-3f56d48a701a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:38.973223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8a3a1-dbfb-4cc6-bbaa-3f56d48a701a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:38.979736 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 358/719] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Sep 3 20:33:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:39.869841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f2064-5b38-44b8-bfea-5b285826d42d None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:39.870362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f2064-5b38-44b8-bfea-5b285826d42d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:39.870472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f2064-5b38-44b8-bfea-5b285826d42d None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:39.872076 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 362/720] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:39.877522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece572e-cc79-448f-bfe9-f6bb29843c4a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:39.877984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece572e-cc79-448f-bfe9-f6bb29843c4a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:39.878180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece572e-cc79-448f-bfe9-f6bb29843c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:39.902217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8ece572e-cc79-448f-bfe9-f6bb29843c4a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:39.903425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8ece572e-cc79-448f-bfe9-f6bb29843c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:39.910280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:40.010016 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 359/721] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:40.445039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f2064-5b38-44b8-bfea-5b285826d42d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:40.487373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:40.582776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f2064-5b38-44b8-bfea-5b285826d42d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:40.631946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab68d5ef-f758-4059-b1a6-36e78553e77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:40.632594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68d5ef-f758-4059-b1a6-36e78553e77e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:40.632880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68d5ef-f758-4059-b1a6-36e78553e77e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:40.633187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68d5ef-f758-4059-b1a6-36e78553e77e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:40.650197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab68d5ef-f758-4059-b1a6-36e78553e77e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-ab68d5ef-f758-4059-b1a6-36e78553e77e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:40.655075 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:40.655987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:40.656820 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 363/722] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:40.662295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece572e-cc79-448f-bfe9-f6bb29843c4a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:40.832817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece572e-cc79-448f-bfe9-f6bb29843c4a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:40.877164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b0c3e-0e3b-44d4-a127-3eafb536d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:40.877786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b0c3e-0e3b-44d4-a127-3eafb536d48e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:40.878334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b0c3e-0e3b-44d4-a127-3eafb536d48e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:40.878334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b0c3e-0e3b-44d4-a127-3eafb536d48e None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:40.878987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b0c3e-0e3b-44d4-a127-3eafb536d48e None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:40.879294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b0c3e-0e3b-44d4-a127-3eafb536d48e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:40.880663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b0c3e-0e3b-44d4-a127-3eafb536d48e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:40.889247 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 360/723] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:40.895577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab68d5ef-f758-4059-b1a6-36e78553e77e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:40.957562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915cf3b-4696-4cb7-bf05-8f372aeb88dd None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:40.958249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915cf3b-4696-4cb7-bf05-8f372aeb88dd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:40.958737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915cf3b-4696-4cb7-bf05-8f372aeb88dd None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:40.960542 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 361/724] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:33:40.965643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64b8be-240a-4b0d-af5c-168218632234 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:40.966262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64b8be-240a-4b0d-af5c-168218632234 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:40.966749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64b8be-240a-4b0d-af5c-168218632234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:40.974086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:40.989024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8d64b8be-240a-4b0d-af5c-168218632234 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:40.991126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8d64b8be-240a-4b0d-af5c-168218632234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:41.072689 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 362/725] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:41.093102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:41.116510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab68d5ef-f758-4059-b1a6-36e78553e77e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:41.177978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f3bc0-ea38-4ffd-9bf2-9fbc2db987be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:41.178627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f3bc0-ea38-4ffd-9bf2-9fbc2db987be None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:41.178918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f3bc0-ea38-4ffd-9bf2-9fbc2db987be None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:41.179203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f3bc0-ea38-4ffd-9bf2-9fbc2db987be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:41.196047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f3bc0-ea38-4ffd-9bf2-9fbc2db987be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:41.197593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f3bc0-ea38-4ffd-9bf2-9fbc2db987be None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-8e4f3bc0-ea38-4ffd-9bf2-9fbc2db987be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:41.221069 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:41.222093 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:41.223059 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:41.223059 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:41.223059 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:41.223059 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 364/726] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:41.227909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64b8be-240a-4b0d-af5c-168218632234 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:41.373054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64b8be-240a-4b0d-af5c-168218632234 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:41.549759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c3bc9-be46-43e3-bed6-6ce58fbdb1d7 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:41.550628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c3bc9-be46-43e3-bed6-6ce58fbdb1d7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:41.552255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c3bc9-be46-43e3-bed6-6ce58fbdb1d7 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:41.554582 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 365/727] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:33:41.560760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:41.562282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9f2d7-5c7c-430c-a018-7c7995c8a677 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:41.562971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9f2d7-5c7c-430c-a018-7c7995c8a677 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:41.563165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9f2d7-5c7c-430c-a018-7c7995c8a677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:41.565806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd756885-a3b5-4c38-a997-8352dffe76e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:41.566612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd756885-a3b5-4c38-a997-8352dffe76e6 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:41.567218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd756885-a3b5-4c38-a997-8352dffe76e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:41.567749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd756885-a3b5-4c38-a997-8352dffe76e6 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:41.568663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd756885-a3b5-4c38-a997-8352dffe76e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:41.569152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd756885-a3b5-4c38-a997-8352dffe76e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:41.570745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd756885-a3b5-4c38-a997-8352dffe76e6 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:41.589838 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 363/728] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:41.591779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd756885-a3b5-4c38-a997-8352dffe76e6 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:41.599865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-49f9f2d7-5c7c-430c-a018-7c7995c8a677 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:41.601115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-49f9f2d7-5c7c-430c-a018-7c7995c8a677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:41.693949 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 366/729] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:41.695146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9f2d7-5c7c-430c-a018-7c7995c8a677 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:41.746446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd756885-a3b5-4c38-a997-8352dffe76e6 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:41.768560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:41.807590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9f2d7-5c7c-430c-a018-7c7995c8a677 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:41.808798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fceaa9-4e8e-47c1-8fc4-2fe801febf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:41.809520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fceaa9-4e8e-47c1-8fc4-2fe801febf3c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:41.809814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fceaa9-4e8e-47c1-8fc4-2fe801febf3c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:41.810097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fceaa9-4e8e-47c1-8fc4-2fe801febf3c None admin] PATH_INFO: `/v3/users/nova` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:41.822971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fceaa9-4e8e-47c1-8fc4-2fe801febf3c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:41.824702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fceaa9-4e8e-47c1-8fc4-2fe801febf3c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-d9fceaa9-4e8e-47c1-8fc4-2fe801febf3c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:41.839394 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:41.840366 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:41.841336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:41.841336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:41.841336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:41.841336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:41.841336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:41.841336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:41.841336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:41.841336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:41.841336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:33:41.841336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:41.841336 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 364/730] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 3 20:33:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:41.841919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4aa1a-b7d3-427b-b58b-dfff369fd27d None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:41.841919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4aa1a-b7d3-427b-b58b-dfff369fd27d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:41.842858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4aa1a-b7d3-427b-b58b-dfff369fd27d None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:41.843615 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 365/731] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:41.845033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa6e43-64f5-4e70-b165-4ec2bd193e15 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:41.845383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa6e43-64f5-4e70-b165-4ec2bd193e15 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:41.845677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa6e43-64f5-4e70-b165-4ec2bd193e15 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:41.847033 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 366/732] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:41.849662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa6e43-64f5-4e70-b165-4ec2bd193e15 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:41.856254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e13fd-775c-4485-b520-2e1cf8a43d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:41.856929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e13fd-775c-4485-b520-2e1cf8a43d7b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:41.857227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e13fd-775c-4485-b520-2e1cf8a43d7b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:41.857508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e13fd-775c-4485-b520-2e1cf8a43d7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:41.869769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e13fd-775c-4485-b520-2e1cf8a43d7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-e87e13fd-775c-4485-b520-2e1cf8a43d7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:41.874694 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:41.875594 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:41.876483 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 367/733] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:41.877214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc1627-a259-4e14-b458-60df2dcea262 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:41.877290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc1627-a259-4e14-b458-60df2dcea262 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:41.877892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc1627-a259-4e14-b458-60df2dcea262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:41.895760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2edc1627-a259-4e14-b458-60df2dcea262 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:41.896988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2edc1627-a259-4e14-b458-60df2dcea262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:41.915587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:41.966655 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 368/734] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:41.967774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cab144-4d7e-45ff-9aa1-5674d9731518 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:41.968072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cab144-4d7e-45ff-9aa1-5674d9731518 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:41.968362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cab144-4d7e-45ff-9aa1-5674d9731518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:41.980920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa6e43-64f5-4e70-b165-4ec2bd193e15 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:41.987324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b0cab144-4d7e-45ff-9aa1-5674d9731518 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:41.988538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b0cab144-4d7e-45ff-9aa1-5674d9731518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:42.021463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524a2d8-81a1-4f38-bd13-e37c70039ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:42.022340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524a2d8-81a1-4f38-bd13-e37c70039ef4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:42.022720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524a2d8-81a1-4f38-bd13-e37c70039ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:42.023000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524a2d8-81a1-4f38-bd13-e37c70039ef4 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:42.023905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524a2d8-81a1-4f38-bd13-e37c70039ef4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:42.024307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524a2d8-81a1-4f38-bd13-e37c70039ef4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:42.025771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524a2d8-81a1-4f38-bd13-e37c70039ef4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:42.047534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:42.049948 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 367/735] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 3 20:33:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:42.050842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524a2d8-81a1-4f38-bd13-e37c70039ef4 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:42.064415 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 369/736] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 20:33:42.065525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cab144-4d7e-45ff-9aa1-5674d9731518 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:42.191938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524a2d8-81a1-4f38-bd13-e37c70039ef4 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:42.204660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:42.217136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cab144-4d7e-45ff-9aa1-5674d9731518 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:42.233800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e420dcf-8d22-497c-a0bc-b91d3019bb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:42.234571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e420dcf-8d22-497c-a0bc-b91d3019bb7c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:42.234958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e420dcf-8d22-497c-a0bc-b91d3019bb7c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:42.235540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e420dcf-8d22-497c-a0bc-b91d3019bb7c None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:42.236341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e420dcf-8d22-497c-a0bc-b91d3019bb7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:42.237128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e420dcf-8d22-497c-a0bc-b91d3019bb7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:42.238902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e420dcf-8d22-497c-a0bc-b91d3019bb7c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:42.249740 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 368/737] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:42.251444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e420dcf-8d22-497c-a0bc-b91d3019bb7c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:42.260757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed736c4-548d-4fb5-9ec8-ba0274bcc5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:42.261767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed736c4-548d-4fb5-9ec8-ba0274bcc5de None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:42.262723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed736c4-548d-4fb5-9ec8-ba0274bcc5de None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:42.263240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed736c4-548d-4fb5-9ec8-ba0274bcc5de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:42.277467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed736c4-548d-4fb5-9ec8-ba0274bcc5de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:42.279980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed736c4-548d-4fb5-9ec8-ba0274bcc5de None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-9ed736c4-548d-4fb5-9ec8-ba0274bcc5de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:42.300461 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:42.301548 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:42.302463 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:42.302463 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:42.302463 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:42.302463 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 370/738] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:42.303318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed736c4-548d-4fb5-9ec8-ba0274bcc5de None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:42.347509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:42.367769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e420dcf-8d22-497c-a0bc-b91d3019bb7c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:42.406165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d9297-f440-4d96-8af2-8b89c7051c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:42.406909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d9297-f440-4d96-8af2-8b89c7051c89 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:42.407692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d9297-f440-4d96-8af2-8b89c7051c89 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:42.407965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d9297-f440-4d96-8af2-8b89c7051c89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:42.412999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed736c4-548d-4fb5-9ec8-ba0274bcc5de None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:42.418459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d9297-f440-4d96-8af2-8b89c7051c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-df5d9297-f440-4d96-8af2-8b89c7051c89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:42.424821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:42.425785 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 369/739] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:42.429901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d9297-f440-4d96-8af2-8b89c7051c89 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:42.454149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dee2a-b846-44b9-8cea-ad9608d111a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:42.454789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dee2a-b846-44b9-8cea-ad9608d111a8 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:42.455088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dee2a-b846-44b9-8cea-ad9608d111a8 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:42.455356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dee2a-b846-44b9-8cea-ad9608d111a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:42.459165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:42.466477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dee2a-b846-44b9-8cea-ad9608d111a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:42.467994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dee2a-b846-44b9-8cea-ad9608d111a8 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-771dee2a-b846-44b9-8cea-ad9608d111a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:42.482722 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:42.483667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:42.484550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:42.484550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:42.484550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:42.484978 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 371/740] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:42.486479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dee2a-b846-44b9-8cea-ad9608d111a8 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:42.528086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d9297-f440-4d96-8af2-8b89c7051c89 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:42.560794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d19bbf-cf2d-4407-91ad-e70f5e189be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:42.561437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d19bbf-cf2d-4407-91ad-e70f5e189be9 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:42.561724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d19bbf-cf2d-4407-91ad-e70f5e189be9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:42.562004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d19bbf-cf2d-4407-91ad-e70f5e189be9 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:42.562665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d19bbf-cf2d-4407-91ad-e70f5e189be9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:42.564066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d19bbf-cf2d-4407-91ad-e70f5e189be9 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:42.576818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:42.579852 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 370/741] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:33:42 2022] POST /identity/v3/services => generated 251 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:33:42.580974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d19bbf-cf2d-4407-91ad-e70f5e189be9 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:42.586991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dee2a-b846-44b9-8cea-ad9608d111a8 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:42.635711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793adbd-0f23-4a36-9fb9-54b6fae0c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:42.636818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793adbd-0f23-4a36-9fb9-54b6fae0c301 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:42.637241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793adbd-0f23-4a36-9fb9-54b6fae0c301 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:42.637582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793adbd-0f23-4a36-9fb9-54b6fae0c301 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:42.638256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793adbd-0f23-4a36-9fb9-54b6fae0c301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:42.638578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793adbd-0f23-4a36-9fb9-54b6fae0c301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:42.640126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793adbd-0f23-4a36-9fb9-54b6fae0c301 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:42.660315 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 372/742] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:42.660315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793adbd-0f23-4a36-9fb9-54b6fae0c301 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:42.701152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:42.714950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d19bbf-cf2d-4407-91ad-e70f5e189be9 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:42.754859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b520fe3-4fd4-4612-aaee-b254745cd1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:42.755467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b520fe3-4fd4-4612-aaee-b254745cd1ed None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:42.755757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b520fe3-4fd4-4612-aaee-b254745cd1ed None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:42.756034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b520fe3-4fd4-4612-aaee-b254745cd1ed None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:42.756710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b520fe3-4fd4-4612-aaee-b254745cd1ed None admin] RBAC: Adding query filter params (name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:42.757049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b520fe3-4fd4-4612-aaee-b254745cd1ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:42.758479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b520fe3-4fd4-4612-aaee-b254745cd1ed None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:42.769289 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 371/743] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:42.770501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b520fe3-4fd4-4612-aaee-b254745cd1ed None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:42.776078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793adbd-0f23-4a36-9fb9-54b6fae0c301 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:42.814438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65978eb-d80d-4558-a651-bf8bffdbc825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:42.815052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65978eb-d80d-4558-a651-bf8bffdbc825 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:42.815332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65978eb-d80d-4558-a651-bf8bffdbc825 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:42.815606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65978eb-d80d-4558-a651-bf8bffdbc825 None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:42.816260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65978eb-d80d-4558-a651-bf8bffdbc825 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:42.817684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65978eb-d80d-4558-a651-bf8bffdbc825 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:42.835405 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 373/744] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:33:42 2022] POST /identity/v3/services => generated 240 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:33:42.836237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65978eb-d80d-4558-a651-bf8bffdbc825 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:42.857485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:42.895270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b520fe3-4fd4-4612-aaee-b254745cd1ed None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:42.956167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192993c-85f9-4158-beff-6fecc3b7de80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:42.956839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192993c-85f9-4158-beff-6fecc3b7de80 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:42.957378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192993c-85f9-4158-beff-6fecc3b7de80 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:42.957640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192993c-85f9-4158-beff-6fecc3b7de80 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:42.958298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192993c-85f9-4158-beff-6fecc3b7de80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:42.958629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192993c-85f9-4158-beff-6fecc3b7de80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:42.960818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192993c-85f9-4158-beff-6fecc3b7de80 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:42.978382 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 372/745] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:42.979498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192993c-85f9-4158-beff-6fecc3b7de80 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:42.987027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65978eb-d80d-4558-a651-bf8bffdbc825 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:42.996951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:43.035971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fb1f1-3a0f-4f4c-99a9-6f33fd1158b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:43.037043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fb1f1-3a0f-4f4c-99a9-6f33fd1158b3 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:43.037543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fb1f1-3a0f-4f4c-99a9-6f33fd1158b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:43.038030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fb1f1-3a0f-4f4c-99a9-6f33fd1158b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:43.046146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fb1f1-3a0f-4f4c-99a9-6f33fd1158b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:43.047789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fb1f1-3a0f-4f4c-99a9-6f33fd1158b3 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-c31fb1f1-3a0f-4f4c-99a9-6f33fd1158b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:43.062612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:43.064071 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:43.064996 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:43.064996 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:43.064996 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:43.064996 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:43.064996 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:43.064996 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:43.064996 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:43.064996 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:43.067449 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 374/746] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:43.068490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fb1f1-3a0f-4f4c-99a9-6f33fd1158b3 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:43.126434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192993c-85f9-4158-beff-6fecc3b7de80 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:43.159731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:43.165889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548e13b-bca0-4a84-b0bb-31f5927b4bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:43.166527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548e13b-bca0-4a84-b0bb-31f5927b4bfe None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:43.166814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548e13b-bca0-4a84-b0bb-31f5927b4bfe None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:43.167171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548e13b-bca0-4a84-b0bb-31f5927b4bfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:43.179517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548e13b-bca0-4a84-b0bb-31f5927b4bfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:43.181139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548e13b-bca0-4a84-b0bb-31f5927b4bfe None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-c548e13b-bca0-4a84-b0bb-31f5927b4bfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:43.197958 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:43.198922 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:43.199794 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:43.199794 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:43.199794 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:43.199794 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 373/747] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:43.200452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548e13b-bca0-4a84-b0bb-31f5927b4bfe None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:43.204998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fb1f1-3a0f-4f4c-99a9-6f33fd1158b3 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:43.251079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392c2e8-dcde-4290-a2f9-ddabf5946631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:43.251821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392c2e8-dcde-4290-a2f9-ddabf5946631 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:43.252118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392c2e8-dcde-4290-a2f9-ddabf5946631 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:43.252402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392c2e8-dcde-4290-a2f9-ddabf5946631 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:43.261042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392c2e8-dcde-4290-a2f9-ddabf5946631 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:43.262595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392c2e8-dcde-4290-a2f9-ddabf5946631 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:43.270502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-5392c2e8-dcde-4290-a2f9-ddabf5946631 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:43.276577 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:43.278341 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:43.279668 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:43.279668 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:43.279668 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:43.279668 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:43.279668 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:43.279668 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:43.279668 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:43.279668 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:43.279668 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 20:33:43.279668 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:43.282786 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 375/748] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:43.283810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392c2e8-dcde-4290-a2f9-ddabf5946631 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:43.313987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548e13b-bca0-4a84-b0bb-31f5927b4bfe None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:43.351011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031f0be-4550-4239-b50e-c42020db5c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:43.351630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031f0be-4550-4239-b50e-c42020db5c17 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:43.351915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031f0be-4550-4239-b50e-c42020db5c17 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:43.352194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031f0be-4550-4239-b50e-c42020db5c17 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:43.352888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031f0be-4550-4239-b50e-c42020db5c17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:43.353219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031f0be-4550-4239-b50e-c42020db5c17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:43.354973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031f0be-4550-4239-b50e-c42020db5c17 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:43.372759 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 374/749] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:43.373790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031f0be-4550-4239-b50e-c42020db5c17 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:43.407911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392c2e8-dcde-4290-a2f9-ddabf5946631 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:43.442894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:43.452433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba009fdf-7886-41c8-b26f-d44d42821b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:43.453413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba009fdf-7886-41c8-b26f-d44d42821b0e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:43.454468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba009fdf-7886-41c8-b26f-d44d42821b0e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:43.454966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba009fdf-7886-41c8-b26f-d44d42821b0e None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:43.455842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba009fdf-7886-41c8-b26f-d44d42821b0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:43.456369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba009fdf-7886-41c8-b26f-d44d42821b0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:43.458304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba009fdf-7886-41c8-b26f-d44d42821b0e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:43.472247 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 376/750] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:43.473427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba009fdf-7886-41c8-b26f-d44d42821b0e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:43.485750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031f0be-4550-4239-b50e-c42020db5c17 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:43.509911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:43.523488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb136c-ae7a-42bc-84b0-ab8e348f3a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:43.524156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb136c-ae7a-42bc-84b0-ab8e348f3a28 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:43.524447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb136c-ae7a-42bc-84b0-ab8e348f3a28 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:43.524727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb136c-ae7a-42bc-84b0-ab8e348f3a28 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:43.525605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb136c-ae7a-42bc-84b0-ab8e348f3a28 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:43.525917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb136c-ae7a-42bc-84b0-ab8e348f3a28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:43.527473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb136c-ae7a-42bc-84b0-ab8e348f3a28 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:43.546865 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 375/751] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Sep 3 20:33:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:43.547882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb136c-ae7a-42bc-84b0-ab8e348f3a28 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:43.585286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba009fdf-7886-41c8-b26f-d44d42821b0e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:43.623802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b35d12-9fc0-486c-a504-9c79e9dd2fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:43.624879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b35d12-9fc0-486c-a504-9c79e9dd2fce None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:43.625400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b35d12-9fc0-486c-a504-9c79e9dd2fce None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:43.625936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b35d12-9fc0-486c-a504-9c79e9dd2fce None admin] PATH_INFO: `/v3/projects/1e040b2fca724c58a2660441a72cd350/users/a56702329eae48819121dcb58f7e3070/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:43.628157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b35d12-9fc0-486c-a504-9c79e9dd2fce None admin] RBAC: Authorizing `identity:create_grant(project_id=1e040b2fca724c58a2660441a72cd350, user_id=a56702329eae48819121dcb58f7e3070, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:43.630035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b35d12-9fc0-486c-a504-9c79e9dd2fce None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:43.646378 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 377/752] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:33:43 2022] PUT /identity/v3/projects/1e040b2fca724c58a2660441a72cd350/users/a56702329eae48819121dcb58f7e3070/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:33:43.647552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b35d12-9fc0-486c-a504-9c79e9dd2fce None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:43.671551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb136c-ae7a-42bc-84b0-ab8e348f3a28 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:43.705435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:43.755051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70899ec7-97ff-4ae8-8e6e-6b696f0332ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:43.755705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70899ec7-97ff-4ae8-8e6e-6b696f0332ca None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:43.755993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70899ec7-97ff-4ae8-8e6e-6b696f0332ca None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:43.756273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70899ec7-97ff-4ae8-8e6e-6b696f0332ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:43.768269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b35d12-9fc0-486c-a504-9c79e9dd2fce None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:43.769213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70899ec7-97ff-4ae8-8e6e-6b696f0332ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:43.770902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70899ec7-97ff-4ae8-8e6e-6b696f0332ca None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-70899ec7-97ff-4ae8-8e6e-6b696f0332ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:43.794999 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:43.796061 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:43.797031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:43.797031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:43.797031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:43.797031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:43.797031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:43.797031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:43.797031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:43.797031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:43.797031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 20:33:43.797031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:43.801267 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 376/753] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 3 20:33:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:43.802458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70899ec7-97ff-4ae8-8e6e-6b696f0332ca None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:43.807900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab33d4a-6d11-4520-942c-faa2d2b77437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:43.808526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab33d4a-6d11-4520-942c-faa2d2b77437 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:43.808875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab33d4a-6d11-4520-942c-faa2d2b77437 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:43.809190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab33d4a-6d11-4520-942c-faa2d2b77437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:43.885513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab33d4a-6d11-4520-942c-faa2d2b77437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:43.886181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:43.886919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab33d4a-6d11-4520-942c-faa2d2b77437 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:43.937985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70899ec7-97ff-4ae8-8e6e-6b696f0332ca None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:43.939497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:43.978885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fa13e-3b9e-4e27-ace4-0559f16e98c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:43.979516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fa13e-3b9e-4e27-ace4-0559f16e98c0 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:43.979806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fa13e-3b9e-4e27-ace4-0559f16e98c0 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:43.980085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fa13e-3b9e-4e27-ace4-0559f16e98c0 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:43.981000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4fa13e-3b9e-4e27-ace4-0559f16e98c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:43.981316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4fa13e-3b9e-4e27-ace4-0559f16e98c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:43.982926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4fa13e-3b9e-4e27-ace4-0559f16e98c0 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:43.999675 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 378/754] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:33:44.003337 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 377/755] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Sep 3 20:33:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:44.008656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab33d4a-6d11-4520-942c-faa2d2b77437 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:44.010120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fa13e-3b9e-4e27-ace4-0559f16e98c0 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:44.143182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:44.149128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fa13e-3b9e-4e27-ace4-0559f16e98c0 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:44.157960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab33d4a-6d11-4520-942c-faa2d2b77437 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:44.194229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67d4bb-79ba-4dd0-aa12-ccc38becd392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:44.194868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67d4bb-79ba-4dd0-aa12-ccc38becd392 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:44.195153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67d4bb-79ba-4dd0-aa12-ccc38becd392 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:44.195435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67d4bb-79ba-4dd0-aa12-ccc38becd392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:44.206773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67d4bb-79ba-4dd0-aa12-ccc38becd392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:44.208508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67d4bb-79ba-4dd0-aa12-ccc38becd392 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:44.213215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94228ade-c9bf-4876-b823-a831462845c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:44.213841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94228ade-c9bf-4876-b823-a831462845c1 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:44.214130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94228ade-c9bf-4876-b823-a831462845c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:44.214407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94228ade-c9bf-4876-b823-a831462845c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-af67d4bb-79ba-4dd0-aa12-ccc38becd392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:44.236450 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:44.237453 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:44.238331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:44.238331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:44.238331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:44.238331 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 378/756] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:44.243282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67d4bb-79ba-4dd0-aa12-ccc38becd392 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:44.253847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:44.342108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94228ade-c9bf-4876-b823-a831462845c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:44.343615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94228ade-c9bf-4876-b823-a831462845c1 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:44.429012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67d4bb-79ba-4dd0-aa12-ccc38becd392 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:44.491361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf0120-ba3e-486c-a1fd-1b37ecd61099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:44.491984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf0120-ba3e-486c-a1fd-1b37ecd61099 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:44.492275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf0120-ba3e-486c-a1fd-1b37ecd61099 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:44.492556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf0120-ba3e-486c-a1fd-1b37ecd61099 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:44.493251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf0120-ba3e-486c-a1fd-1b37ecd61099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:44.493564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf0120-ba3e-486c-a1fd-1b37ecd61099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:44.494959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf0120-ba3e-486c-a1fd-1b37ecd61099 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:44.511608 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 379/757] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 20:33:44.515048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94228ade-c9bf-4876-b823-a831462845c1 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:44.518058 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 379/758] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:44.523218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf0120-ba3e-486c-a1fd-1b37ecd61099 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:44.544326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:44.598998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:44.648319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94228ade-c9bf-4876-b823-a831462845c1 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:44.686069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb0c40-f841-461f-a95c-2add797fecb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:44.686620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb0c40-f841-461f-a95c-2add797fecb1 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:44.686913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb0c40-f841-461f-a95c-2add797fecb1 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:44.687197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb0c40-f841-461f-a95c-2add797fecb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:44.697724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb0c40-f841-461f-a95c-2add797fecb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:44.699177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb0c40-f841-461f-a95c-2add797fecb1 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-c4fb0c40-f841-461f-a95c-2add797fecb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:44.714641 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:44.715616 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:44.716492 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:44.716492 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:44.716492 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:44.716492 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 380/759] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:44.721907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb0c40-f841-461f-a95c-2add797fecb1 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:44.742687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf0120-ba3e-486c-a1fd-1b37ecd61099 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:44.812690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af2889-df0b-4624-bdc6-26493231b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:44.813914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af2889-df0b-4624-bdc6-26493231b048 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:44.814202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af2889-df0b-4624-bdc6-26493231b048 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:44.814480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af2889-df0b-4624-bdc6-26493231b048 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:44.830207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af2889-df0b-4624-bdc6-26493231b048 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:44.831735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af2889-df0b-4624-bdc6-26493231b048 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a6af2889-df0b-4624-bdc6-26493231b048 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:44.858163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:44.859134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:44.860572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:44.860572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:44.860572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:44.860572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:44.860572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:44.860572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:44.860572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:44.860572 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:44.870904 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 380/760] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:44.871875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af2889-df0b-4624-bdc6-26493231b048 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:44.918026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:44.922036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb0c40-f841-461f-a95c-2add797fecb1 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:44.981613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14154477-f563-4942-9710-099718804ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:44.982243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14154477-f563-4942-9710-099718804ea7 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:44.982541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14154477-f563-4942-9710-099718804ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:44.982827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14154477-f563-4942-9710-099718804ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:44.983475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14154477-f563-4942-9710-099718804ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:44.983790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14154477-f563-4942-9710-099718804ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:44.985722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14154477-f563-4942-9710-099718804ea7 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:44.991103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:45.015400 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 381/761] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:45.025880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14154477-f563-4942-9710-099718804ea7 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:45.038586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af2889-df0b-4624-bdc6-26493231b048 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:45.079516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7ad6a-3174-45c8-9aeb-240dd37d2bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:45.080597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7ad6a-3174-45c8-9aeb-240dd37d2bb7 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:45.081172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7ad6a-3174-45c8-9aeb-240dd37d2bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:45.081480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7ad6a-3174-45c8-9aeb-240dd37d2bb7 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:45.082700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7ad6a-3174-45c8-9aeb-240dd37d2bb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:45.083026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7ad6a-3174-45c8-9aeb-240dd37d2bb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:45.085231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7ad6a-3174-45c8-9aeb-240dd37d2bb7 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:45.102873 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 381/762] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:45.104533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02435d9e-539b-4ac0-b57c-82d033d9e7cf None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:45.104826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02435d9e-539b-4ac0-b57c-82d033d9e7cf None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:45.105132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02435d9e-539b-4ac0-b57c-82d033d9e7cf None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:45.107194 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 382/763] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:45.109323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02435d9e-539b-4ac0-b57c-82d033d9e7cf None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:45.178754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14154477-f563-4942-9710-099718804ea7 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:45.216783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35541b05-d51e-4ed5-ad07-6505faf81eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:45.217709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35541b05-d51e-4ed5-ad07-6505faf81eed None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:45.218214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35541b05-d51e-4ed5-ad07-6505faf81eed None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:45.218676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35541b05-d51e-4ed5-ad07-6505faf81eed None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:45.226403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35541b05-d51e-4ed5-ad07-6505faf81eed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:45.228072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35541b05-d51e-4ed5-ad07-6505faf81eed None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:45.239193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02435d9e-539b-4ac0-b57c-82d033d9e7cf None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-35541b05-d51e-4ed5-ad07-6505faf81eed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:45.240757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:45.241637 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:45.242523 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:45.242523 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:45.242523 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:45.242523 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:45.242523 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:45.242523 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:45.242523 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:45.242523 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:45.245084 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 382/764] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:45.246494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242b43c-1945-4f8c-908b-6dec0c640069 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:45.246974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242b43c-1945-4f8c-908b-6dec0c640069 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:45.247433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242b43c-1945-4f8c-908b-6dec0c640069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:45.266066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6242b43c-1945-4f8c-908b-6dec0c640069 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:45.267706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6242b43c-1945-4f8c-908b-6dec0c640069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:45.280278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128821ed-4c68-4485-bc29-d7369c94ee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:45.281057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128821ed-4c68-4485-bc29-d7369c94ee16 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:45.281408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128821ed-4c68-4485-bc29-d7369c94ee16 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:45.281754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128821ed-4c68-4485-bc29-d7369c94ee16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:45.283226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128821ed-4c68-4485-bc29-d7369c94ee16 None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=83865463b2a84f99a136b73dffeeb378) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:45.283602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128821ed-4c68-4485-bc29-d7369c94ee16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:45.284560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:45.286416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128821ed-4c68-4485-bc29-d7369c94ee16 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:45.297148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:45.297471 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 383/765] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:45 2022] GET /identity/v3/role_assignments?user.id=83865463b2a84f99a136b73dffeeb378&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=bc768522dc984239be4a1b39a45ab437 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:45.299894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128821ed-4c68-4485-bc29-d7369c94ee16 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:45.372134 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 383/766] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:33:45.373666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c07a0d-4cb4-4ffa-bd29-0e5a7f68c555 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:45.373962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c07a0d-4cb4-4ffa-bd29-0e5a7f68c555 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:45.374247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c07a0d-4cb4-4ffa-bd29-0e5a7f68c555 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:45.379822 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 384/767] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:45.381461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a21609-75dd-44ea-96cf-ff68328a3d71 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:45.381757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a21609-75dd-44ea-96cf-ff68328a3d71 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:45.382042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a21609-75dd-44ea-96cf-ff68328a3d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:45.403886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d5a21609-75dd-44ea-96cf-ff68328a3d71 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:45.405349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d5a21609-75dd-44ea-96cf-ff68328a3d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:45.499067 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 385/768] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:33:45.508748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128821ed-4c68-4485-bc29-d7369c94ee16 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:45.545170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683aa99-5545-493a-a5ba-7e6cc589827a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:45.545993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683aa99-5545-493a-a5ba-7e6cc589827a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:45.546448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683aa99-5545-493a-a5ba-7e6cc589827a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:45.546924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683aa99-5545-493a-a5ba-7e6cc589827a None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:45.548303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683aa99-5545-493a-a5ba-7e6cc589827a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:45.548754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683aa99-5545-493a-a5ba-7e6cc589827a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:45.550406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683aa99-5545-493a-a5ba-7e6cc589827a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:45.562565 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 384/769] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:45.569393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a21609-75dd-44ea-96cf-ff68328a3d71 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:45.691083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:45.714992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a21609-75dd-44ea-96cf-ff68328a3d71 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:45.756462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49428715-17af-4508-a720-97740ca22ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:45.757012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49428715-17af-4508-a720-97740ca22ce5 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:45.757296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49428715-17af-4508-a720-97740ca22ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:45.757574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49428715-17af-4508-a720-97740ca22ce5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:45.758307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49428715-17af-4508-a720-97740ca22ce5 None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=9388372667f8454498b2e4480187b47c) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:45.758618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49428715-17af-4508-a720-97740ca22ce5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:45.760047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49428715-17af-4508-a720-97740ca22ce5 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:45.765798 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 386/770] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:45 2022] GET /identity/v3/role_assignments?user.id=9388372667f8454498b2e4480187b47c&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:45.782775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683aa99-5545-493a-a5ba-7e6cc589827a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:45.875263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:45.941910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683aa99-5545-493a-a5ba-7e6cc589827a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:45.947788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49428715-17af-4508-a720-97740ca22ce5 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:46.003592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e957d-8fc8-4851-929e-5827ee338347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:46.004750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e957d-8fc8-4851-929e-5827ee338347 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:46.005071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e957d-8fc8-4851-929e-5827ee338347 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:46.005354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e957d-8fc8-4851-929e-5827ee338347 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:46.005959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e957d-8fc8-4851-929e-5827ee338347 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:46.007542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e957d-8fc8-4851-929e-5827ee338347 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-736e957d-8fc8-4851-929e-5827ee338347 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:46.022933 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:46.023942 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 20:33:46.024753 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:46.026123 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 385/771] 10.222.0.16 () {56 vars in 1129 bytes} [Sat Sep 3 20:33:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:46.030292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e957d-8fc8-4851-929e-5827ee338347 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:46.111332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49428715-17af-4508-a720-97740ca22ce5 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:46.177508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b7bef-6924-48bb-8da1-c817944734ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:46.178163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b7bef-6924-48bb-8da1-c817944734ea None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:46.178451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b7bef-6924-48bb-8da1-c817944734ea None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:46.178729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b7bef-6924-48bb-8da1-c817944734ea None admin] PATH_INFO: `/v3/services/network` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:46.179325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b7bef-6924-48bb-8da1-c817944734ea None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:46.181580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b7bef-6924-48bb-8da1-c817944734ea None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-d14b7bef-6924-48bb-8da1-c817944734ea None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:46.196273 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:46.197317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:46.198143 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 387/772] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:45 2022] GET /identity/v3/services/network => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:46.206152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b7bef-6924-48bb-8da1-c817944734ea None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:46.209146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e957d-8fc8-4851-929e-5827ee338347 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:46.224396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:46.229530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:46.252018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66277102-9c51-4905-8e47-9381263078b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:46.252763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66277102-9c51-4905-8e47-9381263078b1 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:46.253190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66277102-9c51-4905-8e47-9381263078b1 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:46.253575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66277102-9c51-4905-8e47-9381263078b1 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:46.254517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66277102-9c51-4905-8e47-9381263078b1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:46.254832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66277102-9c51-4905-8e47-9381263078b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:46.256533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66277102-9c51-4905-8e47-9381263078b1 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:46.262299 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 386/773] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Sep 3 20:33:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:46.268461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66277102-9c51-4905-8e47-9381263078b1 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:46.369613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b7bef-6924-48bb-8da1-c817944734ea None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:46.439929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f965663-decc-46b9-bff7-0f5f26ec8f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:46.441526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f965663-decc-46b9-bff7-0f5f26ec8f1f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:46.442204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f965663-decc-46b9-bff7-0f5f26ec8f1f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:46.443347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f965663-decc-46b9-bff7-0f5f26ec8f1f None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:46.444201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f965663-decc-46b9-bff7-0f5f26ec8f1f None admin] RBAC: Adding query filter params (name=network) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:46.444861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f965663-decc-46b9-bff7-0f5f26ec8f1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:46.447376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f965663-decc-46b9-bff7-0f5f26ec8f1f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:46.455680 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 388/774] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:33:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:46.463121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66277102-9c51-4905-8e47-9381263078b1 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:46.464230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f965663-decc-46b9-bff7-0f5f26ec8f1f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:46.518354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b486f-6ebe-4d26-b1c8-a5782c0a44c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:46.519300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b486f-6ebe-4d26-b1c8-a5782c0a44c7 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:46.521128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b486f-6ebe-4d26-b1c8-a5782c0a44c7 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:46.521651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b486f-6ebe-4d26-b1c8-a5782c0a44c7 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:46.523407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b486f-6ebe-4d26-b1c8-a5782c0a44c7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:46.524595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b486f-6ebe-4d26-b1c8-a5782c0a44c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:46.526170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b486f-6ebe-4d26-b1c8-a5782c0a44c7 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:46.540865 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 387/775] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Sep 3 20:33:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:46.553831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b486f-6ebe-4d26-b1c8-a5782c0a44c7 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:46.593462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:46.627961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:46.630869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f965663-decc-46b9-bff7-0f5f26ec8f1f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:46.675708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609fd4f-d166-4339-8287-0c1a48d6647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:46.676575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609fd4f-d166-4339-8287-0c1a48d6647f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:46.677079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609fd4f-d166-4339-8287-0c1a48d6647f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:46.677510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609fd4f-d166-4339-8287-0c1a48d6647f None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:46.678345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609fd4f-d166-4339-8287-0c1a48d6647f None admin] RBAC: Adding query filter params (type=network) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:46.678914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609fd4f-d166-4339-8287-0c1a48d6647f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:46.680616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609fd4f-d166-4339-8287-0c1a48d6647f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:46.686836 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 389/776] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:33:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:46.688551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577e0d6-8b8a-456f-927b-21939a0f9263 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:46.689039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577e0d6-8b8a-456f-927b-21939a0f9263 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:46.689488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577e0d6-8b8a-456f-927b-21939a0f9263 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:46.691365 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 390/777] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:46.696956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577e0d6-8b8a-456f-927b-21939a0f9263 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:46.702112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b486f-6ebe-4d26-b1c8-a5782c0a44c7 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:46.740043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac936e3-76cc-446f-8231-0ee3f88b72a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:46.741106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac936e3-76cc-446f-8231-0ee3f88b72a4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:46.741600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac936e3-76cc-446f-8231-0ee3f88b72a4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:46.742266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac936e3-76cc-446f-8231-0ee3f88b72a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:46.743469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac936e3-76cc-446f-8231-0ee3f88b72a4 None admin] RBAC: Adding query filter params (interface=public, service_id=52fbe4e489c24de4afb00d3620e33ed4, region_id=RegionOne) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:46.744964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac936e3-76cc-446f-8231-0ee3f88b72a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:46.746560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac936e3-76cc-446f-8231-0ee3f88b72a4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:46.761364 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 388/778] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 3 20:33:46 2022] GET /identity/v3/endpoints?service_id=52fbe4e489c24de4afb00d3620e33ed4&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:46.763083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa9eb2-6bf8-4a43-84cc-d09c21b77464 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:46.763372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa9eb2-6bf8-4a43-84cc-d09c21b77464 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:46.763669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa9eb2-6bf8-4a43-84cc-d09c21b77464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:46.782999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-29aa9eb2-6bf8-4a43-84cc-d09c21b77464 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:46.784124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-29aa9eb2-6bf8-4a43-84cc-d09c21b77464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:46.825745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577e0d6-8b8a-456f-927b-21939a0f9263 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:46.861153 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 389/779] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:33:46.862174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa9eb2-6bf8-4a43-84cc-d09c21b77464 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:46.866090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3898f-097f-4d20-b3eb-e81cab698488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:46.866941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3898f-097f-4d20-b3eb-e81cab698488 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:46.867554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3898f-097f-4d20-b3eb-e81cab698488 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:46.868038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3898f-097f-4d20-b3eb-e81cab698488 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:46.869040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3898f-097f-4d20-b3eb-e81cab698488 None admin] RBAC: Adding query filter params (interface=public, service_id=ea08da3e0654497c9b05aea727fa703d, region_id=RegionOne) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:46.869676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3898f-097f-4d20-b3eb-e81cab698488 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:46.871435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3898f-097f-4d20-b3eb-e81cab698488 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:46.877489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:46.882318 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 391/780] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 3 20:33:46 2022] GET /identity/v3/endpoints?service_id=ea08da3e0654497c9b05aea727fa703d&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:46.888941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3898f-097f-4d20-b3eb-e81cab698488 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:46.930774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:47.000011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3898f-097f-4d20-b3eb-e81cab698488 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:47.031936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa9eb2-6bf8-4a43-84cc-d09c21b77464 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:47.043570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0a971-f708-4c57-a3f1-e3dde14a2f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:47.044440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0a971-f708-4c57-a3f1-e3dde14a2f6f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:47.044930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0a971-f708-4c57-a3f1-e3dde14a2f6f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:47.045389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0a971-f708-4c57-a3f1-e3dde14a2f6f None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:47.047868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0a971-f708-4c57-a3f1-e3dde14a2f6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:47.049466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0a971-f708-4c57-a3f1-e3dde14a2f6f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:47.064990 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 392/781] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 3 20:33:46 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:47.105486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c6a5a-2c69-477b-8a4d-6a075fe69885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:47.106496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c6a5a-2c69-477b-8a4d-6a075fe69885 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:47.106995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c6a5a-2c69-477b-8a4d-6a075fe69885 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:47.107457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c6a5a-2c69-477b-8a4d-6a075fe69885 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:47.108289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c6a5a-2c69-477b-8a4d-6a075fe69885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:47.110476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c6a5a-2c69-477b-8a4d-6a075fe69885 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:47.122569 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 390/782] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 3 20:33:46 2022] GET /identity/v3/services => generated 772 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:47.336369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0a971-f708-4c57-a3f1-e3dde14a2f6f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:47.410754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:47.461068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0a971-f708-4c57-a3f1-e3dde14a2f6f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:47.499730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de0b0c-af6c-4a0d-a232-645c00ea2a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:47.500416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0b0c-af6c-4a0d-a232-645c00ea2a11 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:47.501268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0b0c-af6c-4a0d-a232-645c00ea2a11 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:47.501553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0b0c-af6c-4a0d-a232-645c00ea2a11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:47.512321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de0b0c-af6c-4a0d-a232-645c00ea2a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-61de0b0c-af6c-4a0d-a232-645c00ea2a11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:47.518297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:47.519212 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:47.520020 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 393/783] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:47.527075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c6a5a-2c69-477b-8a4d-6a075fe69885 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:47.679935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c6a5a-2c69-477b-8a4d-6a075fe69885 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:47.683125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:47.730598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e598d8d-6ad0-4998-8859-860a8d7ff56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:47.731238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e598d8d-6ad0-4998-8859-860a8d7ff56b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:47.731530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e598d8d-6ad0-4998-8859-860a8d7ff56b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:47.731807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e598d8d-6ad0-4998-8859-860a8d7ff56b None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:47.732498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e598d8d-6ad0-4998-8859-860a8d7ff56b None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:47.732810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e598d8d-6ad0-4998-8859-860a8d7ff56b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:47.734228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e598d8d-6ad0-4998-8859-860a8d7ff56b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:47.748734 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 391/784] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:47.754640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de0b0c-af6c-4a0d-a232-645c00ea2a11 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:47.884563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39133579-7fac-4dfb-a51a-5701868e04d6 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:47.884834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39133579-7fac-4dfb-a51a-5701868e04d6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:47.885252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39133579-7fac-4dfb-a51a-5701868e04d6 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:47.886768 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 392/785] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:33:47.891981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb869a9f-e330-4485-bdb2-5a94df93a183 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:47.892280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb869a9f-e330-4485-bdb2-5a94df93a183 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:47.892564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb869a9f-e330-4485-bdb2-5a94df93a183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:47.901817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de0b0c-af6c-4a0d-a232-645c00ea2a11 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:47.915282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fb869a9f-e330-4485-bdb2-5a94df93a183 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:47.916515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fb869a9f-e330-4485-bdb2-5a94df93a183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:47.940447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:47.952983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefca40-29b5-4a22-a2e2-f750ce60db67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:47.953636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefca40-29b5-4a22-a2e2-f750ce60db67 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:47.953961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefca40-29b5-4a22-a2e2-f750ce60db67 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:47.954244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefca40-29b5-4a22-a2e2-f750ce60db67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:47.970200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefca40-29b5-4a22-a2e2-f750ce60db67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:47.972213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefca40-29b5-4a22-a2e2-f750ce60db67 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-2aefca40-29b5-4a22-a2e2-f750ce60db67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:47.999717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:48.001550 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:48.002817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:48.002817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:48.002817 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:48.006030 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 394/786] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:48.006872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefca40-29b5-4a22-a2e2-f750ce60db67 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:48.025313 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 393/787] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:33:48.176593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefca40-29b5-4a22-a2e2-f750ce60db67 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:48.224789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e114ed-d84b-4bdd-acc0-adf2f89518fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:48.225364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e114ed-d84b-4bdd-acc0-adf2f89518fd None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:48.225651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e114ed-d84b-4bdd-acc0-adf2f89518fd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:48.225929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e114ed-d84b-4bdd-acc0-adf2f89518fd None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:48.226603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e114ed-d84b-4bdd-acc0-adf2f89518fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:48.226911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e114ed-d84b-4bdd-acc0-adf2f89518fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:48.228331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e114ed-d84b-4bdd-acc0-adf2f89518fd None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:48.245388 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 395/788] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:48.251848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb869a9f-e330-4485-bdb2-5a94df93a183 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:48.360254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:48.370555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb869a9f-e330-4485-bdb2-5a94df93a183 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:48.392581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e114ed-d84b-4bdd-acc0-adf2f89518fd None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:48.412370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceba312-8f05-4697-8d1c-d051b9471fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:48.413152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceba312-8f05-4697-8d1c-d051b9471fcc None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:48.413443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceba312-8f05-4697-8d1c-d051b9471fcc None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:48.413722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceba312-8f05-4697-8d1c-d051b9471fcc None admin] PATH_INFO: `/v3/users/nova` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:48.421419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceba312-8f05-4697-8d1c-d051b9471fcc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:48.423510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceba312-8f05-4697-8d1c-d051b9471fcc None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:48.432534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-3ceba312-8f05-4697-8d1c-d051b9471fcc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:48.436474 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:48.438943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:48.439894 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:48.439894 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:48.439894 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:48.439894 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:48.439894 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:48.439894 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:48.439894 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:48.439894 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:48.439894 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:33:48.439894 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:48.442290 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 394/789] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 3 20:33:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:48.443532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db383f3-9d67-48c1-8b6b-78a8d5871d44 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:48.443829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db383f3-9d67-48c1-8b6b-78a8d5871d44 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:48.444112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db383f3-9d67-48c1-8b6b-78a8d5871d44 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:48.446905 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 395/790] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:48.447631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db383f3-9d67-48c1-8b6b-78a8d5871d44 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:48.527630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e114ed-d84b-4bdd-acc0-adf2f89518fd None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:48.564362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285edf9-f92c-4996-b29f-1c405509a0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:48.565035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285edf9-f92c-4996-b29f-1c405509a0a1 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:48.565325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285edf9-f92c-4996-b29f-1c405509a0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:48.565600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285edf9-f92c-4996-b29f-1c405509a0a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:48.571598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db383f3-9d67-48c1-8b6b-78a8d5871d44 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:48.576437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285edf9-f92c-4996-b29f-1c405509a0a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-5285edf9-f92c-4996-b29f-1c405509a0a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:48.582514 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:48.583422 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:48.584243 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 396/791] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:48.585388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9faf5c3-a5f1-4e19-9feb-5199019edd4c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:48.586340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9faf5c3-a5f1-4e19-9feb-5199019edd4c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:48.586340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9faf5c3-a5f1-4e19-9feb-5199019edd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:48.605490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b9faf5c3-a5f1-4e19-9feb-5199019edd4c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:48.606867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b9faf5c3-a5f1-4e19-9feb-5199019edd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:48.622017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd06cf-c6cc-43a0-b4da-3968e68981c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:48.623192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd06cf-c6cc-43a0-b4da-3968e68981c5 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:48.623668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd06cf-c6cc-43a0-b4da-3968e68981c5 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:48.624370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd06cf-c6cc-43a0-b4da-3968e68981c5 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:48.626331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fd06cf-c6cc-43a0-b4da-3968e68981c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:48.627106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fd06cf-c6cc-43a0-b4da-3968e68981c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:48.629411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fd06cf-c6cc-43a0-b4da-3968e68981c5 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:48.654160 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 396/792] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 3 20:33:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:48.655825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd06cf-c6cc-43a0-b4da-3968e68981c5 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:48.677393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:48.690773 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 397/793] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:33:48.691753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9faf5c3-a5f1-4e19-9feb-5199019edd4c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:48.734715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:48.847435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd06cf-c6cc-43a0-b4da-3968e68981c5 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:48.882926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9faf5c3-a5f1-4e19-9feb-5199019edd4c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:48.886369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdba601-f053-4b6b-913c-2dd7f881ef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:48.887304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdba601-f053-4b6b-913c-2dd7f881ef37 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:48.887806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdba601-f053-4b6b-913c-2dd7f881ef37 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:48.888312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdba601-f053-4b6b-913c-2dd7f881ef37 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:48.889212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdba601-f053-4b6b-913c-2dd7f881ef37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:48.889742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdba601-f053-4b6b-913c-2dd7f881ef37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:48.891921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdba601-f053-4b6b-913c-2dd7f881ef37 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:48.901219 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 397/794] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:48.910398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdba601-f053-4b6b-913c-2dd7f881ef37 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:48.954715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79590232-735e-4429-9dcf-7fbb6014d58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:48.956612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79590232-735e-4429-9dcf-7fbb6014d58d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:48.957898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79590232-735e-4429-9dcf-7fbb6014d58d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:48.958421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79590232-735e-4429-9dcf-7fbb6014d58d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:49.042667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79590232-735e-4429-9dcf-7fbb6014d58d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:49.044611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79590232-735e-4429-9dcf-7fbb6014d58d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:49.074555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:49.095175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:49.129175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdba601-f053-4b6b-913c-2dd7f881ef37 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:49.168486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe22f44-9255-4b84-91f3-0e024a70298c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:49.169392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe22f44-9255-4b84-91f3-0e024a70298c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:49.169750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe22f44-9255-4b84-91f3-0e024a70298c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:49.170053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe22f44-9255-4b84-91f3-0e024a70298c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:49.177182 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 398/795] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:33:49.178741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79590232-735e-4429-9dcf-7fbb6014d58d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:49.182209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe22f44-9255-4b84-91f3-0e024a70298c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:49.183865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe22f44-9255-4b84-91f3-0e024a70298c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-6fe22f44-9255-4b84-91f3-0e024a70298c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:49.198943 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:49.200143 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:49.201272 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:49.201272 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:49.201272 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:49.202664 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 398/796] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:49.203718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe22f44-9255-4b84-91f3-0e024a70298c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:49.309012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79590232-735e-4429-9dcf-7fbb6014d58d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:49.312737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe22f44-9255-4b84-91f3-0e024a70298c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:49.341302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:49.344398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:49.356565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf56bc8-081a-4079-89bb-69d22320f8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:49.357238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf56bc8-081a-4079-89bb-69d22320f8e8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:49.357526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf56bc8-081a-4079-89bb-69d22320f8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:49.357806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf56bc8-081a-4079-89bb-69d22320f8e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:49.360879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8461b9-33e0-4812-8f49-0e219620debe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:49.361825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8461b9-33e0-4812-8f49-0e219620debe None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:49.362305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8461b9-33e0-4812-8f49-0e219620debe None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:49.362773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8461b9-33e0-4812-8f49-0e219620debe None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:49.369657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf56bc8-081a-4079-89bb-69d22320f8e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:49.371126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf56bc8-081a-4079-89bb-69d22320f8e8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:49.375058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8461b9-33e0-4812-8f49-0e219620debe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-2d8461b9-33e0-4812-8f49-0e219620debe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:49.381108 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:49.382045 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:49.382867 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:49.382867 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:49.382867 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:49.382867 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:49.382867 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:49.382867 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:49.382867 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:49.382867 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:49.382867 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:49.382867 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:49.382867 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:49.382867 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:49.385469 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 399/797] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-9cf56bc8-081a-4079-89bb-69d22320f8e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:49.386442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:49.387346 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:49.388262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:49.388262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:49.388262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:49.388978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8461b9-33e0-4812-8f49-0e219620debe None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:49.393063 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 399/798] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:49.393981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf56bc8-081a-4079-89bb-69d22320f8e8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:49.498749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf56bc8-081a-4079-89bb-69d22320f8e8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:49.506194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8461b9-33e0-4812-8f49-0e219620debe None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:49.534938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f6828-bc59-493e-8d3c-040409888d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:49.535775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f6828-bc59-493e-8d3c-040409888d49 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:49.535886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f6828-bc59-493e-8d3c-040409888d49 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:49.536102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f6828-bc59-493e-8d3c-040409888d49 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:49.536772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f6828-bc59-493e-8d3c-040409888d49 None admin] RBAC: Adding query filter params (name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:49.537193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f6828-bc59-493e-8d3c-040409888d49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:49.538512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f6828-bc59-493e-8d3c-040409888d49 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:49.549105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722a0bc-67da-4f9c-8b9e-397a6957a31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:49.550004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722a0bc-67da-4f9c-8b9e-397a6957a31e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:49.550500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722a0bc-67da-4f9c-8b9e-397a6957a31e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:49.550971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722a0bc-67da-4f9c-8b9e-397a6957a31e None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:49.551576 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 400/799] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:49.552389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722a0bc-67da-4f9c-8b9e-397a6957a31e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:49.552495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f6828-bc59-493e-8d3c-040409888d49 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:49.553204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722a0bc-67da-4f9c-8b9e-397a6957a31e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:49.554804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722a0bc-67da-4f9c-8b9e-397a6957a31e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:49.567981 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 400/800] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:49.569293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722a0bc-67da-4f9c-8b9e-397a6957a31e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:49.598348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:49.624008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:49.658889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f6828-bc59-493e-8d3c-040409888d49 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:49.684422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722a0bc-67da-4f9c-8b9e-397a6957a31e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:49.694452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a173f-611b-44ee-8102-8eaf1af294ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:49.695342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a173f-611b-44ee-8102-8eaf1af294ec None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:49.695342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a173f-611b-44ee-8102-8eaf1af294ec None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:49.695618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a173f-611b-44ee-8102-8eaf1af294ec None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:49.696265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a173f-611b-44ee-8102-8eaf1af294ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:49.696575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a173f-611b-44ee-8102-8eaf1af294ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:49.698051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a173f-611b-44ee-8102-8eaf1af294ec None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:49.713807 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 401/801] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:49.714726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a173f-611b-44ee-8102-8eaf1af294ec None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:49.728858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754411be-834d-4c8b-971b-8cd7ca2a60e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:49.729968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754411be-834d-4c8b-971b-8cd7ca2a60e6 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:49.730465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754411be-834d-4c8b-971b-8cd7ca2a60e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:49.730971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754411be-834d-4c8b-971b-8cd7ca2a60e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:49.744255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754411be-834d-4c8b-971b-8cd7ca2a60e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:49.746125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754411be-834d-4c8b-971b-8cd7ca2a60e6 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-754411be-834d-4c8b-971b-8cd7ca2a60e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:49.762356 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:49.763348 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:49.764220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:49.764220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:49.764220 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:49.766831 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 401/802] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:49.767832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754411be-834d-4c8b-971b-8cd7ca2a60e6 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:49.835954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a173f-611b-44ee-8102-8eaf1af294ec None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:49.838133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:49.877554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:49.882241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7fd06-9cbd-4306-a22c-77c8f1d2a50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:49.882982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7fd06-9cbd-4306-a22c-77c8f1d2a50e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:49.883271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7fd06-9cbd-4306-a22c-77c8f1d2a50e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:49.883609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7fd06-9cbd-4306-a22c-77c8f1d2a50e None admin] PATH_INFO: `/v3/users/swift` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:49.883713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754411be-834d-4c8b-971b-8cd7ca2a60e6 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:49.892861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7fd06-9cbd-4306-a22c-77c8f1d2a50e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:49.894258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7fd06-9cbd-4306-a22c-77c8f1d2a50e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-5aa7fd06-9cbd-4306-a22c-77c8f1d2a50e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:49.906533 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:49.907537 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:49.908455 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:49.908455 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:49.908455 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:49.908455 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:49.908455 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:49.908455 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:49.908455 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:49.908455 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:49.908455 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 20:33:49.908455 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:49.908455 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 402/803] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:49.909110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7fd06-9cbd-4306-a22c-77c8f1d2a50e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:49.927918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01716195-dd15-4fdc-ade0-48ef4029612e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:49.928560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01716195-dd15-4fdc-ade0-48ef4029612e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:49.928851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01716195-dd15-4fdc-ade0-48ef4029612e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:49.929169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01716195-dd15-4fdc-ade0-48ef4029612e None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:49.936258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01716195-dd15-4fdc-ade0-48ef4029612e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:49.937826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01716195-dd15-4fdc-ade0-48ef4029612e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-01716195-dd15-4fdc-ade0-48ef4029612e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:49.949575 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:49.950619 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:49.951844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:49.951844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:49.951844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:49.951844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:49.951844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:49.951844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:49.951844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:49.951844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:49.951844 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 402/804] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:49.952427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736cdb6-bb8f-47dc-93a0-8c3915b57507 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:49.952427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736cdb6-bb8f-47dc-93a0-8c3915b57507 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:49.952585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736cdb6-bb8f-47dc-93a0-8c3915b57507 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:49.954695 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 403/805] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:49.955546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb0169-337b-4013-b4bc-61c4bfae86d1 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:49.955857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb0169-337b-4013-b4bc-61c4bfae86d1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:49.956380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb0169-337b-4013-b4bc-61c4bfae86d1 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:49.959060 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 404/806] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:49.959060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb0169-337b-4013-b4bc-61c4bfae86d1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:50.016842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7fd06-9cbd-4306-a22c-77c8f1d2a50e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:50.049449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea753a-6b0a-4eb1-974f-92d1ca19b7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:50.050055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea753a-6b0a-4eb1-974f-92d1ca19b7e8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:50.050342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea753a-6b0a-4eb1-974f-92d1ca19b7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:50.050616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea753a-6b0a-4eb1-974f-92d1ca19b7e8 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:50.051344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea753a-6b0a-4eb1-974f-92d1ca19b7e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:50.051654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea753a-6b0a-4eb1-974f-92d1ca19b7e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:50.053085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea753a-6b0a-4eb1-974f-92d1ca19b7e8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:50.060244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb0169-337b-4013-b4bc-61c4bfae86d1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:50.064862 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 403/807] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:50.065663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea753a-6b0a-4eb1-974f-92d1ca19b7e8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:50.089356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:50.097698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213890ef-4908-4045-8a72-89ae76f82e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:50.098298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213890ef-4908-4045-8a72-89ae76f82e87 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:50.098580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213890ef-4908-4045-8a72-89ae76f82e87 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:50.098850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213890ef-4908-4045-8a72-89ae76f82e87 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:50.099563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213890ef-4908-4045-8a72-89ae76f82e87 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:50.099870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213890ef-4908-4045-8a72-89ae76f82e87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:50.101326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213890ef-4908-4045-8a72-89ae76f82e87 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:50.118886 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 405/808] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Sep 3 20:33:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:50.120288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765a4eb-26f7-4be3-9b14-3c7acc8a1283 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:50.120580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765a4eb-26f7-4be3-9b14-3c7acc8a1283 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:50.120857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765a4eb-26f7-4be3-9b14-3c7acc8a1283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:50.140076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3765a4eb-26f7-4be3-9b14-3c7acc8a1283 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:50.141312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3765a4eb-26f7-4be3-9b14-3c7acc8a1283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:50.152462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:50.170606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea753a-6b0a-4eb1-974f-92d1ca19b7e8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:50.205407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0e3b0-6de7-42ef-82da-3f64e8cdd681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:50.206013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0e3b0-6de7-42ef-82da-3f64e8cdd681 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:50.206294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0e3b0-6de7-42ef-82da-3f64e8cdd681 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:50.206568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0e3b0-6de7-42ef-82da-3f64e8cdd681 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:50.207509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0e3b0-6de7-42ef-82da-3f64e8cdd681 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:50.207814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0e3b0-6de7-42ef-82da-3f64e8cdd681 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:50.209277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0e3b0-6de7-42ef-82da-3f64e8cdd681 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:50.209996 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 406/809] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:33:50.211209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b9da4-17c1-4d98-9edf-ecbc5ec1c661 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:50.211494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b9da4-17c1-4d98-9edf-ecbc5ec1c661 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:50.211778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b9da4-17c1-4d98-9edf-ecbc5ec1c661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:50.220993 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 404/810] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:50.221934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0e3b0-6de7-42ef-82da-3f64e8cdd681 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:50.234807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6a9b9da4-17c1-4d98-9edf-ecbc5ec1c661 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:50.236498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6a9b9da4-17c1-4d98-9edf-ecbc5ec1c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:50.304714 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 407/811] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:33:50.305879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b9da4-17c1-4d98-9edf-ecbc5ec1c661 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:50.320190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:50.346130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0e3b0-6de7-42ef-82da-3f64e8cdd681 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:50.389848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07482fa-8a70-47eb-805b-d0f71b2b946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:50.391161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:50.391982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07482fa-8a70-47eb-805b-d0f71b2b946e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:50.392413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07482fa-8a70-47eb-805b-d0f71b2b946e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:50.392829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07482fa-8a70-47eb-805b-d0f71b2b946e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:50.403597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07482fa-8a70-47eb-805b-d0f71b2b946e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:50.406113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07482fa-8a70-47eb-805b-d0f71b2b946e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-e07482fa-8a70-47eb-805b-d0f71b2b946e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:50.421380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:50.422297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:50.423740 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:50.423740 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:50.423740 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:50.423740 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:50.423740 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:50.423740 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:50.423740 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:50.423740 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:50.423740 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 20:33:50.423740 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:50.423740 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 405/812] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 3 20:33:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:50.424326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07482fa-8a70-47eb-805b-d0f71b2b946e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:50.424812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b9da4-17c1-4d98-9edf-ecbc5ec1c661 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:50.471622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273ce32-624b-4f5a-8c85-93a4284c3b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:50.472579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273ce32-624b-4f5a-8c85-93a4284c3b92 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:50.473036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273ce32-624b-4f5a-8c85-93a4284c3b92 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:50.473711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273ce32-624b-4f5a-8c85-93a4284c3b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:50.487559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273ce32-624b-4f5a-8c85-93a4284c3b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:50.489976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273ce32-624b-4f5a-8c85-93a4284c3b92 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-0273ce32-624b-4f5a-8c85-93a4284c3b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:50.513506 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:50.515015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:50.516066 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:50.516066 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:50.516066 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:50.522420 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 408/813] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:50.522420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273ce32-624b-4f5a-8c85-93a4284c3b92 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:50.539932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07482fa-8a70-47eb-805b-d0f71b2b946e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:50.577631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4511e1-8f9d-4bd4-8038-520d77255dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:50.578367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4511e1-8f9d-4bd4-8038-520d77255dc7 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:50.578691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4511e1-8f9d-4bd4-8038-520d77255dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:50.579133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4511e1-8f9d-4bd4-8038-520d77255dc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:50.580151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4511e1-8f9d-4bd4-8038-520d77255dc7 None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=a56702329eae48819121dcb58f7e3070) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:50.580474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4511e1-8f9d-4bd4-8038-520d77255dc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:50.581938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4511e1-8f9d-4bd4-8038-520d77255dc7 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:50.597329 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 406/814] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:50 2022] GET /identity/v3/role_assignments?user.id=a56702329eae48819121dcb58f7e3070&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=bc768522dc984239be4a1b39a45ab437 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:50.599598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4511e1-8f9d-4bd4-8038-520d77255dc7 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:50.613507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:50.639488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273ce32-624b-4f5a-8c85-93a4284c3b92 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:50.670870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:50.681977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26396b3a-596e-41f8-9f9e-4aab82bb9db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:50.682613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26396b3a-596e-41f8-9f9e-4aab82bb9db0 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:50.682968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26396b3a-596e-41f8-9f9e-4aab82bb9db0 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:50.683257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26396b3a-596e-41f8-9f9e-4aab82bb9db0 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:50.684005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26396b3a-596e-41f8-9f9e-4aab82bb9db0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:50.684315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26396b3a-596e-41f8-9f9e-4aab82bb9db0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:50.685840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26396b3a-596e-41f8-9f9e-4aab82bb9db0 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:50.706475 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 409/815] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Sep 3 20:33:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:50.712147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26396b3a-596e-41f8-9f9e-4aab82bb9db0 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:50.723375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4511e1-8f9d-4bd4-8038-520d77255dc7 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:50.760317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab0692-a87e-43b7-b686-b797d3e017d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:50.761068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab0692-a87e-43b7-b686-b797d3e017d4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:50.761349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab0692-a87e-43b7-b686-b797d3e017d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:50.761625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab0692-a87e-43b7-b686-b797d3e017d4 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:50.762277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab0692-a87e-43b7-b686-b797d3e017d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:50.762583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab0692-a87e-43b7-b686-b797d3e017d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:50.763954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab0692-a87e-43b7-b686-b797d3e017d4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:50.778794 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 407/816] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:50.780271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab0692-a87e-43b7-b686-b797d3e017d4 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:50.826831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26396b3a-596e-41f8-9f9e-4aab82bb9db0 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:50.849937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:50.868930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90df32-1bec-4e9c-a18e-f75f4e446d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:50.869731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90df32-1bec-4e9c-a18e-f75f4e446d79 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:50.870016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90df32-1bec-4e9c-a18e-f75f4e446d79 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:50.870366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90df32-1bec-4e9c-a18e-f75f4e446d79 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:50.871046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90df32-1bec-4e9c-a18e-f75f4e446d79 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:50.872498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90df32-1bec-4e9c-a18e-f75f4e446d79 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-fc90df32-1bec-4e9c-a18e-f75f4e446d79 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:50.881560 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:50.882508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:50.883325 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 410/817] 10.222.0.16 () {56 vars in 1129 bytes} [Sat Sep 3 20:33:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:50.884094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90df32-1bec-4e9c-a18e-f75f4e446d79 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:50.893095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab0692-a87e-43b7-b686-b797d3e017d4 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:50.937257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ac651-cff7-456b-b700-900e890db94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:50.938362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:50.939077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ac651-cff7-456b-b700-900e890db94c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:50.939201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ac651-cff7-456b-b700-900e890db94c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:50.939685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ac651-cff7-456b-b700-900e890db94c None admin] PATH_INFO: `/v3/services/network` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:50.940270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ac651-cff7-456b-b700-900e890db94c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:50.942195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ac651-cff7-456b-b700-900e890db94c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-688ac651-cff7-456b-b700-900e890db94c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:50.954447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:50.955436 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:50.961802 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 408/818] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:50 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:50.962588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ac651-cff7-456b-b700-900e890db94c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:50.992225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90df32-1bec-4e9c-a18e-f75f4e446d79 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:51.030121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5debcdf-6862-4f54-8634-02129033c056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:51.030768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5debcdf-6862-4f54-8634-02129033c056 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:51.031058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5debcdf-6862-4f54-8634-02129033c056 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:51.031338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5debcdf-6862-4f54-8634-02129033c056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:51.042699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5debcdf-6862-4f54-8634-02129033c056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:51.044168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5debcdf-6862-4f54-8634-02129033c056 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-a5debcdf-6862-4f54-8634-02129033c056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:51.058232 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:51.059176 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:51.060053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:51.060053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:51.060053 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:51.060053 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 411/819] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:51.060284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5debcdf-6862-4f54-8634-02129033c056 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:51.072393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ac651-cff7-456b-b700-900e890db94c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:51.102796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:51.109390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6dda3-4773-4e33-9147-2157c9a45c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:51.110062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6dda3-4773-4e33-9147-2157c9a45c4e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:51.110350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6dda3-4773-4e33-9147-2157c9a45c4e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:51.110631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6dda3-4773-4e33-9147-2157c9a45c4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:51.120389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6dda3-4773-4e33-9147-2157c9a45c4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:51.120389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6dda3-4773-4e33-9147-2157c9a45c4e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-87c6dda3-4773-4e33-9147-2157c9a45c4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:51.133349 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:51.134281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:51.135663 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:51.135663 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:51.135663 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:51.135663 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:51.135663 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:51.135663 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:51.135663 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:51.135663 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:51.135663 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 409/820] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:51.135663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6dda3-4773-4e33-9147-2157c9a45c4e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:51.185255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5debcdf-6862-4f54-8634-02129033c056 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:51.186788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:51.222056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de581f2-a570-493d-a9b9-e9f945a46c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:51.222671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de581f2-a570-493d-a9b9-e9f945a46c2f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:51.223472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de581f2-a570-493d-a9b9-e9f945a46c2f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:51.223757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de581f2-a570-493d-a9b9-e9f945a46c2f None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:51.224392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de581f2-a570-493d-a9b9-e9f945a46c2f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:51.224703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de581f2-a570-493d-a9b9-e9f945a46c2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:51.226975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de581f2-a570-493d-a9b9-e9f945a46c2f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:51.237752 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 412/821] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Sep 3 20:33:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:51.237752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de581f2-a570-493d-a9b9-e9f945a46c2f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:51.248813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6dda3-4773-4e33-9147-2157c9a45c4e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:51.285363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84c7a8-95de-4326-b295-b4e0209e0c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:51.286063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84c7a8-95de-4326-b295-b4e0209e0c2f None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:51.286355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84c7a8-95de-4326-b295-b4e0209e0c2f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:51.286638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84c7a8-95de-4326-b295-b4e0209e0c2f None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:51.287277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84c7a8-95de-4326-b295-b4e0209e0c2f None admin] RBAC: Adding query filter params (name=network) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:51.287589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84c7a8-95de-4326-b295-b4e0209e0c2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:51.288994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84c7a8-95de-4326-b295-b4e0209e0c2f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:51.294007 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 410/822] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:33:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:51.294800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84c7a8-95de-4326-b295-b4e0209e0c2f None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:51.346439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de581f2-a570-493d-a9b9-e9f945a46c2f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:51.354803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:51.383145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b07c53-b643-4dc7-bc44-7d25dcbd8ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:51.383869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b07c53-b643-4dc7-bc44-7d25dcbd8ae6 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:51.384158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b07c53-b643-4dc7-bc44-7d25dcbd8ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:51.384642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b07c53-b643-4dc7-bc44-7d25dcbd8ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:51.385212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b07c53-b643-4dc7-bc44-7d25dcbd8ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:51.385614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b07c53-b643-4dc7-bc44-7d25dcbd8ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:51.386922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b07c53-b643-4dc7-bc44-7d25dcbd8ae6 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:51.405714 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 413/823] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:51.405714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b07c53-b643-4dc7-bc44-7d25dcbd8ae6 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:51.408971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84c7a8-95de-4326-b295-b4e0209e0c2f None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:51.433603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:51.448477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845125f-cb66-41fa-b31d-4ff84549aa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:51.449025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845125f-cb66-41fa-b31d-4ff84549aa5f None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:51.449314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845125f-cb66-41fa-b31d-4ff84549aa5f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:51.449590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845125f-cb66-41fa-b31d-4ff84549aa5f None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:51.450286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845125f-cb66-41fa-b31d-4ff84549aa5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:51.450750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845125f-cb66-41fa-b31d-4ff84549aa5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:51.452188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845125f-cb66-41fa-b31d-4ff84549aa5f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:51.469918 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 411/824] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:51.469918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845125f-cb66-41fa-b31d-4ff84549aa5f None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:51.517429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b07c53-b643-4dc7-bc44-7d25dcbd8ae6 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:51.551394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3487263-f072-472e-ae6a-86802f743114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:51.552008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3487263-f072-472e-ae6a-86802f743114 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:51.552298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3487263-f072-472e-ae6a-86802f743114 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:51.552584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3487263-f072-472e-ae6a-86802f743114 None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:51.553257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3487263-f072-472e-ae6a-86802f743114 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:51.553570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3487263-f072-472e-ae6a-86802f743114 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:51.554947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3487263-f072-472e-ae6a-86802f743114 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:51.559726 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 414/825] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Sep 3 20:33:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:51.560510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3487263-f072-472e-ae6a-86802f743114 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:51.590967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845125f-cb66-41fa-b31d-4ff84549aa5f None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:51.606141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:51.629330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071ec79-8e41-4f80-9ae6-6e852e9bc7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:51.629939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071ec79-8e41-4f80-9ae6-6e852e9bc7fb None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:51.630228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071ec79-8e41-4f80-9ae6-6e852e9bc7fb None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:51.630504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071ec79-8e41-4f80-9ae6-6e852e9bc7fb None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:51.631137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071ec79-8e41-4f80-9ae6-6e852e9bc7fb None admin] RBAC: Adding query filter params (type=network) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:51.631443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071ec79-8e41-4f80-9ae6-6e852e9bc7fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:51.632811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071ec79-8e41-4f80-9ae6-6e852e9bc7fb None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:51.643632 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 412/826] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:33:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:51.643632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071ec79-8e41-4f80-9ae6-6e852e9bc7fb None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:51.660767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:51.671579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3487263-f072-472e-ae6a-86802f743114 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:51.711849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e959c4-e3d7-4731-8957-be738e480dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:51.712598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e959c4-e3d7-4731-8957-be738e480dfd None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:51.712889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e959c4-e3d7-4731-8957-be738e480dfd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:51.713193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e959c4-e3d7-4731-8957-be738e480dfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:51.720407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e959c4-e3d7-4731-8957-be738e480dfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:51.721897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e959c4-e3d7-4731-8957-be738e480dfd None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-60e959c4-e3d7-4731-8957-be738e480dfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:51.734400 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:51.735991 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:51.736990 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:51.736990 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:51.736990 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:51.736990 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:51.736990 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:51.736990 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:51.736990 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:51.736990 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:51.736990 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 415/827] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:51.736990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e959c4-e3d7-4731-8957-be738e480dfd None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:51.768944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071ec79-8e41-4f80-9ae6-6e852e9bc7fb None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:51.809580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe7231-8816-4b1a-9251-f2fdbe27a5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:51.810563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe7231-8816-4b1a-9251-f2fdbe27a5ef None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:51.811052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe7231-8816-4b1a-9251-f2fdbe27a5ef None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:51.811521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe7231-8816-4b1a-9251-f2fdbe27a5ef None admin] PATH_INFO: `/v3/projects/1e040b2fca724c58a2660441a72cd350/users/83865463b2a84f99a136b73dffeeb378/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:51.814255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe7231-8816-4b1a-9251-f2fdbe27a5ef None admin] RBAC: Authorizing `identity:create_grant(project_id=1e040b2fca724c58a2660441a72cd350, user_id=83865463b2a84f99a136b73dffeeb378, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:51.815700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe7231-8816-4b1a-9251-f2fdbe27a5ef None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:51.840487 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 413/828] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:33:51 2022] PUT /identity/v3/projects/1e040b2fca724c58a2660441a72cd350/users/83865463b2a84f99a136b73dffeeb378/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:33:51.840487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe7231-8816-4b1a-9251-f2fdbe27a5ef None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:51.874312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e959c4-e3d7-4731-8957-be738e480dfd None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:51.878631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:51.937463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4f692-f3dc-49a8-8cfa-7a9842cc7a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:51.938688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4f692-f3dc-49a8-8cfa-7a9842cc7a6d None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:51.939214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4f692-f3dc-49a8-8cfa-7a9842cc7a6d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:51.939585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4f692-f3dc-49a8-8cfa-7a9842cc7a6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:51.940726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4f692-f3dc-49a8-8cfa-7a9842cc7a6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:51.942502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4f692-f3dc-49a8-8cfa-7a9842cc7a6d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:51.958835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe7231-8816-4b1a-9251-f2fdbe27a5ef None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:51.982228 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 416/829] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:33:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:33:51.983747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4f692-f3dc-49a8-8cfa-7a9842cc7a6d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:51.998862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:52.011434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98beb06-c084-46d5-a128-8d575224e0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:52.012748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98beb06-c084-46d5-a128-8d575224e0ff None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:52.012954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98beb06-c084-46d5-a128-8d575224e0ff None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:52.013339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98beb06-c084-46d5-a128-8d575224e0ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:52.014192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98beb06-c084-46d5-a128-8d575224e0ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:52.016146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98beb06-c084-46d5-a128-8d575224e0ff None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:52.044841 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 414/830] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:33:51 2022] POST /identity/v3/endpoints => generated 337 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:33:52.123635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4f692-f3dc-49a8-8cfa-7a9842cc7a6d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:52.175851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c71df-9d6a-4e6e-8adb-c2b15c0d8477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:52.176790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c71df-9d6a-4e6e-8adb-c2b15c0d8477 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:52.177285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c71df-9d6a-4e6e-8adb-c2b15c0d8477 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:52.177730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c71df-9d6a-4e6e-8adb-c2b15c0d8477 None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:52.178622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c71df-9d6a-4e6e-8adb-c2b15c0d8477 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:52.179101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c71df-9d6a-4e6e-8adb-c2b15c0d8477 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:52.180791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c71df-9d6a-4e6e-8adb-c2b15c0d8477 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:52.196228 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 417/831] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:52.201990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98beb06-c084-46d5-a128-8d575224e0ff None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:52.314582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:52.325837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98beb06-c084-46d5-a128-8d575224e0ff None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:52.365308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cab3f-3a37-4c1c-9db8-73933a2e4804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:52.366081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cab3f-3a37-4c1c-9db8-73933a2e4804 None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:52.366331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cab3f-3a37-4c1c-9db8-73933a2e4804 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:52.366618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cab3f-3a37-4c1c-9db8-73933a2e4804 None admin] PATH_INFO: `/v3/projects/1e040b2fca724c58a2660441a72cd350/users/9388372667f8454498b2e4480187b47c/roles/a211c6f9ebbc4889b636c26cd511e0e3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:52.382443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cab3f-3a37-4c1c-9db8-73933a2e4804 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e040b2fca724c58a2660441a72cd350, user_id=9388372667f8454498b2e4480187b47c, role_id=a211c6f9ebbc4889b636c26cd511e0e3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:52.383931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cab3f-3a37-4c1c-9db8-73933a2e4804 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:52.400605 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 415/832] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:33:52 2022] PUT /identity/v3/projects/1e040b2fca724c58a2660441a72cd350/users/9388372667f8454498b2e4480187b47c/roles/a211c6f9ebbc4889b636c26cd511e0e3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:33:52.913759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e00d45-72b8-463e-b368-228935ea6ce2 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:52.914309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e00d45-72b8-463e-b368-228935ea6ce2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:52.914309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e00d45-72b8-463e-b368-228935ea6ce2 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:52.915895 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 418/833] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:52.921179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6e4a7-c18a-4c96-b8e9-6c58a6c9506c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:52.921479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6e4a7-c18a-4c96-b8e9-6c58a6c9506c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:52.921767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6e4a7-c18a-4c96-b8e9-6c58a6c9506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:52.942409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0aa6e4a7-c18a-4c96-b8e9-6c58a6c9506c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:52.943617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0aa6e4a7-c18a-4c96-b8e9-6c58a6c9506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:53.013933 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 416/834] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:33:53.342280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e00d45-72b8-463e-b368-228935ea6ce2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:53.405172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:53.446069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e00d45-72b8-463e-b368-228935ea6ce2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:53.481065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a39c9-f5e2-499f-b07b-7e81267a7ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:53.482020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a39c9-f5e2-499f-b07b-7e81267a7ebf None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:53.482520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a39c9-f5e2-499f-b07b-7e81267a7ebf None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:53.482992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a39c9-f5e2-499f-b07b-7e81267a7ebf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:53.483791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a39c9-f5e2-499f-b07b-7e81267a7ebf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:53.485440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a39c9-f5e2-499f-b07b-7e81267a7ebf None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-495a39c9-f5e2-499f-b07b-7e81267a7ebf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:53.494476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:53.495440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 20:33:53.496308 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:53.499042 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 419/835] 10.222.0.16 () {56 vars in 1132 bytes} [Sat Sep 3 20:33:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:53.507079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6e4a7-c18a-4c96-b8e9-6c58a6c9506c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:53.597623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:53.612648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6e4a7-c18a-4c96-b8e9-6c58a6c9506c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:53.649864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744fc26-ebdb-4df1-9293-37e6529d1ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:53.650486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744fc26-ebdb-4df1-9293-37e6529d1ef4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:53.650774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744fc26-ebdb-4df1-9293-37e6529d1ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:53.651056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744fc26-ebdb-4df1-9293-37e6529d1ef4 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:53.651692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744fc26-ebdb-4df1-9293-37e6529d1ef4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:53.652005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744fc26-ebdb-4df1-9293-37e6529d1ef4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:53.653430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744fc26-ebdb-4df1-9293-37e6529d1ef4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:53.659045 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 417/836] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 3 20:33:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:53.664558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a39c9-f5e2-499f-b07b-7e81267a7ebf None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:53.772327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a39c9-f5e2-499f-b07b-7e81267a7ebf None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:53.802311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:53.811178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad1df8-6863-4bcc-867e-33312a5fda6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:53.811908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad1df8-6863-4bcc-867e-33312a5fda6c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:53.812257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad1df8-6863-4bcc-867e-33312a5fda6c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:53.812536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad1df8-6863-4bcc-867e-33312a5fda6c None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:53.813299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad1df8-6863-4bcc-867e-33312a5fda6c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:53.813548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad1df8-6863-4bcc-867e-33312a5fda6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:53.815228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad1df8-6863-4bcc-867e-33312a5fda6c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:53.820459 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 420/837] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 3 20:33:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:54.406622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366a0e3-243e-42e6-bebb-8fcf7aae9d25 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:54.407370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366a0e3-243e-42e6-bebb-8fcf7aae9d25 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:54.407936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366a0e3-243e-42e6-bebb-8fcf7aae9d25 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:54.409809 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 418/838] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:54.415175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64279b-95ac-47b9-8db1-1abbadaa08b6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:54.416041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64279b-95ac-47b9-8db1-1abbadaa08b6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:54.416537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64279b-95ac-47b9-8db1-1abbadaa08b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:54.435369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6f64279b-95ac-47b9-8db1-1abbadaa08b6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:54.436947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6f64279b-95ac-47b9-8db1-1abbadaa08b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:54.440576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09510d66-aa95-4cad-a2ef-fb5c2e6db057 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:54.441155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09510d66-aa95-4cad-a2ef-fb5c2e6db057 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:54.441665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09510d66-aa95-4cad-a2ef-fb5c2e6db057 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:54.443430 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 419/839] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:33:54.448632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dca7a-3a37-47c4-9cd9-7233dab0dd02 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:54.449184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dca7a-3a37-47c4-9cd9-7233dab0dd02 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:54.449687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dca7a-3a37-47c4-9cd9-7233dab0dd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:54.469115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7a8dca7a-3a37-47c4-9cd9-7233dab0dd02 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:54.470669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7a8dca7a-3a37-47c4-9cd9-7233dab0dd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:54.517059 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 421/840] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:33:54.546891 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 420/841] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:33:54.670536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193555c-1cf4-4cd3-bbab-544ecfb3b063 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:54.670759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193555c-1cf4-4cd3-bbab-544ecfb3b063 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:54.671099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193555c-1cf4-4cd3-bbab-544ecfb3b063 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:54.672547 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 422/842] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:54.678314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d5903-4777-46d0-9eea-f0c634368fc4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:54.678611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d5903-4777-46d0-9eea-f0c634368fc4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:54.679112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d5903-4777-46d0-9eea-f0c634368fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:54.698555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d71d5903-4777-46d0-9eea-f0c634368fc4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:54.700063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d71d5903-4777-46d0-9eea-f0c634368fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:54.761659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:54.771029 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 421/843] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:33:54.844081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193555c-1cf4-4cd3-bbab-544ecfb3b063 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:54.890947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d5903-4777-46d0-9eea-f0c634368fc4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:54.956345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193555c-1cf4-4cd3-bbab-544ecfb3b063 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:54.985762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:54.996748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84810ec-8602-41a2-b1ae-99aed0af063f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:54.997653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84810ec-8602-41a2-b1ae-99aed0af063f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:54.999741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84810ec-8602-41a2-b1ae-99aed0af063f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:54.999741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84810ec-8602-41a2-b1ae-99aed0af063f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:54.999741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84810ec-8602-41a2-b1ae-99aed0af063f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:55.000300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84810ec-8602-41a2-b1ae-99aed0af063f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-e84810ec-8602-41a2-b1ae-99aed0af063f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:55.013761 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:55.014877 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 20:33:55.015699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:55.017261 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 423/844] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:33:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:55.018777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d5903-4777-46d0-9eea-f0c634368fc4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:55.019316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84810ec-8602-41a2-b1ae-99aed0af063f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:55.062464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d870a53-44a6-4e4f-b6b2-d07f4f6655b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:55.063128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d870a53-44a6-4e4f-b6b2-d07f4f6655b8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:55.063421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d870a53-44a6-4e4f-b6b2-d07f4f6655b8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:55.064013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d870a53-44a6-4e4f-b6b2-d07f4f6655b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:55.076039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d870a53-44a6-4e4f-b6b2-d07f4f6655b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-7d870a53-44a6-4e4f-b6b2-d07f4f6655b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:55.082677 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:55.083615 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:55.084484 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 422/845] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:55.091552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d870a53-44a6-4e4f-b6b2-d07f4f6655b8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:55.147494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84810ec-8602-41a2-b1ae-99aed0af063f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:55.163721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:55.183349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5163f-9ff9-4bdc-93ea-d8d91b835bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:55.184033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5163f-9ff9-4bdc-93ea-d8d91b835bce None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:55.184387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5163f-9ff9-4bdc-93ea-d8d91b835bce None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:55.184732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5163f-9ff9-4bdc-93ea-d8d91b835bce None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:55.185462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5163f-9ff9-4bdc-93ea-d8d91b835bce None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:55.185856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5163f-9ff9-4bdc-93ea-d8d91b835bce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:55.187280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5163f-9ff9-4bdc-93ea-d8d91b835bce None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:55.192226 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 424/846] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:33:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:55.193121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5163f-9ff9-4bdc-93ea-d8d91b835bce None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:55.219766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d870a53-44a6-4e4f-b6b2-d07f4f6655b8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:55.244309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:55.260563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced1354-fd26-4291-ad8e-eae747a3cef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:55.261576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced1354-fd26-4291-ad8e-eae747a3cef0 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:55.261862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced1354-fd26-4291-ad8e-eae747a3cef0 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:55.262143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced1354-fd26-4291-ad8e-eae747a3cef0 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:55.263003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced1354-fd26-4291-ad8e-eae747a3cef0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:55.263289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced1354-fd26-4291-ad8e-eae747a3cef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:55.264773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced1354-fd26-4291-ad8e-eae747a3cef0 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:55.278351 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 423/847] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:33:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:55.278351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced1354-fd26-4291-ad8e-eae747a3cef0 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:55.302805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5163f-9ff9-4bdc-93ea-d8d91b835bce None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:55.341553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223cb02-242d-48f0-ad12-e243ccf17036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:55.342197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223cb02-242d-48f0-ad12-e243ccf17036 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:55.342484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223cb02-242d-48f0-ad12-e243ccf17036 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:55.342766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223cb02-242d-48f0-ad12-e243ccf17036 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:55.353114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223cb02-242d-48f0-ad12-e243ccf17036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-3223cb02-242d-48f0-ad12-e243ccf17036 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:33:55.358362 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:55.359517 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:55.360937 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 425/848] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:33:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:55.361799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223cb02-242d-48f0-ad12-e243ccf17036 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:55.390260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced1354-fd26-4291-ad8e-eae747a3cef0 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:55.408775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:55.433056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3fc9f-fcac-4307-93f4-b8bc7bb98faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:55.433712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3fc9f-fcac-4307-93f4-b8bc7bb98faf None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:55.434164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3fc9f-fcac-4307-93f4-b8bc7bb98faf None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:55.434565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3fc9f-fcac-4307-93f4-b8bc7bb98faf None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:55.435525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3fc9f-fcac-4307-93f4-b8bc7bb98faf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:55.435987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3fc9f-fcac-4307-93f4-b8bc7bb98faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:55.438323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3fc9f-fcac-4307-93f4-b8bc7bb98faf None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:55.444411 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 424/849] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:33:55 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:55.445061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3fc9f-fcac-4307-93f4-b8bc7bb98faf None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:55.493080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223cb02-242d-48f0-ad12-e243ccf17036 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:55.507376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:55.546461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e7ef0-3009-4cbc-8671-54ac9b57fe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:55.547666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e7ef0-3009-4cbc-8671-54ac9b57fe7a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:55.548171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e7ef0-3009-4cbc-8671-54ac9b57fe7a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:55.548674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e7ef0-3009-4cbc-8671-54ac9b57fe7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:55.561681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e7ef0-3009-4cbc-8671-54ac9b57fe7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:55.563589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e7ef0-3009-4cbc-8671-54ac9b57fe7a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:55.563884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3fc9f-fcac-4307-93f4-b8bc7bb98faf None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-de2e7ef0-3009-4cbc-8671-54ac9b57fe7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:55.581600 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:55.582891 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:55.583779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:55.583779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:55.583779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:55.586241 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 426/850] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:55.593047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e7ef0-3009-4cbc-8671-54ac9b57fe7a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:55.604463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81509b55-092f-4014-a0c9-eafa23eaa940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:55.605549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81509b55-092f-4014-a0c9-eafa23eaa940 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:55.606008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81509b55-092f-4014-a0c9-eafa23eaa940 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:55.606434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81509b55-092f-4014-a0c9-eafa23eaa940 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:55.607284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81509b55-092f-4014-a0c9-eafa23eaa940 None admin] RBAC: Adding query filter params (name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:55.607750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81509b55-092f-4014-a0c9-eafa23eaa940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:55.609518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81509b55-092f-4014-a0c9-eafa23eaa940 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:55.619864 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 425/851] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:33:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:55.626176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81509b55-092f-4014-a0c9-eafa23eaa940 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:55.686554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:55.724921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e7ef0-3009-4cbc-8671-54ac9b57fe7a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:55.764126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0456406-e6f1-413e-8a61-5e35dc2c9461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:55.764662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0456406-e6f1-413e-8a61-5e35dc2c9461 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:55.764974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0456406-e6f1-413e-8a61-5e35dc2c9461 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:55.765254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0456406-e6f1-413e-8a61-5e35dc2c9461 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:55.765904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0456406-e6f1-413e-8a61-5e35dc2c9461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:55.766838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0456406-e6f1-413e-8a61-5e35dc2c9461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:55.768227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0456406-e6f1-413e-8a61-5e35dc2c9461 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:55.774529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81509b55-092f-4014-a0c9-eafa23eaa940 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:55.786242 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 427/852] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:55.788255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0456406-e6f1-413e-8a61-5e35dc2c9461 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:55.816958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f3865-ca9a-4ddf-b49d-ca8023554aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:55.817664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f3865-ca9a-4ddf-b49d-ca8023554aee None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:55.817955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f3865-ca9a-4ddf-b49d-ca8023554aee None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:55.818282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f3865-ca9a-4ddf-b49d-ca8023554aee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:55.819931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:55.831730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f3865-ca9a-4ddf-b49d-ca8023554aee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:55.833591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f3865-ca9a-4ddf-b49d-ca8023554aee None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-2f5f3865-ca9a-4ddf-b49d-ca8023554aee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:55.851801 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:55.852782 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:55.853690 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:55.853690 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:55.853690 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:55.853690 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 426/853] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:33:55.859600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f3865-ca9a-4ddf-b49d-ca8023554aee None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:55.939417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0456406-e6f1-413e-8a61-5e35dc2c9461 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:55.981711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2fd75-207d-498c-8e79-56be9377aa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:55.982728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2fd75-207d-498c-8e79-56be9377aa19 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:55.983248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2fd75-207d-498c-8e79-56be9377aa19 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:55.983769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2fd75-207d-498c-8e79-56be9377aa19 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:55.986903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:55.993860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f3865-ca9a-4ddf-b49d-ca8023554aee None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:55.997012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2fd75-207d-498c-8e79-56be9377aa19 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:55.997012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2fd75-207d-498c-8e79-56be9377aa19 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-8dd2fd75-207d-498c-8e79-56be9377aa19 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:56.007391 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:56.008377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:56.009393 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:56.009393 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:56.009393 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:56.009393 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:56.009393 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:56.009393 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:56.009393 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:56.009393 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:56.009393 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 20:33:56.009393 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:56.010732 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 428/854] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:33:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:56.017400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2fd75-207d-498c-8e79-56be9377aa19 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:56.039265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d4df8-008b-44f8-9fa6-51c6503a28c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:56.039943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d4df8-008b-44f8-9fa6-51c6503a28c2 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:56.040237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d4df8-008b-44f8-9fa6-51c6503a28c2 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:56.040519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d4df8-008b-44f8-9fa6-51c6503a28c2 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:56.041315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d4df8-008b-44f8-9fa6-51c6503a28c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:56.041816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d4df8-008b-44f8-9fa6-51c6503a28c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:56.043509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d4df8-008b-44f8-9fa6-51c6503a28c2 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:56.061305 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 427/855] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:56.062217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d4df8-008b-44f8-9fa6-51c6503a28c2 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:56.109574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:56.138875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2fd75-207d-498c-8e79-56be9377aa19 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:56.174329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d4df8-008b-44f8-9fa6-51c6503a28c2 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:56.175824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580a5aa-ebaf-49d1-b798-5a124adcff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:56.176436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580a5aa-ebaf-49d1-b798-5a124adcff96 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:56.176734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580a5aa-ebaf-49d1-b798-5a124adcff96 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:56.177045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580a5aa-ebaf-49d1-b798-5a124adcff96 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:56.177796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580a5aa-ebaf-49d1-b798-5a124adcff96 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:56.178092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580a5aa-ebaf-49d1-b798-5a124adcff96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:56.179535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580a5aa-ebaf-49d1-b798-5a124adcff96 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:56.197387 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 429/856] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Sep 3 20:33:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:56.198625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfec924-23e9-471b-a2f5-f2666a60c866 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:56.198917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfec924-23e9-471b-a2f5-f2666a60c866 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:56.199198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfec924-23e9-471b-a2f5-f2666a60c866 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:56.200546 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 430/857] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:56.203374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfec924-23e9-471b-a2f5-f2666a60c866 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:56.218405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193dc7a-450d-4c2b-94a4-a21a4ddffe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:56.219049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193dc7a-450d-4c2b-94a4-a21a4ddffe1f None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:56.219388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193dc7a-450d-4c2b-94a4-a21a4ddffe1f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:56.219672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193dc7a-450d-4c2b-94a4-a21a4ddffe1f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:56.227344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193dc7a-450d-4c2b-94a4-a21a4ddffe1f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:56.228794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193dc7a-450d-4c2b-94a4-a21a4ddffe1f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-f193dc7a-450d-4c2b-94a4-a21a4ddffe1f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:33:56.240440 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:56.247049 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 428/858] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 3 20:33:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b5bc7-4512-4f8f-9681-94121c7264cb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:56.247996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b5bc7-4512-4f8f-9681-94121c7264cb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:56.248722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b5bc7-4512-4f8f-9681-94121c7264cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:56.263614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e05b5bc7-4512-4f8f-9681-94121c7264cb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:56.264824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e05b5bc7-4512-4f8f-9681-94121c7264cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:56.274924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:56.319281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfec924-23e9-471b-a2f5-f2666a60c866 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:56.338128 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 429/859] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:33:56.342253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b5bc7-4512-4f8f-9681-94121c7264cb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:56.356161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e81cc-686b-43b9-b208-aaeca3d97237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:56.356547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e81cc-686b-43b9-b208-aaeca3d97237 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:56.356837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e81cc-686b-43b9-b208-aaeca3d97237 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:56.357200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e81cc-686b-43b9-b208-aaeca3d97237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:56.366929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:56.447249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e81cc-686b-43b9-b208-aaeca3d97237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:56.449394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e81cc-686b-43b9-b208-aaeca3d97237 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:56.464354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b5bc7-4512-4f8f-9681-94121c7264cb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:56.501228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ca4f8-0e41-4c8d-a963-4a893a885349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:56.501931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ca4f8-0e41-4c8d-a963-4a893a885349 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:56.502291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ca4f8-0e41-4c8d-a963-4a893a885349 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:56.502642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ca4f8-0e41-4c8d-a963-4a893a885349 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:56.503448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ca4f8-0e41-4c8d-a963-4a893a885349 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:56.503831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ca4f8-0e41-4c8d-a963-4a893a885349 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:56.505513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ca4f8-0e41-4c8d-a963-4a893a885349 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:56.528870 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 430/860] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Sep 3 20:33:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:56.530142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ca4f8-0e41-4c8d-a963-4a893a885349 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:56.532917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:56.592828 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 431/861] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 20:33:56.594618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e81cc-686b-43b9-b208-aaeca3d97237 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:56.654165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:56.659172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ca4f8-0e41-4c8d-a963-4a893a885349 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:56.696493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eced39-6b93-4301-93ea-c29dea823520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:56.697501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eced39-6b93-4301-93ea-c29dea823520 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:56.698000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eced39-6b93-4301-93ea-c29dea823520 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:56.698523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eced39-6b93-4301-93ea-c29dea823520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:56.726191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e81cc-686b-43b9-b208-aaeca3d97237 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:56.762319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491faef-948e-4d30-bb5b-a26c921a4b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:56.762958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491faef-948e-4d30-bb5b-a26c921a4b98 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:56.763250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491faef-948e-4d30-bb5b-a26c921a4b98 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:56.763529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491faef-948e-4d30-bb5b-a26c921a4b98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:56.773926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eced39-6b93-4301-93ea-c29dea823520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:56.774282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491faef-948e-4d30-bb5b-a26c921a4b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:56.775398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eced39-6b93-4301-93ea-c29dea823520 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:56.775727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491faef-948e-4d30-bb5b-a26c921a4b98 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:56.777552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-a491faef-948e-4d30-bb5b-a26c921a4b98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:56.792793 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:56.794023 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:56.794915 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:56.794915 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:56.794915 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:56.794915 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 432/862] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:56.794915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491faef-948e-4d30-bb5b-a26c921a4b98 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:56.893819 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 431/863] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:33:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 20:33:56.893819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eced39-6b93-4301-93ea-c29dea823520 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:56.894075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:56.903043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491faef-948e-4d30-bb5b-a26c921a4b98 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:56.937312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88aac5b-6ffa-4347-bd69-65db519b2464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:56.938937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88aac5b-6ffa-4347-bd69-65db519b2464 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:56.939239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88aac5b-6ffa-4347-bd69-65db519b2464 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:56.939520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88aac5b-6ffa-4347-bd69-65db519b2464 None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:56.940175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88aac5b-6ffa-4347-bd69-65db519b2464 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:56.942271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88aac5b-6ffa-4347-bd69-65db519b2464 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:56.956996 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 433/864] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:33:56 2022] POST /identity/v3/services => generated 269 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:33:56.958139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88aac5b-6ffa-4347-bd69-65db519b2464 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:57.032426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eced39-6b93-4301-93ea-c29dea823520 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:57.035182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:57.088333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a687659-02cd-4dce-8420-bc34174fbfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:57.089059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a687659-02cd-4dce-8420-bc34174fbfed None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:57.089364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a687659-02cd-4dce-8420-bc34174fbfed None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:57.089679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a687659-02cd-4dce-8420-bc34174fbfed None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:57.090344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a687659-02cd-4dce-8420-bc34174fbfed None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:57.090663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a687659-02cd-4dce-8420-bc34174fbfed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:57.092106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a687659-02cd-4dce-8420-bc34174fbfed None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:57.100345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88aac5b-6ffa-4347-bd69-65db519b2464 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:57.109688 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 432/865] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:57.111735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a687659-02cd-4dce-8420-bc34174fbfed None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:57.168641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51ac80-8fea-4677-bea2-80ca921d8bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:57.169356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51ac80-8fea-4677-bea2-80ca921d8bee None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:57.169641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51ac80-8fea-4677-bea2-80ca921d8bee None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:57.169921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51ac80-8fea-4677-bea2-80ca921d8bee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:57.182913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51ac80-8fea-4677-bea2-80ca921d8bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:57.184387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51ac80-8fea-4677-bea2-80ca921d8bee None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-5b51ac80-8fea-4677-bea2-80ca921d8bee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:33:57.203375 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:57.204961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:33:57.205871 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:33:57.205871 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:33:57.205871 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:57.205871 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 434/866] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:57.213520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51ac80-8fea-4677-bea2-80ca921d8bee None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:57.218080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:57.250139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a687659-02cd-4dce-8420-bc34174fbfed None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:57.288457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547aac3-2610-44f4-a8c9-2fc028163a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:57.289151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547aac3-2610-44f4-a8c9-2fc028163a34 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:57.289466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547aac3-2610-44f4-a8c9-2fc028163a34 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:57.289760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547aac3-2610-44f4-a8c9-2fc028163a34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:57.297105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547aac3-2610-44f4-a8c9-2fc028163a34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:57.299083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547aac3-2610-44f4-a8c9-2fc028163a34 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-6547aac3-2610-44f4-a8c9-2fc028163a34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:57.311295 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:57.312225 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:57.313682 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:57.313682 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:57.313682 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:57.313682 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:57.313682 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:57.313682 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:57.313682 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:57.313682 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:33:57.313682 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 433/867] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:57.318987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547aac3-2610-44f4-a8c9-2fc028163a34 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:57.341024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:57.352985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51ac80-8fea-4677-bea2-80ca921d8bee None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:57.389080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acaa1fa-f8ef-4bb5-8d1f-3b4ab8f9aa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:57.389697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaa1fa-f8ef-4bb5-8d1f-3b4ab8f9aa2f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:57.389989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaa1fa-f8ef-4bb5-8d1f-3b4ab8f9aa2f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:57.390271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaa1fa-f8ef-4bb5-8d1f-3b4ab8f9aa2f None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:57.391860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaa1fa-f8ef-4bb5-8d1f-3b4ab8f9aa2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:57.391860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaa1fa-f8ef-4bb5-8d1f-3b4ab8f9aa2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:57.393264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaa1fa-f8ef-4bb5-8d1f-3b4ab8f9aa2f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:57.410206 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 435/868] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:33:57.411245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acaa1fa-f8ef-4bb5-8d1f-3b4ab8f9aa2f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:57.459061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547aac3-2610-44f4-a8c9-2fc028163a34 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:57.481023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:57.513143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4877a-07ac-4e37-b77c-ebc5d1ebd863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:57.513918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4877a-07ac-4e37-b77c-ebc5d1ebd863 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:57.514545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4877a-07ac-4e37-b77c-ebc5d1ebd863 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:57.514833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4877a-07ac-4e37-b77c-ebc5d1ebd863 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:57.515820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4877a-07ac-4e37-b77c-ebc5d1ebd863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:57.516701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4877a-07ac-4e37-b77c-ebc5d1ebd863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:57.518681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4877a-07ac-4e37-b77c-ebc5d1ebd863 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:57.524058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acaa1fa-f8ef-4bb5-8d1f-3b4ab8f9aa2f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:57.534728 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 434/869] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:57.541640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4877a-07ac-4e37-b77c-ebc5d1ebd863 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:57.575526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d332e-84bb-4c97-ba33-a27b5b8d32fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:57.576342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d332e-84bb-4c97-ba33-a27b5b8d32fd None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:57.576643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d332e-84bb-4c97-ba33-a27b5b8d32fd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:57.577083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d332e-84bb-4c97-ba33-a27b5b8d32fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:57.589069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d332e-84bb-4c97-ba33-a27b5b8d32fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:57.590599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d332e-84bb-4c97-ba33-a27b5b8d32fd None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-613d332e-84bb-4c97-ba33-a27b5b8d32fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:33:57.606734 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:57.608123 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:57.609474 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:33:57.609474 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:33:57.609474 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:33:57.609474 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:33:57.609474 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:33:57.609474 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:33:57.609474 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:33:57.609474 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:57.609474 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 436/870] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:57.614011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d332e-84bb-4c97-ba33-a27b5b8d32fd None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:57.642231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:57.665760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4877a-07ac-4e37-b77c-ebc5d1ebd863 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:57.702750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfbbed-37f2-4d68-963d-62f2722d16b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:57.703105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfbbed-37f2-4d68-963d-62f2722d16b4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:57.703390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfbbed-37f2-4d68-963d-62f2722d16b4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:57.704192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfbbed-37f2-4d68-963d-62f2722d16b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:57.704955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfbbed-37f2-4d68-963d-62f2722d16b4 None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=83865463b2a84f99a136b73dffeeb378) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:57.705279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfbbed-37f2-4d68-963d-62f2722d16b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:57.706728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfbbed-37f2-4d68-963d-62f2722d16b4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:57.720935 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 435/871] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:57 2022] GET /identity/v3/role_assignments?user.id=83865463b2a84f99a136b73dffeeb378&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=bc768522dc984239be4a1b39a45ab437 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:57.734142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fc969-2996-4328-b5fa-47fc13b186da None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:57.734497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fc969-2996-4328-b5fa-47fc13b186da None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:57.734715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fc969-2996-4328-b5fa-47fc13b186da None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:57.736137 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 436/872] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:57.742627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcc20f-a123-4489-85bc-33131b0b4b0b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:57.743032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d332e-84bb-4c97-ba33-a27b5b8d32fd None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:57.743547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcc20f-a123-4489-85bc-33131b0b4b0b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:57.744012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcc20f-a123-4489-85bc-33131b0b4b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:57.765248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2dfcc20f-a123-4489-85bc-33131b0b4b0b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:57.767166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2dfcc20f-a123-4489-85bc-33131b0b4b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:57.768987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:57.782335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb609a9-f090-491d-9bf5-1a1ae53132dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:57.783228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb609a9-f090-491d-9bf5-1a1ae53132dd None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:57.783688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb609a9-f090-491d-9bf5-1a1ae53132dd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:57.784113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb609a9-f090-491d-9bf5-1a1ae53132dd None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:57.785049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb609a9-f090-491d-9bf5-1a1ae53132dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:57.785367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb609a9-f090-491d-9bf5-1a1ae53132dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:57.786862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb609a9-f090-491d-9bf5-1a1ae53132dd None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:57.800724 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 437/873] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:33:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:57.807366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb609a9-f090-491d-9bf5-1a1ae53132dd None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:57.853134 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 437/874] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:57 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:33:57.921281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb609a9-f090-491d-9bf5-1a1ae53132dd None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:57.959320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a07ff3-d4ad-4bc8-8e08-27a6135455f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:57.959937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a07ff3-d4ad-4bc8-8e08-27a6135455f5 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:57.960226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a07ff3-d4ad-4bc8-8e08-27a6135455f5 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:57.960506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a07ff3-d4ad-4bc8-8e08-27a6135455f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:57.961288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a07ff3-d4ad-4bc8-8e08-27a6135455f5 None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=9388372667f8454498b2e4480187b47c) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:33:57.961607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a07ff3-d4ad-4bc8-8e08-27a6135455f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:57.963045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a07ff3-d4ad-4bc8-8e08-27a6135455f5 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:57.970084 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 438/875] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:33:57 2022] GET /identity/v3/role_assignments?user.id=9388372667f8454498b2e4480187b47c&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:33:58.202731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcc20f-a123-4489-85bc-33131b0b4b0b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:58.283365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:58.315588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcc20f-a123-4489-85bc-33131b0b4b0b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:58.351406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6377b6e-e64d-445d-8648-e717a10481d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:58.352023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6377b6e-e64d-445d-8648-e717a10481d9 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:58.352308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6377b6e-e64d-445d-8648-e717a10481d9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:58.352584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6377b6e-e64d-445d-8648-e717a10481d9 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:58.353290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6377b6e-e64d-445d-8648-e717a10481d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:58.354671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6377b6e-e64d-445d-8648-e717a10481d9 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:58.373332 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 438/876] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:33:58 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:33:59.264061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fe9fd-cc70-4597-b375-20403dc38839 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:59.264570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fe9fd-cc70-4597-b375-20403dc38839 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:59.264646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fe9fd-cc70-4597-b375-20403dc38839 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:59.266357 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 439/877] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:33:59.271646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679861db-bcbb-40c5-80e9-4e1408dd1983 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:59.272068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679861db-bcbb-40c5-80e9-4e1408dd1983 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:59.272259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679861db-bcbb-40c5-80e9-4e1408dd1983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:59.290645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-679861db-bcbb-40c5-80e9-4e1408dd1983 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:33:59.291839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-679861db-bcbb-40c5-80e9-4e1408dd1983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:33:59.370238 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 439/878] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:33:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:33:59.698140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fe9fd-cc70-4597-b375-20403dc38839 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:59.750542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:59.808765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fe9fd-cc70-4597-b375-20403dc38839 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:33:59.845111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b34d6-3e13-404b-8c79-59026ef2a6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:33:59.846119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b34d6-3e13-404b-8c79-59026ef2a6c0 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:33:59.846622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b34d6-3e13-404b-8c79-59026ef2a6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:33:59.847097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b34d6-3e13-404b-8c79-59026ef2a6c0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:33:59.847892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b34d6-3e13-404b-8c79-59026ef2a6c0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:33:59.849591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b34d6-3e13-404b-8c79-59026ef2a6c0 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-e60b34d6-3e13-404b-8c79-59026ef2a6c0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:33:59.858840 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:33:59.860312 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 20:33:59.861222 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:33:59.863916 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 440/879] 10.222.0.16 () {56 vars in 1132 bytes} [Sat Sep 3 20:33:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:33:59.869844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679861db-bcbb-40c5-80e9-4e1408dd1983 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:33:59.934596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:33:59.986563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679861db-bcbb-40c5-80e9-4e1408dd1983 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:00.023382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74e6c0-351e-40d1-aad0-5a89f50af541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:00.024399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74e6c0-351e-40d1-aad0-5a89f50af541 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:00.024883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74e6c0-351e-40d1-aad0-5a89f50af541 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:00.025413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74e6c0-351e-40d1-aad0-5a89f50af541 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:00.026262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74e6c0-351e-40d1-aad0-5a89f50af541 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:00.026767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74e6c0-351e-40d1-aad0-5a89f50af541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:00.028328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74e6c0-351e-40d1-aad0-5a89f50af541 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:00.034246 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 440/880] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 3 20:33:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:00.042009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b34d6-3e13-404b-8c79-59026ef2a6c0 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:00.098411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78399c6-95b3-41bb-b3b7-ddd6651704f6 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:00.099060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78399c6-95b3-41bb-b3b7-ddd6651704f6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:00.099573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78399c6-95b3-41bb-b3b7-ddd6651704f6 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:00.101993 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 441/881] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:00.106832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee217a6-c2b1-4784-958f-16144c7e72a2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:00.107126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee217a6-c2b1-4784-958f-16144c7e72a2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:00.107409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee217a6-c2b1-4784-958f-16144c7e72a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:00.130228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-dee217a6-c2b1-4784-958f-16144c7e72a2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:00.131516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dee217a6-c2b1-4784-958f-16144c7e72a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:00.167012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b34d6-3e13-404b-8c79-59026ef2a6c0 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:00.178786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:00.204626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1870be-ecf2-45b4-8ae2-457d326ed940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:00.205280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1870be-ecf2-45b4-8ae2-457d326ed940 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:00.205567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1870be-ecf2-45b4-8ae2-457d326ed940 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:00.205867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1870be-ecf2-45b4-8ae2-457d326ed940 None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:00.206580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1870be-ecf2-45b4-8ae2-457d326ed940 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:00.206888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1870be-ecf2-45b4-8ae2-457d326ed940 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:00.208521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1870be-ecf2-45b4-8ae2-457d326ed940 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:00.214108 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 441/882] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 3 20:34:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:00.215062 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 442/883] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:00.222076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1870be-ecf2-45b4-8ae2-457d326ed940 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:00.335203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1870be-ecf2-45b4-8ae2-457d326ed940 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:00.370739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8572069-827a-4936-bf71-5baddc13f7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:00.371350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8572069-827a-4936-bf71-5baddc13f7c0 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:00.371634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8572069-827a-4936-bf71-5baddc13f7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:00.371913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8572069-827a-4936-bf71-5baddc13f7c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:00.372612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8572069-827a-4936-bf71-5baddc13f7c0 None admin] RBAC: Adding query filter params (interface=public, service_id=dde7809ec97943278da09344805070f7, region_id=RegionOne) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:00.372945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8572069-827a-4936-bf71-5baddc13f7c0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:00.374322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8572069-827a-4936-bf71-5baddc13f7c0 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:00.379845 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 442/884] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 3 20:34:00 2022] GET /identity/v3/endpoints?service_id=dde7809ec97943278da09344805070f7&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:00.385256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee217a6-c2b1-4784-958f-16144c7e72a2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:00.413389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d52e0-5368-485c-8afd-dc1a65e355bd None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:00.413696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d52e0-5368-485c-8afd-dc1a65e355bd None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:00.413965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d52e0-5368-485c-8afd-dc1a65e355bd None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:00.415433 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 443/885] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:34:00.421482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe866cf-b9ce-4a3b-8c82-6285d307662b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:00.421777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe866cf-b9ce-4a3b-8c82-6285d307662b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:00.422056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe866cf-b9ce-4a3b-8c82-6285d307662b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:00.441796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1fe866cf-b9ce-4a3b-8c82-6285d307662b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:00.443601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1fe866cf-b9ce-4a3b-8c82-6285d307662b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:00.456234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:00.487935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:00.524014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee217a6-c2b1-4784-958f-16144c7e72a2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:00.525234 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 444/886] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:00.563645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe98e67-7c56-40b4-a626-6f9f0ff59fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:00.565058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe98e67-7c56-40b4-a626-6f9f0ff59fae None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:00.565491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe98e67-7c56-40b4-a626-6f9f0ff59fae None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:00.565956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe98e67-7c56-40b4-a626-6f9f0ff59fae None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:00.566702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe98e67-7c56-40b4-a626-6f9f0ff59fae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:00.568205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe98e67-7c56-40b4-a626-6f9f0ff59fae None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:00.579377 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 443/887] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 3 20:34:00 2022] GET /identity/v3/services => generated 1268 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:34:00.588773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe866cf-b9ce-4a3b-8c82-6285d307662b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:00.622769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3142a4-8bb8-4c11-bd3f-83fad26ba02b None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:00.623062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3142a4-8bb8-4c11-bd3f-83fad26ba02b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:00.623342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3142a4-8bb8-4c11-bd3f-83fad26ba02b None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:00.629033 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 444/888] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:34:00.630927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d2676-ceec-467a-851a-d87c296c2927 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:00.631224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d2676-ceec-467a-851a-d87c296c2927 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:00.631515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d2676-ceec-467a-851a-d87c296c2927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:00.651497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e88d2676-ceec-467a-851a-d87c296c2927 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:00.652737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e88d2676-ceec-467a-851a-d87c296c2927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:00.712049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe866cf-b9ce-4a3b-8c82-6285d307662b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:00.728707 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 445/889] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:00.752280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41667c5f-788b-4965-99e5-2977bc0ba927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:00.752967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41667c5f-788b-4965-99e5-2977bc0ba927 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:00.753259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41667c5f-788b-4965-99e5-2977bc0ba927 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:00.753551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41667c5f-788b-4965-99e5-2977bc0ba927 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:00.754136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41667c5f-788b-4965-99e5-2977bc0ba927 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:00.755518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41667c5f-788b-4965-99e5-2977bc0ba927 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-41667c5f-788b-4965-99e5-2977bc0ba927 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:00.765040 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:00.766717 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:00.768219 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 445/890] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 3 20:34:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:00.772552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d2676-ceec-467a-851a-d87c296c2927 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:00.821164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:00.881042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d2676-ceec-467a-851a-d87c296c2927 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:00.893773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41667c5f-788b-4965-99e5-2977bc0ba927 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:00.918833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bff05c-bd34-46a8-9ee5-f467388f435f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:00.919817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bff05c-bd34-46a8-9ee5-f467388f435f None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:00.920308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bff05c-bd34-46a8-9ee5-f467388f435f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:00.920813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bff05c-bd34-46a8-9ee5-f467388f435f None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:00.921683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bff05c-bd34-46a8-9ee5-f467388f435f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:00.922788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bff05c-bd34-46a8-9ee5-f467388f435f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:00.924452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bff05c-bd34-46a8-9ee5-f467388f435f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:00.929594 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 446/891] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 3 20:34:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:00.933825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:00.935262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bff05c-bd34-46a8-9ee5-f467388f435f None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:01.027923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41667c5f-788b-4965-99e5-2977bc0ba927 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:01.047671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bff05c-bd34-46a8-9ee5-f467388f435f None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:01.058961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:01.075241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f819c-c2ea-45ac-8ce6-a161393ec273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:01.075878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f819c-c2ea-45ac-8ce6-a161393ec273 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:01.076160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f819c-c2ea-45ac-8ce6-a161393ec273 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:01.076437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f819c-c2ea-45ac-8ce6-a161393ec273 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:01.085230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bd0a8-42f6-4792-900a-e755338e59fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:01.085840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bd0a8-42f6-4792-900a-e755338e59fa None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:01.086126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bd0a8-42f6-4792-900a-e755338e59fa None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:01.086406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bd0a8-42f6-4792-900a-e755338e59fa None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:01.087041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933bd0a8-42f6-4792-900a-e755338e59fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:01.087351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933bd0a8-42f6-4792-900a-e755338e59fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:01.087745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f819c-c2ea-45ac-8ce6-a161393ec273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:34:01.088727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933bd0a8-42f6-4792-900a-e755338e59fa None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-545f819c-c2ea-45ac-8ce6-a161393ec273 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:34:01.093014 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:01.093900 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:01.095297 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 446/892] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Sep 3 20:34:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:01.095989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f819c-c2ea-45ac-8ce6-a161393ec273 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:01.101983 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 447/893] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 3 20:34:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:01.105978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bd0a8-42f6-4792-900a-e755338e59fa None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:01.226026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:01.237759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f819c-c2ea-45ac-8ce6-a161393ec273 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:01.240573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bd0a8-42f6-4792-900a-e755338e59fa None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:01.277904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf4e0f-206e-4067-9ff4-b29c042d01ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:01.278914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf4e0f-206e-4067-9ff4-b29c042d01ec None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:01.279163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23eacd-a863-4680-bea3-ed7fb79757f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:01.279660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf4e0f-206e-4067-9ff4-b29c042d01ec None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:01.279785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23eacd-a863-4680-bea3-ed7fb79757f3 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:01.280053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23eacd-a863-4680-bea3-ed7fb79757f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:01.280324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23eacd-a863-4680-bea3-ed7fb79757f3 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:01.280677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf4e0f-206e-4067-9ff4-b29c042d01ec None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:01.281042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23eacd-a863-4680-bea3-ed7fb79757f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:01.281354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23eacd-a863-4680-bea3-ed7fb79757f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:01.282455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf4e0f-206e-4067-9ff4-b29c042d01ec None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:01.283301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23eacd-a863-4680-bea3-ed7fb79757f3 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:01.284324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf4e0f-206e-4067-9ff4-b29c042d01ec None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:01.292401 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 448/894] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 3 20:34:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-60bf4e0f-206e-4067-9ff4-b29c042d01ec None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:01.295301 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:01.296241 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 20:34:01.298214 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:01.299148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23eacd-a863-4680-bea3-ed7fb79757f3 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:01.302687 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 447/895] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:34:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:01.305375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf4e0f-206e-4067-9ff4-b29c042d01ec None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:01.341280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:01.414987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23eacd-a863-4680-bea3-ed7fb79757f3 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:01.434213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf4e0f-206e-4067-9ff4-b29c042d01ec None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:01.452795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf9068-1f24-4446-8044-53905e5996f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:01.453628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf9068-1f24-4446-8044-53905e5996f4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:01.454016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf9068-1f24-4446-8044-53905e5996f4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:01.454392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf9068-1f24-4446-8044-53905e5996f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:01.466367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faf9068-1f24-4446-8044-53905e5996f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:01.468039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faf9068-1f24-4446-8044-53905e5996f4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:01.475798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a4bbe-fdca-46a4-ac7d-9b452dd4c57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:01.476423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a4bbe-fdca-46a4-ac7d-9b452dd4c57d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:01.476711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a4bbe-fdca-46a4-ac7d-9b452dd4c57d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:01.477023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a4bbe-fdca-46a4-ac7d-9b452dd4c57d None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:01.478151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a4bbe-fdca-46a4-ac7d-9b452dd4c57d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:01.478462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a4bbe-fdca-46a4-ac7d-9b452dd4c57d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:01.480140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a4bbe-fdca-46a4-ac7d-9b452dd4c57d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-8faf9068-1f24-4446-8044-53905e5996f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:01.484385 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:01.486097 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:01.487181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:01.487181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:01.487181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:01.488467 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 448/896] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:34:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:01.488571 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 449/897] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:01.494242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a4bbe-fdca-46a4-ac7d-9b452dd4c57d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:01.494366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf9068-1f24-4446-8044-53905e5996f4 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:01.516954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:01.599515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a4bbe-fdca-46a4-ac7d-9b452dd4c57d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:01.601439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:01.619089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf9068-1f24-4446-8044-53905e5996f4 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:01.639879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb5b1e-3c8a-4886-aa06-b841432b666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:01.640763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb5b1e-3c8a-4886-aa06-b841432b666f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:01.641776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb5b1e-3c8a-4886-aa06-b841432b666f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:01.642421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb5b1e-3c8a-4886-aa06-b841432b666f None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:01.643266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb5b1e-3c8a-4886-aa06-b841432b666f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:01.643810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb5b1e-3c8a-4886-aa06-b841432b666f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:01.645375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb5b1e-3c8a-4886-aa06-b841432b666f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:01.661070 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 449/898] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:01.663865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb5b1e-3c8a-4886-aa06-b841432b666f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:01.665847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258cbbd-ebff-4117-8d6c-78b22f96625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:01.666415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258cbbd-ebff-4117-8d6c-78b22f96625a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:01.666700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258cbbd-ebff-4117-8d6c-78b22f96625a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:01.667097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258cbbd-ebff-4117-8d6c-78b22f96625a None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:01.667780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258cbbd-ebff-4117-8d6c-78b22f96625a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:01.668098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258cbbd-ebff-4117-8d6c-78b22f96625a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:01.669993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258cbbd-ebff-4117-8d6c-78b22f96625a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:01.676544 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 450/899] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:34:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:01.682731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258cbbd-ebff-4117-8d6c-78b22f96625a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:01.760400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:01.784232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb5b1e-3c8a-4886-aa06-b841432b666f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:01.797323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258cbbd-ebff-4117-8d6c-78b22f96625a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:01.823128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096b3ce-9588-4c86-90d2-226b730fd4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:01.824223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096b3ce-9588-4c86-90d2-226b730fd4eb None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:01.824720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096b3ce-9588-4c86-90d2-226b730fd4eb None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:01.825296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096b3ce-9588-4c86-90d2-226b730fd4eb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:01.834170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096b3ce-9588-4c86-90d2-226b730fd4eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:01.834411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fa8ce-6a67-4da4-96ae-bd0972d4de7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:01.835027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fa8ce-6a67-4da4-96ae-bd0972d4de7d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:01.835312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fa8ce-6a67-4da4-96ae-bd0972d4de7d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:01.835593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fa8ce-6a67-4da4-96ae-bd0972d4de7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:01.836302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fa8ce-6a67-4da4-96ae-bd0972d4de7d None admin] RBAC: Adding query filter params (interface=public, service_id=b11134f9d2f748af904fbc00a9397eb8, region_id=RegionOne) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:01.836617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fa8ce-6a67-4da4-96ae-bd0972d4de7d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:01.837310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096b3ce-9588-4c86-90d2-226b730fd4eb None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:01.838013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fa8ce-6a67-4da4-96ae-bd0972d4de7d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:01.843638 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 451/900] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 3 20:34:01 2022] GET /identity/v3/endpoints?service_id=b11134f9d2f748af904fbc00a9397eb8&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:01.849032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fa8ce-6a67-4da4-96ae-bd0972d4de7d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-0096b3ce-9588-4c86-90d2-226b730fd4eb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:34:01.854555 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:01.855723 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:01.856865 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:01.856865 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:01.856865 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:34:01.856865 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:34:01.856865 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:34:01.856865 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:34:01.856865 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:34:01.856865 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:34:01.856865 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 20:34:01.856865 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:01.858935 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 450/901] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 3 20:34:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:01.864126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096b3ce-9588-4c86-90d2-226b730fd4eb None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:01.869462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:01.961849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fa8ce-6a67-4da4-96ae-bd0972d4de7d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:01.998802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc351621-90c9-48a7-91c1-ba61ea533fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:01.999432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc351621-90c9-48a7-91c1-ba61ea533fc9 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:01.999715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc351621-90c9-48a7-91c1-ba61ea533fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:01.999991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc351621-90c9-48a7-91c1-ba61ea533fc9 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:02.001148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc351621-90c9-48a7-91c1-ba61ea533fc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:02.002031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096b3ce-9588-4c86-90d2-226b730fd4eb None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:02.002515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc351621-90c9-48a7-91c1-ba61ea533fc9 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:02.017201 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 452/902] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 3 20:34:01 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:34:02.041088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee8b5c-a09c-4af4-98a0-872676d6a0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:02.042099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee8b5c-a09c-4af4-98a0-872676d6a0af None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:02.042595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee8b5c-a09c-4af4-98a0-872676d6a0af None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:02.043084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee8b5c-a09c-4af4-98a0-872676d6a0af None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:02.044075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee8b5c-a09c-4af4-98a0-872676d6a0af None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:02.044633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee8b5c-a09c-4af4-98a0-872676d6a0af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:02.046358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee8b5c-a09c-4af4-98a0-872676d6a0af None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:02.067800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:02.068125 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 451/903] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Sep 3 20:34:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:02.073339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc351621-90c9-48a7-91c1-ba61ea533fc9 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:02.168684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:02.192580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc351621-90c9-48a7-91c1-ba61ea533fc9 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:02.230506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406d5bc-e207-4974-8cee-b141fe905174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:02.231055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406d5bc-e207-4974-8cee-b141fe905174 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:02.231346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406d5bc-e207-4974-8cee-b141fe905174 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:02.231631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406d5bc-e207-4974-8cee-b141fe905174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:02.304471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406d5bc-e207-4974-8cee-b141fe905174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:02.305982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406d5bc-e207-4974-8cee-b141fe905174 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:02.393087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:02.414980 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 453/904] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:34:02 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:34:02.420340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee8b5c-a09c-4af4-98a0-872676d6a0af None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:02.524437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee8b5c-a09c-4af4-98a0-872676d6a0af None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:02.566743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b237b16-6ad0-41aa-b948-c512e317961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:02.567754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b237b16-6ad0-41aa-b948-c512e317961c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:02.568266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b237b16-6ad0-41aa-b948-c512e317961c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:02.568745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b237b16-6ad0-41aa-b948-c512e317961c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:02.579616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b237b16-6ad0-41aa-b948-c512e317961c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:02.581378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b237b16-6ad0-41aa-b948-c512e317961c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-8b237b16-6ad0-41aa-b948-c512e317961c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:02.597233 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:02.598187 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:02.599073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:02.599073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:02.599073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:02.601104 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 452/905] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:02.612196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406d5bc-e207-4974-8cee-b141fe905174 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:02.709872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406d5bc-e207-4974-8cee-b141fe905174 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:02.745804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5cd99-d93e-4413-a7bc-501a50f4d1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:02.746434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5cd99-d93e-4413-a7bc-501a50f4d1de None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:02.746720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5cd99-d93e-4413-a7bc-501a50f4d1de None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:02.746997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5cd99-d93e-4413-a7bc-501a50f4d1de None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:02.747656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5cd99-d93e-4413-a7bc-501a50f4d1de None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:02.747963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5cd99-d93e-4413-a7bc-501a50f4d1de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:02.749435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5cd99-d93e-4413-a7bc-501a50f4d1de None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:02.768731 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 454/906] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:02.770241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b237b16-6ad0-41aa-b948-c512e317961c None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:02.822055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:02.878192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b237b16-6ad0-41aa-b948-c512e317961c None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:02.913788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a478c4e-5790-4c22-b57f-54032460a7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:02.914429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a478c4e-5790-4c22-b57f-54032460a7a1 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:02.914718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a478c4e-5790-4c22-b57f-54032460a7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:02.914998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a478c4e-5790-4c22-b57f-54032460a7a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:02.921919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a478c4e-5790-4c22-b57f-54032460a7a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:02.923412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a478c4e-5790-4c22-b57f-54032460a7a1 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-9a478c4e-5790-4c22-b57f-54032460a7a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:34:02.934881 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:02.935789 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:02.936693 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:34:02.936693 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:34:02.936693 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:34:02.936693 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:34:02.936693 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:34:02.936693 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:34:02.936693 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:02.936693 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:02.936693 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 453/907] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:34:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:02.941316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9630744-2cd8-428e-8239-809735221217 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:02.941838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9630744-2cd8-428e-8239-809735221217 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:02.942121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9630744-2cd8-428e-8239-809735221217 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:02.942931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a478c4e-5790-4c22-b57f-54032460a7a1 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:02.943594 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 455/908] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:02.948588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d6a89-dc03-4658-a2f8-536f6ac95c8e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:02.949146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d6a89-dc03-4658-a2f8-536f6ac95c8e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:02.949642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d6a89-dc03-4658-a2f8-536f6ac95c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:02.971187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a31d6a89-dc03-4658-a2f8-536f6ac95c8e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:02.972960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a31d6a89-dc03-4658-a2f8-536f6ac95c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:03.012116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:03.049065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a478c4e-5790-4c22-b57f-54032460a7a1 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:03.052126 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 456/909] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:03.065822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:03.085187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c37333-31da-4645-bc4e-e5d32eae55bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:03.085734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c37333-31da-4645-bc4e-e5d32eae55bf None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:03.086028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c37333-31da-4645-bc4e-e5d32eae55bf None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:03.086311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c37333-31da-4645-bc4e-e5d32eae55bf None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:03.087014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c37333-31da-4645-bc4e-e5d32eae55bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:03.087437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c37333-31da-4645-bc4e-e5d32eae55bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:03.088890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c37333-31da-4645-bc4e-e5d32eae55bf None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:03.102170 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 454/910] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:34:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:03.108445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d6a89-dc03-4658-a2f8-536f6ac95c8e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:03.218906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d6a89-dc03-4658-a2f8-536f6ac95c8e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:03.260475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6b0dd-3baa-47a7-92e8-14f1a37dc531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:03.261520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b0dd-3baa-47a7-92e8-14f1a37dc531 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:03.262007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b0dd-3baa-47a7-92e8-14f1a37dc531 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:03.262483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b0dd-3baa-47a7-92e8-14f1a37dc531 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:03.263439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6b0dd-3baa-47a7-92e8-14f1a37dc531 None admin] RBAC: Adding query filter params (role.id=95f5b6cfc093417cbb9882b6897e875f, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=9388372667f8454498b2e4480187b47c) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:03.263950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6b0dd-3baa-47a7-92e8-14f1a37dc531 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:03.265618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6b0dd-3baa-47a7-92e8-14f1a37dc531 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:03.277026 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 457/911] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:34:03 2022] GET /identity/v3/role_assignments?user.id=9388372667f8454498b2e4480187b47c&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=95f5b6cfc093417cbb9882b6897e875f => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:03.399179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c37333-31da-4645-bc4e-e5d32eae55bf None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:03.480252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9352e7-69b6-447e-ab51-29fcd6d7cef7 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:03.480763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9352e7-69b6-447e-ab51-29fcd6d7cef7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:03.480839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9352e7-69b6-447e-ab51-29fcd6d7cef7 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:03.482526 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 458/912] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:03.488249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ba120-e8d7-4cd6-92d1-4edab1b3fe3d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:03.488547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ba120-e8d7-4cd6-92d1-4edab1b3fe3d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:03.488841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ba120-e8d7-4cd6-92d1-4edab1b3fe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:03.507099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-253ba120-e8d7-4cd6-92d1-4edab1b3fe3d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:03.508303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-253ba120-e8d7-4cd6-92d1-4edab1b3fe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:03.513495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c37333-31da-4645-bc4e-e5d32eae55bf None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:03.539915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:03.550505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf5743-8c4d-407f-94ab-fb7349db4525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:03.551150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf5743-8c4d-407f-94ab-fb7349db4525 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:03.551441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf5743-8c4d-407f-94ab-fb7349db4525 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:03.551724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf5743-8c4d-407f-94ab-fb7349db4525 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:03.552936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf5743-8c4d-407f-94ab-fb7349db4525 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:03.553738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf5743-8c4d-407f-94ab-fb7349db4525 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-8eaf5743-8c4d-407f-94ab-fb7349db4525 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:03.562908 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:03.564039 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:03.564995 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 455/913] 10.222.0.16 () {56 vars in 1132 bytes} [Sat Sep 3 20:34:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:03.569817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf5743-8c4d-407f-94ab-fb7349db4525 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:03.596807 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 459/914] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:03.613358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:03.680349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf5743-8c4d-407f-94ab-fb7349db4525 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:03.715809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e0a87-ad25-47c5-bbb2-3afc2056719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:03.716424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e0a87-ad25-47c5-bbb2-3afc2056719c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:03.716711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e0a87-ad25-47c5-bbb2-3afc2056719c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:03.717024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e0a87-ad25-47c5-bbb2-3afc2056719c None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:03.717657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e0a87-ad25-47c5-bbb2-3afc2056719c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:03.717967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e0a87-ad25-47c5-bbb2-3afc2056719c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:03.720107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e0a87-ad25-47c5-bbb2-3afc2056719c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:03.725402 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 456/915] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 3 20:34:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:03.731415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ba120-e8d7-4cd6-92d1-4edab1b3fe3d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:03.844751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ba120-e8d7-4cd6-92d1-4edab1b3fe3d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:03.880369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d2755-c7f6-4e9e-97ed-cacf2bca3748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:03.881402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d2755-c7f6-4e9e-97ed-cacf2bca3748 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:03.881895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d2755-c7f6-4e9e-97ed-cacf2bca3748 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:03.882391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d2755-c7f6-4e9e-97ed-cacf2bca3748 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:03.883236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d2755-c7f6-4e9e-97ed-cacf2bca3748 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:03.883761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d2755-c7f6-4e9e-97ed-cacf2bca3748 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:03.885363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d2755-c7f6-4e9e-97ed-cacf2bca3748 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:03.891589 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 460/916] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 3 20:34:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:03.897450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e0a87-ad25-47c5-bbb2-3afc2056719c None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:03.959393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d2755-c7f6-4e9e-97ed-cacf2bca3748 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:03.991328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:04.001819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:04.005836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e0a87-ad25-47c5-bbb2-3afc2056719c None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:04.042225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b938601-3a12-4f63-aca3-40a8f632efcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:04.042843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b938601-3a12-4f63-aca3-40a8f632efcf None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:04.043131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b938601-3a12-4f63-aca3-40a8f632efcf None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:04.043409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b938601-3a12-4f63-aca3-40a8f632efcf None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:04.044101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b938601-3a12-4f63-aca3-40a8f632efcf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:04.045509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b938601-3a12-4f63-aca3-40a8f632efcf None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:04.069162 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 457/917] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:34:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:34:04.082146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d2755-c7f6-4e9e-97ed-cacf2bca3748 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:04.137713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e748c-ab4e-439b-91d9-90ad1971f234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:04.138340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e748c-ab4e-439b-91d9-90ad1971f234 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:04.138629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e748c-ab4e-439b-91d9-90ad1971f234 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:04.138907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e748c-ab4e-439b-91d9-90ad1971f234 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:04.139705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e748c-ab4e-439b-91d9-90ad1971f234 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:04.141724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e748c-ab4e-439b-91d9-90ad1971f234 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:04.165487 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 461/918] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:34:03 2022] POST /identity/v3/services => generated 241 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:34:04.401636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7f0a7-f680-413a-b040-c65a99348b08 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:04.401981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7f0a7-f680-413a-b040-c65a99348b08 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:04.402159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7f0a7-f680-413a-b040-c65a99348b08 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:04.403731 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 458/919] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:04.410022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658685ec-d972-4045-8e0a-14dc7bdb8a83 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:04.410313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658685ec-d972-4045-8e0a-14dc7bdb8a83 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:04.410590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658685ec-d972-4045-8e0a-14dc7bdb8a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:04.429464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-658685ec-d972-4045-8e0a-14dc7bdb8a83 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:04.430660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-658685ec-d972-4045-8e0a-14dc7bdb8a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:04.513565 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 462/920] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:04.837511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7f0a7-f680-413a-b040-c65a99348b08 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:04.961330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7f0a7-f680-413a-b040-c65a99348b08 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:04.996598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4ff15-97fe-42e0-996b-3a9c05e90e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:04.997286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4ff15-97fe-42e0-996b-3a9c05e90e09 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:04.997573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4ff15-97fe-42e0-996b-3a9c05e90e09 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:04.997851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4ff15-97fe-42e0-996b-3a9c05e90e09 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:04.998499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4ff15-97fe-42e0-996b-3a9c05e90e09 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:04.999865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4ff15-97fe-42e0-996b-3a9c05e90e09 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-a5e4ff15-97fe-42e0-996b-3a9c05e90e09 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:05.009248 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:05.011457 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:05.013419 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 459/921] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:34:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:05.016920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658685ec-d972-4045-8e0a-14dc7bdb8a83 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:05.028857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:05.126719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658685ec-d972-4045-8e0a-14dc7bdb8a83 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:05.164584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c6d1c-b1e0-4ffd-bb76-6042d15ccf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:05.165246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c6d1c-b1e0-4ffd-bb76-6042d15ccf77 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:05.165527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c6d1c-b1e0-4ffd-bb76-6042d15ccf77 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:05.165805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c6d1c-b1e0-4ffd-bb76-6042d15ccf77 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:05.166443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c6d1c-b1e0-4ffd-bb76-6042d15ccf77 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:05.166807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c6d1c-b1e0-4ffd-bb76-6042d15ccf77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:05.168196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c6d1c-b1e0-4ffd-bb76-6042d15ccf77 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:05.173521 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 463/922] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:34:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:05.178985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4ff15-97fe-42e0-996b-3a9c05e90e09 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:05.198605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:05.287989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4ff15-97fe-42e0-996b-3a9c05e90e09 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:05.327449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed0eb5-16f5-410f-9d70-53af82ae67da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:05.328065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed0eb5-16f5-410f-9d70-53af82ae67da None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:05.328860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed0eb5-16f5-410f-9d70-53af82ae67da None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:05.329216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed0eb5-16f5-410f-9d70-53af82ae67da None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:05.329858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed0eb5-16f5-410f-9d70-53af82ae67da None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:05.330169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed0eb5-16f5-410f-9d70-53af82ae67da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:05.331623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed0eb5-16f5-410f-9d70-53af82ae67da None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:05.342728 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 460/923] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 3 20:34:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:05.344922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c6d1c-b1e0-4ffd-bb76-6042d15ccf77 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:05.423844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:05.453935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c6d1c-b1e0-4ffd-bb76-6042d15ccf77 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:05.492054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6964bc-a1a2-4671-90d3-46d7b62bc809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:05.492978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6964bc-a1a2-4671-90d3-46d7b62bc809 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:05.493431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6964bc-a1a2-4671-90d3-46d7b62bc809 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:05.493859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6964bc-a1a2-4671-90d3-46d7b62bc809 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:05.494734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6964bc-a1a2-4671-90d3-46d7b62bc809 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:05.496320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6964bc-a1a2-4671-90d3-46d7b62bc809 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:05.521069 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 464/924] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:34:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:34:05.621969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba5137-c751-4119-ba05-f4ea3b098faf None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:05.622427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba5137-c751-4119-ba05-f4ea3b098faf None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:05.622804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba5137-c751-4119-ba05-f4ea3b098faf None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:05.624565 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 461/925] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:05.630314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dffb8-ba0f-4b03-a8ea-7a9ebfbf9e70 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:05.630605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dffb8-ba0f-4b03-a8ea-7a9ebfbf9e70 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:05.630883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dffb8-ba0f-4b03-a8ea-7a9ebfbf9e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:05.650889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-806dffb8-ba0f-4b03-a8ea-7a9ebfbf9e70 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:05.652119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-806dffb8-ba0f-4b03-a8ea-7a9ebfbf9e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:05.735546 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 465/926] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:06.059871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba5137-c751-4119-ba05-f4ea3b098faf None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:06.146879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:06.168723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba5137-c751-4119-ba05-f4ea3b098faf None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:06.206442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b5801-d345-4c69-a43f-138bff2b20e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:06.207091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b5801-d345-4c69-a43f-138bff2b20e9 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:06.207631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b5801-d345-4c69-a43f-138bff2b20e9 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:06.207914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b5801-d345-4c69-a43f-138bff2b20e9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:06.218439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b5801-d345-4c69-a43f-138bff2b20e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-030b5801-d345-4c69-a43f-138bff2b20e9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:34:06.223987 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:06.224886 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:06.225770 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 462/927] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Sep 3 20:34:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:06.232973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dffb8-ba0f-4b03-a8ea-7a9ebfbf9e70 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:06.294247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:06.351586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dffb8-ba0f-4b03-a8ea-7a9ebfbf9e70 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:06.390474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ea1ab-e77a-4a06-beec-f7f8de8a956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:06.391674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ea1ab-e77a-4a06-beec-f7f8de8a956f None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:06.391963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ea1ab-e77a-4a06-beec-f7f8de8a956f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:06.392245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ea1ab-e77a-4a06-beec-f7f8de8a956f None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:06.392974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ea1ab-e77a-4a06-beec-f7f8de8a956f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:06.393293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ea1ab-e77a-4a06-beec-f7f8de8a956f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:06.394740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ea1ab-e77a-4a06-beec-f7f8de8a956f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:06.404875 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 466/928] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 3 20:34:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:06.414230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b5801-d345-4c69-a43f-138bff2b20e9 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:06.463977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabbf7a-39d2-483d-a7c4-fef6ccd3822d None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:06.464264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabbf7a-39d2-483d-a7c4-fef6ccd3822d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:06.464703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabbf7a-39d2-483d-a7c4-fef6ccd3822d None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:06.466442 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 467/929] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:34:06.471403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d7f1d-a397-48d9-9bd3-e6c841dba421 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:06.471883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d7f1d-a397-48d9-9bd3-e6c841dba421 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:06.472323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d7f1d-a397-48d9-9bd3-e6c841dba421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:06.497045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ec0d7f1d-a397-48d9-9bd3-e6c841dba421 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:06.498973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ec0d7f1d-a397-48d9-9bd3-e6c841dba421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:06.524538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b5801-d345-4c69-a43f-138bff2b20e9 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:06.560164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88192f00-ccfc-4826-a778-6f74587ab3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:06.560935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88192f00-ccfc-4826-a778-6f74587ab3ca None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:06.561321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88192f00-ccfc-4826-a778-6f74587ab3ca None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:06.561581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88192f00-ccfc-4826-a778-6f74587ab3ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:06.573661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88192f00-ccfc-4826-a778-6f74587ab3ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:06.579785 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 468/930] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:06.580304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88192f00-ccfc-4826-a778-6f74587ab3ca None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:06.580385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dae2d0-d345-414e-8ace-de374bbd5e70 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:06.580385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dae2d0-d345-414e-8ace-de374bbd5e70 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:06.580385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dae2d0-d345-414e-8ace-de374bbd5e70 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:06.580385 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 469/931] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:06.583500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:06.583709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75d667-5d06-491d-b630-1ce624e98573 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:06.584005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75d667-5d06-491d-b630-1ce624e98573 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:06.584287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75d667-5d06-491d-b630-1ce624e98573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-88192f00-ccfc-4826-a778-6f74587ab3ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:06.602671 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:06.603828 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:06.605264 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:06.605264 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:06.605264 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:06.605550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8e75d667-5d06-491d-b630-1ce624e98573 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:06.605550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8e75d667-5d06-491d-b630-1ce624e98573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:06.615137 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 463/932] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:06.616324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88192f00-ccfc-4826-a778-6f74587ab3ca None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:06.654189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:06.685041 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 470/933] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:06.740769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88192f00-ccfc-4826-a778-6f74587ab3ca None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:06.776735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6982b-29ee-4bad-bee9-c636a9125873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:06.777652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6982b-29ee-4bad-bee9-c636a9125873 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:06.778116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6982b-29ee-4bad-bee9-c636a9125873 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:06.778549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6982b-29ee-4bad-bee9-c636a9125873 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:06.779358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6982b-29ee-4bad-bee9-c636a9125873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:06.779814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6982b-29ee-4bad-bee9-c636a9125873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:06.781384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6982b-29ee-4bad-bee9-c636a9125873 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:06.794768 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 464/934] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:06.802142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75d667-5d06-491d-b630-1ce624e98573 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:06.902960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75d667-5d06-491d-b630-1ce624e98573 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:06.914523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6982b-29ee-4bad-bee9-c636a9125873 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:06.940596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be90800-b66d-47f3-b489-b35a62c9778a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:06.941603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be90800-b66d-47f3-b489-b35a62c9778a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:06.942101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be90800-b66d-47f3-b489-b35a62c9778a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:06.942586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be90800-b66d-47f3-b489-b35a62c9778a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:06.950161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be90800-b66d-47f3-b489-b35a62c9778a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:06.951714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be90800-b66d-47f3-b489-b35a62c9778a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-1be90800-b66d-47f3-b489-b35a62c9778a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:34:06.964229 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:06.965449 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:06.966779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:06.966779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:06.966779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:34:06.966779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:34:06.966779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:34:06.966779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:34:06.966779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:34:06.966779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:34:06.966779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 20:34:06.966779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:06.969557 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 471/935] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 3 20:34:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:06.972327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be90800-b66d-47f3-b489-b35a62c9778a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:06.989556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:07.016713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:07.046350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6982b-29ee-4bad-bee9-c636a9125873 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:07.088516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be90800-b66d-47f3-b489-b35a62c9778a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:07.102514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ce3ce-024c-4fee-a65f-fcbdea82a539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:07.104335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ce3ce-024c-4fee-a65f-fcbdea82a539 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:07.104791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ce3ce-024c-4fee-a65f-fcbdea82a539 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:07.105320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ce3ce-024c-4fee-a65f-fcbdea82a539 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:07.106404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ce3ce-024c-4fee-a65f-fcbdea82a539 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:07.108729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ce3ce-024c-4fee-a65f-fcbdea82a539 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-cd0ce3ce-024c-4fee-a65f-fcbdea82a539 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:07.127764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:07.128993 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:07.130271 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:07.135495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481e909-bc12-4e34-9fe3-0c4b1c0dad6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:07.136210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481e909-bc12-4e34-9fe3-0c4b1c0dad6b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:07.136590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481e909-bc12-4e34-9fe3-0c4b1c0dad6b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:07.136991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481e909-bc12-4e34-9fe3-0c4b1c0dad6b None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:07.137899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481e909-bc12-4e34-9fe3-0c4b1c0dad6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:07.138290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481e909-bc12-4e34-9fe3-0c4b1c0dad6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:07.144849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481e909-bc12-4e34-9fe3-0c4b1c0dad6b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:07.145492 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 465/936] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 3 20:34:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:07.147374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ce3ce-024c-4fee-a65f-fcbdea82a539 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:07.166501 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 472/937] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Sep 3 20:34:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:07.166501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481e909-bc12-4e34-9fe3-0c4b1c0dad6b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:07.278563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ce3ce-024c-4fee-a65f-fcbdea82a539 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:07.282796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:07.293068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481e909-bc12-4e34-9fe3-0c4b1c0dad6b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:07.301882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:07.335158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24581799-50ea-4d14-91d0-658c6ecd6621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:07.335807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24581799-50ea-4d14-91d0-658c6ecd6621 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:07.336090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24581799-50ea-4d14-91d0-658c6ecd6621 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:07.336364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24581799-50ea-4d14-91d0-658c6ecd6621 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:07.337040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24581799-50ea-4d14-91d0-658c6ecd6621 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:07.338383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24581799-50ea-4d14-91d0-658c6ecd6621 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:07.345296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aee4a9-ae4f-4e8b-a47e-fc8432e2078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:07.345907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aee4a9-ae4f-4e8b-a47e-fc8432e2078c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:07.346189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aee4a9-ae4f-4e8b-a47e-fc8432e2078c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:07.346463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aee4a9-ae4f-4e8b-a47e-fc8432e2078c None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:07.347089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aee4a9-ae4f-4e8b-a47e-fc8432e2078c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-24581799-50ea-4d14-91d0-658c6ecd6621 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:07.347346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:07.348366 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 20:34:07.349262 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:07.351170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aee4a9-ae4f-4e8b-a47e-fc8432e2078c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:07.351252 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 466/938] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:34:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:07.351323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aee4a9-ae4f-4e8b-a47e-fc8432e2078c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:07.351398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24581799-50ea-4d14-91d0-658c6ecd6621 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:07.363755 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 473/939] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 3 20:34:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:07.363755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aee4a9-ae4f-4e8b-a47e-fc8432e2078c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:07.481366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aee4a9-ae4f-4e8b-a47e-fc8432e2078c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:07.502572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24581799-50ea-4d14-91d0-658c6ecd6621 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:07.519454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fa4d0-d91b-4a7f-9b0c-cc7edd27b339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:07.520075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fa4d0-d91b-4a7f-9b0c-cc7edd27b339 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:07.520368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fa4d0-d91b-4a7f-9b0c-cc7edd27b339 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:07.520653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fa4d0-d91b-4a7f-9b0c-cc7edd27b339 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:07.521316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fa4d0-d91b-4a7f-9b0c-cc7edd27b339 None admin] RBAC: Adding query filter params (name=compute) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:07.521630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fa4d0-d91b-4a7f-9b0c-cc7edd27b339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:07.523009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fa4d0-d91b-4a7f-9b0c-cc7edd27b339 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:07.528947 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 474/940] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:34:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:07.528947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fa4d0-d91b-4a7f-9b0c-cc7edd27b339 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:07.548150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be9842-290f-4f5a-ab5c-b0642a45174b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:07.548790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be9842-290f-4f5a-ab5c-b0642a45174b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:07.549099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be9842-290f-4f5a-ab5c-b0642a45174b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:07.549370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be9842-290f-4f5a-ab5c-b0642a45174b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:07.559979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be9842-290f-4f5a-ab5c-b0642a45174b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:07.560527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:07.561256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be9842-290f-4f5a-ab5c-b0642a45174b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-e6be9842-290f-4f5a-ab5c-b0642a45174b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:07.579800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:07.580825 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:07.581746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:07.581746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:07.581746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:07.581746 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 467/941] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:07.582849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be9842-290f-4f5a-ab5c-b0642a45174b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:07.620094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:07.644530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fa4d0-d91b-4a7f-9b0c-cc7edd27b339 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:07.686764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6a865-d00c-4919-9c3b-858c0cb3a1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:07.687683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6a865-d00c-4919-9c3b-858c0cb3a1e2 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:07.688662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6a865-d00c-4919-9c3b-858c0cb3a1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:07.689244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6a865-d00c-4919-9c3b-858c0cb3a1e2 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:07.689944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6a865-d00c-4919-9c3b-858c0cb3a1e2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:07.690260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6a865-d00c-4919-9c3b-858c0cb3a1e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:07.690701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be9842-290f-4f5a-ab5c-b0642a45174b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:07.691701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6a865-d00c-4919-9c3b-858c0cb3a1e2 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:07.702442 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 475/942] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 3 20:34:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:07.702442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6a865-d00c-4919-9c3b-858c0cb3a1e2 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:07.727004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f8f0a-b4a1-46d7-9d3a-de9596135c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:07.727703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f8f0a-b4a1-46d7-9d3a-de9596135c2c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:07.728455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f8f0a-b4a1-46d7-9d3a-de9596135c2c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:07.728742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f8f0a-b4a1-46d7-9d3a-de9596135c2c None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:07.729402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f8f0a-b4a1-46d7-9d3a-de9596135c2c None admin] RBAC: Adding query filter params (type=compute) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:07.729711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f8f0a-b4a1-46d7-9d3a-de9596135c2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:07.731100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f8f0a-b4a1-46d7-9d3a-de9596135c2c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:07.741053 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 468/943] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:34:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:07.741053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f8f0a-b4a1-46d7-9d3a-de9596135c2c None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:07.823012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:07.842137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6a865-d00c-4919-9c3b-858c0cb3a1e2 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:07.874381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f8f0a-b4a1-46d7-9d3a-de9596135c2c None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:07.892675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68bcda-0f62-4691-a61a-59cd18dd3888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:07.893466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68bcda-0f62-4691-a61a-59cd18dd3888 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:07.893822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68bcda-0f62-4691-a61a-59cd18dd3888 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:07.894170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68bcda-0f62-4691-a61a-59cd18dd3888 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:07.895119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68bcda-0f62-4691-a61a-59cd18dd3888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:07.895536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68bcda-0f62-4691-a61a-59cd18dd3888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:07.897433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68bcda-0f62-4691-a61a-59cd18dd3888 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:07.905323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:07.918218 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 476/944] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:07.919271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047a250-7a19-451d-8c0a-add951f9aadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:07.920125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047a250-7a19-451d-8c0a-add951f9aadc None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:07.920326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144796f-1cee-41f4-8995-883df45556df None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:07.920631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144796f-1cee-41f4-8995-883df45556df None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:07.920847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047a250-7a19-451d-8c0a-add951f9aadc None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:07.920981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144796f-1cee-41f4-8995-883df45556df None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:07.921240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047a250-7a19-451d-8c0a-add951f9aadc None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:07.922049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047a250-7a19-451d-8c0a-add951f9aadc None admin] RBAC: Adding query filter params (interface=public, service_id=69433461b5264eb780cfaf7008502360, region_id=RegionOne) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:07.923828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047a250-7a19-451d-8c0a-add951f9aadc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:07.923938 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 477/945] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:07.924345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047a250-7a19-451d-8c0a-add951f9aadc None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:07.926737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144796f-1cee-41f4-8995-883df45556df None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:07.931754 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 469/946] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 3 20:34:07 2022] GET /identity/v3/endpoints?service_id=69433461b5264eb780cfaf7008502360&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:07.933041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9f6e0-1111-4159-83d0-cd8a2003885d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:07.933336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9f6e0-1111-4159-83d0-cd8a2003885d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:07.933619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9f6e0-1111-4159-83d0-cd8a2003885d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:07.954568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-73b9f6e0-1111-4159-83d0-cd8a2003885d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:07.955787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-73b9f6e0-1111-4159-83d0-cd8a2003885d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:08.030227 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 470/947] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:08.031382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9f6e0-1111-4159-83d0-cd8a2003885d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:08.041509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144796f-1cee-41f4-8995-883df45556df None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:08.077816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f69497-84cc-4a48-8f51-66ba754a1ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:08.079227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f69497-84cc-4a48-8f51-66ba754a1ed4 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:08.079684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f69497-84cc-4a48-8f51-66ba754a1ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:08.080155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f69497-84cc-4a48-8f51-66ba754a1ed4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:08.087997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f69497-84cc-4a48-8f51-66ba754a1ed4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:08.089706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f69497-84cc-4a48-8f51-66ba754a1ed4 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-02f69497-84cc-4a48-8f51-66ba754a1ed4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:34:08.102789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:08.103683 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:08.104571 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:34:08.104571 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:34:08.104571 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:34:08.104571 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:34:08.104571 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:34:08.104571 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:34:08.104571 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:08.104571 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:08.112714 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 478/948] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:34:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:08.113882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f69497-84cc-4a48-8f51-66ba754a1ed4 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:08.135015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:08.137565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:08.147354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9f6e0-1111-4159-83d0-cd8a2003885d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:08.184576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5d753-6a42-46e9-8c6b-5b8f3901f127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:08.185562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5d753-6a42-46e9-8c6b-5b8f3901f127 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:08.186002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5d753-6a42-46e9-8c6b-5b8f3901f127 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:08.186501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5d753-6a42-46e9-8c6b-5b8f3901f127 None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:08.187326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5d753-6a42-46e9-8c6b-5b8f3901f127 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:08.188996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5d753-6a42-46e9-8c6b-5b8f3901f127 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:08.195991 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 471/949] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 3 20:34:08 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:34:08.239917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f69497-84cc-4a48-8f51-66ba754a1ed4 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:08.278925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d9d81-daf3-4d37-b389-4afdcb37307f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:08.279556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d9d81-daf3-4d37-b389-4afdcb37307f None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:08.279838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d9d81-daf3-4d37-b389-4afdcb37307f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:08.280109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d9d81-daf3-4d37-b389-4afdcb37307f None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:08.280881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d9d81-daf3-4d37-b389-4afdcb37307f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:08.281163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d9d81-daf3-4d37-b389-4afdcb37307f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:08.283200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d9d81-daf3-4d37-b389-4afdcb37307f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:08.297701 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 479/950] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:34:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:08.303718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5d753-6a42-46e9-8c6b-5b8f3901f127 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:08.400993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d9d81-daf3-4d37-b389-4afdcb37307f None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:08.425753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5d753-6a42-46e9-8c6b-5b8f3901f127 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:08.462104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7de4d-dead-48af-897a-ac1711c04178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:08.462780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7de4d-dead-48af-897a-ac1711c04178 None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:08.463067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7de4d-dead-48af-897a-ac1711c04178 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:08.463347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7de4d-dead-48af-897a-ac1711c04178 None admin] PATH_INFO: `/v3/projects/1e040b2fca724c58a2660441a72cd350/users/9388372667f8454498b2e4480187b47c/roles/95f5b6cfc093417cbb9882b6897e875f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:08.466828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7de4d-dead-48af-897a-ac1711c04178 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e040b2fca724c58a2660441a72cd350, user_id=9388372667f8454498b2e4480187b47c, role_id=95f5b6cfc093417cbb9882b6897e875f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:08.468305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7de4d-dead-48af-897a-ac1711c04178 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:08.483528 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 472/951] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:34:08 2022] PUT /identity/v3/projects/1e040b2fca724c58a2660441a72cd350/users/9388372667f8454498b2e4480187b47c/roles/95f5b6cfc093417cbb9882b6897e875f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:34:08.505218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:08.549764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d9d81-daf3-4d37-b389-4afdcb37307f None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:08.590579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1b4b7-e487-45be-abd0-fd90d1635980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:08.591623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1b4b7-e487-45be-abd0-fd90d1635980 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:08.592129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1b4b7-e487-45be-abd0-fd90d1635980 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:08.592611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1b4b7-e487-45be-abd0-fd90d1635980 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:08.593535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1b4b7-e487-45be-abd0-fd90d1635980 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:08.595766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1b4b7-e487-45be-abd0-fd90d1635980 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:08.598757 ncdv-master-1 devstack@keystone.service[99210]: WARNING keystone.server.flask.common [None req-a3a1b4b7-e487-45be-abd0-fd90d1635980 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 03 20:34:08.619533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.sql.core [None req-a3a1b4b7-e487-45be-abd0-fd90d1635980 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 03 20:34:08.619533 ncdv-master-1 devstack@keystone.service[99210]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 03 20:34:08.619533 ncdv-master-1 devstack@keystone.service[99210]: [parameters: {'id': '1fe9d0d714324f0a8690a99e217a158c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 03 20:34:08.619533 ncdv-master-1 devstack@keystone.service[99210]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99210) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a3a1b4b7-e487-45be-abd0-fd90d1635980 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1fe9d0d714324f0a8690a99e217a158c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1fe9d0d714324f0a8690a99e217a158c. Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application conn.query(q) Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 20:34:08.628499 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application result.read() Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 03 20:34:08.629451 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application next(self.gen) Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application yield resource Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application next(self.gen) Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.session.rollback() Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.force_reraise() Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise self.value Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application session.commit() Sep 03 20:34:08.630549 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._prepare_impl() Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.session.flush() Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._flush(objects) Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application compat.raise_( Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application flush_context.execute() Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 03 20:34:08.631462 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rec.execute(self) Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 03 20:34:08.632387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application conn.query(q) Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application result.read() Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 20:34:08.633768 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [parameters: {'id': '1fe9d0d714324f0a8690a99e217a158c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:08.634722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:08.635507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:08.636648 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:08.636648 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 03 20:34:08.636648 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 03 20:34:08.636648 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:08.636648 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:08.636648 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 03 20:34:08.636648 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 03 20:34:08.636648 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1fe9d0d714324f0a8690a99e217a158c. Sep 03 20:34:08.636648 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:08.639128 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 480/952] 10.222.0.16 () {60 vars in 1140 bytes} [Sat Sep 3 20:34:08 2022] POST /identity/v3/projects => generated 262 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:08.645396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7de4d-dead-48af-897a-ac1711c04178 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:08.667320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:08.763075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7de4d-dead-48af-897a-ac1711c04178 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:08.799325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa51dfd-2920-4cc1-b62d-39965ec59dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:08.800203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51dfd-2920-4cc1-b62d-39965ec59dfa None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:08.800535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51dfd-2920-4cc1-b62d-39965ec59dfa None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:08.800932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51dfd-2920-4cc1-b62d-39965ec59dfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:08.808304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51dfd-2920-4cc1-b62d-39965ec59dfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:08.809911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51dfd-2920-4cc1-b62d-39965ec59dfa None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-faa51dfd-2920-4cc1-b62d-39965ec59dfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:34:08.822219 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:08.823317 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:08.824420 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:34:08.824420 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:34:08.824420 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:34:08.824420 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:34:08.824420 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:34:08.824420 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:34:08.824420 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:08.824420 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:08.826111 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 473/953] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:34:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:08.832913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1b4b7-e487-45be-abd0-fd90d1635980 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:08.940950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1b4b7-e487-45be-abd0-fd90d1635980 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:08.966594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:08.979590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a626290-dce1-4dbc-b3aa-b2aea359da0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:08.980408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a626290-dce1-4dbc-b3aa-b2aea359da0b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:08.980698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a626290-dce1-4dbc-b3aa-b2aea359da0b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:08.981278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a626290-dce1-4dbc-b3aa-b2aea359da0b None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:08.982054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a626290-dce1-4dbc-b3aa-b2aea359da0b None admin] RBAC: Adding query filter params (name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:08.982576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a626290-dce1-4dbc-b3aa-b2aea359da0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:08.984505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a626290-dce1-4dbc-b3aa-b2aea359da0b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:09.006372 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 481/954] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:34:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:10.014687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5afeea-291e-47e0-aa17-08e328d5c4f5 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:10.015207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5afeea-291e-47e0-aa17-08e328d5c4f5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:10.015285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5afeea-291e-47e0-aa17-08e328d5c4f5 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:10.016916 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 474/955] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:10.022433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8886d9-7d8c-4294-889d-c3f31df11937 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:10.022830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8886d9-7d8c-4294-889d-c3f31df11937 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:10.023012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8886d9-7d8c-4294-889d-c3f31df11937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:10.042692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7e8886d9-7d8c-4294-889d-c3f31df11937 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:10.044235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7e8886d9-7d8c-4294-889d-c3f31df11937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:10.116096 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 482/956] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:10.453944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5afeea-291e-47e0-aa17-08e328d5c4f5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:10.541081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:10.566776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5afeea-291e-47e0-aa17-08e328d5c4f5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:10.603001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80208703-a630-44e5-94a7-98365662c65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:10.603968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80208703-a630-44e5-94a7-98365662c65f None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:10.604460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80208703-a630-44e5-94a7-98365662c65f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:10.604966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80208703-a630-44e5-94a7-98365662c65f None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:10.605859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80208703-a630-44e5-94a7-98365662c65f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:10.607442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80208703-a630-44e5-94a7-98365662c65f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:10.616929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6aeecf-5b6f-44f0-be40-aaca5a473e8b None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:10.617319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6aeecf-5b6f-44f0-be40-aaca5a473e8b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:10.617520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6aeecf-5b6f-44f0-be40-aaca5a473e8b None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:10.620324 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 483/957] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:10.625026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64a83-b86d-42e4-ae85-61059856f744 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:10.625332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64a83-b86d-42e4-ae85-61059856f744 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:10.625538 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 475/958] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:34:10 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:34:10.625643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64a83-b86d-42e4-ae85-61059856f744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:10.651816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-91c64a83-b86d-42e4-ae85-61059856f744 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:10.653747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-91c64a83-b86d-42e4-ae85-61059856f744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:10.745806 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 484/959] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:10.811032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a34251-2807-49e8-b9a4-717115777662 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:10.811363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a34251-2807-49e8-b9a4-717115777662 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:10.811608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a34251-2807-49e8-b9a4-717115777662 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:10.813653 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 476/960] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:10.818988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286feb5-4a35-42f9-9ea0-cda53734a76c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:10.819288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286feb5-4a35-42f9-9ea0-cda53734a76c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:10.819562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286feb5-4a35-42f9-9ea0-cda53734a76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:10.839220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c286feb5-4a35-42f9-9ea0-cda53734a76c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:10.840508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c286feb5-4a35-42f9-9ea0-cda53734a76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:10.877418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:10.912844 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 485/961] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:11.089591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a34251-2807-49e8-b9a4-717115777662 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:11.219632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a34251-2807-49e8-b9a4-717115777662 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:11.263068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67b39e-7020-423d-b26f-3bd54a236602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:11.263705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67b39e-7020-423d-b26f-3bd54a236602 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:11.263988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67b39e-7020-423d-b26f-3bd54a236602 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:11.264262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67b39e-7020-423d-b26f-3bd54a236602 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:11.264929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67b39e-7020-423d-b26f-3bd54a236602 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:11.266349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67b39e-7020-423d-b26f-3bd54a236602 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:11.269302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-af67b39e-7020-423d-b26f-3bd54a236602 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:11.278338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:11.279917 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:11.281449 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 477/962] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 3 20:34:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:11.283242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286feb5-4a35-42f9-9ea0-cda53734a76c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:11.286151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67b39e-7020-423d-b26f-3bd54a236602 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:11.389676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67b39e-7020-423d-b26f-3bd54a236602 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:11.402068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286feb5-4a35-42f9-9ea0-cda53734a76c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:11.425725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6773f9-be14-4911-a397-2cf55d9c88fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:11.426401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6773f9-be14-4911-a397-2cf55d9c88fd None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:11.426725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6773f9-be14-4911-a397-2cf55d9c88fd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:11.427035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6773f9-be14-4911-a397-2cf55d9c88fd None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:11.427809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6773f9-be14-4911-a397-2cf55d9c88fd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:11.432221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6773f9-be14-4911-a397-2cf55d9c88fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:11.432221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6773f9-be14-4911-a397-2cf55d9c88fd None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:11.442807 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 478/963] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 3 20:34:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:11.442807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2f5ba-5e68-4468-befc-537b4b12320e None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:11.442807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2f5ba-5e68-4468-befc-537b4b12320e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:11.442807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2f5ba-5e68-4468-befc-537b4b12320e None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:11.442807 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 479/964] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:11.443538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2f5ba-5e68-4468-befc-537b4b12320e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:11.449886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04232875-4344-42af-bff9-323f79423ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:11.450532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04232875-4344-42af-bff9-323f79423ee9 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:11.450820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04232875-4344-42af-bff9-323f79423ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:11.451100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04232875-4344-42af-bff9-323f79423ee9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:11.461882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04232875-4344-42af-bff9-323f79423ee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-04232875-4344-42af-bff9-323f79423ee9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:34:11.466709 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:11.467625 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:11.468442 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 486/965] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Sep 3 20:34:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:11.469703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d9e8a-cec5-4d6e-92dc-d66a7df3d97d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:11.469780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d9e8a-cec5-4d6e-92dc-d66a7df3d97d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:11.470469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d9e8a-cec5-4d6e-92dc-d66a7df3d97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:11.487939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-797d9e8a-cec5-4d6e-92dc-d66a7df3d97d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:11.489237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-797d9e8a-cec5-4d6e-92dc-d66a7df3d97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:11.514777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:11.519150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:11.566792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2f5ba-5e68-4468-befc-537b4b12320e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:11.569841 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 487/966] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:11.570606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d9e8a-cec5-4d6e-92dc-d66a7df3d97d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:11.603186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f83a70-55c3-446f-8746-56906483b614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:11.603911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f83a70-55c3-446f-8746-56906483b614 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:11.604271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f83a70-55c3-446f-8746-56906483b614 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:11.604679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f83a70-55c3-446f-8746-56906483b614 None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:11.605360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f83a70-55c3-446f-8746-56906483b614 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:11.605795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f83a70-55c3-446f-8746-56906483b614 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:11.607303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f83a70-55c3-446f-8746-56906483b614 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:11.618284 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 480/967] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 3 20:34:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:11.619200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f83a70-55c3-446f-8746-56906483b614 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:11.701370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d9e8a-cec5-4d6e-92dc-d66a7df3d97d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:11.729401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f83a70-55c3-446f-8746-56906483b614 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:11.739045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edec1b1-aed4-42b4-9e2d-6eaa76454442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:11.739924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edec1b1-aed4-42b4-9e2d-6eaa76454442 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:11.740393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edec1b1-aed4-42b4-9e2d-6eaa76454442 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:11.740851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edec1b1-aed4-42b4-9e2d-6eaa76454442 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:11.741781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edec1b1-aed4-42b4-9e2d-6eaa76454442 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:11.742393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edec1b1-aed4-42b4-9e2d-6eaa76454442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:11.743969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edec1b1-aed4-42b4-9e2d-6eaa76454442 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:11.758212 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 488/968] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 3 20:34:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:11.758627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:11.759633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edec1b1-aed4-42b4-9e2d-6eaa76454442 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:11.768705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b735da6-ca29-4f85-bb1f-29827af1e5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:11.769401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b735da6-ca29-4f85-bb1f-29827af1e5f9 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:11.769705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b735da6-ca29-4f85-bb1f-29827af1e5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:11.770257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b735da6-ca29-4f85-bb1f-29827af1e5f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:11.770644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b735da6-ca29-4f85-bb1f-29827af1e5f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:11.772092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b735da6-ca29-4f85-bb1f-29827af1e5f9 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:11.788019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:11.805013 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 481/969] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:34:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:34:11.884611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edec1b1-aed4-42b4-9e2d-6eaa76454442 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:11.904297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b735da6-ca29-4f85-bb1f-29827af1e5f9 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:11.933116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff24c8-346c-4321-8d20-d316a0f1a3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:11.934188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff24c8-346c-4321-8d20-d316a0f1a3d3 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:11.934678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff24c8-346c-4321-8d20-d316a0f1a3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:11.935143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff24c8-346c-4321-8d20-d316a0f1a3d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:11.947382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff24c8-346c-4321-8d20-d316a0f1a3d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:11.949239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff24c8-346c-4321-8d20-d316a0f1a3d3 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-01ff24c8-346c-4321-8d20-d316a0f1a3d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:11.966220 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:11.967210 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:11.968081 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:11.968081 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:11.968081 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:11.970341 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 489/970] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:11.977135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff24c8-346c-4321-8d20-d316a0f1a3d3 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:12.048316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b735da6-ca29-4f85-bb1f-29827af1e5f9 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:12.069942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:12.089376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738692ff-7a3a-48ce-ac9e-de4d62ff8457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:12.089998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738692ff-7a3a-48ce-ac9e-de4d62ff8457 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:12.090550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738692ff-7a3a-48ce-ac9e-de4d62ff8457 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:12.090550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738692ff-7a3a-48ce-ac9e-de4d62ff8457 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:12.091196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738692ff-7a3a-48ce-ac9e-de4d62ff8457 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:12.092640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738692ff-7a3a-48ce-ac9e-de4d62ff8457 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:12.095844 ncdv-master-1 devstack@keystone.service[99211]: WARNING keystone.server.flask.common [None req-738692ff-7a3a-48ce-ac9e-de4d62ff8457 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 03 20:34:12.109176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.sql.core [None req-738692ff-7a3a-48ce-ac9e-de4d62ff8457 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 03 20:34:12.109176 ncdv-master-1 devstack@keystone.service[99211]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 03 20:34:12.109176 ncdv-master-1 devstack@keystone.service[99211]: [parameters: {'user_id': '31b96cb602d841eabb9f60c6ce675566', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 03 20:34:12.109176 ncdv-master-1 devstack@keystone.service[99211]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99211) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 03 20:34:12.115753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff24c8-346c-4321-8d20-d316a0f1a3d3 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-738692ff-7a3a-48ce-ac9e-de4d62ff8457 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application conn.query(q) Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application result.read() Sep 03 20:34:12.129406 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 20:34:12.130380 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application next(self.gen) Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application yield resource Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application next(self.gen) Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.session.rollback() Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.force_reraise() Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise self.value Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application session.commit() Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 03 20:34:12.131416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._prepare_impl() Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.session.flush() Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._flush(objects) Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application compat.raise_( Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application flush_context.execute() Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rec.execute(self) Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 03 20:34:12.132309 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 20:34:12.133236 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application conn.query(q) Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application result.read() Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 20:34:12.134082 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [parameters: {'user_id': '31b96cb602d841eabb9f60c6ce675566', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:12.135412 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 03 20:34:12.136231 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 03 20:34:12.146180 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 03 20:34:12.146180 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 03 20:34:12.146180 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 03 20:34:12.146180 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 03 20:34:12.146180 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 03 20:34:12.146180 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:12.146180 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 482/971] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Sep 3 20:34:11 2022] POST /identity/v3/users => generated 160 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:12.146180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738692ff-7a3a-48ce-ac9e-de4d62ff8457 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:12.179672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6eb76d-bddc-4ebb-accf-697d982bb378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:12.180330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6eb76d-bddc-4ebb-accf-697d982bb378 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:12.180622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6eb76d-bddc-4ebb-accf-697d982bb378 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:12.181129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6eb76d-bddc-4ebb-accf-697d982bb378 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:12.182148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6eb76d-bddc-4ebb-accf-697d982bb378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:12.182468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6eb76d-bddc-4ebb-accf-697d982bb378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:12.184215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6eb76d-bddc-4ebb-accf-697d982bb378 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:12.199209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:12.200652 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 490/972] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:12.207632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6eb76d-bddc-4ebb-accf-697d982bb378 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:12.264874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738692ff-7a3a-48ce-ac9e-de4d62ff8457 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:12.313208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dac29-90c0-4e70-84d7-4496a1adeede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:12.314031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dac29-90c0-4e70-84d7-4496a1adeede None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:12.314779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dac29-90c0-4e70-84d7-4496a1adeede None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:12.315119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dac29-90c0-4e70-84d7-4496a1adeede None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:12.325992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dac29-90c0-4e70-84d7-4496a1adeede None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:12.328191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dac29-90c0-4e70-84d7-4496a1adeede None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:12.338395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6eb76d-bddc-4ebb-accf-697d982bb378 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-f39dac29-90c0-4e70-84d7-4496a1adeede None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:34:12.345361 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:12.346857 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:12.348441 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:12.348441 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:12.348441 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:34:12.348441 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:34:12.348441 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:34:12.348441 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:34:12.348441 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:34:12.348441 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:34:12.348441 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 20:34:12.348441 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:12.355091 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 483/973] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:34:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:12.355091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dac29-90c0-4e70-84d7-4496a1adeede None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:12.376040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:12.383650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c534c-1168-4a3b-a29c-9bf99c2ecb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:12.384370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c534c-1168-4a3b-a29c-9bf99c2ecb7e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:12.384718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c534c-1168-4a3b-a29c-9bf99c2ecb7e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:12.385104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c534c-1168-4a3b-a29c-9bf99c2ecb7e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:12.393812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c534c-1168-4a3b-a29c-9bf99c2ecb7e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:12.395256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c534c-1168-4a3b-a29c-9bf99c2ecb7e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-556c534c-1168-4a3b-a29c-9bf99c2ecb7e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:34:12.409742 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:12.410716 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:12.411662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:12.411662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:12.411662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:34:12.411662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:34:12.411662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:34:12.411662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:34:12.411662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:34:12.411662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:34:12.411662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 20:34:12.411662 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:12.414230 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 491/974] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 3 20:34:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:12.424931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c534c-1168-4a3b-a29c-9bf99c2ecb7e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:12.491742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dac29-90c0-4e70-84d7-4496a1adeede None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:12.501875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:12.538948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c534c-1168-4a3b-a29c-9bf99c2ecb7e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:12.540487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cb982-fd89-498b-b759-0bcb07afee84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:12.542313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cb982-fd89-498b-b759-0bcb07afee84 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:12.542751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cb982-fd89-498b-b759-0bcb07afee84 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:12.543170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cb982-fd89-498b-b759-0bcb07afee84 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:12.544037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cb982-fd89-498b-b759-0bcb07afee84 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:12.544486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cb982-fd89-498b-b759-0bcb07afee84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:12.546093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cb982-fd89-498b-b759-0bcb07afee84 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:12.580481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8303e-e469-4b0c-a93a-9a9729b5cb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:12.581398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8303e-e469-4b0c-a93a-9a9729b5cb37 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:12.581860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8303e-e469-4b0c-a93a-9a9729b5cb37 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:12.582963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8303e-e469-4b0c-a93a-9a9729b5cb37 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:12.584125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8303e-e469-4b0c-a93a-9a9729b5cb37 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:12.584584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8303e-e469-4b0c-a93a-9a9729b5cb37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:12.586185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8303e-e469-4b0c-a93a-9a9729b5cb37 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:12.593006 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 484/975] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 3 20:34:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:12.612249 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 492/976] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Sep 3 20:34:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:12.618408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cb982-fd89-498b-b759-0bcb07afee84 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:12.730768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cb982-fd89-498b-b759-0bcb07afee84 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:12.770912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5b607-2e5d-4cbe-97c7-7e6bec78c693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:12.771549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5b607-2e5d-4cbe-97c7-7e6bec78c693 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:12.771833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5b607-2e5d-4cbe-97c7-7e6bec78c693 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:12.772108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5b607-2e5d-4cbe-97c7-7e6bec78c693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:12.811080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:12.851319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5b607-2e5d-4cbe-97c7-7e6bec78c693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:12.853228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5b607-2e5d-4cbe-97c7-7e6bec78c693 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:12.932656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed7062-3e78-44fe-9403-7da6ede7d467 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:12.933050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed7062-3e78-44fe-9403-7da6ede7d467 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:12.933293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed7062-3e78-44fe-9403-7da6ede7d467 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:12.934899 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 493/977] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:34:12.943184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8b2b1-9fe5-477b-849a-f63629c2987c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:12.944037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8b2b1-9fe5-477b-849a-f63629c2987c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:12.944572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8b2b1-9fe5-477b-849a-f63629c2987c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:12.962226 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 485/978] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:34:12 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:34:12.964054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4ef8b2b1-9fe5-477b-849a-f63629c2987c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:12.965370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4ef8b2b1-9fe5-477b-849a-f63629c2987c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:12.969579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5b607-2e5d-4cbe-97c7-7e6bec78c693 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:13.014802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:13.036350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:13.042915 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 494/979] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:13.077034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5b607-2e5d-4cbe-97c7-7e6bec78c693 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:13.112321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffb640-2bfa-426b-aae1-22b9124b9d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:13.112990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffb640-2bfa-426b-aae1-22b9124b9d75 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:13.113252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffb640-2bfa-426b-aae1-22b9124b9d75 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:13.113531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffb640-2bfa-426b-aae1-22b9124b9d75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:13.124191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffb640-2bfa-426b-aae1-22b9124b9d75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:13.125698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffb640-2bfa-426b-aae1-22b9124b9d75 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-e4ffb640-2bfa-426b-aae1-22b9124b9d75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:13.144128 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:13.149845 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:13.150811 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:13.150811 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:13.150811 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:13.150811 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 486/980] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:13.152495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8b2b1-9fe5-477b-849a-f63629c2987c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:13.253708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8b2b1-9fe5-477b-849a-f63629c2987c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:13.288193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211772a-7f57-49e2-b99d-683a8bec03e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:13.288801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211772a-7f57-49e2-b99d-683a8bec03e2 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:13.289115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211772a-7f57-49e2-b99d-683a8bec03e2 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:13.289397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211772a-7f57-49e2-b99d-683a8bec03e2 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:13.290039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211772a-7f57-49e2-b99d-683a8bec03e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:13.290348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211772a-7f57-49e2-b99d-683a8bec03e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:13.291716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211772a-7f57-49e2-b99d-683a8bec03e2 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:13.303730 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 495/981] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:13.309062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffb640-2bfa-426b-aae1-22b9124b9d75 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:13.367029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211772a-7f57-49e2-b99d-683a8bec03e2 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:13.416000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffb640-2bfa-426b-aae1-22b9124b9d75 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:13.417933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:13.434882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:13.461150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6becf-f134-484b-a2e3-2d68cbb1f210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:13.461822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6becf-f134-484b-a2e3-2d68cbb1f210 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:13.462109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6becf-f134-484b-a2e3-2d68cbb1f210 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:13.462383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6becf-f134-484b-a2e3-2d68cbb1f210 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:13.470483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6becf-f134-484b-a2e3-2d68cbb1f210 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:13.471964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6becf-f134-484b-a2e3-2d68cbb1f210 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:13.481876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211772a-7f57-49e2-b99d-683a8bec03e2 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-abc6becf-f134-484b-a2e3-2d68cbb1f210 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:34:13.483861 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:13.484778 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:13.485702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:34:13.485702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:34:13.485702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:34:13.485702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:34:13.485702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:34:13.485702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:34:13.485702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:13.485702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:13.485702 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 487/982] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:34:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:13.490842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6becf-f134-484b-a2e3-2d68cbb1f210 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:13.520017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc953df7-b334-4c16-87fd-8b5d9d17fcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:13.520651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc953df7-b334-4c16-87fd-8b5d9d17fcb6 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:13.520959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc953df7-b334-4c16-87fd-8b5d9d17fcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:13.521243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc953df7-b334-4c16-87fd-8b5d9d17fcb6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:13.521829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc953df7-b334-4c16-87fd-8b5d9d17fcb6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:13.523194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc953df7-b334-4c16-87fd-8b5d9d17fcb6 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-cc953df7-b334-4c16-87fd-8b5d9d17fcb6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:13.532447 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:13.533667 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 20:34:13.534872 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:13.538583 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 496/983] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:34:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:13.539361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc953df7-b334-4c16-87fd-8b5d9d17fcb6 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:13.598645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6becf-f134-484b-a2e3-2d68cbb1f210 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:13.633535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb510890-2810-48e5-a556-43d855fd3224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:13.634076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb510890-2810-48e5-a556-43d855fd3224 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:13.634366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb510890-2810-48e5-a556-43d855fd3224 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:13.634650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb510890-2810-48e5-a556-43d855fd3224 None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:13.635348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb510890-2810-48e5-a556-43d855fd3224 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:13.635666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb510890-2810-48e5-a556-43d855fd3224 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:13.637132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb510890-2810-48e5-a556-43d855fd3224 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:13.648135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc953df7-b334-4c16-87fd-8b5d9d17fcb6 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:13.649866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:13.655211 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 488/984] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:34:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:13.656223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb510890-2810-48e5-a556-43d855fd3224 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:13.693316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e35a8-7210-4e2c-b668-45136da4bc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:13.693930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e35a8-7210-4e2c-b668-45136da4bc9e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:13.694217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e35a8-7210-4e2c-b668-45136da4bc9e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:13.694494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e35a8-7210-4e2c-b668-45136da4bc9e None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:13.695131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e35a8-7210-4e2c-b668-45136da4bc9e None admin] RBAC: Adding query filter params (name=compute) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:13.695439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e35a8-7210-4e2c-b668-45136da4bc9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:13.695698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:13.696846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e35a8-7210-4e2c-b668-45136da4bc9e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:13.706779 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 497/985] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:34:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:13.708054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e35a8-7210-4e2c-b668-45136da4bc9e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:13.772743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb510890-2810-48e5-a556-43d855fd3224 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:13.811495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e35a8-7210-4e2c-b668-45136da4bc9e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:13.812460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c44cb-7ab1-49c5-b8cd-aad17e01bdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:13.813134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c44cb-7ab1-49c5-b8cd-aad17e01bdd9 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:13.813426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c44cb-7ab1-49c5-b8cd-aad17e01bdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:13.813714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c44cb-7ab1-49c5-b8cd-aad17e01bdd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:13.814445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c44cb-7ab1-49c5-b8cd-aad17e01bdd9 None admin] RBAC: Adding query filter params (role.id=95f5b6cfc093417cbb9882b6897e875f, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=9388372667f8454498b2e4480187b47c) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:13.814766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c44cb-7ab1-49c5-b8cd-aad17e01bdd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:13.816300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c44cb-7ab1-49c5-b8cd-aad17e01bdd9 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:13.822544 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 489/986] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:34:13 2022] GET /identity/v3/role_assignments?user.id=9388372667f8454498b2e4480187b47c&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=95f5b6cfc093417cbb9882b6897e875f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:13.860572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ebd9e-8972-4f7e-b16f-b07fc7c8b02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:13.861238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ebd9e-8972-4f7e-b16f-b07fc7c8b02a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:13.861523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ebd9e-8972-4f7e-b16f-b07fc7c8b02a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:13.861809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ebd9e-8972-4f7e-b16f-b07fc7c8b02a None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:13.862445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ebd9e-8972-4f7e-b16f-b07fc7c8b02a None admin] RBAC: Adding query filter params (type=compute) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:13.862756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ebd9e-8972-4f7e-b16f-b07fc7c8b02a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:13.864158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ebd9e-8972-4f7e-b16f-b07fc7c8b02a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:13.870066 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 498/987] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:34:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:13.875891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c44cb-7ab1-49c5-b8cd-aad17e01bdd9 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:13.985792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:13.990056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c44cb-7ab1-49c5-b8cd-aad17e01bdd9 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:14.028833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fa05e-1181-45c1-8428-c18cccad2f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:14.029482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fa05e-1181-45c1-8428-c18cccad2f55 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:14.029770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fa05e-1181-45c1-8428-c18cccad2f55 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:14.030055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fa05e-1181-45c1-8428-c18cccad2f55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:14.030770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fa05e-1181-45c1-8428-c18cccad2f55 None admin] RBAC: Adding query filter params (interface=public, service_id=22a944bf63374a138e56379ba8633ada, region_id=RegionOne) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:14.031088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fa05e-1181-45c1-8428-c18cccad2f55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:14.033028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fa05e-1181-45c1-8428-c18cccad2f55 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:14.038793 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 490/988] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 3 20:34:13 2022] GET /identity/v3/endpoints?service_id=22a944bf63374a138e56379ba8633ada&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:14.044288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ebd9e-8972-4f7e-b16f-b07fc7c8b02a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:14.077003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:14.140155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757834de-71c7-4b0b-a6ef-624d4cb55c14 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:14.140568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757834de-71c7-4b0b-a6ef-624d4cb55c14 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:14.141288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757834de-71c7-4b0b-a6ef-624d4cb55c14 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:14.142745 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 491/989] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:34:14.147535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45030b-5b68-4b8c-ba84-a987067d966d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:14.147831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45030b-5b68-4b8c-ba84-a987067d966d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:14.148119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45030b-5b68-4b8c-ba84-a987067d966d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:14.157821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ebd9e-8972-4f7e-b16f-b07fc7c8b02a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:14.167052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1e45030b-5b68-4b8c-ba84-a987067d966d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:14.168590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1e45030b-5b68-4b8c-ba84-a987067d966d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:14.195702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c8f2f-09c0-4b8b-96ae-dad843f45eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:14.196316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c8f2f-09c0-4b8b-96ae-dad843f45eaa None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:14.196603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c8f2f-09c0-4b8b-96ae-dad843f45eaa None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:14.197115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c8f2f-09c0-4b8b-96ae-dad843f45eaa None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:14.197517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c8f2f-09c0-4b8b-96ae-dad843f45eaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:14.198891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c8f2f-09c0-4b8b-96ae-dad843f45eaa None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:14.205968 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 499/990] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 3 20:34:14 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:34:14.249533 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 492/991] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:14.580362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c8f2f-09c0-4b8b-96ae-dad843f45eaa None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:14.676122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:14.684264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c8f2f-09c0-4b8b-96ae-dad843f45eaa None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:14.719628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00848271-4923-422b-9dd2-7c3ec42a4888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:14.720265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00848271-4923-422b-9dd2-7c3ec42a4888 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:14.720554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00848271-4923-422b-9dd2-7c3ec42a4888 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:14.720835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00848271-4923-422b-9dd2-7c3ec42a4888 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:14.721456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00848271-4923-422b-9dd2-7c3ec42a4888 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:14.722823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00848271-4923-422b-9dd2-7c3ec42a4888 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-00848271-4923-422b-9dd2-7c3ec42a4888 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:14.731814 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:14.732779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:14.733952 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:14.737268 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 500/992] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 3 20:34:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:14.738912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45030b-5b68-4b8c-ba84-a987067d966d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:14.833849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:14.842144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45030b-5b68-4b8c-ba84-a987067d966d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:14.881588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77471f-8c20-4692-bb26-9153fb7b747c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:14.882305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77471f-8c20-4692-bb26-9153fb7b747c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:14.882599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77471f-8c20-4692-bb26-9153fb7b747c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:14.882878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77471f-8c20-4692-bb26-9153fb7b747c None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:14.883519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77471f-8c20-4692-bb26-9153fb7b747c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:14.883830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77471f-8c20-4692-bb26-9153fb7b747c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:14.885226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77471f-8c20-4692-bb26-9153fb7b747c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:14.894803 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 493/993] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 3 20:34:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:14.896082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00848271-4923-422b-9dd2-7c3ec42a4888 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:15.003542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00848271-4923-422b-9dd2-7c3ec42a4888 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:15.037602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625bc3f-1c1b-4e3f-b4ce-bd85b117bdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:15.038212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625bc3f-1c1b-4e3f-b4ce-bd85b117bdb9 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:15.038498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625bc3f-1c1b-4e3f-b4ce-bd85b117bdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:15.038773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625bc3f-1c1b-4e3f-b4ce-bd85b117bdb9 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:15.039403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625bc3f-1c1b-4e3f-b4ce-bd85b117bdb9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:15.039711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625bc3f-1c1b-4e3f-b4ce-bd85b117bdb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:15.041113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625bc3f-1c1b-4e3f-b4ce-bd85b117bdb9 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:15.045856 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 501/994] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 3 20:34:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:15.052654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77471f-8c20-4692-bb26-9153fb7b747c None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:15.159133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77471f-8c20-4692-bb26-9153fb7b747c None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:15.192727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e4b79-88d1-40b4-ba0e-f42d224437e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:15.193372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e4b79-88d1-40b4-ba0e-f42d224437e9 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:15.193664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e4b79-88d1-40b4-ba0e-f42d224437e9 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:15.193941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e4b79-88d1-40b4-ba0e-f42d224437e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:15.194646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e4b79-88d1-40b4-ba0e-f42d224437e9 None admin] RBAC: Adding query filter params (interface=admin, service_id=69433461b5264eb780cfaf7008502360, region_id=RegionOne) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:15.194963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e4b79-88d1-40b4-ba0e-f42d224437e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:15.196358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e4b79-88d1-40b4-ba0e-f42d224437e9 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:15.202530 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 494/995] 10.222.0.16 () {56 vars in 1249 bytes} [Sat Sep 3 20:34:15 2022] GET /identity/v3/endpoints?service_id=69433461b5264eb780cfaf7008502360&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:15.207980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625bc3f-1c1b-4e3f-b4ce-bd85b117bdb9 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:15.233823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:15.315434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625bc3f-1c1b-4e3f-b4ce-bd85b117bdb9 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:15.350315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc315-0aaf-4ade-bbf4-c9b925b5347e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:15.350928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc315-0aaf-4ade-bbf4-c9b925b5347e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:15.351212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc315-0aaf-4ade-bbf4-c9b925b5347e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:15.351490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc315-0aaf-4ade-bbf4-c9b925b5347e None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:15.352097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc315-0aaf-4ade-bbf4-c9b925b5347e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:15.353583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc315-0aaf-4ade-bbf4-c9b925b5347e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:15.360072 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 502/996] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 3 20:34:15 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:34:16.098970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bb26a-7b5b-4067-bd6f-e4169ce987ff None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:16.099370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bb26a-7b5b-4067-bd6f-e4169ce987ff None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:16.099546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bb26a-7b5b-4067-bd6f-e4169ce987ff None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:16.101060 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 495/997] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:16.106070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6ce63-cb05-46ce-b7bd-5acb0c9155c3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:16.106498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6ce63-cb05-46ce-b7bd-5acb0c9155c3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:16.106690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6ce63-cb05-46ce-b7bd-5acb0c9155c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:16.126772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c7f6ce63-cb05-46ce-b7bd-5acb0c9155c3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:16.127983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c7f6ce63-cb05-46ce-b7bd-5acb0c9155c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:16.204943 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 503/998] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:16.405299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043b3f5-809b-4d0b-93ad-37d595e4b147 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:16.405790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043b3f5-809b-4d0b-93ad-37d595e4b147 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:16.405790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043b3f5-809b-4d0b-93ad-37d595e4b147 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:16.407242 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 496/999] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:16.412618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31696c64-837b-4d8a-97ee-a6b49ded5bea None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:16.412959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31696c64-837b-4d8a-97ee-a6b49ded5bea None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:16.413350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31696c64-837b-4d8a-97ee-a6b49ded5bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:16.432477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-31696c64-837b-4d8a-97ee-a6b49ded5bea None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:16.433709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-31696c64-837b-4d8a-97ee-a6b49ded5bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:16.445756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:16.509445 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 504/1000] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:16.534813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043b3f5-809b-4d0b-93ad-37d595e4b147 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:16.560134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:16.638952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043b3f5-809b-4d0b-93ad-37d595e4b147 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:16.673366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57925e66-3f7a-4d6a-b4eb-1c376cb2c4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:16.673931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57925e66-3f7a-4d6a-b4eb-1c376cb2c4f4 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:16.674343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57925e66-3f7a-4d6a-b4eb-1c376cb2c4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:16.674501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57925e66-3f7a-4d6a-b4eb-1c376cb2c4f4 None admin] PATH_INFO: `/v3/services/image` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:16.675079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57925e66-3f7a-4d6a-b4eb-1c376cb2c4f4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:16.676649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57925e66-3f7a-4d6a-b4eb-1c376cb2c4f4 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-57925e66-3f7a-4d6a-b4eb-1c376cb2c4f4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:16.685358 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:16.686344 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:16.690016 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 497/1001] 10.222.0.16 () {56 vars in 1105 bytes} [Sat Sep 3 20:34:16 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:16.692376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31696c64-837b-4d8a-97ee-a6b49ded5bea None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:16.790329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31696c64-837b-4d8a-97ee-a6b49ded5bea None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:16.823342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd2c84-46d8-45fe-9379-1bb22a3d3d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:16.823946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd2c84-46d8-45fe-9379-1bb22a3d3d46 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:16.824228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd2c84-46d8-45fe-9379-1bb22a3d3d46 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:16.824503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd2c84-46d8-45fe-9379-1bb22a3d3d46 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:16.825151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd2c84-46d8-45fe-9379-1bb22a3d3d46 None admin] RBAC: Adding query filter params (name=image) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:16.825458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd2c84-46d8-45fe-9379-1bb22a3d3d46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:16.826819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd2c84-46d8-45fe-9379-1bb22a3d3d46 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:16.836939 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 505/1002] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:16.837980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57925e66-3f7a-4d6a-b4eb-1c376cb2c4f4 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:16.839716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd2c84-46d8-45fe-9379-1bb22a3d3d46 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:16.852042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:16.925415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:16.942452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57925e66-3f7a-4d6a-b4eb-1c376cb2c4f4 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:16.945696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd2c84-46d8-45fe-9379-1bb22a3d3d46 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:16.977574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91739648-b4af-4754-adcc-63b10299dbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:16.978191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91739648-b4af-4754-adcc-63b10299dbee None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:16.978476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91739648-b4af-4754-adcc-63b10299dbee None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:16.978756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91739648-b4af-4754-adcc-63b10299dbee None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:16.979384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91739648-b4af-4754-adcc-63b10299dbee None admin] RBAC: Adding query filter params (type=image) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:16.979694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91739648-b4af-4754-adcc-63b10299dbee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:16.981115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91739648-b4af-4754-adcc-63b10299dbee None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:16.985458 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 498/1003] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:16.987402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b550c-f75e-47ec-b792-d29f1c5f8171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:16.988046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b550c-f75e-47ec-b792-d29f1c5f8171 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:16.988327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b550c-f75e-47ec-b792-d29f1c5f8171 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:16.988600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b550c-f75e-47ec-b792-d29f1c5f8171 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:16.989222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b550c-f75e-47ec-b792-d29f1c5f8171 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:16.990613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b550c-f75e-47ec-b792-d29f1c5f8171 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-cb3b550c-f75e-47ec-b792-d29f1c5f8171 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:17.004483 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:17.005610 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:17.006441 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 506/1004] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:34:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:17.018194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91739648-b4af-4754-adcc-63b10299dbee None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:17.139385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91739648-b4af-4754-adcc-63b10299dbee None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:17.179328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8eedf6-ffe4-4d55-a69e-0c352f376cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:17.179875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8eedf6-ffe4-4d55-a69e-0c352f376cea None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:17.180167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8eedf6-ffe4-4d55-a69e-0c352f376cea None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:17.180453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8eedf6-ffe4-4d55-a69e-0c352f376cea None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:17.181139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8eedf6-ffe4-4d55-a69e-0c352f376cea None admin] RBAC: Adding query filter params (name=compute) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:17.181453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8eedf6-ffe4-4d55-a69e-0c352f376cea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:17.182825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8eedf6-ffe4-4d55-a69e-0c352f376cea None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:17.188613 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 499/1005] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:34:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:17.194506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b550c-f75e-47ec-b792-d29f1c5f8171 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:17.273634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:17.300604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b550c-f75e-47ec-b792-d29f1c5f8171 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:17.335152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93830962-8ab8-4c60-9237-aaf82f92b861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:17.335759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93830962-8ab8-4c60-9237-aaf82f92b861 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:17.336045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93830962-8ab8-4c60-9237-aaf82f92b861 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:17.336323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93830962-8ab8-4c60-9237-aaf82f92b861 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:17.336987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93830962-8ab8-4c60-9237-aaf82f92b861 None admin] RBAC: Adding query filter params (type=compute) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:17.337299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93830962-8ab8-4c60-9237-aaf82f92b861 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:17.338744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93830962-8ab8-4c60-9237-aaf82f92b861 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:17.348876 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 507/1006] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 3 20:34:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:17.350522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8eedf6-ffe4-4d55-a69e-0c352f376cea None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:17.367746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:17.461061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8eedf6-ffe4-4d55-a69e-0c352f376cea None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:17.497470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b35982-a28b-41f5-8c15-9c0a1e1a7a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:17.498443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b35982-a28b-41f5-8c15-9c0a1e1a7a13 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:17.498933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b35982-a28b-41f5-8c15-9c0a1e1a7a13 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:17.499434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b35982-a28b-41f5-8c15-9c0a1e1a7a13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:17.500890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b35982-a28b-41f5-8c15-9c0a1e1a7a13 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:17.502496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b35982-a28b-41f5-8c15-9c0a1e1a7a13 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:17.525797 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 500/1007] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:34:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:34:17.554708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99fc94-032a-4075-a9df-b4cf1cf395c9 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:17.555079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99fc94-032a-4075-a9df-b4cf1cf395c9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:17.555284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99fc94-032a-4075-a9df-b4cf1cf395c9 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:17.556852 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 508/1008] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:17.563247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79008a5c-4b4f-4cef-9617-50d70c643a9e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:17.563552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79008a5c-4b4f-4cef-9617-50d70c643a9e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:17.563824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79008a5c-4b4f-4cef-9617-50d70c643a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:17.584514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-79008a5c-4b4f-4cef-9617-50d70c643a9e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:17.585854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-79008a5c-4b4f-4cef-9617-50d70c643a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:17.657300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:17.671880 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 501/1009] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:17 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:18.030587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99fc94-032a-4075-a9df-b4cf1cf395c9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:18.157977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99fc94-032a-4075-a9df-b4cf1cf395c9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:18.191554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93b0f8-ec62-4094-aad7-b823188c25ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:18.192188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93b0f8-ec62-4094-aad7-b823188c25ef None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:18.192495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93b0f8-ec62-4094-aad7-b823188c25ef None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:18.192756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93b0f8-ec62-4094-aad7-b823188c25ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:18.193385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93b0f8-ec62-4094-aad7-b823188c25ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:18.194764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93b0f8-ec62-4094-aad7-b823188c25ef None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-0d93b0f8-ec62-4094-aad7-b823188c25ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:18.204457 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:18.205477 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:18.206303 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 509/1010] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 3 20:34:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:18.214143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79008a5c-4b4f-4cef-9617-50d70c643a9e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:18.317323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79008a5c-4b4f-4cef-9617-50d70c643a9e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:18.351790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b57a3a-0472-4646-b80d-c9fd1c077949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:18.352399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57a3a-0472-4646-b80d-c9fd1c077949 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:18.352687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57a3a-0472-4646-b80d-c9fd1c077949 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:18.353057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57a3a-0472-4646-b80d-c9fd1c077949 None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:18.353686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b57a3a-0472-4646-b80d-c9fd1c077949 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:18.353993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b57a3a-0472-4646-b80d-c9fd1c077949 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:18.355351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b57a3a-0472-4646-b80d-c9fd1c077949 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:18.360231 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 502/1011] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 3 20:34:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:18.366096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93b0f8-ec62-4094-aad7-b823188c25ef None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:18.369500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:18.468943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93b0f8-ec62-4094-aad7-b823188c25ef None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:18.507361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dac9c2-ad66-4e15-a4f0-032d1e4d9dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:18.507973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dac9c2-ad66-4e15-a4f0-032d1e4d9dba None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:18.508261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dac9c2-ad66-4e15-a4f0-032d1e4d9dba None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:18.508541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dac9c2-ad66-4e15-a4f0-032d1e4d9dba None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:18.509206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dac9c2-ad66-4e15-a4f0-032d1e4d9dba None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:18.509519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dac9c2-ad66-4e15-a4f0-032d1e4d9dba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:18.510894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dac9c2-ad66-4e15-a4f0-032d1e4d9dba None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:18.515726 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 510/1012] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 3 20:34:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:18.522199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b57a3a-0472-4646-b80d-c9fd1c077949 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:18.588824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:18.631771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b57a3a-0472-4646-b80d-c9fd1c077949 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:18.665472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bed971-26f9-48a0-95f5-d5a5b9f01d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:18.666096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bed971-26f9-48a0-95f5-d5a5b9f01d6f None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:18.666379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bed971-26f9-48a0-95f5-d5a5b9f01d6f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:18.666653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bed971-26f9-48a0-95f5-d5a5b9f01d6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:18.667334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bed971-26f9-48a0-95f5-d5a5b9f01d6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:18.668699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bed971-26f9-48a0-95f5-d5a5b9f01d6f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:18.687036 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 503/1013] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:34:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:34:19.232011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e792595-4574-41c6-8188-0828c6a9f72a None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:19.232524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e792595-4574-41c6-8188-0828c6a9f72a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:19.232632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e792595-4574-41c6-8188-0828c6a9f72a None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:19.234056 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 511/1014] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:19.238945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10863cc1-88d8-4135-b193-93ed4e1c9e27 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:19.239244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10863cc1-88d8-4135-b193-93ed4e1c9e27 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:19.239522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10863cc1-88d8-4135-b193-93ed4e1c9e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:19.258263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-10863cc1-88d8-4135-b193-93ed4e1c9e27 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:19.259449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-10863cc1-88d8-4135-b193-93ed4e1c9e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:19.332285 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 504/1015] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:19 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:19.662209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e792595-4574-41c6-8188-0828c6a9f72a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:19.737023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:19.775696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e792595-4574-41c6-8188-0828c6a9f72a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:19.815920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883257b5-8123-45bc-b176-b73ea44b7893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:19.816542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883257b5-8123-45bc-b176-b73ea44b7893 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:19.816824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883257b5-8123-45bc-b176-b73ea44b7893 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:19.817127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883257b5-8123-45bc-b176-b73ea44b7893 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:19.817801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883257b5-8123-45bc-b176-b73ea44b7893 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:19.819182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883257b5-8123-45bc-b176-b73ea44b7893 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:19.840936 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 512/1016] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:34:19 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:34:19.888040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c29560-bc7e-437b-b165-0549ea3f4eff None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:19.888574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c29560-bc7e-437b-b165-0549ea3f4eff None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:19.888679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c29560-bc7e-437b-b165-0549ea3f4eff None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:19.890166 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 505/1017] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:19.896428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fa5f1-7b37-470c-a44b-4cc96b533af1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:19.896744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fa5f1-7b37-470c-a44b-4cc96b533af1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:19.897044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fa5f1-7b37-470c-a44b-4cc96b533af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:19.916527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-793fa5f1-7b37-470c-a44b-4cc96b533af1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:19.917710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-793fa5f1-7b37-470c-a44b-4cc96b533af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:20.002161 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 513/1018] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:20.337862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c29560-bc7e-437b-b165-0549ea3f4eff None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:20.389238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:20.448994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c29560-bc7e-437b-b165-0549ea3f4eff None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:20.484070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371077b7-cda1-4d68-86cb-dbb1d7677b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:20.485301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371077b7-cda1-4d68-86cb-dbb1d7677b75 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:20.485593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371077b7-cda1-4d68-86cb-dbb1d7677b75 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:20.486362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371077b7-cda1-4d68-86cb-dbb1d7677b75 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:20.496316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371077b7-cda1-4d68-86cb-dbb1d7677b75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-371077b7-cda1-4d68-86cb-dbb1d7677b75 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:34:20.501431 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:20.502346 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:20.503191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:20.503191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:20.503191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:20.503191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:20.503191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:34:20.503191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:34:20.503191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:34:20.503191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:34:20.503191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:34:20.503191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:34:20.503191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 20:34:20.503191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:20.505229 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 506/1019] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Sep 3 20:34:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:20.508508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fa5f1-7b37-470c-a44b-4cc96b533af1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:20.562348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:20.613627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fa5f1-7b37-470c-a44b-4cc96b533af1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:20.646871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb32b0-818f-488e-bfba-8dba3af27184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:20.647479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb32b0-818f-488e-bfba-8dba3af27184 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:20.647817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb32b0-818f-488e-bfba-8dba3af27184 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:20.648186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb32b0-818f-488e-bfba-8dba3af27184 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:20.648991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb32b0-818f-488e-bfba-8dba3af27184 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:20.649307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb32b0-818f-488e-bfba-8dba3af27184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:20.650689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb32b0-818f-488e-bfba-8dba3af27184 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:20.663822 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 514/1020] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 3 20:34:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:20.665054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371077b7-cda1-4d68-86cb-dbb1d7677b75 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:20.772699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371077b7-cda1-4d68-86cb-dbb1d7677b75 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:20.781787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:20.812285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d882fce-3b98-47f6-8548-3413861d6532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:20.812969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d882fce-3b98-47f6-8548-3413861d6532 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:20.813257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d882fce-3b98-47f6-8548-3413861d6532 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:20.813535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d882fce-3b98-47f6-8548-3413861d6532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:20.823512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d882fce-3b98-47f6-8548-3413861d6532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:20.824998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d882fce-3b98-47f6-8548-3413861d6532 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-9d882fce-3b98-47f6-8548-3413861d6532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:20.839565 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:20.840528 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:20.841476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:20.841476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:20.841476 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:20.845221 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 507/1021] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:20.846825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb32b0-818f-488e-bfba-8dba3af27184 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:20.921609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3326de-b323-4141-b1f3-1538412696a2 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:20.922102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3326de-b323-4141-b1f3-1538412696a2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:20.922201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3326de-b323-4141-b1f3-1538412696a2 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:20.923613 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 508/1022] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:34:20.928547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a53abd-bf23-49b9-bc0e-66bc2ec8dd3d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:20.928845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a53abd-bf23-49b9-bc0e-66bc2ec8dd3d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:20.929166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a53abd-bf23-49b9-bc0e-66bc2ec8dd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:20.947339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-30a53abd-bf23-49b9-bc0e-66bc2ec8dd3d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:20.948543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-30a53abd-bf23-49b9-bc0e-66bc2ec8dd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:20.955294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb32b0-818f-488e-bfba-8dba3af27184 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:20.973928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:20.998025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f6527-12f4-4e47-a53c-bc3934415952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:20.998660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f6527-12f4-4e47-a53c-bc3934415952 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:20.999053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f6527-12f4-4e47-a53c-bc3934415952 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:20.999341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f6527-12f4-4e47-a53c-bc3934415952 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:21.000055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f6527-12f4-4e47-a53c-bc3934415952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:21.000372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f6527-12f4-4e47-a53c-bc3934415952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:21.001845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f6527-12f4-4e47-a53c-bc3934415952 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:21.014805 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 515/1023] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:21.020473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f6527-12f4-4e47-a53c-bc3934415952 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:21.041221 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 509/1024] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:21.124074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f6527-12f4-4e47-a53c-bc3934415952 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:21.160782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc28a05-dfad-47fc-b1b5-bcf59f0271c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:21.161457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc28a05-dfad-47fc-b1b5-bcf59f0271c1 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:21.161728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc28a05-dfad-47fc-b1b5-bcf59f0271c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:21.162032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc28a05-dfad-47fc-b1b5-bcf59f0271c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:21.169158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc28a05-dfad-47fc-b1b5-bcf59f0271c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:21.170591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc28a05-dfad-47fc-b1b5-bcf59f0271c1 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-ddc28a05-dfad-47fc-b1b5-bcf59f0271c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:34:21.183407 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:21.184380 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:21.185384 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:21.185384 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:21.185384 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:34:21.185384 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:34:21.185384 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:34:21.185384 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:34:21.185384 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:34:21.185384 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:34:21.185384 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:34:21.185384 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:21.189213 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 516/1025] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 3 20:34:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:21.190696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a53abd-bf23-49b9-bc0e-66bc2ec8dd3d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:21.281156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:21.296152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a53abd-bf23-49b9-bc0e-66bc2ec8dd3d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:21.331732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc8c89-8c47-4dbe-875b-87c4d37ab2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:21.332344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc8c89-8c47-4dbe-875b-87c4d37ab2e6 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:21.332907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc8c89-8c47-4dbe-875b-87c4d37ab2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:21.333209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc8c89-8c47-4dbe-875b-87c4d37ab2e6 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:21.333935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc8c89-8c47-4dbe-875b-87c4d37ab2e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:21.334246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc8c89-8c47-4dbe-875b-87c4d37ab2e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:21.335672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc8c89-8c47-4dbe-875b-87c4d37ab2e6 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:21.357669 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 510/1026] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 3 20:34:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:21.359110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc28a05-dfad-47fc-b1b5-bcf59f0271c1 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:21.360927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc8c89-8c47-4dbe-875b-87c4d37ab2e6 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:21.402817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:21.488872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc28a05-dfad-47fc-b1b5-bcf59f0271c1 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:21.495806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc8c89-8c47-4dbe-875b-87c4d37ab2e6 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:21.526928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e119262-9a6c-46df-8b91-2bbdf0bd2bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:21.528547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e119262-9a6c-46df-8b91-2bbdf0bd2bea None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:21.529096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e119262-9a6c-46df-8b91-2bbdf0bd2bea None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:21.529595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e119262-9a6c-46df-8b91-2bbdf0bd2bea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:21.529710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f22996-eb83-412c-b157-7d23da573246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:21.530257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f22996-eb83-412c-b157-7d23da573246 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:21.530548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f22996-eb83-412c-b157-7d23da573246 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:21.530830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f22996-eb83-412c-b157-7d23da573246 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:21.536319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f22996-eb83-412c-b157-7d23da573246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:21.542591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f22996-eb83-412c-b157-7d23da573246 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:21.544686 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 511/1027] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:21.551324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f22996-eb83-412c-b157-7d23da573246 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:21.569319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:21.622060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e119262-9a6c-46df-8b91-2bbdf0bd2bea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:21.624494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e119262-9a6c-46df-8b91-2bbdf0bd2bea None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:21.664953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:21.685209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f22996-eb83-412c-b157-7d23da573246 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:21.724197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739a9e9-1dfe-4d48-90a2-c2c94d8e243c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:21.724841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739a9e9-1dfe-4d48-90a2-c2c94d8e243c None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:21.725155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739a9e9-1dfe-4d48-90a2-c2c94d8e243c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:21.725500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739a9e9-1dfe-4d48-90a2-c2c94d8e243c None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:21.726156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739a9e9-1dfe-4d48-90a2-c2c94d8e243c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:21.727575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739a9e9-1dfe-4d48-90a2-c2c94d8e243c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:21.742286 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 517/1028] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:34:21 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 20:34:21.747744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e119262-9a6c-46df-8b91-2bbdf0bd2bea None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:21.772958 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 512/1029] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:34:21 2022] POST /identity/v3/projects => generated 311 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:34:21.874155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e119262-9a6c-46df-8b91-2bbdf0bd2bea None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:21.911132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095b89c-ec60-4f5c-9d54-264fea017218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:21.911807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095b89c-ec60-4f5c-9d54-264fea017218 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:21.912098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095b89c-ec60-4f5c-9d54-264fea017218 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:21.912383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095b89c-ec60-4f5c-9d54-264fea017218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:21.914794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:21.927226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095b89c-ec60-4f5c-9d54-264fea017218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:21.928857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095b89c-ec60-4f5c-9d54-264fea017218 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-d095b89c-ec60-4f5c-9d54-264fea017218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:21.945027 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:21.946054 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:21.952180 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:21.952180 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:21.952180 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:21.952180 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 518/1030] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:21.952420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739a9e9-1dfe-4d48-90a2-c2c94d8e243c None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:22.034048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:22.087684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739a9e9-1dfe-4d48-90a2-c2c94d8e243c None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:22.139901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecd19c-58f4-4de2-bfac-da8138d46b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:22.140524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecd19c-58f4-4de2-bfac-da8138d46b78 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:22.141435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecd19c-58f4-4de2-bfac-da8138d46b78 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:22.141751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecd19c-58f4-4de2-bfac-da8138d46b78 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:22.142399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecd19c-58f4-4de2-bfac-da8138d46b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:22.142707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecd19c-58f4-4de2-bfac-da8138d46b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:22.144103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecd19c-58f4-4de2-bfac-da8138d46b78 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:22.157403 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 513/1031] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:22.165781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095b89c-ec60-4f5c-9d54-264fea017218 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:22.179956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfea76-7e3b-42e1-a24a-f2c26052c66f None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:22.180250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfea76-7e3b-42e1-a24a-f2c26052c66f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:22.180536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfea76-7e3b-42e1-a24a-f2c26052c66f None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:22.181981 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 514/1032] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:34:22.187223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbeb50-eb80-45c6-9e43-bf9231a1dd66 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:22.188034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbeb50-eb80-45c6-9e43-bf9231a1dd66 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:22.188326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbeb50-eb80-45c6-9e43-bf9231a1dd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:22.207220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-43cbeb50-eb80-45c6-9e43-bf9231a1dd66 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:22.208675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-43cbeb50-eb80-45c6-9e43-bf9231a1dd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:22.278975 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 515/1033] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:22.280321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095b89c-ec60-4f5c-9d54-264fea017218 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:22.315223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c0104-aae8-4ae7-b4e4-83965a3b7bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:22.315775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c0104-aae8-4ae7-b4e4-83965a3b7bf2 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:22.316060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c0104-aae8-4ae7-b4e4-83965a3b7bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:22.316338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c0104-aae8-4ae7-b4e4-83965a3b7bf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:22.323104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c0104-aae8-4ae7-b4e4-83965a3b7bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:22.324562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c0104-aae8-4ae7-b4e4-83965a3b7bf2 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-416c0104-aae8-4ae7-b4e4-83965a3b7bf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:34:22.336241 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:22.342261 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:22.343395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:34:22.343395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:34:22.343395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:34:22.343395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:34:22.343395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:34:22.343395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:34:22.343395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:22.343395 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:22.343395 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 519/1034] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:34:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:22.347821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbeb50-eb80-45c6-9e43-bf9231a1dd66 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:22.416403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:22.475582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbeb50-eb80-45c6-9e43-bf9231a1dd66 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:22.510438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442546c-ff2e-48ef-bea4-6f7d1dccbf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:22.511048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442546c-ff2e-48ef-bea4-6f7d1dccbf0d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:22.511327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442546c-ff2e-48ef-bea4-6f7d1dccbf0d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:22.511598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442546c-ff2e-48ef-bea4-6f7d1dccbf0d None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:22.512280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442546c-ff2e-48ef-bea4-6f7d1dccbf0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:22.512587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442546c-ff2e-48ef-bea4-6f7d1dccbf0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:22.514041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442546c-ff2e-48ef-bea4-6f7d1dccbf0d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:22.525647 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 516/1035] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:34:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:22.531491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c0104-aae8-4ae7-b4e4-83965a3b7bf2 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:22.586352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:22.642355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c0104-aae8-4ae7-b4e4-83965a3b7bf2 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:22.675318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e3765-c492-411c-a61b-be5ca92133ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:22.675958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e3765-c492-411c-a61b-be5ca92133ba None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:22.676216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e3765-c492-411c-a61b-be5ca92133ba None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:22.676478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e3765-c492-411c-a61b-be5ca92133ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:22.677338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e3765-c492-411c-a61b-be5ca92133ba None admin] RBAC: Adding query filter params (role.id=95f5b6cfc093417cbb9882b6897e875f, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=a56702329eae48819121dcb58f7e3070) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:22.677689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e3765-c492-411c-a61b-be5ca92133ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:22.679151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e3765-c492-411c-a61b-be5ca92133ba None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:22.692933 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 520/1036] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:34:22 2022] GET /identity/v3/role_assignments?user.id=a56702329eae48819121dcb58f7e3070&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=95f5b6cfc093417cbb9882b6897e875f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:22.820227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442546c-ff2e-48ef-bea4-6f7d1dccbf0d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:22.949409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442546c-ff2e-48ef-bea4-6f7d1dccbf0d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:22.966439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:22.994119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e205e-b5e7-4c4d-9d91-fa7453a11070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:22.994119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e205e-b5e7-4c4d-9d91-fa7453a11070 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:22.994119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e205e-b5e7-4c4d-9d91-fa7453a11070 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:22.994119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e205e-b5e7-4c4d-9d91-fa7453a11070 None admin] PATH_INFO: `/v3/services/image` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:22.994586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e205e-b5e7-4c4d-9d91-fa7453a11070 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:22.995228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e205e-b5e7-4c4d-9d91-fa7453a11070 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-406e205e-b5e7-4c4d-9d91-fa7453a11070 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:23.006497 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:23.007667 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:23.008576 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 517/1037] 10.222.0.16 () {56 vars in 1105 bytes} [Sat Sep 3 20:34:22 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:23.014242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e3765-c492-411c-a61b-be5ca92133ba None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:23.124509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e3765-c492-411c-a61b-be5ca92133ba None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:23.137372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:23.163559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a614d-38cd-401e-9eb5-d4fd888b8dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:23.164180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a614d-38cd-401e-9eb5-d4fd888b8dce None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:23.164466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a614d-38cd-401e-9eb5-d4fd888b8dce None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:23.164747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a614d-38cd-401e-9eb5-d4fd888b8dce None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:23.165417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a614d-38cd-401e-9eb5-d4fd888b8dce None admin] RBAC: Adding query filter params (name=image) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:23.165731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a614d-38cd-401e-9eb5-d4fd888b8dce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:23.167095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a614d-38cd-401e-9eb5-d4fd888b8dce None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:23.176973 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 521/1038] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:23.177856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e205e-b5e7-4c4d-9d91-fa7453a11070 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:23.279949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e205e-b5e7-4c4d-9d91-fa7453a11070 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:23.314227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c619cf-eaba-4b67-8293-31a40d0adaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:23.314845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c619cf-eaba-4b67-8293-31a40d0adaa1 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:23.315135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c619cf-eaba-4b67-8293-31a40d0adaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:23.315413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c619cf-eaba-4b67-8293-31a40d0adaa1 None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:23.316040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c619cf-eaba-4b67-8293-31a40d0adaa1 None admin] RBAC: Adding query filter params (type=image) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:23.316362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c619cf-eaba-4b67-8293-31a40d0adaa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:23.317742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c619cf-eaba-4b67-8293-31a40d0adaa1 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:23.322664 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 518/1039] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:23.329254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a614d-38cd-401e-9eb5-d4fd888b8dce None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:23.430621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a614d-38cd-401e-9eb5-d4fd888b8dce None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:23.465412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc733a-d283-46e6-8988-2da5225b4d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:23.466030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc733a-d283-46e6-8988-2da5225b4d85 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:23.466318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc733a-d283-46e6-8988-2da5225b4d85 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:23.466598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc733a-d283-46e6-8988-2da5225b4d85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:23.467309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc733a-d283-46e6-8988-2da5225b4d85 None admin] RBAC: Adding query filter params (interface=public, service_id=847ac3ed00d24e6aa853f29ea4e827d8, region_id=RegionOne) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:23.467626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc733a-d283-46e6-8988-2da5225b4d85 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:23.469543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc733a-d283-46e6-8988-2da5225b4d85 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:23.480921 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 522/1040] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 3 20:34:23 2022] GET /identity/v3/endpoints?service_id=847ac3ed00d24e6aa853f29ea4e827d8&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:23.483529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c619cf-eaba-4b67-8293-31a40d0adaa1 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:23.532946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:23.606954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c619cf-eaba-4b67-8293-31a40d0adaa1 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:23.643155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610e9fe-7563-4c0a-8ef0-91b5392ca360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:23.643768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610e9fe-7563-4c0a-8ef0-91b5392ca360 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:23.644055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610e9fe-7563-4c0a-8ef0-91b5392ca360 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:23.644321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610e9fe-7563-4c0a-8ef0-91b5392ca360 None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:23.644948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610e9fe-7563-4c0a-8ef0-91b5392ca360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:23.646399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610e9fe-7563-4c0a-8ef0-91b5392ca360 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:23.660930 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 519/1041] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 3 20:34:23 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:34:24.137310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec1251-9476-44df-9b2d-5ea122fab130 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:24.137747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec1251-9476-44df-9b2d-5ea122fab130 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:24.137931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec1251-9476-44df-9b2d-5ea122fab130 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:24.139367 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 523/1042] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:24.145016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166c028-9260-41a8-957a-3e6101e24bf6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:24.145424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166c028-9260-41a8-957a-3e6101e24bf6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:24.145598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166c028-9260-41a8-957a-3e6101e24bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:24.164088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4166c028-9260-41a8-957a-3e6101e24bf6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:24.165294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4166c028-9260-41a8-957a-3e6101e24bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:24.233898 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 520/1043] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:24.573362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec1251-9476-44df-9b2d-5ea122fab130 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:24.617855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:24.682948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec1251-9476-44df-9b2d-5ea122fab130 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:24.717967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1cfb8-5181-4d69-bd68-6319e9214dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:24.718629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1cfb8-5181-4d69-bd68-6319e9214dd0 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:24.718932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1cfb8-5181-4d69-bd68-6319e9214dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:24.719216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1cfb8-5181-4d69-bd68-6319e9214dd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:24.720352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1cfb8-5181-4d69-bd68-6319e9214dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:24.721822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1cfb8-5181-4d69-bd68-6319e9214dd0 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:24.725046 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 524/1044] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:24.729234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166c028-9260-41a8-957a-3e6101e24bf6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:24.774056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:24.840183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166c028-9260-41a8-957a-3e6101e24bf6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:24.884123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c619aaa-2c5d-45e1-971c-befe9b033dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:24.884753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c619aaa-2c5d-45e1-971c-befe9b033dd5 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:24.885076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c619aaa-2c5d-45e1-971c-befe9b033dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:24.885359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c619aaa-2c5d-45e1-971c-befe9b033dd5 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:24.886039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c619aaa-2c5d-45e1-971c-befe9b033dd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:24.887485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c619aaa-2c5d-45e1-971c-befe9b033dd5 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:24.910495 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 521/1045] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:34:24 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:34:25.036218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1156-2867-45d6-a6d4-56b980e48237 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:25.036554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1156-2867-45d6-a6d4-56b980e48237 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:25.036785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1156-2867-45d6-a6d4-56b980e48237 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:25.038362 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 525/1046] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:25.044807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842452a3-a9a7-45c9-bbfb-43b5f8853722 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:25.045174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842452a3-a9a7-45c9-bbfb-43b5f8853722 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:25.045441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842452a3-a9a7-45c9-bbfb-43b5f8853722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:25.065973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-842452a3-a9a7-45c9-bbfb-43b5f8853722 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:25.067228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-842452a3-a9a7-45c9-bbfb-43b5f8853722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:25.149252 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 522/1047] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:25.492505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1156-2867-45d6-a6d4-56b980e48237 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:25.597686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1156-2867-45d6-a6d4-56b980e48237 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:25.605642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:25.633258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01091e3b-c360-4e8b-913e-e2b10e95d8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:25.633895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01091e3b-c360-4e8b-913e-e2b10e95d8b9 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:25.634180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01091e3b-c360-4e8b-913e-e2b10e95d8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:25.634457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01091e3b-c360-4e8b-913e-e2b10e95d8b9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:25.645525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01091e3b-c360-4e8b-913e-e2b10e95d8b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-01091e3b-c360-4e8b-913e-e2b10e95d8b9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:34:25.650500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:25.651465 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:25.652335 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 526/1048] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Sep 3 20:34:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:25.659568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842452a3-a9a7-45c9-bbfb-43b5f8853722 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:25.686869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:25.770182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842452a3-a9a7-45c9-bbfb-43b5f8853722 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:25.803874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0bbe21-3fbe-4f47-b3af-da8138b4cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:25.804488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0bbe21-3fbe-4f47-b3af-da8138b4cac8 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:25.804788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0bbe21-3fbe-4f47-b3af-da8138b4cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:25.805083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0bbe21-3fbe-4f47-b3af-da8138b4cac8 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:25.805754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0bbe21-3fbe-4f47-b3af-da8138b4cac8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:25.806059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0bbe21-3fbe-4f47-b3af-da8138b4cac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:25.807425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0bbe21-3fbe-4f47-b3af-da8138b4cac8 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:25.820942 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 523/1049] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 3 20:34:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:25.822096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01091e3b-c360-4e8b-913e-e2b10e95d8b9 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:25.850261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb10a49-6dab-4c7d-bcdc-8322587ced86 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:25.850570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb10a49-6dab-4c7d-bcdc-8322587ced86 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:25.850830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb10a49-6dab-4c7d-bcdc-8322587ced86 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:25.852389 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 524/1050] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:34:25.857124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62122d6b-c81f-4fad-95ee-af7daa28b5ab None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:25.857454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62122d6b-c81f-4fad-95ee-af7daa28b5ab None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:25.857710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62122d6b-c81f-4fad-95ee-af7daa28b5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:25.876193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-62122d6b-c81f-4fad-95ee-af7daa28b5ab None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:25.877451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-62122d6b-c81f-4fad-95ee-af7daa28b5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:25.929586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01091e3b-c360-4e8b-913e-e2b10e95d8b9 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:25.947456 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 525/1051] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:25.966888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c855d-ced8-491a-a0dc-1142b51243f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:25.967803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c855d-ced8-491a-a0dc-1142b51243f8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:25.968347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c855d-ced8-491a-a0dc-1142b51243f8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:25.968822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c855d-ced8-491a-a0dc-1142b51243f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:25.979390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c855d-ced8-491a-a0dc-1142b51243f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:25.981087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c855d-ced8-491a-a0dc-1142b51243f8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-137c855d-ced8-491a-a0dc-1142b51243f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:25.996136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:25.997160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:25.998045 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:25.998045 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:25.998045 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:25.998220 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 527/1052] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:26.004143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62122d6b-c81f-4fad-95ee-af7daa28b5ab None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:26.024356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:26.116306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62122d6b-c81f-4fad-95ee-af7daa28b5ab None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:26.164385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec90d1d-c70a-449a-921e-30420b3d498c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:26.165048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec90d1d-c70a-449a-921e-30420b3d498c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:26.165344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec90d1d-c70a-449a-921e-30420b3d498c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:26.165573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec90d1d-c70a-449a-921e-30420b3d498c None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:26.166214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec90d1d-c70a-449a-921e-30420b3d498c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:26.166520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec90d1d-c70a-449a-921e-30420b3d498c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:26.167880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec90d1d-c70a-449a-921e-30420b3d498c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:26.184074 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 526/1053] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:26.185702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c855d-ced8-491a-a0dc-1142b51243f8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:26.206030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:26.277255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec90d1d-c70a-449a-921e-30420b3d498c None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:26.290237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c855d-ced8-491a-a0dc-1142b51243f8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:26.326056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64583b-d709-41cb-a6eb-de9bf7f3e73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:26.326681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64583b-d709-41cb-a6eb-de9bf7f3e73e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:26.326965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64583b-d709-41cb-a6eb-de9bf7f3e73e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:26.327243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64583b-d709-41cb-a6eb-de9bf7f3e73e None admin] PATH_INFO: `/v3/users/nova` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:26.335734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64583b-d709-41cb-a6eb-de9bf7f3e73e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:26.337208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64583b-d709-41cb-a6eb-de9bf7f3e73e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-0d64583b-d709-41cb-a6eb-de9bf7f3e73e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:34:26.350208 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:26.351174 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:26.352127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:26.352127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:26.352127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:34:26.352127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:34:26.352127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:34:26.352127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:34:26.352127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:34:26.352127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:34:26.352127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:34:26.352127 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:26.352127 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 528/1054] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 3 20:34:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:26.357698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64583b-d709-41cb-a6eb-de9bf7f3e73e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:26.364441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:26.391237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec90d1d-c70a-449a-921e-30420b3d498c None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:26.427607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98398021-e844-4a5b-bf12-2d5886c0b374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:26.428243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98398021-e844-4a5b-bf12-2d5886c0b374 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:26.428526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98398021-e844-4a5b-bf12-2d5886c0b374 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:26.428796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98398021-e844-4a5b-bf12-2d5886c0b374 None admin] PATH_INFO: `/v3/services/image` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:26.429396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98398021-e844-4a5b-bf12-2d5886c0b374 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:26.430757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98398021-e844-4a5b-bf12-2d5886c0b374 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:26.436161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-98398021-e844-4a5b-bf12-2d5886c0b374 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:26.441460 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:26.442691 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:26.443846 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 527/1055] 10.222.0.16 () {56 vars in 1105 bytes} [Sat Sep 3 20:34:26 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:26.448263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98398021-e844-4a5b-bf12-2d5886c0b374 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:26.465519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64583b-d709-41cb-a6eb-de9bf7f3e73e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:26.504652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4c30a-b169-460b-b6d3-71b6688c169c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:26.505356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4c30a-b169-460b-b6d3-71b6688c169c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:26.505628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4c30a-b169-460b-b6d3-71b6688c169c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:26.505908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4c30a-b169-460b-b6d3-71b6688c169c None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:26.506636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4c30a-b169-460b-b6d3-71b6688c169c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:26.506965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4c30a-b169-460b-b6d3-71b6688c169c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:26.508394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4c30a-b169-460b-b6d3-71b6688c169c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:26.530938 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 529/1056] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 3 20:34:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:26.531890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4c30a-b169-460b-b6d3-71b6688c169c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:26.565728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98398021-e844-4a5b-bf12-2d5886c0b374 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:26.602962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224145b-5fcc-498a-bb89-6515d5d6b6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:26.603572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224145b-5fcc-498a-bb89-6515d5d6b6a7 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:26.603859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224145b-5fcc-498a-bb89-6515d5d6b6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:26.604135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224145b-5fcc-498a-bb89-6515d5d6b6a7 None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:26.604866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224145b-5fcc-498a-bb89-6515d5d6b6a7 None admin] RBAC: Adding query filter params (name=image) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:26.605204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224145b-5fcc-498a-bb89-6515d5d6b6a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:26.606687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224145b-5fcc-498a-bb89-6515d5d6b6a7 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:26.612252 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 528/1057] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:26.617828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224145b-5fcc-498a-bb89-6515d5d6b6a7 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:26.622190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:26.636394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4c30a-b169-460b-b6d3-71b6688c169c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:26.673468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832da95-bf45-4409-9c9d-92859583ed01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:26.674102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832da95-bf45-4409-9c9d-92859583ed01 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:26.674382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832da95-bf45-4409-9c9d-92859583ed01 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:26.674751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832da95-bf45-4409-9c9d-92859583ed01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:26.677756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:26.685992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832da95-bf45-4409-9c9d-92859583ed01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:26.687970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832da95-bf45-4409-9c9d-92859583ed01 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a832da95-bf45-4409-9c9d-92859583ed01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:26.705156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:26.706247 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:26.707136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:26.707136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:26.707136 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:26.707136 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 530/1058] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:26.712186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832da95-bf45-4409-9c9d-92859583ed01 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:26.741166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224145b-5fcc-498a-bb89-6515d5d6b6a7 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:26.779326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f63009-d97d-4cf7-8172-dd0787b92c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:26.779948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f63009-d97d-4cf7-8172-dd0787b92c45 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:26.780245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f63009-d97d-4cf7-8172-dd0787b92c45 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:26.780527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f63009-d97d-4cf7-8172-dd0787b92c45 None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:26.781277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f63009-d97d-4cf7-8172-dd0787b92c45 None admin] RBAC: Adding query filter params (type=image) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:26.781539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f63009-d97d-4cf7-8172-dd0787b92c45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:26.782926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f63009-d97d-4cf7-8172-dd0787b92c45 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:26.793239 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 529/1059] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:26.794210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f63009-d97d-4cf7-8172-dd0787b92c45 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:26.826283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832da95-bf45-4409-9c9d-92859583ed01 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:26.864215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6b37b-bc81-439a-812e-686195c6ae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:26.864836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6b37b-bc81-439a-812e-686195c6ae98 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:26.865157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6b37b-bc81-439a-812e-686195c6ae98 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:26.865441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6b37b-bc81-439a-812e-686195c6ae98 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:26.866016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:26.866128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6b37b-bc81-439a-812e-686195c6ae98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:26.866390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6b37b-bc81-439a-812e-686195c6ae98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:26.867774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6b37b-bc81-439a-812e-686195c6ae98 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:26.880936 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 531/1060] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:26.886548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6b37b-bc81-439a-812e-686195c6ae98 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:26.902977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f63009-d97d-4cf7-8172-dd0787b92c45 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:26.938647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8124c-effa-4e30-8767-1838873b2c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:26.939311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8124c-effa-4e30-8767-1838873b2c34 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:26.939593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8124c-effa-4e30-8767-1838873b2c34 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:26.939719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:26.939940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8124c-effa-4e30-8767-1838873b2c34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:26.940521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8124c-effa-4e30-8767-1838873b2c34 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:26.941953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8124c-effa-4e30-8767-1838873b2c34 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:26.962860 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 530/1061] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:34:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:34:27.000697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6b37b-bc81-439a-812e-686195c6ae98 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:27.052453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43af08a-fb63-4f7a-ad4f-fd1a6cca3601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:27.053081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43af08a-fb63-4f7a-ad4f-fd1a6cca3601 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:27.053388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43af08a-fb63-4f7a-ad4f-fd1a6cca3601 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:27.053652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43af08a-fb63-4f7a-ad4f-fd1a6cca3601 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:27.062002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43af08a-fb63-4f7a-ad4f-fd1a6cca3601 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:27.063533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43af08a-fb63-4f7a-ad4f-fd1a6cca3601 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a43af08a-fb63-4f7a-ad4f-fd1a6cca3601 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:34:27.076559 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:27.077741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:27.079119 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:34:27.079119 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:34:27.079119 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:34:27.079119 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:34:27.079119 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:34:27.079119 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:34:27.079119 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:27.079119 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:27.081256 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 532/1062] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:34:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:27.084622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8124c-effa-4e30-8767-1838873b2c34 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:27.202425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8124c-effa-4e30-8767-1838873b2c34 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:27.216775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e2748-79b8-4402-b525-b786b069624c None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:27.217133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e2748-79b8-4402-b525-b786b069624c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:27.217813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e2748-79b8-4402-b525-b786b069624c None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:27.220572 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 533/1063] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:34:27.225551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c80ad-fea2-43de-81b2-7130de5343cc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:27.225839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c80ad-fea2-43de-81b2-7130de5343cc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:27.226132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c80ad-fea2-43de-81b2-7130de5343cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:27.228577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:27.241935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16bb38-5202-4aa6-a8e6-1a96e03c3130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:27.242553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16bb38-5202-4aa6-a8e6-1a96e03c3130 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:27.242838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16bb38-5202-4aa6-a8e6-1a96e03c3130 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:27.243114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16bb38-5202-4aa6-a8e6-1a96e03c3130 None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:27.243807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16bb38-5202-4aa6-a8e6-1a96e03c3130 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:27.244117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16bb38-5202-4aa6-a8e6-1a96e03c3130 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:27.245315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-530c80ad-fea2-43de-81b2-7130de5343cc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:27.245584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16bb38-5202-4aa6-a8e6-1a96e03c3130 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:27.246505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-530c80ad-fea2-43de-81b2-7130de5343cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:27.264933 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 531/1064] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:27.267829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16bb38-5202-4aa6-a8e6-1a96e03c3130 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:27.321165 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 534/1065] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:27.373060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16bb38-5202-4aa6-a8e6-1a96e03c3130 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:27.407673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e971b16-6bfe-442b-9e59-2aa20abed2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:27.408332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e971b16-6bfe-442b-9e59-2aa20abed2f8 None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:27.408617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e971b16-6bfe-442b-9e59-2aa20abed2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:27.408915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e971b16-6bfe-442b-9e59-2aa20abed2f8 None admin] PATH_INFO: `/v3/projects/1e040b2fca724c58a2660441a72cd350/users/a56702329eae48819121dcb58f7e3070/roles/95f5b6cfc093417cbb9882b6897e875f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:27.411101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e971b16-6bfe-442b-9e59-2aa20abed2f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e040b2fca724c58a2660441a72cd350, user_id=a56702329eae48819121dcb58f7e3070, role_id=95f5b6cfc093417cbb9882b6897e875f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:27.412538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e971b16-6bfe-442b-9e59-2aa20abed2f8 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:27.426996 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 532/1066] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:34:27 2022] PUT /identity/v3/projects/1e040b2fca724c58a2660441a72cd350/users/a56702329eae48819121dcb58f7e3070/roles/95f5b6cfc093417cbb9882b6897e875f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:34:27.654133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c80ad-fea2-43de-81b2-7130de5343cc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:27.689966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:27.758163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c80ad-fea2-43de-81b2-7130de5343cc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:27.791277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffe72f-bce6-4a8b-b055-db0d0af97fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:27.791908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffe72f-bce6-4a8b-b055-db0d0af97fa9 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:27.792191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffe72f-bce6-4a8b-b055-db0d0af97fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:27.792469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffe72f-bce6-4a8b-b055-db0d0af97fa9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:27.803052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffe72f-bce6-4a8b-b055-db0d0af97fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-f5ffe72f-bce6-4a8b-b055-db0d0af97fa9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:34:27.807895 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:27.808796 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:27.810000 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:27.810000 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:27.810000 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:27.810000 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:27.810000 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:34:27.810000 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:34:27.810000 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:34:27.810000 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:34:27.810000 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:34:27.810000 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:34:27.810000 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:34:27.810000 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:27.814859 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 535/1067] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:34:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:27.815998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e971b16-6bfe-442b-9e59-2aa20abed2f8 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:27.862653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:27.920149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e971b16-6bfe-442b-9e59-2aa20abed2f8 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:27.955502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c967423-29af-48d1-b0a8-0996accd5aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:27.956118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c967423-29af-48d1-b0a8-0996accd5aeb None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:27.956435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c967423-29af-48d1-b0a8-0996accd5aeb None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:27.956733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c967423-29af-48d1-b0a8-0996accd5aeb None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:27.957429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c967423-29af-48d1-b0a8-0996accd5aeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:27.958006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c967423-29af-48d1-b0a8-0996accd5aeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:27.959533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c967423-29af-48d1-b0a8-0996accd5aeb None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:27.973813 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 533/1068] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:34:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:27.975108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffe72f-bce6-4a8b-b055-db0d0af97fa9 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:28.075732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:28.084340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffe72f-bce6-4a8b-b055-db0d0af97fa9 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:28.120936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daaac9-e0b7-4942-8260-dc9f80de4de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:28.120936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daaac9-e0b7-4942-8260-dc9f80de4de1 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:28.120936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daaac9-e0b7-4942-8260-dc9f80de4de1 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:28.121310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daaac9-e0b7-4942-8260-dc9f80de4de1 None admin] PATH_INFO: `/v3/users/5db9696088af44f792d2ca5b4e890b2c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:28.138209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4daaac9-e0b7-4942-8260-dc9f80de4de1 None admin] RBAC: Authorizing `identity:get_user(user_id=5db9696088af44f792d2ca5b4e890b2c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:28.139670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4daaac9-e0b7-4942-8260-dc9f80de4de1 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:28.142132 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 536/1069] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:34:27 2022] GET /identity/v3/users/5db9696088af44f792d2ca5b4e890b2c => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:28.148075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c967423-29af-48d1-b0a8-0996accd5aeb None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:28.249092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c967423-29af-48d1-b0a8-0996accd5aeb None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:28.256927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:28.285560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216cf6a-711a-449c-8c10-0530fcbf04d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:28.286163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216cf6a-711a-449c-8c10-0530fcbf04d2 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:28.287092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216cf6a-711a-449c-8c10-0530fcbf04d2 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:28.287092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216cf6a-711a-449c-8c10-0530fcbf04d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:28.355061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216cf6a-711a-449c-8c10-0530fcbf04d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:28.356502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216cf6a-711a-449c-8c10-0530fcbf04d2 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:28.459628 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 534/1070] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:34:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 20:34:28.461624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daaac9-e0b7-4942-8260-dc9f80de4de1 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:28.567952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daaac9-e0b7-4942-8260-dc9f80de4de1 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:28.601855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecec484-2fbc-4a26-95b1-80864bf31de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:28.602493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecec484-2fbc-4a26-95b1-80864bf31de0 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:28.602778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecec484-2fbc-4a26-95b1-80864bf31de0 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:28.603055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecec484-2fbc-4a26-95b1-80864bf31de0 None admin] PATH_INFO: `/v3/projects/00c4df8abc174fa5ae37e967ec06cf10` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:28.604182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecec484-2fbc-4a26-95b1-80864bf31de0 None admin] RBAC: Authorizing `identity:get_project(project_id=00c4df8abc174fa5ae37e967ec06cf10)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:28.605649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecec484-2fbc-4a26-95b1-80864bf31de0 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:28.612890 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 537/1071] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:34:28 2022] GET /identity/v3/projects/00c4df8abc174fa5ae37e967ec06cf10 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:28.614173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216cf6a-711a-449c-8c10-0530fcbf04d2 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:28.622999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:28.717738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216cf6a-711a-449c-8c10-0530fcbf04d2 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:28.752554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f0ccd-c62a-4184-9f46-669f34045106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:28.753208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f0ccd-c62a-4184-9f46-669f34045106 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:28.753493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f0ccd-c62a-4184-9f46-669f34045106 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:28.753793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f0ccd-c62a-4184-9f46-669f34045106 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:28.754522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f0ccd-c62a-4184-9f46-669f34045106 None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=00c4df8abc174fa5ae37e967ec06cf10, user.id=5db9696088af44f792d2ca5b4e890b2c) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:28.754833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f0ccd-c62a-4184-9f46-669f34045106 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:28.756290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f0ccd-c62a-4184-9f46-669f34045106 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:28.767539 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 535/1072] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:34:28 2022] GET /identity/v3/role_assignments?user.id=5db9696088af44f792d2ca5b4e890b2c&scope.project.id=00c4df8abc174fa5ae37e967ec06cf10&role.id=bc768522dc984239be4a1b39a45ab437 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:29.217777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3bb37-23f2-4525-9ce5-574a851372e9 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:29.218292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3bb37-23f2-4525-9ce5-574a851372e9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:29.218369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3bb37-23f2-4525-9ce5-574a851372e9 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:29.220243 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 538/1073] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:29.227237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258be81b-342c-42c8-9a43-4362ec191660 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:29.227610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258be81b-342c-42c8-9a43-4362ec191660 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:29.227980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258be81b-342c-42c8-9a43-4362ec191660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:29.246463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-258be81b-342c-42c8-9a43-4362ec191660 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:29.247667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-258be81b-342c-42c8-9a43-4362ec191660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:29.314278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:29.324941 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 536/1074] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:29.620851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8568d8-0194-43de-b128-3e3891921efa None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:29.621270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8568d8-0194-43de-b128-3e3891921efa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:29.621476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8568d8-0194-43de-b128-3e3891921efa None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:29.622928 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 539/1075] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:29.627905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20cc67-440e-4233-ad99-504fcdf12eff None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:29.628203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20cc67-440e-4233-ad99-504fcdf12eff None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:29.628483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20cc67-440e-4233-ad99-504fcdf12eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:29.646186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0c20cc67-440e-4233-ad99-504fcdf12eff None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:29.647362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0c20cc67-440e-4233-ad99-504fcdf12eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:29.651319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8568d8-0194-43de-b128-3e3891921efa None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:29.676100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:29.718520 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 537/1076] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:29.766064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8568d8-0194-43de-b128-3e3891921efa None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:29.803114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48c125-550b-4b74-83d7-3b1b6dff31c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:29.803727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48c125-550b-4b74-83d7-3b1b6dff31c8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:29.804007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48c125-550b-4b74-83d7-3b1b6dff31c8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:29.804288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48c125-550b-4b74-83d7-3b1b6dff31c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:29.878845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48c125-550b-4b74-83d7-3b1b6dff31c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:29.880219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48c125-550b-4b74-83d7-3b1b6dff31c8 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:29.901494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:29.988946 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 540/1077] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:34:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 20:34:29.992464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20cc67-440e-4233-ad99-504fcdf12eff None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:30.046843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48c125-550b-4b74-83d7-3b1b6dff31c8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:30.101188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20cc67-440e-4233-ad99-504fcdf12eff None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:30.137781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:30.145506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8509dd-67dc-4aaf-abf4-d76fbb3b96aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:30.146141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8509dd-67dc-4aaf-abf4-d76fbb3b96aa None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:30.146422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8509dd-67dc-4aaf-abf4-d76fbb3b96aa None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:30.146696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8509dd-67dc-4aaf-abf4-d76fbb3b96aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:30.159012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8509dd-67dc-4aaf-abf4-d76fbb3b96aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:30.160510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8509dd-67dc-4aaf-abf4-d76fbb3b96aa None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:30.164580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48c125-550b-4b74-83d7-3b1b6dff31c8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-7d8509dd-67dc-4aaf-abf4-d76fbb3b96aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:30.176149 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:30.177175 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:30.178808 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:30.178808 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:30.178808 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:30.178808 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 538/1078] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:30.183089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8509dd-67dc-4aaf-abf4-d76fbb3b96aa None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:30.188034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:30.205417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc75ac-6a71-4ac6-acdf-fb8b508a6e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:30.206064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc75ac-6a71-4ac6-acdf-fb8b508a6e94 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:30.206343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc75ac-6a71-4ac6-acdf-fb8b508a6e94 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:30.206620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc75ac-6a71-4ac6-acdf-fb8b508a6e94 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:30.216924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc75ac-6a71-4ac6-acdf-fb8b508a6e94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-53bc75ac-6a71-4ac6-acdf-fb8b508a6e94 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:34:30.222741 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:30.229277 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:30.230134 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 541/1079] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Sep 3 20:34:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:30.230853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc75ac-6a71-4ac6-acdf-fb8b508a6e94 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:30.288498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8509dd-67dc-4aaf-abf4-d76fbb3b96aa None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:30.328712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113587b1-b086-4138-b9bd-e6705dc55c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:30.329404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113587b1-b086-4138-b9bd-e6705dc55c7e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:30.329677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113587b1-b086-4138-b9bd-e6705dc55c7e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:30.329903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113587b1-b086-4138-b9bd-e6705dc55c7e None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:30.330549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113587b1-b086-4138-b9bd-e6705dc55c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:30.330855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113587b1-b086-4138-b9bd-e6705dc55c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:30.332226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113587b1-b086-4138-b9bd-e6705dc55c7e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:30.339595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc75ac-6a71-4ac6-acdf-fb8b508a6e94 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:30.352952 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 539/1080] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:30.385318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135fb74-a82f-4cfc-aed6-9f928f798330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:30.385980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135fb74-a82f-4cfc-aed6-9f928f798330 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:30.386338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135fb74-a82f-4cfc-aed6-9f928f798330 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:30.386690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135fb74-a82f-4cfc-aed6-9f928f798330 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:30.387446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135fb74-a82f-4cfc-aed6-9f928f798330 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:30.387828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135fb74-a82f-4cfc-aed6-9f928f798330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:30.389380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135fb74-a82f-4cfc-aed6-9f928f798330 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:30.404362 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 542/1081] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 3 20:34:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:30.406127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113587b1-b086-4138-b9bd-e6705dc55c7e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:30.455740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:30.521576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113587b1-b086-4138-b9bd-e6705dc55c7e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:30.559970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7af7b-22c7-4d64-a1f8-f96b05d2050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:30.560606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7af7b-22c7-4d64-a1f8-f96b05d2050c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:30.560887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7af7b-22c7-4d64-a1f8-f96b05d2050c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:30.561215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7af7b-22c7-4d64-a1f8-f96b05d2050c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:30.572281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7af7b-22c7-4d64-a1f8-f96b05d2050c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:30.573744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7af7b-22c7-4d64-a1f8-f96b05d2050c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-aac7af7b-22c7-4d64-a1f8-f96b05d2050c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:30.589579 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:30.590757 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:30.592015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:30.592015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:30.592015 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:30.592015 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 540/1082] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:30.596495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135fb74-a82f-4cfc-aed6-9f928f798330 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:30.645667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:30.707869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135fb74-a82f-4cfc-aed6-9f928f798330 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:30.740836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0286e5-c184-42d5-ba5e-b2a777079fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:30.741480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0286e5-c184-42d5-ba5e-b2a777079fbd None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:30.741788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0286e5-c184-42d5-ba5e-b2a777079fbd None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:30.742124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0286e5-c184-42d5-ba5e-b2a777079fbd None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:30.742768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0286e5-c184-42d5-ba5e-b2a777079fbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:30.743074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0286e5-c184-42d5-ba5e-b2a777079fbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:30.744435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0286e5-c184-42d5-ba5e-b2a777079fbd None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:30.756497 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 543/1083] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:30.762350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7af7b-22c7-4d64-a1f8-f96b05d2050c None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:30.858020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:30.865707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7af7b-22c7-4d64-a1f8-f96b05d2050c None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:30.899093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57636f-8ccf-486b-b391-0467305b18a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:30.899651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57636f-8ccf-486b-b391-0467305b18a3 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:30.899934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57636f-8ccf-486b-b391-0467305b18a3 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:30.900210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57636f-8ccf-486b-b391-0467305b18a3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:30.907426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57636f-8ccf-486b-b391-0467305b18a3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:30.908858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57636f-8ccf-486b-b391-0467305b18a3 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-0c57636f-8ccf-486b-b391-0467305b18a3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:34:30.919625 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:30.920709 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:30.922378 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:30.922378 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:30.922378 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:34:30.922378 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:34:30.922378 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:34:30.922378 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:34:30.922378 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:34:30.922378 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:34:30.922378 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 20:34:30.922378 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:30.925055 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 541/1084] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 3 20:34:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:30.926917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0286e5-c184-42d5-ba5e-b2a777079fbd None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:30.946121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276a65b-b6ee-4a9a-9a9e-dd2bc5fb8fa8 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:30.946421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276a65b-b6ee-4a9a-9a9e-dd2bc5fb8fa8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:30.946698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276a65b-b6ee-4a9a-9a9e-dd2bc5fb8fa8 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:30.948041 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 542/1085] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:34:30.953100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4458903-680a-4448-a4db-e1de0afe97d1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:30.953386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4458903-680a-4448-a4db-e1de0afe97d1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:30.953663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4458903-680a-4448-a4db-e1de0afe97d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:30.971953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e4458903-680a-4448-a4db-e1de0afe97d1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:30.973182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e4458903-680a-4448-a4db-e1de0afe97d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:31.026050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0286e5-c184-42d5-ba5e-b2a777079fbd None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:31.027575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:31.041740 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 543/1086] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:31.061751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6d772-91ce-4c1b-a0d1-6f4bb1541fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:31.062900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6d772-91ce-4c1b-a0d1-6f4bb1541fbe None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:31.063233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6d772-91ce-4c1b-a0d1-6f4bb1541fbe None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:31.063516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6d772-91ce-4c1b-a0d1-6f4bb1541fbe None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:31.064247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6d772-91ce-4c1b-a0d1-6f4bb1541fbe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:31.064561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6d772-91ce-4c1b-a0d1-6f4bb1541fbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:31.066032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6d772-91ce-4c1b-a0d1-6f4bb1541fbe None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:31.086791 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 544/1087] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 3 20:34:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:31.092488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4458903-680a-4448-a4db-e1de0afe97d1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:31.175980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:31.192687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4458903-680a-4448-a4db-e1de0afe97d1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:31.225750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969728a6-18c2-4630-846d-ed00e9eb1c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:31.226353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969728a6-18c2-4630-846d-ed00e9eb1c41 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:31.226634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969728a6-18c2-4630-846d-ed00e9eb1c41 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:31.226908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969728a6-18c2-4630-846d-ed00e9eb1c41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:31.290839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969728a6-18c2-4630-846d-ed00e9eb1c41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:31.292270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969728a6-18c2-4630-846d-ed00e9eb1c41 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:31.372420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:31.373058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6d772-91ce-4c1b-a0d1-6f4bb1541fbe None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:31.390144 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 544/1088] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:34:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:34:31.396300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969728a6-18c2-4630-846d-ed00e9eb1c41 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:31.471455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6d772-91ce-4c1b-a0d1-6f4bb1541fbe None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:31.494648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969728a6-18c2-4630-846d-ed00e9eb1c41 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:31.504747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ff51a-ea00-43d8-939e-7637c5cdf0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:31.505447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ff51a-ea00-43d8-939e-7637c5cdf0d7 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:31.505741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ff51a-ea00-43d8-939e-7637c5cdf0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:31.506022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ff51a-ea00-43d8-939e-7637c5cdf0d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:31.516439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ff51a-ea00-43d8-939e-7637c5cdf0d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-143ff51a-ea00-43d8-939e-7637c5cdf0d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:34:31.521268 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:31.522370 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:31.523364 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 545/1089] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:34:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:31.528225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ff51a-ea00-43d8-939e-7637c5cdf0d7 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:31.533781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966307b1-c1cc-4652-9588-f026683e475e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:31.534412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966307b1-c1cc-4652-9588-f026683e475e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:31.534695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966307b1-c1cc-4652-9588-f026683e475e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:31.534970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966307b1-c1cc-4652-9588-f026683e475e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:31.544870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966307b1-c1cc-4652-9588-f026683e475e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:31.546331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966307b1-c1cc-4652-9588-f026683e475e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:31.549650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-966307b1-c1cc-4652-9588-f026683e475e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:34:31.560583 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:31.561735 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:34:31.562695 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:34:31.562695 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:34:31.562695 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:31.562695 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 545/1090] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:31.567280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966307b1-c1cc-4652-9588-f026683e475e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:31.601818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:31.630794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ff51a-ea00-43d8-939e-7637c5cdf0d7 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:31.665521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032763d-7f86-429b-bd58-a598edcea28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:31.666121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032763d-7f86-429b-bd58-a598edcea28c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:31.666402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032763d-7f86-429b-bd58-a598edcea28c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:31.666677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032763d-7f86-429b-bd58-a598edcea28c None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:31.667354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032763d-7f86-429b-bd58-a598edcea28c None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:31.667658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032763d-7f86-429b-bd58-a598edcea28c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:31.668480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966307b1-c1cc-4652-9588-f026683e475e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:31.669035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032763d-7f86-429b-bd58-a598edcea28c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:31.681262 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 546/1091] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:34:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:31.683253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032763d-7f86-429b-bd58-a598edcea28c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:31.706195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc01113-20d1-4608-ac0f-2a2d3ce21e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:31.706801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc01113-20d1-4608-ac0f-2a2d3ce21e1f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:31.707089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc01113-20d1-4608-ac0f-2a2d3ce21e1f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:31.707367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc01113-20d1-4608-ac0f-2a2d3ce21e1f None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:31.708267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc01113-20d1-4608-ac0f-2a2d3ce21e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:31.708574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc01113-20d1-4608-ac0f-2a2d3ce21e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:31.709975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc01113-20d1-4608-ac0f-2a2d3ce21e1f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:31.722058 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 546/1092] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:31.727448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc01113-20d1-4608-ac0f-2a2d3ce21e1f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:31.763741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:31.782645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032763d-7f86-429b-bd58-a598edcea28c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:31.815459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5bbe8-3c6e-4698-a678-75fa02c58731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:31.816085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5bbe8-3c6e-4698-a678-75fa02c58731 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:31.816368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5bbe8-3c6e-4698-a678-75fa02c58731 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:31.816642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5bbe8-3c6e-4698-a678-75fa02c58731 None admin] PATH_INFO: `/v3/users/5db9696088af44f792d2ca5b4e890b2c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:31.817782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5bbe8-3c6e-4698-a678-75fa02c58731 None admin] RBAC: Authorizing `identity:get_user(user_id=5db9696088af44f792d2ca5b4e890b2c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:31.819291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5bbe8-3c6e-4698-a678-75fa02c58731 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:31.819832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:31.820932 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 547/1093] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:34:31 2022] GET /identity/v3/users/5db9696088af44f792d2ca5b4e890b2c => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:31.825739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc01113-20d1-4608-ac0f-2a2d3ce21e1f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:31.826942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5bbe8-3c6e-4698-a678-75fa02c58731 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:31.860391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b5c1a-4c72-4593-92fe-a85ca0ba1565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:31.861060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b5c1a-4c72-4593-92fe-a85ca0ba1565 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:31.861348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b5c1a-4c72-4593-92fe-a85ca0ba1565 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:31.861625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b5c1a-4c72-4593-92fe-a85ca0ba1565 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:31.868306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b5c1a-4c72-4593-92fe-a85ca0ba1565 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:31.869793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b5c1a-4c72-4593-92fe-a85ca0ba1565 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-4e1b5c1a-4c72-4593-92fe-a85ca0ba1565 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:34:31.881487 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:31.882377 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:31.883256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:34:31.883256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:34:31.883256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:34:31.883256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:34:31.883256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:34:31.883256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:34:31.883256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:34:31.883256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:34:31.883256 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 547/1094] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:34:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:34:31.888295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b5c1a-4c72-4593-92fe-a85ca0ba1565 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:31.930636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5bbe8-3c6e-4698-a678-75fa02c58731 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:31.964831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034cb70-2b2c-4d17-ac25-31c66e21335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:31.965483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034cb70-2b2c-4d17-ac25-31c66e21335d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:31.965772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034cb70-2b2c-4d17-ac25-31c66e21335d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:31.966058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034cb70-2b2c-4d17-ac25-31c66e21335d None admin] PATH_INFO: `/v3/projects/00c4df8abc174fa5ae37e967ec06cf10` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:31.967179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034cb70-2b2c-4d17-ac25-31c66e21335d None admin] RBAC: Authorizing `identity:get_project(project_id=00c4df8abc174fa5ae37e967ec06cf10)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:31.968643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034cb70-2b2c-4d17-ac25-31c66e21335d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:31.970308 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 548/1095] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:34:31 2022] GET /identity/v3/projects/00c4df8abc174fa5ae37e967ec06cf10 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:31.976289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034cb70-2b2c-4d17-ac25-31c66e21335d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:31.986061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b5c1a-4c72-4593-92fe-a85ca0ba1565 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:31.994137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:32.021649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2214d-f32c-4f53-8949-fd80a6f709a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:32.022244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2214d-f32c-4f53-8949-fd80a6f709a8 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:32.022527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2214d-f32c-4f53-8949-fd80a6f709a8 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:32.022800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2214d-f32c-4f53-8949-fd80a6f709a8 None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:32.023474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2214d-f32c-4f53-8949-fd80a6f709a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:32.023777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2214d-f32c-4f53-8949-fd80a6f709a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:32.025227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2214d-f32c-4f53-8949-fd80a6f709a8 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:32.036235 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 548/1096] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:32.042221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2214d-f32c-4f53-8949-fd80a6f709a8 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:32.059501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:32.075181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034cb70-2b2c-4d17-ac25-31c66e21335d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:32.110078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d67bd1-019e-4bb5-ab48-461ca14b23f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:32.110737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d67bd1-019e-4bb5-ab48-461ca14b23f2 None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:32.111026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d67bd1-019e-4bb5-ab48-461ca14b23f2 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:32.111305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d67bd1-019e-4bb5-ab48-461ca14b23f2 None admin] PATH_INFO: `/v3/projects/00c4df8abc174fa5ae37e967ec06cf10/users/5db9696088af44f792d2ca5b4e890b2c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:32.113252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d67bd1-019e-4bb5-ab48-461ca14b23f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=00c4df8abc174fa5ae37e967ec06cf10, user_id=5db9696088af44f792d2ca5b4e890b2c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:32.114693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d67bd1-019e-4bb5-ab48-461ca14b23f2 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:32.136987 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 549/1097] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:34:31 2022] PUT /identity/v3/projects/00c4df8abc174fa5ae37e967ec06cf10/users/5db9696088af44f792d2ca5b4e890b2c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:34:32.146406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2214d-f32c-4f53-8949-fd80a6f709a8 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:32.183431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d27f0-9072-47d8-87f6-d1dd38bf55ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:32.184061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d27f0-9072-47d8-87f6-d1dd38bf55ff None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:32.184348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d27f0-9072-47d8-87f6-d1dd38bf55ff None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:32.184626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d27f0-9072-47d8-87f6-d1dd38bf55ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:32.185613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d27f0-9072-47d8-87f6-d1dd38bf55ff None admin] RBAC: Adding query filter params (role.id=95f5b6cfc093417cbb9882b6897e875f, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=a56702329eae48819121dcb58f7e3070) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:32.186016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d27f0-9072-47d8-87f6-d1dd38bf55ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:32.188077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d27f0-9072-47d8-87f6-d1dd38bf55ff None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:32.194834 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 549/1098] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:34:32 2022] GET /identity/v3/role_assignments?user.id=a56702329eae48819121dcb58f7e3070&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=95f5b6cfc093417cbb9882b6897e875f => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:34:34.329080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8e03b-0983-4907-b54b-cfaf6f3e488a None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:34.329624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8e03b-0983-4907-b54b-cfaf6f3e488a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:34.329624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8e03b-0983-4907-b54b-cfaf6f3e488a None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:34.330967 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 550/1099] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:34.336468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d62f5b-ce5a-4f66-803b-e795a14e6446 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:34.336963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d62f5b-ce5a-4f66-803b-e795a14e6446 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:34.337052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d62f5b-ce5a-4f66-803b-e795a14e6446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:34.354891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a8d62f5b-ce5a-4f66-803b-e795a14e6446 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:34.356096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a8d62f5b-ce5a-4f66-803b-e795a14e6446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:34.420778 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 550/1100] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:34.755074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8e03b-0983-4907-b54b-cfaf6f3e488a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:34.835342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:34.858450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8e03b-0983-4907-b54b-cfaf6f3e488a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:34.892293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65022349-7f94-4364-9469-f71a5a22772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:34.892944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65022349-7f94-4364-9469-f71a5a22772c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:34.893236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65022349-7f94-4364-9469-f71a5a22772c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:34.893516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65022349-7f94-4364-9469-f71a5a22772c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:34.904141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65022349-7f94-4364-9469-f71a5a22772c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-65022349-7f94-4364-9469-f71a5a22772c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:34:34.909713 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:34.910817 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:34.911870 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 551/1101] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:34:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:34.916945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d62f5b-ce5a-4f66-803b-e795a14e6446 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:34.928956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:35.017715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d62f5b-ce5a-4f66-803b-e795a14e6446 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:35.051397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c4ad4-fdda-4e5f-8dc3-3468e8414f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:35.052006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c4ad4-fdda-4e5f-8dc3-3468e8414f15 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:35.052292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c4ad4-fdda-4e5f-8dc3-3468e8414f15 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:35.052573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c4ad4-fdda-4e5f-8dc3-3468e8414f15 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:35.053272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c4ad4-fdda-4e5f-8dc3-3468e8414f15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:35.053580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c4ad4-fdda-4e5f-8dc3-3468e8414f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:35.054940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c4ad4-fdda-4e5f-8dc3-3468e8414f15 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:35.062789 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 551/1102] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:34:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:35.068781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65022349-7f94-4364-9469-f71a5a22772c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:35.167650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65022349-7f94-4364-9469-f71a5a22772c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:35.199314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50a194-0b1d-4802-b7bd-9cea9eeb3a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:35.199933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50a194-0b1d-4802-b7bd-9cea9eeb3a22 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:35.200225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50a194-0b1d-4802-b7bd-9cea9eeb3a22 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:35.200501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50a194-0b1d-4802-b7bd-9cea9eeb3a22 None admin] PATH_INFO: `/v3/users/5db9696088af44f792d2ca5b4e890b2c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:35.201665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50a194-0b1d-4802-b7bd-9cea9eeb3a22 None admin] RBAC: Authorizing `identity:get_user(user_id=5db9696088af44f792d2ca5b4e890b2c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:35.203658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50a194-0b1d-4802-b7bd-9cea9eeb3a22 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:35.205265 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 552/1103] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:34:35 2022] GET /identity/v3/users/5db9696088af44f792d2ca5b4e890b2c => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:35.211361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c4ad4-fdda-4e5f-8dc3-3468e8414f15 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:35.279468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:35.308632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c4ad4-fdda-4e5f-8dc3-3468e8414f15 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:35.341236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af1bc6-2b79-466e-ac8f-53e1b79c41d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:35.341817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af1bc6-2b79-466e-ac8f-53e1b79c41d8 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:35.342109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af1bc6-2b79-466e-ac8f-53e1b79c41d8 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:35.342387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af1bc6-2b79-466e-ac8f-53e1b79c41d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:35.407892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af1bc6-2b79-466e-ac8f-53e1b79c41d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:35.409355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af1bc6-2b79-466e-ac8f-53e1b79c41d8 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:35.474814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:35.504195 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 552/1104] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:34:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:34:35.510039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50a194-0b1d-4802-b7bd-9cea9eeb3a22 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:35.520678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:35.608379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50a194-0b1d-4802-b7bd-9cea9eeb3a22 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:35.641046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2427244-338b-4f18-8395-6b18f6751008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:35.641685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2427244-338b-4f18-8395-6b18f6751008 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:35.641971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2427244-338b-4f18-8395-6b18f6751008 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:35.642249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2427244-338b-4f18-8395-6b18f6751008 None admin] PATH_INFO: `/v3/projects/00c4df8abc174fa5ae37e967ec06cf10` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:35.643362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2427244-338b-4f18-8395-6b18f6751008 None admin] RBAC: Authorizing `identity:get_project(project_id=00c4df8abc174fa5ae37e967ec06cf10)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:35.644786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2427244-338b-4f18-8395-6b18f6751008 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:35.646424 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 553/1105] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:34:35 2022] GET /identity/v3/projects/00c4df8abc174fa5ae37e967ec06cf10 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:35.652398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af1bc6-2b79-466e-ac8f-53e1b79c41d8 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:35.750105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af1bc6-2b79-466e-ac8f-53e1b79c41d8 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:35.782422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44edcc-96ec-4945-9c28-689f48ebdd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:35.783015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44edcc-96ec-4945-9c28-689f48ebdd9d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:35.783295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44edcc-96ec-4945-9c28-689f48ebdd9d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:35.783567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44edcc-96ec-4945-9c28-689f48ebdd9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:35.784277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44edcc-96ec-4945-9c28-689f48ebdd9d None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=00c4df8abc174fa5ae37e967ec06cf10, user.id=5db9696088af44f792d2ca5b4e890b2c) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:35.784581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44edcc-96ec-4945-9c28-689f48ebdd9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:35.786013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44edcc-96ec-4945-9c28-689f48ebdd9d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:35.791510 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 553/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:34:35 2022] GET /identity/v3/role_assignments?user.id=5db9696088af44f792d2ca5b4e890b2c&scope.project.id=00c4df8abc174fa5ae37e967ec06cf10&role.id=bc768522dc984239be4a1b39a45ab437 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:37.986323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165db8c0-38d8-4ad1-a6a9-e8764c0a6242 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:37.986903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165db8c0-38d8-4ad1-a6a9-e8764c0a6242 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:37.986903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165db8c0-38d8-4ad1-a6a9-e8764c0a6242 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:37.988238 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 554/1107] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:37.993298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c53c95-32bc-459c-9a33-e18c965fe4a4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:37.993698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c53c95-32bc-459c-9a33-e18c965fe4a4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:37.993866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c53c95-32bc-459c-9a33-e18c965fe4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:38.011683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a9c53c95-32bc-459c-9a33-e18c965fe4a4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:38.012855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a9c53c95-32bc-459c-9a33-e18c965fe4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:38.040604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:38.077623 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 554/1108] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:38.404195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165db8c0-38d8-4ad1-a6a9-e8764c0a6242 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:38.482722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:38.512095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165db8c0-38d8-4ad1-a6a9-e8764c0a6242 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:38.544851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c2c9c-6018-404d-a3cc-aa3a3837b5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:38.545569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c2c9c-6018-404d-a3cc-aa3a3837b5d6 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:38.545854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c2c9c-6018-404d-a3cc-aa3a3837b5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:38.546130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c2c9c-6018-404d-a3cc-aa3a3837b5d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:38.547261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c2c9c-6018-404d-a3cc-aa3a3837b5d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:38.548690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c2c9c-6018-404d-a3cc-aa3a3837b5d6 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:38.550279 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 555/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:38.556208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c53c95-32bc-459c-9a33-e18c965fe4a4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:38.652413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c53c95-32bc-459c-9a33-e18c965fe4a4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:38.686919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f5f6f-1c18-4033-8136-a79db337ed13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:38.687957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f5f6f-1c18-4033-8136-a79db337ed13 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:38.688244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f5f6f-1c18-4033-8136-a79db337ed13 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:38.688524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f5f6f-1c18-4033-8136-a79db337ed13 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:38.689223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f5f6f-1c18-4033-8136-a79db337ed13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:38.690657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f5f6f-1c18-4033-8136-a79db337ed13 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:38.712389 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 555/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:34:38 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:34:40.926243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f1245-d488-4f41-8303-b940aa28c68d None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:40.926807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f1245-d488-4f41-8303-b940aa28c68d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:40.926807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f1245-d488-4f41-8303-b940aa28c68d None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:40.928130 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 556/1111] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:40.933352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b9076-0571-4e58-8d1c-47f4d75d2e57 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:40.933722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b9076-0571-4e58-8d1c-47f4d75d2e57 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:40.933923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b9076-0571-4e58-8d1c-47f4d75d2e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:40.952293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-752b9076-0571-4e58-8d1c-47f4d75d2e57 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:40.954136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-752b9076-0571-4e58-8d1c-47f4d75d2e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:40.970492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:41.020643 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 556/1112] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:41.355587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f1245-d488-4f41-8303-b940aa28c68d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:41.454821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f1245-d488-4f41-8303-b940aa28c68d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:41.487397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d98478-51c5-4c02-b22a-8ffdc60a34e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:41.488009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d98478-51c5-4c02-b22a-8ffdc60a34e5 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:41.488291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d98478-51c5-4c02-b22a-8ffdc60a34e5 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:41.488565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d98478-51c5-4c02-b22a-8ffdc60a34e5 None admin] PATH_INFO: `/v3/roles/542f58433b2747d2abd5d405ebabdea1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:41.489971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d98478-51c5-4c02-b22a-8ffdc60a34e5 None admin] RBAC: Authorizing `identity:get_role(role_id=542f58433b2747d2abd5d405ebabdea1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:41.491331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d98478-51c5-4c02-b22a-8ffdc60a34e5 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:41.492626 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 557/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:34:41 2022] GET /identity/v3/roles/542f58433b2747d2abd5d405ebabdea1 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:41.498372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b9076-0571-4e58-8d1c-47f4d75d2e57 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:41.592865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b9076-0571-4e58-8d1c-47f4d75d2e57 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:41.625090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b96ba-0f10-45ba-96e7-39e84d02d763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:41.625674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b96ba-0f10-45ba-96e7-39e84d02d763 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:41.625957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b96ba-0f10-45ba-96e7-39e84d02d763 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:41.626231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b96ba-0f10-45ba-96e7-39e84d02d763 None admin] PATH_INFO: `/v3/users/8e49d824214d4d168930c56c590e739d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:41.638674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b96ba-0f10-45ba-96e7-39e84d02d763 None admin] RBAC: Authorizing `identity:get_user(user_id=8e49d824214d4d168930c56c590e739d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:41.640087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b96ba-0f10-45ba-96e7-39e84d02d763 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:41.642263 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 557/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:34:41 2022] GET /identity/v3/users/8e49d824214d4d168930c56c590e739d => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:41.647848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d98478-51c5-4c02-b22a-8ffdc60a34e5 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:41.659717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:41.747859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d98478-51c5-4c02-b22a-8ffdc60a34e5 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:41.781465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ea572-0018-44d7-9620-ee67b3d4faa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:41.782064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ea572-0018-44d7-9620-ee67b3d4faa5 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:41.782351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ea572-0018-44d7-9620-ee67b3d4faa5 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:41.782629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ea572-0018-44d7-9620-ee67b3d4faa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:41.848495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ea572-0018-44d7-9620-ee67b3d4faa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:41.849970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ea572-0018-44d7-9620-ee67b3d4faa5 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:41.861222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:41.944994 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 558/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:34:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:34:41.951968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b96ba-0f10-45ba-96e7-39e84d02d763 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:41.964261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:42.050846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b96ba-0f10-45ba-96e7-39e84d02d763 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:42.084635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af712b-baa5-4cd4-9a59-e73e4cdabadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:42.085286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af712b-baa5-4cd4-9a59-e73e4cdabadd None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:42.085574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af712b-baa5-4cd4-9a59-e73e4cdabadd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:42.085853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af712b-baa5-4cd4-9a59-e73e4cdabadd None admin] PATH_INFO: `/v3/projects/00c4df8abc174fa5ae37e967ec06cf10` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:42.086973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af712b-baa5-4cd4-9a59-e73e4cdabadd None admin] RBAC: Authorizing `identity:get_project(project_id=00c4df8abc174fa5ae37e967ec06cf10)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:42.088396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af712b-baa5-4cd4-9a59-e73e4cdabadd None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:42.090216 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 558/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:34:41 2022] GET /identity/v3/projects/00c4df8abc174fa5ae37e967ec06cf10 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:42.096384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ea572-0018-44d7-9620-ee67b3d4faa5 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:42.198987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ea572-0018-44d7-9620-ee67b3d4faa5 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:42.209476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:42.232244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa6e89-2001-4e4d-b8bf-194f830dc98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:42.232840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa6e89-2001-4e4d-b8bf-194f830dc98b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:42.233157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa6e89-2001-4e4d-b8bf-194f830dc98b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:42.233429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa6e89-2001-4e4d-b8bf-194f830dc98b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:42.234763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa6e89-2001-4e4d-b8bf-194f830dc98b None admin] RBAC: Adding query filter params (role.id=542f58433b2747d2abd5d405ebabdea1, scope.project.id=00c4df8abc174fa5ae37e967ec06cf10, user.id=8e49d824214d4d168930c56c590e739d) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:42.235071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa6e89-2001-4e4d-b8bf-194f830dc98b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:42.236490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa6e89-2001-4e4d-b8bf-194f830dc98b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:42.242096 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 559/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:34:42 2022] GET /identity/v3/role_assignments?user.id=8e49d824214d4d168930c56c590e739d&scope.project.id=00c4df8abc174fa5ae37e967ec06cf10&role.id=542f58433b2747d2abd5d405ebabdea1 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:44.407838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d5ded-6d24-4205-a2ad-24f95cb2f455 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:44.408380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d5ded-6d24-4205-a2ad-24f95cb2f455 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:44.408380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d5ded-6d24-4205-a2ad-24f95cb2f455 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:44.409659 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 559/1118] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:44.414756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e05ad-c5cf-4c6b-9a25-00cffb6446cc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:44.415121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e05ad-c5cf-4c6b-9a25-00cffb6446cc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:44.415312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e05ad-c5cf-4c6b-9a25-00cffb6446cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:44.433546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cf9e05ad-c5cf-4c6b-9a25-00cffb6446cc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:44.434725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cf9e05ad-c5cf-4c6b-9a25-00cffb6446cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:44.499006 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 560/1119] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:44.833667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d5ded-6d24-4205-a2ad-24f95cb2f455 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:44.912374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:44.934997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d5ded-6d24-4205-a2ad-24f95cb2f455 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:44.968352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19150a3b-65b8-4c01-99ba-92d67c07fbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:44.969006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19150a3b-65b8-4c01-99ba-92d67c07fbc8 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:44.969291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19150a3b-65b8-4c01-99ba-92d67c07fbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:44.969568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19150a3b-65b8-4c01-99ba-92d67c07fbc8 None admin] PATH_INFO: `/v3/roles/542f58433b2747d2abd5d405ebabdea1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:44.970959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19150a3b-65b8-4c01-99ba-92d67c07fbc8 None admin] RBAC: Authorizing `identity:get_role(role_id=542f58433b2747d2abd5d405ebabdea1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:44.972320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19150a3b-65b8-4c01-99ba-92d67c07fbc8 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:44.973651 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 560/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:34:44 2022] GET /identity/v3/roles/542f58433b2747d2abd5d405ebabdea1 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:44.979561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e05ad-c5cf-4c6b-9a25-00cffb6446cc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:45.075447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e05ad-c5cf-4c6b-9a25-00cffb6446cc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:45.077720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:45.108447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa55e1-baad-4505-af41-c266b5c8dd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:45.109531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa55e1-baad-4505-af41-c266b5c8dd40 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:45.109814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa55e1-baad-4505-af41-c266b5c8dd40 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:45.110086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa55e1-baad-4505-af41-c266b5c8dd40 None admin] PATH_INFO: `/v3/users/8e49d824214d4d168930c56c590e739d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:45.111191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa55e1-baad-4505-af41-c266b5c8dd40 None admin] RBAC: Authorizing `identity:get_user(user_id=8e49d824214d4d168930c56c590e739d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:45.112616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa55e1-baad-4505-af41-c266b5c8dd40 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:45.114191 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 561/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:34:44 2022] GET /identity/v3/users/8e49d824214d4d168930c56c590e739d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:45.120010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19150a3b-65b8-4c01-99ba-92d67c07fbc8 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:45.227017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19150a3b-65b8-4c01-99ba-92d67c07fbc8 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:45.260753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286a271-7952-4e8f-b308-7bf68b0e817e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:45.261414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286a271-7952-4e8f-b308-7bf68b0e817e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:45.261704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286a271-7952-4e8f-b308-7bf68b0e817e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:45.261985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286a271-7952-4e8f-b308-7bf68b0e817e None admin] PATH_INFO: `/v3/projects/00c4df8abc174fa5ae37e967ec06cf10` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:45.263109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286a271-7952-4e8f-b308-7bf68b0e817e None admin] RBAC: Authorizing `identity:get_project(project_id=00c4df8abc174fa5ae37e967ec06cf10)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:45.264555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286a271-7952-4e8f-b308-7bf68b0e817e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:45.266242 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 561/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:34:45 2022] GET /identity/v3/projects/00c4df8abc174fa5ae37e967ec06cf10 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:45.272230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa55e1-baad-4505-af41-c266b5c8dd40 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:45.367618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa55e1-baad-4505-af41-c266b5c8dd40 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:45.400406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc252ff0-55ac-413b-95c4-9a57a0dcd76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:45.401079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc252ff0-55ac-413b-95c4-9a57a0dcd76d None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:45.401363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc252ff0-55ac-413b-95c4-9a57a0dcd76d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:45.401640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc252ff0-55ac-413b-95c4-9a57a0dcd76d None admin] PATH_INFO: `/v3/projects/00c4df8abc174fa5ae37e967ec06cf10/users/8e49d824214d4d168930c56c590e739d/roles/542f58433b2747d2abd5d405ebabdea1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:45.403742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc252ff0-55ac-413b-95c4-9a57a0dcd76d None admin] RBAC: Authorizing `identity:create_grant(project_id=00c4df8abc174fa5ae37e967ec06cf10, user_id=8e49d824214d4d168930c56c590e739d, role_id=542f58433b2747d2abd5d405ebabdea1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:45.405197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc252ff0-55ac-413b-95c4-9a57a0dcd76d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:45.417885 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 562/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:34:45 2022] PUT /identity/v3/projects/00c4df8abc174fa5ae37e967ec06cf10/users/8e49d824214d4d168930c56c590e739d/roles/542f58433b2747d2abd5d405ebabdea1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:34:47.817807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c38078-594b-4413-8b4a-bb0f7bfe957c None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:47.818357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c38078-594b-4413-8b4a-bb0f7bfe957c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:47.818357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c38078-594b-4413-8b4a-bb0f7bfe957c None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:47.819704 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 562/1124] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:47.825064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd84cb-457f-4fb2-b7b9-b63e64b46957 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:47.825472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd84cb-457f-4fb2-b7b9-b63e64b46957 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:47.825613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd84cb-457f-4fb2-b7b9-b63e64b46957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:47.844215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3ebd84cb-457f-4fb2-b7b9-b63e64b46957 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:47.845432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3ebd84cb-457f-4fb2-b7b9-b63e64b46957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:47.870379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:47.910135 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 563/1125] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:34:48.246875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c38078-594b-4413-8b4a-bb0f7bfe957c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:48.264112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:48.348497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c38078-594b-4413-8b4a-bb0f7bfe957c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:48.381670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b8b6c-26af-4a38-ac17-cebb53dcf0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:48.382300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b8b6c-26af-4a38-ac17-cebb53dcf0b7 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:48.382588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b8b6c-26af-4a38-ac17-cebb53dcf0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:48.382919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b8b6c-26af-4a38-ac17-cebb53dcf0b7 None admin] PATH_INFO: `/v3/roles/542f58433b2747d2abd5d405ebabdea1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:48.384273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b8b6c-26af-4a38-ac17-cebb53dcf0b7 None admin] RBAC: Authorizing `identity:get_role(role_id=542f58433b2747d2abd5d405ebabdea1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:48.385673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b8b6c-26af-4a38-ac17-cebb53dcf0b7 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:48.386990 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 563/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:34:48 2022] GET /identity/v3/roles/542f58433b2747d2abd5d405ebabdea1 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:48.392845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd84cb-457f-4fb2-b7b9-b63e64b46957 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:48.487931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd84cb-457f-4fb2-b7b9-b63e64b46957 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:48.521030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eea62-5e49-4254-bf20-b1512c3f9f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:48.521650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eea62-5e49-4254-bf20-b1512c3f9f2e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:48.521933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eea62-5e49-4254-bf20-b1512c3f9f2e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:48.522210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eea62-5e49-4254-bf20-b1512c3f9f2e None admin] PATH_INFO: `/v3/users/8e49d824214d4d168930c56c590e739d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:48.523313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786eea62-5e49-4254-bf20-b1512c3f9f2e None admin] RBAC: Authorizing `identity:get_user(user_id=8e49d824214d4d168930c56c590e739d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:48.524738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786eea62-5e49-4254-bf20-b1512c3f9f2e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:48.526311 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 564/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:34:48 2022] GET /identity/v3/users/8e49d824214d4d168930c56c590e739d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:48.532154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b8b6c-26af-4a38-ac17-cebb53dcf0b7 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:48.613685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:48.632190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b8b6c-26af-4a38-ac17-cebb53dcf0b7 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:48.665381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24db944-8dc7-431f-b265-e010df26a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:48.665935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24db944-8dc7-431f-b265-e010df26a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:48.666221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24db944-8dc7-431f-b265-e010df26a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:48.666501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24db944-8dc7-431f-b265-e010df26a9a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:48.732480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24db944-8dc7-431f-b265-e010df26a9a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:48.734476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24db944-8dc7-431f-b265-e010df26a9a0 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:48.812879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:48.831555 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 564/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:34:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:34:48.838084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eea62-5e49-4254-bf20-b1512c3f9f2e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:48.867705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:48.937148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eea62-5e49-4254-bf20-b1512c3f9f2e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:48.970995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f0bff-5227-4bf2-9098-0c08d0e21bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:48.971621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f0bff-5227-4bf2-9098-0c08d0e21bac None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:48.971903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f0bff-5227-4bf2-9098-0c08d0e21bac None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:48.972179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f0bff-5227-4bf2-9098-0c08d0e21bac None admin] PATH_INFO: `/v3/projects/00c4df8abc174fa5ae37e967ec06cf10` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:48.973324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f0bff-5227-4bf2-9098-0c08d0e21bac None admin] RBAC: Authorizing `identity:get_project(project_id=00c4df8abc174fa5ae37e967ec06cf10)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:48.974758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f0bff-5227-4bf2-9098-0c08d0e21bac None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:48.976517 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 565/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:34:48 2022] GET /identity/v3/projects/00c4df8abc174fa5ae37e967ec06cf10 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:48.982751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24db944-8dc7-431f-b265-e010df26a9a0 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:49.077102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24db944-8dc7-431f-b265-e010df26a9a0 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:49.110699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f172a-6c5c-47f9-9916-44af34f0ba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:49.111287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f172a-6c5c-47f9-9916-44af34f0ba1a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:49.111597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f172a-6c5c-47f9-9916-44af34f0ba1a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:49.111873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f172a-6c5c-47f9-9916-44af34f0ba1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:49.112584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f172a-6c5c-47f9-9916-44af34f0ba1a None admin] RBAC: Adding query filter params (role.id=542f58433b2747d2abd5d405ebabdea1, scope.project.id=00c4df8abc174fa5ae37e967ec06cf10, user.id=8e49d824214d4d168930c56c590e739d) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:49.112922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f172a-6c5c-47f9-9916-44af34f0ba1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:49.114326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f172a-6c5c-47f9-9916-44af34f0ba1a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:49.120155 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 565/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:34:48 2022] GET /identity/v3/role_assignments?user.id=8e49d824214d4d168930c56c590e739d&scope.project.id=00c4df8abc174fa5ae37e967ec06cf10&role.id=542f58433b2747d2abd5d405ebabdea1 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:51.384142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924b712-0e72-4176-86ac-edfad01e91b4 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:51.384665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924b712-0e72-4176-86ac-edfad01e91b4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:51.384665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924b712-0e72-4176-86ac-edfad01e91b4 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:51.385964 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 566/1131] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:51.392191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98aa85-5b39-4db5-b1ea-a136d165fa8f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:51.392601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98aa85-5b39-4db5-b1ea-a136d165fa8f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:51.392744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98aa85-5b39-4db5-b1ea-a136d165fa8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:51.411379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-de98aa85-5b39-4db5-b1ea-a136d165fa8f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:51.412626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-de98aa85-5b39-4db5-b1ea-a136d165fa8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:51.453234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:51.477999 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 566/1132] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:51.805950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924b712-0e72-4176-86ac-edfad01e91b4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:51.905290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:51.910537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924b712-0e72-4176-86ac-edfad01e91b4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:51.943892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fb667-5738-4d17-afbc-4abc69088599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:51.944516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fb667-5738-4d17-afbc-4abc69088599 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:51.944801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fb667-5738-4d17-afbc-4abc69088599 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:51.945110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fb667-5738-4d17-afbc-4abc69088599 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:51.946223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fb667-5738-4d17-afbc-4abc69088599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:51.947654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fb667-5738-4d17-afbc-4abc69088599 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:51.949247 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 567/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:51.955040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98aa85-5b39-4db5-b1ea-a136d165fa8f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:52.051856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98aa85-5b39-4db5-b1ea-a136d165fa8f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:52.085156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b359b-069e-4cf5-ae71-6b2915e522dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:52.085698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b359b-069e-4cf5-ae71-6b2915e522dd None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:52.085987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b359b-069e-4cf5-ae71-6b2915e522dd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:52.086267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b359b-069e-4cf5-ae71-6b2915e522dd None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:52.086921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b359b-069e-4cf5-ae71-6b2915e522dd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:52.088349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b359b-069e-4cf5-ae71-6b2915e522dd None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:52.114593 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 567/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:34:51 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:34:54.345549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad4182-078a-4928-8864-0046192100ad None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:54.346128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad4182-078a-4928-8864-0046192100ad None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:54.346128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad4182-078a-4928-8864-0046192100ad None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:54.347430 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 568/1135] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:54.352527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552e4f8-767e-4c52-b9e4-06abf3f1371d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:54.353002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552e4f8-767e-4c52-b9e4-06abf3f1371d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:54.353137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552e4f8-767e-4c52-b9e4-06abf3f1371d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:54.370988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8552e4f8-767e-4c52-b9e4-06abf3f1371d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:54.372185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8552e4f8-767e-4c52-b9e4-06abf3f1371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:54.395520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:54.439139 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 568/1136] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:54.770152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad4182-078a-4928-8864-0046192100ad None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:54.868163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad4182-078a-4928-8864-0046192100ad None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:54.902279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a75043-371b-40f3-ae9d-7dbac86b0a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:54.902898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a75043-371b-40f3-ae9d-7dbac86b0a7b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:54.903181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a75043-371b-40f3-ae9d-7dbac86b0a7b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:54.903459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a75043-371b-40f3-ae9d-7dbac86b0a7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:54.904560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a75043-371b-40f3-ae9d-7dbac86b0a7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:54.906742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a75043-371b-40f3-ae9d-7dbac86b0a7b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:54.908297 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 569/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:34:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:54.914088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552e4f8-767e-4c52-b9e4-06abf3f1371d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:55.010027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552e4f8-767e-4c52-b9e4-06abf3f1371d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:55.042866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8e860-c1f8-4e04-a203-284c53e54b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:55.043407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8e860-c1f8-4e04-a203-284c53e54b52 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:55.043694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8e860-c1f8-4e04-a203-284c53e54b52 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:55.043970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8e860-c1f8-4e04-a203-284c53e54b52 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:55.044623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8e860-c1f8-4e04-a203-284c53e54b52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:55.046069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8e860-c1f8-4e04-a203-284c53e54b52 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:55.067142 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 569/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:34:54 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:34:57.293491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487f2ed-68d3-4638-985f-05727428dca5 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:57.294037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487f2ed-68d3-4638-985f-05727428dca5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:57.294037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487f2ed-68d3-4638-985f-05727428dca5 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:57.295366 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 570/1139] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:34:57.300577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6294e-d89e-4edc-b02f-f517ae674a7b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:57.300988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6294e-d89e-4edc-b02f-f517ae674a7b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:57.301179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6294e-d89e-4edc-b02f-f517ae674a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:57.319532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-44a6294e-d89e-4edc-b02f-f517ae674a7b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:34:57.320733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-44a6294e-d89e-4edc-b02f-f517ae674a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:34:57.330623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:57.388061 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 570/1140] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:34:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:34:57.731189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487f2ed-68d3-4638-985f-05727428dca5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:57.765649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:57.831080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487f2ed-68d3-4638-985f-05727428dca5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:57.865277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b8747-528b-47ca-960b-7206145e564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:57.865900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b8747-528b-47ca-960b-7206145e564a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:57.866185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b8747-528b-47ca-960b-7206145e564a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:57.866462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b8747-528b-47ca-960b-7206145e564a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:57.876170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b8747-528b-47ca-960b-7206145e564a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-cc7b8747-528b-47ca-960b-7206145e564a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:34:57.880965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:34:57.882204 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:34:57.883497 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 571/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:34:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:34:57.888064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6294e-d89e-4edc-b02f-f517ae674a7b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:57.983774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6294e-d89e-4edc-b02f-f517ae674a7b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:58.016643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927add06-1b23-4b4a-a33f-41fa2546e7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:58.017253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927add06-1b23-4b4a-a33f-41fa2546e7c9 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:58.017535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927add06-1b23-4b4a-a33f-41fa2546e7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:58.017810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927add06-1b23-4b4a-a33f-41fa2546e7c9 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:58.018463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927add06-1b23-4b4a-a33f-41fa2546e7c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:58.018766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927add06-1b23-4b4a-a33f-41fa2546e7c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:58.020109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927add06-1b23-4b4a-a33f-41fa2546e7c9 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:58.027740 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 571/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:34:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:58.033574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b8747-528b-47ca-960b-7206145e564a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:58.123787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:58.131484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b8747-528b-47ca-960b-7206145e564a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:58.164879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea29a8e-2e76-47ce-8291-7f299f1916b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:58.165532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea29a8e-2e76-47ce-8291-7f299f1916b3 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:58.165821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea29a8e-2e76-47ce-8291-7f299f1916b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:58.166101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea29a8e-2e76-47ce-8291-7f299f1916b3 None admin] PATH_INFO: `/v3/users/2ccc4f2a895c47c686d249a55bea05ab` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:58.178638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea29a8e-2e76-47ce-8291-7f299f1916b3 None admin] RBAC: Authorizing `identity:get_user(user_id=2ccc4f2a895c47c686d249a55bea05ab)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:58.180073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea29a8e-2e76-47ce-8291-7f299f1916b3 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:58.181705 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 572/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:34:58 2022] GET /identity/v3/users/2ccc4f2a895c47c686d249a55bea05ab => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:58.187971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927add06-1b23-4b4a-a33f-41fa2546e7c9 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:58.197074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:58.287001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927add06-1b23-4b4a-a33f-41fa2546e7c9 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:58.318955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef639936-ab3d-4d58-bb83-d4e748a2ecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:58.319550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef639936-ab3d-4d58-bb83-d4e748a2ecf3 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:58.319844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef639936-ab3d-4d58-bb83-d4e748a2ecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:58.320141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef639936-ab3d-4d58-bb83-d4e748a2ecf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:58.385613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef639936-ab3d-4d58-bb83-d4e748a2ecf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:58.387051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef639936-ab3d-4d58-bb83-d4e748a2ecf3 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:58.394379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:58.481722 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 572/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:34:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:34:58.488192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea29a8e-2e76-47ce-8291-7f299f1916b3 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:58.584979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea29a8e-2e76-47ce-8291-7f299f1916b3 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:58.617659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3b636-e65c-488b-b4c0-d44b80b1aa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:58.618274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3b636-e65c-488b-b4c0-d44b80b1aa8d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:58.618560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3b636-e65c-488b-b4c0-d44b80b1aa8d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:58.618839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3b636-e65c-488b-b4c0-d44b80b1aa8d None admin] PATH_INFO: `/v3/projects/90658151376541c88410cd2fa06c3036` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:58.619945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3b636-e65c-488b-b4c0-d44b80b1aa8d None admin] RBAC: Authorizing `identity:get_project(project_id=90658151376541c88410cd2fa06c3036)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:58.621406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3b636-e65c-488b-b4c0-d44b80b1aa8d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:58.623002 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 573/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:34:58 2022] GET /identity/v3/projects/90658151376541c88410cd2fa06c3036 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:34:58.629669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef639936-ab3d-4d58-bb83-d4e748a2ecf3 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:34:58.724447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef639936-ab3d-4d58-bb83-d4e748a2ecf3 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:34:58.731611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:34:58.758617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ef9f7-d2a3-4609-88b1-301e463de77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:34:58.759208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ef9f7-d2a3-4609-88b1-301e463de77c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:34:58.759491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ef9f7-d2a3-4609-88b1-301e463de77c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:34:58.759766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ef9f7-d2a3-4609-88b1-301e463de77c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:34:58.760476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ef9f7-d2a3-4609-88b1-301e463de77c None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=90658151376541c88410cd2fa06c3036, user.id=2ccc4f2a895c47c686d249a55bea05ab) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:34:58.760789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ef9f7-d2a3-4609-88b1-301e463de77c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:34:58.762231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ef9f7-d2a3-4609-88b1-301e463de77c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:34:58.767399 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 573/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:34:58 2022] GET /identity/v3/role_assignments?user.id=2ccc4f2a895c47c686d249a55bea05ab&scope.project.id=90658151376541c88410cd2fa06c3036&role.id=bc768522dc984239be4a1b39a45ab437 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:00.930516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bc379-9154-4d50-96a3-461023e40b72 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:00.931091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bc379-9154-4d50-96a3-461023e40b72 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:00.931091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bc379-9154-4d50-96a3-461023e40b72 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:00.932383 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 574/1147] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:35:00.937588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee0307-e524-4044-a489-80c3aa80f164 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:00.937986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee0307-e524-4044-a489-80c3aa80f164 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:00.938147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee0307-e524-4044-a489-80c3aa80f164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:00.956550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7aee0307-e524-4044-a489-80c3aa80f164 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:35:00.957770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7aee0307-e524-4044-a489-80c3aa80f164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:35:01.022420 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 574/1148] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:35:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:35:01.353637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bc379-9154-4d50-96a3-461023e40b72 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:01.375756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:01.451954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bc379-9154-4d50-96a3-461023e40b72 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:01.484633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc91b3d-f4cb-4fc7-9691-0c9dd83e712e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:01.485275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc91b3d-f4cb-4fc7-9691-0c9dd83e712e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:01.485560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc91b3d-f4cb-4fc7-9691-0c9dd83e712e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:01.485834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc91b3d-f4cb-4fc7-9691-0c9dd83e712e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:01.495535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc91b3d-f4cb-4fc7-9691-0c9dd83e712e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-ccc91b3d-f4cb-4fc7-9691-0c9dd83e712e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:35:01.500936 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:35:01.502023 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:35:01.503114 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 575/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:35:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:35:01.507980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee0307-e524-4044-a489-80c3aa80f164 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:01.601544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:01.604517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee0307-e524-4044-a489-80c3aa80f164 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:01.638511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf71101-2c2c-42b9-94a1-cb7fb5f18b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:01.639104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf71101-2c2c-42b9-94a1-cb7fb5f18b5f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:01.639389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf71101-2c2c-42b9-94a1-cb7fb5f18b5f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:01.639664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf71101-2c2c-42b9-94a1-cb7fb5f18b5f None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:01.640352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf71101-2c2c-42b9-94a1-cb7fb5f18b5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:35:01.640659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf71101-2c2c-42b9-94a1-cb7fb5f18b5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:01.642055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf71101-2c2c-42b9-94a1-cb7fb5f18b5f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:01.649931 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 575/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:35:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:01.655634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc91b3d-f4cb-4fc7-9691-0c9dd83e712e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:01.734308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:01.753441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc91b3d-f4cb-4fc7-9691-0c9dd83e712e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:01.785974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6448de0-d55a-4c23-8a54-b459f3c3c7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:01.786601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6448de0-d55a-4c23-8a54-b459f3c3c7f7 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:01.786866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6448de0-d55a-4c23-8a54-b459f3c3c7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:01.787137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6448de0-d55a-4c23-8a54-b459f3c3c7f7 None admin] PATH_INFO: `/v3/users/2ccc4f2a895c47c686d249a55bea05ab` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:01.788241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6448de0-d55a-4c23-8a54-b459f3c3c7f7 None admin] RBAC: Authorizing `identity:get_user(user_id=2ccc4f2a895c47c686d249a55bea05ab)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:01.789690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6448de0-d55a-4c23-8a54-b459f3c3c7f7 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:01.791225 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 576/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:35:01 2022] GET /identity/v3/users/2ccc4f2a895c47c686d249a55bea05ab => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:01.797552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf71101-2c2c-42b9-94a1-cb7fb5f18b5f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:01.893254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf71101-2c2c-42b9-94a1-cb7fb5f18b5f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:01.925936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbdd60-f262-459a-96ab-8ac94abd2e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:01.926552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbdd60-f262-459a-96ab-8ac94abd2e7c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:01.926833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbdd60-f262-459a-96ab-8ac94abd2e7c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:01.927107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbdd60-f262-459a-96ab-8ac94abd2e7c None admin] PATH_INFO: `/v3/projects/90658151376541c88410cd2fa06c3036` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:01.928215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbdd60-f262-459a-96ab-8ac94abd2e7c None admin] RBAC: Authorizing `identity:get_project(project_id=90658151376541c88410cd2fa06c3036)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:01.929655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbdd60-f262-459a-96ab-8ac94abd2e7c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:01.931243 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 576/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:35:01 2022] GET /identity/v3/projects/90658151376541c88410cd2fa06c3036 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:01.937495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6448de0-d55a-4c23-8a54-b459f3c3c7f7 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:02.032787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6448de0-d55a-4c23-8a54-b459f3c3c7f7 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:02.065150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa0c0e-c1d8-4d7d-b4d0-445a3e1de9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:02.065778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa0c0e-c1d8-4d7d-b4d0-445a3e1de9cb None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:02.066058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa0c0e-c1d8-4d7d-b4d0-445a3e1de9cb None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:02.066331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa0c0e-c1d8-4d7d-b4d0-445a3e1de9cb None admin] PATH_INFO: `/v3/projects/90658151376541c88410cd2fa06c3036/users/2ccc4f2a895c47c686d249a55bea05ab/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:02.068201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa0c0e-c1d8-4d7d-b4d0-445a3e1de9cb None admin] RBAC: Authorizing `identity:create_grant(project_id=90658151376541c88410cd2fa06c3036, user_id=2ccc4f2a895c47c686d249a55bea05ab, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:02.069645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa0c0e-c1d8-4d7d-b4d0-445a3e1de9cb None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:02.082305 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 577/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:35:01 2022] PUT /identity/v3/projects/90658151376541c88410cd2fa06c3036/users/2ccc4f2a895c47c686d249a55bea05ab/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:35:04.224877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e473cf-3af3-4168-ab56-623173647388 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:04.225452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e473cf-3af3-4168-ab56-623173647388 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:04.225452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e473cf-3af3-4168-ab56-623173647388 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:04.226773 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 577/1154] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:35:04.231924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce20f0-7dd6-4d14-96fe-b38d510237bd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:04.232362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce20f0-7dd6-4d14-96fe-b38d510237bd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:04.232488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce20f0-7dd6-4d14-96fe-b38d510237bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:04.250821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-75ce20f0-7dd6-4d14-96fe-b38d510237bd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:35:04.252010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-75ce20f0-7dd6-4d14-96fe-b38d510237bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:35:04.253976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:04.321237 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 578/1155] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:35:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:35:04.658991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e473cf-3af3-4168-ab56-623173647388 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:04.690653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:04.760359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e473cf-3af3-4168-ab56-623173647388 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:04.793466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c39dcc-a7d7-4717-807a-bc343a67f8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:04.794091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c39dcc-a7d7-4717-807a-bc343a67f8a0 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:04.794377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c39dcc-a7d7-4717-807a-bc343a67f8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:04.794657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c39dcc-a7d7-4717-807a-bc343a67f8a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:04.805092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c39dcc-a7d7-4717-807a-bc343a67f8a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-87c39dcc-a7d7-4717-807a-bc343a67f8a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:35:04.809902 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:35:04.811084 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:35:04.812700 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 578/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:35:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:35:04.817753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce20f0-7dd6-4d14-96fe-b38d510237bd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:04.913371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce20f0-7dd6-4d14-96fe-b38d510237bd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:04.946790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8062caf-4fd5-4778-a43f-f5abefb0e2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:04.947388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8062caf-4fd5-4778-a43f-f5abefb0e2f9 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:04.947673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8062caf-4fd5-4778-a43f-f5abefb0e2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:04.947951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8062caf-4fd5-4778-a43f-f5abefb0e2f9 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:04.948616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8062caf-4fd5-4778-a43f-f5abefb0e2f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:35:04.948960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8062caf-4fd5-4778-a43f-f5abefb0e2f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:04.950326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8062caf-4fd5-4778-a43f-f5abefb0e2f9 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:04.955760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:04.959539 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 579/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:35:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:04.965983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c39dcc-a7d7-4717-807a-bc343a67f8a0 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:05.055642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:05.065290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c39dcc-a7d7-4717-807a-bc343a67f8a0 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:05.098129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60a402-d751-4aaa-9655-3de57a7d8bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:05.098752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60a402-d751-4aaa-9655-3de57a7d8bce None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:05.099041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60a402-d751-4aaa-9655-3de57a7d8bce None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:05.099317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60a402-d751-4aaa-9655-3de57a7d8bce None admin] PATH_INFO: `/v3/users/2ccc4f2a895c47c686d249a55bea05ab` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:05.100428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60a402-d751-4aaa-9655-3de57a7d8bce None admin] RBAC: Authorizing `identity:get_user(user_id=2ccc4f2a895c47c686d249a55bea05ab)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:05.101883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60a402-d751-4aaa-9655-3de57a7d8bce None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:05.103443 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 579/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:35:04 2022] GET /identity/v3/users/2ccc4f2a895c47c686d249a55bea05ab => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:05.109969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8062caf-4fd5-4778-a43f-f5abefb0e2f9 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:05.206673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8062caf-4fd5-4778-a43f-f5abefb0e2f9 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:05.239139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9224205-5fa2-4f83-b819-bfeb48925195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:05.239739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9224205-5fa2-4f83-b819-bfeb48925195 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:05.240021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9224205-5fa2-4f83-b819-bfeb48925195 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:05.240296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9224205-5fa2-4f83-b819-bfeb48925195 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:05.303913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:05.306490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9224205-5fa2-4f83-b819-bfeb48925195 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:05.308291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9224205-5fa2-4f83-b819-bfeb48925195 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:05.403059 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 580/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:35:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:35:05.409089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60a402-d751-4aaa-9655-3de57a7d8bce None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:05.506965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60a402-d751-4aaa-9655-3de57a7d8bce None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:05.539307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b23b5-d128-4ca0-823a-97f6cb46e5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:05.540011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b23b5-d128-4ca0-823a-97f6cb46e5fd None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:05.540306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b23b5-d128-4ca0-823a-97f6cb46e5fd None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:05.540586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b23b5-d128-4ca0-823a-97f6cb46e5fd None admin] PATH_INFO: `/v3/projects/90658151376541c88410cd2fa06c3036` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:05.541737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b23b5-d128-4ca0-823a-97f6cb46e5fd None admin] RBAC: Authorizing `identity:get_project(project_id=90658151376541c88410cd2fa06c3036)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:05.543169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b23b5-d128-4ca0-823a-97f6cb46e5fd None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:05.544776 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 580/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:35:05 2022] GET /identity/v3/projects/90658151376541c88410cd2fa06c3036 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:05.551275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9224205-5fa2-4f83-b819-bfeb48925195 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:05.642796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:05.645869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9224205-5fa2-4f83-b819-bfeb48925195 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:05.679950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acd365-2fb0-4571-a757-834a2437f4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:05.680544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acd365-2fb0-4571-a757-834a2437f4a9 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:05.680831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acd365-2fb0-4571-a757-834a2437f4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:05.681162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acd365-2fb0-4571-a757-834a2437f4a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:05.681858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acd365-2fb0-4571-a757-834a2437f4a9 None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=90658151376541c88410cd2fa06c3036, user.id=2ccc4f2a895c47c686d249a55bea05ab) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:35:05.682169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acd365-2fb0-4571-a757-834a2437f4a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:05.683590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acd365-2fb0-4571-a757-834a2437f4a9 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:05.689162 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 581/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:35:05 2022] GET /identity/v3/role_assignments?user.id=2ccc4f2a895c47c686d249a55bea05ab&scope.project.id=90658151376541c88410cd2fa06c3036&role.id=bc768522dc984239be4a1b39a45ab437 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:07.880683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4bb99-4448-4393-af2f-b0583e6c2ec9 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:07.881272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4bb99-4448-4393-af2f-b0583e6c2ec9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:07.881272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4bb99-4448-4393-af2f-b0583e6c2ec9 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:07.882621 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 581/1162] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:35:07.887903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f55c5-64f2-4289-92e1-dec1c57f7518 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:07.888298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f55c5-64f2-4289-92e1-dec1c57f7518 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:07.888489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f55c5-64f2-4289-92e1-dec1c57f7518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:07.906851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-395f55c5-64f2-4289-92e1-dec1c57f7518 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:35:07.908051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-395f55c5-64f2-4289-92e1-dec1c57f7518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:35:07.972590 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 582/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:35:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:35:08.301941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4bb99-4448-4393-af2f-b0583e6c2ec9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:08.330846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:08.403781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4bb99-4448-4393-af2f-b0583e6c2ec9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:08.435887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f8ca2-b3ac-44f3-83b7-795339eaa86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:08.436485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f8ca2-b3ac-44f3-83b7-795339eaa86b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:08.437320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f8ca2-b3ac-44f3-83b7-795339eaa86b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:08.437600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f8ca2-b3ac-44f3-83b7-795339eaa86b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:08.502038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f8ca2-b3ac-44f3-83b7-795339eaa86b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:08.503472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f8ca2-b3ac-44f3-83b7-795339eaa86b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:08.528159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:08.598792 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 582/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:35:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:35:08.604787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f55c5-64f2-4289-92e1-dec1c57f7518 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:08.701757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:08.709232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f55c5-64f2-4289-92e1-dec1c57f7518 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:08.741050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5eaca-b761-4ea2-abf8-899189762652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:08.741671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5eaca-b761-4ea2-abf8-899189762652 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:08.741953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5eaca-b761-4ea2-abf8-899189762652 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:08.742227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5eaca-b761-4ea2-abf8-899189762652 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:08.748708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5eaca-b761-4ea2-abf8-899189762652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:08.750665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5eaca-b761-4ea2-abf8-899189762652 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-52b5eaca-b761-4ea2-abf8-899189762652 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:35:08.762297 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:35:08.763500 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:35:08.764527 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 583/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 3 20:35:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:35:08.769872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f8ca2-b3ac-44f3-83b7-795339eaa86b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:08.864780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f8ca2-b3ac-44f3-83b7-795339eaa86b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:08.890284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:08.898599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78161347-cd05-4e77-968e-dab9cc1b511f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:08.899196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78161347-cd05-4e77-968e-dab9cc1b511f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:08.899479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78161347-cd05-4e77-968e-dab9cc1b511f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:08.899794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78161347-cd05-4e77-968e-dab9cc1b511f None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:08.900379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78161347-cd05-4e77-968e-dab9cc1b511f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:35:08.900682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78161347-cd05-4e77-968e-dab9cc1b511f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:08.902069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78161347-cd05-4e77-968e-dab9cc1b511f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:08.908957 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 583/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 3 20:35:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:11.082566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ba89d-dd58-4dea-8507-d6ff4970603a None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:11.083362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ba89d-dd58-4dea-8507-d6ff4970603a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:11.083362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ba89d-dd58-4dea-8507-d6ff4970603a None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:11.084714 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 584/1167] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:35:11.089799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e0369-4b88-45d2-9234-b48f91a9d275 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:11.090229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e0369-4b88-45d2-9234-b48f91a9d275 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:11.090366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e0369-4b88-45d2-9234-b48f91a9d275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:11.108142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7c8e0369-4b88-45d2-9234-b48f91a9d275 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:35:11.109892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7c8e0369-4b88-45d2-9234-b48f91a9d275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:35:11.175041 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 584/1168] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:35:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:35:11.505144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ba89d-dd58-4dea-8507-d6ff4970603a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:11.607663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ba89d-dd58-4dea-8507-d6ff4970603a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:11.640590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c9a5e-8ecb-40b6-9013-56bcdc34c545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:11.641204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c9a5e-8ecb-40b6-9013-56bcdc34c545 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:11.641486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c9a5e-8ecb-40b6-9013-56bcdc34c545 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:11.641756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c9a5e-8ecb-40b6-9013-56bcdc34c545 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:11.642418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c9a5e-8ecb-40b6-9013-56bcdc34c545 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:11.643775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c9a5e-8ecb-40b6-9013-56bcdc34c545 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:11.658915 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 585/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Sat Sep 3 20:35:11 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:35:13.863515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b21a0-ebc6-43f0-805f-1ec591f007dc None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:13.864038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b21a0-ebc6-43f0-805f-1ec591f007dc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:13.864130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b21a0-ebc6-43f0-805f-1ec591f007dc None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:13.865458 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 585/1170] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:35:13.870802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49e50e-ff5c-4600-a287-0cbde7c47496 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:13.871178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49e50e-ff5c-4600-a287-0cbde7c47496 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:13.871361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49e50e-ff5c-4600-a287-0cbde7c47496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:13.889022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8b49e50e-ff5c-4600-a287-0cbde7c47496 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:35:13.890228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8b49e50e-ff5c-4600-a287-0cbde7c47496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:35:13.899052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:13.955464 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 586/1171] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:35:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:35:14.284729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b21a0-ebc6-43f0-805f-1ec591f007dc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:14.386125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b21a0-ebc6-43f0-805f-1ec591f007dc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:14.409061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:14.423198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c013e-ea63-48c0-a567-37cfc3bd9446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:14.423828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c013e-ea63-48c0-a567-37cfc3bd9446 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:14.424111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c013e-ea63-48c0-a567-37cfc3bd9446 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:14.424387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c013e-ea63-48c0-a567-37cfc3bd9446 None admin] PATH_INFO: `/v3/domains/dd5df7dde3424b6087474f925072227e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:14.434722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c013e-ea63-48c0-a567-37cfc3bd9446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd5df7dde3424b6087474f925072227e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:14.436155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c013e-ea63-48c0-a567-37cfc3bd9446 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:14.437749 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 586/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Sat Sep 3 20:35:14 2022] GET /identity/v3/domains/dd5df7dde3424b6087474f925072227e => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:14.443817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49e50e-ff5c-4600-a287-0cbde7c47496 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:14.540177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49e50e-ff5c-4600-a287-0cbde7c47496 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:14.572650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c99bf7-7bfc-495f-bd34-812bd0426a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:14.573276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c99bf7-7bfc-495f-bd34-812bd0426a6f None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:14.573566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c99bf7-7bfc-495f-bd34-812bd0426a6f None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:14.573845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c99bf7-7bfc-495f-bd34-812bd0426a6f None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:14.574484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c99bf7-7bfc-495f-bd34-812bd0426a6f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:14.575913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c99bf7-7bfc-495f-bd34-812bd0426a6f None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:14.591270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:14.603927 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 587/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:35:14 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:35:16.812924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549552b9-e837-4d0a-b7e2-47373772112c None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:16.813488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549552b9-e837-4d0a-b7e2-47373772112c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:16.813488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549552b9-e837-4d0a-b7e2-47373772112c None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:16.814813 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 587/1174] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:35:16.820103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a12c8b-ade6-4efc-8476-478e0b6bb25f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:16.820494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a12c8b-ade6-4efc-8476-478e0b6bb25f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:16.820675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a12c8b-ade6-4efc-8476-478e0b6bb25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:16.839505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a9a12c8b-ade6-4efc-8476-478e0b6bb25f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:35:16.840704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a9a12c8b-ade6-4efc-8476-478e0b6bb25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:35:16.905623 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 588/1175] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:35:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:35:17.234212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549552b9-e837-4d0a-b7e2-47373772112c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:17.335497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549552b9-e837-4d0a-b7e2-47373772112c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:17.368245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4042f0-2b6f-4d54-9c77-d82f76eefdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:17.368865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4042f0-2b6f-4d54-9c77-d82f76eefdff None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:17.369185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4042f0-2b6f-4d54-9c77-d82f76eefdff None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:17.369463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4042f0-2b6f-4d54-9c77-d82f76eefdff None admin] PATH_INFO: `/v3/domains/dd5df7dde3424b6087474f925072227e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:17.370582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4042f0-2b6f-4d54-9c77-d82f76eefdff None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd5df7dde3424b6087474f925072227e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:17.372029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4042f0-2b6f-4d54-9c77-d82f76eefdff None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:17.373602 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 588/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Sat Sep 3 20:35:17 2022] GET /identity/v3/domains/dd5df7dde3424b6087474f925072227e => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:17.380058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a12c8b-ade6-4efc-8476-478e0b6bb25f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:17.476154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a12c8b-ade6-4efc-8476-478e0b6bb25f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:17.501138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:17.510224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893035f-3988-49fe-89c6-1e0219e96fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:17.510822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893035f-3988-49fe-89c6-1e0219e96fc5 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:17.511106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893035f-3988-49fe-89c6-1e0219e96fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:17.511381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893035f-3988-49fe-89c6-1e0219e96fc5 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:17.512029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893035f-3988-49fe-89c6-1e0219e96fc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:17.513484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893035f-3988-49fe-89c6-1e0219e96fc5 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:17.534312 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 589/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:35:17 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:35:19.749218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a403c7-7332-42ec-9fd5-9461eec8bab2 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:19.749763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a403c7-7332-42ec-9fd5-9461eec8bab2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:19.749763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a403c7-7332-42ec-9fd5-9461eec8bab2 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:19.751089 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 589/1178] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:35:19.756252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32cafc-ad39-438c-85c8-8d15b4cb4783 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:19.756696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32cafc-ad39-438c-85c8-8d15b4cb4783 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:19.756798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32cafc-ad39-438c-85c8-8d15b4cb4783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:19.774634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bf32cafc-ad39-438c-85c8-8d15b4cb4783 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:35:19.784409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bf32cafc-ad39-438c-85c8-8d15b4cb4783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:35:19.851044 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 590/1179] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:35:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:35:20.179824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a403c7-7332-42ec-9fd5-9461eec8bab2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:20.229424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:20.278992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a403c7-7332-42ec-9fd5-9461eec8bab2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:20.311423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40daf65-aec0-4109-ad39-90459693e445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:20.312036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40daf65-aec0-4109-ad39-90459693e445 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:20.312320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40daf65-aec0-4109-ad39-90459693e445 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:20.312597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40daf65-aec0-4109-ad39-90459693e445 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:20.322275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40daf65-aec0-4109-ad39-90459693e445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-a40daf65-aec0-4109-ad39-90459693e445 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:35:20.327508 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:35:20.328661 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:35:20.329707 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 590/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:35:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:35:20.334868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32cafc-ad39-438c-85c8-8d15b4cb4783 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:20.432682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32cafc-ad39-438c-85c8-8d15b4cb4783 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:20.445036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:20.467155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa8cb3-e233-4329-9814-6fb81fcb3fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:20.467810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa8cb3-e233-4329-9814-6fb81fcb3fa7 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:20.468092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa8cb3-e233-4329-9814-6fb81fcb3fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:20.468363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa8cb3-e233-4329-9814-6fb81fcb3fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:20.469578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa8cb3-e233-4329-9814-6fb81fcb3fa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:35:20.469883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa8cb3-e233-4329-9814-6fb81fcb3fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:20.471242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa8cb3-e233-4329-9814-6fb81fcb3fa7 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:20.479070 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 591/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:35:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:20.485023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40daf65-aec0-4109-ad39-90459693e445 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:20.582584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40daf65-aec0-4109-ad39-90459693e445 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:20.591688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:20.615741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f67b7-74f1-4a76-954e-d0ebbf5c0134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:20.616363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f67b7-74f1-4a76-954e-d0ebbf5c0134 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:20.616648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f67b7-74f1-4a76-954e-d0ebbf5c0134 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:20.616949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f67b7-74f1-4a76-954e-d0ebbf5c0134 None admin] PATH_INFO: `/v3/users/addc8db13c3d4178a5d0c06f1d285cef` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:20.629950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f67b7-74f1-4a76-954e-d0ebbf5c0134 None admin] RBAC: Authorizing `identity:get_user(user_id=addc8db13c3d4178a5d0c06f1d285cef)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:20.631388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f67b7-74f1-4a76-954e-d0ebbf5c0134 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:20.632989 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 591/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:35:20 2022] GET /identity/v3/users/addc8db13c3d4178a5d0c06f1d285cef => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:20.638844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa8cb3-e233-4329-9814-6fb81fcb3fa7 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:20.736056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa8cb3-e233-4329-9814-6fb81fcb3fa7 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:20.769184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd3758-d0d0-4229-80c4-7d976dfd0a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:20.769783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd3758-d0d0-4229-80c4-7d976dfd0a6b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:20.770071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd3758-d0d0-4229-80c4-7d976dfd0a6b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:20.770349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd3758-d0d0-4229-80c4-7d976dfd0a6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:20.810497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:20.836287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd3758-d0d0-4229-80c4-7d976dfd0a6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:20.837751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd3758-d0d0-4229-80c4-7d976dfd0a6b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:20.931304 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 592/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:35:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:35:20.937776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f67b7-74f1-4a76-954e-d0ebbf5c0134 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:21.035011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f67b7-74f1-4a76-954e-d0ebbf5c0134 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:21.068412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95acb491-447a-4c1b-99fb-cdbe49e284c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:21.069063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acb491-447a-4c1b-99fb-cdbe49e284c6 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:21.069359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acb491-447a-4c1b-99fb-cdbe49e284c6 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:21.069641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acb491-447a-4c1b-99fb-cdbe49e284c6 None admin] PATH_INFO: `/v3/projects/f5b7ad87389843b99f96b294e9bef2a7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:21.070758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95acb491-447a-4c1b-99fb-cdbe49e284c6 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b7ad87389843b99f96b294e9bef2a7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:21.072174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95acb491-447a-4c1b-99fb-cdbe49e284c6 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:21.073804 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 592/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:35:20 2022] GET /identity/v3/projects/f5b7ad87389843b99f96b294e9bef2a7 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:21.079855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd3758-d0d0-4229-80c4-7d976dfd0a6b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:21.154245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:21.179293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd3758-d0d0-4229-80c4-7d976dfd0a6b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:21.212562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29d347-0efd-435b-94ae-84ad3e5641db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:21.213196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29d347-0efd-435b-94ae-84ad3e5641db None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:21.213482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29d347-0efd-435b-94ae-84ad3e5641db None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:21.213756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29d347-0efd-435b-94ae-84ad3e5641db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:21.214475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29d347-0efd-435b-94ae-84ad3e5641db None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=f5b7ad87389843b99f96b294e9bef2a7, user.id=addc8db13c3d4178a5d0c06f1d285cef) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:35:21.214784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29d347-0efd-435b-94ae-84ad3e5641db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:21.216202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29d347-0efd-435b-94ae-84ad3e5641db None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:21.221964 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 593/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:35:21 2022] GET /identity/v3/role_assignments?user.id=addc8db13c3d4178a5d0c06f1d285cef&scope.project.id=f5b7ad87389843b99f96b294e9bef2a7&role.id=bc768522dc984239be4a1b39a45ab437 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:23.376416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e39be-8dcd-450b-afaf-7ab00d315e1f None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:23.376961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e39be-8dcd-450b-afaf-7ab00d315e1f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:23.377092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e39be-8dcd-450b-afaf-7ab00d315e1f None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:23.378330 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 593/1186] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:35:23.383608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7782480-ba43-41ed-9b98-c4f86c256a79 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:23.383997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7782480-ba43-41ed-9b98-c4f86c256a79 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:23.384123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7782480-ba43-41ed-9b98-c4f86c256a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:23.401895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c7782480-ba43-41ed-9b98-c4f86c256a79 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:35:23.403092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c7782480-ba43-41ed-9b98-c4f86c256a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:35:23.470098 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 594/1187] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:35:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:35:23.803830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e39be-8dcd-450b-afaf-7ab00d315e1f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:23.837963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:23.903467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e39be-8dcd-450b-afaf-7ab00d315e1f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:23.936325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192421e4-d9ab-4610-a0a1-a160f9ab1b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:23.936960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192421e4-d9ab-4610-a0a1-a160f9ab1b3f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:23.937245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192421e4-d9ab-4610-a0a1-a160f9ab1b3f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:23.937516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192421e4-d9ab-4610-a0a1-a160f9ab1b3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:23.947138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192421e4-d9ab-4610-a0a1-a160f9ab1b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-192421e4-d9ab-4610-a0a1-a160f9ab1b3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:35:23.952321 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:35:23.953484 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:35:23.954386 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 594/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:35:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:35:23.959706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7782480-ba43-41ed-9b98-c4f86c256a79 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:24.023292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:24.055505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7782480-ba43-41ed-9b98-c4f86c256a79 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:24.089222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6ef22-4d96-4742-a91a-95786c3386bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:24.089824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6ef22-4d96-4742-a91a-95786c3386bd None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:24.090109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6ef22-4d96-4742-a91a-95786c3386bd None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:24.090383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6ef22-4d96-4742-a91a-95786c3386bd None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:24.091041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6ef22-4d96-4742-a91a-95786c3386bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:35:24.091347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6ef22-4d96-4742-a91a-95786c3386bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:24.093278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6ef22-4d96-4742-a91a-95786c3386bd None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:24.101296 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 595/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:35:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:24.107128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192421e4-d9ab-4610-a0a1-a160f9ab1b3f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:24.198139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:24.205817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192421e4-d9ab-4610-a0a1-a160f9ab1b3f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:24.239075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9098fc-2dfe-4976-86f8-434d10afe1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:24.239698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9098fc-2dfe-4976-86f8-434d10afe1ee None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:24.239983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9098fc-2dfe-4976-86f8-434d10afe1ee None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:24.240288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9098fc-2dfe-4976-86f8-434d10afe1ee None admin] PATH_INFO: `/v3/users/addc8db13c3d4178a5d0c06f1d285cef` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:24.241432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9098fc-2dfe-4976-86f8-434d10afe1ee None admin] RBAC: Authorizing `identity:get_user(user_id=addc8db13c3d4178a5d0c06f1d285cef)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:24.242869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9098fc-2dfe-4976-86f8-434d10afe1ee None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:24.244432 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 595/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:35:24 2022] GET /identity/v3/users/addc8db13c3d4178a5d0c06f1d285cef => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:24.250664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6ef22-4d96-4742-a91a-95786c3386bd None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:24.347053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6ef22-4d96-4742-a91a-95786c3386bd None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:24.373558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:24.382275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fd623-ad60-47d2-9cf1-936747912bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:24.383316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fd623-ad60-47d2-9cf1-936747912bb1 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:24.383683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fd623-ad60-47d2-9cf1-936747912bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:24.384234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fd623-ad60-47d2-9cf1-936747912bb1 None admin] PATH_INFO: `/v3/projects/f5b7ad87389843b99f96b294e9bef2a7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:24.385131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60fd623-ad60-47d2-9cf1-936747912bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b7ad87389843b99f96b294e9bef2a7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:24.387462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60fd623-ad60-47d2-9cf1-936747912bb1 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:24.391043 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 596/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:35:24 2022] GET /identity/v3/projects/f5b7ad87389843b99f96b294e9bef2a7 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:24.397228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9098fc-2dfe-4976-86f8-434d10afe1ee None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:24.499867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9098fc-2dfe-4976-86f8-434d10afe1ee None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:24.533524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26605a13-667c-4b89-9eeb-c0154f8dc0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:24.534191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26605a13-667c-4b89-9eeb-c0154f8dc0ef None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:24.534477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26605a13-667c-4b89-9eeb-c0154f8dc0ef None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:24.534757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26605a13-667c-4b89-9eeb-c0154f8dc0ef None admin] PATH_INFO: `/v3/projects/f5b7ad87389843b99f96b294e9bef2a7/users/addc8db13c3d4178a5d0c06f1d285cef/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:24.536655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26605a13-667c-4b89-9eeb-c0154f8dc0ef None admin] RBAC: Authorizing `identity:create_grant(project_id=f5b7ad87389843b99f96b294e9bef2a7, user_id=addc8db13c3d4178a5d0c06f1d285cef, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:24.538132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26605a13-667c-4b89-9eeb-c0154f8dc0ef None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:24.551506 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 596/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:35:24 2022] PUT /identity/v3/projects/f5b7ad87389843b99f96b294e9bef2a7/users/addc8db13c3d4178a5d0c06f1d285cef/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:35:26.708396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51120c5b-05b6-416a-9dfb-32e008509398 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:26.708989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51120c5b-05b6-416a-9dfb-32e008509398 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:26.708989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51120c5b-05b6-416a-9dfb-32e008509398 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:26.710335 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 597/1193] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:35:26.715675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418bb06-449a-4985-a2dc-06532d625c24 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:26.716078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418bb06-449a-4985-a2dc-06532d625c24 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:26.716222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418bb06-449a-4985-a2dc-06532d625c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:26.733927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c418bb06-449a-4985-a2dc-06532d625c24 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:35:26.735106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c418bb06-449a-4985-a2dc-06532d625c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:35:26.751524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:26.799672 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 597/1194] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:35:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:35:27.128413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51120c5b-05b6-416a-9dfb-32e008509398 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:27.226655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51120c5b-05b6-416a-9dfb-32e008509398 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:27.259335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679b9db-f65c-4f5e-9c60-07c015b35c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:27.259961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679b9db-f65c-4f5e-9c60-07c015b35c61 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:27.260248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679b9db-f65c-4f5e-9c60-07c015b35c61 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:27.260529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679b9db-f65c-4f5e-9c60-07c015b35c61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:27.270353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679b9db-f65c-4f5e-9c60-07c015b35c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-4679b9db-f65c-4f5e-9c60-07c015b35c61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:35:27.275167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:35:27.276264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:35:27.277331 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 598/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:35:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:35:27.282768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418bb06-449a-4985-a2dc-06532d625c24 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:27.378526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418bb06-449a-4985-a2dc-06532d625c24 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:27.411179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9ebe1-be15-4470-89e6-bb6cfc116170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:27.411766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9ebe1-be15-4470-89e6-bb6cfc116170 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:27.412047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9ebe1-be15-4470-89e6-bb6cfc116170 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:27.412325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9ebe1-be15-4470-89e6-bb6cfc116170 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:27.413008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9ebe1-be15-4470-89e6-bb6cfc116170 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:35:27.413311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9ebe1-be15-4470-89e6-bb6cfc116170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:27.414657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9ebe1-be15-4470-89e6-bb6cfc116170 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:27.422320 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 598/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:35:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:27.428653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679b9db-f65c-4f5e-9c60-07c015b35c61 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:27.486957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:27.527855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679b9db-f65c-4f5e-9c60-07c015b35c61 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:27.560070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb342d3-d97c-4fbd-89e2-51520f32aa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:27.560692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb342d3-d97c-4fbd-89e2-51520f32aa77 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:27.561492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb342d3-d97c-4fbd-89e2-51520f32aa77 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:27.561782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb342d3-d97c-4fbd-89e2-51520f32aa77 None admin] PATH_INFO: `/v3/users/addc8db13c3d4178a5d0c06f1d285cef` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:27.562896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb342d3-d97c-4fbd-89e2-51520f32aa77 None admin] RBAC: Authorizing `identity:get_user(user_id=addc8db13c3d4178a5d0c06f1d285cef)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:27.564325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb342d3-d97c-4fbd-89e2-51520f32aa77 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:27.565958 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 599/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 3 20:35:27 2022] GET /identity/v3/users/addc8db13c3d4178a5d0c06f1d285cef => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:27.571929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9ebe1-be15-4470-89e6-bb6cfc116170 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:27.588339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:27.668836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9ebe1-be15-4470-89e6-bb6cfc116170 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:27.702374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7b087-1d0b-4c28-9084-92ce0ade6c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:27.702978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7b087-1d0b-4c28-9084-92ce0ade6c6a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:27.703265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7b087-1d0b-4c28-9084-92ce0ade6c6a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:27.703544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7b087-1d0b-4c28-9084-92ce0ade6c6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:27.769833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7b087-1d0b-4c28-9084-92ce0ade6c6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:27.771270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7b087-1d0b-4c28-9084-92ce0ade6c6a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:27.785433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:27.865660 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 599/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:35:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:35:27.871582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb342d3-d97c-4fbd-89e2-51520f32aa77 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:27.968660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb342d3-d97c-4fbd-89e2-51520f32aa77 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:27.992703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:28.002208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b2a33-43cf-46f1-854d-3bae6dea8d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:28.002826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b2a33-43cf-46f1-854d-3bae6dea8d06 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:28.003107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b2a33-43cf-46f1-854d-3bae6dea8d06 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:28.003380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b2a33-43cf-46f1-854d-3bae6dea8d06 None admin] PATH_INFO: `/v3/projects/f5b7ad87389843b99f96b294e9bef2a7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:28.004473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b2a33-43cf-46f1-854d-3bae6dea8d06 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b7ad87389843b99f96b294e9bef2a7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:28.005915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b2a33-43cf-46f1-854d-3bae6dea8d06 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:28.007519 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 600/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 3 20:35:27 2022] GET /identity/v3/projects/f5b7ad87389843b99f96b294e9bef2a7 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:28.013567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7b087-1d0b-4c28-9084-92ce0ade6c6a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:35:28.129260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7b087-1d0b-4c28-9084-92ce0ade6c6a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:35:28.143385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:28.163180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc91023-6f1b-4787-adba-cc8765a9bad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:35:28.163776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc91023-6f1b-4787-adba-cc8765a9bad6 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:35:28.164057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc91023-6f1b-4787-adba-cc8765a9bad6 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:35:28.164333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc91023-6f1b-4787-adba-cc8765a9bad6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:35:28.165079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc91023-6f1b-4787-adba-cc8765a9bad6 None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=f5b7ad87389843b99f96b294e9bef2a7, user.id=addc8db13c3d4178a5d0c06f1d285cef) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:35:28.165385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc91023-6f1b-4787-adba-cc8765a9bad6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:35:28.166905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc91023-6f1b-4787-adba-cc8765a9bad6 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:35:28.173036 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 600/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:35:28 2022] GET /identity/v3/role_assignments?user.id=addc8db13c3d4178a5d0c06f1d285cef&scope.project.id=f5b7ad87389843b99f96b294e9bef2a7&role.id=bc768522dc984239be4a1b39a45ab437 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:35:37.997690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:35:38.169948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=11 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:30.306526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae34ffb-1e23-42f9-9c07-27a26fa9583e None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:30.307158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae34ffb-1e23-42f9-9c07-27a26fa9583e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:30.307158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae34ffb-1e23-42f9-9c07-27a26fa9583e None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:30.308418 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 601/1201] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:36:30.314290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77909535-3268-460f-b1fd-64fa29877f24 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:30.314741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77909535-3268-460f-b1fd-64fa29877f24 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:30.314851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77909535-3268-460f-b1fd-64fa29877f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:30.337075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-77909535-3268-460f-b1fd-64fa29877f24 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:36:30.338549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-77909535-3268-460f-b1fd-64fa29877f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:36:30.437014 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 601/1202] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:36:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:36:30.768114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae34ffb-1e23-42f9-9c07-27a26fa9583e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:30.888294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae34ffb-1e23-42f9-9c07-27a26fa9583e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:30.938439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293192d-ff5a-4d9b-939a-06836a00f673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:30.939620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293192d-ff5a-4d9b-939a-06836a00f673 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:30.940012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293192d-ff5a-4d9b-939a-06836a00f673 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:30.940215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293192d-ff5a-4d9b-939a-06836a00f673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:30.960546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293192d-ff5a-4d9b-939a-06836a00f673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:30.962377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293192d-ff5a-4d9b-939a-06836a00f673 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-2293192d-ff5a-4d9b-939a-06836a00f673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:30.980556 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:30.987311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:30.988311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:30.988311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:30.988311 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:30.988311 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 602/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:30.990139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77909535-3268-460f-b1fd-64fa29877f24 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:31.129471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77909535-3268-460f-b1fd-64fa29877f24 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:31.165961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:31.175171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b0e5c-6553-4350-8348-7febf79d259a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:31.176016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b0e5c-6553-4350-8348-7febf79d259a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:31.176455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b0e5c-6553-4350-8348-7febf79d259a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:31.176822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b0e5c-6553-4350-8348-7febf79d259a None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:31.177607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b0e5c-6553-4350-8348-7febf79d259a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:31.177983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b0e5c-6553-4350-8348-7febf79d259a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:31.179465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b0e5c-6553-4350-8348-7febf79d259a None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:31.198935 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 602/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:31.200705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293192d-ff5a-4d9b-939a-06836a00f673 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:31.268535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:31.326013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293192d-ff5a-4d9b-939a-06836a00f673 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:31.368401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b3b51-f352-45d8-9a2d-8c1d6f7ff8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:31.369417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b3b51-f352-45d8-9a2d-8c1d6f7ff8dc None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:31.369936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b3b51-f352-45d8-9a2d-8c1d6f7ff8dc None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:31.370407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b3b51-f352-45d8-9a2d-8c1d6f7ff8dc None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:31.371236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b3b51-f352-45d8-9a2d-8c1d6f7ff8dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:31.373040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b3b51-f352-45d8-9a2d-8c1d6f7ff8dc None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:31.401190 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 603/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 3 20:36:31 2022] POST /identity/v3/users => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:36:33.897481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac71e4-abe4-4fcf-8720-52f93580fcd1 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:33.898068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac71e4-abe4-4fcf-8720-52f93580fcd1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:33.898068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac71e4-abe4-4fcf-8720-52f93580fcd1 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:33.899702 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 603/1206] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:36:33.906266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d335a-acfa-48ff-95ff-3449481f8593 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:33.908745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d335a-acfa-48ff-95ff-3449481f8593 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:33.908745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d335a-acfa-48ff-95ff-3449481f8593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:33.927482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-724d335a-acfa-48ff-95ff-3449481f8593 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:36:33.928824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-724d335a-acfa-48ff-95ff-3449481f8593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:36:34.012230 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 604/1207] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:36:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:36:34.377847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac71e4-abe4-4fcf-8720-52f93580fcd1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:34.502308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac71e4-abe4-4fcf-8720-52f93580fcd1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:34.542234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5e8c0-8db3-4855-b7cb-e9ed7cd59d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:34.542926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5e8c0-8db3-4855-b7cb-e9ed7cd59d3d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:34.543206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5e8c0-8db3-4855-b7cb-e9ed7cd59d3d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:34.543478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5e8c0-8db3-4855-b7cb-e9ed7cd59d3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:34.555198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5e8c0-8db3-4855-b7cb-e9ed7cd59d3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-c4f5e8c0-8db3-4855-b7cb-e9ed7cd59d3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:36:34.561238 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:34.562217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:34.563746 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 604/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:36:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:34.569002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d335a-acfa-48ff-95ff-3449481f8593 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:34.612804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:34.677317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d335a-acfa-48ff-95ff-3449481f8593 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:34.718899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acade0a0-76f3-406f-b7cd-83a3a8e267b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:34.719517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acade0a0-76f3-406f-b7cd-83a3a8e267b9 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:34.719844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acade0a0-76f3-406f-b7cd-83a3a8e267b9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:34.720082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acade0a0-76f3-406f-b7cd-83a3a8e267b9 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:34.720769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acade0a0-76f3-406f-b7cd-83a3a8e267b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:34.721129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acade0a0-76f3-406f-b7cd-83a3a8e267b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:34.722527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acade0a0-76f3-406f-b7cd-83a3a8e267b9 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:34.731005 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 605/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:36:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:34.738009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5e8c0-8db3-4855-b7cb-e9ed7cd59d3d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:34.794446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:34.851627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5e8c0-8db3-4855-b7cb-e9ed7cd59d3d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:34.888030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abd1ed-48eb-42de-a6be-f899285d7836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:34.888672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abd1ed-48eb-42de-a6be-f899285d7836 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:34.888987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abd1ed-48eb-42de-a6be-f899285d7836 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:34.889273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abd1ed-48eb-42de-a6be-f899285d7836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:34.900924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abd1ed-48eb-42de-a6be-f899285d7836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:34.902398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abd1ed-48eb-42de-a6be-f899285d7836 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-53abd1ed-48eb-42de-a6be-f899285d7836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:34.918098 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:34.920500 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:34.921951 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:34.921951 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:34.921951 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:34.921951 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 605/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:34.925806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acade0a0-76f3-406f-b7cd-83a3a8e267b9 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:35.044453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:35.047901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acade0a0-76f3-406f-b7cd-83a3a8e267b9 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:35.100739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b6730-2457-4720-8163-f5d572409f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:35.101435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b6730-2457-4720-8163-f5d572409f5e None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:35.101730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b6730-2457-4720-8163-f5d572409f5e None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:35.102017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b6730-2457-4720-8163-f5d572409f5e None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:35.102686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b6730-2457-4720-8163-f5d572409f5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:35.102995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b6730-2457-4720-8163-f5d572409f5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:35.104708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b6730-2457-4720-8163-f5d572409f5e None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:35.119577 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 606/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:36:35.125503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abd1ed-48eb-42de-a6be-f899285d7836 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:35.241745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abd1ed-48eb-42de-a6be-f899285d7836 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:35.243691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:35.283216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410c83-af62-4930-85d8-b4f65fba1e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:35.284054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410c83-af62-4930-85d8-b4f65fba1e5e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:35.284445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410c83-af62-4930-85d8-b4f65fba1e5e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:35.284842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410c83-af62-4930-85d8-b4f65fba1e5e None admin] PATH_INFO: `/v3/users/placement` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:35.293640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53410c83-af62-4930-85d8-b4f65fba1e5e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:35.295366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53410c83-af62-4930-85d8-b4f65fba1e5e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-53410c83-af62-4930-85d8-b4f65fba1e5e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:36:35.307741 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:35.309171 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:35.310498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:35.310498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:35.310498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:36:35.310498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:36:35.310498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:36:35.310498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:36:35.310498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:36:35.310498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:36:35.310498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 20:36:35.310498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:35.310498 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 606/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:36:35.315811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b6730-2457-4720-8163-f5d572409f5e None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:35.429502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b6730-2457-4720-8163-f5d572409f5e None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:35.479485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39078d9-deef-400b-a58b-74ee4b571dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:35.480232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39078d9-deef-400b-a58b-74ee4b571dce None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:35.480462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39078d9-deef-400b-a58b-74ee4b571dce None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:35.481269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39078d9-deef-400b-a58b-74ee4b571dce None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:35.482056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39078d9-deef-400b-a58b-74ee4b571dce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:35.482665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39078d9-deef-400b-a58b-74ee4b571dce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:35.484484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39078d9-deef-400b-a58b-74ee4b571dce None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:35.519046 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 607/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Sep 3 20:36:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:35.526346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410c83-af62-4930-85d8-b4f65fba1e5e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:35.653310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410c83-af62-4930-85d8-b4f65fba1e5e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:35.693428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603ad35-bb62-42ec-8982-30296aabb507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:35.694406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603ad35-bb62-42ec-8982-30296aabb507 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:35.695079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603ad35-bb62-42ec-8982-30296aabb507 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:35.695560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603ad35-bb62-42ec-8982-30296aabb507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:35.735042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:35.779984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603ad35-bb62-42ec-8982-30296aabb507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:35.781867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603ad35-bb62-42ec-8982-30296aabb507 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:35.905907 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 607/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:36:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:36:35.907023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39078d9-deef-400b-a58b-74ee4b571dce None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:35.931886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:36.045658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39078d9-deef-400b-a58b-74ee4b571dce None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:36.085369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c35bd-4518-4f55-ac49-aff7feb3335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:36.085989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c35bd-4518-4f55-ac49-aff7feb3335d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:36.086322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c35bd-4518-4f55-ac49-aff7feb3335d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:36.086602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c35bd-4518-4f55-ac49-aff7feb3335d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:36.098945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c35bd-4518-4f55-ac49-aff7feb3335d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:36.100631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c35bd-4518-4f55-ac49-aff7feb3335d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-fe9c35bd-4518-4f55-ac49-aff7feb3335d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:36.125869 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:36.127253 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:36.128191 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:36.128191 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:36.128191 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:36.128191 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 608/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:36.133829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603ad35-bb62-42ec-8982-30296aabb507 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:36.209128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:36.254339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603ad35-bb62-42ec-8982-30296aabb507 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:36.295118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b62ad-eb22-4f43-979b-f2b0b4e4969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:36.295823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b62ad-eb22-4f43-979b-f2b0b4e4969f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:36.296182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b62ad-eb22-4f43-979b-f2b0b4e4969f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:36.296531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b62ad-eb22-4f43-979b-f2b0b4e4969f None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:36.297281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b62ad-eb22-4f43-979b-f2b0b4e4969f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:36.297664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b62ad-eb22-4f43-979b-f2b0b4e4969f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:36.299105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b62ad-eb22-4f43-979b-f2b0b4e4969f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:36.312976 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 608/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:36.319066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c35bd-4518-4f55-ac49-aff7feb3335d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:36.401409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:36.442345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c35bd-4518-4f55-ac49-aff7feb3335d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:36.497545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c409b-52db-43df-b302-bfe50abee731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:36.498380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c409b-52db-43df-b302-bfe50abee731 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:36.498770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c409b-52db-43df-b302-bfe50abee731 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:36.499416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c409b-52db-43df-b302-bfe50abee731 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:36.508353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c409b-52db-43df-b302-bfe50abee731 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:36.510995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c409b-52db-43df-b302-bfe50abee731 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-769c409b-52db-43df-b302-bfe50abee731 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:36:36.524732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:36.525886 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:36.527158 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:36:36.527158 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:36:36.527158 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:36:36.527158 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:36:36.527158 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:36:36.527158 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:36:36.527158 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:36:36.527158 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:36.527158 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 609/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:36:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:36:36.531895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b62ad-eb22-4f43-979b-f2b0b4e4969f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:36.650543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b62ad-eb22-4f43-979b-f2b0b4e4969f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:36.679340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:36.695518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986c678-2fe6-4565-9a33-06e81f1a6617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:36.695884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986c678-2fe6-4565-9a33-06e81f1a6617 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:36.695978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986c678-2fe6-4565-9a33-06e81f1a6617 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:36.696235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986c678-2fe6-4565-9a33-06e81f1a6617 None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:36.697121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986c678-2fe6-4565-9a33-06e81f1a6617 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:36.697471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986c678-2fe6-4565-9a33-06e81f1a6617 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:36.699038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986c678-2fe6-4565-9a33-06e81f1a6617 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:36.715222 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 609/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:36:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:36.721624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c409b-52db-43df-b302-bfe50abee731 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:36.859544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c409b-52db-43df-b302-bfe50abee731 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:36.889038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:36.897721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7f953-3b1b-4233-a2d9-e985b40714aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:36.898356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7f953-3b1b-4233-a2d9-e985b40714aa None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:36.898644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7f953-3b1b-4233-a2d9-e985b40714aa None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:36.898932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7f953-3b1b-4233-a2d9-e985b40714aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:36.899664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7f953-3b1b-4233-a2d9-e985b40714aa None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=6dfb57e006f344808f7a2ef41b246d5e) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:36.899975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7f953-3b1b-4233-a2d9-e985b40714aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:36.901547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7f953-3b1b-4233-a2d9-e985b40714aa None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:36.908249 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 610/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:36:36 2022] GET /identity/v3/role_assignments?user.id=6dfb57e006f344808f7a2ef41b246d5e&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:39.125190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bca1d-ef9f-469a-81a5-71a9be4201cd None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:39.125835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bca1d-ef9f-469a-81a5-71a9be4201cd None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:39.125835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bca1d-ef9f-469a-81a5-71a9be4201cd None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:39.127091 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 610/1220] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:36:39.136429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01603ade-1792-476b-822b-0741361a9765 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:39.136429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01603ade-1792-476b-822b-0741361a9765 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:39.136429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01603ade-1792-476b-822b-0741361a9765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:39.153866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-01603ade-1792-476b-822b-0741361a9765 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:36:39.155082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-01603ade-1792-476b-822b-0741361a9765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:36:39.233120 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 611/1221] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:36:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:36:39.555272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bca1d-ef9f-469a-81a5-71a9be4201cd None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:39.677209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bca1d-ef9f-469a-81a5-71a9be4201cd None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:39.727321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec939536-5058-4824-a0ad-4ce75cc16196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:39.727971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec939536-5058-4824-a0ad-4ce75cc16196 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:39.728256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec939536-5058-4824-a0ad-4ce75cc16196 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:39.728576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec939536-5058-4824-a0ad-4ce75cc16196 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:39.740170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec939536-5058-4824-a0ad-4ce75cc16196 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-ec939536-5058-4824-a0ad-4ce75cc16196 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:36:39.745146 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:39.746115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:39.747018 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:39.747018 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:39.747018 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:39.747018 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:39.747018 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:36:39.747018 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:36:39.747018 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:36:39.747018 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:36:39.747018 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:36:39.747018 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:36:39.747018 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:36:39.747018 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:39.751742 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 611/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:36:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:39.752750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01603ade-1792-476b-822b-0741361a9765 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:39.863488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01603ade-1792-476b-822b-0741361a9765 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:39.889845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:39.901815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc3195-f0cd-45f5-aa05-29ad38402d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:39.902431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3195-f0cd-45f5-aa05-29ad38402d54 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:39.902717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3195-f0cd-45f5-aa05-29ad38402d54 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:39.903537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3195-f0cd-45f5-aa05-29ad38402d54 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:39.904264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc3195-f0cd-45f5-aa05-29ad38402d54 None admin] RBAC: Adding query filter params (name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:39.904573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc3195-f0cd-45f5-aa05-29ad38402d54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:39.905980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc3195-f0cd-45f5-aa05-29ad38402d54 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:39.914331 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 612/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:36:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:39.920610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec939536-5058-4824-a0ad-4ce75cc16196 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:39.990509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:40.061043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec939536-5058-4824-a0ad-4ce75cc16196 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:40.114407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e050e11-05dd-470b-9a78-598ee28d62be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:40.115139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e050e11-05dd-470b-9a78-598ee28d62be None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:40.115554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e050e11-05dd-470b-9a78-598ee28d62be None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:40.115861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e050e11-05dd-470b-9a78-598ee28d62be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:40.129269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e050e11-05dd-470b-9a78-598ee28d62be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:40.130742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e050e11-05dd-470b-9a78-598ee28d62be None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-5e050e11-05dd-470b-9a78-598ee28d62be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:40.149099 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:40.152985 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:40.154162 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:40.154162 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:40.154162 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:40.154162 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 612/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:40.156271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc3195-f0cd-45f5-aa05-29ad38402d54 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:40.264333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc3195-f0cd-45f5-aa05-29ad38402d54 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:40.299428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16dc83-ae28-4425-b74b-42dc79ace35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:40.300039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16dc83-ae28-4425-b74b-42dc79ace35c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:40.300321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16dc83-ae28-4425-b74b-42dc79ace35c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:40.300597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16dc83-ae28-4425-b74b-42dc79ace35c None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:40.301409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16dc83-ae28-4425-b74b-42dc79ace35c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:40.301721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16dc83-ae28-4425-b74b-42dc79ace35c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:40.303091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16dc83-ae28-4425-b74b-42dc79ace35c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:40.321163 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 613/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:40.322385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e050e11-05dd-470b-9a78-598ee28d62be None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:40.421695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:40.429259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e050e11-05dd-470b-9a78-598ee28d62be None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:40.467416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30adb6-5362-4e49-82d9-4a24ad74edeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:40.468051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30adb6-5362-4e49-82d9-4a24ad74edeb None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:40.468333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30adb6-5362-4e49-82d9-4a24ad74edeb None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:40.468606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30adb6-5362-4e49-82d9-4a24ad74edeb None admin] PATH_INFO: `/v3/users/placement` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:40.475510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30adb6-5362-4e49-82d9-4a24ad74edeb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:40.476998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30adb6-5362-4e49-82d9-4a24ad74edeb None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-2f30adb6-5362-4e49-82d9-4a24ad74edeb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:36:40.489526 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:40.490712 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:40.491823 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:40.491823 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:40.491823 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:36:40.491823 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:36:40.491823 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:36:40.491823 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:36:40.491823 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:36:40.491823 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:36:40.491823 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 20:36:40.491823 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:40.491823 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 613/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:40.497431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16dc83-ae28-4425-b74b-42dc79ace35c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:40.544448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:40.600542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16dc83-ae28-4425-b74b-42dc79ace35c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:40.634365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853b42e-6946-4c0b-87dd-42841ad5e4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:40.634995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853b42e-6946-4c0b-87dd-42841ad5e4b0 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:40.635243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853b42e-6946-4c0b-87dd-42841ad5e4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:40.635517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853b42e-6946-4c0b-87dd-42841ad5e4b0 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:40.636230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853b42e-6946-4c0b-87dd-42841ad5e4b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:40.636538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853b42e-6946-4c0b-87dd-42841ad5e4b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:40.638518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853b42e-6946-4c0b-87dd-42841ad5e4b0 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:40.656464 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 614/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Sep 3 20:36:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:40.663369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30adb6-5362-4e49-82d9-4a24ad74edeb None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:40.783498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30adb6-5362-4e49-82d9-4a24ad74edeb None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:40.821878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56446c62-6962-42ed-9ce4-f3fe03ce5af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:40.822551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56446c62-6962-42ed-9ce4-f3fe03ce5af1 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:40.822857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56446c62-6962-42ed-9ce4-f3fe03ce5af1 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:40.823206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56446c62-6962-42ed-9ce4-f3fe03ce5af1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:40.835789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56446c62-6962-42ed-9ce4-f3fe03ce5af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:40.837367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56446c62-6962-42ed-9ce4-f3fe03ce5af1 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-56446c62-6962-42ed-9ce4-f3fe03ce5af1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:40.853012 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:40.854165 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:40.855458 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:40.855458 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:40.855458 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:40.855458 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 614/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:40.860656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853b42e-6946-4c0b-87dd-42841ad5e4b0 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:40.973140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853b42e-6946-4c0b-87dd-42841ad5e4b0 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:40.975707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:41.009797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff05dd8-7c45-45ae-b07e-61cf6747bc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:41.010418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff05dd8-7c45-45ae-b07e-61cf6747bc2b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:41.010723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff05dd8-7c45-45ae-b07e-61cf6747bc2b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:41.010982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff05dd8-7c45-45ae-b07e-61cf6747bc2b None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:41.011627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff05dd8-7c45-45ae-b07e-61cf6747bc2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:41.011936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff05dd8-7c45-45ae-b07e-61cf6747bc2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:41.013913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff05dd8-7c45-45ae-b07e-61cf6747bc2b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:41.025973 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 615/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:36:41.031866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56446c62-6962-42ed-9ce4-f3fe03ce5af1 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:41.041931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:41.144426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56446c62-6962-42ed-9ce4-f3fe03ce5af1 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:41.183757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61e93e-bd80-446f-9ae0-963f9cbc23b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:41.185282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61e93e-bd80-446f-9ae0-963f9cbc23b8 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:41.185652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61e93e-bd80-446f-9ae0-963f9cbc23b8 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:41.185939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61e93e-bd80-446f-9ae0-963f9cbc23b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:41.194093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61e93e-bd80-446f-9ae0-963f9cbc23b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:41.195943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61e93e-bd80-446f-9ae0-963f9cbc23b8 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-0f61e93e-bd80-446f-9ae0-963f9cbc23b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:36:41.209494 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:41.210573 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:41.216445 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:36:41.216445 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:36:41.216445 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:36:41.216445 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:36:41.216445 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:36:41.216445 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:36:41.216445 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:36:41.216445 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:41.216445 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 615/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:36:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:36:41.216968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff05dd8-7c45-45ae-b07e-61cf6747bc2b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:41.331105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff05dd8-7c45-45ae-b07e-61cf6747bc2b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:41.368520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618bd46-3ec0-4164-b4e8-ac694204fc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:41.369152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618bd46-3ec0-4164-b4e8-ac694204fc83 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:41.369433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618bd46-3ec0-4164-b4e8-ac694204fc83 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:41.369702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618bd46-3ec0-4164-b4e8-ac694204fc83 None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:41.370412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618bd46-3ec0-4164-b4e8-ac694204fc83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:41.370678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618bd46-3ec0-4164-b4e8-ac694204fc83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:41.372147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618bd46-3ec0-4164-b4e8-ac694204fc83 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:41.384811 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 616/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:36:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:36:41.391692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61e93e-bd80-446f-9ae0-963f9cbc23b8 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:41.474446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:41.511235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61e93e-bd80-446f-9ae0-963f9cbc23b8 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:41.546778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89736cd3-4179-4e5c-bed5-4e1bd68033b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:41.547431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89736cd3-4179-4e5c-bed5-4e1bd68033b5 None admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:41.547712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89736cd3-4179-4e5c-bed5-4e1bd68033b5 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:41.547989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89736cd3-4179-4e5c-bed5-4e1bd68033b5 None admin] PATH_INFO: `/v3/projects/1e040b2fca724c58a2660441a72cd350/users/6dfb57e006f344808f7a2ef41b246d5e/roles/a211c6f9ebbc4889b636c26cd511e0e3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:41.563025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89736cd3-4179-4e5c-bed5-4e1bd68033b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e040b2fca724c58a2660441a72cd350, user_id=6dfb57e006f344808f7a2ef41b246d5e, role_id=a211c6f9ebbc4889b636c26cd511e0e3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:41.564481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89736cd3-4179-4e5c-bed5-4e1bd68033b5 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:41.577656 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 616/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:36:41 2022] PUT /identity/v3/projects/1e040b2fca724c58a2660441a72cd350/users/6dfb57e006f344808f7a2ef41b246d5e/roles/a211c6f9ebbc4889b636c26cd511e0e3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:36:43.746489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6272d-bc4e-4b5d-a396-2a4b4e4238c7 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:43.747668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6272d-bc4e-4b5d-a396-2a4b4e4238c7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:43.747914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6272d-bc4e-4b5d-a396-2a4b4e4238c7 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:43.749399 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 617/1233] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:36:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:36:43.755184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e5998-9a7e-4769-9932-9175f9966a95 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:43.755668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e5998-9a7e-4769-9932-9175f9966a95 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:43.755778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e5998-9a7e-4769-9932-9175f9966a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:43.775681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-372e5998-9a7e-4769-9932-9175f9966a95 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:36:43.776962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-372e5998-9a7e-4769-9932-9175f9966a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:36:43.851212 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 617/1234] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:36:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:36:44.217777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6272d-bc4e-4b5d-a396-2a4b4e4238c7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:44.247329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:44.335944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6272d-bc4e-4b5d-a396-2a4b4e4238c7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:44.371012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1944-2fb8-4fb1-9ca7-2f2624a932b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:44.372594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1944-2fb8-4fb1-9ca7-2f2624a932b8 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:44.373048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1944-2fb8-4fb1-9ca7-2f2624a932b8 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:44.373366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1944-2fb8-4fb1-9ca7-2f2624a932b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:44.384285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb1944-2fb8-4fb1-9ca7-2f2624a932b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-6dbb1944-2fb8-4fb1-9ca7-2f2624a932b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:36:44.389714 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:44.390838 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:44.391722 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 618/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 3 20:36:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:44.399549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e5998-9a7e-4769-9932-9175f9966a95 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:44.448419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:44.506313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e5998-9a7e-4769-9932-9175f9966a95 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:44.544244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73babc75-08d9-451f-8f25-45f6ce82e1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:44.544925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73babc75-08d9-451f-8f25-45f6ce82e1cf None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:44.545222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73babc75-08d9-451f-8f25-45f6ce82e1cf None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:44.545506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73babc75-08d9-451f-8f25-45f6ce82e1cf None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:44.546184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73babc75-08d9-451f-8f25-45f6ce82e1cf None admin] RBAC: Adding query filter params (name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:44.546492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73babc75-08d9-451f-8f25-45f6ce82e1cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:44.548005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73babc75-08d9-451f-8f25-45f6ce82e1cf None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:44.557397 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 618/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 3 20:36:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:36:44.563462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1944-2fb8-4fb1-9ca7-2f2624a932b8 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:44.656334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:44.673569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1944-2fb8-4fb1-9ca7-2f2624a932b8 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:44.712693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbfc0e-3d91-404b-acb2-0ee2d4d1c3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:44.713380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbfc0e-3d91-404b-acb2-0ee2d4d1c3dd None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:44.713670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbfc0e-3d91-404b-acb2-0ee2d4d1c3dd None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:44.713911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbfc0e-3d91-404b-acb2-0ee2d4d1c3dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:44.725844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbfc0e-3d91-404b-acb2-0ee2d4d1c3dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:44.727312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbfc0e-3d91-404b-acb2-0ee2d4d1c3dd None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-1dfbfc0e-3d91-404b-acb2-0ee2d4d1c3dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:44.743884 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:44.745117 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:44.746359 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:44.746359 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:44.746359 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:44.746359 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 619/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:44.753289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73babc75-08d9-451f-8f25-45f6ce82e1cf None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:44.862200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73babc75-08d9-451f-8f25-45f6ce82e1cf None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:44.872093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:44.899990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbfb12-a642-4921-89ee-c57bfeab8573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:44.900738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbfb12-a642-4921-89ee-c57bfeab8573 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:44.901080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbfb12-a642-4921-89ee-c57bfeab8573 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:44.901370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbfb12-a642-4921-89ee-c57bfeab8573 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:44.902107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbfb12-a642-4921-89ee-c57bfeab8573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:44.902493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbfb12-a642-4921-89ee-c57bfeab8573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:44.904632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbfb12-a642-4921-89ee-c57bfeab8573 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:44.917384 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 619/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:44.922991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbfc0e-3d91-404b-acb2-0ee2d4d1c3dd None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:45.030670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbfc0e-3d91-404b-acb2-0ee2d4d1c3dd None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:45.066479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acf85c-6268-4239-bec9-6bd109c61f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:45.067141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acf85c-6268-4239-bec9-6bd109c61f27 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:45.067399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acf85c-6268-4239-bec9-6bd109c61f27 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:45.067695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acf85c-6268-4239-bec9-6bd109c61f27 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:45.070603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:45.076819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acf85c-6268-4239-bec9-6bd109c61f27 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:45.078344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acf85c-6268-4239-bec9-6bd109c61f27 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-18acf85c-6268-4239-bec9-6bd109c61f27 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:36:45.089944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:45.091472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:45.092431 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:45.092431 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:45.092431 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:36:45.092431 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:36:45.092431 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:36:45.092431 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:36:45.092431 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:36:45.092431 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:36:45.092431 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 20:36:45.092431 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:45.092431 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 620/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:45.099260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbfb12-a642-4921-89ee-c57bfeab8573 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:45.210985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbfb12-a642-4921-89ee-c57bfeab8573 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:45.252930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002e6cc-53a6-4aff-8911-222daba5e225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:45.253803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002e6cc-53a6-4aff-8911-222daba5e225 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:45.253973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002e6cc-53a6-4aff-8911-222daba5e225 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:45.254242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002e6cc-53a6-4aff-8911-222daba5e225 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:45.254964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002e6cc-53a6-4aff-8911-222daba5e225 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:45.255271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002e6cc-53a6-4aff-8911-222daba5e225 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:45.256828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002e6cc-53a6-4aff-8911-222daba5e225 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:45.275033 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 620/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Sep 3 20:36:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:45.281005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acf85c-6268-4239-bec9-6bd109c61f27 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:45.387858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acf85c-6268-4239-bec9-6bd109c61f27 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:45.425679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb3612-af54-48dc-82bc-568c2f2da13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:45.426294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb3612-af54-48dc-82bc-568c2f2da13c None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:45.426578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb3612-af54-48dc-82bc-568c2f2da13c None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:45.426853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb3612-af54-48dc-82bc-568c2f2da13c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:45.492780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:45.500740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb3612-af54-48dc-82bc-568c2f2da13c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:45.502817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb3612-af54-48dc-82bc-568c2f2da13c None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:45.612187 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 621/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:36:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:36:45.618104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002e6cc-53a6-4aff-8911-222daba5e225 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:45.649484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:45.730655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002e6cc-53a6-4aff-8911-222daba5e225 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:45.770716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290abd9-ebc7-414a-8d07-5d35de432012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:45.771288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290abd9-ebc7-414a-8d07-5d35de432012 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:45.771572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290abd9-ebc7-414a-8d07-5d35de432012 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:45.771852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290abd9-ebc7-414a-8d07-5d35de432012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:45.783035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290abd9-ebc7-414a-8d07-5d35de432012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:45.784745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290abd9-ebc7-414a-8d07-5d35de432012 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-f290abd9-ebc7-414a-8d07-5d35de432012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:45.800670 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:45.801899 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:45.803223 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:45.803223 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:45.803223 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:45.803223 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 621/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:45.807507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb3612-af54-48dc-82bc-568c2f2da13c None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:45.916967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:45.923516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb3612-af54-48dc-82bc-568c2f2da13c None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:45.961753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814900c-911c-49ad-8a5a-0e891a1c1ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:45.962446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814900c-911c-49ad-8a5a-0e891a1c1ee5 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:45.962794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814900c-911c-49ad-8a5a-0e891a1c1ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:45.963116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814900c-911c-49ad-8a5a-0e891a1c1ee5 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:45.963876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814900c-911c-49ad-8a5a-0e891a1c1ee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:45.964189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814900c-911c-49ad-8a5a-0e891a1c1ee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:45.966205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814900c-911c-49ad-8a5a-0e891a1c1ee5 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:45.984464 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 622/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:45.985686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290abd9-ebc7-414a-8d07-5d35de432012 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:46.077241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:46.104096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290abd9-ebc7-414a-8d07-5d35de432012 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:46.144019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107e015-e093-4910-a9c8-94e61d73d4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:46.144691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107e015-e093-4910-a9c8-94e61d73d4fb None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:46.145016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107e015-e093-4910-a9c8-94e61d73d4fb None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:46.145397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107e015-e093-4910-a9c8-94e61d73d4fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:46.152627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107e015-e093-4910-a9c8-94e61d73d4fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:46.154634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107e015-e093-4910-a9c8-94e61d73d4fb None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-a107e015-e093-4910-a9c8-94e61d73d4fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:36:46.167118 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:46.168153 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:46.169685 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:36:46.169685 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:36:46.169685 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:36:46.169685 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:36:46.169685 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:36:46.169685 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:36:46.169685 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:36:46.169685 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:46.169685 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 622/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:36:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:36:46.174957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814900c-911c-49ad-8a5a-0e891a1c1ee5 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:46.313291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814900c-911c-49ad-8a5a-0e891a1c1ee5 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:46.348675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7597486-4150-4b84-a860-a4274ebbc8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:46.349347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7597486-4150-4b84-a860-a4274ebbc8fa None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:46.349634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7597486-4150-4b84-a860-a4274ebbc8fa None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:46.349930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7597486-4150-4b84-a860-a4274ebbc8fa None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:46.350651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7597486-4150-4b84-a860-a4274ebbc8fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:46.351004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7597486-4150-4b84-a860-a4274ebbc8fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:46.352482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7597486-4150-4b84-a860-a4274ebbc8fa None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:46.368350 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 623/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:36:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:36:46.375135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107e015-e093-4910-a9c8-94e61d73d4fb None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:46.490690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107e015-e093-4910-a9c8-94e61d73d4fb None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:46.528579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaeace5-551c-4562-bbad-992adc3a5c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:46.529249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeace5-551c-4562-bbad-992adc3a5c1c None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:46.529512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeace5-551c-4562-bbad-992adc3a5c1c None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:46.529835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeace5-551c-4562-bbad-992adc3a5c1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:46.530513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaeace5-551c-4562-bbad-992adc3a5c1c None admin] RBAC: Adding query filter params (role.id=a211c6f9ebbc4889b636c26cd511e0e3, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=6dfb57e006f344808f7a2ef41b246d5e) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:46.530827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaeace5-551c-4562-bbad-992adc3a5c1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:46.532262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaeace5-551c-4562-bbad-992adc3a5c1c None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:46.538677 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 623/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:36:46 2022] GET /identity/v3/role_assignments?user.id=6dfb57e006f344808f7a2ef41b246d5e&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=a211c6f9ebbc4889b636c26cd511e0e3 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:48.892181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bb14a-43fc-4e99-93e3-cd488c60d1fc None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:48.892721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bb14a-43fc-4e99-93e3-cd488c60d1fc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:48.892721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bb14a-43fc-4e99-93e3-cd488c60d1fc None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:48.894110 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 624/1247] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:36:48.900126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9e795-eb35-4545-9de9-e3dd27f01805 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:48.900583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9e795-eb35-4545-9de9-e3dd27f01805 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:48.900683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9e795-eb35-4545-9de9-e3dd27f01805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:48.907665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:48.920230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ded9e795-eb35-4545-9de9-e3dd27f01805 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:36:48.921497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ded9e795-eb35-4545-9de9-e3dd27f01805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:36:48.990841 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 624/1248] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:36:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:36:49.360577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bb14a-43fc-4e99-93e3-cd488c60d1fc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:49.379407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:49.468192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bb14a-43fc-4e99-93e3-cd488c60d1fc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:49.504586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b6455-2f03-4cfa-a8f8-72d161543d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:49.505306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6455-2f03-4cfa-a8f8-72d161543d1a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:49.505591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6455-2f03-4cfa-a8f8-72d161543d1a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:49.505867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6455-2f03-4cfa-a8f8-72d161543d1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:49.516922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b6455-2f03-4cfa-a8f8-72d161543d1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-449b6455-2f03-4cfa-a8f8-72d161543d1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:36:49.521809 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:49.522882 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:49.523928 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 625/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:36:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:49.529610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9e795-eb35-4545-9de9-e3dd27f01805 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:49.637635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9e795-eb35-4545-9de9-e3dd27f01805 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:49.674849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:49.681153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56a56d-0cb2-4fee-85ad-a7d96a21bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:49.681866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56a56d-0cb2-4fee-85ad-a7d96a21bfc2 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:49.682227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56a56d-0cb2-4fee-85ad-a7d96a21bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:49.682574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56a56d-0cb2-4fee-85ad-a7d96a21bfc2 None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:49.683405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56a56d-0cb2-4fee-85ad-a7d96a21bfc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:49.683741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56a56d-0cb2-4fee-85ad-a7d96a21bfc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:49.685370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56a56d-0cb2-4fee-85ad-a7d96a21bfc2 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:49.694867 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 625/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:36:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:49.701374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b6455-2f03-4cfa-a8f8-72d161543d1a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:49.796573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:49.822538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b6455-2f03-4cfa-a8f8-72d161543d1a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:49.858468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d11318-5da0-4976-a898-a10531ee3adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:49.859125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d11318-5da0-4976-a898-a10531ee3adf None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:49.859482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d11318-5da0-4976-a898-a10531ee3adf None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:49.859712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d11318-5da0-4976-a898-a10531ee3adf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:49.871040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d11318-5da0-4976-a898-a10531ee3adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:49.873065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d11318-5da0-4976-a898-a10531ee3adf None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-d2d11318-5da0-4976-a898-a10531ee3adf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:49.890476 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:49.891464 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:49.893376 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:49.893376 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:49.893376 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:49.893376 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 626/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:36:49.897634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56a56d-0cb2-4fee-85ad-a7d96a21bfc2 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:50.012955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56a56d-0cb2-4fee-85ad-a7d96a21bfc2 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:50.052626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2610a4-0817-434d-b51c-eebd5f0836fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:50.053365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2610a4-0817-434d-b51c-eebd5f0836fc None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:50.053631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2610a4-0817-434d-b51c-eebd5f0836fc None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:50.053912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2610a4-0817-434d-b51c-eebd5f0836fc None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:50.054709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2610a4-0817-434d-b51c-eebd5f0836fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:50.054964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2610a4-0817-434d-b51c-eebd5f0836fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:50.056363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2610a4-0817-434d-b51c-eebd5f0836fc None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:50.068726 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 626/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:50.074962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d11318-5da0-4976-a898-a10531ee3adf None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:50.187064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d11318-5da0-4976-a898-a10531ee3adf None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:50.214873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:50.224608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1b907-2552-47ee-9269-99a49b7461e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:50.225370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1b907-2552-47ee-9269-99a49b7461e0 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:50.225653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1b907-2552-47ee-9269-99a49b7461e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:50.226006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1b907-2552-47ee-9269-99a49b7461e0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:50.233678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1b907-2552-47ee-9269-99a49b7461e0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:50.235136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1b907-2552-47ee-9269-99a49b7461e0 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-0ee1b907-2552-47ee-9269-99a49b7461e0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:36:50.247264 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:50.248434 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:50.249799 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:50.249799 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:50.249799 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:36:50.249799 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:36:50.249799 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:36:50.249799 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:36:50.249799 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:36:50.249799 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:36:50.249799 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 20:36:50.249799 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:50.249799 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 627/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:50.254579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2610a4-0817-434d-b51c-eebd5f0836fc None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:50.317419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:50.369023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2610a4-0817-434d-b51c-eebd5f0836fc None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:50.405070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127cce0-4b7b-4958-b4c8-686159992910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:50.405692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127cce0-4b7b-4958-b4c8-686159992910 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:50.406047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127cce0-4b7b-4958-b4c8-686159992910 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:50.406413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127cce0-4b7b-4958-b4c8-686159992910 None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:50.407203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127cce0-4b7b-4958-b4c8-686159992910 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:50.407516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127cce0-4b7b-4958-b4c8-686159992910 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:50.409015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127cce0-4b7b-4958-b4c8-686159992910 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:50.428719 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 627/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Sep 3 20:36:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:50.435399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1b907-2552-47ee-9269-99a49b7461e0 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:50.542730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1b907-2552-47ee-9269-99a49b7461e0 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:50.578483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d72595-c4b9-40a4-b3c8-a942045c1aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:50.579097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d72595-c4b9-40a4-b3c8-a942045c1aab None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:50.579380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d72595-c4b9-40a4-b3c8-a942045c1aab None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:50.579651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d72595-c4b9-40a4-b3c8-a942045c1aab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:50.639234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:50.652182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d72595-c4b9-40a4-b3c8-a942045c1aab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:50.653684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d72595-c4b9-40a4-b3c8-a942045c1aab None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:50.766927 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 628/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:36:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:36:50.772915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127cce0-4b7b-4958-b4c8-686159992910 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:50.877283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127cce0-4b7b-4958-b4c8-686159992910 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:50.890462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:50.912589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9299c9c-223d-402a-9018-d336c7f6b272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:50.913396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9299c9c-223d-402a-9018-d336c7f6b272 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:50.913680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9299c9c-223d-402a-9018-d336c7f6b272 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:50.913957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9299c9c-223d-402a-9018-d336c7f6b272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:50.924430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9299c9c-223d-402a-9018-d336c7f6b272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:50.925897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9299c9c-223d-402a-9018-d336c7f6b272 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-f9299c9c-223d-402a-9018-d336c7f6b272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:50.940626 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:50.941769 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:50.942961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:50.942961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:50.942961 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:50.942961 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 628/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:50.947501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d72595-c4b9-40a4-b3c8-a942045c1aab None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:51.039440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:51.052132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d72595-c4b9-40a4-b3c8-a942045c1aab None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:51.087513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d598b-9149-4180-b220-6391e245005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:51.088121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d598b-9149-4180-b220-6391e245005b None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:51.088407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d598b-9149-4180-b220-6391e245005b None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:51.088685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d598b-9149-4180-b220-6391e245005b None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:51.089368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d598b-9149-4180-b220-6391e245005b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:51.089671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d598b-9149-4180-b220-6391e245005b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:51.091596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d598b-9149-4180-b220-6391e245005b None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:51.103062 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 629/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:51.108853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9299c9c-223d-402a-9018-d336c7f6b272 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:51.211464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9299c9c-223d-402a-9018-d336c7f6b272 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:51.244389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2da299-e9c8-4737-b2ec-43f395ee4d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:51.245045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2da299-e9c8-4737-b2ec-43f395ee4d6e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:51.245331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2da299-e9c8-4737-b2ec-43f395ee4d6e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:51.245608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2da299-e9c8-4737-b2ec-43f395ee4d6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:51.251954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2da299-e9c8-4737-b2ec-43f395ee4d6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:51.253426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2da299-e9c8-4737-b2ec-43f395ee4d6e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-6e2da299-e9c8-4737-b2ec-43f395ee4d6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:36:51.264350 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:51.265397 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:51.266612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:36:51.266612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:36:51.266612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:36:51.266612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:36:51.266612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:36:51.266612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:36:51.266612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:36:51.266612 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:51.266612 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 629/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:36:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:51.271312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d598b-9149-4180-b220-6391e245005b None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:51.367647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d598b-9149-4180-b220-6391e245005b None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:51.400287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3b9d4-c983-4076-85af-5d6ab8a4196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:51.400826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3b9d4-c983-4076-85af-5d6ab8a4196a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:51.401139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3b9d4-c983-4076-85af-5d6ab8a4196a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:51.401418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3b9d4-c983-4076-85af-5d6ab8a4196a None admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:51.402107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3b9d4-c983-4076-85af-5d6ab8a4196a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:51.402417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3b9d4-c983-4076-85af-5d6ab8a4196a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:51.403878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3b9d4-c983-4076-85af-5d6ab8a4196a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:51.415287 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 630/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:36:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:51.421168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2da299-e9c8-4737-b2ec-43f395ee4d6e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:51.446646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:51.520464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2da299-e9c8-4737-b2ec-43f395ee4d6e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:51.552215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71f7d8-9f97-4b87-8f78-88d410d4843d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:51.552805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71f7d8-9f97-4b87-8f78-88d410d4843d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:51.553120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71f7d8-9f97-4b87-8f78-88d410d4843d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:51.553395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71f7d8-9f97-4b87-8f78-88d410d4843d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:51.554112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71f7d8-9f97-4b87-8f78-88d410d4843d None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=6dfb57e006f344808f7a2ef41b246d5e) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:51.554422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71f7d8-9f97-4b87-8f78-88d410d4843d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:51.555842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71f7d8-9f97-4b87-8f78-88d410d4843d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:51.561609 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 630/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:36:51 2022] GET /identity/v3/role_assignments?user.id=6dfb57e006f344808f7a2ef41b246d5e&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=bc768522dc984239be4a1b39a45ab437 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:53.702087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a57d45-367f-4535-a0c5-8d0ca4405a79 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:53.702643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a57d45-367f-4535-a0c5-8d0ca4405a79 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:53.702756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a57d45-367f-4535-a0c5-8d0ca4405a79 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:53.704177 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 631/1261] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:36:53.710287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e7fc6-9349-49d2-be7f-c52b67b21667 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:53.710684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e7fc6-9349-49d2-be7f-c52b67b21667 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:53.710851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e7fc6-9349-49d2-be7f-c52b67b21667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:53.733134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ac5e7fc6-9349-49d2-be7f-c52b67b21667 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:36:53.734672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ac5e7fc6-9349-49d2-be7f-c52b67b21667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:36:53.805588 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 631/1262] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:36:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:36:54.133849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a57d45-367f-4535-a0c5-8d0ca4405a79 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:54.145547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:54.234064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a57d45-367f-4535-a0c5-8d0ca4405a79 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:54.267486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486094e-c67b-42ba-91fe-da7e756ea5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:54.268107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486094e-c67b-42ba-91fe-da7e756ea5b9 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:54.268412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486094e-c67b-42ba-91fe-da7e756ea5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:54.268687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486094e-c67b-42ba-91fe-da7e756ea5b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:54.278425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486094e-c67b-42ba-91fe-da7e756ea5b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-2486094e-c67b-42ba-91fe-da7e756ea5b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:36:54.283160 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:54.284223 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:54.285281 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 632/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:36:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:54.290596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e7fc6-9349-49d2-be7f-c52b67b21667 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:54.308274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:54.386774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e7fc6-9349-49d2-be7f-c52b67b21667 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:54.419412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea961a1-3b2c-4797-818b-4c833c46b11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:54.420004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea961a1-3b2c-4797-818b-4c833c46b11d None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:54.420288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea961a1-3b2c-4797-818b-4c833c46b11d None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:54.420567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea961a1-3b2c-4797-818b-4c833c46b11d None admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:54.421258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea961a1-3b2c-4797-818b-4c833c46b11d None admin] RBAC: Adding query filter params (name=admin) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:54.421565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea961a1-3b2c-4797-818b-4c833c46b11d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:54.422927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea961a1-3b2c-4797-818b-4c833c46b11d None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:54.430658 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 632/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:36:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:54.436723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486094e-c67b-42ba-91fe-da7e756ea5b9 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:54.502194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:54.534163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486094e-c67b-42ba-91fe-da7e756ea5b9 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:54.566504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304163f-2302-4475-a6d5-4c103d6f666a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:54.567119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304163f-2302-4475-a6d5-4c103d6f666a None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:54.567405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304163f-2302-4475-a6d5-4c103d6f666a None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:54.567687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304163f-2302-4475-a6d5-4c103d6f666a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:54.578055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304163f-2302-4475-a6d5-4c103d6f666a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:54.579500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304163f-2302-4475-a6d5-4c103d6f666a None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-f304163f-2302-4475-a6d5-4c103d6f666a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:54.593837 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:54.594970 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:54.596090 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:54.596090 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:54.596090 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:54.596090 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 633/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:54.600504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea961a1-3b2c-4797-818b-4c833c46b11d None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:54.679634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:54.702203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea961a1-3b2c-4797-818b-4c833c46b11d None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:54.741711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b17c7-72a4-470c-9f14-cc5340ed1e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:54.742346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b17c7-72a4-470c-9f14-cc5340ed1e02 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:54.742593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b17c7-72a4-470c-9f14-cc5340ed1e02 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:54.742873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b17c7-72a4-470c-9f14-cc5340ed1e02 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:54.743513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b17c7-72a4-470c-9f14-cc5340ed1e02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:54.743820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b17c7-72a4-470c-9f14-cc5340ed1e02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:54.745222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b17c7-72a4-470c-9f14-cc5340ed1e02 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:54.757358 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 633/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:54.763288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304163f-2302-4475-a6d5-4c103d6f666a None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:54.860008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304163f-2302-4475-a6d5-4c103d6f666a None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:54.879650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:54.894310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a8e3d-609d-47a7-ad7d-31a412ec08e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:54.894935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a8e3d-609d-47a7-ad7d-31a412ec08e3 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:54.895219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a8e3d-609d-47a7-ad7d-31a412ec08e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:54.895496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a8e3d-609d-47a7-ad7d-31a412ec08e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:54.903523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a8e3d-609d-47a7-ad7d-31a412ec08e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:54.904995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a8e3d-609d-47a7-ad7d-31a412ec08e3 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-858a8e3d-609d-47a7-ad7d-31a412ec08e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:36:54.915789 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:54.916924 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:54.918609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:54.918609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:54.918609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:36:54.918609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:36:54.918609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:36:54.918609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:36:54.918609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:36:54.918609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:36:54.918609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 20:36:54.918609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:54.918609 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 634/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:54.923148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b17c7-72a4-470c-9f14-cc5340ed1e02 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:55.020213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b17c7-72a4-470c-9f14-cc5340ed1e02 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:55.053652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a3547-7d96-42f4-8ac3-04bf16e7622e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:55.054244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a3547-7d96-42f4-8ac3-04bf16e7622e None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:55.054523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a3547-7d96-42f4-8ac3-04bf16e7622e None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:55.054807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a3547-7d96-42f4-8ac3-04bf16e7622e None admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:55.055682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a3547-7d96-42f4-8ac3-04bf16e7622e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:55.056125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a3547-7d96-42f4-8ac3-04bf16e7622e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:55.057603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a3547-7d96-42f4-8ac3-04bf16e7622e None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:55.066136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:55.074317 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 634/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Sep 3 20:36:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:55.079962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a8e3d-609d-47a7-ad7d-31a412ec08e3 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:55.179169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a8e3d-609d-47a7-ad7d-31a412ec08e3 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:55.211021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd313c-a07e-4c4f-bdff-acfe4765088d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:55.211638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd313c-a07e-4c4f-bdff-acfe4765088d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:55.211925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd313c-a07e-4c4f-bdff-acfe4765088d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:55.212201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd313c-a07e-4c4f-bdff-acfe4765088d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:55.222751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd313c-a07e-4c4f-bdff-acfe4765088d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:55.224190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd313c-a07e-4c4f-bdff-acfe4765088d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-2cfd313c-a07e-4c4f-bdff-acfe4765088d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:55.238414 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:55.239555 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:55.240663 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:55.240663 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:55.240663 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:55.240663 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 635/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:55.245257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a3547-7d96-42f4-8ac3-04bf16e7622e None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:55.343451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a3547-7d96-42f4-8ac3-04bf16e7622e None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:55.376516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f409b-8ef2-4d5e-80c4-92d01b616121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:55.377138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f409b-8ef2-4d5e-80c4-92d01b616121 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:55.377421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f409b-8ef2-4d5e-80c4-92d01b616121 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:55.377697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f409b-8ef2-4d5e-80c4-92d01b616121 None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:55.378362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f409b-8ef2-4d5e-80c4-92d01b616121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:55.378667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f409b-8ef2-4d5e-80c4-92d01b616121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:55.380017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f409b-8ef2-4d5e-80c4-92d01b616121 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:55.390810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 635/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:55.396432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd313c-a07e-4c4f-bdff-acfe4765088d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:55.427019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:55.494355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd313c-a07e-4c4f-bdff-acfe4765088d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:55.527341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae2f96-6460-4e2d-a30e-1d1840d9e2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:55.528505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae2f96-6460-4e2d-a30e-1d1840d9e2d0 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:55.528786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae2f96-6460-4e2d-a30e-1d1840d9e2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:55.529089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae2f96-6460-4e2d-a30e-1d1840d9e2d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:55.535476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae2f96-6460-4e2d-a30e-1d1840d9e2d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:55.536988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae2f96-6460-4e2d-a30e-1d1840d9e2d0 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-35ae2f96-6460-4e2d-a30e-1d1840d9e2d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:36:55.547881 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:55.548880 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:55.549976 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:36:55.549976 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:36:55.549976 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:36:55.549976 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:36:55.549976 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:36:55.549976 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:36:55.549976 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:36:55.549976 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:55.549976 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 636/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:36:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:55.554664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f409b-8ef2-4d5e-80c4-92d01b616121 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:55.612288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:55.653566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f409b-8ef2-4d5e-80c4-92d01b616121 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:55.686771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e4677-6227-4f86-9d2b-0e5cc9336755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:55.687368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e4677-6227-4f86-9d2b-0e5cc9336755 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:55.687654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e4677-6227-4f86-9d2b-0e5cc9336755 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:55.687930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e4677-6227-4f86-9d2b-0e5cc9336755 None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:55.688625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e4677-6227-4f86-9d2b-0e5cc9336755 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:55.688952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e4677-6227-4f86-9d2b-0e5cc9336755 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:55.690386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e4677-6227-4f86-9d2b-0e5cc9336755 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:55.703103 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 636/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:36:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:55.710473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae2f96-6460-4e2d-a30e-1d1840d9e2d0 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:55.799582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:55.808455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae2f96-6460-4e2d-a30e-1d1840d9e2d0 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:55.841907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96096fa0-7d81-498d-83e6-aed090ee9d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:55.842568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96096fa0-7d81-498d-83e6-aed090ee9d75 None admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:55.842836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96096fa0-7d81-498d-83e6-aed090ee9d75 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:55.843112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96096fa0-7d81-498d-83e6-aed090ee9d75 None admin] PATH_INFO: `/v3/projects/1e040b2fca724c58a2660441a72cd350/users/6dfb57e006f344808f7a2ef41b246d5e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:55.845016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96096fa0-7d81-498d-83e6-aed090ee9d75 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e040b2fca724c58a2660441a72cd350, user_id=6dfb57e006f344808f7a2ef41b246d5e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:55.846447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96096fa0-7d81-498d-83e6-aed090ee9d75 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:55.858697 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 637/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 3 20:36:55 2022] PUT /identity/v3/projects/1e040b2fca724c58a2660441a72cd350/users/6dfb57e006f344808f7a2ef41b246d5e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:36:58.010569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ff17c-62ea-43b3-b559-aa5b52ceb9c8 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:58.011149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ff17c-62ea-43b3-b559-aa5b52ceb9c8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:58.011149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ff17c-62ea-43b3-b559-aa5b52ceb9c8 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:58.013060 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 637/1274] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:36:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:36:58.018264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cc17b-1d64-4525-934d-2f4afd7faa09 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:58.018660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cc17b-1d64-4525-934d-2f4afd7faa09 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:58.018829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cc17b-1d64-4525-934d-2f4afd7faa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:58.036620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-928cc17b-1d64-4525-934d-2f4afd7faa09 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:36:58.037873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-928cc17b-1d64-4525-934d-2f4afd7faa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:36:58.102729 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 638/1275] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:36:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:36:58.431290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ff17c-62ea-43b3-b559-aa5b52ceb9c8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:58.530056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ff17c-62ea-43b3-b559-aa5b52ceb9c8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:58.547074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:58.564136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7009cc1-73f0-421f-8647-360092d43d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:58.564755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7009cc1-73f0-421f-8647-360092d43d8a None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:58.565079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7009cc1-73f0-421f-8647-360092d43d8a None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:58.565359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7009cc1-73f0-421f-8647-360092d43d8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:58.575128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7009cc1-73f0-421f-8647-360092d43d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-a7009cc1-73f0-421f-8647-360092d43d8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 20:36:58.579916 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:58.581085 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:58.582305 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 638/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 3 20:36:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:58.587061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cc17b-1d64-4525-934d-2f4afd7faa09 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:58.664576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:58.683309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cc17b-1d64-4525-934d-2f4afd7faa09 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:58.720731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64068f90-15b3-4ff1-b1d6-9a4c0faa1736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:58.721454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64068f90-15b3-4ff1-b1d6-9a4c0faa1736 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:58.721855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64068f90-15b3-4ff1-b1d6-9a4c0faa1736 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:58.722127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64068f90-15b3-4ff1-b1d6-9a4c0faa1736 None admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:58.722855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64068f90-15b3-4ff1-b1d6-9a4c0faa1736 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:58.723245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64068f90-15b3-4ff1-b1d6-9a4c0faa1736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:58.724765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64068f90-15b3-4ff1-b1d6-9a4c0faa1736 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:58.734434 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 639/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 3 20:36:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:58.740703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7009cc1-73f0-421f-8647-360092d43d8a None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:58.840135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7009cc1-73f0-421f-8647-360092d43d8a None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:58.872602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faef1a-625f-42cd-a02c-d7e381c6dee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:58.873266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faef1a-625f-42cd-a02c-d7e381c6dee6 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:58.873548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faef1a-625f-42cd-a02c-d7e381c6dee6 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:58.873821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faef1a-625f-42cd-a02c-d7e381c6dee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:58.883306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63faef1a-625f-42cd-a02c-d7e381c6dee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:58.884736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63faef1a-625f-42cd-a02c-d7e381c6dee6 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-63faef1a-625f-42cd-a02c-d7e381c6dee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:58.899386 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:58.900405 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:58.901347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:58.901347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:58.901347 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:58.901347 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 639/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 20:36:58.906370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64068f90-15b3-4ff1-b1d6-9a4c0faa1736 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:59.002247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64068f90-15b3-4ff1-b1d6-9a4c0faa1736 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:59.034046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f10107-f29e-40d9-a2ce-2a0ca70de833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:59.034654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f10107-f29e-40d9-a2ce-2a0ca70de833 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:59.034934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f10107-f29e-40d9-a2ce-2a0ca70de833 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:59.035208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f10107-f29e-40d9-a2ce-2a0ca70de833 None admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:59.035842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f10107-f29e-40d9-a2ce-2a0ca70de833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:59.036145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f10107-f29e-40d9-a2ce-2a0ca70de833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:59.037535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f10107-f29e-40d9-a2ce-2a0ca70de833 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:59.048638 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 640/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:59.057246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faef1a-625f-42cd-a02c-d7e381c6dee6 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:59.088436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:59.157924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faef1a-625f-42cd-a02c-d7e381c6dee6 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:59.195215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91000701-cc31-47c9-a68a-3fcb78afd939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:59.195836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91000701-cc31-47c9-a68a-3fcb78afd939 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:59.196118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91000701-cc31-47c9-a68a-3fcb78afd939 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:59.196392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91000701-cc31-47c9-a68a-3fcb78afd939 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:59.203477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91000701-cc31-47c9-a68a-3fcb78afd939 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:59.204920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91000701-cc31-47c9-a68a-3fcb78afd939 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-91000701-cc31-47c9-a68a-3fcb78afd939 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 20:36:59.215922 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:59.217073 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:59.218021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:59.218021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:59.218021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 20:36:59.218021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 20:36:59.218021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 20:36:59.218021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 20:36:59.218021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 20:36:59.218021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 20:36:59.218021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 20:36:59.218021 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:36:59.218021 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 640/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:59.223027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f10107-f29e-40d9-a2ce-2a0ca70de833 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:59.226829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:59.321718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f10107-f29e-40d9-a2ce-2a0ca70de833 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:59.355827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1f7ca-99e5-4e03-980f-e7b4ba9f8666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:59.356432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1f7ca-99e5-4e03-980f-e7b4ba9f8666 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:59.356728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1f7ca-99e5-4e03-980f-e7b4ba9f8666 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:59.357040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1f7ca-99e5-4e03-980f-e7b4ba9f8666 None admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:59.357763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1f7ca-99e5-4e03-980f-e7b4ba9f8666 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:59.358072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1f7ca-99e5-4e03-980f-e7b4ba9f8666 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:59.359503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1f7ca-99e5-4e03-980f-e7b4ba9f8666 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:59.375783 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 641/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Sep 3 20:36:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:36:59.381498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91000701-cc31-47c9-a68a-3fcb78afd939 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:59.470564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:59.479841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91000701-cc31-47c9-a68a-3fcb78afd939 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:59.511771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc11b25-46b8-4e6b-a0d6-b132d2410b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:59.512919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc11b25-46b8-4e6b-a0d6-b132d2410b02 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:59.513203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc11b25-46b8-4e6b-a0d6-b132d2410b02 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:59.513475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc11b25-46b8-4e6b-a0d6-b132d2410b02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:59.578119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc11b25-46b8-4e6b-a0d6-b132d2410b02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:59.579471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc11b25-46b8-4e6b-a0d6-b132d2410b02 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:59.664372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:59.674534 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 641/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 3 20:36:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 20:36:59.680352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1f7ca-99e5-4e03-980f-e7b4ba9f8666 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:59.741787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:36:59.780349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1f7ca-99e5-4e03-980f-e7b4ba9f8666 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:59.813448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf551e-a702-47e0-bd60-eaa5af1d1a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:59.814071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf551e-a702-47e0-bd60-eaa5af1d1a5d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:59.814360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf551e-a702-47e0-bd60-eaa5af1d1a5d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:59.814641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf551e-a702-47e0-bd60-eaa5af1d1a5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:59.824299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf551e-a702-47e0-bd60-eaa5af1d1a5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:59.826295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf551e-a702-47e0-bd60-eaa5af1d1a5d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-4cdf551e-a702-47e0-bd60-eaa5af1d1a5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 20:36:59.840781 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:36:59.841906 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 20:36:59.842885 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 20:36:59.842885 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 20:36:59.842885 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:36:59.842885 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 642/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:36:59.847461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc11b25-46b8-4e6b-a0d6-b132d2410b02 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:36:59.946550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc11b25-46b8-4e6b-a0d6-b132d2410b02 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:36:59.979255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8167c-45c8-4ba9-9f39-b54a41191a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:36:59.979862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8167c-45c8-4ba9-9f39-b54a41191a0b None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:36:59.980149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8167c-45c8-4ba9-9f39-b54a41191a0b None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:36:59.980427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8167c-45c8-4ba9-9f39-b54a41191a0b None admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:36:59.981579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8167c-45c8-4ba9-9f39-b54a41191a0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:36:59.981889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8167c-45c8-4ba9-9f39-b54a41191a0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:36:59.983253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8167c-45c8-4ba9-9f39-b54a41191a0b None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:36:59.994208 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 642/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 3 20:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:37:00.000617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf551e-a702-47e0-bd60-eaa5af1d1a5d None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:00.097746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf551e-a702-47e0-bd60-eaa5af1d1a5d None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:00.111432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:37:00.131917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737379dc-1280-43a8-a862-f4368b2a9fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:00.132549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737379dc-1280-43a8-a862-f4368b2a9fe3 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:00.132822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737379dc-1280-43a8-a862-f4368b2a9fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:00.133140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737379dc-1280-43a8-a862-f4368b2a9fe3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:00.139658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737379dc-1280-43a8-a862-f4368b2a9fe3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:00.141133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737379dc-1280-43a8-a862-f4368b2a9fe3 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-737379dc-1280-43a8-a862-f4368b2a9fe3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 20:37:00.152932 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:37:00.153956 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:37:00.155060 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 20:37:00.155060 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 20:37:00.155060 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 20:37:00.155060 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 20:37:00.155060 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 20:37:00.155060 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 20:37:00.155060 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 20:37:00.155060 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 20:37:00.155060 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 643/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 3 20:36:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:37:00.159856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8167c-45c8-4ba9-9f39-b54a41191a0b None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:00.209890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:37:00.258949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8167c-45c8-4ba9-9f39-b54a41191a0b None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:00.291370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f4b08-2815-472f-8780-7267e8862248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:00.292007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f4b08-2815-472f-8780-7267e8862248 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:00.292297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f4b08-2815-472f-8780-7267e8862248 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:00.292571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f4b08-2815-472f-8780-7267e8862248 None admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:00.293824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f4b08-2815-472f-8780-7267e8862248 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:37:00.294134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f4b08-2815-472f-8780-7267e8862248 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:00.295551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f4b08-2815-472f-8780-7267e8862248 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:00.307037 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 643/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 3 20:37:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:37:00.313052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737379dc-1280-43a8-a862-f4368b2a9fe3 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:00.408794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737379dc-1280-43a8-a862-f4368b2a9fe3 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:00.440658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb153bf-f9f4-4702-ac49-1847f4f63368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:00.441361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb153bf-f9f4-4702-ac49-1847f4f63368 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:00.441645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb153bf-f9f4-4702-ac49-1847f4f63368 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:00.441919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb153bf-f9f4-4702-ac49-1847f4f63368 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:00.442646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb153bf-f9f4-4702-ac49-1847f4f63368 None admin] RBAC: Adding query filter params (role.id=bc768522dc984239be4a1b39a45ab437, scope.project.id=1e040b2fca724c58a2660441a72cd350, user.id=6dfb57e006f344808f7a2ef41b246d5e) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:37:00.442953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb153bf-f9f4-4702-ac49-1847f4f63368 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:00.444393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb153bf-f9f4-4702-ac49-1847f4f63368 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:00.450097 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 644/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 3 20:37:00 2022] GET /identity/v3/role_assignments?user.id=6dfb57e006f344808f7a2ef41b246d5e&scope.project.id=1e040b2fca724c58a2660441a72cd350&role.id=bc768522dc984239be4a1b39a45ab437 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:37:02.622444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d067a-707d-4a67-9e6d-86f1deb553db None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:02.622999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d067a-707d-4a67-9e6d-86f1deb553db None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:02.622999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d067a-707d-4a67-9e6d-86f1deb553db None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:02.624314 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 644/1288] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:37:02.629431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd0eb7-aa09-47b4-8dfd-95f55a202d83 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:02.629858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd0eb7-aa09-47b4-8dfd-95f55a202d83 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:02.630019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd0eb7-aa09-47b4-8dfd-95f55a202d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:02.648245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-99cd0eb7-aa09-47b4-8dfd-95f55a202d83 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:37:02.649473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-99cd0eb7-aa09-47b4-8dfd-95f55a202d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:37:02.677472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:37:02.721501 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 645/1289] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:37:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:37:03.053456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d067a-707d-4a67-9e6d-86f1deb553db None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:03.154942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d067a-707d-4a67-9e6d-86f1deb553db None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:03.165322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:37:03.189482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160dca0-4a6e-46f7-b9b5-2d65d824bd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:03.190109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160dca0-4a6e-46f7-b9b5-2d65d824bd9f None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:03.190391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160dca0-4a6e-46f7-b9b5-2d65d824bd9f None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:03.190665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160dca0-4a6e-46f7-b9b5-2d65d824bd9f None admin] PATH_INFO: `/v3/services/placement` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:03.191240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160dca0-4a6e-46f7-b9b5-2d65d824bd9f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:03.192610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160dca0-4a6e-46f7-b9b5-2d65d824bd9f None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-4160dca0-4a6e-46f7-b9b5-2d65d824bd9f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:37:03.201981 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:37:03.203189 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:37:03.204247 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 645/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 3 20:37:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:37:03.209338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd0eb7-aa09-47b4-8dfd-95f55a202d83 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:03.306682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd0eb7-aa09-47b4-8dfd-95f55a202d83 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:03.340513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd6359-7902-4085-990a-bb5b7088812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:03.341149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd6359-7902-4085-990a-bb5b7088812d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:03.341441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd6359-7902-4085-990a-bb5b7088812d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:03.341724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd6359-7902-4085-990a-bb5b7088812d None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:03.342361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd6359-7902-4085-990a-bb5b7088812d None admin] RBAC: Adding query filter params (name=placement) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:37:03.342672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd6359-7902-4085-990a-bb5b7088812d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:03.344037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd6359-7902-4085-990a-bb5b7088812d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:03.349114 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 646/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Sat Sep 3 20:37:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:37:03.354956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160dca0-4a6e-46f7-b9b5-2d65d824bd9f None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:03.452675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160dca0-4a6e-46f7-b9b5-2d65d824bd9f None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:03.484641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2894e-7884-41a4-b4fa-d9aaf9e46569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:03.485262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2894e-7884-41a4-b4fa-d9aaf9e46569 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:03.485545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2894e-7884-41a4-b4fa-d9aaf9e46569 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:03.485821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2894e-7884-41a4-b4fa-d9aaf9e46569 None admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:03.486438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2894e-7884-41a4-b4fa-d9aaf9e46569 None admin] RBAC: Adding query filter params (type=placement) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:37:03.486739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2894e-7884-41a4-b4fa-d9aaf9e46569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:03.488089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2894e-7884-41a4-b4fa-d9aaf9e46569 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:03.492289 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 646/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Sat Sep 3 20:37:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:37:05.638388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167432dd-b797-4fb0-b08b-5e6b8a5ad454 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:05.638945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167432dd-b797-4fb0-b08b-5e6b8a5ad454 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:05.638945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167432dd-b797-4fb0-b08b-5e6b8a5ad454 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:05.640251 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 647/1293] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:37:05.645260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d4e86-55f5-4c7c-b524-6479dd869a98 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:05.645693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d4e86-55f5-4c7c-b524-6479dd869a98 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:05.645811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d4e86-55f5-4c7c-b524-6479dd869a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:05.663625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c11d4e86-55f5-4c7c-b524-6479dd869a98 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:37:05.664801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c11d4e86-55f5-4c7c-b524-6479dd869a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:37:05.694344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:37:05.738314 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 647/1294] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:37:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 20:37:06.072416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167432dd-b797-4fb0-b08b-5e6b8a5ad454 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:06.103032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:37:06.174119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167432dd-b797-4fb0-b08b-5e6b8a5ad454 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:06.207542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b71e08-16d2-41a1-9732-1bdab217a827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:06.208141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b71e08-16d2-41a1-9732-1bdab217a827 None admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:06.208675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b71e08-16d2-41a1-9732-1bdab217a827 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:06.208980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b71e08-16d2-41a1-9732-1bdab217a827 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:06.209639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b71e08-16d2-41a1-9732-1bdab217a827 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:06.210994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b71e08-16d2-41a1-9732-1bdab217a827 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:06.224926 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 648/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 3 20:37:06 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:37:08.399799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd5176-13bb-49e8-b96d-b610d5dfce92 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:08.400300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd5176-13bb-49e8-b96d-b610d5dfce92 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:08.400300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd5176-13bb-49e8-b96d-b610d5dfce92 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:08.401643 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 648/1296] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:37:08.406800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a819c99-53ef-4d60-a50b-b4544893ce49 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:08.407217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a819c99-53ef-4d60-a50b-b4544893ce49 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:08.407349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a819c99-53ef-4d60-a50b-b4544893ce49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:08.425486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7a819c99-53ef-4d60-a50b-b4544893ce49 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:37:08.426663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7a819c99-53ef-4d60-a50b-b4544893ce49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:37:08.497227 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 649/1297] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:37:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:37:08.831567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd5176-13bb-49e8-b96d-b610d5dfce92 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:08.931825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd5176-13bb-49e8-b96d-b610d5dfce92 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:08.964822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158de9f-7653-40f5-9944-6d66a79f7f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:08.965468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158de9f-7653-40f5-9944-6d66a79f7f27 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:08.965752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158de9f-7653-40f5-9944-6d66a79f7f27 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:08.966031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158de9f-7653-40f5-9944-6d66a79f7f27 None admin] PATH_INFO: `/v3/services/placement` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:08.966606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158de9f-7653-40f5-9944-6d66a79f7f27 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:08.967961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158de9f-7653-40f5-9944-6d66a79f7f27 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-7158de9f-7653-40f5-9944-6d66a79f7f27 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:37:08.976511 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:37:08.977677 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:37:08.978521 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 649/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 3 20:37:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:37:08.984161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a819c99-53ef-4d60-a50b-b4544893ce49 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:09.002654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:37:09.081786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a819c99-53ef-4d60-a50b-b4544893ce49 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:09.115437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a97032-ed4e-485e-8bb9-0906e66c48c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:09.116038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a97032-ed4e-485e-8bb9-0906e66c48c2 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:09.116324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a97032-ed4e-485e-8bb9-0906e66c48c2 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:09.116598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a97032-ed4e-485e-8bb9-0906e66c48c2 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:09.117259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a97032-ed4e-485e-8bb9-0906e66c48c2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:37:09.117567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a97032-ed4e-485e-8bb9-0906e66c48c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:09.118943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a97032-ed4e-485e-8bb9-0906e66c48c2 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:09.126306 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 650/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Sat Sep 3 20:37:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:37:09.133018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158de9f-7653-40f5-9944-6d66a79f7f27 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:09.164875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:37:09.233753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158de9f-7653-40f5-9944-6d66a79f7f27 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:09.266777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384bc16-9cff-4bbf-b688-8d1452959a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:09.267326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384bc16-9cff-4bbf-b688-8d1452959a61 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:09.267612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384bc16-9cff-4bbf-b688-8d1452959a61 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:09.267892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384bc16-9cff-4bbf-b688-8d1452959a61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:09.268581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384bc16-9cff-4bbf-b688-8d1452959a61 None admin] RBAC: Adding query filter params (interface=public, service_id=09e88fa7d6664c53876ee6db7122bb4a, region_id=RegionOne) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:37:09.268917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384bc16-9cff-4bbf-b688-8d1452959a61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:09.270278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384bc16-9cff-4bbf-b688-8d1452959a61 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:09.275125 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 650/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 3 20:37:09 2022] GET /identity/v3/endpoints?service_id=09e88fa7d6664c53876ee6db7122bb4a&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:37:09.280982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a97032-ed4e-485e-8bb9-0906e66c48c2 None admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:09.358507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:37:09.377456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a97032-ed4e-485e-8bb9-0906e66c48c2 None admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:09.411193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de5a7b-6c60-4a46-8965-4b1d56d9974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:09.411800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de5a7b-6c60-4a46-8965-4b1d56d9974d None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:09.412076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de5a7b-6c60-4a46-8965-4b1d56d9974d None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:09.412353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de5a7b-6c60-4a46-8965-4b1d56d9974d None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:09.412990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de5a7b-6c60-4a46-8965-4b1d56d9974d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:09.414363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de5a7b-6c60-4a46-8965-4b1d56d9974d None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:09.421035 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 651/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 3 20:37:09 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:37:11.590578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cb683-b1ba-48e9-bb3f-3b5871a2eca6 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:11.591115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cb683-b1ba-48e9-bb3f-3b5871a2eca6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:11.591115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cb683-b1ba-48e9-bb3f-3b5871a2eca6 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:11.592476 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 651/1302] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:37:11.597727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d69ed0-ce9c-46b6-b94a-a75a48759957 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:11.598157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d69ed0-ce9c-46b6-b94a-a75a48759957 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:11.598281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d69ed0-ce9c-46b6-b94a-a75a48759957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:11.615864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-14d69ed0-ce9c-46b6-b94a-a75a48759957 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:37:11.617095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-14d69ed0-ce9c-46b6-b94a-a75a48759957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:37:11.682459 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 652/1303] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:37:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 20:37:12.008088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cb683-b1ba-48e9-bb3f-3b5871a2eca6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:12.104976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:37:12.112734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cb683-b1ba-48e9-bb3f-3b5871a2eca6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:12.146286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c59567-1f12-4618-98c0-e48a593d6449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:12.146914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c59567-1f12-4618-98c0-e48a593d6449 None admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:12.147198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c59567-1f12-4618-98c0-e48a593d6449 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:12.147473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c59567-1f12-4618-98c0-e48a593d6449 None admin] PATH_INFO: `/v3/services/placement` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:12.148051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c59567-1f12-4618-98c0-e48a593d6449 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:12.149446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c59567-1f12-4618-98c0-e48a593d6449 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-69c59567-1f12-4618-98c0-e48a593d6449 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 20:37:12.158602 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 20:37:12.159779 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 20:37:12.161009 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 652/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 3 20:37:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 20:37:12.165712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d69ed0-ce9c-46b6-b94a-a75a48759957 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:12.238136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:37:12.260464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d69ed0-ce9c-46b6-b94a-a75a48759957 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:12.292590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e5060-b2c1-40dd-bbdb-6acceb207299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:12.293226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e5060-b2c1-40dd-bbdb-6acceb207299 None admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:12.293509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e5060-b2c1-40dd-bbdb-6acceb207299 None admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:12.293781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e5060-b2c1-40dd-bbdb-6acceb207299 None admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:12.294403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e5060-b2c1-40dd-bbdb-6acceb207299 None admin] RBAC: Adding query filter params (name=placement) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:37:12.294706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e5060-b2c1-40dd-bbdb-6acceb207299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:12.296055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e5060-b2c1-40dd-bbdb-6acceb207299 None admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:12.300955 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 653/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Sat Sep 3 20:37:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:37:12.306737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c59567-1f12-4618-98c0-e48a593d6449 None admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:37:12.410785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c59567-1f12-4618-98c0-e48a593d6449 None admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:37:12.445130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c05388-bf37-4dd7-89d2-1f9ea85e3617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:37:12.445729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c05388-bf37-4dd7-89d2-1f9ea85e3617 None admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:37:12.446013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c05388-bf37-4dd7-89d2-1f9ea85e3617 None admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:37:12.446291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c05388-bf37-4dd7-89d2-1f9ea85e3617 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:37:12.446959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c05388-bf37-4dd7-89d2-1f9ea85e3617 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:37:12.448328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c05388-bf37-4dd7-89d2-1f9ea85e3617 None admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:37:12.467857 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 653/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:37:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:37:22.298089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=30 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:37:22.463584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:24.818938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47504d-a893-4ea8-a785-2c72f7fbe740 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:24.819515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47504d-a893-4ea8-a785-2c72f7fbe740 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:24.819515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47504d-a893-4ea8-a785-2c72f7fbe740 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:24.825957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc1a0e-f301-4ea7-9706-ba3ed43a088e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:24.826398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc1a0e-f301-4ea7-9706-ba3ed43a088e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:24.826587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc1a0e-f301-4ea7-9706-ba3ed43a088e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:24.831315 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 654/1307] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:38:24 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:38:24.900770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4cfc1a0e-f301-4ea7-9706-ba3ed43a088e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:38:24.902228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4cfc1a0e-f301-4ea7-9706-ba3ed43a088e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:25.030765 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 654/1308] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:38:25.402115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12f849-8c4d-4763-9ce6-8fe8461843a3 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:25.402419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12f849-8c4d-4763-9ce6-8fe8461843a3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:25.402686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12f849-8c4d-4763-9ce6-8fe8461843a3 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:25.404171 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 655/1309] 10.222.0.16 () {54 vars in 965 bytes} [Sat Sep 3 20:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:38:25.410693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb687c7e-0346-453f-8eac-7c9658d3dea7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:25.411065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb687c7e-0346-453f-8eac-7c9658d3dea7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:25.411451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb687c7e-0346-453f-8eac-7c9658d3dea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:25.492500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bb687c7e-0346-453f-8eac-7c9658d3dea7 None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:38:25.493882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bb687c7e-0346-453f-8eac-7c9658d3dea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:25.520170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:25.579913 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 655/1310] 10.222.0.16 () {58 vars in 1062 bytes} [Sat Sep 3 20:38:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:38:25.589702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12f849-8c4d-4763-9ce6-8fe8461843a3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:38:25.591553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7a12f849-8c4d-4763-9ce6-8fe8461843a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:25.719454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12f849-8c4d-4763-9ce6-8fe8461843a3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:38:25.798173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff240f32-3715-4d15-a27d-7d7a272d8cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:38:25.798789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff240f32-3715-4d15-a27d-7d7a272d8cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:25.799075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff240f32-3715-4d15-a27d-7d7a272d8cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:25.799352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff240f32-3715-4d15-a27d-7d7a272d8cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:25.801003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ff240f32-3715-4d15-a27d-7d7a272d8cd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:25.861474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:25.926762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff240f32-3715-4d15-a27d-7d7a272d8cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:38:25.929416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff240f32-3715-4d15-a27d-7d7a272d8cd0 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:38:25.948359 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 656/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:38:28.400216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca607f7f-bc53-497e-b008-09f337d93eac None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:28.400833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca607f7f-bc53-497e-b008-09f337d93eac None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:28.400833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca607f7f-bc53-497e-b008-09f337d93eac None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:28.403705 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 656/1312] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:38:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:38:28.408656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516ffb9-3d73-43ea-adcb-7b8a7b9ad0be None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:28.409149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516ffb9-3d73-43ea-adcb-7b8a7b9ad0be None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:28.409235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516ffb9-3d73-43ea-adcb-7b8a7b9ad0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:28.494136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e516ffb9-3d73-43ea-adcb-7b8a7b9ad0be None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:38:28.496123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e516ffb9-3d73-43ea-adcb-7b8a7b9ad0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:28.598568 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 657/1313] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:38:28.939316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca607f7f-bc53-497e-b008-09f337d93eac None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:38:28.941015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ca607f7f-bc53-497e-b008-09f337d93eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:29.055275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca607f7f-bc53-497e-b008-09f337d93eac None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:38:29.137203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:29.139947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c8daa-015e-40f6-9109-027308c2d940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:38:29.140712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c8daa-015e-40f6-9109-027308c2d940 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:29.141621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c8daa-015e-40f6-9109-027308c2d940 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:29.142664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c8daa-015e-40f6-9109-027308c2d940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:29.144765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2d7c8daa-015e-40f6-9109-027308c2d940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:29.260479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c8daa-015e-40f6-9109-027308c2d940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:38:29.262270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c8daa-015e-40f6-9109-027308c2d940 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:38:29.281806 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 657/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:38:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:38:31.850527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d151c4-4f39-4492-ba4f-eaa0e32cb585 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:31.851083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d151c4-4f39-4492-ba4f-eaa0e32cb585 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:31.851083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d151c4-4f39-4492-ba4f-eaa0e32cb585 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:31.852427 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 658/1315] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:38:31.857684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2dd28-967c-42ae-9424-0b4cac8ad7ea None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:31.858075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2dd28-967c-42ae-9424-0b4cac8ad7ea None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:31.858248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2dd28-967c-42ae-9424-0b4cac8ad7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:31.927764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-81f2dd28-967c-42ae-9424-0b4cac8ad7ea None None] MFA Rules not processed for user `c2d9267d86294911a8b40405544b5a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:38:31.929027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-81f2dd28-967c-42ae-9424-0b4cac8ad7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:31.980565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:32.024621 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 658/1316] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:38:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:38:32.366662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d151c4-4f39-4492-ba4f-eaa0e32cb585 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:38:32.368217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e3d151c4-4f39-4492-ba4f-eaa0e32cb585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:32.378494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:32.471538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d151c4-4f39-4492-ba4f-eaa0e32cb585 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:38:32.542468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d475a0-a8ab-40ad-8507-ef8d8c1a6fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:38:32.543070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d475a0-a8ab-40ad-8507-ef8d8c1a6fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:32.543353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d475a0-a8ab-40ad-8507-ef8d8c1a6fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:32.543630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d475a0-a8ab-40ad-8507-ef8d8c1a6fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:32.545296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-04d475a0-a8ab-40ad-8507-ef8d8c1a6fa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:32.607474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:32.654358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d475a0-a8ab-40ad-8507-ef8d8c1a6fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:38:32.655812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d475a0-a8ab-40ad-8507-ef8d8c1a6fa0 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:38:32.673529 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 659/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:38:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:38:35.396835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52511914-874f-4dd6-bb4c-752456b3e046 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:35.397411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52511914-874f-4dd6-bb4c-752456b3e046 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:35.397411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52511914-874f-4dd6-bb4c-752456b3e046 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:35.398738 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 659/1318] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:38:35.404355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3608a8-cb61-4878-af1b-2f99f2046e26 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:35.404816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3608a8-cb61-4878-af1b-2f99f2046e26 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:35.404938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3608a8-cb61-4878-af1b-2f99f2046e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:35.469764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ab3608a8-cb61-4878-af1b-2f99f2046e26 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:38:35.470989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ab3608a8-cb61-4878-af1b-2f99f2046e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:35.564657 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 660/1319] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:38:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:38:35.910392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52511914-874f-4dd6-bb4c-752456b3e046 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:38:35.911908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-52511914-874f-4dd6-bb4c-752456b3e046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:36.013703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52511914-874f-4dd6-bb4c-752456b3e046 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:38:36.086081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f613132-193b-4427-88db-4fbefa1903ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:38:36.086685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f613132-193b-4427-88db-4fbefa1903ea service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:36.086971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f613132-193b-4427-88db-4fbefa1903ea service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:36.087247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f613132-193b-4427-88db-4fbefa1903ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:36.089415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2f613132-193b-4427-88db-4fbefa1903ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:36.096418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:36.194842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f613132-193b-4427-88db-4fbefa1903ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:38:36.196269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f613132-193b-4427-88db-4fbefa1903ea service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:38:36.215366 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 660/1320] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:38:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:38:39.046141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7d937-7de0-4abe-888c-858ca86b6d9e None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:39.046714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7d937-7de0-4abe-888c-858ca86b6d9e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:39.046714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7d937-7de0-4abe-888c-858ca86b6d9e None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:39.048041 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 661/1321] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:38:39.053120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3546458-eb1f-4465-87e4-8ffbb0e08092 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:39.053544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3546458-eb1f-4465-87e4-8ffbb0e08092 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:39.053686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3546458-eb1f-4465-87e4-8ffbb0e08092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:39.123475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d3546458-eb1f-4465-87e4-8ffbb0e08092 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:38:39.124717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d3546458-eb1f-4465-87e4-8ffbb0e08092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:39.183346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:39.223308 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 661/1322] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:38:39.573877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7d937-7de0-4abe-888c-858ca86b6d9e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:38:39.575422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-aed7d937-7de0-4abe-888c-858ca86b6d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:39.594951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:39.679466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7d937-7de0-4abe-888c-858ca86b6d9e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:38:39.757108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17953f92-fadb-4de8-a8ea-0ffcf78f9f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:38:39.757719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953f92-fadb-4de8-a8ea-0ffcf78f9f61 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:39.758009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953f92-fadb-4de8-a8ea-0ffcf78f9f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:39.758288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953f92-fadb-4de8-a8ea-0ffcf78f9f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:39.759903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-17953f92-fadb-4de8-a8ea-0ffcf78f9f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:39.831030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:39.869999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17953f92-fadb-4de8-a8ea-0ffcf78f9f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:38:39.871459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17953f92-fadb-4de8-a8ea-0ffcf78f9f61 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:38:39.891370 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 662/1323] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:38:42.144431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715ff77-5989-41ad-83d0-5218cf0a189d None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:42.144850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715ff77-5989-41ad-83d0-5218cf0a189d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:42.145052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715ff77-5989-41ad-83d0-5218cf0a189d None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:42.146383 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 662/1324] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:38:42.151642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afccd13-900f-4e18-b8fc-ae79042a85de None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:42.152117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afccd13-900f-4e18-b8fc-ae79042a85de None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:42.152329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afccd13-900f-4e18-b8fc-ae79042a85de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:42.218241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0afccd13-900f-4e18-b8fc-ae79042a85de None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:38:42.219482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0afccd13-900f-4e18-b8fc-ae79042a85de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:42.315611 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 663/1325] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:38:42.666400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715ff77-5989-41ad-83d0-5218cf0a189d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:38:42.668010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f715ff77-5989-41ad-83d0-5218cf0a189d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:42.774523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715ff77-5989-41ad-83d0-5218cf0a189d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:38:42.851326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd86eb-1890-457f-8505-b389905d9416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:38:42.851937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd86eb-1890-457f-8505-b389905d9416 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:42.852220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd86eb-1890-457f-8505-b389905d9416 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:42.852497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd86eb-1890-457f-8505-b389905d9416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:42.854170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2ccd86eb-1890-457f-8505-b389905d9416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:42.870843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:42.968193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd86eb-1890-457f-8505-b389905d9416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:38:42.969671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd86eb-1890-457f-8505-b389905d9416 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:38:42.990008 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 663/1326] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:38:45.226558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1355c9-0828-4ef4-8ca0-fafe56f2216c None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:45.227256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1355c9-0828-4ef4-8ca0-fafe56f2216c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:45.227256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1355c9-0828-4ef4-8ca0-fafe56f2216c None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:45.231687 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 664/1327] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:38:45.234332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c2db4-869f-4bd9-8f8a-1ed2f7b28870 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:45.234858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c2db4-869f-4bd9-8f8a-1ed2f7b28870 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:45.234972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c2db4-869f-4bd9-8f8a-1ed2f7b28870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:45.300356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-410c2db4-869f-4bd9-8f8a-1ed2f7b28870 None None] MFA Rules not processed for user `c2d9267d86294911a8b40405544b5a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:38:45.301647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-410c2db4-869f-4bd9-8f8a-1ed2f7b28870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:45.374859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:45.408240 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 664/1328] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:38:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:38:45.774376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cca5b-457d-41eb-a182-e8054af81548 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:45.774669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cca5b-457d-41eb-a182-e8054af81548 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:45.774950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cca5b-457d-41eb-a182-e8054af81548 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:45.776848 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 665/1329] 10.222.0.16 () {54 vars in 965 bytes} [Sat Sep 3 20:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:38:45.782852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701281d4-02a5-4287-be2b-3931892a966a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:45.783218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701281d4-02a5-4287-be2b-3931892a966a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:45.783593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701281d4-02a5-4287-be2b-3931892a966a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:45.861508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-701281d4-02a5-4287-be2b-3931892a966a None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:38:45.862719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-701281d4-02a5-4287-be2b-3931892a966a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:45.937195 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 665/1330] 10.222.0.16 () {58 vars in 1062 bytes} [Sat Sep 3 20:38:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:38:45.945570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cca5b-457d-41eb-a182-e8054af81548 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:38:45.947097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-487cca5b-457d-41eb-a182-e8054af81548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:45.978339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:46.053938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cca5b-457d-41eb-a182-e8054af81548 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:38:46.131330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa6b38-5404-4c01-8964-81ec658b159c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:38:46.131942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6b38-5404-4c01-8964-81ec658b159c service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:46.132226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6b38-5404-4c01-8964-81ec658b159c service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:46.132557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6b38-5404-4c01-8964-81ec658b159c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:46.134142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9cfa6b38-5404-4c01-8964-81ec658b159c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:46.214017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:46.246359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa6b38-5404-4c01-8964-81ec658b159c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:38:46.247832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa6b38-5404-4c01-8964-81ec658b159c service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:38:46.266214 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 666/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:38:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:38:49.076631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d995aea-ae2b-47b5-814e-5174b8a3cc74 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:49.077740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d995aea-ae2b-47b5-814e-5174b8a3cc74 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:49.078056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d995aea-ae2b-47b5-814e-5174b8a3cc74 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:49.079891 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 666/1332] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:38:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:38:49.085424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b0301-5d55-4d35-a04b-aeae72ef6b35 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:49.085910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b0301-5d55-4d35-a04b-aeae72ef6b35 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:49.085989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b0301-5d55-4d35-a04b-aeae72ef6b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:49.172218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-240b0301-5d55-4d35-a04b-aeae72ef6b35 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:38:49.173499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-240b0301-5d55-4d35-a04b-aeae72ef6b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:49.284494 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 667/1333] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:38:49.625918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d995aea-ae2b-47b5-814e-5174b8a3cc74 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:38:49.627482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2d995aea-ae2b-47b5-814e-5174b8a3cc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:49.696849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:49.741405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d995aea-ae2b-47b5-814e-5174b8a3cc74 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:38:49.814532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ab074-1f38-4aef-8685-756573c2d833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:38:49.815147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ab074-1f38-4aef-8685-756573c2d833 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:49.815439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ab074-1f38-4aef-8685-756573c2d833 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:49.815707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ab074-1f38-4aef-8685-756573c2d833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:49.817343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6c5ab074-1f38-4aef-8685-756573c2d833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:49.925435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ab074-1f38-4aef-8685-756573c2d833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:38:49.926804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ab074-1f38-4aef-8685-756573c2d833 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:38:49.937329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:49.948705 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 667/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:38:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:38:52.170039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d67e37-7c42-4f7e-8d10-b0717e9bc9ad None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:52.170550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d67e37-7c42-4f7e-8d10-b0717e9bc9ad None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:52.170646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d67e37-7c42-4f7e-8d10-b0717e9bc9ad None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:52.171935 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 668/1335] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:38:52.176983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143838ff-fd79-4855-bdb1-08c19577521d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:52.177408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143838ff-fd79-4855-bdb1-08c19577521d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:52.177548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143838ff-fd79-4855-bdb1-08c19577521d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:52.243049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-143838ff-fd79-4855-bdb1-08c19577521d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:38:52.244321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-143838ff-fd79-4855-bdb1-08c19577521d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:52.339395 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 668/1336] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:38:52.689218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d67e37-7c42-4f7e-8d10-b0717e9bc9ad None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:38:52.690764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-61d67e37-7c42-4f7e-8d10-b0717e9bc9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:52.731173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:52.796237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d67e37-7c42-4f7e-8d10-b0717e9bc9ad None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:38:52.867343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1266a-25e2-4414-b640-783328f6476b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:38:52.867952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1266a-25e2-4414-b640-783328f6476b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:52.868242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1266a-25e2-4414-b640-783328f6476b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:52.868522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1266a-25e2-4414-b640-783328f6476b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:52.870697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-00e1266a-25e2-4414-b640-783328f6476b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:52.954274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:52.983498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1266a-25e2-4414-b640-783328f6476b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:38:52.985021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1266a-25e2-4414-b640-783328f6476b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:38:53.005885 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 669/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:38:55.605134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e83ec4-f8bc-4a1b-bc2d-8f63d7959060 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:55.605642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e83ec4-f8bc-4a1b-bc2d-8f63d7959060 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:55.605642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e83ec4-f8bc-4a1b-bc2d-8f63d7959060 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:55.606950 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 669/1338] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:38:55.611999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4287146-f701-40bb-8f0a-b71d00130671 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:55.612442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4287146-f701-40bb-8f0a-b71d00130671 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:55.612537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4287146-f701-40bb-8f0a-b71d00130671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:55.676980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e4287146-f701-40bb-8f0a-b71d00130671 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:38:55.678179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e4287146-f701-40bb-8f0a-b71d00130671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:55.779957 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 670/1339] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:38:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:38:56.118255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e83ec4-f8bc-4a1b-bc2d-8f63d7959060 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:38:56.120378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-64e83ec4-f8bc-4a1b-bc2d-8f63d7959060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:56.197188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:38:56.224789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e83ec4-f8bc-4a1b-bc2d-8f63d7959060 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:38:56.296504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ab90-5472-4136-9a15-8d612afc0d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:38:56.297181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ab90-5472-4136-9a15-8d612afc0d05 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:38:56.297469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ab90-5472-4136-9a15-8d612afc0d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:38:56.297755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ab90-5472-4136-9a15-8d612afc0d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:38:56.299372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f9f0ab90-5472-4136-9a15-8d612afc0d05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:38:56.406805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0ab90-5472-4136-9a15-8d612afc0d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:38:56.408221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0ab90-5472-4136-9a15-8d612afc0d05 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:38:56.428349 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 670/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:38:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:00.629006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbd570-88da-42f3-8039-5bcd942b6434 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:00.629558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbd570-88da-42f3-8039-5bcd942b6434 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:00.629558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbd570-88da-42f3-8039-5bcd942b6434 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:00.630795 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 671/1341] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:00.635939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e04b37-6053-4503-a465-9604cc5467d9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:00.636395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e04b37-6053-4503-a465-9604cc5467d9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:00.636492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e04b37-6053-4503-a465-9604cc5467d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:00.642496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:00.704837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c5e04b37-6053-4503-a465-9604cc5467d9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:00.706242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c5e04b37-6053-4503-a465-9604cc5467d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:00.802326 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 671/1342] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:01.145725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbd570-88da-42f3-8039-5bcd942b6434 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:01.147343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1edbd570-88da-42f3-8039-5bcd942b6434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:01.195191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:01.250679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbd570-88da-42f3-8039-5bcd942b6434 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:01.325608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b52a9-8f23-49e1-bdc2-4a763eeb3023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:01.326218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b52a9-8f23-49e1-bdc2-4a763eeb3023 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:01.326506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b52a9-8f23-49e1-bdc2-4a763eeb3023 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:01.326785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b52a9-8f23-49e1-bdc2-4a763eeb3023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:01.328479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-951b52a9-8f23-49e1-bdc2-4a763eeb3023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:01.422332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:01.439819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b52a9-8f23-49e1-bdc2-4a763eeb3023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:01.441318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b52a9-8f23-49e1-bdc2-4a763eeb3023 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:01.461597 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 672/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:04.294289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbcb1e-369a-46e6-ac17-9e66397be9f8 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:04.294852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbcb1e-369a-46e6-ac17-9e66397be9f8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:04.294852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbcb1e-369a-46e6-ac17-9e66397be9f8 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:04.296185 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 672/1344] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:04.301349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5581c9c-d70d-4d11-950c-1e4ee62111e1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:04.301792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5581c9c-d70d-4d11-950c-1e4ee62111e1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:04.302060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5581c9c-d70d-4d11-950c-1e4ee62111e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:04.369630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b5581c9c-d70d-4d11-950c-1e4ee62111e1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:04.370842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b5581c9c-d70d-4d11-950c-1e4ee62111e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:04.474549 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 673/1345] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:04.828706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbcb1e-369a-46e6-ac17-9e66397be9f8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:04.830327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-92cbcb1e-369a-46e6-ac17-9e66397be9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:04.917678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:04.935964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbcb1e-369a-46e6-ac17-9e66397be9f8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:05.012841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef526194-7b61-44d7-b78e-743e8639ba42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:05.013472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef526194-7b61-44d7-b78e-743e8639ba42 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:05.013760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef526194-7b61-44d7-b78e-743e8639ba42 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:05.014040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef526194-7b61-44d7-b78e-743e8639ba42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:05.015654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ef526194-7b61-44d7-b78e-743e8639ba42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:05.129995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef526194-7b61-44d7-b78e-743e8639ba42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:05.131366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef526194-7b61-44d7-b78e-743e8639ba42 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:05.149777 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 673/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:09.577736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528e649-bbfb-4957-96c1-c76d50b24c14 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:09.578252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528e649-bbfb-4957-96c1-c76d50b24c14 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:09.578252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528e649-bbfb-4957-96c1-c76d50b24c14 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:09.579573 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 674/1347] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:09.586136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfc2b0-b6aa-40c9-a3e7-e14b25b5ef02 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:09.586526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfc2b0-b6aa-40c9-a3e7-e14b25b5ef02 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:09.586702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfc2b0-b6aa-40c9-a3e7-e14b25b5ef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:09.602154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:09.654281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-31dfc2b0-b6aa-40c9-a3e7-e14b25b5ef02 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:09.655530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-31dfc2b0-b6aa-40c9-a3e7-e14b25b5ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:09.758707 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 674/1348] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:39:10.112574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528e649-bbfb-4957-96c1-c76d50b24c14 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:10.114218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a528e649-bbfb-4957-96c1-c76d50b24c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:10.179421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:10.230307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528e649-bbfb-4957-96c1-c76d50b24c14 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:10.307831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f918d-ac11-4288-97cd-42bfba921011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:10.308357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f918d-ac11-4288-97cd-42bfba921011 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:10.308648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f918d-ac11-4288-97cd-42bfba921011 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:10.308953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f918d-ac11-4288-97cd-42bfba921011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:10.310594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-822f918d-ac11-4288-97cd-42bfba921011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:10.418784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:10.422908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f918d-ac11-4288-97cd-42bfba921011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:10.424375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f918d-ac11-4288-97cd-42bfba921011 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:10.442341 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 675/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:15.996114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab88c0-acc9-4c1d-9100-4dbc95e5c078 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:15.996604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab88c0-acc9-4c1d-9100-4dbc95e5c078 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:15.996826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab88c0-acc9-4c1d-9100-4dbc95e5c078 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:15.998491 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 675/1350] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:16.003367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12947d34-a0b3-445d-bf66-69ea6e7f5134 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:16.003787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12947d34-a0b3-445d-bf66-69ea6e7f5134 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:16.003978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12947d34-a0b3-445d-bf66-69ea6e7f5134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:16.088381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-12947d34-a0b3-445d-bf66-69ea6e7f5134 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:16.089961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-12947d34-a0b3-445d-bf66-69ea6e7f5134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:16.213498 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 676/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:39:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:19.756812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=59 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:20.781042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7a79f-725c-410e-b3a0-8fe8c1211e88 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:20.782053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7a79f-725c-410e-b3a0-8fe8c1211e88 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:20.782395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7a79f-725c-410e-b3a0-8fe8c1211e88 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:20.784023 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 676/1352] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:39:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:20.789053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2219f-c6b2-456f-b44a-d5350ba8a6f8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:20.789476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2219f-c6b2-456f-b44a-d5350ba8a6f8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:20.789641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2219f-c6b2-456f-b44a-d5350ba8a6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:20.849614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:20.877745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b9d2219f-c6b2-456f-b44a-d5350ba8a6f8 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:20.878995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b9d2219f-c6b2-456f-b44a-d5350ba8a6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:20.996759 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 677/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:39:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:22.271037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdbf73-7301-495f-9217-6a2e386768a8 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:22.271655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdbf73-7301-495f-9217-6a2e386768a8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:22.271809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdbf73-7301-495f-9217-6a2e386768a8 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:22.273601 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 677/1354] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:22.279234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac059536-e476-46a8-bca6-92e0ba218c0e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:22.279698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac059536-e476-46a8-bca6-92e0ba218c0e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:22.279780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac059536-e476-46a8-bca6-92e0ba218c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:22.354820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ac059536-e476-46a8-bca6-92e0ba218c0e None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:22.356018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ac059536-e476-46a8-bca6-92e0ba218c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:22.398994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:22.465646 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 678/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:39:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:26.239746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52f66c-b602-43a2-98dd-dcc54b97bfb1 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:26.240180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52f66c-b602-43a2-98dd-dcc54b97bfb1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:26.240294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52f66c-b602-43a2-98dd-dcc54b97bfb1 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:26.245881 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 678/1356] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:26.250117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386bfb8-79bc-445e-9c66-f6980409be94 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:26.251664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386bfb8-79bc-445e-9c66-f6980409be94 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:26.252321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386bfb8-79bc-445e-9c66-f6980409be94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:26.338977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c386bfb8-79bc-445e-9c66-f6980409be94 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:26.340571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c386bfb8-79bc-445e-9c66-f6980409be94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:26.442401 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 679/1357] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:26.807669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77dcec-1626-4564-9f7e-fe9d9639e181 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:26.808411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77dcec-1626-4564-9f7e-fe9d9639e181 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:26.809152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77dcec-1626-4564-9f7e-fe9d9639e181 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:26.811333 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 679/1358] 10.222.0.16 () {54 vars in 962 bytes} [Sat Sep 3 20:39:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:26.818361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b6a7-2c3b-470e-ad24-b1c578aa21f6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:26.818652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b6a7-2c3b-470e-ad24-b1c578aa21f6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:26.818931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b6a7-2c3b-470e-ad24-b1c578aa21f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:26.837694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:26.907613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6912b6a7-2c3b-470e-ad24-b1c578aa21f6 None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:26.908975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6912b6a7-2c3b-470e-ad24-b1c578aa21f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:27.039584 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 680/1359] 10.222.0.16 () {58 vars in 1059 bytes} [Sat Sep 3 20:39:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:39:27.048790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77dcec-1626-4564-9f7e-fe9d9639e181 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:27.051465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-aa77dcec-1626-4564-9f7e-fe9d9639e181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:27.217358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77dcec-1626-4564-9f7e-fe9d9639e181 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:27.296464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:27.345042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e3f7d-f1e8-474c-b421-7ff934d5f3b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:27.345750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e3f7d-f1e8-474c-b421-7ff934d5f3b2 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:27.346110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e3f7d-f1e8-474c-b421-7ff934d5f3b2 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:27.346469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e3f7d-f1e8-474c-b421-7ff934d5f3b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:27.348263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3b1e3f7d-f1e8-474c-b421-7ff934d5f3b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:27.446248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e3f7d-f1e8-474c-b421-7ff934d5f3b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:27.447971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e3f7d-f1e8-474c-b421-7ff934d5f3b2 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:27.467864 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 680/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:39:29.908373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adebf9-f0ec-4466-8ee0-3df49a2fb7bc None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:29.908827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adebf9-f0ec-4466-8ee0-3df49a2fb7bc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:29.908999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adebf9-f0ec-4466-8ee0-3df49a2fb7bc None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:29.910350 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 681/1361] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:29.915814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f774f-e3c1-42da-ac16-b1835a379a63 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:29.916277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f774f-e3c1-42da-ac16-b1835a379a63 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:29.916358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f774f-e3c1-42da-ac16-b1835a379a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:29.997231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2b8f774f-e3c1-42da-ac16-b1835a379a63 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:29.998576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2b8f774f-e3c1-42da-ac16-b1835a379a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:30.056464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:30.136936 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 681/1362] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:30.496436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adebf9-f0ec-4466-8ee0-3df49a2fb7bc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:30.498178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-02adebf9-f0ec-4466-8ee0-3df49a2fb7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:30.565054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:30.654040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adebf9-f0ec-4466-8ee0-3df49a2fb7bc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:30.780598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591c620-7c74-4620-aff1-8f8e7593909c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:30.781242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591c620-7c74-4620-aff1-8f8e7593909c service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:30.781522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591c620-7c74-4620-aff1-8f8e7593909c service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:30.781798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591c620-7c74-4620-aff1-8f8e7593909c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:30.783404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7591c620-7c74-4620-aff1-8f8e7593909c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:30.806926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:30.882015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591c620-7c74-4620-aff1-8f8e7593909c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:30.883951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591c620-7c74-4620-aff1-8f8e7593909c service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:30.903922 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 682/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:33.450468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b1dc7-c787-473e-98a5-bb32d0e2cf9a None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:33.451010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b1dc7-c787-473e-98a5-bb32d0e2cf9a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:33.451010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b1dc7-c787-473e-98a5-bb32d0e2cf9a None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:33.452368 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 682/1364] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:33.458242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce6716-7c81-4a1a-b0e8-636f915ea388 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:33.458662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce6716-7c81-4a1a-b0e8-636f915ea388 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:33.458805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce6716-7c81-4a1a-b0e8-636f915ea388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:33.553434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7bce6716-7c81-4a1a-b0e8-636f915ea388 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:33.554771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7bce6716-7c81-4a1a-b0e8-636f915ea388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:33.657310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:33.669625 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 683/1365] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:34.016484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6988b-24b2-4694-80f6-3ddaae50c372 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:34.017259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6988b-24b2-4694-80f6-3ddaae50c372 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:34.017393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6988b-24b2-4694-80f6-3ddaae50c372 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:34.018743 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 683/1366] 10.222.0.16 () {54 vars in 962 bytes} [Sat Sep 3 20:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:34.025392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9db8d1-b784-424f-b83d-15307657a3df None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:34.025757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9db8d1-b784-424f-b83d-15307657a3df None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:34.026109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9db8d1-b784-424f-b83d-15307657a3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:34.107411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3b9db8d1-b784-424f-b83d-15307657a3df None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:34.108705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3b9db8d1-b784-424f-b83d-15307657a3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:34.239433 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 684/1367] 10.222.0.16 () {58 vars in 1059 bytes} [Sat Sep 3 20:39:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:34.250673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6988b-24b2-4694-80f6-3ddaae50c372 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:34.250905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ffe6988b-24b2-4694-80f6-3ddaae50c372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:34.466187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6988b-24b2-4694-80f6-3ddaae50c372 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:34.491566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:34.608334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfb6d0-f8ba-45db-98c7-52156e9ceeb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:34.608877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfb6d0-f8ba-45db-98c7-52156e9ceeb8 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:34.609198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfb6d0-f8ba-45db-98c7-52156e9ceeb8 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:34.609478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfb6d0-f8ba-45db-98c7-52156e9ceeb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:34.611104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f7cfb6d0-f8ba-45db-98c7-52156e9ceeb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:34.729077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfb6d0-f8ba-45db-98c7-52156e9ceeb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:34.730841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfb6d0-f8ba-45db-98c7-52156e9ceeb8 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:34.751859 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 684/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:37.231069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e4ced-337e-44f7-b38b-95abd9efcabe None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:37.231691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e4ced-337e-44f7-b38b-95abd9efcabe None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:37.231691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e4ced-337e-44f7-b38b-95abd9efcabe None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:37.233225 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 685/1369] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:37.238491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4b484-fba8-4e18-8d96-a4b243a7160f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:37.239362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4b484-fba8-4e18-8d96-a4b243a7160f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:37.239782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4b484-fba8-4e18-8d96-a4b243a7160f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:37.273538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:37.324518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d3a4b484-fba8-4e18-8d96-a4b243a7160f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:37.325804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d3a4b484-fba8-4e18-8d96-a4b243a7160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:37.494865 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 685/1370] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:37.872118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e4ced-337e-44f7-b38b-95abd9efcabe None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:37.874124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a66e4ced-337e-44f7-b38b-95abd9efcabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:37.938848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:38.065002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e4ced-337e-44f7-b38b-95abd9efcabe None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:38.190011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ab07b-1f9a-4fee-a43e-ae35c9f8f725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:38.191118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ab07b-1f9a-4fee-a43e-ae35c9f8f725 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:38.191634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ab07b-1f9a-4fee-a43e-ae35c9f8f725 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:38.192144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ab07b-1f9a-4fee-a43e-ae35c9f8f725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:38.194067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9e0ab07b-1f9a-4fee-a43e-ae35c9f8f725 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:38.203945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:38.293047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ab07b-1f9a-4fee-a43e-ae35c9f8f725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:38.294664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ab07b-1f9a-4fee-a43e-ae35c9f8f725 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:38.325114 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 686/1371] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:39.308979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267c8a9-2405-4ce1-84d5-62ee9bcfa5e3 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:39.309936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267c8a9-2405-4ce1-84d5-62ee9bcfa5e3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:39.309936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267c8a9-2405-4ce1-84d5-62ee9bcfa5e3 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:39.310942 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 686/1372] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:39.316417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71994fb5-80e0-4f83-b720-b9ff350e77c6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:39.316935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71994fb5-80e0-4f83-b720-b9ff350e77c6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:39.317036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71994fb5-80e0-4f83-b720-b9ff350e77c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:39.386662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-71994fb5-80e0-4f83-b720-b9ff350e77c6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:39.388363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-71994fb5-80e0-4f83-b720-b9ff350e77c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:39.473087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:39.495301 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 687/1373] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:40.160829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654e156-e05b-40ee-81dd-d6169bd6f60b None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:40.161242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654e156-e05b-40ee-81dd-d6169bd6f60b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:40.161599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654e156-e05b-40ee-81dd-d6169bd6f60b None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:40.163381 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 687/1374] 10.222.0.16 () {54 vars in 964 bytes} [Sat Sep 3 20:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:40.169463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87517e14-1878-4e2e-90e7-929e19bc3dba None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:40.169864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87517e14-1878-4e2e-90e7-929e19bc3dba None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:40.170245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87517e14-1878-4e2e-90e7-929e19bc3dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:40.261530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-87517e14-1878-4e2e-90e7-929e19bc3dba None None] MFA Rules not processed for user `133a1ff64e224593be467eb7c27cbb09`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:40.262661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-87517e14-1878-4e2e-90e7-929e19bc3dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:40.346975 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 688/1375] 10.222.0.16 () {58 vars in 1061 bytes} [Sat Sep 3 20:39:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:40.354795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654e156-e05b-40ee-81dd-d6169bd6f60b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:40.356387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9654e156-e05b-40ee-81dd-d6169bd6f60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:40.469658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654e156-e05b-40ee-81dd-d6169bd6f60b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:40.474119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:40.560657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec108b28-e2ed-4fe0-8614-463191362036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:40.561473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec108b28-e2ed-4fe0-8614-463191362036 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:40.561833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec108b28-e2ed-4fe0-8614-463191362036 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:40.562188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec108b28-e2ed-4fe0-8614-463191362036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:40.564119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ec108b28-e2ed-4fe0-8614-463191362036 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:40.692286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec108b28-e2ed-4fe0-8614-463191362036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:40.694410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec108b28-e2ed-4fe0-8614-463191362036 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:40.722287 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 688/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:40.739655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7eefe-b1cc-40e7-9ea5-c3d44552064b None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:40.739954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7eefe-b1cc-40e7-9ea5-c3d44552064b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:40.740260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7eefe-b1cc-40e7-9ea5-c3d44552064b None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:40.741768 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 689/1377] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:40.747340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d84257-2dbd-4c2c-ac4e-12ce3f9120af None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:40.747633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d84257-2dbd-4c2c-ac4e-12ce3f9120af None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:40.747914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d84257-2dbd-4c2c-ac4e-12ce3f9120af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:40.799638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:40.826784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-74d84257-2dbd-4c2c-ac4e-12ce3f9120af None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:40.828282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-74d84257-2dbd-4c2c-ac4e-12ce3f9120af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:40.967963 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 689/1378] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:41.331977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7eefe-b1cc-40e7-9ea5-c3d44552064b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:41.333655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0da7eefe-b1cc-40e7-9ea5-c3d44552064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:41.435996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:41.526143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7eefe-b1cc-40e7-9ea5-c3d44552064b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:41.646558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884f324-a3d4-466c-b9ca-9545aabe76bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:41.647163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884f324-a3d4-466c-b9ca-9545aabe76bd service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:41.647446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884f324-a3d4-466c-b9ca-9545aabe76bd service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:41.647719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884f324-a3d4-466c-b9ca-9545aabe76bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:41.649398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5884f324-a3d4-466c-b9ca-9545aabe76bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:41.690994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:41.739392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884f324-a3d4-466c-b9ca-9545aabe76bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:41.741032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884f324-a3d4-466c-b9ca-9545aabe76bd service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:41.759438 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 690/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:44.077217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812603e-b62f-4f9c-a95a-ddb3d003fe43 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:44.077810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812603e-b62f-4f9c-a95a-ddb3d003fe43 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:44.077810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812603e-b62f-4f9c-a95a-ddb3d003fe43 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:44.079135 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 690/1380] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:44.084283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67011202-ff21-440c-8252-82c29edef999 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:44.085011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67011202-ff21-440c-8252-82c29edef999 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:44.085141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67011202-ff21-440c-8252-82c29edef999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:44.157923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-67011202-ff21-440c-8252-82c29edef999 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:44.159268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-67011202-ff21-440c-8252-82c29edef999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:44.280993 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 691/1381] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:44.625673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812603e-b62f-4f9c-a95a-ddb3d003fe43 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:44.627202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1812603e-b62f-4f9c-a95a-ddb3d003fe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:44.754008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:44.804138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812603e-b62f-4f9c-a95a-ddb3d003fe43 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:44.938471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82572731-c9af-416b-b152-df01f4baa5df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:44.939034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82572731-c9af-416b-b152-df01f4baa5df service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:44.939321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82572731-c9af-416b-b152-df01f4baa5df service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:44.939601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82572731-c9af-416b-b152-df01f4baa5df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:44.941522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-82572731-c9af-416b-b152-df01f4baa5df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:45.033638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:45.053805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82572731-c9af-416b-b152-df01f4baa5df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:45.056374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82572731-c9af-416b-b152-df01f4baa5df service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:45.080216 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 691/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:47.341537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab668bf1-c899-4340-82db-67612daff048 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:47.342182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab668bf1-c899-4340-82db-67612daff048 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:47.342182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab668bf1-c899-4340-82db-67612daff048 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:47.343729 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 692/1383] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:47.349733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c206d-cc95-4c5a-84f6-69aa3a7b2677 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:47.350207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c206d-cc95-4c5a-84f6-69aa3a7b2677 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:47.350294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c206d-cc95-4c5a-84f6-69aa3a7b2677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:47.421591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6d0c206d-cc95-4c5a-84f6-69aa3a7b2677 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:47.422914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6d0c206d-cc95-4c5a-84f6-69aa3a7b2677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:47.525806 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 692/1384] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:47.882958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab668bf1-c899-4340-82db-67612daff048 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:47.884529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ab668bf1-c899-4340-82db-67612daff048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:47.905063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:48.051804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab668bf1-c899-4340-82db-67612daff048 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:48.169936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:48.196209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f5052-e5e2-4443-95a7-3055497bec19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:48.196819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f5052-e5e2-4443-95a7-3055497bec19 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:48.197175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f5052-e5e2-4443-95a7-3055497bec19 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:48.197464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f5052-e5e2-4443-95a7-3055497bec19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:48.199356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-374f5052-e5e2-4443-95a7-3055497bec19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:48.297810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f5052-e5e2-4443-95a7-3055497bec19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:48.299501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f5052-e5e2-4443-95a7-3055497bec19 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:48.320260 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 693/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:50.859273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fc003-1c5b-4185-ac8b-f201c163ad03 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:50.859945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fc003-1c5b-4185-ac8b-f201c163ad03 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:50.859945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fc003-1c5b-4185-ac8b-f201c163ad03 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:50.861319 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 693/1386] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:50.866804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112c68-8229-4f94-9ea5-af7a2f493bc3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:50.867263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112c68-8229-4f94-9ea5-af7a2f493bc3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:50.867444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112c68-8229-4f94-9ea5-af7a2f493bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:50.948277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9c112c68-8229-4f94-9ea5-af7a2f493bc3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:50.949856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9c112c68-8229-4f94-9ea5-af7a2f493bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:51.003928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:51.058196 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 694/1387] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:51.412498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fc003-1c5b-4185-ac8b-f201c163ad03 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:51.414110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-069fc003-1c5b-4185-ac8b-f201c163ad03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:51.472163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:51.585134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fc003-1c5b-4185-ac8b-f201c163ad03 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:51.717285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f41ab-14e8-447e-b014-8818a96f77b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:51.717907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f41ab-14e8-447e-b014-8818a96f77b1 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:51.718191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f41ab-14e8-447e-b014-8818a96f77b1 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:51.718467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f41ab-14e8-447e-b014-8818a96f77b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:51.720078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-940f41ab-14e8-447e-b014-8818a96f77b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:51.733037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:51.820287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f41ab-14e8-447e-b014-8818a96f77b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:51.822412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f41ab-14e8-447e-b014-8818a96f77b1 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:51.841191 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 694/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:54.226546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24695c09-0ea0-40b4-90ec-93b2b64be5e6 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:54.227308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24695c09-0ea0-40b4-90ec-93b2b64be5e6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:54.227308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24695c09-0ea0-40b4-90ec-93b2b64be5e6 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:54.228571 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 695/1389] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:54.233875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fffdf-61d3-4aaf-b2f9-8dce0867b5fa None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:54.234400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fffdf-61d3-4aaf-b2f9-8dce0867b5fa None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:54.234400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fffdf-61d3-4aaf-b2f9-8dce0867b5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:54.300638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e19fffdf-61d3-4aaf-b2f9-8dce0867b5fa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:54.309097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e19fffdf-61d3-4aaf-b2f9-8dce0867b5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:54.409351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:54.421499 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 695/1390] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:54.793373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24695c09-0ea0-40b4-90ec-93b2b64be5e6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:54.794989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-24695c09-0ea0-40b4-90ec-93b2b64be5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:54.950652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24695c09-0ea0-40b4-90ec-93b2b64be5e6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:54.989517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:55.092527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc46104-3c97-4741-b95d-c72828631f14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:55.093095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc46104-3c97-4741-b95d-c72828631f14 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:55.093384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc46104-3c97-4741-b95d-c72828631f14 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:55.093679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc46104-3c97-4741-b95d-c72828631f14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:55.095315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-acc46104-3c97-4741-b95d-c72828631f14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:55.191227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc46104-3c97-4741-b95d-c72828631f14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:55.192844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc46104-3c97-4741-b95d-c72828631f14 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:55.212208 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 696/1391] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:39:57.330205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad86ffe-ac21-4f3a-a03a-62f5212b9415 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:57.330820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad86ffe-ac21-4f3a-a03a-62f5212b9415 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:57.330820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad86ffe-ac21-4f3a-a03a-62f5212b9415 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:57.332041 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 696/1392] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:57.338524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c0e81-d0fb-4b94-9dbe-a64446e396c2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:57.339982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c0e81-d0fb-4b94-9dbe-a64446e396c2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:57.339982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c0e81-d0fb-4b94-9dbe-a64446e396c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:57.399860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:57.414294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-388c0e81-d0fb-4b94-9dbe-a64446e396c2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:57.416674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-388c0e81-d0fb-4b94-9dbe-a64446e396c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:57.428558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b8e50-7c0f-48fe-ab1f-7d13fb820edd None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:57.428841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b8e50-7c0f-48fe-ab1f-7d13fb820edd None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:57.429154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b8e50-7c0f-48fe-ab1f-7d13fb820edd None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:57.430540 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 697/1393] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:57.435425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8de625-cb42-4be6-ad92-b48d06af77d9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:57.435713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8de625-cb42-4be6-ad92-b48d06af77d9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:57.435992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8de625-cb42-4be6-ad92-b48d06af77d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:57.506253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4e8de625-cb42-4be6-ad92-b48d06af77d9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:57.507480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4e8de625-cb42-4be6-ad92-b48d06af77d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:57.516154 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 697/1394] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:57.604242 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 698/1395] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:39:57.880475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb34ac-deaf-404c-8688-e5d8a8d7ba36 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:57.880768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb34ac-deaf-404c-8688-e5d8a8d7ba36 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:57.881080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb34ac-deaf-404c-8688-e5d8a8d7ba36 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:57.882426 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 698/1396] 10.222.0.16 () {54 vars in 961 bytes} [Sat Sep 3 20:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:57.888361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354132e-fab1-4c6c-85b4-234fab260f09 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:57.888730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354132e-fab1-4c6c-85b4-234fab260f09 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:57.889086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354132e-fab1-4c6c-85b4-234fab260f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:57.959470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb34ac-deaf-404c-8688-e5d8a8d7ba36 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:57.961300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0adb34ac-deaf-404c-8688-e5d8a8d7ba36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:57.966337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b354132e-fab1-4c6c-85b4-234fab260f09 None None] MFA Rules not processed for user `7a36c3c09b2d425fbf6ccbe051c293d4`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:57.967557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b354132e-fab1-4c6c-85b4-234fab260f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:57.980171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:58.053589 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 699/1397] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Sep 3 20:39:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:58.061668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354132e-fab1-4c6c-85b4-234fab260f09 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:58.069231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354132e-fab1-4c6c-85b4-234fab260f09 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:58.084370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:58.114583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb34ac-deaf-404c-8688-e5d8a8d7ba36 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:58.148130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240a1a5-2ced-4a01-bd31-bec3944958df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:58.148857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240a1a5-2ced-4a01-bd31-bec3944958df service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:58.149769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240a1a5-2ced-4a01-bd31-bec3944958df service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:58.150143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240a1a5-2ced-4a01-bd31-bec3944958df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:58.151866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e240a1a5-2ced-4a01-bd31-bec3944958df service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:58.234963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:58.237305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461c938-b630-4005-8996-456741cf60e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:58.237995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461c938-b630-4005-8996-456741cf60e0 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:58.238369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461c938-b630-4005-8996-456741cf60e0 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:58.238719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461c938-b630-4005-8996-456741cf60e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:58.240404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6461c938-b630-4005-8996-456741cf60e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:58.265019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240a1a5-2ced-4a01-bd31-bec3944958df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:58.266499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240a1a5-2ced-4a01-bd31-bec3944958df service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:58.284950 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 700/1398] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:39:58.321719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:58.331957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461c938-b630-4005-8996-456741cf60e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:58.333629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461c938-b630-4005-8996-456741cf60e0 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:58.352538 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 699/1399] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:39:58.885309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09820639-7bc7-49a9-9727-45459dab020c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:58.885817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09820639-7bc7-49a9-9727-45459dab020c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:58.885923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09820639-7bc7-49a9-9727-45459dab020c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:58.952017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-09820639-7bc7-49a9-9727-45459dab020c None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:58.953277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-09820639-7bc7-49a9-9727-45459dab020c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:59.043129 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 701/1400] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:39:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:59.091700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b282ac-9479-49e3-91f6-e248bc58c7ea None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:59.091994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b282ac-9479-49e3-91f6-e248bc58c7ea None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:59.092309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b282ac-9479-49e3-91f6-e248bc58c7ea None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:59.093742 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 700/1401] 10.222.0.16 () {54 vars in 943 bytes} [Sat Sep 3 20:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:39:59.099936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99b7b3-c192-4c08-8e70-656430c41f4f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:59.100327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99b7b3-c192-4c08-8e70-656430c41f4f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:59.100672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99b7b3-c192-4c08-8e70-656430c41f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:59.166753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:59.181095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0f99b7b3-c192-4c08-8e70-656430c41f4f None None] MFA Rules not processed for user `83865463b2a84f99a136b73dffeeb378`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:39:59.182300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0f99b7b3-c192-4c08-8e70-656430c41f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:59.286316 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 702/1402] 10.222.0.16 () {58 vars in 1040 bytes} [Sat Sep 3 20:39:59 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:39:59.296166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b282ac-9479-49e3-91f6-e248bc58c7ea None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:39:59.297719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-96b282ac-9479-49e3-91f6-e248bc58c7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:59.450003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b282ac-9479-49e3-91f6-e248bc58c7ea None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:39:59.518986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:39:59.558242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd497548-67a2-4260-9c03-99aa8c29d8a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:39:59.558943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd497548-67a2-4260-9c03-99aa8c29d8a7 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:39:59.559308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd497548-67a2-4260-9c03-99aa8c29d8a7 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:39:59.559664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd497548-67a2-4260-9c03-99aa8c29d8a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:39:59.561567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cd497548-67a2-4260-9c03-99aa8c29d8a7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:39:59.648226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd497548-67a2-4260-9c03-99aa8c29d8a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:39:59.650374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd497548-67a2-4260-9c03-99aa8c29d8a7 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:39:59.658392 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 701/1403] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:39:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:40:00.668973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6889f69-7d11-4eeb-b93f-edb7451d1c10 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:00.669655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6889f69-7d11-4eeb-b93f-edb7451d1c10 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:00.669655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6889f69-7d11-4eeb-b93f-edb7451d1c10 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:00.671022 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 703/1404] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:00.676548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434979-a3cb-48aa-a897-0c3981712e0b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:00.677060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434979-a3cb-48aa-a897-0c3981712e0b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:00.677167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434979-a3cb-48aa-a897-0c3981712e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:00.749695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-44434979-a3cb-48aa-a897-0c3981712e0b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:00.751121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-44434979-a3cb-48aa-a897-0c3981712e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:00.817905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:00.863472 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 702/1405] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:01.218815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6889f69-7d11-4eeb-b93f-edb7451d1c10 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:01.220272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c6889f69-7d11-4eeb-b93f-edb7451d1c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:01.343958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:01.412607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6889f69-7d11-4eeb-b93f-edb7451d1c10 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:01.550122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67f5fa-5030-42f6-ad81-00b0b4ff1131 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:01.550656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67f5fa-5030-42f6-ad81-00b0b4ff1131 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:01.550965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67f5fa-5030-42f6-ad81-00b0b4ff1131 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:01.551220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67f5fa-5030-42f6-ad81-00b0b4ff1131 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:01.552837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6d67f5fa-5030-42f6-ad81-00b0b4ff1131 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:01.635333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:01.654915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67f5fa-5030-42f6-ad81-00b0b4ff1131 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:01.656924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67f5fa-5030-42f6-ad81-00b0b4ff1131 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:01.676436 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 704/1406] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:03.976036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedba28-997f-4c86-b55e-4ac85325a500 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:03.976699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedba28-997f-4c86-b55e-4ac85325a500 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:03.977041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedba28-997f-4c86-b55e-4ac85325a500 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:03.978636 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 703/1407] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:03.983685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3e7e4-dcac-4de8-bc9e-a39f7cc957a8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:03.984174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3e7e4-dcac-4de8-bc9e-a39f7cc957a8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:03.984275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3e7e4-dcac-4de8-bc9e-a39f7cc957a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:04.056737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-70b3e7e4-dcac-4de8-bc9e-a39f7cc957a8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:04.058130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-70b3e7e4-dcac-4de8-bc9e-a39f7cc957a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:04.176053 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 705/1408] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:04.584261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedba28-997f-4c86-b55e-4ac85325a500 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:04.584261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7eedba28-997f-4c86-b55e-4ac85325a500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:04.779114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedba28-997f-4c86-b55e-4ac85325a500 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:04.829023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:04.913098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e206d-4c70-43c8-a5c9-5cee75159faa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:04.913828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e206d-4c70-43c8-a5c9-5cee75159faa service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:04.914187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e206d-4c70-43c8-a5c9-5cee75159faa service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:04.914503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e206d-4c70-43c8-a5c9-5cee75159faa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:04.916128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-809e206d-4c70-43c8-a5c9-5cee75159faa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:05.016842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e206d-4c70-43c8-a5c9-5cee75159faa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:05.018498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e206d-4c70-43c8-a5c9-5cee75159faa service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:05.037036 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 704/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:40:14.174189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=78 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:15.034448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=80 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:23.006095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbcafc-f5b4-48cd-bb71-1d387d458f9c None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:23.006637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbcafc-f5b4-48cd-bb71-1d387d458f9c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:23.006637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbcafc-f5b4-48cd-bb71-1d387d458f9c None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:23.008033 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 706/1410] 10.222.0.16 () {54 vars in 918 bytes} [Sat Sep 3 20:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:23.013032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36c657-ded6-4418-8fc4-b5456dfb9069 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:23.013473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36c657-ded6-4418-8fc4-b5456dfb9069 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:23.013581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36c657-ded6-4418-8fc4-b5456dfb9069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:23.090747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fb36c657-ded6-4418-8fc4-b5456dfb9069 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:23.091969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fb36c657-ded6-4418-8fc4-b5456dfb9069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:23.196828 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 705/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Sat Sep 3 20:40:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:23.211807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed9932-8a24-4cd2-b183-0f6c9ee06d9b None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:23.212098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed9932-8a24-4cd2-b183-0f6c9ee06d9b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:23.212378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed9932-8a24-4cd2-b183-0f6c9ee06d9b None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:23.213764 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 707/1412] 10.222.0.16 () {54 vars in 918 bytes} [Sat Sep 3 20:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:23.218254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72626ed5-c220-489a-a92d-0768dc17fab7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:23.218546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72626ed5-c220-489a-a92d-0768dc17fab7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:23.218825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72626ed5-c220-489a-a92d-0768dc17fab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:23.289732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:23.292472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-72626ed5-c220-489a-a92d-0768dc17fab7 None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:23.294261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-72626ed5-c220-489a-a92d-0768dc17fab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:23.408646 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 706/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Sat Sep 3 20:40:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:40:26.054074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6224381-f640-49ab-8dbd-c58978fc560f None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:26.055162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6224381-f640-49ab-8dbd-c58978fc560f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:26.055162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6224381-f640-49ab-8dbd-c58978fc560f None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:26.056439 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:26.061808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a0687-3b24-4206-a127-94919808d3e6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:26.062211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a0687-3b24-4206-a127-94919808d3e6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:26.062360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a0687-3b24-4206-a127-94919808d3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:26.128426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d23a0687-3b24-4206-a127-94919808d3e6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:26.129669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d23a0687-3b24-4206-a127-94919808d3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:26.185990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:26.228414 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:26.583890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6224381-f640-49ab-8dbd-c58978fc560f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:26.586252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b6224381-f640-49ab-8dbd-c58978fc560f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:26.738023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6224381-f640-49ab-8dbd-c58978fc560f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:26.818643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:26.860003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96739ae-c360-4e9b-a321-85755c4a6e03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:26.860553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96739ae-c360-4e9b-a321-85755c4a6e03 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:26.860869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96739ae-c360-4e9b-a321-85755c4a6e03 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:26.861159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96739ae-c360-4e9b-a321-85755c4a6e03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:26.862782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b96739ae-c360-4e9b-a321-85755c4a6e03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:26.951165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96739ae-c360-4e9b-a321-85755c4a6e03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:26.952752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96739ae-c360-4e9b-a321-85755c4a6e03 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:26.970875 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 709/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:27.970345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4c686-5b4e-4a61-a583-70117e96af47 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:27.970846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4c686-5b4e-4a61-a583-70117e96af47 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:27.970960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4c686-5b4e-4a61-a583-70117e96af47 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:27.972302 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 20:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:27.977933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af7d93-56e3-44a3-a741-9981dab43237 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:27.978316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af7d93-56e3-44a3-a741-9981dab43237 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:27.978502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af7d93-56e3-44a3-a741-9981dab43237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:28.056963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a5af7d93-56e3-44a3-a741-9981dab43237 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:28.058203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a5af7d93-56e3-44a3-a741-9981dab43237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:28.085491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:28.157414 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 20:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:28.397783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4c686-5b4e-4a61-a583-70117e96af47 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:28.399277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a0e4c686-5b4e-4a61-a583-70117e96af47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:28.542310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4c686-5b4e-4a61-a583-70117e96af47 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:28.582412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:28.655174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01164f71-161e-43b8-8523-3def8ea378e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:28.655727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01164f71-161e-43b8-8523-3def8ea378e5 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:28.656014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01164f71-161e-43b8-8523-3def8ea378e5 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:28.656295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01164f71-161e-43b8-8523-3def8ea378e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:28.657939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-01164f71-161e-43b8-8523-3def8ea378e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:28.746057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01164f71-161e-43b8-8523-3def8ea378e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:28.747658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01164f71-161e-43b8-8523-3def8ea378e5 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:28.765077 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 709/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:29.657789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519d78-dec3-4bac-b43b-56861eec4978 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:29.658330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519d78-dec3-4bac-b43b-56861eec4978 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:29.658330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519d78-dec3-4bac-b43b-56861eec4978 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:29.659705 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 20:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:29.665252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb2701-0eda-417f-8d19-8c90ef156fe4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:29.665723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb2701-0eda-417f-8d19-8c90ef156fe4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:29.665828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb2701-0eda-417f-8d19-8c90ef156fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:29.718823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:29.741009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-87fb2701-0eda-417f-8d19-8c90ef156fe4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:29.742243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-87fb2701-0eda-417f-8d19-8c90ef156fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:29.837941 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 20:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:29.998331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2519d78-dec3-4bac-b43b-56861eec4978 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:29.999842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e2519d78-dec3-4bac-b43b-56861eec4978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:30.144310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2519d78-dec3-4bac-b43b-56861eec4978 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:30.157206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:30.259811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72384043-4bfd-405f-8e95-f34be9c4f5c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:30.260342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72384043-4bfd-405f-8e95-f34be9c4f5c7 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:30.260624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72384043-4bfd-405f-8e95-f34be9c4f5c7 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:30.260928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72384043-4bfd-405f-8e95-f34be9c4f5c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:30.262501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-72384043-4bfd-405f-8e95-f34be9c4f5c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:30.351079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72384043-4bfd-405f-8e95-f34be9c4f5c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:30.352660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72384043-4bfd-405f-8e95-f34be9c4f5c7 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:30.370793 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 712/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:31.256811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720930f1-d3ee-4243-8d3c-8ae4e5ebee29 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:31.257347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720930f1-d3ee-4243-8d3c-8ae4e5ebee29 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:31.257347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720930f1-d3ee-4243-8d3c-8ae4e5ebee29 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:31.259261 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 20:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:31.264191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b182be-61a1-4476-94f2-26526e283953 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:31.264632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b182be-61a1-4476-94f2-26526e283953 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:31.264753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b182be-61a1-4476-94f2-26526e283953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:31.284569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:31.341416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-62b182be-61a1-4476-94f2-26526e283953 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:31.342870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-62b182be-61a1-4476-94f2-26526e283953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:31.441487 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 20:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:40:31.599975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720930f1-d3ee-4243-8d3c-8ae4e5ebee29 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:31.602055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-720930f1-d3ee-4243-8d3c-8ae4e5ebee29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:31.712522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:31.748620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720930f1-d3ee-4243-8d3c-8ae4e5ebee29 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:31.862251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8940f26-d351-47f3-9ba7-8ccf59505b05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:31.862784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8940f26-d351-47f3-9ba7-8ccf59505b05 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:31.863065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8940f26-d351-47f3-9ba7-8ccf59505b05 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:31.863340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8940f26-d351-47f3-9ba7-8ccf59505b05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:31.864964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b8940f26-d351-47f3-9ba7-8ccf59505b05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:31.938780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:31.952694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8940f26-d351-47f3-9ba7-8ccf59505b05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:31.954408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8940f26-d351-47f3-9ba7-8ccf59505b05 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:31.972973 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 712/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:32.840579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74016078-5660-468b-9765-2ecbd2f9996a None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:32.841172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74016078-5660-468b-9765-2ecbd2f9996a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:32.841172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74016078-5660-468b-9765-2ecbd2f9996a None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:32.842568 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 20:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:32.847517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b87240-503f-4727-9a89-4a33808c1353 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:32.847922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b87240-503f-4727-9a89-4a33808c1353 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:32.848079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b87240-503f-4727-9a89-4a33808c1353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:32.923415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-24b87240-503f-4727-9a89-4a33808c1353 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:32.924643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-24b87240-503f-4727-9a89-4a33808c1353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:33.020345 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 20:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:33.177419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74016078-5660-468b-9765-2ecbd2f9996a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:33.178942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-74016078-5660-468b-9765-2ecbd2f9996a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:33.387071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:33.449826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74016078-5660-468b-9765-2ecbd2f9996a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:33.560061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d95ac-6e90-4d7e-b018-7baddad4ac2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:33.560666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d95ac-6e90-4d7e-b018-7baddad4ac2b service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:33.560973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d95ac-6e90-4d7e-b018-7baddad4ac2b service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:33.561247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d95ac-6e90-4d7e-b018-7baddad4ac2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:33.562825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fe0d95ac-6e90-4d7e-b018-7baddad4ac2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:33.609897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:33.650895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d95ac-6e90-4d7e-b018-7baddad4ac2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:33.652411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d95ac-6e90-4d7e-b018-7baddad4ac2b service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:33.669964 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 715/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:34.489929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2529e38-5e20-42de-981a-5d6e12e06706 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:34.490486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2529e38-5e20-42de-981a-5d6e12e06706 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:34.490486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2529e38-5e20-42de-981a-5d6e12e06706 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:34.491821 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 20:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:34.496975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f207d0-4629-40d2-93f7-fc1d8ef521bb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:34.497445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f207d0-4629-40d2-93f7-fc1d8ef521bb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:34.497552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f207d0-4629-40d2-93f7-fc1d8ef521bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:34.573005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-18f207d0-4629-40d2-93f7-fc1d8ef521bb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:34.574236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-18f207d0-4629-40d2-93f7-fc1d8ef521bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:34.670129 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 20:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:34.833662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2529e38-5e20-42de-981a-5d6e12e06706 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:34.835407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e2529e38-5e20-42de-981a-5d6e12e06706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:34.883091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:34.979596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2529e38-5e20-42de-981a-5d6e12e06706 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:35.104285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414e481-724d-4465-9cde-0b4b66425a37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:35.104843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414e481-724d-4465-9cde-0b4b66425a37 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:35.105156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414e481-724d-4465-9cde-0b4b66425a37 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:35.105432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414e481-724d-4465-9cde-0b4b66425a37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:35.107127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a414e481-724d-4465-9cde-0b4b66425a37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:35.114207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:35.200471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414e481-724d-4465-9cde-0b4b66425a37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:35.202126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414e481-724d-4465-9cde-0b4b66425a37 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:35.220392 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 715/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:36.037138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577c757-6bf3-4a75-8e3e-4e29434c0599 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:36.037675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577c757-6bf3-4a75-8e3e-4e29434c0599 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:36.037675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577c757-6bf3-4a75-8e3e-4e29434c0599 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:36.039038 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 20:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:36.044545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f110b7-be85-416a-ada2-31368cd46e38 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:36.045020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f110b7-be85-416a-ada2-31368cd46e38 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:36.045136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f110b7-be85-416a-ada2-31368cd46e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:36.123825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e5f110b7-be85-416a-ada2-31368cd46e38 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:36.125116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e5f110b7-be85-416a-ada2-31368cd46e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:36.192789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:36.222037 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 20:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:36.379216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577c757-6bf3-4a75-8e3e-4e29434c0599 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:36.380700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4577c757-6bf3-4a75-8e3e-4e29434c0599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:36.399795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:36.529911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577c757-6bf3-4a75-8e3e-4e29434c0599 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:36.624297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:36.650873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848ded6-d537-444b-bbb0-47c51b69d9d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:36.651407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848ded6-d537-444b-bbb0-47c51b69d9d2 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:36.651687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848ded6-d537-444b-bbb0-47c51b69d9d2 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:36.651964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848ded6-d537-444b-bbb0-47c51b69d9d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:36.653578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4848ded6-d537-444b-bbb0-47c51b69d9d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:36.741171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848ded6-d537-444b-bbb0-47c51b69d9d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:36.742878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848ded6-d537-444b-bbb0-47c51b69d9d2 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:36.761147 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 718/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:37.571819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48cd99-d7bf-4f7a-9443-e9aac2795739 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:37.572464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48cd99-d7bf-4f7a-9443-e9aac2795739 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:37.572464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48cd99-d7bf-4f7a-9443-e9aac2795739 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:37.573761 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 20:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:37.578525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a239108-c375-4cbb-b268-2f6fd9a63206 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:37.578960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a239108-c375-4cbb-b268-2f6fd9a63206 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:37.579234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a239108-c375-4cbb-b268-2f6fd9a63206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:37.654107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8a239108-c375-4cbb-b268-2f6fd9a63206 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:37.655318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8a239108-c375-4cbb-b268-2f6fd9a63206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:37.695981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:37.752176 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 20:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:37.911158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48cd99-d7bf-4f7a-9443-e9aac2795739 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:37.912679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ea48cd99-d7bf-4f7a-9443-e9aac2795739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:38.058973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48cd99-d7bf-4f7a-9443-e9aac2795739 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:38.113011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:38.171307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096038f4-3276-48e8-8903-c080205f7e7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:38.171916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096038f4-3276-48e8-8903-c080205f7e7e service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:38.172197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096038f4-3276-48e8-8903-c080205f7e7e service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:38.172470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096038f4-3276-48e8-8903-c080205f7e7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:38.174143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-096038f4-3276-48e8-8903-c080205f7e7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:38.261572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096038f4-3276-48e8-8903-c080205f7e7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:38.263206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096038f4-3276-48e8-8903-c080205f7e7e service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:38.281281 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 718/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:39.095243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a9a71-fad6-457a-af39-8e5c033a90ff None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:39.095764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a9a71-fad6-457a-af39-8e5c033a90ff None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:39.095886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a9a71-fad6-457a-af39-8e5c033a90ff None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:39.097234 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 20:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:39.102262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddb0ae-7cea-4a36-8076-384df2cbdbaa None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:39.102670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddb0ae-7cea-4a36-8076-384df2cbdbaa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:39.102832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddb0ae-7cea-4a36-8076-384df2cbdbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:39.179937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-21ddb0ae-7cea-4a36-8076-384df2cbdbaa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:39.181692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-21ddb0ae-7cea-4a36-8076-384df2cbdbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:39.183713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:39.278214 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 20:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:39.437056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a9a71-fad6-457a-af39-8e5c033a90ff None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:39.438561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2f6a9a71-fad6-457a-af39-8e5c033a90ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:39.583880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a9a71-fad6-457a-af39-8e5c033a90ff None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:39.611176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:39.696472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc185214-b15b-4bc4-a63d-35c15dfe7e31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:39.697104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185214-b15b-4bc4-a63d-35c15dfe7e31 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:39.697394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185214-b15b-4bc4-a63d-35c15dfe7e31 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:39.697674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185214-b15b-4bc4-a63d-35c15dfe7e31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:39.699272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bc185214-b15b-4bc4-a63d-35c15dfe7e31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:39.787405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc185214-b15b-4bc4-a63d-35c15dfe7e31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:39.789028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc185214-b15b-4bc4-a63d-35c15dfe7e31 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:39.806882 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 721/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:41.896027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094881e-f067-4983-be2c-0a51018be881 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:41.896576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094881e-f067-4983-be2c-0a51018be881 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:41.896576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094881e-f067-4983-be2c-0a51018be881 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:41.897944 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:41.904031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e96a3-0f1c-4c83-b7e0-763b20b06ef7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:41.904453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e96a3-0f1c-4c83-b7e0-763b20b06ef7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:41.904609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e96a3-0f1c-4c83-b7e0-763b20b06ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:41.944791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:41.970860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9f8e96a3-0f1c-4c83-b7e0-763b20b06ef7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:41.972105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9f8e96a3-0f1c-4c83-b7e0-763b20b06ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:42.066696 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:42.421025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094881e-f067-4983-be2c-0a51018be881 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:42.422626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1094881e-f067-4983-be2c-0a51018be881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:42.527483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094881e-f067-4983-be2c-0a51018be881 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:42.559349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:42.599905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e4d93-b621-4aca-97c9-e592a54dc364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:42.600513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e4d93-b621-4aca-97c9-e592a54dc364 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:42.600797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e4d93-b621-4aca-97c9-e592a54dc364 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:42.601099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e4d93-b621-4aca-97c9-e592a54dc364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:42.603250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7d8e4d93-b621-4aca-97c9-e592a54dc364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:42.708336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e4d93-b621-4aca-97c9-e592a54dc364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:42.709821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e4d93-b621-4aca-97c9-e592a54dc364 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:42.728152 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 721/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:45.073865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649814f-3743-4358-bf1d-5a6cbbf90af8 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:45.074465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649814f-3743-4358-bf1d-5a6cbbf90af8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:45.074465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649814f-3743-4358-bf1d-5a6cbbf90af8 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:45.075821 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:45.081237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72583bed-2ffc-4692-8f3c-33e9d77590dd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:45.081670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72583bed-2ffc-4692-8f3c-33e9d77590dd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:45.081801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72583bed-2ffc-4692-8f3c-33e9d77590dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:45.149564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-72583bed-2ffc-4692-8f3c-33e9d77590dd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:45.150781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-72583bed-2ffc-4692-8f3c-33e9d77590dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:45.163358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:45.247497 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:45.593299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649814f-3743-4358-bf1d-5a6cbbf90af8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:45.594873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4649814f-3743-4358-bf1d-5a6cbbf90af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:45.697649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649814f-3743-4358-bf1d-5a6cbbf90af8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:45.709706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:45.771451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702404a-6194-425d-befe-616ba2e30e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:45.772092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702404a-6194-425d-befe-616ba2e30e07 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:45.772373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702404a-6194-425d-befe-616ba2e30e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:45.772647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702404a-6194-425d-befe-616ba2e30e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:45.774899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8702404a-6194-425d-befe-616ba2e30e07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:45.882599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702404a-6194-425d-befe-616ba2e30e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:45.884460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702404a-6194-425d-befe-616ba2e30e07 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:45.903603 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 724/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:48.402513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d73ef-3e53-4594-bd1d-9e0f631836ef None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:48.403049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d73ef-3e53-4594-bd1d-9e0f631836ef None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:48.403049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d73ef-3e53-4594-bd1d-9e0f631836ef None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:48.404354 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:48.409840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ceae-2a4f-4b9f-abe5-948a45af830f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:48.410266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ceae-2a4f-4b9f-abe5-948a45af830f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:48.410399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ceae-2a4f-4b9f-abe5-948a45af830f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:48.458691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:48.480549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-68e1ceae-2a4f-4b9f-abe5-948a45af830f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:48.481783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-68e1ceae-2a4f-4b9f-abe5-948a45af830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:48.577344 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:48.924263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d73ef-3e53-4594-bd1d-9e0f631836ef None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:48.925865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f28d73ef-3e53-4594-bd1d-9e0f631836ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:49.152957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d73ef-3e53-4594-bd1d-9e0f631836ef None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:49.192254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:49.226139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fbbfc-f72f-4b95-888b-eb84ecddf750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:49.226738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fbbfc-f72f-4b95-888b-eb84ecddf750 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:49.227626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fbbfc-f72f-4b95-888b-eb84ecddf750 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:49.227902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fbbfc-f72f-4b95-888b-eb84ecddf750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:49.229534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f56fbbfc-f72f-4b95-888b-eb84ecddf750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:49.334538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fbbfc-f72f-4b95-888b-eb84ecddf750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:49.336221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fbbfc-f72f-4b95-888b-eb84ecddf750 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:49.354098 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 724/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:51.684563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3212a-05fa-4267-986f-3722f0627fb4 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:51.685179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3212a-05fa-4267-986f-3722f0627fb4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:51.685179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3212a-05fa-4267-986f-3722f0627fb4 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:51.686722 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:51.691942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407bd1f-60d0-4a40-9ef3-1733739ecf32 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:51.692408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407bd1f-60d0-4a40-9ef3-1733739ecf32 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:51.692534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407bd1f-60d0-4a40-9ef3-1733739ecf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:51.759526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f407bd1f-60d0-4a40-9ef3-1733739ecf32 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:51.760749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f407bd1f-60d0-4a40-9ef3-1733739ecf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:51.781842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:51.860345 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:52.209609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3212a-05fa-4267-986f-3722f0627fb4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:52.211435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-70d3212a-05fa-4267-986f-3722f0627fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:52.317278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3212a-05fa-4267-986f-3722f0627fb4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:52.337088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:52.390321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b41171-cdb5-47cb-a468-d8105f1908f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:52.390855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b41171-cdb5-47cb-a468-d8105f1908f6 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:52.391138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b41171-cdb5-47cb-a468-d8105f1908f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:52.391411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b41171-cdb5-47cb-a468-d8105f1908f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:52.393041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-43b41171-cdb5-47cb-a468-d8105f1908f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:52.501194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b41171-cdb5-47cb-a468-d8105f1908f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:52.502670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b41171-cdb5-47cb-a468-d8105f1908f6 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:52.521021 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 727/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:54.768319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bd211-cd0b-412a-b2d3-df09ceed1153 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:54.768876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bd211-cd0b-412a-b2d3-df09ceed1153 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:54.768876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bd211-cd0b-412a-b2d3-df09ceed1153 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:54.770331 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:54.776304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbf38f-b593-40eb-8a5f-922c57639519 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:54.776732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbf38f-b593-40eb-8a5f-922c57639519 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:54.776860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbf38f-b593-40eb-8a5f-922c57639519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:54.839140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:54.842154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a5cbf38f-b593-40eb-8a5f-922c57639519 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:54.843921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a5cbf38f-b593-40eb-8a5f-922c57639519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:54.938630 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:40:55.283849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bd211-cd0b-412a-b2d3-df09ceed1153 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:55.285418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-df4bd211-cd0b-412a-b2d3-df09ceed1153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:55.389990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bd211-cd0b-412a-b2d3-df09ceed1153 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:55.437884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:55.463090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01ca1b-404d-4c60-9488-5feb4929a82d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:55.463690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01ca1b-404d-4c60-9488-5feb4929a82d service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:55.463967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01ca1b-404d-4c60-9488-5feb4929a82d service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:55.464237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01ca1b-404d-4c60-9488-5feb4929a82d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:55.465867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1b01ca1b-404d-4c60-9488-5feb4929a82d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:55.572600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01ca1b-404d-4c60-9488-5feb4929a82d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:55.574077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01ca1b-404d-4c60-9488-5feb4929a82d service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:55.592678 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 727/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:40:57.822477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecc191-2695-4315-8681-28874be59b24 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:57.823044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecc191-2695-4315-8681-28874be59b24 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:57.823044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecc191-2695-4315-8681-28874be59b24 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:57.824364 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:40:57.829634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d367d88-c3bd-435e-ab7c-9b61293feff2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:57.830153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d367d88-c3bd-435e-ab7c-9b61293feff2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:57.830265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d367d88-c3bd-435e-ab7c-9b61293feff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:57.899922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5d367d88-c3bd-435e-ab7c-9b61293feff2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:40:57.901183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5d367d88-c3bd-435e-ab7c-9b61293feff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:57.930560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:57.998137 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:40:58.347921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecc191-2695-4315-8681-28874be59b24 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:40:58.349515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b7ecc191-2695-4315-8681-28874be59b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:58.456770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecc191-2695-4315-8681-28874be59b24 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:40:58.485955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:40:58.532074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d50cb-2632-4949-8f19-83216ed455a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:40:58.532684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d50cb-2632-4949-8f19-83216ed455a8 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:40:58.533006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d50cb-2632-4949-8f19-83216ed455a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:40:58.533288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d50cb-2632-4949-8f19-83216ed455a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:40:58.534902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5b6d50cb-2632-4949-8f19-83216ed455a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:40:58.663408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d50cb-2632-4949-8f19-83216ed455a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:40:58.665393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d50cb-2632-4949-8f19-83216ed455a8 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:40:58.683971 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 730/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:40:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:41:00.907676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e650e2-15de-4fa3-a1ca-b05a45ad7636 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:00.908245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e650e2-15de-4fa3-a1ca-b05a45ad7636 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:00.908245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e650e2-15de-4fa3-a1ca-b05a45ad7636 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:00.909625 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:41:00.914976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b8f4d-baf0-42fd-947f-886d3508bb7e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:00.915398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b8f4d-baf0-42fd-947f-886d3508bb7e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:00.915523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b8f4d-baf0-42fd-947f-886d3508bb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:00.980309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4f8b8f4d-baf0-42fd-947f-886d3508bb7e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:41:00.981592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4f8b8f4d-baf0-42fd-947f-886d3508bb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:00.991118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:01.079928 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:41:01.423631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e650e2-15de-4fa3-a1ca-b05a45ad7636 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:41:01.425204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f9e650e2-15de-4fa3-a1ca-b05a45ad7636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:01.530467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e650e2-15de-4fa3-a1ca-b05a45ad7636 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:41:01.587176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:01.605589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab3153-9a5c-4864-9913-4710f6a9a773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:41:01.606200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab3153-9a5c-4864-9913-4710f6a9a773 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:01.606485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab3153-9a5c-4864-9913-4710f6a9a773 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:01.606761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab3153-9a5c-4864-9913-4710f6a9a773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:01.608370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3eab3153-9a5c-4864-9913-4710f6a9a773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:01.717310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab3153-9a5c-4864-9913-4710f6a9a773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:41:01.718808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab3153-9a5c-4864-9913-4710f6a9a773 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:41:01.736642 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 730/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:41:03.979404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1214e-e0de-484e-a480-00b0a7fd7916 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:03.979969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1214e-e0de-484e-a480-00b0a7fd7916 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:03.979969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1214e-e0de-484e-a480-00b0a7fd7916 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:03.981393 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:41:03.986815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb9f97-fa87-4ba5-b775-f0e04cba2dce None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:03.987217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb9f97-fa87-4ba5-b775-f0e04cba2dce None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:03.987373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb9f97-fa87-4ba5-b775-f0e04cba2dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:04.051732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c8fb9f97-fa87-4ba5-b775-f0e04cba2dce None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:41:04.052959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c8fb9f97-fa87-4ba5-b775-f0e04cba2dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:04.090323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:04.147193 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:41:04.494686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1214e-e0de-484e-a480-00b0a7fd7916 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:41:04.496252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-83e1214e-e0de-484e-a480-00b0a7fd7916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:04.598735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1214e-e0de-484e-a480-00b0a7fd7916 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:41:04.637725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:04.672735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1516a8-5829-46ad-85d1-3eebfc76b72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:41:04.673366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1516a8-5829-46ad-85d1-3eebfc76b72b service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:04.673645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1516a8-5829-46ad-85d1-3eebfc76b72b service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:04.673920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1516a8-5829-46ad-85d1-3eebfc76b72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:04.675512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6b1516a8-5829-46ad-85d1-3eebfc76b72b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:04.782602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1516a8-5829-46ad-85d1-3eebfc76b72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:41:04.783958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1516a8-5829-46ad-85d1-3eebfc76b72b service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:41:04.802026 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 733/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:41:07.035361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb4f40-1642-4e3e-b671-0059031be8dc None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:07.035868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb4f40-1642-4e3e-b671-0059031be8dc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:07.035993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb4f40-1642-4e3e-b671-0059031be8dc None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:07.037313 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:41:07.043326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a338671-a50a-40f8-87b7-3c0b79b6d74b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:07.043712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a338671-a50a-40f8-87b7-3c0b79b6d74b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:07.043911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a338671-a50a-40f8-87b7-3c0b79b6d74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:07.108667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1a338671-a50a-40f8-87b7-3c0b79b6d74b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:41:07.109909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1a338671-a50a-40f8-87b7-3c0b79b6d74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:07.128089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:07.206937 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:41:07.555531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb4f40-1642-4e3e-b671-0059031be8dc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:41:07.557104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dceb4f40-1642-4e3e-b671-0059031be8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:07.659516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb4f40-1642-4e3e-b671-0059031be8dc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:41:07.725223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:07.732670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9435617-24f3-4f9a-b959-caf65abb5bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:41:07.733300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9435617-24f3-4f9a-b959-caf65abb5bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:07.733590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9435617-24f3-4f9a-b959-caf65abb5bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:07.733864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9435617-24f3-4f9a-b959-caf65abb5bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:07.735473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d9435617-24f3-4f9a-b959-caf65abb5bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:07.843290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9435617-24f3-4f9a-b959-caf65abb5bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:41:07.844711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9435617-24f3-4f9a-b959-caf65abb5bc8 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:41:07.863459 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 733/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:41:10.064052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518008d-d85b-4be9-b3fc-a79082839298 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:10.064635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518008d-d85b-4be9-b3fc-a79082839298 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:10.064635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518008d-d85b-4be9-b3fc-a79082839298 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:10.065997 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:41:10.071272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a349e9a-8258-4bd7-99e9-06c17fde5095 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:10.071699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a349e9a-8258-4bd7-99e9-06c17fde5095 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:10.071824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a349e9a-8258-4bd7-99e9-06c17fde5095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:10.137032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4a349e9a-8258-4bd7-99e9-06c17fde5095 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:41:10.138239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4a349e9a-8258-4bd7-99e9-06c17fde5095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:10.184934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:10.232717 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:41:10.575949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518008d-d85b-4be9-b3fc-a79082839298 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:41:10.577522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8518008d-d85b-4be9-b3fc-a79082839298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:10.680862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518008d-d85b-4be9-b3fc-a79082839298 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:41:10.730335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:10.756570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae292aaa-ae79-425a-8c2e-e2624af4a051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:41:10.757216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae292aaa-ae79-425a-8c2e-e2624af4a051 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:10.757502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae292aaa-ae79-425a-8c2e-e2624af4a051 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:10.757778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae292aaa-ae79-425a-8c2e-e2624af4a051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:10.759377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ae292aaa-ae79-425a-8c2e-e2624af4a051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:10.867913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae292aaa-ae79-425a-8c2e-e2624af4a051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:41:10.869321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae292aaa-ae79-425a-8c2e-e2624af4a051 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:41:10.886682 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 736/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:41:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:41:20.230994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=22 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:20.884492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=66 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:34.991194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b3983-e075-42c4-b596-e1b289aff381 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:34.991793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b3983-e075-42c4-b596-e1b289aff381 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:34.991793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b3983-e075-42c4-b596-e1b289aff381 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:34.993148 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 735/1471] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:41:34.998188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac545804-955c-46e4-8861-f3511f8e72d4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:34.998606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac545804-955c-46e4-8861-f3511f8e72d4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:34.998736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac545804-955c-46e4-8861-f3511f8e72d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:35.068814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ac545804-955c-46e4-8861-f3511f8e72d4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:41:35.070561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ac545804-955c-46e4-8861-f3511f8e72d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:35.167728 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 737/1472] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:41:35.535656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25047aa-0a0b-42ba-bee4-7dee56c37542 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:35.535941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25047aa-0a0b-42ba-bee4-7dee56c37542 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:35.536218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25047aa-0a0b-42ba-bee4-7dee56c37542 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:35.537589 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 736/1473] 10.222.0.16 () {54 vars in 961 bytes} [Sat Sep 3 20:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:41:35.542436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f8448-d4d9-4756-9aeb-59ce67a12f27 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:35.542731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f8448-d4d9-4756-9aeb-59ce67a12f27 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:35.543011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f8448-d4d9-4756-9aeb-59ce67a12f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:35.618553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-151f8448-d4d9-4756-9aeb-59ce67a12f27 None None] MFA Rules not processed for user `7a36c3c09b2d425fbf6ccbe051c293d4`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:41:35.619755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-151f8448-d4d9-4756-9aeb-59ce67a12f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:35.640049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:35.696971 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 738/1474] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Sep 3 20:41:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:41:35.704400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25047aa-0a0b-42ba-bee4-7dee56c37542 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:41:35.706020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d25047aa-0a0b-42ba-bee4-7dee56c37542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:35.806819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25047aa-0a0b-42ba-bee4-7dee56c37542 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:41:35.877416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea728db7-abca-416d-bc7e-2defc0e667b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:41:35.878000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea728db7-abca-416d-bc7e-2defc0e667b9 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:35.878280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea728db7-abca-416d-bc7e-2defc0e667b9 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:35.878555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea728db7-abca-416d-bc7e-2defc0e667b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:35.880166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ea728db7-abca-416d-bc7e-2defc0e667b9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:35.932876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:35.986882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea728db7-abca-416d-bc7e-2defc0e667b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:41:35.988305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea728db7-abca-416d-bc7e-2defc0e667b9 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:41:36.008101 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 737/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:41:38.323393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f0c85-9641-4e16-a4db-218ca847897c None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:38.323916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f0c85-9641-4e16-a4db-218ca847897c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:38.323916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f0c85-9641-4e16-a4db-218ca847897c None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:38.325240 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 739/1476] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:41:38.330515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70ae3d-724f-45b8-b217-72309a07eb50 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:38.330914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70ae3d-724f-45b8-b217-72309a07eb50 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:38.331106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70ae3d-724f-45b8-b217-72309a07eb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:38.396381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5f70ae3d-724f-45b8-b217-72309a07eb50 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:41:38.397628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5f70ae3d-724f-45b8-b217-72309a07eb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:38.492984 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 738/1477] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:41:38.828377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f0c85-9641-4e16-a4db-218ca847897c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:41:38.829974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c42f0c85-9641-4e16-a4db-218ca847897c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:38.956137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f0c85-9641-4e16-a4db-218ca847897c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:41:39.004611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:39.049991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90624dc0-c057-4001-ac4b-1653603c76c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:41:39.050600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90624dc0-c057-4001-ac4b-1653603c76c9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:39.050886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90624dc0-c057-4001-ac4b-1653603c76c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:39.051189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90624dc0-c057-4001-ac4b-1653603c76c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:39.051822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90624dc0-c057-4001-ac4b-1653603c76c9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:41:39.053640 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-90624dc0-c057-4001-ac4b-1653603c76c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:41:39.053640 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 20:41:39.053640 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 20:41:39.054823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90624dc0-c057-4001-ac4b-1653603c76c9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:41:39.073586 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 740/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 3 20:41:38 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:41:41.286958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96e226-da23-4072-aa44-a564aa98578e None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:41.287471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96e226-da23-4072-aa44-a564aa98578e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:41.287471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96e226-da23-4072-aa44-a564aa98578e None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:41.288814 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 739/1479] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:41:41.294047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32251596-2b0c-4fdf-971c-31d80602a8ab None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:41.294477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32251596-2b0c-4fdf-971c-31d80602a8ab None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:41.294613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32251596-2b0c-4fdf-971c-31d80602a8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:41.360256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-32251596-2b0c-4fdf-971c-31d80602a8ab None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:41:41.361511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-32251596-2b0c-4fdf-971c-31d80602a8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:41.457759 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 741/1480] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:41:41.814306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96e226-da23-4072-aa44-a564aa98578e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:41:41.816137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6b96e226-da23-4072-aa44-a564aa98578e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:41.826427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:41.959312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96e226-da23-4072-aa44-a564aa98578e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:41:42.038545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:42.071649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc093d8-5af9-44ad-9e98-71a44f8cc463 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:41:42.072190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc093d8-5af9-44ad-9e98-71a44f8cc463 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:42.072473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc093d8-5af9-44ad-9e98-71a44f8cc463 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:42.072749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc093d8-5af9-44ad-9e98-71a44f8cc463 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:42.074392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6dc093d8-5af9-44ad-9e98-71a44f8cc463 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:42.162749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc093d8-5af9-44ad-9e98-71a44f8cc463 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:41:42.164361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc093d8-5af9-44ad-9e98-71a44f8cc463 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:41:42.182703 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 740/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:41:44.558367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76a0a5-5978-48b4-83b0-1ee53abd6104 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:44.559023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76a0a5-5978-48b4-83b0-1ee53abd6104 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:44.559023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76a0a5-5978-48b4-83b0-1ee53abd6104 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:44.560256 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 742/1482] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:41:44.565680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c205e-5e32-4db8-89a7-fba53ced1c5f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:44.566140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c205e-5e32-4db8-89a7-fba53ced1c5f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:44.566397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c205e-5e32-4db8-89a7-fba53ced1c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:44.635212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f39c205e-5e32-4db8-89a7-fba53ced1c5f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:41:44.636436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f39c205e-5e32-4db8-89a7-fba53ced1c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:44.683093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:44.735309 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 741/1483] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:41:45.087474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76a0a5-5978-48b4-83b0-1ee53abd6104 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:41:45.089107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7d76a0a5-5978-48b4-83b0-1ee53abd6104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:45.112358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:45.193964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76a0a5-5978-48b4-83b0-1ee53abd6104 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:41:45.267964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8791f3-9199-426e-8f69-c6478c717572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:41:45.268569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8791f3-9199-426e-8f69-c6478c717572 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:45.268852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8791f3-9199-426e-8f69-c6478c717572 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:45.269176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8791f3-9199-426e-8f69-c6478c717572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:45.270789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1a8791f3-9199-426e-8f69-c6478c717572 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:45.343892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:45.378275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8791f3-9199-426e-8f69-c6478c717572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:41:45.379707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8791f3-9199-426e-8f69-c6478c717572 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:41:45.398968 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 743/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:41:47.650719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacab6e0-22e6-4f85-b56d-179540c29cde None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:47.651279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacab6e0-22e6-4f85-b56d-179540c29cde None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:47.651279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacab6e0-22e6-4f85-b56d-179540c29cde None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:47.652610 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 742/1485] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:41:47.657807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f351f94-be39-4ffe-a8ba-6bcd97fa6d76 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:47.658248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f351f94-be39-4ffe-a8ba-6bcd97fa6d76 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:47.658356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f351f94-be39-4ffe-a8ba-6bcd97fa6d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:47.729196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5f351f94-be39-4ffe-a8ba-6bcd97fa6d76 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:41:47.730406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5f351f94-be39-4ffe-a8ba-6bcd97fa6d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:47.825943 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 744/1486] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:41:48.195034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacab6e0-22e6-4f85-b56d-179540c29cde None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:41:48.196580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bacab6e0-22e6-4f85-b56d-179540c29cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:48.298679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacab6e0-22e6-4f85-b56d-179540c29cde None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:41:48.370623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:48.372574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eff96d-ce0d-4f25-a78f-1e9804ca9458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:41:48.373382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eff96d-ce0d-4f25-a78f-1e9804ca9458 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:48.373913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eff96d-ce0d-4f25-a78f-1e9804ca9458 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:48.374236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eff96d-ce0d-4f25-a78f-1e9804ca9458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:48.376324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-21eff96d-ce0d-4f25-a78f-1e9804ca9458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:48.483686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eff96d-ce0d-4f25-a78f-1e9804ca9458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:41:48.485157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eff96d-ce0d-4f25-a78f-1e9804ca9458 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:41:48.502976 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 743/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:41:50.832215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5793ce-fb43-4790-9de5-a4f3d8b096f7 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:50.832781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5793ce-fb43-4790-9de5-a4f3d8b096f7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:50.832781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5793ce-fb43-4790-9de5-a4f3d8b096f7 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:50.834273 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 745/1488] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:41:50.839898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad83db7-fbd4-4258-9495-9150a3ba3023 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:50.840300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad83db7-fbd4-4258-9495-9150a3ba3023 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:50.840461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad83db7-fbd4-4258-9495-9150a3ba3023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:50.907059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8ad83db7-fbd4-4258-9495-9150a3ba3023 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:41:50.908283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8ad83db7-fbd4-4258-9495-9150a3ba3023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:50.961074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:51.004869 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 744/1489] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:41:51.357205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5793ce-fb43-4790-9de5-a4f3d8b096f7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:41:51.358761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5e5793ce-fb43-4790-9de5-a4f3d8b096f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:51.393754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:51.462492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5793ce-fb43-4790-9de5-a4f3d8b096f7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:41:51.533625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1af72-fe97-4fc0-94fe-2a3702680569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:41:51.534225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1af72-fe97-4fc0-94fe-2a3702680569 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:51.534508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1af72-fe97-4fc0-94fe-2a3702680569 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:51.534785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1af72-fe97-4fc0-94fe-2a3702680569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:51.536365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7da1af72-fe97-4fc0-94fe-2a3702680569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:51.620783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:51.650990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1af72-fe97-4fc0-94fe-2a3702680569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:41:51.652370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1af72-fe97-4fc0-94fe-2a3702680569 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:41:51.670682 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 746/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:41:54.065576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee3602-b7f6-4249-bba4-8124096e1079 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:54.066157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee3602-b7f6-4249-bba4-8124096e1079 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:54.066157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee3602-b7f6-4249-bba4-8124096e1079 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:54.067483 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 745/1491] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 3 20:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:41:54.072634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3396c-36e2-4bac-b415-8bd9c7de8c45 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:54.073093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3396c-36e2-4bac-b415-8bd9c7de8c45 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:54.073230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3396c-36e2-4bac-b415-8bd9c7de8c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:54.139314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1ed3396c-36e2-4bac-b415-8bd9c7de8c45 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:41:54.140516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1ed3396c-36e2-4bac-b415-8bd9c7de8c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:54.236378 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 747/1492] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 3 20:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:41:54.586421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee3602-b7f6-4249-bba4-8124096e1079 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:41:54.587963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-96ee3602-b7f6-4249-bba4-8124096e1079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:54.688924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee3602-b7f6-4249-bba4-8124096e1079 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:41:54.764491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876da53-2249-44d5-8864-7f89c9f655e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:41:54.765137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876da53-2249-44d5-8864-7f89c9f655e4 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:41:54.765426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876da53-2249-44d5-8864-7f89c9f655e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:41:54.765709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876da53-2249-44d5-8864-7f89c9f655e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:41:54.767334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5876da53-2249-44d5-8864-7f89c9f655e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:41:54.777053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:41:54.875811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876da53-2249-44d5-8864-7f89c9f655e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:41:54.877202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876da53-2249-44d5-8864-7f89c9f655e4 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:41:54.895121 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 746/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:42:04.234629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=82 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:42:04.892870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=50 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:43:08.618976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c172a6f-e01e-4237-b9aa-31df9b681de8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:08.619567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c172a6f-e01e-4237-b9aa-31df9b681de8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:08.619567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c172a6f-e01e-4237-b9aa-31df9b681de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:08.694543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3c172a6f-e01e-4237-b9aa-31df9b681de8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:43:08.695767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3c172a6f-e01e-4237-b9aa-31df9b681de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:08.794324 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 748/1494] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:43:08.802294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a6527-9635-414a-b6a8-8e0b7a3d5fa3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:08.803396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a6527-9635-414a-b6a8-8e0b7a3d5fa3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:08.803518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a6527-9635-414a-b6a8-8e0b7a3d5fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:08.877663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8e8a6527-9635-414a-b6a8-8e0b7a3d5fa3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:43:08.878799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8e8a6527-9635-414a-b6a8-8e0b7a3d5fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:08.974309 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 747/1495] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:43:08.979605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c172a6f-e01e-4237-b9aa-31df9b681de8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:43:08.980842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3c172a6f-e01e-4237-b9aa-31df9b681de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:08.992220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c172a6f-e01e-4237-b9aa-31df9b681de8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:43:09.055716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:43:09.089791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f2651-b7b5-4e12-a85e-55c88802c813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:43:09.090403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f2651-b7b5-4e12-a85e-55c88802c813 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:09.090691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f2651-b7b5-4e12-a85e-55c88802c813 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:09.090970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f2651-b7b5-4e12-a85e-55c88802c813 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:09.091618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f2651-b7b5-4e12-a85e-55c88802c813 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:43:09.091926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f2651-b7b5-4e12-a85e-55c88802c813 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:43:09.092972 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-9d2f2651-b7b5-4e12-a85e-55c88802c813 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:43:09.092972 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 20:43:09.092972 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 20:43:09.093874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f2651-b7b5-4e12-a85e-55c88802c813 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:43:09.106155 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 749/1496] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:43:09.115010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a6527-9635-414a-b6a8-8e0b7a3d5fa3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:43:09.124330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a6527-9635-414a-b6a8-8e0b7a3d5fa3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:43:09.185302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:43:09.219653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e3b9a-beec-48cf-862c-c07c805c517c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:43:09.220262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e3b9a-beec-48cf-862c-c07c805c517c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:09.220549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e3b9a-beec-48cf-862c-c07c805c517c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:09.220826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e3b9a-beec-48cf-862c-c07c805c517c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:09.221501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e3b9a-beec-48cf-862c-c07c805c517c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:43:09.223285 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-d29e3b9a-beec-48cf-862c-c07c805c517c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:43:09.223285 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 20:43:09.223285 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 20:43:09.224555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e3b9a-beec-48cf-862c-c07c805c517c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:43:09.279159 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 748/1497] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:43:09 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:43:09.283522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f2651-b7b5-4e12-a85e-55c88802c813 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:43:09.285179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9d2f2651-b7b5-4e12-a85e-55c88802c813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:09.319745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f2651-b7b5-4e12-a85e-55c88802c813 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:43:09.413003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52e764-73d8-401d-81e5-5b214067bea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:43:09.413538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52e764-73d8-401d-81e5-5b214067bea0 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:09.413822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52e764-73d8-401d-81e5-5b214067bea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:09.414100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52e764-73d8-401d-81e5-5b214067bea0 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:09.414734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db52e764-73d8-401d-81e5-5b214067bea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:43:09.415743 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-db52e764-73d8-401d-81e5-5b214067bea0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:43:09.415743 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 20:43:09.415743 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 20:43:09.416978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db52e764-73d8-401d-81e5-5b214067bea0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:43:09.457815 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 750/1498] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:43:09 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:43:09.462783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e3b9a-beec-48cf-862c-c07c805c517c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:43:09.464303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d29e3b9a-beec-48cf-862c-c07c805c517c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:09.529540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e3b9a-beec-48cf-862c-c07c805c517c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:43:09.624534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:43:09.638560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea552d-19fd-4b1a-b39d-55fd5f0c00d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:43:09.639410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea552d-19fd-4b1a-b39d-55fd5f0c00d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:09.639874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea552d-19fd-4b1a-b39d-55fd5f0c00d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:09.640325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea552d-19fd-4b1a-b39d-55fd5f0c00d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:09.641022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea552d-19fd-4b1a-b39d-55fd5f0c00d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:43:09.642071 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-57ea552d-19fd-4b1a-b39d-55fd5f0c00d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:43:09.642071 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 20:43:09.642071 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 20:43:09.642978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea552d-19fd-4b1a-b39d-55fd5f0c00d9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:43:09.655735 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 749/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:43:09.659785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52e764-73d8-401d-81e5-5b214067bea0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:43:09.661351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-db52e764-73d8-401d-81e5-5b214067bea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:09.673350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52e764-73d8-401d-81e5-5b214067bea0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:43:09.700408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:43:09.772611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db97c4-daf6-4e47-9a41-68a55eef4dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:43:09.773254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db97c4-daf6-4e47-9a41-68a55eef4dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:09.773539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db97c4-daf6-4e47-9a41-68a55eef4dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:09.773813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db97c4-daf6-4e47-9a41-68a55eef4dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:09.774454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db97c4-daf6-4e47-9a41-68a55eef4dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:43:09.775465 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-62db97c4-daf6-4e47-9a41-68a55eef4dc7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:43:09.775465 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 20:43:09.775465 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 20:43:09.776350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db97c4-daf6-4e47-9a41-68a55eef4dc7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:43:09.788720 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 751/1500] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:43:09.792934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea552d-19fd-4b1a-b39d-55fd5f0c00d9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:43:09.794460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-57ea552d-19fd-4b1a-b39d-55fd5f0c00d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:09.827817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea552d-19fd-4b1a-b39d-55fd5f0c00d9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:43:09.919785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d81ea-6d92-49ff-a51e-31e0eab6eba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:43:09.920434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d81ea-6d92-49ff-a51e-31e0eab6eba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:09.920715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d81ea-6d92-49ff-a51e-31e0eab6eba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:09.921022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d81ea-6d92-49ff-a51e-31e0eab6eba1 admin admin] PATH_INFO: `/v3/projects/2743a6265df745ae98d00096cd14ecd6/users/4c0067ceb83a4850864d337d9a986b4a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:09.943816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d81ea-6d92-49ff-a51e-31e0eab6eba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2743a6265df745ae98d00096cd14ecd6, user_id=4c0067ceb83a4850864d337d9a986b4a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:43:09.945061 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-de9d81ea-6d92-49ff-a51e-31e0eab6eba1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:43:09.945061 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 20:43:09.945061 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 20:43:09.947022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d81ea-6d92-49ff-a51e-31e0eab6eba1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:43:09.972641 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 750/1501] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:43:09 2022] PUT /identity/v3/projects/2743a6265df745ae98d00096cd14ecd6/users/4c0067ceb83a4850864d337d9a986b4a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:43:09.988043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33db8c-93d5-4ae0-b35e-6efad1258438 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:09.988343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33db8c-93d5-4ae0-b35e-6efad1258438 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:09.988627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33db8c-93d5-4ae0-b35e-6efad1258438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:10.045854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6c33db8c-93d5-4ae0-b35e-6efad1258438 None None] MFA Rules not processed for user `4c0067ceb83a4850864d337d9a986b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:43:10.047109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6c33db8c-93d5-4ae0-b35e-6efad1258438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:10.125965 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 752/1502] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:43:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:43:10.139171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb3a70-e69b-4f8b-8326-a412d6143f84 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:10.139470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb3a70-e69b-4f8b-8326-a412d6143f84 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:10.139757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb3a70-e69b-4f8b-8326-a412d6143f84 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:10.141141 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 751/1503] 10.222.0.16 () {54 vars in 964 bytes} [Sat Sep 3 20:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:43:10.146019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745c45b-2b53-4a7a-9e12-be4c81645a5f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:10.146315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745c45b-2b53-4a7a-9e12-be4c81645a5f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:10.146593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745c45b-2b53-4a7a-9e12-be4c81645a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:10.178230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:43:10.221977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9745c45b-2b53-4a7a-9e12-be4c81645a5f None None] MFA Rules not processed for user `133a1ff64e224593be467eb7c27cbb09`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:43:10.223176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9745c45b-2b53-4a7a-9e12-be4c81645a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:10.300099 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 753/1504] 10.222.0.16 () {58 vars in 1061 bytes} [Sat Sep 3 20:43:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:43:10.308484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb3a70-e69b-4f8b-8326-a412d6143f84 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:43:10.310056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-03fb3a70-e69b-4f8b-8326-a412d6143f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:10.365440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:43:10.420154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb3a70-e69b-4f8b-8326-a412d6143f84 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:43:10.493853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615568c-9eb6-4b64-8f09-6cd8c961a885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:43:10.494386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615568c-9eb6-4b64-8f09-6cd8c961a885 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:10.494667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615568c-9eb6-4b64-8f09-6cd8c961a885 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:10.494942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615568c-9eb6-4b64-8f09-6cd8c961a885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:10.496531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e615568c-9eb6-4b64-8f09-6cd8c961a885 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:10.594764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615568c-9eb6-4b64-8f09-6cd8c961a885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:43:10.601308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615568c-9eb6-4b64-8f09-6cd8c961a885 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:43:10.605157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:43:10.620550 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 752/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:43:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:43:10.666567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d1afd-0946-4b66-82fd-f418471e6df5 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:10.666981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d1afd-0946-4b66-82fd-f418471e6df5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:10.667134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d1afd-0946-4b66-82fd-f418471e6df5 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:10.668392 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 754/1506] 10.222.0.16 () {52 vars in 830 bytes} [Sat Sep 3 20:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:43:10.671921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615568c-9eb6-4b64-8f09-6cd8c961a885 service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:43:10.673158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e615568c-9eb6-4b64-8f09-6cd8c961a885 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:10.770282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615568c-9eb6-4b64-8f09-6cd8c961a885 service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:43:10.861289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:43:10.861927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:10.862177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:10.862447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] PATH_INFO: `/v3/users/4c0067ceb83a4850864d337d9a986b4a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:10.874542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c0067ceb83a4850864d337d9a986b4a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:43:10.875565 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:43:10.875565 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 20:43:10.875565 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 20:43:10.876774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:43:10.886723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:43:10.942349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c0067ceb83a4850864d337d9a986b4a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:43:10.967608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c0067ceb83a4850864d337d9a986b4a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:43:10.977024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c0067ceb83a4850864d337d9a986b4a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:43:10.977568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c0067ceb83a4850864d337d9a986b4a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:43:10.978072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c0067ceb83a4850864d337d9a986b4a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:43:10.996955 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 753/1507] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:43:10 2022] DELETE /identity/v3/users/4c0067ceb83a4850864d337d9a986b4a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:43:11.001337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e5af9-1c0b-4626-80a7-f126b89e10c9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:11.001651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e5af9-1c0b-4626-80a7-f126b89e10c9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:11.001935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e5af9-1c0b-4626-80a7-f126b89e10c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:11.058910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9d0e5af9-1c0b-4626-80a7-f126b89e10c9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:43:11.060066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9d0e5af9-1c0b-4626-80a7-f126b89e10c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:11.136678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:43:11.150335 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 755/1508] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:43:11.159753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:43:11.161018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:11.259063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f78780-f8f2-4649-9831-b9e577a398cb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:43:11.312419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:43:11.334041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c5d5d-2b01-40bf-96f8-dc4ce74f68d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:43:11.334650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c5d5d-2b01-40bf-96f8-dc4ce74f68d3 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:11.334934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c5d5d-2b01-40bf-96f8-dc4ce74f68d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:11.335213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c5d5d-2b01-40bf-96f8-dc4ce74f68d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:11.337106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c29c5d5d-2b01-40bf-96f8-dc4ce74f68d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:11.443928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c5d5d-2b01-40bf-96f8-dc4ce74f68d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:43:11.445404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c5d5d-2b01-40bf-96f8-dc4ce74f68d3 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:43:11.463456 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 754/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:43:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:43:11.861623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e5af9-1c0b-4626-80a7-f126b89e10c9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:43:11.863018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9d0e5af9-1c0b-4626-80a7-f126b89e10c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:43:11.875088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e5af9-1c0b-4626-80a7-f126b89e10c9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:43:11.970980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:43:11.971613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:43:11.971897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:43:11.972173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] PATH_INFO: `/v3/projects/2743a6265df745ae98d00096cd14ecd6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:43:11.982569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2743a6265df745ae98d00096cd14ecd6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:43:11.983601 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:43:11.983601 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 20:43:11.983601 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 20:43:11.984819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:43:12.043568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2743a6265df745ae98d00096cd14ecd6', 'request_id': 'req-332a03e9-eba8-4dc5-8687-3c708db80e60'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:43:12.052324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2743a6265df745ae98d00096cd14ecd6', 'request_id': 'req-332a03e9-eba8-4dc5-8687-3c708db80e60'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:43:12.065218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2743a6265df745ae98d00096cd14ecd6', 'request_id': 'req-332a03e9-eba8-4dc5-8687-3c708db80e60'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:43:12.087735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] The token cache is being invalidate because project 2743a6265df745ae98d00096cd14ecd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:43:12.088026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:43:12.088483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-332a03e9-eba8-4dc5-8687-3c708db80e60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:43:12.089903 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 756/1510] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:43:11 2022] DELETE /identity/v3/projects/2743a6265df745ae98d00096cd14ecd6 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:43:21.461001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=62 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:43:22.085510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:46:33.299033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79019946-861e-48be-9215-9088ef167a87 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:46:33.299631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79019946-861e-48be-9215-9088ef167a87 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:46:33.299631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79019946-861e-48be-9215-9088ef167a87 None None] PATH_INFO: `/v3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:46:33.300880 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 755/1511] 10.222.0.42 () {54 vars in 925 bytes} [Sat Sep 3 20:46:33 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:46:33.306489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b53c29-37d2-4a38-83e1-30a2a6891256 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:46:33.306978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b53c29-37d2-4a38-83e1-30a2a6891256 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:46:33.307081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b53c29-37d2-4a38-83e1-30a2a6891256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:46:33.387448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b0b53c29-37d2-4a38-83e1-30a2a6891256 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:46:33.388598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b0b53c29-37d2-4a38-83e1-30a2a6891256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:46:33.501362 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 757/1512] 10.222.0.42 () {58 vars in 1015 bytes} [Sat Sep 3 20:46:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:46:37.596312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93136cf9-9a4e-4816-976b-1dc183060e8b None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:46:37.596930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93136cf9-9a4e-4816-976b-1dc183060e8b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:46:37.597042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93136cf9-9a4e-4816-976b-1dc183060e8b None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:46:37.598292 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 756/1513] 10.222.0.16 () {54 vars in 961 bytes} [Sat Sep 3 20:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:46:37.603253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd632e0-2cbc-4ef5-ba7a-c62f3f1dcf6a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:46:37.603708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd632e0-2cbc-4ef5-ba7a-c62f3f1dcf6a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:46:37.603824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd632e0-2cbc-4ef5-ba7a-c62f3f1dcf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:46:37.624102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6bd632e0-2cbc-4ef5-ba7a-c62f3f1dcf6a None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:46:37.625347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6bd632e0-2cbc-4ef5-ba7a-c62f3f1dcf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:46:37.635923 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 758/1514] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Sep 3 20:46:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:46:37.643833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93136cf9-9a4e-4816-976b-1dc183060e8b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:46:37.660335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93136cf9-9a4e-4816-976b-1dc183060e8b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:46:37.665223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd7db4-4c72-47b6-b00c-0bf8cad7d3b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:46:37.665818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd7db4-4c72-47b6-b00c-0bf8cad7d3b9 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:46:37.666103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd7db4-4c72-47b6-b00c-0bf8cad7d3b9 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:46:37.666378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd7db4-4c72-47b6-b00c-0bf8cad7d3b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:46:37.675159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd7db4-4c72-47b6-b00c-0bf8cad7d3b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:46:37.678000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd7db4-4c72-47b6-b00c-0bf8cad7d3b9 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:46:37.687634 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 757/1515] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:46:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:46:47.619252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=77 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:46:47.683673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:46:48.178891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb122d-ab1f-48de-b713-a37e71ffff72 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:46:48.179186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb122d-ab1f-48de-b713-a37e71ffff72 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:46:48.179471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb122d-ab1f-48de-b713-a37e71ffff72 None None] PATH_INFO: `/v3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:46:48.180826 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 759/1516] 10.222.0.30 () {54 vars in 925 bytes} [Sat Sep 3 20:46:48 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:46:48.186925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5228c4f-430b-4012-87b1-7ea04d9c6663 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:46:48.187215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5228c4f-430b-4012-87b1-7ea04d9c6663 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:46:48.187493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5228c4f-430b-4012-87b1-7ea04d9c6663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:46:48.206562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c5228c4f-430b-4012-87b1-7ea04d9c6663 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:46:48.207759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c5228c4f-430b-4012-87b1-7ea04d9c6663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:46:48.216905 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 758/1517] 10.222.0.30 () {58 vars in 1015 bytes} [Sat Sep 3 20:46:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:46:57.214811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b438a-1775-4245-a41a-7f05960e5850 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:46:57.215258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b438a-1775-4245-a41a-7f05960e5850 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:46:57.215369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b438a-1775-4245-a41a-7f05960e5850 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:46:57.216694 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 760/1518] 10.222.0.16 () {54 vars in 961 bytes} [Sat Sep 3 20:46:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:46:57.221547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae468cb9-d38e-4e85-b806-6c822a5dccbd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:46:57.222044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae468cb9-d38e-4e85-b806-6c822a5dccbd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:46:57.222142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae468cb9-d38e-4e85-b806-6c822a5dccbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:46:57.241815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ae468cb9-d38e-4e85-b806-6c822a5dccbd None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:46:57.243049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ae468cb9-d38e-4e85-b806-6c822a5dccbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:46:57.251623 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 759/1519] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Sep 3 20:46:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:46:57.258861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b438a-1775-4245-a41a-7f05960e5850 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:46:57.271934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b438a-1775-4245-a41a-7f05960e5850 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:46:57.277412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a767e-8438-4ee7-a478-b8c065430fb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:46:57.278112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a767e-8438-4ee7-a478-b8c065430fb1 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:46:57.278398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a767e-8438-4ee7-a478-b8c065430fb1 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:46:57.278671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a767e-8438-4ee7-a478-b8c065430fb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:46:57.286779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a767e-8438-4ee7-a478-b8c065430fb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:46:57.288372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a767e-8438-4ee7-a478-b8c065430fb1 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:46:57.298664 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 761/1520] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:46:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:47:07.236554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:47:07.294674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:47:08.404612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828d63d-cf1e-4f67-84df-2fde80a12677 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:47:08.405077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828d63d-cf1e-4f67-84df-2fde80a12677 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:47:08.405224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828d63d-cf1e-4f67-84df-2fde80a12677 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:47:08.406582 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 760/1521] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 20:47:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:47:08.411507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf15beb-013d-4703-9417-a37ce82558e7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:47:08.411920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf15beb-013d-4703-9417-a37ce82558e7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:47:08.412061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf15beb-013d-4703-9417-a37ce82558e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:47:08.472656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6cf15beb-013d-4703-9417-a37ce82558e7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:47:08.473923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6cf15beb-013d-4703-9417-a37ce82558e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:47:08.548912 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 762/1522] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 20:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:47:08.709624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828d63d-cf1e-4f67-84df-2fde80a12677 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:47:08.711403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5828d63d-cf1e-4f67-84df-2fde80a12677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:47:08.820232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828d63d-cf1e-4f67-84df-2fde80a12677 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:47:08.827043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ebde6-a59d-495f-8365-ee1601dccacd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:47:08.827864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ebde6-a59d-495f-8365-ee1601dccacd service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:47:08.828235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ebde6-a59d-495f-8365-ee1601dccacd service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:47:08.828512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ebde6-a59d-495f-8365-ee1601dccacd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:47:08.838064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ebde6-a59d-495f-8365-ee1601dccacd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:47:08.839977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ebde6-a59d-495f-8365-ee1601dccacd service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:47:08.850037 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 761/1523] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:47:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:47:09.687460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9130a-15e4-46d8-98fa-6eb911d8fdd5 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:47:09.687996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9130a-15e4-46d8-98fa-6eb911d8fdd5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:47:09.688093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9130a-15e4-46d8-98fa-6eb911d8fdd5 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:47:09.689412 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 763/1524] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 20:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:47:09.694270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152c4e3-466b-4da0-9dc7-2cdc17d6b545 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:47:09.694692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152c4e3-466b-4da0-9dc7-2cdc17d6b545 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:47:09.694837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152c4e3-466b-4da0-9dc7-2cdc17d6b545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:47:09.714689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8152c4e3-466b-4da0-9dc7-2cdc17d6b545 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:47:09.716376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8152c4e3-466b-4da0-9dc7-2cdc17d6b545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:47:09.724047 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 762/1525] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 20:47:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:47:09.899591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9130a-15e4-46d8-98fa-6eb911d8fdd5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:47:09.914652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9130a-15e4-46d8-98fa-6eb911d8fdd5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:47:09.920164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ae6ca-3b95-4267-9f21-1ea25e6d7869 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:47:09.920765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ae6ca-3b95-4267-9f21-1ea25e6d7869 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:47:09.921089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ae6ca-3b95-4267-9f21-1ea25e6d7869 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:47:09.921371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ae6ca-3b95-4267-9f21-1ea25e6d7869 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:47:09.929383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ae6ca-3b95-4267-9f21-1ea25e6d7869 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:47:09.930977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ae6ca-3b95-4267-9f21-1ea25e6d7869 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:47:09.940313 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 764/1526] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:47:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:47:13.860836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c938d-fad0-46f9-a075-67b43fff6b23 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:47:13.861438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c938d-fad0-46f9-a075-67b43fff6b23 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:47:13.861438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c938d-fad0-46f9-a075-67b43fff6b23 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:47:13.862803 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 763/1527] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:47:13.867988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451facf5-dbd1-47fd-a7de-7e4ed4371f69 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:47:13.868420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451facf5-dbd1-47fd-a7de-7e4ed4371f69 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:47:13.868532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451facf5-dbd1-47fd-a7de-7e4ed4371f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:47:13.887865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-451facf5-dbd1-47fd-a7de-7e4ed4371f69 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:47:13.889101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-451facf5-dbd1-47fd-a7de-7e4ed4371f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:47:13.897060 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 765/1528] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:47:14.247654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c938d-fad0-46f9-a075-67b43fff6b23 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:47:14.249376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-134c938d-fad0-46f9-a075-67b43fff6b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:47:14.326505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c938d-fad0-46f9-a075-67b43fff6b23 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:47:14.330034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcc479-20de-4d17-8499-824dcea2ad91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:47:14.330627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcc479-20de-4d17-8499-824dcea2ad91 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:47:14.330912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcc479-20de-4d17-8499-824dcea2ad91 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:47:14.331187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcc479-20de-4d17-8499-824dcea2ad91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:47:14.340051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcc479-20de-4d17-8499-824dcea2ad91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:47:14.342172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcc479-20de-4d17-8499-824dcea2ad91 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:47:14.351274 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 764/1529] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:47:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:47:16.512111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bb107-8ef2-4b70-b5bc-a5ad73b41664 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:47:16.512667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bb107-8ef2-4b70-b5bc-a5ad73b41664 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:47:16.512667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bb107-8ef2-4b70-b5bc-a5ad73b41664 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:47:16.514259 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 766/1530] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 20:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:47:16.519337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671e47d-09ea-4b4f-8f89-1019968ae3cd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:47:16.519817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671e47d-09ea-4b4f-8f89-1019968ae3cd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:47:16.519924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671e47d-09ea-4b4f-8f89-1019968ae3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:47:16.539378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6671e47d-09ea-4b4f-8f89-1019968ae3cd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:47:16.540578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6671e47d-09ea-4b4f-8f89-1019968ae3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:47:16.550586 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 765/1531] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 20:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:47:16.907630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bb107-8ef2-4b70-b5bc-a5ad73b41664 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:47:16.918406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bb107-8ef2-4b70-b5bc-a5ad73b41664 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:47:16.922531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef13a8-d325-4980-8619-d9203e4e7ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:47:16.923123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef13a8-d325-4980-8619-d9203e4e7ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:47:16.923403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef13a8-d325-4980-8619-d9203e4e7ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:47:16.923676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef13a8-d325-4980-8619-d9203e4e7ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:47:16.932407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef13a8-d325-4980-8619-d9203e4e7ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:47:16.933849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef13a8-d325-4980-8619-d9203e4e7ba3 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:47:16.943032 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 767/1532] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:47:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:47:26.534276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:47:26.939104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=65 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:07.139784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf3046-61c1-44de-b6f1-a20f4cac72e4 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:07.140271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf3046-61c1-44de-b6f1-a20f4cac72e4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:07.140355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf3046-61c1-44de-b6f1-a20f4cac72e4 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:07.141714 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 766/1533] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:48:07.147047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e1f1d-6e49-4b07-be65-6ad500f1e440 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:07.147497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e1f1d-6e49-4b07-be65-6ad500f1e440 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:07.147602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e1f1d-6e49-4b07-be65-6ad500f1e440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:07.167287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-582e1f1d-6e49-4b07-be65-6ad500f1e440 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:07.168477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-582e1f1d-6e49-4b07-be65-6ad500f1e440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:07.178050 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 768/1534] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:48:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:17.162520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:40.619113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eab934-414f-4727-9b0f-35f7829816d2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:40.620143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eab934-414f-4727-9b0f-35f7829816d2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:40.620649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eab934-414f-4727-9b0f-35f7829816d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:40.659825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-87eab934-414f-4727-9b0f-35f7829816d2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:40.661164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-87eab934-414f-4727-9b0f-35f7829816d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:40.676718 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 767/1535] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:48:40.686490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b32ba3-87cb-4af1-93cd-0cf37f4104f8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:40.687488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b32ba3-87cb-4af1-93cd-0cf37f4104f8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:40.688000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b32ba3-87cb-4af1-93cd-0cf37f4104f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:40.713373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-11b32ba3-87cb-4af1-93cd-0cf37f4104f8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:40.714599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-11b32ba3-87cb-4af1-93cd-0cf37f4104f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:40.726991 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 769/1536] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:40.731912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eab934-414f-4727-9b0f-35f7829816d2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:40.751824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eab934-414f-4727-9b0f-35f7829816d2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:40.760352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2eba75-a420-4eb5-80b9-82f7942d93bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:40.761154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2eba75-a420-4eb5-80b9-82f7942d93bd admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:40.761441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2eba75-a420-4eb5-80b9-82f7942d93bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:40.761719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2eba75-a420-4eb5-80b9-82f7942d93bd admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:40.762379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2eba75-a420-4eb5-80b9-82f7942d93bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:48:40.762702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2eba75-a420-4eb5-80b9-82f7942d93bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:40.763767 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-0f2eba75-a420-4eb5-80b9-82f7942d93bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:48:40.763767 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 20:48:40.763767 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 20:48:40.764664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2eba75-a420-4eb5-80b9-82f7942d93bd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:40.785484 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 768/1537] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:48:40.794020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b32ba3-87cb-4af1-93cd-0cf37f4104f8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:40.808839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b32ba3-87cb-4af1-93cd-0cf37f4104f8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:40.814787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2ee6a-7795-4cac-b85e-d5ce61872150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:40.817551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2ee6a-7795-4cac-b85e-d5ce61872150 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:40.818108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2ee6a-7795-4cac-b85e-d5ce61872150 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:40.818581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2ee6a-7795-4cac-b85e-d5ce61872150 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:40.819442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2ee6a-7795-4cac-b85e-d5ce61872150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:40.820764 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-81c2ee6a-7795-4cac-b85e-d5ce61872150 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:48:40.820764 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 20:48:40.820764 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 20:48:40.822362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2ee6a-7795-4cac-b85e-d5ce61872150 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:40.877443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88854f-476d-4427-ab8b-38af9f2465d2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:40.877807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88854f-476d-4427-ab8b-38af9f2465d2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:40.878145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88854f-476d-4427-ab8b-38af9f2465d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:40.878950 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 770/1538] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:40 2022] POST /identity/v3/projects => generated 397 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:40.886254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2ee6a-7795-4cac-b85e-d5ce61872150 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:40.912966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1e88854f-476d-4427-ab8b-38af9f2465d2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:40.912966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1e88854f-476d-4427-ab8b-38af9f2465d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:40.991828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2ee6a-7795-4cac-b85e-d5ce61872150 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.001685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a298ded-8d55-4fdd-8c92-a073b89d8c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.003315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a298ded-8d55-4fdd-8c92-a073b89d8c97 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.003811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a298ded-8d55-4fdd-8c92-a073b89d8c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.004318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a298ded-8d55-4fdd-8c92-a073b89d8c97 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.005238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a298ded-8d55-4fdd-8c92-a073b89d8c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.007893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a298ded-8d55-4fdd-8c92-a073b89d8c97 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.046247 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 771/1539] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:40 2022] POST /identity/v3/users => generated 299 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:48:41.047913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177ee80-49b6-4763-bd1e-af832a5a0b7c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.048380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177ee80-49b6-4763-bd1e-af832a5a0b7c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.048907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177ee80-49b6-4763-bd1e-af832a5a0b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.055201 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 769/1540] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.055201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71ca80-e9d7-4870-890d-3f35231329a6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.055201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71ca80-e9d7-4870-890d-3f35231329a6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.055201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71ca80-e9d7-4870-890d-3f35231329a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.087833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8177ee80-49b6-4763-bd1e-af832a5a0b7c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.089172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8177ee80-49b6-4763-bd1e-af832a5a0b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.092669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6f71ca80-e9d7-4870-890d-3f35231329a6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.094015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6f71ca80-e9d7-4870-890d-3f35231329a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.099227 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 772/1541] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.100747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feec95a-7a63-4278-9c62-91561e0552c3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.101083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feec95a-7a63-4278-9c62-91561e0552c3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.101365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feec95a-7a63-4278-9c62-91561e0552c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.109265 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 770/1542] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.110223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71ca80-e9d7-4870-890d-3f35231329a6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.126036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71ca80-e9d7-4870-890d-3f35231329a6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.141654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d8a05-8ae1-4f9e-8733-4b6a6261dcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.145496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d8a05-8ae1-4f9e-8733-4b6a6261dcbc admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.145786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d8a05-8ae1-4f9e-8733-4b6a6261dcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.146065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d8a05-8ae1-4f9e-8733-4b6a6261dcbc admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.146734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d8a05-8ae1-4f9e-8733-4b6a6261dcbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.149776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d8a05-8ae1-4f9e-8733-4b6a6261dcbc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.150004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7feec95a-7a63-4278-9c62-91561e0552c3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.151253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7feec95a-7a63-4278-9c62-91561e0552c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.164606 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 771/1543] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:41.165752 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 773/1544] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.165976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985f710-2ea7-4ecb-87cc-dee54b7a9690 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.166324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985f710-2ea7-4ecb-87cc-dee54b7a9690 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.166609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985f710-2ea7-4ecb-87cc-dee54b7a9690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.167035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12474f8-5c5e-42ee-a34d-4b70170734b5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.167346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12474f8-5c5e-42ee-a34d-4b70170734b5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.167630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12474f8-5c5e-42ee-a34d-4b70170734b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.195328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f12474f8-5c5e-42ee-a34d-4b70170734b5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.195529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b985f710-2ea7-4ecb-87cc-dee54b7a9690 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.196552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f12474f8-5c5e-42ee-a34d-4b70170734b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.196656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b985f710-2ea7-4ecb-87cc-dee54b7a9690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.204718 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 772/1545] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.205934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fd469-8458-4cc6-a658-d5e2ab9d0761 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.206226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fd469-8458-4cc6-a658-d5e2ab9d0761 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.206504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fd469-8458-4cc6-a658-d5e2ab9d0761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.213640 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 774/1546] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.213640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c706a13-c662-4606-8b97-861016bfd2e0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.213640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c706a13-c662-4606-8b97-861016bfd2e0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.213640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c706a13-c662-4606-8b97-861016bfd2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.231469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-919fd469-8458-4cc6-a658-d5e2ab9d0761 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.233277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-919fd469-8458-4cc6-a658-d5e2ab9d0761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.235306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3c706a13-c662-4606-8b97-861016bfd2e0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.236485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3c706a13-c662-4606-8b97-861016bfd2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.243273 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 773/1547] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.244469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f47185-526b-463d-95c3-dfe575a6a0a1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.244927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f47185-526b-463d-95c3-dfe575a6a0a1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.245148 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 775/1548] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.245249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f47185-526b-463d-95c3-dfe575a6a0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.245892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c706a13-c662-4606-8b97-861016bfd2e0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.259067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c706a13-c662-4606-8b97-861016bfd2e0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.264064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e030ef-268c-41d8-93d7-b021d76f80ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.264672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e030ef-268c-41d8-93d7-b021d76f80ec admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.264998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e030ef-268c-41d8-93d7-b021d76f80ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.265276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e030ef-268c-41d8-93d7-b021d76f80ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.266115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e030ef-268c-41d8-93d7-b021d76f80ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.266350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-72f47185-526b-463d-95c3-dfe575a6a0a1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.267558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-72f47185-526b-463d-95c3-dfe575a6a0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.267747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e030ef-268c-41d8-93d7-b021d76f80ec admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.276311 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 774/1549] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.277381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f521b33-af53-492b-9f08-b674478835e0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.277673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f521b33-af53-492b-9f08-b674478835e0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.278127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f521b33-af53-492b-9f08-b674478835e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.278227 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 776/1550] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:41.280068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de23683-ea21-4638-83d4-6e1d2b0534d0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.280068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de23683-ea21-4638-83d4-6e1d2b0534d0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.280068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de23683-ea21-4638-83d4-6e1d2b0534d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.299379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6f521b33-af53-492b-9f08-b674478835e0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.299818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6de23683-ea21-4638-83d4-6e1d2b0534d0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.300590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6f521b33-af53-492b-9f08-b674478835e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.301176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6de23683-ea21-4638-83d4-6e1d2b0534d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.308495 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 775/1551] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.309367 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 777/1552] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.310024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de23683-ea21-4638-83d4-6e1d2b0534d0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.313433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3e6c7-0356-4f2d-a544-3144d7d3c8ad None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.313725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3e6c7-0356-4f2d-a544-3144d7d3c8ad None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.314004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3e6c7-0356-4f2d-a544-3144d7d3c8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.320526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de23683-ea21-4638-83d4-6e1d2b0534d0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.325178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea18e4-1f9d-4621-b4ec-c78767148afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.325820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea18e4-1f9d-4621-b4ec-c78767148afe admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.326136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea18e4-1f9d-4621-b4ec-c78767148afe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.326416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea18e4-1f9d-4621-b4ec-c78767148afe admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.327130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea18e4-1f9d-4621-b4ec-c78767148afe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:48:41.327442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea18e4-1f9d-4621-b4ec-c78767148afe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.329055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea18e4-1f9d-4621-b4ec-c78767148afe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.333211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-36b3e6c7-0356-4f2d-a544-3144d7d3c8ad None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.334410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-36b3e6c7-0356-4f2d-a544-3144d7d3c8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.340721 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 778/1553] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:48:41.341536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea18e4-1f9d-4621-b4ec-c78767148afe admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.345947 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 776/1554] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.345947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ad0bb-257d-4d6f-ad92-9ca17c77e25b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.345947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ad0bb-257d-4d6f-ad92-9ca17c77e25b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.345947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ad0bb-257d-4d6f-ad92-9ca17c77e25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.353438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea18e4-1f9d-4621-b4ec-c78767148afe admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.358688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c63a3d-9376-4604-a750-545ead7899e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.359287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c63a3d-9376-4604-a750-545ead7899e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.359572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c63a3d-9376-4604-a750-545ead7899e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.359850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c63a3d-9376-4604-a750-545ead7899e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.360490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c63a3d-9376-4604-a750-545ead7899e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:48:41.360799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c63a3d-9376-4604-a750-545ead7899e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.362427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c63a3d-9376-4604-a750-545ead7899e7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.363958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9e6ad0bb-257d-4d6f-ad92-9ca17c77e25b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.365168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9e6ad0bb-257d-4d6f-ad92-9ca17c77e25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.372910 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 777/1555] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.374232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ad0bb-257d-4d6f-ad92-9ca17c77e25b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.383858 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 779/1556] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:48:41.383858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c63a3d-9376-4604-a750-545ead7899e7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.384091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ad0bb-257d-4d6f-ad92-9ca17c77e25b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.388678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f33e9-3f8d-4410-b126-8513a7e5421a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.389247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c63a3d-9376-4604-a750-545ead7899e7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.389376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f33e9-3f8d-4410-b126-8513a7e5421a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.389565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f33e9-3f8d-4410-b126-8513a7e5421a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.389841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f33e9-3f8d-4410-b126-8513a7e5421a admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.390484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f33e9-3f8d-4410-b126-8513a7e5421a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:48:41.390790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f33e9-3f8d-4410-b126-8513a7e5421a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.392394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f33e9-3f8d-4410-b126-8513a7e5421a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.393934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e56cf9-47d7-4a50-ba8e-964f3ce8c65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.394573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e56cf9-47d7-4a50-ba8e-964f3ce8c65b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.394860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e56cf9-47d7-4a50-ba8e-964f3ce8c65b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.395132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e56cf9-47d7-4a50-ba8e-964f3ce8c65b admin admin] PATH_INFO: `/v3/projects/90b3649ee46d439d9dccdb7ef0e27291/users/4aa6221685824a758e0b8df8ed0d7dca/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.403817 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 778/1557] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:48:41.404710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f33e9-3f8d-4410-b126-8513a7e5421a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.410486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e56cf9-47d7-4a50-ba8e-964f3ce8c65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b3649ee46d439d9dccdb7ef0e27291, user_id=4aa6221685824a758e0b8df8ed0d7dca, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.411522 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-63e56cf9-47d7-4a50-ba8e-964f3ce8c65b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:48:41.411522 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 20:48:41.411522 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 20:48:41.413510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e56cf9-47d7-4a50-ba8e-964f3ce8c65b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.415798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f33e9-3f8d-4410-b126-8513a7e5421a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.421056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83444e6-fd0a-4070-9fcb-ae5ffde47396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.421650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83444e6-fd0a-4070-9fcb-ae5ffde47396 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.421934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83444e6-fd0a-4070-9fcb-ae5ffde47396 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.422206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83444e6-fd0a-4070-9fcb-ae5ffde47396 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.422845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83444e6-fd0a-4070-9fcb-ae5ffde47396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:48:41.423159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83444e6-fd0a-4070-9fcb-ae5ffde47396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.424731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83444e6-fd0a-4070-9fcb-ae5ffde47396 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.437541 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 780/1558] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:41 2022] PUT /identity/v3/projects/90b3649ee46d439d9dccdb7ef0e27291/users/4aa6221685824a758e0b8df8ed0d7dca/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:41.437541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4c5e8-18e9-49cf-bdb7-a21027528586 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.437541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4c5e8-18e9-49cf-bdb7-a21027528586 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.437541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4c5e8-18e9-49cf-bdb7-a21027528586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.455127 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 779/1559] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:48:41.455127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83444e6-fd0a-4070-9fcb-ae5ffde47396 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.455411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5fc4c5e8-18e9-49cf-bdb7-a21027528586 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.455411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5fc4c5e8-18e9-49cf-bdb7-a21027528586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.459766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83444e6-fd0a-4070-9fcb-ae5ffde47396 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.463846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:41.511147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:41.513866 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 781/1560] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.515484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4c5e8-18e9-49cf-bdb7-a21027528586 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.522066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f872b0f-197e-4bd6-b267-0792ab1afd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.522950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f872b0f-197e-4bd6-b267-0792ab1afd02 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.523340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f872b0f-197e-4bd6-b267-0792ab1afd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.523703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f872b0f-197e-4bd6-b267-0792ab1afd02 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.524362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f872b0f-197e-4bd6-b267-0792ab1afd02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:48:41.524759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f872b0f-197e-4bd6-b267-0792ab1afd02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.526529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4c5e8-18e9-49cf-bdb7-a21027528586 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.527217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f872b0f-197e-4bd6-b267-0792ab1afd02 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.532389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c5f62-bd5a-4a27-9c8e-1ac7e631270b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.533183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c5f62-bd5a-4a27-9c8e-1ac7e631270b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.533473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c5f62-bd5a-4a27-9c8e-1ac7e631270b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.533753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c5f62-bd5a-4a27-9c8e-1ac7e631270b admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.534391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c5f62-bd5a-4a27-9c8e-1ac7e631270b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.536291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c5f62-bd5a-4a27-9c8e-1ac7e631270b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.539684 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 780/1561] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:48:41.540487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f872b0f-197e-4bd6-b267-0792ab1afd02 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.556072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f872b0f-197e-4bd6-b267-0792ab1afd02 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.560916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26195c84-0fde-4394-a6a1-e8a9f58912da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.561576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26195c84-0fde-4394-a6a1-e8a9f58912da admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.561862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26195c84-0fde-4394-a6a1-e8a9f58912da admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.562141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26195c84-0fde-4394-a6a1-e8a9f58912da admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.562800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26195c84-0fde-4394-a6a1-e8a9f58912da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:48:41.563161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26195c84-0fde-4394-a6a1-e8a9f58912da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.564785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26195c84-0fde-4394-a6a1-e8a9f58912da admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.572451 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 782/1562] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:41.572451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c5f62-bd5a-4a27-9c8e-1ac7e631270b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.577990 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 781/1563] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:48:41.578760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26195c84-0fde-4394-a6a1-e8a9f58912da admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.588331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26195c84-0fde-4394-a6a1-e8a9f58912da admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.637818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c5f62-bd5a-4a27-9c8e-1ac7e631270b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.642599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e5838-bf1e-4f12-b51d-32ea9ee8a472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.643143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e5838-bf1e-4f12-b51d-32ea9ee8a472 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.643426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e5838-bf1e-4f12-b51d-32ea9ee8a472 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.643697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e5838-bf1e-4f12-b51d-32ea9ee8a472 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.644351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e5838-bf1e-4f12-b51d-32ea9ee8a472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.646049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a843a1-95d5-42ce-a71f-022e4ac84baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.646552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e5838-bf1e-4f12-b51d-32ea9ee8a472 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.646664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a843a1-95d5-42ce-a71f-022e4ac84baa admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.646955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a843a1-95d5-42ce-a71f-022e4ac84baa admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.647229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a843a1-95d5-42ce-a71f-022e4ac84baa admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.647907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a843a1-95d5-42ce-a71f-022e4ac84baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.649907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a843a1-95d5-42ce-a71f-022e4ac84baa admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.674896 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 783/1564] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:41.676531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b10ad-c0d8-49c5-8992-d527c4d60d9c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.677126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b10ad-c0d8-49c5-8992-d527c4d60d9c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.677410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b10ad-c0d8-49c5-8992-d527c4d60d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.680252 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 782/1565] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:41.681062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a843a1-95d5-42ce-a71f-022e4ac84baa admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.709636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d10b10ad-c0d8-49c5-8992-d527c4d60d9c None None] MFA Rules not processed for user `4aa6221685824a758e0b8df8ed0d7dca`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:41.710861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d10b10ad-c0d8-49c5-8992-d527c4d60d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:41.749732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a843a1-95d5-42ce-a71f-022e4ac84baa admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.754568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb044a-bbe2-4195-adc4-3c65724d9b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.755186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb044a-bbe2-4195-adc4-3c65724d9b39 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.755478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb044a-bbe2-4195-adc4-3c65724d9b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.755761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb044a-bbe2-4195-adc4-3c65724d9b39 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.756408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb044a-bbe2-4195-adc4-3c65724d9b39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.758368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb044a-bbe2-4195-adc4-3c65724d9b39 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.776040 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 784/1566] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:41.777017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b10ad-c0d8-49c5-8992-d527c4d60d9c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.791611 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 783/1567] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:41.791611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb044a-bbe2-4195-adc4-3c65724d9b39 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.793068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b10ad-c0d8-49c5-8992-d527c4d60d9c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.802741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:41.842274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:41.855585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166129c-15ee-4437-bb51-28010c89c0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.856195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166129c-15ee-4437-bb51-28010c89c0fe admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.856484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166129c-15ee-4437-bb51-28010c89c0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.856627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb044a-bbe2-4195-adc4-3c65724d9b39 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.856765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166129c-15ee-4437-bb51-28010c89c0fe admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.858014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166129c-15ee-4437-bb51-28010c89c0fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:48:41.858358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166129c-15ee-4437-bb51-28010c89c0fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.859969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166129c-15ee-4437-bb51-28010c89c0fe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.861976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91c438-15a2-432d-a539-5ab9eec9d26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.862578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91c438-15a2-432d-a539-5ab9eec9d26c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.862899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91c438-15a2-432d-a539-5ab9eec9d26c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.863343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91c438-15a2-432d-a539-5ab9eec9d26c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.864012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91c438-15a2-432d-a539-5ab9eec9d26c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.866026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91c438-15a2-432d-a539-5ab9eec9d26c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.872957 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 785/1568] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:48:41.872957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166129c-15ee-4437-bb51-28010c89c0fe admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.882158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166129c-15ee-4437-bb51-28010c89c0fe admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.886816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050dae2-410d-4bee-9206-0fededc2fef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.887403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050dae2-410d-4bee-9206-0fededc2fef4 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.887691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050dae2-410d-4bee-9206-0fededc2fef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.887965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050dae2-410d-4bee-9206-0fededc2fef4 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.889245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050dae2-410d-4bee-9206-0fededc2fef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.891326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050dae2-410d-4bee-9206-0fededc2fef4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.903259 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 784/1569] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:41.903259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91c438-15a2-432d-a539-5ab9eec9d26c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.914067 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 786/1570] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:41.914933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050dae2-410d-4bee-9206-0fededc2fef4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:41.924449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050dae2-410d-4bee-9206-0fededc2fef4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.963538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91c438-15a2-432d-a539-5ab9eec9d26c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:41.968973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f49f99-4efb-418e-a2c4-f49d6f5296f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.969580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f49f99-4efb-418e-a2c4-f49d6f5296f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.969864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f49f99-4efb-418e-a2c4-f49d6f5296f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.970142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f49f99-4efb-418e-a2c4-f49d6f5296f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.970776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f49f99-4efb-418e-a2c4-f49d6f5296f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.972690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f49f99-4efb-418e-a2c4-f49d6f5296f6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:41.982167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fe529-7423-4532-83d3-404fcdefbdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:41.982809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fe529-7423-4532-83d3-404fcdefbdaa admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:41.983093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fe529-7423-4532-83d3-404fcdefbdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:41.983369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fe529-7423-4532-83d3-404fcdefbdaa admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:41.984003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fe529-7423-4532-83d3-404fcdefbdaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:41.985959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fe529-7423-4532-83d3-404fcdefbdaa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.006570 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 785/1571] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/projects => generated 395 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:42.006570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f49f99-4efb-418e-a2c4-f49d6f5296f6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.011153 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 787/1572] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:41 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:42.011153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fe529-7423-4532-83d3-404fcdefbdaa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.021660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fe529-7423-4532-83d3-404fcdefbdaa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.071211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f49f99-4efb-418e-a2c4-f49d6f5296f6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.075966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623e505-90d8-4826-ad68-1f34cc0e7882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.077128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623e505-90d8-4826-ad68-1f34cc0e7882 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.077416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623e505-90d8-4826-ad68-1f34cc0e7882 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.077696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623e505-90d8-4826-ad68-1f34cc0e7882 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.078250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7207284-fa8b-473b-849a-7480c23050cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.078356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623e505-90d8-4826-ad68-1f34cc0e7882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.078907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7207284-fa8b-473b-849a-7480c23050cc admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.079200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7207284-fa8b-473b-849a-7480c23050cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.079337 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-d623e505-90d8-4826-ad68-1f34cc0e7882 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:48:42.079337 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 20:48:42.079337 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 20:48:42.079551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7207284-fa8b-473b-849a-7480c23050cc admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.080095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7207284-fa8b-473b-849a-7480c23050cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.080564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623e505-90d8-4826-ad68-1f34cc0e7882 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.082069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7207284-fa8b-473b-849a-7480c23050cc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.104797 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 786/1573] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:42 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:42.105690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623e505-90d8-4826-ad68-1f34cc0e7882 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.114254 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 788/1574] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:42 2022] POST /identity/v3/projects => generated 399 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:42.115067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7207284-fa8b-473b-849a-7480c23050cc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.115661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623e505-90d8-4826-ad68-1f34cc0e7882 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.119270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:42.171584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:42.184761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630577ff-d071-4443-a8af-511fef19853f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.185416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630577ff-d071-4443-a8af-511fef19853f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.185730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630577ff-d071-4443-a8af-511fef19853f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.186012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630577ff-d071-4443-a8af-511fef19853f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.186654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630577ff-d071-4443-a8af-511fef19853f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.188597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630577ff-d071-4443-a8af-511fef19853f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.192439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7207284-fa8b-473b-849a-7480c23050cc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.197190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca8c2c-ee8e-440a-b2d3-12aaba07e36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.197799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca8c2c-ee8e-440a-b2d3-12aaba07e36f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.198076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca8c2c-ee8e-440a-b2d3-12aaba07e36f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.198667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca8c2c-ee8e-440a-b2d3-12aaba07e36f admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.199000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca8c2c-ee8e-440a-b2d3-12aaba07e36f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.201735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca8c2c-ee8e-440a-b2d3-12aaba07e36f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.209900 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 787/1575] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:42 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:42.210702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630577ff-d071-4443-a8af-511fef19853f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.220943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630577ff-d071-4443-a8af-511fef19853f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.225726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c75f3-5cb8-45e4-93c9-1443499281d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.226318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c75f3-5cb8-45e4-93c9-1443499281d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.226606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c75f3-5cb8-45e4-93c9-1443499281d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.226888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c75f3-5cb8-45e4-93c9-1443499281d1 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.227513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c75f3-5cb8-45e4-93c9-1443499281d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.229724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c75f3-5cb8-45e4-93c9-1443499281d1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.233011 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 789/1576] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:42 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:42.237014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca8c2c-ee8e-440a-b2d3-12aaba07e36f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.252591 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 788/1577] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:42 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:42.253419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c75f3-5cb8-45e4-93c9-1443499281d1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.263449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c75f3-5cb8-45e4-93c9-1443499281d1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.302244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca8c2c-ee8e-440a-b2d3-12aaba07e36f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.307194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dff82-9eec-4c26-82a0-5b9d682b12d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.307798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dff82-9eec-4c26-82a0-5b9d682b12d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.308078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dff82-9eec-4c26-82a0-5b9d682b12d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.308352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dff82-9eec-4c26-82a0-5b9d682b12d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.309104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56dff82-9eec-4c26-82a0-5b9d682b12d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.311319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56dff82-9eec-4c26-82a0-5b9d682b12d6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.321590 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 790/1578] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.322563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dff82-9eec-4c26-82a0-5b9d682b12d6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.328574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70edae-09a0-4df5-ba72-85b09e1295b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.329288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70edae-09a0-4df5-ba72-85b09e1295b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.329601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70edae-09a0-4df5-ba72-85b09e1295b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.329898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70edae-09a0-4df5-ba72-85b09e1295b0 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.330574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70edae-09a0-4df5-ba72-85b09e1295b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.332525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70edae-09a0-4df5-ba72-85b09e1295b0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.333208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dff82-9eec-4c26-82a0-5b9d682b12d6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.337795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f200e00-4573-47a8-9c73-de35af02223e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.338383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f200e00-4573-47a8-9c73-de35af02223e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.338671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f200e00-4573-47a8-9c73-de35af02223e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.338981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f200e00-4573-47a8-9c73-de35af02223e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.339640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f200e00-4573-47a8-9c73-de35af02223e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.341507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f200e00-4573-47a8-9c73-de35af02223e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.352685 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 791/1579] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.353523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f200e00-4573-47a8-9c73-de35af02223e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.357538 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 789/1580] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:42 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:42.358538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70edae-09a0-4df5-ba72-85b09e1295b0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.363225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f200e00-4573-47a8-9c73-de35af02223e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.368658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721598e-e864-41df-bc73-42f8593ae49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.369000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70edae-09a0-4df5-ba72-85b09e1295b0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.369389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721598e-e864-41df-bc73-42f8593ae49e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.369685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721598e-e864-41df-bc73-42f8593ae49e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.370011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721598e-e864-41df-bc73-42f8593ae49e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.370706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721598e-e864-41df-bc73-42f8593ae49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.372407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721598e-e864-41df-bc73-42f8593ae49e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.375539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3ed66-90ce-48b6-972a-59d3af90e948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.376190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3ed66-90ce-48b6-972a-59d3af90e948 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.376478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3ed66-90ce-48b6-972a-59d3af90e948 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.376755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3ed66-90ce-48b6-972a-59d3af90e948 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.377433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3ed66-90ce-48b6-972a-59d3af90e948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.379341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3ed66-90ce-48b6-972a-59d3af90e948 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.386245 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 792/1581] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.386245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721598e-e864-41df-bc73-42f8593ae49e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.394386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721598e-e864-41df-bc73-42f8593ae49e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.399146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979dfb2-aa82-4dd0-8b70-ee4da24cd1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.399734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979dfb2-aa82-4dd0-8b70-ee4da24cd1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.400025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979dfb2-aa82-4dd0-8b70-ee4da24cd1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.400307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979dfb2-aa82-4dd0-8b70-ee4da24cd1b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.400973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979dfb2-aa82-4dd0-8b70-ee4da24cd1b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.404114 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 790/1582] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:42 2022] POST /identity/v3/users => generated 306 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:42.404114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3ed66-90ce-48b6-972a-59d3af90e948 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.404363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979dfb2-aa82-4dd0-8b70-ee4da24cd1b5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.415657 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 793/1583] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.415657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979dfb2-aa82-4dd0-8b70-ee4da24cd1b5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.415903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3ed66-90ce-48b6-972a-59d3af90e948 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.420118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7715e-5c6b-4754-a187-1b1d6e26c27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.420718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7715e-5c6b-4754-a187-1b1d6e26c27d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.421224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7715e-5c6b-4754-a187-1b1d6e26c27d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.421503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7715e-5c6b-4754-a187-1b1d6e26c27d admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.422178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7715e-5c6b-4754-a187-1b1d6e26c27d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.423086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979dfb2-aa82-4dd0-8b70-ee4da24cd1b5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.424040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7715e-5c6b-4754-a187-1b1d6e26c27d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.427734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f359aff-e490-45be-ad69-124d92e16c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.428320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f359aff-e490-45be-ad69-124d92e16c96 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.428610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f359aff-e490-45be-ad69-124d92e16c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.428910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f359aff-e490-45be-ad69-124d92e16c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.429547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f359aff-e490-45be-ad69-124d92e16c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.431116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f359aff-e490-45be-ad69-124d92e16c96 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.443638 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 794/1584] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.444390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f359aff-e490-45be-ad69-124d92e16c96 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.450153 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 791/1585] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:42 2022] POST /identity/v3/users => generated 292 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:42.451114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7715e-5c6b-4754-a187-1b1d6e26c27d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.455520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f359aff-e490-45be-ad69-124d92e16c96 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.460425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f62aa-c642-4be9-b9b0-110e00ca337c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.461089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f62aa-c642-4be9-b9b0-110e00ca337c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.461379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f62aa-c642-4be9-b9b0-110e00ca337c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.461659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f62aa-c642-4be9-b9b0-110e00ca337c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.462295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f62aa-c642-4be9-b9b0-110e00ca337c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.462603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7715e-5c6b-4754-a187-1b1d6e26c27d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.463885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f62aa-c642-4be9-b9b0-110e00ca337c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.467915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ba77b-0874-4c8a-b725-50b79991cab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.468511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ba77b-0874-4c8a-b725-50b79991cab6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.468804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ba77b-0874-4c8a-b725-50b79991cab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.469112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ba77b-0874-4c8a-b725-50b79991cab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.469759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ba77b-0874-4c8a-b725-50b79991cab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.471361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ba77b-0874-4c8a-b725-50b79991cab6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.474113 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 795/1586] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.474895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f62aa-c642-4be9-b9b0-110e00ca337c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.482250 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 792/1587] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.483099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ba77b-0874-4c8a-b725-50b79991cab6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.485007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f62aa-c642-4be9-b9b0-110e00ca337c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.490056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19577ba9-2919-408e-a6f7-4b8c4ced0495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.490651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19577ba9-2919-408e-a6f7-4b8c4ced0495 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.490941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19577ba9-2919-408e-a6f7-4b8c4ced0495 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.491225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19577ba9-2919-408e-a6f7-4b8c4ced0495 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.491887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19577ba9-2919-408e-a6f7-4b8c4ced0495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.492366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ba77b-0874-4c8a-b725-50b79991cab6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.493514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19577ba9-2919-408e-a6f7-4b8c4ced0495 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.497002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06991827-1345-44a4-9ee4-f67249ffda62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.497583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06991827-1345-44a4-9ee4-f67249ffda62 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.497874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06991827-1345-44a4-9ee4-f67249ffda62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.498706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06991827-1345-44a4-9ee4-f67249ffda62 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.499340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06991827-1345-44a4-9ee4-f67249ffda62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.500931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06991827-1345-44a4-9ee4-f67249ffda62 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.507294 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 796/1588] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.507294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19577ba9-2919-408e-a6f7-4b8c4ced0495 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.512608 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 793/1589] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.513346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06991827-1345-44a4-9ee4-f67249ffda62 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.516527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19577ba9-2919-408e-a6f7-4b8c4ced0495 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.521501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a12bf-e3e6-4e66-bb91-2b32018ed6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.522128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a12bf-e3e6-4e66-bb91-2b32018ed6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.522412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a12bf-e3e6-4e66-bb91-2b32018ed6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.522692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a12bf-e3e6-4e66-bb91-2b32018ed6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.523325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a12bf-e3e6-4e66-bb91-2b32018ed6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.523765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06991827-1345-44a4-9ee4-f67249ffda62 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.524937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a12bf-e3e6-4e66-bb91-2b32018ed6d7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.528506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec693c6-efe6-47bf-b72a-0e8724c3cda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.529128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec693c6-efe6-47bf-b72a-0e8724c3cda8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.529452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec693c6-efe6-47bf-b72a-0e8724c3cda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.529735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec693c6-efe6-47bf-b72a-0e8724c3cda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.530371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec693c6-efe6-47bf-b72a-0e8724c3cda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.531958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec693c6-efe6-47bf-b72a-0e8724c3cda8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.538273 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 797/1590] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.538273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a12bf-e3e6-4e66-bb91-2b32018ed6d7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.545213 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 794/1591] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.545213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec693c6-efe6-47bf-b72a-0e8724c3cda8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.547569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a12bf-e3e6-4e66-bb91-2b32018ed6d7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.552357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec693c6-efe6-47bf-b72a-0e8724c3cda8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.552764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd55043-ed4f-4286-b125-c9b85a8d48aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.553399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd55043-ed4f-4286-b125-c9b85a8d48aa admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.553744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd55043-ed4f-4286-b125-c9b85a8d48aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.554036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd55043-ed4f-4286-b125-c9b85a8d48aa admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.554670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd55043-ed4f-4286-b125-c9b85a8d48aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.556428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd55043-ed4f-4286-b125-c9b85a8d48aa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.557085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a443418-dfeb-41b7-a4b1-e404eb365279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.557671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a443418-dfeb-41b7-a4b1-e404eb365279 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.557924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:42.558033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a443418-dfeb-41b7-a4b1-e404eb365279 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.558218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a443418-dfeb-41b7-a4b1-e404eb365279 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.558849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a443418-dfeb-41b7-a4b1-e404eb365279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.561110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a443418-dfeb-41b7-a4b1-e404eb365279 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.572598 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 798/1592] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.572598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd55043-ed4f-4286-b125-c9b85a8d48aa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.577039 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 795/1593] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.577039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a443418-dfeb-41b7-a4b1-e404eb365279 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.581092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd55043-ed4f-4286-b125-c9b85a8d48aa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.584962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a443418-dfeb-41b7-a4b1-e404eb365279 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.585660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b8a18-9098-4d5f-a8d0-ce75258ef7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.586339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b8a18-9098-4d5f-a8d0-ce75258ef7b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.586589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b8a18-9098-4d5f-a8d0-ce75258ef7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.586865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b8a18-9098-4d5f-a8d0-ce75258ef7b4 admin admin] PATH_INFO: `/v3/projects/18ee41edde4146b2a8cf7dbde2b5f5d2/users/5959424dab254b8e82add6ae6490b936/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.589616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc65743-39d9-40d4-b9ef-d737c3b478b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.590513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc65743-39d9-40d4-b9ef-d737c3b478b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.590801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc65743-39d9-40d4-b9ef-d737c3b478b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.591087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc65743-39d9-40d4-b9ef-d737c3b478b8 admin admin] PATH_INFO: `/v3/projects/123134ed03c540b79122e8ecaa021d3d/users/c34f1d07f48c4b4dadf97fd38709fd33/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.601514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b8a18-9098-4d5f-a8d0-ce75258ef7b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ee41edde4146b2a8cf7dbde2b5f5d2, user_id=5959424dab254b8e82add6ae6490b936, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.604162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b8a18-9098-4d5f-a8d0-ce75258ef7b4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.605976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc65743-39d9-40d4-b9ef-d737c3b478b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=123134ed03c540b79122e8ecaa021d3d, user_id=c34f1d07f48c4b4dadf97fd38709fd33, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.608685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc65743-39d9-40d4-b9ef-d737c3b478b8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.632177 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 799/1594] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:42 2022] PUT /identity/v3/projects/18ee41edde4146b2a8cf7dbde2b5f5d2/users/5959424dab254b8e82add6ae6490b936/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:42.632177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b8a18-9098-4d5f-a8d0-ce75258ef7b4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.632471 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 796/1595] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:42 2022] PUT /identity/v3/projects/123134ed03c540b79122e8ecaa021d3d/users/c34f1d07f48c4b4dadf97fd38709fd33/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:42.632471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc65743-39d9-40d4-b9ef-d737c3b478b8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.643222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:INSERT=16 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:42.698864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc65743-39d9-40d4-b9ef-d737c3b478b8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.701897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b8a18-9098-4d5f-a8d0-ce75258ef7b4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.703592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c53eb1-a7fc-4279-a8a4-e9e11d0fe7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.704255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c53eb1-a7fc-4279-a8a4-e9e11d0fe7ce admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.704544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c53eb1-a7fc-4279-a8a4-e9e11d0fe7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.704822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c53eb1-a7fc-4279-a8a4-e9e11d0fe7ce admin admin] PATH_INFO: `/v3/projects/1dacf31e1f0b4a4f9b27fc1f49006170/users/403b2dcf1b3f4e1fbead88207efa9ea0/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.706460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04eeae2-d339-462e-8aa1-f0614232dd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.707051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04eeae2-d339-462e-8aa1-f0614232dd27 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.707334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04eeae2-d339-462e-8aa1-f0614232dd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.708380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04eeae2-d339-462e-8aa1-f0614232dd27 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.708380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04eeae2-d339-462e-8aa1-f0614232dd27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.709889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04eeae2-d339-462e-8aa1-f0614232dd27 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.719848 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 800/1596] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.720295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c53eb1-a7fc-4279-a8a4-e9e11d0fe7ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dacf31e1f0b4a4f9b27fc1f49006170, user_id=403b2dcf1b3f4e1fbead88207efa9ea0, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.720735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04eeae2-d339-462e-8aa1-f0614232dd27 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.723528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c53eb1-a7fc-4279-a8a4-e9e11d0fe7ce admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.730480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04eeae2-d339-462e-8aa1-f0614232dd27 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.737274 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 797/1597] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:42 2022] PUT /identity/v3/projects/1dacf31e1f0b4a4f9b27fc1f49006170/users/403b2dcf1b3f4e1fbead88207efa9ea0/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:42.745775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c53eb1-a7fc-4279-a8a4-e9e11d0fe7ce admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.789347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b431f7-504e-4137-8a55-605aaa16345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.790010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b431f7-504e-4137-8a55-605aaa16345e admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.790345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b431f7-504e-4137-8a55-605aaa16345e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.790638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b431f7-504e-4137-8a55-605aaa16345e admin admin] PATH_INFO: `/v3/projects/0ecb6d714e9f4670ab6a27cac2269f96/users/622edda288714384a6a7aefae0c5be77/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.805467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b431f7-504e-4137-8a55-605aaa16345e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ecb6d714e9f4670ab6a27cac2269f96, user_id=622edda288714384a6a7aefae0c5be77, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.808180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b431f7-504e-4137-8a55-605aaa16345e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.812146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c53eb1-a7fc-4279-a8a4-e9e11d0fe7ce admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.817840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d58be-8ce2-42f4-b3f8-d817ba704096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.818523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d58be-8ce2-42f4-b3f8-d817ba704096 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.818815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d58be-8ce2-42f4-b3f8-d817ba704096 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.819101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d58be-8ce2-42f4-b3f8-d817ba704096 admin admin] PATH_INFO: `/v3/projects/993b7f76855c47e58427d7542a0f41c2/users/4c3a73b54fd54a23808657c7ed9e62cb/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.823025 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 801/1598] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:42 2022] PUT /identity/v3/projects/0ecb6d714e9f4670ab6a27cac2269f96/users/622edda288714384a6a7aefae0c5be77/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:42.823616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b431f7-504e-4137-8a55-605aaa16345e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.837177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d58be-8ce2-42f4-b3f8-d817ba704096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993b7f76855c47e58427d7542a0f41c2, user_id=4c3a73b54fd54a23808657c7ed9e62cb, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.839866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d58be-8ce2-42f4-b3f8-d817ba704096 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.854964 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 798/1599] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:42 2022] PUT /identity/v3/projects/993b7f76855c47e58427d7542a0f41c2/users/4c3a73b54fd54a23808657c7ed9e62cb/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:42.858294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d58be-8ce2-42f4-b3f8-d817ba704096 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:42.880278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:42.892824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b431f7-504e-4137-8a55-605aaa16345e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.924567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d58be-8ce2-42f4-b3f8-d817ba704096 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:42.929304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc6f37-374b-4fa5-83e0-5e065e40c132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.929918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc6f37-374b-4fa5-83e0-5e065e40c132 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.930206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc6f37-374b-4fa5-83e0-5e065e40c132 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.930487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc6f37-374b-4fa5-83e0-5e065e40c132 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.931136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc6f37-374b-4fa5-83e0-5e065e40c132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.932753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc6f37-374b-4fa5-83e0-5e065e40c132 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.943555 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 799/1600] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:42.944738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40fc04-822c-4bcf-ac41-86b64da059fd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.945076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40fc04-822c-4bcf-ac41-86b64da059fd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.945356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40fc04-822c-4bcf-ac41-86b64da059fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.955911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a5b34-147a-4166-9a43-8de721ff69de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:42.956511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a5b34-147a-4166-9a43-8de721ff69de admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.956820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a5b34-147a-4166-9a43-8de721ff69de admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.957124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a5b34-147a-4166-9a43-8de721ff69de admin admin] PATH_INFO: `/v3/projects/1270b5dbd21b49d0b595a2aebb99255c/users/c0f18ad3c7a64133bd7c5048a1fc566d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:42.964818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:42.973649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a5b34-147a-4166-9a43-8de721ff69de admin admin] RBAC: Authorizing `identity:create_grant(project_id=1270b5dbd21b49d0b595a2aebb99255c, user_id=c0f18ad3c7a64133bd7c5048a1fc566d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:42.975076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2d40fc04-822c-4bcf-ac41-86b64da059fd None None] MFA Rules not processed for user `5959424dab254b8e82add6ae6490b936`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:42.976303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2d40fc04-822c-4bcf-ac41-86b64da059fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:42.976388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a5b34-147a-4166-9a43-8de721ff69de admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:42.989988 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 802/1601] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:42 2022] PUT /identity/v3/projects/1270b5dbd21b49d0b595a2aebb99255c/users/c0f18ad3c7a64133bd7c5048a1fc566d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:42.991223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336600b-ba22-458a-a74f-c1c696c7a90d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:42.991586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336600b-ba22-458a-a74f-c1c696c7a90d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:42.991856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336600b-ba22-458a-a74f-c1c696c7a90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.022659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8336600b-ba22-458a-a74f-c1c696c7a90d None None] MFA Rules not processed for user `c34f1d07f48c4b4dadf97fd38709fd33`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:43.023886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8336600b-ba22-458a-a74f-c1c696c7a90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:43.050856 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 800/1602] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:43.051767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40fc04-822c-4bcf-ac41-86b64da059fd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.062657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40fc04-822c-4bcf-ac41-86b64da059fd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.090596 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 803/1603] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:43.092326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45118a-a179-48d4-b013-ca61f1228a88 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.092623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45118a-a179-48d4-b013-ca61f1228a88 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.092923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45118a-a179-48d4-b013-ca61f1228a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.122878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e619a-2085-41bc-bd7b-2b69a8d71fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.123634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e619a-2085-41bc-bd7b-2b69a8d71fee admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.123923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e619a-2085-41bc-bd7b-2b69a8d71fee admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.124196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7c45118a-a179-48d4-b013-ca61f1228a88 None None] MFA Rules not processed for user `403b2dcf1b3f4e1fbead88207efa9ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:43.124314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e619a-2085-41bc-bd7b-2b69a8d71fee admin admin] PATH_INFO: `/v3/projects/d3c752a733314acd887b75faf52c9a33/users/9c3be59ecad04039bf1777f2874efedb/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.125589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7c45118a-a179-48d4-b013-ca61f1228a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:43.140031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e619a-2085-41bc-bd7b-2b69a8d71fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c752a733314acd887b75faf52c9a33, user_id=9c3be59ecad04039bf1777f2874efedb, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.142764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e619a-2085-41bc-bd7b-2b69a8d71fee admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.162330 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 801/1604] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:43 2022] PUT /identity/v3/projects/d3c752a733314acd887b75faf52c9a33/users/9c3be59ecad04039bf1777f2874efedb/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:43.162330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e619a-2085-41bc-bd7b-2b69a8d71fee admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.200255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:43.204644 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 804/1605] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:43.209948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba627d49-3c01-4023-83ae-5d6b98e79a91 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.210318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba627d49-3c01-4023-83ae-5d6b98e79a91 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.210479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba627d49-3c01-4023-83ae-5d6b98e79a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.223148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e619a-2085-41bc-bd7b-2b69a8d71fee admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.227845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c3ebd-39e2-4436-83a2-634ca6b57278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.228457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c3ebd-39e2-4436-83a2-634ca6b57278 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.228746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c3ebd-39e2-4436-83a2-634ca6b57278 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.229164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c3ebd-39e2-4436-83a2-634ca6b57278 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.229805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c3ebd-39e2-4436-83a2-634ca6b57278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.231503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c3ebd-39e2-4436-83a2-634ca6b57278 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.239452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ba627d49-3c01-4023-83ae-5d6b98e79a91 None None] MFA Rules not processed for user `4c3a73b54fd54a23808657c7ed9e62cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:43.240769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ba627d49-3c01-4023-83ae-5d6b98e79a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:43.242227 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 802/1606] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:43.243020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c3ebd-39e2-4436-83a2-634ca6b57278 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.252556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c3ebd-39e2-4436-83a2-634ca6b57278 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.257179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f631c-c109-4502-b949-cd502b5af898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.257806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f631c-c109-4502-b949-cd502b5af898 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.258099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f631c-c109-4502-b949-cd502b5af898 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.258383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f631c-c109-4502-b949-cd502b5af898 admin admin] PATH_INFO: `/v3/projects/496175c02dcf4a7cb855531695233167/users/e9543728c6844cff9c97693fc37d35f4/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.272620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f631c-c109-4502-b949-cd502b5af898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=496175c02dcf4a7cb855531695233167, user_id=e9543728c6844cff9c97693fc37d35f4, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.275359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f631c-c109-4502-b949-cd502b5af898 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.277544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:43.289486 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 803/1607] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:43 2022] PUT /identity/v3/projects/496175c02dcf4a7cb855531695233167/users/e9543728c6844cff9c97693fc37d35f4/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:43.291020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f436b17-51a4-414c-9a49-083b934390db None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.291315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f436b17-51a4-414c-9a49-083b934390db None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.291648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f436b17-51a4-414c-9a49-083b934390db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.308930 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 805/1608] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:43.309792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba627d49-3c01-4023-83ae-5d6b98e79a91 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.320756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba627d49-3c01-4023-83ae-5d6b98e79a91 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.324239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8f436b17-51a4-414c-9a49-083b934390db None None] MFA Rules not processed for user `c0f18ad3c7a64133bd7c5048a1fc566d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:43.325752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8f436b17-51a4-414c-9a49-083b934390db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:43.377318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec7772-4b9d-4de3-872f-46086376c785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.377924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec7772-4b9d-4de3-872f-46086376c785 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.378212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec7772-4b9d-4de3-872f-46086376c785 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.378603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec7772-4b9d-4de3-872f-46086376c785 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.379235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec7772-4b9d-4de3-872f-46086376c785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.381170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec7772-4b9d-4de3-872f-46086376c785 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.393337 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 804/1609] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:43.394376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f436b17-51a4-414c-9a49-083b934390db None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.405628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f436b17-51a4-414c-9a49-083b934390db None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.410714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6ac9c-8a57-49eb-b794-7f3538e0088c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.411323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6ac9c-8a57-49eb-b794-7f3538e0088c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.411613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6ac9c-8a57-49eb-b794-7f3538e0088c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.411864 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 806/1610] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:43.411971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6ac9c-8a57-49eb-b794-7f3538e0088c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.412588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6ac9c-8a57-49eb-b794-7f3538e0088c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.413126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b1f49-470a-43fd-a3d3-fd7ad2f8e7b8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.413421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b1f49-470a-43fd-a3d3-fd7ad2f8e7b8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.413699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b1f49-470a-43fd-a3d3-fd7ad2f8e7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.414571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6ac9c-8a57-49eb-b794-7f3538e0088c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.440909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-098b1f49-470a-43fd-a3d3-fd7ad2f8e7b8 None None] MFA Rules not processed for user `9c3be59ecad04039bf1777f2874efedb`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:43.442137 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 805/1611] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:43.442288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-098b1f49-470a-43fd-a3d3-fd7ad2f8e7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:43.442951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6ac9c-8a57-49eb-b794-7f3538e0088c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.503845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6ac9c-8a57-49eb-b794-7f3538e0088c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.506030 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 807/1612] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:43.506841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b1f49-470a-43fd-a3d3-fd7ad2f8e7b8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.512111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3221c-e636-4e11-bfb9-a3c232d325c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.512730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3221c-e636-4e11-bfb9-a3c232d325c3 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.513056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3221c-e636-4e11-bfb9-a3c232d325c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.513162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:43.513339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3221c-e636-4e11-bfb9-a3c232d325c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.514502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3221c-e636-4e11-bfb9-a3c232d325c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.516584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3221c-e636-4e11-bfb9-a3c232d325c3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.518625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b1f49-470a-43fd-a3d3-fd7ad2f8e7b8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.524452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda40f7-c781-4a34-8066-f9e8337fffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.525133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda40f7-c781-4a34-8066-f9e8337fffb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.525419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda40f7-c781-4a34-8066-f9e8337fffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.525695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda40f7-c781-4a34-8066-f9e8337fffb6 admin admin] PATH_INFO: `/v3/projects/0ecb6d714e9f4670ab6a27cac2269f96/users/622edda288714384a6a7aefae0c5be77/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.527639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda40f7-c781-4a34-8066-f9e8337fffb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ecb6d714e9f4670ab6a27cac2269f96, user_id=622edda288714384a6a7aefae0c5be77, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.530920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda40f7-c781-4a34-8066-f9e8337fffb6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.541905 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 808/1613] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:43 2022] PUT /identity/v3/projects/0ecb6d714e9f4670ab6a27cac2269f96/users/622edda288714384a6a7aefae0c5be77/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:43.543088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a615a3-c565-4e44-81dd-516037d4ee94 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.543374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a615a3-c565-4e44-81dd-516037d4ee94 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.543648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a615a3-c565-4e44-81dd-516037d4ee94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.547608 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 806/1614] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:43.547608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3221c-e636-4e11-bfb9-a3c232d325c3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.572405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-12a615a3-c565-4e44-81dd-516037d4ee94 None None] MFA Rules not processed for user `e9543728c6844cff9c97693fc37d35f4`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:43.573696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-12a615a3-c565-4e44-81dd-516037d4ee94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:43.595803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:43.611316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3221c-e636-4e11-bfb9-a3c232d325c3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.616393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549bd64-91b6-4fc0-b393-b8c4114e3e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.617287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549bd64-91b6-4fc0-b393-b8c4114e3e31 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.617599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549bd64-91b6-4fc0-b393-b8c4114e3e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.617870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549bd64-91b6-4fc0-b393-b8c4114e3e31 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.618517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549bd64-91b6-4fc0-b393-b8c4114e3e31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.620453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549bd64-91b6-4fc0-b393-b8c4114e3e31 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.641806 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 809/1615] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:43.642787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a615a3-c565-4e44-81dd-516037d4ee94 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.653506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a615a3-c565-4e44-81dd-516037d4ee94 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.655010 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 807/1616] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:43.655880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549bd64-91b6-4fc0-b393-b8c4114e3e31 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.713619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41667a-8252-437f-aa3e-6da841c3aa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.714285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41667a-8252-437f-aa3e-6da841c3aa53 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.714503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41667a-8252-437f-aa3e-6da841c3aa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.714776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41667a-8252-437f-aa3e-6da841c3aa53 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.715664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41667a-8252-437f-aa3e-6da841c3aa53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.718169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41667a-8252-437f-aa3e-6da841c3aa53 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.718672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549bd64-91b6-4fc0-b393-b8c4114e3e31 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.723672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924bb40-6a8d-44d4-9e5c-7972e6393b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.724273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924bb40-6a8d-44d4-9e5c-7972e6393b1b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.724558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924bb40-6a8d-44d4-9e5c-7972e6393b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.724832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924bb40-6a8d-44d4-9e5c-7972e6393b1b admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.725512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924bb40-6a8d-44d4-9e5c-7972e6393b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.727410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924bb40-6a8d-44d4-9e5c-7972e6393b1b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.745151 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 810/1617] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:43.746140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41667a-8252-437f-aa3e-6da841c3aa53 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.751807 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 808/1618] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:43.752874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924bb40-6a8d-44d4-9e5c-7972e6393b1b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.762680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924bb40-6a8d-44d4-9e5c-7972e6393b1b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.809187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41667a-8252-437f-aa3e-6da841c3aa53 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.814473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928de6c-03f0-4df9-bc6a-69c88e2397cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.815084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928de6c-03f0-4df9-bc6a-69c88e2397cf admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.815364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928de6c-03f0-4df9-bc6a-69c88e2397cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.815639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928de6c-03f0-4df9-bc6a-69c88e2397cf admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.816267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928de6c-03f0-4df9-bc6a-69c88e2397cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.818483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928de6c-03f0-4df9-bc6a-69c88e2397cf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.823964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe44e4-e9e9-48d4-ae35-295e6cfb0ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.824584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe44e4-e9e9-48d4-ae35-295e6cfb0ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.824873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe44e4-e9e9-48d4-ae35-295e6cfb0ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.825260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe44e4-e9e9-48d4-ae35-295e6cfb0ca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.825926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe44e4-e9e9-48d4-ae35-295e6cfb0ca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.828456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe44e4-e9e9-48d4-ae35-295e6cfb0ca8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.839844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:43.851098 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 811/1619] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:43.852071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928de6c-03f0-4df9-bc6a-69c88e2397cf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.861606 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 809/1620] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/projects => generated 395 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:43.862185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928de6c-03f0-4df9-bc6a-69c88e2397cf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.863067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f2961-69a2-45bb-9b55-69900dfb1fe0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.863463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f2961-69a2-45bb-9b55-69900dfb1fe0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.863842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f2961-69a2-45bb-9b55-69900dfb1fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.896434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c26f2961-69a2-45bb-9b55-69900dfb1fe0 None None] MFA Rules not processed for user `622edda288714384a6a7aefae0c5be77`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:43.897713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c26f2961-69a2-45bb-9b55-69900dfb1fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:43.930446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d7862-de08-4532-a7d3-394a543e70b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.931064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d7862-de08-4532-a7d3-394a543e70b1 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.931352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d7862-de08-4532-a7d3-394a543e70b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.931632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d7862-de08-4532-a7d3-394a543e70b1 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.932274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d7862-de08-4532-a7d3-394a543e70b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.932829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:43.934297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d7862-de08-4532-a7d3-394a543e70b1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:43.957614 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 812/1621] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:43.958556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d7862-de08-4532-a7d3-394a543e70b1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.971818 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 810/1622] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:43.975780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f2961-69a2-45bb-9b55-69900dfb1fe0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:43.982972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f2961-69a2-45bb-9b55-69900dfb1fe0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:43.987599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775d7dd-52b1-4e37-9373-772f51363593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:43.988200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775d7dd-52b1-4e37-9373-772f51363593 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:43.988713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775d7dd-52b1-4e37-9373-772f51363593 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:43.988815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775d7dd-52b1-4e37-9373-772f51363593 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:43.989566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775d7dd-52b1-4e37-9373-772f51363593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:43.991882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775d7dd-52b1-4e37-9373-772f51363593 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.016969 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 811/1623] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:43 2022] POST /identity/v3/users => generated 300 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:44.017869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775d7dd-52b1-4e37-9373-772f51363593 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.021906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d7862-de08-4532-a7d3-394a543e70b1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.025601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cd5c1-92ea-448d-8962-008e53955bdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.026212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cd5c1-92ea-448d-8962-008e53955bdc service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.026499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cd5c1-92ea-448d-8962-008e53955bdc service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.026812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cd5c1-92ea-448d-8962-008e53955bdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.029264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775d7dd-52b1-4e37-9373-772f51363593 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.033872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec12ae-d176-400f-b748-40b97f5e19d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.034453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec12ae-d176-400f-b748-40b97f5e19d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.034761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec12ae-d176-400f-b748-40b97f5e19d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.035034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec12ae-d176-400f-b748-40b97f5e19d4 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.035651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec12ae-d176-400f-b748-40b97f5e19d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.037594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec12ae-d176-400f-b748-40b97f5e19d4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.058807 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 812/1624] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:44 2022] POST /identity/v3/users => generated 297 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:44.059860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec12ae-d176-400f-b748-40b97f5e19d4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.069749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec12ae-d176-400f-b748-40b97f5e19d4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.075177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06e924-e378-4d8d-ab3d-12c279716960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.075596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06e924-e378-4d8d-ab3d-12c279716960 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.075888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06e924-e378-4d8d-ab3d-12c279716960 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.076723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06e924-e378-4d8d-ab3d-12c279716960 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.077418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06e924-e378-4d8d-ab3d-12c279716960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.079044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06e924-e378-4d8d-ab3d-12c279716960 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.086966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cd5c1-92ea-448d-8962-008e53955bdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.088412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cd5c1-92ea-448d-8962-008e53955bdc service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.090456 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 813/1625] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:44.091161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06e924-e378-4d8d-ab3d-12c279716960 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.098364 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 813/1626] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:44.099222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cd5c1-92ea-448d-8962-008e53955bdc service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.100926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06e924-e378-4d8d-ab3d-12c279716960 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.106741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed78b74-a1a9-4d0a-a632-9fb793be2fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.107377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed78b74-a1a9-4d0a-a632-9fb793be2fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.107652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed78b74-a1a9-4d0a-a632-9fb793be2fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.107961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed78b74-a1a9-4d0a-a632-9fb793be2fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.108675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed78b74-a1a9-4d0a-a632-9fb793be2fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.109888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cd5c1-92ea-448d-8962-008e53955bdc service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.110360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed78b74-a1a9-4d0a-a632-9fb793be2fe6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.115458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce315de1-2a40-4d8c-ada9-1f07ad5999ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.116073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce315de1-2a40-4d8c-ada9-1f07ad5999ed admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.120768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce315de1-2a40-4d8c-ada9-1f07ad5999ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.120768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce315de1-2a40-4d8c-ada9-1f07ad5999ed admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.120768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce315de1-2a40-4d8c-ada9-1f07ad5999ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.120768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce315de1-2a40-4d8c-ada9-1f07ad5999ed admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.121209 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 814/1627] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:44.125642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed78b74-a1a9-4d0a-a632-9fb793be2fe6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.136372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed78b74-a1a9-4d0a-a632-9fb793be2fe6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.143090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38856-332f-4f41-8987-81deae38a4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.151521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38856-332f-4f41-8987-81deae38a4be admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.151860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38856-332f-4f41-8987-81deae38a4be admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.152214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38856-332f-4f41-8987-81deae38a4be admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.152964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d38856-332f-4f41-8987-81deae38a4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.154879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d38856-332f-4f41-8987-81deae38a4be admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.160681 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 814/1628] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:44 2022] POST /identity/v3/users => generated 304 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:44.166415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0372fa9-476d-485e-b0d7-892d6dfabd76 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.167011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0372fa9-476d-485e-b0d7-892d6dfabd76 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.168255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0372fa9-476d-485e-b0d7-892d6dfabd76 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.170636 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 815/1629] 10.222.0.16 () {54 vars in 962 bytes} [Sat Sep 3 20:48:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:48:44.171674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0372fa9-476d-485e-b0d7-892d6dfabd76 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.179585 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 815/1630] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:44.179585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38856-332f-4f41-8987-81deae38a4be admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.193854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0372fa9-476d-485e-b0d7-892d6dfabd76 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.195514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38856-332f-4f41-8987-81deae38a4be admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.199239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc0c9f-6721-47ef-b1d2-268bc587af0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.200008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc0c9f-6721-47ef-b1d2-268bc587af0f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.200232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbea36a-5930-4c37-b6e6-1094747938c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.200628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc0c9f-6721-47ef-b1d2-268bc587af0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.200844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbea36a-5930-4c37-b6e6-1094747938c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.201156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc0c9f-6721-47ef-b1d2-268bc587af0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.201242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbea36a-5930-4c37-b6e6-1094747938c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.201483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbea36a-5930-4c37-b6e6-1094747938c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.202164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc0c9f-6721-47ef-b1d2-268bc587af0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.202248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbea36a-5930-4c37-b6e6-1094747938c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.203799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbea36a-5930-4c37-b6e6-1094747938c7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.205204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc0c9f-6721-47ef-b1d2-268bc587af0f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.219259 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 816/1631] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:44.219259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbea36a-5930-4c37-b6e6-1094747938c7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.219521 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 816/1632] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:44.219852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc0c9f-6721-47ef-b1d2-268bc587af0f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.230964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc0c9f-6721-47ef-b1d2-268bc587af0f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.233701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbea36a-5930-4c37-b6e6-1094747938c7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.238337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae401c-f2ac-4477-9741-d164c375b389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.238503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421477c6-d815-4a0c-8f5d-3d8e48465939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.239012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae401c-f2ac-4477-9741-d164c375b389 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.239284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421477c6-d815-4a0c-8f5d-3d8e48465939 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.239393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae401c-f2ac-4477-9741-d164c375b389 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.239565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae401c-f2ac-4477-9741-d164c375b389 admin admin] PATH_INFO: `/v3/projects/b9ea36ea140a41f1b3e471eb4cf980ef/users/8cb69666ecf14b9bb5a96603c8163182/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.239742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421477c6-d815-4a0c-8f5d-3d8e48465939 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.240112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421477c6-d815-4a0c-8f5d-3d8e48465939 admin admin] PATH_INFO: `/v3/projects/9414f023ef63477b9062610de91f9ede/users/ff31a8e9b6584e059808c9ad887bd698/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.252501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:44.257141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae401c-f2ac-4477-9741-d164c375b389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9ea36ea140a41f1b3e471eb4cf980ef, user_id=8cb69666ecf14b9bb5a96603c8163182, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.259982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae401c-f2ac-4477-9741-d164c375b389 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.261524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421477c6-d815-4a0c-8f5d-3d8e48465939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9414f023ef63477b9062610de91f9ede, user_id=ff31a8e9b6584e059808c9ad887bd698, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.265407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421477c6-d815-4a0c-8f5d-3d8e48465939 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.274609 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 817/1633] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:44 2022] PUT /identity/v3/projects/b9ea36ea140a41f1b3e471eb4cf980ef/users/8cb69666ecf14b9bb5a96603c8163182/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:44.278527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae401c-f2ac-4477-9741-d164c375b389 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.284795 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 817/1634] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:44 2022] PUT /identity/v3/projects/9414f023ef63477b9062610de91f9ede/users/ff31a8e9b6584e059808c9ad887bd698/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:44.284795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2632-ce22-40fb-86b2-0855feb9803d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.284795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2632-ce22-40fb-86b2-0855feb9803d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.284795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2632-ce22-40fb-86b2-0855feb9803d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.331760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-acbe2632-ce22-40fb-86b2-0855feb9803d None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:44.333135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-acbe2632-ce22-40fb-86b2-0855feb9803d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:44.354930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae401c-f2ac-4477-9741-d164c375b389 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.392818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:44.416113 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 818/1635] 10.222.0.16 () {58 vars in 1059 bytes} [Sat Sep 3 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:44.416996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe2632-ce22-40fb-86b2-0855feb9803d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.422937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61f72a-55b9-40fa-93ee-ea7471f6b816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.423635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61f72a-55b9-40fa-93ee-ea7471f6b816 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.423996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61f72a-55b9-40fa-93ee-ea7471f6b816 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.424365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61f72a-55b9-40fa-93ee-ea7471f6b816 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.425041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61f72a-55b9-40fa-93ee-ea7471f6b816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.428973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe2632-ce22-40fb-86b2-0855feb9803d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.429906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61f72a-55b9-40fa-93ee-ea7471f6b816 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.434647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed5b80-ea89-48c7-8df8-b47d4b36457d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.436087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed5b80-ea89-48c7-8df8-b47d4b36457d admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.436458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed5b80-ea89-48c7-8df8-b47d4b36457d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.437153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed5b80-ea89-48c7-8df8-b47d4b36457d admin admin] PATH_INFO: `/v3/projects/76995bf5b0394ecf93109e9dc87b8219/users/57bb8fae155a469a972a4feddbfffbb6/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.442175 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 818/1636] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:44.445832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61f72a-55b9-40fa-93ee-ea7471f6b816 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.453268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed5b80-ea89-48c7-8df8-b47d4b36457d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76995bf5b0394ecf93109e9dc87b8219, user_id=57bb8fae155a469a972a4feddbfffbb6, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.455762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61f72a-55b9-40fa-93ee-ea7471f6b816 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.455937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed5b80-ea89-48c7-8df8-b47d4b36457d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.460390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33998917-9b5f-4061-8c5a-b85dffa3a5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.461211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998917-9b5f-4061-8c5a-b85dffa3a5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.461450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998917-9b5f-4061-8c5a-b85dffa3a5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.461733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998917-9b5f-4061-8c5a-b85dffa3a5b7 admin admin] PATH_INFO: `/v3/projects/5b3de50f698a4292858f898d092256af/users/2276474f1cb7428cb6b479b7926609a7/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.469499 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 819/1637] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:44 2022] PUT /identity/v3/projects/76995bf5b0394ecf93109e9dc87b8219/users/57bb8fae155a469a972a4feddbfffbb6/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:44.470352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed5b80-ea89-48c7-8df8-b47d4b36457d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.479661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33998917-9b5f-4061-8c5a-b85dffa3a5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b3de50f698a4292858f898d092256af, user_id=2276474f1cb7428cb6b479b7926609a7, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.483187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33998917-9b5f-4061-8c5a-b85dffa3a5b7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.496199 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 819/1638] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:44 2022] PUT /identity/v3/projects/5b3de50f698a4292858f898d092256af/users/2276474f1cb7428cb6b479b7926609a7/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:44.497223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33998917-9b5f-4061-8c5a-b85dffa3a5b7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.537328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed5b80-ea89-48c7-8df8-b47d4b36457d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.560253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33998917-9b5f-4061-8c5a-b85dffa3a5b7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.565680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5533777-bbad-4f4d-8b7d-ed0137690e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.566291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5533777-bbad-4f4d-8b7d-ed0137690e59 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.566582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5533777-bbad-4f4d-8b7d-ed0137690e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.566863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5533777-bbad-4f4d-8b7d-ed0137690e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.567506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5533777-bbad-4f4d-8b7d-ed0137690e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.569134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5533777-bbad-4f4d-8b7d-ed0137690e59 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.578909 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 820/1639] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:44.579697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5533777-bbad-4f4d-8b7d-ed0137690e59 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.583021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:44.590412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5533777-bbad-4f4d-8b7d-ed0137690e59 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.595562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47152ef4-fdb5-4f3d-b446-0c7a2504b451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.596153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47152ef4-fdb5-4f3d-b446-0c7a2504b451 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.596441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47152ef4-fdb5-4f3d-b446-0c7a2504b451 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.596719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47152ef4-fdb5-4f3d-b446-0c7a2504b451 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.597392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47152ef4-fdb5-4f3d-b446-0c7a2504b451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.597725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d5e75-2ee9-4c21-8a1b-0b62b6b70f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.598424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d5e75-2ee9-4c21-8a1b-0b62b6b70f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.598742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d5e75-2ee9-4c21-8a1b-0b62b6b70f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.598964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47152ef4-fdb5-4f3d-b446-0c7a2504b451 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.599063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d5e75-2ee9-4c21-8a1b-0b62b6b70f34 admin admin] PATH_INFO: `/v3/projects/a293794868cd4c749b6eaa82e23e0500/users/8b5729e885694c2caf13e018c54a95bf/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.609427 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 821/1640] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:44.610224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47152ef4-fdb5-4f3d-b446-0c7a2504b451 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.616511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d5e75-2ee9-4c21-8a1b-0b62b6b70f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a293794868cd4c749b6eaa82e23e0500, user_id=8b5729e885694c2caf13e018c54a95bf, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.619281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d5e75-2ee9-4c21-8a1b-0b62b6b70f34 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.632502 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 820/1641] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:44 2022] PUT /identity/v3/projects/a293794868cd4c749b6eaa82e23e0500/users/8b5729e885694c2caf13e018c54a95bf/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:44.635587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d5e75-2ee9-4c21-8a1b-0b62b6b70f34 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.686517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47152ef4-fdb5-4f3d-b446-0c7a2504b451 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.694729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d5e75-2ee9-4c21-8a1b-0b62b6b70f34 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.699399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.700031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.700315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.700592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] PATH_INFO: `/v3/users/e9543728c6844cff9c97693fc37d35f4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.701770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9543728c6844cff9c97693fc37d35f4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.702795 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:48:44.702795 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 20:48:44.702795 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 20:48:44.704005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.715960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:44.756662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1b255-c42f-403e-ac5c-d8ebfb952b4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.757373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1b255-c42f-403e-ac5c-d8ebfb952b4b service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.757597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1b255-c42f-403e-ac5c-d8ebfb952b4b service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.757862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1b255-c42f-403e-ac5c-d8ebfb952b4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.765654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9543728c6844cff9c97693fc37d35f4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:44.785207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9543728c6844cff9c97693fc37d35f4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:44.785901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9543728c6844cff9c97693fc37d35f4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:44.796380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9543728c6844cff9c97693fc37d35f4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:44.805519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9543728c6844cff9c97693fc37d35f4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:44.812459 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 821/1642] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:48:44 2022] DELETE /identity/v3/users/e9543728c6844cff9c97693fc37d35f4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:44.816231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.817624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:44.819998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1b255-c42f-403e-ac5c-d8ebfb952b4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.821779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1b255-c42f-403e-ac5c-d8ebfb952b4b service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.832688 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 822/1643] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:44 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:44.833585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1b255-c42f-403e-ac5c-d8ebfb952b4b service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.844621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1b255-c42f-403e-ac5c-d8ebfb952b4b service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.877059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:44.893565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cf676-3839-47cb-abab-09699e33e897 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:44.898917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12485df0-4178-4e8e-9ae9-04069e82c593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.899620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12485df0-4178-4e8e-9ae9-04069e82c593 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.899940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12485df0-4178-4e8e-9ae9-04069e82c593 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.900220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12485df0-4178-4e8e-9ae9-04069e82c593 admin admin] PATH_INFO: `/v3/projects/04232849b122407b8256919ca8724783/users/a430e034949d4b619b1620b1021bcf45/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.912274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12925c-2357-4a8b-9ec9-4285b495172c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:44.913098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12925c-2357-4a8b-9ec9-4285b495172c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:44.913528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12925c-2357-4a8b-9ec9-4285b495172c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:44.914016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12925c-2357-4a8b-9ec9-4285b495172c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:44.914516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12485df0-4178-4e8e-9ae9-04069e82c593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04232849b122407b8256919ca8724783, user_id=a430e034949d4b619b1620b1021bcf45, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.914998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12925c-2357-4a8b-9ec9-4285b495172c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:44.916953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12925c-2357-4a8b-9ec9-4285b495172c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.917231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12485df0-4178-4e8e-9ae9-04069e82c593 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:44.929694 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 823/1644] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:44.930817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12925c-2357-4a8b-9ec9-4285b495172c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.932665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5c12925c-2357-4a8b-9ec9-4285b495172c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:44.933238 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 822/1645] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:44 2022] PUT /identity/v3/projects/04232849b122407b8256919ca8724783/users/a430e034949d4b619b1620b1021bcf45/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:44.934208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12485df0-4178-4e8e-9ae9-04069e82c593 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:44.936767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-12485df0-4178-4e8e-9ae9-04069e82c593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:44.947075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12925c-2357-4a8b-9ec9-4285b495172c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:45.004432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12485df0-4178-4e8e-9ae9-04069e82c593 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:45.009603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7e44b-6ecc-4a32-af2c-efd6beab29dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:45.010288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef904664-6550-4145-87e1-cad6acfe581c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:45.010397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7e44b-6ecc-4a32-af2c-efd6beab29dc admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.010792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7e44b-6ecc-4a32-af2c-efd6beab29dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.011148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7e44b-6ecc-4a32-af2c-efd6beab29dc admin admin] PATH_INFO: `/v3/projects/b9ea36ea140a41f1b3e471eb4cf980ef/users/8cb69666ecf14b9bb5a96603c8163182/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.011231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef904664-6550-4145-87e1-cad6acfe581c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.011586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef904664-6550-4145-87e1-cad6acfe581c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.011942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef904664-6550-4145-87e1-cad6acfe581c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.012667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef904664-6550-4145-87e1-cad6acfe581c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:45.013370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7e44b-6ecc-4a32-af2c-efd6beab29dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9ea36ea140a41f1b3e471eb4cf980ef, user_id=8cb69666ecf14b9bb5a96603c8163182, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:45.014467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef904664-6550-4145-87e1-cad6acfe581c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:45.015928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:45.016694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7e44b-6ecc-4a32-af2c-efd6beab29dc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:45.026651 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 824/1646] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:45.027528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef904664-6550-4145-87e1-cad6acfe581c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:45.036928 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 823/1647] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:44 2022] PUT /identity/v3/projects/b9ea36ea140a41f1b3e471eb4cf980ef/users/8cb69666ecf14b9bb5a96603c8163182/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:45.036928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7e44b-6ecc-4a32-af2c-efd6beab29dc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:45.037167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ef904664-6550-4145-87e1-cad6acfe581c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:45.039231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ddb7e44b-6ecc-4a32-af2c-efd6beab29dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:45.042897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef904664-6550-4145-87e1-cad6acfe581c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:45.110885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb776f27-0719-4260-bb31-2403c9fe6a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:45.111935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb776f27-0719-4260-bb31-2403c9fe6a77 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.112286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb776f27-0719-4260-bb31-2403c9fe6a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.112633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb776f27-0719-4260-bb31-2403c9fe6a77 admin admin] PATH_INFO: `/v3/projects/9414f023ef63477b9062610de91f9ede/users/ff31a8e9b6584e059808c9ad887bd698/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.114767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb776f27-0719-4260-bb31-2403c9fe6a77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9414f023ef63477b9062610de91f9ede, user_id=ff31a8e9b6584e059808c9ad887bd698, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:45.118517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb776f27-0719-4260-bb31-2403c9fe6a77 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:45.121740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7e44b-6ecc-4a32-af2c-efd6beab29dc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:45.128070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c01240-62e3-4c10-820c-62784a89678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:45.129378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c01240-62e3-4c10-820c-62784a89678d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.129378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c01240-62e3-4c10-820c-62784a89678d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.129743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c01240-62e3-4c10-820c-62784a89678d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.132855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c01240-62e3-4c10-820c-62784a89678d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:45.132855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c01240-62e3-4c10-820c-62784a89678d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:45.138272 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 825/1648] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:45 2022] PUT /identity/v3/projects/9414f023ef63477b9062610de91f9ede/users/ff31a8e9b6584e059808c9ad887bd698/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:45.139780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9316580-4113-4ebd-b976-4aaaa00b2f43 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.140158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9316580-4113-4ebd-b976-4aaaa00b2f43 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.140491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9316580-4113-4ebd-b976-4aaaa00b2f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.152276 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 824/1649] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:45.152422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c01240-62e3-4c10-820c-62784a89678d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:45.154151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f4c01240-62e3-4c10-820c-62784a89678d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:45.165674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e9316580-4113-4ebd-b976-4aaaa00b2f43 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:45.166981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e9316580-4113-4ebd-b976-4aaaa00b2f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:45.167842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c01240-62e3-4c10-820c-62784a89678d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:45.233456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dd348-5870-4aed-b813-784dea73c438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:45.234345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dd348-5870-4aed-b813-784dea73c438 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.234782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dd348-5870-4aed-b813-784dea73c438 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.235067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dd348-5870-4aed-b813-784dea73c438 admin admin] PATH_INFO: `/v3/projects/76995bf5b0394ecf93109e9dc87b8219/users/57bb8fae155a469a972a4feddbfffbb6/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.237371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dd348-5870-4aed-b813-784dea73c438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76995bf5b0394ecf93109e9dc87b8219, user_id=57bb8fae155a469a972a4feddbfffbb6, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:45.242250 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 826/1650] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:45.242250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9316580-4113-4ebd-b976-4aaaa00b2f43 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:45.242477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dd348-5870-4aed-b813-784dea73c438 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:45.252264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:45.265596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9316580-4113-4ebd-b976-4aaaa00b2f43 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:45.266676 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 825/1651] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:45 2022] PUT /identity/v3/projects/76995bf5b0394ecf93109e9dc87b8219/users/57bb8fae155a469a972a4feddbfffbb6/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:45.267659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dd348-5870-4aed-b813-784dea73c438 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:45.338207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f150c-dfcd-4f6f-9467-e8ee4e5242e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:45.339002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f150c-dfcd-4f6f-9467-e8ee4e5242e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.339373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f150c-dfcd-4f6f-9467-e8ee4e5242e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.339640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f150c-dfcd-4f6f-9467-e8ee4e5242e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.340551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f150c-dfcd-4f6f-9467-e8ee4e5242e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:45.341251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dd348-5870-4aed-b813-784dea73c438 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:45.343213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f150c-dfcd-4f6f-9467-e8ee4e5242e2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:45.346632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41746f8-35ac-4f7b-8bb7-4c18bafc34e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:45.347293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41746f8-35ac-4f7b-8bb7-4c18bafc34e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.347578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41746f8-35ac-4f7b-8bb7-4c18bafc34e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.347868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41746f8-35ac-4f7b-8bb7-4c18bafc34e4 admin admin] PATH_INFO: `/v3/projects/5b3de50f698a4292858f898d092256af/users/2276474f1cb7428cb6b479b7926609a7/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.351237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41746f8-35ac-4f7b-8bb7-4c18bafc34e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b3de50f698a4292858f898d092256af, user_id=2276474f1cb7428cb6b479b7926609a7, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:45.353769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41746f8-35ac-4f7b-8bb7-4c18bafc34e4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:45.355466 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 827/1652] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:45.356641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55133b8-da5a-463a-9fb7-a1d83130303f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.357090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55133b8-da5a-463a-9fb7-a1d83130303f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.357533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55133b8-da5a-463a-9fb7-a1d83130303f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.370635 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 826/1653] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:45 2022] PUT /identity/v3/projects/5b3de50f698a4292858f898d092256af/users/2276474f1cb7428cb6b479b7926609a7/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:45.372234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39aec51-e069-4e2f-81dc-6a0120223510 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.372531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39aec51-e069-4e2f-81dc-6a0120223510 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.372816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39aec51-e069-4e2f-81dc-6a0120223510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.399142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d55133b8-da5a-463a-9fb7-a1d83130303f None None] MFA Rules not processed for user `8cb69666ecf14b9bb5a96603c8163182`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:45.405397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d55133b8-da5a-463a-9fb7-a1d83130303f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:45.407623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a39aec51-e069-4e2f-81dc-6a0120223510 None None] MFA Rules not processed for user `ff31a8e9b6584e059808c9ad887bd698`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:45.408834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a39aec51-e069-4e2f-81dc-6a0120223510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:45.422170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:45.500190 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 827/1654] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:45.501310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39aec51-e069-4e2f-81dc-6a0120223510 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:45.517037 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 828/1655] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:45.518325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55133b8-da5a-463a-9fb7-a1d83130303f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:45.519674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d55133b8-da5a-463a-9fb7-a1d83130303f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:45.583673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55133b8-da5a-463a-9fb7-a1d83130303f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:45.587630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6878f-e9aa-45e0-942f-4d5fc147d5eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:45.588333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6878f-e9aa-45e0-942f-4d5fc147d5eb service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.588561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6878f-e9aa-45e0-942f-4d5fc147d5eb service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.588862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6878f-e9aa-45e0-942f-4d5fc147d5eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.596067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39aec51-e069-4e2f-81dc-6a0120223510 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:45.598333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6878f-e9aa-45e0-942f-4d5fc147d5eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:45.599747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6878f-e9aa-45e0-942f-4d5fc147d5eb service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:45.600664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ea4c3-643a-43c8-b0f3-6bb33405758e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:45.601359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ea4c3-643a-43c8-b0f3-6bb33405758e admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.601658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ea4c3-643a-43c8-b0f3-6bb33405758e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.601938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ea4c3-643a-43c8-b0f3-6bb33405758e admin admin] PATH_INFO: `/v3/projects/a293794868cd4c749b6eaa82e23e0500/users/8b5729e885694c2caf13e018c54a95bf/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.603839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ea4c3-643a-43c8-b0f3-6bb33405758e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a293794868cd4c749b6eaa82e23e0500, user_id=8b5729e885694c2caf13e018c54a95bf, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:45.606546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ea4c3-643a-43c8-b0f3-6bb33405758e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:45.609707 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 829/1656] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:45.611221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede8683-1560-4898-8122-7ddd0a11502b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.611558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede8683-1560-4898-8122-7ddd0a11502b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.611893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede8683-1560-4898-8122-7ddd0a11502b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.618302 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 828/1657] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:45 2022] PUT /identity/v3/projects/a293794868cd4c749b6eaa82e23e0500/users/8b5729e885694c2caf13e018c54a95bf/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:45.619249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ea4c3-643a-43c8-b0f3-6bb33405758e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:45.634556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:45.645705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2ede8683-1560-4898-8122-7ddd0a11502b None None] MFA Rules not processed for user `57bb8fae155a469a972a4feddbfffbb6`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:45.647062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2ede8683-1560-4898-8122-7ddd0a11502b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:45.703605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ea4c3-643a-43c8-b0f3-6bb33405758e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:45.710268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6bb6e-f6a7-4fd1-9392-f0ebc4b7853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:45.711199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6bb6e-f6a7-4fd1-9392-f0ebc4b7853d admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.711414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6bb6e-f6a7-4fd1-9392-f0ebc4b7853d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.711770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6bb6e-f6a7-4fd1-9392-f0ebc4b7853d admin admin] PATH_INFO: `/v3/projects/04232849b122407b8256919ca8724783/users/a430e034949d4b619b1620b1021bcf45/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.714320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6bb6e-f6a7-4fd1-9392-f0ebc4b7853d admin admin] RBAC: Authorizing `identity:create_grant(project_id=04232849b122407b8256919ca8724783, user_id=a430e034949d4b619b1620b1021bcf45, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:45.717556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6bb6e-f6a7-4fd1-9392-f0ebc4b7853d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:45.730060 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 829/1658] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:45 2022] PUT /identity/v3/projects/04232849b122407b8256919ca8724783/users/a430e034949d4b619b1620b1021bcf45/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:45.731477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762714c-d7e8-4e1f-8c5e-3b9c3e98c66e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.731781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762714c-d7e8-4e1f-8c5e-3b9c3e98c66e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.732058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762714c-d7e8-4e1f-8c5e-3b9c3e98c66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.735004 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 830/1659] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:45.736380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c85fd-7822-4986-94ff-6fc86fc18e0c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.736679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c85fd-7822-4986-94ff-6fc86fc18e0c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.737000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c85fd-7822-4986-94ff-6fc86fc18e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.756985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b87c85fd-7822-4986-94ff-6fc86fc18e0c None None] MFA Rules not processed for user `c34f1d07f48c4b4dadf97fd38709fd33`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:45.758221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b87c85fd-7822-4986-94ff-6fc86fc18e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:45.771096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1762714c-d7e8-4e1f-8c5e-3b9c3e98c66e None None] MFA Rules not processed for user `2276474f1cb7428cb6b479b7926609a7`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:45.772129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1762714c-d7e8-4e1f-8c5e-3b9c3e98c66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:45.808549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:45.836008 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 831/1660] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:45.840643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba96e7c-f6a9-4b51-a90e-aa9660e560df None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.841052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba96e7c-f6a9-4b51-a90e-aa9660e560df None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.841458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba96e7c-f6a9-4b51-a90e-aa9660e560df None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.843402 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 832/1661] 10.222.0.16 () {54 vars in 962 bytes} [Sat Sep 3 20:48:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:48:45.844755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b56b68-65d8-4875-b67a-b2563196a4b7 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.845085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b56b68-65d8-4875-b67a-b2563196a4b7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.845368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b56b68-65d8-4875-b67a-b2563196a4b7 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.847330 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 833/1662] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:48:45.848446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7daf4-b9f7-47d7-a824-9bb83d3f8052 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.848742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7daf4-b9f7-47d7-a824-9bb83d3f8052 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.849050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7daf4-b9f7-47d7-a824-9bb83d3f8052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.869387 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 830/1663] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:45.869387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5124a0-6ca7-4b78-9188-02ad1343696f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.869387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5124a0-6ca7-4b78-9188-02ad1343696f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.869387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5124a0-6ca7-4b78-9188-02ad1343696f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.881503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-83d7daf4-b9f7-47d7-a824-9bb83d3f8052 None None] MFA Rules not processed for user `8b5729e885694c2caf13e018c54a95bf`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:45.882832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-83d7daf4-b9f7-47d7-a824-9bb83d3f8052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:45.900284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bb5124a0-6ca7-4b78-9188-02ad1343696f None None] MFA Rules not processed for user `a430e034949d4b619b1620b1021bcf45`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:45.902106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bb5124a0-6ca7-4b78-9188-02ad1343696f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:45.973846 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 834/1664] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:45.975421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c078f25-5a78-4568-a80e-72f9cec9bbe4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.976126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c078f25-5a78-4568-a80e-72f9cec9bbe4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.976468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c078f25-5a78-4568-a80e-72f9cec9bbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.987598 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 831/1665] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:45.988844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc3f7d-a879-4628-a92c-4d0d82e3d45c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:45.989162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc3f7d-a879-4628-a92c-4d0d82e3d45c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:45.989443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc3f7d-a879-4628-a92c-4d0d82e3d45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:45.999983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3c078f25-5a78-4568-a80e-72f9cec9bbe4 None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:46.001239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3c078f25-5a78-4568-a80e-72f9cec9bbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.009631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8ecc3f7d-a879-4628-a92c-4d0d82e3d45c None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:46.010820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8ecc3f7d-a879-4628-a92c-4d0d82e3d45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.020308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:46.070088 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 835/1666] 10.222.0.16 () {58 vars in 1059 bytes} [Sat Sep 3 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:46.071117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c078f25-5a78-4568-a80e-72f9cec9bbe4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.072430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3c078f25-5a78-4568-a80e-72f9cec9bbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.084741 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 832/1667] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:46.086496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e3165-e92e-4555-abdb-7ebf48577c66 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.086798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e3165-e92e-4555-abdb-7ebf48577c66 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.087083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e3165-e92e-4555-abdb-7ebf48577c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.108658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4a9e3165-e92e-4555-abdb-7ebf48577c66 None None] MFA Rules not processed for user `4c3a73b54fd54a23808657c7ed9e62cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:46.109916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4a9e3165-e92e-4555-abdb-7ebf48577c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.146642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c078f25-5a78-4568-a80e-72f9cec9bbe4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.151664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.152303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.152594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.152878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] PATH_INFO: `/v3/projects/496175c02dcf4a7cb855531695233167` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.154064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=496175c02dcf4a7cb855531695233167)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.155599 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 20:48:46.155599 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 20:48:46.155599 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 20:48:46.156826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.159215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:46.166923 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 833/1668] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:46.169047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e3165-e92e-4555-abdb-7ebf48577c66 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.183984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e3165-e92e-4555-abdb-7ebf48577c66 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.190416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c667e-3637-4e79-9a0d-8c8f583ee9c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.191107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c667e-3637-4e79-9a0d-8c8f583ee9c8 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.192188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c667e-3637-4e79-9a0d-8c8f583ee9c8 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.192548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c667e-3637-4e79-9a0d-8c8f583ee9c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.215268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '496175c02dcf4a7cb855531695233167', 'request_id': 'req-92bb5759-082f-4b87-954b-f9e4d0b340d5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:46.224852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '496175c02dcf4a7cb855531695233167', 'request_id': 'req-92bb5759-082f-4b87-954b-f9e4d0b340d5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:46.240080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '496175c02dcf4a7cb855531695233167', 'request_id': 'req-92bb5759-082f-4b87-954b-f9e4d0b340d5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:46.251701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] The token cache is being invalidate because project 496175c02dcf4a7cb855531695233167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:48:46.251999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:46.252669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:46.254222 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 836/1669] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:48:46 2022] DELETE /identity/v3/projects/496175c02dcf4a7cb855531695233167 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:46.254335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c667e-3637-4e79-9a0d-8c8f583ee9c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.255951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c667e-3637-4e79-9a0d-8c8f583ee9c8 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.260226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.261621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.266265 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 834/1670] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:46.267782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19daca3e-ccaa-4383-95a5-6034798054b9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.268607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19daca3e-ccaa-4383-95a5-6034798054b9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.268912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19daca3e-ccaa-4383-95a5-6034798054b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.289309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-19daca3e-ccaa-4383-95a5-6034798054b9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:46.290539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-19daca3e-ccaa-4383-95a5-6034798054b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.337665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:46.346732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb5759-082f-4b87-954b-f9e4d0b340d5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.352173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d630ca-38b7-4c94-9422-fea392955b71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.353017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d630ca-38b7-4c94-9422-fea392955b71 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.360687 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 835/1671] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:46.360828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d630ca-38b7-4c94-9422-fea392955b71 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.360948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19daca3e-ccaa-4383-95a5-6034798054b9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.361040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d630ca-38b7-4c94-9422-fea392955b71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.361210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-19daca3e-ccaa-4383-95a5-6034798054b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.361294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-38d630ca-38b7-4c94-9422-fea392955b71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.369169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d630ca-38b7-4c94-9422-fea392955b71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.370855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d630ca-38b7-4c94-9422-fea392955b71 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.380805 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 837/1672] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:46.383245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692b707-b7d7-45c2-b496-a00198b33d37 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.383706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692b707-b7d7-45c2-b496-a00198b33d37 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.384009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692b707-b7d7-45c2-b496-a00198b33d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.406099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8692b707-b7d7-45c2-b496-a00198b33d37 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:46.407313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8692b707-b7d7-45c2-b496-a00198b33d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.415866 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 838/1673] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:46.419981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692b707-b7d7-45c2-b496-a00198b33d37 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.434186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19daca3e-ccaa-4383-95a5-6034798054b9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.434319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692b707-b7d7-45c2-b496-a00198b33d37 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.439274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ebe4f-0299-4be6-8105-395c16d82059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.439878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ebe4f-0299-4be6-8105-395c16d82059 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.440161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ebe4f-0299-4be6-8105-395c16d82059 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.440265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b48cc-d901-45c7-83b4-a64764330ddc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.440442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ebe4f-0299-4be6-8105-395c16d82059 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.440810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b48cc-d901-45c7-83b4-a64764330ddc service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.441117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ebe4f-0299-4be6-8105-395c16d82059 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:48:46.441223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b48cc-d901-45c7-83b4-a64764330ddc service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.441405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b48cc-d901-45c7-83b4-a64764330ddc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.441506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ebe4f-0299-4be6-8105-395c16d82059 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.443009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ebe4f-0299-4be6-8105-395c16d82059 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.443236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-471b48cc-d901-45c7-83b4-a64764330ddc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.456948 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 839/1674] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:48:46.461192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ebe4f-0299-4be6-8105-395c16d82059 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.475560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ebe4f-0299-4be6-8105-395c16d82059 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.480137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879e97c-9c2f-4b88-a11a-8ec0a2db0763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.481056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879e97c-9c2f-4b88-a11a-8ec0a2db0763 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.481506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879e97c-9c2f-4b88-a11a-8ec0a2db0763 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.481950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879e97c-9c2f-4b88-a11a-8ec0a2db0763 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.482740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879e97c-9c2f-4b88-a11a-8ec0a2db0763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.484647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879e97c-9c2f-4b88-a11a-8ec0a2db0763 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.512536 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 840/1675] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:46 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:46.514084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879e97c-9c2f-4b88-a11a-8ec0a2db0763 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.515864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3879e97c-9c2f-4b88-a11a-8ec0a2db0763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.527880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b48cc-d901-45c7-83b4-a64764330ddc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.530280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b48cc-d901-45c7-83b4-a64764330ddc service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.540394 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 836/1676] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:46.541309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b48cc-d901-45c7-83b4-a64764330ddc service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.551107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b48cc-d901-45c7-83b4-a64764330ddc service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.561916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:46.605455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879e97c-9c2f-4b88-a11a-8ec0a2db0763 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.619037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce12c2-b670-4edb-bc96-37842756823e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.619037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce12c2-b670-4edb-bc96-37842756823e service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.619037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce12c2-b670-4edb-bc96-37842756823e service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.619037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce12c2-b670-4edb-bc96-37842756823e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.619391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-47ce12c2-b670-4edb-bc96-37842756823e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.628526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a65b38-9111-4dec-b2b6-ca578fa3e0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.629242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a65b38-9111-4dec-b2b6-ca578fa3e0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.629509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a65b38-9111-4dec-b2b6-ca578fa3e0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.629889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a65b38-9111-4dec-b2b6-ca578fa3e0f1 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.630570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a65b38-9111-4dec-b2b6-ca578fa3e0f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.632510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a65b38-9111-4dec-b2b6-ca578fa3e0f1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.659201 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 837/1677] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:46 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:46.659201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a65b38-9111-4dec-b2b6-ca578fa3e0f1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.671412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a65b38-9111-4dec-b2b6-ca578fa3e0f1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.677068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a3bdd-de7c-4910-8665-a8e984a90e23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.677675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a3bdd-de7c-4910-8665-a8e984a90e23 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.677965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a3bdd-de7c-4910-8665-a8e984a90e23 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.678245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a3bdd-de7c-4910-8665-a8e984a90e23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.678713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce12c2-b670-4edb-bc96-37842756823e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.680058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-075a3bdd-de7c-4910-8665-a8e984a90e23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.680333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce12c2-b670-4edb-bc96-37842756823e service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.689735 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 841/1678] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:46.690558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:46.693819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce12c2-b670-4edb-bc96-37842756823e service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.706855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce12c2-b670-4edb-bc96-37842756823e service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.712070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7c6c8-4e43-47d7-819c-095fd82b7878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.712750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7c6c8-4e43-47d7-819c-095fd82b7878 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.713139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7c6c8-4e43-47d7-819c-095fd82b7878 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.713487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7c6c8-4e43-47d7-819c-095fd82b7878 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.714205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7c6c8-4e43-47d7-819c-095fd82b7878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.716129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7c6c8-4e43-47d7-819c-095fd82b7878 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.728791 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 842/1679] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:46.733753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7c6c8-4e43-47d7-819c-095fd82b7878 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.744882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7c6c8-4e43-47d7-819c-095fd82b7878 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.753968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a70282-8905-423f-ac3a-c89ffae5f5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.754484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a70282-8905-423f-ac3a-c89ffae5f5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.754784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a70282-8905-423f-ac3a-c89ffae5f5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.755068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a70282-8905-423f-ac3a-c89ffae5f5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.755754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a70282-8905-423f-ac3a-c89ffae5f5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.757203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a3bdd-de7c-4910-8665-a8e984a90e23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.757578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a70282-8905-423f-ac3a-c89ffae5f5a1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.758982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a3bdd-de7c-4910-8665-a8e984a90e23 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.769076 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 843/1680] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:48:46.769370 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 838/1681] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:46 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:46.770792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21521a-019d-4cad-910d-2af195bb9e2a None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.771159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21521a-019d-4cad-910d-2af195bb9e2a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.771405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21521a-019d-4cad-910d-2af195bb9e2a None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.772696 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 844/1682] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:48:46.773465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a3bdd-de7c-4910-8665-a8e984a90e23 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.777747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc69a2-1b08-4275-af1f-2c6dc33d364e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.778047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc69a2-1b08-4275-af1f-2c6dc33d364e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.778338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc69a2-1b08-4275-af1f-2c6dc33d364e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.784845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a3bdd-de7c-4910-8665-a8e984a90e23 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.790292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89ae58-8582-4663-b7cd-a348a896051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.791016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89ae58-8582-4663-b7cd-a348a896051c admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.791244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89ae58-8582-4663-b7cd-a348a896051c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.791527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89ae58-8582-4663-b7cd-a348a896051c admin admin] PATH_INFO: `/v3/projects/33dd8b3c5f2346fcbcac1f6f7d61390b/users/a999bf00dd8845db88dcd3eb919e569d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.801319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-53cc69a2-1b08-4275-af1f-2c6dc33d364e None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:46.802599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-53cc69a2-1b08-4275-af1f-2c6dc33d364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.808540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89ae58-8582-4663-b7cd-a348a896051c admin admin] RBAC: Authorizing `identity:create_grant(project_id=33dd8b3c5f2346fcbcac1f6f7d61390b, user_id=a999bf00dd8845db88dcd3eb919e569d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.811764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89ae58-8582-4663-b7cd-a348a896051c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.813778 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 845/1683] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:48:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:48:46.829014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc69a2-1b08-4275-af1f-2c6dc33d364e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.829014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-53cc69a2-1b08-4275-af1f-2c6dc33d364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.839284 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 839/1684] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:46 2022] PUT /identity/v3/projects/33dd8b3c5f2346fcbcac1f6f7d61390b/users/a999bf00dd8845db88dcd3eb919e569d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:48:46.840463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89ae58-8582-4663-b7cd-a348a896051c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:46.908121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc69a2-1b08-4275-af1f-2c6dc33d364e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.917941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89ae58-8582-4663-b7cd-a348a896051c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:46.923510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d6c7e-cb92-4c47-ba3b-b0b0e2e0ce6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.924117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d6c7e-cb92-4c47-ba3b-b0b0e2e0ce6e service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.924418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d6c7e-cb92-4c47-ba3b-b0b0e2e0ce6e service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.924671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d6c7e-cb92-4c47-ba3b-b0b0e2e0ce6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.926423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-912d6c7e-cb92-4c47-ba3b-b0b0e2e0ce6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:46.949775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:46.958758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03196ca-461f-4abe-9233-e1ef622e5939 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:46.959458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03196ca-461f-4abe-9233-e1ef622e5939 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.959829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03196ca-461f-4abe-9233-e1ef622e5939 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.960211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03196ca-461f-4abe-9233-e1ef622e5939 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.970723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03196ca-461f-4abe-9233-e1ef622e5939 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.972168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03196ca-461f-4abe-9233-e1ef622e5939 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:46.982916 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 846/1685] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:48:46.984119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59170a24-27dc-430a-8913-d498fda0351e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:46.984404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59170a24-27dc-430a-8913-d498fda0351e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:46.984727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59170a24-27dc-430a-8913-d498fda0351e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:46.993887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d6c7e-cb92-4c47-ba3b-b0b0e2e0ce6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:46.995490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d6c7e-cb92-4c47-ba3b-b0b0e2e0ce6e service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:47.006155 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 840/1686] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:46 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:47.014684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-59170a24-27dc-430a-8913-d498fda0351e None None] MFA Rules not processed for user `a999bf00dd8845db88dcd3eb919e569d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:47.015961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-59170a24-27dc-430a-8913-d498fda0351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:47.038611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d6c7e-cb92-4c47-ba3b-b0b0e2e0ce6e service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.049980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d6c7e-cb92-4c47-ba3b-b0b0e2e0ce6e service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:47.056168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7475ed4-0002-4926-b1ca-8b9a08c8dc57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:47.056788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7475ed4-0002-4926-b1ca-8b9a08c8dc57 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.057181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7475ed4-0002-4926-b1ca-8b9a08c8dc57 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.057581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7475ed4-0002-4926-b1ca-8b9a08c8dc57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.059288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e7475ed4-0002-4926-b1ca-8b9a08c8dc57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:47.073818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:47.090812 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 847/1687] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:47.095253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59170a24-27dc-430a-8913-d498fda0351e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.128348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7475ed4-0002-4926-b1ca-8b9a08c8dc57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:47.132679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7475ed4-0002-4926-b1ca-8b9a08c8dc57 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:47.143639 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 841/1688] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:47.176591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59170a24-27dc-430a-8913-d498fda0351e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:47.182320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cd4bf-5970-4f9a-8220-6793ee75dd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:47.182955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cd4bf-5970-4f9a-8220-6793ee75dd2f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.183240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cd4bf-5970-4f9a-8220-6793ee75dd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.183614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cd4bf-5970-4f9a-8220-6793ee75dd2f admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.184192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cd4bf-5970-4f9a-8220-6793ee75dd2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:47.186779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cd4bf-5970-4f9a-8220-6793ee75dd2f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:47.192942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b57d3-b078-4519-97c9-789219dbb588 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.193243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b57d3-b078-4519-97c9-789219dbb588 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.193541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b57d3-b078-4519-97c9-789219dbb588 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.195044 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 842/1689] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:48:47.201178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144ba53-76bb-4f36-a1d5-53a1f480ef5d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.201932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144ba53-76bb-4f36-a1d5-53a1f480ef5d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.202317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144ba53-76bb-4f36-a1d5-53a1f480ef5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.218012 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 848/1690] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:47 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:47.221918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cd4bf-5970-4f9a-8220-6793ee75dd2f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.225629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f144ba53-76bb-4f36-a1d5-53a1f480ef5d None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:47.226426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f144ba53-76bb-4f36-a1d5-53a1f480ef5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:47.295294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cd4bf-5970-4f9a-8220-6793ee75dd2f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:47.300220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a4cfe-5d2f-4594-85cf-b07211db6bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:47.300924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a4cfe-5d2f-4594-85cf-b07211db6bcb admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.301213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a4cfe-5d2f-4594-85cf-b07211db6bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.301485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a4cfe-5d2f-4594-85cf-b07211db6bcb admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.302170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a4cfe-5d2f-4594-85cf-b07211db6bcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:47.304155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a4cfe-5d2f-4594-85cf-b07211db6bcb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:47.309258 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 843/1691] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:48:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:48:47.318712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:47.321802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144ba53-76bb-4f36-a1d5-53a1f480ef5d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.323473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f144ba53-76bb-4f36-a1d5-53a1f480ef5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:47.337311 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 849/1692] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:47 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:48:47.338422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a4cfe-5d2f-4594-85cf-b07211db6bcb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.350743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a4cfe-5d2f-4594-85cf-b07211db6bcb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:47.355738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5e470-c699-42c9-9ce8-8cde0d49e183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:47.356438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5e470-c699-42c9-9ce8-8cde0d49e183 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.356787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5e470-c699-42c9-9ce8-8cde0d49e183 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.357153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5e470-c699-42c9-9ce8-8cde0d49e183 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.358511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5e470-c699-42c9-9ce8-8cde0d49e183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:47.360173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5e470-c699-42c9-9ce8-8cde0d49e183 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:47.376533 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 850/1693] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:47.377587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5e470-c699-42c9-9ce8-8cde0d49e183 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.388207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5e470-c699-42c9-9ce8-8cde0d49e183 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:47.389867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144ba53-76bb-4f36-a1d5-53a1f480ef5d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:47.393071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095d4a5-3761-41f9-8535-31182e24a290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:47.393544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e559f71-7a56-40b9-b6b9-7ed88830577a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:47.393868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095d4a5-3761-41f9-8535-31182e24a290 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.394227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095d4a5-3761-41f9-8535-31182e24a290 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.394314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e559f71-7a56-40b9-b6b9-7ed88830577a service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.394632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095d4a5-3761-41f9-8535-31182e24a290 admin admin] PATH_INFO: `/v3/projects/cb64c00a5b19455190f23bc28eebbae4/users/eaac1c1ed2d84496b415a7824b01b469/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.394735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e559f71-7a56-40b9-b6b9-7ed88830577a service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.395044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e559f71-7a56-40b9-b6b9-7ed88830577a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.407706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e559f71-7a56-40b9-b6b9-7ed88830577a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:47.409332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e559f71-7a56-40b9-b6b9-7ed88830577a service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:47.409965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095d4a5-3761-41f9-8535-31182e24a290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb64c00a5b19455190f23bc28eebbae4, user_id=eaac1c1ed2d84496b415a7824b01b469, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:47.412775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095d4a5-3761-41f9-8535-31182e24a290 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:47.421305 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 844/1694] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:48:47.441501 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 851/1695] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:47 2022] PUT /identity/v3/projects/cb64c00a5b19455190f23bc28eebbae4/users/eaac1c1ed2d84496b415a7824b01b469/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:48:47.445630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e559f71-7a56-40b9-b6b9-7ed88830577a service glance] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.456751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e559f71-7a56-40b9-b6b9-7ed88830577a service glance] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:47.513707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:47.527070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aec59-e87b-439b-9cf6-fc989733d504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:47.527795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aec59-e87b-439b-9cf6-fc989733d504 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.528184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aec59-e87b-439b-9cf6-fc989733d504 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.528575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aec59-e87b-439b-9cf6-fc989733d504 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.530028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493aec59-e87b-439b-9cf6-fc989733d504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:47.531761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493aec59-e87b-439b-9cf6-fc989733d504 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:47.543312 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 845/1696] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:47.548668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095d4a5-3761-41f9-8535-31182e24a290 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.559135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095d4a5-3761-41f9-8535-31182e24a290 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:47.564598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77832913-7b07-4a79-92ed-6a7ca26af69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:47.565405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77832913-7b07-4a79-92ed-6a7ca26af69e admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.565843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77832913-7b07-4a79-92ed-6a7ca26af69e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.566240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77832913-7b07-4a79-92ed-6a7ca26af69e admin admin] PATH_INFO: `/v3/projects/cb64c00a5b19455190f23bc28eebbae4/users/eaac1c1ed2d84496b415a7824b01b469/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.568268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77832913-7b07-4a79-92ed-6a7ca26af69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb64c00a5b19455190f23bc28eebbae4, user_id=eaac1c1ed2d84496b415a7824b01b469, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:47.571118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77832913-7b07-4a79-92ed-6a7ca26af69e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:47.589353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aec59-e87b-439b-9cf6-fc989733d504 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.596000 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 852/1697] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:47 2022] PUT /identity/v3/projects/cb64c00a5b19455190f23bc28eebbae4/users/eaac1c1ed2d84496b415a7824b01b469/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:47.618476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb42edd-f629-4599-b73a-124e0e3977c9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.618894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb42edd-f629-4599-b73a-124e0e3977c9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.619348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb42edd-f629-4599-b73a-124e0e3977c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.671716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0cb42edd-f629-4599-b73a-124e0e3977c9 None None] MFA Rules not processed for user `eaac1c1ed2d84496b415a7824b01b469`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:47.674260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0cb42edd-f629-4599-b73a-124e0e3977c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:47.692018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aec59-e87b-439b-9cf6-fc989733d504 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:47.761197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e007e-c608-435a-9a0d-228e5abff1de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:47.761819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e007e-c608-435a-9a0d-228e5abff1de service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.762171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e007e-c608-435a-9a0d-228e5abff1de service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.762518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e007e-c608-435a-9a0d-228e5abff1de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.764570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ee3e007e-c608-435a-9a0d-228e5abff1de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:47.765072 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 853/1698] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:47.765746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb42edd-f629-4599-b73a-124e0e3977c9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.767026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0cb42edd-f629-4599-b73a-124e0e3977c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:47.856762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e007e-c608-435a-9a0d-228e5abff1de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:47.859657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e007e-c608-435a-9a0d-228e5abff1de service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:47.860361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb42edd-f629-4599-b73a-124e0e3977c9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:47.867713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68519a96-dc80-4e2d-bb14-b1e791ec033f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:47.868502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68519a96-dc80-4e2d-bb14-b1e791ec033f service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.868798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68519a96-dc80-4e2d-bb14-b1e791ec033f service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.869126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68519a96-dc80-4e2d-bb14-b1e791ec033f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.871029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-68519a96-dc80-4e2d-bb14-b1e791ec033f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:47.874985 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 846/1699] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:47.907833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68519a96-dc80-4e2d-bb14-b1e791ec033f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:47.910191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68519a96-dc80-4e2d-bb14-b1e791ec033f service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:47.910672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e007e-c608-435a-9a0d-228e5abff1de service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.922040 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 854/1700] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:47.925504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e007e-c608-435a-9a0d-228e5abff1de service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:47.931827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c1969-0630-400b-b512-ad00b94efd10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:47.932541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c1969-0630-400b-b512-ad00b94efd10 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:47.933511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c1969-0630-400b-b512-ad00b94efd10 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:47.934062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c1969-0630-400b-b512-ad00b94efd10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:47.945360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68519a96-dc80-4e2d-bb14-b1e791ec033f service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.945360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-68519a96-dc80-4e2d-bb14-b1e791ec033f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:47.962241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c1969-0630-400b-b512-ad00b94efd10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:47.964468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c1969-0630-400b-b512-ad00b94efd10 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:47.977896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:47.980690 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 847/1701] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:48:47.981614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c1969-0630-400b-b512-ad00b94efd10 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:47.994823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c1969-0630-400b-b512-ad00b94efd10 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:48.000624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705dc3e9-5d64-4082-b61f-57e4b0020be9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:48.001328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705dc3e9-5d64-4082-b61f-57e4b0020be9 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:48.001591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705dc3e9-5d64-4082-b61f-57e4b0020be9 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:48.001803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705dc3e9-5d64-4082-b61f-57e4b0020be9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:48.007107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:48.078463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68519a96-dc80-4e2d-bb14-b1e791ec033f service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:48.078941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705dc3e9-5d64-4082-b61f-57e4b0020be9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:48.081233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705dc3e9-5d64-4082-b61f-57e4b0020be9 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:48.083541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20deb7bc-9f67-41fd-b6d6-b1ed1b289663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:48.084192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deb7bc-9f67-41fd-b6d6-b1ed1b289663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:48.084484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deb7bc-9f67-41fd-b6d6-b1ed1b289663 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:48.084765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deb7bc-9f67-41fd-b6d6-b1ed1b289663 admin admin] PATH_INFO: `/v3/users/9c3be59ecad04039bf1777f2874efedb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:48.085949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20deb7bc-9f67-41fd-b6d6-b1ed1b289663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c3be59ecad04039bf1777f2874efedb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:48.087992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20deb7bc-9f67-41fd-b6d6-b1ed1b289663 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:48.094307 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 848/1702] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:48.125572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705dc3e9-5d64-4082-b61f-57e4b0020be9 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:48.155271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705dc3e9-5d64-4082-b61f-57e4b0020be9 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:48.182202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-20deb7bc-9f67-41fd-b6d6-b1ed1b289663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c3be59ecad04039bf1777f2874efedb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.198548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128de0a-6a27-4a9f-bb79-2b663d6a4439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:48.198548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128de0a-6a27-4a9f-bb79-2b663d6a4439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:48.198548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128de0a-6a27-4a9f-bb79-2b663d6a4439 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:48.198548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128de0a-6a27-4a9f-bb79-2b663d6a4439 admin admin] PATH_INFO: `/v3/users/a999bf00dd8845db88dcd3eb919e569d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:48.198939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128de0a-6a27-4a9f-bb79-2b663d6a4439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a999bf00dd8845db88dcd3eb919e569d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:48.198939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128de0a-6a27-4a9f-bb79-2b663d6a4439 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:48.226146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-20deb7bc-9f67-41fd-b6d6-b1ed1b289663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c3be59ecad04039bf1777f2874efedb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.244208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-20deb7bc-9f67-41fd-b6d6-b1ed1b289663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c3be59ecad04039bf1777f2874efedb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.244946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-20deb7bc-9f67-41fd-b6d6-b1ed1b289663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c3be59ecad04039bf1777f2874efedb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.246220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-20deb7bc-9f67-41fd-b6d6-b1ed1b289663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c3be59ecad04039bf1777f2874efedb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.283596 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 855/1703] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:48:47 2022] DELETE /identity/v3/users/9c3be59ecad04039bf1777f2874efedb => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:48.283698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b128de0a-6a27-4a9f-bb79-2b663d6a4439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a999bf00dd8845db88dcd3eb919e569d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.285009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a0b10-e086-4281-bba5-cb94abddfd25 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:48.285766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a0b10-e086-4281-bba5-cb94abddfd25 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:48.286058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a0b10-e086-4281-bba5-cb94abddfd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:48.304161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b128de0a-6a27-4a9f-bb79-2b663d6a4439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a999bf00dd8845db88dcd3eb919e569d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.307102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b128de0a-6a27-4a9f-bb79-2b663d6a4439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a999bf00dd8845db88dcd3eb919e569d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.345394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b67a0b10-e086-4281-bba5-cb94abddfd25 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:48.345921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b128de0a-6a27-4a9f-bb79-2b663d6a4439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a999bf00dd8845db88dcd3eb919e569d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.346284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b67a0b10-e086-4281-bba5-cb94abddfd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:48.359978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b128de0a-6a27-4a9f-bb79-2b663d6a4439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a999bf00dd8845db88dcd3eb919e569d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.371717 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 849/1704] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:48:48 2022] DELETE /identity/v3/users/a999bf00dd8845db88dcd3eb919e569d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:48.377154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665b1fa-7026-4bc9-9141-50552cbb71e4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:48.377639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665b1fa-7026-4bc9-9141-50552cbb71e4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:48.378068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665b1fa-7026-4bc9-9141-50552cbb71e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:48.398949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1665b1fa-7026-4bc9-9141-50552cbb71e4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:48.400461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1665b1fa-7026-4bc9-9141-50552cbb71e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:48.432288 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 856/1705] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:48:48.443305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a0b10-e086-4281-bba5-cb94abddfd25 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:48.445738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b67a0b10-e086-4281-bba5-cb94abddfd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:48.545092 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 850/1706] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:48.545274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:48.552816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665b1fa-7026-4bc9-9141-50552cbb71e4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:48.554891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1665b1fa-7026-4bc9-9141-50552cbb71e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:48.583637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a0b10-e086-4281-bba5-cb94abddfd25 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:48.595659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:48.646692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665b1fa-7026-4bc9-9141-50552cbb71e4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:48.651262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b018f-3f9e-4ba1-8215-99d1d9816ba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:48.651942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b018f-3f9e-4ba1-8215-99d1d9816ba0 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:48.654870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b018f-3f9e-4ba1-8215-99d1d9816ba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:48.655144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b018f-3f9e-4ba1-8215-99d1d9816ba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:48.662619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b62a7d-ddf0-4e3d-aca5-8c6a24a1afdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:48.663252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b62a7d-ddf0-4e3d-aca5-8c6a24a1afdd service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:48.663540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b62a7d-ddf0-4e3d-aca5-8c6a24a1afdd service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:48.663819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b62a7d-ddf0-4e3d-aca5-8c6a24a1afdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:48.680735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b62a7d-ddf0-4e3d-aca5-8c6a24a1afdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:48.682688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b62a7d-ddf0-4e3d-aca5-8c6a24a1afdd service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:48.696533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b018f-3f9e-4ba1-8215-99d1d9816ba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:48.697098 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 857/1707] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:48.698021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b62a7d-ddf0-4e3d-aca5-8c6a24a1afdd service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:48.699917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b5b62a7d-ddf0-4e3d-aca5-8c6a24a1afdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:48.710040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b018f-3f9e-4ba1-8215-99d1d9816ba0 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:48.721356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b62a7d-ddf0-4e3d-aca5-8c6a24a1afdd service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:48.730015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46f96b-f11a-4e3d-957c-45864a00b43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:48.730608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46f96b-f11a-4e3d-957c-45864a00b43e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:48.731044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46f96b-f11a-4e3d-957c-45864a00b43e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:48.731277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46f96b-f11a-4e3d-957c-45864a00b43e admin admin] PATH_INFO: `/v3/users/5959424dab254b8e82add6ae6490b936` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:48.735913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46f96b-f11a-4e3d-957c-45864a00b43e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5959424dab254b8e82add6ae6490b936)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:48.738930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46f96b-f11a-4e3d-957c-45864a00b43e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:48.743362 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 851/1708] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:48:48.823868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3e46f96b-f11a-4e3d-957c-45864a00b43e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5959424dab254b8e82add6ae6490b936'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.834387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b018f-3f9e-4ba1-8215-99d1d9816ba0 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:48.839575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-360b018f-3f9e-4ba1-8215-99d1d9816ba0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:48.865859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3e46f96b-f11a-4e3d-957c-45864a00b43e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5959424dab254b8e82add6ae6490b936'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.893333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3e46f96b-f11a-4e3d-957c-45864a00b43e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5959424dab254b8e82add6ae6490b936'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.893996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3e46f96b-f11a-4e3d-957c-45864a00b43e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5959424dab254b8e82add6ae6490b936'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.894641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3e46f96b-f11a-4e3d-957c-45864a00b43e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5959424dab254b8e82add6ae6490b936'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:48.920926 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 858/1709] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:48:48 2022] DELETE /identity/v3/users/5959424dab254b8e82add6ae6490b936 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:48.926109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7f40c-6c4b-4aa4-96cb-4b86bf16a3e8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:48.926409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7f40c-6c4b-4aa4-96cb-4b86bf16a3e8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:48.926844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7f40c-6c4b-4aa4-96cb-4b86bf16a3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:48.960925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-58e7f40c-6c4b-4aa4-96cb-4b86bf16a3e8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:48.960925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-58e7f40c-6c4b-4aa4-96cb-4b86bf16a3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:48.999396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b018f-3f9e-4ba1-8215-99d1d9816ba0 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:49.068373 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 859/1710] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:49.080240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7f40c-6c4b-4aa4-96cb-4b86bf16a3e8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:49.081987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-58e7f40c-6c4b-4aa4-96cb-4b86bf16a3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:49.101926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:49.191765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817ef0f-cd3c-42be-92ce-35713bacdc94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:49.192102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7f40c-6c4b-4aa4-96cb-4b86bf16a3e8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:49.192511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817ef0f-cd3c-42be-92ce-35713bacdc94 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:49.193082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817ef0f-cd3c-42be-92ce-35713bacdc94 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:49.193438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817ef0f-cd3c-42be-92ce-35713bacdc94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:49.198501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79061fc-7374-40c8-a9e8-e53d97b6a5f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:49.199305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79061fc-7374-40c8-a9e8-e53d97b6a5f2 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:49.199920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79061fc-7374-40c8-a9e8-e53d97b6a5f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:49.200350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79061fc-7374-40c8-a9e8-e53d97b6a5f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:49.202452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d817ef0f-cd3c-42be-92ce-35713bacdc94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:49.214804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79061fc-7374-40c8-a9e8-e53d97b6a5f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:49.219263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79061fc-7374-40c8-a9e8-e53d97b6a5f2 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:49.257418 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 860/1711] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:49.261002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88475b-b1e2-4aeb-9555-fad4c30fd754 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:49.261165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88475b-b1e2-4aeb-9555-fad4c30fd754 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:49.261456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88475b-b1e2-4aeb-9555-fad4c30fd754 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:49.265226 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 861/1712] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:48:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:48:49.271491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2347e-0a84-4d85-b1e2-d9189d4695d4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:49.272524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2347e-0a84-4d85-b1e2-d9189d4695d4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:49.273078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2347e-0a84-4d85-b1e2-d9189d4695d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:49.343558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:49.346887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6ec2347e-0a84-4d85-b1e2-d9189d4695d4 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:49.349080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817ef0f-cd3c-42be-92ce-35713bacdc94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:49.349336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6ec2347e-0a84-4d85-b1e2-d9189d4695d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:49.350730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817ef0f-cd3c-42be-92ce-35713bacdc94 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:49.378205 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 852/1713] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:48 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:48:49.424871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817ef0f-cd3c-42be-92ce-35713bacdc94 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:49.426137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d817ef0f-cd3c-42be-92ce-35713bacdc94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:49.450451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817ef0f-cd3c-42be-92ce-35713bacdc94 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:49.457073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:49.458027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:49.458443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:49.458878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] PATH_INFO: `/v3/projects/33dd8b3c5f2346fcbcac1f6f7d61390b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:49.460556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33dd8b3c5f2346fcbcac1f6f7d61390b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:49.463132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:49.559465 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 862/1714] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:48:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:49.562898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2347e-0a84-4d85-b1e2-d9189d4695d4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:49.567141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6ec2347e-0a84-4d85-b1e2-d9189d4695d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:49.591464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2347e-0a84-4d85-b1e2-d9189d4695d4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:49.599775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:49.600525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:49.601003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:49.601349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] PATH_INFO: `/v3/projects/d3c752a733314acd887b75faf52c9a33` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:49.602912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3c752a733314acd887b75faf52c9a33)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:49.606396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:49.617137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33dd8b3c5f2346fcbcac1f6f7d61390b', 'request_id': 'req-0f5b970c-2edc-48ad-a1e1-9f91254df155'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:49.643182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33dd8b3c5f2346fcbcac1f6f7d61390b', 'request_id': 'req-0f5b970c-2edc-48ad-a1e1-9f91254df155'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:49.655790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33dd8b3c5f2346fcbcac1f6f7d61390b', 'request_id': 'req-0f5b970c-2edc-48ad-a1e1-9f91254df155'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:49.677196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] The token cache is being invalidate because project 33dd8b3c5f2346fcbcac1f6f7d61390b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:48:49.677973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:49.684771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:49.692861 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 853/1715] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:48:49 2022] DELETE /identity/v3/projects/33dd8b3c5f2346fcbcac1f6f7d61390b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:49.695715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:49.700536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:49.703494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3c752a733314acd887b75faf52c9a33', 'request_id': 'req-faa86a27-a2b1-4461-a2ad-d4c44842ea72'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:49.730560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3c752a733314acd887b75faf52c9a33', 'request_id': 'req-faa86a27-a2b1-4461-a2ad-d4c44842ea72'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:49.743828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3c752a733314acd887b75faf52c9a33', 'request_id': 'req-faa86a27-a2b1-4461-a2ad-d4c44842ea72'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:49.765545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] The token cache is being invalidate because project d3c752a733314acd887b75faf52c9a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:48:49.766635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:49.767551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:49.769345 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 863/1716] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:48:49 2022] DELETE /identity/v3/projects/d3c752a733314acd887b75faf52c9a33 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:49.770867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:49.773277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:49.857341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:49.885848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b970c-2edc-48ad-a1e1-9f91254df155 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:49.959441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86a27-a2b1-4461-a2ad-d4c44842ea72 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:49.972140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:49.972920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:49.973215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:49.973499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] PATH_INFO: `/v3/users/eaac1c1ed2d84496b415a7824b01b469` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:49.976081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaac1c1ed2d84496b415a7824b01b469)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:49.978525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:50.000874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceb292-ea07-4615-b752-9b07c87242db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:50.001541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceb292-ea07-4615-b752-9b07c87242db service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:50.001822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceb292-ea07-4615-b752-9b07c87242db service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:50.002097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceb292-ea07-4615-b752-9b07c87242db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:50.003883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-48ceb292-ea07-4615-b752-9b07c87242db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:50.021461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ceb292-ea07-4615-b752-9b07c87242db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:50.023801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ceb292-ea07-4615-b752-9b07c87242db service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:50.036662 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 854/1717] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:50.037662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceb292-ea07-4615-b752-9b07c87242db service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:50.038902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaac1c1ed2d84496b415a7824b01b469'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.039037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-48ceb292-ea07-4615-b752-9b07c87242db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:50.063281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceb292-ea07-4615-b752-9b07c87242db service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:50.066332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaac1c1ed2d84496b415a7824b01b469'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.075633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85061f-436a-4499-a5c3-a6bbe11b8b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:50.077051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85061f-436a-4499-a5c3-a6bbe11b8b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:50.077355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85061f-436a-4499-a5c3-a6bbe11b8b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:50.077929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85061f-436a-4499-a5c3-a6bbe11b8b6a admin admin] PATH_INFO: `/v3/users/a430e034949d4b619b1620b1021bcf45` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:50.078882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaac1c1ed2d84496b415a7824b01b469'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.079702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaac1c1ed2d84496b415a7824b01b469'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.079840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85061f-436a-4499-a5c3-a6bbe11b8b6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a430e034949d4b619b1620b1021bcf45)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:50.080360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaac1c1ed2d84496b415a7824b01b469'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.082388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85061f-436a-4499-a5c3-a6bbe11b8b6a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:50.094914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:50.107143 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 864/1718] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:48:49 2022] DELETE /identity/v3/users/eaac1c1ed2d84496b415a7824b01b469 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:50.107143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:50.119338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:50.193160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9e85061f-436a-4499-a5c3-a6bbe11b8b6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a430e034949d4b619b1620b1021bcf45'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.219773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9e85061f-436a-4499-a5c3-a6bbe11b8b6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a430e034949d4b619b1620b1021bcf45'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.221291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9e85061f-436a-4499-a5c3-a6bbe11b8b6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a430e034949d4b619b1620b1021bcf45'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.227327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226260e-4d83-4825-a3af-04a7144b7337 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:50.233856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:50.234520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:50.234906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:50.235306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] PATH_INFO: `/v3/projects/18ee41edde4146b2a8cf7dbde2b5f5d2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:50.236719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18ee41edde4146b2a8cf7dbde2b5f5d2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:50.239195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:50.241576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9e85061f-436a-4499-a5c3-a6bbe11b8b6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a430e034949d4b619b1620b1021bcf45'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.257176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9e85061f-436a-4499-a5c3-a6bbe11b8b6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a430e034949d4b619b1620b1021bcf45'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.274146 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 855/1719] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:48:50 2022] DELETE /identity/v3/users/a430e034949d4b619b1620b1021bcf45 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:50.275439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9a1b6-6c88-4b1a-9530-2a88078dd324 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:50.275789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9a1b6-6c88-4b1a-9530-2a88078dd324 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:50.276125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9a1b6-6c88-4b1a-9530-2a88078dd324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:50.328110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f3f9a1b6-6c88-4b1a-9530-2a88078dd324 None None] MFA Rules not processed for user `403b2dcf1b3f4e1fbead88207efa9ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:50.331743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f3f9a1b6-6c88-4b1a-9530-2a88078dd324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:50.388160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18ee41edde4146b2a8cf7dbde2b5f5d2', 'request_id': 'req-c3289767-7dcb-4bbd-b0db-734a6eced696'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.393782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:50.408562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18ee41edde4146b2a8cf7dbde2b5f5d2', 'request_id': 'req-c3289767-7dcb-4bbd-b0db-734a6eced696'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.428178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18ee41edde4146b2a8cf7dbde2b5f5d2', 'request_id': 'req-c3289767-7dcb-4bbd-b0db-734a6eced696'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.459876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] The token cache is being invalidate because project 18ee41edde4146b2a8cf7dbde2b5f5d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:48:50.460329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.462103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.469159 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 865/1720] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:48:50 2022] DELETE /identity/v3/projects/18ee41edde4146b2a8cf7dbde2b5f5d2 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:50.473154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:50.474626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:50.525079 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 856/1721] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:50.540769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9a1b6-6c88-4b1a-9530-2a88078dd324 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:50.542316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f3f9a1b6-6c88-4b1a-9530-2a88078dd324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:50.630669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3289767-7dcb-4bbd-b0db-734a6eced696 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:50.636746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:50.637448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:50.637738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:50.638017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] PATH_INFO: `/v3/users/8cb69666ecf14b9bb5a96603c8163182` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:50.639281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb69666ecf14b9bb5a96603c8163182)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:50.647741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:50.688933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9a1b6-6c88-4b1a-9530-2a88078dd324 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:50.739967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:50.752084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb69666ecf14b9bb5a96603c8163182'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.778354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb69666ecf14b9bb5a96603c8163182'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.791222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb69666ecf14b9bb5a96603c8163182'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.792389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb69666ecf14b9bb5a96603c8163182'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.793409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb69666ecf14b9bb5a96603c8163182'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:50.800488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8133c6d-9c40-463d-a361-266627d72c37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:50.801508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8133c6d-9c40-463d-a361-266627d72c37 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:50.801927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8133c6d-9c40-463d-a361-266627d72c37 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:50.802372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8133c6d-9c40-463d-a361-266627d72c37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:50.804570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a8133c6d-9c40-463d-a361-266627d72c37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:50.827098 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 866/1722] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:48:50 2022] DELETE /identity/v3/users/8cb69666ecf14b9bb5a96603c8163182 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:50.828259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:50.832680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:50.892123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8133c6d-9c40-463d-a361-266627d72c37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:50.895340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8133c6d-9c40-463d-a361-266627d72c37 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:50.908280 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 857/1723] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:48:50.924059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8133c6d-9c40-463d-a361-266627d72c37 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:50.926352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a8133c6d-9c40-463d-a361-266627d72c37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:50.936359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da6147-9ea7-4d78-9e91-1e8df296736a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:50.942158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:50.943424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:50.943577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8133c6d-9c40-463d-a361-266627d72c37 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:50.943899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:50.944285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99 admin admin] PATH_INFO: `/v3/projects/cb64c00a5b19455190f23bc28eebbae4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:50.947508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb64c00a5b19455190f23bc28eebbae4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:50.948423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:50.949136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b078af-b80f-476b-a254-4bc06142b440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:50.950081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b078af-b80f-476b-a254-4bc06142b440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:50.950859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b078af-b80f-476b-a254-4bc06142b440 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:50.951661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:50.951827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b078af-b80f-476b-a254-4bc06142b440 admin admin] PATH_INFO: `/v3/projects/04232849b122407b8256919ca8724783` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:50.953481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b078af-b80f-476b-a254-4bc06142b440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04232849b122407b8256919ca8724783)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:50.956340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b078af-b80f-476b-a254-4bc06142b440 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:51.055240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb64c00a5b19455190f23bc28eebbae4', 'request_id': 'req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.075153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb64c00a5b19455190f23bc28eebbae4', 'request_id': 'req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.084839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-25b078af-b80f-476b-a254-4bc06142b440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04232849b122407b8256919ca8724783', 'request_id': 'req-25b078af-b80f-476b-a254-4bc06142b440'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.096745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-25b078af-b80f-476b-a254-4bc06142b440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04232849b122407b8256919ca8724783', 'request_id': 'req-25b078af-b80f-476b-a254-4bc06142b440'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.097816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb64c00a5b19455190f23bc28eebbae4', 'request_id': 'req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.108757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-25b078af-b80f-476b-a254-4bc06142b440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04232849b122407b8256919ca8724783', 'request_id': 'req-25b078af-b80f-476b-a254-4bc06142b440'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.125521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-25b078af-b80f-476b-a254-4bc06142b440 admin admin] The token cache is being invalidate because project 04232849b122407b8256919ca8724783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:48:51.126051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-25b078af-b80f-476b-a254-4bc06142b440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.126826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-25b078af-b80f-476b-a254-4bc06142b440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.128763 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 858/1724] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:48:50 2022] DELETE /identity/v3/projects/04232849b122407b8256919ca8724783 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:51.134898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955006c-943b-4e48-950a-7d9585960e78 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.135473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955006c-943b-4e48-950a-7d9585960e78 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.135864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955006c-943b-4e48-950a-7d9585960e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.136771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99 admin admin] The token cache is being invalidate because project cb64c00a5b19455190f23bc28eebbae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:48:51.137121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.138292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d148a1b7-3b2f-4be2-8e0e-29a8c2c2bd99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.145244 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 867/1725] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:48:50 2022] DELETE /identity/v3/projects/cb64c00a5b19455190f23bc28eebbae4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:51.148626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c485f-2dd6-4517-b945-6c51f41d6787 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.149039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c485f-2dd6-4517-b945-6c51f41d6787 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.149398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c485f-2dd6-4517-b945-6c51f41d6787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.171644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1955006c-943b-4e48-950a-7d9585960e78 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:51.172029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1955006c-943b-4e48-950a-7d9585960e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:51.178873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-961c485f-2dd6-4517-b945-6c51f41d6787 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:51.180472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-961c485f-2dd6-4517-b945-6c51f41d6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:51.264768 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 868/1726] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:51.271846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5d914-557a-4d9d-a520-9317f0c7171b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.272446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5d914-557a-4d9d-a520-9317f0c7171b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.272992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5d914-557a-4d9d-a520-9317f0c7171b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.295662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:51.312742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7dc5d914-557a-4d9d-a520-9317f0c7171b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:51.314539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7dc5d914-557a-4d9d-a520-9317f0c7171b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:51.326942 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 869/1727] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:48:51.329897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5d914-557a-4d9d-a520-9317f0c7171b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:51.332164 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 859/1728] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:48:51.341217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324476b-0486-4046-bc10-c240b79b9f9c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.342362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324476b-0486-4046-bc10-c240b79b9f9c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.342900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324476b-0486-4046-bc10-c240b79b9f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.346266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5d914-557a-4d9d-a520-9317f0c7171b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:51.352363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba743d-d858-43bd-96ab-bf109faf51f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:51.353058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba743d-d858-43bd-96ab-bf109faf51f9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.353349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba743d-d858-43bd-96ab-bf109faf51f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.353630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba743d-d858-43bd-96ab-bf109faf51f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.354352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba743d-d858-43bd-96ab-bf109faf51f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:48:51.354668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba743d-d858-43bd-96ab-bf109faf51f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:51.356422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba743d-d858-43bd-96ab-bf109faf51f9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:51.373559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a324476b-0486-4046-bc10-c240b79b9f9c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:51.375251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a324476b-0486-4046-bc10-c240b79b9f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:51.386018 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 860/1729] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:51.393774 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 870/1730] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:48:51.394525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324476b-0486-4046-bc10-c240b79b9f9c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:51.403689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba743d-d858-43bd-96ab-bf109faf51f9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:51.406808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324476b-0486-4046-bc10-c240b79b9f9c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:51.419701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3428c4-3794-4488-8734-b2a3114a7c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:51.420347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3428c4-3794-4488-8734-b2a3114a7c55 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.420634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3428c4-3794-4488-8734-b2a3114a7c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.422141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3428c4-3794-4488-8734-b2a3114a7c55 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.422820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3428c4-3794-4488-8734-b2a3114a7c55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:48:51.423135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3428c4-3794-4488-8734-b2a3114a7c55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:51.424760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3428c4-3794-4488-8734-b2a3114a7c55 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:51.429181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba743d-d858-43bd-96ab-bf109faf51f9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:51.450313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc7563-58c8-4589-8659-2a70598db8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:51.450313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc7563-58c8-4589-8659-2a70598db8a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.450313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc7563-58c8-4589-8659-2a70598db8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.450313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc7563-58c8-4589-8659-2a70598db8a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.450735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc7563-58c8-4589-8659-2a70598db8a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:51.452773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc7563-58c8-4589-8659-2a70598db8a9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:51.464512 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 861/1731] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:48:51.466253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3428c4-3794-4488-8734-b2a3114a7c55 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:51.481773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3428c4-3794-4488-8734-b2a3114a7c55 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:51.488239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f377c18-e79a-40c8-a6f2-3fcd98fbbd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:51.488863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f377c18-e79a-40c8-a6f2-3fcd98fbbd92 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.489194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f377c18-e79a-40c8-a6f2-3fcd98fbbd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.489459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f377c18-e79a-40c8-a6f2-3fcd98fbbd92 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.490106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f377c18-e79a-40c8-a6f2-3fcd98fbbd92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:51.493788 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 871/1732] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:51 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:51.493925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f377c18-e79a-40c8-a6f2-3fcd98fbbd92 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:51.505637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc7563-58c8-4589-8659-2a70598db8a9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:51.514517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2cdc7563-58c8-4589-8659-2a70598db8a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:51.527280 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 862/1733] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:51 2022] POST /identity/v3/projects => generated 403 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:51.531792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f377c18-e79a-40c8-a6f2-3fcd98fbbd92 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:51.588527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:51.600958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f377c18-e79a-40c8-a6f2-3fcd98fbbd92 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:51.605912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72f6f9-15a5-4540-891f-b0df1679dd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:51.606536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72f6f9-15a5-4540-891f-b0df1679dd16 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.606850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72f6f9-15a5-4540-891f-b0df1679dd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.607134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72f6f9-15a5-4540-891f-b0df1679dd16 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.607902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72f6f9-15a5-4540-891f-b0df1679dd16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:51.609896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72f6f9-15a5-4540-891f-b0df1679dd16 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:51.611991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc7563-58c8-4589-8659-2a70598db8a9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:51.618267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:51.619566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.620730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.621151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] PATH_INFO: `/v3/projects/b9ea36ea140a41f1b3e471eb4cf980ef` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.624196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9ea36ea140a41f1b3e471eb4cf980ef)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:51.626817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:51.646163 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 863/1734] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:51 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:51.647394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72f6f9-15a5-4540-891f-b0df1679dd16 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:51.659778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72f6f9-15a5-4540-891f-b0df1679dd16 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:51.665813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78841e52-ba67-428f-a515-b231577f38e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:51.666922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78841e52-ba67-428f-a515-b231577f38e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.667205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78841e52-ba67-428f-a515-b231577f38e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.667484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78841e52-ba67-428f-a515-b231577f38e1 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.668282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78841e52-ba67-428f-a515-b231577f38e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:51.671161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78841e52-ba67-428f-a515-b231577f38e1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:51.751174 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 864/1735] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:51 2022] POST /identity/v3/users => generated 302 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:51.755585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78841e52-ba67-428f-a515-b231577f38e1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:51.762374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9ea36ea140a41f1b3e471eb4cf980ef', 'request_id': 'req-6817f588-e832-4bef-8dc7-b50965922a69'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.769469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78841e52-ba67-428f-a515-b231577f38e1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:51.780517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9ea36ea140a41f1b3e471eb4cf980ef', 'request_id': 'req-6817f588-e832-4bef-8dc7-b50965922a69'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.789505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9ea36ea140a41f1b3e471eb4cf980ef', 'request_id': 'req-6817f588-e832-4bef-8dc7-b50965922a69'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.813616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] The token cache is being invalidate because project b9ea36ea140a41f1b3e471eb4cf980ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:48:51.813936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.816746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:51.825470 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 872/1736] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:48:51 2022] DELETE /identity/v3/projects/b9ea36ea140a41f1b3e471eb4cf980ef => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:51.826781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:51.833205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:51.917179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382242a-56b1-424e-9b63-0889b5360863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:51.918306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382242a-56b1-424e-9b63-0889b5360863 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.918808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382242a-56b1-424e-9b63-0889b5360863 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.919303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382242a-56b1-424e-9b63-0889b5360863 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.920674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382242a-56b1-424e-9b63-0889b5360863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:51.922660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382242a-56b1-424e-9b63-0889b5360863 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:51.936623 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 865/1737] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:51.938353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936fdaa-3968-435f-a622-319fe3a348f5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.938807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936fdaa-3968-435f-a622-319fe3a348f5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.939211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936fdaa-3968-435f-a622-319fe3a348f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.945485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817f588-e832-4bef-8dc7-b50965922a69 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:51.955951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee0662-9914-4321-bcf3-3b57f5003eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:51.956639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee0662-9914-4321-bcf3-3b57f5003eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:51.956956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee0662-9914-4321-bcf3-3b57f5003eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:51.957239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee0662-9914-4321-bcf3-3b57f5003eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:51.957907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee0662-9914-4321-bcf3-3b57f5003eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:51.959659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee0662-9914-4321-bcf3-3b57f5003eb1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:51.964962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c936fdaa-3968-435f-a622-319fe3a348f5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:51.966738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c936fdaa-3968-435f-a622-319fe3a348f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:51.972698 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 873/1738] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:51.973957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee0662-9914-4321-bcf3-3b57f5003eb1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:51.976213 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 866/1739] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:51.977225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936fdaa-3968-435f-a622-319fe3a348f5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:51.992056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2fee0662-9914-4321-bcf3-3b57f5003eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:52.016455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936fdaa-3968-435f-a622-319fe3a348f5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:52.017056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee0662-9914-4321-bcf3-3b57f5003eb1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:52.041276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9989b-5913-4aa7-bc0d-15e44a20e7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:52.041842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9989b-5913-4aa7-bc0d-15e44a20e7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:52.042128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9989b-5913-4aa7-bc0d-15e44a20e7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:52.042413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9989b-5913-4aa7-bc0d-15e44a20e7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:52.043995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f110e2-86b2-4773-bd62-bb9174c695aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:52.050513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f110e2-86b2-4773-bd62-bb9174c695aa admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:52.050973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f110e2-86b2-4773-bd62-bb9174c695aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:52.051266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f110e2-86b2-4773-bd62-bb9174c695aa admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:52.051996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f110e2-86b2-4773-bd62-bb9174c695aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:52.055006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9989b-5913-4aa7-bc0d-15e44a20e7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:52.056989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9989b-5913-4aa7-bc0d-15e44a20e7f2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:52.064512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f110e2-86b2-4773-bd62-bb9174c695aa admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:52.069329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:52.100609 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 867/1740] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:52.106104 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 874/1741] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:52.106269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b1803-d3ff-4f69-b9d4-7562913c457b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:52.106269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b1803-d3ff-4f69-b9d4-7562913c457b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:52.106269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b1803-d3ff-4f69-b9d4-7562913c457b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:52.110188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aaaadc-a729-4830-b3bd-278a7eb168ba None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:52.110480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aaaadc-a729-4830-b3bd-278a7eb168ba None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:52.110758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aaaadc-a729-4830-b3bd-278a7eb168ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:52.145779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-130b1803-d3ff-4f69-b9d4-7562913c457b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:52.149716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-130b1803-d3ff-4f69-b9d4-7562913c457b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:52.173729 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 868/1742] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:52.175923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b1803-d3ff-4f69-b9d4-7562913c457b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:52.198355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b1803-d3ff-4f69-b9d4-7562913c457b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:52.219609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32493b99-5d86-4232-9aba-f2faf6bfdca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:52.221705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32493b99-5d86-4232-9aba-f2faf6bfdca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:52.222800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32493b99-5d86-4232-9aba-f2faf6bfdca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:52.222800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32493b99-5d86-4232-9aba-f2faf6bfdca6 admin admin] PATH_INFO: `/v3/projects/913c519428fd49a7a6f098b4112afc1a/users/45b55868ea8a492182510bfd0133d74f/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:52.225608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b7aaaadc-a729-4830-b3bd-278a7eb168ba None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:52.226873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b7aaaadc-a729-4830-b3bd-278a7eb168ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:52.257448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32493b99-5d86-4232-9aba-f2faf6bfdca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913c519428fd49a7a6f098b4112afc1a, user_id=45b55868ea8a492182510bfd0133d74f, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:52.264459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32493b99-5d86-4232-9aba-f2faf6bfdca6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:52.290798 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 869/1743] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:52 2022] PUT /identity/v3/projects/913c519428fd49a7a6f098b4112afc1a/users/45b55868ea8a492182510bfd0133d74f/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:52.291841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32493b99-5d86-4232-9aba-f2faf6bfdca6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:52.323345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:52.385711 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 875/1744] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:52.387001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aaaadc-a729-4830-b3bd-278a7eb168ba None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:52.434422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aaaadc-a729-4830-b3bd-278a7eb168ba None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:52.490872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32493b99-5d86-4232-9aba-f2faf6bfdca6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:52.498046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9b78b-3ff0-4c4a-a4a3-45a79fd21491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:52.499354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9b78b-3ff0-4c4a-a4a3-45a79fd21491 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:52.500762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9b78b-3ff0-4c4a-a4a3-45a79fd21491 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:52.501255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9b78b-3ff0-4c4a-a4a3-45a79fd21491 admin admin] PATH_INFO: `/v3/projects/e005609cdb7945ebbf75b805621a2592/users/64b7035c42e04fef99215dbd84b841c3/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:52.520972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9b78b-3ff0-4c4a-a4a3-45a79fd21491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e005609cdb7945ebbf75b805621a2592, user_id=64b7035c42e04fef99215dbd84b841c3, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:52.525379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9b78b-3ff0-4c4a-a4a3-45a79fd21491 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:52.569212 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 870/1745] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:52 2022] PUT /identity/v3/projects/e005609cdb7945ebbf75b805621a2592/users/64b7035c42e04fef99215dbd84b841c3/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:52.571494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67eadc-dd5f-4342-9f23-62f353f787fa None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:52.572096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67eadc-dd5f-4342-9f23-62f353f787fa None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:52.572547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67eadc-dd5f-4342-9f23-62f353f787fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:52.602711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7b67eadc-dd5f-4342-9f23-62f353f787fa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:52.606108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7b67eadc-dd5f-4342-9f23-62f353f787fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:52.628182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab168b-c17c-4ec0-9451-70ed435ae2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:52.629249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab168b-c17c-4ec0-9451-70ed435ae2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:52.629768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab168b-c17c-4ec0-9451-70ed435ae2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:52.630232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab168b-c17c-4ec0-9451-70ed435ae2b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:52.632227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab168b-c17c-4ec0-9451-70ed435ae2b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:48:52.632646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab168b-c17c-4ec0-9451-70ed435ae2b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:52.634429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab168b-c17c-4ec0-9451-70ed435ae2b1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:52.659181 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 876/1746] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:48:52.659181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab168b-c17c-4ec0-9451-70ed435ae2b1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:52.661725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-48ab168b-c17c-4ec0-9451-70ed435ae2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:52.752857 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 871/1747] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:52.754752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078dd8f-6af1-4c0f-b566-8e53484900e1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:52.755256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078dd8f-6af1-4c0f-b566-8e53484900e1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:52.755684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078dd8f-6af1-4c0f-b566-8e53484900e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:52.799696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b078dd8f-6af1-4c0f-b566-8e53484900e1 None None] MFA Rules not processed for user `64b7035c42e04fef99215dbd84b841c3`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:52.801231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b078dd8f-6af1-4c0f-b566-8e53484900e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:52.865840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:52.919035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab168b-c17c-4ec0-9451-70ed435ae2b1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:52.924187 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 872/1748] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:52.925323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078dd8f-6af1-4c0f-b566-8e53484900e1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:52.941973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078dd8f-6af1-4c0f-b566-8e53484900e1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:52.949027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbeb76-5ae0-4ec9-8c07-cca5e0f2c2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:52.950166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbeb76-5ae0-4ec9-8c07-cca5e0f2c2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:52.950560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbeb76-5ae0-4ec9-8c07-cca5e0f2c2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:52.951629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbeb76-5ae0-4ec9-8c07-cca5e0f2c2c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:52.952389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbeb76-5ae0-4ec9-8c07-cca5e0f2c2c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:52.954656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbeb76-5ae0-4ec9-8c07-cca5e0f2c2c4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:52.993968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:53.011516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c60605-0daf-4af8-b7b2-5abab92da260 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:53.012324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c60605-0daf-4af8-b7b2-5abab92da260 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.012614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c60605-0daf-4af8-b7b2-5abab92da260 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.012890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c60605-0daf-4af8-b7b2-5abab92da260 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:53.017625 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 873/1749] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:52 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:53.018713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbeb76-5ae0-4ec9-8c07-cca5e0f2c2c4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:53.021393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-51bbeb76-5ae0-4ec9-8c07-cca5e0f2c2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:53.109185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbeb76-5ae0-4ec9-8c07-cca5e0f2c2c4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:53.112320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c60605-0daf-4af8-b7b2-5abab92da260 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:53.114107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c60605-0daf-4af8-b7b2-5abab92da260 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:53.114893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117f1f9-10ab-4dba-9941-b5c3d20a5b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:53.115697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117f1f9-10ab-4dba-9941-b5c3d20a5b07 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.116075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117f1f9-10ab-4dba-9941-b5c3d20a5b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.116445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117f1f9-10ab-4dba-9941-b5c3d20a5b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:53.136193 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 877/1750] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:48:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:48:53.137378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c60605-0daf-4af8-b7b2-5abab92da260 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:53.152875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c60605-0daf-4af8-b7b2-5abab92da260 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:53.197407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117f1f9-10ab-4dba-9941-b5c3d20a5b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:53.199377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117f1f9-10ab-4dba-9941-b5c3d20a5b07 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:53.225693 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 874/1751] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:53.243069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117f1f9-10ab-4dba-9941-b5c3d20a5b07 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:53.261629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117f1f9-10ab-4dba-9941-b5c3d20a5b07 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:53.275057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e43b43-9c5e-4a92-b043-150a62fcff9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:53.275929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e43b43-9c5e-4a92-b043-150a62fcff9f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.276348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e43b43-9c5e-4a92-b043-150a62fcff9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.276745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e43b43-9c5e-4a92-b043-150a62fcff9f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:53.278513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e43b43-9c5e-4a92-b043-150a62fcff9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:53.283654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e43b43-9c5e-4a92-b043-150a62fcff9f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:53.295034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c634231-613f-4c7d-8f16-655f618a356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:53.295661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c634231-613f-4c7d-8f16-655f618a356a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.295954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c634231-613f-4c7d-8f16-655f618a356a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.296236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c634231-613f-4c7d-8f16-655f618a356a admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:53.297072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c634231-613f-4c7d-8f16-655f618a356a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:53.298862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c634231-613f-4c7d-8f16-655f618a356a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:53.341013 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 875/1752] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:53 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:53.342525 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 878/1753] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:53 2022] POST /identity/v3/projects => generated 377 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:53.346167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e43b43-9c5e-4a92-b043-150a62fcff9f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:53.349895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c634231-613f-4c7d-8f16-655f618a356a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:53.370188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e43b43-9c5e-4a92-b043-150a62fcff9f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:53.490191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:53.509615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c634231-613f-4c7d-8f16-655f618a356a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:53.532307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dcaca3-792f-4293-be2c-f6d999247680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:53.532973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcaca3-792f-4293-be2c-f6d999247680 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.533267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcaca3-792f-4293-be2c-f6d999247680 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.533552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcaca3-792f-4293-be2c-f6d999247680 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:53.536374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e3d06-f1ac-4d4f-916d-321c9d6e48ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:53.538149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e3d06-f1ac-4d4f-916d-321c9d6e48ef admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.538608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e3d06-f1ac-4d4f-916d-321c9d6e48ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.538898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e3d06-f1ac-4d4f-916d-321c9d6e48ef admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:53.539906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dcaca3-792f-4293-be2c-f6d999247680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:53.541617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e3d06-f1ac-4d4f-916d-321c9d6e48ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:53.545394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e3d06-f1ac-4d4f-916d-321c9d6e48ef admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:53.547095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dcaca3-792f-4293-be2c-f6d999247680 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:53.573768 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 876/1754] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:48:53.577970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dcaca3-792f-4293-be2c-f6d999247680 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:53.608877 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 879/1755] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:53 2022] POST /identity/v3/users => generated 295 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:53.609762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dcaca3-792f-4293-be2c-f6d999247680 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:53.612873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e3d06-f1ac-4d4f-916d-321c9d6e48ef admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:53.618062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bd56e-279b-49cc-9e60-5d4c77ec6cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:53.625249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bd56e-279b-49cc-9e60-5d4c77ec6cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.625779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bd56e-279b-49cc-9e60-5d4c77ec6cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.626073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bd56e-279b-49cc-9e60-5d4c77ec6cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:53.626923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bd56e-279b-49cc-9e60-5d4c77ec6cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:53.639917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bd56e-279b-49cc-9e60-5d4c77ec6cf5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:53.644004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e3d06-f1ac-4d4f-916d-321c9d6e48ef admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:53.660411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb187c-7721-40f6-849a-29cf50fed97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:53.661216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb187c-7721-40f6-849a-29cf50fed97f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.661511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb187c-7721-40f6-849a-29cf50fed97f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.661793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb187c-7721-40f6-849a-29cf50fed97f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:53.662455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb187c-7721-40f6-849a-29cf50fed97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:53.669053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb187c-7721-40f6-849a-29cf50fed97f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:53.673693 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 877/1756] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:53.673851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:53.674871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bd56e-279b-49cc-9e60-5d4c77ec6cf5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:53.699353 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 880/1757] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:53.700115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bd56e-279b-49cc-9e60-5d4c77ec6cf5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:53.704408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb187c-7721-40f6-849a-29cf50fed97f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:53.710501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a10118-ff85-4469-b9d5-8788518fbfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:53.712469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a10118-ff85-4469-b9d5-8788518fbfdb admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.713171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a10118-ff85-4469-b9d5-8788518fbfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.714551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a10118-ff85-4469-b9d5-8788518fbfdb admin admin] PATH_INFO: `/v3/projects/961e40fa8faf4830963a431fcddd43fa/users/a35b1565100f482d955648755c2f04ef/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:53.729445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb187c-7721-40f6-849a-29cf50fed97f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:53.746517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87694b-e9e3-4a54-8ed7-625f1648a5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:53.747214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87694b-e9e3-4a54-8ed7-625f1648a5a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.747505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87694b-e9e3-4a54-8ed7-625f1648a5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.748319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87694b-e9e3-4a54-8ed7-625f1648a5a0 admin admin] PATH_INFO: `/v3/projects/2ea65b7e844f4c95b6d7022fe811d9bb/users/5151406eb4554dac8aeb2fcd9ca205ae/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:53.762114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a10118-ff85-4469-b9d5-8788518fbfdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=961e40fa8faf4830963a431fcddd43fa, user_id=a35b1565100f482d955648755c2f04ef, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:53.765816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a10118-ff85-4469-b9d5-8788518fbfdb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:53.782973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87694b-e9e3-4a54-8ed7-625f1648a5a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ea65b7e844f4c95b6d7022fe811d9bb, user_id=5151406eb4554dac8aeb2fcd9ca205ae, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:53.791392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87694b-e9e3-4a54-8ed7-625f1648a5a0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:53.802755 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 878/1758] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:53 2022] PUT /identity/v3/projects/961e40fa8faf4830963a431fcddd43fa/users/a35b1565100f482d955648755c2f04ef/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:53.805994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5faaaa-e670-47b3-9a80-d9f69a531165 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.805994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5faaaa-e670-47b3-9a80-d9f69a531165 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.806587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5faaaa-e670-47b3-9a80-d9f69a531165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:53.821106 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 881/1759] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:53 2022] PUT /identity/v3/projects/2ea65b7e844f4c95b6d7022fe811d9bb/users/5151406eb4554dac8aeb2fcd9ca205ae/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:48:53.825265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87694b-e9e3-4a54-8ed7-625f1648a5a0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:53.939064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5b5faaaa-e670-47b3-9a80-d9f69a531165 None None] MFA Rules not processed for user `a35b1565100f482d955648755c2f04ef`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:53.944681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5b5faaaa-e670-47b3-9a80-d9f69a531165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:53.967931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87694b-e9e3-4a54-8ed7-625f1648a5a0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:53.973419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093197f-99d0-4cdf-850d-118eb6a5e236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:53.974154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093197f-99d0-4cdf-850d-118eb6a5e236 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.974335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093197f-99d0-4cdf-850d-118eb6a5e236 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.974719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093197f-99d0-4cdf-850d-118eb6a5e236 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:53.975511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093197f-99d0-4cdf-850d-118eb6a5e236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:53.977530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093197f-99d0-4cdf-850d-118eb6a5e236 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:53.993811 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 882/1760] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:48:53.998415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f2cfe-981f-4cf3-adef-5d8756ccb8a9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:53.998976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f2cfe-981f-4cf3-adef-5d8756ccb8a9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:53.999239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f2cfe-981f-4cf3-adef-5d8756ccb8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:54.039850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-525f2cfe-981f-4cf3-adef-5d8756ccb8a9 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:54.042227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-525f2cfe-981f-4cf3-adef-5d8756ccb8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:54.214178 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 879/1761] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:54.214178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5faaaa-e670-47b3-9a80-d9f69a531165 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:54.246173 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 883/1762] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:48:54.247179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f2cfe-981f-4cf3-adef-5d8756ccb8a9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:54.249078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5faaaa-e670-47b3-9a80-d9f69a531165 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:54.264820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b13df-849e-4c4a-bf66-f5aabc113451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:54.268564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b13df-849e-4c4a-bf66-f5aabc113451 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:54.269191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b13df-849e-4c4a-bf66-f5aabc113451 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:54.269480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b13df-849e-4c4a-bf66-f5aabc113451 admin admin] PATH_INFO: `/v3/projects/2ea65b7e844f4c95b6d7022fe811d9bb/users/5151406eb4554dac8aeb2fcd9ca205ae/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:54.274207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b13df-849e-4c4a-bf66-f5aabc113451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ea65b7e844f4c95b6d7022fe811d9bb, user_id=5151406eb4554dac8aeb2fcd9ca205ae, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:54.276963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b13df-849e-4c4a-bf66-f5aabc113451 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:54.283574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f2cfe-981f-4cf3-adef-5d8756ccb8a9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:54.295310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce144b-1b7e-441f-8b52-0c33d62b2dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:54.296277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce144b-1b7e-441f-8b52-0c33d62b2dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:54.296575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce144b-1b7e-441f-8b52-0c33d62b2dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:54.296857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce144b-1b7e-441f-8b52-0c33d62b2dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:54.299752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce144b-1b7e-441f-8b52-0c33d62b2dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:54.302893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce144b-1b7e-441f-8b52-0c33d62b2dd7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:54.307887 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 880/1763] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:54 2022] PUT /identity/v3/projects/2ea65b7e844f4c95b6d7022fe811d9bb/users/5151406eb4554dac8aeb2fcd9ca205ae/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:54.308877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b13df-849e-4c4a-bf66-f5aabc113451 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:54.400777 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 884/1764] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:48:54 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:54.404869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c10db1-f04f-4d35-ad7e-e5827e08960e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:54.405191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c10db1-f04f-4d35-ad7e-e5827e08960e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:54.405509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c10db1-f04f-4d35-ad7e-e5827e08960e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:54.421142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b13df-849e-4c4a-bf66-f5aabc113451 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:54.481871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:54.566093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e2c10db1-f04f-4d35-ad7e-e5827e08960e None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:54.571626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e2c10db1-f04f-4d35-ad7e-e5827e08960e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:54.585478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ead989-0f2b-4038-9d74-52f00f923369 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:54.586174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ead989-0f2b-4038-9d74-52f00f923369 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:54.586423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ead989-0f2b-4038-9d74-52f00f923369 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:54.586878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ead989-0f2b-4038-9d74-52f00f923369 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:54.681539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:54.728852 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 885/1765] 10.222.0.30 () {58 vars in 1015 bytes} [Sat Sep 3 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:54.730557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958ab83-2e4f-4834-be4c-ba12a61f6d8a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:54.731479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958ab83-2e4f-4834-be4c-ba12a61f6d8a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:54.732975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ead989-0f2b-4038-9d74-52f00f923369 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:54.734778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ead989-0f2b-4038-9d74-52f00f923369 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:54.736179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958ab83-2e4f-4834-be4c-ba12a61f6d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:54.746352 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 881/1766] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:48:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:48:54.747529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ead989-0f2b-4038-9d74-52f00f923369 service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:54.829755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6958ab83-2e4f-4834-be4c-ba12a61f6d8a None None] MFA Rules not processed for user `5151406eb4554dac8aeb2fcd9ca205ae`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:54.831370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6958ab83-2e4f-4834-be4c-ba12a61f6d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:54.886300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ead989-0f2b-4038-9d74-52f00f923369 service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:54.894267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17bc4c-e05d-4a68-b930-0010dc8ccc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:54.895303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17bc4c-e05d-4a68-b930-0010dc8ccc54 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:54.895857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17bc4c-e05d-4a68-b930-0010dc8ccc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:54.896350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17bc4c-e05d-4a68-b930-0010dc8ccc54 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:54.897256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17bc4c-e05d-4a68-b930-0010dc8ccc54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:54.899624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17bc4c-e05d-4a68-b930-0010dc8ccc54 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:54.947736 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 882/1767] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:48:54 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:48:54.952148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17bc4c-e05d-4a68-b930-0010dc8ccc54 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:54.959993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-af17bc4c-e05d-4a68-b930-0010dc8ccc54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:55.036157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17bc4c-e05d-4a68-b930-0010dc8ccc54 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:55.052089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf73552-dbc6-4e18-ad6c-a2d5641a1742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:55.054708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf73552-dbc6-4e18-ad6c-a2d5641a1742 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:55.055071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf73552-dbc6-4e18-ad6c-a2d5641a1742 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:55.056839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf73552-dbc6-4e18-ad6c-a2d5641a1742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:55.058608 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 886/1768] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:55.059772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958ab83-2e4f-4834-be4c-ba12a61f6d8a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:55.080288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf73552-dbc6-4e18-ad6c-a2d5641a1742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:55.083998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf73552-dbc6-4e18-ad6c-a2d5641a1742 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:55.085731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958ab83-2e4f-4834-be4c-ba12a61f6d8a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:55.102912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff4d81-ba48-4588-95a5-d77dccd79727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:55.103511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff4d81-ba48-4588-95a5-d77dccd79727 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:55.103802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff4d81-ba48-4588-95a5-d77dccd79727 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:55.104187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff4d81-ba48-4588-95a5-d77dccd79727 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:55.104858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff4d81-ba48-4588-95a5-d77dccd79727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:55.106782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff4d81-ba48-4588-95a5-d77dccd79727 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:55.125087 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 883/1769] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:55.126056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf73552-dbc6-4e18-ad6c-a2d5641a1742 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:55.134473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bcf73552-dbc6-4e18-ad6c-a2d5641a1742 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:55.142137 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 887/1770] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:55.142983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff4d81-ba48-4588-95a5-d77dccd79727 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:55.179987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff4d81-ba48-4588-95a5-d77dccd79727 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:55.191885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650279e-fe7b-4dd7-94cc-ad9e97524b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:55.193302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650279e-fe7b-4dd7-94cc-ad9e97524b59 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:55.195262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650279e-fe7b-4dd7-94cc-ad9e97524b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:55.196055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650279e-fe7b-4dd7-94cc-ad9e97524b59 admin admin] PATH_INFO: `/v3/projects/0142c22d5bbc410db839863c4cf85d04/users/de8a166cd63c4c77a695d57ea00ba649/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:55.253763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650279e-fe7b-4dd7-94cc-ad9e97524b59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0142c22d5bbc410db839863c4cf85d04, user_id=de8a166cd63c4c77a695d57ea00ba649, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:55.260760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650279e-fe7b-4dd7-94cc-ad9e97524b59 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:55.288303 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 888/1771] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:55 2022] PUT /identity/v3/projects/0142c22d5bbc410db839863c4cf85d04/users/de8a166cd63c4c77a695d57ea00ba649/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:55.289560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650279e-fe7b-4dd7-94cc-ad9e97524b59 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:55.290110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:55.376351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf73552-dbc6-4e18-ad6c-a2d5641a1742 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:55.409007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650279e-fe7b-4dd7-94cc-ad9e97524b59 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:55.416802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a54315-a28b-4f23-8094-df500e7fac67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:55.418161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54315-a28b-4f23-8094-df500e7fac67 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:55.418598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54315-a28b-4f23-8094-df500e7fac67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:55.419126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54315-a28b-4f23-8094-df500e7fac67 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:55.420393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a54315-a28b-4f23-8094-df500e7fac67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:55.423402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a54315-a28b-4f23-8094-df500e7fac67 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:55.447932 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 889/1772] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:48:55.450050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a54315-a28b-4f23-8094-df500e7fac67 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:55.470022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a54315-a28b-4f23-8094-df500e7fac67 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:55.479712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383843b5-97db-4f77-bbd4-41589a50b358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:55.487060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383843b5-97db-4f77-bbd4-41589a50b358 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:55.488502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383843b5-97db-4f77-bbd4-41589a50b358 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:55.488965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383843b5-97db-4f77-bbd4-41589a50b358 admin admin] PATH_INFO: `/v3/projects/0142c22d5bbc410db839863c4cf85d04/users/de8a166cd63c4c77a695d57ea00ba649/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:55.499611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383843b5-97db-4f77-bbd4-41589a50b358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0142c22d5bbc410db839863c4cf85d04, user_id=de8a166cd63c4c77a695d57ea00ba649, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:55.514025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383843b5-97db-4f77-bbd4-41589a50b358 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:55.532639 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 890/1773] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:48:55 2022] PUT /identity/v3/projects/0142c22d5bbc410db839863c4cf85d04/users/de8a166cd63c4c77a695d57ea00ba649/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:55.546237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87be82e-443e-46a4-8674-2a1c089a8f0e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:55.546576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87be82e-443e-46a4-8674-2a1c089a8f0e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:55.546821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87be82e-443e-46a4-8674-2a1c089a8f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:55.568811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce72404-c2a6-438f-a36c-d0476eaf897e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:55.570958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce72404-c2a6-438f-a36c-d0476eaf897e service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:55.572601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce72404-c2a6-438f-a36c-d0476eaf897e service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:55.573538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce72404-c2a6-438f-a36c-d0476eaf897e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:55.627473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f87be82e-443e-46a4-8674-2a1c089a8f0e None None] MFA Rules not processed for user `de8a166cd63c4c77a695d57ea00ba649`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:55.631230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f87be82e-443e-46a4-8674-2a1c089a8f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:55.653273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:55.740273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce72404-c2a6-438f-a36c-d0476eaf897e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:55.745785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce72404-c2a6-438f-a36c-d0476eaf897e service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:55.776174 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 884/1774] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:55.924956 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 891/1775] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:56.271282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce72404-c2a6-438f-a36c-d0476eaf897e service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:56.273959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6ce72404-c2a6-438f-a36c-d0476eaf897e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:56.293184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce72404-c2a6-438f-a36c-d0476eaf897e service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:56.479800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee24ac-7796-40d0-9d6b-27dc9db58dc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:56.480597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee24ac-7796-40d0-9d6b-27dc9db58dc7 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:56.481194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee24ac-7796-40d0-9d6b-27dc9db58dc7 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:56.481602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee24ac-7796-40d0-9d6b-27dc9db58dc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:56.501664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee24ac-7796-40d0-9d6b-27dc9db58dc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:56.503360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee24ac-7796-40d0-9d6b-27dc9db58dc7 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:56.534586 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 885/1776] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:56.718259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87be82e-443e-46a4-8674-2a1c089a8f0e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:56.738145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87be82e-443e-46a4-8674-2a1c089a8f0e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:56.744308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b989c47-c4f8-4a47-a7ad-35e8e1aa02be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:56.745040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b989c47-c4f8-4a47-a7ad-35e8e1aa02be service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:56.745338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b989c47-c4f8-4a47-a7ad-35e8e1aa02be service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:56.745622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b989c47-c4f8-4a47-a7ad-35e8e1aa02be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:56.847935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b989c47-c4f8-4a47-a7ad-35e8e1aa02be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:56.849852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b989c47-c4f8-4a47-a7ad-35e8e1aa02be service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:56.874366 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 892/1777] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:57.403773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee24ac-7796-40d0-9d6b-27dc9db58dc7 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:57.418657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:57.690614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422495f3-0a42-4e4c-a864-b2f5c24b6a3c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:57.690897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422495f3-0a42-4e4c-a864-b2f5c24b6a3c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:57.691192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422495f3-0a42-4e4c-a864-b2f5c24b6a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:57.700137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee24ac-7796-40d0-9d6b-27dc9db58dc7 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:57.707475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d0e1d-0333-4b10-b07c-bca448b6aa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:57.708828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d0e1d-0333-4b10-b07c-bca448b6aa4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:57.709254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d0e1d-0333-4b10-b07c-bca448b6aa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:57.709641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d0e1d-0333-4b10-b07c-bca448b6aa4c admin admin] PATH_INFO: `/v3/users/a35b1565100f482d955648755c2f04ef` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:57.710996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d0e1d-0333-4b10-b07c-bca448b6aa4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a35b1565100f482d955648755c2f04ef)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:57.714417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d0e1d-0333-4b10-b07c-bca448b6aa4c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:57.765371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-422495f3-0a42-4e4c-a864-b2f5c24b6a3c None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:57.766598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-422495f3-0a42-4e4c-a864-b2f5c24b6a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:57.822446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae8d0e1d-0333-4b10-b07c-bca448b6aa4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a35b1565100f482d955648755c2f04ef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:57.864841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae8d0e1d-0333-4b10-b07c-bca448b6aa4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a35b1565100f482d955648755c2f04ef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:57.867400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae8d0e1d-0333-4b10-b07c-bca448b6aa4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a35b1565100f482d955648755c2f04ef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:57.884144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae8d0e1d-0333-4b10-b07c-bca448b6aa4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a35b1565100f482d955648755c2f04ef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:57.901447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae8d0e1d-0333-4b10-b07c-bca448b6aa4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a35b1565100f482d955648755c2f04ef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:48:57.916583 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 886/1778] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:48:57 2022] DELETE /identity/v3/users/a35b1565100f482d955648755c2f04ef => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:48:57.921890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06884af4-6b54-4004-bcf7-979373435800 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:57.922369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06884af4-6b54-4004-bcf7-979373435800 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:57.922706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06884af4-6b54-4004-bcf7-979373435800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:57.960274 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 893/1779] 10.222.0.42 () {58 vars in 1015 bytes} [Sat Sep 3 20:48:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:57.972294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-06884af4-6b54-4004-bcf7-979373435800 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:57.974555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-06884af4-6b54-4004-bcf7-979373435800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:57.976671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422495f3-0a42-4e4c-a864-b2f5c24b6a3c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:57.978449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-422495f3-0a42-4e4c-a864-b2f5c24b6a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:58.015773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422495f3-0a42-4e4c-a864-b2f5c24b6a3c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:58.112155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a01e8a-0de1-414f-9966-2f3851e7c21b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:58.113977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a01e8a-0de1-414f-9966-2f3851e7c21b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:58.114415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a01e8a-0de1-414f-9966-2f3851e7c21b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:58.114842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a01e8a-0de1-414f-9966-2f3851e7c21b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:58.128586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a01e8a-0de1-414f-9966-2f3851e7c21b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:58.130290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a01e8a-0de1-414f-9966-2f3851e7c21b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:58.134190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:58.142533 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 894/1780] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:48:58.158257 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 887/1781] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:58.187879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a01e8a-0de1-414f-9966-2f3851e7c21b service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:58.189683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a0a01e8a-0de1-414f-9966-2f3851e7c21b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:58.203530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a01e8a-0de1-414f-9966-2f3851e7c21b service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:58.224225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41489721-3abd-4961-b5cd-50db7cec9710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:58.225851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41489721-3abd-4961-b5cd-50db7cec9710 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:58.228861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41489721-3abd-4961-b5cd-50db7cec9710 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:58.229355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41489721-3abd-4961-b5cd-50db7cec9710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:58.269185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41489721-3abd-4961-b5cd-50db7cec9710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:58.273143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41489721-3abd-4961-b5cd-50db7cec9710 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:58.281134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06884af4-6b54-4004-bcf7-979373435800 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:58.284686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-06884af4-6b54-4004-bcf7-979373435800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:58.321132 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 895/1782] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:48:58.343095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:48:58.402333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06884af4-6b54-4004-bcf7-979373435800 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:58.410528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe18fe-0622-48dd-8caf-3b2ab172fd02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:58.411233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe18fe-0622-48dd-8caf-3b2ab172fd02 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:58.411468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe18fe-0622-48dd-8caf-3b2ab172fd02 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:58.411710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe18fe-0622-48dd-8caf-3b2ab172fd02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:58.416135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-48fe18fe-0622-48dd-8caf-3b2ab172fd02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:58.518735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe18fe-0622-48dd-8caf-3b2ab172fd02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:58.521144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe18fe-0622-48dd-8caf-3b2ab172fd02 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:58.537669 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 888/1783] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:48:59.512023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23393ed-04e0-4126-b224-c3e938ffdc24 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:59.512740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23393ed-04e0-4126-b224-c3e938ffdc24 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:59.512740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23393ed-04e0-4126-b224-c3e938ffdc24 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:59.517684 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 896/1784] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:48:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:48:59.525017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e03b8-6486-418c-8979-f9c74e31a843 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:59.525566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e03b8-6486-418c-8979-f9c74e31a843 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:59.525566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e03b8-6486-418c-8979-f9c74e31a843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:59.570013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-180e03b8-6486-418c-8979-f9c74e31a843 None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:59.585228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-180e03b8-6486-418c-8979-f9c74e31a843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:59.592020 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 889/1785] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:48:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:48:59.616023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79b1bc-d56b-4005-bd28-a09b9347136b None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:59.617165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79b1bc-d56b-4005-bd28-a09b9347136b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:59.617589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79b1bc-d56b-4005-bd28-a09b9347136b None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:59.619565 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 897/1786] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:48:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:48:59.766503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e03b8-6486-418c-8979-f9c74e31a843 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:48:59.768581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-180e03b8-6486-418c-8979-f9c74e31a843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:59.779900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdede8-3f19-41e5-8869-422535299a7e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:59.780288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdede8-3f19-41e5-8869-422535299a7e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:59.781034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdede8-3f19-41e5-8869-422535299a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:59.802433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e03b8-6486-418c-8979-f9c74e31a843 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:48:59.808313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5fcdede8-3f19-41e5-8869-422535299a7e None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:48:59.812770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5fcdede8-3f19-41e5-8869-422535299a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:48:59.821996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624119d-913d-40c3-9eec-9ea347d30739 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:48:59.825455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624119d-913d-40c3-9eec-9ea347d30739 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:48:59.825850 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 898/1787] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:48:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:48:59.826393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624119d-913d-40c3-9eec-9ea347d30739 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:48:59.831468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624119d-913d-40c3-9eec-9ea347d30739 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:48:59.848961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624119d-913d-40c3-9eec-9ea347d30739 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:48:59.851124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624119d-913d-40c3-9eec-9ea347d30739 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:48:59.867081 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 890/1788] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:48:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:00.056392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdede8-3f19-41e5-8869-422535299a7e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:00.069589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdede8-3f19-41e5-8869-422535299a7e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:00.105145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc43e58-a399-41bf-b049-b5ef366e42a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:00.105145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc43e58-a399-41bf-b049-b5ef366e42a9 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:00.113269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc43e58-a399-41bf-b049-b5ef366e42a9 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:00.113817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc43e58-a399-41bf-b049-b5ef366e42a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:00.141385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc43e58-a399-41bf-b049-b5ef366e42a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:00.144639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc43e58-a399-41bf-b049-b5ef366e42a9 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:00.161860 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 899/1789] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:00.262642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624119d-913d-40c3-9eec-9ea347d30739 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:00.265883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4624119d-913d-40c3-9eec-9ea347d30739 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:00.337873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624119d-913d-40c3-9eec-9ea347d30739 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:00.366658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:00.367331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:00.367618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:00.367899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] PATH_INFO: `/v3/projects/961e40fa8faf4830963a431fcddd43fa` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:00.378330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=961e40fa8faf4830963a431fcddd43fa)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:00.381760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:00.533607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '961e40fa8faf4830963a431fcddd43fa', 'request_id': 'req-53d9bcd6-6959-4748-8e37-0ce949511d42'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:00.558977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '961e40fa8faf4830963a431fcddd43fa', 'request_id': 'req-53d9bcd6-6959-4748-8e37-0ce949511d42'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:00.589626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '961e40fa8faf4830963a431fcddd43fa', 'request_id': 'req-53d9bcd6-6959-4748-8e37-0ce949511d42'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:00.609207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] The token cache is being invalidate because project 961e40fa8faf4830963a431fcddd43fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:00.609781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:00.610621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:00.613435 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 891/1790] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:00 2022] DELETE /identity/v3/projects/961e40fa8faf4830963a431fcddd43fa => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:00.619388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc43e58-a399-41bf-b049-b5ef366e42a9 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:00.638388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc43e58-a399-41bf-b049-b5ef366e42a9 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:00.825618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:00.827743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:00.828038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:00.828382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] PATH_INFO: `/v3/users/de8a166cd63c4c77a695d57ea00ba649` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:00.835197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de8a166cd63c4c77a695d57ea00ba649)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:00.837404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:00.950784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de8a166cd63c4c77a695d57ea00ba649'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:00.980999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de8a166cd63c4c77a695d57ea00ba649'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:00.995815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de8a166cd63c4c77a695d57ea00ba649'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:01.001431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de8a166cd63c4c77a695d57ea00ba649'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:01.002166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de8a166cd63c4c77a695d57ea00ba649'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:01.031777 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 900/1791] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:00 2022] DELETE /identity/v3/users/de8a166cd63c4c77a695d57ea00ba649 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:03.013040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:03.015782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:03.051263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9bcd6-6959-4748-8e37-0ce949511d42 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:03.085729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:03.247631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3d733-7d00-4e99-a3b9-ed25472d4614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:03.250970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3d733-7d00-4e99-a3b9-ed25472d4614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:03.251887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3d733-7d00-4e99-a3b9-ed25472d4614 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:03.253449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3d733-7d00-4e99-a3b9-ed25472d4614 admin admin] PATH_INFO: `/v3/users/64b7035c42e04fef99215dbd84b841c3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:03.255896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3d733-7d00-4e99-a3b9-ed25472d4614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b7035c42e04fef99215dbd84b841c3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:03.261304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:03.262995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:03.277960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3d733-7d00-4e99-a3b9-ed25472d4614 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:03.322099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d4f54-57a2-4c62-b905-2a0ab1df4f46 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:03.346123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16993bf3-6371-4e2b-8efc-3a260e76bf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:03.347194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16993bf3-6371-4e2b-8efc-3a260e76bf52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:03.349452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16993bf3-6371-4e2b-8efc-3a260e76bf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:03.350116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16993bf3-6371-4e2b-8efc-3a260e76bf52 admin admin] PATH_INFO: `/v3/projects/0142c22d5bbc410db839863c4cf85d04` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:03.354482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16993bf3-6371-4e2b-8efc-3a260e76bf52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0142c22d5bbc410db839863c4cf85d04)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:03.361446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16993bf3-6371-4e2b-8efc-3a260e76bf52 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:03.439334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d4c3d733-7d00-4e99-a3b9-ed25472d4614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b7035c42e04fef99215dbd84b841c3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:03.490985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d4c3d733-7d00-4e99-a3b9-ed25472d4614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b7035c42e04fef99215dbd84b841c3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:03.491783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d4c3d733-7d00-4e99-a3b9-ed25472d4614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b7035c42e04fef99215dbd84b841c3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:03.514197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d4c3d733-7d00-4e99-a3b9-ed25472d4614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b7035c42e04fef99215dbd84b841c3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:03.531375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d4c3d733-7d00-4e99-a3b9-ed25472d4614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b7035c42e04fef99215dbd84b841c3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:03.539586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:03.545058 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 892/1792] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:03 2022] DELETE /identity/v3/users/64b7035c42e04fef99215dbd84b841c3 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:03.547488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c9b62-2d8c-4a5a-8d78-3ab6d6ef9fbb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:03.549255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c9b62-2d8c-4a5a-8d78-3ab6d6ef9fbb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:03.550048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c9b62-2d8c-4a5a-8d78-3ab6d6ef9fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:03.575574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16993bf3-6371-4e2b-8efc-3a260e76bf52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0142c22d5bbc410db839863c4cf85d04', 'request_id': 'req-16993bf3-6371-4e2b-8efc-3a260e76bf52'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:03.602180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16993bf3-6371-4e2b-8efc-3a260e76bf52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0142c22d5bbc410db839863c4cf85d04', 'request_id': 'req-16993bf3-6371-4e2b-8efc-3a260e76bf52'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:03.617509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-694c9b62-2d8c-4a5a-8d78-3ab6d6ef9fbb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:03.619079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-694c9b62-2d8c-4a5a-8d78-3ab6d6ef9fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:03.631451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16993bf3-6371-4e2b-8efc-3a260e76bf52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0142c22d5bbc410db839863c4cf85d04', 'request_id': 'req-16993bf3-6371-4e2b-8efc-3a260e76bf52'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:03.677325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16993bf3-6371-4e2b-8efc-3a260e76bf52 admin admin] The token cache is being invalidate because project 0142c22d5bbc410db839863c4cf85d04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:03.679880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16993bf3-6371-4e2b-8efc-3a260e76bf52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:03.692919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16993bf3-6371-4e2b-8efc-3a260e76bf52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:03.702084 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 901/1793] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:03 2022] DELETE /identity/v3/projects/0142c22d5bbc410db839863c4cf85d04 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:03.706906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294e562-e514-4da7-8e81-21de9a654f31 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:03.707511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294e562-e514-4da7-8e81-21de9a654f31 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:03.708053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294e562-e514-4da7-8e81-21de9a654f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:03.744622 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 893/1794] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:03.769412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2294e562-e514-4da7-8e81-21de9a654f31 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:03.772125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2294e562-e514-4da7-8e81-21de9a654f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:03.829813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9032ee-7ebe-4c67-8565-61966e895d24 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:03.829813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9032ee-7ebe-4c67-8565-61966e895d24 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:03.829813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9032ee-7ebe-4c67-8565-61966e895d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:03.846422 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 902/1795] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:03.866627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294e562-e514-4da7-8e81-21de9a654f31 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:03.868272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2294e562-e514-4da7-8e81-21de9a654f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:03.915378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cc9032ee-7ebe-4c67-8565-61966e895d24 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:03.925671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cc9032ee-7ebe-4c67-8565-61966e895d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:03.984473 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 894/1796] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:03.988569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9032ee-7ebe-4c67-8565-61966e895d24 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:04.002577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9032ee-7ebe-4c67-8565-61966e895d24 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:04.030703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294e562-e514-4da7-8e81-21de9a654f31 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:04.035176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b09f7-7547-40df-90d3-c38b3dbe3766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:04.035887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b09f7-7547-40df-90d3-c38b3dbe3766 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:04.036228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47aad6e-b9bf-4297-86f7-6bf350f93a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:04.036731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b09f7-7547-40df-90d3-c38b3dbe3766 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:04.036731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b09f7-7547-40df-90d3-c38b3dbe3766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:04.039278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47aad6e-b9bf-4297-86f7-6bf350f93a5c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:04.041122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47aad6e-b9bf-4297-86f7-6bf350f93a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:04.041583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47aad6e-b9bf-4297-86f7-6bf350f93a5c admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:04.043298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47aad6e-b9bf-4297-86f7-6bf350f93a5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:04.048880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b09f7-7547-40df-90d3-c38b3dbe3766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:04.049836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47aad6e-b9bf-4297-86f7-6bf350f93a5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:04.057217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47aad6e-b9bf-4297-86f7-6bf350f93a5c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:04.064805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b09f7-7547-40df-90d3-c38b3dbe3766 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:04.079705 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 903/1797] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:04.124752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b09f7-7547-40df-90d3-c38b3dbe3766 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:04.137250 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 895/1798] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:49:04.142621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b09f7-7547-40df-90d3-c38b3dbe3766 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:04.150413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed4440-86ea-4b03-b022-00182c9ef818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:04.151463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed4440-86ea-4b03-b022-00182c9ef818 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:04.151974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed4440-86ea-4b03-b022-00182c9ef818 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:04.152486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed4440-86ea-4b03-b022-00182c9ef818 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:04.153481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed4440-86ea-4b03-b022-00182c9ef818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:04.157180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed4440-86ea-4b03-b022-00182c9ef818 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:04.209153 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 904/1799] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:04 2022] POST /identity/v3/projects => generated 399 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:49:04.212989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47aad6e-b9bf-4297-86f7-6bf350f93a5c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:04.231229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47aad6e-b9bf-4297-86f7-6bf350f93a5c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:04.339555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:04.444459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4914168-170a-48a9-86b8-89698dc3e83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:04.445142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4914168-170a-48a9-86b8-89698dc3e83a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:04.446002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4914168-170a-48a9-86b8-89698dc3e83a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:04.446349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4914168-170a-48a9-86b8-89698dc3e83a admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:04.447002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4914168-170a-48a9-86b8-89698dc3e83a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:04.450586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4914168-170a-48a9-86b8-89698dc3e83a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:04.485002 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 896/1800] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:04 2022] POST /identity/v3/users => generated 300 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:04.489112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed4440-86ea-4b03-b022-00182c9ef818 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:04.510117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed4440-86ea-4b03-b022-00182c9ef818 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:04.516662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a01801-55f2-4234-a791-59cf654d4825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:04.517704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a01801-55f2-4234-a791-59cf654d4825 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:04.518271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a01801-55f2-4234-a791-59cf654d4825 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:04.519041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a01801-55f2-4234-a791-59cf654d4825 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:04.520242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a01801-55f2-4234-a791-59cf654d4825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:04.522805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a01801-55f2-4234-a791-59cf654d4825 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:04.539026 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 905/1801] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:04.542658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4914168-170a-48a9-86b8-89698dc3e83a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:04.566252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4914168-170a-48a9-86b8-89698dc3e83a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:04.602200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0c337-33e8-4a02-8dd5-14349fb4e53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:04.606492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0c337-33e8-4a02-8dd5-14349fb4e53b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:04.607621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0c337-33e8-4a02-8dd5-14349fb4e53b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:04.609990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0c337-33e8-4a02-8dd5-14349fb4e53b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:04.611925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0c337-33e8-4a02-8dd5-14349fb4e53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:04.618877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0c337-33e8-4a02-8dd5-14349fb4e53b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:04.691941 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 897/1802] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:04.694679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a01801-55f2-4234-a791-59cf654d4825 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:04.713509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a01801-55f2-4234-a791-59cf654d4825 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:04.751433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2391a9d-ecb8-4f4d-9992-dcc008e645d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:04.752541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2391a9d-ecb8-4f4d-9992-dcc008e645d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:04.753071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2391a9d-ecb8-4f4d-9992-dcc008e645d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:04.753595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2391a9d-ecb8-4f4d-9992-dcc008e645d9 admin admin] PATH_INFO: `/v3/projects/0cb94fbc17264a278102253f0f61a150/users/e5380ce0bd6d4157997494de588c5156/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:04.806670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2391a9d-ecb8-4f4d-9992-dcc008e645d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb94fbc17264a278102253f0f61a150, user_id=e5380ce0bd6d4157997494de588c5156, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:04.814127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2391a9d-ecb8-4f4d-9992-dcc008e645d9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:04.841086 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 906/1803] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:04 2022] PUT /identity/v3/projects/0cb94fbc17264a278102253f0f61a150/users/e5380ce0bd6d4157997494de588c5156/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:04.855056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fbd87-5a44-4cf4-8613-180c6e4b6dd7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:04.855604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fbd87-5a44-4cf4-8613-180c6e4b6dd7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:04.856038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fbd87-5a44-4cf4-8613-180c6e4b6dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:04.921974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6a9fbd87-5a44-4cf4-8613-180c6e4b6dd7 None None] MFA Rules not processed for user `e5380ce0bd6d4157997494de588c5156`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:04.926719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6a9fbd87-5a44-4cf4-8613-180c6e4b6dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:05.158344 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 898/1804] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:04 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:05.160495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2391a9d-ecb8-4f4d-9992-dcc008e645d9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:05.349924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2391a9d-ecb8-4f4d-9992-dcc008e645d9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:05.361868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf9f0b-ecae-4275-9f53-1da95e83c3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:05.362580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf9f0b-ecae-4275-9f53-1da95e83c3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:05.365851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf9f0b-ecae-4275-9f53-1da95e83c3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:05.366223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf9f0b-ecae-4275-9f53-1da95e83c3f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:05.366926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf9f0b-ecae-4275-9f53-1da95e83c3f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:05.387393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf9f0b-ecae-4275-9f53-1da95e83c3f4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:05.431032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:05.491140 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 907/1805] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:05 2022] POST /identity/v3/projects => generated 399 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:05.492734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fbd87-5a44-4cf4-8613-180c6e4b6dd7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:05.532810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fbd87-5a44-4cf4-8613-180c6e4b6dd7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:05.764369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbe4d0-de98-45aa-9d87-efc9d8c047f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:05.766676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbe4d0-de98-45aa-9d87-efc9d8c047f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:05.767084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbe4d0-de98-45aa-9d87-efc9d8c047f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:05.768401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbe4d0-de98-45aa-9d87-efc9d8c047f7 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:05.769292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbe4d0-de98-45aa-9d87-efc9d8c047f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:05.775374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbe4d0-de98-45aa-9d87-efc9d8c047f7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:05.847847 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 899/1806] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:05 2022] POST /identity/v3/users => generated 306 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:49:05.854680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf9f0b-ecae-4275-9f53-1da95e83c3f4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:05.874390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf9f0b-ecae-4275-9f53-1da95e83c3f4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:05.888938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdec516-4363-4100-935b-f36d21fd6cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:05.889616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdec516-4363-4100-935b-f36d21fd6cff admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:05.889928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdec516-4363-4100-935b-f36d21fd6cff admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:05.890211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdec516-4363-4100-935b-f36d21fd6cff admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:05.890873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdec516-4363-4100-935b-f36d21fd6cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:05.896512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdec516-4363-4100-935b-f36d21fd6cff admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:05.939324 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 908/1807] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:05.944976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbe4d0-de98-45aa-9d87-efc9d8c047f7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:05.970102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbe4d0-de98-45aa-9d87-efc9d8c047f7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:05.978885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f237c0-b2a5-4eb9-9acf-a8c2f27d9ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:05.981063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f237c0-b2a5-4eb9-9acf-a8c2f27d9ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:05.982740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f237c0-b2a5-4eb9-9acf-a8c2f27d9ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:05.983117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f237c0-b2a5-4eb9-9acf-a8c2f27d9ff9 admin admin] PATH_INFO: `/v3/projects/739e0a081fd141eda9ac0290ac96d585/users/81ddde2d139447c584f28323f40f1fc5/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:06.023741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f237c0-b2a5-4eb9-9acf-a8c2f27d9ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=739e0a081fd141eda9ac0290ac96d585, user_id=81ddde2d139447c584f28323f40f1fc5, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:06.029187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f237c0-b2a5-4eb9-9acf-a8c2f27d9ff9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:06.038473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:06.063249 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 900/1808] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:05 2022] PUT /identity/v3/projects/739e0a081fd141eda9ac0290ac96d585/users/81ddde2d139447c584f28323f40f1fc5/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:06.065493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdec516-4363-4100-935b-f36d21fd6cff admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:06.085172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdec516-4363-4100-935b-f36d21fd6cff admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:06.110263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f237c0-b2a5-4eb9-9acf-a8c2f27d9ff9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:06.112215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c2f237c0-b2a5-4eb9-9acf-a8c2f27d9ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:06.220990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ec930-a6e4-4bbb-a4ed-b454637fa2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:06.221604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ec930-a6e4-4bbb-a4ed-b454637fa2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:06.222049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ec930-a6e4-4bbb-a4ed-b454637fa2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:06.222367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ec930-a6e4-4bbb-a4ed-b454637fa2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:06.223581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ec930-a6e4-4bbb-a4ed-b454637fa2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:06.226405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ec930-a6e4-4bbb-a4ed-b454637fa2c4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:06.240953 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 909/1809] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:06.248046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ec930-a6e4-4bbb-a4ed-b454637fa2c4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:06.277776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ec930-a6e4-4bbb-a4ed-b454637fa2c4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:06.294370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3533e2-7938-4f18-b451-aa84ac8fd3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:06.295284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3533e2-7938-4f18-b451-aa84ac8fd3ff admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:06.296404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3533e2-7938-4f18-b451-aa84ac8fd3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:06.297733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3533e2-7938-4f18-b451-aa84ac8fd3ff admin admin] PATH_INFO: `/v3/projects/739e0a081fd141eda9ac0290ac96d585/users/81ddde2d139447c584f28323f40f1fc5/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:06.305156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3533e2-7938-4f18-b451-aa84ac8fd3ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=739e0a081fd141eda9ac0290ac96d585, user_id=81ddde2d139447c584f28323f40f1fc5, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:06.309967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3533e2-7938-4f18-b451-aa84ac8fd3ff admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:06.370302 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 910/1810] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:06 2022] PUT /identity/v3/projects/739e0a081fd141eda9ac0290ac96d585/users/81ddde2d139447c584f28323f40f1fc5/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:06.383037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeb587-7726-4ad7-9a2e-05e348c4f28b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:06.383390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeb587-7726-4ad7-9a2e-05e348c4f28b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:06.383738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeb587-7726-4ad7-9a2e-05e348c4f28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:06.460418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f237c0-b2a5-4eb9-9acf-a8c2f27d9ff9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:06.480593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8bbeb587-7726-4ad7-9a2e-05e348c4f28b None None] MFA Rules not processed for user `81ddde2d139447c584f28323f40f1fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:06.483548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8bbeb587-7726-4ad7-9a2e-05e348c4f28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:06.654721 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 911/1811] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:06 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:06.667418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbeb587-7726-4ad7-9a2e-05e348c4f28b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:06.671441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8bbeb587-7726-4ad7-9a2e-05e348c4f28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:06.674918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:06.677869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:06.678995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:06.680887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] PATH_INFO: `/v3/projects/e005609cdb7945ebbf75b805621a2592` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:06.686610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e005609cdb7945ebbf75b805621a2592)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:06.693312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:06.799911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:06.862081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbeb587-7726-4ad7-9a2e-05e348c4f28b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:06.872252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e005609cdb7945ebbf75b805621a2592', 'request_id': 'req-b81d1e40-554c-4cce-af17-e0c5d411ebdd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:06.890093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e005609cdb7945ebbf75b805621a2592', 'request_id': 'req-b81d1e40-554c-4cce-af17-e0c5d411ebdd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:06.905075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e005609cdb7945ebbf75b805621a2592', 'request_id': 'req-b81d1e40-554c-4cce-af17-e0c5d411ebdd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:06.920395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] The token cache is being invalidate because project e005609cdb7945ebbf75b805621a2592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:06.920732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:06.921933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:06.924338 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 901/1812] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:06 2022] DELETE /identity/v3/projects/e005609cdb7945ebbf75b805621a2592 => generated 0 bytes in 815 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:06.929417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:06.931529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:06.979171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a71f7-ce7f-4214-ba63-533b33c13c6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:06.980172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a71f7-ce7f-4214-ba63-533b33c13c6f service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:06.980665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a71f7-ce7f-4214-ba63-533b33c13c6f service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:06.981242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a71f7-ce7f-4214-ba63-533b33c13c6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:06.985058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bd9a71f7-ce7f-4214-ba63-533b33c13c6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:07.064439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a71f7-ce7f-4214-ba63-533b33c13c6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:07.066453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a71f7-ce7f-4214-ba63-533b33c13c6f service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:07.094979 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 912/1813] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:06 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:07.160393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:07.188178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a71f7-ce7f-4214-ba63-533b33c13c6f service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:07.189761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bd9a71f7-ce7f-4214-ba63-533b33c13c6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:07.226256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a71f7-ce7f-4214-ba63-533b33c13c6f service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:07.240287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bf352-3a23-4195-8d0f-4174a8321dbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:07.240960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bf352-3a23-4195-8d0f-4174a8321dbb service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:07.241096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bf352-3a23-4195-8d0f-4174a8321dbb service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:07.241400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bf352-3a23-4195-8d0f-4174a8321dbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:07.252389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d1e40-554c-4cce-af17-e0c5d411ebdd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:07.262182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-375bf352-3a23-4195-8d0f-4174a8321dbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:07.265185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac1aef-7386-4d8a-813c-f745adbdcd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:07.265921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac1aef-7386-4d8a-813c-f745adbdcd48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:07.266278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac1aef-7386-4d8a-813c-f745adbdcd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:07.266561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac1aef-7386-4d8a-813c-f745adbdcd48 admin admin] PATH_INFO: `/v3/users/5151406eb4554dac8aeb2fcd9ca205ae` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:07.270937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac1aef-7386-4d8a-813c-f745adbdcd48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5151406eb4554dac8aeb2fcd9ca205ae)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:07.272978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac1aef-7386-4d8a-813c-f745adbdcd48 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:07.433253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-05ac1aef-7386-4d8a-813c-f745adbdcd48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5151406eb4554dac8aeb2fcd9ca205ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:07.455328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-05ac1aef-7386-4d8a-813c-f745adbdcd48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5151406eb4554dac8aeb2fcd9ca205ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:07.456820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-05ac1aef-7386-4d8a-813c-f745adbdcd48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5151406eb4554dac8aeb2fcd9ca205ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:07.470105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-05ac1aef-7386-4d8a-813c-f745adbdcd48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5151406eb4554dac8aeb2fcd9ca205ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:07.481847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bf352-3a23-4195-8d0f-4174a8321dbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:07.483726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bf352-3a23-4195-8d0f-4174a8321dbb service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:07.486889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-05ac1aef-7386-4d8a-813c-f745adbdcd48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5151406eb4554dac8aeb2fcd9ca205ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:07.505782 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 913/1814] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:07 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:49:07.508289 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 902/1815] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:06 2022] DELETE /identity/v3/users/5151406eb4554dac8aeb2fcd9ca205ae => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:08.095342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bf352-3a23-4195-8d0f-4174a8321dbb service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:08.097322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-375bf352-3a23-4195-8d0f-4174a8321dbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:08.162286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bf352-3a23-4195-8d0f-4174a8321dbb service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:08.193286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:08.274412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:08.276438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:08.276913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:08.277931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] PATH_INFO: `/v3/users/e5380ce0bd6d4157997494de588c5156` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:08.281158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5380ce0bd6d4157997494de588c5156)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:08.287375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:08.410194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5380ce0bd6d4157997494de588c5156'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:08.434451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5380ce0bd6d4157997494de588c5156'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:08.461443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5380ce0bd6d4157997494de588c5156'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:08.464655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5380ce0bd6d4157997494de588c5156'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:08.465720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5380ce0bd6d4157997494de588c5156'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:08.483606 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 914/1816] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:08 2022] DELETE /identity/v3/users/e5380ce0bd6d4157997494de588c5156 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:08.487991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30ad59-4294-4d8f-9b94-aa46bdf25c46 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:08.488949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30ad59-4294-4d8f-9b94-aa46bdf25c46 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:08.489468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30ad59-4294-4d8f-9b94-aa46bdf25c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:08.514566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2e30ad59-4294-4d8f-9b94-aa46bdf25c46 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:08.516483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2e30ad59-4294-4d8f-9b94-aa46bdf25c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:08.683988 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 903/1817] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:08.709381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:08.711638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:08.871349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30ad59-4294-4d8f-9b94-aa46bdf25c46 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:08.874203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2e30ad59-4294-4d8f-9b94-aa46bdf25c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:08.930853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30ad59-4294-4d8f-9b94-aa46bdf25c46 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:08.957655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe97898-48ff-40a1-9630-dd3cef4d7399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:08.958368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe97898-48ff-40a1-9630-dd3cef4d7399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:08.958662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe97898-48ff-40a1-9630-dd3cef4d7399 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:08.958946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe97898-48ff-40a1-9630-dd3cef4d7399 admin admin] PATH_INFO: `/v3/projects/2ea65b7e844f4c95b6d7022fe811d9bb` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:08.966532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe97898-48ff-40a1-9630-dd3cef4d7399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ea65b7e844f4c95b6d7022fe811d9bb)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:08.968499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe97898-48ff-40a1-9630-dd3cef4d7399 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:09.016268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ef139-a46e-4abf-a6fb-0712d3cfd393 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:09.021684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22565ead-a206-453d-9ddb-fa7cc94e001a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:09.023423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22565ead-a206-453d-9ddb-fa7cc94e001a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:09.025765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22565ead-a206-453d-9ddb-fa7cc94e001a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:09.026310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22565ead-a206-453d-9ddb-fa7cc94e001a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:09.051173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22565ead-a206-453d-9ddb-fa7cc94e001a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:09.053035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22565ead-a206-453d-9ddb-fa7cc94e001a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:09.054585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:09.056515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fbe97898-48ff-40a1-9630-dd3cef4d7399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ea65b7e844f4c95b6d7022fe811d9bb', 'request_id': 'req-fbe97898-48ff-40a1-9630-dd3cef4d7399'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:09.082939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fbe97898-48ff-40a1-9630-dd3cef4d7399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ea65b7e844f4c95b6d7022fe811d9bb', 'request_id': 'req-fbe97898-48ff-40a1-9630-dd3cef4d7399'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:09.093499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fbe97898-48ff-40a1-9630-dd3cef4d7399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ea65b7e844f4c95b6d7022fe811d9bb', 'request_id': 'req-fbe97898-48ff-40a1-9630-dd3cef4d7399'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:09.094437 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 915/1818] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:09.130894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fbe97898-48ff-40a1-9630-dd3cef4d7399 admin admin] The token cache is being invalidate because project 2ea65b7e844f4c95b6d7022fe811d9bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:09.131515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fbe97898-48ff-40a1-9630-dd3cef4d7399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:09.132486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fbe97898-48ff-40a1-9630-dd3cef4d7399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:09.137906 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 904/1819] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:08 2022] DELETE /identity/v3/projects/2ea65b7e844f4c95b6d7022fe811d9bb => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:09.143124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ec0cd-9cfd-43ca-955c-87c7cf18d86f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:09.144123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ec0cd-9cfd-43ca-955c-87c7cf18d86f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:09.144715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ec0cd-9cfd-43ca-955c-87c7cf18d86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:09.213886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d54ec0cd-9cfd-43ca-955c-87c7cf18d86f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:09.215787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d54ec0cd-9cfd-43ca-955c-87c7cf18d86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:09.407463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:09.516773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9f0a5-64cb-4b6a-86fb-378f09e27216 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:09.520332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9f0a5-64cb-4b6a-86fb-378f09e27216 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:09.520931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9f0a5-64cb-4b6a-86fb-378f09e27216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:09.523168 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 916/1820] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:09.596687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-78e9f0a5-64cb-4b6a-86fb-378f09e27216 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:09.598085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-78e9f0a5-64cb-4b6a-86fb-378f09e27216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:09.651230 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 905/1821] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:09.655419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ec0cd-9cfd-43ca-955c-87c7cf18d86f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:09.674681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ec0cd-9cfd-43ca-955c-87c7cf18d86f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:09.709923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc581e-2af6-4449-9925-c44d56af93b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:09.711470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc581e-2af6-4449-9925-c44d56af93b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:09.711988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc581e-2af6-4449-9925-c44d56af93b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:09.712469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc581e-2af6-4449-9925-c44d56af93b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:09.713398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc581e-2af6-4449-9925-c44d56af93b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:09.713949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc581e-2af6-4449-9925-c44d56af93b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:09.716082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc581e-2af6-4449-9925-c44d56af93b5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:09.770791 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 917/1822] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:09.771605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9f0a5-64cb-4b6a-86fb-378f09e27216 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:09.787990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9f0a5-64cb-4b6a-86fb-378f09e27216 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:09.813995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af371a-a31b-43c0-9c34-551f7b1ef2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:09.814643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af371a-a31b-43c0-9c34-551f7b1ef2eb admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:09.814932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af371a-a31b-43c0-9c34-551f7b1ef2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:09.815207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af371a-a31b-43c0-9c34-551f7b1ef2eb admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:09.815867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af371a-a31b-43c0-9c34-551f7b1ef2eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:09.836011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af371a-a31b-43c0-9c34-551f7b1ef2eb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:09.914910 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 906/1823] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:09 2022] POST /identity/v3/projects => generated 393 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:09.920163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc581e-2af6-4449-9925-c44d56af93b5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:09.945459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc581e-2af6-4449-9925-c44d56af93b5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:10.044363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a02a49-9155-4aa9-8eb0-3f241c10fd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:10.045063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a02a49-9155-4aa9-8eb0-3f241c10fd0d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:10.045420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a02a49-9155-4aa9-8eb0-3f241c10fd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:10.045706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a02a49-9155-4aa9-8eb0-3f241c10fd0d admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:10.046452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a02a49-9155-4aa9-8eb0-3f241c10fd0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:10.048758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a02a49-9155-4aa9-8eb0-3f241c10fd0d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:10.095733 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 918/1824] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:09 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:10.100449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af371a-a31b-43c0-9c34-551f7b1ef2eb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:10.115143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af371a-a31b-43c0-9c34-551f7b1ef2eb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:10.125369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054b674-9cf4-456f-a0b7-c8eeeed89a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:10.126125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054b674-9cf4-456f-a0b7-c8eeeed89a92 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:10.126492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054b674-9cf4-456f-a0b7-c8eeeed89a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:10.126849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054b674-9cf4-456f-a0b7-c8eeeed89a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:10.127893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054b674-9cf4-456f-a0b7-c8eeeed89a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:10.130248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054b674-9cf4-456f-a0b7-c8eeeed89a92 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:10.149865 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 907/1825] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:10.150698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a02a49-9155-4aa9-8eb0-3f241c10fd0d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:10.162373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a02a49-9155-4aa9-8eb0-3f241c10fd0d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:10.168038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23957c-f4c7-49a8-90a9-60926fa9dd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:10.168864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23957c-f4c7-49a8-90a9-60926fa9dd80 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:10.169179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23957c-f4c7-49a8-90a9-60926fa9dd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:10.169495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23957c-f4c7-49a8-90a9-60926fa9dd80 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:10.170160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23957c-f4c7-49a8-90a9-60926fa9dd80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:10.171839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23957c-f4c7-49a8-90a9-60926fa9dd80 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:10.184152 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 919/1826] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:49:10.189529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054b674-9cf4-456f-a0b7-c8eeeed89a92 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:10.203901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054b674-9cf4-456f-a0b7-c8eeeed89a92 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:10.208692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099006dc-37c7-4940-ac5d-f3e0484f57db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:10.209461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099006dc-37c7-4940-ac5d-f3e0484f57db admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:10.209783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099006dc-37c7-4940-ac5d-f3e0484f57db admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:10.210067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099006dc-37c7-4940-ac5d-f3e0484f57db admin admin] PATH_INFO: `/v3/projects/2a28735e9d204354a30066460fd3d271/users/fc148ca2d15445f7860e84a40d1cdde7/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:10.227224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099006dc-37c7-4940-ac5d-f3e0484f57db admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a28735e9d204354a30066460fd3d271, user_id=fc148ca2d15445f7860e84a40d1cdde7, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:10.229978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099006dc-37c7-4940-ac5d-f3e0484f57db admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:10.244799 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 908/1827] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:10 2022] PUT /identity/v3/projects/2a28735e9d204354a30066460fd3d271/users/fc148ca2d15445f7860e84a40d1cdde7/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:10.255771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61171a1-ef2f-4b84-80a3-250d5e5d3fa0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:10.256066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61171a1-ef2f-4b84-80a3-250d5e5d3fa0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:10.256378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61171a1-ef2f-4b84-80a3-250d5e5d3fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:10.283370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099006dc-37c7-4940-ac5d-f3e0484f57db admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:10.285240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-099006dc-37c7-4940-ac5d-f3e0484f57db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:10.305886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c61171a1-ef2f-4b84-80a3-250d5e5d3fa0 None None] MFA Rules not processed for user `fc148ca2d15445f7860e84a40d1cdde7`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:10.307162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c61171a1-ef2f-4b84-80a3-250d5e5d3fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:10.383874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099006dc-37c7-4940-ac5d-f3e0484f57db admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:10.386866 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 920/1828] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:10.388181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abc4e7-4a0e-4520-8d73-9d00c72a93a4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:10.389077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abc4e7-4a0e-4520-8d73-9d00c72a93a4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:10.389334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abc4e7-4a0e-4520-8d73-9d00c72a93a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:10.389882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:10.390796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:10.391382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:10.391949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] PATH_INFO: `/v3/projects/0cb94fbc17264a278102253f0f61a150` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:10.393185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cb94fbc17264a278102253f0f61a150)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:10.395100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:10.420428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d0abc4e7-4a0e-4520-8d73-9d00c72a93a4 None None] MFA Rules not processed for user `45b55868ea8a492182510bfd0133d74f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:10.421721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d0abc4e7-4a0e-4520-8d73-9d00c72a93a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:10.454119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:10.459302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cb94fbc17264a278102253f0f61a150', 'request_id': 'req-fefbf92b-e10d-42be-969a-ffaecca66a0a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:10.471437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cb94fbc17264a278102253f0f61a150', 'request_id': 'req-fefbf92b-e10d-42be-969a-ffaecca66a0a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:10.481177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cb94fbc17264a278102253f0f61a150', 'request_id': 'req-fefbf92b-e10d-42be-969a-ffaecca66a0a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:10.495169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] The token cache is being invalidate because project 0cb94fbc17264a278102253f0f61a150 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:10.495465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:10.496167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:10.498653 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 909/1829] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:10 2022] DELETE /identity/v3/projects/0cb94fbc17264a278102253f0f61a150 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:10.502579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:10.504030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:10.510583 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 921/1830] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:10.511364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abc4e7-4a0e-4520-8d73-9d00c72a93a4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:10.515454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:10.523836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abc4e7-4a0e-4520-8d73-9d00c72a93a4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:10.593585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:10.594368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:10.594656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:10.594935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] PATH_INFO: `/v3/users/81ddde2d139447c584f28323f40f1fc5` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:10.595286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbf92b-e10d-42be-969a-ffaecca66a0a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:10.596462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ddde2d139447c584f28323f40f1fc5)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:10.598446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:10.600338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc1443-c8f0-4c3d-9d4b-2a2ce9325797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:10.601365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc1443-c8f0-4c3d-9d4b-2a2ce9325797 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:10.601919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc1443-c8f0-4c3d-9d4b-2a2ce9325797 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:10.602203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc1443-c8f0-4c3d-9d4b-2a2ce9325797 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:10.602844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc1443-c8f0-4c3d-9d4b-2a2ce9325797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:10.604942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc1443-c8f0-4c3d-9d4b-2a2ce9325797 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:10.657850 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 910/1831] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:10 2022] POST /identity/v3/projects => generated 393 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:10.658733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc1443-c8f0-4c3d-9d4b-2a2ce9325797 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:10.660871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-96bc1443-c8f0-4c3d-9d4b-2a2ce9325797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:10.682465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ddde2d139447c584f28323f40f1fc5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:10.699104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ddde2d139447c584f28323f40f1fc5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:10.712992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ddde2d139447c584f28323f40f1fc5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:10.714001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ddde2d139447c584f28323f40f1fc5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:10.714647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ddde2d139447c584f28323f40f1fc5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:10.738953 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 922/1832] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:10 2022] DELETE /identity/v3/users/81ddde2d139447c584f28323f40f1fc5 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:10.740007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:10.741504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:10.768561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc1443-c8f0-4c3d-9d4b-2a2ce9325797 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:10.822448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:10.850491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226058a7-2de1-4bcc-afb0-bdc75c7d2272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:10.851500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226058a7-2de1-4bcc-afb0-bdc75c7d2272 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:10.852687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226058a7-2de1-4bcc-afb0-bdc75c7d2272 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:10.853352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226058a7-2de1-4bcc-afb0-bdc75c7d2272 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:10.854293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226058a7-2de1-4bcc-afb0-bdc75c7d2272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:10.858992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226058a7-2de1-4bcc-afb0-bdc75c7d2272 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:10.862553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267c086-63eb-4e53-8fc6-56c56a8b78b7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:10.877034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910115df-bd35-4220-9342-169fdf072189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:10.878615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910115df-bd35-4220-9342-169fdf072189 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:10.878963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910115df-bd35-4220-9342-169fdf072189 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:10.879309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910115df-bd35-4220-9342-169fdf072189 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:10.880353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910115df-bd35-4220-9342-169fdf072189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:10.883440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910115df-bd35-4220-9342-169fdf072189 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:10.905794 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 911/1833] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:10 2022] POST /identity/v3/projects => generated 403 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:10.910045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226058a7-2de1-4bcc-afb0-bdc75c7d2272 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:10.913988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-226058a7-2de1-4bcc-afb0-bdc75c7d2272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:10.914205 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 923/1834] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:10 2022] POST /identity/v3/users => generated 303 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:10.917942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910115df-bd35-4220-9342-169fdf072189 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:10.931643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910115df-bd35-4220-9342-169fdf072189 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:10.960504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:11.007760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe186ed9-9dcd-4f99-8982-3da288b75e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:11.008430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe186ed9-9dcd-4f99-8982-3da288b75e02 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.008720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe186ed9-9dcd-4f99-8982-3da288b75e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.009049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe186ed9-9dcd-4f99-8982-3da288b75e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.009737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe186ed9-9dcd-4f99-8982-3da288b75e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:11.011978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe186ed9-9dcd-4f99-8982-3da288b75e02 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:11.026472 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 924/1835] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:11.035753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe186ed9-9dcd-4f99-8982-3da288b75e02 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:11.054948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226058a7-2de1-4bcc-afb0-bdc75c7d2272 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:11.055606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe186ed9-9dcd-4f99-8982-3da288b75e02 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:11.071817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a95fe-345e-42a4-b5a1-f523c8802e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:11.072504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a95fe-345e-42a4-b5a1-f523c8802e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.073240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a95fe-345e-42a4-b5a1-f523c8802e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.073240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a95fe-345e-42a4-b5a1-f523c8802e35 admin admin] PATH_INFO: `/v3/projects/6db583b83bcd4030bf0f9a70f6943210/users/5993923715d94398a246e19ecbf59106/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.080196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe878b43-3a1d-47cf-9e2a-bd50a7cb55fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:11.081925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe878b43-3a1d-47cf-9e2a-bd50a7cb55fa admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.081925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe878b43-3a1d-47cf-9e2a-bd50a7cb55fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.082954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe878b43-3a1d-47cf-9e2a-bd50a7cb55fa admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.084473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe878b43-3a1d-47cf-9e2a-bd50a7cb55fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:11.087525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe878b43-3a1d-47cf-9e2a-bd50a7cb55fa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:11.108588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a95fe-345e-42a4-b5a1-f523c8802e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db583b83bcd4030bf0f9a70f6943210, user_id=5993923715d94398a246e19ecbf59106, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:11.111702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a95fe-345e-42a4-b5a1-f523c8802e35 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:11.136535 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 912/1836] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:10 2022] POST /identity/v3/users => generated 308 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:49:11.140334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe878b43-3a1d-47cf-9e2a-bd50a7cb55fa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:11.150038 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 925/1837] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:11 2022] PUT /identity/v3/projects/6db583b83bcd4030bf0f9a70f6943210/users/5993923715d94398a246e19ecbf59106/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:11.155596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a95fe-345e-42a4-b5a1-f523c8802e35 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:11.163761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe878b43-3a1d-47cf-9e2a-bd50a7cb55fa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:11.250599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cf99b-5187-4c7e-8238-8b4172627a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:11.251620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cf99b-5187-4c7e-8238-8b4172627a38 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.252117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cf99b-5187-4c7e-8238-8b4172627a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.252884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cf99b-5187-4c7e-8238-8b4172627a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.253909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cf99b-5187-4c7e-8238-8b4172627a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:11.255743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cf99b-5187-4c7e-8238-8b4172627a38 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:11.274241 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 913/1838] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:11.278230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cf99b-5187-4c7e-8238-8b4172627a38 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:11.286556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a95fe-345e-42a4-b5a1-f523c8802e35 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:11.291799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91661574-ea24-4276-b14b-92143d899d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:11.292433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91661574-ea24-4276-b14b-92143d899d0b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.292871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91661574-ea24-4276-b14b-92143d899d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.293187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91661574-ea24-4276-b14b-92143d899d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.293841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91661574-ea24-4276-b14b-92143d899d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:11.296134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91661574-ea24-4276-b14b-92143d899d0b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:11.314577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cf99b-5187-4c7e-8238-8b4172627a38 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:11.322312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4714df-c089-494f-a2fa-188f1de32d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:11.323302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4714df-c089-494f-a2fa-188f1de32d52 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.323803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4714df-c089-494f-a2fa-188f1de32d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.324292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4714df-c089-494f-a2fa-188f1de32d52 admin admin] PATH_INFO: `/v3/projects/f58d0034099b4acd95ed5b1fee4fc9a5/users/928517e539f24ab2b0f1bd3033051a00/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.333609 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 926/1839] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:11.338101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91661574-ea24-4276-b14b-92143d899d0b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:11.359816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4714df-c089-494f-a2fa-188f1de32d52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58d0034099b4acd95ed5b1fee4fc9a5, user_id=928517e539f24ab2b0f1bd3033051a00, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:11.360688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91661574-ea24-4276-b14b-92143d899d0b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:11.364462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4714df-c089-494f-a2fa-188f1de32d52 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:11.374651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38948158-26a1-4e2c-bb89-57e6d9e26011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:11.377384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38948158-26a1-4e2c-bb89-57e6d9e26011 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.377718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38948158-26a1-4e2c-bb89-57e6d9e26011 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.378242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38948158-26a1-4e2c-bb89-57e6d9e26011 admin admin] PATH_INFO: `/v3/projects/6db583b83bcd4030bf0f9a70f6943210/users/5993923715d94398a246e19ecbf59106/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.384928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38948158-26a1-4e2c-bb89-57e6d9e26011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db583b83bcd4030bf0f9a70f6943210, user_id=5993923715d94398a246e19ecbf59106, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:11.386935 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 914/1840] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:11 2022] PUT /identity/v3/projects/f58d0034099b4acd95ed5b1fee4fc9a5/users/928517e539f24ab2b0f1bd3033051a00/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:11.390378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38948158-26a1-4e2c-bb89-57e6d9e26011 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:11.392436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4714df-c089-494f-a2fa-188f1de32d52 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:11.429429 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 927/1841] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:11 2022] PUT /identity/v3/projects/6db583b83bcd4030bf0f9a70f6943210/users/5993923715d94398a246e19ecbf59106/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:11.440460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408f8b-5b63-498f-b6a8-7f2cd7430b7b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.441374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408f8b-5b63-498f-b6a8-7f2cd7430b7b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.441993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408f8b-5b63-498f-b6a8-7f2cd7430b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.493479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0f408f8b-5b63-498f-b6a8-7f2cd7430b7b None None] MFA Rules not processed for user `5993923715d94398a246e19ecbf59106`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:11.495224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0f408f8b-5b63-498f-b6a8-7f2cd7430b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:11.519671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:11.543097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4714df-c089-494f-a2fa-188f1de32d52 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:11.588012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:11.619438 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 928/1842] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:11.620612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f408f8b-5b63-498f-b6a8-7f2cd7430b7b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:11.622066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0f408f8b-5b63-498f-b6a8-7f2cd7430b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:11.625662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c63977-0943-446a-8cd6-bf164bdfb134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:11.626360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63977-0943-446a-8cd6-bf164bdfb134 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.626716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63977-0943-446a-8cd6-bf164bdfb134 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.627130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63977-0943-446a-8cd6-bf164bdfb134 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.627793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c63977-0943-446a-8cd6-bf164bdfb134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:11.629536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c63977-0943-446a-8cd6-bf164bdfb134 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:11.641160 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 915/1843] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:11.642023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c63977-0943-446a-8cd6-bf164bdfb134 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:11.643604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f408f8b-5b63-498f-b6a8-7f2cd7430b7b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:11.650769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:11.651574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.651889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.652207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] PATH_INFO: `/v3/projects/739e0a081fd141eda9ac0290ac96d585` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.654695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=739e0a081fd141eda9ac0290ac96d585)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:11.656993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:11.660789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c63977-0943-446a-8cd6-bf164bdfb134 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:11.670002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1072d1-76c2-4cb7-ae0e-e3b2bed5c9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:11.670643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1072d1-76c2-4cb7-ae0e-e3b2bed5c9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.671247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1072d1-76c2-4cb7-ae0e-e3b2bed5c9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.671503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1072d1-76c2-4cb7-ae0e-e3b2bed5c9b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.672483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1072d1-76c2-4cb7-ae0e-e3b2bed5c9b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:11.674291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1072d1-76c2-4cb7-ae0e-e3b2bed5c9b3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:11.705864 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 916/1844] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:11 2022] POST /identity/v3/projects => generated 395 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:11.706664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1072d1-76c2-4cb7-ae0e-e3b2bed5c9b3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:11.735530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '739e0a081fd141eda9ac0290ac96d585', 'request_id': 'req-a8196ff2-60b2-49d0-a67b-a5842fdd9300'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:11.744066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '739e0a081fd141eda9ac0290ac96d585', 'request_id': 'req-a8196ff2-60b2-49d0-a67b-a5842fdd9300'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:11.751673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '739e0a081fd141eda9ac0290ac96d585', 'request_id': 'req-a8196ff2-60b2-49d0-a67b-a5842fdd9300'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:11.763931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] The token cache is being invalidate because project 739e0a081fd141eda9ac0290ac96d585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:11.764235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:11.764925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:11.768970 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 929/1845] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:11 2022] DELETE /identity/v3/projects/739e0a081fd141eda9ac0290ac96d585 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:11.770795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:11.772381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:11.785131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1072d1-76c2-4cb7-ae0e-e3b2bed5c9b3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:11.847974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86af25-53ea-48a5-9421-1b2f27056d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:11.848646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86af25-53ea-48a5-9421-1b2f27056d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.848960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86af25-53ea-48a5-9421-1b2f27056d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.849244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86af25-53ea-48a5-9421-1b2f27056d39 admin admin] PATH_INFO: `/v3/projects/f58d0034099b4acd95ed5b1fee4fc9a5/users/928517e539f24ab2b0f1bd3033051a00/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.851228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86af25-53ea-48a5-9421-1b2f27056d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58d0034099b4acd95ed5b1fee4fc9a5, user_id=928517e539f24ab2b0f1bd3033051a00, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:11.853950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86af25-53ea-48a5-9421-1b2f27056d39 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:11.860924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8196ff2-60b2-49d0-a67b-a5842fdd9300 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:11.865140 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 917/1846] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:11 2022] PUT /identity/v3/projects/f58d0034099b4acd95ed5b1fee4fc9a5/users/928517e539f24ab2b0f1bd3033051a00/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:11.866182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f062e1-53b6-4ac0-9c0a-fa1db0afb1e9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.866475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f062e1-53b6-4ac0-9c0a-fa1db0afb1e9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.866757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f062e1-53b6-4ac0-9c0a-fa1db0afb1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.899983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-22f062e1-53b6-4ac0-9c0a-fa1db0afb1e9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:11.901968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-22f062e1-53b6-4ac0-9c0a-fa1db0afb1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:11.940183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:11.964842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:11.979140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178dbebd-0c0f-4b69-b9e3-31682a4402e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:11.980121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dbebd-0c0f-4b69-b9e3-31682a4402e7 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:11.980619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dbebd-0c0f-4b69-b9e3-31682a4402e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:11.981268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dbebd-0c0f-4b69-b9e3-31682a4402e7 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:11.982365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dbebd-0c0f-4b69-b9e3-31682a4402e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:11.985337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dbebd-0c0f-4b69-b9e3-31682a4402e7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:11.992947 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 918/1847] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:12.001353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dc64a-1704-48fd-8d1b-f99f46e069d4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.001964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dc64a-1704-48fd-8d1b-f99f46e069d4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.002498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dc64a-1704-48fd-8d1b-f99f46e069d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.019880 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 930/1848] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:11 2022] POST /identity/v3/users => generated 298 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:12.021100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178dbebd-0c0f-4b69-b9e3-31682a4402e7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:12.034558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-861dc64a-1704-48fd-8d1b-f99f46e069d4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:12.036023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-861dc64a-1704-48fd-8d1b-f99f46e069d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:12.037409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178dbebd-0c0f-4b69-b9e3-31682a4402e7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:12.046867 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 919/1849] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:12.051913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07602f3f-4433-486b-86e0-9969467f7f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:12.052207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dc64a-1704-48fd-8d1b-f99f46e069d4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:12.052796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07602f3f-4433-486b-86e0-9969467f7f75 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.057655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07602f3f-4433-486b-86e0-9969467f7f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.057655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07602f3f-4433-486b-86e0-9969467f7f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.058064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07602f3f-4433-486b-86e0-9969467f7f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:12.059718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07602f3f-4433-486b-86e0-9969467f7f75 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:12.063880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dc64a-1704-48fd-8d1b-f99f46e069d4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:12.070489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f5bc4-539e-40dd-bf50-918054f445ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:12.071279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f5bc4-539e-40dd-bf50-918054f445ed admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.071568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f5bc4-539e-40dd-bf50-918054f445ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.071899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f5bc4-539e-40dd-bf50-918054f445ed admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.072571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f5bc4-539e-40dd-bf50-918054f445ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:12.072887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f5bc4-539e-40dd-bf50-918054f445ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:12.075235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f5bc4-539e-40dd-bf50-918054f445ed admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:12.087512 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 931/1850] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:12.089253 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 920/1851] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:12.094701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f5bc4-539e-40dd-bf50-918054f445ed admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:12.095732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07602f3f-4433-486b-86e0-9969467f7f75 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:12.107857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f5bc4-539e-40dd-bf50-918054f445ed admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:12.112555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07602f3f-4433-486b-86e0-9969467f7f75 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:12.114137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991ede7-b77f-4edb-b2b2-7817ad65df28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:12.114907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991ede7-b77f-4edb-b2b2-7817ad65df28 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.115714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991ede7-b77f-4edb-b2b2-7817ad65df28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.115860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991ede7-b77f-4edb-b2b2-7817ad65df28 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.116918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991ede7-b77f-4edb-b2b2-7817ad65df28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:12.119573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991ede7-b77f-4edb-b2b2-7817ad65df28 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:12.120307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980164b2-4560-48ac-bb96-47e1d49db282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:12.120951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980164b2-4560-48ac-bb96-47e1d49db282 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.121241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980164b2-4560-48ac-bb96-47e1d49db282 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.121516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980164b2-4560-48ac-bb96-47e1d49db282 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.122603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980164b2-4560-48ac-bb96-47e1d49db282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:12.124383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980164b2-4560-48ac-bb96-47e1d49db282 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:12.152953 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 932/1852] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:49:12.155663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980164b2-4560-48ac-bb96-47e1d49db282 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:12.165943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980164b2-4560-48ac-bb96-47e1d49db282 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:12.174680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25243a51-28ea-4412-b976-f87008131f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:12.175596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25243a51-28ea-4412-b976-f87008131f32 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.175917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25243a51-28ea-4412-b976-f87008131f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.176177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25243a51-28ea-4412-b976-f87008131f32 admin admin] PATH_INFO: `/v3/projects/4b46258375af4678b58c037fb587d677/users/e939063cdd1a4e8ea576fde05f1b6d60/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.181322 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 921/1853] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:12 2022] POST /identity/v3/projects => generated 409 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:49:12.186698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991ede7-b77f-4edb-b2b2-7817ad65df28 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:12.202047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25243a51-28ea-4412-b976-f87008131f32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b46258375af4678b58c037fb587d677, user_id=e939063cdd1a4e8ea576fde05f1b6d60, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:12.205417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25243a51-28ea-4412-b976-f87008131f32 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:12.227045 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 933/1854] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:12 2022] PUT /identity/v3/projects/4b46258375af4678b58c037fb587d677/users/e939063cdd1a4e8ea576fde05f1b6d60/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:12.238273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e165b7-256a-435c-bdd7-7fc68ee6fcec None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.238562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e165b7-256a-435c-bdd7-7fc68ee6fcec None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.238845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e165b7-256a-435c-bdd7-7fc68ee6fcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.270487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-73e165b7-256a-435c-bdd7-7fc68ee6fcec None None] MFA Rules not processed for user `e939063cdd1a4e8ea576fde05f1b6d60`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:12.270669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991ede7-b77f-4edb-b2b2-7817ad65df28 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:12.271729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-73e165b7-256a-435c-bdd7-7fc68ee6fcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:12.405815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5aaca-7ba6-46c5-9ef0-78f0cbfdadcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:12.406411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5aaca-7ba6-46c5-9ef0-78f0cbfdadcf admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.406724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5aaca-7ba6-46c5-9ef0-78f0cbfdadcf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.408135 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 934/1855] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:12 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:12.408247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5aaca-7ba6-46c5-9ef0-78f0cbfdadcf admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.408680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5aaca-7ba6-46c5-9ef0-78f0cbfdadcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:12.408784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e165b7-256a-435c-bdd7-7fc68ee6fcec None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:12.410284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-73e165b7-256a-435c-bdd7-7fc68ee6fcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:12.411301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5aaca-7ba6-46c5-9ef0-78f0cbfdadcf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:12.458059 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 922/1856] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:12 2022] POST /identity/v3/users => generated 305 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:12.460139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5aaca-7ba6-46c5-9ef0-78f0cbfdadcf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:12.486827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5aaca-7ba6-46c5-9ef0-78f0cbfdadcf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:12.492114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca678dd3-5b76-457a-a7d4-9b5b71497975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:12.492761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca678dd3-5b76-457a-a7d4-9b5b71497975 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.493082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca678dd3-5b76-457a-a7d4-9b5b71497975 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.493366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca678dd3-5b76-457a-a7d4-9b5b71497975 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.499755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca678dd3-5b76-457a-a7d4-9b5b71497975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:12.501521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca678dd3-5b76-457a-a7d4-9b5b71497975 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:12.512105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e165b7-256a-435c-bdd7-7fc68ee6fcec None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:12.519982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c7028-c37e-4a56-bc6f-fa1b84d42246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:12.521072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c7028-c37e-4a56-bc6f-fa1b84d42246 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.521591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c7028-c37e-4a56-bc6f-fa1b84d42246 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.522138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c7028-c37e-4a56-bc6f-fa1b84d42246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.526534 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 923/1857] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:12.528388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca678dd3-5b76-457a-a7d4-9b5b71497975 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:12.529005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8c6c7028-c37e-4a56-bc6f-fa1b84d42246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:12.546619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca678dd3-5b76-457a-a7d4-9b5b71497975 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:12.547339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:12.565338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b40244-d33b-4b75-a0a0-09ee4b050bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:12.565975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b40244-d33b-4b75-a0a0-09ee4b050bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.566268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b40244-d33b-4b75-a0a0-09ee4b050bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.566606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b40244-d33b-4b75-a0a0-09ee4b050bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.567271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b40244-d33b-4b75-a0a0-09ee4b050bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:12.568969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b40244-d33b-4b75-a0a0-09ee4b050bd8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:12.587225 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 924/1858] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:12.588350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b40244-d33b-4b75-a0a0-09ee4b050bd8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:12.601452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b40244-d33b-4b75-a0a0-09ee4b050bd8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:12.609845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7b499-3d92-4759-add4-4eb6943b6b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:12.609845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7b499-3d92-4759-add4-4eb6943b6b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.609845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7b499-3d92-4759-add4-4eb6943b6b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.610415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7b499-3d92-4759-add4-4eb6943b6b12 admin admin] PATH_INFO: `/v3/projects/3f1a033dec094ceb9050178fd2a086ba/users/ef9328825c7c4f26baeb1e0e05f9451d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.626728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c7028-c37e-4a56-bc6f-fa1b84d42246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:12.628660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c7028-c37e-4a56-bc6f-fa1b84d42246 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:12.640854 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 935/1859] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:12 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:12.641761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7b499-3d92-4759-add4-4eb6943b6b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1a033dec094ceb9050178fd2a086ba, user_id=ef9328825c7c4f26baeb1e0e05f9451d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:12.644980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7b499-3d92-4759-add4-4eb6943b6b12 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:12.652088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:12.676611 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 925/1860] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:12 2022] PUT /identity/v3/projects/3f1a033dec094ceb9050178fd2a086ba/users/ef9328825c7c4f26baeb1e0e05f9451d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:12.688402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84025f-b8dd-4147-91c3-4212d5eacb90 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.688697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84025f-b8dd-4147-91c3-4212d5eacb90 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.690038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84025f-b8dd-4147-91c3-4212d5eacb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.737866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0ad76-7e78-49f2-bad4-58013a5b27ee None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.738157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0ad76-7e78-49f2-bad4-58013a5b27ee None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.738443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0ad76-7e78-49f2-bad4-58013a5b27ee None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.739843 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 926/1861] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:49:12.740789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4e84025f-b8dd-4147-91c3-4212d5eacb90 None None] MFA Rules not processed for user `ef9328825c7c4f26baeb1e0e05f9451d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:12.742368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4e84025f-b8dd-4147-91c3-4212d5eacb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:12.746424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0ad76-7e78-49f2-bad4-58013a5b27ee None None] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:12.834991 ncdv-master-1 devstack@keystone.service[99211]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-56a0ad76-7e78-49f2-bad4-58013a5b27ee None None] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Sep 03 20:49:12.835222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0ad76-7e78-49f2-bad4-58013a5b27ee None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:12.855551 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 936/1862] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:12 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:12.860313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84025f-b8dd-4147-91c3-4212d5eacb90 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:12.951043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0ad76-7e78-49f2-bad4-58013a5b27ee None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:12.956665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dfacf-6854-42e8-a85b-1f2cb0eaaf79 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5993923715d94398a246e19ecbf59106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6db583b83bcd4030bf0f9a70f6943210', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:12.957649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dfacf-6854-42e8-a85b-1f2cb0eaaf79 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.958237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dfacf-6854-42e8-a85b-1f2cb0eaaf79 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.958599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dfacf-6854-42e8-a85b-1f2cb0eaaf79 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] PATH_INFO: `/v3/projects/2a28735e9d204354a30066460fd3d271` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.959846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dfacf-6854-42e8-a85b-1f2cb0eaaf79 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: Authorizing `identity:get_project(project_id=2a28735e9d204354a30066460fd3d271)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:12.962130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dfacf-6854-42e8-a85b-1f2cb0eaaf79 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:12.964259 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 927/1863] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:12 2022] GET /identity/v3/projects/2a28735e9d204354a30066460fd3d271 => generated 393 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:12.986535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84025f-b8dd-4147-91c3-4212d5eacb90 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:12.992655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a90b8b-ef30-4c2f-b05d-d98a4627c52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:12.993391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a90b8b-ef30-4c2f-b05d-d98a4627c52d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:12.993928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a90b8b-ef30-4c2f-b05d-d98a4627c52d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:12.994217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a90b8b-ef30-4c2f-b05d-d98a4627c52d admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:12.994991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a90b8b-ef30-4c2f-b05d-d98a4627c52d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:12.997061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a90b8b-ef30-4c2f-b05d-d98a4627c52d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:13.027249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36753b4-77eb-4fa9-bc5b-06ba86986a6d None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:13.027540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36753b4-77eb-4fa9-bc5b-06ba86986a6d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:13.027818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36753b4-77eb-4fa9-bc5b-06ba86986a6d None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:13.030034 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 928/1864] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:13.037031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36753b4-77eb-4fa9-bc5b-06ba86986a6d None None] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:13.039967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a36753b4-77eb-4fa9-bc5b-06ba86986a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:13.049129 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 937/1865] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:12 2022] POST /identity/v3/projects => generated 411 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:49:13.051093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a90b8b-ef30-4c2f-b05d-d98a4627c52d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:13.086520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:13.143776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a90b8b-ef30-4c2f-b05d-d98a4627c52d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:13.153437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7a5cd-dedc-4283-9618-edaa817b307d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:13.154741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7a5cd-dedc-4283-9618-edaa817b307d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:13.155032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7a5cd-dedc-4283-9618-edaa817b307d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:13.155315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7a5cd-dedc-4283-9618-edaa817b307d admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:13.156347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7a5cd-dedc-4283-9618-edaa817b307d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:13.160313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7a5cd-dedc-4283-9618-edaa817b307d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:13.190733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36753b4-77eb-4fa9-bc5b-06ba86986a6d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:13.195033 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 938/1866] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:13 2022] POST /identity/v3/users => generated 312 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:13.203390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7a5cd-dedc-4283-9618-edaa817b307d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:13.221831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7a5cd-dedc-4283-9618-edaa817b307d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:13.227688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7256406-8560-42b4-b672-b0d5bc233e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:13.228246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7256406-8560-42b4-b672-b0d5bc233e25 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:13.228535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7256406-8560-42b4-b672-b0d5bc233e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:13.228818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7256406-8560-42b4-b672-b0d5bc233e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:13.229511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7256406-8560-42b4-b672-b0d5bc233e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:13.231191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7256406-8560-42b4-b672-b0d5bc233e25 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:13.247205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:13.251063 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 939/1867] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:13.252467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7256406-8560-42b4-b672-b0d5bc233e25 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:13.265836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7256406-8560-42b4-b672-b0d5bc233e25 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:13.273623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ca36b-2f9e-4be9-bc39-ea41f0b6dfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:13.274850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ca36b-2f9e-4be9-bc39-ea41f0b6dfb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:13.274850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ca36b-2f9e-4be9-bc39-ea41f0b6dfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:13.274850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ca36b-2f9e-4be9-bc39-ea41f0b6dfb1 admin admin] PATH_INFO: `/v3/projects/2989465b737c4318aaab7f0effc2162d/users/42220281055e46809b88a7511275f0a2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:13.296256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ca36b-2f9e-4be9-bc39-ea41f0b6dfb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2989465b737c4318aaab7f0effc2162d, user_id=42220281055e46809b88a7511275f0a2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:13.302219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ca36b-2f9e-4be9-bc39-ea41f0b6dfb1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:13.310967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36753b4-77eb-4fa9-bc5b-06ba86986a6d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:13.321221 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 940/1868] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:13 2022] PUT /identity/v3/projects/2989465b737c4318aaab7f0effc2162d/users/42220281055e46809b88a7511275f0a2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:13.324711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ca36b-2f9e-4be9-bc39-ea41f0b6dfb1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:13.387379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b4d3b-9aea-474a-ac8a-7f7b1143e531 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5993923715d94398a246e19ecbf59106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6db583b83bcd4030bf0f9a70f6943210', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:13.388887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b4d3b-9aea-474a-ac8a-7f7b1143e531 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:13.389466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b4d3b-9aea-474a-ac8a-7f7b1143e531 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:13.389947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b4d3b-9aea-474a-ac8a-7f7b1143e531 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] PATH_INFO: `/v3/projects/2a28735e9d204354a30066460fd3d271` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:13.391698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b4d3b-9aea-474a-ac8a-7f7b1143e531 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: Authorizing `identity:get_project(project_id=2a28735e9d204354a30066460fd3d271)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:13.395836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b4d3b-9aea-474a-ac8a-7f7b1143e531 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:13.398130 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 929/1869] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:13 2022] GET /identity/v3/projects/2a28735e9d204354a30066460fd3d271 => generated 393 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:49:13.422686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ca36b-2f9e-4be9-bc39-ea41f0b6dfb1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:13.428697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67612f1d-b311-462a-b6b1-b076a5f14267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:13.429353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67612f1d-b311-462a-b6b1-b076a5f14267 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:13.429811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67612f1d-b311-462a-b6b1-b076a5f14267 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:13.430260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67612f1d-b311-462a-b6b1-b076a5f14267 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:13.431509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67612f1d-b311-462a-b6b1-b076a5f14267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:13.433704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67612f1d-b311-462a-b6b1-b076a5f14267 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:13.448105 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 941/1870] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:49:13.452488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b4d3b-9aea-474a-ac8a-7f7b1143e531 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:13.460241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bce806-f915-478b-aced-57d102cdf4b7 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:13.460820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bce806-f915-478b-aced-57d102cdf4b7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:13.461158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bce806-f915-478b-aced-57d102cdf4b7 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:13.462836 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 942/1871] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:13.464466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b4d3b-9aea-474a-ac8a-7f7b1143e531 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:13.469448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bce806-f915-478b-aced-57d102cdf4b7 None None] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:13.472469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4975b18-242e-48a5-b382-1c14d27c0c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:13.473488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4975b18-242e-48a5-b382-1c14d27c0c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:13.473850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4975b18-242e-48a5-b382-1c14d27c0c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:13.474970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4975b18-242e-48a5-b382-1c14d27c0c7a admin admin] PATH_INFO: `/v3/projects/2989465b737c4318aaab7f0effc2162d/users/42220281055e46809b88a7511275f0a2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:13.478430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4975b18-242e-48a5-b382-1c14d27c0c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2989465b737c4318aaab7f0effc2162d, user_id=42220281055e46809b88a7511275f0a2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:13.483741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4975b18-242e-48a5-b382-1c14d27c0c7a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:13.499170 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 930/1872] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:13 2022] PUT /identity/v3/projects/2989465b737c4318aaab7f0effc2162d/users/42220281055e46809b88a7511275f0a2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:13.510585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c64125-5128-4cc8-9bc6-43c0a1e828b4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:13.510877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c64125-5128-4cc8-9bc6-43c0a1e828b4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:13.511162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c64125-5128-4cc8-9bc6-43c0a1e828b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:13.562890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-76c64125-5128-4cc8-9bc6-43c0a1e828b4 None None] MFA Rules not processed for user `42220281055e46809b88a7511275f0a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:13.564376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-76c64125-5128-4cc8-9bc6-43c0a1e828b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:13.568195 ncdv-master-1 devstack@keystone.service[99210]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-70bce806-f915-478b-aced-57d102cdf4b7 None None] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Sep 03 20:49:13.568672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bce806-f915-478b-aced-57d102cdf4b7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:13.584248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bce806-f915-478b-aced-57d102cdf4b7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:13.596676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:13.674348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e43c57-e549-487c-8da6-1cc8cfb45040 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5993923715d94398a246e19ecbf59106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6db583b83bcd4030bf0f9a70f6943210', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:13.680563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e43c57-e549-487c-8da6-1cc8cfb45040 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:13.681045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e43c57-e549-487c-8da6-1cc8cfb45040 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:13.681601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e43c57-e549-487c-8da6-1cc8cfb45040 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] PATH_INFO: `/v3/projects/2a28735e9d204354a30066460fd3d271` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:13.683257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e43c57-e549-487c-8da6-1cc8cfb45040 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: Authorizing `identity:get_project(project_id=2a28735e9d204354a30066460fd3d271)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:13.686352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e43c57-e549-487c-8da6-1cc8cfb45040 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:13.689060 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 943/1873] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:13 2022] GET /identity/v3/projects/2a28735e9d204354a30066460fd3d271 => generated 393 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:49:13.693580 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 931/1874] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:13.732383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e43c57-e549-487c-8da6-1cc8cfb45040 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:13.847813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e43c57-e549-487c-8da6-1cc8cfb45040 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:13.856617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230c73e-9579-4a70-9ba1-fa69d7851834 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:13.857515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230c73e-9579-4a70-9ba1-fa69d7851834 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:13.857971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230c73e-9579-4a70-9ba1-fa69d7851834 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:13.858412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230c73e-9579-4a70-9ba1-fa69d7851834 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:13.870394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230c73e-9579-4a70-9ba1-fa69d7851834 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:13.875034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230c73e-9579-4a70-9ba1-fa69d7851834 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:13.895580 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 944/1875] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:13 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:13.965102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c64125-5128-4cc8-9bc6-43c0a1e828b4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:13.982598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c64125-5128-4cc8-9bc6-43c0a1e828b4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:13.990980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f12ee-a37b-4f86-802b-2bdbf38813cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:13.992559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f12ee-a37b-4f86-802b-2bdbf38813cc service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:13.992949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f12ee-a37b-4f86-802b-2bdbf38813cc service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:13.993459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f12ee-a37b-4f86-802b-2bdbf38813cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:14.068638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:14.091673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f12ee-a37b-4f86-802b-2bdbf38813cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:14.093424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f12ee-a37b-4f86-802b-2bdbf38813cc service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:14.106072 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 932/1876] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:13 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:14.191510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230c73e-9579-4a70-9ba1-fa69d7851834 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:14.288427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230c73e-9579-4a70-9ba1-fa69d7851834 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:14.297980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28780070-3218-48d6-8023-a4768626008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:14.298704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28780070-3218-48d6-8023-a4768626008c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:14.298993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28780070-3218-48d6-8023-a4768626008c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:14.299274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28780070-3218-48d6-8023-a4768626008c admin admin] PATH_INFO: `/v3/users/ef9328825c7c4f26baeb1e0e05f9451d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:14.300497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28780070-3218-48d6-8023-a4768626008c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef9328825c7c4f26baeb1e0e05f9451d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:14.303695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28780070-3218-48d6-8023-a4768626008c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:14.373720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:14.385437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28780070-3218-48d6-8023-a4768626008c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef9328825c7c4f26baeb1e0e05f9451d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:14.409947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28780070-3218-48d6-8023-a4768626008c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef9328825c7c4f26baeb1e0e05f9451d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:14.424321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28780070-3218-48d6-8023-a4768626008c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef9328825c7c4f26baeb1e0e05f9451d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:14.425454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28780070-3218-48d6-8023-a4768626008c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef9328825c7c4f26baeb1e0e05f9451d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:14.426420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28780070-3218-48d6-8023-a4768626008c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef9328825c7c4f26baeb1e0e05f9451d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:14.441702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f12ee-a37b-4f86-802b-2bdbf38813cc service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:14.455764 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 945/1877] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:14 2022] DELETE /identity/v3/users/ef9328825c7c4f26baeb1e0e05f9451d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:14.457666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628baf-4f9c-4879-8f73-b77f7f996d23 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:14.458057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628baf-4f9c-4879-8f73-b77f7f996d23 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:14.458416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628baf-4f9c-4879-8f73-b77f7f996d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:14.460450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f12ee-a37b-4f86-802b-2bdbf38813cc service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:14.483750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0a628baf-4f9c-4879-8f73-b77f7f996d23 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:14.485311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0a628baf-4f9c-4879-8f73-b77f7f996d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:14.590013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b4376-9bbc-4f6b-8af0-7ad4e9db0242 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:14.591118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b4376-9bbc-4f6b-8af0-7ad4e9db0242 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:14.592026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b4376-9bbc-4f6b-8af0-7ad4e9db0242 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:14.592872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b4376-9bbc-4f6b-8af0-7ad4e9db0242 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:14.596699 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 946/1878] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:14.598268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-444b4376-9bbc-4f6b-8af0-7ad4e9db0242 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:14.603496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628baf-4f9c-4879-8f73-b77f7f996d23 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:14.614430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0a628baf-4f9c-4879-8f73-b77f7f996d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:14.749541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628baf-4f9c-4879-8f73-b77f7f996d23 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:14.753697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019fbf8-bf49-44b3-b409-b381912e5f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:14.754340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019fbf8-bf49-44b3-b409-b381912e5f40 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:14.754623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019fbf8-bf49-44b3-b409-b381912e5f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:14.754919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019fbf8-bf49-44b3-b409-b381912e5f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:14.765214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019fbf8-bf49-44b3-b409-b381912e5f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:14.773703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019fbf8-bf49-44b3-b409-b381912e5f40 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:14.775697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b4376-9bbc-4f6b-8af0-7ad4e9db0242 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:14.777368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b4376-9bbc-4f6b-8af0-7ad4e9db0242 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:14.791227 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 947/1879] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:49:14.800644 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 933/1880] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:49:15.041538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686479e-b0ad-48cc-81cc-c023a525efee None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:15.041832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686479e-b0ad-48cc-81cc-c023a525efee None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:15.042137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686479e-b0ad-48cc-81cc-c023a525efee None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:15.045184 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 948/1881] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:15.050721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b4376-9bbc-4f6b-8af0-7ad4e9db0242 service nova] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:15.052258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-444b4376-9bbc-4f6b-8af0-7ad4e9db0242 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:15.080033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b4376-9bbc-4f6b-8af0-7ad4e9db0242 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:15.082371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-444b4376-9bbc-4f6b-8af0-7ad4e9db0242 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:15.186816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b4376-9bbc-4f6b-8af0-7ad4e9db0242 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:15.194891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf90456-c9c0-4838-b6c5-968a3f8ae1f1 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5993923715d94398a246e19ecbf59106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6db583b83bcd4030bf0f9a70f6943210', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:15.195857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf90456-c9c0-4838-b6c5-968a3f8ae1f1 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:15.195857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf90456-c9c0-4838-b6c5-968a3f8ae1f1 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:15.196196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf90456-c9c0-4838-b6c5-968a3f8ae1f1 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] PATH_INFO: `/v3/projects/2a28735e9d204354a30066460fd3d271` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:15.197649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf90456-c9c0-4838-b6c5-968a3f8ae1f1 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: Authorizing `identity:get_project(project_id=2a28735e9d204354a30066460fd3d271)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:15.199893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf90456-c9c0-4838-b6c5-968a3f8ae1f1 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:15.202985 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 934/1882] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:15 2022] GET /identity/v3/projects/2a28735e9d204354a30066460fd3d271 => generated 393 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:49:15.335702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686479e-b0ad-48cc-81cc-c023a525efee None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:15.338703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2686479e-b0ad-48cc-81cc-c023a525efee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:15.369374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1a6d-cc84-4610-ba71-6d577648f157 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:15.370124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1a6d-cc84-4610-ba71-6d577648f157 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:15.371297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1a6d-cc84-4610-ba71-6d577648f157 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:15.373622 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 935/1883] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:15.374369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686479e-b0ad-48cc-81cc-c023a525efee None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:15.380134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1a6d-cc84-4610-ba71-6d577648f157 None None] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:15.382620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d3ab1a6d-cc84-4610-ba71-6d577648f157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:15.407980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9090c0-682c-4d19-9bf0-ba713963589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:15.410188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9090c0-682c-4d19-9bf0-ba713963589a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:15.410743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9090c0-682c-4d19-9bf0-ba713963589a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:15.411257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9090c0-682c-4d19-9bf0-ba713963589a admin admin] PATH_INFO: `/v3/projects/3f1a033dec094ceb9050178fd2a086ba` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:15.414066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9090c0-682c-4d19-9bf0-ba713963589a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1a033dec094ceb9050178fd2a086ba)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:15.416394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9090c0-682c-4d19-9bf0-ba713963589a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:15.462995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1a6d-cc84-4610-ba71-6d577648f157 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:15.486320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1a6d-cc84-4610-ba71-6d577648f157 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:15.507236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1012d-18de-48dc-a7b1-e234336c8e42 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5993923715d94398a246e19ecbf59106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6db583b83bcd4030bf0f9a70f6943210', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:15.507949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1012d-18de-48dc-a7b1-e234336c8e42 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:15.508242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1012d-18de-48dc-a7b1-e234336c8e42 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:15.508526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1012d-18de-48dc-a7b1-e234336c8e42 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] PATH_INFO: `/v3/projects/2a28735e9d204354a30066460fd3d271` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:15.509923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1012d-18de-48dc-a7b1-e234336c8e42 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: Authorizing `identity:get_project(project_id=2a28735e9d204354a30066460fd3d271)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:15.512048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1012d-18de-48dc-a7b1-e234336c8e42 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:15.521391 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 936/1884] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:15 2022] GET /identity/v3/projects/2a28735e9d204354a30066460fd3d271 => generated 393 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:49:15.579066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-be9090c0-682c-4d19-9bf0-ba713963589a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1a033dec094ceb9050178fd2a086ba', 'request_id': 'req-be9090c0-682c-4d19-9bf0-ba713963589a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:15.589286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:15.626456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-be9090c0-682c-4d19-9bf0-ba713963589a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1a033dec094ceb9050178fd2a086ba', 'request_id': 'req-be9090c0-682c-4d19-9bf0-ba713963589a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:15.644831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-be9090c0-682c-4d19-9bf0-ba713963589a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1a033dec094ceb9050178fd2a086ba', 'request_id': 'req-be9090c0-682c-4d19-9bf0-ba713963589a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:15.671652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-be9090c0-682c-4d19-9bf0-ba713963589a admin admin] The token cache is being invalidate because project 3f1a033dec094ceb9050178fd2a086ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:15.672037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-be9090c0-682c-4d19-9bf0-ba713963589a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:15.673888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-be9090c0-682c-4d19-9bf0-ba713963589a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:15.678927 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 949/1885] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:15 2022] DELETE /identity/v3/projects/3f1a033dec094ceb9050178fd2a086ba => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:15.693134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1012d-18de-48dc-a7b1-e234336c8e42 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:15.699225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7aa1012d-18de-48dc-a7b1-e234336c8e42 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:15.735566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1012d-18de-48dc-a7b1-e234336c8e42 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:15.749310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:15.777805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933de169-a456-4fa1-92b0-3b5910283cda None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:15.778116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933de169-a456-4fa1-92b0-3b5910283cda None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:15.778393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933de169-a456-4fa1-92b0-3b5910283cda None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:15.781771 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 950/1886] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:49:15.791765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933de169-a456-4fa1-92b0-3b5910283cda None None] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:15.794165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-933de169-a456-4fa1-92b0-3b5910283cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:15.866954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:15.867766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:15.868559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:15.868559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] PATH_INFO: `/v3/users/42220281055e46809b88a7511275f0a2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:15.879176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42220281055e46809b88a7511275f0a2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:15.881632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:15.990021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933de169-a456-4fa1-92b0-3b5910283cda None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:15.993319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-933de169-a456-4fa1-92b0-3b5910283cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:16.006791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42220281055e46809b88a7511275f0a2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:16.031852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42220281055e46809b88a7511275f0a2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:16.032714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42220281055e46809b88a7511275f0a2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:16.042695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42220281055e46809b88a7511275f0a2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:16.059689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42220281055e46809b88a7511275f0a2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:16.077891 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 937/1887] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:15 2022] DELETE /identity/v3/users/42220281055e46809b88a7511275f0a2 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:16.148918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933de169-a456-4fa1-92b0-3b5910283cda None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:16.227886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bc1fe-f4b2-48a9-b6a9-6808e33db707 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5993923715d94398a246e19ecbf59106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6db583b83bcd4030bf0f9a70f6943210', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:16.228558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bc1fe-f4b2-48a9-b6a9-6808e33db707 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:16.228860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bc1fe-f4b2-48a9-b6a9-6808e33db707 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:16.229173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bc1fe-f4b2-48a9-b6a9-6808e33db707 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] PATH_INFO: `/v3/projects/4b46258375af4678b58c037fb587d677` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:16.230550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bc1fe-f4b2-48a9-b6a9-6808e33db707 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: Authorizing `identity:get_project(project_id=4b46258375af4678b58c037fb587d677)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:16.233375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bc1fe-f4b2-48a9-b6a9-6808e33db707 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:16.250171 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 951/1888] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:15 2022] GET /identity/v3/projects/4b46258375af4678b58c037fb587d677 => generated 395 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:49:16.557614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:16.557614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:16.749515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1dcb2b-8026-460b-8ac7-85343ef12881 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:16.760472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f351845-1408-49ad-b5d0-5d6d670c03ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:16.761216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f351845-1408-49ad-b5d0-5d6d670c03ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:16.761507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f351845-1408-49ad-b5d0-5d6d670c03ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:16.761848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f351845-1408-49ad-b5d0-5d6d670c03ff admin admin] PATH_INFO: `/v3/users/fc148ca2d15445f7860e84a40d1cdde7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:16.764728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f351845-1408-49ad-b5d0-5d6d670c03ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc148ca2d15445f7860e84a40d1cdde7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:16.766644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f351845-1408-49ad-b5d0-5d6d670c03ff admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:16.859615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:16.894104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4f351845-1408-49ad-b5d0-5d6d670c03ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc148ca2d15445f7860e84a40d1cdde7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:16.931789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4f351845-1408-49ad-b5d0-5d6d670c03ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc148ca2d15445f7860e84a40d1cdde7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:16.932553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4f351845-1408-49ad-b5d0-5d6d670c03ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc148ca2d15445f7860e84a40d1cdde7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:16.943029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bc1fe-f4b2-48a9-b6a9-6808e33db707 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:16.946414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4d3bc1fe-f4b2-48a9-b6a9-6808e33db707 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:16.958029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4f351845-1408-49ad-b5d0-5d6d670c03ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc148ca2d15445f7860e84a40d1cdde7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:16.981745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bc1fe-f4b2-48a9-b6a9-6808e33db707 tempest-FlavorsAccessNegativeTestJSON-746619449 tempest-FlavorsAccessNegativeTestJSON-746619449-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:16.982121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4f351845-1408-49ad-b5d0-5d6d670c03ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc148ca2d15445f7860e84a40d1cdde7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:16.997158 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 938/1889] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:16 2022] DELETE /identity/v3/users/fc148ca2d15445f7860e84a40d1cdde7 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:17.001796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658df4a5-1b67-42ca-b086-80faca6ad94a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:17.002019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658df4a5-1b67-42ca-b086-80faca6ad94a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:17.002315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658df4a5-1b67-42ca-b086-80faca6ad94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:17.009174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada8632-1c9e-41a3-ab52-9d59b30f995e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:17.011466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada8632-1c9e-41a3-ab52-9d59b30f995e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:17.011768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada8632-1c9e-41a3-ab52-9d59b30f995e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:17.012111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada8632-1c9e-41a3-ab52-9d59b30f995e admin admin] PATH_INFO: `/v3/projects/2989465b737c4318aaab7f0effc2162d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:17.029014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada8632-1c9e-41a3-ab52-9d59b30f995e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2989465b737c4318aaab7f0effc2162d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:17.031501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada8632-1c9e-41a3-ab52-9d59b30f995e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:17.050206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-658df4a5-1b67-42ca-b086-80faca6ad94a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:17.053029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-658df4a5-1b67-42ca-b086-80faca6ad94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:17.065957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:17.118572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fada8632-1c9e-41a3-ab52-9d59b30f995e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2989465b737c4318aaab7f0effc2162d', 'request_id': 'req-fada8632-1c9e-41a3-ab52-9d59b30f995e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:17.134182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fada8632-1c9e-41a3-ab52-9d59b30f995e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2989465b737c4318aaab7f0effc2162d', 'request_id': 'req-fada8632-1c9e-41a3-ab52-9d59b30f995e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:17.154055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fada8632-1c9e-41a3-ab52-9d59b30f995e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2989465b737c4318aaab7f0effc2162d', 'request_id': 'req-fada8632-1c9e-41a3-ab52-9d59b30f995e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:17.177976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fada8632-1c9e-41a3-ab52-9d59b30f995e admin admin] The token cache is being invalidate because project 2989465b737c4318aaab7f0effc2162d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:17.178284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fada8632-1c9e-41a3-ab52-9d59b30f995e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:17.180022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fada8632-1c9e-41a3-ab52-9d59b30f995e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:17.189247 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 952/1890] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:16 2022] DELETE /identity/v3/projects/2989465b737c4318aaab7f0effc2162d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:17.189432 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 939/1891] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:17.191023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736e9cf-23fd-4476-b549-b0a93aba4847 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:17.191324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736e9cf-23fd-4476-b549-b0a93aba4847 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:17.191691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736e9cf-23fd-4476-b549-b0a93aba4847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:17.198205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658df4a5-1b67-42ca-b086-80faca6ad94a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:17.199800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-658df4a5-1b67-42ca-b086-80faca6ad94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:17.239578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9736e9cf-23fd-4476-b549-b0a93aba4847 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:17.240861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9736e9cf-23fd-4476-b549-b0a93aba4847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:17.319584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658df4a5-1b67-42ca-b086-80faca6ad94a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:17.387500 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 953/1892] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:17.390233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077124a9-cd29-417b-8228-3f0a52dfb9f3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:17.390541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077124a9-cd29-417b-8228-3f0a52dfb9f3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:17.390872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077124a9-cd29-417b-8228-3f0a52dfb9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:17.432811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-077124a9-cd29-417b-8228-3f0a52dfb9f3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:17.434969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-077124a9-cd29-417b-8228-3f0a52dfb9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:17.436548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c1870-00e8-4252-875d-1cd2457a23de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:17.437242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c1870-00e8-4252-875d-1cd2457a23de service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:17.437529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c1870-00e8-4252-875d-1cd2457a23de service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:17.437831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c1870-00e8-4252-875d-1cd2457a23de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:17.440848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-038c1870-00e8-4252-875d-1cd2457a23de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:17.451850 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 954/1893] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:17.458567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077124a9-cd29-417b-8228-3f0a52dfb9f3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:17.474453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c1870-00e8-4252-875d-1cd2457a23de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:17.479130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c1870-00e8-4252-875d-1cd2457a23de service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:17.482223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077124a9-cd29-417b-8228-3f0a52dfb9f3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:17.507021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138c21e-20f7-4cdd-8825-e44ce58559cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:17.509839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138c21e-20f7-4cdd-8825-e44ce58559cb admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:17.509839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138c21e-20f7-4cdd-8825-e44ce58559cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:17.509839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138c21e-20f7-4cdd-8825-e44ce58559cb admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:17.515391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138c21e-20f7-4cdd-8825-e44ce58559cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:17.515391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138c21e-20f7-4cdd-8825-e44ce58559cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:17.523361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138c21e-20f7-4cdd-8825-e44ce58559cb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:17.524706 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 940/1894] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:17.566203 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 955/1895] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:17.574289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c1870-00e8-4252-875d-1cd2457a23de service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:17.595628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c1870-00e8-4252-875d-1cd2457a23de service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:17.605524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fee8d5-70a0-4ca1-b90b-4fa413fb45b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:17.606707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fee8d5-70a0-4ca1-b90b-4fa413fb45b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:17.607191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fee8d5-70a0-4ca1-b90b-4fa413fb45b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:17.607660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fee8d5-70a0-4ca1-b90b-4fa413fb45b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:17.608602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fee8d5-70a0-4ca1-b90b-4fa413fb45b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:17.610901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fee8d5-70a0-4ca1-b90b-4fa413fb45b0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:17.629875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:17.673995 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 941/1896] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:17 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:17.678157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138c21e-20f7-4cdd-8825-e44ce58559cb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:17.691662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138c21e-20f7-4cdd-8825-e44ce58559cb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:17.827506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc1fea-fb81-4764-a7de-a75069c0cfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:17.831160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc1fea-fb81-4764-a7de-a75069c0cfa0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:17.833675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc1fea-fb81-4764-a7de-a75069c0cfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:17.834081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc1fea-fb81-4764-a7de-a75069c0cfa0 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:17.834865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc1fea-fb81-4764-a7de-a75069c0cfa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:17.838835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc1fea-fb81-4764-a7de-a75069c0cfa0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:17.890674 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 956/1897] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:17 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:17.894762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fee8d5-70a0-4ca1-b90b-4fa413fb45b0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:17.911800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fee8d5-70a0-4ca1-b90b-4fa413fb45b0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:17.927829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5d03e-f855-4f48-bda8-35e851acbe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:17.928687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5d03e-f855-4f48-bda8-35e851acbe1c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:17.929079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5d03e-f855-4f48-bda8-35e851acbe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:17.929367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5d03e-f855-4f48-bda8-35e851acbe1c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:17.930180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5d03e-f855-4f48-bda8-35e851acbe1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:17.931844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5d03e-f855-4f48-bda8-35e851acbe1c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:17.953977 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 942/1898] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:17.957654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc1fea-fb81-4764-a7de-a75069c0cfa0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:17.979992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc1fea-fb81-4764-a7de-a75069c0cfa0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:18.020086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806ef0b-eb9a-41c1-b6b9-b8882947b652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:18.026247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5d03e-f855-4f48-bda8-35e851acbe1c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:18.029147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-16f5d03e-f855-4f48-bda8-35e851acbe1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:18.039551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806ef0b-eb9a-41c1-b6b9-b8882947b652 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:18.039767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806ef0b-eb9a-41c1-b6b9-b8882947b652 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:18.042614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806ef0b-eb9a-41c1-b6b9-b8882947b652 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:18.054673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806ef0b-eb9a-41c1-b6b9-b8882947b652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:18.060873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806ef0b-eb9a-41c1-b6b9-b8882947b652 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:18.066520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5d03e-f855-4f48-bda8-35e851acbe1c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:18.080598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:18.081302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:18.081592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:18.081876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] PATH_INFO: `/v3/projects/2a28735e9d204354a30066460fd3d271` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:18.087046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a28735e9d204354a30066460fd3d271)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:18.090010 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 957/1899] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:49:18.090671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:18.095109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806ef0b-eb9a-41c1-b6b9-b8882947b652 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:18.123709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806ef0b-eb9a-41c1-b6b9-b8882947b652 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:18.135291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a0680-b502-4298-8839-7d7060165239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:18.135981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a0680-b502-4298-8839-7d7060165239 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:18.136502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a0680-b502-4298-8839-7d7060165239 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:18.136797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a0680-b502-4298-8839-7d7060165239 admin admin] PATH_INFO: `/v3/projects/c4f4634039eb4925bb80ed9c890a43b6/users/52b13d0749224aafb3e2814cec7ed1e3/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:18.142552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:18.162260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a0680-b502-4298-8839-7d7060165239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f4634039eb4925bb80ed9c890a43b6, user_id=52b13d0749224aafb3e2814cec7ed1e3, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:18.167083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a0680-b502-4298-8839-7d7060165239 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:18.201950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a28735e9d204354a30066460fd3d271', 'request_id': 'req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:18.203034 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 958/1900] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:18 2022] PUT /identity/v3/projects/c4f4634039eb4925bb80ed9c890a43b6/users/52b13d0749224aafb3e2814cec7ed1e3/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:18.213054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59dd28-727a-4530-89b7-d5e60a7c5901 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:18.213411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59dd28-727a-4530-89b7-d5e60a7c5901 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:18.213849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59dd28-727a-4530-89b7-d5e60a7c5901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:18.220546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a28735e9d204354a30066460fd3d271', 'request_id': 'req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:18.231078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a28735e9d204354a30066460fd3d271', 'request_id': 'req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:18.255693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] The token cache is being invalidate because project 2a28735e9d204354a30066460fd3d271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:18.256045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:18.261296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:18.267355 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 943/1901] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:18 2022] DELETE /identity/v3/projects/2a28735e9d204354a30066460fd3d271 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:18.271847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:18.273809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:18.290112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2c59dd28-727a-4530-89b7-d5e60a7c5901 None None] MFA Rules not processed for user `52b13d0749224aafb3e2814cec7ed1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:18.292102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2c59dd28-727a-4530-89b7-d5e60a7c5901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:18.389570 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 959/1902] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:18.400705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59dd28-727a-4530-89b7-d5e60a7c5901 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:18.403406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2c59dd28-727a-4530-89b7-d5e60a7c5901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:18.470992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b8ffb-b3de-4bf4-9a51-fab8b41e2d3d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:18.489175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59dd28-727a-4530-89b7-d5e60a7c5901 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:18.495609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cc08b-295a-4976-857e-33b913b3ee33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:18.496984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc08b-295a-4976-857e-33b913b3ee33 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:18.497466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc08b-295a-4976-857e-33b913b3ee33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:18.498304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc08b-295a-4976-857e-33b913b3ee33 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:18.499041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cc08b-295a-4976-857e-33b913b3ee33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:18.501088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cc08b-295a-4976-857e-33b913b3ee33 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:18.545425 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 960/1903] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:18 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:18.549124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cc08b-295a-4976-857e-33b913b3ee33 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:18.591137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:18.591610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:18.592197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:18.592730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] PATH_INFO: `/v3/users/5993923715d94398a246e19ecbf59106` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:18.596843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5993923715d94398a246e19ecbf59106)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:18.598860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:18.618242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:18.683941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cc08b-295a-4976-857e-33b913b3ee33 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:18.698189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57eeaa-d72d-479f-8ba5-081de261a71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:18.700381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57eeaa-d72d-479f-8ba5-081de261a71e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:18.701429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57eeaa-d72d-479f-8ba5-081de261a71e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:18.702437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57eeaa-d72d-479f-8ba5-081de261a71e admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:18.704214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57eeaa-d72d-479f-8ba5-081de261a71e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:18.708407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57eeaa-d72d-479f-8ba5-081de261a71e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:18.718094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5993923715d94398a246e19ecbf59106'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:18.727161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:18.745008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5993923715d94398a246e19ecbf59106'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:18.747205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5993923715d94398a246e19ecbf59106'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:18.756600 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 961/1904] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:18 2022] POST /identity/v3/users => generated 300 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:18.760657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57eeaa-d72d-479f-8ba5-081de261a71e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:18.764227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ad57eeaa-d72d-479f-8ba5-081de261a71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:18.772325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5993923715d94398a246e19ecbf59106'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:18.793845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5993923715d94398a246e19ecbf59106'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:18.803996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57eeaa-d72d-479f-8ba5-081de261a71e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:18.808377 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 944/1905] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:18 2022] DELETE /identity/v3/users/5993923715d94398a246e19ecbf59106 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:18.923241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ca995-7f0c-484b-a24c-34077dc62e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:18.924199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ca995-7f0c-484b-a24c-34077dc62e6b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:18.925254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ca995-7f0c-484b-a24c-34077dc62e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:18.925694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ca995-7f0c-484b-a24c-34077dc62e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:18.928476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ca995-7f0c-484b-a24c-34077dc62e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:18.933503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ca995-7f0c-484b-a24c-34077dc62e6b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:18.959556 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 962/1906] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:18.963478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:18.976131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7277c-b37d-4b53-ae19-eadb1e0a735d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:18.990280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2493f-d695-4046-abc3-61fedcea8db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:18.991360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2493f-d695-4046-abc3-61fedcea8db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:18.991901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2493f-d695-4046-abc3-61fedcea8db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:18.992403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2493f-d695-4046-abc3-61fedcea8db0 admin admin] PATH_INFO: `/v3/projects/ca6d1aa1e81e4acda7bd88ee577fabe6/users/092b8d9c33104cbbaec18d9d8b4ed7ef/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:19.027911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2493f-d695-4046-abc3-61fedcea8db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6d1aa1e81e4acda7bd88ee577fabe6, user_id=092b8d9c33104cbbaec18d9d8b4ed7ef, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:19.033576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2493f-d695-4046-abc3-61fedcea8db0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:19.054948 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 945/1907] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:18 2022] PUT /identity/v3/projects/ca6d1aa1e81e4acda7bd88ee577fabe6/users/092b8d9c33104cbbaec18d9d8b4ed7ef/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:19.058890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ca995-7f0c-484b-a24c-34077dc62e6b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:19.076389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ca995-7f0c-484b-a24c-34077dc62e6b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:19.249780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c33a9c-0532-4427-8ab8-b8e9b6d2487b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:19.250677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c33a9c-0532-4427-8ab8-b8e9b6d2487b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:19.250977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c33a9c-0532-4427-8ab8-b8e9b6d2487b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:19.251261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c33a9c-0532-4427-8ab8-b8e9b6d2487b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:19.251923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c33a9c-0532-4427-8ab8-b8e9b6d2487b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:19.253592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c33a9c-0532-4427-8ab8-b8e9b6d2487b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:19.277293 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 963/1908] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:19.286038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2493f-d695-4046-abc3-61fedcea8db0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:19.297020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a48b97-e270-4ded-899e-a75a94b171d2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:19.297317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a48b97-e270-4ded-899e-a75a94b171d2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:19.297599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a48b97-e270-4ded-899e-a75a94b171d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:19.304444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2493f-d695-4046-abc3-61fedcea8db0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:19.316260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e775202-1f28-458e-ae07-32fa53d1d1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:19.316914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e775202-1f28-458e-ae07-32fa53d1d1c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:19.317938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e775202-1f28-458e-ae07-32fa53d1d1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:19.317938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e775202-1f28-458e-ae07-32fa53d1d1c5 admin admin] PATH_INFO: `/v3/projects/ca6d1aa1e81e4acda7bd88ee577fabe6/users/092b8d9c33104cbbaec18d9d8b4ed7ef/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:19.323623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e775202-1f28-458e-ae07-32fa53d1d1c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6d1aa1e81e4acda7bd88ee577fabe6, user_id=092b8d9c33104cbbaec18d9d8b4ed7ef, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:19.326930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e775202-1f28-458e-ae07-32fa53d1d1c5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:19.353725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-21a48b97-e270-4ded-899e-a75a94b171d2 None None] MFA Rules not processed for user `928517e539f24ab2b0f1bd3033051a00`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:19.355014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-21a48b97-e270-4ded-899e-a75a94b171d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:19.368677 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 946/1909] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:19 2022] PUT /identity/v3/projects/ca6d1aa1e81e4acda7bd88ee577fabe6/users/092b8d9c33104cbbaec18d9d8b4ed7ef/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:19.369761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e775202-1f28-458e-ae07-32fa53d1d1c5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:19.371691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2e775202-1f28-458e-ae07-32fa53d1d1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:19.462072 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 964/1910] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:19 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:19.464751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f8230-2a66-4de4-83b2-4fc4149345ad None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:19.466877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f8230-2a66-4de4-83b2-4fc4149345ad None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:19.467279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f8230-2a66-4de4-83b2-4fc4149345ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:19.491105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:19.501470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e775202-1f28-458e-ae07-32fa53d1d1c5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:19.517589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-617f8230-2a66-4de4-83b2-4fc4149345ad None None] MFA Rules not processed for user `092b8d9c33104cbbaec18d9d8b4ed7ef`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:19.519099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-617f8230-2a66-4de4-83b2-4fc4149345ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:19.524269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:19.526175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:19.527714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:19.528164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] PATH_INFO: `/v3/projects/6db583b83bcd4030bf0f9a70f6943210` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:19.530279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db583b83bcd4030bf0f9a70f6943210)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:19.532386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:19.621915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db583b83bcd4030bf0f9a70f6943210', 'request_id': 'req-e333bb29-2220-4aaf-8afd-50152cc8df48'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:19.624473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:19.635691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db583b83bcd4030bf0f9a70f6943210', 'request_id': 'req-e333bb29-2220-4aaf-8afd-50152cc8df48'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:19.645113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db583b83bcd4030bf0f9a70f6943210', 'request_id': 'req-e333bb29-2220-4aaf-8afd-50152cc8df48'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:19.647390 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 965/1911] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:19.661370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f8230-2a66-4de4-83b2-4fc4149345ad None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:19.663915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] The token cache is being invalidate because project 6db583b83bcd4030bf0f9a70f6943210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:19.664230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:19.664481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-617f8230-2a66-4de4-83b2-4fc4149345ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:19.666329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:19.669527 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 947/1912] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:19 2022] DELETE /identity/v3/projects/6db583b83bcd4030bf0f9a70f6943210 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:19.671113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:19.672246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:19.768356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333bb29-2220-4aaf-8afd-50152cc8df48 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:19.774518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da550ac-ba08-4f40-a50e-1fcbcfba5212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:19.775601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da550ac-ba08-4f40-a50e-1fcbcfba5212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:19.776054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da550ac-ba08-4f40-a50e-1fcbcfba5212 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:19.776395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da550ac-ba08-4f40-a50e-1fcbcfba5212 admin admin] PATH_INFO: `/v3/users/e939063cdd1a4e8ea576fde05f1b6d60` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:19.778345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da550ac-ba08-4f40-a50e-1fcbcfba5212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e939063cdd1a4e8ea576fde05f1b6d60)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:19.781384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da550ac-ba08-4f40-a50e-1fcbcfba5212 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:19.796955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f8230-2a66-4de4-83b2-4fc4149345ad None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:19.890497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9da550ac-ba08-4f40-a50e-1fcbcfba5212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e939063cdd1a4e8ea576fde05f1b6d60'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:19.903325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cbf21-6586-40eb-a45d-768a1120ccd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:19.904145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cbf21-6586-40eb-a45d-768a1120ccd8 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:19.904442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cbf21-6586-40eb-a45d-768a1120ccd8 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:19.904730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cbf21-6586-40eb-a45d-768a1120ccd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:19.907545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-104cbf21-6586-40eb-a45d-768a1120ccd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:19.918701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9da550ac-ba08-4f40-a50e-1fcbcfba5212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e939063cdd1a4e8ea576fde05f1b6d60'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:19.920069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9da550ac-ba08-4f40-a50e-1fcbcfba5212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e939063cdd1a4e8ea576fde05f1b6d60'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:19.927568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9da550ac-ba08-4f40-a50e-1fcbcfba5212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e939063cdd1a4e8ea576fde05f1b6d60'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:19.937878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9da550ac-ba08-4f40-a50e-1fcbcfba5212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e939063cdd1a4e8ea576fde05f1b6d60'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:19.945535 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 948/1913] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:19 2022] DELETE /identity/v3/users/e939063cdd1a4e8ea576fde05f1b6d60 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:19.946854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23fd5b-8507-4792-a745-9cfbbcae4f83 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:19.947145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23fd5b-8507-4792-a745-9cfbbcae4f83 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:19.947424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23fd5b-8507-4792-a745-9cfbbcae4f83 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:19.953305 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 949/1914] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:49:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:19.961534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d57b-4952-4611-8770-a9b863d59ddc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:19.962483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d57b-4952-4611-8770-a9b863d59ddc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:19.962990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d57b-4952-4611-8770-a9b863d59ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:19.969621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:20.013460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3354d57b-4952-4611-8770-a9b863d59ddc None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:20.014703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3354d57b-4952-4611-8770-a9b863d59ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:20.058967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104cbf21-6586-40eb-a45d-768a1120ccd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:20.060274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104cbf21-6586-40eb-a45d-768a1120ccd8 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:20.077956 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 966/1915] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:20.177807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:20.225081 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 950/1916] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:49:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:20.504172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cbf21-6586-40eb-a45d-768a1120ccd8 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:20.526093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cbf21-6586-40eb-a45d-768a1120ccd8 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:20.543390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354d57b-4952-4611-8770-a9b863d59ddc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:20.544922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3354d57b-4952-4611-8770-a9b863d59ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:20.574123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354d57b-4952-4611-8770-a9b863d59ddc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:20.586021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478c229-f957-43f0-8e4e-9472d3be0932 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:20.586631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478c229-f957-43f0-8e4e-9472d3be0932 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:20.586916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478c229-f957-43f0-8e4e-9472d3be0932 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:20.587275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478c229-f957-43f0-8e4e-9472d3be0932 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:20.609216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478c229-f957-43f0-8e4e-9472d3be0932 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:20.610573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478c229-f957-43f0-8e4e-9472d3be0932 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:20.647651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6638d-cd12-4e36-a72f-7d265a1b8303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:20.648406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6638d-cd12-4e36-a72f-7d265a1b8303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:20.648775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6638d-cd12-4e36-a72f-7d265a1b8303 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:20.648994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6638d-cd12-4e36-a72f-7d265a1b8303 admin admin] PATH_INFO: `/v3/projects/4b46258375af4678b58c037fb587d677` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:20.650848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6638d-cd12-4e36-a72f-7d265a1b8303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b46258375af4678b58c037fb587d677)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:20.652855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6638d-cd12-4e36-a72f-7d265a1b8303 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:20.665107 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 951/1917] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:20.754173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-efb6638d-cd12-4e36-a72f-7d265a1b8303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b46258375af4678b58c037fb587d677', 'request_id': 'req-efb6638d-cd12-4e36-a72f-7d265a1b8303'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:20.776387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-efb6638d-cd12-4e36-a72f-7d265a1b8303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b46258375af4678b58c037fb587d677', 'request_id': 'req-efb6638d-cd12-4e36-a72f-7d265a1b8303'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:20.801652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-efb6638d-cd12-4e36-a72f-7d265a1b8303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b46258375af4678b58c037fb587d677', 'request_id': 'req-efb6638d-cd12-4e36-a72f-7d265a1b8303'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:20.818584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-efb6638d-cd12-4e36-a72f-7d265a1b8303 admin admin] The token cache is being invalidate because project 4b46258375af4678b58c037fb587d677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:20.818959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-efb6638d-cd12-4e36-a72f-7d265a1b8303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:20.819789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-efb6638d-cd12-4e36-a72f-7d265a1b8303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:20.827511 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 967/1918] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:20 2022] DELETE /identity/v3/projects/4b46258375af4678b58c037fb587d677 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:20.831796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0e827-9688-45db-9ce0-11d26d2ad858 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:20.832318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0e827-9688-45db-9ce0-11d26d2ad858 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:20.833009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0e827-9688-45db-9ce0-11d26d2ad858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:20.884053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9ce0e827-9688-45db-9ce0-11d26d2ad858 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:20.885496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9ce0e827-9688-45db-9ce0-11d26d2ad858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:20.975616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dbc0c-defa-4fa4-bed0-d781220bc854 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:20.976672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dbc0c-defa-4fa4-bed0-d781220bc854 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:20.977055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dbc0c-defa-4fa4-bed0-d781220bc854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:20.980804 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 952/1919] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:21.002429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-789dbc0c-defa-4fa4-bed0-d781220bc854 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:21.004212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-789dbc0c-defa-4fa4-bed0-d781220bc854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:21.018032 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 968/1920] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:21.022138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0e827-9688-45db-9ce0-11d26d2ad858 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:21.041958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0e827-9688-45db-9ce0-11d26d2ad858 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:21.043297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dbc0c-defa-4fa4-bed0-d781220bc854 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:21.046134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-789dbc0c-defa-4fa4-bed0-d781220bc854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:21.056755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686c005-16e6-4ade-8173-4df4cbc6c7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:21.058775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686c005-16e6-4ade-8173-4df4cbc6c7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:21.058775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686c005-16e6-4ade-8173-4df4cbc6c7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:21.059580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686c005-16e6-4ade-8173-4df4cbc6c7f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:21.060843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686c005-16e6-4ade-8173-4df4cbc6c7f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:21.061278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686c005-16e6-4ade-8173-4df4cbc6c7f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:21.063098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686c005-16e6-4ade-8173-4df4cbc6c7f2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:21.079050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dbc0c-defa-4fa4-bed0-d781220bc854 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:21.095735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6866e8f-f259-4a19-be8c-5fe675e9ce46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:21.096809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6866e8f-f259-4a19-be8c-5fe675e9ce46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:21.097119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6866e8f-f259-4a19-be8c-5fe675e9ce46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:21.097527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6866e8f-f259-4a19-be8c-5fe675e9ce46 admin admin] PATH_INFO: `/v3/users/52b13d0749224aafb3e2814cec7ed1e3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:21.098061 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 953/1921] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:21.103679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686c005-16e6-4ade-8173-4df4cbc6c7f2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:21.112089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6866e8f-f259-4a19-be8c-5fe675e9ce46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52b13d0749224aafb3e2814cec7ed1e3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:21.122938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6866e8f-f259-4a19-be8c-5fe675e9ce46 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:21.142916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686c005-16e6-4ade-8173-4df4cbc6c7f2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:21.152945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ab6ac-6035-4cdb-87fd-a5432b3ba4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:21.152945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ab6ac-6035-4cdb-87fd-a5432b3ba4cd admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:21.152945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ab6ac-6035-4cdb-87fd-a5432b3ba4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:21.153341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ab6ac-6035-4cdb-87fd-a5432b3ba4cd admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:21.153736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ab6ac-6035-4cdb-87fd-a5432b3ba4cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:21.155676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ab6ac-6035-4cdb-87fd-a5432b3ba4cd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:21.200055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:21.207554 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 954/1922] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:21 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:21.212566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ab6ac-6035-4cdb-87fd-a5432b3ba4cd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:21.219020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a6866e8f-f259-4a19-be8c-5fe675e9ce46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52b13d0749224aafb3e2814cec7ed1e3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:21.261831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a6866e8f-f259-4a19-be8c-5fe675e9ce46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52b13d0749224aafb3e2814cec7ed1e3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:21.278208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a6866e8f-f259-4a19-be8c-5fe675e9ce46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52b13d0749224aafb3e2814cec7ed1e3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:21.279437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a6866e8f-f259-4a19-be8c-5fe675e9ce46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52b13d0749224aafb3e2814cec7ed1e3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:21.280239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a6866e8f-f259-4a19-be8c-5fe675e9ce46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52b13d0749224aafb3e2814cec7ed1e3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:21.302618 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 969/1923] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:21 2022] DELETE /identity/v3/users/52b13d0749224aafb3e2814cec7ed1e3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:21.322114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdbb21-1d2b-4c1f-844b-5f50bad27729 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:21.322505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdbb21-1d2b-4c1f-844b-5f50bad27729 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:21.322796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdbb21-1d2b-4c1f-844b-5f50bad27729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:21.348581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ab6ac-6035-4cdb-87fd-a5432b3ba4cd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:21.349157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9fbdbb21-1d2b-4c1f-844b-5f50bad27729 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:21.350429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9fbdbb21-1d2b-4c1f-844b-5f50bad27729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:21.414940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:21.431573 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 970/1924] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:21.433329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70f2b9-6939-4b85-b51c-c31a3210475f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:21.434118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70f2b9-6939-4b85-b51c-c31a3210475f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:21.434550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70f2b9-6939-4b85-b51c-c31a3210475f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:21.436050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70f2b9-6939-4b85-b51c-c31a3210475f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:21.436860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70f2b9-6939-4b85-b51c-c31a3210475f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:21.439010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70f2b9-6939-4b85-b51c-c31a3210475f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:21.442305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdbb21-1d2b-4c1f-844b-5f50bad27729 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:21.443903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9fbdbb21-1d2b-4c1f-844b-5f50bad27729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:21.471452 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 955/1925] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:21 2022] POST /identity/v3/users => generated 294 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:21.475029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70f2b9-6939-4b85-b51c-c31a3210475f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:21.480391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ef70f2b9-6939-4b85-b51c-c31a3210475f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:21.504424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70f2b9-6939-4b85-b51c-c31a3210475f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:21.517724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb21608-11ac-462c-a2c6-10ed34bbe3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:21.518808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21608-11ac-462c-a2c6-10ed34bbe3cf admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:21.520060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21608-11ac-462c-a2c6-10ed34bbe3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:21.520562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21608-11ac-462c-a2c6-10ed34bbe3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:21.523648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb21608-11ac-462c-a2c6-10ed34bbe3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:21.527133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb21608-11ac-462c-a2c6-10ed34bbe3cf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:21.541182 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 956/1926] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:21.549178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdbb21-1d2b-4c1f-844b-5f50bad27729 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:21.553115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d0a53-e8d1-4492-9df3-d53396001197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:21.553751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d0a53-e8d1-4492-9df3-d53396001197 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:21.554033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d0a53-e8d1-4492-9df3-d53396001197 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:21.554307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d0a53-e8d1-4492-9df3-d53396001197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:21.558117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb21608-11ac-462c-a2c6-10ed34bbe3cf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:21.566923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d0a53-e8d1-4492-9df3-d53396001197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:21.569665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d0a53-e8d1-4492-9df3-d53396001197 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:21.582842 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 971/1927] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:21.589190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb21608-11ac-462c-a2c6-10ed34bbe3cf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:21.611875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb94c9-2627-47d8-98f1-af3b26e86a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:21.612605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb94c9-2627-47d8-98f1-af3b26e86a72 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:21.612922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb94c9-2627-47d8-98f1-af3b26e86a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:21.613217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb94c9-2627-47d8-98f1-af3b26e86a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:21.614613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb94c9-2627-47d8-98f1-af3b26e86a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:21.616256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb94c9-2627-47d8-98f1-af3b26e86a72 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:21.655342 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 957/1928] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:21.656774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d0a53-e8d1-4492-9df3-d53396001197 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:21.671638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d0a53-e8d1-4492-9df3-d53396001197 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:21.689967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04842a-0e3b-4bcd-8115-7857fb2445b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:21.692560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04842a-0e3b-4bcd-8115-7857fb2445b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:21.693052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04842a-0e3b-4bcd-8115-7857fb2445b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:21.694044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04842a-0e3b-4bcd-8115-7857fb2445b2 admin admin] PATH_INFO: `/v3/projects/3f2e65e42b6f424084b1908581c853f2/users/1bcd7ba470ae456bbefd5ff0f8966fb1/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:21.754326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04842a-0e3b-4bcd-8115-7857fb2445b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f2e65e42b6f424084b1908581c853f2, user_id=1bcd7ba470ae456bbefd5ff0f8966fb1, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:21.757798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04842a-0e3b-4bcd-8115-7857fb2445b2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:21.787223 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 972/1929] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:21 2022] PUT /identity/v3/projects/3f2e65e42b6f424084b1908581c853f2/users/1bcd7ba470ae456bbefd5ff0f8966fb1/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:21.794723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17462c-8a6f-4dfd-9167-7c4603ac9017 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:21.795014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17462c-8a6f-4dfd-9167-7c4603ac9017 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:21.795292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17462c-8a6f-4dfd-9167-7c4603ac9017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:21.881695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3c17462c-8a6f-4dfd-9167-7c4603ac9017 None None] MFA Rules not processed for user `1bcd7ba470ae456bbefd5ff0f8966fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:21.884795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3c17462c-8a6f-4dfd-9167-7c4603ac9017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:22.043967 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 958/1930] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:22.048888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04842a-0e3b-4bcd-8115-7857fb2445b2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:22.114733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17462c-8a6f-4dfd-9167-7c4603ac9017 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:22.127871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3c17462c-8a6f-4dfd-9167-7c4603ac9017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:22.146769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:22.197772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04842a-0e3b-4bcd-8115-7857fb2445b2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:22.209116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29504850-4939-496b-834e-d21c40c9ca72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:22.209852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29504850-4939-496b-834e-d21c40c9ca72 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:22.210239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29504850-4939-496b-834e-d21c40c9ca72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:22.210616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29504850-4939-496b-834e-d21c40c9ca72 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:22.211382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29504850-4939-496b-834e-d21c40c9ca72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:22.213934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29504850-4939-496b-834e-d21c40c9ca72 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:22.253054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17462c-8a6f-4dfd-9167-7c4603ac9017 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:22.260699 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 973/1931] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:22 2022] POST /identity/v3/projects => generated 389 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:22.260996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:22.261844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:22.262121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29504850-4939-496b-834e-d21c40c9ca72 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:22.262603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:22.263084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] PATH_INFO: `/v3/projects/c4f4634039eb4925bb80ed9c890a43b6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:22.264493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f4634039eb4925bb80ed9c890a43b6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:22.269427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:22.338909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:22.340676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f4634039eb4925bb80ed9c890a43b6', 'request_id': 'req-0875aba2-e413-409e-b425-1c9c56c7abc3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:22.352425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f4634039eb4925bb80ed9c890a43b6', 'request_id': 'req-0875aba2-e413-409e-b425-1c9c56c7abc3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:22.382661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f4634039eb4925bb80ed9c890a43b6', 'request_id': 'req-0875aba2-e413-409e-b425-1c9c56c7abc3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:22.400284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] The token cache is being invalidate because project c4f4634039eb4925bb80ed9c890a43b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:22.400821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:22.402345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:22.404304 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 959/1932] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:22 2022] DELETE /identity/v3/projects/c4f4634039eb4925bb80ed9c890a43b6 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:22.408775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:22.410486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:22.426538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29504850-4939-496b-834e-d21c40c9ca72 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:22.508966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cac2c-13f2-44f0-ab01-6900d4a835fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:22.509936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cac2c-13f2-44f0-ab01-6900d4a835fd admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:22.510358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cac2c-13f2-44f0-ab01-6900d4a835fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:22.510777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cac2c-13f2-44f0-ab01-6900d4a835fd admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:22.511587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cac2c-13f2-44f0-ab01-6900d4a835fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:22.514023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cac2c-13f2-44f0-ab01-6900d4a835fd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:22.515084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875aba2-e413-409e-b425-1c9c56c7abc3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:22.520772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:22.521746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:22.522194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:22.522617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] PATH_INFO: `/v3/users/092b8d9c33104cbbaec18d9d8b4ed7ef` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:22.524549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=092b8d9c33104cbbaec18d9d8b4ed7ef)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:22.526934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:22.549358 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 974/1933] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:22 2022] POST /identity/v3/users => generated 301 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:22.554911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cac2c-13f2-44f0-ab01-6900d4a835fd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:22.558643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-164cac2c-13f2-44f0-ab01-6900d4a835fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:22.576525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cac2c-13f2-44f0-ab01-6900d4a835fd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:22.584987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1caae-f6d7-4819-b54f-4923b5b54502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:22.586557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1caae-f6d7-4819-b54f-4923b5b54502 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:22.586988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1caae-f6d7-4819-b54f-4923b5b54502 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:22.587358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1caae-f6d7-4819-b54f-4923b5b54502 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:22.589109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1caae-f6d7-4819-b54f-4923b5b54502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:22.592620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1caae-f6d7-4819-b54f-4923b5b54502 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:22.624217 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 975/1934] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:22.625339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1caae-f6d7-4819-b54f-4923b5b54502 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:22.628036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '092b8d9c33104cbbaec18d9d8b4ed7ef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:22.639628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1caae-f6d7-4819-b54f-4923b5b54502 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:22.647490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193590b-5e2d-4f1a-838c-c59741158668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:22.648252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193590b-5e2d-4f1a-838c-c59741158668 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:22.648858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193590b-5e2d-4f1a-838c-c59741158668 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:22.649181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193590b-5e2d-4f1a-838c-c59741158668 admin admin] PATH_INFO: `/v3/projects/01257be8071444c089e63b0614a07130/users/36cff37f33cc458e97c47a9b047b7081/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:22.649934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '092b8d9c33104cbbaec18d9d8b4ed7ef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:22.650983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '092b8d9c33104cbbaec18d9d8b4ed7ef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:22.663406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '092b8d9c33104cbbaec18d9d8b4ed7ef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:22.673191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193590b-5e2d-4f1a-838c-c59741158668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01257be8071444c089e63b0614a07130, user_id=36cff37f33cc458e97c47a9b047b7081, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:22.675940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193590b-5e2d-4f1a-838c-c59741158668 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:22.677054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '092b8d9c33104cbbaec18d9d8b4ed7ef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:22.691946 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 960/1935] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:22 2022] DELETE /identity/v3/users/092b8d9c33104cbbaec18d9d8b4ed7ef => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:22.699986 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 976/1936] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:22 2022] PUT /identity/v3/projects/01257be8071444c089e63b0614a07130/users/36cff37f33cc458e97c47a9b047b7081/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:22.704639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:22.706898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:22.817394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:22.827019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd64f87-7813-466f-949b-22f65654fe4d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:22.833479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e6758-e259-476f-b74a-9c6b9f0297a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:22.834486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e6758-e259-476f-b74a-9c6b9f0297a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:22.835124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e6758-e259-476f-b74a-9c6b9f0297a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:22.835695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e6758-e259-476f-b74a-9c6b9f0297a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:22.836869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e6758-e259-476f-b74a-9c6b9f0297a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:22.838673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e6758-e259-476f-b74a-9c6b9f0297a4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:22.878059 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 961/1937] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:22.879112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193590b-5e2d-4f1a-838c-c59741158668 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:22.946590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:23.021450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193590b-5e2d-4f1a-838c-c59741158668 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:23.029363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba6d37-937b-4b71-91f5-f07bf34fab71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:23.030435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba6d37-937b-4b71-91f5-f07bf34fab71 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:23.030950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba6d37-937b-4b71-91f5-f07bf34fab71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:23.031338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba6d37-937b-4b71-91f5-f07bf34fab71 admin admin] PATH_INFO: `/v3/projects/01257be8071444c089e63b0614a07130/users/36cff37f33cc458e97c47a9b047b7081/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:23.037179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba6d37-937b-4b71-91f5-f07bf34fab71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01257be8071444c089e63b0614a07130, user_id=36cff37f33cc458e97c47a9b047b7081, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:23.041715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba6d37-937b-4b71-91f5-f07bf34fab71 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:23.056448 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 977/1938] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:22 2022] PUT /identity/v3/projects/01257be8071444c089e63b0614a07130/users/36cff37f33cc458e97c47a9b047b7081/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:23.066861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c5a81-df72-480a-af45-46a2eccfd744 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:23.067332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c5a81-df72-480a-af45-46a2eccfd744 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:23.067476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c5a81-df72-480a-af45-46a2eccfd744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:23.110167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2d3c5a81-df72-480a-af45-46a2eccfd744 None None] MFA Rules not processed for user `36cff37f33cc458e97c47a9b047b7081`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:23.112149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2d3c5a81-df72-480a-af45-46a2eccfd744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:23.206241 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 962/1939] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:23 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:23.217981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba6d37-937b-4b71-91f5-f07bf34fab71 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:23.221748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-25ba6d37-937b-4b71-91f5-f07bf34fab71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:23.360710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba6d37-937b-4b71-91f5-f07bf34fab71 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:23.372716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79402123-2ba0-41c7-a27b-cfff9162b57a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:23.373401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79402123-2ba0-41c7-a27b-cfff9162b57a service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:23.373691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79402123-2ba0-41c7-a27b-cfff9162b57a service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:23.373979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79402123-2ba0-41c7-a27b-cfff9162b57a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:23.395541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79402123-2ba0-41c7-a27b-cfff9162b57a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:23.397232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79402123-2ba0-41c7-a27b-cfff9162b57a service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:23.420007 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 978/1940] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:23 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:23.473328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c5a81-df72-480a-af45-46a2eccfd744 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:23.491966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c5a81-df72-480a-af45-46a2eccfd744 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:23.501805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420af20-94f5-4be7-a1bc-9382aa461892 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:23.502451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420af20-94f5-4be7-a1bc-9382aa461892 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:23.502738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420af20-94f5-4be7-a1bc-9382aa461892 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:23.503018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420af20-94f5-4be7-a1bc-9382aa461892 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:23.505614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4420af20-94f5-4be7-a1bc-9382aa461892 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:23.601520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79402123-2ba0-41c7-a27b-cfff9162b57a service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:23.603820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-79402123-2ba0-41c7-a27b-cfff9162b57a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:23.678338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420af20-94f5-4be7-a1bc-9382aa461892 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:23.680460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420af20-94f5-4be7-a1bc-9382aa461892 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:23.703811 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 963/1941] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:23.780740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79402123-2ba0-41c7-a27b-cfff9162b57a service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:23.798055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94513d-6830-412d-a482-9c60baa739ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:23.798917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94513d-6830-412d-a482-9c60baa739ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:23.799211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94513d-6830-412d-a482-9c60baa739ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:23.799490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94513d-6830-412d-a482-9c60baa739ec admin admin] PATH_INFO: `/v3/projects/ca6d1aa1e81e4acda7bd88ee577fabe6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:23.806250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94513d-6830-412d-a482-9c60baa739ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca6d1aa1e81e4acda7bd88ee577fabe6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:23.809305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94513d-6830-412d-a482-9c60baa739ec admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:23.959304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:23.964707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ec94513d-6830-412d-a482-9c60baa739ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca6d1aa1e81e4acda7bd88ee577fabe6', 'request_id': 'req-ec94513d-6830-412d-a482-9c60baa739ec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:23.985025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ec94513d-6830-412d-a482-9c60baa739ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca6d1aa1e81e4acda7bd88ee577fabe6', 'request_id': 'req-ec94513d-6830-412d-a482-9c60baa739ec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:24.005810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ec94513d-6830-412d-a482-9c60baa739ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca6d1aa1e81e4acda7bd88ee577fabe6', 'request_id': 'req-ec94513d-6830-412d-a482-9c60baa739ec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:24.030981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ec94513d-6830-412d-a482-9c60baa739ec admin admin] The token cache is being invalidate because project ca6d1aa1e81e4acda7bd88ee577fabe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:24.031551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ec94513d-6830-412d-a482-9c60baa739ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:24.033704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ec94513d-6830-412d-a482-9c60baa739ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:24.037892 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 979/1942] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:23 2022] DELETE /identity/v3/projects/ca6d1aa1e81e4acda7bd88ee577fabe6 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:24.039272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29890fe-bcd8-4d2c-a3af-ba772df24114 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:24.039579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29890fe-bcd8-4d2c-a3af-ba772df24114 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:24.039866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29890fe-bcd8-4d2c-a3af-ba772df24114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:24.094102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a29890fe-bcd8-4d2c-a3af-ba772df24114 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:24.099079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a29890fe-bcd8-4d2c-a3af-ba772df24114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:24.285691 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 964/1943] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:24.286908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0581319-5264-4787-89a8-41740eae85af None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:24.287383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0581319-5264-4787-89a8-41740eae85af None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:24.287458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0581319-5264-4787-89a8-41740eae85af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:24.310067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29890fe-bcd8-4d2c-a3af-ba772df24114 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:24.310441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a29890fe-bcd8-4d2c-a3af-ba772df24114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:24.324549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29890fe-bcd8-4d2c-a3af-ba772df24114 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:24.332226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4460de4-d333-434e-bbc9-c6942b73d728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:24.333040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4460de4-d333-434e-bbc9-c6942b73d728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:24.333328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4460de4-d333-434e-bbc9-c6942b73d728 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:24.333596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4460de4-d333-434e-bbc9-c6942b73d728 admin admin] PATH_INFO: `/v3/users/1bcd7ba470ae456bbefd5ff0f8966fb1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:24.336364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4460de4-d333-434e-bbc9-c6942b73d728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bcd7ba470ae456bbefd5ff0f8966fb1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:24.338232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a0581319-5264-4787-89a8-41740eae85af None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:24.338940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4460de4-d333-434e-bbc9-c6942b73d728 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:24.339980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a0581319-5264-4787-89a8-41740eae85af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:24.355004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:24.360877 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 980/1944] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:24.365404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0581319-5264-4787-89a8-41740eae85af None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:24.390966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0581319-5264-4787-89a8-41740eae85af None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:24.406971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64befb9e-91e7-4588-a41d-28a70ff84fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:24.407629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64befb9e-91e7-4588-a41d-28a70ff84fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:24.407912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64befb9e-91e7-4588-a41d-28a70ff84fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:24.408183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64befb9e-91e7-4588-a41d-28a70ff84fb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:24.408885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64befb9e-91e7-4588-a41d-28a70ff84fb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:24.409238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64befb9e-91e7-4588-a41d-28a70ff84fb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:24.412074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64befb9e-91e7-4588-a41d-28a70ff84fb6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:24.449737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f4460de4-d333-434e-bbc9-c6942b73d728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bcd7ba470ae456bbefd5ff0f8966fb1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:24.455216 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 981/1945] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:24.457409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64befb9e-91e7-4588-a41d-28a70ff84fb6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:24.478092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64befb9e-91e7-4588-a41d-28a70ff84fb6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:24.491517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f4460de4-d333-434e-bbc9-c6942b73d728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bcd7ba470ae456bbefd5ff0f8966fb1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:24.492667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f4460de4-d333-434e-bbc9-c6942b73d728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bcd7ba470ae456bbefd5ff0f8966fb1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:24.492948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c8b59-ceb9-4fca-8ea6-7df9ca603039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:24.493594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c8b59-ceb9-4fca-8ea6-7df9ca603039 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:24.493883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c8b59-ceb9-4fca-8ea6-7df9ca603039 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:24.494165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c8b59-ceb9-4fca-8ea6-7df9ca603039 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:24.494830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c8b59-ceb9-4fca-8ea6-7df9ca603039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:24.496781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c8b59-ceb9-4fca-8ea6-7df9ca603039 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:24.509222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f4460de4-d333-434e-bbc9-c6942b73d728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bcd7ba470ae456bbefd5ff0f8966fb1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:24.523519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f4460de4-d333-434e-bbc9-c6942b73d728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bcd7ba470ae456bbefd5ff0f8966fb1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:24.545353 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 965/1946] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:24 2022] DELETE /identity/v3/users/1bcd7ba470ae456bbefd5ff0f8966fb1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:24.551314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03df706-5b79-415d-a7f5-0e07aaad73df None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:24.551314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03df706-5b79-415d-a7f5-0e07aaad73df None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:24.554143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03df706-5b79-415d-a7f5-0e07aaad73df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:24.561347 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 982/1947] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:24 2022] POST /identity/v3/projects => generated 403 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:24.565298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c8b59-ceb9-4fca-8ea6-7df9ca603039 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:24.566991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2d1c8b59-ceb9-4fca-8ea6-7df9ca603039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:24.592380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e03df706-5b79-415d-a7f5-0e07aaad73df None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:24.594674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e03df706-5b79-415d-a7f5-0e07aaad73df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:24.665108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c8b59-ceb9-4fca-8ea6-7df9ca603039 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:24.674891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371a445-781e-4e74-83de-eac1631e968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:24.675525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371a445-781e-4e74-83de-eac1631e968c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:24.675811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371a445-781e-4e74-83de-eac1631e968c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:24.676086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371a445-781e-4e74-83de-eac1631e968c admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:24.677445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371a445-781e-4e74-83de-eac1631e968c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:24.680177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371a445-781e-4e74-83de-eac1631e968c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:24.697464 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 966/1948] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:24.714638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03df706-5b79-415d-a7f5-0e07aaad73df None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:24.716939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e03df706-5b79-415d-a7f5-0e07aaad73df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:24.737861 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 983/1949] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:24 2022] POST /identity/v3/users => generated 302 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:24.741608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371a445-781e-4e74-83de-eac1631e968c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:24.759735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371a445-781e-4e74-83de-eac1631e968c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:24.783594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd7e94-018d-4ea9-87dd-504ae8e18336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:24.784232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7e94-018d-4ea9-87dd-504ae8e18336 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:24.784524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7e94-018d-4ea9-87dd-504ae8e18336 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:24.784805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7e94-018d-4ea9-87dd-504ae8e18336 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:24.785876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd7e94-018d-4ea9-87dd-504ae8e18336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:24.788180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd7e94-018d-4ea9-87dd-504ae8e18336 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:24.803157 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 984/1950] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:24.804555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd7e94-018d-4ea9-87dd-504ae8e18336 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:24.819437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03df706-5b79-415d-a7f5-0e07aaad73df None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:24.823816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b13ab-c80d-43fa-8258-5c051683d76f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:24.825102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b13ab-c80d-43fa-8258-5c051683d76f service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:24.825819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b13ab-c80d-43fa-8258-5c051683d76f service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:24.825819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b13ab-c80d-43fa-8258-5c051683d76f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:24.836878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd7e94-018d-4ea9-87dd-504ae8e18336 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:24.837147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b13ab-c80d-43fa-8258-5c051683d76f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:24.838771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b13ab-c80d-43fa-8258-5c051683d76f service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:24.850510 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 967/1951] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:24.851103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e46874-01db-4e8e-81ca-99f2f6062b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:24.851719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e46874-01db-4e8e-81ca-99f2f6062b9e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:24.852013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e46874-01db-4e8e-81ca-99f2f6062b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:24.852293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e46874-01db-4e8e-81ca-99f2f6062b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:24.853586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e46874-01db-4e8e-81ca-99f2f6062b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:24.855803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e46874-01db-4e8e-81ca-99f2f6062b9e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:24.873469 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 985/1952] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:24.875622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b13ab-c80d-43fa-8258-5c051683d76f service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:24.889174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b13ab-c80d-43fa-8258-5c051683d76f service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:24.895643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29996a84-9563-458d-8a02-a8b563df034b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:24.896590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29996a84-9563-458d-8a02-a8b563df034b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:24.896909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29996a84-9563-458d-8a02-a8b563df034b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:24.897230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29996a84-9563-458d-8a02-a8b563df034b admin admin] PATH_INFO: `/v3/projects/4aa79c05b5a7408c9748ec9d3dfc4283/users/9387ac00c53d4d6687289d53f615f5a2/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:24.919642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29996a84-9563-458d-8a02-a8b563df034b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa79c05b5a7408c9748ec9d3dfc4283, user_id=9387ac00c53d4d6687289d53f615f5a2, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:24.922541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29996a84-9563-458d-8a02-a8b563df034b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:24.942645 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 968/1953] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:24 2022] PUT /identity/v3/projects/4aa79c05b5a7408c9748ec9d3dfc4283/users/9387ac00c53d4d6687289d53f615f5a2/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:24.955302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89f57e-9ce8-46fa-9002-2bdba006e35e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:24.955599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89f57e-9ce8-46fa-9002-2bdba006e35e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:24.955879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89f57e-9ce8-46fa-9002-2bdba006e35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:25.014387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cd89f57e-9ce8-46fa-9002-2bdba006e35e None None] MFA Rules not processed for user `9387ac00c53d4d6687289d53f615f5a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:25.018894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cd89f57e-9ce8-46fa-9002-2bdba006e35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:25.088748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:25.142109 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 986/1954] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:24 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:25.143365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29996a84-9563-458d-8a02-a8b563df034b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:25.172473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:25.287994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29996a84-9563-458d-8a02-a8b563df034b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:25.293664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb092d63-24fc-4a09-945b-46fe89e4e2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:25.294685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb092d63-24fc-4a09-945b-46fe89e4e2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:25.295067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb092d63-24fc-4a09-945b-46fe89e4e2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:25.295448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb092d63-24fc-4a09-945b-46fe89e4e2e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:25.297327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb092d63-24fc-4a09-945b-46fe89e4e2e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:25.300769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb092d63-24fc-4a09-945b-46fe89e4e2e6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:25.353049 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 969/1955] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:25 2022] POST /identity/v3/projects => generated 401 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:25.355194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89f57e-9ce8-46fa-9002-2bdba006e35e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:25.369153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89f57e-9ce8-46fa-9002-2bdba006e35e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:25.473290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb092d63-24fc-4a09-945b-46fe89e4e2e6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:25.476501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-eb092d63-24fc-4a09-945b-46fe89e4e2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:25.495890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163035d-9211-4270-a7c8-f0faade24c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:25.497002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163035d-9211-4270-a7c8-f0faade24c54 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:25.497539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163035d-9211-4270-a7c8-f0faade24c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:25.497995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163035d-9211-4270-a7c8-f0faade24c54 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:25.498950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163035d-9211-4270-a7c8-f0faade24c54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:25.504946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163035d-9211-4270-a7c8-f0faade24c54 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:25.563997 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 987/1956] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:25 2022] POST /identity/v3/users => generated 307 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:25.566102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163035d-9211-4270-a7c8-f0faade24c54 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:25.592043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163035d-9211-4270-a7c8-f0faade24c54 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:25.602522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e4ccb-f0a6-4400-9e7a-48db49b0cba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:25.603154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e4ccb-f0a6-4400-9e7a-48db49b0cba7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:25.603361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e4ccb-f0a6-4400-9e7a-48db49b0cba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:25.603646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e4ccb-f0a6-4400-9e7a-48db49b0cba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:25.604299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e4ccb-f0a6-4400-9e7a-48db49b0cba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:25.606072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e4ccb-f0a6-4400-9e7a-48db49b0cba7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:25.635441 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 988/1957] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:25.636368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e4ccb-f0a6-4400-9e7a-48db49b0cba7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:25.659476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e4ccb-f0a6-4400-9e7a-48db49b0cba7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:25.659629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb092d63-24fc-4a09-945b-46fe89e4e2e6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:25.664693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:25.665508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:25.666031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:25.666434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] PATH_INFO: `/v3/projects/3f2e65e42b6f424084b1908581c853f2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:25.667817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754b4b6-c913-494a-a6ca-ae58b9f06ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:25.668146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f2e65e42b6f424084b1908581c853f2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:25.668423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754b4b6-c913-494a-a6ca-ae58b9f06ec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:25.668716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754b4b6-c913-494a-a6ca-ae58b9f06ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:25.669014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754b4b6-c913-494a-a6ca-ae58b9f06ec8 admin admin] PATH_INFO: `/v3/projects/177c55cd1fa34ac88cdd3e787ca97621/users/4fcdc60b97c14a8f9c0bddac59d5fe95/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:25.671323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:25.704930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754b4b6-c913-494a-a6ca-ae58b9f06ec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=177c55cd1fa34ac88cdd3e787ca97621, user_id=4fcdc60b97c14a8f9c0bddac59d5fe95, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:25.718035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754b4b6-c913-494a-a6ca-ae58b9f06ec8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:25.743513 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 989/1958] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:25 2022] PUT /identity/v3/projects/177c55cd1fa34ac88cdd3e787ca97621/users/4fcdc60b97c14a8f9c0bddac59d5fe95/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:25.748793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754b4b6-c913-494a-a6ca-ae58b9f06ec8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:25.785207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f2e65e42b6f424084b1908581c853f2', 'request_id': 'req-4aa01199-0de3-4890-9a6a-f7011acaf7ab'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:25.813954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f2e65e42b6f424084b1908581c853f2', 'request_id': 'req-4aa01199-0de3-4890-9a6a-f7011acaf7ab'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:25.835143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f2e65e42b6f424084b1908581c853f2', 'request_id': 'req-4aa01199-0de3-4890-9a6a-f7011acaf7ab'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:25.857964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] The token cache is being invalidate because project 3f2e65e42b6f424084b1908581c853f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:25.858216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:25.860156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:25.865085 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 970/1959] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:25 2022] DELETE /identity/v3/projects/3f2e65e42b6f424084b1908581c853f2 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:25.870931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:25.871791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:25.918544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754b4b6-c913-494a-a6ca-ae58b9f06ec8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:25.938302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:26.018800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa01199-0de3-4890-9a6a-f7011acaf7ab admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:26.023914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:26.024753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:26.025216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:26.026068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] PATH_INFO: `/v3/users/36cff37f33cc458e97c47a9b047b7081` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:26.027429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36cff37f33cc458e97c47a9b047b7081)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:26.030609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:26.043081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:26.075702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735857a-bc0b-4313-a2b6-6cfd6559363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:26.076947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735857a-bc0b-4313-a2b6-6cfd6559363d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:26.077241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735857a-bc0b-4313-a2b6-6cfd6559363d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:26.077799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735857a-bc0b-4313-a2b6-6cfd6559363d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:26.078464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735857a-bc0b-4313-a2b6-6cfd6559363d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:26.081089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735857a-bc0b-4313-a2b6-6cfd6559363d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:26.101862 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 990/1960] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:26.106626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735857a-bc0b-4313-a2b6-6cfd6559363d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:26.110296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5735857a-bc0b-4313-a2b6-6cfd6559363d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:26.128536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36cff37f33cc458e97c47a9b047b7081'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:26.132840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735857a-bc0b-4313-a2b6-6cfd6559363d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:26.142172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496c9b7-56e5-4f80-996b-bcce72d70606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:26.143757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496c9b7-56e5-4f80-996b-bcce72d70606 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:26.144052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496c9b7-56e5-4f80-996b-bcce72d70606 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:26.144335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496c9b7-56e5-4f80-996b-bcce72d70606 admin admin] PATH_INFO: `/v3/projects/177c55cd1fa34ac88cdd3e787ca97621/users/4fcdc60b97c14a8f9c0bddac59d5fe95/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:26.144690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36cff37f33cc458e97c47a9b047b7081'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:26.146107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36cff37f33cc458e97c47a9b047b7081'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:26.146379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496c9b7-56e5-4f80-996b-bcce72d70606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=177c55cd1fa34ac88cdd3e787ca97621, user_id=4fcdc60b97c14a8f9c0bddac59d5fe95, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:26.150752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496c9b7-56e5-4f80-996b-bcce72d70606 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:26.157120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36cff37f33cc458e97c47a9b047b7081'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:26.177671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36cff37f33cc458e97c47a9b047b7081'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:26.178094 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 991/1961] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:26 2022] PUT /identity/v3/projects/177c55cd1fa34ac88cdd3e787ca97621/users/4fcdc60b97c14a8f9c0bddac59d5fe95/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:26.188316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb712ae-9805-4d62-9fc4-f3e5ab0410dd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:26.188968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb712ae-9805-4d62-9fc4-f3e5ab0410dd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:26.190442 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 971/1962] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:25 2022] DELETE /identity/v3/users/36cff37f33cc458e97c47a9b047b7081 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:26.193944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb712ae-9805-4d62-9fc4-f3e5ab0410dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:26.244384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-abb712ae-9805-4d62-9fc4-f3e5ab0410dd None None] MFA Rules not processed for user `4fcdc60b97c14a8f9c0bddac59d5fe95`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:26.246594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-abb712ae-9805-4d62-9fc4-f3e5ab0410dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:26.371529 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 992/1963] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:26.385586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:26.388442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:26.519466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d204efc-48dd-4e18-87c7-cad98e7bc506 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:26.526720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a91b01-f35c-4aac-a93c-42fd67cc1dcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:26.527392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a91b01-f35c-4aac-a93c-42fd67cc1dcd service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:26.527679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a91b01-f35c-4aac-a93c-42fd67cc1dcd service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:26.527960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a91b01-f35c-4aac-a93c-42fd67cc1dcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:26.538514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a91b01-f35c-4aac-a93c-42fd67cc1dcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:26.540214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a91b01-f35c-4aac-a93c-42fd67cc1dcd service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:26.553513 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 972/1964] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:26 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:26.627762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb712ae-9805-4d62-9fc4-f3e5ab0410dd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:26.630056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-abb712ae-9805-4d62-9fc4-f3e5ab0410dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:26.756078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb712ae-9805-4d62-9fc4-f3e5ab0410dd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:26.762023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e081968-e809-46e4-b2b3-487ecb679e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:26.763071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e081968-e809-46e4-b2b3-487ecb679e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:26.763684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e081968-e809-46e4-b2b3-487ecb679e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:26.764172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e081968-e809-46e4-b2b3-487ecb679e56 admin admin] PATH_INFO: `/v3/projects/01257be8071444c089e63b0614a07130` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:26.765889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e081968-e809-46e4-b2b3-487ecb679e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01257be8071444c089e63b0614a07130)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:26.769002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e081968-e809-46e4-b2b3-487ecb679e56 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:26.831593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926794f-1b60-4de9-b543-6b87e732e66f None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:26.831879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926794f-1b60-4de9-b543-6b87e732e66f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:26.832160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926794f-1b60-4de9-b543-6b87e732e66f None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:26.835090 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 973/1965] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:49:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:26.843697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfb9f0-0a83-4393-9370-70a20d587ad8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:26.844072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfb9f0-0a83-4393-9370-70a20d587ad8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:26.844270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfb9f0-0a83-4393-9370-70a20d587ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:26.861262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e081968-e809-46e4-b2b3-487ecb679e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01257be8071444c089e63b0614a07130', 'request_id': 'req-5e081968-e809-46e4-b2b3-487ecb679e56'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:26.886846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-99dfb9f0-0a83-4393-9370-70a20d587ad8 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:26.890942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-99dfb9f0-0a83-4393-9370-70a20d587ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:26.892373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e081968-e809-46e4-b2b3-487ecb679e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01257be8071444c089e63b0614a07130', 'request_id': 'req-5e081968-e809-46e4-b2b3-487ecb679e56'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:26.902881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e081968-e809-46e4-b2b3-487ecb679e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01257be8071444c089e63b0614a07130', 'request_id': 'req-5e081968-e809-46e4-b2b3-487ecb679e56'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:26.907735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:26.920297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:26.938463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e081968-e809-46e4-b2b3-487ecb679e56 admin admin] The token cache is being invalidate because project 01257be8071444c089e63b0614a07130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:26.938674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e081968-e809-46e4-b2b3-487ecb679e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:26.939249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e081968-e809-46e4-b2b3-487ecb679e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:26.944935 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 993/1966] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:26 2022] DELETE /identity/v3/projects/01257be8071444c089e63b0614a07130 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:26.946422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdcf2c-2dac-448b-b135-de65b2e77603 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:26.946716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdcf2c-2dac-448b-b135-de65b2e77603 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:26.948187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdcf2c-2dac-448b-b135-de65b2e77603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.000404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1abdcf2c-2dac-448b-b135-de65b2e77603 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:27.003851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1abdcf2c-2dac-448b-b135-de65b2e77603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:27.015545 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 974/1967] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:27.062534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfb9f0-0a83-4393-9370-70a20d587ad8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.064533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-99dfb9f0-0a83-4393-9370-70a20d587ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:27.084466 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 994/1968] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:27.087968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfb9f0-0a83-4393-9370-70a20d587ad8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:27.092279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594fd74-c284-4eac-8055-edeb5c4b1cf2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.092577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594fd74-c284-4eac-8055-edeb5c4b1cf2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.092862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594fd74-c284-4eac-8055-edeb5c4b1cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.102228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aac6c3-0b1a-44d4-a509-c2a5d550b9dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:27.102743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aac6c3-0b1a-44d4-a509-c2a5d550b9dd service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.103017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aac6c3-0b1a-44d4-a509-c2a5d550b9dd service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.103293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aac6c3-0b1a-44d4-a509-c2a5d550b9dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.117957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aac6c3-0b1a-44d4-a509-c2a5d550b9dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:27.122373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aac6c3-0b1a-44d4-a509-c2a5d550b9dd service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:27.133494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c594fd74-c284-4eac-8055-edeb5c4b1cf2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:27.133494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c594fd74-c284-4eac-8055-edeb5c4b1cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:27.133810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 975/1969] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:49:27.143662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aac6c3-0b1a-44d4-a509-c2a5d550b9dd service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.145511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-62aac6c3-0b1a-44d4-a509-c2a5d550b9dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:27.147399 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 995/1970] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:27.151533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594fd74-c284-4eac-8055-edeb5c4b1cf2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.161509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594fd74-c284-4eac-8055-edeb5c4b1cf2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:27.166338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443f3a1-cf87-4494-97f1-92364e4a1f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:27.166940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443f3a1-cf87-4494-97f1-92364e4a1f39 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.167227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443f3a1-cf87-4494-97f1-92364e4a1f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.167805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443f3a1-cf87-4494-97f1-92364e4a1f39 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.168571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443f3a1-cf87-4494-97f1-92364e4a1f39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:27.168772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443f3a1-cf87-4494-97f1-92364e4a1f39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:27.170410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443f3a1-cf87-4494-97f1-92364e4a1f39 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:27.189776 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 996/1971] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:49:27.191261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443f3a1-cf87-4494-97f1-92364e4a1f39 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.204201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443f3a1-cf87-4494-97f1-92364e4a1f39 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:27.209324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a3317-113b-4d88-97ee-e3c8693f465f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:27.210621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a3317-113b-4d88-97ee-e3c8693f465f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.210621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a3317-113b-4d88-97ee-e3c8693f465f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.210621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a3317-113b-4d88-97ee-e3c8693f465f admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.211278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a3317-113b-4d88-97ee-e3c8693f465f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:27.213234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a3317-113b-4d88-97ee-e3c8693f465f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:27.234808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aac6c3-0b1a-44d4-a509-c2a5d550b9dd service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:27.241525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ea797-4037-4cb6-8188-6963c3562f9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:27.242226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ea797-4037-4cb6-8188-6963c3562f9f service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.242575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ea797-4037-4cb6-8188-6963c3562f9f service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.242920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ea797-4037-4cb6-8188-6963c3562f9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.244487 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 997/1972] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:27 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:27.244818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-289ea797-4037-4cb6-8188-6963c3562f9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:27.248478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a3317-113b-4d88-97ee-e3c8693f465f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.319005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a3317-113b-4d88-97ee-e3c8693f465f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:27.319217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ea797-4037-4cb6-8188-6963c3562f9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:27.320844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ea797-4037-4cb6-8188-6963c3562f9f service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:27.323751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e06013-c306-4b64-8b46-b39f52488d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:27.324579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e06013-c306-4b64-8b46-b39f52488d2d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.324875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e06013-c306-4b64-8b46-b39f52488d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.325188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e06013-c306-4b64-8b46-b39f52488d2d admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.326393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e06013-c306-4b64-8b46-b39f52488d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:27.328347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e06013-c306-4b64-8b46-b39f52488d2d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:27.332362 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 976/1973] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:27 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:49:27.355220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ea797-4037-4cb6-8188-6963c3562f9f service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.357091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-289ea797-4037-4cb6-8188-6963c3562f9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:27.359769 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 998/1974] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:27 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:49:27.364015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e06013-c306-4b64-8b46-b39f52488d2d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.377205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e06013-c306-4b64-8b46-b39f52488d2d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:27.378510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ea797-4037-4cb6-8188-6963c3562f9f service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:27.382487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b86693-92f4-4048-8883-a04c617f1729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:27.383444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b86693-92f4-4048-8883-a04c617f1729 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.383867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b86693-92f4-4048-8883-a04c617f1729 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.384163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b86693-92f4-4048-8883-a04c617f1729 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.384574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:27.385002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b86693-92f4-4048-8883-a04c617f1729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:27.385382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.385854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.386427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] PATH_INFO: `/v3/users/9387ac00c53d4d6687289d53f615f5a2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.387076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b86693-92f4-4048-8883-a04c617f1729 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:27.387712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9387ac00c53d4d6687289d53f615f5a2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:27.391765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:27.401141 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 999/1975] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:27.406121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b86693-92f4-4048-8883-a04c617f1729 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.434068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b86693-92f4-4048-8883-a04c617f1729 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:27.446211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fc214-c973-437d-b35f-b850e4e4efa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:27.447132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fc214-c973-437d-b35f-b850e4e4efa8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.447425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fc214-c973-437d-b35f-b850e4e4efa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.447853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fc214-c973-437d-b35f-b850e4e4efa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.448806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fc214-c973-437d-b35f-b850e4e4efa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:27.451016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fc214-c973-437d-b35f-b850e4e4efa8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:27.466390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:27.468459 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1000/1976] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:27.473116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fc214-c973-437d-b35f-b850e4e4efa8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.477470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9387ac00c53d4d6687289d53f615f5a2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:27.488744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fc214-c973-437d-b35f-b850e4e4efa8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:27.498359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f9c2d-2788-46ba-912b-b973ae87eefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:27.499544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f9c2d-2788-46ba-912b-b973ae87eefd admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.500447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f9c2d-2788-46ba-912b-b973ae87eefd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.500835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f9c2d-2788-46ba-912b-b973ae87eefd admin admin] PATH_INFO: `/v3/projects/4196dbc6c42746ffab45efab01240e1e/users/627bcee7d4d244cf8af381923a0d906b/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.504599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9387ac00c53d4d6687289d53f615f5a2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:27.505502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9387ac00c53d4d6687289d53f615f5a2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:27.514569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9387ac00c53d4d6687289d53f615f5a2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:27.523839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:27.528658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9387ac00c53d4d6687289d53f615f5a2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:27.531957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f9c2d-2788-46ba-912b-b973ae87eefd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4196dbc6c42746ffab45efab01240e1e, user_id=627bcee7d4d244cf8af381923a0d906b, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:27.537871 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 977/1977] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:27 2022] DELETE /identity/v3/users/9387ac00c53d4d6687289d53f615f5a2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:27.538998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.540521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:27.541076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f9c2d-2788-46ba-912b-b973ae87eefd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:27.566678 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1001/1978] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:27 2022] PUT /identity/v3/projects/4196dbc6c42746ffab45efab01240e1e/users/627bcee7d4d244cf8af381923a0d906b/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:27.574830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2a38d-3dde-4309-ac16-0886ba69ea85 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.575927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2a38d-3dde-4309-ac16-0886ba69ea85 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.576414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2a38d-3dde-4309-ac16-0886ba69ea85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.598727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c3d2a38d-3dde-4309-ac16-0886ba69ea85 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:27.600095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c3d2a38d-3dde-4309-ac16-0886ba69ea85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:27.653876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5387328-f8e4-4a8e-a6fb-3d9591319f68 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:27.666162 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1002/1979] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:27.667726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4277cf-959a-4d8b-a929-e826ab3de756 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.668090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4277cf-959a-4d8b-a929-e826ab3de756 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.668443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4277cf-959a-4d8b-a929-e826ab3de756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.701832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ec4277cf-959a-4d8b-a929-e826ab3de756 None None] MFA Rules not processed for user `627bcee7d4d244cf8af381923a0d906b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:27.703107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ec4277cf-959a-4d8b-a929-e826ab3de756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:27.734108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7e996-9fb2-4b8d-b643-254bd47e5ccc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:27.734860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7e996-9fb2-4b8d-b643-254bd47e5ccc service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.735126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7e996-9fb2-4b8d-b643-254bd47e5ccc service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.735308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7e996-9fb2-4b8d-b643-254bd47e5ccc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.737123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d6e7e996-9fb2-4b8d-b643-254bd47e5ccc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:27.779476 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1003/1980] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:27.781926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4277cf-959a-4d8b-a929-e826ab3de756 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.784594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ec4277cf-959a-4d8b-a929-e826ab3de756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:27.794717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:27.809363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7e996-9fb2-4b8d-b643-254bd47e5ccc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:27.811811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7e996-9fb2-4b8d-b643-254bd47e5ccc service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:27.828273 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 978/1981] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:27.829473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7e996-9fb2-4b8d-b643-254bd47e5ccc service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.830962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d6e7e996-9fb2-4b8d-b643-254bd47e5ccc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:27.855105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7e996-9fb2-4b8d-b643-254bd47e5ccc service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:27.858590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4277cf-959a-4d8b-a929-e826ab3de756 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:27.862764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538378aa-5804-4c9c-aa63-2abc1a8cf9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:27.863968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538378aa-5804-4c9c-aa63-2abc1a8cf9b5 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.864328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538378aa-5804-4c9c-aa63-2abc1a8cf9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.864964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bd1f-bc3b-4927-b4cf-0053278be162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:27.865105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538378aa-5804-4c9c-aa63-2abc1a8cf9b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.865801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538378aa-5804-4c9c-aa63-2abc1a8cf9b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:27.866335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bd1f-bc3b-4927-b4cf-0053278be162 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:27.866686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bd1f-bc3b-4927-b4cf-0053278be162 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:27.867003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bd1f-bc3b-4927-b4cf-0053278be162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:27.868642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538378aa-5804-4c9c-aa63-2abc1a8cf9b5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:27.881263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52bd1f-bc3b-4927-b4cf-0053278be162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:27.884302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52bd1f-bc3b-4927-b4cf-0053278be162 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:27.909499 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 979/1982] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:27 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:27.913728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538378aa-5804-4c9c-aa63-2abc1a8cf9b5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:27.922868 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1004/1983] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:28.000482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca15f1-4f53-47aa-97ab-4786b3d834aa None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:28.001109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca15f1-4f53-47aa-97ab-4786b3d834aa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:28.001576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca15f1-4f53-47aa-97ab-4786b3d834aa None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:28.003184 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1005/1984] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:28.009293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538378aa-5804-4c9c-aa63-2abc1a8cf9b5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:28.012351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96e9e7-ef4f-4119-b2de-78684b122ba1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:28.013266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96e9e7-ef4f-4119-b2de-78684b122ba1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:28.013612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96e9e7-ef4f-4119-b2de-78684b122ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:28.019504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb40326-9356-47cd-9810-bac63b41e342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:28.020418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb40326-9356-47cd-9810-bac63b41e342 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:28.021101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb40326-9356-47cd-9810-bac63b41e342 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:28.021631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb40326-9356-47cd-9810-bac63b41e342 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:28.023986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb40326-9356-47cd-9810-bac63b41e342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:28.026028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb40326-9356-47cd-9810-bac63b41e342 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:28.036883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-be96e9e7-ef4f-4119-b2de-78684b122ba1 None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:28.038754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-be96e9e7-ef4f-4119-b2de-78684b122ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:28.059476 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 980/1985] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:27 2022] POST /identity/v3/users => generated 298 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:28.064514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb40326-9356-47cd-9810-bac63b41e342 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:28.069366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:28.100614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb40326-9356-47cd-9810-bac63b41e342 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:28.110365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42a858-44a7-4c71-9418-2f2d8a888c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:28.111022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42a858-44a7-4c71-9418-2f2d8a888c14 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:28.111308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42a858-44a7-4c71-9418-2f2d8a888c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:28.111589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42a858-44a7-4c71-9418-2f2d8a888c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:28.112872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42a858-44a7-4c71-9418-2f2d8a888c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:28.114886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42a858-44a7-4c71-9418-2f2d8a888c14 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:28.133553 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 981/1986] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:28.134524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42a858-44a7-4c71-9418-2f2d8a888c14 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:28.149066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42a858-44a7-4c71-9418-2f2d8a888c14 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:28.155132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a70c4-24b5-4af4-83b0-24768bbf05a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:28.155665 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1006/1987] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:49:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:28.155964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a70c4-24b5-4af4-83b0-24768bbf05a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:28.156324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a70c4-24b5-4af4-83b0-24768bbf05a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:28.156692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a70c4-24b5-4af4-83b0-24768bbf05a7 admin admin] PATH_INFO: `/v3/projects/c6d57a0da6584f0ea47cf6fe296dd28c/users/29023b1313fb4f3cb1a6a7967a05127e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:28.174774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a70c4-24b5-4af4-83b0-24768bbf05a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d57a0da6584f0ea47cf6fe296dd28c, user_id=29023b1313fb4f3cb1a6a7967a05127e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:28.177854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a70c4-24b5-4af4-83b0-24768bbf05a7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:28.182983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96e9e7-ef4f-4119-b2de-78684b122ba1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:28.185399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-be96e9e7-ef4f-4119-b2de-78684b122ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:28.192976 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 982/1988] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:28 2022] PUT /identity/v3/projects/c6d57a0da6584f0ea47cf6fe296dd28c/users/29023b1313fb4f3cb1a6a7967a05127e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:28.197758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a70c4-24b5-4af4-83b0-24768bbf05a7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:28.303463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a70c4-24b5-4af4-83b0-24768bbf05a7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:28.326529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65116cdb-f816-4871-990b-ab8ef3062789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:28.327172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65116cdb-f816-4871-990b-ab8ef3062789 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:28.327459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65116cdb-f816-4871-990b-ab8ef3062789 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:28.327739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65116cdb-f816-4871-990b-ab8ef3062789 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:28.328494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65116cdb-f816-4871-990b-ab8ef3062789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:28.330229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65116cdb-f816-4871-990b-ab8ef3062789 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:28.335727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96e9e7-ef4f-4119-b2de-78684b122ba1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:28.345829 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 983/1989] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:28.346707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65116cdb-f816-4871-990b-ab8ef3062789 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:28.348761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-65116cdb-f816-4871-990b-ab8ef3062789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:28.371896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:28.426540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed1215-4ada-4b68-bf3a-0665863d5075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:28.428244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed1215-4ada-4b68-bf3a-0665863d5075 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:28.428653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed1215-4ada-4b68-bf3a-0665863d5075 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:28.429057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65116cdb-f816-4871-990b-ab8ef3062789 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:28.433349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c9e3c-a0a9-41ab-8ca6-1fcad810d5a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:28.434219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c9e3c-a0a9-41ab-8ca6-1fcad810d5a9 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:28.434805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c9e3c-a0a9-41ab-8ca6-1fcad810d5a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:28.435730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c9e3c-a0a9-41ab-8ca6-1fcad810d5a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:28.441439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed1215-4ada-4b68-bf3a-0665863d5075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:28.445586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-93ed1215-4ada-4b68-bf3a-0665863d5075 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:28.446077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c9e3c-a0a9-41ab-8ca6-1fcad810d5a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:28.448407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c9e3c-a0a9-41ab-8ca6-1fcad810d5a9 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:28.460650 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 984/1990] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:28.461838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c9e3c-a0a9-41ab-8ca6-1fcad810d5a9 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:28.475524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c9e3c-a0a9-41ab-8ca6-1fcad810d5a9 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:28.482683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d06154-1509-42a4-8e63-5f2259909efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:28.483725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d06154-1509-42a4-8e63-5f2259909efe admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:28.484484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d06154-1509-42a4-8e63-5f2259909efe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:28.485094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d06154-1509-42a4-8e63-5f2259909efe admin admin] PATH_INFO: `/v3/projects/c6d57a0da6584f0ea47cf6fe296dd28c/users/29023b1313fb4f3cb1a6a7967a05127e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:28.487799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d06154-1509-42a4-8e63-5f2259909efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d57a0da6584f0ea47cf6fe296dd28c, user_id=29023b1313fb4f3cb1a6a7967a05127e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:28.491267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d06154-1509-42a4-8e63-5f2259909efe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:28.515995 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 985/1991] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:28 2022] PUT /identity/v3/projects/c6d57a0da6584f0ea47cf6fe296dd28c/users/29023b1313fb4f3cb1a6a7967a05127e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:28.518734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b79327-c765-4319-8180-67cdf606fb00 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:28.519042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b79327-c765-4319-8180-67cdf606fb00 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:28.519330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b79327-c765-4319-8180-67cdf606fb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:28.577456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-07b79327-c765-4319-8180-67cdf606fb00 None None] MFA Rules not processed for user `29023b1313fb4f3cb1a6a7967a05127e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:28.578914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-07b79327-c765-4319-8180-67cdf606fb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:28.619360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed1215-4ada-4b68-bf3a-0665863d5075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:28.634449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed1215-4ada-4b68-bf3a-0665863d5075 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:28.651880 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1007/1992] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:49:28.715236 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 986/1993] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:28.808794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed1215-4ada-4b68-bf3a-0665863d5075 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:28.810281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-93ed1215-4ada-4b68-bf3a-0665863d5075 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:28.826430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:28.834028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed1215-4ada-4b68-bf3a-0665863d5075 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:28.885450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b79327-c765-4319-8180-67cdf606fb00 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:28.931714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:28.932363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:28.932754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:28.933084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] PATH_INFO: `/v3/projects/4aa79c05b5a7408c9748ec9d3dfc4283` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:28.934236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aa79c05b5a7408c9748ec9d3dfc4283)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:28.936447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:28.997793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b79327-c765-4319-8180-67cdf606fb00 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:29.014889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aa79c05b5a7408c9748ec9d3dfc4283', 'request_id': 'req-41542846-ef6a-4d3a-b1e2-f4deb8758e70'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.027981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aa79c05b5a7408c9748ec9d3dfc4283', 'request_id': 'req-41542846-ef6a-4d3a-b1e2-f4deb8758e70'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.040580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aa79c05b5a7408c9748ec9d3dfc4283', 'request_id': 'req-41542846-ef6a-4d3a-b1e2-f4deb8758e70'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.052636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:29.066957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] The token cache is being invalidate because project 4aa79c05b5a7408c9748ec9d3dfc4283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:29.067565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.069331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.072513 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1008/1994] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:28 2022] DELETE /identity/v3/projects/4aa79c05b5a7408c9748ec9d3dfc4283 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:29.077216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:29.079010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:29.109571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3b42f-9ada-465e-bc23-1a2e884ac27b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:29.110909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3b42f-9ada-465e-bc23-1a2e884ac27b service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:29.111353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3b42f-9ada-465e-bc23-1a2e884ac27b service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:29.111788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3b42f-9ada-465e-bc23-1a2e884ac27b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:29.115019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-acf3b42f-9ada-465e-bc23-1a2e884ac27b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:29.208823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41542846-ef6a-4d3a-b1e2-f4deb8758e70 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:29.219886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6459d-fe5f-4516-b077-8ffa79e08185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:29.220855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6459d-fe5f-4516-b077-8ffa79e08185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:29.221489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6459d-fe5f-4516-b077-8ffa79e08185 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:29.221846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6459d-fe5f-4516-b077-8ffa79e08185 admin admin] PATH_INFO: `/v3/users/4fcdc60b97c14a8f9c0bddac59d5fe95` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:29.223426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6459d-fe5f-4516-b077-8ffa79e08185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fcdc60b97c14a8f9c0bddac59d5fe95)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:29.225727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6459d-fe5f-4516-b077-8ffa79e08185 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:29.236776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3b42f-9ada-465e-bc23-1a2e884ac27b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:29.238444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3b42f-9ada-465e-bc23-1a2e884ac27b service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:29.257202 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 987/1995] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:49:29.259334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e35482-83d0-4a8e-a9b4-a3a901dde964 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:29.259764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e35482-83d0-4a8e-a9b4-a3a901dde964 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:29.260111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e35482-83d0-4a8e-a9b4-a3a901dde964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:29.284764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:29.299144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1dc6459d-fe5f-4516-b077-8ffa79e08185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fcdc60b97c14a8f9c0bddac59d5fe95'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.307704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c8e35482-83d0-4a8e-a9b4-a3a901dde964 None None] MFA Rules not processed for user `4c3a73b54fd54a23808657c7ed9e62cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:29.309669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c8e35482-83d0-4a8e-a9b4-a3a901dde964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:29.319163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1dc6459d-fe5f-4516-b077-8ffa79e08185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fcdc60b97c14a8f9c0bddac59d5fe95'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.332821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1dc6459d-fe5f-4516-b077-8ffa79e08185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fcdc60b97c14a8f9c0bddac59d5fe95'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.333579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1dc6459d-fe5f-4516-b077-8ffa79e08185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fcdc60b97c14a8f9c0bddac59d5fe95'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.334703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1dc6459d-fe5f-4516-b077-8ffa79e08185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fcdc60b97c14a8f9c0bddac59d5fe95'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.356718 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1009/1996] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:29 2022] DELETE /identity/v3/users/4fcdc60b97c14a8f9c0bddac59d5fe95 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:29.361988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83803a-3cbe-464c-803c-2c87a6712c39 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:29.362292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83803a-3cbe-464c-803c-2c87a6712c39 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:29.362576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83803a-3cbe-464c-803c-2c87a6712c39 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:29.363871 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1010/1997] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:29.371147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83803a-3cbe-464c-803c-2c87a6712c39 None None] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:29.373313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1e83803a-3cbe-464c-803c-2c87a6712c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:29.399129 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 988/1998] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:29.462073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83803a-3cbe-464c-803c-2c87a6712c39 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:29.464233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1e83803a-3cbe-464c-803c-2c87a6712c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:29.577040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83803a-3cbe-464c-803c-2c87a6712c39 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:29.582998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c25da-13ad-4f03-b232-54eeba1ed4fe tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29023b1313fb4f3cb1a6a7967a05127e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d57a0da6584f0ea47cf6fe296dd28c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:29.583835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c25da-13ad-4f03-b232-54eeba1ed4fe tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:29.584282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c25da-13ad-4f03-b232-54eeba1ed4fe tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:29.585017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c25da-13ad-4f03-b232-54eeba1ed4fe tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] PATH_INFO: `/v3/projects/4196dbc6c42746ffab45efab01240e1e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:29.586880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c25da-13ad-4f03-b232-54eeba1ed4fe tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorizing `identity:get_project(project_id=4196dbc6c42746ffab45efab01240e1e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:29.589420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c25da-13ad-4f03-b232-54eeba1ed4fe tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:29.592287 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1011/1999] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:29 2022] GET /identity/v3/projects/4196dbc6c42746ffab45efab01240e1e => generated 381 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:29.634984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596f0a0-5bbd-4a95-9eee-3b370fa51996 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:29.635275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596f0a0-5bbd-4a95-9eee-3b370fa51996 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:29.635558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596f0a0-5bbd-4a95-9eee-3b370fa51996 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:29.637232 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 989/2000] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:29.642799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c25da-13ad-4f03-b232-54eeba1ed4fe tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:29.658651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c25da-13ad-4f03-b232-54eeba1ed4fe tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:29.678384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c25da-13ad-4f03-b232-54eeba1ed4fe tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:29.692261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144f9d0-30b8-4702-8876-bfcd71379542 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29023b1313fb4f3cb1a6a7967a05127e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d57a0da6584f0ea47cf6fe296dd28c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:29.692772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144f9d0-30b8-4702-8876-bfcd71379542 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:29.693245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144f9d0-30b8-4702-8876-bfcd71379542 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:29.693596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144f9d0-30b8-4702-8876-bfcd71379542 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] PATH_INFO: `/v3/projects/4196dbc6c42746ffab45efab01240e1e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:29.697511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144f9d0-30b8-4702-8876-bfcd71379542 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorizing `identity:get_project(project_id=4196dbc6c42746ffab45efab01240e1e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:29.697511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144f9d0-30b8-4702-8876-bfcd71379542 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:29.699147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596f0a0-5bbd-4a95-9eee-3b370fa51996 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:29.699276 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1012/2001] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:29 2022] GET /identity/v3/projects/4196dbc6c42746ffab45efab01240e1e => generated 381 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:29.701673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8596f0a0-5bbd-4a95-9eee-3b370fa51996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:29.783231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596f0a0-5bbd-4a95-9eee-3b370fa51996 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:29.789713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979b6af-fe86-44a4-900f-bb4a2a4db77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:29.790542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979b6af-fe86-44a4-900f-bb4a2a4db77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:29.791213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979b6af-fe86-44a4-900f-bb4a2a4db77f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:29.791553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979b6af-fe86-44a4-900f-bb4a2a4db77f admin admin] PATH_INFO: `/v3/projects/177c55cd1fa34ac88cdd3e787ca97621` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:29.793178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979b6af-fe86-44a4-900f-bb4a2a4db77f admin admin] RBAC: Authorizing `identity:delete_project(project_id=177c55cd1fa34ac88cdd3e787ca97621)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:29.794860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979b6af-fe86-44a4-900f-bb4a2a4db77f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:29.807447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:29.821211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74a581-4f1e-41d4-99ed-b300d63d8e26 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:29.822299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74a581-4f1e-41d4-99ed-b300d63d8e26 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:29.822674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74a581-4f1e-41d4-99ed-b300d63d8e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:29.848609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2b74a581-4f1e-41d4-99ed-b300d63d8e26 None None] MFA Rules not processed for user `627bcee7d4d244cf8af381923a0d906b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:29.850145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b74a581-4f1e-41d4-99ed-b300d63d8e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:29.854186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5979b6af-fe86-44a4-900f-bb4a2a4db77f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '177c55cd1fa34ac88cdd3e787ca97621', 'request_id': 'req-5979b6af-fe86-44a4-900f-bb4a2a4db77f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.868348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5979b6af-fe86-44a4-900f-bb4a2a4db77f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '177c55cd1fa34ac88cdd3e787ca97621', 'request_id': 'req-5979b6af-fe86-44a4-900f-bb4a2a4db77f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.879128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5979b6af-fe86-44a4-900f-bb4a2a4db77f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '177c55cd1fa34ac88cdd3e787ca97621', 'request_id': 'req-5979b6af-fe86-44a4-900f-bb4a2a4db77f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.902131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5979b6af-fe86-44a4-900f-bb4a2a4db77f admin admin] The token cache is being invalidate because project 177c55cd1fa34ac88cdd3e787ca97621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:29.902443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5979b6af-fe86-44a4-900f-bb4a2a4db77f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.903219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5979b6af-fe86-44a4-900f-bb4a2a4db77f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:29.910066 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 990/2002] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:29 2022] DELETE /identity/v3/projects/177c55cd1fa34ac88cdd3e787ca97621 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:29.911717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fe85b-d799-46ca-bee7-7470708debeb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:29.912018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fe85b-d799-46ca-bee7-7470708debeb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:29.912355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fe85b-d799-46ca-bee7-7470708debeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:29.914405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:29.948279 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1013/2003] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:29.950234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c17fe85b-d799-46ca-bee7-7470708debeb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:29.951553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c17fe85b-d799-46ca-bee7-7470708debeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:30.074605 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 991/2004] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:30.081555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ddd4b-24c0-4688-99a9-b54445184534 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:30.082284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ddd4b-24c0-4688-99a9-b54445184534 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:30.083637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ddd4b-24c0-4688-99a9-b54445184534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:30.107971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fe85b-d799-46ca-bee7-7470708debeb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:30.110208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c17fe85b-d799-46ca-bee7-7470708debeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:30.145480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-eb2ddd4b-24c0-4688-99a9-b54445184534 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:30.146586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-eb2ddd4b-24c0-4688-99a9-b54445184534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:30.174339 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1014/2005] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:30.179040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ddd4b-24c0-4688-99a9-b54445184534 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:30.198203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ddd4b-24c0-4688-99a9-b54445184534 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:30.214254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fab3d-7752-4600-ac0c-c7b51f0f554c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:30.216064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fab3d-7752-4600-ac0c-c7b51f0f554c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:30.216612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fab3d-7752-4600-ac0c-c7b51f0f554c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:30.217130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fab3d-7752-4600-ac0c-c7b51f0f554c admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:30.219102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fab3d-7752-4600-ac0c-c7b51f0f554c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:30.221012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fab3d-7752-4600-ac0c-c7b51f0f554c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:30.223289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fab3d-7752-4600-ac0c-c7b51f0f554c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:30.248431 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1015/2006] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:30.256740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fab3d-7752-4600-ac0c-c7b51f0f554c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:30.284429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fab3d-7752-4600-ac0c-c7b51f0f554c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:30.293873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fe85b-d799-46ca-bee7-7470708debeb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:30.297285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7d02c-d7ee-46b2-ba30-94f84142270d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:30.297919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7d02c-d7ee-46b2-ba30-94f84142270d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:30.298654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7d02c-d7ee-46b2-ba30-94f84142270d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:30.298654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7d02c-d7ee-46b2-ba30-94f84142270d admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:30.299129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7d02c-d7ee-46b2-ba30-94f84142270d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:30.304941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7d02c-d7ee-46b2-ba30-94f84142270d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:30.309258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4d14c-acdb-496f-84e8-1452393d1559 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:30.309950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d14c-acdb-496f-84e8-1452393d1559 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:30.310239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d14c-acdb-496f-84e8-1452393d1559 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:30.310519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d14c-acdb-496f-84e8-1452393d1559 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:30.314967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-aec4d14c-acdb-496f-84e8-1452393d1559 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:30.339608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4d14c-acdb-496f-84e8-1452393d1559 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:30.343026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4d14c-acdb-496f-84e8-1452393d1559 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:30.362978 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 992/2007] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:30.389744 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1016/2008] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:30 2022] POST /identity/v3/projects => generated 413 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:30.390547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4d14c-acdb-496f-84e8-1452393d1559 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:30.411667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4d14c-acdb-496f-84e8-1452393d1559 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:30.470687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:30.567788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07accf6b-b01e-4a03-945f-f320d563d85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:30.568443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07accf6b-b01e-4a03-945f-f320d563d85b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:30.570941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07accf6b-b01e-4a03-945f-f320d563d85b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:30.571383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07accf6b-b01e-4a03-945f-f320d563d85b admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:30.573597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07accf6b-b01e-4a03-945f-f320d563d85b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:30.579440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07accf6b-b01e-4a03-945f-f320d563d85b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:30.624686 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 993/2009] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:30 2022] POST /identity/v3/users => generated 307 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:49:30.626186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7d02c-d7ee-46b2-ba30-94f84142270d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:30.648693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7d02c-d7ee-46b2-ba30-94f84142270d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:30.667634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de351ba-3184-4020-b2a4-e0ad0ef72e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:30.674968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de351ba-3184-4020-b2a4-e0ad0ef72e96 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:30.675288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de351ba-3184-4020-b2a4-e0ad0ef72e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:30.677142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de351ba-3184-4020-b2a4-e0ad0ef72e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:30.677875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de351ba-3184-4020-b2a4-e0ad0ef72e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:30.682552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de351ba-3184-4020-b2a4-e0ad0ef72e96 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:30.709223 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1017/2010] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:30.713720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07accf6b-b01e-4a03-945f-f320d563d85b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:30.734561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07accf6b-b01e-4a03-945f-f320d563d85b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:30.746149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44f768-0209-43e9-8e1c-0253658a49df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:30.747508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44f768-0209-43e9-8e1c-0253658a49df admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:30.747625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44f768-0209-43e9-8e1c-0253658a49df admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:30.747905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44f768-0209-43e9-8e1c-0253658a49df admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:30.748615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44f768-0209-43e9-8e1c-0253658a49df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:30.752999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44f768-0209-43e9-8e1c-0253658a49df admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:30.790800 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 994/2011] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:30.794685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de351ba-3184-4020-b2a4-e0ad0ef72e96 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:30.818324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de351ba-3184-4020-b2a4-e0ad0ef72e96 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:30.852690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccca5a4-9ff9-46f3-a0c5-f823ed71f997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:30.853425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccca5a4-9ff9-46f3-a0c5-f823ed71f997 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:30.853717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccca5a4-9ff9-46f3-a0c5-f823ed71f997 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:30.853990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccca5a4-9ff9-46f3-a0c5-f823ed71f997 admin admin] PATH_INFO: `/v3/projects/8d0ccd04fad8493fbef0a8afc12e0a38/users/78b60bdc00024f6f9cac34b07b8940ac/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:30.908361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccca5a4-9ff9-46f3-a0c5-f823ed71f997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0ccd04fad8493fbef0a8afc12e0a38, user_id=78b60bdc00024f6f9cac34b07b8940ac, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:30.913727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccca5a4-9ff9-46f3-a0c5-f823ed71f997 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:30.952074 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1018/2012] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:30 2022] PUT /identity/v3/projects/8d0ccd04fad8493fbef0a8afc12e0a38/users/78b60bdc00024f6f9cac34b07b8940ac/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:30.962995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da348374-6618-4fc8-b3f2-5a87c3c1adc5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:30.963292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da348374-6618-4fc8-b3f2-5a87c3c1adc5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:30.963573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da348374-6618-4fc8-b3f2-5a87c3c1adc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:31.004029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-da348374-6618-4fc8-b3f2-5a87c3c1adc5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:31.007173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-da348374-6618-4fc8-b3f2-5a87c3c1adc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:31.144805 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 995/2013] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:31.155001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccca5a4-9ff9-46f3-a0c5-f823ed71f997 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:31.157072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bccca5a4-9ff9-46f3-a0c5-f823ed71f997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:31.230333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccca5a4-9ff9-46f3-a0c5-f823ed71f997 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:31.243144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4300233-c7b7-494b-a711-788da511d18e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:31.244199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4300233-c7b7-494b-a711-788da511d18e service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:31.244863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4300233-c7b7-494b-a711-788da511d18e service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:31.245275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4300233-c7b7-494b-a711-788da511d18e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:31.269326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4300233-c7b7-494b-a711-788da511d18e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:31.279055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4300233-c7b7-494b-a711-788da511d18e service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:31.300365 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1019/2014] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:31.356401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93359a-ade8-4b12-9a3e-39cc76efc98b None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:31.357323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93359a-ade8-4b12-9a3e-39cc76efc98b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:31.357776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93359a-ade8-4b12-9a3e-39cc76efc98b None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:31.359545 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 996/2015] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:31.372026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4300233-c7b7-494b-a711-788da511d18e service neutron] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:31.374086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e4300233-c7b7-494b-a711-788da511d18e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:31.574451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4300233-c7b7-494b-a711-788da511d18e service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:31.576153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e4300233-c7b7-494b-a711-788da511d18e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:31.613462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:31.748052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4300233-c7b7-494b-a711-788da511d18e service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:31.756520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4495f-c8a1-4adc-90b2-41a73ce9e5a0 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29023b1313fb4f3cb1a6a7967a05127e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d57a0da6584f0ea47cf6fe296dd28c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:31.758086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4495f-c8a1-4adc-90b2-41a73ce9e5a0 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:31.759073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4495f-c8a1-4adc-90b2-41a73ce9e5a0 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:31.759774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4495f-c8a1-4adc-90b2-41a73ce9e5a0 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] PATH_INFO: `/v3/projects/4196dbc6c42746ffab45efab01240e1e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:31.762041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4495f-c8a1-4adc-90b2-41a73ce9e5a0 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorizing `identity:get_project(project_id=4196dbc6c42746ffab45efab01240e1e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:31.766873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4495f-c8a1-4adc-90b2-41a73ce9e5a0 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:31.769905 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1020/2016] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:31 2022] GET /identity/v3/projects/4196dbc6c42746ffab45efab01240e1e => generated 381 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:32.015497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70234ff0-52d0-4a81-af01-44c2a1001291 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:32.015853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70234ff0-52d0-4a81-af01-44c2a1001291 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:32.016211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70234ff0-52d0-4a81-af01-44c2a1001291 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:32.018251 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 997/2017] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:32.025612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4495f-c8a1-4adc-90b2-41a73ce9e5a0 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:32.046568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4495f-c8a1-4adc-90b2-41a73ce9e5a0 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:32.077408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4495f-c8a1-4adc-90b2-41a73ce9e5a0 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:32.104142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153dc45a-a779-462b-a94c-0b198d3986db tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29023b1313fb4f3cb1a6a7967a05127e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d57a0da6584f0ea47cf6fe296dd28c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:32.107784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153dc45a-a779-462b-a94c-0b198d3986db tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:32.108107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153dc45a-a779-462b-a94c-0b198d3986db tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:32.108397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153dc45a-a779-462b-a94c-0b198d3986db tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] PATH_INFO: `/v3/projects/4196dbc6c42746ffab45efab01240e1e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:32.109888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153dc45a-a779-462b-a94c-0b198d3986db tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorizing `identity:get_project(project_id=4196dbc6c42746ffab45efab01240e1e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:32.114428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153dc45a-a779-462b-a94c-0b198d3986db tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:32.122609 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1021/2018] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:32 2022] GET /identity/v3/projects/4196dbc6c42746ffab45efab01240e1e => generated 381 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:32.170641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25defb-5026-487d-8802-84c0bf5c28b1 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:32.171594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25defb-5026-487d-8802-84c0bf5c28b1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:32.171879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25defb-5026-487d-8802-84c0bf5c28b1 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:32.173277 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 998/2019] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:32.181914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153dc45a-a779-462b-a94c-0b198d3986db tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:32.213189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153dc45a-a779-462b-a94c-0b198d3986db tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:32.228505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153dc45a-a779-462b-a94c-0b198d3986db tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:32.238235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928690-e7c0-4ba7-845d-4cdb4cc47e48 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29023b1313fb4f3cb1a6a7967a05127e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d57a0da6584f0ea47cf6fe296dd28c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:32.239045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928690-e7c0-4ba7-845d-4cdb4cc47e48 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:32.239267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928690-e7c0-4ba7-845d-4cdb4cc47e48 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:32.239428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928690-e7c0-4ba7-845d-4cdb4cc47e48 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] PATH_INFO: `/v3/projects/4196dbc6c42746ffab45efab01240e1e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:32.241940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2928690-e7c0-4ba7-845d-4cdb4cc47e48 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorizing `identity:get_project(project_id=4196dbc6c42746ffab45efab01240e1e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:32.244947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2928690-e7c0-4ba7-845d-4cdb4cc47e48 tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:32.247608 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1022/2020] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:32 2022] GET /identity/v3/projects/4196dbc6c42746ffab45efab01240e1e => generated 381 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:32.406488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2235c50-f22e-46b2-9f35-2d1a692b15f6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:32.407024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2235c50-f22e-46b2-9f35-2d1a692b15f6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:32.407024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2235c50-f22e-46b2-9f35-2d1a692b15f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:32.445145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f2235c50-f22e-46b2-9f35-2d1a692b15f6 None None] MFA Rules not processed for user `627bcee7d4d244cf8af381923a0d906b`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:32.446401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f2235c50-f22e-46b2-9f35-2d1a692b15f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:32.540292 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 999/2021] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:33.371492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a7cc9-dcd4-4923-9d10-5cc8d4335ef4 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:33.372113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a7cc9-dcd4-4923-9d10-5cc8d4335ef4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:33.372450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a7cc9-dcd4-4923-9d10-5cc8d4335ef4 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:33.375550 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1023/2022] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:33.399337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2235c50-f22e-46b2-9f35-2d1a692b15f6 None None] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:33.418188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2235c50-f22e-46b2-9f35-2d1a692b15f6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:33.441723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2235c50-f22e-46b2-9f35-2d1a692b15f6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:33.454375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699042f1-ce8f-4119-be03-6798f526f06d tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29023b1313fb4f3cb1a6a7967a05127e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d57a0da6584f0ea47cf6fe296dd28c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:33.454988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699042f1-ce8f-4119-be03-6798f526f06d tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:33.455276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699042f1-ce8f-4119-be03-6798f526f06d tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:33.455585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699042f1-ce8f-4119-be03-6798f526f06d tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] PATH_INFO: `/v3/projects/4196dbc6c42746ffab45efab01240e1e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:33.465384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699042f1-ce8f-4119-be03-6798f526f06d tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorizing `identity:get_project(project_id=4196dbc6c42746ffab45efab01240e1e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:33.468268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699042f1-ce8f-4119-be03-6798f526f06d tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:33.479116 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1000/2023] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:33 2022] GET /identity/v3/projects/4196dbc6c42746ffab45efab01240e1e => generated 381 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:33.809528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a8209-5253-4d84-80b3-8997f509353c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:33.810141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a8209-5253-4d84-80b3-8997f509353c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:33.810636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a8209-5253-4d84-80b3-8997f509353c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:33.821287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1d9e8-8f36-4bcf-9d2e-fc7bc4559b8d None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:33.821660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1d9e8-8f36-4bcf-9d2e-fc7bc4559b8d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:33.821980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1d9e8-8f36-4bcf-9d2e-fc7bc4559b8d None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:33.823591 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1001/2024] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:49:33.829508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1d9e8-8f36-4bcf-9d2e-fc7bc4559b8d None None] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:33.875456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1d9e8-8f36-4bcf-9d2e-fc7bc4559b8d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:33.884312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-246a8209-5253-4d84-80b3-8997f509353c None None] MFA Rules not processed for user `c34f1d07f48c4b4dadf97fd38709fd33`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:33.889433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-246a8209-5253-4d84-80b3-8997f509353c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:33.905813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1d9e8-8f36-4bcf-9d2e-fc7bc4559b8d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:33.917102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5b06a-33aa-4deb-b33a-8de91bad6a1a tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29023b1313fb4f3cb1a6a7967a05127e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d57a0da6584f0ea47cf6fe296dd28c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:33.919289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b06a-33aa-4deb-b33a-8de91bad6a1a tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:33.919583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b06a-33aa-4deb-b33a-8de91bad6a1a tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:33.919926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b06a-33aa-4deb-b33a-8de91bad6a1a tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] PATH_INFO: `/v3/projects/4196dbc6c42746ffab45efab01240e1e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:33.922522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5b06a-33aa-4deb-b33a-8de91bad6a1a tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorizing `identity:get_project(project_id=4196dbc6c42746ffab45efab01240e1e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:33.928092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5b06a-33aa-4deb-b33a-8de91bad6a1a tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:33.934491 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1002/2025] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:33 2022] GET /identity/v3/projects/4196dbc6c42746ffab45efab01240e1e => generated 381 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:49:34.034545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6602887-e51e-4aa9-8603-193b6aa76c3d None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:34.035148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6602887-e51e-4aa9-8603-193b6aa76c3d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:34.035681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6602887-e51e-4aa9-8603-193b6aa76c3d None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:34.038253 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1003/2026] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:49:34.044838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6602887-e51e-4aa9-8603-193b6aa76c3d None None] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:34.066835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6602887-e51e-4aa9-8603-193b6aa76c3d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:34.083358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6602887-e51e-4aa9-8603-193b6aa76c3d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:34.103912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d316f-8cee-47d6-acc8-f0e21a16651a tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29023b1313fb4f3cb1a6a7967a05127e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d57a0da6584f0ea47cf6fe296dd28c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:34.106325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d316f-8cee-47d6-acc8-f0e21a16651a tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:34.106982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d316f-8cee-47d6-acc8-f0e21a16651a tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:34.108055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d316f-8cee-47d6-acc8-f0e21a16651a tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] PATH_INFO: `/v3/projects/4196dbc6c42746ffab45efab01240e1e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:34.111655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d316f-8cee-47d6-acc8-f0e21a16651a tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorizing `identity:get_project(project_id=4196dbc6c42746ffab45efab01240e1e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:34.118366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d316f-8cee-47d6-acc8-f0e21a16651a tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:34.123456 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1024/2027] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:34.129894 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1004/2028] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:34 2022] GET /identity/v3/projects/4196dbc6c42746ffab45efab01240e1e => generated 381 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:49:34.238422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d28934-853d-4852-9d02-c7bfcd88ea3b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:34.238871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d28934-853d-4852-9d02-c7bfcd88ea3b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:34.239276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d28934-853d-4852-9d02-c7bfcd88ea3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:34.282949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c8d28934-853d-4852-9d02-c7bfcd88ea3b None None] MFA Rules not processed for user `627bcee7d4d244cf8af381923a0d906b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:34.284172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c8d28934-853d-4852-9d02-c7bfcd88ea3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:34.301930 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1025/2029] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:34.475635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eef10-59d3-487a-b7a0-bed0c381b001 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:34.476186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eef10-59d3-487a-b7a0-bed0c381b001 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:34.476186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eef10-59d3-487a-b7a0-bed0c381b001 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:34.478046 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1005/2030] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:34.499561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b14ec-8e0d-4113-bc15-9a2d4018c1fb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:34.500562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b14ec-8e0d-4113-bc15-9a2d4018c1fb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:34.501279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b14ec-8e0d-4113-bc15-9a2d4018c1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:34.518058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78eef10-59d3-487a-b7a0-bed0c381b001 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:34.520243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c78eef10-59d3-487a-b7a0-bed0c381b001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:34.528291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-215b14ec-8e0d-4113-bc15-9a2d4018c1fb None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:34.530318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-215b14ec-8e0d-4113-bc15-9a2d4018c1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:34.549996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78eef10-59d3-487a-b7a0-bed0c381b001 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:34.568367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:34.570057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:34.570354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:34.570636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] PATH_INFO: `/v3/users/45b55868ea8a492182510bfd0133d74f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:34.574828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45b55868ea8a492182510bfd0133d74f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:34.583452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:34.614134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:34.650488 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1026/2031] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:34.672366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b14ec-8e0d-4113-bc15-9a2d4018c1fb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:34.675264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-215b14ec-8e0d-4113-bc15-9a2d4018c1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:34.686022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45b55868ea8a492182510bfd0133d74f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:34.696113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b14ec-8e0d-4113-bc15-9a2d4018c1fb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:34.706887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae998b6-0b28-4515-b004-51234f8063d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:34.708504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae998b6-0b28-4515-b004-51234f8063d2 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:34.709066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae998b6-0b28-4515-b004-51234f8063d2 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:34.709556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae998b6-0b28-4515-b004-51234f8063d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:34.712334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45b55868ea8a492182510bfd0133d74f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:34.713116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45b55868ea8a492182510bfd0133d74f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:34.723968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae998b6-0b28-4515-b004-51234f8063d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:34.728715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae998b6-0b28-4515-b004-51234f8063d2 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:34.728936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45b55868ea8a492182510bfd0133d74f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:34.753720 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1027/2032] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:34.755607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45b55868ea8a492182510bfd0133d74f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:34.767831 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1006/2033] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:34 2022] DELETE /identity/v3/users/45b55868ea8a492182510bfd0133d74f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:35.310188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae998b6-0b28-4515-b004-51234f8063d2 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:35.332628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae998b6-0b28-4515-b004-51234f8063d2 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:35.370330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:35.477679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:35.479463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:35.481262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:35.481759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8 admin admin] PATH_INFO: `/v3/projects/913c519428fd49a7a6f098b4112afc1a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:35.485031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=913c519428fd49a7a6f098b4112afc1a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:35.491128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:35.543826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:35.545988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:35.569929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34559b-8ed0-402c-b7a8-10868d27be35 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:35.586338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:35.589316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:35.590394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:35.591706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] PATH_INFO: `/v3/users/c0f18ad3c7a64133bd7c5048a1fc566d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:35.594875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0f18ad3c7a64133bd7c5048a1fc566d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:35.602601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:35.638910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913c519428fd49a7a6f098b4112afc1a', 'request_id': 'req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:35.658775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913c519428fd49a7a6f098b4112afc1a', 'request_id': 'req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:35.682659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913c519428fd49a7a6f098b4112afc1a', 'request_id': 'req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:35.726706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8 admin admin] The token cache is being invalidate because project 913c519428fd49a7a6f098b4112afc1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:35.728997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:35.731292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e3cb4e2f-d999-4098-b5e0-30451aa0b3b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:35.737692 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1028/2034] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:35 2022] DELETE /identity/v3/projects/913c519428fd49a7a6f098b4112afc1a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:35.739781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dea86a-ef50-4d8c-8c5d-bd9625654efc None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:35.740087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dea86a-ef50-4d8c-8c5d-bd9625654efc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:35.742350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dea86a-ef50-4d8c-8c5d-bd9625654efc None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:35.746871 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1029/2035] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:35.749852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dea86a-ef50-4d8c-8c5d-bd9625654efc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:35.751337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0f18ad3c7a64133bd7c5048a1fc566d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:35.756300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c0dea86a-ef50-4d8c-8c5d-bd9625654efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:35.820454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0f18ad3c7a64133bd7c5048a1fc566d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:35.822087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0f18ad3c7a64133bd7c5048a1fc566d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:35.845171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0f18ad3c7a64133bd7c5048a1fc566d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:35.870173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0f18ad3c7a64133bd7c5048a1fc566d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:35.891094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dea86a-ef50-4d8c-8c5d-bd9625654efc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:35.900874 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1007/2036] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:35 2022] DELETE /identity/v3/users/c0f18ad3c7a64133bd7c5048a1fc566d => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:35.903038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:49:35.906619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:35.993245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:36.023487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095276ef-a180-44cd-8fbc-adecc42b3868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:36.024480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095276ef-a180-44cd-8fbc-adecc42b3868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:36.024772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095276ef-a180-44cd-8fbc-adecc42b3868 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:36.025513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095276ef-a180-44cd-8fbc-adecc42b3868 admin admin] PATH_INFO: `/v3/users/928517e539f24ab2b0f1bd3033051a00` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:36.027834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095276ef-a180-44cd-8fbc-adecc42b3868 admin admin] RBAC: Authorizing `identity:delete_user(user_id=928517e539f24ab2b0f1bd3033051a00)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:36.031053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095276ef-a180-44cd-8fbc-adecc42b3868 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:36.125064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-095276ef-a180-44cd-8fbc-adecc42b3868 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '928517e539f24ab2b0f1bd3033051a00'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:36.149539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-095276ef-a180-44cd-8fbc-adecc42b3868 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '928517e539f24ab2b0f1bd3033051a00'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:36.168400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:36.189439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-095276ef-a180-44cd-8fbc-adecc42b3868 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '928517e539f24ab2b0f1bd3033051a00'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:36.191266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:36.191400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-095276ef-a180-44cd-8fbc-adecc42b3868 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '928517e539f24ab2b0f1bd3033051a00'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:36.192476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-095276ef-a180-44cd-8fbc-adecc42b3868 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '928517e539f24ab2b0f1bd3033051a00'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:36.194000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:36.223200 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1030/2037] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:35 2022] DELETE /identity/v3/users/928517e539f24ab2b0f1bd3033051a00 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:36.224859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b107ad8-09a6-4b7b-889e-1064de229fd0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:36.225283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b107ad8-09a6-4b7b-889e-1064de229fd0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:36.225496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b107ad8-09a6-4b7b-889e-1064de229fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:36.265809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0b107ad8-09a6-4b7b-889e-1064de229fd0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:36.269377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0b107ad8-09a6-4b7b-889e-1064de229fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:36.336834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3230b02-c341-4bb0-be24-b159e9a2803f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:36.436436 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1031/2038] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:36.449217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b107ad8-09a6-4b7b-889e-1064de229fd0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:36.451005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0b107ad8-09a6-4b7b-889e-1064de229fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:36.452029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd5871-ad68-4e3c-831e-bf103d4e733f tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29023b1313fb4f3cb1a6a7967a05127e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d57a0da6584f0ea47cf6fe296dd28c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:36.452990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd5871-ad68-4e3c-831e-bf103d4e733f tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:36.453287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd5871-ad68-4e3c-831e-bf103d4e733f tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:36.453650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd5871-ad68-4e3c-831e-bf103d4e733f tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] PATH_INFO: `/v3/projects/4196dbc6c42746ffab45efab01240e1e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:36.455062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd5871-ad68-4e3c-831e-bf103d4e733f tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorizing `identity:get_project(project_id=4196dbc6c42746ffab45efab01240e1e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:36.457952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd5871-ad68-4e3c-831e-bf103d4e733f tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:36.460691 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1008/2039] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:49:35 2022] GET /identity/v3/projects/4196dbc6c42746ffab45efab01240e1e => generated 381 bytes in 559 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:36.539478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b107ad8-09a6-4b7b-889e-1064de229fd0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:36.543418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac3f3e-c1c3-4333-ab2d-56525d2cd4d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:36.544638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac3f3e-c1c3-4333-ab2d-56525d2cd4d2 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:36.544976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac3f3e-c1c3-4333-ab2d-56525d2cd4d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:36.545238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac3f3e-c1c3-4333-ab2d-56525d2cd4d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:36.555904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac3f3e-c1c3-4333-ab2d-56525d2cd4d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:36.558621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac3f3e-c1c3-4333-ab2d-56525d2cd4d2 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:36.575166 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1032/2040] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:36.690271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd5871-ad68-4e3c-831e-bf103d4e733f tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:36.694035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f4bd5871-ad68-4e3c-831e-bf103d4e733f tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:36.732334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd5871-ad68-4e3c-831e-bf103d4e733f tempest-QuotasAdminNegativeTest-2135470950 tempest-QuotasAdminNegativeTest-2135470950-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:36.747924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:36.748788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:36.749114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:36.751137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] PATH_INFO: `/v3/users/627bcee7d4d244cf8af381923a0d906b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:36.753952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=627bcee7d4d244cf8af381923a0d906b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:36.759892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:36.878696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '627bcee7d4d244cf8af381923a0d906b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:36.921679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '627bcee7d4d244cf8af381923a0d906b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:36.923092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '627bcee7d4d244cf8af381923a0d906b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:36.945795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '627bcee7d4d244cf8af381923a0d906b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:36.974350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '627bcee7d4d244cf8af381923a0d906b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:37.000738 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1009/2041] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:36 2022] DELETE /identity/v3/users/627bcee7d4d244cf8af381923a0d906b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:37.005320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad40c91-6763-43e2-a2e9-dad0c348887d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:37.005810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad40c91-6763-43e2-a2e9-dad0c348887d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:37.005907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad40c91-6763-43e2-a2e9-dad0c348887d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:37.035328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fad40c91-6763-43e2-a2e9-dad0c348887d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:37.036574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fad40c91-6763-43e2-a2e9-dad0c348887d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:37.053288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:37.140086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:37.141739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:37.177085 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1033/2042] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:37.194893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:37.200715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad40c91-6763-43e2-a2e9-dad0c348887d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:37.202783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fad40c91-6763-43e2-a2e9-dad0c348887d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:37.317224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad40c91-6763-43e2-a2e9-dad0c348887d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:37.323936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1982441-652d-4a26-8e51-171845127be7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:37.327365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f835d0-566b-494d-80fb-3ff10475570c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:37.329205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f835d0-566b-494d-80fb-3ff10475570c service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:37.329528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f835d0-566b-494d-80fb-3ff10475570c service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:37.329821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f835d0-566b-494d-80fb-3ff10475570c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:37.334528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:37.335897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:37.336187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:37.336472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] PATH_INFO: `/v3/projects/1270b5dbd21b49d0b595a2aebb99255c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:37.338125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1270b5dbd21b49d0b595a2aebb99255c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:37.345180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:37.366064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f835d0-566b-494d-80fb-3ff10475570c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:37.372692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f835d0-566b-494d-80fb-3ff10475570c service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:37.415416 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1034/2043] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:37.420215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f835d0-566b-494d-80fb-3ff10475570c service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:37.425964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a7f835d0-566b-494d-80fb-3ff10475570c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:37.461483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f835d0-566b-494d-80fb-3ff10475570c service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:37.475504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:37.476553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:37.476855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:37.477182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] PATH_INFO: `/v3/projects/f58d0034099b4acd95ed5b1fee4fc9a5` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:37.479453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f58d0034099b4acd95ed5b1fee4fc9a5)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:37.500151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:37.506589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1270b5dbd21b49d0b595a2aebb99255c', 'request_id': 'req-e69da2a5-f3b9-4b63-9770-1121977fd53c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:37.533283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1270b5dbd21b49d0b595a2aebb99255c', 'request_id': 'req-e69da2a5-f3b9-4b63-9770-1121977fd53c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:37.551562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1270b5dbd21b49d0b595a2aebb99255c', 'request_id': 'req-e69da2a5-f3b9-4b63-9770-1121977fd53c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:37.581109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] The token cache is being invalidate because project 1270b5dbd21b49d0b595a2aebb99255c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:37.584844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:37.586636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:37.591645 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1010/2044] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:37 2022] DELETE /identity/v3/projects/1270b5dbd21b49d0b595a2aebb99255c => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:37.595014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:37.599294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:37.714158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f58d0034099b4acd95ed5b1fee4fc9a5', 'request_id': 'req-1fe466b0-d482-4c37-ba76-d8ec237da914'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:37.750138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f58d0034099b4acd95ed5b1fee4fc9a5', 'request_id': 'req-1fe466b0-d482-4c37-ba76-d8ec237da914'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:37.765491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f58d0034099b4acd95ed5b1fee4fc9a5', 'request_id': 'req-1fe466b0-d482-4c37-ba76-d8ec237da914'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:37.804819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69da2a5-f3b9-4b63-9770-1121977fd53c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:37.839561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] The token cache is being invalidate because project f58d0034099b4acd95ed5b1fee4fc9a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:37.839863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:37.845397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:37.852102 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1035/2045] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:37 2022] DELETE /identity/v3/projects/f58d0034099b4acd95ed5b1fee4fc9a5 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:37.852102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:37.852759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:37.963879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361831c-467f-4e3c-a2be-8f2e1a6c38ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:37.964547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361831c-467f-4e3c-a2be-8f2e1a6c38ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:37.965291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361831c-467f-4e3c-a2be-8f2e1a6c38ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:37.965465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361831c-467f-4e3c-a2be-8f2e1a6c38ff admin admin] PATH_INFO: `/v3/users/4aa6221685824a758e0b8df8ed0d7dca` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:37.967431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361831c-467f-4e3c-a2be-8f2e1a6c38ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aa6221685824a758e0b8df8ed0d7dca)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:37.969391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361831c-467f-4e3c-a2be-8f2e1a6c38ff admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:37.971233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:38.000262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:38.065347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe466b0-d482-4c37-ba76-d8ec237da914 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:38.086706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:38.087545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:38.087844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:38.088130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] PATH_INFO: `/v3/users/2276474f1cb7428cb6b479b7926609a7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:38.091336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2276474f1cb7428cb6b479b7926609a7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:38.097647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:38.104459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d361831c-467f-4e3c-a2be-8f2e1a6c38ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aa6221685824a758e0b8df8ed0d7dca'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.127911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d361831c-467f-4e3c-a2be-8f2e1a6c38ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aa6221685824a758e0b8df8ed0d7dca'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.129127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d361831c-467f-4e3c-a2be-8f2e1a6c38ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aa6221685824a758e0b8df8ed0d7dca'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.143124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d361831c-467f-4e3c-a2be-8f2e1a6c38ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aa6221685824a758e0b8df8ed0d7dca'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.171611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d361831c-467f-4e3c-a2be-8f2e1a6c38ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aa6221685824a758e0b8df8ed0d7dca'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.184015 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1011/2046] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:37 2022] DELETE /identity/v3/users/4aa6221685824a758e0b8df8ed0d7dca => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:38.186293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9ae30-71fc-4667-90e6-919bcc6b9932 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:38.186814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9ae30-71fc-4667-90e6-919bcc6b9932 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:38.187299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9ae30-71fc-4667-90e6-919bcc6b9932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:38.210899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2276474f1cb7428cb6b479b7926609a7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.229654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b2a9ae30-71fc-4667-90e6-919bcc6b9932 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:38.229868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2276474f1cb7428cb6b479b7926609a7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.231367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b2a9ae30-71fc-4667-90e6-919bcc6b9932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:38.242755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2276474f1cb7428cb6b479b7926609a7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.244220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2276474f1cb7428cb6b479b7926609a7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.249556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2276474f1cb7428cb6b479b7926609a7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.273334 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1036/2047] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:37 2022] DELETE /identity/v3/users/2276474f1cb7428cb6b479b7926609a7 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:38.277602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:38.283261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:38.358861 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1012/2048] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:38.369593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576632c-65a7-4f4e-98e3-fe73e8c7a3f6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:38.369593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576632c-65a7-4f4e-98e3-fe73e8c7a3f6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:38.370123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576632c-65a7-4f4e-98e3-fe73e8c7a3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:38.413066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d576632c-65a7-4f4e-98e3-fe73e8c7a3f6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:38.414566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d576632c-65a7-4f4e-98e3-fe73e8c7a3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:38.440421 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1013/2049] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:38.441384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576632c-65a7-4f4e-98e3-fe73e8c7a3f6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:38.444321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b2f5c-4fbf-4f0a-9732-fd159c1e94d9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:38.454636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d576632c-65a7-4f4e-98e3-fe73e8c7a3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:38.465208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:38.468533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:38.468829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:38.469149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] PATH_INFO: `/v3/users/403b2dcf1b3f4e1fbead88207efa9ea0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:38.472302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] RBAC: Authorizing `identity:delete_user(user_id=403b2dcf1b3f4e1fbead88207efa9ea0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:38.475535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:38.483122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576632c-65a7-4f4e-98e3-fe73e8c7a3f6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:38.494779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:38.495431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:38.495709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:38.495985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e admin admin] PATH_INFO: `/v3/projects/4196dbc6c42746ffab45efab01240e1e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:38.499289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4196dbc6c42746ffab45efab01240e1e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:38.501285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:38.567016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4196dbc6c42746ffab45efab01240e1e', 'request_id': 'req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.568341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '403b2dcf1b3f4e1fbead88207efa9ea0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.579356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4196dbc6c42746ffab45efab01240e1e', 'request_id': 'req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.587719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:38.591270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4196dbc6c42746ffab45efab01240e1e', 'request_id': 'req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.593597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '403b2dcf1b3f4e1fbead88207efa9ea0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.602801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:38.608851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '403b2dcf1b3f4e1fbead88207efa9ea0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.609827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '403b2dcf1b3f4e1fbead88207efa9ea0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.611082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '403b2dcf1b3f4e1fbead88207efa9ea0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.611685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e admin admin] The token cache is being invalidate because project 4196dbc6c42746ffab45efab01240e1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:38.612284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.613589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b15dcdfc-385c-45f0-a568-ec7fdcb7ce9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.615844 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1014/2050] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:38 2022] DELETE /identity/v3/projects/4196dbc6c42746ffab45efab01240e1e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:38.617473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f23c8-ba65-4571-9fc9-73e89b0565b3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:38.617947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f23c8-ba65-4571-9fc9-73e89b0565b3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:38.618227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f23c8-ba65-4571-9fc9-73e89b0565b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:38.634153 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1037/2051] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:38 2022] DELETE /identity/v3/users/403b2dcf1b3f4e1fbead88207efa9ea0 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:38.634153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:38.635724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:38.643461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-395f23c8-ba65-4571-9fc9-73e89b0565b3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:38.645103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-395f23c8-ba65-4571-9fc9-73e89b0565b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:38.709126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc96251-3398-4920-97cc-bcf0c9bec76b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:38.713022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d6259-aeb1-4c0e-acd4-5c051ead4109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:38.713783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d6259-aeb1-4c0e-acd4-5c051ead4109 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:38.714137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d6259-aeb1-4c0e-acd4-5c051ead4109 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:38.714643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d6259-aeb1-4c0e-acd4-5c051ead4109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:38.717016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-745d6259-aeb1-4c0e-acd4-5c051ead4109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:38.721282 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1015/2052] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:38.721282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f23c8-ba65-4571-9fc9-73e89b0565b3 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:38.721282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-395f23c8-ba65-4571-9fc9-73e89b0565b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:38.733096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d6259-aeb1-4c0e-acd4-5c051ead4109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:38.734399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f23c8-ba65-4571-9fc9-73e89b0565b3 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:38.734849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d6259-aeb1-4c0e-acd4-5c051ead4109 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:38.740147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:38.740933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:38.741291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:38.741640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] PATH_INFO: `/v3/users/29023b1313fb4f3cb1a6a7967a05127e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:38.743135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29023b1313fb4f3cb1a6a7967a05127e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:38.745332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:38.747462 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1038/2053] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:38.748738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0be3b7-2405-46d8-bd7b-9fe036cf0440 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:38.749271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0be3b7-2405-46d8-bd7b-9fe036cf0440 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:38.749662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0be3b7-2405-46d8-bd7b-9fe036cf0440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:38.772419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ad0be3b7-2405-46d8-bd7b-9fe036cf0440 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:38.773961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ad0be3b7-2405-46d8-bd7b-9fe036cf0440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:38.784826 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1039/2054] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:38.789722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0be3b7-2405-46d8-bd7b-9fe036cf0440 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:38.802872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0be3b7-2405-46d8-bd7b-9fe036cf0440 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:38.812116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fe1f3-8f12-4aa0-bbfd-700b956f8d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:38.813015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fe1f3-8f12-4aa0-bbfd-700b956f8d0c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:38.814121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fe1f3-8f12-4aa0-bbfd-700b956f8d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:38.814468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fe1f3-8f12-4aa0-bbfd-700b956f8d0c admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:38.815360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fe1f3-8f12-4aa0-bbfd-700b956f8d0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:38.815728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29023b1313fb4f3cb1a6a7967a05127e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.816183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fe1f3-8f12-4aa0-bbfd-700b956f8d0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:38.817973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fe1f3-8f12-4aa0-bbfd-700b956f8d0c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:38.839859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29023b1313fb4f3cb1a6a7967a05127e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.843004 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1040/2055] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:38.843541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29023b1313fb4f3cb1a6a7967a05127e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.844849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fe1f3-8f12-4aa0-bbfd-700b956f8d0c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:38.847419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a03fe1f3-8f12-4aa0-bbfd-700b956f8d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:38.857757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29023b1313fb4f3cb1a6a7967a05127e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.865523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fe1f3-8f12-4aa0-bbfd-700b956f8d0c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:38.870478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd52017-07d3-431d-9796-344fa015e1c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:38.871293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd52017-07d3-431d-9796-344fa015e1c1 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:38.871452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29023b1313fb4f3cb1a6a7967a05127e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:38.872748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd52017-07d3-431d-9796-344fa015e1c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:38.873174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd52017-07d3-431d-9796-344fa015e1c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:38.876104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6dd52017-07d3-431d-9796-344fa015e1c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:38.888768 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1016/2056] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:38 2022] DELETE /identity/v3/users/29023b1313fb4f3cb1a6a7967a05127e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:38.891225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:38.893558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd52017-07d3-431d-9796-344fa015e1c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:38.894151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:38.895199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd52017-07d3-431d-9796-344fa015e1c1 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:38.909924 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1041/2057] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:38.996288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf95ab8-1d8a-4f0f-86aa-dd5695f4bc73 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:39.003597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff40f07-deba-4347-9199-09d7f2f897f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:39.004649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff40f07-deba-4347-9199-09d7f2f897f3 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:39.005244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff40f07-deba-4347-9199-09d7f2f897f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:39.005747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff40f07-deba-4347-9199-09d7f2f897f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:39.006599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff40f07-deba-4347-9199-09d7f2f897f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:39.009127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff40f07-deba-4347-9199-09d7f2f897f3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:39.031922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:39.065782 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1017/2058] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:38 2022] POST /identity/v3/projects => generated 371 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:39.070693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd52017-07d3-431d-9796-344fa015e1c1 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:39.082733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd52017-07d3-431d-9796-344fa015e1c1 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:39.182475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca13185-496e-42c1-9698-c7cd2466eb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:39.183299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca13185-496e-42c1-9698-c7cd2466eb45 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:39.184217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca13185-496e-42c1-9698-c7cd2466eb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:39.184217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca13185-496e-42c1-9698-c7cd2466eb45 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:39.185874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca13185-496e-42c1-9698-c7cd2466eb45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:39.188951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca13185-496e-42c1-9698-c7cd2466eb45 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:39.223920 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1042/2059] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:39 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:39.225384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff40f07-deba-4347-9199-09d7f2f897f3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:39.238379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff40f07-deba-4347-9199-09d7f2f897f3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:39.260219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b90c5e-34c2-469e-b052-35f03641828e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:39.260876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b90c5e-34c2-469e-b052-35f03641828e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:39.261181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b90c5e-34c2-469e-b052-35f03641828e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:39.261453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b90c5e-34c2-469e-b052-35f03641828e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:39.262774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b90c5e-34c2-469e-b052-35f03641828e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:39.264377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b90c5e-34c2-469e-b052-35f03641828e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:39.278836 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1018/2060] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:49:39.283681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca13185-496e-42c1-9698-c7cd2466eb45 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:39.298367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca13185-496e-42c1-9698-c7cd2466eb45 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:39.307433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182d9ef-ab36-4302-a447-4091b013f9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:39.308890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182d9ef-ab36-4302-a447-4091b013f9de admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:39.309430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182d9ef-ab36-4302-a447-4091b013f9de admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:39.310099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182d9ef-ab36-4302-a447-4091b013f9de admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:39.311025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182d9ef-ab36-4302-a447-4091b013f9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:39.314649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182d9ef-ab36-4302-a447-4091b013f9de admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:39.327287 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1043/2061] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:39.331437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b90c5e-34c2-469e-b052-35f03641828e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:39.345913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b90c5e-34c2-469e-b052-35f03641828e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:39.351432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa157a-8870-4c59-9799-f023737ca408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:39.353033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa157a-8870-4c59-9799-f023737ca408 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:39.353490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa157a-8870-4c59-9799-f023737ca408 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:39.353956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa157a-8870-4c59-9799-f023737ca408 admin admin] PATH_INFO: `/v3/projects/6a7e10b9d8594d16a4c1eb63196ea329/users/90d7e4c2e577467f949be3dc480cc7a2/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:39.372642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa157a-8870-4c59-9799-f023737ca408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7e10b9d8594d16a4c1eb63196ea329, user_id=90d7e4c2e577467f949be3dc480cc7a2, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:39.376077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa157a-8870-4c59-9799-f023737ca408 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:39.397142 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1019/2062] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:39 2022] PUT /identity/v3/projects/6a7e10b9d8594d16a4c1eb63196ea329/users/90d7e4c2e577467f949be3dc480cc7a2/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:39.398341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c527b7-c7b2-4d3c-8d27-f62c24181a52 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:39.398639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c527b7-c7b2-4d3c-8d27-f62c24181a52 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:39.398916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c527b7-c7b2-4d3c-8d27-f62c24181a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:39.418729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-71c527b7-c7b2-4d3c-8d27-f62c24181a52 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:39.420513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-71c527b7-c7b2-4d3c-8d27-f62c24181a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:39.442457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:39.525501 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1044/2063] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:39.548777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa157a-8870-4c59-9799-f023737ca408 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:39.562836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c527b7-c7b2-4d3c-8d27-f62c24181a52 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:39.565028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-71c527b7-c7b2-4d3c-8d27-f62c24181a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:39.597926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c527b7-c7b2-4d3c-8d27-f62c24181a52 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:39.605117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:39.606052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:39.607038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:39.607707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] PATH_INFO: `/v3/projects/90b3649ee46d439d9dccdb7ef0e27291` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:39.609649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90b3649ee46d439d9dccdb7ef0e27291)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:39.612458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:39.711183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa157a-8870-4c59-9799-f023737ca408 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:39.735230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90b3649ee46d439d9dccdb7ef0e27291', 'request_id': 'req-f847f632-dbca-40a5-9d84-c6811e5487f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:39.748975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90b3649ee46d439d9dccdb7ef0e27291', 'request_id': 'req-f847f632-dbca-40a5-9d84-c6811e5487f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:39.768955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90b3649ee46d439d9dccdb7ef0e27291', 'request_id': 'req-f847f632-dbca-40a5-9d84-c6811e5487f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:39.793284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71bc8cb-3279-4caa-ad81-0ec77c697116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:39.794395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bc8cb-3279-4caa-ad81-0ec77c697116 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:39.794842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bc8cb-3279-4caa-ad81-0ec77c697116 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:39.795319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bc8cb-3279-4caa-ad81-0ec77c697116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:39.801441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] The token cache is being invalidate because project 90b3649ee46d439d9dccdb7ef0e27291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:39.801902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:39.804229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:39.810528 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1045/2064] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:39 2022] DELETE /identity/v3/projects/90b3649ee46d439d9dccdb7ef0e27291 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:39.811406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:39.813549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:39.877741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71bc8cb-3279-4caa-ad81-0ec77c697116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:39.879941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71bc8cb-3279-4caa-ad81-0ec77c697116 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:39.894578 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1020/2065] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:49:39.896227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71bc8cb-3279-4caa-ad81-0ec77c697116 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:39.898419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d71bc8cb-3279-4caa-ad81-0ec77c697116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:39.920078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71bc8cb-3279-4caa-ad81-0ec77c697116 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:39.930406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:39.931483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:39.932082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:39.932587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] PATH_INFO: `/v3/projects/5b3de50f698a4292858f898d092256af` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:39.936676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b3de50f698a4292858f898d092256af)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:39.941906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:39.986192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:40.007108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847f632-dbca-40a5-9d84-c6811e5487f0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:40.031561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b3de50f698a4292858f898d092256af', 'request_id': 'req-1810279f-afd5-4ab0-94c7-c443670bfd8c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.047224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b3de50f698a4292858f898d092256af', 'request_id': 'req-1810279f-afd5-4ab0-94c7-c443670bfd8c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.069925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b3de50f698a4292858f898d092256af', 'request_id': 'req-1810279f-afd5-4ab0-94c7-c443670bfd8c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.087359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] The token cache is being invalidate because project 5b3de50f698a4292858f898d092256af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:40.087660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.097502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.099332 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1021/2066] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:39 2022] DELETE /identity/v3/projects/5b3de50f698a4292858f898d092256af => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:40.100019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:40.101498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:40.139344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:40.150610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:40.151298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:40.151592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:40.151992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] PATH_INFO: `/v3/projects/1dacf31e1f0b4a4f9b27fc1f49006170` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:40.153773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dacf31e1f0b4a4f9b27fc1f49006170)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:40.159768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:40.239963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dacf31e1f0b4a4f9b27fc1f49006170', 'request_id': 'req-89227747-7981-49d5-acf6-3dc92a04907e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.256763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dacf31e1f0b4a4f9b27fc1f49006170', 'request_id': 'req-89227747-7981-49d5-acf6-3dc92a04907e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.274189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dacf31e1f0b4a4f9b27fc1f49006170', 'request_id': 'req-89227747-7981-49d5-acf6-3dc92a04907e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.291751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] The token cache is being invalidate because project 1dacf31e1f0b4a4f9b27fc1f49006170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:40.292197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.292786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810279f-afd5-4ab0-94c7-c443670bfd8c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:40.292933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.301794 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1046/2067] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:39 2022] DELETE /identity/v3/projects/1dacf31e1f0b4a4f9b27fc1f49006170 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:40.301794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:40.307206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:40.412822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a05524-ce26-4768-9b15-38430b2ac7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:40.413498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a05524-ce26-4768-9b15-38430b2ac7ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:40.413783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a05524-ce26-4768-9b15-38430b2ac7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:40.414062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a05524-ce26-4768-9b15-38430b2ac7ce admin admin] PATH_INFO: `/v3/users/622edda288714384a6a7aefae0c5be77` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:40.431641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a05524-ce26-4768-9b15-38430b2ac7ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=622edda288714384a6a7aefae0c5be77)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:40.434283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a05524-ce26-4768-9b15-38430b2ac7ce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:40.493717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89227747-7981-49d5-acf6-3dc92a04907e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:40.503560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:40.504366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:40.504799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:40.505127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] PATH_INFO: `/v3/projects/c6d57a0da6584f0ea47cf6fe296dd28c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:40.506521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d57a0da6584f0ea47cf6fe296dd28c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:40.515177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:40.526967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f0a05524-ce26-4768-9b15-38430b2ac7ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '622edda288714384a6a7aefae0c5be77'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.536394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:40.543299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f0a05524-ce26-4768-9b15-38430b2ac7ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '622edda288714384a6a7aefae0c5be77'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.545946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f0a05524-ce26-4768-9b15-38430b2ac7ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '622edda288714384a6a7aefae0c5be77'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.566348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f0a05524-ce26-4768-9b15-38430b2ac7ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '622edda288714384a6a7aefae0c5be77'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.580142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f0a05524-ce26-4768-9b15-38430b2ac7ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '622edda288714384a6a7aefae0c5be77'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.594107 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1022/2068] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:40 2022] DELETE /identity/v3/users/622edda288714384a6a7aefae0c5be77 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:40.597406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d643e-6cf3-4ba8-a7bd-4ca2f0cdf002 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:40.598306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d643e-6cf3-4ba8-a7bd-4ca2f0cdf002 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:40.599208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d57a0da6584f0ea47cf6fe296dd28c', 'request_id': 'req-f3b6287f-97b4-4af1-9c1e-447d62d1f312'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.599389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d643e-6cf3-4ba8-a7bd-4ca2f0cdf002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:40.612676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d57a0da6584f0ea47cf6fe296dd28c', 'request_id': 'req-f3b6287f-97b4-4af1-9c1e-447d62d1f312'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.621792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d57a0da6584f0ea47cf6fe296dd28c', 'request_id': 'req-f3b6287f-97b4-4af1-9c1e-447d62d1f312'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.637632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] The token cache is being invalidate because project c6d57a0da6584f0ea47cf6fe296dd28c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:40.638038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.639671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:40.645242 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1047/2069] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:40 2022] DELETE /identity/v3/projects/c6d57a0da6584f0ea47cf6fe296dd28c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:40.645242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:40.645242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:40.650421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f67d643e-6cf3-4ba8-a7bd-4ca2f0cdf002 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:40.653458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f67d643e-6cf3-4ba8-a7bd-4ca2f0cdf002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:40.677551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:40.740738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6287f-97b4-4af1-9c1e-447d62d1f312 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:40.842286 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1023/2070] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:40.842286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9d790-3ecb-4206-8256-ca2740c088fa None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:40.842286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9d790-3ecb-4206-8256-ca2740c088fa None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:40.842286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9d790-3ecb-4206-8256-ca2740c088fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:40.897154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e0e9d790-3ecb-4206-8256-ca2740c088fa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:40.898606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e0e9d790-3ecb-4206-8256-ca2740c088fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:40.905482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:40.915548 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1024/2071] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:40.915771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:40.915874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a5166-2716-4d52-8fb5-75e786ddeabd None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:40.915874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a5166-2716-4d52-8fb5-75e786ddeabd None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:40.915874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a5166-2716-4d52-8fb5-75e786ddeabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:40.916113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:40.916113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] PATH_INFO: `/v3/users/57bb8fae155a469a972a4feddbfffbb6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:40.916113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57bb8fae155a469a972a4feddbfffbb6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:40.922346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:40.932939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e99a5166-2716-4d52-8fb5-75e786ddeabd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:40.934500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e99a5166-2716-4d52-8fb5-75e786ddeabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:40.947938 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1025/2072] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:40.949507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb5243-7d47-49ed-9319-eb6bfdeceee0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:40.949895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb5243-7d47-49ed-9319-eb6bfdeceee0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:40.950340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb5243-7d47-49ed-9319-eb6bfdeceee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:40.973350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d2cb5243-7d47-49ed-9319-eb6bfdeceee0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:40.975001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d2cb5243-7d47-49ed-9319-eb6bfdeceee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:40.988877 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1026/2073] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:40.988877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb5243-7d47-49ed-9319-eb6bfdeceee0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:40.998790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb5243-7d47-49ed-9319-eb6bfdeceee0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:41.017222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f0574-a5eb-44ea-8229-66929282feaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:41.019366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f0574-a5eb-44ea-8229-66929282feaa admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:41.020584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f0574-a5eb-44ea-8229-66929282feaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:41.021378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f0574-a5eb-44ea-8229-66929282feaa admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:41.022336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f0574-a5eb-44ea-8229-66929282feaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:41.023689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57bb8fae155a469a972a4feddbfffbb6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:41.025145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f0574-a5eb-44ea-8229-66929282feaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:41.029802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f0574-a5eb-44ea-8229-66929282feaa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:41.053174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57bb8fae155a469a972a4feddbfffbb6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:41.055340 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1027/2074] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:41.057882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f0574-a5eb-44ea-8229-66929282feaa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:41.071436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57bb8fae155a469a972a4feddbfffbb6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:41.071648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f0574-a5eb-44ea-8229-66929282feaa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:41.074553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57bb8fae155a469a972a4feddbfffbb6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:41.075428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57bb8fae155a469a972a4feddbfffbb6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:41.078470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c1792-6e9e-4be9-b2ef-f5652874f133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:41.079091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c1792-6e9e-4be9-b2ef-f5652874f133 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:41.079386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c1792-6e9e-4be9-b2ef-f5652874f133 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:41.079723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c1792-6e9e-4be9-b2ef-f5652874f133 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:41.080706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c1792-6e9e-4be9-b2ef-f5652874f133 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:41.081055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c1792-6e9e-4be9-b2ef-f5652874f133 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:41.083406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c1792-6e9e-4be9-b2ef-f5652874f133 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:41.107099 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1048/2075] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:40 2022] DELETE /identity/v3/users/57bb8fae155a469a972a4feddbfffbb6 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:41.107099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:41.107393 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1028/2076] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:41.111760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:41.115179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c1792-6e9e-4be9-b2ef-f5652874f133 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:41.124227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7c4c1792-6e9e-4be9-b2ef-f5652874f133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:41.170109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c1792-6e9e-4be9-b2ef-f5652874f133 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:41.225179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:41.275997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be953d-2dd0-45df-a099-e651f90fe0a7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:41.293022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ebe4d-588d-4aaf-a08e-de590d350907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:41.294786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ebe4d-588d-4aaf-a08e-de590d350907 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:41.294786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ebe4d-588d-4aaf-a08e-de590d350907 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:41.295226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ebe4d-588d-4aaf-a08e-de590d350907 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:41.298518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ebe4d-588d-4aaf-a08e-de590d350907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:41.302685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ebe4d-588d-4aaf-a08e-de590d350907 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:41.320085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f9cd0-289a-4ffc-b569-45b14243e2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:41.324604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f9cd0-289a-4ffc-b569-45b14243e2fa admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:41.325565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f9cd0-289a-4ffc-b569-45b14243e2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:41.326086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f9cd0-289a-4ffc-b569-45b14243e2fa admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:41.326766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f9cd0-289a-4ffc-b569-45b14243e2fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:41.328710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f9cd0-289a-4ffc-b569-45b14243e2fa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:41.354593 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1049/2077] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:41 2022] POST /identity/v3/projects => generated 385 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:41.359850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ebe4d-588d-4aaf-a08e-de590d350907 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:41.402128 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1029/2078] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:41 2022] POST /identity/v3/projects => generated 393 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:41.407049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f9cd0-289a-4ffc-b569-45b14243e2fa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:41.430997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:41.493702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ebe4d-588d-4aaf-a08e-de590d350907 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:41.524562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f9cd0-289a-4ffc-b569-45b14243e2fa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:41.533770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052700a-9b7b-420b-b4e0-ea40e40963c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:41.534387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052700a-9b7b-420b-b4e0-ea40e40963c1 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:41.534678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052700a-9b7b-420b-b4e0-ea40e40963c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:41.535022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052700a-9b7b-420b-b4e0-ea40e40963c1 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:41.535686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052700a-9b7b-420b-b4e0-ea40e40963c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:41.538116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052700a-9b7b-420b-b4e0-ea40e40963c1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:41.581124 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1030/2079] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:41 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:49:41.584249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052700a-9b7b-420b-b4e0-ea40e40963c1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:41.587675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1052700a-9b7b-420b-b4e0-ea40e40963c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:41.615975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1c718-687b-45e7-99a7-bf270bc3eff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:41.616685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1c718-687b-45e7-99a7-bf270bc3eff6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:41.619643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1c718-687b-45e7-99a7-bf270bc3eff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:41.621011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1c718-687b-45e7-99a7-bf270bc3eff6 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:41.623645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052700a-9b7b-420b-b4e0-ea40e40963c1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:41.623805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1c718-687b-45e7-99a7-bf270bc3eff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:41.625801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1c718-687b-45e7-99a7-bf270bc3eff6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:41.629531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:41.630648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:41.630878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:41.631266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] PATH_INFO: `/v3/projects/76995bf5b0394ecf93109e9dc87b8219` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:41.634618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] RBAC: Authorizing `identity:delete_project(project_id=76995bf5b0394ecf93109e9dc87b8219)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:41.637140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:41.664397 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1050/2080] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:41 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:41.664397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1c718-687b-45e7-99a7-bf270bc3eff6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:41.676887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1c718-687b-45e7-99a7-bf270bc3eff6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:41.683402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b80ef6-8ffa-4289-86df-6f1291e2b32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:41.684029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b80ef6-8ffa-4289-86df-6f1291e2b32f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:41.684318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b80ef6-8ffa-4289-86df-6f1291e2b32f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:41.685155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b80ef6-8ffa-4289-86df-6f1291e2b32f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:41.685822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b80ef6-8ffa-4289-86df-6f1291e2b32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:41.687756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b80ef6-8ffa-4289-86df-6f1291e2b32f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:41.706682 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1051/2081] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:41.706682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b80ef6-8ffa-4289-86df-6f1291e2b32f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:41.716886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b80ef6-8ffa-4289-86df-6f1291e2b32f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:41.722901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcdf46-d7f8-4e56-9d84-39e1bd3d0533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:41.723617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcdf46-d7f8-4e56-9d84-39e1bd3d0533 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:41.723958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcdf46-d7f8-4e56-9d84-39e1bd3d0533 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:41.725060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcdf46-d7f8-4e56-9d84-39e1bd3d0533 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:41.725060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcdf46-d7f8-4e56-9d84-39e1bd3d0533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:41.732587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcdf46-d7f8-4e56-9d84-39e1bd3d0533 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:41.749140 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1052/2082] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:41.749140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcdf46-d7f8-4e56-9d84-39e1bd3d0533 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:41.764118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcdf46-d7f8-4e56-9d84-39e1bd3d0533 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:41.766454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76995bf5b0394ecf93109e9dc87b8219', 'request_id': 'req-9f4c8190-2fa7-45f4-838c-327f51d3957d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:41.778020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76995bf5b0394ecf93109e9dc87b8219', 'request_id': 'req-9f4c8190-2fa7-45f4-838c-327f51d3957d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:41.788508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76995bf5b0394ecf93109e9dc87b8219', 'request_id': 'req-9f4c8190-2fa7-45f4-838c-327f51d3957d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:41.805084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] The token cache is being invalidate because project 76995bf5b0394ecf93109e9dc87b8219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:41.805465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:41.806528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:41.809975 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1031/2083] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:41 2022] DELETE /identity/v3/projects/76995bf5b0394ecf93109e9dc87b8219 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:41.810860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:41.812441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:41.839899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:41.870304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4dc67-093c-4a4c-92a6-d47f4b4d9b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:41.871232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4dc67-093c-4a4c-92a6-d47f4b4d9b17 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:41.872503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4dc67-093c-4a4c-92a6-d47f4b4d9b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:41.873095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4dc67-093c-4a4c-92a6-d47f4b4d9b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:41.874349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4dc67-093c-4a4c-92a6-d47f4b4d9b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:41.876349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4dc67-093c-4a4c-92a6-d47f4b4d9b17 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:41.906562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c8190-2fa7-45f4-838c-327f51d3957d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:41.930895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864302a-76aa-44eb-9c48-32887930ce20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:41.931603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864302a-76aa-44eb-9c48-32887930ce20 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:41.931899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864302a-76aa-44eb-9c48-32887930ce20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:41.932189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864302a-76aa-44eb-9c48-32887930ce20 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:41.933582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864302a-76aa-44eb-9c48-32887930ce20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:41.935690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864302a-76aa-44eb-9c48-32887930ce20 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:41.947645 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1053/2084] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:41.952001 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1032/2085] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:41.953390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0129a0-5a34-430f-94b0-544a0fdafe10 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:41.953693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0129a0-5a34-430f-94b0-544a0fdafe10 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:41.953975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0129a0-5a34-430f-94b0-544a0fdafe10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:41.960754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4dc67-093c-4a4c-92a6-d47f4b4d9b17 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:41.966003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-01d4dc67-093c-4a4c-92a6-d47f4b4d9b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:41.982943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-dd0129a0-5a34-430f-94b0-544a0fdafe10 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:41.984204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dd0129a0-5a34-430f-94b0-544a0fdafe10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:41.994892 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1033/2086] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:41.995966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0129a0-5a34-430f-94b0-544a0fdafe10 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:42.004595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4dc67-093c-4a4c-92a6-d47f4b4d9b17 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:42.010061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5e12f-231a-4b43-ba93-72545ff223a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:42.011809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5e12f-231a-4b43-ba93-72545ff223a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.012096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5e12f-231a-4b43-ba93-72545ff223a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.012447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5e12f-231a-4b43-ba93-72545ff223a0 admin admin] PATH_INFO: `/v3/projects/a72b0776bdcd4644afed4d9cbc966e4e/users/cb51295631fd43fd8323ea06097a632e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.022977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0129a0-5a34-430f-94b0-544a0fdafe10 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:42.033562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83cf9d-115b-4b45-b61a-e1db93d1cd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:42.034395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83cf9d-115b-4b45-b61a-e1db93d1cd71 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.034789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83cf9d-115b-4b45-b61a-e1db93d1cd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.035175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83cf9d-115b-4b45-b61a-e1db93d1cd71 admin admin] PATH_INFO: `/v3/projects/82f4ba0629f24f7998a67a6a8820bc22/users/b919b648394b49c394e45c96793e8434/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.039196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5e12f-231a-4b43-ba93-72545ff223a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a72b0776bdcd4644afed4d9cbc966e4e, user_id=cb51295631fd43fd8323ea06097a632e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:42.042054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5e12f-231a-4b43-ba93-72545ff223a0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:42.058285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:42.062349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83cf9d-115b-4b45-b61a-e1db93d1cd71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f4ba0629f24f7998a67a6a8820bc22, user_id=b919b648394b49c394e45c96793e8434, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:42.068099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83cf9d-115b-4b45-b61a-e1db93d1cd71 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:42.074807 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1054/2087] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:41 2022] PUT /identity/v3/projects/a72b0776bdcd4644afed4d9cbc966e4e/users/cb51295631fd43fd8323ea06097a632e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:42.077169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcde2bc-eef9-43c8-a04a-ab203f7dc489 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.078959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcde2bc-eef9-43c8-a04a-ab203f7dc489 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.079395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcde2bc-eef9-43c8-a04a-ab203f7dc489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.103223 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1034/2088] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:41 2022] PUT /identity/v3/projects/82f4ba0629f24f7998a67a6a8820bc22/users/b919b648394b49c394e45c96793e8434/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:42.105847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d146a-47f3-4b18-955c-c41356e2a181 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.107238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d146a-47f3-4b18-955c-c41356e2a181 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.107735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d146a-47f3-4b18-955c-c41356e2a181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.129310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-adcde2bc-eef9-43c8-a04a-ab203f7dc489 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:42.132493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-adcde2bc-eef9-43c8-a04a-ab203f7dc489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:42.193420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4c3d146a-47f3-4b18-955c-c41356e2a181 None None] MFA Rules not processed for user `cb51295631fd43fd8323ea06097a632e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:42.194550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4c3d146a-47f3-4b18-955c-c41356e2a181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:42.296679 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1055/2089] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:42.298555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405a313-dc2d-4634-b10d-205ed1eadc3c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.298854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405a313-dc2d-4634-b10d-205ed1eadc3c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.299299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405a313-dc2d-4634-b10d-205ed1eadc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.333108 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1035/2090] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:42.334516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d146a-47f3-4b18-955c-c41356e2a181 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:42.336966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4c3d146a-47f3-4b18-955c-c41356e2a181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:42.346739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f405a313-dc2d-4634-b10d-205ed1eadc3c None None] MFA Rules not processed for user `b919b648394b49c394e45c96793e8434`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:42.347716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f405a313-dc2d-4634-b10d-205ed1eadc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:42.368726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d146a-47f3-4b18-955c-c41356e2a181 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:42.387501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:42.388278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.388751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.389178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] PATH_INFO: `/v3/projects/0ecb6d714e9f4670ab6a27cac2269f96` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.390615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ecb6d714e9f4670ab6a27cac2269f96)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:42.392840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:42.457955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ecb6d714e9f4670ab6a27cac2269f96', 'request_id': 'req-e561ced3-7222-4223-ac67-0f8f67d2eeb1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:42.461682 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1056/2091] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:42.462928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405a313-dc2d-4634-b10d-205ed1eadc3c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:42.476914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ecb6d714e9f4670ab6a27cac2269f96', 'request_id': 'req-e561ced3-7222-4223-ac67-0f8f67d2eeb1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:42.479327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405a313-dc2d-4634-b10d-205ed1eadc3c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:42.488012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ecb6d714e9f4670ab6a27cac2269f96', 'request_id': 'req-e561ced3-7222-4223-ac67-0f8f67d2eeb1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:42.503334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] The token cache is being invalidate because project 0ecb6d714e9f4670ab6a27cac2269f96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:42.503576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:42.504184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:42.506171 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1036/2092] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:42 2022] DELETE /identity/v3/projects/0ecb6d714e9f4670ab6a27cac2269f96 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:42.506505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:42.510489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:42.551967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:42.569777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f245a4-b1bb-469f-8bf7-4b284cb9a0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:42.571190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f245a4-b1bb-469f-8bf7-4b284cb9a0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.571631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f245a4-b1bb-469f-8bf7-4b284cb9a0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.571879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f245a4-b1bb-469f-8bf7-4b284cb9a0c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.572711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f245a4-b1bb-469f-8bf7-4b284cb9a0c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:42.573100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f245a4-b1bb-469f-8bf7-4b284cb9a0c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:42.575079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f245a4-b1bb-469f-8bf7-4b284cb9a0c4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:42.602467 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1057/2093] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:42.603765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f245a4-b1bb-469f-8bf7-4b284cb9a0c4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:42.606370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-22f245a4-b1bb-469f-8bf7-4b284cb9a0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:42.608075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561ced3-7222-4223-ac67-0f8f67d2eeb1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:42.614692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddf6fc-aa79-4bcb-a699-9a536392ec2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:42.615865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddf6fc-aa79-4bcb-a699-9a536392ec2d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.616524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddf6fc-aa79-4bcb-a699-9a536392ec2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.617860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddf6fc-aa79-4bcb-a699-9a536392ec2d admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.620688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddf6fc-aa79-4bcb-a699-9a536392ec2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:42.623396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddf6fc-aa79-4bcb-a699-9a536392ec2d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:42.635607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:42.641046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f245a4-b1bb-469f-8bf7-4b284cb9a0c4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:42.673439 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1037/2094] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:42 2022] POST /identity/v3/projects => generated 391 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:42.675310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59118-ae54-44f5-b965-96cffa62d357 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.675612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59118-ae54-44f5-b965-96cffa62d357 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.676532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59118-ae54-44f5-b965-96cffa62d357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.677432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78a610-de92-4cc0-83c7-43c3fad89bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:42.679489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78a610-de92-4cc0-83c7-43c3fad89bee admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.679869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78a610-de92-4cc0-83c7-43c3fad89bee admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.680321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78a610-de92-4cc0-83c7-43c3fad89bee admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.681084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c78a610-de92-4cc0-83c7-43c3fad89bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:42.683620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c78a610-de92-4cc0-83c7-43c3fad89bee admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:42.706701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6eb59118-ae54-44f5-b965-96cffa62d357 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:42.708416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6eb59118-ae54-44f5-b965-96cffa62d357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:42.726469 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1058/2095] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:42 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:42.727402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78a610-de92-4cc0-83c7-43c3fad89bee admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:42.729413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7c78a610-de92-4cc0-83c7-43c3fad89bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:42.794136 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1038/2096] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:42.795662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb59118-ae54-44f5-b965-96cffa62d357 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:42.809904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb59118-ae54-44f5-b965-96cffa62d357 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:42.810873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78a610-de92-4cc0-83c7-43c3fad89bee admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:42.817158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4f6d0-ff86-4d81-b7a8-af10fed54d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:42.818193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f6d0-ff86-4d81-b7a8-af10fed54d28 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.818808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f6d0-ff86-4d81-b7a8-af10fed54d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.819283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f6d0-ff86-4d81-b7a8-af10fed54d28 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.820273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e60d6-85f4-4e93-ba35-864370ba1b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:42.820994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e60d6-85f4-4e93-ba35-864370ba1b97 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.821381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e60d6-85f4-4e93-ba35-864370ba1b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.821838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e60d6-85f4-4e93-ba35-864370ba1b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.822634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934e60d6-85f4-4e93-ba35-864370ba1b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:42.823023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4f6d0-ff86-4d81-b7a8-af10fed54d28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:42.826043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934e60d6-85f4-4e93-ba35-864370ba1b97 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:42.836783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4f6d0-ff86-4d81-b7a8-af10fed54d28 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:42.879739 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1059/2097] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:42 2022] POST /identity/v3/projects => generated 369 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:42.881498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e60d6-85f4-4e93-ba35-864370ba1b97 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:42.899555 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1039/2098] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:42 2022] POST /identity/v3/users => generated 302 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:42.904731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2df5c-f5a3-4075-856b-931bff20edbc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:42.905066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2df5c-f5a3-4075-856b-931bff20edbc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:42.905407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2df5c-f5a3-4075-856b-931bff20edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:42.960315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-05e2df5c-f5a3-4075-856b-931bff20edbc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:42.964029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-05e2df5c-f5a3-4075-856b-931bff20edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:43.039216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e60d6-85f4-4e93-ba35-864370ba1b97 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.058987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65aedc-ce4f-4b00-a3e4-e347ecb3dd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.059662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65aedc-ce4f-4b00-a3e4-e347ecb3dd99 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.060007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65aedc-ce4f-4b00-a3e4-e347ecb3dd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.060236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65aedc-ce4f-4b00-a3e4-e347ecb3dd99 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.061076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65aedc-ce4f-4b00-a3e4-e347ecb3dd99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.063055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65aedc-ce4f-4b00-a3e4-e347ecb3dd99 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:43.105167 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1040/2099] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:43.106660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2df5c-f5a3-4075-856b-931bff20edbc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.114197 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1060/2100] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:42 2022] POST /identity/v3/users => generated 298 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:43.114197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65aedc-ce4f-4b00-a3e4-e347ecb3dd99 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.120531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2df5c-f5a3-4075-856b-931bff20edbc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.126536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71927816-28d0-4d4e-ba1d-a75abcee7ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.127123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71927816-28d0-4d4e-ba1d-a75abcee7ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.127678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71927816-28d0-4d4e-ba1d-a75abcee7ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.128664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71927816-28d0-4d4e-ba1d-a75abcee7ce0 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.129432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71927816-28d0-4d4e-ba1d-a75abcee7ce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.132035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71927816-28d0-4d4e-ba1d-a75abcee7ce0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:43.138632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65aedc-ce4f-4b00-a3e4-e347ecb3dd99 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.164831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a08607-1c86-48cf-8cbf-56ff11a8057e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.169837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a08607-1c86-48cf-8cbf-56ff11a8057e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.170217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a08607-1c86-48cf-8cbf-56ff11a8057e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.170607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a08607-1c86-48cf-8cbf-56ff11a8057e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.172758 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1041/2101] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:43 2022] POST /identity/v3/users => generated 285 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:43.172758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71927816-28d0-4d4e-ba1d-a75abcee7ce0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.185309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a08607-1c86-48cf-8cbf-56ff11a8057e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.187041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a08607-1c86-48cf-8cbf-56ff11a8057e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:43.187300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71927816-28d0-4d4e-ba1d-a75abcee7ce0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.194350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d994b49-37d0-4105-9844-ec2d6619b98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.195822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d994b49-37d0-4105-9844-ec2d6619b98d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.196415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d994b49-37d0-4105-9844-ec2d6619b98d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.196863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d994b49-37d0-4105-9844-ec2d6619b98d admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.197965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d994b49-37d0-4105-9844-ec2d6619b98d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:43.198438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d994b49-37d0-4105-9844-ec2d6619b98d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.207163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d994b49-37d0-4105-9844-ec2d6619b98d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:43.220740 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1061/2102] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:43.227407 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1042/2103] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:43.228286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d994b49-37d0-4105-9844-ec2d6619b98d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.235505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a08607-1c86-48cf-8cbf-56ff11a8057e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.250699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d994b49-37d0-4105-9844-ec2d6619b98d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.269061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a08607-1c86-48cf-8cbf-56ff11a8057e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.274973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184785f-52ed-4b45-a09b-2b219837a802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.275703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184785f-52ed-4b45-a09b-2b219837a802 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.276151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184785f-52ed-4b45-a09b-2b219837a802 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.276432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184785f-52ed-4b45-a09b-2b219837a802 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.277146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184785f-52ed-4b45-a09b-2b219837a802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.279467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184785f-52ed-4b45-a09b-2b219837a802 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:43.280014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5ba27-0c1f-4829-94fe-888596a93344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.280757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5ba27-0c1f-4829-94fe-888596a93344 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.281436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5ba27-0c1f-4829-94fe-888596a93344 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.281726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5ba27-0c1f-4829-94fe-888596a93344 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.282939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5ba27-0c1f-4829-94fe-888596a93344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.284562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5ba27-0c1f-4829-94fe-888596a93344 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:43.291592 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1062/2104] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:43.292521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184785f-52ed-4b45-a09b-2b219837a802 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.298353 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1043/2105] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:43.299322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5ba27-0c1f-4829-94fe-888596a93344 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.314371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184785f-52ed-4b45-a09b-2b219837a802 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.316796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5ba27-0c1f-4829-94fe-888596a93344 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.327868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ffe28-96eb-4479-81e3-fa1c6c295a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.328471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb698570-1a9d-46f2-a5a5-d9804f1c1430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.328612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ffe28-96eb-4479-81e3-fa1c6c295a70 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.329013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ffe28-96eb-4479-81e3-fa1c6c295a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.329266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb698570-1a9d-46f2-a5a5-d9804f1c1430 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.329393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ffe28-96eb-4479-81e3-fa1c6c295a70 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.329638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb698570-1a9d-46f2-a5a5-d9804f1c1430 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.329886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb698570-1a9d-46f2-a5a5-d9804f1c1430 admin admin] PATH_INFO: `/v3/projects/3d2d0567544d4db487c2d63a34078bb9/users/3643165fa95e4f16a8fc2e5c7765d71d/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.330240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ffe28-96eb-4479-81e3-fa1c6c295a70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.332605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ffe28-96eb-4479-81e3-fa1c6c295a70 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:43.355223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:43.366317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb698570-1a9d-46f2-a5a5-d9804f1c1430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2d0567544d4db487c2d63a34078bb9, user_id=3643165fa95e4f16a8fc2e5c7765d71d, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.376201 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1044/2106] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:43 2022] POST /identity/v3/projects => generated 387 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:43.384481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb698570-1a9d-46f2-a5a5-d9804f1c1430 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:43.386968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ffe28-96eb-4479-81e3-fa1c6c295a70 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.409763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:43.425114 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1063/2107] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:43 2022] PUT /identity/v3/projects/3d2d0567544d4db487c2d63a34078bb9/users/3643165fa95e4f16a8fc2e5c7765d71d/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:43.426444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb698570-1a9d-46f2-a5a5-d9804f1c1430 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.487396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ffe28-96eb-4479-81e3-fa1c6c295a70 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.587620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb698570-1a9d-46f2-a5a5-d9804f1c1430 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.594955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5a953-6881-4681-bdee-d68b109fe10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.596183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5a953-6881-4681-bdee-d68b109fe10e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.596482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5a953-6881-4681-bdee-d68b109fe10e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.596768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5a953-6881-4681-bdee-d68b109fe10e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.598236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5a953-6881-4681-bdee-d68b109fe10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.600099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5a953-6881-4681-bdee-d68b109fe10e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:43.618481 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1064/2108] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:43.619442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5a953-6881-4681-bdee-d68b109fe10e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.636498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5a953-6881-4681-bdee-d68b109fe10e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.643525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f50118-9ee9-4880-ba38-6429b520250d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.644261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f50118-9ee9-4880-ba38-6429b520250d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.644570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecc118-1274-4318-b35d-acdf3ff4edae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.644774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f50118-9ee9-4880-ba38-6429b520250d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.645040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f50118-9ee9-4880-ba38-6429b520250d admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.645281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecc118-1274-4318-b35d-acdf3ff4edae admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.645566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecc118-1274-4318-b35d-acdf3ff4edae admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.645788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f50118-9ee9-4880-ba38-6429b520250d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.645888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecc118-1274-4318-b35d-acdf3ff4edae admin admin] PATH_INFO: `/v3/projects/ca60026c0e9f45789817024071bb004b/users/0a0e6949fe5a4b00a10810c2a9bdf8df/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.648008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f50118-9ee9-4880-ba38-6429b520250d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:43.680619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecc118-1274-4318-b35d-acdf3ff4edae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca60026c0e9f45789817024071bb004b, user_id=0a0e6949fe5a4b00a10810c2a9bdf8df, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.683757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecc118-1274-4318-b35d-acdf3ff4edae admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:43.707328 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1065/2109] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:43 2022] POST /identity/v3/users => generated 294 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:43.708762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f50118-9ee9-4880-ba38-6429b520250d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.732157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f50118-9ee9-4880-ba38-6429b520250d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.737465 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1045/2110] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:43 2022] PUT /identity/v3/projects/ca60026c0e9f45789817024071bb004b/users/0a0e6949fe5a4b00a10810c2a9bdf8df/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:43.738148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecc118-1274-4318-b35d-acdf3ff4edae admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.893771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f53a2-8fdc-4686-9a8e-98042c72d16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.894549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f53a2-8fdc-4686-9a8e-98042c72d16f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.895518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f53a2-8fdc-4686-9a8e-98042c72d16f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.895518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f53a2-8fdc-4686-9a8e-98042c72d16f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.897010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f53a2-8fdc-4686-9a8e-98042c72d16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.899283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f53a2-8fdc-4686-9a8e-98042c72d16f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:43.933063 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1066/2111] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:43.933063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f53a2-8fdc-4686-9a8e-98042c72d16f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:43.950231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecc118-1274-4318-b35d-acdf3ff4edae admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.956401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f53a2-8fdc-4686-9a8e-98042c72d16f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:43.963873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad176dea-eb8c-44f4-95d3-887dd7c0b19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.964833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad176dea-eb8c-44f4-95d3-887dd7c0b19a admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.965149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad176dea-eb8c-44f4-95d3-887dd7c0b19a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.965493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad176dea-eb8c-44f4-95d3-887dd7c0b19a admin admin] PATH_INFO: `/v3/projects/c99a78f2b7d74d2fafc7426cb9046181/users/d872a5a6ae57458f94090871d0ee4b55/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.975480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c16658-6026-4765-a82e-dfe7bb86aeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:43.976326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c16658-6026-4765-a82e-dfe7bb86aeb5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:43.976688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c16658-6026-4765-a82e-dfe7bb86aeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:43.977098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c16658-6026-4765-a82e-dfe7bb86aeb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:43.978018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c16658-6026-4765-a82e-dfe7bb86aeb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:43.980411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c16658-6026-4765-a82e-dfe7bb86aeb5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:44.005008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad176dea-eb8c-44f4-95d3-887dd7c0b19a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99a78f2b7d74d2fafc7426cb9046181, user_id=d872a5a6ae57458f94090871d0ee4b55, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:44.009372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad176dea-eb8c-44f4-95d3-887dd7c0b19a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:44.009545 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1067/2112] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:44.009630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c16658-6026-4765-a82e-dfe7bb86aeb5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:44.039198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c16658-6026-4765-a82e-dfe7bb86aeb5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:44.055141 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1046/2113] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:43 2022] PUT /identity/v3/projects/c99a78f2b7d74d2fafc7426cb9046181/users/d872a5a6ae57458f94090871d0ee4b55/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:44.056159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad176dea-eb8c-44f4-95d3-887dd7c0b19a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:44.113979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:44.159321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:44.182816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b982caa-d6d1-458a-b6dc-d0fd61b39967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:44.183069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b982caa-d6d1-458a-b6dc-d0fd61b39967 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:44.183968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b982caa-d6d1-458a-b6dc-d0fd61b39967 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:44.184255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b982caa-d6d1-458a-b6dc-d0fd61b39967 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:44.185487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b982caa-d6d1-458a-b6dc-d0fd61b39967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:44.186638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b982caa-d6d1-458a-b6dc-d0fd61b39967 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:44.189722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad176dea-eb8c-44f4-95d3-887dd7c0b19a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:44.201081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351661ab-0f33-468e-82a3-f07aaf05e06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:44.201635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351661ab-0f33-468e-82a3-f07aaf05e06e admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:44.201926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351661ab-0f33-468e-82a3-f07aaf05e06e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:44.202206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351661ab-0f33-468e-82a3-f07aaf05e06e admin admin] PATH_INFO: `/v3/projects/3d2d0567544d4db487c2d63a34078bb9/users/3643165fa95e4f16a8fc2e5c7765d71d/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:44.207928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351661ab-0f33-468e-82a3-f07aaf05e06e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2d0567544d4db487c2d63a34078bb9, user_id=3643165fa95e4f16a8fc2e5c7765d71d, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:44.211625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351661ab-0f33-468e-82a3-f07aaf05e06e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:44.214709 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1068/2114] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:44.216189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b982caa-d6d1-458a-b6dc-d0fd61b39967 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:44.233107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b982caa-d6d1-458a-b6dc-d0fd61b39967 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:44.237504 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1047/2115] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:44 2022] PUT /identity/v3/projects/3d2d0567544d4db487c2d63a34078bb9/users/3643165fa95e4f16a8fc2e5c7765d71d/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:44.238939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c839f2a-59d2-460c-872a-067b59160eb0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:44.239234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c839f2a-59d2-460c-872a-067b59160eb0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:44.239512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c839f2a-59d2-460c-872a-067b59160eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:44.317845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5c839f2a-59d2-460c-872a-067b59160eb0 None None] MFA Rules not processed for user `d872a5a6ae57458f94090871d0ee4b55`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:44.319108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5c839f2a-59d2-460c-872a-067b59160eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:44.391302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44594e98-d12c-4268-8c75-8c49f42fee12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:44.391953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44594e98-d12c-4268-8c75-8c49f42fee12 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:44.392240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44594e98-d12c-4268-8c75-8c49f42fee12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:44.392517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44594e98-d12c-4268-8c75-8c49f42fee12 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:44.393216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44594e98-d12c-4268-8c75-8c49f42fee12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:44.394844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44594e98-d12c-4268-8c75-8c49f42fee12 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:44.441494 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1069/2116] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:44.449873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44594e98-d12c-4268-8c75-8c49f42fee12 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:44.478316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44594e98-d12c-4268-8c75-8c49f42fee12 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:44.494767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92164ea-34e7-4870-864a-d9b829d91f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:44.494767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92164ea-34e7-4870-864a-d9b829d91f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:44.494767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92164ea-34e7-4870-864a-d9b829d91f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:44.494767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92164ea-34e7-4870-864a-d9b829d91f76 admin admin] PATH_INFO: `/v3/projects/ca60026c0e9f45789817024071bb004b/users/0a0e6949fe5a4b00a10810c2a9bdf8df/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:44.495215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92164ea-34e7-4870-864a-d9b829d91f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca60026c0e9f45789817024071bb004b, user_id=0a0e6949fe5a4b00a10810c2a9bdf8df, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:44.495579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92164ea-34e7-4870-864a-d9b829d91f76 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:44.512699 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1048/2117] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:44.514378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bb0c0-7aa2-4b5d-bc4c-d152ab75164b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:44.514753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bb0c0-7aa2-4b5d-bc4c-d152ab75164b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:44.515114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bb0c0-7aa2-4b5d-bc4c-d152ab75164b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:44.530660 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1070/2118] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:44 2022] PUT /identity/v3/projects/ca60026c0e9f45789817024071bb004b/users/0a0e6949fe5a4b00a10810c2a9bdf8df/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:44.535281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92164ea-34e7-4870-864a-d9b829d91f76 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:44.566828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a61bb0c0-7aa2-4b5d-bc4c-d152ab75164b None None] MFA Rules not processed for user `3643165fa95e4f16a8fc2e5c7765d71d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:44.568835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a61bb0c0-7aa2-4b5d-bc4c-d152ab75164b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:44.685101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92164ea-34e7-4870-864a-d9b829d91f76 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:44.704704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d6623-7570-48fc-aaa8-05e29739bcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:44.705441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d6623-7570-48fc-aaa8-05e29739bcbe admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:44.705866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d6623-7570-48fc-aaa8-05e29739bcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:44.706122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d6623-7570-48fc-aaa8-05e29739bcbe admin admin] PATH_INFO: `/v3/projects/f2253308775749df8038947933483108/users/35606939a51e4322b8a38f76659dac57/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:44.740660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d6623-7570-48fc-aaa8-05e29739bcbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2253308775749df8038947933483108, user_id=35606939a51e4322b8a38f76659dac57, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:44.747008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d6623-7570-48fc-aaa8-05e29739bcbe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:44.753569 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1049/2119] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:44 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:44.766527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bb0c0-7aa2-4b5d-bc4c-d152ab75164b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:44.780487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bb0c0-7aa2-4b5d-bc4c-d152ab75164b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:44.787199 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1071/2120] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:44 2022] PUT /identity/v3/projects/f2253308775749df8038947933483108/users/35606939a51e4322b8a38f76659dac57/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:44.793089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac84d8-583d-4796-a3e4-2643903aa75c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:44.793394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac84d8-583d-4796-a3e4-2643903aa75c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:44.794219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac84d8-583d-4796-a3e4-2643903aa75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:44.843274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:44.872117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f2ac84d8-583d-4796-a3e4-2643903aa75c None None] MFA Rules not processed for user `0a0e6949fe5a4b00a10810c2a9bdf8df`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:44.876510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f2ac84d8-583d-4796-a3e4-2643903aa75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:44.906155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb7bcd-2e53-4904-be34-dd07c6d3f202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:44.908043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb7bcd-2e53-4904-be34-dd07c6d3f202 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:44.909644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb7bcd-2e53-4904-be34-dd07c6d3f202 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:44.911057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb7bcd-2e53-4904-be34-dd07c6d3f202 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:44.911967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb7bcd-2e53-4904-be34-dd07c6d3f202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:44.916082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb7bcd-2e53-4904-be34-dd07c6d3f202 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:44.967640 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1050/2121] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:44 2022] POST /identity/v3/projects => generated 371 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:44.969864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe51c9-b9e5-4ff2-9962-29d4839cbdc7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:44.970276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe51c9-b9e5-4ff2-9962-29d4839cbdc7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:44.970653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe51c9-b9e5-4ff2-9962-29d4839cbdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:44.978853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:45.021127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cebe51c9-b9e5-4ff2-9962-29d4839cbdc7 None None] MFA Rules not processed for user `35606939a51e4322b8a38f76659dac57`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:45.026472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cebe51c9-b9e5-4ff2-9962-29d4839cbdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:45.039843 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1072/2122] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:45.042006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea311f-8845-4067-903b-2100619ec0d4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:45.044621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea311f-8845-4067-903b-2100619ec0d4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:45.045099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea311f-8845-4067-903b-2100619ec0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:45.079073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3dea311f-8845-4067-903b-2100619ec0d4 None None] MFA Rules not processed for user `cb51295631fd43fd8323ea06097a632e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:45.081253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3dea311f-8845-4067-903b-2100619ec0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:45.181638 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1051/2123] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:45.182641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe51c9-b9e5-4ff2-9962-29d4839cbdc7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:45.183743 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1073/2124] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:45.185574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea311f-8845-4067-903b-2100619ec0d4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:45.187222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3dea311f-8845-4067-903b-2100619ec0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:45.298945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe51c9-b9e5-4ff2-9962-29d4839cbdc7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:45.304695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38708038-6263-4f53-9dce-220f808b10ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:45.305553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38708038-6263-4f53-9dce-220f808b10ae admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:45.305846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38708038-6263-4f53-9dce-220f808b10ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:45.306131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38708038-6263-4f53-9dce-220f808b10ae admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:45.306786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38708038-6263-4f53-9dce-220f808b10ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:45.308775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38708038-6263-4f53-9dce-220f808b10ae admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:45.345306 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1052/2125] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:45 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:45.346160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38708038-6263-4f53-9dce-220f808b10ae admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:45.367345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38708038-6263-4f53-9dce-220f808b10ae admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:45.369361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea311f-8845-4067-903b-2100619ec0d4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:45.379765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b3fad-77a8-4c71-ba7b-6d240efc3701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:45.380384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b3fad-77a8-4c71-ba7b-6d240efc3701 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:45.380672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b3fad-77a8-4c71-ba7b-6d240efc3701 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:45.381160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b3fad-77a8-4c71-ba7b-6d240efc3701 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:45.381612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b3fad-77a8-4c71-ba7b-6d240efc3701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:45.383613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b3fad-77a8-4c71-ba7b-6d240efc3701 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:45.385784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe9d10-be58-4766-b42d-869e4225beda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:45.386963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe9d10-be58-4766-b42d-869e4225beda service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:45.387256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe9d10-be58-4766-b42d-869e4225beda service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:45.387632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe9d10-be58-4766-b42d-869e4225beda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:45.439058 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1053/2126] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:45 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:45.443472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b3fad-77a8-4c71-ba7b-6d240efc3701 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:45.446336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5f3b3fad-77a8-4c71-ba7b-6d240efc3701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:45.473867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:45.502050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe9d10-be58-4766-b42d-869e4225beda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:45.504686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe9d10-be58-4766-b42d-869e4225beda service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:45.534653 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1074/2127] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:45.535835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe9d10-be58-4766-b42d-869e4225beda service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:45.561408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe9d10-be58-4766-b42d-869e4225beda service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:45.595037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:45.600437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b3fad-77a8-4c71-ba7b-6d240efc3701 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:45.616471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a039b35-5971-43d6-a4a5-ede837ab78af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:45.617240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a039b35-5971-43d6-a4a5-ede837ab78af service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:45.617531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a039b35-5971-43d6-a4a5-ede837ab78af service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:45.617810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a039b35-5971-43d6-a4a5-ede837ab78af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:45.623081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6a039b35-5971-43d6-a4a5-ede837ab78af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:45.681496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7272a3-b2a1-43db-9969-6c278b027979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:45.682132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7272a3-b2a1-43db-9969-6c278b027979 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:45.682416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7272a3-b2a1-43db-9969-6c278b027979 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:45.682699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7272a3-b2a1-43db-9969-6c278b027979 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:45.683362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7272a3-b2a1-43db-9969-6c278b027979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:45.688472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7272a3-b2a1-43db-9969-6c278b027979 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:45.735535 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1075/2128] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:45 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:45.737291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312dde0-7e6f-4c76-a333-8874560a5be6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:45.737770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312dde0-7e6f-4c76-a333-8874560a5be6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:45.738550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312dde0-7e6f-4c76-a333-8874560a5be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:45.775585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a039b35-5971-43d6-a4a5-ede837ab78af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:45.779943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a039b35-5971-43d6-a4a5-ede837ab78af service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:45.791047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b312dde0-7e6f-4c76-a333-8874560a5be6 None None] MFA Rules not processed for user `78b60bdc00024f6f9cac34b07b8940ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:45.792319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b312dde0-7e6f-4c76-a333-8874560a5be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:45.805059 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1054/2129] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:45.805917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a039b35-5971-43d6-a4a5-ede837ab78af service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:45.826723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a039b35-5971-43d6-a4a5-ede837ab78af service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:45.849828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3723a-560f-4cb0-b95a-5385eea5accf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:45.850469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3723a-560f-4cb0-b95a-5385eea5accf admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:45.850796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3723a-560f-4cb0-b95a-5385eea5accf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:45.851078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3723a-560f-4cb0-b95a-5385eea5accf admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:45.851733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3723a-560f-4cb0-b95a-5385eea5accf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:45.853713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3723a-560f-4cb0-b95a-5385eea5accf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:45.911610 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1076/2130] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:45 2022] POST /identity/v3/auth/tokens => generated 3290 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:45.912632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312dde0-7e6f-4c76-a333-8874560a5be6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:45.926726 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1055/2131] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:45 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:45.928557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cb469-46f3-4d40-b3bb-f1bdcb7a3f5b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:45.928557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cb469-46f3-4d40-b3bb-f1bdcb7a3f5b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:45.928776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cb469-46f3-4d40-b3bb-f1bdcb7a3f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:45.940037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312dde0-7e6f-4c76-a333-8874560a5be6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:45.952601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbcd4e-20a7-4329-8d9d-364326c35f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:45.953629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbcd4e-20a7-4329-8d9d-364326c35f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:45.954164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbcd4e-20a7-4329-8d9d-364326c35f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:45.954971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbcd4e-20a7-4329-8d9d-364326c35f9c admin admin] PATH_INFO: `/v3/projects/ec1df3b7183f4faf930d6449579bc92c/users/cfe151e1bbc64633991ff43c0001840c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:45.962554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0e1cb469-46f3-4d40-b3bb-f1bdcb7a3f5b None None] MFA Rules not processed for user `78b60bdc00024f6f9cac34b07b8940ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:45.964754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0e1cb469-46f3-4d40-b3bb-f1bdcb7a3f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:45.978084 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1056/2132] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:45 2022] POST /identity/v3/auth/tokens => generated 3290 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:45.978983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cb469-46f3-4d40-b3bb-f1bdcb7a3f5b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:45.980322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbcd4e-20a7-4329-8d9d-364326c35f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec1df3b7183f4faf930d6449579bc92c, user_id=cfe151e1bbc64633991ff43c0001840c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:45.986627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbcd4e-20a7-4329-8d9d-364326c35f9c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:46.001332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cb469-46f3-4d40-b3bb-f1bdcb7a3f5b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:46.010079 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1077/2133] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:45 2022] PUT /identity/v3/projects/ec1df3b7183f4faf930d6449579bc92c/users/cfe151e1bbc64633991ff43c0001840c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:46.014409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbcd4e-20a7-4329-8d9d-364326c35f9c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:46.148429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8ea0f-1139-4d63-a95a-f85a476ec23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:46.149118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8ea0f-1139-4d63-a95a-f85a476ec23c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:46.149464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8ea0f-1139-4d63-a95a-f85a476ec23c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:46.149748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8ea0f-1139-4d63-a95a-f85a476ec23c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:46.150588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8ea0f-1139-4d63-a95a-f85a476ec23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:46.152296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8ea0f-1139-4d63-a95a-f85a476ec23c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:46.163099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbcd4e-20a7-4329-8d9d-364326c35f9c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:46.171205 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1057/2134] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:45 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:46.172328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8ea0f-1139-4d63-a95a-f85a476ec23c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:46.173930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd1590-1693-4f22-934c-90781b94b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:46.174574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd1590-1693-4f22-934c-90781b94b3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:46.174978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd1590-1693-4f22-934c-90781b94b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:46.175262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd1590-1693-4f22-934c-90781b94b3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:46.176119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd1590-1693-4f22-934c-90781b94b3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:46.177761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd1590-1693-4f22-934c-90781b94b3a1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:46.184869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8ea0f-1139-4d63-a95a-f85a476ec23c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:46.191734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf068a9-34ac-4ca8-bb1e-fa221303ca8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:46.193106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf068a9-34ac-4ca8-bb1e-fa221303ca8c admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:46.193474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf068a9-34ac-4ca8-bb1e-fa221303ca8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:46.193865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf068a9-34ac-4ca8-bb1e-fa221303ca8c admin admin] PATH_INFO: `/v3/projects/1056967c67b440debf877be118e7e1a7/users/143b5a8df4704dd1a0e57e183718a64c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:46.211128 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1078/2135] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:49:46.213720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd1590-1693-4f22-934c-90781b94b3a1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:46.219910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf068a9-34ac-4ca8-bb1e-fa221303ca8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1056967c67b440debf877be118e7e1a7, user_id=143b5a8df4704dd1a0e57e183718a64c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:46.223196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf068a9-34ac-4ca8-bb1e-fa221303ca8c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:46.235276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:46.258920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd1590-1693-4f22-934c-90781b94b3a1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:46.266283 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1058/2136] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:46 2022] PUT /identity/v3/projects/1056967c67b440debf877be118e7e1a7/users/143b5a8df4704dd1a0e57e183718a64c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:46.267236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf068a9-34ac-4ca8-bb1e-fa221303ca8c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:46.297284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:46.405357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf068a9-34ac-4ca8-bb1e-fa221303ca8c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:46.406334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cfadb-b005-4836-a370-5f6eff797ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:46.407030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cfadb-b005-4836-a370-5f6eff797ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:46.407320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cfadb-b005-4836-a370-5f6eff797ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:46.409108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cfadb-b005-4836-a370-5f6eff797ffe admin admin] PATH_INFO: `/v3/projects/ec1df3b7183f4faf930d6449579bc92c/users/cfe151e1bbc64633991ff43c0001840c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:46.415141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cfadb-b005-4836-a370-5f6eff797ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec1df3b7183f4faf930d6449579bc92c, user_id=cfe151e1bbc64633991ff43c0001840c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:46.421304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d14b39-963b-49b6-9602-ebe29cad5f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:46.421934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d14b39-963b-49b6-9602-ebe29cad5f7a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:46.422224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d14b39-963b-49b6-9602-ebe29cad5f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:46.422509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d14b39-963b-49b6-9602-ebe29cad5f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:46.426030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cfadb-b005-4836-a370-5f6eff797ffe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:46.443619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d14b39-963b-49b6-9602-ebe29cad5f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:46.447701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d14b39-963b-49b6-9602-ebe29cad5f7a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:46.454152 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1079/2137] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:46 2022] PUT /identity/v3/projects/ec1df3b7183f4faf930d6449579bc92c/users/cfe151e1bbc64633991ff43c0001840c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:46.461770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cfadb-b005-4836-a370-5f6eff797ffe admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:46.499949 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1059/2138] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:46.500182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0bbcf-ab05-4996-b74d-1cb6b69a535d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:46.500470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0bbcf-ab05-4996-b74d-1cb6b69a535d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:46.500756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0bbcf-ab05-4996-b74d-1cb6b69a535d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:46.572008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2fa0bbcf-ab05-4996-b74d-1cb6b69a535d None None] MFA Rules not processed for user `cfe151e1bbc64633991ff43c0001840c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:46.573353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2fa0bbcf-ab05-4996-b74d-1cb6b69a535d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:46.674594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cfadb-b005-4836-a370-5f6eff797ffe admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:46.695323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a594f-bead-4ba6-9017-e769bcbf8f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:46.695960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a594f-bead-4ba6-9017-e769bcbf8f67 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:46.696257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a594f-bead-4ba6-9017-e769bcbf8f67 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:46.696538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a594f-bead-4ba6-9017-e769bcbf8f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:46.778221 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1060/2139] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:46.780086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0bbcf-ab05-4996-b74d-1cb6b69a535d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:46.884073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a594f-bead-4ba6-9017-e769bcbf8f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:46.886719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a594f-bead-4ba6-9017-e769bcbf8f67 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:46.902159 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1080/2140] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:46 2022] GET /identity/v3/auth/tokens => generated 3290 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:46.948162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0bbcf-ab05-4996-b74d-1cb6b69a535d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:46.962907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adcbfd-5f8f-4701-82de-df6068104973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:46.964278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adcbfd-5f8f-4701-82de-df6068104973 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:46.965944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adcbfd-5f8f-4701-82de-df6068104973 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:46.967601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adcbfd-5f8f-4701-82de-df6068104973 admin admin] PATH_INFO: `/v3/projects/1056967c67b440debf877be118e7e1a7/users/143b5a8df4704dd1a0e57e183718a64c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:46.974634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adcbfd-5f8f-4701-82de-df6068104973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1056967c67b440debf877be118e7e1a7, user_id=143b5a8df4704dd1a0e57e183718a64c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:46.983016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adcbfd-5f8f-4701-82de-df6068104973 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:47.010517 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1061/2141] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:46 2022] PUT /identity/v3/projects/1056967c67b440debf877be118e7e1a7/users/143b5a8df4704dd1a0e57e183718a64c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:47.024797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f204e-25d2-4095-ba5f-01fa826d1a98 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:47.025915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f204e-25d2-4095-ba5f-01fa826d1a98 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:47.026365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f204e-25d2-4095-ba5f-01fa826d1a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:47.085797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6d9f204e-25d2-4095-ba5f-01fa826d1a98 None None] MFA Rules not processed for user `143b5a8df4704dd1a0e57e183718a64c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:47.087066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6d9f204e-25d2-4095-ba5f-01fa826d1a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:47.182425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:47.241453 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1081/2142] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:47.359104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adcbfd-5f8f-4701-82de-df6068104973 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:47.378895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:47.491745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adcbfd-5f8f-4701-82de-df6068104973 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:47.498339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6dca5b-fc9d-4cd8-bbbe-b929f5673f54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:47.499245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6dca5b-fc9d-4cd8-bbbe-b929f5673f54 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:47.499361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6dca5b-fc9d-4cd8-bbbe-b929f5673f54 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:47.499627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6dca5b-fc9d-4cd8-bbbe-b929f5673f54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:47.606062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6dca5b-fc9d-4cd8-bbbe-b929f5673f54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:47.608985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6dca5b-fc9d-4cd8-bbbe-b929f5673f54 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:47.623283 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1062/2143] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:47.952359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f204e-25d2-4095-ba5f-01fa826d1a98 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:47.988494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f204e-25d2-4095-ba5f-01fa826d1a98 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:48.004395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01345c4c-19e0-4555-84f9-ea4690e2b842 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:48.010947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01345c4c-19e0-4555-84f9-ea4690e2b842 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:48.012679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01345c4c-19e0-4555-84f9-ea4690e2b842 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:48.013856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01345c4c-19e0-4555-84f9-ea4690e2b842 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:48.040920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01345c4c-19e0-4555-84f9-ea4690e2b842 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:48.042163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01345c4c-19e0-4555-84f9-ea4690e2b842 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:48.102788 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1082/2144] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:48.492938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6dca5b-fc9d-4cd8-bbbe-b929f5673f54 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:48.562634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01345c4c-19e0-4555-84f9-ea4690e2b842 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:48.630304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6dca5b-fc9d-4cd8-bbbe-b929f5673f54 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:48.646712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f2ff0-79b4-485e-bee8-be40f1fa1ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:48.647343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f2ff0-79b4-485e-bee8-be40f1fa1ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:48.648041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f2ff0-79b4-485e-bee8-be40f1fa1ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:48.648041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f2ff0-79b4-485e-bee8-be40f1fa1ec9 admin admin] PATH_INFO: `/v3/users/d872a5a6ae57458f94090871d0ee4b55` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:48.651954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f2ff0-79b4-485e-bee8-be40f1fa1ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d872a5a6ae57458f94090871d0ee4b55)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:48.655008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f2ff0-79b4-485e-bee8-be40f1fa1ec9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:48.665667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01345c4c-19e0-4555-84f9-ea4690e2b842 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:48.677171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae2e34-4a04-48fe-8136-8ed26cfba6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:48.678267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae2e34-4a04-48fe-8136-8ed26cfba6f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:48.679400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae2e34-4a04-48fe-8136-8ed26cfba6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:48.679926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae2e34-4a04-48fe-8136-8ed26cfba6f6 admin admin] PATH_INFO: `/v3/users/b919b648394b49c394e45c96793e8434` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:48.681414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae2e34-4a04-48fe-8136-8ed26cfba6f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b919b648394b49c394e45c96793e8434)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:48.684958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae2e34-4a04-48fe-8136-8ed26cfba6f6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:48.769191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-749f2ff0-79b4-485e-bee8-be40f1fa1ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd872a5a6ae57458f94090871d0ee4b55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:48.771137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:48.789616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-749f2ff0-79b4-485e-bee8-be40f1fa1ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd872a5a6ae57458f94090871d0ee4b55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:48.790599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-749f2ff0-79b4-485e-bee8-be40f1fa1ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd872a5a6ae57458f94090871d0ee4b55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:48.800373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-749f2ff0-79b4-485e-bee8-be40f1fa1ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd872a5a6ae57458f94090871d0ee4b55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:48.811756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-749f2ff0-79b4-485e-bee8-be40f1fa1ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd872a5a6ae57458f94090871d0ee4b55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:48.818801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-08ae2e34-4a04-48fe-8136-8ed26cfba6f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b919b648394b49c394e45c96793e8434'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:48.829209 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1063/2145] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:48 2022] DELETE /identity/v3/users/d872a5a6ae57458f94090871d0ee4b55 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:48.833071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ba2fe-8c97-424d-948a-11905f8b860a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:48.833553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ba2fe-8c97-424d-948a-11905f8b860a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:48.833975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ba2fe-8c97-424d-948a-11905f8b860a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:48.852418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-08ae2e34-4a04-48fe-8136-8ed26cfba6f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b919b648394b49c394e45c96793e8434'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:48.863895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-08ae2e34-4a04-48fe-8136-8ed26cfba6f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b919b648394b49c394e45c96793e8434'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:48.872717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-08ae2e34-4a04-48fe-8136-8ed26cfba6f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b919b648394b49c394e45c96793e8434'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:48.874009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-08ae2e34-4a04-48fe-8136-8ed26cfba6f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b919b648394b49c394e45c96793e8434'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:48.904274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b96ba2fe-8c97-424d-948a-11905f8b860a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:48.909355 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1083/2146] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:48 2022] DELETE /identity/v3/users/b919b648394b49c394e45c96793e8434 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:48.910715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b96ba2fe-8c97-424d-948a-11905f8b860a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:48.914363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702359a7-0521-435a-af87-2be2ad8cd6d2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:48.914881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702359a7-0521-435a-af87-2be2ad8cd6d2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:48.915621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702359a7-0521-435a-af87-2be2ad8cd6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:48.975524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-702359a7-0521-435a-af87-2be2ad8cd6d2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:48.978267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-702359a7-0521-435a-af87-2be2ad8cd6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:49.084042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:49.093642 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1064/2147] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:49.105393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ba2fe-8c97-424d-948a-11905f8b860a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:49.107395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b96ba2fe-8c97-424d-948a-11905f8b860a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:49.179419 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1084/2148] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:49.188114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702359a7-0521-435a-af87-2be2ad8cd6d2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:49.190068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-702359a7-0521-435a-af87-2be2ad8cd6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:49.325356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702359a7-0521-435a-af87-2be2ad8cd6d2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:49.331063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dffb1-bc5a-4875-9159-b47c587355a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:49.332157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dffb1-bc5a-4875-9159-b47c587355a4 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:49.332847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dffb1-bc5a-4875-9159-b47c587355a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:49.333264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dffb1-bc5a-4875-9159-b47c587355a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:49.366964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dffb1-bc5a-4875-9159-b47c587355a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:49.377092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ba2fe-8c97-424d-948a-11905f8b860a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:49.377700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dffb1-bc5a-4875-9159-b47c587355a4 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:49.390087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38de54c-69fc-4050-939d-fd5648755a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:49.391465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38de54c-69fc-4050-939d-fd5648755a0a service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:49.391745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38de54c-69fc-4050-939d-fd5648755a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:49.392024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38de54c-69fc-4050-939d-fd5648755a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:49.415131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38de54c-69fc-4050-939d-fd5648755a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:49.415364 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1085/2149] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:49.416807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38de54c-69fc-4050-939d-fd5648755a0a service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:49.446649 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1065/2150] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:50.136248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dffb1-bc5a-4875-9159-b47c587355a4 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:50.139952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-254dffb1-bc5a-4875-9159-b47c587355a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:50.373389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dffb1-bc5a-4875-9159-b47c587355a4 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:50.395848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17178279-db4a-4071-a762-812f448f7f54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:50.396536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17178279-db4a-4071-a762-812f448f7f54 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:50.398485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17178279-db4a-4071-a762-812f448f7f54 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:50.398852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17178279-db4a-4071-a762-812f448f7f54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:50.403513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-17178279-db4a-4071-a762-812f448f7f54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:50.638427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17178279-db4a-4071-a762-812f448f7f54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:50.641090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17178279-db4a-4071-a762-812f448f7f54 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:50.717523 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1086/2151] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:50.960610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38de54c-69fc-4050-939d-fd5648755a0a service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:50.963199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e38de54c-69fc-4050-939d-fd5648755a0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:51.020054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38de54c-69fc-4050-939d-fd5648755a0a service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:51.033430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:51.034097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:51.034784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:51.037106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] PATH_INFO: `/v3/projects/82f4ba0629f24f7998a67a6a8820bc22` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:51.039783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f4ba0629f24f7998a67a6a8820bc22)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:51.043788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:51.126274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303af9ac-f5a3-4f9b-b28d-24bc62a790e0 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:51.140095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303af9ac-f5a3-4f9b-b28d-24bc62a790e0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:51.140822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303af9ac-f5a3-4f9b-b28d-24bc62a790e0 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:51.144666 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1087/2152] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:49:51 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:49:51.152287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd433df-9ecd-4112-84a7-d7afa493092f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:51.155409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd433df-9ecd-4112-84a7-d7afa493092f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:51.155951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd433df-9ecd-4112-84a7-d7afa493092f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:51.183031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f4ba0629f24f7998a67a6a8820bc22', 'request_id': 'req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.199414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f4ba0629f24f7998a67a6a8820bc22', 'request_id': 'req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.214135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f4ba0629f24f7998a67a6a8820bc22', 'request_id': 'req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.219567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:51.228519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ffd433df-9ecd-4112-84a7-d7afa493092f None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:51.229961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ffd433df-9ecd-4112-84a7-d7afa493092f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:51.248509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] The token cache is being invalidate because project 82f4ba0629f24f7998a67a6a8820bc22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:51.248831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.251286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.256200 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1066/2153] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:50 2022] DELETE /identity/v3/projects/82f4ba0629f24f7998a67a6a8820bc22 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:51.260012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:51.264052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:51.275686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:51.385474 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1088/2154] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:49:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:51.402115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd433df-9ecd-4112-84a7-d7afa493092f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:51.404149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ffd433df-9ecd-4112-84a7-d7afa493092f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:51.408714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dc1ab-3ccc-410b-a27d-b553ff0722d6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:51.428854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:51.430701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:51.431040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:51.431939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd433df-9ecd-4112-84a7-d7afa493092f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:51.433605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] PATH_INFO: `/v3/users/0a0e6949fe5a4b00a10810c2a9bdf8df` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:51.435891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a0e6949fe5a4b00a10810c2a9bdf8df)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:51.438364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:51.445340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9cb19-4aaf-43b6-a756-e124c9bf480e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:51.445828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9cb19-4aaf-43b6-a756-e124c9bf480e service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:51.446029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9cb19-4aaf-43b6-a756-e124c9bf480e service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:51.446252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9cb19-4aaf-43b6-a756-e124c9bf480e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:51.467542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9cb19-4aaf-43b6-a756-e124c9bf480e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:51.469051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9cb19-4aaf-43b6-a756-e124c9bf480e service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:51.484464 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1089/2155] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:49:51.485386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9cb19-4aaf-43b6-a756-e124c9bf480e service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:51.486884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cab9cb19-4aaf-43b6-a756-e124c9bf480e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:51.515330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9cb19-4aaf-43b6-a756-e124c9bf480e service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:51.532799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:51.533564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:51.533923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:51.534325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] PATH_INFO: `/v3/projects/c99a78f2b7d74d2fafc7426cb9046181` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:51.535625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99a78f2b7d74d2fafc7426cb9046181)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:51.537945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:51.544616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a0e6949fe5a4b00a10810c2a9bdf8df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.578364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a0e6949fe5a4b00a10810c2a9bdf8df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.579371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a0e6949fe5a4b00a10810c2a9bdf8df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.586962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a0e6949fe5a4b00a10810c2a9bdf8df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.596803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a0e6949fe5a4b00a10810c2a9bdf8df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.598804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99a78f2b7d74d2fafc7426cb9046181', 'request_id': 'req-44ac2aa6-5f70-49fb-a530-c15071b82f5f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.608433 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1067/2156] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:51 2022] DELETE /identity/v3/users/0a0e6949fe5a4b00a10810c2a9bdf8df => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:51.609429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:51.610824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:51.620208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99a78f2b7d74d2fafc7426cb9046181', 'request_id': 'req-44ac2aa6-5f70-49fb-a530-c15071b82f5f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.635279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99a78f2b7d74d2fafc7426cb9046181', 'request_id': 'req-44ac2aa6-5f70-49fb-a530-c15071b82f5f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.671581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] The token cache is being invalidate because project c99a78f2b7d74d2fafc7426cb9046181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:51.672158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.673072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:51.675604 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1090/2157] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:51 2022] DELETE /identity/v3/projects/c99a78f2b7d74d2fafc7426cb9046181 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:51.680024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:51.682624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:51.898901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46832b9-e354-41f5-96f2-ddfb2ba9dc50 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:51.914904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416c904-ff88-4f5f-bafa-e3b6b971d5e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:51.915676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416c904-ff88-4f5f-bafa-e3b6b971d5e9 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:51.916776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416c904-ff88-4f5f-bafa-e3b6b971d5e9 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:51.917212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416c904-ff88-4f5f-bafa-e3b6b971d5e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:51.921772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac2aa6-5f70-49fb-a530-c15071b82f5f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:51.922033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5416c904-ff88-4f5f-bafa-e3b6b971d5e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:51.939003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:52.013872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:52.070046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:52.070935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:52.071376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:52.072027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] PATH_INFO: `/v3/users/cfe151e1bbc64633991ff43c0001840c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:52.075661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe151e1bbc64633991ff43c0001840c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:52.079209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416c904-ff88-4f5f-bafa-e3b6b971d5e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:52.080141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:52.081225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416c904-ff88-4f5f-bafa-e3b6b971d5e9 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:52.098089 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1068/2158] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:51 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:52.170919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe151e1bbc64633991ff43c0001840c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:52.240456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe151e1bbc64633991ff43c0001840c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:52.252737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe151e1bbc64633991ff43c0001840c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:52.253487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe151e1bbc64633991ff43c0001840c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:52.254209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe151e1bbc64633991ff43c0001840c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:52.331600 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1091/2159] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:51 2022] DELETE /identity/v3/users/cfe151e1bbc64633991ff43c0001840c => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:52.727817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2f9de-1fa0-4fca-9b1a-15555c78f4b5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:52.728211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2f9de-1fa0-4fca-9b1a-15555c78f4b5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:52.728554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2f9de-1fa0-4fca-9b1a-15555c78f4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:52.748716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:52.750369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:52.843942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ccc2f9de-1fa0-4fca-9b1a-15555c78f4b5 None None] MFA Rules not processed for user `90d7e4c2e577467f949be3dc480cc7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:52.850388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ccc2f9de-1fa0-4fca-9b1a-15555c78f4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:53.005545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832c0c6-c340-40b9-ae00-29e6a319e410 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:53.019083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae82c8-691c-4c92-8169-97675396bef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:53.019753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae82c8-691c-4c92-8169-97675396bef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:53.021649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae82c8-691c-4c92-8169-97675396bef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:53.021957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae82c8-691c-4c92-8169-97675396bef5 admin admin] PATH_INFO: `/v3/projects/ca60026c0e9f45789817024071bb004b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:53.027231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae82c8-691c-4c92-8169-97675396bef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca60026c0e9f45789817024071bb004b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:53.031149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae82c8-691c-4c92-8169-97675396bef5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:53.049889 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1069/2160] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:53.059753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2f9de-1fa0-4fca-9b1a-15555c78f4b5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:53.064822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ccc2f9de-1fa0-4fca-9b1a-15555c78f4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:53.084459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2f9de-1fa0-4fca-9b1a-15555c78f4b5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:53.092321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26791fe3-901a-4741-baa2-e5f1a2e7e722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:53.093644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26791fe3-901a-4741-baa2-e5f1a2e7e722 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:53.094467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26791fe3-901a-4741-baa2-e5f1a2e7e722 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:53.094954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26791fe3-901a-4741-baa2-e5f1a2e7e722 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:53.096311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26791fe3-901a-4741-baa2-e5f1a2e7e722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:53.099737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26791fe3-901a-4741-baa2-e5f1a2e7e722 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:53.157361 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1070/2161] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:53 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:53.162304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26791fe3-901a-4741-baa2-e5f1a2e7e722 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:53.177730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-23ae82c8-691c-4c92-8169-97675396bef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca60026c0e9f45789817024071bb004b', 'request_id': 'req-23ae82c8-691c-4c92-8169-97675396bef5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:53.193759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-23ae82c8-691c-4c92-8169-97675396bef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca60026c0e9f45789817024071bb004b', 'request_id': 'req-23ae82c8-691c-4c92-8169-97675396bef5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:53.210911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-23ae82c8-691c-4c92-8169-97675396bef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca60026c0e9f45789817024071bb004b', 'request_id': 'req-23ae82c8-691c-4c92-8169-97675396bef5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:53.220655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:53.240244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-23ae82c8-691c-4c92-8169-97675396bef5 admin admin] The token cache is being invalidate because project ca60026c0e9f45789817024071bb004b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:53.240601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-23ae82c8-691c-4c92-8169-97675396bef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:53.242599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-23ae82c8-691c-4c92-8169-97675396bef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:53.251097 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1092/2162] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:52 2022] DELETE /identity/v3/projects/ca60026c0e9f45789817024071bb004b => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:53.256365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c93d73-91c1-40d4-80ca-9c8a091a812e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:53.256679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c93d73-91c1-40d4-80ca-9c8a091a812e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:53.257028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c93d73-91c1-40d4-80ca-9c8a091a812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:53.276260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26791fe3-901a-4741-baa2-e5f1a2e7e722 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:53.282307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9791f-e698-483e-a5a9-21ccb5fa800d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:53.282936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9791f-e698-483e-a5a9-21ccb5fa800d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:53.283219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9791f-e698-483e-a5a9-21ccb5fa800d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:53.283591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9791f-e698-483e-a5a9-21ccb5fa800d admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:53.284740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9791f-e698-483e-a5a9-21ccb5fa800d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:53.287472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9791f-e698-483e-a5a9-21ccb5fa800d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:53.310227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-15c93d73-91c1-40d4-80ca-9c8a091a812e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:53.313728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:53.315897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-15c93d73-91c1-40d4-80ca-9c8a091a812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:53.326946 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1071/2163] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:53 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:49:53.334621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9791f-e698-483e-a5a9-21ccb5fa800d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:53.340488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-51a9791f-e698-483e-a5a9-21ccb5fa800d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:53.341703 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1093/2164] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:53.349703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156d4be-10dc-429b-afc2-c0954770ad15 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:53.350083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156d4be-10dc-429b-afc2-c0954770ad15 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:53.350893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156d4be-10dc-429b-afc2-c0954770ad15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:53.363790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9791f-e698-483e-a5a9-21ccb5fa800d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:53.383071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06433427-e384-45aa-aa57-aa97e4107d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:53.384616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06433427-e384-45aa-aa57-aa97e4107d1c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:53.385741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8156d4be-10dc-429b-afc2-c0954770ad15 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:53.386308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06433427-e384-45aa-aa57-aa97e4107d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:53.389597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8156d4be-10dc-429b-afc2-c0954770ad15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:53.391538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06433427-e384-45aa-aa57-aa97e4107d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:53.392571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06433427-e384-45aa-aa57-aa97e4107d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:53.395282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06433427-e384-45aa-aa57-aa97e4107d1c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:53.409767 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1094/2165] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:53.413960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156d4be-10dc-429b-afc2-c0954770ad15 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:53.415826 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1072/2166] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:53.420548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06433427-e384-45aa-aa57-aa97e4107d1c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:53.436046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156d4be-10dc-429b-afc2-c0954770ad15 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:53.444873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5a630-c39f-4f37-84e0-22c092481a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:53.445481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06433427-e384-45aa-aa57-aa97e4107d1c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:53.445598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5a630-c39f-4f37-84e0-22c092481a8d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:53.445946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5a630-c39f-4f37-84e0-22c092481a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:53.446424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5a630-c39f-4f37-84e0-22c092481a8d admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:53.447141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5a630-c39f-4f37-84e0-22c092481a8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:53.447542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5a630-c39f-4f37-84e0-22c092481a8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:53.449339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5a630-c39f-4f37-84e0-22c092481a8d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:53.457956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f5905-2c8c-4b2a-a0c1-4f14ce33e18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:53.460224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f5905-2c8c-4b2a-a0c1-4f14ce33e18a admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:53.460709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f5905-2c8c-4b2a-a0c1-4f14ce33e18a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:53.461256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f5905-2c8c-4b2a-a0c1-4f14ce33e18a admin admin] PATH_INFO: `/v3/projects/587a114de1624c6fb97b479e2395ef77/users/f1b49ae269454b179df6ebdc94a487ba/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:53.479075 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1095/2167] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:53.483740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5a630-c39f-4f37-84e0-22c092481a8d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:53.492874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f5905-2c8c-4b2a-a0c1-4f14ce33e18a admin admin] RBAC: Authorizing `identity:create_grant(project_id=587a114de1624c6fb97b479e2395ef77, user_id=f1b49ae269454b179df6ebdc94a487ba, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:53.497485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5a630-c39f-4f37-84e0-22c092481a8d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:53.500049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f5905-2c8c-4b2a-a0c1-4f14ce33e18a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:53.508817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbfd4f-4d5e-4f18-8e0f-3bb341a6df54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:53.509408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbfd4f-4d5e-4f18-8e0f-3bb341a6df54 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:53.509787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbfd4f-4d5e-4f18-8e0f-3bb341a6df54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:53.510147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbfd4f-4d5e-4f18-8e0f-3bb341a6df54 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:53.510867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbfd4f-4d5e-4f18-8e0f-3bb341a6df54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:53.513603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbfd4f-4d5e-4f18-8e0f-3bb341a6df54 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:53.527743 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1073/2168] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:53 2022] PUT /identity/v3/projects/587a114de1624c6fb97b479e2395ef77/users/f1b49ae269454b179df6ebdc94a487ba/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:53.533432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f5905-2c8c-4b2a-a0c1-4f14ce33e18a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:53.563532 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1096/2169] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:53 2022] POST /identity/v3/projects => generated 401 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:49:53.564703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbfd4f-4d5e-4f18-8e0f-3bb341a6df54 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:53.567018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-55bbfd4f-4d5e-4f18-8e0f-3bb341a6df54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:53.707983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f5905-2c8c-4b2a-a0c1-4f14ce33e18a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:53.779468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbfd4f-4d5e-4f18-8e0f-3bb341a6df54 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:53.795006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:53.796987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:53.797354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:53.799568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] PATH_INFO: `/v3/projects/ec1df3b7183f4faf930d6449579bc92c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:53.804262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec1df3b7183f4faf930d6449579bc92c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:53.810072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:53.825452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ace1f7-1e4d-42c1-8489-51a17ecf420e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:53.825987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ace1f7-1e4d-42c1-8489-51a17ecf420e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:53.826275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ace1f7-1e4d-42c1-8489-51a17ecf420e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:53.826553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ace1f7-1e4d-42c1-8489-51a17ecf420e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:53.827213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ace1f7-1e4d-42c1-8489-51a17ecf420e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:53.828829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ace1f7-1e4d-42c1-8489-51a17ecf420e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:53.857175 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1074/2170] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:53.857175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ace1f7-1e4d-42c1-8489-51a17ecf420e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:53.875128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ace1f7-1e4d-42c1-8489-51a17ecf420e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:53.881279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66cea3-dac0-4cbe-8edf-f83390da6741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:53.882025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66cea3-dac0-4cbe-8edf-f83390da6741 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:53.885056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66cea3-dac0-4cbe-8edf-f83390da6741 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:53.886755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66cea3-dac0-4cbe-8edf-f83390da6741 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:53.888745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66cea3-dac0-4cbe-8edf-f83390da6741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:53.893877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66cea3-dac0-4cbe-8edf-f83390da6741 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:53.919428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec1df3b7183f4faf930d6449579bc92c', 'request_id': 'req-4341ebac-b82b-4e32-813d-e411abee24be'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:53.930068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec1df3b7183f4faf930d6449579bc92c', 'request_id': 'req-4341ebac-b82b-4e32-813d-e411abee24be'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:53.944504 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1075/2171] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:53 2022] POST /identity/v3/users => generated 301 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:53.944504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66cea3-dac0-4cbe-8edf-f83390da6741 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:53.945351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec1df3b7183f4faf930d6449579bc92c', 'request_id': 'req-4341ebac-b82b-4e32-813d-e411abee24be'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:53.963709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66cea3-dac0-4cbe-8edf-f83390da6741 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:53.969836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] The token cache is being invalidate because project ec1df3b7183f4faf930d6449579bc92c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:53.970136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:53.971172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:53.973130 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1097/2172] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:53 2022] DELETE /identity/v3/projects/ec1df3b7183f4faf930d6449579bc92c => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:53.973927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:53.975290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:54.037811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:54.067869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:54.082774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd84ed3-dc6d-4c64-9db4-ccb2bb209dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:54.083722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd84ed3-dc6d-4c64-9db4-ccb2bb209dda admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:54.084019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd84ed3-dc6d-4c64-9db4-ccb2bb209dda admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:54.084307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd84ed3-dc6d-4c64-9db4-ccb2bb209dda admin admin] PATH_INFO: `/v3/projects/587a114de1624c6fb97b479e2395ef77/users/f1b49ae269454b179df6ebdc94a487ba/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:54.087381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd84ed3-dc6d-4c64-9db4-ccb2bb209dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=587a114de1624c6fb97b479e2395ef77, user_id=f1b49ae269454b179df6ebdc94a487ba, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:54.091562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd84ed3-dc6d-4c64-9db4-ccb2bb209dda admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:54.105430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341ebac-b82b-4e32-813d-e411abee24be admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:54.122342 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1076/2173] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:53 2022] PUT /identity/v3/projects/587a114de1624c6fb97b479e2395ef77/users/f1b49ae269454b179df6ebdc94a487ba/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:54.125177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b0d61-fbae-4c64-b325-07a088e53872 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:54.125479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b0d61-fbae-4c64-b325-07a088e53872 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:54.125768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b0d61-fbae-4c64-b325-07a088e53872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:54.160745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-642b0d61-fbae-4c64-b325-07a088e53872 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:54.162347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-642b0d61-fbae-4c64-b325-07a088e53872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:54.269880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24ceb1-0e2a-4240-a657-0cb3f896cecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:54.270385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24ceb1-0e2a-4240-a657-0cb3f896cecb admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:54.270385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24ceb1-0e2a-4240-a657-0cb3f896cecb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:54.270675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24ceb1-0e2a-4240-a657-0cb3f896cecb admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:54.271283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24ceb1-0e2a-4240-a657-0cb3f896cecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:54.272932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24ceb1-0e2a-4240-a657-0cb3f896cecb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:54.279089 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1077/2174] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:54.286506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408c0fc-1ec0-4c2f-8af4-a34c248eec82 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:54.287019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408c0fc-1ec0-4c2f-8af4-a34c248eec82 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:54.287456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408c0fc-1ec0-4c2f-8af4-a34c248eec82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:54.312986 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1098/2175] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:54.314026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24ceb1-0e2a-4240-a657-0cb3f896cecb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:54.340121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24ceb1-0e2a-4240-a657-0cb3f896cecb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:54.367024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4408c0fc-1ec0-4c2f-8af4-a34c248eec82 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:54.368256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4408c0fc-1ec0-4c2f-8af4-a34c248eec82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:54.369806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38613737-5dbb-452a-9ef8-992b7e756e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:54.370426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38613737-5dbb-452a-9ef8-992b7e756e41 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:54.370711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38613737-5dbb-452a-9ef8-992b7e756e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:54.370986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38613737-5dbb-452a-9ef8-992b7e756e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:54.371679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38613737-5dbb-452a-9ef8-992b7e756e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:54.373361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38613737-5dbb-452a-9ef8-992b7e756e41 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:54.404140 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1099/2176] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:54.405836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38613737-5dbb-452a-9ef8-992b7e756e41 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:54.412851 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1078/2177] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:54.413790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408c0fc-1ec0-4c2f-8af4-a34c248eec82 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:54.426848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38613737-5dbb-452a-9ef8-992b7e756e41 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:54.439933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408c0fc-1ec0-4c2f-8af4-a34c248eec82 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:54.446307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271fd2a-7e34-4ca8-9677-3c0f6b11c33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:54.448752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271fd2a-7e34-4ca8-9677-3c0f6b11c33c admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:54.449359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271fd2a-7e34-4ca8-9677-3c0f6b11c33c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:54.450436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271fd2a-7e34-4ca8-9677-3c0f6b11c33c admin admin] PATH_INFO: `/v3/projects/0f86e3171bdc40d6b4b303e6bf5e4a2c/users/8756e3fd985440d1a29cb5f5b94a6ac1/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:54.463646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2818f7-ec2d-4671-9f4d-324bb5de145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:54.465016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2818f7-ec2d-4671-9f4d-324bb5de145c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:54.468806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2818f7-ec2d-4671-9f4d-324bb5de145c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:54.469350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2818f7-ec2d-4671-9f4d-324bb5de145c admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:54.477817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2818f7-ec2d-4671-9f4d-324bb5de145c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:54.480700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2818f7-ec2d-4671-9f4d-324bb5de145c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:54.486654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2818f7-ec2d-4671-9f4d-324bb5de145c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:54.496514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271fd2a-7e34-4ca8-9677-3c0f6b11c33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f86e3171bdc40d6b4b303e6bf5e4a2c, user_id=8756e3fd985440d1a29cb5f5b94a6ac1, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:54.500240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271fd2a-7e34-4ca8-9677-3c0f6b11c33c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:54.532597 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1100/2178] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:54 2022] PUT /identity/v3/projects/0f86e3171bdc40d6b4b303e6bf5e4a2c/users/8756e3fd985440d1a29cb5f5b94a6ac1/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:54.545453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a515bfc-1ec1-44b3-b501-61a8ae352bff None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:54.545886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a515bfc-1ec1-44b3-b501-61a8ae352bff None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:54.546187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a515bfc-1ec1-44b3-b501-61a8ae352bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:54.552127 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1079/2179] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:54.557965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2818f7-ec2d-4671-9f4d-324bb5de145c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:54.586301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2818f7-ec2d-4671-9f4d-324bb5de145c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:54.602119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3a515bfc-1ec1-44b3-b501-61a8ae352bff None None] MFA Rules not processed for user `8756e3fd985440d1a29cb5f5b94a6ac1`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:54.603942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3a515bfc-1ec1-44b3-b501-61a8ae352bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:54.699904 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1101/2180] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:54 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:49:54.700849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a515bfc-1ec1-44b3-b501-61a8ae352bff None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:54.703935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3a515bfc-1ec1-44b3-b501-61a8ae352bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:54.748588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e9d46-de50-484a-b917-71160fd044bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:54.749835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e9d46-de50-484a-b917-71160fd044bd admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:54.751550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e9d46-de50-484a-b917-71160fd044bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:54.751671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:54.752156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e9d46-de50-484a-b917-71160fd044bd admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:54.755956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e9d46-de50-484a-b917-71160fd044bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:54.762118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e9d46-de50-484a-b917-71160fd044bd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:54.805018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a515bfc-1ec1-44b3-b501-61a8ae352bff None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:54.810984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:54.811689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:54.811983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:54.812265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] PATH_INFO: `/v3/users/35606939a51e4322b8a38f76659dac57` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:54.813703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35606939a51e4322b8a38f76659dac57)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:54.816709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:54.853619 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1080/2181] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:54 2022] POST /identity/v3/projects => generated 403 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:54.853619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e9d46-de50-484a-b917-71160fd044bd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:54.888966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35606939a51e4322b8a38f76659dac57'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:54.905955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35606939a51e4322b8a38f76659dac57'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:54.926149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35606939a51e4322b8a38f76659dac57'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:54.926764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35606939a51e4322b8a38f76659dac57'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:54.927456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35606939a51e4322b8a38f76659dac57'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:54.970671 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1102/2182] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:54 2022] DELETE /identity/v3/users/35606939a51e4322b8a38f76659dac57 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:54.971967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:54.973439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:54.993182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:55.039002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e9d46-de50-484a-b917-71160fd044bd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:55.099836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a216c-315f-4d17-b5b5-0fcbd00f06b2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:55.111179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512fb2a-b7c8-47e5-b5bd-962f5a5d73de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:55.112989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512fb2a-b7c8-47e5-b5bd-962f5a5d73de admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:55.112989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512fb2a-b7c8-47e5-b5bd-962f5a5d73de admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:55.113340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512fb2a-b7c8-47e5-b5bd-962f5a5d73de admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:55.113853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512fb2a-b7c8-47e5-b5bd-962f5a5d73de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:55.118378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512fb2a-b7c8-47e5-b5bd-962f5a5d73de admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:55.170690 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1103/2183] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:54 2022] POST /identity/v3/users => generated 302 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:55.172157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7feaa6a-6cd5-4d8d-8f17-f7a259087fac None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:55.172854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7feaa6a-6cd5-4d8d-8f17-f7a259087fac None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:55.173168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7feaa6a-6cd5-4d8d-8f17-f7a259087fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:55.218305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a7feaa6a-6cd5-4d8d-8f17-f7a259087fac None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:55.221719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe6d5f-8c7b-4fbe-995c-eeaedc16bd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:55.222394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe6d5f-8c7b-4fbe-995c-eeaedc16bd40 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:55.222754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe6d5f-8c7b-4fbe-995c-eeaedc16bd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:55.223104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe6d5f-8c7b-4fbe-995c-eeaedc16bd40 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:55.223559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a7feaa6a-6cd5-4d8d-8f17-f7a259087fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:55.223901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe6d5f-8c7b-4fbe-995c-eeaedc16bd40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:55.225960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe6d5f-8c7b-4fbe-995c-eeaedc16bd40 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:55.247594 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1104/2184] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:55.247594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7feaa6a-6cd5-4d8d-8f17-f7a259087fac None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:55.264594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7feaa6a-6cd5-4d8d-8f17-f7a259087fac None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:55.275708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc2c89-45f1-44e1-b08e-85b64ae7fb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:55.276554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc2c89-45f1-44e1-b08e-85b64ae7fb03 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:55.276846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc2c89-45f1-44e1-b08e-85b64ae7fb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:55.277166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc2c89-45f1-44e1-b08e-85b64ae7fb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:55.277883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc2c89-45f1-44e1-b08e-85b64ae7fb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:55.282097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc2c89-45f1-44e1-b08e-85b64ae7fb03 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:55.283677 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1081/2185] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:54 2022] POST /identity/v3/projects => generated 403 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:55.284655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe6d5f-8c7b-4fbe-995c-eeaedc16bd40 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:55.290997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-38fe6d5f-8c7b-4fbe-995c-eeaedc16bd40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:55.310840 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1105/2186] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:55.310840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc2c89-45f1-44e1-b08e-85b64ae7fb03 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:55.330409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-11dc2c89-45f1-44e1-b08e-85b64ae7fb03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:55.348714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc2c89-45f1-44e1-b08e-85b64ae7fb03 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:55.394736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:55.455920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce09e69-6dd3-4729-8203-3b408bfe2d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:55.456695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce09e69-6dd3-4729-8203-3b408bfe2d0b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:55.457021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce09e69-6dd3-4729-8203-3b408bfe2d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:55.457304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce09e69-6dd3-4729-8203-3b408bfe2d0b admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:55.458138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce09e69-6dd3-4729-8203-3b408bfe2d0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:55.460121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce09e69-6dd3-4729-8203-3b408bfe2d0b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:55.485225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe6d5f-8c7b-4fbe-995c-eeaedc16bd40 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:55.499840 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1106/2187] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:55 2022] POST /identity/v3/users => generated 308 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:55.501510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56375f2-b15e-4023-af61-e269e13d5fba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:55.502343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56375f2-b15e-4023-af61-e269e13d5fba service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:55.503056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56375f2-b15e-4023-af61-e269e13d5fba service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:55.503449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56375f2-b15e-4023-af61-e269e13d5fba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:55.504135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce09e69-6dd3-4729-8203-3b408bfe2d0b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:55.529375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56375f2-b15e-4023-af61-e269e13d5fba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:55.530357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce09e69-6dd3-4729-8203-3b408bfe2d0b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:55.531115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56375f2-b15e-4023-af61-e269e13d5fba service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:55.541788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cd6b2-3094-42f2-ad83-a2103df5f29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:55.543641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cd6b2-3094-42f2-ad83-a2103df5f29e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:55.546700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cd6b2-3094-42f2-ad83-a2103df5f29e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:55.547046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cd6b2-3094-42f2-ad83-a2103df5f29e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:55.548200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cd6b2-3094-42f2-ad83-a2103df5f29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:55.548964 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1082/2188] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:55.549844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56375f2-b15e-4023-af61-e269e13d5fba service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:55.554737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cd6b2-3094-42f2-ad83-a2103df5f29e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:55.576190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56375f2-b15e-4023-af61-e269e13d5fba service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:55.577673 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1107/2189] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:55.579338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cd6b2-3094-42f2-ad83-a2103df5f29e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:55.587672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5826c89-0e1d-4fc3-9c8b-8e6f71b92edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:55.588507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5826c89-0e1d-4fc3-9c8b-8e6f71b92edd admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:55.588928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5826c89-0e1d-4fc3-9c8b-8e6f71b92edd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:55.589308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5826c89-0e1d-4fc3-9c8b-8e6f71b92edd admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:55.591293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5826c89-0e1d-4fc3-9c8b-8e6f71b92edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:55.595818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5826c89-0e1d-4fc3-9c8b-8e6f71b92edd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:55.599668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cd6b2-3094-42f2-ad83-a2103df5f29e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:55.607571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ced050-960b-4715-ab11-6e440ab3d589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:55.608663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ced050-960b-4715-ab11-6e440ab3d589 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:55.609347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ced050-960b-4715-ab11-6e440ab3d589 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:55.610042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ced050-960b-4715-ab11-6e440ab3d589 admin admin] PATH_INFO: `/v3/projects/68cdc132e95244478bf01490de3e0941/users/ee0fdf107484448d862ac3216df0d9dc/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:55.618395 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1083/2190] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:55.629980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5826c89-0e1d-4fc3-9c8b-8e6f71b92edd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:55.636029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ced050-960b-4715-ab11-6e440ab3d589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68cdc132e95244478bf01490de3e0941, user_id=ee0fdf107484448d862ac3216df0d9dc, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:55.640922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ced050-960b-4715-ab11-6e440ab3d589 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:55.644741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5826c89-0e1d-4fc3-9c8b-8e6f71b92edd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:55.662640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c21877-8fea-49b4-b79d-7824158c156d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:55.665037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c21877-8fea-49b4-b79d-7824158c156d admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:55.666630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c21877-8fea-49b4-b79d-7824158c156d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:55.668690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c21877-8fea-49b4-b79d-7824158c156d admin admin] PATH_INFO: `/v3/projects/e5de9aca20464f10b2ccdf65707c0443/users/20a2b116593b4601ac0b957f56e8ad7c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:55.678196 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1108/2191] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:55 2022] PUT /identity/v3/projects/68cdc132e95244478bf01490de3e0941/users/ee0fdf107484448d862ac3216df0d9dc/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:55.681661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027c3f7-f3de-4471-ab59-4da1028f0510 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:55.681986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027c3f7-f3de-4471-ab59-4da1028f0510 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:55.682287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027c3f7-f3de-4471-ab59-4da1028f0510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:55.725900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c21877-8fea-49b4-b79d-7824158c156d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5de9aca20464f10b2ccdf65707c0443, user_id=20a2b116593b4601ac0b957f56e8ad7c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:55.733611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c21877-8fea-49b4-b79d-7824158c156d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:55.747690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7027c3f7-f3de-4471-ab59-4da1028f0510 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:55.754694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7027c3f7-f3de-4471-ab59-4da1028f0510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:55.768639 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1084/2192] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:55 2022] PUT /identity/v3/projects/e5de9aca20464f10b2ccdf65707c0443/users/20a2b116593b4601ac0b957f56e8ad7c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:55.784885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c21877-8fea-49b4-b79d-7824158c156d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:55.951671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:55.970962 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1109/2193] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:55.983055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c21877-8fea-49b4-b79d-7824158c156d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:55.991113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027c3f7-f3de-4471-ab59-4da1028f0510 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:55.993656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7027c3f7-f3de-4471-ab59-4da1028f0510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:56.009296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56282db4-af48-4157-abf4-d5f4fc0a38ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:56.009939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56282db4-af48-4157-abf4-d5f4fc0a38ff admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:56.012030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56282db4-af48-4157-abf4-d5f4fc0a38ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:56.017918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56282db4-af48-4157-abf4-d5f4fc0a38ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:56.020511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56282db4-af48-4157-abf4-d5f4fc0a38ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:56.023406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56282db4-af48-4157-abf4-d5f4fc0a38ff admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:56.060957 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1085/2194] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:49:56.065466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56282db4-af48-4157-abf4-d5f4fc0a38ff admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:56.095398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56282db4-af48-4157-abf4-d5f4fc0a38ff admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:56.110246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dafdee-4396-47d0-bb04-7c0d0b6066f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:56.110875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dafdee-4396-47d0-bb04-7c0d0b6066f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:56.111180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dafdee-4396-47d0-bb04-7c0d0b6066f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:56.111464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dafdee-4396-47d0-bb04-7c0d0b6066f6 admin admin] PATH_INFO: `/v3/projects/e5de9aca20464f10b2ccdf65707c0443/users/20a2b116593b4601ac0b957f56e8ad7c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:56.126471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dafdee-4396-47d0-bb04-7c0d0b6066f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5de9aca20464f10b2ccdf65707c0443, user_id=20a2b116593b4601ac0b957f56e8ad7c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:56.136910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dafdee-4396-47d0-bb04-7c0d0b6066f6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:56.154625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027c3f7-f3de-4471-ab59-4da1028f0510 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:56.160421 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1086/2195] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:49:56 2022] PUT /identity/v3/projects/e5de9aca20464f10b2ccdf65707c0443/users/20a2b116593b4601ac0b957f56e8ad7c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:56.171594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362e58-aba1-4582-ae4b-317c64b024d5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:56.172033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362e58-aba1-4582-ae4b-317c64b024d5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:56.172478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362e58-aba1-4582-ae4b-317c64b024d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:56.198740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:56.225863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06c636-94ea-4f6b-862d-9eaceab91086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:56.226842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06c636-94ea-4f6b-862d-9eaceab91086 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:56.227237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06c636-94ea-4f6b-862d-9eaceab91086 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:56.228164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06c636-94ea-4f6b-862d-9eaceab91086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:56.241760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-96362e58-aba1-4582-ae4b-317c64b024d5 None None] MFA Rules not processed for user `20a2b116593b4601ac0b957f56e8ad7c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:56.244083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-96362e58-aba1-4582-ae4b-317c64b024d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:56.325553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06c636-94ea-4f6b-862d-9eaceab91086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:56.329173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06c636-94ea-4f6b-862d-9eaceab91086 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:56.377789 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1087/2196] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:56 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:56.378200 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1110/2197] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:56.388843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96362e58-aba1-4582-ae4b-317c64b024d5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:56.390731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-96362e58-aba1-4582-ae4b-317c64b024d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:56.495516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96362e58-aba1-4582-ae4b-317c64b024d5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:56.502015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0fb4d-bd1d-4839-8f83-67b198c07fc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:56.502887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0fb4d-bd1d-4839-8f83-67b198c07fc6 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:56.503002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0fb4d-bd1d-4839-8f83-67b198c07fc6 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:56.503206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0fb4d-bd1d-4839-8f83-67b198c07fc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:56.515010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0fb4d-bd1d-4839-8f83-67b198c07fc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:56.516801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0fb4d-bd1d-4839-8f83-67b198c07fc6 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:56.542009 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1088/2198] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:56 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:56.592357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06c636-94ea-4f6b-862d-9eaceab91086 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:56.607797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06c636-94ea-4f6b-862d-9eaceab91086 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:56.615595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcda8c2-743d-45c3-a9e0-1ca293ef3e85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:56.616200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcda8c2-743d-45c3-a9e0-1ca293ef3e85 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:56.617156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcda8c2-743d-45c3-a9e0-1ca293ef3e85 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:56.617437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcda8c2-743d-45c3-a9e0-1ca293ef3e85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:56.619375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3dcda8c2-743d-45c3-a9e0-1ca293ef3e85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:56.704953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0fb4d-bd1d-4839-8f83-67b198c07fc6 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:56.706770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-08e0fb4d-bd1d-4839-8f83-67b198c07fc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:56.725627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcda8c2-743d-45c3-a9e0-1ca293ef3e85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:56.729598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcda8c2-743d-45c3-a9e0-1ca293ef3e85 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:56.731640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0fb4d-bd1d-4839-8f83-67b198c07fc6 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:56.749926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:56.753581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:56.753581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:56.755009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] PATH_INFO: `/v3/projects/f2253308775749df8038947933483108` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:56.758468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2253308775749df8038947933483108)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:56.781949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:56.810559 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1111/2199] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:56 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:56.874626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:56.896827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2253308775749df8038947933483108', 'request_id': 'req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:56.910132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2253308775749df8038947933483108', 'request_id': 'req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:56.927716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2253308775749df8038947933483108', 'request_id': 'req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:56.960068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] The token cache is being invalidate because project f2253308775749df8038947933483108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:56.960556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:56.961446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:56.967552 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1089/2200] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:56 2022] DELETE /identity/v3/projects/f2253308775749df8038947933483108 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:56.968184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcda8c2-743d-45c3-a9e0-1ca293ef3e85 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:56.984352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcda8c2-743d-45c3-a9e0-1ca293ef3e85 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:57.108888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:57.109648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:57.109890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:57.110169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] PATH_INFO: `/v3/users/143b5a8df4704dd1a0e57e183718a64c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:57.116556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=143b5a8df4704dd1a0e57e183718a64c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:57.130810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:57.192053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:57.244026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '143b5a8df4704dd1a0e57e183718a64c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:57.266827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '143b5a8df4704dd1a0e57e183718a64c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:57.280552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '143b5a8df4704dd1a0e57e183718a64c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:57.281525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '143b5a8df4704dd1a0e57e183718a64c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:57.282654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '143b5a8df4704dd1a0e57e183718a64c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:57.302185 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1112/2201] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:56 2022] DELETE /identity/v3/users/143b5a8df4704dd1a0e57e183718a64c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:58.758255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:58.759955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:58.807550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f6c64-2fb2-4e51-93dd-f8d3992088c2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:58.927490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:58.932696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:59.007025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:59.009469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:59.010260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:59.010629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021 admin admin] PATH_INFO: `/v3/projects/1056967c67b440debf877be118e7e1a7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:59.013650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1056967c67b440debf877be118e7e1a7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:59.021577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:59.091547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960340fa-fd21-411c-b8ef-245ae2faa675 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:59.105653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e6901-cfb5-49b4-ba5c-386cedc5c949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:59.106632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e6901-cfb5-49b4-ba5c-386cedc5c949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:59.107063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e6901-cfb5-49b4-ba5c-386cedc5c949 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:59.107529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e6901-cfb5-49b4-ba5c-386cedc5c949 admin admin] PATH_INFO: `/v3/users/8756e3fd985440d1a29cb5f5b94a6ac1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:59.109374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e6901-cfb5-49b4-ba5c-386cedc5c949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8756e3fd985440d1a29cb5f5b94a6ac1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:59.111518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e6901-cfb5-49b4-ba5c-386cedc5c949 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:59.124515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1056967c67b440debf877be118e7e1a7', 'request_id': 'req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:59.141641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1056967c67b440debf877be118e7e1a7', 'request_id': 'req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:59.176232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1056967c67b440debf877be118e7e1a7', 'request_id': 'req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:59.188704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021 admin admin] The token cache is being invalidate because project 1056967c67b440debf877be118e7e1a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:49:59.189035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:59.189712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fcdfdad9-10c5-449b-b11c-c5a7c7b6b021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:59.191499 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1090/2202] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:49:58 2022] DELETE /identity/v3/projects/1056967c67b440debf877be118e7e1a7 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:49:59.196679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901b9c3-ecd3-4182-bf5d-847373d332b8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:59.197198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901b9c3-ecd3-4182-bf5d-847373d332b8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:59.197589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901b9c3-ecd3-4182-bf5d-847373d332b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:59.236395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8901b9c3-ecd3-4182-bf5d-847373d332b8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:59.240223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8901b9c3-ecd3-4182-bf5d-847373d332b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:59.242192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ba2e6901-cfb5-49b4-ba5c-386cedc5c949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8756e3fd985440d1a29cb5f5b94a6ac1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:59.281620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ba2e6901-cfb5-49b4-ba5c-386cedc5c949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8756e3fd985440d1a29cb5f5b94a6ac1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:59.298825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ba2e6901-cfb5-49b4-ba5c-386cedc5c949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8756e3fd985440d1a29cb5f5b94a6ac1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:59.299489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ba2e6901-cfb5-49b4-ba5c-386cedc5c949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8756e3fd985440d1a29cb5f5b94a6ac1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:59.300221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ba2e6901-cfb5-49b4-ba5c-386cedc5c949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8756e3fd985440d1a29cb5f5b94a6ac1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:49:59.327192 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1113/2203] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:49:58 2022] DELETE /identity/v3/users/8756e3fd985440d1a29cb5f5b94a6ac1 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:49:59.333113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d988a5-f20e-48e0-916a-cb0af323f4fb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:59.333113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d988a5-f20e-48e0-916a-cb0af323f4fb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:59.333113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d988a5-f20e-48e0-916a-cb0af323f4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:59.358646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:59.377757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-53d988a5-f20e-48e0-916a-cb0af323f4fb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:59.379017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-53d988a5-f20e-48e0-916a-cb0af323f4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:59.382062 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1091/2204] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:59.390609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544a2c2-e5dc-4753-aade-9477f785e56a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:59.391171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544a2c2-e5dc-4753-aade-9477f785e56a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:59.391626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544a2c2-e5dc-4753-aade-9477f785e56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:59.405449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:49:59.430856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e544a2c2-e5dc-4753-aade-9477f785e56a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:49:59.432358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e544a2c2-e5dc-4753-aade-9477f785e56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:59.493072 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1114/2205] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:59.510244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d988a5-f20e-48e0-916a-cb0af323f4fb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:59.514771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-53d988a5-f20e-48e0-916a-cb0af323f4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:49:59.558995 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1092/2206] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:49:59.560922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544a2c2-e5dc-4753-aade-9477f785e56a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:59.576222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544a2c2-e5dc-4753-aade-9477f785e56a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:59.581779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250df732-11c4-44fa-87b5-c5a9fc5d4770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:59.582530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250df732-11c4-44fa-87b5-c5a9fc5d4770 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:59.582924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250df732-11c4-44fa-87b5-c5a9fc5d4770 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:59.583312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250df732-11c4-44fa-87b5-c5a9fc5d4770 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:59.584082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250df732-11c4-44fa-87b5-c5a9fc5d4770 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:49:59.584669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250df732-11c4-44fa-87b5-c5a9fc5d4770 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:59.586463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250df732-11c4-44fa-87b5-c5a9fc5d4770 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:59.601411 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1093/2207] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:49:59.605433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250df732-11c4-44fa-87b5-c5a9fc5d4770 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:59.617555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250df732-11c4-44fa-87b5-c5a9fc5d4770 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:59.618820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d988a5-f20e-48e0-916a-cb0af323f4fb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:59.625813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fc1e4-1542-47bd-a1ea-c5bea69e528d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:59.627299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fc1e4-1542-47bd-a1ea-c5bea69e528d service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:59.627594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fc1e4-1542-47bd-a1ea-c5bea69e528d service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:59.627870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fc1e4-1542-47bd-a1ea-c5bea69e528d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:59.629798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20440cd0-059c-49d3-83e9-182085bcd701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:59.630412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20440cd0-059c-49d3-83e9-182085bcd701 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:59.630693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20440cd0-059c-49d3-83e9-182085bcd701 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:59.630967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20440cd0-059c-49d3-83e9-182085bcd701 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:59.632169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20440cd0-059c-49d3-83e9-182085bcd701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:59.638709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20440cd0-059c-49d3-83e9-182085bcd701 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:59.653588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fc1e4-1542-47bd-a1ea-c5bea69e528d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:59.655550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fc1e4-1542-47bd-a1ea-c5bea69e528d service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:59.689078 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1115/2208] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:49:59.706722 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1094/2209] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:49:59 2022] POST /identity/v3/projects => generated 409 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:49:59.711379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fc1e4-1542-47bd-a1ea-c5bea69e528d service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:59.736427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fc1e4-1542-47bd-a1ea-c5bea69e528d service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:59.848824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222df37d-679e-49ac-9280-17951b9cf40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:59.849921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222df37d-679e-49ac-9280-17951b9cf40c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:59.850318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222df37d-679e-49ac-9280-17951b9cf40c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:59.851051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222df37d-679e-49ac-9280-17951b9cf40c admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:59.851523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222df37d-679e-49ac-9280-17951b9cf40c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:59.860850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222df37d-679e-49ac-9280-17951b9cf40c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:59.913544 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1116/2210] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:49:59 2022] POST /identity/v3/users => generated 305 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:49:59.918030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20440cd0-059c-49d3-83e9-182085bcd701 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:59.934666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20440cd0-059c-49d3-83e9-182085bcd701 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:49:59.945195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481c1e7-465b-4b72-a3b5-c7e9ae06d47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:49:59.945833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481c1e7-465b-4b72-a3b5-c7e9ae06d47c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:49:59.946118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481c1e7-465b-4b72-a3b5-c7e9ae06d47c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:49:59.946394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481c1e7-465b-4b72-a3b5-c7e9ae06d47c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:49:59.949959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481c1e7-465b-4b72-a3b5-c7e9ae06d47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:49:59.959892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481c1e7-465b-4b72-a3b5-c7e9ae06d47c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:49:59.987964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222df37d-679e-49ac-9280-17951b9cf40c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:49:59.993344 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1095/2211] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:00.030436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222df37d-679e-49ac-9280-17951b9cf40c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:00.049031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a29047-5f7f-4b92-8ebf-85b212e5afa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:00.050954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a29047-5f7f-4b92-8ebf-85b212e5afa5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:00.052853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a29047-5f7f-4b92-8ebf-85b212e5afa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:00.053782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a29047-5f7f-4b92-8ebf-85b212e5afa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:00.056551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a29047-5f7f-4b92-8ebf-85b212e5afa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:00.063829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a29047-5f7f-4b92-8ebf-85b212e5afa5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:00.103283 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1117/2212] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:00.106751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481c1e7-465b-4b72-a3b5-c7e9ae06d47c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:00.122409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481c1e7-465b-4b72-a3b5-c7e9ae06d47c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:00.140257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de124b8c-1b60-4ba6-b5b0-94cfd6d781ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:00.141180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de124b8c-1b60-4ba6-b5b0-94cfd6d781ff admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:00.141685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de124b8c-1b60-4ba6-b5b0-94cfd6d781ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:00.142070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de124b8c-1b60-4ba6-b5b0-94cfd6d781ff admin admin] PATH_INFO: `/v3/projects/b7cc297574f444839169119af615179f/users/7dd518c8e7c94a8dbc2d484d4fec1ea9/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:00.179146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de124b8c-1b60-4ba6-b5b0-94cfd6d781ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7cc297574f444839169119af615179f, user_id=7dd518c8e7c94a8dbc2d484d4fec1ea9, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:00.182133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de124b8c-1b60-4ba6-b5b0-94cfd6d781ff admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:00.228714 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1096/2213] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:00 2022] PUT /identity/v3/projects/b7cc297574f444839169119af615179f/users/7dd518c8e7c94a8dbc2d484d4fec1ea9/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:00.239637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad83af-1519-4b4a-8101-33b8108dcf38 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:00.240169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad83af-1519-4b4a-8101-33b8108dcf38 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:00.240586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad83af-1519-4b4a-8101-33b8108dcf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:00.345044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-80ad83af-1519-4b4a-8101-33b8108dcf38 None None] MFA Rules not processed for user `7dd518c8e7c94a8dbc2d484d4fec1ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:00.346630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-80ad83af-1519-4b4a-8101-33b8108dcf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:00.458491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:00.562097 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1118/2214] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:00 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:00.567321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de124b8c-1b60-4ba6-b5b0-94cfd6d781ff admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:00.694307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de124b8c-1b60-4ba6-b5b0-94cfd6d781ff admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:00.699360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d3e26-456c-490c-aa83-b407788b0c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:00.700566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d3e26-456c-490c-aa83-b407788b0c58 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:00.701058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d3e26-456c-490c-aa83-b407788b0c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:00.701223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d3e26-456c-490c-aa83-b407788b0c58 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:00.701771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d3e26-456c-490c-aa83-b407788b0c58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:00.703711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d3e26-456c-490c-aa83-b407788b0c58 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:00.739396 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1097/2215] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:00 2022] POST /identity/v3/projects => generated 407 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:00.743549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad83af-1519-4b4a-8101-33b8108dcf38 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:00.760725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad83af-1519-4b4a-8101-33b8108dcf38 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:00.887282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afc3f7-c6cb-458d-92c8-5bed0c1dde79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:00.888161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afc3f7-c6cb-458d-92c8-5bed0c1dde79 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:00.888642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afc3f7-c6cb-458d-92c8-5bed0c1dde79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:00.889100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afc3f7-c6cb-458d-92c8-5bed0c1dde79 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:00.889899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afc3f7-c6cb-458d-92c8-5bed0c1dde79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:00.893761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afc3f7-c6cb-458d-92c8-5bed0c1dde79 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:00.948209 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1119/2216] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:00 2022] POST /identity/v3/users => generated 310 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:00.953010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d3e26-456c-490c-aa83-b407788b0c58 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:00.988370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d3e26-456c-490c-aa83-b407788b0c58 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:01.000694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f86d20-45d3-452e-ba3b-daab755bf31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:01.015387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f86d20-45d3-452e-ba3b-daab755bf31e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:01.016063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f86d20-45d3-452e-ba3b-daab755bf31e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:01.016505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f86d20-45d3-452e-ba3b-daab755bf31e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:01.017758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f86d20-45d3-452e-ba3b-daab755bf31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:01.020426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f86d20-45d3-452e-ba3b-daab755bf31e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:01.049529 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1098/2217] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:01.050219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afc3f7-c6cb-458d-92c8-5bed0c1dde79 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:01.071209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afc3f7-c6cb-458d-92c8-5bed0c1dde79 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:01.082151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89846e7c-d5a3-4cb7-917c-70795a095d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:01.085077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89846e7c-d5a3-4cb7-917c-70795a095d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:01.085726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89846e7c-d5a3-4cb7-917c-70795a095d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:01.086219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89846e7c-d5a3-4cb7-917c-70795a095d03 admin admin] PATH_INFO: `/v3/projects/ed95a87adbf74a5da729e65a31b04a5c/users/f36c6355836349169e2f409b5e306f1a/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:01.101748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f86d20-45d3-452e-ba3b-daab755bf31e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:01.103638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-84f86d20-45d3-452e-ba3b-daab755bf31e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:01.116517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:01.133796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89846e7c-d5a3-4cb7-917c-70795a095d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed95a87adbf74a5da729e65a31b04a5c, user_id=f36c6355836349169e2f409b5e306f1a, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:01.136622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89846e7c-d5a3-4cb7-917c-70795a095d03 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:01.137264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f86d20-45d3-452e-ba3b-daab755bf31e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:01.147842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:01.149063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:01.149419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:01.149795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] PATH_INFO: `/v3/projects/0f86e3171bdc40d6b4b303e6bf5e4a2c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:01.153762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f86e3171bdc40d6b4b303e6bf5e4a2c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:01.172182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:01.178949 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1120/2218] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:01 2022] PUT /identity/v3/projects/ed95a87adbf74a5da729e65a31b04a5c/users/f36c6355836349169e2f409b5e306f1a/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:01.183289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89846e7c-d5a3-4cb7-917c-70795a095d03 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:01.264931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f86e3171bdc40d6b4b303e6bf5e4a2c', 'request_id': 'req-5193493e-c019-4e4e-9019-1ab9f2c37581'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:01.279119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f86e3171bdc40d6b4b303e6bf5e4a2c', 'request_id': 'req-5193493e-c019-4e4e-9019-1ab9f2c37581'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:01.287249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f86e3171bdc40d6b4b303e6bf5e4a2c', 'request_id': 'req-5193493e-c019-4e4e-9019-1ab9f2c37581'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:01.303111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89846e7c-d5a3-4cb7-917c-70795a095d03 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:01.314499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] The token cache is being invalidate because project 0f86e3171bdc40d6b4b303e6bf5e4a2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:01.315130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:01.316016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:01.323458 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1099/2219] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:01 2022] DELETE /identity/v3/projects/0f86e3171bdc40d6b4b303e6bf5e4a2c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:01.324845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:01.326525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:01.387651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:01.418601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193493e-c019-4e4e-9019-1ab9f2c37581 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:01.424067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:01.424916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:01.425460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:01.425911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] PATH_INFO: `/v3/users/20a2b116593b4601ac0b957f56e8ad7c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:01.429254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] RBAC: Authorizing `identity:delete_user(user_id=20a2b116593b4601ac0b957f56e8ad7c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:01.429413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81152b14-0f9c-4081-907a-13bfd4507b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:01.430393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81152b14-0f9c-4081-907a-13bfd4507b39 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:01.430393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81152b14-0f9c-4081-907a-13bfd4507b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:01.430815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81152b14-0f9c-4081-907a-13bfd4507b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:01.433364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:01.434607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81152b14-0f9c-4081-907a-13bfd4507b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:01.436225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81152b14-0f9c-4081-907a-13bfd4507b39 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:01.464006 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1121/2220] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:01.464933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81152b14-0f9c-4081-907a-13bfd4507b39 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:01.467351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-81152b14-0f9c-4081-907a-13bfd4507b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:01.518641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81152b14-0f9c-4081-907a-13bfd4507b39 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:01.522180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20a2b116593b4601ac0b957f56e8ad7c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:01.532558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503cd184-658d-40d2-8097-0e597a46c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:01.533287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cd184-658d-40d2-8097-0e597a46c1c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:01.533576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cd184-658d-40d2-8097-0e597a46c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:01.533862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cd184-658d-40d2-8097-0e597a46c1c6 admin admin] PATH_INFO: `/v3/projects/ed95a87adbf74a5da729e65a31b04a5c/users/f36c6355836349169e2f409b5e306f1a/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:01.537922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503cd184-658d-40d2-8097-0e597a46c1c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed95a87adbf74a5da729e65a31b04a5c, user_id=f36c6355836349169e2f409b5e306f1a, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:01.543447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503cd184-658d-40d2-8097-0e597a46c1c6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:01.552217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20a2b116593b4601ac0b957f56e8ad7c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:01.554877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20a2b116593b4601ac0b957f56e8ad7c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:01.565049 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1122/2221] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:01 2022] PUT /identity/v3/projects/ed95a87adbf74a5da729e65a31b04a5c/users/f36c6355836349169e2f409b5e306f1a/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:01.572029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20a2b116593b4601ac0b957f56e8ad7c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:01.578906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c37fd-c9fa-497b-a713-18d0d67a30e3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:01.579254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c37fd-c9fa-497b-a713-18d0d67a30e3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:01.579541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c37fd-c9fa-497b-a713-18d0d67a30e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:01.589493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20a2b116593b4601ac0b957f56e8ad7c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:01.613546 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1100/2222] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:01 2022] DELETE /identity/v3/users/20a2b116593b4601ac0b957f56e8ad7c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:01.635208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e65c37fd-c9fa-497b-a713-18d0d67a30e3 None None] MFA Rules not processed for user `f36c6355836349169e2f409b5e306f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:01.636534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e65c37fd-c9fa-497b-a713-18d0d67a30e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:01.763069 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1123/2223] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:01 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:01.770792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:01.774063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:01.796578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:01.956148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9722311-efe7-4583-8298-a208a611415a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:01.970066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fecefc-7062-485e-bcb8-eebaff18fe18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:01.970915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fecefc-7062-485e-bcb8-eebaff18fe18 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:01.971209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fecefc-7062-485e-bcb8-eebaff18fe18 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:01.971652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fecefc-7062-485e-bcb8-eebaff18fe18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:01.990356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fecefc-7062-485e-bcb8-eebaff18fe18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:01.992194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fecefc-7062-485e-bcb8-eebaff18fe18 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:02.027060 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1101/2224] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:01 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:02.307915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c37fd-c9fa-497b-a713-18d0d67a30e3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:02.310098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e65c37fd-c9fa-497b-a713-18d0d67a30e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:02.358945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c37fd-c9fa-497b-a713-18d0d67a30e3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:02.374216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a1ab6-876e-48cc-98fb-025c7a032366 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:02.375271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a1ab6-876e-48cc-98fb-025c7a032366 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:02.377417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a1ab6-876e-48cc-98fb-025c7a032366 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:02.378016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a1ab6-876e-48cc-98fb-025c7a032366 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:02.383283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-731a1ab6-876e-48cc-98fb-025c7a032366 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:02.560388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a1ab6-876e-48cc-98fb-025c7a032366 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:02.562254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a1ab6-876e-48cc-98fb-025c7a032366 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:02.592233 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1124/2225] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:02 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:02.694580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fecefc-7062-485e-bcb8-eebaff18fe18 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:02.696490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f4fecefc-7062-485e-bcb8-eebaff18fe18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:02.905661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fecefc-7062-485e-bcb8-eebaff18fe18 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:02.936530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46443230-bb7f-4fde-bf3e-6cc9878c52ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:02.937237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46443230-bb7f-4fde-bf3e-6cc9878c52ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:02.937528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46443230-bb7f-4fde-bf3e-6cc9878c52ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:02.940752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46443230-bb7f-4fde-bf3e-6cc9878c52ac admin admin] PATH_INFO: `/v3/projects/e5de9aca20464f10b2ccdf65707c0443` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:02.946248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46443230-bb7f-4fde-bf3e-6cc9878c52ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5de9aca20464f10b2ccdf65707c0443)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:02.948997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46443230-bb7f-4fde-bf3e-6cc9878c52ac admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:03.039773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-46443230-bb7f-4fde-bf3e-6cc9878c52ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5de9aca20464f10b2ccdf65707c0443', 'request_id': 'req-46443230-bb7f-4fde-bf3e-6cc9878c52ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:03.049191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-46443230-bb7f-4fde-bf3e-6cc9878c52ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5de9aca20464f10b2ccdf65707c0443', 'request_id': 'req-46443230-bb7f-4fde-bf3e-6cc9878c52ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:03.057474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-46443230-bb7f-4fde-bf3e-6cc9878c52ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5de9aca20464f10b2ccdf65707c0443', 'request_id': 'req-46443230-bb7f-4fde-bf3e-6cc9878c52ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:03.069873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-46443230-bb7f-4fde-bf3e-6cc9878c52ac admin admin] The token cache is being invalidate because project e5de9aca20464f10b2ccdf65707c0443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:03.070382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-46443230-bb7f-4fde-bf3e-6cc9878c52ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:03.071328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-46443230-bb7f-4fde-bf3e-6cc9878c52ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:03.073614 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1102/2226] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:02 2022] DELETE /identity/v3/projects/e5de9aca20464f10b2ccdf65707c0443 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:03.079725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f387a3-6e28-43e5-b2bf-e4c5104e5a89 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:03.080035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f387a3-6e28-43e5-b2bf-e4c5104e5a89 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:03.080366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f387a3-6e28-43e5-b2bf-e4c5104e5a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:03.157808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-10f387a3-6e28-43e5-b2bf-e4c5104e5a89 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:03.164752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-10f387a3-6e28-43e5-b2bf-e4c5104e5a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:03.243479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:03.263656 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1125/2227] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:03.271856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713f3f4-167e-4077-9a3b-249d9ae02544 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:03.272090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713f3f4-167e-4077-9a3b-249d9ae02544 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:03.272381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713f3f4-167e-4077-9a3b-249d9ae02544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:03.351068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e713f3f4-167e-4077-9a3b-249d9ae02544 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:03.352432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e713f3f4-167e-4077-9a3b-249d9ae02544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:03.381132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f387a3-6e28-43e5-b2bf-e4c5104e5a89 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:03.382827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-10f387a3-6e28-43e5-b2bf-e4c5104e5a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:03.389716 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1103/2228] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:03.390967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713f3f4-167e-4077-9a3b-249d9ae02544 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:03.407197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713f3f4-167e-4077-9a3b-249d9ae02544 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:03.423404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ec326-6c65-4cfb-b29f-a943fac45c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:03.424062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ec326-6c65-4cfb-b29f-a943fac45c6b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:03.424345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ec326-6c65-4cfb-b29f-a943fac45c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:03.424988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ec326-6c65-4cfb-b29f-a943fac45c6b admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:03.425283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ec326-6c65-4cfb-b29f-a943fac45c6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:03.425590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ec326-6c65-4cfb-b29f-a943fac45c6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:03.427199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ec326-6c65-4cfb-b29f-a943fac45c6b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:03.435601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f387a3-6e28-43e5-b2bf-e4c5104e5a89 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:03.447399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:03.456230 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1104/2229] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:50:03.460754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53b39a-3d30-4755-ac6c-84abe7533cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:03.462077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ec326-6c65-4cfb-b29f-a943fac45c6b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:03.462464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53b39a-3d30-4755-ac6c-84abe7533cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:03.462820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53b39a-3d30-4755-ac6c-84abe7533cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:03.466247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53b39a-3d30-4755-ac6c-84abe7533cd8 admin admin] PATH_INFO: `/v3/users/7dd518c8e7c94a8dbc2d484d4fec1ea9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:03.471102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53b39a-3d30-4755-ac6c-84abe7533cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dd518c8e7c94a8dbc2d484d4fec1ea9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:03.479114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53b39a-3d30-4755-ac6c-84abe7533cd8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:03.494052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ec326-6c65-4cfb-b29f-a943fac45c6b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:03.509376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0763d-d2bc-4005-9882-0ae02e92d378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:03.510010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0763d-d2bc-4005-9882-0ae02e92d378 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:03.510292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0763d-d2bc-4005-9882-0ae02e92d378 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:03.510564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0763d-d2bc-4005-9882-0ae02e92d378 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:03.515421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0763d-d2bc-4005-9882-0ae02e92d378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:03.521039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0763d-d2bc-4005-9882-0ae02e92d378 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:03.574422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8f53b39a-3d30-4755-ac6c-84abe7533cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dd518c8e7c94a8dbc2d484d4fec1ea9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:03.584435 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1105/2230] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:03 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:03.585505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0763d-d2bc-4005-9882-0ae02e92d378 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:03.594190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8f53b39a-3d30-4755-ac6c-84abe7533cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dd518c8e7c94a8dbc2d484d4fec1ea9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:03.604702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8f53b39a-3d30-4755-ac6c-84abe7533cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dd518c8e7c94a8dbc2d484d4fec1ea9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:03.605559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8f53b39a-3d30-4755-ac6c-84abe7533cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dd518c8e7c94a8dbc2d484d4fec1ea9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:03.606695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8f53b39a-3d30-4755-ac6c-84abe7533cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dd518c8e7c94a8dbc2d484d4fec1ea9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:03.624993 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1126/2231] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:03 2022] DELETE /identity/v3/users/7dd518c8e7c94a8dbc2d484d4fec1ea9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:50:03.630042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e967632-1978-475c-bb28-cbf1dfae2ded None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:03.630447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e967632-1978-475c-bb28-cbf1dfae2ded None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:03.630794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e967632-1978-475c-bb28-cbf1dfae2ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:03.654279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9e967632-1978-475c-bb28-cbf1dfae2ded None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:03.655819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9e967632-1978-475c-bb28-cbf1dfae2ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:03.699371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0763d-d2bc-4005-9882-0ae02e92d378 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:03.756673 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1127/2232] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:50:03.768116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e967632-1978-475c-bb28-cbf1dfae2ded None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:03.770075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9e967632-1978-475c-bb28-cbf1dfae2ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:03.791012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325dad8-0434-4fc2-ba86-e6d67b98a42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:03.792150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325dad8-0434-4fc2-ba86-e6d67b98a42e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:03.792793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325dad8-0434-4fc2-ba86-e6d67b98a42e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:03.793121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325dad8-0434-4fc2-ba86-e6d67b98a42e admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:03.794412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325dad8-0434-4fc2-ba86-e6d67b98a42e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:03.797454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325dad8-0434-4fc2-ba86-e6d67b98a42e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:03.831262 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1106/2233] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:03 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:03.833132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325dad8-0434-4fc2-ba86-e6d67b98a42e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:03.835400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3325dad8-0434-4fc2-ba86-e6d67b98a42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:03.850287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325dad8-0434-4fc2-ba86-e6d67b98a42e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:03.855848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a0054-1494-4ede-89c4-10fc5f431598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:03.856617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a0054-1494-4ede-89c4-10fc5f431598 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:03.857158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a0054-1494-4ede-89c4-10fc5f431598 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:03.857441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a0054-1494-4ede-89c4-10fc5f431598 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:03.858103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a0054-1494-4ede-89c4-10fc5f431598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:03.859767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a0054-1494-4ede-89c4-10fc5f431598 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:03.869188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e967632-1978-475c-bb28-cbf1dfae2ded None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:03.872665 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1107/2234] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:03.874073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae706678-f981-4219-a83f-0062c2a6ea03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:03.875183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae706678-f981-4219-a83f-0062c2a6ea03 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:03.875444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae706678-f981-4219-a83f-0062c2a6ea03 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:03.875730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae706678-f981-4219-a83f-0062c2a6ea03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:03.876824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a0054-1494-4ede-89c4-10fc5f431598 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:03.897673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae706678-f981-4219-a83f-0062c2a6ea03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:03.898449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a0054-1494-4ede-89c4-10fc5f431598 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:03.899203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae706678-f981-4219-a83f-0062c2a6ea03 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:03.905315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c94722-ec49-40ea-acf7-f6f9ac1d5f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:03.906578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c94722-ec49-40ea-acf7-f6f9ac1d5f81 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:03.906941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c94722-ec49-40ea-acf7-f6f9ac1d5f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:03.907420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c94722-ec49-40ea-acf7-f6f9ac1d5f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:03.908406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c94722-ec49-40ea-acf7-f6f9ac1d5f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:03.910257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c94722-ec49-40ea-acf7-f6f9ac1d5f81 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:03.912791 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1128/2235] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:50:03.923822 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1108/2236] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:03.928797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae706678-f981-4219-a83f-0062c2a6ea03 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:03.951786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae706678-f981-4219-a83f-0062c2a6ea03 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:03.959815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41694d25-3ad9-4e50-9445-717eb1b3fbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:03.960642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41694d25-3ad9-4e50-9445-717eb1b3fbf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:03.961078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41694d25-3ad9-4e50-9445-717eb1b3fbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:03.961454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41694d25-3ad9-4e50-9445-717eb1b3fbf9 admin admin] PATH_INFO: `/v3/projects/3fb8975b0bc94e448ec0b9076504abf4/users/5ca11a4dc0a54f74a176a2aeb1974cf6/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:03.964784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:03.999370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41694d25-3ad9-4e50-9445-717eb1b3fbf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb8975b0bc94e448ec0b9076504abf4, user_id=5ca11a4dc0a54f74a176a2aeb1974cf6, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:04.016427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41694d25-3ad9-4e50-9445-717eb1b3fbf9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:04.077530 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1129/2237] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:03 2022] PUT /identity/v3/projects/3fb8975b0bc94e448ec0b9076504abf4/users/5ca11a4dc0a54f74a176a2aeb1974cf6/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:50:04.087412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5d76a-8eb1-4dfb-93a7-ac1712878041 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:04.087870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5d76a-8eb1-4dfb-93a7-ac1712878041 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:04.087955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5d76a-8eb1-4dfb-93a7-ac1712878041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:04.201954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5fa5d76a-8eb1-4dfb-93a7-ac1712878041 None None] MFA Rules not processed for user `5ca11a4dc0a54f74a176a2aeb1974cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:04.203218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5fa5d76a-8eb1-4dfb-93a7-ac1712878041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:04.372548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:04.427295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41694d25-3ad9-4e50-9445-717eb1b3fbf9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:04.435239 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1109/2238] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:04.547033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41694d25-3ad9-4e50-9445-717eb1b3fbf9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:04.560512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49c803-e83b-4022-9cd3-578e958effa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:04.561916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49c803-e83b-4022-9cd3-578e958effa0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:04.562210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49c803-e83b-4022-9cd3-578e958effa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:04.562486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49c803-e83b-4022-9cd3-578e958effa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:04.563302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49c803-e83b-4022-9cd3-578e958effa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:04.565474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49c803-e83b-4022-9cd3-578e958effa0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:04.615689 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1130/2239] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:04 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:04.616960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5d76a-8eb1-4dfb-93a7-ac1712878041 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:04.634816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5d76a-8eb1-4dfb-93a7-ac1712878041 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:04.776378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72bf37-071d-4981-bdf3-dc5df882baec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:04.778373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72bf37-071d-4981-bdf3-dc5df882baec admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:04.778612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72bf37-071d-4981-bdf3-dc5df882baec admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:04.778945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72bf37-071d-4981-bdf3-dc5df882baec admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:04.779637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72bf37-071d-4981-bdf3-dc5df882baec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:04.783864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72bf37-071d-4981-bdf3-dc5df882baec admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:04.829045 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1110/2240] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:04 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:50:04.831014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49c803-e83b-4022-9cd3-578e958effa0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:04.845084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49c803-e83b-4022-9cd3-578e958effa0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:04.861162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4916df5-bff6-44c6-af03-665a4d980eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:04.861953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4916df5-bff6-44c6-af03-665a4d980eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:04.862285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4916df5-bff6-44c6-af03-665a4d980eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:04.862725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4916df5-bff6-44c6-af03-665a4d980eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:04.863381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4916df5-bff6-44c6-af03-665a4d980eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:04.868051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4916df5-bff6-44c6-af03-665a4d980eb5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:04.899750 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1131/2241] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:04.899955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72bf37-071d-4981-bdf3-dc5df882baec admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:04.916305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72bf37-071d-4981-bdf3-dc5df882baec admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:04.927901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b952ae8-cca2-481d-9173-cb41ed932237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:04.928332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b952ae8-cca2-481d-9173-cb41ed932237 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:04.928629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b952ae8-cca2-481d-9173-cb41ed932237 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:04.931401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b952ae8-cca2-481d-9173-cb41ed932237 admin admin] PATH_INFO: `/v3/projects/27c2d8d3613c4945ace46a1ec4f70d67/users/42d03397519e435483e59bb0911dac61/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:04.978552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b952ae8-cca2-481d-9173-cb41ed932237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c2d8d3613c4945ace46a1ec4f70d67, user_id=42d03397519e435483e59bb0911dac61, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:04.981442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b952ae8-cca2-481d-9173-cb41ed932237 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:05.016210 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1111/2242] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:04 2022] PUT /identity/v3/projects/27c2d8d3613c4945ace46a1ec4f70d67/users/42d03397519e435483e59bb0911dac61/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:05.016869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4916df5-bff6-44c6-af03-665a4d980eb5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:05.034653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4916df5-bff6-44c6-af03-665a4d980eb5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:05.162913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b952ae8-cca2-481d-9173-cb41ed932237 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:05.168273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7b952ae8-cca2-481d-9173-cb41ed932237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:05.189805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c846a-1865-4017-ba37-80644dab8da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:05.190435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c846a-1865-4017-ba37-80644dab8da7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:05.190723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c846a-1865-4017-ba37-80644dab8da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:05.191002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c846a-1865-4017-ba37-80644dab8da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:05.191655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c846a-1865-4017-ba37-80644dab8da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:05.193369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c846a-1865-4017-ba37-80644dab8da7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:05.211628 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1132/2243] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:05.223955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c846a-1865-4017-ba37-80644dab8da7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:05.237755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c846a-1865-4017-ba37-80644dab8da7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:05.245254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb5cdc-d798-4012-9aab-5e409789e511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:05.246187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb5cdc-d798-4012-9aab-5e409789e511 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:05.247318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb5cdc-d798-4012-9aab-5e409789e511 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:05.247772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb5cdc-d798-4012-9aab-5e409789e511 admin admin] PATH_INFO: `/v3/projects/27c2d8d3613c4945ace46a1ec4f70d67/users/42d03397519e435483e59bb0911dac61/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:05.250327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb5cdc-d798-4012-9aab-5e409789e511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c2d8d3613c4945ace46a1ec4f70d67, user_id=42d03397519e435483e59bb0911dac61, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:05.254001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb5cdc-d798-4012-9aab-5e409789e511 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:05.259224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b952ae8-cca2-481d-9173-cb41ed932237 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:05.274979 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1133/2244] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:05 2022] PUT /identity/v3/projects/27c2d8d3613c4945ace46a1ec4f70d67/users/42d03397519e435483e59bb0911dac61/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:05.276246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb5cdc-d798-4012-9aab-5e409789e511 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:05.278775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b9cb5cdc-d798-4012-9aab-5e409789e511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:05.283782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cbc2e-aa9e-44b3-b95f-106183ca3327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:05.289633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cbc2e-aa9e-44b3-b95f-106183ca3327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:05.290219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cbc2e-aa9e-44b3-b95f-106183ca3327 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:05.290423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cbc2e-aa9e-44b3-b95f-106183ca3327 admin admin] PATH_INFO: `/v3/users/c34f1d07f48c4b4dadf97fd38709fd33` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:05.291927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cbc2e-aa9e-44b3-b95f-106183ca3327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c34f1d07f48c4b4dadf97fd38709fd33)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:05.295464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cbc2e-aa9e-44b3-b95f-106183ca3327 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:05.355134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:05.365138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-125cbc2e-aa9e-44b3-b95f-106183ca3327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34f1d07f48c4b4dadf97fd38709fd33'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.367742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:05.385355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb5cdc-d798-4012-9aab-5e409789e511 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:05.387278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-125cbc2e-aa9e-44b3-b95f-106183ca3327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34f1d07f48c4b4dadf97fd38709fd33'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.388343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-125cbc2e-aa9e-44b3-b95f-106183ca3327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34f1d07f48c4b4dadf97fd38709fd33'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.391769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332e19c-572f-49d0-ab76-146a15d188ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:05.392566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332e19c-572f-49d0-ab76-146a15d188ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:05.393010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332e19c-572f-49d0-ab76-146a15d188ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:05.393425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332e19c-572f-49d0-ab76-146a15d188ed admin admin] PATH_INFO: `/v3/projects/b7cc297574f444839169119af615179f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:05.394889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332e19c-572f-49d0-ab76-146a15d188ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7cc297574f444839169119af615179f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:05.397059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332e19c-572f-49d0-ab76-146a15d188ed admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:05.403091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-125cbc2e-aa9e-44b3-b95f-106183ca3327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34f1d07f48c4b4dadf97fd38709fd33'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.427925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-125cbc2e-aa9e-44b3-b95f-106183ca3327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34f1d07f48c4b4dadf97fd38709fd33'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.447330 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1112/2245] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:05 2022] DELETE /identity/v3/users/c34f1d07f48c4b4dadf97fd38709fd33 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:05.447655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb11be-7a35-466a-87fe-5b8cde2f6194 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:05.447954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb11be-7a35-466a-87fe-5b8cde2f6194 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:05.448239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb11be-7a35-466a-87fe-5b8cde2f6194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:05.464679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a332e19c-572f-49d0-ab76-146a15d188ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7cc297574f444839169119af615179f', 'request_id': 'req-a332e19c-572f-49d0-ab76-146a15d188ed'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.477395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a332e19c-572f-49d0-ab76-146a15d188ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7cc297574f444839169119af615179f', 'request_id': 'req-a332e19c-572f-49d0-ab76-146a15d188ed'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.490129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a332e19c-572f-49d0-ab76-146a15d188ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7cc297574f444839169119af615179f', 'request_id': 'req-a332e19c-572f-49d0-ab76-146a15d188ed'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.511661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a332e19c-572f-49d0-ab76-146a15d188ed admin admin] The token cache is being invalidate because project b7cc297574f444839169119af615179f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:05.512055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a332e19c-572f-49d0-ab76-146a15d188ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.513528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a332e19c-572f-49d0-ab76-146a15d188ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.518559 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1134/2246] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:05 2022] DELETE /identity/v3/projects/b7cc297574f444839169119af615179f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:05.518559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7caf2-6059-4eff-8e7e-cf54ab5e86ef None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:05.518559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7caf2-6059-4eff-8e7e-cf54ab5e86ef None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:05.518559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7caf2-6059-4eff-8e7e-cf54ab5e86ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:05.546747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5dcb11be-7a35-466a-87fe-5b8cde2f6194 None None] MFA Rules not processed for user `42d03397519e435483e59bb0911dac61`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:05.548030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5dcb11be-7a35-466a-87fe-5b8cde2f6194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:05.552077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d0a7caf2-6059-4eff-8e7e-cf54ab5e86ef None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:05.553932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d0a7caf2-6059-4eff-8e7e-cf54ab5e86ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:05.635370 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1135/2247] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:05.636216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7caf2-6059-4eff-8e7e-cf54ab5e86ef None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:05.638637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d0a7caf2-6059-4eff-8e7e-cf54ab5e86ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:05.655957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7caf2-6059-4eff-8e7e-cf54ab5e86ef None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:05.668438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:05.677504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:05.677856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:05.678123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] PATH_INFO: `/v3/users/f36c6355836349169e2f409b5e306f1a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:05.679505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f36c6355836349169e2f409b5e306f1a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:05.681579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:05.765051 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1113/2248] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:05.765051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb11be-7a35-466a-87fe-5b8cde2f6194 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:05.779975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5dcb11be-7a35-466a-87fe-5b8cde2f6194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:05.792675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f36c6355836349169e2f409b5e306f1a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.815818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f36c6355836349169e2f409b5e306f1a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.819276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:05.830453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f36c6355836349169e2f409b5e306f1a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.831169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f36c6355836349169e2f409b5e306f1a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.831899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f36c6355836349169e2f409b5e306f1a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:05.860432 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1136/2249] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:05 2022] DELETE /identity/v3/users/f36c6355836349169e2f409b5e306f1a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:05.873578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:05.877640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:05.951647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb11be-7a35-466a-87fe-5b8cde2f6194 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:05.995266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c19f5-6c6a-433b-a853-3eb556ba6f02 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:06.001135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:06.001994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:06.002530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:06.002973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] PATH_INFO: `/v3/users/4c3a73b54fd54a23808657c7ed9e62cb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:06.004338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c3a73b54fd54a23808657c7ed9e62cb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:06.006944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:06.073914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c3a73b54fd54a23808657c7ed9e62cb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:06.078281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0827942-bed4-4ede-9817-cd63d9283644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:06.078942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0827942-bed4-4ede-9817-cd63d9283644 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:06.079101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0827942-bed4-4ede-9817-cd63d9283644 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:06.079382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0827942-bed4-4ede-9817-cd63d9283644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:06.081358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d0827942-bed4-4ede-9817-cd63d9283644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:06.092500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c3a73b54fd54a23808657c7ed9e62cb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:06.103712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c3a73b54fd54a23808657c7ed9e62cb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:06.105049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c3a73b54fd54a23808657c7ed9e62cb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:06.105759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c3a73b54fd54a23808657c7ed9e62cb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:06.117424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0827942-bed4-4ede-9817-cd63d9283644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:06.118903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0827942-bed4-4ede-9817-cd63d9283644 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:06.127411 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1137/2250] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:05 2022] DELETE /identity/v3/users/4c3a73b54fd54a23808657c7ed9e62cb => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:06.127411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:06.133565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:06.148756 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1114/2251] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:06.151902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0479d-d077-4454-b1f6-0e4fd11ff76b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:06.152194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0479d-d077-4454-b1f6-0e4fd11ff76b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:06.152742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0479d-d077-4454-b1f6-0e4fd11ff76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:06.192260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b7f0479d-d077-4454-b1f6-0e4fd11ff76b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:06.201321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b7f0479d-d077-4454-b1f6-0e4fd11ff76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:06.218331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:06.228414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:06.275738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0387f2-5f04-43e9-a8bc-eeafe1e5700c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:06.289568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f24f8-9f39-43c6-9005-3869a074c6c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:06.290209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f24f8-9f39-43c6-9005-3869a074c6c6 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:06.290500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f24f8-9f39-43c6-9005-3869a074c6c6 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:06.290782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f24f8-9f39-43c6-9005-3869a074c6c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:06.292666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-432f24f8-9f39-43c6-9005-3869a074c6c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:06.340160 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1115/2252] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:06.356724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0479d-d077-4454-b1f6-0e4fd11ff76b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:06.360048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b7f0479d-d077-4454-b1f6-0e4fd11ff76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:06.426985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f24f8-9f39-43c6-9005-3869a074c6c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:06.428658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f24f8-9f39-43c6-9005-3869a074c6c6 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:06.453512 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1138/2253] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:06.472735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0479d-d077-4454-b1f6-0e4fd11ff76b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:06.477065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d22b2-117f-4aa4-8722-b41a7587b102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:06.477821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d22b2-117f-4aa4-8722-b41a7587b102 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:06.478212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d22b2-117f-4aa4-8722-b41a7587b102 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:06.478596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d22b2-117f-4aa4-8722-b41a7587b102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:06.494038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d22b2-117f-4aa4-8722-b41a7587b102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:06.497226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d22b2-117f-4aa4-8722-b41a7587b102 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:06.511261 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1116/2254] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:50:07.089362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f24f8-9f39-43c6-9005-3869a074c6c6 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:07.091760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-432f24f8-9f39-43c6-9005-3869a074c6c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:07.115775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f24f8-9f39-43c6-9005-3869a074c6c6 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:07.163837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:07.164644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:07.165065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:07.165448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] PATH_INFO: `/v3/projects/123134ed03c540b79122e8ecaa021d3d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:07.171457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=123134ed03c540b79122e8ecaa021d3d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:07.174224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:07.234092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d22b2-117f-4aa4-8722-b41a7587b102 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:07.235633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a20d22b2-117f-4aa4-8722-b41a7587b102 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:07.258600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d22b2-117f-4aa4-8722-b41a7587b102 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:07.276408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:07.277482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:07.278032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:07.279404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] PATH_INFO: `/v3/projects/993b7f76855c47e58427d7542a0f41c2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:07.281263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=993b7f76855c47e58427d7542a0f41c2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:07.283385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:07.314177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '123134ed03c540b79122e8ecaa021d3d', 'request_id': 'req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.330241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '123134ed03c540b79122e8ecaa021d3d', 'request_id': 'req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.344638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '123134ed03c540b79122e8ecaa021d3d', 'request_id': 'req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.377229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] The token cache is being invalidate because project 123134ed03c540b79122e8ecaa021d3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:07.377531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.378460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993b7f76855c47e58427d7542a0f41c2', 'request_id': 'req-94634cec-f5a0-477b-9f5f-4acbbc3978ba'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.381592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.389247 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1139/2255] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:07 2022] DELETE /identity/v3/projects/123134ed03c540b79122e8ecaa021d3d => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:07.389247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:07.391320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:07.398897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993b7f76855c47e58427d7542a0f41c2', 'request_id': 'req-94634cec-f5a0-477b-9f5f-4acbbc3978ba'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.408676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993b7f76855c47e58427d7542a0f41c2', 'request_id': 'req-94634cec-f5a0-477b-9f5f-4acbbc3978ba'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.425018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] The token cache is being invalidate because project 993b7f76855c47e58427d7542a0f41c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:07.425624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.435112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.441079 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1117/2256] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:07 2022] DELETE /identity/v3/projects/993b7f76855c47e58427d7542a0f41c2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:07.442561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:07.446433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:07.546681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:07.571266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dac74-a376-4ad6-a3b9-5b33dcf4db33 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:07.583418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:07.584084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:07.584374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:07.584660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] PATH_INFO: `/v3/users/5ca11a4dc0a54f74a176a2aeb1974cf6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:07.585878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca11a4dc0a54f74a176a2aeb1974cf6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:07.587829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:07.592302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94634cec-f5a0-477b-9f5f-4acbbc3978ba admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:07.598360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:07.599169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:07.599990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:07.600375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] PATH_INFO: `/v3/users/ff31a8e9b6584e059808c9ad887bd698` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:07.602240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff31a8e9b6584e059808c9ad887bd698)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:07.604021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:07.634018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:07.687211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca11a4dc0a54f74a176a2aeb1974cf6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.695229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff31a8e9b6584e059808c9ad887bd698'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.713159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca11a4dc0a54f74a176a2aeb1974cf6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.722562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca11a4dc0a54f74a176a2aeb1974cf6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.723695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca11a4dc0a54f74a176a2aeb1974cf6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.724866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca11a4dc0a54f74a176a2aeb1974cf6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.749653 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1140/2257] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:07 2022] DELETE /identity/v3/users/5ca11a4dc0a54f74a176a2aeb1974cf6 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:07.750761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:07.752927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:07.754624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff31a8e9b6584e059808c9ad887bd698'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.756115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff31a8e9b6584e059808c9ad887bd698'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.788933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff31a8e9b6584e059808c9ad887bd698'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.805355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff31a8e9b6584e059808c9ad887bd698'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:07.819055 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1118/2258] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:07 2022] DELETE /identity/v3/users/ff31a8e9b6584e059808c9ad887bd698 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:07.823079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:07.825067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:07.876482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dbd23-b56d-43d1-8c95-2899279df0a0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:08.005832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed76fd-0d2d-4589-acb2-71cba2ac57e3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:08.046484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611eb41-9da4-4660-8290-726555d6fd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:08.048856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611eb41-9da4-4660-8290-726555d6fd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:08.049344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611eb41-9da4-4660-8290-726555d6fd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:08.051766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611eb41-9da4-4660-8290-726555d6fd7f admin admin] PATH_INFO: `/v3/projects/ed95a87adbf74a5da729e65a31b04a5c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:08.056718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611eb41-9da4-4660-8290-726555d6fd7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed95a87adbf74a5da729e65a31b04a5c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:08.062942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611eb41-9da4-4660-8290-726555d6fd7f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:08.114496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16d6f7-1794-4a77-988d-6e8f15683743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:08.116236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16d6f7-1794-4a77-988d-6e8f15683743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:08.116810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16d6f7-1794-4a77-988d-6e8f15683743 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:08.117349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16d6f7-1794-4a77-988d-6e8f15683743 admin admin] PATH_INFO: `/v3/users/8b5729e885694c2caf13e018c54a95bf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:08.123942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16d6f7-1794-4a77-988d-6e8f15683743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b5729e885694c2caf13e018c54a95bf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:08.133939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16d6f7-1794-4a77-988d-6e8f15683743 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:08.199283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b611eb41-9da4-4660-8290-726555d6fd7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed95a87adbf74a5da729e65a31b04a5c', 'request_id': 'req-b611eb41-9da4-4660-8290-726555d6fd7f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.217462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b611eb41-9da4-4660-8290-726555d6fd7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed95a87adbf74a5da729e65a31b04a5c', 'request_id': 'req-b611eb41-9da4-4660-8290-726555d6fd7f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.223388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:08.241731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b611eb41-9da4-4660-8290-726555d6fd7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed95a87adbf74a5da729e65a31b04a5c', 'request_id': 'req-b611eb41-9da4-4660-8290-726555d6fd7f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.248594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:08.266891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a16d6f7-1794-4a77-988d-6e8f15683743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b5729e885694c2caf13e018c54a95bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.278873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b611eb41-9da4-4660-8290-726555d6fd7f admin admin] The token cache is being invalidate because project ed95a87adbf74a5da729e65a31b04a5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:08.279177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b611eb41-9da4-4660-8290-726555d6fd7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.284940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b611eb41-9da4-4660-8290-726555d6fd7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.287866 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1119/2259] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:07 2022] DELETE /identity/v3/projects/ed95a87adbf74a5da729e65a31b04a5c => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:08.288930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa6f47-81f5-4476-ab2d-a3c5c6cf4cba None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:08.289230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa6f47-81f5-4476-ab2d-a3c5c6cf4cba None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:08.289510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa6f47-81f5-4476-ab2d-a3c5c6cf4cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:08.313162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a16d6f7-1794-4a77-988d-6e8f15683743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b5729e885694c2caf13e018c54a95bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.330468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a16d6f7-1794-4a77-988d-6e8f15683743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b5729e885694c2caf13e018c54a95bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.331728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a16d6f7-1794-4a77-988d-6e8f15683743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b5729e885694c2caf13e018c54a95bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.332721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a16d6f7-1794-4a77-988d-6e8f15683743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b5729e885694c2caf13e018c54a95bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.359590 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1141/2260] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:07 2022] DELETE /identity/v3/users/8b5729e885694c2caf13e018c54a95bf => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:08.369656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1672b-f4c5-42e2-ac1c-cf12ebeb0e54 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:08.369943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1672b-f4c5-42e2-ac1c-cf12ebeb0e54 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:08.370224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1672b-f4c5-42e2-ac1c-cf12ebeb0e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:08.390936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5caa6f47-81f5-4476-ab2d-a3c5c6cf4cba None None] MFA Rules not processed for user `f1b49ae269454b179df6ebdc94a487ba`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:08.392431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5caa6f47-81f5-4476-ab2d-a3c5c6cf4cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:08.418247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-56c1672b-f4c5-42e2-ac1c-cf12ebeb0e54 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:08.421731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-56c1672b-f4c5-42e2-ac1c-cf12ebeb0e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:08.553426 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1142/2261] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:08.555131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1eb6ca-b71a-4896-b471-657a341eadb5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:08.555584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1eb6ca-b71a-4896-b471-657a341eadb5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:08.556046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1eb6ca-b71a-4896-b471-657a341eadb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:08.576582 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1120/2262] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:08.577660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa6f47-81f5-4476-ab2d-a3c5c6cf4cba None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:08.580689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5caa6f47-81f5-4476-ab2d-a3c5c6cf4cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:08.606290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ea1eb6ca-b71a-4896-b471-657a341eadb5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:08.608051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ea1eb6ca-b71a-4896-b471-657a341eadb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:08.614586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa6f47-81f5-4476-ab2d-a3c5c6cf4cba None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:08.625011 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1143/2263] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:08.631299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1eb6ca-b71a-4896-b471-657a341eadb5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:08.631585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:08.631585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:08.631585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:08.631585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40 admin admin] PATH_INFO: `/v3/projects/9414f023ef63477b9062610de91f9ede` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:08.631944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ea1eb6ca-b71a-4896-b471-657a341eadb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:08.632022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9414f023ef63477b9062610de91f9ede)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:08.634172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:08.742077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1eb6ca-b71a-4896-b471-657a341eadb5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:08.743496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9414f023ef63477b9062610de91f9ede', 'request_id': 'req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.755613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9414f023ef63477b9062610de91f9ede', 'request_id': 'req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.770186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9414f023ef63477b9062610de91f9ede', 'request_id': 'req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.796191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40 admin admin] The token cache is being invalidate because project 9414f023ef63477b9062610de91f9ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:08.798650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.800596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78ab4ead-d242-4a6a-b3c6-93c4ba91ca40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:08.803452 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1121/2264] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:08 2022] DELETE /identity/v3/projects/9414f023ef63477b9062610de91f9ede => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:08.804463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe2a24-54aa-46b9-9e99-bf4dc17fdf9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:08.805129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe2a24-54aa-46b9-9e99-bf4dc17fdf9a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:08.805420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe2a24-54aa-46b9-9e99-bf4dc17fdf9a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:08.805703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe2a24-54aa-46b9-9e99-bf4dc17fdf9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:08.807329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41629ab0-a284-4216-9ac5-ffb0bd1b1a52 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:08.807735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41629ab0-a284-4216-9ac5-ffb0bd1b1a52 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:08.807854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dbfe2a24-54aa-46b9-9e99-bf4dc17fdf9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:08.813468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41629ab0-a284-4216-9ac5-ffb0bd1b1a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:08.839584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:08.854793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-41629ab0-a284-4216-9ac5-ffb0bd1b1a52 None None] MFA Rules not processed for user `90d7e4c2e577467f949be3dc480cc7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:08.856709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-41629ab0-a284-4216-9ac5-ffb0bd1b1a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:08.884580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe2a24-54aa-46b9-9e99-bf4dc17fdf9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:08.886014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe2a24-54aa-46b9-9e99-bf4dc17fdf9a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:08.898538 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1144/2265] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:08.899855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b14c0-1c63-4b8d-8bb0-96f861239f20 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:08.900157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b14c0-1c63-4b8d-8bb0-96f861239f20 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:08.900440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b14c0-1c63-4b8d-8bb0-96f861239f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:08.925445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-124b14c0-1c63-4b8d-8bb0-96f861239f20 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:08.926927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-124b14c0-1c63-4b8d-8bb0-96f861239f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:08.937678 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1145/2266] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:08.937895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b14c0-1c63-4b8d-8bb0-96f861239f20 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:08.939320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-124b14c0-1c63-4b8d-8bb0-96f861239f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:08.953618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b14c0-1c63-4b8d-8bb0-96f861239f20 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:08.958803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:08.959648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:08.959796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:08.960066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] PATH_INFO: `/v3/projects/a293794868cd4c749b6eaa82e23e0500` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:08.961529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a293794868cd4c749b6eaa82e23e0500)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:08.963745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:08.967097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:08.989139 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1122/2267] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:08.989139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6389c-2b3d-42d8-8125-3c9553b4bf68 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:08.989139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6389c-2b3d-42d8-8125-3c9553b4bf68 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:08.989139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6389c-2b3d-42d8-8125-3c9553b4bf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:09.037774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a4c6389c-2b3d-42d8-8125-3c9553b4bf68 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:09.039046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a4c6389c-2b3d-42d8-8125-3c9553b4bf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:09.053794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a293794868cd4c749b6eaa82e23e0500', 'request_id': 'req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:09.065887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a293794868cd4c749b6eaa82e23e0500', 'request_id': 'req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:09.086616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a293794868cd4c749b6eaa82e23e0500', 'request_id': 'req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:09.113435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] The token cache is being invalidate because project a293794868cd4c749b6eaa82e23e0500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:09.114951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:09.117108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:09.122519 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1146/2268] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:08 2022] DELETE /identity/v3/projects/a293794868cd4c749b6eaa82e23e0500 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:09.126232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:09.130564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:09.269541 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1123/2269] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:09.274119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6389c-2b3d-42d8-8125-3c9553b4bf68 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:09.287967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a4c6389c-2b3d-42d8-8125-3c9553b4bf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:09.416137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fcc7d-78d3-4bc5-ba7b-9b9cbd60c91d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:09.432417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddc5a9-f50e-4025-bc95-b44c415f69f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:09.433314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddc5a9-f50e-4025-bc95-b44c415f69f6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:09.433613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddc5a9-f50e-4025-bc95-b44c415f69f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:09.434585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddc5a9-f50e-4025-bc95-b44c415f69f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:09.435306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddc5a9-f50e-4025-bc95-b44c415f69f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:09.435622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddc5a9-f50e-4025-bc95-b44c415f69f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:09.437370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddc5a9-f50e-4025-bc95-b44c415f69f6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:09.467790 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1147/2270] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:09.469144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5c7db-fe62-49be-bd16-6181b146b72b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:09.469445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5c7db-fe62-49be-bd16-6181b146b72b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:09.469725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5c7db-fe62-49be-bd16-6181b146b72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:09.511059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6389c-2b3d-42d8-8125-3c9553b4bf68 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:09.525707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2360a2-11e9-425a-b5cf-73ef48025841 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:09.526352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2360a2-11e9-425a-b5cf-73ef48025841 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:09.526693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2360a2-11e9-425a-b5cf-73ef48025841 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:09.526917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2360a2-11e9-425a-b5cf-73ef48025841 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:09.533132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9d2360a2-11e9-425a-b5cf-73ef48025841 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:09.546512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e7e5c7db-fe62-49be-bd16-6181b146b72b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:09.548123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e7e5c7db-fe62-49be-bd16-6181b146b72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:09.569257 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1148/2271] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:09.571203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92befde3-b06b-4b65-af6d-de5e6c174856 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:09.571527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92befde3-b06b-4b65-af6d-de5e6c174856 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:09.571815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92befde3-b06b-4b65-af6d-de5e6c174856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:09.630760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-92befde3-b06b-4b65-af6d-de5e6c174856 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:09.634319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-92befde3-b06b-4b65-af6d-de5e6c174856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:09.655688 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1149/2272] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:09.657261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92befde3-b06b-4b65-af6d-de5e6c174856 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:09.660414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-92befde3-b06b-4b65-af6d-de5e6c174856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:09.707143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92befde3-b06b-4b65-af6d-de5e6c174856 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:09.719110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0d292-51c9-47aa-b1f3-972596dec34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:09.720349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0d292-51c9-47aa-b1f3-972596dec34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:09.720643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0d292-51c9-47aa-b1f3-972596dec34a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:09.721419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0d292-51c9-47aa-b1f3-972596dec34a admin admin] PATH_INFO: `/v3/projects/3fb8975b0bc94e448ec0b9076504abf4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:09.725013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0d292-51c9-47aa-b1f3-972596dec34a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fb8975b0bc94e448ec0b9076504abf4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:09.728470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0d292-51c9-47aa-b1f3-972596dec34a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:09.743824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2360a2-11e9-425a-b5cf-73ef48025841 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:09.746136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2360a2-11e9-425a-b5cf-73ef48025841 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:09.779352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:09.783438 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1124/2273] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:09.786056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2360a2-11e9-425a-b5cf-73ef48025841 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:09.809674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2360a2-11e9-425a-b5cf-73ef48025841 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:09.842485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a15723-7e46-4977-b298-0d66a13a88fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:09.843277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a15723-7e46-4977-b298-0d66a13a88fc admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:09.843403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a15723-7e46-4977-b298-0d66a13a88fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:09.843624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a15723-7e46-4977-b298-0d66a13a88fc admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:09.844276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a15723-7e46-4977-b298-0d66a13a88fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:09.867377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a15723-7e46-4977-b298-0d66a13a88fc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:09.899138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58f0d292-51c9-47aa-b1f3-972596dec34a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fb8975b0bc94e448ec0b9076504abf4', 'request_id': 'req-58f0d292-51c9-47aa-b1f3-972596dec34a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:09.919337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58f0d292-51c9-47aa-b1f3-972596dec34a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fb8975b0bc94e448ec0b9076504abf4', 'request_id': 'req-58f0d292-51c9-47aa-b1f3-972596dec34a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:09.936580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58f0d292-51c9-47aa-b1f3-972596dec34a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fb8975b0bc94e448ec0b9076504abf4', 'request_id': 'req-58f0d292-51c9-47aa-b1f3-972596dec34a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:09.975908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58f0d292-51c9-47aa-b1f3-972596dec34a admin admin] The token cache is being invalidate because project 3fb8975b0bc94e448ec0b9076504abf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:09.976292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58f0d292-51c9-47aa-b1f3-972596dec34a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:09.978699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58f0d292-51c9-47aa-b1f3-972596dec34a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:09.983312 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1150/2274] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:09 2022] DELETE /identity/v3/projects/3fb8975b0bc94e448ec0b9076504abf4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:09.984701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23370cc4-5df4-41b6-bc1a-0f139c33c344 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:09.985023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23370cc4-5df4-41b6-bc1a-0f139c33c344 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:09.985309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23370cc4-5df4-41b6-bc1a-0f139c33c344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:09.987603 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1125/2275] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:09 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:10.002298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a15723-7e46-4977-b298-0d66a13a88fc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:10.014143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b0a15723-7e46-4977-b298-0d66a13a88fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:10.083786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-23370cc4-5df4-41b6-bc1a-0f139c33c344 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:10.085413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:10.086186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-23370cc4-5df4-41b6-bc1a-0f139c33c344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:10.226517 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1151/2276] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:10.227714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23370cc4-5df4-41b6-bc1a-0f139c33c344 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:10.229273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-23370cc4-5df4-41b6-bc1a-0f139c33c344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:10.244539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a15723-7e46-4977-b298-0d66a13a88fc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:10.254323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23370cc4-5df4-41b6-bc1a-0f139c33c344 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:10.257096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320861b3-75fc-42a6-8edc-8bf8147e6ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:10.257748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320861b3-75fc-42a6-8edc-8bf8147e6ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:10.258039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320861b3-75fc-42a6-8edc-8bf8147e6ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:10.258320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320861b3-75fc-42a6-8edc-8bf8147e6ed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:10.258992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320861b3-75fc-42a6-8edc-8bf8147e6ed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:10.259305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320861b3-75fc-42a6-8edc-8bf8147e6ed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:10.261007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320861b3-75fc-42a6-8edc-8bf8147e6ed3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:10.267121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:10.274114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:10.274343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:10.274632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] PATH_INFO: `/v3/users/42d03397519e435483e59bb0911dac61` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:10.276032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=42d03397519e435483e59bb0911dac61)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:10.279220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:10.288337 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1126/2277] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:10.289227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320861b3-75fc-42a6-8edc-8bf8147e6ed3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:10.294167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-320861b3-75fc-42a6-8edc-8bf8147e6ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:10.319666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320861b3-75fc-42a6-8edc-8bf8147e6ed3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:10.335624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185fa09-3564-4491-968f-3b9ae778dc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:10.336279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185fa09-3564-4491-968f-3b9ae778dc95 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:10.336573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185fa09-3564-4491-968f-3b9ae778dc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:10.336859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185fa09-3564-4491-968f-3b9ae778dc95 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:10.337545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185fa09-3564-4491-968f-3b9ae778dc95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:10.339768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185fa09-3564-4491-968f-3b9ae778dc95 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:10.353959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42d03397519e435483e59bb0911dac61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:10.377212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42d03397519e435483e59bb0911dac61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:10.385752 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1127/2278] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:10 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:10.387848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e366c3-1218-47bf-ad98-dc4c723a0fad None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:10.388150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e366c3-1218-47bf-ad98-dc4c723a0fad None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:10.388431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e366c3-1218-47bf-ad98-dc4c723a0fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:10.388801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42d03397519e435483e59bb0911dac61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:10.389860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42d03397519e435483e59bb0911dac61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:10.390566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42d03397519e435483e59bb0911dac61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:10.421007 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1152/2279] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:10 2022] DELETE /identity/v3/users/42d03397519e435483e59bb0911dac61 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:10.422394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:10.424152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:10.450823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e2e366c3-1218-47bf-ad98-dc4c723a0fad None None] MFA Rules not processed for user `ee0fdf107484448d862ac3216df0d9dc`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:10.452088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e2e366c3-1218-47bf-ad98-dc4c723a0fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:10.476367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:10.634716 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1128/2280] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:10 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:10.635856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e366c3-1218-47bf-ad98-dc4c723a0fad None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:10.637499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e2e366c3-1218-47bf-ad98-dc4c723a0fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:10.661212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e366c3-1218-47bf-ad98-dc4c723a0fad None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:10.663292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392663e-fc25-42a5-95d8-b98744cd5f9e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:10.666707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e875c82-6127-4ba3-85ec-1cb9ba9c7d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:10.669752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e875c82-6127-4ba3-85ec-1cb9ba9c7d78 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:10.670617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e875c82-6127-4ba3-85ec-1cb9ba9c7d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:10.671060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e875c82-6127-4ba3-85ec-1cb9ba9c7d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:10.671934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e875c82-6127-4ba3-85ec-1cb9ba9c7d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:10.674522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e875c82-6127-4ba3-85ec-1cb9ba9c7d78 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:10.681641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b62338-55c0-4f09-b631-8a61f5fccdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:10.682135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b62338-55c0-4f09-b631-8a61f5fccdf7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:10.683590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b62338-55c0-4f09-b631-8a61f5fccdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:10.684088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b62338-55c0-4f09-b631-8a61f5fccdf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:10.686554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b62338-55c0-4f09-b631-8a61f5fccdf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:10.687552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b62338-55c0-4f09-b631-8a61f5fccdf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:10.694735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b62338-55c0-4f09-b631-8a61f5fccdf7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:10.726269 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1129/2281] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:10 2022] POST /identity/v3/projects => generated 393 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:10.726758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e875c82-6127-4ba3-85ec-1cb9ba9c7d78 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:10.732094 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1153/2282] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:10.733173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8e875c82-6127-4ba3-85ec-1cb9ba9c7d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:10.736165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b62338-55c0-4f09-b631-8a61f5fccdf7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:10.741319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-06b62338-55c0-4f09-b631-8a61f5fccdf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:10.772133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b62338-55c0-4f09-b631-8a61f5fccdf7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:10.905267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e875c82-6127-4ba3-85ec-1cb9ba9c7d78 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:10.919709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae04659-27df-4a0e-aab2-dc82e9b0b828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:10.920804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae04659-27df-4a0e-aab2-dc82e9b0b828 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:10.922813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae04659-27df-4a0e-aab2-dc82e9b0b828 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:10.924257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae04659-27df-4a0e-aab2-dc82e9b0b828 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:10.925715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae04659-27df-4a0e-aab2-dc82e9b0b828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:10.928237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae04659-27df-4a0e-aab2-dc82e9b0b828 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:10.930567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a621c1-16e6-4d4c-ada3-e1c6d855ce58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:10.932474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a621c1-16e6-4d4c-ada3-e1c6d855ce58 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:10.932869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a621c1-16e6-4d4c-ada3-e1c6d855ce58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:10.933366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:10.936471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a621c1-16e6-4d4c-ada3-e1c6d855ce58 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:10.937296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a621c1-16e6-4d4c-ada3-e1c6d855ce58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:10.945698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a621c1-16e6-4d4c-ada3-e1c6d855ce58 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:10.958401 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1130/2283] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:10.962998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae04659-27df-4a0e-aab2-dc82e9b0b828 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:10.986014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae04659-27df-4a0e-aab2-dc82e9b0b828 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:11.000705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082906f-7177-429a-8e6f-07b75cecc916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:11.001350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082906f-7177-429a-8e6f-07b75cecc916 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.003016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082906f-7177-429a-8e6f-07b75cecc916 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.003400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082906f-7177-429a-8e6f-07b75cecc916 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.004544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082906f-7177-429a-8e6f-07b75cecc916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:11.008400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082906f-7177-429a-8e6f-07b75cecc916 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:11.016862 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1154/2284] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:10 2022] POST /identity/v3/projects => generated 403 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:11.016862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a621c1-16e6-4d4c-ada3-e1c6d855ce58 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:11.045345 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1131/2285] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:10 2022] POST /identity/v3/users => generated 297 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:11.045925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082906f-7177-429a-8e6f-07b75cecc916 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:11.089185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082906f-7177-429a-8e6f-07b75cecc916 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:11.126398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a621c1-16e6-4d4c-ada3-e1c6d855ce58 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:11.132756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6a946-9a8b-468b-8a57-358333da8215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:11.133950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a946-9a8b-468b-8a57-358333da8215 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.134450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a946-9a8b-468b-8a57-358333da8215 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.134920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a946-9a8b-468b-8a57-358333da8215 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.135897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6a946-9a8b-468b-8a57-358333da8215 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:11.139281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6a946-9a8b-468b-8a57-358333da8215 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:11.166036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04372ed7-132a-4046-b939-bfe1c30f9f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:11.167105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04372ed7-132a-4046-b939-bfe1c30f9f52 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.167405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04372ed7-132a-4046-b939-bfe1c30f9f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.168009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04372ed7-132a-4046-b939-bfe1c30f9f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.170042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04372ed7-132a-4046-b939-bfe1c30f9f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:11.172180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04372ed7-132a-4046-b939-bfe1c30f9f52 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:11.181375 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1155/2286] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:11 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:11.182913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6a946-9a8b-468b-8a57-358333da8215 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:11.185602 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1132/2287] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:11.186522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04372ed7-132a-4046-b939-bfe1c30f9f52 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:11.188302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-04372ed7-132a-4046-b939-bfe1c30f9f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:11.203325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04372ed7-132a-4046-b939-bfe1c30f9f52 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:11.221369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:11.303722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6a946-9a8b-468b-8a57-358333da8215 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:11.310023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:11.310688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.310983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.311268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] PATH_INFO: `/v3/projects/27c2d8d3613c4945ace46a1ec4f70d67` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.311605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2648c-cdd7-4f0d-abc4-24e762049d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:11.312298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2648c-cdd7-4f0d-abc4-24e762049d42 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.312486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27c2d8d3613c4945ace46a1ec4f70d67)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:11.312649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2648c-cdd7-4f0d-abc4-24e762049d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.313078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2648c-cdd7-4f0d-abc4-24e762049d42 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.313785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2648c-cdd7-4f0d-abc4-24e762049d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:11.314659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:11.315857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2648c-cdd7-4f0d-abc4-24e762049d42 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:11.346158 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1156/2288] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:11 2022] POST /identity/v3/users => generated 308 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:11.351329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2648c-cdd7-4f0d-abc4-24e762049d42 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:11.364276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2648c-cdd7-4f0d-abc4-24e762049d42 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:11.371172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cfa2d-36d9-4b11-8ab0-8290cf8c7128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:11.372344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cfa2d-36d9-4b11-8ab0-8290cf8c7128 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.372629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cfa2d-36d9-4b11-8ab0-8290cf8c7128 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.372954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cfa2d-36d9-4b11-8ab0-8290cf8c7128 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.373614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cfa2d-36d9-4b11-8ab0-8290cf8c7128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:11.376377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cfa2d-36d9-4b11-8ab0-8290cf8c7128 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:11.392939 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1157/2289] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:11.392939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cfa2d-36d9-4b11-8ab0-8290cf8c7128 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:11.401148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27c2d8d3613c4945ace46a1ec4f70d67', 'request_id': 'req-2acd4629-c043-4fc6-9557-f6bf520b4c65'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:11.401355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cfa2d-36d9-4b11-8ab0-8290cf8c7128 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:11.420182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27c2d8d3613c4945ace46a1ec4f70d67', 'request_id': 'req-2acd4629-c043-4fc6-9557-f6bf520b4c65'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:11.431491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27c2d8d3613c4945ace46a1ec4f70d67', 'request_id': 'req-2acd4629-c043-4fc6-9557-f6bf520b4c65'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:11.455778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] The token cache is being invalidate because project 27c2d8d3613c4945ace46a1ec4f70d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:11.456268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:11.459680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:11.464507 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1133/2290] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:11 2022] DELETE /identity/v3/projects/27c2d8d3613c4945ace46a1ec4f70d67 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:11.464507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:11.468459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:11.478862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:11.497913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77649c0a-3e8a-4804-92df-bd955bfdc09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:11.498556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77649c0a-3e8a-4804-92df-bd955bfdc09b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.499423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77649c0a-3e8a-4804-92df-bd955bfdc09b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.499718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77649c0a-3e8a-4804-92df-bd955bfdc09b admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.500416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77649c0a-3e8a-4804-92df-bd955bfdc09b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:11.505001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77649c0a-3e8a-4804-92df-bd955bfdc09b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:11.562035 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1158/2291] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:11 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:11.562741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77649c0a-3e8a-4804-92df-bd955bfdc09b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:11.566766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-77649c0a-3e8a-4804-92df-bd955bfdc09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:11.608530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77649c0a-3e8a-4804-92df-bd955bfdc09b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:11.628053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd4629-c043-4fc6-9557-f6bf520b4c65 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:11.636433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44094b1b-20ff-4d47-b08f-b03d1d9407dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:11.637208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44094b1b-20ff-4d47-b08f-b03d1d9407dc admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.637486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44094b1b-20ff-4d47-b08f-b03d1d9407dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.637775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44094b1b-20ff-4d47-b08f-b03d1d9407dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.642083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609da8d-d894-41fb-90f0-1786c740e7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:11.644758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44094b1b-20ff-4d47-b08f-b03d1d9407dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:11.645690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609da8d-d894-41fb-90f0-1786c740e7e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.645877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609da8d-d894-41fb-90f0-1786c740e7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.648684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44094b1b-20ff-4d47-b08f-b03d1d9407dc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:11.649676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609da8d-d894-41fb-90f0-1786c740e7e5 admin admin] PATH_INFO: `/v3/projects/ca0998946ce24a8da8b4fba9e9f79101/users/1ee2df69ab474fc9aa85624983fd8fa6/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.668856 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1159/2292] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:11.668856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44094b1b-20ff-4d47-b08f-b03d1d9407dc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:11.671718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-44094b1b-20ff-4d47-b08f-b03d1d9407dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:11.679378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609da8d-d894-41fb-90f0-1786c740e7e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0998946ce24a8da8b4fba9e9f79101, user_id=1ee2df69ab474fc9aa85624983fd8fa6, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:11.692121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44094b1b-20ff-4d47-b08f-b03d1d9407dc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:11.696242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609da8d-d894-41fb-90f0-1786c740e7e5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:11.709009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9b5a6-8bf2-4ab8-bb06-3fcce1eff70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:11.709939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9b5a6-8bf2-4ab8-bb06-3fcce1eff70f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.710398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9b5a6-8bf2-4ab8-bb06-3fcce1eff70f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.710719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9b5a6-8bf2-4ab8-bb06-3fcce1eff70f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.711522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9b5a6-8bf2-4ab8-bb06-3fcce1eff70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:11.713701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9b5a6-8bf2-4ab8-bb06-3fcce1eff70f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:11.728527 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1134/2293] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:11 2022] PUT /identity/v3/projects/ca0998946ce24a8da8b4fba9e9f79101/users/1ee2df69ab474fc9aa85624983fd8fa6/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:11.730448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5c311-de5f-46e1-b0c8-a89a3b4a427d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.730744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5c311-de5f-46e1-b0c8-a89a3b4a427d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.731580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5c311-de5f-46e1-b0c8-a89a3b4a427d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.741480 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1160/2294] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:11.742625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9b5a6-8bf2-4ab8-bb06-3fcce1eff70f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:11.746195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c7b9b5a6-8bf2-4ab8-bb06-3fcce1eff70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:11.760349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-05f5c311-de5f-46e1-b0c8-a89a3b4a427d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:11.761931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-05f5c311-de5f-46e1-b0c8-a89a3b4a427d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:11.765358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9b5a6-8bf2-4ab8-bb06-3fcce1eff70f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:11.902106 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1135/2295] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:11.902106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5c311-de5f-46e1-b0c8-a89a3b4a427d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:11.921396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5c311-de5f-46e1-b0c8-a89a3b4a427d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:11.932433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9baef-e3b5-41bd-9b28-2b785ed4c9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:11.933191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9baef-e3b5-41bd-9b28-2b785ed4c9f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.933537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9baef-e3b5-41bd-9b28-2b785ed4c9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.933869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9baef-e3b5-41bd-9b28-2b785ed4c9f0 admin admin] PATH_INFO: `/v3/projects/ae38955142824be086f297cc4310991d/users/30ebd40778e3492199eef13eb0ab3310/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.936486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dd4d1-1b4f-437e-8199-b05c46dd25ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:11.937137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dd4d1-1b4f-437e-8199-b05c46dd25ce admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.937428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dd4d1-1b4f-437e-8199-b05c46dd25ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.937708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dd4d1-1b4f-437e-8199-b05c46dd25ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.938370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dd4d1-1b4f-437e-8199-b05c46dd25ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:11.939994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dd4d1-1b4f-437e-8199-b05c46dd25ce admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:11.974502 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1161/2296] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:11.976237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e59704-6fb8-491b-a651-f2f0c6349380 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:11.976699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e59704-6fb8-491b-a651-f2f0c6349380 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:11.977352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e59704-6fb8-491b-a651-f2f0c6349380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:11.983086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9baef-e3b5-41bd-9b28-2b785ed4c9f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae38955142824be086f297cc4310991d, user_id=30ebd40778e3492199eef13eb0ab3310, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:11.986002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9baef-e3b5-41bd-9b28-2b785ed4c9f0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:11.990725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:12.005742 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1136/2297] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:11 2022] PUT /identity/v3/projects/ae38955142824be086f297cc4310991d/users/30ebd40778e3492199eef13eb0ab3310/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:12.007187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9baef-e3b5-41bd-9b28-2b785ed4c9f0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:12.050286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-39e59704-6fb8-491b-a651-f2f0c6349380 None None] MFA Rules not processed for user `1ee2df69ab474fc9aa85624983fd8fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:12.051623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-39e59704-6fb8-491b-a651-f2f0c6349380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:12.192084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9baef-e3b5-41bd-9b28-2b785ed4c9f0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:12.218491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0879f9f-70c9-45f8-84e4-cf2c74d3d123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:12.219687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0879f9f-70c9-45f8-84e4-cf2c74d3d123 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:12.220002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0879f9f-70c9-45f8-84e4-cf2c74d3d123 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:12.220527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0879f9f-70c9-45f8-84e4-cf2c74d3d123 admin admin] PATH_INFO: `/v3/projects/ec21d1602f7b4dc5b825dcb479e67d95/users/08d418072145419ba6c07d5a06f6bc61/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:12.226284 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1162/2298] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:12.231945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f1b2c-a6f1-400d-b81c-646aeb7cf78c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:12.232230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f1b2c-a6f1-400d-b81c-646aeb7cf78c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:12.232501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f1b2c-a6f1-400d-b81c-646aeb7cf78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:12.238905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:12.250427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0879f9f-70c9-45f8-84e4-cf2c74d3d123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec21d1602f7b4dc5b825dcb479e67d95, user_id=08d418072145419ba6c07d5a06f6bc61, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:12.254619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0879f9f-70c9-45f8-84e4-cf2c74d3d123 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:12.270578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c98f1b2c-a6f1-400d-b81c-646aeb7cf78c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:12.272220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c98f1b2c-a6f1-400d-b81c-646aeb7cf78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:12.274923 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1137/2299] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:12 2022] PUT /identity/v3/projects/ec21d1602f7b4dc5b825dcb479e67d95/users/08d418072145419ba6c07d5a06f6bc61/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:12.275875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0879f9f-70c9-45f8-84e4-cf2c74d3d123 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:12.379552 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1163/2300] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:12.379552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f1b2c-a6f1-400d-b81c-646aeb7cf78c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:12.391700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f1b2c-a6f1-400d-b81c-646aeb7cf78c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:12.398380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38046b-7cdb-4a7c-8c4e-29df0b820f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:12.400249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38046b-7cdb-4a7c-8c4e-29df0b820f7f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:12.400616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38046b-7cdb-4a7c-8c4e-29df0b820f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:12.401018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38046b-7cdb-4a7c-8c4e-29df0b820f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:12.401743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38046b-7cdb-4a7c-8c4e-29df0b820f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:12.404384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38046b-7cdb-4a7c-8c4e-29df0b820f7f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:12.430194 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1164/2301] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:12.431434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38046b-7cdb-4a7c-8c4e-29df0b820f7f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:12.461605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38046b-7cdb-4a7c-8c4e-29df0b820f7f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:12.488056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e442159-5f31-400b-861b-fe26399d6cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:12.488851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e442159-5f31-400b-861b-fe26399d6cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:12.489242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e442159-5f31-400b-861b-fe26399d6cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:12.489467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e442159-5f31-400b-861b-fe26399d6cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:12.490121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e442159-5f31-400b-861b-fe26399d6cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:12.490322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0879f9f-70c9-45f8-84e4-cf2c74d3d123 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:12.492218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e442159-5f31-400b-861b-fe26399d6cd1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:12.508980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681166e-658d-4c03-961d-39393e8bc4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:12.509946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681166e-658d-4c03-961d-39393e8bc4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:12.510231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681166e-658d-4c03-961d-39393e8bc4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:12.510509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681166e-658d-4c03-961d-39393e8bc4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:12.514415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681166e-658d-4c03-961d-39393e8bc4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:12.518283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681166e-658d-4c03-961d-39393e8bc4e5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:12.542700 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1165/2302] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:12 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:12.544840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa78a72-54ab-41b5-ade0-d8f7a8078e35 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:12.545177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa78a72-54ab-41b5-ade0-d8f7a8078e35 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:12.545459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa78a72-54ab-41b5-ade0-d8f7a8078e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:12.573078 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1138/2303] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:12.573937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681166e-658d-4c03-961d-39393e8bc4e5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:12.604638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2aa78a72-54ab-41b5-ade0-d8f7a8078e35 None None] MFA Rules not processed for user `08d418072145419ba6c07d5a06f6bc61`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:12.605439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681166e-658d-4c03-961d-39393e8bc4e5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:12.606040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2aa78a72-54ab-41b5-ade0-d8f7a8078e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:12.694455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:12.743132 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1166/2304] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:12.743921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa78a72-54ab-41b5-ade0-d8f7a8078e35 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:12.757841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb822df1-afef-4748-9ab3-7a784ae3086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:12.768068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb822df1-afef-4748-9ab3-7a784ae3086e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:12.768342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb822df1-afef-4748-9ab3-7a784ae3086e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:12.768562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb822df1-afef-4748-9ab3-7a784ae3086e admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:12.774748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb822df1-afef-4748-9ab3-7a784ae3086e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:12.776707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb822df1-afef-4748-9ab3-7a784ae3086e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:12.781815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb822df1-afef-4748-9ab3-7a784ae3086e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:12.827501 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1139/2305] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:12.829262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb822df1-afef-4748-9ab3-7a784ae3086e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:12.850974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa78a72-54ab-41b5-ade0-d8f7a8078e35 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:12.860552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f608e51-e9a5-4e9a-b212-9636fa19dd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:12.862150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f608e51-e9a5-4e9a-b212-9636fa19dd16 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:12.862611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f608e51-e9a5-4e9a-b212-9636fa19dd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:12.868617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb822df1-afef-4748-9ab3-7a784ae3086e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:12.870937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f608e51-e9a5-4e9a-b212-9636fa19dd16 admin admin] PATH_INFO: `/v3/projects/ae38955142824be086f297cc4310991d/users/30ebd40778e3492199eef13eb0ab3310/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:12.876942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f608e51-e9a5-4e9a-b212-9636fa19dd16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae38955142824be086f297cc4310991d, user_id=30ebd40778e3492199eef13eb0ab3310, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:12.877355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadf05e-44c1-4a15-93d5-63b22f4d736c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:12.878260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadf05e-44c1-4a15-93d5-63b22f4d736c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:12.878683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadf05e-44c1-4a15-93d5-63b22f4d736c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:12.879270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadf05e-44c1-4a15-93d5-63b22f4d736c admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:12.881013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadf05e-44c1-4a15-93d5-63b22f4d736c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:12.883373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f608e51-e9a5-4e9a-b212-9636fa19dd16 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:12.884190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadf05e-44c1-4a15-93d5-63b22f4d736c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:12.907824 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1167/2306] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:12 2022] PUT /identity/v3/projects/ae38955142824be086f297cc4310991d/users/30ebd40778e3492199eef13eb0ab3310/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:12.908482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f608e51-e9a5-4e9a-b212-9636fa19dd16 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:12.917131 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1140/2307] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:12 2022] POST /identity/v3/users => generated 294 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:12.917820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadf05e-44c1-4a15-93d5-63b22f4d736c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:12.930937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadf05e-44c1-4a15-93d5-63b22f4d736c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:13.008227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2328a73-397f-4c53-bad6-6eb9dba30bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:13.008865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2328a73-397f-4c53-bad6-6eb9dba30bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:13.009184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2328a73-397f-4c53-bad6-6eb9dba30bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:13.009464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2328a73-397f-4c53-bad6-6eb9dba30bf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:13.010110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2328a73-397f-4c53-bad6-6eb9dba30bf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:13.012221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2328a73-397f-4c53-bad6-6eb9dba30bf5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:13.016330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f608e51-e9a5-4e9a-b212-9636fa19dd16 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:13.023701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a27d63-d801-4a44-a3b1-cdade5f05089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:13.024744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:13.025117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a27d63-d801-4a44-a3b1-cdade5f05089 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:13.025555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a27d63-d801-4a44-a3b1-cdade5f05089 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:13.025851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a27d63-d801-4a44-a3b1-cdade5f05089 admin admin] PATH_INFO: `/v3/projects/f4d12291e32b4d9a91938e11c958eb96/users/ecdab6643a234862a797ebb53aaf5681/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:13.060426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a27d63-d801-4a44-a3b1-cdade5f05089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d12291e32b4d9a91938e11c958eb96, user_id=ecdab6643a234862a797ebb53aaf5681, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:13.066340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a27d63-d801-4a44-a3b1-cdade5f05089 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:13.072475 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1141/2308] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:12 2022] POST /identity/v3/projects => generated 391 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:13.074669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2328a73-397f-4c53-bad6-6eb9dba30bf5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:13.092489 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1168/2309] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:12 2022] PUT /identity/v3/projects/f4d12291e32b4d9a91938e11c958eb96/users/ecdab6643a234862a797ebb53aaf5681/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:13.093868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a27d63-d801-4a44-a3b1-cdade5f05089 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:13.247143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a27d63-d801-4a44-a3b1-cdade5f05089 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:13.249991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2328a73-397f-4c53-bad6-6eb9dba30bf5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:13.253790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377cfdb-14cb-4b6a-808e-40aa0cb1ffb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:13.255168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377cfdb-14cb-4b6a-808e-40aa0cb1ffb2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:13.255647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377cfdb-14cb-4b6a-808e-40aa0cb1ffb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:13.256257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377cfdb-14cb-4b6a-808e-40aa0cb1ffb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:13.257014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377cfdb-14cb-4b6a-808e-40aa0cb1ffb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:13.259103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377cfdb-14cb-4b6a-808e-40aa0cb1ffb2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:13.262792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9d6ef-9b36-4a60-a53f-e621dad1cd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:13.263487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9d6ef-9b36-4a60-a53f-e621dad1cd5a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:13.263773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9d6ef-9b36-4a60-a53f-e621dad1cd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:13.264052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9d6ef-9b36-4a60-a53f-e621dad1cd5a admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:13.264796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9d6ef-9b36-4a60-a53f-e621dad1cd5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:13.273085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9d6ef-9b36-4a60-a53f-e621dad1cd5a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:13.278305 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1169/2310] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:13.279420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377cfdb-14cb-4b6a-808e-40aa0cb1ffb2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:13.300224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:13.313443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377cfdb-14cb-4b6a-808e-40aa0cb1ffb2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:13.324671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fb653-5509-41d4-b270-544a7275df6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:13.326405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fb653-5509-41d4-b270-544a7275df6e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:13.327605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fb653-5509-41d4-b270-544a7275df6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:13.328163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fb653-5509-41d4-b270-544a7275df6e admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:13.328852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fb653-5509-41d4-b270-544a7275df6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:13.334040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fb653-5509-41d4-b270-544a7275df6e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:13.334466 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1142/2311] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:13 2022] POST /identity/v3/projects => generated 375 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:13.336794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81328879-a57f-4e76-89ba-33abe39ea10d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:13.337426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81328879-a57f-4e76-89ba-33abe39ea10d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:13.337726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81328879-a57f-4e76-89ba-33abe39ea10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:13.410745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-81328879-a57f-4e76-89ba-33abe39ea10d None None] MFA Rules not processed for user `ecdab6643a234862a797ebb53aaf5681`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:13.414987 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1170/2312] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:13 2022] POST /identity/v3/users => generated 302 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:13.415815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fb653-5509-41d4-b270-544a7275df6e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:13.418339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-81328879-a57f-4e76-89ba-33abe39ea10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:13.451811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fb653-5509-41d4-b270-544a7275df6e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:13.544666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b8ce6-2b70-4bc0-8412-d303c022e081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:13.546288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b8ce6-2b70-4bc0-8412-d303c022e081 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:13.546783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b8ce6-2b70-4bc0-8412-d303c022e081 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:13.547266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b8ce6-2b70-4bc0-8412-d303c022e081 admin admin] PATH_INFO: `/v3/projects/ad8850110d7c49ed92be6077d7204301/users/ac23076fca324df1885d72bbf28aeac8/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:13.574344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b8ce6-2b70-4bc0-8412-d303c022e081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8850110d7c49ed92be6077d7204301, user_id=ac23076fca324df1885d72bbf28aeac8, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:13.578326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b8ce6-2b70-4bc0-8412-d303c022e081 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:13.579264 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1143/2313] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:13.580185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81328879-a57f-4e76-89ba-33abe39ea10d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:13.600023 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1171/2314] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:13 2022] PUT /identity/v3/projects/ad8850110d7c49ed92be6077d7204301/users/ac23076fca324df1885d72bbf28aeac8/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:13.600210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81328879-a57f-4e76-89ba-33abe39ea10d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:13.601284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b8ce6-2b70-4bc0-8412-d303c022e081 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:13.718506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b8ce6-2b70-4bc0-8412-d303c022e081 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:13.724084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b25c5-9cf7-4b93-bd9f-c1a1f2b06eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:13.724716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b25c5-9cf7-4b93-bd9f-c1a1f2b06eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:13.725032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b25c5-9cf7-4b93-bd9f-c1a1f2b06eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:13.725433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b25c5-9cf7-4b93-bd9f-c1a1f2b06eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:13.726266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b25c5-9cf7-4b93-bd9f-c1a1f2b06eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:13.728045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b25c5-9cf7-4b93-bd9f-c1a1f2b06eb7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:13.739704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6792ff-2f61-4900-9838-532afb6273f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:13.740391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6792ff-2f61-4900-9838-532afb6273f4 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:13.740684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6792ff-2f61-4900-9838-532afb6273f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:13.740988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6792ff-2f61-4900-9838-532afb6273f4 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:13.742550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6792ff-2f61-4900-9838-532afb6273f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:13.752855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6792ff-2f61-4900-9838-532afb6273f4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:13.761690 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1172/2315] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:13.762941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b25c5-9cf7-4b93-bd9f-c1a1f2b06eb7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:13.767123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:13.777969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b25c5-9cf7-4b93-bd9f-c1a1f2b06eb7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:13.784837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d219f-4d21-4c00-84b6-1f98f60a5bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:13.786001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d219f-4d21-4c00-84b6-1f98f60a5bed admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:13.786554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d219f-4d21-4c00-84b6-1f98f60a5bed admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:13.787045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d219f-4d21-4c00-84b6-1f98f60a5bed admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:13.788143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d219f-4d21-4c00-84b6-1f98f60a5bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:13.790469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d219f-4d21-4c00-84b6-1f98f60a5bed admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:13.807613 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1144/2316] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:13 2022] POST /identity/v3/users => generated 288 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:13.810829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6792ff-2f61-4900-9838-532afb6273f4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:13.830780 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1173/2317] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:13 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:13.831780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d219f-4d21-4c00-84b6-1f98f60a5bed admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:13.840690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6792ff-2f61-4900-9838-532afb6273f4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:13.904394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:13.927343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d219f-4d21-4c00-84b6-1f98f60a5bed admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:13.940226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6333da-fe1c-419f-a0c2-32655cacc303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:13.945457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6333da-fe1c-419f-a0c2-32655cacc303 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:13.945855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6333da-fe1c-419f-a0c2-32655cacc303 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:13.946403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6333da-fe1c-419f-a0c2-32655cacc303 admin admin] PATH_INFO: `/v3/projects/469527875c434c4988fafccdfb0a4e86/users/3fa2f5c506814884bf38ae55e6022c6d/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:13.957642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5eebe-febe-40aa-8e76-1af71ffb6250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:13.958262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5eebe-febe-40aa-8e76-1af71ffb6250 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:13.958549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5eebe-febe-40aa-8e76-1af71ffb6250 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:13.958830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5eebe-febe-40aa-8e76-1af71ffb6250 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:13.959483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5eebe-febe-40aa-8e76-1af71ffb6250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:13.962702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5eebe-febe-40aa-8e76-1af71ffb6250 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:13.972327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6333da-fe1c-419f-a0c2-32655cacc303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=469527875c434c4988fafccdfb0a4e86, user_id=3fa2f5c506814884bf38ae55e6022c6d, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:13.975880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6333da-fe1c-419f-a0c2-32655cacc303 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:13.982459 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1145/2318] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:13.983759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5eebe-febe-40aa-8e76-1af71ffb6250 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:13.998614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5eebe-febe-40aa-8e76-1af71ffb6250 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:13.999233 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1174/2319] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:13 2022] PUT /identity/v3/projects/469527875c434c4988fafccdfb0a4e86/users/3fa2f5c506814884bf38ae55e6022c6d/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:14.000610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6333da-fe1c-419f-a0c2-32655cacc303 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:14.150848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7d3a1-3bc2-4207-8261-6706847f4c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:14.151072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7d3a1-3bc2-4207-8261-6706847f4c25 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:14.151360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7d3a1-3bc2-4207-8261-6706847f4c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:14.151645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7d3a1-3bc2-4207-8261-6706847f4c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:14.152305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7d3a1-3bc2-4207-8261-6706847f4c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:14.154143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7d3a1-3bc2-4207-8261-6706847f4c25 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:14.167211 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1146/2320] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:14.168436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7d3a1-3bc2-4207-8261-6706847f4c25 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:14.172434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6333da-fe1c-419f-a0c2-32655cacc303 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:14.181952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06920746-15d2-4399-839c-c45fd196f90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:14.182486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7d3a1-3bc2-4207-8261-6706847f4c25 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:14.183294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06920746-15d2-4399-839c-c45fd196f90e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:14.184352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06920746-15d2-4399-839c-c45fd196f90e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:14.185102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06920746-15d2-4399-839c-c45fd196f90e admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:14.186137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06920746-15d2-4399-839c-c45fd196f90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:14.189953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06920746-15d2-4399-839c-c45fd196f90e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:14.192001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f386c4-1a44-4029-bf61-08f302628ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:14.192748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f386c4-1a44-4029-bf61-08f302628ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:14.193199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f386c4-1a44-4029-bf61-08f302628ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:14.193513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f386c4-1a44-4029-bf61-08f302628ac2 admin admin] PATH_INFO: `/v3/projects/ad8850110d7c49ed92be6077d7204301/users/ac23076fca324df1885d72bbf28aeac8/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:14.196685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f386c4-1a44-4029-bf61-08f302628ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8850110d7c49ed92be6077d7204301, user_id=ac23076fca324df1885d72bbf28aeac8, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:14.200357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f386c4-1a44-4029-bf61-08f302628ac2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:14.223920 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1175/2322] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:13 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:14.224654 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1147/2322] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:14 2022] PUT /identity/v3/projects/ad8850110d7c49ed92be6077d7204301/users/ac23076fca324df1885d72bbf28aeac8/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:14.225532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06920746-15d2-4399-839c-c45fd196f90e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:14.228572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f386c4-1a44-4029-bf61-08f302628ac2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:14.245614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06920746-15d2-4399-839c-c45fd196f90e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:14.326178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f386c4-1a44-4029-bf61-08f302628ac2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:14.338588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce330b03-0a4f-478d-acb8-afba1d55062d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:14.339112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25babc78-24e9-4a04-9b0f-fbfa9d5cfef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:14.340053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce330b03-0a4f-478d-acb8-afba1d55062d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:14.340560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25babc78-24e9-4a04-9b0f-fbfa9d5cfef2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:14.341548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25babc78-24e9-4a04-9b0f-fbfa9d5cfef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:14.341843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25babc78-24e9-4a04-9b0f-fbfa9d5cfef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:14.341981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce330b03-0a4f-478d-acb8-afba1d55062d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:14.343607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25babc78-24e9-4a04-9b0f-fbfa9d5cfef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:14.349856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce330b03-0a4f-478d-acb8-afba1d55062d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:14.350868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25babc78-24e9-4a04-9b0f-fbfa9d5cfef2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:14.351879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce330b03-0a4f-478d-acb8-afba1d55062d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:14.354421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce330b03-0a4f-478d-acb8-afba1d55062d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:14.376095 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1176/2323] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:14.377595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce330b03-0a4f-478d-acb8-afba1d55062d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:14.381607 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1148/2324] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:14.385189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7ecec-5ba7-4f0d-acce-951c96459905 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:14.385502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7ecec-5ba7-4f0d-acce-951c96459905 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:14.385785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7ecec-5ba7-4f0d-acce-951c96459905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:14.393817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce330b03-0a4f-478d-acb8-afba1d55062d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:14.401283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e838f5-b0c0-410c-b6d9-e4d5fa9c6fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:14.404229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e838f5-b0c0-410c-b6d9-e4d5fa9c6fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:14.404755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e838f5-b0c0-410c-b6d9-e4d5fa9c6fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:14.405336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e838f5-b0c0-410c-b6d9-e4d5fa9c6fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:14.411222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:14.420107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e838f5-b0c0-410c-b6d9-e4d5fa9c6fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:14.434950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e838f5-b0c0-410c-b6d9-e4d5fa9c6fb5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:14.455863 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1177/2325] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:14.457182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e838f5-b0c0-410c-b6d9-e4d5fa9c6fb5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:14.475167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e838f5-b0c0-410c-b6d9-e4d5fa9c6fb5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:14.475378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8ec7ecec-5ba7-4f0d-acce-951c96459905 None None] MFA Rules not processed for user `ac23076fca324df1885d72bbf28aeac8`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:14.476574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8ec7ecec-5ba7-4f0d-acce-951c96459905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:14.482933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186ca60-6eec-43bc-b288-4ca30c2f6159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:14.484574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186ca60-6eec-43bc-b288-4ca30c2f6159 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:14.485023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186ca60-6eec-43bc-b288-4ca30c2f6159 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:14.485432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186ca60-6eec-43bc-b288-4ca30c2f6159 admin admin] PATH_INFO: `/v3/projects/469527875c434c4988fafccdfb0a4e86/users/3fa2f5c506814884bf38ae55e6022c6d/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:14.487606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186ca60-6eec-43bc-b288-4ca30c2f6159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=469527875c434c4988fafccdfb0a4e86, user_id=3fa2f5c506814884bf38ae55e6022c6d, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:14.490765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186ca60-6eec-43bc-b288-4ca30c2f6159 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:14.509865 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1178/2326] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:14 2022] PUT /identity/v3/projects/469527875c434c4988fafccdfb0a4e86/users/3fa2f5c506814884bf38ae55e6022c6d/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:14.512526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186ca60-6eec-43bc-b288-4ca30c2f6159 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:14.605071 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1149/2327] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:14.606048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7ecec-5ba7-4f0d-acce-951c96459905 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:14.613605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:14.635759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7ecec-5ba7-4f0d-acce-951c96459905 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:14.694648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186ca60-6eec-43bc-b288-4ca30c2f6159 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:14.706405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd75b6-de3f-40a3-ab7b-86729743092f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:14.707103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd75b6-de3f-40a3-ab7b-86729743092f admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:14.707389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd75b6-de3f-40a3-ab7b-86729743092f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:14.707665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd75b6-de3f-40a3-ab7b-86729743092f admin admin] PATH_INFO: `/v3/projects/e96ec8c49f384e7d81244fd2476d0496/users/2e11c33c786746e5b3effedd32674582/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:14.737325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff6b7a-253d-48ca-b43b-ded5906544e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:14.738428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff6b7a-253d-48ca-b43b-ded5906544e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:14.738883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff6b7a-253d-48ca-b43b-ded5906544e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:14.739430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff6b7a-253d-48ca-b43b-ded5906544e4 admin admin] PATH_INFO: `/v3/projects/006c8f409597497d9ab1c774da8e1240/users/2eb6f6196ffe40e7bc9e67a31d88ddef/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:14.749941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd75b6-de3f-40a3-ab7b-86729743092f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96ec8c49f384e7d81244fd2476d0496, user_id=2e11c33c786746e5b3effedd32674582, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:14.756638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd75b6-de3f-40a3-ab7b-86729743092f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:14.762161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff6b7a-253d-48ca-b43b-ded5906544e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=006c8f409597497d9ab1c774da8e1240, user_id=2eb6f6196ffe40e7bc9e67a31d88ddef, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:14.765179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff6b7a-253d-48ca-b43b-ded5906544e4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:14.784684 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1179/2328] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:14 2022] PUT /identity/v3/projects/e96ec8c49f384e7d81244fd2476d0496/users/2e11c33c786746e5b3effedd32674582/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:14.786025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32fcb4-02a3-40ba-a219-fa704080b93d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:14.786321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32fcb4-02a3-40ba-a219-fa704080b93d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:14.786606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32fcb4-02a3-40ba-a219-fa704080b93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:14.837932 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1150/2329] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:14 2022] PUT /identity/v3/projects/006c8f409597497d9ab1c774da8e1240/users/2eb6f6196ffe40e7bc9e67a31d88ddef/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:14.839259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defceb68-d191-4e06-9f75-ba9d2ce7a057 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:14.839549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defceb68-d191-4e06-9f75-ba9d2ce7a057 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:14.839830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defceb68-d191-4e06-9f75-ba9d2ce7a057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:14.869246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3a32fcb4-02a3-40ba-a219-fa704080b93d None None] MFA Rules not processed for user `3fa2f5c506814884bf38ae55e6022c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:14.870563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3a32fcb4-02a3-40ba-a219-fa704080b93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:14.922525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-defceb68-d191-4e06-9f75-ba9d2ce7a057 None None] MFA Rules not processed for user `2e11c33c786746e5b3effedd32674582`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:14.927464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-defceb68-d191-4e06-9f75-ba9d2ce7a057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:14.956972 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1180/2330] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:14 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:14.957872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32fcb4-02a3-40ba-a219-fa704080b93d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:14.959434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3a32fcb4-02a3-40ba-a219-fa704080b93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:15.061815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32fcb4-02a3-40ba-a219-fa704080b93d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:15.073586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ea408-bc14-4417-87ea-e5ecc581e18f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:15.074503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ea408-bc14-4417-87ea-e5ecc581e18f service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:15.075294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ea408-bc14-4417-87ea-e5ecc581e18f service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:15.075578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ea408-bc14-4417-87ea-e5ecc581e18f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:15.090318 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1151/2331] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:15.095068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defceb68-d191-4e06-9f75-ba9d2ce7a057 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:15.107234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:15.119060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:15.166643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ea408-bc14-4417-87ea-e5ecc581e18f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:15.169084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ea408-bc14-4417-87ea-e5ecc581e18f service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:15.182839 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1181/2332] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:15.200736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ea408-bc14-4417-87ea-e5ecc581e18f service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:15.242118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defceb68-d191-4e06-9f75-ba9d2ce7a057 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:15.256532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959b461-e892-4e01-b76a-be2df4efb78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:15.258849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959b461-e892-4e01-b76a-be2df4efb78d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:15.260328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959b461-e892-4e01-b76a-be2df4efb78d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:15.260814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959b461-e892-4e01-b76a-be2df4efb78d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:15.262687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959b461-e892-4e01-b76a-be2df4efb78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:15.265477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959b461-e892-4e01-b76a-be2df4efb78d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:15.303227 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1152/2333] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:15.304508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959b461-e892-4e01-b76a-be2df4efb78d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:15.332034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959b461-e892-4e01-b76a-be2df4efb78d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:15.347740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394b927-3146-4c54-ad09-5d1d40af5649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:15.349527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394b927-3146-4c54-ad09-5d1d40af5649 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:15.351726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394b927-3146-4c54-ad09-5d1d40af5649 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:15.352430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394b927-3146-4c54-ad09-5d1d40af5649 admin admin] PATH_INFO: `/v3/projects/006c8f409597497d9ab1c774da8e1240/users/2eb6f6196ffe40e7bc9e67a31d88ddef/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:15.358407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4394b927-3146-4c54-ad09-5d1d40af5649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=006c8f409597497d9ab1c774da8e1240, user_id=2eb6f6196ffe40e7bc9e67a31d88ddef, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:15.365002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4394b927-3146-4c54-ad09-5d1d40af5649 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:15.372863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ea408-bc14-4417-87ea-e5ecc581e18f service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:15.381555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2919fe-0ac9-4877-b578-9a28afc3795f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:15.382188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2919fe-0ac9-4877-b578-9a28afc3795f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:15.382475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2919fe-0ac9-4877-b578-9a28afc3795f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:15.382773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2919fe-0ac9-4877-b578-9a28afc3795f admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:15.383405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2919fe-0ac9-4877-b578-9a28afc3795f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:15.385729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2919fe-0ac9-4877-b578-9a28afc3795f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:15.402164 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1153/2334] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:15 2022] PUT /identity/v3/projects/006c8f409597497d9ab1c774da8e1240/users/2eb6f6196ffe40e7bc9e67a31d88ddef/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:15.403424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394b927-3146-4c54-ad09-5d1d40af5649 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:15.426847 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1182/2335] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:15 2022] POST /identity/v3/projects => generated 373 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:15.428494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7899792-f43a-431d-bcc7-21da18fb883e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:15.428797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7899792-f43a-431d-bcc7-21da18fb883e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:15.429108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7899792-f43a-431d-bcc7-21da18fb883e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:15.470986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d7899792-f43a-431d-bcc7-21da18fb883e None None] MFA Rules not processed for user `2eb6f6196ffe40e7bc9e67a31d88ddef`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:15.472408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d7899792-f43a-431d-bcc7-21da18fb883e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:15.580256 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1183/2336] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:15.581307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7899792-f43a-431d-bcc7-21da18fb883e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:15.587042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394b927-3146-4c54-ad09-5d1d40af5649 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:15.639690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:15.676067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7899792-f43a-431d-bcc7-21da18fb883e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:15.682533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b356d1-d590-45ab-860c-10951ad9a62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:15.683182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b356d1-d590-45ab-860c-10951ad9a62a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:15.683468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b356d1-d590-45ab-860c-10951ad9a62a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:15.683788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b356d1-d590-45ab-860c-10951ad9a62a admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:15.684525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b356d1-d590-45ab-860c-10951ad9a62a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:15.687341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b356d1-d590-45ab-860c-10951ad9a62a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:15.722999 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1184/2337] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:15 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:15.730528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b356d1-d590-45ab-860c-10951ad9a62a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:15.741658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ae428-5f56-4727-8b24-418caeda71b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:15.742580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ae428-5f56-4727-8b24-418caeda71b1 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:15.742890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ae428-5f56-4727-8b24-418caeda71b1 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:15.743153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ae428-5f56-4727-8b24-418caeda71b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:15.753222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b356d1-d590-45ab-860c-10951ad9a62a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:15.760753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a116c-e05b-4154-9515-ba5a83298ee5 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb6f6196ffe40e7bc9e67a31d88ddef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '006c8f409597497d9ab1c774da8e1240', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:15.762467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a116c-e05b-4154-9515-ba5a83298ee5 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:15.763147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a116c-e05b-4154-9515-ba5a83298ee5 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:15.763554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a116c-e05b-4154-9515-ba5a83298ee5 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:15.764344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a116c-e05b-4154-9515-ba5a83298ee5 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:15.765297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a116c-e05b-4154-9515-ba5a83298ee5 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:15.768148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a116c-e05b-4154-9515-ba5a83298ee5 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:15.789826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:15.797981 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1185/2338] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:15.802491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a116c-e05b-4154-9515-ba5a83298ee5 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:15.826287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a116c-e05b-4154-9515-ba5a83298ee5 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:15.832489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503df3e9-0763-4d14-9322-714f15cbb9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:15.834118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503df3e9-0763-4d14-9322-714f15cbb9cd admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:15.834524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503df3e9-0763-4d14-9322-714f15cbb9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:15.834912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503df3e9-0763-4d14-9322-714f15cbb9cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:15.835765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503df3e9-0763-4d14-9322-714f15cbb9cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:15.837796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503df3e9-0763-4d14-9322-714f15cbb9cd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:15.863583 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1186/2339] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:15.869546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503df3e9-0763-4d14-9322-714f15cbb9cd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:15.877353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ae428-5f56-4727-8b24-418caeda71b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:15.879214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ae428-5f56-4727-8b24-418caeda71b1 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:15.894200 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1154/2340] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:15 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:15.894443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503df3e9-0763-4d14-9322-714f15cbb9cd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:15.895264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ae428-5f56-4727-8b24-418caeda71b1 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:15.904666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6a62b-c22e-4f43-ab22-65218bc2e20d tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb6f6196ffe40e7bc9e67a31d88ddef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '006c8f409597497d9ab1c774da8e1240', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:15.905403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6a62b-c22e-4f43-ab22-65218bc2e20d tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:15.905602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6a62b-c22e-4f43-ab22-65218bc2e20d tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:15.905886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6a62b-c22e-4f43-ab22-65218bc2e20d tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:15.906534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6a62b-c22e-4f43-ab22-65218bc2e20d tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:15.908463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6a62b-c22e-4f43-ab22-65218bc2e20d tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:15.926308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ae428-5f56-4727-8b24-418caeda71b1 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:15.932947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e47a1-1bf5-4c0d-ad72-afc46df75c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:15.934740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e47a1-1bf5-4c0d-ad72-afc46df75c83 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:15.936647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e47a1-1bf5-4c0d-ad72-afc46df75c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:15.937109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e47a1-1bf5-4c0d-ad72-afc46df75c83 admin admin] PATH_INFO: `/v3/projects/baf7e7f35c3f42b0a853dc2a0c196753/users/2f91eb6e45a14d1e8c18c9b94ae1567e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:15.962149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e47a1-1bf5-4c0d-ad72-afc46df75c83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf7e7f35c3f42b0a853dc2a0c196753, user_id=2f91eb6e45a14d1e8c18c9b94ae1567e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:15.963060 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1187/2341] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:15 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:15.965881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e47a1-1bf5-4c0d-ad72-afc46df75c83 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:15.967008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6a62b-c22e-4f43-ab22-65218bc2e20d tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:15.996999 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1155/2342] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:15 2022] PUT /identity/v3/projects/baf7e7f35c3f42b0a853dc2a0c196753/users/2f91eb6e45a14d1e8c18c9b94ae1567e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:15.999867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e47a1-1bf5-4c0d-ad72-afc46df75c83 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:16.091938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6a62b-c22e-4f43-ab22-65218bc2e20d tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:16.154148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e47a1-1bf5-4c0d-ad72-afc46df75c83 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:16.163478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b914344-368f-493c-acf2-6da24689c352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:16.164560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b914344-368f-493c-acf2-6da24689c352 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:16.165119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b914344-368f-493c-acf2-6da24689c352 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:16.166077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b914344-368f-493c-acf2-6da24689c352 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:16.169367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b914344-368f-493c-acf2-6da24689c352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:16.171736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b914344-368f-493c-acf2-6da24689c352 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:16.224747 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1156/2343] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:16.225506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b914344-368f-493c-acf2-6da24689c352 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:16.257374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b914344-368f-493c-acf2-6da24689c352 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:16.264776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2187b7-4b0c-4b7a-a568-cd5d44685ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:16.266298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2187b7-4b0c-4b7a-a568-cd5d44685ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:16.266768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2187b7-4b0c-4b7a-a568-cd5d44685ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:16.268276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2187b7-4b0c-4b7a-a568-cd5d44685ed1 admin admin] PATH_INFO: `/v3/projects/baf7e7f35c3f42b0a853dc2a0c196753/users/2f91eb6e45a14d1e8c18c9b94ae1567e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:16.271842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2187b7-4b0c-4b7a-a568-cd5d44685ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf7e7f35c3f42b0a853dc2a0c196753, user_id=2f91eb6e45a14d1e8c18c9b94ae1567e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:16.275746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2187b7-4b0c-4b7a-a568-cd5d44685ed1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:16.307781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6d5cd-e642-459b-a44b-f339c95a0da7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb6f6196ffe40e7bc9e67a31d88ddef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '006c8f409597497d9ab1c774da8e1240', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:16.308459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6d5cd-e642-459b-a44b-f339c95a0da7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:16.309320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6d5cd-e642-459b-a44b-f339c95a0da7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:16.309599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6d5cd-e642-459b-a44b-f339c95a0da7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:16.310278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6d5cd-e642-459b-a44b-f339c95a0da7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:16.310592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6d5cd-e642-459b-a44b-f339c95a0da7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:16.320726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6d5cd-e642-459b-a44b-f339c95a0da7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:16.326670 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1157/2344] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:16 2022] PUT /identity/v3/projects/baf7e7f35c3f42b0a853dc2a0c196753/users/2f91eb6e45a14d1e8c18c9b94ae1567e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:16.329791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde282a-a72b-48e7-8698-41f9b020ae5e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:16.330260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde282a-a72b-48e7-8698-41f9b020ae5e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:16.330722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde282a-a72b-48e7-8698-41f9b020ae5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:16.340002 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1188/2345] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:50:16.386629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ddde282a-a72b-48e7-8698-41f9b020ae5e None None] MFA Rules not processed for user `2f91eb6e45a14d1e8c18c9b94ae1567e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:16.389419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ddde282a-a72b-48e7-8698-41f9b020ae5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:16.547613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:16.556485 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1158/2346] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:16.569795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5bd9-ba2f-464d-88bc-9ff943e0a419 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:16.570266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5bd9-ba2f-464d-88bc-9ff943e0a419 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:16.570363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5bd9-ba2f-464d-88bc-9ff943e0a419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:16.598816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-928e5bd9-ba2f-464d-88bc-9ff943e0a419 None None] MFA Rules not processed for user `2e11c33c786746e5b3effedd32674582`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:16.600580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-928e5bd9-ba2f-464d-88bc-9ff943e0a419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:16.638540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:16.696286 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1189/2347] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:17.022433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde282a-a72b-48e7-8698-41f9b020ae5e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:17.229205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde282a-a72b-48e7-8698-41f9b020ae5e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:17.236351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646c5bb-c5ac-47d8-833b-4b5c898b5aca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:17.237039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646c5bb-c5ac-47d8-833b-4b5c898b5aca service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:17.237324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646c5bb-c5ac-47d8-833b-4b5c898b5aca service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:17.237601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646c5bb-c5ac-47d8-833b-4b5c898b5aca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:17.317675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646c5bb-c5ac-47d8-833b-4b5c898b5aca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:17.319892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646c5bb-c5ac-47d8-833b-4b5c898b5aca service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:17.333246 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1159/2348] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:17.350362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959767d-0296-42ad-a6e7-3ad8b8e4230d None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:17.350664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959767d-0296-42ad-a6e7-3ad8b8e4230d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:17.350949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959767d-0296-42ad-a6e7-3ad8b8e4230d None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:17.352462 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1190/2349] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:50:17.357956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646c5bb-c5ac-47d8-833b-4b5c898b5aca service nova] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:50:17.378063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646c5bb-c5ac-47d8-833b-4b5c898b5aca service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:17.387771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646c5bb-c5ac-47d8-833b-4b5c898b5aca service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:17.393005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecb2b2-6758-4dfb-8297-776936fba40b tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb6f6196ffe40e7bc9e67a31d88ddef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '006c8f409597497d9ab1c774da8e1240', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:17.393838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecb2b2-6758-4dfb-8297-776936fba40b tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:17.394226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecb2b2-6758-4dfb-8297-776936fba40b tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:17.394614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecb2b2-6758-4dfb-8297-776936fba40b tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] PATH_INFO: `/v3/projects/f1b9304b7bd04cbc82441062c0b4af3e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:17.396630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecb2b2-6758-4dfb-8297-776936fba40b tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorizing `identity:get_project(project_id=f1b9304b7bd04cbc82441062c0b4af3e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:17.398276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecb2b2-6758-4dfb-8297-776936fba40b tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:17.400280 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1160/2350] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:50:17 2022] GET /identity/v3/projects/f1b9304b7bd04cbc82441062c0b4af3e => generated 369 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:17.518224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959767d-0296-42ad-a6e7-3ad8b8e4230d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:17.535555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959767d-0296-42ad-a6e7-3ad8b8e4230d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:17.543992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f21711-9858-40b3-98ac-54e25cbc71b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:17.544619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f21711-9858-40b3-98ac-54e25cbc71b2 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:17.545280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f21711-9858-40b3-98ac-54e25cbc71b2 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:17.545430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f21711-9858-40b3-98ac-54e25cbc71b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:17.558109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f21711-9858-40b3-98ac-54e25cbc71b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:17.560418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f21711-9858-40b3-98ac-54e25cbc71b2 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:17.577339 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1191/2351] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:18.238462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3b4e2-c7a7-4895-8996-3a2c18b22c31 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:18.239929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3b4e2-c7a7-4895-8996-3a2c18b22c31 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:18.240442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3b4e2-c7a7-4895-8996-3a2c18b22c31 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:18.244408 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1161/2352] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:50:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:50:18.250323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f21711-9858-40b3-98ac-54e25cbc71b2 service nova] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:50:18.268538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f21711-9858-40b3-98ac-54e25cbc71b2 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:18.283754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f21711-9858-40b3-98ac-54e25cbc71b2 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:18.294125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18faf190-24e0-4c66-be65-75c5781764ad tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb6f6196ffe40e7bc9e67a31d88ddef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '006c8f409597497d9ab1c774da8e1240', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:18.295046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18faf190-24e0-4c66-be65-75c5781764ad tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:18.295172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18faf190-24e0-4c66-be65-75c5781764ad tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:18.295707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18faf190-24e0-4c66-be65-75c5781764ad tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] PATH_INFO: `/v3/projects/f1b9304b7bd04cbc82441062c0b4af3e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:18.299454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18faf190-24e0-4c66-be65-75c5781764ad tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorizing `identity:get_project(project_id=f1b9304b7bd04cbc82441062c0b4af3e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:18.302426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18faf190-24e0-4c66-be65-75c5781764ad tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:18.306024 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1192/2353] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:50:18 2022] GET /identity/v3/projects/f1b9304b7bd04cbc82441062c0b4af3e => generated 369 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:18.365018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3b4e2-c7a7-4895-8996-3a2c18b22c31 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:18.379287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3b4e2-c7a7-4895-8996-3a2c18b22c31 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:18.384590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a46dc6-dbe2-40d6-8060-3991384bcbb3 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb6f6196ffe40e7bc9e67a31d88ddef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '006c8f409597497d9ab1c774da8e1240', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:18.386838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a46dc6-dbe2-40d6-8060-3991384bcbb3 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:18.387615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a46dc6-dbe2-40d6-8060-3991384bcbb3 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:18.387615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a46dc6-dbe2-40d6-8060-3991384bcbb3 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:18.388116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a46dc6-dbe2-40d6-8060-3991384bcbb3 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:18.388434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a46dc6-dbe2-40d6-8060-3991384bcbb3 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:18.390067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a46dc6-dbe2-40d6-8060-3991384bcbb3 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:18.423341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18faf190-24e0-4c66-be65-75c5781764ad tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:18.431786 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1162/2354] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:18.438578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18faf190-24e0-4c66-be65-75c5781764ad tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:18.444628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5941b-fe59-4817-b70d-c297af41c629 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb6f6196ffe40e7bc9e67a31d88ddef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '006c8f409597497d9ab1c774da8e1240', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:18.445266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5941b-fe59-4817-b70d-c297af41c629 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:18.445553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5941b-fe59-4817-b70d-c297af41c629 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:18.445829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5941b-fe59-4817-b70d-c297af41c629 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:18.446820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5941b-fe59-4817-b70d-c297af41c629 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:18.449699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5941b-fe59-4817-b70d-c297af41c629 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:18.515987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a46dc6-dbe2-40d6-8060-3991384bcbb3 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:18.533473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a46dc6-dbe2-40d6-8060-3991384bcbb3 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:18.533765 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1193/2355] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:18 2022] POST /identity/v3/users => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:18.547318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca457cc4-64ee-4e07-b98a-834e911aaca7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb6f6196ffe40e7bc9e67a31d88ddef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '006c8f409597497d9ab1c774da8e1240', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:18.569781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca457cc4-64ee-4e07-b98a-834e911aaca7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:18.571198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca457cc4-64ee-4e07-b98a-834e911aaca7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:18.571588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca457cc4-64ee-4e07-b98a-834e911aaca7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:18.573527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca457cc4-64ee-4e07-b98a-834e911aaca7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:18.574681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca457cc4-64ee-4e07-b98a-834e911aaca7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:18.578829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca457cc4-64ee-4e07-b98a-834e911aaca7 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:18.644147 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1163/2356] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:19.381347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5941b-fe59-4817-b70d-c297af41c629 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:19.407426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054c10e-d521-4d7a-8889-84973e2f6206 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:19.408597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054c10e-d521-4d7a-8889-84973e2f6206 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:19.409263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054c10e-d521-4d7a-8889-84973e2f6206 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:19.411620 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1164/2357] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:50:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:50:19.421138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054c10e-d521-4d7a-8889-84973e2f6206 None None] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:50:19.462079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054c10e-d521-4d7a-8889-84973e2f6206 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:19.493661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054c10e-d521-4d7a-8889-84973e2f6206 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:19.511537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c824d3b-fee3-49cb-b275-47bd07c57f16 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb6f6196ffe40e7bc9e67a31d88ddef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '006c8f409597497d9ab1c774da8e1240', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:19.513759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c824d3b-fee3-49cb-b275-47bd07c57f16 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:19.514737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c824d3b-fee3-49cb-b275-47bd07c57f16 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:19.516411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c824d3b-fee3-49cb-b275-47bd07c57f16 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] PATH_INFO: `/v3/projects/f1b9304b7bd04cbc82441062c0b4af3e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:19.518006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c824d3b-fee3-49cb-b275-47bd07c57f16 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorizing `identity:get_project(project_id=f1b9304b7bd04cbc82441062c0b4af3e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:19.525810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c824d3b-fee3-49cb-b275-47bd07c57f16 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:19.531538 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1165/2358] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:50:19 2022] GET /identity/v3/projects/f1b9304b7bd04cbc82441062c0b4af3e => generated 369 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:19.557535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5941b-fe59-4817-b70d-c297af41c629 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:19.564974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:19.566057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:19.566558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:19.566975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] PATH_INFO: `/v3/users/08d418072145419ba6c07d5a06f6bc61` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:19.568322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d418072145419ba6c07d5a06f6bc61)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:19.571492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:19.646118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d418072145419ba6c07d5a06f6bc61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:19.665131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d418072145419ba6c07d5a06f6bc61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:19.675745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d418072145419ba6c07d5a06f6bc61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:19.676967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d418072145419ba6c07d5a06f6bc61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:19.677896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d418072145419ba6c07d5a06f6bc61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:19.695621 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1194/2359] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:19 2022] DELETE /identity/v3/users/08d418072145419ba6c07d5a06f6bc61 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:19.700949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397079f-ee5d-471f-ba2f-634d4ecf60da None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:19.701308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397079f-ee5d-471f-ba2f-634d4ecf60da None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:19.701649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397079f-ee5d-471f-ba2f-634d4ecf60da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:19.737125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c397079f-ee5d-471f-ba2f-634d4ecf60da None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:19.738780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c397079f-ee5d-471f-ba2f-634d4ecf60da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:19.817071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:19.846924 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1166/2360] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:19.857115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:19.858762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:19.871941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:19.954188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c033a94-3c79-420f-bd21-5f51d46c34b9 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:19.954828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c033a94-3c79-420f-bd21-5f51d46c34b9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:19.957221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c033a94-3c79-420f-bd21-5f51d46c34b9 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:19.967420 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1167/2361] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:50:19 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:50:19.968628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8f28e-b4f3-4323-860f-d5fc246e83f4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:19.980458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c033a94-3c79-420f-bd21-5f51d46c34b9 None None] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:50:19.982427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bf027-cb8d-4dd8-a894-4c68f66e1b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:19.983824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bf027-cb8d-4dd8-a894-4c68f66e1b7f service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:19.984159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bf027-cb8d-4dd8-a894-4c68f66e1b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:19.984441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bf027-cb8d-4dd8-a894-4c68f66e1b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:19.985806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8c033a94-3c79-420f-bd21-5f51d46c34b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:20.006172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83bf027-cb8d-4dd8-a894-4c68f66e1b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:20.020788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83bf027-cb8d-4dd8-a894-4c68f66e1b7f service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:20.053272 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1195/2362] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:20.213009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c033a94-3c79-420f-bd21-5f51d46c34b9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:20.216390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8c033a94-3c79-420f-bd21-5f51d46c34b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:20.300111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c033a94-3c79-420f-bd21-5f51d46c34b9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:20.309668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a57e47-d758-4ed2-9756-43759e4cee92 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb6f6196ffe40e7bc9e67a31d88ddef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '006c8f409597497d9ab1c774da8e1240', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:20.311609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a57e47-d758-4ed2-9756-43759e4cee92 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:20.312201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a57e47-d758-4ed2-9756-43759e4cee92 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:20.312811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a57e47-d758-4ed2-9756-43759e4cee92 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] PATH_INFO: `/v3/projects/f1b9304b7bd04cbc82441062c0b4af3e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:20.315127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a57e47-d758-4ed2-9756-43759e4cee92 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorizing `identity:get_project(project_id=f1b9304b7bd04cbc82441062c0b4af3e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:20.318869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a57e47-d758-4ed2-9756-43759e4cee92 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:20.322184 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1168/2363] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:50:19 2022] GET /identity/v3/projects/f1b9304b7bd04cbc82441062c0b4af3e => generated 369 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:20.375235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bf027-cb8d-4dd8-a894-4c68f66e1b7f service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:20.391107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bf027-cb8d-4dd8-a894-4c68f66e1b7f service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:20.406515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb6f6196ffe40e7bc9e67a31d88ddef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '006c8f409597497d9ab1c774da8e1240', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:20.407469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:20.407469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:20.407711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] PATH_INFO: `/v3/users/f392a042281840d3ba35a06caf31005b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:20.451354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorizing `identity:delete_user(user_id=f392a042281840d3ba35a06caf31005b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:20.457119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:20.564962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f392a042281840d3ba35a06caf31005b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:20.601966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f392a042281840d3ba35a06caf31005b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:20.622571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f392a042281840d3ba35a06caf31005b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:20.623471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f392a042281840d3ba35a06caf31005b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:20.625261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f392a042281840d3ba35a06caf31005b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:20.662981 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1196/2364] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:20 2022] DELETE /identity/v3/users/f392a042281840d3ba35a06caf31005b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:20.663399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a57e47-d758-4ed2-9756-43759e4cee92 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:20.666840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d3a57e47-d758-4ed2-9756-43759e4cee92 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:20.687690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a57e47-d758-4ed2-9756-43759e4cee92 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:20.774454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb6f6196ffe40e7bc9e67a31d88ddef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '006c8f409597497d9ab1c774da8e1240', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:20.775791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:20.776301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:20.776993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] PATH_INFO: `/v3/projects/f1b9304b7bd04cbc82441062c0b4af3e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:20.780041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f1b9304b7bd04cbc82441062c0b4af3e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:20.783449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:20.856779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:20.911460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b9304b7bd04cbc82441062c0b4af3e', 'request_id': 'req-b63ccbf5-7c5f-4b62-8c95-261b3031a338'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:20.929949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b9304b7bd04cbc82441062c0b4af3e', 'request_id': 'req-b63ccbf5-7c5f-4b62-8c95-261b3031a338'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:20.954512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b9304b7bd04cbc82441062c0b4af3e', 'request_id': 'req-b63ccbf5-7c5f-4b62-8c95-261b3031a338'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:20.987332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] The token cache is being invalidate because project f1b9304b7bd04cbc82441062c0b4af3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:20.987627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:20.988345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:20.991894 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1169/2365] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:20 2022] DELETE /identity/v3/projects/f1b9304b7bd04cbc82441062c0b4af3e => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:20.999658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:21.001524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:21.178392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d075a9d-8265-4a23-8213-e699710c6585 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:21.178392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:21.243522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6164492-a3a4-4761-a722-015145b1e68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:21.244258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6164492-a3a4-4761-a722-015145b1e68a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:21.244550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6164492-a3a4-4761-a722-015145b1e68a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:21.244829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6164492-a3a4-4761-a722-015145b1e68a admin admin] PATH_INFO: `/v3/users/ecdab6643a234862a797ebb53aaf5681` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:21.247082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6164492-a3a4-4761-a722-015145b1e68a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecdab6643a234862a797ebb53aaf5681)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:21.252886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6164492-a3a4-4761-a722-015145b1e68a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:21.339775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:21.342273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:21.368939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ccbf5-7c5f-4b62-8c95-261b3031a338 tempest-QuotasAdminTestV236-1585191909 tempest-QuotasAdminTestV236-1585191909-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:21.375238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:21.376143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:21.376585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:21.377401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f6164492-a3a4-4761-a722-015145b1e68a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecdab6643a234862a797ebb53aaf5681'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.377878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] PATH_INFO: `/v3/users/1ee2df69ab474fc9aa85624983fd8fa6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:21.380930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee2df69ab474fc9aa85624983fd8fa6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:21.384071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:21.409964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f6164492-a3a4-4761-a722-015145b1e68a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecdab6643a234862a797ebb53aaf5681'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.422382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f6164492-a3a4-4761-a722-015145b1e68a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecdab6643a234862a797ebb53aaf5681'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.423148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f6164492-a3a4-4761-a722-015145b1e68a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecdab6643a234862a797ebb53aaf5681'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.424476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f6164492-a3a4-4761-a722-015145b1e68a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecdab6643a234862a797ebb53aaf5681'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.462178 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1197/2366] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:20 2022] DELETE /identity/v3/users/ecdab6643a234862a797ebb53aaf5681 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:21.469408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa102cb-e3f0-49b6-b817-c4de9e46af63 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:21.469572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa102cb-e3f0-49b6-b817-c4de9e46af63 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:21.469800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa102cb-e3f0-49b6-b817-c4de9e46af63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:21.490270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee2df69ab474fc9aa85624983fd8fa6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.525264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-daa102cb-e3f0-49b6-b817-c4de9e46af63 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:21.525811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee2df69ab474fc9aa85624983fd8fa6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.526756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee2df69ab474fc9aa85624983fd8fa6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.528062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-daa102cb-e3f0-49b6-b817-c4de9e46af63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:21.538303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee2df69ab474fc9aa85624983fd8fa6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.552709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee2df69ab474fc9aa85624983fd8fa6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.568453 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1170/2367] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:21 2022] DELETE /identity/v3/users/1ee2df69ab474fc9aa85624983fd8fa6 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:21.570255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:21.572830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:21.667343 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1198/2368] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:21.669507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c34f1-023b-46dc-b101-03a6d8d7dd70 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:21.670017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c34f1-023b-46dc-b101-03a6d8d7dd70 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:21.670500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c34f1-023b-46dc-b101-03a6d8d7dd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:21.674480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95f008-7419-4e26-bad1-ea76146042d3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:21.685808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:21.686386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:21.686679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:21.691338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] PATH_INFO: `/v3/projects/ec21d1602f7b4dc5b825dcb479e67d95` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:21.693753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec21d1602f7b4dc5b825dcb479e67d95)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:21.696370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:21.705788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-614c34f1-023b-46dc-b101-03a6d8d7dd70 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:21.707322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-614c34f1-023b-46dc-b101-03a6d8d7dd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:21.732304 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1199/2369] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:21.737794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c34f1-023b-46dc-b101-03a6d8d7dd70 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:21.739314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-614c34f1-023b-46dc-b101-03a6d8d7dd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:21.776475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:21.813069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:21.821936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec21d1602f7b4dc5b825dcb479e67d95', 'request_id': 'req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.848600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec21d1602f7b4dc5b825dcb479e67d95', 'request_id': 'req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.850593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c34f1-023b-46dc-b101-03a6d8d7dd70 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:21.860661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec21d1602f7b4dc5b825dcb479e67d95', 'request_id': 'req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.880614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] The token cache is being invalidate because project ec21d1602f7b4dc5b825dcb479e67d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:21.881036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.881741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:21.890737 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1171/2370] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:21 2022] DELETE /identity/v3/projects/ec21d1602f7b4dc5b825dcb479e67d95 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:21.891019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:21.927607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:21.955245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44837e5-de73-4091-bc0d-fcb9b92e5dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:21.955878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44837e5-de73-4091-bc0d-fcb9b92e5dda service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:21.956165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44837e5-de73-4091-bc0d-fcb9b92e5dda service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:21.956442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44837e5-de73-4091-bc0d-fcb9b92e5dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:21.965622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b44837e5-de73-4091-bc0d-fcb9b92e5dda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:22.102661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65d456-4b92-4ff0-bdb4-e8c02a7311b2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:22.110808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44837e5-de73-4091-bc0d-fcb9b92e5dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:22.111245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9497f78-5ce6-4814-aa8b-171267f951f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:22.113119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9497f78-5ce6-4814-aa8b-171267f951f6 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:22.113545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9497f78-5ce6-4814-aa8b-171267f951f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:22.114661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9497f78-5ce6-4814-aa8b-171267f951f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:22.114962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44837e5-de73-4091-bc0d-fcb9b92e5dda service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:22.118989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e9497f78-5ce6-4814-aa8b-171267f951f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:22.142570 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1200/2371] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:50:22.144735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44837e5-de73-4091-bc0d-fcb9b92e5dda service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:22.146749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b44837e5-de73-4091-bc0d-fcb9b92e5dda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:22.148715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9497f78-5ce6-4814-aa8b-171267f951f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:22.154568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9497f78-5ce6-4814-aa8b-171267f951f6 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:22.175857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44837e5-de73-4091-bc0d-fcb9b92e5dda service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:22.184770 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1172/2372] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:22.186824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094d529-2a43-4296-b0d4-d53b4d7a5200 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:22.187718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094d529-2a43-4296-b0d4-d53b4d7a5200 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:22.188607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094d529-2a43-4296-b0d4-d53b4d7a5200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:22.193150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:22.195491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:22.195491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:22.195491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] PATH_INFO: `/v3/users/3fa2f5c506814884bf38ae55e6022c6d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:22.197974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa2f5c506814884bf38ae55e6022c6d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:22.205172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:22.238981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1094d529-2a43-4296-b0d4-d53b4d7a5200 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:22.240502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1094d529-2a43-4296-b0d4-d53b4d7a5200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:22.335476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa2f5c506814884bf38ae55e6022c6d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:22.366911 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1173/2373] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:22.367702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa2f5c506814884bf38ae55e6022c6d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:22.393392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa2f5c506814884bf38ae55e6022c6d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:22.394190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa2f5c506814884bf38ae55e6022c6d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:22.394668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa2f5c506814884bf38ae55e6022c6d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:22.413474 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1201/2374] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:22 2022] DELETE /identity/v3/users/3fa2f5c506814884bf38ae55e6022c6d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:22.441946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094d529-2a43-4296-b0d4-d53b4d7a5200 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:22.443468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1094d529-2a43-4296-b0d4-d53b4d7a5200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:22.529194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094d529-2a43-4296-b0d4-d53b4d7a5200 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:22.535489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:22.669045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc09618-ec64-4f50-81d4-3b23cc741422 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:22.670233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc09618-ec64-4f50-81d4-3b23cc741422 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:22.670676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc09618-ec64-4f50-81d4-3b23cc741422 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:22.670969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc09618-ec64-4f50-81d4-3b23cc741422 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:22.673320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-acc09618-ec64-4f50-81d4-3b23cc741422 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:22.755078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc09618-ec64-4f50-81d4-3b23cc741422 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:22.756723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc09618-ec64-4f50-81d4-3b23cc741422 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:22.770626 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1174/2375] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:50:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:50:23.572080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:23.574177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:23.629543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc09618-ec64-4f50-81d4-3b23cc741422 service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:23.631316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-acc09618-ec64-4f50-81d4-3b23cc741422 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:23.656682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:23.825720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a5d53-10ed-48dd-aff0-130f7728c9ae admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:23.832095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:23.833171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:23.833687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:23.834191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] PATH_INFO: `/v3/projects/f4d12291e32b4d9a91938e11c958eb96` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:23.840778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d12291e32b4d9a91938e11c958eb96)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:23.843706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:23.931529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d12291e32b4d9a91938e11c958eb96', 'request_id': 'req-c9278e25-bc9e-48ab-b046-1911596477e1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:23.935668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc09618-ec64-4f50-81d4-3b23cc741422 service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:23.950423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d12291e32b4d9a91938e11c958eb96', 'request_id': 'req-c9278e25-bc9e-48ab-b046-1911596477e1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:23.962945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d12291e32b4d9a91938e11c958eb96', 'request_id': 'req-c9278e25-bc9e-48ab-b046-1911596477e1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.032711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] The token cache is being invalidate because project f4d12291e32b4d9a91938e11c958eb96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:24.033507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.035151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.041270 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1202/2376] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:23 2022] DELETE /identity/v3/projects/f4d12291e32b4d9a91938e11c958eb96 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:24.046675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:24.048372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:24.241901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9278e25-bc9e-48ab-b046-1911596477e1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:24.259916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:24.265151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:24.265364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:24.265701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:24.266549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] PATH_INFO: `/v3/users/2eb6f6196ffe40e7bc9e67a31d88ddef` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:24.270729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb6f6196ffe40e7bc9e67a31d88ddef)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:24.274951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:24.293733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:24.294404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:24.298924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:24.299220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] PATH_INFO: `/v3/projects/ca0998946ce24a8da8b4fba9e9f79101` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:24.310922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca0998946ce24a8da8b4fba9e9f79101)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:24.317334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:24.407662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:24.415732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb6f6196ffe40e7bc9e67a31d88ddef'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.442658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb6f6196ffe40e7bc9e67a31d88ddef'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.464718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb6f6196ffe40e7bc9e67a31d88ddef'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.466688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb6f6196ffe40e7bc9e67a31d88ddef'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.467404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb6f6196ffe40e7bc9e67a31d88ddef'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.468118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca0998946ce24a8da8b4fba9e9f79101', 'request_id': 'req-1785564f-995c-4de3-9d8a-17803b91c46c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.500617 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1203/2377] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:24 2022] DELETE /identity/v3/users/2eb6f6196ffe40e7bc9e67a31d88ddef => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:50:24.503805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:24.505354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca0998946ce24a8da8b4fba9e9f79101', 'request_id': 'req-1785564f-995c-4de3-9d8a-17803b91c46c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.505712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:24.530928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca0998946ce24a8da8b4fba9e9f79101', 'request_id': 'req-1785564f-995c-4de3-9d8a-17803b91c46c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.584447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] The token cache is being invalidate because project ca0998946ce24a8da8b4fba9e9f79101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:24.584742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.592832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.600674 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1175/2378] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:23 2022] DELETE /identity/v3/projects/ca0998946ce24a8da8b4fba9e9f79101 => generated 0 bytes in 972 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:24.604349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:24.608431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:24.675479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59cc70-73d3-4f8f-8985-6d64d1164df6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:24.686027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74853425-e2d2-437b-aab2-6bdccfc5aa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:24.686930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74853425-e2d2-437b-aab2-6bdccfc5aa57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:24.687225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74853425-e2d2-437b-aab2-6bdccfc5aa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:24.687506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74853425-e2d2-437b-aab2-6bdccfc5aa57 admin admin] PATH_INFO: `/v3/projects/469527875c434c4988fafccdfb0a4e86` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:24.694180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74853425-e2d2-437b-aab2-6bdccfc5aa57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=469527875c434c4988fafccdfb0a4e86)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:24.696473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74853425-e2d2-437b-aab2-6bdccfc5aa57 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:24.803921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-74853425-e2d2-437b-aab2-6bdccfc5aa57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '469527875c434c4988fafccdfb0a4e86', 'request_id': 'req-74853425-e2d2-437b-aab2-6bdccfc5aa57'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.820426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-74853425-e2d2-437b-aab2-6bdccfc5aa57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '469527875c434c4988fafccdfb0a4e86', 'request_id': 'req-74853425-e2d2-437b-aab2-6bdccfc5aa57'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.828688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-74853425-e2d2-437b-aab2-6bdccfc5aa57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '469527875c434c4988fafccdfb0a4e86', 'request_id': 'req-74853425-e2d2-437b-aab2-6bdccfc5aa57'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.840725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785564f-995c-4de3-9d8a-17803b91c46c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:24.848215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-74853425-e2d2-437b-aab2-6bdccfc5aa57 admin admin] The token cache is being invalidate because project 469527875c434c4988fafccdfb0a4e86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:24.849357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-74853425-e2d2-437b-aab2-6bdccfc5aa57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.849473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-74853425-e2d2-437b-aab2-6bdccfc5aa57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:24.853215 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1204/2379] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:24 2022] DELETE /identity/v3/projects/469527875c434c4988fafccdfb0a4e86 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:24.860653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e9190-39d9-417e-a65a-5942e1057eff None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:24.861567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e9190-39d9-417e-a65a-5942e1057eff None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:24.864200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e9190-39d9-417e-a65a-5942e1057eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:24.912568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4c3e9190-39d9-417e-a65a-5942e1057eff None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:24.915056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4c3e9190-39d9-417e-a65a-5942e1057eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:24.919652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:24.920576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:24.921122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:24.921617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] PATH_INFO: `/v3/users/ac23076fca324df1885d72bbf28aeac8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:24.924667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac23076fca324df1885d72bbf28aeac8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:24.927655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:24.946078 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1205/2380] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:24.954386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841dff8-005b-4024-9f3a-b9b035fecfbd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:24.955187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841dff8-005b-4024-9f3a-b9b035fecfbd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:24.955687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841dff8-005b-4024-9f3a-b9b035fecfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:24.993199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4841dff8-005b-4024-9f3a-b9b035fecfbd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:24.994939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4841dff8-005b-4024-9f3a-b9b035fecfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:25.025438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:25.058484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac23076fca324df1885d72bbf28aeac8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:25.064718 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1206/2381] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:50:25.071594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841dff8-005b-4024-9f3a-b9b035fecfbd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:25.105256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841dff8-005b-4024-9f3a-b9b035fecfbd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:25.111262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac23076fca324df1885d72bbf28aeac8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:25.112819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac23076fca324df1885d72bbf28aeac8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:25.122869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a02b92-c607-4d4e-830e-945a9d33662d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:25.123469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a02b92-c607-4d4e-830e-945a9d33662d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:25.127245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a02b92-c607-4d4e-830e-945a9d33662d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:25.134272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a02b92-c607-4d4e-830e-945a9d33662d admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:25.134272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a02b92-c607-4d4e-830e-945a9d33662d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:25.134272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a02b92-c607-4d4e-830e-945a9d33662d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:25.134272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a02b92-c607-4d4e-830e-945a9d33662d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:25.136289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac23076fca324df1885d72bbf28aeac8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:25.156702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac23076fca324df1885d72bbf28aeac8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:25.166758 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1176/2382] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:24 2022] DELETE /identity/v3/users/ac23076fca324df1885d72bbf28aeac8 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:25.167569 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1207/2383] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:50:25.172064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:25.173962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:25.284023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beceaf1c-a5db-4c45-9fc0-fab92d6dba93 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:25.291612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaa79d-f515-4d13-9bc7-f7dd6bfba621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:25.292629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaa79d-f515-4d13-9bc7-f7dd6bfba621 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:25.294761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaa79d-f515-4d13-9bc7-f7dd6bfba621 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:25.295694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaa79d-f515-4d13-9bc7-f7dd6bfba621 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:25.296508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbaa79d-f515-4d13-9bc7-f7dd6bfba621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:25.305708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbaa79d-f515-4d13-9bc7-f7dd6bfba621 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:25.373155 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1177/2384] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:25 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:25.377202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a02b92-c607-4d4e-830e-945a9d33662d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:25.403879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a02b92-c607-4d4e-830e-945a9d33662d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:25.474783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:25.574974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210bef8-09aa-4918-8ddf-024dced6debb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:25.576306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210bef8-09aa-4918-8ddf-024dced6debb admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:25.576793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210bef8-09aa-4918-8ddf-024dced6debb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:25.578706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210bef8-09aa-4918-8ddf-024dced6debb admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:25.580868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210bef8-09aa-4918-8ddf-024dced6debb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:25.583386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210bef8-09aa-4918-8ddf-024dced6debb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:25.664707 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1208/2385] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:25 2022] POST /identity/v3/users => generated 291 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:25.668998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaa79d-f515-4d13-9bc7-f7dd6bfba621 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:25.688222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaa79d-f515-4d13-9bc7-f7dd6bfba621 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:25.730186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1d37b-24fc-4b8f-a89e-16435002f599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:25.734011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1d37b-24fc-4b8f-a89e-16435002f599 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:25.734468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1d37b-24fc-4b8f-a89e-16435002f599 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:25.735098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1d37b-24fc-4b8f-a89e-16435002f599 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:25.735941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1d37b-24fc-4b8f-a89e-16435002f599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:25.737921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1d37b-24fc-4b8f-a89e-16435002f599 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:25.802835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210bef8-09aa-4918-8ddf-024dced6debb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:25.806694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5210bef8-09aa-4918-8ddf-024dced6debb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:25.829806 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1178/2386] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:25.838579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1d37b-24fc-4b8f-a89e-16435002f599 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:25.841560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210bef8-09aa-4918-8ddf-024dced6debb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:25.865474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1d37b-24fc-4b8f-a89e-16435002f599 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:25.870426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:25.872855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:25.874882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:25.875629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f19b07-a183-4e4a-ad5e-4b6a4271e9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:25.875771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd admin admin] PATH_INFO: `/v3/projects/006c8f409597497d9ab1c774da8e1240` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:25.876311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f19b07-a183-4e4a-ad5e-4b6a4271e9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:25.876677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f19b07-a183-4e4a-ad5e-4b6a4271e9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:25.876993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f19b07-a183-4e4a-ad5e-4b6a4271e9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:25.877656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f19b07-a183-4e4a-ad5e-4b6a4271e9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:25.879266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f19b07-a183-4e4a-ad5e-4b6a4271e9c5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:25.882736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=006c8f409597497d9ab1c774da8e1240)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:25.890568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:25.911430 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1179/2387] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:25.912575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f19b07-a183-4e4a-ad5e-4b6a4271e9c5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:25.925098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f19b07-a183-4e4a-ad5e-4b6a4271e9c5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:25.932639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7cab2-565e-4364-9c59-5a0f7b46e708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:25.933374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7cab2-565e-4364-9c59-5a0f7b46e708 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:25.933698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7cab2-565e-4364-9c59-5a0f7b46e708 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:25.933984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7cab2-565e-4364-9c59-5a0f7b46e708 admin admin] PATH_INFO: `/v3/projects/4113aadc6ea344ea863152ddbf561958/users/0584a39d42c94900a41e4532682489ce/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:25.971799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7cab2-565e-4364-9c59-5a0f7b46e708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4113aadc6ea344ea863152ddbf561958, user_id=0584a39d42c94900a41e4532682489ce, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:25.985902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7cab2-565e-4364-9c59-5a0f7b46e708 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:26.008882 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1180/2388] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:25 2022] PUT /identity/v3/projects/4113aadc6ea344ea863152ddbf561958/users/0584a39d42c94900a41e4532682489ce/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:50:26.022017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '006c8f409597497d9ab1c774da8e1240', 'request_id': 'req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:26.037332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '006c8f409597497d9ab1c774da8e1240', 'request_id': 'req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:26.042004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de7f96-16e9-4d6c-b667-62972994239a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:26.044616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de7f96-16e9-4d6c-b667-62972994239a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:26.044973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de7f96-16e9-4d6c-b667-62972994239a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:26.057723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '006c8f409597497d9ab1c774da8e1240', 'request_id': 'req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:26.089708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd admin admin] The token cache is being invalidate because project 006c8f409597497d9ab1c774da8e1240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:26.090154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:26.090967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cbd71d28-b08d-4fd7-9f85-c816bd4b58dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:26.104960 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1209/2389] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:25 2022] DELETE /identity/v3/projects/006c8f409597497d9ab1c774da8e1240 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:26.104960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58734e61-1c07-4379-8b4b-333585b6cc81 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:26.104960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58734e61-1c07-4379-8b4b-333585b6cc81 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:26.104960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58734e61-1c07-4379-8b4b-333585b6cc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:26.177680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-58734e61-1c07-4379-8b4b-333585b6cc81 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:26.185750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-58734e61-1c07-4379-8b4b-333585b6cc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:26.186814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-02de7f96-16e9-4d6c-b667-62972994239a None None] MFA Rules not processed for user `0584a39d42c94900a41e4532682489ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:26.189144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-02de7f96-16e9-4d6c-b667-62972994239a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:26.265297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:26.330262 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1181/2390] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:26.334885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de7f96-16e9-4d6c-b667-62972994239a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:26.336330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-02de7f96-16e9-4d6c-b667-62972994239a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:26.417670 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1210/2391] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:26.419875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ff5ab-29f1-4f5d-8600-e285b62fcd92 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:26.420291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ff5ab-29f1-4f5d-8600-e285b62fcd92 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:26.420841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ff5ab-29f1-4f5d-8600-e285b62fcd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:26.463122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-973ff5ab-29f1-4f5d-8600-e285b62fcd92 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:26.464549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-973ff5ab-29f1-4f5d-8600-e285b62fcd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:26.492630 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1211/2392] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:26.500272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ff5ab-29f1-4f5d-8600-e285b62fcd92 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:26.504985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de7f96-16e9-4d6c-b667-62972994239a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:26.516308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ff5ab-29f1-4f5d-8600-e285b62fcd92 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:26.518602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ade488-6b1d-4049-8557-0ae2087a3628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:26.519429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ade488-6b1d-4049-8557-0ae2087a3628 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:26.519877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ade488-6b1d-4049-8557-0ae2087a3628 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:26.520401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ade488-6b1d-4049-8557-0ae2087a3628 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:26.521247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ade488-6b1d-4049-8557-0ae2087a3628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:26.529822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ade488-6b1d-4049-8557-0ae2087a3628 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:26.531156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e45b0c-5c6a-4df2-a5e2-ce328e997353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:26.531908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e45b0c-5c6a-4df2-a5e2-ce328e997353 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:26.532310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e45b0c-5c6a-4df2-a5e2-ce328e997353 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:26.532828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e45b0c-5c6a-4df2-a5e2-ce328e997353 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:26.533728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e45b0c-5c6a-4df2-a5e2-ce328e997353 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:26.534237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e45b0c-5c6a-4df2-a5e2-ce328e997353 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:26.535967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e45b0c-5c6a-4df2-a5e2-ce328e997353 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:26.564586 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1212/2393] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:26.570357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e45b0c-5c6a-4df2-a5e2-ce328e997353 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:26.586601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e45b0c-5c6a-4df2-a5e2-ce328e997353 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:26.597611 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1182/2394] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:26 2022] POST /identity/v3/projects => generated 383 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:50:26.599919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c669bf-0fd2-467f-83b8-5311064d5d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:26.601242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ade488-6b1d-4049-8557-0ae2087a3628 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:26.601908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c669bf-0fd2-467f-83b8-5311064d5d32 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:26.603167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c669bf-0fd2-467f-83b8-5311064d5d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:26.603537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c669bf-0fd2-467f-83b8-5311064d5d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:26.605125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c669bf-0fd2-467f-83b8-5311064d5d32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:26.611678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c669bf-0fd2-467f-83b8-5311064d5d32 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:26.658524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:26.691283 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1213/2395] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:26 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:26.695499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c669bf-0fd2-467f-83b8-5311064d5d32 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:26.770136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ade488-6b1d-4049-8557-0ae2087a3628 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:26.834728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c669bf-0fd2-467f-83b8-5311064d5d32 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:26.841861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada840a-47d1-4ac3-bc8a-dbe9f16ef01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:26.843166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada840a-47d1-4ac3-bc8a-dbe9f16ef01d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:26.844037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada840a-47d1-4ac3-bc8a-dbe9f16ef01d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:26.844488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada840a-47d1-4ac3-bc8a-dbe9f16ef01d admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:26.845341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada840a-47d1-4ac3-bc8a-dbe9f16ef01d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:26.847901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada840a-47d1-4ac3-bc8a-dbe9f16ef01d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:26.892013 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1214/2396] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:26 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:26.900126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada840a-47d1-4ac3-bc8a-dbe9f16ef01d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:26.908277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f1d95-b053-4227-bca6-28922ec17f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:26.908988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f1d95-b053-4227-bca6-28922ec17f36 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:26.909280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f1d95-b053-4227-bca6-28922ec17f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:26.909587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f1d95-b053-4227-bca6-28922ec17f36 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:26.910208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f1d95-b053-4227-bca6-28922ec17f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:26.912164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f1d95-b053-4227-bca6-28922ec17f36 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:26.934328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada840a-47d1-4ac3-bc8a-dbe9f16ef01d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:26.940017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9bb90-8b26-4a5c-956c-38f71ec26862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:26.940647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9bb90-8b26-4a5c-956c-38f71ec26862 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:26.940977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9bb90-8b26-4a5c-956c-38f71ec26862 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:26.941243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9bb90-8b26-4a5c-956c-38f71ec26862 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:26.942117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9bb90-8b26-4a5c-956c-38f71ec26862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:26.943806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9bb90-8b26-4a5c-956c-38f71ec26862 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:26.963538 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1183/2397] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:26 2022] POST /identity/v3/users => generated 298 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:50:26.965751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f1d95-b053-4227-bca6-28922ec17f36 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:26.976200 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1215/2398] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:26.977090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:26.977234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9bb90-8b26-4a5c-956c-38f71ec26862 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:26.988584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f1d95-b053-4227-bca6-28922ec17f36 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:26.996341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721228a8-33bd-4c83-93d1-e75667af5c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:26.997205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721228a8-33bd-4c83-93d1-e75667af5c8e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:26.997615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721228a8-33bd-4c83-93d1-e75667af5c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:26.999496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721228a8-33bd-4c83-93d1-e75667af5c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:27.000366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721228a8-33bd-4c83-93d1-e75667af5c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:27.002180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721228a8-33bd-4c83-93d1-e75667af5c8e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:27.002321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9bb90-8b26-4a5c-956c-38f71ec26862 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:27.016866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035db17d-7315-4fa5-a78a-e3193d3a30a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:27.017674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035db17d-7315-4fa5-a78a-e3193d3a30a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:27.018075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035db17d-7315-4fa5-a78a-e3193d3a30a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:27.018506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035db17d-7315-4fa5-a78a-e3193d3a30a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:27.019282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035db17d-7315-4fa5-a78a-e3193d3a30a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:27.024716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035db17d-7315-4fa5-a78a-e3193d3a30a8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:27.027792 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1184/2399] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:27.029397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721228a8-33bd-4c83-93d1-e75667af5c8e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:27.048045 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1216/2400] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:27.051086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721228a8-33bd-4c83-93d1-e75667af5c8e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:27.052540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035db17d-7315-4fa5-a78a-e3193d3a30a8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:27.069718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035db17d-7315-4fa5-a78a-e3193d3a30a8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:27.079318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371cc47-ac81-426c-943d-c71f49001060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:27.080656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371cc47-ac81-426c-943d-c71f49001060 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:27.081725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371cc47-ac81-426c-943d-c71f49001060 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:27.082592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371cc47-ac81-426c-943d-c71f49001060 admin admin] PATH_INFO: `/v3/projects/7c4932c6af424797b765ff8215cf6266/users/1a36774d43454b31b42fd27c8aa22d1a/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:27.084940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15792865-c1e4-4d19-9f8f-66ecbf41d489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:27.085788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15792865-c1e4-4d19-9f8f-66ecbf41d489 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:27.086386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15792865-c1e4-4d19-9f8f-66ecbf41d489 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:27.088170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15792865-c1e4-4d19-9f8f-66ecbf41d489 admin admin] PATH_INFO: `/v3/projects/7ca286521ec549da9fde582fc148f45a/users/d7956df7a8954bbc9571a8d7c29e3f34/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:27.138074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371cc47-ac81-426c-943d-c71f49001060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c4932c6af424797b765ff8215cf6266, user_id=1a36774d43454b31b42fd27c8aa22d1a, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:27.141417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371cc47-ac81-426c-943d-c71f49001060 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:27.142085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15792865-c1e4-4d19-9f8f-66ecbf41d489 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca286521ec549da9fde582fc148f45a, user_id=d7956df7a8954bbc9571a8d7c29e3f34, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:27.147432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15792865-c1e4-4d19-9f8f-66ecbf41d489 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:27.172814 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1185/2401] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:27 2022] PUT /identity/v3/projects/7c4932c6af424797b765ff8215cf6266/users/1a36774d43454b31b42fd27c8aa22d1a/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:27.179954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371cc47-ac81-426c-943d-c71f49001060 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:27.183236 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1217/2402] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:27 2022] PUT /identity/v3/projects/7ca286521ec549da9fde582fc148f45a/users/d7956df7a8954bbc9571a8d7c29e3f34/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:27.199094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861b705-5318-4c2f-8bb2-e55ecc6f1897 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:27.199912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861b705-5318-4c2f-8bb2-e55ecc6f1897 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:27.200189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861b705-5318-4c2f-8bb2-e55ecc6f1897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:27.289177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9861b705-5318-4c2f-8bb2-e55ecc6f1897 None None] MFA Rules not processed for user `d7956df7a8954bbc9571a8d7c29e3f34`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:27.290414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9861b705-5318-4c2f-8bb2-e55ecc6f1897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:27.379309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371cc47-ac81-426c-943d-c71f49001060 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:27.471251 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1218/2403] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:27.478032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861b705-5318-4c2f-8bb2-e55ecc6f1897 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:27.531223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf3b19-1207-4d8f-a3cc-491ccb3209b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:27.532802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf3b19-1207-4d8f-a3cc-491ccb3209b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:27.533361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf3b19-1207-4d8f-a3cc-491ccb3209b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:27.533846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf3b19-1207-4d8f-a3cc-491ccb3209b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:27.534854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf3b19-1207-4d8f-a3cc-491ccb3209b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:27.539688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf3b19-1207-4d8f-a3cc-491ccb3209b7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:27.556358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:27.559188 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1186/2404] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:27.564308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf3b19-1207-4d8f-a3cc-491ccb3209b7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:27.582457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf3b19-1207-4d8f-a3cc-491ccb3209b7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:27.598453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b38e6-34a7-467a-aa05-2b52ac235670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:27.599266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b38e6-34a7-467a-aa05-2b52ac235670 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:27.601278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b38e6-34a7-467a-aa05-2b52ac235670 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:27.603746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b38e6-34a7-467a-aa05-2b52ac235670 admin admin] PATH_INFO: `/v3/projects/7c4932c6af424797b765ff8215cf6266/users/1a36774d43454b31b42fd27c8aa22d1a/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:27.610947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b38e6-34a7-467a-aa05-2b52ac235670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c4932c6af424797b765ff8215cf6266, user_id=1a36774d43454b31b42fd27c8aa22d1a, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:27.615283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b38e6-34a7-467a-aa05-2b52ac235670 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:27.640964 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1187/2405] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:27 2022] PUT /identity/v3/projects/7c4932c6af424797b765ff8215cf6266/users/1a36774d43454b31b42fd27c8aa22d1a/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:27.651734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74d4c4-0450-4023-8017-eb01231a6a0b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:27.652791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74d4c4-0450-4023-8017-eb01231a6a0b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:27.653239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74d4c4-0450-4023-8017-eb01231a6a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:27.667582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861b705-5318-4c2f-8bb2-e55ecc6f1897 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:27.786969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bc74d4c4-0450-4023-8017-eb01231a6a0b None None] MFA Rules not processed for user `1a36774d43454b31b42fd27c8aa22d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:27.788237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bc74d4c4-0450-4023-8017-eb01231a6a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:27.846657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74e9de-af02-4f89-b458-3a61216ccd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:27.847564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74e9de-af02-4f89-b458-3a61216ccd8d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:27.847861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74e9de-af02-4f89-b458-3a61216ccd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:27.848143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74e9de-af02-4f89-b458-3a61216ccd8d admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:27.848799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74e9de-af02-4f89-b458-3a61216ccd8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:27.850812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74e9de-af02-4f89-b458-3a61216ccd8d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:27.907515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:27.921839 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1219/2406] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:27 2022] POST /identity/v3/projects => generated 377 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:27.923627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74e9de-af02-4f89-b458-3a61216ccd8d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:27.927683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cf74e9de-af02-4f89-b458-3a61216ccd8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:27.966767 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1188/2407] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:27.968368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897da7e3-83b0-4d81-9018-3d196f0d03f4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:27.968646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897da7e3-83b0-4d81-9018-3d196f0d03f4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:27.974457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897da7e3-83b0-4d81-9018-3d196f0d03f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:28.046057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-897da7e3-83b0-4d81-9018-3d196f0d03f4 None None] MFA Rules not processed for user `30ebd40778e3492199eef13eb0ab3310`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:28.048050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-897da7e3-83b0-4d81-9018-3d196f0d03f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:28.083052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74e9de-af02-4f89-b458-3a61216ccd8d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:28.099470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:28.100131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:28.100418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:28.100859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] PATH_INFO: `/v3/projects/ad8850110d7c49ed92be6077d7204301` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:28.105185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8850110d7c49ed92be6077d7204301)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:28.107722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:28.230848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8850110d7c49ed92be6077d7204301', 'request_id': 'req-35533d26-3667-4ba6-9cea-5daa0468e800'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:28.242636 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1189/2408] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:28.244747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897da7e3-83b0-4d81-9018-3d196f0d03f4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:28.250922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8850110d7c49ed92be6077d7204301', 'request_id': 'req-35533d26-3667-4ba6-9cea-5daa0468e800'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:28.262753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897da7e3-83b0-4d81-9018-3d196f0d03f4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:28.269961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8850110d7c49ed92be6077d7204301', 'request_id': 'req-35533d26-3667-4ba6-9cea-5daa0468e800'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:28.273911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:28.309505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] The token cache is being invalidate because project ad8850110d7c49ed92be6077d7204301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:28.310243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:28.312246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:28.314504 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1220/2409] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:27 2022] DELETE /identity/v3/projects/ad8850110d7c49ed92be6077d7204301 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:28.315322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:28.321326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:28.374743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa64990-a0ed-4238-9aca-b9e8da48efdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:28.375472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa64990-a0ed-4238-9aca-b9e8da48efdb admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:28.376308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa64990-a0ed-4238-9aca-b9e8da48efdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:28.376857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa64990-a0ed-4238-9aca-b9e8da48efdb admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:28.377861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa64990-a0ed-4238-9aca-b9e8da48efdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:28.387390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa64990-a0ed-4238-9aca-b9e8da48efdb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:28.443551 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1190/2410] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:28 2022] POST /identity/v3/users => generated 295 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:28.450443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33438f7e-7d56-4f36-88b2-5ca4527a2eb5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:28.450977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33438f7e-7d56-4f36-88b2-5ca4527a2eb5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:28.451497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33438f7e-7d56-4f36-88b2-5ca4527a2eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:28.483412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-33438f7e-7d56-4f36-88b2-5ca4527a2eb5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:28.485116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-33438f7e-7d56-4f36-88b2-5ca4527a2eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:28.503237 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1191/2411] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:28.504651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33438f7e-7d56-4f36-88b2-5ca4527a2eb5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:28.506184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-33438f7e-7d56-4f36-88b2-5ca4527a2eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:28.520170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35533d26-3667-4ba6-9cea-5daa0468e800 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:28.532243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33438f7e-7d56-4f36-88b2-5ca4527a2eb5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:28.540525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f786b0-9415-47ad-9e94-51ab1fc31be2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:28.541207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f786b0-9415-47ad-9e94-51ab1fc31be2 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:28.543664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f786b0-9415-47ad-9e94-51ab1fc31be2 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:28.543972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f786b0-9415-47ad-9e94-51ab1fc31be2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:28.551426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-15f786b0-9415-47ad-9e94-51ab1fc31be2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:28.564694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523167e-962f-40a9-9c84-a9554918a83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:28.565647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523167e-962f-40a9-9c84-a9554918a83b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:28.566285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523167e-962f-40a9-9c84-a9554918a83b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:28.566733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523167e-962f-40a9-9c84-a9554918a83b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:28.567554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523167e-962f-40a9-9c84-a9554918a83b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:28.569374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523167e-962f-40a9-9c84-a9554918a83b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:28.601080 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1192/2412] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:28.612078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04bc51-b7a9-444b-b5d4-44aad57380ab None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:28.612527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04bc51-b7a9-444b-b5d4-44aad57380ab None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:28.612848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04bc51-b7a9-444b-b5d4-44aad57380ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:28.687486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5d04bc51-b7a9-444b-b5d4-44aad57380ab None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:28.687486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5d04bc51-b7a9-444b-b5d4-44aad57380ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:28.714074 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1193/2413] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:28.714999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04bc51-b7a9-444b-b5d4-44aad57380ab None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:28.732890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f786b0-9415-47ad-9e94-51ab1fc31be2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:28.735349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f786b0-9415-47ad-9e94-51ab1fc31be2 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:28.748557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04bc51-b7a9-444b-b5d4-44aad57380ab None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:28.770684 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1221/2414] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:28.772223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f786b0-9415-47ad-9e94-51ab1fc31be2 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:28.785458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddf3e1-8052-4524-9ac6-3d5847c2df54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:28.786652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddf3e1-8052-4524-9ac6-3d5847c2df54 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:28.789885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddf3e1-8052-4524-9ac6-3d5847c2df54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:28.794296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddf3e1-8052-4524-9ac6-3d5847c2df54 admin admin] PATH_INFO: `/v3/projects/0920989729ee4c37a055e13d1dda2d1f/users/e12b04847ebf4d249d1718aef14680e1/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:28.802010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f786b0-9415-47ad-9e94-51ab1fc31be2 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:28.817337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40f813-b555-4709-be31-7f23f1db7cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:28.823363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40f813-b555-4709-be31-7f23f1db7cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:28.823746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40f813-b555-4709-be31-7f23f1db7cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:28.824134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40f813-b555-4709-be31-7f23f1db7cb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:28.824883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40f813-b555-4709-be31-7f23f1db7cb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:28.825478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40f813-b555-4709-be31-7f23f1db7cb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:28.827984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40f813-b555-4709-be31-7f23f1db7cb7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:28.856555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddf3e1-8052-4524-9ac6-3d5847c2df54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0920989729ee4c37a055e13d1dda2d1f, user_id=e12b04847ebf4d249d1718aef14680e1, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:28.867047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddf3e1-8052-4524-9ac6-3d5847c2df54 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:28.897623 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1222/2415] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:28.899511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40f813-b555-4709-be31-7f23f1db7cb7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:28.905064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0a40f813-b555-4709-be31-7f23f1db7cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:28.906571 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1194/2416] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:28 2022] PUT /identity/v3/projects/0920989729ee4c37a055e13d1dda2d1f/users/e12b04847ebf4d249d1718aef14680e1/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:28.907435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddf3e1-8052-4524-9ac6-3d5847c2df54 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:28.948333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:28.957037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40f813-b555-4709-be31-7f23f1db7cb7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:29.096721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44a56e-362a-4d02-b351-2c9bd5f4e5e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:29.098249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44a56e-362a-4d02-b351-2c9bd5f4e5e0 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:29.098692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44a56e-362a-4d02-b351-2c9bd5f4e5e0 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:29.099621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44a56e-362a-4d02-b351-2c9bd5f4e5e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:29.103488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dd44a56e-362a-4d02-b351-2c9bd5f4e5e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:29.108360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddf3e1-8052-4524-9ac6-3d5847c2df54 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:29.114375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855adbc-9acd-4477-b3b6-10cbf66b94bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:29.115131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855adbc-9acd-4477-b3b6-10cbf66b94bf admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:29.115524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855adbc-9acd-4477-b3b6-10cbf66b94bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:29.116744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:29.117707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855adbc-9acd-4477-b3b6-10cbf66b94bf admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:29.122251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855adbc-9acd-4477-b3b6-10cbf66b94bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:29.124117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855adbc-9acd-4477-b3b6-10cbf66b94bf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:29.175009 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1195/2417] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:28 2022] POST /identity/v3/projects => generated 397 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:29.175730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855adbc-9acd-4477-b3b6-10cbf66b94bf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:29.262887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44a56e-362a-4d02-b351-2c9bd5f4e5e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:29.264569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44a56e-362a-4d02-b351-2c9bd5f4e5e0 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:29.276478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855adbc-9acd-4477-b3b6-10cbf66b94bf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:29.287574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d3a61-8c8a-487b-bab1-75ea37fbfb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:29.288465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d3a61-8c8a-487b-bab1-75ea37fbfb37 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:29.288864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d3a61-8c8a-487b-bab1-75ea37fbfb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:29.289318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d3a61-8c8a-487b-bab1-75ea37fbfb37 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:29.290893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d3a61-8c8a-487b-bab1-75ea37fbfb37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:29.295183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d3a61-8c8a-487b-bab1-75ea37fbfb37 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:29.304460 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1223/2418] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:28 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:29.305576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44a56e-362a-4d02-b351-2c9bd5f4e5e0 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:29.313160 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1196/2419] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:29.321788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d3a61-8c8a-487b-bab1-75ea37fbfb37 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:29.331927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d3a61-8c8a-487b-bab1-75ea37fbfb37 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:29.333393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44a56e-362a-4d02-b351-2c9bd5f4e5e0 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:29.338398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d839a20-61ec-4954-9a9c-cc919b7613ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:29.339306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d839a20-61ec-4954-9a9c-cc919b7613ff admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:29.339739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d839a20-61ec-4954-9a9c-cc919b7613ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:29.340227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d839a20-61ec-4954-9a9c-cc919b7613ff admin admin] PATH_INFO: `/v3/projects/0920989729ee4c37a055e13d1dda2d1f/users/e12b04847ebf4d249d1718aef14680e1/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:29.342734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d839a20-61ec-4954-9a9c-cc919b7613ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=0920989729ee4c37a055e13d1dda2d1f, user_id=e12b04847ebf4d249d1718aef14680e1, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:29.345250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ca3a6-6947-46ad-8594-040857ddddcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:29.346538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d839a20-61ec-4954-9a9c-cc919b7613ff admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:29.348807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ca3a6-6947-46ad-8594-040857ddddcb admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:29.349128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ca3a6-6947-46ad-8594-040857ddddcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:29.349408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ca3a6-6947-46ad-8594-040857ddddcb admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:29.350753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ca3a6-6947-46ad-8594-040857ddddcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:29.358411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ca3a6-6947-46ad-8594-040857ddddcb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:29.372158 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1197/2420] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:29 2022] PUT /identity/v3/projects/0920989729ee4c37a055e13d1dda2d1f/users/e12b04847ebf4d249d1718aef14680e1/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:50:29.375063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b17ad-c616-4b1a-b7f2-ee19a52dc116 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:29.375463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b17ad-c616-4b1a-b7f2-ee19a52dc116 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:29.375979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b17ad-c616-4b1a-b7f2-ee19a52dc116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:29.410990 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1224/2421] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:29 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:29.412447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ca3a6-6947-46ad-8594-040857ddddcb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:29.416642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ad2b17ad-c616-4b1a-b7f2-ee19a52dc116 None None] MFA Rules not processed for user `e12b04847ebf4d249d1718aef14680e1`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:29.418904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ad2b17ad-c616-4b1a-b7f2-ee19a52dc116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:29.432601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ca3a6-6947-46ad-8594-040857ddddcb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:29.525002 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1198/2422] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:50:29.533994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc765d-0b4e-4c85-a06b-463baab21612 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:29.534650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc765d-0b4e-4c85-a06b-463baab21612 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:29.535461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc765d-0b4e-4c85-a06b-463baab21612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:29.558028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:29.600572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8bcc765d-0b4e-4c85-a06b-463baab21612 None None] MFA Rules not processed for user `d7956df7a8954bbc9571a8d7c29e3f34`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:29.601812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef99e4d-5d97-4d44-8297-dfbe5071e312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:29.602659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef99e4d-5d97-4d44-8297-dfbe5071e312 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:29.602955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef99e4d-5d97-4d44-8297-dfbe5071e312 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:29.603236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef99e4d-5d97-4d44-8297-dfbe5071e312 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:29.603670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8bcc765d-0b4e-4c85-a06b-463baab21612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:29.603964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef99e4d-5d97-4d44-8297-dfbe5071e312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:29.614394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef99e4d-5d97-4d44-8297-dfbe5071e312 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:29.651648 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1225/2423] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:50:29.654640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef99e4d-5d97-4d44-8297-dfbe5071e312 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:29.661078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2ef99e4d-5d97-4d44-8297-dfbe5071e312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:29.685287 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1199/2424] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:50:29.687795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc765d-0b4e-4c85-a06b-463baab21612 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:29.701826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef99e4d-5d97-4d44-8297-dfbe5071e312 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:29.716029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:29.718524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:29.718846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:29.720689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] PATH_INFO: `/v3/users/0584a39d42c94900a41e4532682489ce` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:29.723159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc765d-0b4e-4c85-a06b-463baab21612 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:29.723414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0584a39d42c94900a41e4532682489ce)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:29.730880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d258e-c99f-42ab-bc1b-7716597664bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:29.731840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d258e-c99f-42ab-bc1b-7716597664bb admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:29.732559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d258e-c99f-42ab-bc1b-7716597664bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:29.733099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d258e-c99f-42ab-bc1b-7716597664bb admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:29.733296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:29.734181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d258e-c99f-42ab-bc1b-7716597664bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:29.736673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d258e-c99f-42ab-bc1b-7716597664bb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:29.752107 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1200/2425] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:29.756457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d258e-c99f-42ab-bc1b-7716597664bb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:29.769855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d258e-c99f-42ab-bc1b-7716597664bb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:29.781849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995e13a-363a-4792-b11d-79cbc3c0d91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:29.783080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995e13a-363a-4792-b11d-79cbc3c0d91b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:29.783714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995e13a-363a-4792-b11d-79cbc3c0d91b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:29.784774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995e13a-363a-4792-b11d-79cbc3c0d91b admin admin] PATH_INFO: `/v3/projects/361f907a187547aabda6cb54eeeb3684/users/778561ee5e474accb31816f357985f44/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:29.806111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995e13a-363a-4792-b11d-79cbc3c0d91b admin admin] RBAC: Authorizing `identity:create_grant(project_id=361f907a187547aabda6cb54eeeb3684, user_id=778561ee5e474accb31816f357985f44, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:29.810653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995e13a-363a-4792-b11d-79cbc3c0d91b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:29.845215 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1201/2426] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:29 2022] PUT /identity/v3/projects/361f907a187547aabda6cb54eeeb3684/users/778561ee5e474accb31816f357985f44/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:29.850334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1836f-9ffc-4859-97b5-14913077cdea None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:29.852118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1836f-9ffc-4859-97b5-14913077cdea None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:29.853199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1836f-9ffc-4859-97b5-14913077cdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:29.867524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0584a39d42c94900a41e4532682489ce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:29.884649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3ed1836f-9ffc-4859-97b5-14913077cdea None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:29.886288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3ed1836f-9ffc-4859-97b5-14913077cdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:29.901275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0584a39d42c94900a41e4532682489ce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:29.915128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0584a39d42c94900a41e4532682489ce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:29.915872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0584a39d42c94900a41e4532682489ce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:29.916998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0584a39d42c94900a41e4532682489ce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:29.939238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:29.946213 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1226/2427] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:29 2022] DELETE /identity/v3/users/0584a39d42c94900a41e4532682489ce => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:29.948758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:29.952307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:30.000436 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1202/2428] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:30.002462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dfd80-22eb-4804-a499-3ecbc5e04c01 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:30.002967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dfd80-22eb-4804-a499-3ecbc5e04c01 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:30.003473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dfd80-22eb-4804-a499-3ecbc5e04c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:30.028544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-140dfd80-22eb-4804-a499-3ecbc5e04c01 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:30.032412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-140dfd80-22eb-4804-a499-3ecbc5e04c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:30.082172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ceacd-1f93-4380-a91b-83499bddf0b4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:30.090399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244df7a-271d-4cce-b430-5bc8a9abf806 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:30.091450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244df7a-271d-4cce-b430-5bc8a9abf806 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:30.091753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244df7a-271d-4cce-b430-5bc8a9abf806 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:30.092032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244df7a-271d-4cce-b430-5bc8a9abf806 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:30.094114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b244df7a-271d-4cce-b430-5bc8a9abf806 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:30.157910 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1203/2429] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:30.157910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dfd80-22eb-4804-a499-3ecbc5e04c01 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:30.164169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-140dfd80-22eb-4804-a499-3ecbc5e04c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:30.173459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244df7a-271d-4cce-b430-5bc8a9abf806 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:30.183205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244df7a-271d-4cce-b430-5bc8a9abf806 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:30.186706 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1227/2430] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:29 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:30.188207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244df7a-271d-4cce-b430-5bc8a9abf806 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:30.189168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b244df7a-271d-4cce-b430-5bc8a9abf806 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:30.262684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:30.276065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244df7a-271d-4cce-b430-5bc8a9abf806 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:30.283636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee6d60-dff5-4ca9-9c1a-c3a39a57d98b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:30.284294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee6d60-dff5-4ca9-9c1a-c3a39a57d98b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:30.284846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee6d60-dff5-4ca9-9c1a-c3a39a57d98b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:30.285170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee6d60-dff5-4ca9-9c1a-c3a39a57d98b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:30.299680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee6d60-dff5-4ca9-9c1a-c3a39a57d98b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:30.301351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee6d60-dff5-4ca9-9c1a-c3a39a57d98b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:30.313582 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1228/2431] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:30.314266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee6d60-dff5-4ca9-9c1a-c3a39a57d98b service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:30.327949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee6d60-dff5-4ca9-9c1a-c3a39a57d98b service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:30.335562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b078019-02bc-4549-a56f-8781be486516 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:30.336236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b078019-02bc-4549-a56f-8781be486516 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:30.336440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b078019-02bc-4549-a56f-8781be486516 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:30.337236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b078019-02bc-4549-a56f-8781be486516 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:30.339576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0b078019-02bc-4549-a56f-8781be486516 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:30.367901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dfd80-22eb-4804-a499-3ecbc5e04c01 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:30.378331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:30.383162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceddf74-85ba-4fdf-a83e-7d4084c42ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:30.384116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceddf74-85ba-4fdf-a83e-7d4084c42ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:30.384417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceddf74-85ba-4fdf-a83e-7d4084c42ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:30.384808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceddf74-85ba-4fdf-a83e-7d4084c42ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:30.410069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceddf74-85ba-4fdf-a83e-7d4084c42ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:30.411573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceddf74-85ba-4fdf-a83e-7d4084c42ca8 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:30.427561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b078019-02bc-4549-a56f-8781be486516 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:30.430238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b078019-02bc-4549-a56f-8781be486516 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:30.442802 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1204/2432] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:30.445769 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1229/2433] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:30.894544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceddf74-85ba-4fdf-a83e-7d4084c42ca8 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:30.918686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceddf74-85ba-4fdf-a83e-7d4084c42ca8 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:30.929800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430164c-7243-426c-9ff0-1c88903f680f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:30.930793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430164c-7243-426c-9ff0-1c88903f680f service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:30.934749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430164c-7243-426c-9ff0-1c88903f680f service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:30.936433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430164c-7243-426c-9ff0-1c88903f680f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:30.946485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8430164c-7243-426c-9ff0-1c88903f680f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:31.197010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430164c-7243-426c-9ff0-1c88903f680f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:31.199006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430164c-7243-426c-9ff0-1c88903f680f service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:31.210573 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1205/2434] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:31.344502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b078019-02bc-4549-a56f-8781be486516 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:31.346238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0b078019-02bc-4549-a56f-8781be486516 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:31.369735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b078019-02bc-4549-a56f-8781be486516 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:31.385043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:31.385831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:31.386121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:31.386399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] PATH_INFO: `/v3/projects/4113aadc6ea344ea863152ddbf561958` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:31.388193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4113aadc6ea344ea863152ddbf561958)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:31.390254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:31.504357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4113aadc6ea344ea863152ddbf561958', 'request_id': 'req-81b1695b-f1ff-428e-8f01-44117f075547'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:31.529066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4113aadc6ea344ea863152ddbf561958', 'request_id': 'req-81b1695b-f1ff-428e-8f01-44117f075547'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:31.545387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4113aadc6ea344ea863152ddbf561958', 'request_id': 'req-81b1695b-f1ff-428e-8f01-44117f075547'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:31.590697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] The token cache is being invalidate because project 4113aadc6ea344ea863152ddbf561958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:31.591092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:31.591992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:31.599432 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1230/2435] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:31 2022] DELETE /identity/v3/projects/4113aadc6ea344ea863152ddbf561958 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:31.599896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430164c-7243-426c-9ff0-1c88903f680f service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:31.619991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430164c-7243-426c-9ff0-1c88903f680f service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:31.775040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd7c63-ed94-4d0d-bfb1-ec22dc9815c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:31.778688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd7c63-ed94-4d0d-bfb1-ec22dc9815c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:31.779465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd7c63-ed94-4d0d-bfb1-ec22dc9815c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:31.779815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd7c63-ed94-4d0d-bfb1-ec22dc9815c8 admin admin] PATH_INFO: `/v3/users/1a36774d43454b31b42fd27c8aa22d1a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:31.785328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd7c63-ed94-4d0d-bfb1-ec22dc9815c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a36774d43454b31b42fd27c8aa22d1a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:31.787376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd7c63-ed94-4d0d-bfb1-ec22dc9815c8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:31.919848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2abd7c63-ed94-4d0d-bfb1-ec22dc9815c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a36774d43454b31b42fd27c8aa22d1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:31.939066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:31.949739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2abd7c63-ed94-4d0d-bfb1-ec22dc9815c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a36774d43454b31b42fd27c8aa22d1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:31.950569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2abd7c63-ed94-4d0d-bfb1-ec22dc9815c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a36774d43454b31b42fd27c8aa22d1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:31.964242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2abd7c63-ed94-4d0d-bfb1-ec22dc9815c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a36774d43454b31b42fd27c8aa22d1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:31.977758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2abd7c63-ed94-4d0d-bfb1-ec22dc9815c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a36774d43454b31b42fd27c8aa22d1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:31.998426 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1206/2436] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:31 2022] DELETE /identity/v3/users/1a36774d43454b31b42fd27c8aa22d1a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:50:33.863670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:33.865250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:33.889023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1695b-f1ff-428e-8f01-44117f075547 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:34.090295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887553de-b70a-4464-aaaa-269851e8bfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:34.091484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887553de-b70a-4464-aaaa-269851e8bfc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:34.091846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887553de-b70a-4464-aaaa-269851e8bfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:34.092210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887553de-b70a-4464-aaaa-269851e8bfc0 admin admin] PATH_INFO: `/v3/projects/7c4932c6af424797b765ff8215cf6266` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:34.093765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887553de-b70a-4464-aaaa-269851e8bfc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c4932c6af424797b765ff8215cf6266)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:34.096236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887553de-b70a-4464-aaaa-269851e8bfc0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:34.273344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-887553de-b70a-4464-aaaa-269851e8bfc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c4932c6af424797b765ff8215cf6266', 'request_id': 'req-887553de-b70a-4464-aaaa-269851e8bfc0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:34.277992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:34.296237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-887553de-b70a-4464-aaaa-269851e8bfc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c4932c6af424797b765ff8215cf6266', 'request_id': 'req-887553de-b70a-4464-aaaa-269851e8bfc0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:34.310737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-887553de-b70a-4464-aaaa-269851e8bfc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c4932c6af424797b765ff8215cf6266', 'request_id': 'req-887553de-b70a-4464-aaaa-269851e8bfc0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:34.326705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-887553de-b70a-4464-aaaa-269851e8bfc0 admin admin] The token cache is being invalidate because project 7c4932c6af424797b765ff8215cf6266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:34.327015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-887553de-b70a-4464-aaaa-269851e8bfc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:34.328123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-887553de-b70a-4464-aaaa-269851e8bfc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:34.335420 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1231/2437] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:33 2022] DELETE /identity/v3/projects/7c4932c6af424797b765ff8215cf6266 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:50:34.340809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d357c-dba3-453c-860b-774bbd5b4bb3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:34.341563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d357c-dba3-453c-860b-774bbd5b4bb3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:34.341673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d357c-dba3-453c-860b-774bbd5b4bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:34.400666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-134d357c-dba3-453c-860b-774bbd5b4bb3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:34.408208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-134d357c-dba3-453c-860b-774bbd5b4bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:34.638925 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1207/2438] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:34.646679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f07ac6-d6fb-46e8-be8a-9cc87fc6f1d2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:34.647214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f07ac6-d6fb-46e8-be8a-9cc87fc6f1d2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:34.647647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f07ac6-d6fb-46e8-be8a-9cc87fc6f1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:34.696324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-96f07ac6-d6fb-46e8-be8a-9cc87fc6f1d2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:34.696324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-96f07ac6-d6fb-46e8-be8a-9cc87fc6f1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:34.722401 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1232/2439] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:34.723682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d357c-dba3-453c-860b-774bbd5b4bb3 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:34.741134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d357c-dba3-453c-860b-774bbd5b4bb3 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:34.748222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7e95f-fb39-4d71-8bbc-5e1a423a2f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:34.749098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7e95f-fb39-4d71-8bbc-5e1a423a2f0a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:34.749558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7e95f-fb39-4d71-8bbc-5e1a423a2f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:34.749983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7e95f-fb39-4d71-8bbc-5e1a423a2f0a admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:34.750865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7e95f-fb39-4d71-8bbc-5e1a423a2f0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:34.751394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7e95f-fb39-4d71-8bbc-5e1a423a2f0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:34.753946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7e95f-fb39-4d71-8bbc-5e1a423a2f0a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:34.780839 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1208/2440] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:50:34.784787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f07ac6-d6fb-46e8-be8a-9cc87fc6f1d2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:34.805649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f07ac6-d6fb-46e8-be8a-9cc87fc6f1d2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:34.828198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f58b1-16f9-468e-8774-9e4ac3949d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:34.829180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f58b1-16f9-468e-8774-9e4ac3949d00 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:34.829479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f58b1-16f9-468e-8774-9e4ac3949d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:34.829758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f58b1-16f9-468e-8774-9e4ac3949d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:34.838610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f58b1-16f9-468e-8774-9e4ac3949d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:34.841870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f58b1-16f9-468e-8774-9e4ac3949d00 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:34.909279 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1233/2441] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:34 2022] POST /identity/v3/projects => generated 359 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:34.910859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7e95f-fb39-4d71-8bbc-5e1a423a2f0a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:34.929272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7e95f-fb39-4d71-8bbc-5e1a423a2f0a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:35.021450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207c4ea-2373-4091-a3ee-0e23b0d6ec46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:35.022302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207c4ea-2373-4091-a3ee-0e23b0d6ec46 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:35.022689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207c4ea-2373-4091-a3ee-0e23b0d6ec46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:35.023083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207c4ea-2373-4091-a3ee-0e23b0d6ec46 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:35.024208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207c4ea-2373-4091-a3ee-0e23b0d6ec46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:35.026719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207c4ea-2373-4091-a3ee-0e23b0d6ec46 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:35.077908 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1209/2442] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:34 2022] POST /identity/v3/users => generated 280 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:35.080020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f58b1-16f9-468e-8774-9e4ac3949d00 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:35.097709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f58b1-16f9-468e-8774-9e4ac3949d00 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:35.115029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b704a-9656-4c4d-af87-7f8dbb1bea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:35.115687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b704a-9656-4c4d-af87-7f8dbb1bea3a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:35.116250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b704a-9656-4c4d-af87-7f8dbb1bea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:35.116536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b704a-9656-4c4d-af87-7f8dbb1bea3a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:35.124273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b704a-9656-4c4d-af87-7f8dbb1bea3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:35.126482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b704a-9656-4c4d-af87-7f8dbb1bea3a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:35.152649 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1234/2443] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:35.154534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207c4ea-2373-4091-a3ee-0e23b0d6ec46 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:35.177632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207c4ea-2373-4091-a3ee-0e23b0d6ec46 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:35.187073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcb94f-a882-480b-8ff5-e53c26224c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:35.190176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcb94f-a882-480b-8ff5-e53c26224c3c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:35.191745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcb94f-a882-480b-8ff5-e53c26224c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:35.192706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcb94f-a882-480b-8ff5-e53c26224c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:35.196469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcb94f-a882-480b-8ff5-e53c26224c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:35.201486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcb94f-a882-480b-8ff5-e53c26224c3c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:35.235450 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1210/2444] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:35.239755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b704a-9656-4c4d-af87-7f8dbb1bea3a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:35.258858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b704a-9656-4c4d-af87-7f8dbb1bea3a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:35.299189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7460fb-fff2-425e-81ae-b3a88771a31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:35.301465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7460fb-fff2-425e-81ae-b3a88771a31a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:35.302759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7460fb-fff2-425e-81ae-b3a88771a31a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:35.303542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7460fb-fff2-425e-81ae-b3a88771a31a admin admin] PATH_INFO: `/v3/projects/d2f0f89b89e6497987fd6e85460d834f/users/bc4b71e6df63444dad8a521ea06cbe43/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:35.362513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7460fb-fff2-425e-81ae-b3a88771a31a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f0f89b89e6497987fd6e85460d834f, user_id=bc4b71e6df63444dad8a521ea06cbe43, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:35.375208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7460fb-fff2-425e-81ae-b3a88771a31a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:35.440704 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1235/2445] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:35 2022] PUT /identity/v3/projects/d2f0f89b89e6497987fd6e85460d834f/users/bc4b71e6df63444dad8a521ea06cbe43/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:35.443759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04727f9d-65d5-42a2-a2a7-d39b2e6ff838 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:35.444659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04727f9d-65d5-42a2-a2a7-d39b2e6ff838 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:35.445164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04727f9d-65d5-42a2-a2a7-d39b2e6ff838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:35.508685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7460fb-fff2-425e-81ae-b3a88771a31a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:35.509502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-04727f9d-65d5-42a2-a2a7-d39b2e6ff838 None None] MFA Rules not processed for user `bc4b71e6df63444dad8a521ea06cbe43`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:35.510743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4f7460fb-fff2-425e-81ae-b3a88771a31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:35.511301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-04727f9d-65d5-42a2-a2a7-d39b2e6ff838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:35.582000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:35.645209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7460fb-fff2-425e-81ae-b3a88771a31a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:35.654049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e08d33-6d48-486c-8640-b05650c03307 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:35.654680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e08d33-6d48-486c-8640-b05650c03307 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:35.654965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e08d33-6d48-486c-8640-b05650c03307 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:35.655239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e08d33-6d48-486c-8640-b05650c03307 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:35.657120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-54e08d33-6d48-486c-8640-b05650c03307 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:35.666290 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1211/2446] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:35.667313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04727f9d-65d5-42a2-a2a7-d39b2e6ff838 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:35.696581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e08d33-6d48-486c-8640-b05650c03307 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:35.700520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e08d33-6d48-486c-8640-b05650c03307 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:35.738772 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1236/2447] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:50:35.947692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04727f9d-65d5-42a2-a2a7-d39b2e6ff838 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:35.963618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8849ec4-d2d4-46f3-a0ad-7a60a44ed6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:35.966702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8849ec4-d2d4-46f3-a0ad-7a60a44ed6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:35.968004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8849ec4-d2d4-46f3-a0ad-7a60a44ed6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:35.968873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8849ec4-d2d4-46f3-a0ad-7a60a44ed6b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:35.972139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8849ec4-d2d4-46f3-a0ad-7a60a44ed6b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:35.978672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8849ec4-d2d4-46f3-a0ad-7a60a44ed6b6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:36.052932 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1212/2448] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:35 2022] POST /identity/v3/projects => generated 361 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:36.057609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e08d33-6d48-486c-8640-b05650c03307 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:36.077507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e08d33-6d48-486c-8640-b05650c03307 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:36.255956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22959c05-427c-4fa4-8ee6-0aa9568d55e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:36.256586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22959c05-427c-4fa4-8ee6-0aa9568d55e7 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:36.256873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22959c05-427c-4fa4-8ee6-0aa9568d55e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:36.257225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22959c05-427c-4fa4-8ee6-0aa9568d55e7 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:36.257870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22959c05-427c-4fa4-8ee6-0aa9568d55e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:36.261324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22959c05-427c-4fa4-8ee6-0aa9568d55e7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:36.346748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:36.379540 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1237/2449] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:36 2022] POST /identity/v3/users => generated 287 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:36.384048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8849ec4-d2d4-46f3-a0ad-7a60a44ed6b6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:36.403712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8849ec4-d2d4-46f3-a0ad-7a60a44ed6b6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:36.414331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10dcea-c0cb-45ee-95e7-99e5322b20b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:36.415015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10dcea-c0cb-45ee-95e7-99e5322b20b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:36.415355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10dcea-c0cb-45ee-95e7-99e5322b20b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:36.415636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10dcea-c0cb-45ee-95e7-99e5322b20b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:36.416394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10dcea-c0cb-45ee-95e7-99e5322b20b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:36.423795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10dcea-c0cb-45ee-95e7-99e5322b20b5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:36.448456 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1213/2450] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:36.453718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22959c05-427c-4fa4-8ee6-0aa9568d55e7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:36.467284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22959c05-427c-4fa4-8ee6-0aa9568d55e7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:36.472653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3a30a-2b93-424c-8862-73f8172a006c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:36.473361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3a30a-2b93-424c-8862-73f8172a006c admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:36.473697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3a30a-2b93-424c-8862-73f8172a006c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:36.475148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3a30a-2b93-424c-8862-73f8172a006c admin admin] PATH_INFO: `/v3/projects/0ce92c62510c48ee98b68b40ca35d085/users/49a0d58461da45a894de8d140ab97893/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:36.526814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3a30a-2b93-424c-8862-73f8172a006c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce92c62510c48ee98b68b40ca35d085, user_id=49a0d58461da45a894de8d140ab97893, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:36.534538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3a30a-2b93-424c-8862-73f8172a006c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:36.661590 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1238/2451] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:36 2022] PUT /identity/v3/projects/0ce92c62510c48ee98b68b40ca35d085/users/49a0d58461da45a894de8d140ab97893/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:36.665835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10dcea-c0cb-45ee-95e7-99e5322b20b5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:36.692588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10dcea-c0cb-45ee-95e7-99e5322b20b5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:36.950277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50696f68-f406-4cbc-ba95-b285cd77825a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:36.957116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50696f68-f406-4cbc-ba95-b285cd77825a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:36.958234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50696f68-f406-4cbc-ba95-b285cd77825a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:36.961004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50696f68-f406-4cbc-ba95-b285cd77825a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:36.970132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50696f68-f406-4cbc-ba95-b285cd77825a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:36.986013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50696f68-f406-4cbc-ba95-b285cd77825a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:37.030994 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1214/2452] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:37.036070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3a30a-2b93-424c-8862-73f8172a006c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:37.064818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237aabd-bba7-4a4a-9727-d2e090b4c60c None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:37.065371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3a30a-2b93-424c-8862-73f8172a006c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:37.065990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237aabd-bba7-4a4a-9727-d2e090b4c60c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:37.066444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237aabd-bba7-4a4a-9727-d2e090b4c60c None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:37.068352 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1215/2453] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:50:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:50:37.076885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c0c5c-44da-430e-b196-5fecc540387f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:37.077477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c0c5c-44da-430e-b196-5fecc540387f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:37.078619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c0c5c-44da-430e-b196-5fecc540387f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:37.091949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fc91b-6012-42a4-849a-ca0cf91ea076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:37.095929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fc91b-6012-42a4-849a-ca0cf91ea076 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:37.096444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fc91b-6012-42a4-849a-ca0cf91ea076 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:37.097035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fc91b-6012-42a4-849a-ca0cf91ea076 admin admin] PATH_INFO: `/v3/projects/0ce92c62510c48ee98b68b40ca35d085/users/49a0d58461da45a894de8d140ab97893/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:37.101982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fc91b-6012-42a4-849a-ca0cf91ea076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce92c62510c48ee98b68b40ca35d085, user_id=49a0d58461da45a894de8d140ab97893, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:37.113796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fc91b-6012-42a4-849a-ca0cf91ea076 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:37.147592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8c6c0c5c-44da-430e-b196-5fecc540387f None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:37.148870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8c6c0c5c-44da-430e-b196-5fecc540387f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:37.165872 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1239/2454] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:37 2022] PUT /identity/v3/projects/0ce92c62510c48ee98b68b40ca35d085/users/49a0d58461da45a894de8d140ab97893/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:37.177325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b42980-ae23-41b5-9bf3-4c3e671f9e1a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:37.177516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b42980-ae23-41b5-9bf3-4c3e671f9e1a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:37.177780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b42980-ae23-41b5-9bf3-4c3e671f9e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:37.237829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:37.253480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-67b42980-ae23-41b5-9bf3-4c3e671f9e1a None None] MFA Rules not processed for user `49a0d58461da45a894de8d140ab97893`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:37.254912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-67b42980-ae23-41b5-9bf3-4c3e671f9e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:37.315851 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1216/2455] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:50:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:37.379280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c0c5c-44da-430e-b196-5fecc540387f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:37.381941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8c6c0c5c-44da-430e-b196-5fecc540387f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:37.448229 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1240/2456] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:37 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:37.562142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c0c5c-44da-430e-b196-5fecc540387f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:37.567902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3832c-e81d-4113-8a77-8d13a744d4f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:37.568979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3832c-e81d-4113-8a77-8d13a744d4f5 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:37.569499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3832c-e81d-4113-8a77-8d13a744d4f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:37.569988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3832c-e81d-4113-8a77-8d13a744d4f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:37.585147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3832c-e81d-4113-8a77-8d13a744d4f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:37.587396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3832c-e81d-4113-8a77-8d13a744d4f5 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:37.616716 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1217/2457] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:37.694591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7bf72-85a1-470c-9a32-a7d71c70913f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:37.694940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7bf72-85a1-470c-9a32-a7d71c70913f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:37.695180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7bf72-85a1-470c-9a32-a7d71c70913f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:37.726510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-14a7bf72-85a1-470c-9a32-a7d71c70913f None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:37.729062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-14a7bf72-85a1-470c-9a32-a7d71c70913f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:37.738145 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1241/2458] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:50:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:37.772728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3832c-e81d-4113-8a77-8d13a744d4f5 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:37.786793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3832c-e81d-4113-8a77-8d13a744d4f5 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:37.842563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4f5e9-4380-44ef-872a-da8805e9f948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:37.846127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4f5e9-4380-44ef-872a-da8805e9f948 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:37.846534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4f5e9-4380-44ef-872a-da8805e9f948 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:37.850649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4f5e9-4380-44ef-872a-da8805e9f948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:37.892920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a4f5e9-4380-44ef-872a-da8805e9f948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:37.898086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a4f5e9-4380-44ef-872a-da8805e9f948 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:37.920596 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1218/2459] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:39.784194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7bf72-85a1-470c-9a32-a7d71c70913f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:39.787781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-14a7bf72-85a1-470c-9a32-a7d71c70913f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:39.810419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7bf72-85a1-470c-9a32-a7d71c70913f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:39.818189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c028b7-8d94-4741-a82d-459439254eca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:39.818917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c028b7-8d94-4741-a82d-459439254eca service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:39.819296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c028b7-8d94-4741-a82d-459439254eca service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:39.819648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c028b7-8d94-4741-a82d-459439254eca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:39.832993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c028b7-8d94-4741-a82d-459439254eca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:39.834883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c028b7-8d94-4741-a82d-459439254eca service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:39.847659 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1242/2460] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:41.559572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4f5e9-4380-44ef-872a-da8805e9f948 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:41.754086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4f5e9-4380-44ef-872a-da8805e9f948 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:41.791607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:41.794947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:41.796329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:41.796620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] PATH_INFO: `/v3/users/bc4b71e6df63444dad8a521ea06cbe43` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:41.804785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc4b71e6df63444dad8a521ea06cbe43)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:41.812937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:41.947135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc4b71e6df63444dad8a521ea06cbe43'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:41.986435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc4b71e6df63444dad8a521ea06cbe43'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:41.987906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc4b71e6df63444dad8a521ea06cbe43'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:41.989721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:42.011060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc4b71e6df63444dad8a521ea06cbe43'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:42.027443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc4b71e6df63444dad8a521ea06cbe43'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:42.052696 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1219/2461] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:41 2022] DELETE /identity/v3/users/bc4b71e6df63444dad8a521ea06cbe43 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:42.054310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488a0c2-89f7-4e40-ab74-82d3336bf4d5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:42.055236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488a0c2-89f7-4e40-ab74-82d3336bf4d5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:42.055719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488a0c2-89f7-4e40-ab74-82d3336bf4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:42.095888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4488a0c2-89f7-4e40-ab74-82d3336bf4d5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:42.097380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4488a0c2-89f7-4e40-ab74-82d3336bf4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:42.260768 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1243/2462] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:42.312293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:42.313801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:42.473142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5ecb6-3dd3-4b6a-9a72-25118b3db21a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:42.503320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bed15a-01d1-4ee6-a5cd-bb6f4ee3646e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:42.503964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bed15a-01d1-4ee6-a5cd-bb6f4ee3646e service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:42.504258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bed15a-01d1-4ee6-a5cd-bb6f4ee3646e service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:42.504540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bed15a-01d1-4ee6-a5cd-bb6f4ee3646e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:42.526857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bed15a-01d1-4ee6-a5cd-bb6f4ee3646e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:42.529124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bed15a-01d1-4ee6-a5cd-bb6f4ee3646e service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:42.540281 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1220/2463] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:43.903888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488a0c2-89f7-4e40-ab74-82d3336bf4d5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:43.905939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4488a0c2-89f7-4e40-ab74-82d3336bf4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:43.940635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488a0c2-89f7-4e40-ab74-82d3336bf4d5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:43.955332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee073551-8fdf-47aa-81da-f75248a1078a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:43.957124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee073551-8fdf-47aa-81da-f75248a1078a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:43.957682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee073551-8fdf-47aa-81da-f75248a1078a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:43.958183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee073551-8fdf-47aa-81da-f75248a1078a admin admin] PATH_INFO: `/v3/projects/d2f0f89b89e6497987fd6e85460d834f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:43.960234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee073551-8fdf-47aa-81da-f75248a1078a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f0f89b89e6497987fd6e85460d834f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:43.963338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee073551-8fdf-47aa-81da-f75248a1078a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:43.991812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:44.078027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee073551-8fdf-47aa-81da-f75248a1078a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f0f89b89e6497987fd6e85460d834f', 'request_id': 'req-ee073551-8fdf-47aa-81da-f75248a1078a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:44.130933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee073551-8fdf-47aa-81da-f75248a1078a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f0f89b89e6497987fd6e85460d834f', 'request_id': 'req-ee073551-8fdf-47aa-81da-f75248a1078a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:44.154073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee073551-8fdf-47aa-81da-f75248a1078a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f0f89b89e6497987fd6e85460d834f', 'request_id': 'req-ee073551-8fdf-47aa-81da-f75248a1078a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:44.200233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee073551-8fdf-47aa-81da-f75248a1078a admin admin] The token cache is being invalidate because project d2f0f89b89e6497987fd6e85460d834f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:44.200580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee073551-8fdf-47aa-81da-f75248a1078a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:44.203118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee073551-8fdf-47aa-81da-f75248a1078a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:44.210639 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1244/2464] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:43 2022] DELETE /identity/v3/projects/d2f0f89b89e6497987fd6e85460d834f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:44.212102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bed15a-01d1-4ee6-a5cd-bb6f4ee3646e service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:44.237426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bed15a-01d1-4ee6-a5cd-bb6f4ee3646e service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:44.434324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdcfa7-f44c-4b04-bb2b-730ccd10e6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:44.436169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ca8de-fdbf-4f27-818a-378188a8a6e8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:44.436454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdcfa7-f44c-4b04-bb2b-730ccd10e6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:44.436823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ca8de-fdbf-4f27-818a-378188a8a6e8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:44.437357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ca8de-fdbf-4f27-818a-378188a8a6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:44.437486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdcfa7-f44c-4b04-bb2b-730ccd10e6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:44.439170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdcfa7-f44c-4b04-bb2b-730ccd10e6a2 admin admin] PATH_INFO: `/v3/users/49a0d58461da45a894de8d140ab97893` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:44.444253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdcfa7-f44c-4b04-bb2b-730ccd10e6a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49a0d58461da45a894de8d140ab97893)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:44.447793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdcfa7-f44c-4b04-bb2b-730ccd10e6a2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:44.514070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ff6ca8de-fdbf-4f27-818a-378188a8a6e8 None None] MFA Rules not processed for user `ee0fdf107484448d862ac3216df0d9dc`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:44.515395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ff6ca8de-fdbf-4f27-818a-378188a8a6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:44.600299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31cdcfa7-f44c-4b04-bb2b-730ccd10e6a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49a0d58461da45a894de8d140ab97893'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:44.630110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31cdcfa7-f44c-4b04-bb2b-730ccd10e6a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49a0d58461da45a894de8d140ab97893'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:44.631638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31cdcfa7-f44c-4b04-bb2b-730ccd10e6a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49a0d58461da45a894de8d140ab97893'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:44.649402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31cdcfa7-f44c-4b04-bb2b-730ccd10e6a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49a0d58461da45a894de8d140ab97893'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:44.664927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31cdcfa7-f44c-4b04-bb2b-730ccd10e6a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49a0d58461da45a894de8d140ab97893'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:44.671455 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1245/2465] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:44 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:44.676840 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1221/2466] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:50:44 2022] DELETE /identity/v3/users/49a0d58461da45a894de8d140ab97893 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:46.679239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ca8de-fdbf-4f27-818a-378188a8a6e8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:46.680926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ff6ca8de-fdbf-4f27-818a-378188a8a6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:46.710166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ca8de-fdbf-4f27-818a-378188a8a6e8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:46.760446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f44f1b-1e8d-4d09-9dba-183254bfa589 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:46.761005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f44f1b-1e8d-4d09-9dba-183254bfa589 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:46.761164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f44f1b-1e8d-4d09-9dba-183254bfa589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:46.780628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:46.835674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-03f44f1b-1e8d-4d09-9dba-183254bfa589 None None] MFA Rules not processed for user `778561ee5e474accb31816f357985f44`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:46.839878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-03f44f1b-1e8d-4d09-9dba-183254bfa589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:46.877919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:46.878807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:46.879333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:46.879775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] PATH_INFO: `/v3/projects/0ce92c62510c48ee98b68b40ca35d085` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:46.886312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce92c62510c48ee98b68b40ca35d085)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:46.905380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:47.019996 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1222/2467] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:46 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:50:47.020416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f44f1b-1e8d-4d09-9dba-183254bfa589 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:47.022038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-03f44f1b-1e8d-4d09-9dba-183254bfa589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:47.056919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce92c62510c48ee98b68b40ca35d085', 'request_id': 'req-dade586e-7548-47e3-aa50-5711eaaa4fec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:47.061815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:47.075178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce92c62510c48ee98b68b40ca35d085', 'request_id': 'req-dade586e-7548-47e3-aa50-5711eaaa4fec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:47.091087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce92c62510c48ee98b68b40ca35d085', 'request_id': 'req-dade586e-7548-47e3-aa50-5711eaaa4fec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:47.125901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] The token cache is being invalidate because project 0ce92c62510c48ee98b68b40ca35d085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:50:47.126266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:47.129058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:50:47.135446 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1246/2468] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:50:46 2022] DELETE /identity/v3/projects/0ce92c62510c48ee98b68b40ca35d085 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:47.137865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:47.139942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:47.216549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f44f1b-1e8d-4d09-9dba-183254bfa589 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:47.313333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade586e-7548-47e3-aa50-5711eaaa4fec admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:47.319214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc60020-4301-4301-a63f-9b003b6bf6ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:47.319372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236aeacb-843f-4186-9bcd-0603a4cca654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:47.319470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc60020-4301-4301-a63f-9b003b6bf6ad service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:47.319549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236aeacb-843f-4186-9bcd-0603a4cca654 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:47.319659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc60020-4301-4301-a63f-9b003b6bf6ad service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:47.319738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236aeacb-843f-4186-9bcd-0603a4cca654 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:47.319816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc60020-4301-4301-a63f-9b003b6bf6ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:47.319895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236aeacb-843f-4186-9bcd-0603a4cca654 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:47.321241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236aeacb-843f-4186-9bcd-0603a4cca654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:47.321853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1dc60020-4301-4301-a63f-9b003b6bf6ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:47.323539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236aeacb-843f-4186-9bcd-0603a4cca654 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:47.370852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:47.374922 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1247/2469] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:47 2022] POST /identity/v3/projects => generated 395 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:47.376617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448af01-014a-41f2-ad1c-04d5db92f555 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:47.377014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448af01-014a-41f2-ad1c-04d5db92f555 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:47.377628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448af01-014a-41f2-ad1c-04d5db92f555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:47.427237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3448af01-014a-41f2-ad1c-04d5db92f555 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:47.432082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3448af01-014a-41f2-ad1c-04d5db92f555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:47.434121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc60020-4301-4301-a63f-9b003b6bf6ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:47.437849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc60020-4301-4301-a63f-9b003b6bf6ad service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:47.470350 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1223/2470] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:47.471443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc60020-4301-4301-a63f-9b003b6bf6ad service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:47.502563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc60020-4301-4301-a63f-9b003b6bf6ad service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:47.524503 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1248/2471] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:47.538393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad26d31-20c4-48cf-aeb0-c35850c3e116 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:47.539144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad26d31-20c4-48cf-aeb0-c35850c3e116 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:47.539450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad26d31-20c4-48cf-aeb0-c35850c3e116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:47.587058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5ad26d31-20c4-48cf-aeb0-c35850c3e116 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:47.588633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5ad26d31-20c4-48cf-aeb0-c35850c3e116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:47.612185 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1249/2472] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:47.616673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad26d31-20c4-48cf-aeb0-c35850c3e116 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:47.633548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282cff0-928b-4e90-8a55-b6ce29e014f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:47.634250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282cff0-928b-4e90-8a55-b6ce29e014f3 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:47.634443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad26d31-20c4-48cf-aeb0-c35850c3e116 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:47.634547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282cff0-928b-4e90-8a55-b6ce29e014f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:47.634805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282cff0-928b-4e90-8a55-b6ce29e014f3 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:47.635460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282cff0-928b-4e90-8a55-b6ce29e014f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:47.639980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282cff0-928b-4e90-8a55-b6ce29e014f3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:47.650724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2325441-b04b-4a99-aa08-dba159c04389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:47.651394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2325441-b04b-4a99-aa08-dba159c04389 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:47.651658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2325441-b04b-4a99-aa08-dba159c04389 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:47.652002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2325441-b04b-4a99-aa08-dba159c04389 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:47.653195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2325441-b04b-4a99-aa08-dba159c04389 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:50:47.653519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2325441-b04b-4a99-aa08-dba159c04389 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:47.655148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2325441-b04b-4a99-aa08-dba159c04389 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:47.680949 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1250/2473] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:50:47.682981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2325441-b04b-4a99-aa08-dba159c04389 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:47.703199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2325441-b04b-4a99-aa08-dba159c04389 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:47.711125 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1224/2474] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:47 2022] POST /identity/v3/users => generated 304 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:47.712508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282cff0-928b-4e90-8a55-b6ce29e014f3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:47.726967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282cff0-928b-4e90-8a55-b6ce29e014f3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:47.733564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0511e-f3d3-45ab-8bb6-3776e3a49c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:47.734406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0511e-f3d3-45ab-8bb6-3776e3a49c22 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:47.735665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0511e-f3d3-45ab-8bb6-3776e3a49c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:47.735956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0511e-f3d3-45ab-8bb6-3776e3a49c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:47.736667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0511e-f3d3-45ab-8bb6-3776e3a49c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:47.738663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0511e-f3d3-45ab-8bb6-3776e3a49c22 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:47.738815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95f98e-b102-41e6-804c-0895d963ca46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:47.739355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95f98e-b102-41e6-804c-0895d963ca46 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:47.739637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95f98e-b102-41e6-804c-0895d963ca46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:47.739919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95f98e-b102-41e6-804c-0895d963ca46 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:47.745213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95f98e-b102-41e6-804c-0895d963ca46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:47.747778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95f98e-b102-41e6-804c-0895d963ca46 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:47.755135 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1225/2475] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:47.759842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0511e-f3d3-45ab-8bb6-3776e3a49c22 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:47.787914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0511e-f3d3-45ab-8bb6-3776e3a49c22 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:47.793711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1026b-8288-497c-b784-12d9bdde2b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:47.794386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1026b-8288-497c-b784-12d9bdde2b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:47.794668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1026b-8288-497c-b784-12d9bdde2b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:47.797215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1026b-8288-497c-b784-12d9bdde2b79 admin admin] PATH_INFO: `/v3/projects/e826490d9b834105834b6e18fa788131/users/0e1dda23b62b4c369b10b00340c6dcbd/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:47.824630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a1026b-8288-497c-b784-12d9bdde2b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e826490d9b834105834b6e18fa788131, user_id=0e1dda23b62b4c369b10b00340c6dcbd, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:47.824997 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1251/2476] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:47 2022] POST /identity/v3/projects => generated 395 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:50:47.828465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a1026b-8288-497c-b784-12d9bdde2b79 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:47.829645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95f98e-b102-41e6-804c-0895d963ca46 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:47.847541 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1226/2477] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:47 2022] PUT /identity/v3/projects/e826490d9b834105834b6e18fa788131/users/0e1dda23b62b4c369b10b00340c6dcbd/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:47.851580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1026b-8288-497c-b784-12d9bdde2b79 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:47.870577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:47.948132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1026b-8288-497c-b784-12d9bdde2b79 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:47.950171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95f98e-b102-41e6-804c-0895d963ca46 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:47.963995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db21cb-9672-4c98-94c3-a85d287a4ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:47.964741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db21cb-9672-4c98-94c3-a85d287a4ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:47.967594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db21cb-9672-4c98-94c3-a85d287a4ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:47.968236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db21cb-9672-4c98-94c3-a85d287a4ea4 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:47.970334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db21cb-9672-4c98-94c3-a85d287a4ea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:47.971758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e372f0-7ce8-44d7-a1f6-860d7c802de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:47.972625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e372f0-7ce8-44d7-a1f6-860d7c802de9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:47.973721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db21cb-9672-4c98-94c3-a85d287a4ea4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:47.974230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e372f0-7ce8-44d7-a1f6-860d7c802de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:47.974707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e372f0-7ce8-44d7-a1f6-860d7c802de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:47.976478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e372f0-7ce8-44d7-a1f6-860d7c802de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:47.979637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e372f0-7ce8-44d7-a1f6-860d7c802de9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:48.005336 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1252/2478] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:47 2022] POST /identity/v3/users => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:50:48.005861 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1227/2479] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:50:48.009953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db21cb-9672-4c98-94c3-a85d287a4ea4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:48.010133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e372f0-7ce8-44d7-a1f6-860d7c802de9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:48.031823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e372f0-7ce8-44d7-a1f6-860d7c802de9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:48.041707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db21cb-9672-4c98-94c3-a85d287a4ea4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:48.057584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec28c9a-b7ba-4080-810a-dfa9732ca201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:48.059090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec28c9a-b7ba-4080-810a-dfa9732ca201 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:48.061265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec28c9a-b7ba-4080-810a-dfa9732ca201 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:48.062350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecd493-64b8-46d3-b2ba-04638d939e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:48.062710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec28c9a-b7ba-4080-810a-dfa9732ca201 admin admin] PATH_INFO: `/v3/projects/e826490d9b834105834b6e18fa788131/users/0e1dda23b62b4c369b10b00340c6dcbd/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:48.064059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecd493-64b8-46d3-b2ba-04638d939e46 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:48.064939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecd493-64b8-46d3-b2ba-04638d939e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:48.065299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecd493-64b8-46d3-b2ba-04638d939e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:48.070924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ecd493-64b8-46d3-b2ba-04638d939e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:48.072510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec28c9a-b7ba-4080-810a-dfa9732ca201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e826490d9b834105834b6e18fa788131, user_id=0e1dda23b62b4c369b10b00340c6dcbd, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:48.073536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ecd493-64b8-46d3-b2ba-04638d939e46 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:48.076882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec28c9a-b7ba-4080-810a-dfa9732ca201 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:48.099509 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1253/2480] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:48.101290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecd493-64b8-46d3-b2ba-04638d939e46 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:48.107125 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1228/2481] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:48 2022] PUT /identity/v3/projects/e826490d9b834105834b6e18fa788131/users/0e1dda23b62b4c369b10b00340c6dcbd/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:48.130855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecd493-64b8-46d3-b2ba-04638d939e46 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:48.222249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b60e8f-1074-485a-8de8-0e0c715d1b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:48.222930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b60e8f-1074-485a-8de8-0e0c715d1b43 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:48.223324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b60e8f-1074-485a-8de8-0e0c715d1b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:48.223750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b60e8f-1074-485a-8de8-0e0c715d1b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:48.224561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b60e8f-1074-485a-8de8-0e0c715d1b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:48.241005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b60e8f-1074-485a-8de8-0e0c715d1b43 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:48.262901 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1254/2482] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:48.263917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec28c9a-b7ba-4080-810a-dfa9732ca201 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:48.270173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:48.283341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec28c9a-b7ba-4080-810a-dfa9732ca201 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:48.298698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675f74e-ce73-401e-ae6f-de90c26afc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:48.299396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675f74e-ce73-401e-ae6f-de90c26afc3a admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:48.299725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675f74e-ce73-401e-ae6f-de90c26afc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:48.300072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675f74e-ce73-401e-ae6f-de90c26afc3a admin admin] PATH_INFO: `/v3/projects/f968aad8b96b4ba9ade8cafd689c1b7f/users/ad3cb96a474d4d04be4e16212a2bc8c9/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:48.337831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675f74e-ce73-401e-ae6f-de90c26afc3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f968aad8b96b4ba9ade8cafd689c1b7f, user_id=ad3cb96a474d4d04be4e16212a2bc8c9, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:48.345691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675f74e-ce73-401e-ae6f-de90c26afc3a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:48.377203 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1229/2483] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:48 2022] PUT /identity/v3/projects/f968aad8b96b4ba9ade8cafd689c1b7f/users/ad3cb96a474d4d04be4e16212a2bc8c9/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:48.387939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ac369-11c8-4139-9894-d0bbfcbe5eab None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:48.388296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ac369-11c8-4139-9894-d0bbfcbe5eab None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:48.388749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ac369-11c8-4139-9894-d0bbfcbe5eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:48.445161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6f9ac369-11c8-4139-9894-d0bbfcbe5eab None None] MFA Rules not processed for user `ad3cb96a474d4d04be4e16212a2bc8c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:48.446791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6f9ac369-11c8-4139-9894-d0bbfcbe5eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:48.558925 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1255/2484] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:48 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:48.559403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675f74e-ce73-401e-ae6f-de90c26afc3a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:48.728663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675f74e-ce73-401e-ae6f-de90c26afc3a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:48.743621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c1f74-dec5-441f-befa-fe0f8a1a7488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:48.744959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c1f74-dec5-441f-befa-fe0f8a1a7488 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:48.745973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c1f74-dec5-441f-befa-fe0f8a1a7488 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:48.746758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c1f74-dec5-441f-befa-fe0f8a1a7488 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:48.748756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c1f74-dec5-441f-befa-fe0f8a1a7488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:48.753859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c1f74-dec5-441f-befa-fe0f8a1a7488 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:48.817633 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1230/2485] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:50:48 2022] POST /identity/v3/projects => generated 393 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:50:48.821564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ac369-11c8-4139-9894-d0bbfcbe5eab None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:48.834089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ac369-11c8-4139-9894-d0bbfcbe5eab None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:48.876761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:48.924525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fa9df-e31b-4542-bbbf-4b112de0a70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:48.925355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fa9df-e31b-4542-bbbf-4b112de0a70e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:48.925827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fa9df-e31b-4542-bbbf-4b112de0a70e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:48.926196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fa9df-e31b-4542-bbbf-4b112de0a70e admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:48.927274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fa9df-e31b-4542-bbbf-4b112de0a70e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:48.930318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fa9df-e31b-4542-bbbf-4b112de0a70e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:48.985610 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1256/2486] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:50:48 2022] POST /identity/v3/users => generated 303 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:50:48.990031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c1f74-dec5-441f-befa-fe0f8a1a7488 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:49.010122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c1f74-dec5-441f-befa-fe0f8a1a7488 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:49.028790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937287f-699f-44d5-ba71-e6673ae8d7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:49.032303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937287f-699f-44d5-ba71-e6673ae8d7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:49.035471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937287f-699f-44d5-ba71-e6673ae8d7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:49.037401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937287f-699f-44d5-ba71-e6673ae8d7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:49.038619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a937287f-699f-44d5-ba71-e6673ae8d7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:49.048165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a937287f-699f-44d5-ba71-e6673ae8d7e2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:49.078247 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1231/2487] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:49.082646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fa9df-e31b-4542-bbbf-4b112de0a70e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:49.099771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fa9df-e31b-4542-bbbf-4b112de0a70e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:49.117780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfbcd5-2c7c-4414-b1da-9fb7420965be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:49.118690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfbcd5-2c7c-4414-b1da-9fb7420965be admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:49.118979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfbcd5-2c7c-4414-b1da-9fb7420965be admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:49.119555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfbcd5-2c7c-4414-b1da-9fb7420965be admin admin] PATH_INFO: `/v3/projects/1c5b42b9fbdf4845bbe196fad7a2873b/users/fa4cc607989842a5ac6041883f9645de/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:49.157895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfbcd5-2c7c-4414-b1da-9fb7420965be admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5b42b9fbdf4845bbe196fad7a2873b, user_id=fa4cc607989842a5ac6041883f9645de, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:49.162027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfbcd5-2c7c-4414-b1da-9fb7420965be admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:49.191063 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1257/2488] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:49 2022] PUT /identity/v3/projects/1c5b42b9fbdf4845bbe196fad7a2873b/users/fa4cc607989842a5ac6041883f9645de/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:49.195628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937287f-699f-44d5-ba71-e6673ae8d7e2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:49.213695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937287f-699f-44d5-ba71-e6673ae8d7e2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:49.328851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b74d0-29d0-4560-8a22-957cffb2d518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:49.329768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b74d0-29d0-4560-8a22-957cffb2d518 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:49.330814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b74d0-29d0-4560-8a22-957cffb2d518 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:49.331109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b74d0-29d0-4560-8a22-957cffb2d518 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:49.331974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b74d0-29d0-4560-8a22-957cffb2d518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:49.334542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b74d0-29d0-4560-8a22-957cffb2d518 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:49.352964 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1232/2489] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:50:49.355147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfbcd5-2c7c-4414-b1da-9fb7420965be admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:49.371648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfbcd5-2c7c-4414-b1da-9fb7420965be admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:49.377391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7d46d-d9fd-4cbb-b35d-db5547871892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:49.378459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d46d-d9fd-4cbb-b35d-db5547871892 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:49.379485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d46d-d9fd-4cbb-b35d-db5547871892 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:49.380158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d46d-d9fd-4cbb-b35d-db5547871892 admin admin] PATH_INFO: `/v3/projects/1c5b42b9fbdf4845bbe196fad7a2873b/users/fa4cc607989842a5ac6041883f9645de/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:49.389294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7d46d-d9fd-4cbb-b35d-db5547871892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5b42b9fbdf4845bbe196fad7a2873b, user_id=fa4cc607989842a5ac6041883f9645de, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:49.392721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7d46d-d9fd-4cbb-b35d-db5547871892 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:49.420091 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1258/2490] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:50:49 2022] PUT /identity/v3/projects/1c5b42b9fbdf4845bbe196fad7a2873b/users/fa4cc607989842a5ac6041883f9645de/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:50:49.432925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927761ac-c718-4e69-a091-2df352095954 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:49.433841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927761ac-c718-4e69-a091-2df352095954 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:49.434148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927761ac-c718-4e69-a091-2df352095954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:49.481900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-927761ac-c718-4e69-a091-2df352095954 None None] MFA Rules not processed for user `fa4cc607989842a5ac6041883f9645de`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:49.483200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-927761ac-c718-4e69-a091-2df352095954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:49.541046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:49.588342 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1233/2491] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:49 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:49.599933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed241eb-7ce0-4871-a836-9dfecad96325 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:49.600371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed241eb-7ce0-4871-a836-9dfecad96325 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:49.600675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed241eb-7ce0-4871-a836-9dfecad96325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:49.650891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5ed241eb-7ce0-4871-a836-9dfecad96325 None None] MFA Rules not processed for user `ad3cb96a474d4d04be4e16212a2bc8c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:49.653569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5ed241eb-7ce0-4871-a836-9dfecad96325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:49.726689 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1259/2492] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:50:49 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:49.866600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927761ac-c718-4e69-a091-2df352095954 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:49.868592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-927761ac-c718-4e69-a091-2df352095954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:50.007395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02ac33-f12b-4c30-9a67-1857073d6aca None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:50.007927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02ac33-f12b-4c30-9a67-1857073d6aca None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:50.007927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02ac33-f12b-4c30-9a67-1857073d6aca None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:50.010031 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1260/2493] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:50:50.018973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fc59a-c223-4f9a-8855-996cf12636ca None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:50.019243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fc59a-c223-4f9a-8855-996cf12636ca None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:50.019529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fc59a-c223-4f9a-8855-996cf12636ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:50.058856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927761ac-c718-4e69-a091-2df352095954 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:50.069984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6a8fc59a-c223-4f9a-8855-996cf12636ca None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:50.070171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32b986-d3a3-410d-8123-d5651fc906e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:50.071585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32b986-d3a3-410d-8123-d5651fc906e6 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:50.071889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32b986-d3a3-410d-8123-d5651fc906e6 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:50.072181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32b986-d3a3-410d-8123-d5651fc906e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:50.073875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6a8fc59a-c223-4f9a-8855-996cf12636ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:50.090567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32b986-d3a3-410d-8123-d5651fc906e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:50.092445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32b986-d3a3-410d-8123-d5651fc906e6 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:50.097540 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1261/2494] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:50:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:50:50.133763 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1234/2495] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:49 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:50.144429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fc59a-c223-4f9a-8855-996cf12636ca None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:50.149462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6a8fc59a-c223-4f9a-8855-996cf12636ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:50.358027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fc59a-c223-4f9a-8855-996cf12636ca None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:50.363806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7c4db-5c4e-43e0-b598-e627a4467c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:50.364997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7c4db-5c4e-43e0-b598-e627a4467c3d service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:50.365440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7c4db-5c4e-43e0-b598-e627a4467c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:50.365728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7c4db-5c4e-43e0-b598-e627a4467c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:50.411259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7c4db-5c4e-43e0-b598-e627a4467c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:50.413340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7c4db-5c4e-43e0-b598-e627a4467c3d service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:50.455335 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1262/2496] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:51.699656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32b986-d3a3-410d-8123-d5651fc906e6 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:51.729361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32b986-d3a3-410d-8123-d5651fc906e6 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:51.744258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622c05d-cf0f-4570-8588-2ee03de0acbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:51.745036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622c05d-cf0f-4570-8588-2ee03de0acbb service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:51.745453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622c05d-cf0f-4570-8588-2ee03de0acbb service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:51.746593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622c05d-cf0f-4570-8588-2ee03de0acbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:51.749958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0622c05d-cf0f-4570-8588-2ee03de0acbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:52.021221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622c05d-cf0f-4570-8588-2ee03de0acbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:52.022929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622c05d-cf0f-4570-8588-2ee03de0acbb service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:52.060723 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1235/2497] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:50:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:50:57.582398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe2cfb-5a0a-4aa6-b8c6-87dab1201747 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:57.582877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe2cfb-5a0a-4aa6-b8c6-87dab1201747 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:57.583035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe2cfb-5a0a-4aa6-b8c6-87dab1201747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:57.629301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7ebe2cfb-5a0a-4aa6-b8c6-87dab1201747 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:50:57.631625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7ebe2cfb-5a0a-4aa6-b8c6-87dab1201747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:57.684238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:57.763112 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1263/2498] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:50:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:50:57.781812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622c05d-cf0f-4570-8588-2ee03de0acbb service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:50:57.783388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0622c05d-cf0f-4570-8588-2ee03de0acbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:50:57.880412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:50:57.908135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622c05d-cf0f-4570-8588-2ee03de0acbb service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:50:57.915181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615506cc-c744-414b-a3e3-ddcb91ea2c74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:50:57.915967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615506cc-c744-414b-a3e3-ddcb91ea2c74 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:50:57.916360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615506cc-c744-414b-a3e3-ddcb91ea2c74 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:50:57.916934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615506cc-c744-414b-a3e3-ddcb91ea2c74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:50:57.956705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615506cc-c744-414b-a3e3-ddcb91ea2c74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:50:57.958823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615506cc-c744-414b-a3e3-ddcb91ea2c74 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:50:57.996600 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1236/2499] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:50:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:51:00.182192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe2cfb-5a0a-4aa6-b8c6-87dab1201747 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:00.184887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7ebe2cfb-5a0a-4aa6-b8c6-87dab1201747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:00.310407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615506cc-c744-414b-a3e3-ddcb91ea2c74 service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:00.314337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-615506cc-c744-414b-a3e3-ddcb91ea2c74 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:00.392490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe2cfb-5a0a-4aa6-b8c6-87dab1201747 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:00.403358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328e018-ee2f-44b8-af90-b4ad137b6a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:00.404207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328e018-ee2f-44b8-af90-b4ad137b6a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:00.410336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328e018-ee2f-44b8-af90-b4ad137b6a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:00.411795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328e018-ee2f-44b8-af90-b4ad137b6a73 admin admin] PATH_INFO: `/v3/users/d7956df7a8954bbc9571a8d7c29e3f34` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:00.414007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328e018-ee2f-44b8-af90-b4ad137b6a73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7956df7a8954bbc9571a8d7c29e3f34)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:00.419521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328e018-ee2f-44b8-af90-b4ad137b6a73 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:00.510597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615506cc-c744-414b-a3e3-ddcb91ea2c74 service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:00.523553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:00.525830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:00.525830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:00.528209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] PATH_INFO: `/v3/users/78b60bdc00024f6f9cac34b07b8940ac` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:00.531205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b60bdc00024f6f9cac34b07b8940ac)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:00.534259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:00.541690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9328e018-ee2f-44b8-af90-b4ad137b6a73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7956df7a8954bbc9571a8d7c29e3f34'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:00.574150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9328e018-ee2f-44b8-af90-b4ad137b6a73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7956df7a8954bbc9571a8d7c29e3f34'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:00.596169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9328e018-ee2f-44b8-af90-b4ad137b6a73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7956df7a8954bbc9571a8d7c29e3f34'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:00.597242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9328e018-ee2f-44b8-af90-b4ad137b6a73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7956df7a8954bbc9571a8d7c29e3f34'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:00.598094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9328e018-ee2f-44b8-af90-b4ad137b6a73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7956df7a8954bbc9571a8d7c29e3f34'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:00.656219 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1264/2500] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:51:00 2022] DELETE /identity/v3/users/d7956df7a8954bbc9571a8d7c29e3f34 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:00.657668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b60bdc00024f6f9cac34b07b8940ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:00.657963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5e811-a965-44f4-b20f-75723ff4a3f2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:00.658383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5e811-a965-44f4-b20f-75723ff4a3f2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:00.658516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5e811-a965-44f4-b20f-75723ff4a3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:00.704601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f2f5e811-a965-44f4-b20f-75723ff4a3f2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:00.705288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b60bdc00024f6f9cac34b07b8940ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:00.705967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f2f5e811-a965-44f4-b20f-75723ff4a3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:00.707811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b60bdc00024f6f9cac34b07b8940ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:00.759283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b60bdc00024f6f9cac34b07b8940ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:00.788555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b60bdc00024f6f9cac34b07b8940ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:00.815850 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1237/2501] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:51:00 2022] DELETE /identity/v3/users/78b60bdc00024f6f9cac34b07b8940ac => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:00.845310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:00.895986 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1265/2502] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:00.989261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:00.990953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:01.196388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a7940-730f-44b0-96cf-75d85ea60234 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:01.203305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62980197-a182-4497-8a02-e1b00ee2c9b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:01.204112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62980197-a182-4497-8a02-e1b00ee2c9b9 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:01.204496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62980197-a182-4497-8a02-e1b00ee2c9b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:01.204890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62980197-a182-4497-8a02-e1b00ee2c9b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:01.224716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:01.416201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62980197-a182-4497-8a02-e1b00ee2c9b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:01.418649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62980197-a182-4497-8a02-e1b00ee2c9b9 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:01.438752 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1238/2503] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:01.752323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5e811-a965-44f4-b20f-75723ff4a3f2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:01.753897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f2f5e811-a965-44f4-b20f-75723ff4a3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:01.773048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5e811-a965-44f4-b20f-75723ff4a3f2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:01.798906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727b59a-e1ee-4ca2-9e0c-22e19da54482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:01.799718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727b59a-e1ee-4ca2-9e0c-22e19da54482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:01.800099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727b59a-e1ee-4ca2-9e0c-22e19da54482 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:01.800798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727b59a-e1ee-4ca2-9e0c-22e19da54482 admin admin] PATH_INFO: `/v3/projects/8d0ccd04fad8493fbef0a8afc12e0a38` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:01.803879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727b59a-e1ee-4ca2-9e0c-22e19da54482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d0ccd04fad8493fbef0a8afc12e0a38)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:01.808719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727b59a-e1ee-4ca2-9e0c-22e19da54482 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:01.923981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9727b59a-e1ee-4ca2-9e0c-22e19da54482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d0ccd04fad8493fbef0a8afc12e0a38', 'request_id': 'req-9727b59a-e1ee-4ca2-9e0c-22e19da54482'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:01.934085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9727b59a-e1ee-4ca2-9e0c-22e19da54482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d0ccd04fad8493fbef0a8afc12e0a38', 'request_id': 'req-9727b59a-e1ee-4ca2-9e0c-22e19da54482'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:01.979955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9727b59a-e1ee-4ca2-9e0c-22e19da54482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d0ccd04fad8493fbef0a8afc12e0a38', 'request_id': 'req-9727b59a-e1ee-4ca2-9e0c-22e19da54482'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:02.018118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9727b59a-e1ee-4ca2-9e0c-22e19da54482 admin admin] The token cache is being invalidate because project 8d0ccd04fad8493fbef0a8afc12e0a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:51:02.020048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9727b59a-e1ee-4ca2-9e0c-22e19da54482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:02.021465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9727b59a-e1ee-4ca2-9e0c-22e19da54482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:02.044133 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1266/2504] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:51:01 2022] DELETE /identity/v3/projects/8d0ccd04fad8493fbef0a8afc12e0a38 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:02.045553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1392e9-a327-49ef-90d3-a01fe78ef6b4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:02.046050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1392e9-a327-49ef-90d3-a01fe78ef6b4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:02.046372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1392e9-a327-49ef-90d3-a01fe78ef6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:02.081135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ae1392e9-a327-49ef-90d3-a01fe78ef6b4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:02.083403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ae1392e9-a327-49ef-90d3-a01fe78ef6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:02.222328 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1239/2505] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:02.225728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d2828-b43f-4af5-a3be-dcd1bfd0b692 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:02.226443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d2828-b43f-4af5-a3be-dcd1bfd0b692 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:02.226959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d2828-b43f-4af5-a3be-dcd1bfd0b692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:02.254429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6a7d2828-b43f-4af5-a3be-dcd1bfd0b692 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:02.256623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6a7d2828-b43f-4af5-a3be-dcd1bfd0b692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:02.274230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1392e9-a327-49ef-90d3-a01fe78ef6b4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:02.275416 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1267/2506] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:02.286205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1392e9-a327-49ef-90d3-a01fe78ef6b4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:02.297716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfda00-af37-4475-8d7d-694759afdddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:02.298354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfda00-af37-4475-8d7d-694759afdddb admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:02.298639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfda00-af37-4475-8d7d-694759afdddb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:02.298918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfda00-af37-4475-8d7d-694759afdddb admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:02.299580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfda00-af37-4475-8d7d-694759afdddb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:51:02.299935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfda00-af37-4475-8d7d-694759afdddb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:02.305572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfda00-af37-4475-8d7d-694759afdddb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:02.336806 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1240/2507] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:51:02.340769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d2828-b43f-4af5-a3be-dcd1bfd0b692 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:02.362196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d2828-b43f-4af5-a3be-dcd1bfd0b692 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:02.368877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534200b-629e-4abb-876d-f792680cf482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:02.369661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534200b-629e-4abb-876d-f792680cf482 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:02.369875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534200b-629e-4abb-876d-f792680cf482 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:02.370141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534200b-629e-4abb-876d-f792680cf482 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:02.377036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534200b-629e-4abb-876d-f792680cf482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:02.377036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534200b-629e-4abb-876d-f792680cf482 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:02.476943 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1268/2508] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:51:02 2022] POST /identity/v3/projects => generated 369 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:51:02.478388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfda00-af37-4475-8d7d-694759afdddb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:02.493371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfda00-af37-4475-8d7d-694759afdddb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:02.617885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ca5ba-02ad-4284-a0cf-f69db873a401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:02.619402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ca5ba-02ad-4284-a0cf-f69db873a401 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:02.621142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ca5ba-02ad-4284-a0cf-f69db873a401 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:02.621686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ca5ba-02ad-4284-a0cf-f69db873a401 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:02.622933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ca5ba-02ad-4284-a0cf-f69db873a401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:02.625247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ca5ba-02ad-4284-a0cf-f69db873a401 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:02.690623 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1241/2509] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:51:02 2022] POST /identity/v3/users => generated 285 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:51:02.691216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534200b-629e-4abb-876d-f792680cf482 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:02.704061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534200b-629e-4abb-876d-f792680cf482 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:02.710233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5f0b2-cbfe-4274-b3ff-0a5c5234492d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:02.711193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5f0b2-cbfe-4274-b3ff-0a5c5234492d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:02.711642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5f0b2-cbfe-4274-b3ff-0a5c5234492d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:02.712391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5f0b2-cbfe-4274-b3ff-0a5c5234492d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:02.713752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5f0b2-cbfe-4274-b3ff-0a5c5234492d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:02.718008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5f0b2-cbfe-4274-b3ff-0a5c5234492d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:02.757587 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1269/2510] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:51:02.759145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ca5ba-02ad-4284-a0cf-f69db873a401 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:02.776808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ca5ba-02ad-4284-a0cf-f69db873a401 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:02.785975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1582c-06fb-4937-adc9-98f27f67d4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:02.788671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1582c-06fb-4937-adc9-98f27f67d4bf admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:02.789894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1582c-06fb-4937-adc9-98f27f67d4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:02.790287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1582c-06fb-4937-adc9-98f27f67d4bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:02.791003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1582c-06fb-4937-adc9-98f27f67d4bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:02.792863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1582c-06fb-4937-adc9-98f27f67d4bf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:02.810299 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1242/2511] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:51:02.811658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5f0b2-cbfe-4274-b3ff-0a5c5234492d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:02.826914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5f0b2-cbfe-4274-b3ff-0a5c5234492d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:02.832906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b0d64-a2a2-4ea1-ae74-bdd6afd3d400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:02.833675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b0d64-a2a2-4ea1-ae74-bdd6afd3d400 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:02.834209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b0d64-a2a2-4ea1-ae74-bdd6afd3d400 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:02.834632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b0d64-a2a2-4ea1-ae74-bdd6afd3d400 admin admin] PATH_INFO: `/v3/projects/68efe54b0a8f4b909e61eaf52898a337/users/c05efa51ab97498eb2293f9a8124a78d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:02.859893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b0d64-a2a2-4ea1-ae74-bdd6afd3d400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68efe54b0a8f4b909e61eaf52898a337, user_id=c05efa51ab97498eb2293f9a8124a78d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:02.863132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b0d64-a2a2-4ea1-ae74-bdd6afd3d400 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:02.924278 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1270/2512] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:51:02 2022] PUT /identity/v3/projects/68efe54b0a8f4b909e61eaf52898a337/users/c05efa51ab97498eb2293f9a8124a78d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:02.927450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e723c-6186-4863-8ae9-d08a18a480ba None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:02.927753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e723c-6186-4863-8ae9-d08a18a480ba None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:02.928042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e723c-6186-4863-8ae9-d08a18a480ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:02.959461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b0d64-a2a2-4ea1-ae74-bdd6afd3d400 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:02.979425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-013b0d64-a2a2-4ea1-ae74-bdd6afd3d400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:03.091820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-120e723c-6186-4863-8ae9-d08a18a480ba None None] MFA Rules not processed for user `c05efa51ab97498eb2293f9a8124a78d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:03.094716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-120e723c-6186-4863-8ae9-d08a18a480ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:03.117319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:03.238693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b0d64-a2a2-4ea1-ae74-bdd6afd3d400 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:03.249788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:03.250444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:03.250731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:03.251008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] PATH_INFO: `/v3/projects/7ca286521ec549da9fde582fc148f45a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:03.253764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca286521ec549da9fde582fc148f45a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:03.255724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:03.275394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:03.297345 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1243/2513] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:03.366770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca286521ec549da9fde582fc148f45a', 'request_id': 'req-6a978724-7c90-4659-90b3-d5ffaaddb508'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:03.378355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca286521ec549da9fde582fc148f45a', 'request_id': 'req-6a978724-7c90-4659-90b3-d5ffaaddb508'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:03.390118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca286521ec549da9fde582fc148f45a', 'request_id': 'req-6a978724-7c90-4659-90b3-d5ffaaddb508'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:03.421351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] The token cache is being invalidate because project 7ca286521ec549da9fde582fc148f45a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:51:03.421708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:03.422486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:03.424721 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1271/2514] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:51:02 2022] DELETE /identity/v3/projects/7ca286521ec549da9fde582fc148f45a => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:03.429399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e723c-6186-4863-8ae9-d08a18a480ba None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:03.451506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e723c-6186-4863-8ae9-d08a18a480ba None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:03.456802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:03.458682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:03.634839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:03.635686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:03.636577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:03.637103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] PATH_INFO: `/v3/users/e12b04847ebf4d249d1718aef14680e1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:03.639430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e12b04847ebf4d249d1718aef14680e1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:03.641365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:03.703106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a978724-7c90-4659-90b3-d5ffaaddb508 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:03.714671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12726826-3ede-4db4-9a34-00907c698a71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:03.715852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12726826-3ede-4db4-9a34-00907c698a71 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:03.716150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12726826-3ede-4db4-9a34-00907c698a71 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:03.716445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12726826-3ede-4db4-9a34-00907c698a71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:03.726644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-12726826-3ede-4db4-9a34-00907c698a71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:03.789237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e12b04847ebf4d249d1718aef14680e1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:03.807393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e12b04847ebf4d249d1718aef14680e1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:03.808488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e12b04847ebf4d249d1718aef14680e1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:03.849428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12726826-3ede-4db4-9a34-00907c698a71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:03.851371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12726826-3ede-4db4-9a34-00907c698a71 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:03.856007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e12b04847ebf4d249d1718aef14680e1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:03.864994 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1272/2515] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:03.869642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e12b04847ebf4d249d1718aef14680e1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:03.883804 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1244/2516] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:51:03 2022] DELETE /identity/v3/users/e12b04847ebf4d249d1718aef14680e1 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:04.032117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21db9c9-cd22-460b-bbcf-4a93decc87be None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:04.032701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21db9c9-cd22-460b-bbcf-4a93decc87be None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:04.032998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21db9c9-cd22-460b-bbcf-4a93decc87be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:04.081910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d21db9c9-cd22-460b-bbcf-4a93decc87be None None] MFA Rules not processed for user `90d7e4c2e577467f949be3dc480cc7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:04.083440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d21db9c9-cd22-460b-bbcf-4a93decc87be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:04.176222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:04.233212 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1273/2517] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:05.149095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:05.150582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:05.158576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21db9c9-cd22-460b-bbcf-4a93decc87be None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:05.161013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d21db9c9-cd22-460b-bbcf-4a93decc87be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:05.202967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:05.252881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570df5b-69aa-4a1f-8538-5e1dd9f5e35c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:05.277528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc04681-a205-4d4a-9434-3afdfc3e1bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:05.282837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc04681-a205-4d4a-9434-3afdfc3e1bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:05.283138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc04681-a205-4d4a-9434-3afdfc3e1bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:05.283427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc04681-a205-4d4a-9434-3afdfc3e1bfa admin admin] PATH_INFO: `/v3/users/c05efa51ab97498eb2293f9a8124a78d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:05.285186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc04681-a205-4d4a-9434-3afdfc3e1bfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c05efa51ab97498eb2293f9a8124a78d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:05.287116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc04681-a205-4d4a-9434-3afdfc3e1bfa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:05.389372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21db9c9-cd22-460b-bbcf-4a93decc87be None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:05.399437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:05.400884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:05.401533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:05.402129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908 admin admin] PATH_INFO: `/v3/projects/0920989729ee4c37a055e13d1dda2d1f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:05.404564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0920989729ee4c37a055e13d1dda2d1f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:05.408367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ddc04681-a205-4d4a-9434-3afdfc3e1bfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c05efa51ab97498eb2293f9a8124a78d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:05.409376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:05.440191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ddc04681-a205-4d4a-9434-3afdfc3e1bfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c05efa51ab97498eb2293f9a8124a78d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:05.447071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ddc04681-a205-4d4a-9434-3afdfc3e1bfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c05efa51ab97498eb2293f9a8124a78d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:05.475180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ddc04681-a205-4d4a-9434-3afdfc3e1bfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c05efa51ab97498eb2293f9a8124a78d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:05.491819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ddc04681-a205-4d4a-9434-3afdfc3e1bfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c05efa51ab97498eb2293f9a8124a78d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:05.503861 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1245/2518] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:51:05 2022] DELETE /identity/v3/users/c05efa51ab97498eb2293f9a8124a78d => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:05.508298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35877de9-5d00-4f67-986c-17d3373a0c85 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:05.508593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35877de9-5d00-4f67-986c-17d3373a0c85 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:05.509031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35877de9-5d00-4f67-986c-17d3373a0c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:05.525728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0920989729ee4c37a055e13d1dda2d1f', 'request_id': 'req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:05.545436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0920989729ee4c37a055e13d1dda2d1f', 'request_id': 'req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:05.564313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0920989729ee4c37a055e13d1dda2d1f', 'request_id': 'req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:05.605203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-35877de9-5d00-4f67-986c-17d3373a0c85 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:05.607597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-35877de9-5d00-4f67-986c-17d3373a0c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:05.612710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908 admin admin] The token cache is being invalidate because project 0920989729ee4c37a055e13d1dda2d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:51:05.613122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:05.614849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1aa98963-ad45-4d75-a7b3-bb2f67e2d908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:05.622253 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1274/2519] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:51:05 2022] DELETE /identity/v3/projects/0920989729ee4c37a055e13d1dda2d1f => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:51:05.624025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ee9d8-1e61-427a-b21b-65be7b818ad6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:05.624314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ee9d8-1e61-427a-b21b-65be7b818ad6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:05.624743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ee9d8-1e61-427a-b21b-65be7b818ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:05.666091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-775ee9d8-1e61-427a-b21b-65be7b818ad6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:05.669287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-775ee9d8-1e61-427a-b21b-65be7b818ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:05.789809 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1246/2520] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:05.832619 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1275/2521] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:51:05.840614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31edf26-9a42-4c82-a7d4-bfa6f2ca4136 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:05.842094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ee9d8-1e61-427a-b21b-65be7b818ad6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:05.842680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31edf26-9a42-4c82-a7d4-bfa6f2ca4136 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:05.843128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31edf26-9a42-4c82-a7d4-bfa6f2ca4136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:05.847349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-775ee9d8-1e61-427a-b21b-65be7b818ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:05.879112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:05.896617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e31edf26-9a42-4c82-a7d4-bfa6f2ca4136 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:05.897903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e31edf26-9a42-4c82-a7d4-bfa6f2ca4136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:05.928117 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1247/2522] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:05.935584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31edf26-9a42-4c82-a7d4-bfa6f2ca4136 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:05.960159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31edf26-9a42-4c82-a7d4-bfa6f2ca4136 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:05.971825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923516a3-dd9f-4e39-a780-69cfe4edf964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:05.972071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923516a3-dd9f-4e39-a780-69cfe4edf964 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:05.972393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923516a3-dd9f-4e39-a780-69cfe4edf964 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:05.972773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923516a3-dd9f-4e39-a780-69cfe4edf964 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:05.973566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923516a3-dd9f-4e39-a780-69cfe4edf964 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:51:05.973995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923516a3-dd9f-4e39-a780-69cfe4edf964 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:05.975827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923516a3-dd9f-4e39-a780-69cfe4edf964 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:06.011342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ee9d8-1e61-427a-b21b-65be7b818ad6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:06.017044 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1248/2523] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:51:06.018107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923516a3-dd9f-4e39-a780-69cfe4edf964 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:06.024529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436e00d-ed1c-4859-a0ab-5682dbb040ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:06.025191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436e00d-ed1c-4859-a0ab-5682dbb040ab service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:06.025482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436e00d-ed1c-4859-a0ab-5682dbb040ab service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:06.025764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436e00d-ed1c-4859-a0ab-5682dbb040ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:06.037675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923516a3-dd9f-4e39-a780-69cfe4edf964 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:06.049838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436e00d-ed1c-4859-a0ab-5682dbb040ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:06.050732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43690621-2cb9-4dbf-8630-c259c04a878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:06.052918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436e00d-ed1c-4859-a0ab-5682dbb040ab service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:06.053362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43690621-2cb9-4dbf-8630-c259c04a878b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:06.053755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43690621-2cb9-4dbf-8630-c259c04a878b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:06.054155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43690621-2cb9-4dbf-8630-c259c04a878b admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:06.054903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43690621-2cb9-4dbf-8630-c259c04a878b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:06.061533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43690621-2cb9-4dbf-8630-c259c04a878b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:06.088787 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1276/2524] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:06.145192 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1249/2525] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:51:06 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:51:06.149164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436e00d-ed1c-4859-a0ab-5682dbb040ab service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:06.171649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436e00d-ed1c-4859-a0ab-5682dbb040ab service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:06.299386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcba04-16a3-4859-8b93-b957969892e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:06.300180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcba04-16a3-4859-8b93-b957969892e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:06.300724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcba04-16a3-4859-8b93-b957969892e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:06.301000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcba04-16a3-4859-8b93-b957969892e3 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:06.301843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcba04-16a3-4859-8b93-b957969892e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:06.304235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcba04-16a3-4859-8b93-b957969892e3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:06.359043 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1277/2526] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:51:06 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:51:06.360342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43690621-2cb9-4dbf-8630-c259c04a878b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:06.370416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:06.378099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43690621-2cb9-4dbf-8630-c259c04a878b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:06.390531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e2c7e-d582-4651-8816-0c8350ff73e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:06.392532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e2c7e-d582-4651-8816-0c8350ff73e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:06.394178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e2c7e-d582-4651-8816-0c8350ff73e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:06.394667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e2c7e-d582-4651-8816-0c8350ff73e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:06.395760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e2c7e-d582-4651-8816-0c8350ff73e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:06.398690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e2c7e-d582-4651-8816-0c8350ff73e7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:06.435257 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1250/2527] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:51:06.439746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcba04-16a3-4859-8b93-b957969892e3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:06.462250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcba04-16a3-4859-8b93-b957969892e3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:06.490544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83433ad-7864-49cf-9c7f-7e877d5358d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:06.490544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83433ad-7864-49cf-9c7f-7e877d5358d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:06.490544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83433ad-7864-49cf-9c7f-7e877d5358d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:06.490544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83433ad-7864-49cf-9c7f-7e877d5358d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:06.491267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83433ad-7864-49cf-9c7f-7e877d5358d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:06.494823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83433ad-7864-49cf-9c7f-7e877d5358d6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:06.534875 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1278/2528] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:51:06.539483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e2c7e-d582-4651-8816-0c8350ff73e7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:06.560357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e2c7e-d582-4651-8816-0c8350ff73e7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:06.570942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d280fa-185b-4d49-ae15-13d3a9c94b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:06.571739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d280fa-185b-4d49-ae15-13d3a9c94b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:06.572031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d280fa-185b-4d49-ae15-13d3a9c94b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:06.572309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d280fa-185b-4d49-ae15-13d3a9c94b8c admin admin] PATH_INFO: `/v3/projects/2e521d306afe4d8f8ac8eef5f554892c/users/257eed38c7c74fd08a15f65deef421cf/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:06.609833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d280fa-185b-4d49-ae15-13d3a9c94b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e521d306afe4d8f8ac8eef5f554892c, user_id=257eed38c7c74fd08a15f65deef421cf, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:06.612542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d280fa-185b-4d49-ae15-13d3a9c94b8c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:06.652585 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1251/2529] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:51:06 2022] PUT /identity/v3/projects/2e521d306afe4d8f8ac8eef5f554892c/users/257eed38c7c74fd08a15f65deef421cf/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:06.664128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a904-8d0c-440b-ac35-02d1ead22c29 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:06.664490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a904-8d0c-440b-ac35-02d1ead22c29 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:06.664772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a904-8d0c-440b-ac35-02d1ead22c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:06.726245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-76c4a904-8d0c-440b-ac35-02d1ead22c29 None None] MFA Rules not processed for user `257eed38c7c74fd08a15f65deef421cf`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:06.727052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-76c4a904-8d0c-440b-ac35-02d1ead22c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:06.861109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7eb4a-a772-4d22-906b-ea3edd11af9b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:06.861702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7eb4a-a772-4d22-906b-ea3edd11af9b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:06.862157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7eb4a-a772-4d22-906b-ea3edd11af9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:06.953150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3da7eb4a-a772-4d22-906b-ea3edd11af9b None None] MFA Rules not processed for user `0e1dda23b62b4c369b10b00340c6dcbd`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:06.954670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3da7eb4a-a772-4d22-906b-ea3edd11af9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:07.057784 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1279/2530] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:07.165565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a904-8d0c-440b-ac35-02d1ead22c29 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:07.169112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-76c4a904-8d0c-440b-ac35-02d1ead22c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:07.196367 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1252/2531] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:06 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:07.205811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7eb4a-a772-4d22-906b-ea3edd11af9b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:07.205811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3da7eb4a-a772-4d22-906b-ea3edd11af9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:07.242494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:07.369590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a904-8d0c-440b-ac35-02d1ead22c29 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:07.384743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36eb270-259b-4750-a159-d0b627580222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:07.386687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36eb270-259b-4750-a159-d0b627580222 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:07.387672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36eb270-259b-4750-a159-d0b627580222 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:07.387977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36eb270-259b-4750-a159-d0b627580222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:07.407797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36eb270-259b-4750-a159-d0b627580222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:07.408531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36eb270-259b-4750-a159-d0b627580222 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:07.440027 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1280/2532] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:51:07.443351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec439765-6809-44a5-855a-2d1054369aaa None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:07.444080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec439765-6809-44a5-855a-2d1054369aaa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:07.444361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec439765-6809-44a5-855a-2d1054369aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:07.495267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ec439765-6809-44a5-855a-2d1054369aaa None None] MFA Rules not processed for user `778561ee5e474accb31816f357985f44`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:07.498802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ec439765-6809-44a5-855a-2d1054369aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:07.532735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7eb4a-a772-4d22-906b-ea3edd11af9b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:07.550367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:07.551331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:07.551685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:07.551959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] PATH_INFO: `/v3/projects/68efe54b0a8f4b909e61eaf52898a337` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:07.555055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=68efe54b0a8f4b909e61eaf52898a337)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:07.559611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:07.642046 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1281/2533] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:07.668189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68efe54b0a8f4b909e61eaf52898a337', 'request_id': 'req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:07.684787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68efe54b0a8f4b909e61eaf52898a337', 'request_id': 'req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:07.686462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:07.693235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68efe54b0a8f4b909e61eaf52898a337', 'request_id': 'req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:07.718204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] The token cache is being invalidate because project 68efe54b0a8f4b909e61eaf52898a337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:51:07.718514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:07.720975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:07.727438 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1253/2534] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:51:07 2022] DELETE /identity/v3/projects/68efe54b0a8f4b909e61eaf52898a337 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:07.740567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82d3d5-d0e1-4e2c-b78a-731f3fc221db None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:07.740953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82d3d5-d0e1-4e2c-b78a-731f3fc221db None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:07.741170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82d3d5-d0e1-4e2c-b78a-731f3fc221db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:07.780799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9a82d3d5-d0e1-4e2c-b78a-731f3fc221db None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:07.794083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9a82d3d5-d0e1-4e2c-b78a-731f3fc221db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:07.893628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:07.895419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:07.962327 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1282/2535] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:07.970994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c0e3f-68a1-4b06-8440-743df6377116 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:07.971285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c0e3f-68a1-4b06-8440-743df6377116 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:07.971609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c0e3f-68a1-4b06-8440-743df6377116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:08.031817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c67c0e3f-68a1-4b06-8440-743df6377116 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:08.041040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c67c0e3f-68a1-4b06-8440-743df6377116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:08.082871 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1283/2536] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:08.086093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c0e3f-68a1-4b06-8440-743df6377116 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:08.092644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfafa87-b2d1-48a2-8c4d-801a25b61d7a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:08.112596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c0e3f-68a1-4b06-8440-743df6377116 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:08.115928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f3307-9abb-4b74-ba90-56bd2408069b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:08.115928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f3307-9abb-4b74-ba90-56bd2408069b service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:08.115928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f3307-9abb-4b74-ba90-56bd2408069b service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:08.115928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f3307-9abb-4b74-ba90-56bd2408069b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:08.117826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3f5f3307-9abb-4b74-ba90-56bd2408069b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:08.133725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65f7d5-a62a-4a8b-8a06-22677e0c50e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:08.137228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65f7d5-a62a-4a8b-8a06-22677e0c50e0 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:08.137342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65f7d5-a62a-4a8b-8a06-22677e0c50e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:08.137610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65f7d5-a62a-4a8b-8a06-22677e0c50e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:08.141845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65f7d5-a62a-4a8b-8a06-22677e0c50e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:51:08.142168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65f7d5-a62a-4a8b-8a06-22677e0c50e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:08.143868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65f7d5-a62a-4a8b-8a06-22677e0c50e0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:08.191283 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1284/2537] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:51:08.195971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65f7d5-a62a-4a8b-8a06-22677e0c50e0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:08.214960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65f7d5-a62a-4a8b-8a06-22677e0c50e0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:08.223364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6cae2-9a4d-4620-9252-65a238a2261a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:08.224348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6cae2-9a4d-4620-9252-65a238a2261a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:08.225005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6cae2-9a4d-4620-9252-65a238a2261a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:08.225577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6cae2-9a4d-4620-9252-65a238a2261a admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:08.226782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6cae2-9a4d-4620-9252-65a238a2261a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:08.231887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6cae2-9a4d-4620-9252-65a238a2261a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:08.296521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f3307-9abb-4b74-ba90-56bd2408069b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:08.304709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f3307-9abb-4b74-ba90-56bd2408069b service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:08.357363 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1254/2538] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:51:08.396032 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1285/2539] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:51:08 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:51:08.397213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f3307-9abb-4b74-ba90-56bd2408069b service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:08.410155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f3307-9abb-4b74-ba90-56bd2408069b service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:08.582082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59866cef-9571-466f-8fd2-dad8025a3197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:08.583411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59866cef-9571-466f-8fd2-dad8025a3197 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:08.583871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59866cef-9571-466f-8fd2-dad8025a3197 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:08.584584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59866cef-9571-466f-8fd2-dad8025a3197 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:08.585548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59866cef-9571-466f-8fd2-dad8025a3197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:08.587814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59866cef-9571-466f-8fd2-dad8025a3197 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:08.627743 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1255/2540] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:51:08 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:51:08.632095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6cae2-9a4d-4620-9252-65a238a2261a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:08.648856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6cae2-9a4d-4620-9252-65a238a2261a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:08.665031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a0763-8efd-4935-85a7-9e9b307a00b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:08.668167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a0763-8efd-4935-85a7-9e9b307a00b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:08.668486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a0763-8efd-4935-85a7-9e9b307a00b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:08.668768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a0763-8efd-4935-85a7-9e9b307a00b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:08.670330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a0763-8efd-4935-85a7-9e9b307a00b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:08.674208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a0763-8efd-4935-85a7-9e9b307a00b6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:08.698104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:08.718261 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1286/2541] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:51:08.718406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59866cef-9571-466f-8fd2-dad8025a3197 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:08.741418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59866cef-9571-466f-8fd2-dad8025a3197 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:08.765494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb1011-84e3-45e0-b7bb-10d571484531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:08.767353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb1011-84e3-45e0-b7bb-10d571484531 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:08.767844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb1011-84e3-45e0-b7bb-10d571484531 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:08.769444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb1011-84e3-45e0-b7bb-10d571484531 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:08.770364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb1011-84e3-45e0-b7bb-10d571484531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:08.772714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb1011-84e3-45e0-b7bb-10d571484531 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:08.780924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:08.794283 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1256/2542] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:51:08.795830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a0763-8efd-4935-85a7-9e9b307a00b6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:08.811543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a0763-8efd-4935-85a7-9e9b307a00b6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:08.816223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6485c1-4e67-4c96-9e92-edae8f85ae42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:08.817098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6485c1-4e67-4c96-9e92-edae8f85ae42 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:08.817388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6485c1-4e67-4c96-9e92-edae8f85ae42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:08.817670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6485c1-4e67-4c96-9e92-edae8f85ae42 admin admin] PATH_INFO: `/v3/projects/ead11d740ca7415d85851b5b2ae62a75/users/33f68efd14004d2e8db0f18c588b6780/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:08.835236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6485c1-4e67-4c96-9e92-edae8f85ae42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ead11d740ca7415d85851b5b2ae62a75, user_id=33f68efd14004d2e8db0f18c588b6780, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:08.838154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6485c1-4e67-4c96-9e92-edae8f85ae42 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:08.866333 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1287/2543] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:51:08 2022] PUT /identity/v3/projects/ead11d740ca7415d85851b5b2ae62a75/users/33f68efd14004d2e8db0f18c588b6780/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:08.935222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c780ce4-282c-49ec-b4c2-564cd620e4d5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:08.935509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c780ce4-282c-49ec-b4c2-564cd620e4d5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:08.935885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c780ce4-282c-49ec-b4c2-564cd620e4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:08.971672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2c780ce4-282c-49ec-b4c2-564cd620e4d5 None None] MFA Rules not processed for user `33f68efd14004d2e8db0f18c588b6780`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:08.972964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2c780ce4-282c-49ec-b4c2-564cd620e4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:09.075964 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1257/2544] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:09.093741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6485c1-4e67-4c96-9e92-edae8f85ae42 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:09.097349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7d6485c1-4e67-4c96-9e92-edae8f85ae42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:09.200133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6485c1-4e67-4c96-9e92-edae8f85ae42 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:09.237718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc189320-25f4-4efb-bbc0-3e66497e2f5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:09.247406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc189320-25f4-4efb-bbc0-3e66497e2f5e service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:09.247851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc189320-25f4-4efb-bbc0-3e66497e2f5e service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:09.250012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc189320-25f4-4efb-bbc0-3e66497e2f5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:09.277162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc189320-25f4-4efb-bbc0-3e66497e2f5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:09.278822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc189320-25f4-4efb-bbc0-3e66497e2f5e service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:09.306176 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1288/2545] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:11.705869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c780ce4-282c-49ec-b4c2-564cd620e4d5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:11.707418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2c780ce4-282c-49ec-b4c2-564cd620e4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:11.803017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c780ce4-282c-49ec-b4c2-564cd620e4d5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:11.807886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:11.808848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:11.809387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:11.810007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] PATH_INFO: `/v3/users/257eed38c7c74fd08a15f65deef421cf` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:11.811518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=257eed38c7c74fd08a15f65deef421cf)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:11.813905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:11.903962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '257eed38c7c74fd08a15f65deef421cf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:11.919640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '257eed38c7c74fd08a15f65deef421cf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:11.920539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '257eed38c7c74fd08a15f65deef421cf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:11.928619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '257eed38c7c74fd08a15f65deef421cf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:11.940060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '257eed38c7c74fd08a15f65deef421cf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:11.947644 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1258/2546] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:51:11 2022] DELETE /identity/v3/users/257eed38c7c74fd08a15f65deef421cf => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:11.952264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bea61-c789-44f9-93a0-5ed9be19e19b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:11.953318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bea61-c789-44f9-93a0-5ed9be19e19b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:11.953829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bea61-c789-44f9-93a0-5ed9be19e19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:11.993589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d75bea61-c789-44f9-93a0-5ed9be19e19b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:11.994887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d75bea61-c789-44f9-93a0-5ed9be19e19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:12.087211 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1289/2547] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:12.105433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:12.107627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:12.134449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:12.295435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94591d-f833-41c7-9046-719b7c961d27 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:12.309028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1c1e0-25db-4380-a69d-9280ac8156f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:12.309028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1c1e0-25db-4380-a69d-9280ac8156f7 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:12.309028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1c1e0-25db-4380-a69d-9280ac8156f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:12.309028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1c1e0-25db-4380-a69d-9280ac8156f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:12.329971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1c1e0-25db-4380-a69d-9280ac8156f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:12.343512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1c1e0-25db-4380-a69d-9280ac8156f7 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:12.410856 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1259/2548] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:13.442457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bea61-c789-44f9-93a0-5ed9be19e19b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:13.444053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d75bea61-c789-44f9-93a0-5ed9be19e19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:13.458148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bea61-c789-44f9-93a0-5ed9be19e19b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:13.463291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfc4ea-489d-40c0-bf68-1a95f28ebef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:13.464770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfc4ea-489d-40c0-bf68-1a95f28ebef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:13.465100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfc4ea-489d-40c0-bf68-1a95f28ebef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:13.465612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfc4ea-489d-40c0-bf68-1a95f28ebef2 admin admin] PATH_INFO: `/v3/users/33f68efd14004d2e8db0f18c588b6780` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:13.467144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfc4ea-489d-40c0-bf68-1a95f28ebef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f68efd14004d2e8db0f18c588b6780)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:13.469361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfc4ea-489d-40c0-bf68-1a95f28ebef2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:13.532887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fabfc4ea-489d-40c0-bf68-1a95f28ebef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f68efd14004d2e8db0f18c588b6780'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:13.543333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:13.554042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fabfc4ea-489d-40c0-bf68-1a95f28ebef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f68efd14004d2e8db0f18c588b6780'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:13.568560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fabfc4ea-489d-40c0-bf68-1a95f28ebef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f68efd14004d2e8db0f18c588b6780'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:13.570851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fabfc4ea-489d-40c0-bf68-1a95f28ebef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f68efd14004d2e8db0f18c588b6780'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:13.571944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fabfc4ea-489d-40c0-bf68-1a95f28ebef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f68efd14004d2e8db0f18c588b6780'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:13.589740 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1290/2549] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:51:13 2022] DELETE /identity/v3/users/33f68efd14004d2e8db0f18c588b6780 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:13.595630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202df99-4f79-487e-95da-f5592c91dd63 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:13.596205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202df99-4f79-487e-95da-f5592c91dd63 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:13.596205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202df99-4f79-487e-95da-f5592c91dd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:13.626446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5202df99-4f79-487e-95da-f5592c91dd63 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:13.628107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5202df99-4f79-487e-95da-f5592c91dd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:13.900956 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1260/2550] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:51:13.916454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09114af-7b13-486a-b109-dd904e34c389 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:13.917519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09114af-7b13-486a-b109-dd904e34c389 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:13.917977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09114af-7b13-486a-b109-dd904e34c389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:13.919393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202df99-4f79-487e-95da-f5592c91dd63 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:13.922720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5202df99-4f79-487e-95da-f5592c91dd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:13.977758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c09114af-7b13-486a-b109-dd904e34c389 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:13.981555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c09114af-7b13-486a-b109-dd904e34c389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:14.076059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202df99-4f79-487e-95da-f5592c91dd63 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:14.088315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2787f8e-0688-4357-acee-e72214743a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:14.090106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2787f8e-0688-4357-acee-e72214743a56 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:14.090390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2787f8e-0688-4357-acee-e72214743a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:14.090663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2787f8e-0688-4357-acee-e72214743a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:14.111152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2787f8e-0688-4357-acee-e72214743a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:14.112734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2787f8e-0688-4357-acee-e72214743a56 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:14.135763 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1261/2551] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:14.140174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2787f8e-0688-4357-acee-e72214743a56 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:14.146801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c2787f8e-0688-4357-acee-e72214743a56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:14.152832 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1291/2552] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:51:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:14.169074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09114af-7b13-486a-b109-dd904e34c389 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:14.171139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c09114af-7b13-486a-b109-dd904e34c389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:14.185669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2787f8e-0688-4357-acee-e72214743a56 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:14.196670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010d0f5-f392-40dc-ab64-59b3c917bc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:14.197333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010d0f5-f392-40dc-ab64-59b3c917bc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:14.197613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010d0f5-f392-40dc-ab64-59b3c917bc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:14.205417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010d0f5-f392-40dc-ab64-59b3c917bc96 admin admin] PATH_INFO: `/v3/projects/2e521d306afe4d8f8ac8eef5f554892c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:14.211233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010d0f5-f392-40dc-ab64-59b3c917bc96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e521d306afe4d8f8ac8eef5f554892c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:14.213714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010d0f5-f392-40dc-ab64-59b3c917bc96 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:14.372454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2010d0f5-f392-40dc-ab64-59b3c917bc96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e521d306afe4d8f8ac8eef5f554892c', 'request_id': 'req-2010d0f5-f392-40dc-ab64-59b3c917bc96'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:14.375257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:14.386546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2010d0f5-f392-40dc-ab64-59b3c917bc96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e521d306afe4d8f8ac8eef5f554892c', 'request_id': 'req-2010d0f5-f392-40dc-ab64-59b3c917bc96'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:14.397092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2010d0f5-f392-40dc-ab64-59b3c917bc96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e521d306afe4d8f8ac8eef5f554892c', 'request_id': 'req-2010d0f5-f392-40dc-ab64-59b3c917bc96'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:14.410351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09114af-7b13-486a-b109-dd904e34c389 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:14.412768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2010d0f5-f392-40dc-ab64-59b3c917bc96 admin admin] The token cache is being invalidate because project 2e521d306afe4d8f8ac8eef5f554892c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:51:14.413336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2010d0f5-f392-40dc-ab64-59b3c917bc96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:14.414499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2010d0f5-f392-40dc-ab64-59b3c917bc96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:14.416660 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1262/2553] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:51:14 2022] DELETE /identity/v3/projects/2e521d306afe4d8f8ac8eef5f554892c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:14.422948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6cb3d-63a4-4c38-836d-5dcf0192dd7c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:14.423695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6cb3d-63a4-4c38-836d-5dcf0192dd7c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:14.424434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6cb3d-63a4-4c38-836d-5dcf0192dd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:14.456273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-42e6cb3d-63a4-4c38-836d-5dcf0192dd7c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:14.458190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-42e6cb3d-63a4-4c38-836d-5dcf0192dd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:14.559377 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1263/2554] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:14.566745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb9bcc-a7f0-40ab-8cbf-4bc8aa273ca4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:14.567034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb9bcc-a7f0-40ab-8cbf-4bc8aa273ca4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:14.569055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb9bcc-a7f0-40ab-8cbf-4bc8aa273ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:14.600631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86433b-e8e9-4a40-92c5-5f04040e1486 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:14.601289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86433b-e8e9-4a40-92c5-5f04040e1486 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:14.601743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86433b-e8e9-4a40-92c5-5f04040e1486 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:14.602334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86433b-e8e9-4a40-92c5-5f04040e1486 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:14.606678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5b86433b-e8e9-4a40-92c5-5f04040e1486 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:14.614951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4ceb9bcc-a7f0-40ab-8cbf-4bc8aa273ca4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:14.620002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4ceb9bcc-a7f0-40ab-8cbf-4bc8aa273ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:14.647611 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1264/2555] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:51:14.649285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb9bcc-a7f0-40ab-8cbf-4bc8aa273ca4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:14.653019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:14.663487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb9bcc-a7f0-40ab-8cbf-4bc8aa273ca4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:14.682254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b88423-fe22-48e1-abf9-b70c559fd1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:14.683866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b88423-fe22-48e1-abf9-b70c559fd1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:14.684345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b88423-fe22-48e1-abf9-b70c559fd1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:14.684843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b88423-fe22-48e1-abf9-b70c559fd1a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:14.686783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b88423-fe22-48e1-abf9-b70c559fd1a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:51:14.687928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b88423-fe22-48e1-abf9-b70c559fd1a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:14.691686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b88423-fe22-48e1-abf9-b70c559fd1a2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:14.732088 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1265/2556] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:51:14.735092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b88423-fe22-48e1-abf9-b70c559fd1a2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:14.780613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b88423-fe22-48e1-abf9-b70c559fd1a2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:14.796696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205275e-aabc-46e4-a1ca-10a542c1d185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:14.797412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205275e-aabc-46e4-a1ca-10a542c1d185 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:14.797675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205275e-aabc-46e4-a1ca-10a542c1d185 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:14.797990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205275e-aabc-46e4-a1ca-10a542c1d185 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:14.798899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205275e-aabc-46e4-a1ca-10a542c1d185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:14.799302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86433b-e8e9-4a40-92c5-5f04040e1486 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:14.800865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205275e-aabc-46e4-a1ca-10a542c1d185 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:14.802496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86433b-e8e9-4a40-92c5-5f04040e1486 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:14.817203 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1292/2557] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:51:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 20:51:14.862476 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1266/2558] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:51:14 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:51:14.868427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86433b-e8e9-4a40-92c5-5f04040e1486 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:14.893881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86433b-e8e9-4a40-92c5-5f04040e1486 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:15.002839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29481441-dfe8-4dc9-b389-fd3d75c817c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:15.003653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29481441-dfe8-4dc9-b389-fd3d75c817c9 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:15.004121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29481441-dfe8-4dc9-b389-fd3d75c817c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:15.004538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29481441-dfe8-4dc9-b389-fd3d75c817c9 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:15.005373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29481441-dfe8-4dc9-b389-fd3d75c817c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:15.009302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29481441-dfe8-4dc9-b389-fd3d75c817c9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:15.052407 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1293/2559] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:51:14 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:51:15.053247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205275e-aabc-46e4-a1ca-10a542c1d185 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:15.065520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205275e-aabc-46e4-a1ca-10a542c1d185 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:15.070862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f3188-6c0a-4d48-9ad6-24f39479f39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:15.071773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f3188-6c0a-4d48-9ad6-24f39479f39e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:15.072241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f3188-6c0a-4d48-9ad6-24f39479f39e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:15.072704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f3188-6c0a-4d48-9ad6-24f39479f39e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:15.074155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f3188-6c0a-4d48-9ad6-24f39479f39e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:15.076311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f3188-6c0a-4d48-9ad6-24f39479f39e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:15.095468 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1267/2560] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:51:15.099762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29481441-dfe8-4dc9-b389-fd3d75c817c9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:15.121924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29481441-dfe8-4dc9-b389-fd3d75c817c9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:15.128072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c08a5-1a6e-4c16-90e5-ecc924acb96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:15.129043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c08a5-1a6e-4c16-90e5-ecc924acb96e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:15.129637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c08a5-1a6e-4c16-90e5-ecc924acb96e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:15.130292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c08a5-1a6e-4c16-90e5-ecc924acb96e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:15.131484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c08a5-1a6e-4c16-90e5-ecc924acb96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:15.134488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c08a5-1a6e-4c16-90e5-ecc924acb96e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:15.154199 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1294/2561] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:51:15.158257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f3188-6c0a-4d48-9ad6-24f39479f39e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:15.182659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f3188-6c0a-4d48-9ad6-24f39479f39e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:15.191389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e675d2-3dcc-423d-a500-b84d31a7ae84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:15.192269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e675d2-3dcc-423d-a500-b84d31a7ae84 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:15.192654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e675d2-3dcc-423d-a500-b84d31a7ae84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:15.194702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e675d2-3dcc-423d-a500-b84d31a7ae84 admin admin] PATH_INFO: `/v3/projects/bcbaf703d0064704a34a858adf2a33fa/users/1c73e775740c40c4a9c088ceddaaf0d8/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:15.223111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e675d2-3dcc-423d-a500-b84d31a7ae84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbaf703d0064704a34a858adf2a33fa, user_id=1c73e775740c40c4a9c088ceddaaf0d8, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:15.229441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e675d2-3dcc-423d-a500-b84d31a7ae84 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:15.258068 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1268/2562] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:51:15 2022] PUT /identity/v3/projects/bcbaf703d0064704a34a858adf2a33fa/users/1c73e775740c40c4a9c088ceddaaf0d8/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:15.260674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfa55d-2292-475a-9678-f6166b7c6402 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:15.261136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfa55d-2292-475a-9678-f6166b7c6402 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:15.261421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfa55d-2292-475a-9678-f6166b7c6402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:15.316720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bbcfa55d-2292-475a-9678-f6166b7c6402 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:15.318366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bbcfa55d-2292-475a-9678-f6166b7c6402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:15.455673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e675d2-3dcc-423d-a500-b84d31a7ae84 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:15.458415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f1e675d2-3dcc-423d-a500-b84d31a7ae84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:15.477420 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1295/2563] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:15.505752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfa55d-2292-475a-9678-f6166b7c6402 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:15.507530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bbcfa55d-2292-475a-9678-f6166b7c6402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:15.597359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:15.612617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e675d2-3dcc-423d-a500-b84d31a7ae84 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:15.626102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47852dc9-78a2-42fe-acc0-833171945d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:15.627808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfa55d-2292-475a-9678-f6166b7c6402 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:15.628620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47852dc9-78a2-42fe-acc0-833171945d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:15.628998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47852dc9-78a2-42fe-acc0-833171945d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:15.629216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47852dc9-78a2-42fe-acc0-833171945d7d admin admin] PATH_INFO: `/v3/projects/ead11d740ca7415d85851b5b2ae62a75` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:15.632379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47852dc9-78a2-42fe-acc0-833171945d7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ead11d740ca7415d85851b5b2ae62a75)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:15.636389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb7c78-68d5-4bf7-8e8a-dcfd8622ed03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:15.638453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb7c78-68d5-4bf7-8e8a-dcfd8622ed03 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:15.638771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47852dc9-78a2-42fe-acc0-833171945d7d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:15.639305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb7c78-68d5-4bf7-8e8a-dcfd8622ed03 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:15.639772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb7c78-68d5-4bf7-8e8a-dcfd8622ed03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:15.642671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:15.661849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb7c78-68d5-4bf7-8e8a-dcfd8622ed03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:15.665019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb7c78-68d5-4bf7-8e8a-dcfd8622ed03 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:15.678690 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1296/2564] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:51:15.751664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-47852dc9-78a2-42fe-acc0-833171945d7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ead11d740ca7415d85851b5b2ae62a75', 'request_id': 'req-47852dc9-78a2-42fe-acc0-833171945d7d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:15.765501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-47852dc9-78a2-42fe-acc0-833171945d7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ead11d740ca7415d85851b5b2ae62a75', 'request_id': 'req-47852dc9-78a2-42fe-acc0-833171945d7d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:15.778700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-47852dc9-78a2-42fe-acc0-833171945d7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ead11d740ca7415d85851b5b2ae62a75', 'request_id': 'req-47852dc9-78a2-42fe-acc0-833171945d7d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:15.802281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-47852dc9-78a2-42fe-acc0-833171945d7d admin admin] The token cache is being invalidate because project ead11d740ca7415d85851b5b2ae62a75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:51:15.802716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-47852dc9-78a2-42fe-acc0-833171945d7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:15.803792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-47852dc9-78a2-42fe-acc0-833171945d7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:15.805897 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1269/2565] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:51:15 2022] DELETE /identity/v3/projects/ead11d740ca7415d85851b5b2ae62a75 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:51:15.811172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e968d-5b96-4cab-a50e-c8010fe1cd6e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:15.811471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e968d-5b96-4cab-a50e-c8010fe1cd6e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:15.811792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e968d-5b96-4cab-a50e-c8010fe1cd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:15.837692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7f8e968d-5b96-4cab-a50e-c8010fe1cd6e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:15.839195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7f8e968d-5b96-4cab-a50e-c8010fe1cd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:15.929170 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1297/2566] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:15.936998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7472c-7df0-48f2-80f0-281fb7cb2a5c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:15.937323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7472c-7df0-48f2-80f0-281fb7cb2a5c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:15.937596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7472c-7df0-48f2-80f0-281fb7cb2a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:15.972798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-11b7472c-7df0-48f2-80f0-281fb7cb2a5c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:15.977791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-11b7472c-7df0-48f2-80f0-281fb7cb2a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:15.995864 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1270/2567] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:16.004383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e968d-5b96-4cab-a50e-c8010fe1cd6e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:16.030595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e968d-5b96-4cab-a50e-c8010fe1cd6e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:16.036170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381a169-886c-4c8a-b926-2226b19f9622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:16.038235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381a169-886c-4c8a-b926-2226b19f9622 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:16.038538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381a169-886c-4c8a-b926-2226b19f9622 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:16.038818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381a169-886c-4c8a-b926-2226b19f9622 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:16.039775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381a169-886c-4c8a-b926-2226b19f9622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:51:16.040382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381a169-886c-4c8a-b926-2226b19f9622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:16.044677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381a169-886c-4c8a-b926-2226b19f9622 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:16.068388 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1298/2568] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:51:16.072367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7472c-7df0-48f2-80f0-281fb7cb2a5c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:16.082995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7472c-7df0-48f2-80f0-281fb7cb2a5c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:16.090716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e47616-5805-4d66-99e6-4e24f6dd383a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:16.091333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e47616-5805-4d66-99e6-4e24f6dd383a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:16.092366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e47616-5805-4d66-99e6-4e24f6dd383a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:16.092366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e47616-5805-4d66-99e6-4e24f6dd383a admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:16.092831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e47616-5805-4d66-99e6-4e24f6dd383a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:16.094901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e47616-5805-4d66-99e6-4e24f6dd383a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:16.148488 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1271/2569] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:51:16 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:51:16.150880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381a169-886c-4c8a-b926-2226b19f9622 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:16.166759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381a169-886c-4c8a-b926-2226b19f9622 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:16.294625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5dd74-f1a6-40bc-a2de-c95e79e43c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:16.295411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5dd74-f1a6-40bc-a2de-c95e79e43c92 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:16.295546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5dd74-f1a6-40bc-a2de-c95e79e43c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:16.295946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5dd74-f1a6-40bc-a2de-c95e79e43c92 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:16.296589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5dd74-f1a6-40bc-a2de-c95e79e43c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:16.298620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5dd74-f1a6-40bc-a2de-c95e79e43c92 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:16.344825 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1299/2570] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:51:16 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:51:16.350810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e47616-5805-4d66-99e6-4e24f6dd383a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:16.368749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e47616-5805-4d66-99e6-4e24f6dd383a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:16.389736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f9e81-7501-427c-a6e5-56f657108cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:16.390739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f9e81-7501-427c-a6e5-56f657108cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:16.391649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f9e81-7501-427c-a6e5-56f657108cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:16.392820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f9e81-7501-427c-a6e5-56f657108cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:16.393740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f9e81-7501-427c-a6e5-56f657108cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:16.396519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f9e81-7501-427c-a6e5-56f657108cb8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:16.428315 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1272/2571] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:51:16.430020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5dd74-f1a6-40bc-a2de-c95e79e43c92 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:16.453517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5dd74-f1a6-40bc-a2de-c95e79e43c92 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:16.462276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa311905-6f4b-457d-b493-f36f1cea45cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:16.463129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa311905-6f4b-457d-b493-f36f1cea45cc admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:16.463697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa311905-6f4b-457d-b493-f36f1cea45cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:16.464221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa311905-6f4b-457d-b493-f36f1cea45cc admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:16.465148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa311905-6f4b-457d-b493-f36f1cea45cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:16.467290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa311905-6f4b-457d-b493-f36f1cea45cc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:16.498300 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1300/2572] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:51:16.503792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f9e81-7501-427c-a6e5-56f657108cb8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:16.530517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f9e81-7501-427c-a6e5-56f657108cb8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:16.548413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344c443-8588-4287-9e73-46163efaa306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:16.549724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344c443-8588-4287-9e73-46163efaa306 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:16.550125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344c443-8588-4287-9e73-46163efaa306 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:16.551324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344c443-8588-4287-9e73-46163efaa306 admin admin] PATH_INFO: `/v3/projects/d196cef49e5040dea8885bcd8f179747/users/10bbfca2d8744acea2587037f190fe90/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:16.580577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344c443-8588-4287-9e73-46163efaa306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d196cef49e5040dea8885bcd8f179747, user_id=10bbfca2d8744acea2587037f190fe90, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:16.593326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344c443-8588-4287-9e73-46163efaa306 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:16.629415 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1273/2573] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:51:16 2022] PUT /identity/v3/projects/d196cef49e5040dea8885bcd8f179747/users/10bbfca2d8744acea2587037f190fe90/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:16.634251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba18eb-05f1-41ce-8814-5372cc40c95a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:16.634760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba18eb-05f1-41ce-8814-5372cc40c95a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:16.634903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba18eb-05f1-41ce-8814-5372cc40c95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:16.679842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-51ba18eb-05f1-41ce-8814-5372cc40c95a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:16.681942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-51ba18eb-05f1-41ce-8814-5372cc40c95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:16.778427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:16.825615 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1301/2574] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:16.854111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344c443-8588-4287-9e73-46163efaa306 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:16.857347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3344c443-8588-4287-9e73-46163efaa306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:16.959094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344c443-8588-4287-9e73-46163efaa306 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:16.969101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f62a1-86f4-4cc6-b3a5-c9bb1b0ebb71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:16.970724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f62a1-86f4-4cc6-b3a5-c9bb1b0ebb71 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:16.979348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f62a1-86f4-4cc6-b3a5-c9bb1b0ebb71 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:16.979964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f62a1-86f4-4cc6-b3a5-c9bb1b0ebb71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:16.998240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f62a1-86f4-4cc6-b3a5-c9bb1b0ebb71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:17.001662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f62a1-86f4-4cc6-b3a5-c9bb1b0ebb71 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:17.015174 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1274/2575] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:24.820555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba18eb-05f1-41ce-8814-5372cc40c95a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:24.822213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-51ba18eb-05f1-41ce-8814-5372cc40c95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:24.851738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba18eb-05f1-41ce-8814-5372cc40c95a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:24.871574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:24.872222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:24.872506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:24.872781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] PATH_INFO: `/v3/users/ad3cb96a474d4d04be4e16212a2bc8c9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:24.874482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad3cb96a474d4d04be4e16212a2bc8c9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:24.883474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:25.006208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad3cb96a474d4d04be4e16212a2bc8c9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:25.045534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad3cb96a474d4d04be4e16212a2bc8c9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:25.059935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad3cb96a474d4d04be4e16212a2bc8c9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:25.060584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad3cb96a474d4d04be4e16212a2bc8c9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:25.061246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad3cb96a474d4d04be4e16212a2bc8c9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:25.126883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7483016-b634-4999-9ffb-ae01726b2521 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:25.127426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7483016-b634-4999-9ffb-ae01726b2521 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:25.127426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7483016-b634-4999-9ffb-ae01726b2521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:25.131821 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1302/2576] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:51:24 2022] DELETE /identity/v3/users/ad3cb96a474d4d04be4e16212a2bc8c9 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:25.166356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c7483016-b634-4999-9ffb-ae01726b2521 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:25.172708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c7483016-b634-4999-9ffb-ae01726b2521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:25.264546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:25.366269 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1275/2577] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:25.377398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:25.379082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:25.525473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed66e4-b84e-4546-8229-2a0e77a67684 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:25.543228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126543a3-08e3-4a30-acf2-5a74c5bb5251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:25.543873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126543a3-08e3-4a30-acf2-5a74c5bb5251 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:25.544168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126543a3-08e3-4a30-acf2-5a74c5bb5251 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:25.544447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126543a3-08e3-4a30-acf2-5a74c5bb5251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:25.557465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126543a3-08e3-4a30-acf2-5a74c5bb5251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:25.559311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126543a3-08e3-4a30-acf2-5a74c5bb5251 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:25.605006 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1303/2578] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:26.231075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46511f4-9c5b-4eb0-a57c-1da0654bf31e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:26.231722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46511f4-9c5b-4eb0-a57c-1da0654bf31e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:26.231722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46511f4-9c5b-4eb0-a57c-1da0654bf31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:26.325664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f46511f4-9c5b-4eb0-a57c-1da0654bf31e None None] MFA Rules not processed for user `1c73e775740c40c4a9c088ceddaaf0d8`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:26.328597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f46511f4-9c5b-4eb0-a57c-1da0654bf31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:26.398197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126543a3-08e3-4a30-acf2-5a74c5bb5251 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:26.400452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-126543a3-08e3-4a30-acf2-5a74c5bb5251 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:26.429478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126543a3-08e3-4a30-acf2-5a74c5bb5251 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:26.445932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:26.447377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:26.447857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:26.448326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] PATH_INFO: `/v3/projects/f968aad8b96b4ba9ade8cafd689c1b7f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:26.450389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f968aad8b96b4ba9ade8cafd689c1b7f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:26.458290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:26.522926 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1276/2579] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:51:26.591567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f968aad8b96b4ba9ade8cafd689c1b7f', 'request_id': 'req-d3fb3c5c-a213-4573-accc-e8511e6caa83'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:26.615174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f968aad8b96b4ba9ade8cafd689c1b7f', 'request_id': 'req-d3fb3c5c-a213-4573-accc-e8511e6caa83'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:26.617723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:26.649426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f968aad8b96b4ba9ade8cafd689c1b7f', 'request_id': 'req-d3fb3c5c-a213-4573-accc-e8511e6caa83'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:26.667404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46511f4-9c5b-4eb0-a57c-1da0654bf31e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:26.669891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f46511f4-9c5b-4eb0-a57c-1da0654bf31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:26.694286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] The token cache is being invalidate because project f968aad8b96b4ba9ade8cafd689c1b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:51:26.704194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:26.705871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:26.709846 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1304/2580] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:51:26 2022] DELETE /identity/v3/projects/f968aad8b96b4ba9ade8cafd689c1b7f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:26.714255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:26.720948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:26.906837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46511f4-9c5b-4eb0-a57c-1da0654bf31e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:26.941798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb3c5c-a213-4573-accc-e8511e6caa83 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:26.970856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:26.971525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:26.971872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:26.972158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] PATH_INFO: `/v3/users/fa4cc607989842a5ac6041883f9645de` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:26.973959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa4cc607989842a5ac6041883f9645de)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:26.977194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:27.049144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:27.072358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e8a1f-306c-483b-9097-441cf8d1ebdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:27.077458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8a1f-306c-483b-9097-441cf8d1ebdb service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:27.078211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8a1f-306c-483b-9097-441cf8d1ebdb service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:27.078855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8a1f-306c-483b-9097-441cf8d1ebdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:27.080148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa4cc607989842a5ac6041883f9645de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:27.081373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b84e8a1f-306c-483b-9097-441cf8d1ebdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:27.110663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa4cc607989842a5ac6041883f9645de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:27.140273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa4cc607989842a5ac6041883f9645de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:27.141707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa4cc607989842a5ac6041883f9645de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:27.145406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa4cc607989842a5ac6041883f9645de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:27.165591 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1305/2581] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:51:26 2022] DELETE /identity/v3/users/fa4cc607989842a5ac6041883f9645de => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:27.185008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e8a1f-306c-483b-9097-441cf8d1ebdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:27.187402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e8a1f-306c-483b-9097-441cf8d1ebdb service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:27.199369 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1277/2582] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:51:28.057944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:28.059495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:28.222475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037268cd-8c32-4847-8ea3-b2e5b766edc1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:28.223390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037268cd-8c32-4847-8ea3-b2e5b766edc1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:28.223825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037268cd-8c32-4847-8ea3-b2e5b766edc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:28.256997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b174b13-c978-401b-8997-39f3d40c9664 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:28.267902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec8d27-1066-415f-ba96-2d99baf55df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:28.268658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec8d27-1066-415f-ba96-2d99baf55df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:28.269345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec8d27-1066-415f-ba96-2d99baf55df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:28.269847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec8d27-1066-415f-ba96-2d99baf55df7 admin admin] PATH_INFO: `/v3/projects/1c5b42b9fbdf4845bbe196fad7a2873b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:28.273534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec8d27-1066-415f-ba96-2d99baf55df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c5b42b9fbdf4845bbe196fad7a2873b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:28.278063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec8d27-1066-415f-ba96-2d99baf55df7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:28.292711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:28.292930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-037268cd-8c32-4847-8ea3-b2e5b766edc1 None None] MFA Rules not processed for user `10bbfca2d8744acea2587037f190fe90`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:28.294139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-037268cd-8c32-4847-8ea3-b2e5b766edc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:28.362450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9aec8d27-1066-415f-ba96-2d99baf55df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c5b42b9fbdf4845bbe196fad7a2873b', 'request_id': 'req-9aec8d27-1066-415f-ba96-2d99baf55df7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:28.372358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9aec8d27-1066-415f-ba96-2d99baf55df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c5b42b9fbdf4845bbe196fad7a2873b', 'request_id': 'req-9aec8d27-1066-415f-ba96-2d99baf55df7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:28.393702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9aec8d27-1066-415f-ba96-2d99baf55df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c5b42b9fbdf4845bbe196fad7a2873b', 'request_id': 'req-9aec8d27-1066-415f-ba96-2d99baf55df7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:28.418972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9aec8d27-1066-415f-ba96-2d99baf55df7 admin admin] The token cache is being invalidate because project 1c5b42b9fbdf4845bbe196fad7a2873b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:51:28.419361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9aec8d27-1066-415f-ba96-2d99baf55df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:28.420203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9aec8d27-1066-415f-ba96-2d99baf55df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:28.422216 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1306/2583] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:51:28 2022] DELETE /identity/v3/projects/1c5b42b9fbdf4845bbe196fad7a2873b => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:28.426595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4319d52-38c1-463f-85c3-c23b4365aa56 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:28.427005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4319d52-38c1-463f-85c3-c23b4365aa56 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:28.427378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4319d52-38c1-463f-85c3-c23b4365aa56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:28.456866 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1278/2584] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:51:28.470603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef436f54-2b7a-4a8b-a959-62715e1b3989 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:28.470894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef436f54-2b7a-4a8b-a959-62715e1b3989 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:28.471172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef436f54-2b7a-4a8b-a959-62715e1b3989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:28.472844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e4319d52-38c1-463f-85c3-c23b4365aa56 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:28.475338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e4319d52-38c1-463f-85c3-c23b4365aa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:28.522217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ef436f54-2b7a-4a8b-a959-62715e1b3989 None None] MFA Rules not processed for user `10bbfca2d8744acea2587037f190fe90`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:28.523308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ef436f54-2b7a-4a8b-a959-62715e1b3989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:28.618849 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1307/2585] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:28.628194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60846fd6-ab3a-473d-a956-7542c6446c5c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:28.628867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60846fd6-ab3a-473d-a956-7542c6446c5c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:28.629952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60846fd6-ab3a-473d-a956-7542c6446c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:28.660528 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1279/2586] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:28.660924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-60846fd6-ab3a-473d-a956-7542c6446c5c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:28.663015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-60846fd6-ab3a-473d-a956-7542c6446c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:28.685022 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1308/2587] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:51:28.688491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef436f54-2b7a-4a8b-a959-62715e1b3989 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:28.701951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef436f54-2b7a-4a8b-a959-62715e1b3989 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:28.708397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b02329-01e7-4cbf-98ac-77fc734865b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:28.709970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b02329-01e7-4cbf-98ac-77fc734865b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:28.710539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b02329-01e7-4cbf-98ac-77fc734865b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:28.711009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b02329-01e7-4cbf-98ac-77fc734865b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:28.711750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b02329-01e7-4cbf-98ac-77fc734865b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:51:28.712152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b02329-01e7-4cbf-98ac-77fc734865b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:28.714527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b02329-01e7-4cbf-98ac-77fc734865b2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:28.732445 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1280/2588] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:51:28.733465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60846fd6-ab3a-473d-a956-7542c6446c5c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:28.747999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60846fd6-ab3a-473d-a956-7542c6446c5c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:28.766117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0fadd-567a-4255-bc0c-0cb304c418a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:28.766760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0fadd-567a-4255-bc0c-0cb304c418a6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:28.767047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0fadd-567a-4255-bc0c-0cb304c418a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:28.767327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0fadd-567a-4255-bc0c-0cb304c418a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:28.767976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0fadd-567a-4255-bc0c-0cb304c418a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:28.770018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0fadd-567a-4255-bc0c-0cb304c418a6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:28.818247 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1309/2589] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:51:28 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:51:28.824513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b02329-01e7-4cbf-98ac-77fc734865b2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:28.838645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b02329-01e7-4cbf-98ac-77fc734865b2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:28.866836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:28.937173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c547d-6630-4ec8-9bb6-aa44c2e2c4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:28.937811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c547d-6630-4ec8-9bb6-aa44c2e2c4bd admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:28.938263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c547d-6630-4ec8-9bb6-aa44c2e2c4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:28.938547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c547d-6630-4ec8-9bb6-aa44c2e2c4bd admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:28.939191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c547d-6630-4ec8-9bb6-aa44c2e2c4bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:28.941899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c547d-6630-4ec8-9bb6-aa44c2e2c4bd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:28.975373 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1281/2590] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:51:28 2022] POST /identity/v3/users => generated 290 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:51:28.979530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0fadd-567a-4255-bc0c-0cb304c418a6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:28.993442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0fadd-567a-4255-bc0c-0cb304c418a6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:29.017218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da532165-512e-4e1e-b6d0-e1ced053c5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:29.017218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da532165-512e-4e1e-b6d0-e1ced053c5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:29.017218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da532165-512e-4e1e-b6d0-e1ced053c5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:29.017218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da532165-512e-4e1e-b6d0-e1ced053c5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:29.017593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da532165-512e-4e1e-b6d0-e1ced053c5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:29.017593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da532165-512e-4e1e-b6d0-e1ced053c5e6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:29.033600 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1310/2591] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:51:29.038116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c547d-6630-4ec8-9bb6-aa44c2e2c4bd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:29.061590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c547d-6630-4ec8-9bb6-aa44c2e2c4bd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:29.077584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2f2dc-953d-4bd9-9c81-60b506da3fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:29.078431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2f2dc-953d-4bd9-9c81-60b506da3fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:29.078865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2f2dc-953d-4bd9-9c81-60b506da3fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:29.079297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2f2dc-953d-4bd9-9c81-60b506da3fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:29.080275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2f2dc-953d-4bd9-9c81-60b506da3fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:29.082060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2f2dc-953d-4bd9-9c81-60b506da3fe8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:29.096000 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1282/2592] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:51:29.100136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da532165-512e-4e1e-b6d0-e1ced053c5e6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:29.113746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da532165-512e-4e1e-b6d0-e1ced053c5e6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:29.126216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d79daf-ace7-4408-8619-6a7428a22276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:29.127024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d79daf-ace7-4408-8619-6a7428a22276 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:29.127312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d79daf-ace7-4408-8619-6a7428a22276 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:29.127596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d79daf-ace7-4408-8619-6a7428a22276 admin admin] PATH_INFO: `/v3/projects/cd0212e91da8419da9a74763d36a990d/users/404086c2409f4135af212d4047c29223/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:29.159743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d79daf-ace7-4408-8619-6a7428a22276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd0212e91da8419da9a74763d36a990d, user_id=404086c2409f4135af212d4047c29223, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:29.162675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d79daf-ace7-4408-8619-6a7428a22276 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:29.207545 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1311/2593] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:51:29 2022] PUT /identity/v3/projects/cd0212e91da8419da9a74763d36a990d/users/404086c2409f4135af212d4047c29223/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:29.209214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa71772-cf09-4ea7-ac47-ae6d5494fdf9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:29.209508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa71772-cf09-4ea7-ac47-ae6d5494fdf9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:29.209810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa71772-cf09-4ea7-ac47-ae6d5494fdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:29.261343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9fa71772-cf09-4ea7-ac47-ae6d5494fdf9 None None] MFA Rules not processed for user `404086c2409f4135af212d4047c29223`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:29.263531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9fa71772-cf09-4ea7-ac47-ae6d5494fdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:29.292535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d79daf-ace7-4408-8619-6a7428a22276 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:29.294359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b2d79daf-ace7-4408-8619-6a7428a22276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:29.337730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:29.388530 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1283/2594] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:29.419750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d79daf-ace7-4408-8619-6a7428a22276 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:29.427579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec8afd-3d90-4ac0-b4d1-17575712deba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:29.428205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec8afd-3d90-4ac0-b4d1-17575712deba service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:29.428807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec8afd-3d90-4ac0-b4d1-17575712deba service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:29.428807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec8afd-3d90-4ac0-b4d1-17575712deba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:29.524038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec8afd-3d90-4ac0-b4d1-17575712deba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:29.525872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec8afd-3d90-4ac0-b4d1-17575712deba service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:29.537929 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1312/2595] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:29.592366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f856fe0-285c-4927-bca8-359c3e452bdb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:29.592664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f856fe0-285c-4927-bca8-359c3e452bdb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:29.592991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f856fe0-285c-4927-bca8-359c3e452bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:29.643739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1f856fe0-285c-4927-bca8-359c3e452bdb None None] MFA Rules not processed for user `2e11c33c786746e5b3effedd32674582`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:29.646453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1f856fe0-285c-4927-bca8-359c3e452bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:29.743336 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1284/2596] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:30.307450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec8afd-3d90-4ac0-b4d1-17575712deba service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:30.325918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec8afd-3d90-4ac0-b4d1-17575712deba service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:30.336300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ddb5d-9063-4cbb-9d4e-13c989537e99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:30.338395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ddb5d-9063-4cbb-9d4e-13c989537e99 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:30.339160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ddb5d-9063-4cbb-9d4e-13c989537e99 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:30.339954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ddb5d-9063-4cbb-9d4e-13c989537e99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:30.353923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ddb5d-9063-4cbb-9d4e-13c989537e99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:30.357510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ddb5d-9063-4cbb-9d4e-13c989537e99 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:30.383492 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1313/2597] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:30.796405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b4af3-8fc2-451d-9802-3ecd7da94bb5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:30.797404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b4af3-8fc2-451d-9802-3ecd7da94bb5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:30.797859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b4af3-8fc2-451d-9802-3ecd7da94bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:30.833082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5b7b4af3-8fc2-451d-9802-3ecd7da94bb5 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:30.849146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5b7b4af3-8fc2-451d-9802-3ecd7da94bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:30.922390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:30.976751 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1285/2598] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:51:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:31.014787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ddb5d-9063-4cbb-9d4e-13c989537e99 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:31.017113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7e6ddb5d-9063-4cbb-9d4e-13c989537e99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:31.155484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ddb5d-9063-4cbb-9d4e-13c989537e99 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:31.165883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0264575-c0c3-443c-8f71-037a8f85667b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:31.167390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0264575-c0c3-443c-8f71-037a8f85667b service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:31.167786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0264575-c0c3-443c-8f71-037a8f85667b service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:31.168546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0264575-c0c3-443c-8f71-037a8f85667b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:31.182074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0264575-c0c3-443c-8f71-037a8f85667b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:31.186568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0264575-c0c3-443c-8f71-037a8f85667b service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:31.198358 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1314/2599] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:51:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:51:31.419235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d7a08-ccc1-4427-80fd-9d4cdf20bdcd None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:31.419565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d7a08-ccc1-4427-80fd-9d4cdf20bdcd None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:31.419891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d7a08-ccc1-4427-80fd-9d4cdf20bdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:31.460499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-428d7a08-ccc1-4427-80fd-9d4cdf20bdcd None None] MFA Rules not processed for user `404086c2409f4135af212d4047c29223`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:31.462041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-428d7a08-ccc1-4427-80fd-9d4cdf20bdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:31.475593 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1286/2600] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:32.819224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dd693-92e7-43b6-aa02-a47028f36f29 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:32.819647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dd693-92e7-43b6-aa02-a47028f36f29 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:32.819810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dd693-92e7-43b6-aa02-a47028f36f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:32.866489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3b4dd693-92e7-43b6-aa02-a47028f36f29 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:32.877838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3b4dd693-92e7-43b6-aa02-a47028f36f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:32.903942 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1315/2601] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:51:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:32.924228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d7a08-ccc1-4427-80fd-9d4cdf20bdcd None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:32.940438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d7a08-ccc1-4427-80fd-9d4cdf20bdcd None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:32.964429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22040c46-54c9-4bd4-9dad-3933642d06a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:32.965571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22040c46-54c9-4bd4-9dad-3933642d06a0 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:32.965571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22040c46-54c9-4bd4-9dad-3933642d06a0 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:32.965820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22040c46-54c9-4bd4-9dad-3933642d06a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:32.987723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22040c46-54c9-4bd4-9dad-3933642d06a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:32.990122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22040c46-54c9-4bd4-9dad-3933642d06a0 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:33.012225 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1287/2602] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:51:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:51:33.203907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dd693-92e7-43b6-aa02-a47028f36f29 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:33.302645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:33.357964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dd693-92e7-43b6-aa02-a47028f36f29 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:33.369280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:33.374266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:33.374567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:33.374880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] PATH_INFO: `/v3/users/404086c2409f4135af212d4047c29223` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:33.376593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=404086c2409f4135af212d4047c29223)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:33.379675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:33.485019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '404086c2409f4135af212d4047c29223'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:33.514760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '404086c2409f4135af212d4047c29223'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:33.531222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '404086c2409f4135af212d4047c29223'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:33.532009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '404086c2409f4135af212d4047c29223'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:33.532657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '404086c2409f4135af212d4047c29223'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:33.565007 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1316/2603] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:51:33 2022] DELETE /identity/v3/users/404086c2409f4135af212d4047c29223 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:33.569314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74320e5c-adc0-461f-bb94-8b1d3f9ac199 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:33.569717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74320e5c-adc0-461f-bb94-8b1d3f9ac199 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:33.570246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74320e5c-adc0-461f-bb94-8b1d3f9ac199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:33.637761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-74320e5c-adc0-461f-bb94-8b1d3f9ac199 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:33.641782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-74320e5c-adc0-461f-bb94-8b1d3f9ac199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:33.820466 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1288/2604] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:33.875602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:33.879741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:33.952834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916818a-64fe-4291-b22e-adcb2f072d93 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:33.953959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916818a-64fe-4291-b22e-adcb2f072d93 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:33.954454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916818a-64fe-4291-b22e-adcb2f072d93 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:33.956434 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1289/2605] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:51:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:51:33.963326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ef640-c2de-4ed1-ab44-9595d7206643 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:33.964039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ef640-c2de-4ed1-ab44-9595d7206643 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:33.965118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ef640-c2de-4ed1-ab44-9595d7206643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:34.019707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-740ef640-c2de-4ed1-ab44-9595d7206643 None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:34.021751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-740ef640-c2de-4ed1-ab44-9595d7206643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:34.051758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf73168-5804-4dd1-bd21-26205ea37cb1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:34.068163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df391039-a26a-4050-a02a-b7a641885246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:34.070617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df391039-a26a-4050-a02a-b7a641885246 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:34.071112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df391039-a26a-4050-a02a-b7a641885246 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:34.071409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df391039-a26a-4050-a02a-b7a641885246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:34.098099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df391039-a26a-4050-a02a-b7a641885246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:34.099629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df391039-a26a-4050-a02a-b7a641885246 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:34.127349 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1317/2606] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:34.204436 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1290/2607] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:51:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:51:34.277519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df391039-a26a-4050-a02a-b7a641885246 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:34.298299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df391039-a26a-4050-a02a-b7a641885246 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:34.317460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222c793-b59c-4d63-b3c7-cc3bcbbcefc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:34.318383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c793-b59c-4d63-b3c7-cc3bcbbcefc4 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:34.320717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c793-b59c-4d63-b3c7-cc3bcbbcefc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:34.322627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c793-b59c-4d63-b3c7-cc3bcbbcefc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:34.354683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222c793-b59c-4d63-b3c7-cc3bcbbcefc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:34.358230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222c793-b59c-4d63-b3c7-cc3bcbbcefc4 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:34.375962 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1318/2608] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:34.412695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86c05f-99ba-44f4-992d-bb33600e07c8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:34.413448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86c05f-99ba-44f4-992d-bb33600e07c8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:34.413889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86c05f-99ba-44f4-992d-bb33600e07c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:34.449730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1c86c05f-99ba-44f4-992d-bb33600e07c8 None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:34.451160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1c86c05f-99ba-44f4-992d-bb33600e07c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:34.469287 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1291/2609] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:51:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:34.498162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222c793-b59c-4d63-b3c7-cc3bcbbcefc4 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:34.513513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222c793-b59c-4d63-b3c7-cc3bcbbcefc4 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:34.521209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bf5f5-e854-41f3-84c7-73f507219aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:34.521969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bf5f5-e854-41f3-84c7-73f507219aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:34.522364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bf5f5-e854-41f3-84c7-73f507219aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:34.522710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bf5f5-e854-41f3-84c7-73f507219aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:34.531985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bf5f5-e854-41f3-84c7-73f507219aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:34.533629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bf5f5-e854-41f3-84c7-73f507219aa0 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:34.544218 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1319/2610] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:35.127100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86c05f-99ba-44f4-992d-bb33600e07c8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:35.134902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1c86c05f-99ba-44f4-992d-bb33600e07c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:35.160494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86c05f-99ba-44f4-992d-bb33600e07c8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:35.166026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:35.166722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:35.167073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:35.167397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd admin admin] PATH_INFO: `/v3/projects/cd0212e91da8419da9a74763d36a990d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:35.169006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd0212e91da8419da9a74763d36a990d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:35.170999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:35.227018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:35.239970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd0212e91da8419da9a74763d36a990d', 'request_id': 'req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:35.252212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd0212e91da8419da9a74763d36a990d', 'request_id': 'req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:35.261081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd0212e91da8419da9a74763d36a990d', 'request_id': 'req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:35.274597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd admin admin] The token cache is being invalidate because project cd0212e91da8419da9a74763d36a990d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:51:35.275348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:35.276296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba1f16b6-0e3a-4397-a518-e32609bcc1bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:35.277959 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1292/2611] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:51:35 2022] DELETE /identity/v3/projects/cd0212e91da8419da9a74763d36a990d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:35.283049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a631dc7-3953-40f6-b617-26c39fb00b45 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:35.283548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a631dc7-3953-40f6-b617-26c39fb00b45 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:35.283655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a631dc7-3953-40f6-b617-26c39fb00b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:35.308147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7a631dc7-3953-40f6-b617-26c39fb00b45 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:35.309471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7a631dc7-3953-40f6-b617-26c39fb00b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:35.348887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:35.383889 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1320/2612] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:35.392414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c5fd4-9173-4085-81e5-d52c9f6bd408 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:35.392730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c5fd4-9173-4085-81e5-d52c9f6bd408 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:35.393043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c5fd4-9173-4085-81e5-d52c9f6bd408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:35.414917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-109c5fd4-9173-4085-81e5-d52c9f6bd408 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:35.416144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-109c5fd4-9173-4085-81e5-d52c9f6bd408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:35.426659 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1293/2613] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:35.431117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a631dc7-3953-40f6-b617-26c39fb00b45 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:35.441551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a631dc7-3953-40f6-b617-26c39fb00b45 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:35.447074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e0633-d31d-451f-902b-c693175fa2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:35.447691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e0633-d31d-451f-902b-c693175fa2ca admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:35.448045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e0633-d31d-451f-902b-c693175fa2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:35.448434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e0633-d31d-451f-902b-c693175fa2ca admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:35.449266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e0633-d31d-451f-902b-c693175fa2ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:51:35.450324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e0633-d31d-451f-902b-c693175fa2ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:35.452574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e0633-d31d-451f-902b-c693175fa2ca admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:35.468705 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1321/2614] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:51:35.472991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c5fd4-9173-4085-81e5-d52c9f6bd408 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:35.495918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c5fd4-9173-4085-81e5-d52c9f6bd408 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:35.508413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca643a-af02-49a4-b72f-51bb5dba3c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:35.509182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca643a-af02-49a4-b72f-51bb5dba3c1d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:35.509590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca643a-af02-49a4-b72f-51bb5dba3c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:35.509940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca643a-af02-49a4-b72f-51bb5dba3c1d admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:35.510817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca643a-af02-49a4-b72f-51bb5dba3c1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:35.518297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca643a-af02-49a4-b72f-51bb5dba3c1d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:35.563609 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1294/2615] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:51:35 2022] POST /identity/v3/projects => generated 391 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:51:35.566631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e0633-d31d-451f-902b-c693175fa2ca admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:35.578691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e0633-d31d-451f-902b-c693175fa2ca admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:35.716403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa7d62-5a56-452f-85a1-3061c0c88e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:35.716627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa7d62-5a56-452f-85a1-3061c0c88e1b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:35.717090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa7d62-5a56-452f-85a1-3061c0c88e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:35.717372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa7d62-5a56-452f-85a1-3061c0c88e1b admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:35.718074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa7d62-5a56-452f-85a1-3061c0c88e1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:35.721060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa7d62-5a56-452f-85a1-3061c0c88e1b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:35.747866 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1322/2616] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:51:35 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:51:35.752099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca643a-af02-49a4-b72f-51bb5dba3c1d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:35.764192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca643a-af02-49a4-b72f-51bb5dba3c1d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:35.769223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31d6ae-acf7-41a2-ab0a-bd5f955b0dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:35.769846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31d6ae-acf7-41a2-ab0a-bd5f955b0dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:35.770131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31d6ae-acf7-41a2-ab0a-bd5f955b0dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:35.770410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31d6ae-acf7-41a2-ab0a-bd5f955b0dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:35.771066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31d6ae-acf7-41a2-ab0a-bd5f955b0dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:35.773040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31d6ae-acf7-41a2-ab0a-bd5f955b0dd6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:35.790976 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1295/2617] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:51:35.795286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa7d62-5a56-452f-85a1-3061c0c88e1b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:35.806162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa7d62-5a56-452f-85a1-3061c0c88e1b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:35.811189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc876bf4-6e61-40b3-a514-aad398cab852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:35.811817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc876bf4-6e61-40b3-a514-aad398cab852 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:35.812105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc876bf4-6e61-40b3-a514-aad398cab852 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:35.812387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc876bf4-6e61-40b3-a514-aad398cab852 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:35.813108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc876bf4-6e61-40b3-a514-aad398cab852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:35.814866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc876bf4-6e61-40b3-a514-aad398cab852 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:35.832175 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1323/2618] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:51:35.836295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31d6ae-acf7-41a2-ab0a-bd5f955b0dd6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:35.847801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31d6ae-acf7-41a2-ab0a-bd5f955b0dd6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:35.853983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b05ed0-4d28-49f7-8534-b12938485f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:35.856658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b05ed0-4d28-49f7-8534-b12938485f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:35.857048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b05ed0-4d28-49f7-8534-b12938485f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:35.857404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b05ed0-4d28-49f7-8534-b12938485f7e admin admin] PATH_INFO: `/v3/projects/47d6e91988314f84866964e998922059/users/9342065f3fb246008d09a2eed524ac22/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:35.883476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b05ed0-4d28-49f7-8534-b12938485f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d6e91988314f84866964e998922059, user_id=9342065f3fb246008d09a2eed524ac22, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:35.886909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b05ed0-4d28-49f7-8534-b12938485f7e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:35.909827 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1296/2619] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:51:35 2022] PUT /identity/v3/projects/47d6e91988314f84866964e998922059/users/9342065f3fb246008d09a2eed524ac22/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:51:35.914810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d66c9-a8ca-4cd5-abc3-06efc298025a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:35.915295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d66c9-a8ca-4cd5-abc3-06efc298025a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:35.915577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d66c9-a8ca-4cd5-abc3-06efc298025a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:35.949182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2c7d66c9-a8ca-4cd5-abc3-06efc298025a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:35.950729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2c7d66c9-a8ca-4cd5-abc3-06efc298025a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:36.038744 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1324/2620] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:36.049292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b05ed0-4d28-49f7-8534-b12938485f7e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:36.051786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-98b05ed0-4d28-49f7-8534-b12938485f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:36.121053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b05ed0-4d28-49f7-8534-b12938485f7e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:36.124831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c41e1a-569c-4892-8ff1-00db10c80d0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:36.126093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c41e1a-569c-4892-8ff1-00db10c80d0b service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:36.126387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c41e1a-569c-4892-8ff1-00db10c80d0b service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:36.126664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c41e1a-569c-4892-8ff1-00db10c80d0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:36.137028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c41e1a-569c-4892-8ff1-00db10c80d0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:36.139099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c41e1a-569c-4892-8ff1-00db10c80d0b service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:36.152663 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1297/2621] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:36.324780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d66c9-a8ca-4cd5-abc3-06efc298025a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:36.326810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2c7d66c9-a8ca-4cd5-abc3-06efc298025a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:36.405961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:36.459610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d66c9-a8ca-4cd5-abc3-06efc298025a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:36.468042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad611a-b69c-4d9b-8576-1011ab5ecdbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:36.468857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad611a-b69c-4d9b-8576-1011ab5ecdbd service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:36.469189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad611a-b69c-4d9b-8576-1011ab5ecdbd service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:36.469477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad611a-b69c-4d9b-8576-1011ab5ecdbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:36.471458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e9ad611a-b69c-4d9b-8576-1011ab5ecdbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:36.558147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad611a-b69c-4d9b-8576-1011ab5ecdbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:36.560096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad611a-b69c-4d9b-8576-1011ab5ecdbd service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:36.572705 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1325/2622] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:36 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:45.688639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d2c51-a3e9-48c8-9294-0da28a8ec330 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:45.689264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d2c51-a3e9-48c8-9294-0da28a8ec330 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:45.689458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d2c51-a3e9-48c8-9294-0da28a8ec330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:45.710326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ca4d2c51-a3e9-48c8-9294-0da28a8ec330 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:45.712181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ca4d2c51-a3e9-48c8-9294-0da28a8ec330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:45.805372 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1298/2623] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:51:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:45.823704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad611a-b69c-4d9b-8576-1011ab5ecdbd service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:45.826415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e9ad611a-b69c-4d9b-8576-1011ab5ecdbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:45.953619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad611a-b69c-4d9b-8576-1011ab5ecdbd service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:45.976965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da760dd-0625-4f35-b7be-926a3a5ec121 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:45.978430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da760dd-0625-4f35-b7be-926a3a5ec121 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:45.978430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da760dd-0625-4f35-b7be-926a3a5ec121 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:45.978971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da760dd-0625-4f35-b7be-926a3a5ec121 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:45.997137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da760dd-0625-4f35-b7be-926a3a5ec121 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:46.001263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da760dd-0625-4f35-b7be-926a3a5ec121 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:46.028069 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1326/2624] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:51:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:51:46.851730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27879f5-670d-4be3-8699-895ab3e47482 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:46.852151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27879f5-670d-4be3-8699-895ab3e47482 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:46.852304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27879f5-670d-4be3-8699-895ab3e47482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:46.955170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:46.979103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c27879f5-670d-4be3-8699-895ab3e47482 None None] MFA Rules not processed for user `9342065f3fb246008d09a2eed524ac22`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:46.981365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c27879f5-670d-4be3-8699-895ab3e47482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:47.148983 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1299/2625] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:47.285294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da760dd-0625-4f35-b7be-926a3a5ec121 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:47.287094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7da760dd-0625-4f35-b7be-926a3a5ec121 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:47.331425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da760dd-0625-4f35-b7be-926a3a5ec121 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:47.349864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d1bec-4e5e-4a94-abd4-020f58c59f8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:47.350913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d1bec-4e5e-4a94-abd4-020f58c59f8a service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:47.351561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d1bec-4e5e-4a94-abd4-020f58c59f8a service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:47.352525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d1bec-4e5e-4a94-abd4-020f58c59f8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:47.366699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d1bec-4e5e-4a94-abd4-020f58c59f8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:47.371542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d1bec-4e5e-4a94-abd4-020f58c59f8a service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:47.396160 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1327/2626] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:53.451451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19289425-0ba7-46a5-bc23-16c5b9cff569 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:53.451987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19289425-0ba7-46a5-bc23-16c5b9cff569 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:53.452797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19289425-0ba7-46a5-bc23-16c5b9cff569 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:53.454639 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1300/2627] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 3 20:51:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:51:53.468659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af812c5-0dcc-43a0-83fb-fb2411ab7ad2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:53.469660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af812c5-0dcc-43a0-83fb-fb2411ab7ad2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:53.473264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af812c5-0dcc-43a0-83fb-fb2411ab7ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:53.520529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2af812c5-0dcc-43a0-83fb-fb2411ab7ad2 None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:53.523787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2af812c5-0dcc-43a0-83fb-fb2411ab7ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:53.552753 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1328/2628] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 20:51:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:51:53.651506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19289425-0ba7-46a5-bc23-16c5b9cff569 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:53.653807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-19289425-0ba7-46a5-bc23-16c5b9cff569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:53.676723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19289425-0ba7-46a5-bc23-16c5b9cff569 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:53.689492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92fdb4-6209-4524-9144-e061c6a3430e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:53.691562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92fdb4-6209-4524-9144-e061c6a3430e service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:53.694024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92fdb4-6209-4524-9144-e061c6a3430e service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:53.694573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92fdb4-6209-4524-9144-e061c6a3430e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:53.724095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92fdb4-6209-4524-9144-e061c6a3430e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:53.726138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92fdb4-6209-4524-9144-e061c6a3430e service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:53.759798 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1301/2629] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:54.711983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af812c5-0dcc-43a0-83fb-fb2411ab7ad2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:54.714489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2af812c5-0dcc-43a0-83fb-fb2411ab7ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:54.850284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:54.925712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af812c5-0dcc-43a0-83fb-fb2411ab7ad2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:54.942364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62cf95-f380-4180-aaa8-13c9226192ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:54.943343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62cf95-f380-4180-aaa8-13c9226192ee service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:54.943814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62cf95-f380-4180-aaa8-13c9226192ee service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:54.944248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62cf95-f380-4180-aaa8-13c9226192ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:54.946436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9d62cf95-f380-4180-aaa8-13c9226192ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:54.977418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62cf95-f380-4180-aaa8-13c9226192ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:54.980377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62cf95-f380-4180-aaa8-13c9226192ee service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:55.003025 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1329/2630] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:56.990358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92fdb4-6209-4524-9144-e061c6a3430e service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:56.993311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fd92fdb4-6209-4524-9144-e061c6a3430e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:57.026650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92fdb4-6209-4524-9144-e061c6a3430e service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:57.049498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ce9c6-4d04-42fd-9743-34d7afcd0fd0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:57.050265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ce9c6-4d04-42fd-9743-34d7afcd0fd0 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:57.051722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ce9c6-4d04-42fd-9743-34d7afcd0fd0 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:57.055787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ce9c6-4d04-42fd-9743-34d7afcd0fd0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:57.074101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-411ce9c6-4d04-42fd-9743-34d7afcd0fd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:57.133597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ce9c6-4d04-42fd-9743-34d7afcd0fd0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:57.139554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ce9c6-4d04-42fd-9743-34d7afcd0fd0 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:57.180417 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1302/2631] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:51:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:51:58.994745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62cf95-f380-4180-aaa8-13c9226192ee service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:59.008921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62cf95-f380-4180-aaa8-13c9226192ee service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:59.030096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:59.033143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:59.037156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:59.037652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] PATH_INFO: `/v3/users/9342065f3fb246008d09a2eed524ac22` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:59.039356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9342065f3fb246008d09a2eed524ac22)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:59.042222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:59.151605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9342065f3fb246008d09a2eed524ac22'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.180285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9342065f3fb246008d09a2eed524ac22'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.193638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9342065f3fb246008d09a2eed524ac22'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.194228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9342065f3fb246008d09a2eed524ac22'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.194989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9342065f3fb246008d09a2eed524ac22'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.231493 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1330/2632] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:51:58 2022] DELETE /identity/v3/users/9342065f3fb246008d09a2eed524ac22 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:59.417240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ce9c6-4d04-42fd-9743-34d7afcd0fd0 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:59.418805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-411ce9c6-4d04-42fd-9743-34d7afcd0fd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:59.445690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ce9c6-4d04-42fd-9743-34d7afcd0fd0 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:59.609370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb0538-1fce-4359-ac25-dfb70d632b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:59.609370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb0538-1fce-4359-ac25-dfb70d632b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:59.609370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb0538-1fce-4359-ac25-dfb70d632b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:59.609370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb0538-1fce-4359-ac25-dfb70d632b63 admin admin] PATH_INFO: `/v3/users/1c73e775740c40c4a9c088ceddaaf0d8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:59.610288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb0538-1fce-4359-ac25-dfb70d632b63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c73e775740c40c4a9c088ceddaaf0d8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:59.610288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb0538-1fce-4359-ac25-dfb70d632b63 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:59.672719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:51:59.675179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:51:59.698743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53fb0538-1fce-4359-ac25-dfb70d632b63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c73e775740c40c4a9c088ceddaaf0d8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.705284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaea00-b41f-42d6-903c-ee364eb14a87 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:51:59.712348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd550ce-7fd0-4985-8fda-71ee35a274c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:51:59.715827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd550ce-7fd0-4985-8fda-71ee35a274c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:59.716999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd550ce-7fd0-4985-8fda-71ee35a274c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:59.718007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd550ce-7fd0-4985-8fda-71ee35a274c0 admin admin] PATH_INFO: `/v3/projects/47d6e91988314f84866964e998922059` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:59.721002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd550ce-7fd0-4985-8fda-71ee35a274c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47d6e91988314f84866964e998922059)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:51:59.721316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53fb0538-1fce-4359-ac25-dfb70d632b63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c73e775740c40c4a9c088ceddaaf0d8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.722064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53fb0538-1fce-4359-ac25-dfb70d632b63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c73e775740c40c4a9c088ceddaaf0d8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.726014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd550ce-7fd0-4985-8fda-71ee35a274c0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:51:59.731599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53fb0538-1fce-4359-ac25-dfb70d632b63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c73e775740c40c4a9c088ceddaaf0d8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.734990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:51:59.745651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53fb0538-1fce-4359-ac25-dfb70d632b63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c73e775740c40c4a9c088ceddaaf0d8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.759787 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1303/2633] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:51:59 2022] DELETE /identity/v3/users/1c73e775740c40c4a9c088ceddaaf0d8 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:59.899231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7fd550ce-7fd0-4985-8fda-71ee35a274c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47d6e91988314f84866964e998922059', 'request_id': 'req-7fd550ce-7fd0-4985-8fda-71ee35a274c0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.909604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7fd550ce-7fd0-4985-8fda-71ee35a274c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47d6e91988314f84866964e998922059', 'request_id': 'req-7fd550ce-7fd0-4985-8fda-71ee35a274c0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.920320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7fd550ce-7fd0-4985-8fda-71ee35a274c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47d6e91988314f84866964e998922059', 'request_id': 'req-7fd550ce-7fd0-4985-8fda-71ee35a274c0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.939600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7fd550ce-7fd0-4985-8fda-71ee35a274c0 admin admin] The token cache is being invalidate because project 47d6e91988314f84866964e998922059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:51:59.939967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7fd550ce-7fd0-4985-8fda-71ee35a274c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.940788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7fd550ce-7fd0-4985-8fda-71ee35a274c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:51:59.943026 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1331/2634] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:51:59 2022] DELETE /identity/v3/projects/47d6e91988314f84866964e998922059 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:51:59.949486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e3559-384a-4053-aae8-ce36298f6fa2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:51:59.949988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e3559-384a-4053-aae8-ce36298f6fa2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:51:59.951176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e3559-384a-4053-aae8-ce36298f6fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:51:59.976971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7c5e3559-384a-4053-aae8-ce36298f6fa2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:51:59.979296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7c5e3559-384a-4053-aae8-ce36298f6fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:00.125173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855a722-711d-407c-9308-2db497b45f76 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:00.125722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855a722-711d-407c-9308-2db497b45f76 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:00.126094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855a722-711d-407c-9308-2db497b45f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:00.130862 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1304/2635] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:00.163397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8855a722-711d-407c-9308-2db497b45f76 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:00.165484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8855a722-711d-407c-9308-2db497b45f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:00.176815 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1332/2636] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:00.180808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e3559-384a-4053-aae8-ce36298f6fa2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:00.192467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e3559-384a-4053-aae8-ce36298f6fa2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:00.206149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d96d3-74dc-43ce-95b9-e2f5059bae80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:00.207041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d96d3-74dc-43ce-95b9-e2f5059bae80 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:00.207336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d96d3-74dc-43ce-95b9-e2f5059bae80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:00.207614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d96d3-74dc-43ce-95b9-e2f5059bae80 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:00.208277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d96d3-74dc-43ce-95b9-e2f5059bae80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:52:00.208584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d96d3-74dc-43ce-95b9-e2f5059bae80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:00.210236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d96d3-74dc-43ce-95b9-e2f5059bae80 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:00.228938 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1305/2637] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:52:00.233114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855a722-711d-407c-9308-2db497b45f76 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:00.245147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855a722-711d-407c-9308-2db497b45f76 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:00.251068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a5ef3-c6fa-4888-8b70-d87ba253eac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:00.252064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a5ef3-c6fa-4888-8b70-d87ba253eac0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:00.252605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a5ef3-c6fa-4888-8b70-d87ba253eac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:00.253720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a5ef3-c6fa-4888-8b70-d87ba253eac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:00.255066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a5ef3-c6fa-4888-8b70-d87ba253eac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:00.257320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a5ef3-c6fa-4888-8b70-d87ba253eac0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:00.288419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:00.305519 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1333/2638] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:52:00 2022] POST /identity/v3/projects => generated 365 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:52:00.309886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d96d3-74dc-43ce-95b9-e2f5059bae80 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:00.332391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d96d3-74dc-43ce-95b9-e2f5059bae80 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:00.499501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f52b9-8048-4f14-b3a4-36f7b710cb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:00.501072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f52b9-8048-4f14-b3a4-36f7b710cb96 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:00.501518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f52b9-8048-4f14-b3a4-36f7b710cb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:00.505263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f52b9-8048-4f14-b3a4-36f7b710cb96 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:00.506296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f52b9-8048-4f14-b3a4-36f7b710cb96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:00.506624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a5ef3-c6fa-4888-8b70-d87ba253eac0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:00.508831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7c1a5ef3-c6fa-4888-8b70-d87ba253eac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:00.509173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f52b9-8048-4f14-b3a4-36f7b710cb96 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:00.538209 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1306/2639] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:52:00 2022] POST /identity/v3/users => generated 283 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:52:00.539727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a5ef3-c6fa-4888-8b70-d87ba253eac0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:00.543262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f52b9-8048-4f14-b3a4-36f7b710cb96 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:00.550992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db4693-79ee-4d9c-812d-e6b315e6ae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:00.552634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db4693-79ee-4d9c-812d-e6b315e6ae2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:00.553122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db4693-79ee-4d9c-812d-e6b315e6ae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:00.553565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db4693-79ee-4d9c-812d-e6b315e6ae2f admin admin] PATH_INFO: `/v3/projects/bcbaf703d0064704a34a858adf2a33fa` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:00.555111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db4693-79ee-4d9c-812d-e6b315e6ae2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcbaf703d0064704a34a858adf2a33fa)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:00.558279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db4693-79ee-4d9c-812d-e6b315e6ae2f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:00.590744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f52b9-8048-4f14-b3a4-36f7b710cb96 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:00.601222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e757ec5-c9a1-42dc-9f3b-fc0edbb2b7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:00.601860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757ec5-c9a1-42dc-9f3b-fc0edbb2b7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:00.602293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757ec5-c9a1-42dc-9f3b-fc0edbb2b7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:00.602675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757ec5-c9a1-42dc-9f3b-fc0edbb2b7e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:00.603491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e757ec5-c9a1-42dc-9f3b-fc0edbb2b7e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:00.605488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e757ec5-c9a1-42dc-9f3b-fc0edbb2b7e5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:00.623266 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1307/2640] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:52:00.624214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e757ec5-c9a1-42dc-9f3b-fc0edbb2b7e5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:00.633946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-10db4693-79ee-4d9c-812d-e6b315e6ae2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcbaf703d0064704a34a858adf2a33fa', 'request_id': 'req-10db4693-79ee-4d9c-812d-e6b315e6ae2f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:00.636600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e757ec5-c9a1-42dc-9f3b-fc0edbb2b7e5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:00.663094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-10db4693-79ee-4d9c-812d-e6b315e6ae2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcbaf703d0064704a34a858adf2a33fa', 'request_id': 'req-10db4693-79ee-4d9c-812d-e6b315e6ae2f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:00.679941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-10db4693-79ee-4d9c-812d-e6b315e6ae2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcbaf703d0064704a34a858adf2a33fa', 'request_id': 'req-10db4693-79ee-4d9c-812d-e6b315e6ae2f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:00.698427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-10db4693-79ee-4d9c-812d-e6b315e6ae2f admin admin] The token cache is being invalidate because project bcbaf703d0064704a34a858adf2a33fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:52:00.698888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-10db4693-79ee-4d9c-812d-e6b315e6ae2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:00.699792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-10db4693-79ee-4d9c-812d-e6b315e6ae2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:00.708360 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1334/2641] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:52:00 2022] DELETE /identity/v3/projects/bcbaf703d0064704a34a858adf2a33fa => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:52:00.711176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcbd5a-0c6a-4db6-bbd0-0bfd61b212e5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:00.711744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcbd5a-0c6a-4db6-bbd0-0bfd61b212e5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:00.712776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcbd5a-0c6a-4db6-bbd0-0bfd61b212e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:00.733674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:00.742718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d6bcbd5a-0c6a-4db6-bbd0-0bfd61b212e5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:00.743887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d6bcbd5a-0c6a-4db6-bbd0-0bfd61b212e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:00.746558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ce428-2346-4175-94e4-d6686d5115ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:00.748065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ce428-2346-4175-94e4-d6686d5115ce admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:00.748660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ce428-2346-4175-94e4-d6686d5115ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:00.749233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ce428-2346-4175-94e4-d6686d5115ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:00.750667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ce428-2346-4175-94e4-d6686d5115ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:00.753236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ce428-2346-4175-94e4-d6686d5115ce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:00.753426 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1335/2642] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:00.763701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1930f2-de25-43e7-993d-46e0d16e2478 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:00.763992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1930f2-de25-43e7-993d-46e0d16e2478 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:00.764276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1930f2-de25-43e7-993d-46e0d16e2478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:00.768965 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1308/2643] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:52:00.770444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ce428-2346-4175-94e4-d6686d5115ce admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:00.772469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b34ce428-2346-4175-94e4-d6686d5115ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:00.791250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ce428-2346-4175-94e4-d6686d5115ce admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:00.796199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fa1930f2-de25-43e7-993d-46e0d16e2478 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:00.797495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fa1930f2-de25-43e7-993d-46e0d16e2478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:00.798720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0c452-2d1f-4fc6-b4e3-94b53a0a3027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:00.799821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0c452-2d1f-4fc6-b4e3-94b53a0a3027 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:00.800453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0c452-2d1f-4fc6-b4e3-94b53a0a3027 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:00.800989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0c452-2d1f-4fc6-b4e3-94b53a0a3027 admin admin] PATH_INFO: `/v3/projects/f17cee6ff91f4292b688b5cc63292039/users/d9688dca94a841cfa29420ad780562cd/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:00.808816 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1336/2644] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:00.813709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1930f2-de25-43e7-993d-46e0d16e2478 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:00.821134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0c452-2d1f-4fc6-b4e3-94b53a0a3027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f17cee6ff91f4292b688b5cc63292039, user_id=d9688dca94a841cfa29420ad780562cd, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:00.825996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0c452-2d1f-4fc6-b4e3-94b53a0a3027 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:00.831370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1930f2-de25-43e7-993d-46e0d16e2478 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:00.852714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054ccc4-a002-4bbd-bb5a-89b82ff1ab02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:00.854751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054ccc4-a002-4bbd-bb5a-89b82ff1ab02 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:00.855703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054ccc4-a002-4bbd-bb5a-89b82ff1ab02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:00.856209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054ccc4-a002-4bbd-bb5a-89b82ff1ab02 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:00.857505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054ccc4-a002-4bbd-bb5a-89b82ff1ab02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:52:00.858043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054ccc4-a002-4bbd-bb5a-89b82ff1ab02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:00.863244 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1309/2645] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:52:00 2022] PUT /identity/v3/projects/f17cee6ff91f4292b688b5cc63292039/users/d9688dca94a841cfa29420ad780562cd/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:00.863441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054ccc4-a002-4bbd-bb5a-89b82ff1ab02 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:00.864934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78eb32-8b20-4eb8-9028-5d1a41d88373 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:00.866519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78eb32-8b20-4eb8-9028-5d1a41d88373 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:00.866519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78eb32-8b20-4eb8-9028-5d1a41d88373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:00.888077 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1337/2646] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:52:00.892117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054ccc4-a002-4bbd-bb5a-89b82ff1ab02 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:00.899241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9d78eb32-8b20-4eb8-9028-5d1a41d88373 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:00.901091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9d78eb32-8b20-4eb8-9028-5d1a41d88373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:00.908996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054ccc4-a002-4bbd-bb5a-89b82ff1ab02 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:01.040943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185bd37-a179-4df8-9085-141a8f43b9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:01.045587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185bd37-a179-4df8-9085-141a8f43b9d8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:01.045587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185bd37-a179-4df8-9085-141a8f43b9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:01.045587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185bd37-a179-4df8-9085-141a8f43b9d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:01.045915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185bd37-a179-4df8-9085-141a8f43b9d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:01.049729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185bd37-a179-4df8-9085-141a8f43b9d8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:01.086066 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1310/2647] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:01.102475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78eb32-8b20-4eb8-9028-5d1a41d88373 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:01.105400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9d78eb32-8b20-4eb8-9028-5d1a41d88373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:01.116683 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1338/2648] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:52:00 2022] POST /identity/v3/projects => generated 403 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:52:01.120128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185bd37-a179-4df8-9085-141a8f43b9d8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:01.205876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:01.256161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185bd37-a179-4df8-9085-141a8f43b9d8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:01.262754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87ac53-45d3-4560-bb1f-cefd581bef8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:01.263420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87ac53-45d3-4560-bb1f-cefd581bef8c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:01.263864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87ac53-45d3-4560-bb1f-cefd581bef8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:01.264484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87ac53-45d3-4560-bb1f-cefd581bef8c admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:01.266334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87ac53-45d3-4560-bb1f-cefd581bef8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:01.268993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87ac53-45d3-4560-bb1f-cefd581bef8c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:01.277073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78eb32-8b20-4eb8-9028-5d1a41d88373 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:01.308868 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1339/2649] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:52:01 2022] POST /identity/v3/users => generated 302 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:52:01.313092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87ac53-45d3-4560-bb1f-cefd581bef8c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:01.332679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87ac53-45d3-4560-bb1f-cefd581bef8c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:01.349564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c53dd-733d-4ae3-93ad-6d91f0254ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:01.350245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c53dd-733d-4ae3-93ad-6d91f0254ffc admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:01.350544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c53dd-733d-4ae3-93ad-6d91f0254ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:01.350826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c53dd-733d-4ae3-93ad-6d91f0254ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:01.351486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c53dd-733d-4ae3-93ad-6d91f0254ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:01.353149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c53dd-733d-4ae3-93ad-6d91f0254ffc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:01.358555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806202f1-a08c-4253-8899-639fed39ad1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:01.359394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806202f1-a08c-4253-8899-639fed39ad1f service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:01.359752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806202f1-a08c-4253-8899-639fed39ad1f service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:01.360123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806202f1-a08c-4253-8899-639fed39ad1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:01.372113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806202f1-a08c-4253-8899-639fed39ad1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:01.373725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806202f1-a08c-4253-8899-639fed39ad1f service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:01.379334 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1340/2650] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:52:01.386216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c53dd-733d-4ae3-93ad-6d91f0254ffc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:01.390907 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1311/2651] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:52:01.408082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c53dd-733d-4ae3-93ad-6d91f0254ffc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:01.422280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce9447-b8d0-4fdc-a38f-017fea71db83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:01.423037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce9447-b8d0-4fdc-a38f-017fea71db83 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:01.423399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce9447-b8d0-4fdc-a38f-017fea71db83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:01.423748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce9447-b8d0-4fdc-a38f-017fea71db83 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:01.424626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce9447-b8d0-4fdc-a38f-017fea71db83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:01.427430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce9447-b8d0-4fdc-a38f-017fea71db83 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:01.459248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806202f1-a08c-4253-8899-639fed39ad1f service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:01.460577 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1341/2652] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:52:01.479078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806202f1-a08c-4253-8899-639fed39ad1f service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:01.497216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea20ab1-43b0-4f75-a09a-98c60a6eb3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:01.498032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea20ab1-43b0-4f75-a09a-98c60a6eb3bc admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:01.499296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea20ab1-43b0-4f75-a09a-98c60a6eb3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:01.499667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea20ab1-43b0-4f75-a09a-98c60a6eb3bc admin admin] PATH_INFO: `/v3/projects/e44f80b00bc14282af09e08925c79c7d/users/b52bd1dc24db4078b99839e483934234/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:01.539233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea20ab1-43b0-4f75-a09a-98c60a6eb3bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e44f80b00bc14282af09e08925c79c7d, user_id=b52bd1dc24db4078b99839e483934234, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:01.543018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea20ab1-43b0-4f75-a09a-98c60a6eb3bc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:01.565609 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1312/2653] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:52:01 2022] PUT /identity/v3/projects/e44f80b00bc14282af09e08925c79c7d/users/b52bd1dc24db4078b99839e483934234/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:52:01.566837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786588e-879c-4c0e-a22f-5f368fe67b3b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:01.567130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786588e-879c-4c0e-a22f-5f368fe67b3b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:01.567419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786588e-879c-4c0e-a22f-5f368fe67b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:01.612972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d786588e-879c-4c0e-a22f-5f368fe67b3b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:01.614163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d786588e-879c-4c0e-a22f-5f368fe67b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:01.727176 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1342/2654] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:01.744950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea20ab1-43b0-4f75-a09a-98c60a6eb3bc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:01.758847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:01.952333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea20ab1-43b0-4f75-a09a-98c60a6eb3bc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:01.968599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cc34f-80ff-4e5d-8258-ac5df6e0a8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:01.969390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cc34f-80ff-4e5d-8258-ac5df6e0a8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:01.969682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cc34f-80ff-4e5d-8258-ac5df6e0a8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:01.970061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cc34f-80ff-4e5d-8258-ac5df6e0a8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:01.986185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cc34f-80ff-4e5d-8258-ac5df6e0a8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:01.988181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cc34f-80ff-4e5d-8258-ac5df6e0a8e9 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:02.000373 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1313/2655] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:52:02.420370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521244d4-061d-4440-9200-342ac69dd014 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:02.420992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521244d4-061d-4440-9200-342ac69dd014 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:02.421224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521244d4-061d-4440-9200-342ac69dd014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:02.451020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-521244d4-061d-4440-9200-342ac69dd014 None None] MFA Rules not processed for user `90d7e4c2e577467f949be3dc480cc7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:02.452515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-521244d4-061d-4440-9200-342ac69dd014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:02.527680 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1343/2656] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:10.740194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cc34f-80ff-4e5d-8258-ac5df6e0a8e9 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:10.741731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fd4cc34f-80ff-4e5d-8258-ac5df6e0a8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:10.778982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cc34f-80ff-4e5d-8258-ac5df6e0a8e9 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:10.800333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3de12a-e954-46d6-bdf0-aed09665a2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:10.801983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3de12a-e954-46d6-bdf0-aed09665a2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:10.803856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3de12a-e954-46d6-bdf0-aed09665a2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:10.804404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3de12a-e954-46d6-bdf0-aed09665a2f6 admin admin] PATH_INFO: `/v3/users/10bbfca2d8744acea2587037f190fe90` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:10.808125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3de12a-e954-46d6-bdf0-aed09665a2f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10bbfca2d8744acea2587037f190fe90)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:10.812011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3de12a-e954-46d6-bdf0-aed09665a2f6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:10.936835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb3de12a-e954-46d6-bdf0-aed09665a2f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10bbfca2d8744acea2587037f190fe90'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:10.988312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb3de12a-e954-46d6-bdf0-aed09665a2f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10bbfca2d8744acea2587037f190fe90'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:10.989404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb3de12a-e954-46d6-bdf0-aed09665a2f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10bbfca2d8744acea2587037f190fe90'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:11.011836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb3de12a-e954-46d6-bdf0-aed09665a2f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10bbfca2d8744acea2587037f190fe90'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:11.023707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb3de12a-e954-46d6-bdf0-aed09665a2f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10bbfca2d8744acea2587037f190fe90'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:11.032988 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1314/2657] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:52:10 2022] DELETE /identity/v3/users/10bbfca2d8744acea2587037f190fe90 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:11.809795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521244d4-061d-4440-9200-342ac69dd014 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:11.824997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521244d4-061d-4440-9200-342ac69dd014 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:11.958432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:11.972707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa6745-9654-4416-b09c-970be455b78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:11.973282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa6745-9654-4416-b09c-970be455b78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:11.973561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa6745-9654-4416-b09c-970be455b78d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:11.981894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa6745-9654-4416-b09c-970be455b78d admin admin] PATH_INFO: `/v3/projects/d196cef49e5040dea8885bcd8f179747` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:11.983719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa6745-9654-4416-b09c-970be455b78d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d196cef49e5040dea8885bcd8f179747)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:11.985958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa6745-9654-4416-b09c-970be455b78d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:12.074892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b2aa6745-9654-4416-b09c-970be455b78d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd196cef49e5040dea8885bcd8f179747', 'request_id': 'req-b2aa6745-9654-4416-b09c-970be455b78d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:12.104770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b2aa6745-9654-4416-b09c-970be455b78d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd196cef49e5040dea8885bcd8f179747', 'request_id': 'req-b2aa6745-9654-4416-b09c-970be455b78d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:12.115235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b2aa6745-9654-4416-b09c-970be455b78d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd196cef49e5040dea8885bcd8f179747', 'request_id': 'req-b2aa6745-9654-4416-b09c-970be455b78d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:12.145515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b2aa6745-9654-4416-b09c-970be455b78d admin admin] The token cache is being invalidate because project d196cef49e5040dea8885bcd8f179747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:52:12.146078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b2aa6745-9654-4416-b09c-970be455b78d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:12.146927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b2aa6745-9654-4416-b09c-970be455b78d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:12.150005 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1344/2658] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:52:11 2022] DELETE /identity/v3/projects/d196cef49e5040dea8885bcd8f179747 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:12.156192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ff41b-9806-494b-8b9a-39d596d53339 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:12.157526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ff41b-9806-494b-8b9a-39d596d53339 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:12.158049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ff41b-9806-494b-8b9a-39d596d53339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:12.184869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-422ff41b-9806-494b-8b9a-39d596d53339 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:12.186743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-422ff41b-9806-494b-8b9a-39d596d53339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:12.306182 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1315/2659] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:12.315187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9cb3c-cfcc-4731-b727-a035091880bb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:12.315490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9cb3c-cfcc-4731-b727-a035091880bb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:12.315776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9cb3c-cfcc-4731-b727-a035091880bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:12.341827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-91c9cb3c-cfcc-4731-b727-a035091880bb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:12.344566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-91c9cb3c-cfcc-4731-b727-a035091880bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:12.358989 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1345/2660] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:12.363375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ff41b-9806-494b-8b9a-39d596d53339 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:12.374841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ff41b-9806-494b-8b9a-39d596d53339 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:12.386039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f340e7-723a-430e-9bda-bab2ec2a6aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:12.387060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f340e7-723a-430e-9bda-bab2ec2a6aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:12.387568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f340e7-723a-430e-9bda-bab2ec2a6aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:12.388171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f340e7-723a-430e-9bda-bab2ec2a6aa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:12.390077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f340e7-723a-430e-9bda-bab2ec2a6aa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:52:12.390634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f340e7-723a-430e-9bda-bab2ec2a6aa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:12.394207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f340e7-723a-430e-9bda-bab2ec2a6aa7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:12.414599 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1316/2661] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:52:12.419334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9cb3c-cfcc-4731-b727-a035091880bb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:12.435820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9cb3c-cfcc-4731-b727-a035091880bb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:12.449583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bd9d8-5d11-43b5-b5a8-c6d48df343f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:12.450786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bd9d8-5d11-43b5-b5a8-c6d48df343f4 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:12.452048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bd9d8-5d11-43b5-b5a8-c6d48df343f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:12.452614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bd9d8-5d11-43b5-b5a8-c6d48df343f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:12.455355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139bd9d8-5d11-43b5-b5a8-c6d48df343f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:12.459097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139bd9d8-5d11-43b5-b5a8-c6d48df343f4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:12.546103 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1346/2662] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:52:12 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:52:12.550450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f340e7-723a-430e-9bda-bab2ec2a6aa7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:12.579677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f340e7-723a-430e-9bda-bab2ec2a6aa7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:12.586752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2c413-0885-4f02-98e9-c49fe948ecdd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:12.587153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2c413-0885-4f02-98e9-c49fe948ecdd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:12.587536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2c413-0885-4f02-98e9-c49fe948ecdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:12.594211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:12.684248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bef2c413-0885-4f02-98e9-c49fe948ecdd None None] MFA Rules not processed for user `d9688dca94a841cfa29420ad780562cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:12.686052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bef2c413-0885-4f02-98e9-c49fe948ecdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:12.805735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c4c49-66e0-4ae4-b3d3-a43ff63537f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:12.806371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c4c49-66e0-4ae4-b3d3-a43ff63537f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:12.807291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c4c49-66e0-4ae4-b3d3-a43ff63537f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:12.807734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c4c49-66e0-4ae4-b3d3-a43ff63537f5 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:12.808531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c4c49-66e0-4ae4-b3d3-a43ff63537f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:12.817429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c4c49-66e0-4ae4-b3d3-a43ff63537f5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:12.863295 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1347/2663] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:12.875000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851b6e1-dd33-4825-b41d-4cc46580b1bd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:12.877218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851b6e1-dd33-4825-b41d-4cc46580b1bd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:12.877749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851b6e1-dd33-4825-b41d-4cc46580b1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:12.886340 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1317/2664] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:52:12 2022] POST /identity/v3/users => generated 291 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:52:12.891819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c4c49-66e0-4ae4-b3d3-a43ff63537f5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:12.913573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c4c49-66e0-4ae4-b3d3-a43ff63537f5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:12.927419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b4199-b0b0-41e5-b5ad-6cbeed97ae40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:12.927817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b4199-b0b0-41e5-b5ad-6cbeed97ae40 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:12.928107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b4199-b0b0-41e5-b5ad-6cbeed97ae40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:12.928389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b4199-b0b0-41e5-b5ad-6cbeed97ae40 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:12.929129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b4199-b0b0-41e5-b5ad-6cbeed97ae40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:12.930806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b4199-b0b0-41e5-b5ad-6cbeed97ae40 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:12.932882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2851b6e1-dd33-4825-b41d-4cc46580b1bd None None] MFA Rules not processed for user `d9688dca94a841cfa29420ad780562cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:12.934976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2851b6e1-dd33-4825-b41d-4cc46580b1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:12.963358 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1348/2666] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:12.963709 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1318/2666] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:52:12.963817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851b6e1-dd33-4825-b41d-4cc46580b1bd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:12.990229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851b6e1-dd33-4825-b41d-4cc46580b1bd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:12.997291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb585-07bb-4a06-b9b6-4880d91fb937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:12.998556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb585-07bb-4a06-b9b6-4880d91fb937 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:12.999786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb585-07bb-4a06-b9b6-4880d91fb937 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:13.006168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb585-07bb-4a06-b9b6-4880d91fb937 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:13.006945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfb585-07bb-4a06-b9b6-4880d91fb937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:13.008716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfb585-07bb-4a06-b9b6-4880d91fb937 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:13.030809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b4199-b0b0-41e5-b5ad-6cbeed97ae40 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:13.032752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2f2b4199-b0b0-41e5-b5ad-6cbeed97ae40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:13.096577 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1349/2667] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:52:13.098296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb585-07bb-4a06-b9b6-4880d91fb937 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:13.146602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb585-07bb-4a06-b9b6-4880d91fb937 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:13.163875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82051e-80a0-458a-917b-728f2cdf700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:13.168995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82051e-80a0-458a-917b-728f2cdf700d admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:13.181555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82051e-80a0-458a-917b-728f2cdf700d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:13.181971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82051e-80a0-458a-917b-728f2cdf700d admin admin] PATH_INFO: `/v3/projects/c55700f621564ad5a52056d7a7854e20/users/86035889fc474973bc31a97989ec26fd/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:13.220989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82051e-80a0-458a-917b-728f2cdf700d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c55700f621564ad5a52056d7a7854e20, user_id=86035889fc474973bc31a97989ec26fd, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:13.227693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82051e-80a0-458a-917b-728f2cdf700d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:13.274885 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1350/2668] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:52:13 2022] PUT /identity/v3/projects/c55700f621564ad5a52056d7a7854e20/users/86035889fc474973bc31a97989ec26fd/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:13.283101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee49374-95c8-42e3-9603-7e43b7e84c64 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:13.283584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee49374-95c8-42e3-9603-7e43b7e84c64 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:13.285329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b4199-b0b0-41e5-b5ad-6cbeed97ae40 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:13.302092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee49374-95c8-42e3-9603-7e43b7e84c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:13.365301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:13.427402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-eee49374-95c8-42e3-9603-7e43b7e84c64 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:13.435055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-eee49374-95c8-42e3-9603-7e43b7e84c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:13.464286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2df5-78c7-4d84-acc7-e2ca6dd6f3f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:13.464944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2df5-78c7-4d84-acc7-e2ca6dd6f3f2 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:13.465242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2df5-78c7-4d84-acc7-e2ca6dd6f3f2 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:13.465521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2df5-78c7-4d84-acc7-e2ca6dd6f3f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:13.512683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:13.599086 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1351/2669] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:13.642598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee49374-95c8-42e3-9603-7e43b7e84c64 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:13.644522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-eee49374-95c8-42e3-9603-7e43b7e84c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:13.691509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2df5-78c7-4d84-acc7-e2ca6dd6f3f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:13.693333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2df5-78c7-4d84-acc7-e2ca6dd6f3f2 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:13.724271 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1319/2670] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:52:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:52:13.826722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee49374-95c8-42e3-9603-7e43b7e84c64 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:13.839883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd30fb1-c73a-4f38-8043-5f734a1e4cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:13.840452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd30fb1-c73a-4f38-8043-5f734a1e4cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:13.843094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd30fb1-c73a-4f38-8043-5f734a1e4cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:13.843094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd30fb1-c73a-4f38-8043-5f734a1e4cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:13.875322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd30fb1-c73a-4f38-8043-5f734a1e4cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:13.882157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd30fb1-c73a-4f38-8043-5f734a1e4cf5 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:13.901908 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1352/2671] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:52:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:52:16.829902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267621b0-8471-49de-a8bf-abdcdb5da9a6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:16.830408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267621b0-8471-49de-a8bf-abdcdb5da9a6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:16.830507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267621b0-8471-49de-a8bf-abdcdb5da9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:16.906030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-267621b0-8471-49de-a8bf-abdcdb5da9a6 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:16.918056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-267621b0-8471-49de-a8bf-abdcdb5da9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:17.143809 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1320/2672] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:52:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:17.158838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd30fb1-c73a-4f38-8043-5f734a1e4cf5 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:17.161760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5fd30fb1-c73a-4f38-8043-5f734a1e4cf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:17.320625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd30fb1-c73a-4f38-8043-5f734a1e4cf5 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:17.338936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c047485-f6b6-4d63-bb3d-561b97851a3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:17.339695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c047485-f6b6-4d63-bb3d-561b97851a3e service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:17.340225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c047485-f6b6-4d63-bb3d-561b97851a3e service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:17.341232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c047485-f6b6-4d63-bb3d-561b97851a3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:17.362097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c047485-f6b6-4d63-bb3d-561b97851a3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:17.369684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c047485-f6b6-4d63-bb3d-561b97851a3e service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:17.399171 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1353/2673] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:52:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:52:20.714660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea92c8-639d-454b-b679-2b1c4733bd1f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:20.716491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea92c8-639d-454b-b679-2b1c4733bd1f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:20.717042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea92c8-639d-454b-b679-2b1c4733bd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:20.755263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4bea92c8-639d-454b-b679-2b1c4733bd1f None None] MFA Rules not processed for user `778561ee5e474accb31816f357985f44`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:20.757573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4bea92c8-639d-454b-b679-2b1c4733bd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:20.929285 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1321/2674] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:20 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:21.232171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b218cc-160e-480e-b2af-fc6def488571 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:21.233256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b218cc-160e-480e-b2af-fc6def488571 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:21.233721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b218cc-160e-480e-b2af-fc6def488571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:21.290747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-08b218cc-160e-480e-b2af-fc6def488571 None None] MFA Rules not processed for user `b52bd1dc24db4078b99839e483934234`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:21.294183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-08b218cc-160e-480e-b2af-fc6def488571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:21.370828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:21.538969 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1354/2675] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:21 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:21.546840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea92c8-639d-454b-b679-2b1c4733bd1f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:21.548975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4bea92c8-639d-454b-b679-2b1c4733bd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:21.583069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea92c8-639d-454b-b679-2b1c4733bd1f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:21.592044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73959f8d-8d44-451a-ae85-2d4d6e8ccd42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:21.593216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73959f8d-8d44-451a-ae85-2d4d6e8ccd42 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:21.593686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73959f8d-8d44-451a-ae85-2d4d6e8ccd42 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:21.594747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73959f8d-8d44-451a-ae85-2d4d6e8ccd42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:21.607325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73959f8d-8d44-451a-ae85-2d4d6e8ccd42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:21.609417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73959f8d-8d44-451a-ae85-2d4d6e8ccd42 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:21.622917 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1322/2676] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:52:21 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:52:28.234168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d51910-80ee-45c6-936b-24fa88ac3209 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:28.234712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d51910-80ee-45c6-936b-24fa88ac3209 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:28.234712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d51910-80ee-45c6-936b-24fa88ac3209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:28.278123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-90d51910-80ee-45c6-936b-24fa88ac3209 None None] MFA Rules not processed for user `ee0fdf107484448d862ac3216df0d9dc`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:28.290400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-90d51910-80ee-45c6-936b-24fa88ac3209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:28.465754 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1355/2677] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:28 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:31.204465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3da139-b633-45ea-8be6-08aa8f766a1e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:31.205065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3da139-b633-45ea-8be6-08aa8f766a1e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:31.205065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3da139-b633-45ea-8be6-08aa8f766a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:31.244095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bb3da139-b633-45ea-8be6-08aa8f766a1e None None] MFA Rules not processed for user `86035889fc474973bc31a97989ec26fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:31.245401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bb3da139-b633-45ea-8be6-08aa8f766a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:31.363010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:31.430485 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1323/2678] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:31.432778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f8890-a384-4057-98d3-20e1e86b35aa None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:31.433328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f8890-a384-4057-98d3-20e1e86b35aa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:31.433452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f8890-a384-4057-98d3-20e1e86b35aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:31.461849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e95f8890-a384-4057-98d3-20e1e86b35aa None None] MFA Rules not processed for user `86035889fc474973bc31a97989ec26fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:31.464876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e95f8890-a384-4057-98d3-20e1e86b35aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:31.475902 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1356/2679] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:31.664870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3da139-b633-45ea-8be6-08aa8f766a1e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:31.689190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3da139-b633-45ea-8be6-08aa8f766a1e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:31.706601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab9be7-2d75-44a3-9721-63c5065a8c5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:31.707817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9be7-2d75-44a3-9721-63c5065a8c5e service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:31.708327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9be7-2d75-44a3-9721-63c5065a8c5e service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:31.708832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9be7-2d75-44a3-9721-63c5065a8c5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:31.729956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab9be7-2d75-44a3-9721-63c5065a8c5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:31.732189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab9be7-2d75-44a3-9721-63c5065a8c5e service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:31.757306 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1324/2680] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:52:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:52:35.987975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f8890-a384-4057-98d3-20e1e86b35aa None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:35.990154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e95f8890-a384-4057-98d3-20e1e86b35aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:36.027154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f8890-a384-4057-98d3-20e1e86b35aa None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:36.059776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abcb0f-eff4-40c0-a3da-84ede568129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:36.060782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abcb0f-eff4-40c0-a3da-84ede568129f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:36.061332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abcb0f-eff4-40c0-a3da-84ede568129f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:36.061840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abcb0f-eff4-40c0-a3da-84ede568129f admin admin] PATH_INFO: `/v3/users/b52bd1dc24db4078b99839e483934234` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:36.063251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abcb0f-eff4-40c0-a3da-84ede568129f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b52bd1dc24db4078b99839e483934234)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:36.066541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abcb0f-eff4-40c0-a3da-84ede568129f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:36.146129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16abcb0f-eff4-40c0-a3da-84ede568129f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b52bd1dc24db4078b99839e483934234'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:36.159984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16abcb0f-eff4-40c0-a3da-84ede568129f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b52bd1dc24db4078b99839e483934234'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:36.171150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16abcb0f-eff4-40c0-a3da-84ede568129f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b52bd1dc24db4078b99839e483934234'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:36.171803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16abcb0f-eff4-40c0-a3da-84ede568129f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b52bd1dc24db4078b99839e483934234'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:36.172608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16abcb0f-eff4-40c0-a3da-84ede568129f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b52bd1dc24db4078b99839e483934234'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:36.190625 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1357/2681] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:52:35 2022] DELETE /identity/v3/users/b52bd1dc24db4078b99839e483934234 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:37.478968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab9be7-2d75-44a3-9721-63c5065a8c5e service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:37.500269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab9be7-2d75-44a3-9721-63c5065a8c5e service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:37.613130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e4533-f100-4e37-824a-de6b2d043f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:37.613876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e4533-f100-4e37-824a-de6b2d043f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:37.614152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e4533-f100-4e37-824a-de6b2d043f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:37.614429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e4533-f100-4e37-824a-de6b2d043f4a admin admin] PATH_INFO: `/v3/projects/e44f80b00bc14282af09e08925c79c7d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:37.621737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e4533-f100-4e37-824a-de6b2d043f4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e44f80b00bc14282af09e08925c79c7d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:37.623388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e4533-f100-4e37-824a-de6b2d043f4a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:37.719452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab4e4533-f100-4e37-824a-de6b2d043f4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e44f80b00bc14282af09e08925c79c7d', 'request_id': 'req-ab4e4533-f100-4e37-824a-de6b2d043f4a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:37.730858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab4e4533-f100-4e37-824a-de6b2d043f4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e44f80b00bc14282af09e08925c79c7d', 'request_id': 'req-ab4e4533-f100-4e37-824a-de6b2d043f4a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:37.747030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab4e4533-f100-4e37-824a-de6b2d043f4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e44f80b00bc14282af09e08925c79c7d', 'request_id': 'req-ab4e4533-f100-4e37-824a-de6b2d043f4a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:37.786341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab4e4533-f100-4e37-824a-de6b2d043f4a admin admin] The token cache is being invalidate because project e44f80b00bc14282af09e08925c79c7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:52:37.787153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab4e4533-f100-4e37-824a-de6b2d043f4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:37.789683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab4e4533-f100-4e37-824a-de6b2d043f4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:37.796047 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1325/2682] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:52:37 2022] DELETE /identity/v3/projects/e44f80b00bc14282af09e08925c79c7d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:37.801308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3893ba5-9b6a-4787-a139-ac073fa70e11 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:37.802815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3893ba5-9b6a-4787-a139-ac073fa70e11 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:37.803250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3893ba5-9b6a-4787-a139-ac073fa70e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:37.848442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d3893ba5-9b6a-4787-a139-ac073fa70e11 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:37.850436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d3893ba5-9b6a-4787-a139-ac073fa70e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:37.952002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:37.999466 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1358/2683] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:38.008529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa27a8-e2ce-448e-b16a-03539027efd2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:38.008819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa27a8-e2ce-448e-b16a-03539027efd2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:38.009667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa27a8-e2ce-448e-b16a-03539027efd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:38.070570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-53aa27a8-e2ce-448e-b16a-03539027efd2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:38.072411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-53aa27a8-e2ce-448e-b16a-03539027efd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:38.100953 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1326/2684] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:38.102623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3893ba5-9b6a-4787-a139-ac073fa70e11 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:38.124407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3893ba5-9b6a-4787-a139-ac073fa70e11 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:38.137641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f680e-39b9-4c1e-bdca-1577aa0e92f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:38.138279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f680e-39b9-4c1e-bdca-1577aa0e92f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:38.138570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f680e-39b9-4c1e-bdca-1577aa0e92f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:38.138844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f680e-39b9-4c1e-bdca-1577aa0e92f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:38.139887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f680e-39b9-4c1e-bdca-1577aa0e92f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:52:38.140358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f680e-39b9-4c1e-bdca-1577aa0e92f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:38.142162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f680e-39b9-4c1e-bdca-1577aa0e92f5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:38.145692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bdc55-bb03-45ae-94a0-9712df357dde None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:38.145968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bdc55-bb03-45ae-94a0-9712df357dde None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:38.146249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bdc55-bb03-45ae-94a0-9712df357dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:38.166445 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1359/2685] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:52:38.170679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f680e-39b9-4c1e-bdca-1577aa0e92f5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:38.187553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f680e-39b9-4c1e-bdca-1577aa0e92f5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:38.201632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7a4bdc55-bb03-45ae-94a0-9712df357dde None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:38.202868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7a4bdc55-bb03-45ae-94a0-9712df357dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:38.205840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91241d9-f1f2-457e-b610-d320bb23ac9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:38.206700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91241d9-f1f2-457e-b610-d320bb23ac9b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:38.207254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91241d9-f1f2-457e-b610-d320bb23ac9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:38.207426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91241d9-f1f2-457e-b610-d320bb23ac9b admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:38.208328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91241d9-f1f2-457e-b610-d320bb23ac9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:38.210198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91241d9-f1f2-457e-b610-d320bb23ac9b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:38.265755 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1360/2686] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:52:38 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:52:38.269534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91241d9-f1f2-457e-b610-d320bb23ac9b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:38.388559 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1327/2687] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:38.401846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91241d9-f1f2-457e-b610-d320bb23ac9b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:38.417658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c5142-141f-483b-8f3c-66d32d067840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:38.418797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c5142-141f-483b-8f3c-66d32d067840 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:38.419241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c5142-141f-483b-8f3c-66d32d067840 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:38.419691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c5142-141f-483b-8f3c-66d32d067840 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:38.420540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c5142-141f-483b-8f3c-66d32d067840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:38.423692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bdc55-bb03-45ae-94a0-9712df357dde None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:38.424830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c5142-141f-483b-8f3c-66d32d067840 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:38.425209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7a4bdc55-bb03-45ae-94a0-9712df357dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:38.430086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:38.473699 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1361/2688] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:52:38 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:52:38.477279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c5142-141f-483b-8f3c-66d32d067840 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:38.510375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c5142-141f-483b-8f3c-66d32d067840 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:38.517239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef40e3-e062-41ec-81f9-aeb55be1c121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:38.518317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef40e3-e062-41ec-81f9-aeb55be1c121 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:38.518819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef40e3-e062-41ec-81f9-aeb55be1c121 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:38.519356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef40e3-e062-41ec-81f9-aeb55be1c121 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:38.521064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef40e3-e062-41ec-81f9-aeb55be1c121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:38.523138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef40e3-e062-41ec-81f9-aeb55be1c121 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:38.538571 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1362/2689] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:52:38.543646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef40e3-e062-41ec-81f9-aeb55be1c121 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:38.569823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef40e3-e062-41ec-81f9-aeb55be1c121 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:38.581766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6796e9-47d7-4eae-a0e6-a9ff1f098952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:38.582833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6796e9-47d7-4eae-a0e6-a9ff1f098952 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:38.583466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6796e9-47d7-4eae-a0e6-a9ff1f098952 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:38.583960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6796e9-47d7-4eae-a0e6-a9ff1f098952 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:38.584823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6796e9-47d7-4eae-a0e6-a9ff1f098952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:38.585204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bdc55-bb03-45ae-94a0-9712df357dde None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:38.587795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6796e9-47d7-4eae-a0e6-a9ff1f098952 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:38.606613 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1363/2690] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:52:38.611463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6796e9-47d7-4eae-a0e6-a9ff1f098952 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:38.626040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6796e9-47d7-4eae-a0e6-a9ff1f098952 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:38.635947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0ca99-d90f-4b8f-b5c1-ac007008d86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:38.637039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0ca99-d90f-4b8f-b5c1-ac007008d86f admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:38.637545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0ca99-d90f-4b8f-b5c1-ac007008d86f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:38.638062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0ca99-d90f-4b8f-b5c1-ac007008d86f admin admin] PATH_INFO: `/v3/projects/44e122acac834fc2843f725beb5daeff/users/ddb7b29dc8464329abda9836b09b907f/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:38.661591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0ca99-d90f-4b8f-b5c1-ac007008d86f admin admin] RBAC: Authorizing `identity:create_grant(project_id=44e122acac834fc2843f725beb5daeff, user_id=ddb7b29dc8464329abda9836b09b907f, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:38.666654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0ca99-d90f-4b8f-b5c1-ac007008d86f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:38.711444 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1364/2691] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:52:38 2022] PUT /identity/v3/projects/44e122acac834fc2843f725beb5daeff/users/ddb7b29dc8464329abda9836b09b907f/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:38.721915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a7f6e-25dd-4073-839f-95798159e888 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:38.722863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a7f6e-25dd-4073-839f-95798159e888 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:38.723154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a7f6e-25dd-4073-839f-95798159e888 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:38.723427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a7f6e-25dd-4073-839f-95798159e888 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:38.723835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bbb1d-ee4f-4076-9ad7-0e0fb08a1598 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:38.724282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bbb1d-ee4f-4076-9ad7-0e0fb08a1598 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:38.724802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bbb1d-ee4f-4076-9ad7-0e0fb08a1598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:38.804375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-267bbb1d-ee4f-4076-9ad7-0e0fb08a1598 None None] MFA Rules not processed for user `ddb7b29dc8464329abda9836b09b907f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:38.805726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-267bbb1d-ee4f-4076-9ad7-0e0fb08a1598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:38.806797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a7f6e-25dd-4073-839f-95798159e888 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:38.809409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a7f6e-25dd-4073-839f-95798159e888 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:38.821216 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1328/2692] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:52:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 20:52:38.884490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:38.965362 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1365/2693] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:38.979782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646dd11-9ebb-43e0-b0f0-47f580324106 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:38.980064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646dd11-9ebb-43e0-b0f0-47f580324106 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:38.980400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646dd11-9ebb-43e0-b0f0-47f580324106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:39.024261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c646dd11-9ebb-43e0-b0f0-47f580324106 None None] MFA Rules not processed for user `ddb7b29dc8464329abda9836b09b907f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:39.026912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c646dd11-9ebb-43e0-b0f0-47f580324106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:39.041107 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1329/2694] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:39.875019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bbb1d-ee4f-4076-9ad7-0e0fb08a1598 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:39.876970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-267bbb1d-ee4f-4076-9ad7-0e0fb08a1598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:40.189188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bbb1d-ee4f-4076-9ad7-0e0fb08a1598 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:40.215048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d846e-c2e6-49b7-bc4f-6369e8a53bb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:40.215944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d846e-c2e6-49b7-bc4f-6369e8a53bb6 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:40.216396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d846e-c2e6-49b7-bc4f-6369e8a53bb6 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:40.216863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d846e-c2e6-49b7-bc4f-6369e8a53bb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:40.241441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d846e-c2e6-49b7-bc4f-6369e8a53bb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:40.244374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d846e-c2e6-49b7-bc4f-6369e8a53bb6 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:40.263121 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1366/2695] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:52:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:52:45.804817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f17b1e-4f4c-4128-ac1a-f2cb502079e4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:45.805350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f17b1e-4f4c-4128-ac1a-f2cb502079e4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:45.805433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f17b1e-4f4c-4128-ac1a-f2cb502079e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:45.833710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b3f17b1e-4f4c-4128-ac1a-f2cb502079e4 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:45.836236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b3f17b1e-4f4c-4128-ac1a-f2cb502079e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:45.887275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:45.920677 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1330/2696] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:52:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:45.939564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d846e-c2e6-49b7-bc4f-6369e8a53bb6 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:46.050516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d846e-c2e6-49b7-bc4f-6369e8a53bb6 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:46.061496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8b2b9-2deb-4274-9d08-d0044d911438 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:46.062405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8b2b9-2deb-4274-9d08-d0044d911438 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:46.062684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8b2b9-2deb-4274-9d08-d0044d911438 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:46.062965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8b2b9-2deb-4274-9d08-d0044d911438 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:46.074018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8b2b9-2deb-4274-9d08-d0044d911438 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:46.075749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8b2b9-2deb-4274-9d08-d0044d911438 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:46.084549 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1367/2697] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:52:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:52:48.888951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f17b1e-4f4c-4128-ac1a-f2cb502079e4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:48.890392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b3f17b1e-4f4c-4128-ac1a-f2cb502079e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:49.073996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f17b1e-4f4c-4128-ac1a-f2cb502079e4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:49.087626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:49.088744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:49.089197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:49.089473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] PATH_INFO: `/v3/users/cb51295631fd43fd8323ea06097a632e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:49.092229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb51295631fd43fd8323ea06097a632e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:49.096159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:49.214134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb51295631fd43fd8323ea06097a632e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:49.243871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb51295631fd43fd8323ea06097a632e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:49.244890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb51295631fd43fd8323ea06097a632e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:49.259455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb51295631fd43fd8323ea06097a632e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:49.276531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb51295631fd43fd8323ea06097a632e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:49.285794 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1331/2698] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:52:48 2022] DELETE /identity/v3/users/cb51295631fd43fd8323ea06097a632e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:49.290889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c087df-bd03-4cae-a279-ff78301ab532 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:49.291418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c087df-bd03-4cae-a279-ff78301ab532 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:49.291418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c087df-bd03-4cae-a279-ff78301ab532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:49.331079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-06c087df-bd03-4cae-a279-ff78301ab532 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:49.333212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-06c087df-bd03-4cae-a279-ff78301ab532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:49.452018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:49.483794 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1368/2699] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:52:49.541669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:49.543772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:49.680113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456fbac-fc3d-4f28-afdb-0f3d4dd322e2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:49.683614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ea5c1-b411-4a21-8b5e-bf05cbba7f4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:49.685966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ea5c1-b411-4a21-8b5e-bf05cbba7f4b service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:49.685966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ea5c1-b411-4a21-8b5e-bf05cbba7f4b service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:49.685966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ea5c1-b411-4a21-8b5e-bf05cbba7f4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:49.702130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ea5c1-b411-4a21-8b5e-bf05cbba7f4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:49.705700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ea5c1-b411-4a21-8b5e-bf05cbba7f4b service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:49.721125 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1332/2700] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:52:50.844203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c087df-bd03-4cae-a279-ff78301ab532 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:50.845775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-06c087df-bd03-4cae-a279-ff78301ab532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:50.870027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c087df-bd03-4cae-a279-ff78301ab532 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:50.891680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:50.894856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:50.896952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:50.899002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] PATH_INFO: `/v3/projects/a72b0776bdcd4644afed4d9cbc966e4e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:50.902427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a72b0776bdcd4644afed4d9cbc966e4e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:50.905502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:50.987640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a72b0776bdcd4644afed4d9cbc966e4e', 'request_id': 'req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:50.998977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a72b0776bdcd4644afed4d9cbc966e4e', 'request_id': 'req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:51.007558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a72b0776bdcd4644afed4d9cbc966e4e', 'request_id': 'req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:51.027741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] The token cache is being invalidate because project a72b0776bdcd4644afed4d9cbc966e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:52:51.028626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:51.029881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:51.032725 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1369/2701] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:52:50 2022] DELETE /identity/v3/projects/a72b0776bdcd4644afed4d9cbc966e4e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:51.036715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ea5c1-b411-4a21-8b5e-bf05cbba7f4b service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:51.049700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ea5c1-b411-4a21-8b5e-bf05cbba7f4b service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:51.061528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:51.183694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a9a47-36f2-44da-b2e1-16dd72cb3b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:51.184701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a9a47-36f2-44da-b2e1-16dd72cb3b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:51.185245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a9a47-36f2-44da-b2e1-16dd72cb3b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:51.185773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a9a47-36f2-44da-b2e1-16dd72cb3b4b admin admin] PATH_INFO: `/v3/users/3643165fa95e4f16a8fc2e5c7765d71d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:51.187360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a9a47-36f2-44da-b2e1-16dd72cb3b4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3643165fa95e4f16a8fc2e5c7765d71d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:51.190005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a9a47-36f2-44da-b2e1-16dd72cb3b4b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:51.289249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1b8a9a47-36f2-44da-b2e1-16dd72cb3b4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3643165fa95e4f16a8fc2e5c7765d71d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:51.332725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1b8a9a47-36f2-44da-b2e1-16dd72cb3b4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3643165fa95e4f16a8fc2e5c7765d71d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:51.338285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1b8a9a47-36f2-44da-b2e1-16dd72cb3b4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3643165fa95e4f16a8fc2e5c7765d71d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:51.354483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1b8a9a47-36f2-44da-b2e1-16dd72cb3b4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3643165fa95e4f16a8fc2e5c7765d71d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:51.367173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1b8a9a47-36f2-44da-b2e1-16dd72cb3b4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3643165fa95e4f16a8fc2e5c7765d71d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:51.384698 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1333/2702] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:52:51 2022] DELETE /identity/v3/users/3643165fa95e4f16a8fc2e5c7765d71d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:52.423764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:52.425325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:52.445186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a8d68-43ba-46e6-bb0e-7c0e5e4c3ef3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:52.731246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d47791-145e-41d0-8961-5b3b1adaa5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:52.732944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d47791-145e-41d0-8961-5b3b1adaa5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:52.732944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d47791-145e-41d0-8961-5b3b1adaa5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:52.732944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d47791-145e-41d0-8961-5b3b1adaa5e2 admin admin] PATH_INFO: `/v3/projects/3d2d0567544d4db487c2d63a34078bb9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:52.733939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d47791-145e-41d0-8961-5b3b1adaa5e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d2d0567544d4db487c2d63a34078bb9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:52.744665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d47791-145e-41d0-8961-5b3b1adaa5e2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:52.837965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1d47791-145e-41d0-8961-5b3b1adaa5e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d2d0567544d4db487c2d63a34078bb9', 'request_id': 'req-a1d47791-145e-41d0-8961-5b3b1adaa5e2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:52.853184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1d47791-145e-41d0-8961-5b3b1adaa5e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d2d0567544d4db487c2d63a34078bb9', 'request_id': 'req-a1d47791-145e-41d0-8961-5b3b1adaa5e2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:52.870635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1d47791-145e-41d0-8961-5b3b1adaa5e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d2d0567544d4db487c2d63a34078bb9', 'request_id': 'req-a1d47791-145e-41d0-8961-5b3b1adaa5e2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:52.896537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1d47791-145e-41d0-8961-5b3b1adaa5e2 admin admin] The token cache is being invalidate because project 3d2d0567544d4db487c2d63a34078bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:52:52.896831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1d47791-145e-41d0-8961-5b3b1adaa5e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:52.897995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1d47791-145e-41d0-8961-5b3b1adaa5e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:52.900934 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1370/2703] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:52:52 2022] DELETE /identity/v3/projects/3d2d0567544d4db487c2d63a34078bb9 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:52.905639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e901db1-62c1-499c-904c-019e53e81ce2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:52.906157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e901db1-62c1-499c-904c-019e53e81ce2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:52.906258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e901db1-62c1-499c-904c-019e53e81ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:52.940619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2e901db1-62c1-499c-904c-019e53e81ce2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:52.941905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2e901db1-62c1-499c-904c-019e53e81ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:53.087733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314f76a-e7f2-48db-8de3-db5e6ce40959 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:53.087913 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1334/2704] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:53.088016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314f76a-e7f2-48db-8de3-db5e6ce40959 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:53.088293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314f76a-e7f2-48db-8de3-db5e6ce40959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:53.104863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:53.133269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e314f76a-e7f2-48db-8de3-db5e6ce40959 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:53.134549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e314f76a-e7f2-48db-8de3-db5e6ce40959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:53.151827 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1371/2705] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:53.156023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e901db1-62c1-499c-904c-019e53e81ce2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:53.178665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e901db1-62c1-499c-904c-019e53e81ce2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:53.190303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed59d93-e65d-4c74-b7aa-56771f7c5cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:53.191392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed59d93-e65d-4c74-b7aa-56771f7c5cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:53.191894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed59d93-e65d-4c74-b7aa-56771f7c5cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:53.192394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed59d93-e65d-4c74-b7aa-56771f7c5cf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:53.193299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed59d93-e65d-4c74-b7aa-56771f7c5cf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:52:53.194950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed59d93-e65d-4c74-b7aa-56771f7c5cf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:53.198633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed59d93-e65d-4c74-b7aa-56771f7c5cf8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:53.234224 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1335/2706] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:52:53.235161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314f76a-e7f2-48db-8de3-db5e6ce40959 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:53.246526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314f76a-e7f2-48db-8de3-db5e6ce40959 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:53.268668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16c181-246b-4322-bd48-c82e200d265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:53.269375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16c181-246b-4322-bd48-c82e200d265e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:53.269660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16c181-246b-4322-bd48-c82e200d265e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:53.269988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16c181-246b-4322-bd48-c82e200d265e admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:53.270623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16c181-246b-4322-bd48-c82e200d265e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:53.273616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16c181-246b-4322-bd48-c82e200d265e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:53.343882 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1372/2707] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:52:53 2022] POST /identity/v3/projects => generated 393 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:52:53.348042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed59d93-e65d-4c74-b7aa-56771f7c5cf8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:53.371266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed59d93-e65d-4c74-b7aa-56771f7c5cf8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:53.407313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:53.510409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c22190-b7a4-46d5-9918-f514568d49e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:53.511082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c22190-b7a4-46d5-9918-f514568d49e4 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:53.511702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c22190-b7a4-46d5-9918-f514568d49e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:53.511702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c22190-b7a4-46d5-9918-f514568d49e4 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:53.512285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c22190-b7a4-46d5-9918-f514568d49e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:53.515055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c22190-b7a4-46d5-9918-f514568d49e4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:53.572062 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1336/2708] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:52:53 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:52:53.576030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16c181-246b-4322-bd48-c82e200d265e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:53.593599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16c181-246b-4322-bd48-c82e200d265e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:53.611747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37311d2-89c2-4a5b-bbcf-90a29248f3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:53.612778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37311d2-89c2-4a5b-bbcf-90a29248f3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:53.613459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37311d2-89c2-4a5b-bbcf-90a29248f3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:53.614071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37311d2-89c2-4a5b-bbcf-90a29248f3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:53.615061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37311d2-89c2-4a5b-bbcf-90a29248f3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:53.623879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37311d2-89c2-4a5b-bbcf-90a29248f3e1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:53.638074 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1373/2709] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:52:53.641857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c22190-b7a4-46d5-9918-f514568d49e4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:53.655146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c22190-b7a4-46d5-9918-f514568d49e4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:53.660622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e046b-2498-4bcf-a28b-92fff5f94475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:53.661558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e046b-2498-4bcf-a28b-92fff5f94475 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:53.662069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e046b-2498-4bcf-a28b-92fff5f94475 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:53.662869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e046b-2498-4bcf-a28b-92fff5f94475 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:53.663693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e046b-2498-4bcf-a28b-92fff5f94475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:53.666404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e046b-2498-4bcf-a28b-92fff5f94475 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:53.685639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37311d2-89c2-4a5b-bbcf-90a29248f3e1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:53.697633 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1337/2710] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:52:53.702049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37311d2-89c2-4a5b-bbcf-90a29248f3e1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:53.717319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9df70-bb6f-40ba-8bf9-de032301dc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:53.718259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9df70-bb6f-40ba-8bf9-de032301dc91 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:53.718548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9df70-bb6f-40ba-8bf9-de032301dc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:53.718819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9df70-bb6f-40ba-8bf9-de032301dc91 admin admin] PATH_INFO: `/v3/projects/29e0ecbfab5b4d5b8376c13a499d541b/users/2415e6e7c5d549e0b8a0a684e4dd58b2/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:53.751439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9df70-bb6f-40ba-8bf9-de032301dc91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29e0ecbfab5b4d5b8376c13a499d541b, user_id=2415e6e7c5d549e0b8a0a684e4dd58b2, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:53.754631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9df70-bb6f-40ba-8bf9-de032301dc91 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:53.799450 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1374/2711] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:52:53 2022] PUT /identity/v3/projects/29e0ecbfab5b4d5b8376c13a499d541b/users/2415e6e7c5d549e0b8a0a684e4dd58b2/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:53.800704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e5d98-0e12-454f-b91f-9e8d4cdfac94 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:53.801194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e5d98-0e12-454f-b91f-9e8d4cdfac94 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:53.801674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e5d98-0e12-454f-b91f-9e8d4cdfac94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:53.866922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-528e5d98-0e12-454f-b91f-9e8d4cdfac94 None None] MFA Rules not processed for user `2415e6e7c5d549e0b8a0a684e4dd58b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:53.873765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-528e5d98-0e12-454f-b91f-9e8d4cdfac94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:54.025996 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1338/2712] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:54.030633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9df70-bb6f-40ba-8bf9-de032301dc91 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:54.172544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9df70-bb6f-40ba-8bf9-de032301dc91 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:54.196926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71682d77-9630-46a1-8dea-e18e0127657f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:54.201747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71682d77-9630-46a1-8dea-e18e0127657f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:54.202636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71682d77-9630-46a1-8dea-e18e0127657f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:54.203108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71682d77-9630-46a1-8dea-e18e0127657f admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:54.203887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71682d77-9630-46a1-8dea-e18e0127657f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:54.206068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71682d77-9630-46a1-8dea-e18e0127657f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:54.273710 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1375/2713] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:52:54 2022] POST /identity/v3/projects => generated 393 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:52:54.274916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e5d98-0e12-454f-b91f-9e8d4cdfac94 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:54.311322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e5d98-0e12-454f-b91f-9e8d4cdfac94 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:54.507699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae063951-0487-4060-8563-93d780429de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:54.508318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae063951-0487-4060-8563-93d780429de5 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:54.509196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae063951-0487-4060-8563-93d780429de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:54.509499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae063951-0487-4060-8563-93d780429de5 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:54.510158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae063951-0487-4060-8563-93d780429de5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:54.512091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae063951-0487-4060-8563-93d780429de5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:54.542620 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1339/2714] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:52:54 2022] POST /identity/v3/users => generated 303 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:52:54.544164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71682d77-9630-46a1-8dea-e18e0127657f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:54.566429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71682d77-9630-46a1-8dea-e18e0127657f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:54.572624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a96f4-961a-4e27-a592-41e81961eae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:54.573293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a96f4-961a-4e27-a592-41e81961eae1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:54.573583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a96f4-961a-4e27-a592-41e81961eae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:54.573861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a96f4-961a-4e27-a592-41e81961eae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:54.574561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a96f4-961a-4e27-a592-41e81961eae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:54.576193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a96f4-961a-4e27-a592-41e81961eae1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:54.591151 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1376/2715] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:52:54.593076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae063951-0487-4060-8563-93d780429de5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:54.619719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae063951-0487-4060-8563-93d780429de5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:54.635987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5169448-52f3-4ba9-8e13-c07b8b172f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:54.637148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5169448-52f3-4ba9-8e13-c07b8b172f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:54.640242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5169448-52f3-4ba9-8e13-c07b8b172f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:54.640788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5169448-52f3-4ba9-8e13-c07b8b172f15 admin admin] PATH_INFO: `/v3/projects/e3fea7ae388847fc9379dc1281a4c38c/users/f69588e23bdf4c5ea0139344d70b42b0/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:54.654561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:54.671730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5169448-52f3-4ba9-8e13-c07b8b172f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3fea7ae388847fc9379dc1281a4c38c, user_id=f69588e23bdf4c5ea0139344d70b42b0, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:54.691956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5169448-52f3-4ba9-8e13-c07b8b172f15 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:54.753721 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1340/2716] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:52:54 2022] PUT /identity/v3/projects/e3fea7ae388847fc9379dc1281a4c38c/users/f69588e23bdf4c5ea0139344d70b42b0/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:54.755084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a96f4-961a-4e27-a592-41e81961eae1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:54.771286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a96f4-961a-4e27-a592-41e81961eae1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:55.079494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff68f2a-a2ee-414f-8f52-88ce229e782c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:55.091199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff68f2a-a2ee-414f-8f52-88ce229e782c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:55.091728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff68f2a-a2ee-414f-8f52-88ce229e782c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:55.092029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff68f2a-a2ee-414f-8f52-88ce229e782c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:55.093130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff68f2a-a2ee-414f-8f52-88ce229e782c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:55.094665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff68f2a-a2ee-414f-8f52-88ce229e782c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:55.144416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5169448-52f3-4ba9-8e13-c07b8b172f15 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:55.146285 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1377/2717] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:52:55.162930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5169448-52f3-4ba9-8e13-c07b8b172f15 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:55.181880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d28029-8345-41b6-b078-9439b9bfe4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:55.182651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d28029-8345-41b6-b078-9439b9bfe4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:55.183039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d28029-8345-41b6-b078-9439b9bfe4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:55.183420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d28029-8345-41b6-b078-9439b9bfe4b8 admin admin] PATH_INFO: `/v3/projects/e3fea7ae388847fc9379dc1281a4c38c/users/f69588e23bdf4c5ea0139344d70b42b0/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:55.190783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d28029-8345-41b6-b078-9439b9bfe4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3fea7ae388847fc9379dc1281a4c38c, user_id=f69588e23bdf4c5ea0139344d70b42b0, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:55.198937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d28029-8345-41b6-b078-9439b9bfe4b8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:55.244177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829955d-cdd4-4319-9f3c-93dd824296e6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:55.244475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829955d-cdd4-4319-9f3c-93dd824296e6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:55.244668 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1341/2718] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:52:55 2022] PUT /identity/v3/projects/e3fea7ae388847fc9379dc1281a4c38c/users/f69588e23bdf4c5ea0139344d70b42b0/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:52:55.244772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829955d-cdd4-4319-9f3c-93dd824296e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:55.319733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:55.332878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e829955d-cdd4-4319-9f3c-93dd824296e6 None None] MFA Rules not processed for user `f69588e23bdf4c5ea0139344d70b42b0`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:55.334169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e829955d-cdd4-4319-9f3c-93dd824296e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:55.580953 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1378/2719] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:55 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:55.605019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d28029-8345-41b6-b078-9439b9bfe4b8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:55.607851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f5d28029-8345-41b6-b078-9439b9bfe4b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:55.756981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d28029-8345-41b6-b078-9439b9bfe4b8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:55.764768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fae0f7-24e8-4975-974e-e3021e428eee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:55.765515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fae0f7-24e8-4975-974e-e3021e428eee service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:55.766489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fae0f7-24e8-4975-974e-e3021e428eee service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:55.766489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fae0f7-24e8-4975-974e-e3021e428eee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:55.785565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fae0f7-24e8-4975-974e-e3021e428eee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:55.787701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fae0f7-24e8-4975-974e-e3021e428eee service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:55.812416 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1342/2720] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:52:55 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:52:56.232008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829955d-cdd4-4319-9f3c-93dd824296e6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:56.238190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e829955d-cdd4-4319-9f3c-93dd824296e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:56.274925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829955d-cdd4-4319-9f3c-93dd824296e6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:56.281410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba0a2b-e837-4266-8f58-aa1ccea6b672 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:56.285789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba0a2b-e837-4266-8f58-aa1ccea6b672 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:56.286330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba0a2b-e837-4266-8f58-aa1ccea6b672 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:56.286807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba0a2b-e837-4266-8f58-aa1ccea6b672 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:56.389755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba0a2b-e837-4266-8f58-aa1ccea6b672 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:56.392403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba0a2b-e837-4266-8f58-aa1ccea6b672 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:56.405612 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1379/2721] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:52:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:52:56.436787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fae0f7-24e8-4975-974e-e3021e428eee service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:56.556538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fae0f7-24e8-4975-974e-e3021e428eee service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:56.561933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:56.562578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:56.562867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:56.563150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] PATH_INFO: `/v3/users/2415e6e7c5d549e0b8a0a684e4dd58b2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:56.564300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2415e6e7c5d549e0b8a0a684e4dd58b2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:56.566403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:56.637815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2415e6e7c5d549e0b8a0a684e4dd58b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:56.652414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:56.655360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2415e6e7c5d549e0b8a0a684e4dd58b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:56.657216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2415e6e7c5d549e0b8a0a684e4dd58b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:56.668470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2415e6e7c5d549e0b8a0a684e4dd58b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:56.679660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2415e6e7c5d549e0b8a0a684e4dd58b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:56.718537 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1343/2722] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:52:56 2022] DELETE /identity/v3/users/2415e6e7c5d549e0b8a0a684e4dd58b2 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:56.725857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a9319-4250-4328-8f08-8ce41d48f01f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:56.726807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a9319-4250-4328-8f08-8ce41d48f01f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:56.727384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a9319-4250-4328-8f08-8ce41d48f01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:56.757716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-474a9319-4250-4328-8f08-8ce41d48f01f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:56.759808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-474a9319-4250-4328-8f08-8ce41d48f01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:56.943365 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1380/2723] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:56.989659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:56.993456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:57.165787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61269d1-e38f-4dc2-973d-d36f7a3b88aa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:57.172516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fee9a9-f146-4bdf-974b-ca03cf8412bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:57.173201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fee9a9-f146-4bdf-974b-ca03cf8412bc service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:57.173492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fee9a9-f146-4bdf-974b-ca03cf8412bc service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:57.173773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fee9a9-f146-4bdf-974b-ca03cf8412bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:57.204850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fee9a9-f146-4bdf-974b-ca03cf8412bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:57.208225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fee9a9-f146-4bdf-974b-ca03cf8412bc service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:57.226664 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1344/2724] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:52:58.313591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a9319-4250-4328-8f08-8ce41d48f01f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:58.315357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-474a9319-4250-4328-8f08-8ce41d48f01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:58.331405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a9319-4250-4328-8f08-8ce41d48f01f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:58.338566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:58.339269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:58.339657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:58.340057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] PATH_INFO: `/v3/projects/29e0ecbfab5b4d5b8376c13a499d541b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:58.341807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29e0ecbfab5b4d5b8376c13a499d541b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:58.344215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:58.395011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:58.413204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29e0ecbfab5b4d5b8376c13a499d541b', 'request_id': 'req-87156d65-db61-4f20-a846-fd880c5128a3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:58.433099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29e0ecbfab5b4d5b8376c13a499d541b', 'request_id': 'req-87156d65-db61-4f20-a846-fd880c5128a3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:58.441627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29e0ecbfab5b4d5b8376c13a499d541b', 'request_id': 'req-87156d65-db61-4f20-a846-fd880c5128a3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:58.454236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] The token cache is being invalidate because project 29e0ecbfab5b4d5b8376c13a499d541b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:52:58.454522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:58.455373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:58.460408 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1381/2725] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:52:58 2022] DELETE /identity/v3/projects/29e0ecbfab5b4d5b8376c13a499d541b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:58.461264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fee9a9-f146-4bdf-974b-ca03cf8412bc service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:58.474829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fee9a9-f146-4bdf-974b-ca03cf8412bc service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:58.584664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b6982-d291-4dd9-b46e-0faf11529c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:58.585476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b6982-d291-4dd9-b46e-0faf11529c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:58.585772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b6982-d291-4dd9-b46e-0faf11529c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:58.586050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b6982-d291-4dd9-b46e-0faf11529c78 admin admin] PATH_INFO: `/v3/users/f69588e23bdf4c5ea0139344d70b42b0` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:58.587380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b6982-d291-4dd9-b46e-0faf11529c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69588e23bdf4c5ea0139344d70b42b0)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:58.589667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b6982-d291-4dd9-b46e-0faf11529c78 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:58.669973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dc2b6982-d291-4dd9-b46e-0faf11529c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69588e23bdf4c5ea0139344d70b42b0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:58.689093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dc2b6982-d291-4dd9-b46e-0faf11529c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69588e23bdf4c5ea0139344d70b42b0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:58.690383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dc2b6982-d291-4dd9-b46e-0faf11529c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69588e23bdf4c5ea0139344d70b42b0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:58.701287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dc2b6982-d291-4dd9-b46e-0faf11529c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69588e23bdf4c5ea0139344d70b42b0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:58.720035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dc2b6982-d291-4dd9-b46e-0faf11529c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69588e23bdf4c5ea0139344d70b42b0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:58.723230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:52:58.737453 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1345/2726] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:52:58 2022] DELETE /identity/v3/users/f69588e23bdf4c5ea0139344d70b42b0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:52:59.105555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:59.107203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:59.135769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87156d65-db61-4f20-a846-fd880c5128a3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:59.259660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9467c1a-426f-4e5c-a635-79f99b56ea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:59.260599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9467c1a-426f-4e5c-a635-79f99b56ea79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:59.261100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9467c1a-426f-4e5c-a635-79f99b56ea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:59.261844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9467c1a-426f-4e5c-a635-79f99b56ea79 admin admin] PATH_INFO: `/v3/projects/e3fea7ae388847fc9379dc1281a4c38c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:59.263575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9467c1a-426f-4e5c-a635-79f99b56ea79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3fea7ae388847fc9379dc1281a4c38c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:59.268767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9467c1a-426f-4e5c-a635-79f99b56ea79 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:59.384918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9467c1a-426f-4e5c-a635-79f99b56ea79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3fea7ae388847fc9379dc1281a4c38c', 'request_id': 'req-f9467c1a-426f-4e5c-a635-79f99b56ea79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:59.397947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9467c1a-426f-4e5c-a635-79f99b56ea79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3fea7ae388847fc9379dc1281a4c38c', 'request_id': 'req-f9467c1a-426f-4e5c-a635-79f99b56ea79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:59.407490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9467c1a-426f-4e5c-a635-79f99b56ea79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3fea7ae388847fc9379dc1281a4c38c', 'request_id': 'req-f9467c1a-426f-4e5c-a635-79f99b56ea79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:59.423440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9467c1a-426f-4e5c-a635-79f99b56ea79 admin admin] The token cache is being invalidate because project e3fea7ae388847fc9379dc1281a4c38c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:52:59.424013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9467c1a-426f-4e5c-a635-79f99b56ea79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:59.424821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9467c1a-426f-4e5c-a635-79f99b56ea79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:52:59.434760 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1382/2727] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:52:59 2022] DELETE /identity/v3/projects/e3fea7ae388847fc9379dc1281a4c38c => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:52:59.439607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa04a61-40cd-41a5-b457-edcbe09f228e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:59.440090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa04a61-40cd-41a5-b457-edcbe09f228e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:59.440544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa04a61-40cd-41a5-b457-edcbe09f228e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:59.489652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-daa04a61-40cd-41a5-b457-edcbe09f228e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:59.491612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-daa04a61-40cd-41a5-b457-edcbe09f228e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:59.559994 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1346/2728] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:59.567558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a164423-de72-4fa9-88d6-75cd9427786a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:59.567842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a164423-de72-4fa9-88d6-75cd9427786a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:59.568330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a164423-de72-4fa9-88d6-75cd9427786a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:59.591655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8a164423-de72-4fa9-88d6-75cd9427786a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:52:59.594203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8a164423-de72-4fa9-88d6-75cd9427786a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:52:59.615565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa04a61-40cd-41a5-b457-edcbe09f228e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:59.616549 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1383/2729] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:52:59.630491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa04a61-40cd-41a5-b457-edcbe09f228e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:59.637593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c21b02-d885-4b0f-9cf5-3bb182901703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:59.639318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c21b02-d885-4b0f-9cf5-3bb182901703 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:59.639849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c21b02-d885-4b0f-9cf5-3bb182901703 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:59.640298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c21b02-d885-4b0f-9cf5-3bb182901703 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:59.641150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c21b02-d885-4b0f-9cf5-3bb182901703 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:52:59.641605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c21b02-d885-4b0f-9cf5-3bb182901703 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:59.645378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c21b02-d885-4b0f-9cf5-3bb182901703 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:59.669158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a164423-de72-4fa9-88d6-75cd9427786a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:59.675339 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1347/2730] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:52:59.682784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a164423-de72-4fa9-88d6-75cd9427786a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:59.688310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed3aba-02aa-4915-9d73-02e7e1958b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:59.689034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed3aba-02aa-4915-9d73-02e7e1958b45 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:59.689390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed3aba-02aa-4915-9d73-02e7e1958b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:59.689901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed3aba-02aa-4915-9d73-02e7e1958b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:59.690622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed3aba-02aa-4915-9d73-02e7e1958b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:59.693561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed3aba-02aa-4915-9d73-02e7e1958b45 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:59.744857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c21b02-d885-4b0f-9cf5-3bb182901703 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:59.751629 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1384/2731] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:52:59 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:52:59.759499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c21b02-d885-4b0f-9cf5-3bb182901703 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:59.870903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e494c-eb8b-4619-b183-6326705b328b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:59.872014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e494c-eb8b-4619-b183-6326705b328b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:59.872407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e494c-eb8b-4619-b183-6326705b328b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:59.872789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e494c-eb8b-4619-b183-6326705b328b admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:59.873839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e494c-eb8b-4619-b183-6326705b328b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:59.877231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e494c-eb8b-4619-b183-6326705b328b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:59.919540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed3aba-02aa-4915-9d73-02e7e1958b45 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:59.931740 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1348/2732] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:52:59 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:52:59.934181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed3aba-02aa-4915-9d73-02e7e1958b45 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:59.939595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24943d-92d4-43e0-a4ba-f5a034e57730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:59.940356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24943d-92d4-43e0-a4ba-f5a034e57730 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:59.940743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24943d-92d4-43e0-a4ba-f5a034e57730 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:59.941299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24943d-92d4-43e0-a4ba-f5a034e57730 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:59.942110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24943d-92d4-43e0-a4ba-f5a034e57730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:59.944383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24943d-92d4-43e0-a4ba-f5a034e57730 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:52:59.962305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e494c-eb8b-4619-b183-6326705b328b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:52:59.973325 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1385/2733] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:52:59.979243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e494c-eb8b-4619-b183-6326705b328b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:52:59.985506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fa67b-febc-4f38-8c23-e3ef1a1fbd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:52:59.986549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa67b-febc-4f38-8c23-e3ef1a1fbd41 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:52:59.987358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa67b-febc-4f38-8c23-e3ef1a1fbd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:52:59.987651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa67b-febc-4f38-8c23-e3ef1a1fbd41 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:52:59.988315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fa67b-febc-4f38-8c23-e3ef1a1fbd41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:52:59.990023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fa67b-febc-4f38-8c23-e3ef1a1fbd41 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:00.014330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24943d-92d4-43e0-a4ba-f5a034e57730 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:00.020742 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1349/2734] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:53:00.028347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24943d-92d4-43e0-a4ba-f5a034e57730 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:00.036853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f853705-1ffe-4a51-b203-d3904db2bd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:00.038255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f853705-1ffe-4a51-b203-d3904db2bd47 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:00.038624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f853705-1ffe-4a51-b203-d3904db2bd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:00.038915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f853705-1ffe-4a51-b203-d3904db2bd47 admin admin] PATH_INFO: `/v3/projects/23ef148ffad044529cb3658ac1ed3107/users/0f8ece445698482795d9b9d5ad5b13aa/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:00.042847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:00.070253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f853705-1ffe-4a51-b203-d3904db2bd47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ef148ffad044529cb3658ac1ed3107, user_id=0f8ece445698482795d9b9d5ad5b13aa, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:00.074415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f853705-1ffe-4a51-b203-d3904db2bd47 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:00.093203 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1386/2735] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:53:00 2022] PUT /identity/v3/projects/23ef148ffad044529cb3658ac1ed3107/users/0f8ece445698482795d9b9d5ad5b13aa/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:53:00.107772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68f8b4-9766-400e-9905-380f204b9211 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:00.108650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68f8b4-9766-400e-9905-380f204b9211 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:00.109112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68f8b4-9766-400e-9905-380f204b9211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:00.168958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ce68f8b4-9766-400e-9905-380f204b9211 None None] MFA Rules not processed for user `0f8ece445698482795d9b9d5ad5b13aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:00.170574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ce68f8b4-9766-400e-9905-380f204b9211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:00.255787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:00.273896 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1350/2736] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:00.326127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f853705-1ffe-4a51-b203-d3904db2bd47 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:00.328270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1f853705-1ffe-4a51-b203-d3904db2bd47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:00.401454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f853705-1ffe-4a51-b203-d3904db2bd47 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:00.406516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aac7bd-2ac0-435c-9cc0-1d6bf829c164 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:00.407419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aac7bd-2ac0-435c-9cc0-1d6bf829c164 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:00.407923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aac7bd-2ac0-435c-9cc0-1d6bf829c164 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:00.408588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aac7bd-2ac0-435c-9cc0-1d6bf829c164 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:00.419014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aac7bd-2ac0-435c-9cc0-1d6bf829c164 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:00.420864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aac7bd-2ac0-435c-9cc0-1d6bf829c164 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:00.442467 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1387/2737] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:53:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:53:01.589388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451da04-8034-4977-8d43-fc23b9b38212 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:01.589992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451da04-8034-4977-8d43-fc23b9b38212 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:01.589992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451da04-8034-4977-8d43-fc23b9b38212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:01.631798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0451da04-8034-4977-8d43-fc23b9b38212 None None] MFA Rules not processed for user `2e11c33c786746e5b3effedd32674582`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:01.633055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0451da04-8034-4977-8d43-fc23b9b38212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:01.679758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab506034-8ea3-49df-99c3-26d0451ae245 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:01.680872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab506034-8ea3-49df-99c3-26d0451ae245 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:01.680872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab506034-8ea3-49df-99c3-26d0451ae245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:01.709300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ab506034-8ea3-49df-99c3-26d0451ae245 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:01.710880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ab506034-8ea3-49df-99c3-26d0451ae245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:01.745959 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1351/2738] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:01.833472 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1388/2739] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:53:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:53:01.849774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451da04-8034-4977-8d43-fc23b9b38212 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:01.852408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0451da04-8034-4977-8d43-fc23b9b38212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:01.981404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451da04-8034-4977-8d43-fc23b9b38212 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:01.988496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e2969-7c00-4ad9-8ea7-dc0b65a9d516 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:01.989164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e2969-7c00-4ad9-8ea7-dc0b65a9d516 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:01.989456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e2969-7c00-4ad9-8ea7-dc0b65a9d516 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:01.989735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e2969-7c00-4ad9-8ea7-dc0b65a9d516 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:02.000678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e2969-7c00-4ad9-8ea7-dc0b65a9d516 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:02.002420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e2969-7c00-4ad9-8ea7-dc0b65a9d516 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:02.011808 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1352/2740] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:53:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:53:03.105275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4cbe9-6ddc-44a5-9a7a-cdaed2b169b1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:03.106270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4cbe9-6ddc-44a5-9a7a-cdaed2b169b1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:03.106896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4cbe9-6ddc-44a5-9a7a-cdaed2b169b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:03.147645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-03a4cbe9-6ddc-44a5-9a7a-cdaed2b169b1 None None] MFA Rules not processed for user `86035889fc474973bc31a97989ec26fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:03.149496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-03a4cbe9-6ddc-44a5-9a7a-cdaed2b169b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:03.229038 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1389/2741] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:03.448965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257a648-91f1-4525-a150-49222faca117 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:03.449494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257a648-91f1-4525-a150-49222faca117 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:03.449849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257a648-91f1-4525-a150-49222faca117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:03.474665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0257a648-91f1-4525-a150-49222faca117 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:03.476105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0257a648-91f1-4525-a150-49222faca117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:03.487188 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1353/2742] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:53:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:53:03.505698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4cbe9-6ddc-44a5-9a7a-cdaed2b169b1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:03.524530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4cbe9-6ddc-44a5-9a7a-cdaed2b169b1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:03.530544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70fdd3-2b3f-4f55-931c-1ef45747cc74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:03.531392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70fdd3-2b3f-4f55-931c-1ef45747cc74 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:03.531853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70fdd3-2b3f-4f55-931c-1ef45747cc74 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:03.532305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70fdd3-2b3f-4f55-931c-1ef45747cc74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:03.542812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70fdd3-2b3f-4f55-931c-1ef45747cc74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:03.544789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70fdd3-2b3f-4f55-931c-1ef45747cc74 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:03.554707 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1390/2743] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:53:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:53:03.592123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d772a8-41b4-4fd5-90bf-e27d55943886 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:03.593336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d772a8-41b4-4fd5-90bf-e27d55943886 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:03.593715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d772a8-41b4-4fd5-90bf-e27d55943886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:03.622891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-17d772a8-41b4-4fd5-90bf-e27d55943886 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:03.624700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-17d772a8-41b4-4fd5-90bf-e27d55943886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:03.634363 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1354/2744] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:53:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:03.655053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70fdd3-2b3f-4f55-931c-1ef45747cc74 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:03.670238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70fdd3-2b3f-4f55-931c-1ef45747cc74 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:03.676524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05d93a-c481-405b-ab1f-732160f493d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:03.677577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05d93a-c481-405b-ab1f-732160f493d9 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:03.678058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05d93a-c481-405b-ab1f-732160f493d9 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:03.678897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05d93a-c481-405b-ab1f-732160f493d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:03.690523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05d93a-c481-405b-ab1f-732160f493d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:03.692998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05d93a-c481-405b-ab1f-732160f493d9 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:03.717780 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1391/2745] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:53:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:53:04.271145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dff6a-743e-475a-b34d-bc6258fced73 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:04.271429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dff6a-743e-475a-b34d-bc6258fced73 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:04.271896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dff6a-743e-475a-b34d-bc6258fced73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:04.302615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-396dff6a-743e-475a-b34d-bc6258fced73 None None] MFA Rules not processed for user `d9688dca94a841cfa29420ad780562cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:04.303914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-396dff6a-743e-475a-b34d-bc6258fced73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:04.410569 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1355/2746] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:05.768155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77010e58-708f-4d1e-880c-cfa05e9a8a17 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:05.768648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77010e58-708f-4d1e-880c-cfa05e9a8a17 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:05.768757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77010e58-708f-4d1e-880c-cfa05e9a8a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:05.807762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-77010e58-708f-4d1e-880c-cfa05e9a8a17 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:05.809031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-77010e58-708f-4d1e-880c-cfa05e9a8a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:05.818493 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1392/2747] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:53:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:05.837433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dff6a-743e-475a-b34d-bc6258fced73 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:05.894895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dff6a-743e-475a-b34d-bc6258fced73 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:05.913579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18b25e-9ed0-4258-b3d7-e63fd74f3c8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:05.914315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18b25e-9ed0-4258-b3d7-e63fd74f3c8b service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:05.915372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18b25e-9ed0-4258-b3d7-e63fd74f3c8b service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:05.916093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18b25e-9ed0-4258-b3d7-e63fd74f3c8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:05.944952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18b25e-9ed0-4258-b3d7-e63fd74f3c8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:05.951387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18b25e-9ed0-4258-b3d7-e63fd74f3c8b service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:05.981108 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1356/2748] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:53:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:53:06.287381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfd65e-beca-4bf7-942c-9bbfd48551dc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:06.287666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfd65e-beca-4bf7-942c-9bbfd48551dc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:06.288151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfd65e-beca-4bf7-942c-9bbfd48551dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:06.298165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:06.315745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-03bfd65e-beca-4bf7-942c-9bbfd48551dc None None] MFA Rules not processed for user `0f8ece445698482795d9b9d5ad5b13aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:06.317600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-03bfd65e-beca-4bf7-942c-9bbfd48551dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:06.325852 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1393/2749] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:06.850713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5d8f0-4da4-429a-8148-ad21be5422ea None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:06.851578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5d8f0-4da4-429a-8148-ad21be5422ea None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:06.851865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5d8f0-4da4-429a-8148-ad21be5422ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:06.874025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f9c5d8f0-4da4-429a-8148-ad21be5422ea None None] MFA Rules not processed for user `0f8ece445698482795d9b9d5ad5b13aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:06.875380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f9c5d8f0-4da4-429a-8148-ad21be5422ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:06.888857 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1357/2750] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:08.262677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7adc20-3469-4462-a2ed-6f58e7b3ea5d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:08.264803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7adc20-3469-4462-a2ed-6f58e7b3ea5d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:08.265304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7adc20-3469-4462-a2ed-6f58e7b3ea5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:08.291806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2b7adc20-3469-4462-a2ed-6f58e7b3ea5d None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:08.293690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b7adc20-3469-4462-a2ed-6f58e7b3ea5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:08.323130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5d8f0-4da4-429a-8148-ad21be5422ea None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:08.331854 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1394/2751] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:53:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:08.332138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:08.340168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5d8f0-4da4-429a-8148-ad21be5422ea None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:08.348592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4916a9-9bbf-438b-9831-2a57c395b63f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:08.349505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4916a9-9bbf-438b-9831-2a57c395b63f service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:08.349982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4916a9-9bbf-438b-9831-2a57c395b63f service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:08.350549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4916a9-9bbf-438b-9831-2a57c395b63f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:08.362482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4916a9-9bbf-438b-9831-2a57c395b63f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:08.364785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4916a9-9bbf-438b-9831-2a57c395b63f service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:08.386523 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1358/2752] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:53:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:53:18.284023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:18.372064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:22.526965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6f25-55ad-4767-ae02-b2a397260447 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:22.527426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6f25-55ad-4767-ae02-b2a397260447 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:22.527530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6f25-55ad-4767-ae02-b2a397260447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:22.571205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1e5e6f25-55ad-4767-ae02-b2a397260447 None None] MFA Rules not processed for user `778561ee5e474accb31816f357985f44`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:22.572630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1e5e6f25-55ad-4767-ae02-b2a397260447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:22.680010 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1395/2753] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:22 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:26.974274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c451f-998b-420a-87d6-f1ff31a47e5e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:26.974758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c451f-998b-420a-87d6-f1ff31a47e5e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:26.974909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c451f-998b-420a-87d6-f1ff31a47e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:27.008923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1e9c451f-998b-420a-87d6-f1ff31a47e5e None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:27.010941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1e9c451f-998b-420a-87d6-f1ff31a47e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:27.018418 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1359/2754] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:53:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:27.032505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6f25-55ad-4767-ae02-b2a397260447 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:27.046740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6f25-55ad-4767-ae02-b2a397260447 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:27.051313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada087f-a1d1-4a76-9c4a-5be3ad6f6eaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:27.052035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada087f-a1d1-4a76-9c4a-5be3ad6f6eaa service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:27.052387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada087f-a1d1-4a76-9c4a-5be3ad6f6eaa service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:27.052738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada087f-a1d1-4a76-9c4a-5be3ad6f6eaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:27.061477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada087f-a1d1-4a76-9c4a-5be3ad6f6eaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:27.063142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada087f-a1d1-4a76-9c4a-5be3ad6f6eaa service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:27.072044 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1396/2755] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:53:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:53:34.300280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea27c10-e3a0-4567-91e8-2246ca8330b6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:34.300821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea27c10-e3a0-4567-91e8-2246ca8330b6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:34.300821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea27c10-e3a0-4567-91e8-2246ca8330b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:34.323534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-eea27c10-e3a0-4567-91e8-2246ca8330b6 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:34.324917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-eea27c10-e3a0-4567-91e8-2246ca8330b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:34.337057 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1360/2756] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:53:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:34.348243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada087f-a1d1-4a76-9c4a-5be3ad6f6eaa service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:34.359550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada087f-a1d1-4a76-9c4a-5be3ad6f6eaa service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:34.366007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eca3d7-0edb-43f0-878d-42164e61da1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:34.366624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eca3d7-0edb-43f0-878d-42164e61da1c service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:34.367041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eca3d7-0edb-43f0-878d-42164e61da1c service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:34.367508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eca3d7-0edb-43f0-878d-42164e61da1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:34.378210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eca3d7-0edb-43f0-878d-42164e61da1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:34.380231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eca3d7-0edb-43f0-878d-42164e61da1c service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:34.393774 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1397/2757] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:53:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:53:44.317410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:44.389021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=37 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:47.161906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea27c10-e3a0-4567-91e8-2246ca8330b6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:47.163916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-eea27c10-e3a0-4567-91e8-2246ca8330b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:47.185851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea27c10-e3a0-4567-91e8-2246ca8330b6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:47.188654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eca3d7-0edb-43f0-878d-42164e61da1c service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:47.190601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-08eca3d7-0edb-43f0-878d-42164e61da1c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:47.198811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778d155-addc-4ab0-991e-cd64c3caf63c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:47.199785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778d155-addc-4ab0-991e-cd64c3caf63c service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:47.203405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778d155-addc-4ab0-991e-cd64c3caf63c service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:47.203893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778d155-addc-4ab0-991e-cd64c3caf63c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:47.206510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f778d155-addc-4ab0-991e-cd64c3caf63c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:47.328380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eca3d7-0edb-43f0-878d-42164e61da1c service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:47.362190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ff19e-5528-47e8-82ae-357ab55a03c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:47.363609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ff19e-5528-47e8-82ae-357ab55a03c1 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:47.364085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ff19e-5528-47e8-82ae-357ab55a03c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:47.364525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ff19e-5528-47e8-82ae-357ab55a03c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:47.367566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5f5ff19e-5528-47e8-82ae-357ab55a03c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:47.393025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ff19e-5528-47e8-82ae-357ab55a03c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:47.397223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ff19e-5528-47e8-82ae-357ab55a03c1 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:47.423403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778d155-addc-4ab0-991e-cd64c3caf63c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:47.424876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778d155-addc-4ab0-991e-cd64c3caf63c service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:47.441127 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1398/2758] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:53:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:53:47.451276 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1361/2759] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:53:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:53:49.436704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2930225-caef-4a4b-a6e6-9a376399c784 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:49.437255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2930225-caef-4a4b-a6e6-9a376399c784 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:49.437452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2930225-caef-4a4b-a6e6-9a376399c784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:49.461986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e2930225-caef-4a4b-a6e6-9a376399c784 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:49.463973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e2930225-caef-4a4b-a6e6-9a376399c784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:49.474561 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1399/2760] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:53:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:53:49.488876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778d155-addc-4ab0-991e-cd64c3caf63c service glance] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:49.503247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778d155-addc-4ab0-991e-cd64c3caf63c service glance] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:49.510459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5961f-03fd-41fb-85cd-d8decd5f05c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:49.511347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5961f-03fd-41fb-85cd-d8decd5f05c8 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:49.511845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5961f-03fd-41fb-85cd-d8decd5f05c8 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:49.512321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5961f-03fd-41fb-85cd-d8decd5f05c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:49.522248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5961f-03fd-41fb-85cd-d8decd5f05c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:49.524315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5961f-03fd-41fb-85cd-d8decd5f05c8 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:49.533677 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1362/2761] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:53:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:53:50.406631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c010b5-509f-4d52-99e5-dce5ce4bcc36 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:50.406913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c010b5-509f-4d52-99e5-dce5ce4bcc36 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:50.407182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c010b5-509f-4d52-99e5-dce5ce4bcc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:50.446234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-19c010b5-509f-4d52-99e5-dce5ce4bcc36 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:50.447448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-19c010b5-509f-4d52-99e5-dce5ce4bcc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:50.464880 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1400/2762] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:50.479777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5961f-03fd-41fb-85cd-d8decd5f05c8 service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:50.497320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5961f-03fd-41fb-85cd-d8decd5f05c8 service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:50.502986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4f90e-9f6d-4f19-8452-0de243112141 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:50.503729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4f90e-9f6d-4f19-8452-0de243112141 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:50.504122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4f90e-9f6d-4f19-8452-0de243112141 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:50.504504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4f90e-9f6d-4f19-8452-0de243112141 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:50.516078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4f90e-9f6d-4f19-8452-0de243112141 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:50.519725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4f90e-9f6d-4f19-8452-0de243112141 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:50.531234 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1363/2763] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:53:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:53:53.925246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c010b5-509f-4d52-99e5-dce5ce4bcc36 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:53.926820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-19c010b5-509f-4d52-99e5-dce5ce4bcc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:54.193319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c010b5-509f-4d52-99e5-dce5ce4bcc36 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:54.210559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e849e-cbe5-4566-9fc6-45ec4c46f20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:54.211665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e849e-cbe5-4566-9fc6-45ec4c46f20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:54.212742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e849e-cbe5-4566-9fc6-45ec4c46f20b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:54.213202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e849e-cbe5-4566-9fc6-45ec4c46f20b admin admin] PATH_INFO: `/v3/users/86035889fc474973bc31a97989ec26fd` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:54.214624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e849e-cbe5-4566-9fc6-45ec4c46f20b admin admin] RBAC: Authorizing `identity:delete_user(user_id=86035889fc474973bc31a97989ec26fd)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:54.216728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e849e-cbe5-4566-9fc6-45ec4c46f20b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:54.424775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-315e849e-cbe5-4566-9fc6-45ec4c46f20b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86035889fc474973bc31a97989ec26fd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:54.484348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-315e849e-cbe5-4566-9fc6-45ec4c46f20b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86035889fc474973bc31a97989ec26fd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:54.495373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-315e849e-cbe5-4566-9fc6-45ec4c46f20b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86035889fc474973bc31a97989ec26fd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:54.498442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-315e849e-cbe5-4566-9fc6-45ec4c46f20b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86035889fc474973bc31a97989ec26fd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:54.502475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-315e849e-cbe5-4566-9fc6-45ec4c46f20b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86035889fc474973bc31a97989ec26fd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:54.535628 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1401/2764] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:53:53 2022] DELETE /identity/v3/users/86035889fc474973bc31a97989ec26fd => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:53:54.986406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4f90e-9f6d-4f19-8452-0de243112141 service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:55.000818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4f90e-9f6d-4f19-8452-0de243112141 service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:55.108705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c925aa-17da-42bf-a1c1-39ad62122b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:55.110599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c925aa-17da-42bf-a1c1-39ad62122b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:55.111051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c925aa-17da-42bf-a1c1-39ad62122b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:55.111520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c925aa-17da-42bf-a1c1-39ad62122b97 admin admin] PATH_INFO: `/v3/projects/c55700f621564ad5a52056d7a7854e20` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:55.113000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c925aa-17da-42bf-a1c1-39ad62122b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c55700f621564ad5a52056d7a7854e20)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:55.115288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c925aa-17da-42bf-a1c1-39ad62122b97 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:55.192060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-20c925aa-17da-42bf-a1c1-39ad62122b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c55700f621564ad5a52056d7a7854e20', 'request_id': 'req-20c925aa-17da-42bf-a1c1-39ad62122b97'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:55.201580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-20c925aa-17da-42bf-a1c1-39ad62122b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c55700f621564ad5a52056d7a7854e20', 'request_id': 'req-20c925aa-17da-42bf-a1c1-39ad62122b97'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:55.209812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-20c925aa-17da-42bf-a1c1-39ad62122b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c55700f621564ad5a52056d7a7854e20', 'request_id': 'req-20c925aa-17da-42bf-a1c1-39ad62122b97'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:55.211284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:55.230692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-20c925aa-17da-42bf-a1c1-39ad62122b97 admin admin] The token cache is being invalidate because project c55700f621564ad5a52056d7a7854e20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:53:55.231053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-20c925aa-17da-42bf-a1c1-39ad62122b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:55.231945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-20c925aa-17da-42bf-a1c1-39ad62122b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:55.237300 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1364/2765] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:53:54 2022] DELETE /identity/v3/projects/c55700f621564ad5a52056d7a7854e20 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:53:55.239256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e71de0-5642-47d7-825b-ce917811fed6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:55.239850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e71de0-5642-47d7-825b-ce917811fed6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:55.239950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e71de0-5642-47d7-825b-ce917811fed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:55.280533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-98e71de0-5642-47d7-825b-ce917811fed6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:55.282676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-98e71de0-5642-47d7-825b-ce917811fed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:55.289785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:55.411711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2f62d-5753-4fc3-b232-a717049878f6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:55.412833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2f62d-5753-4fc3-b232-a717049878f6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:55.413906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2f62d-5753-4fc3-b232-a717049878f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:55.418748 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1402/2766] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:55.465611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-16c2f62d-5753-4fc3-b232-a717049878f6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:55.466993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-16c2f62d-5753-4fc3-b232-a717049878f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:55.483784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e71de0-5642-47d7-825b-ce917811fed6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:55.489879 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1365/2767] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:55.498778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e71de0-5642-47d7-825b-ce917811fed6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:55.507875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15afab-7c4c-4f70-8615-e54060ed5578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:55.507875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15afab-7c4c-4f70-8615-e54060ed5578 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:55.507875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15afab-7c4c-4f70-8615-e54060ed5578 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:55.507875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15afab-7c4c-4f70-8615-e54060ed5578 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:55.509013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15afab-7c4c-4f70-8615-e54060ed5578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:53:55.509653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15afab-7c4c-4f70-8615-e54060ed5578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:55.511482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15afab-7c4c-4f70-8615-e54060ed5578 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:55.539555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2f62d-5753-4fc3-b232-a717049878f6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:55.551816 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1403/2768] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:53:55.555612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2f62d-5753-4fc3-b232-a717049878f6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:55.562253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816dfce-bb67-4870-a853-9edf2aa2068e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:55.563541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816dfce-bb67-4870-a853-9edf2aa2068e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:55.564144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816dfce-bb67-4870-a853-9edf2aa2068e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:55.565100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816dfce-bb67-4870-a853-9edf2aa2068e admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:55.566889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816dfce-bb67-4870-a853-9edf2aa2068e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:55.570766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816dfce-bb67-4870-a853-9edf2aa2068e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:55.618043 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1366/2769] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:53:55 2022] POST /identity/v3/projects => generated 397 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:53:55.622292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15afab-7c4c-4f70-8615-e54060ed5578 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:55.641432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15afab-7c4c-4f70-8615-e54060ed5578 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:55.729098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a5f6a-d1e6-486b-84d8-ec75e4ad3f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:55.729728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5f6a-d1e6-486b-84d8-ec75e4ad3f38 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:55.730016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5f6a-d1e6-486b-84d8-ec75e4ad3f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:55.730295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5f6a-d1e6-486b-84d8-ec75e4ad3f38 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:55.730934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a5f6a-d1e6-486b-84d8-ec75e4ad3f38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:55.748179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a5f6a-d1e6-486b-84d8-ec75e4ad3f38 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:55.839680 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1404/2770] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:53:55 2022] POST /identity/v3/users => generated 299 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:53:55.843807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816dfce-bb67-4870-a853-9edf2aa2068e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:55.859974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816dfce-bb67-4870-a853-9edf2aa2068e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:55.875651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf409844-fd61-456c-b7a3-a233290ff32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:55.876394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf409844-fd61-456c-b7a3-a233290ff32b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:55.876754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf409844-fd61-456c-b7a3-a233290ff32b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:55.877265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf409844-fd61-456c-b7a3-a233290ff32b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:55.877992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf409844-fd61-456c-b7a3-a233290ff32b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:55.883054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf409844-fd61-456c-b7a3-a233290ff32b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:55.910529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a5f6a-d1e6-486b-84d8-ec75e4ad3f38 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:55.911032 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1367/2771] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:53:55.912079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf409844-fd61-456c-b7a3-a233290ff32b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:55.919602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bf409844-fd61-456c-b7a3-a233290ff32b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:55.930159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a5f6a-d1e6-486b-84d8-ec75e4ad3f38 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:55.943047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02093d0b-87a4-4fb6-99fd-20d65d69d7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:55.945129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02093d0b-87a4-4fb6-99fd-20d65d69d7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:55.945626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02093d0b-87a4-4fb6-99fd-20d65d69d7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:55.946090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02093d0b-87a4-4fb6-99fd-20d65d69d7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:55.946928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02093d0b-87a4-4fb6-99fd-20d65d69d7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:55.950294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02093d0b-87a4-4fb6-99fd-20d65d69d7b9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:55.962492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf409844-fd61-456c-b7a3-a233290ff32b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:55.968473 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1405/2772] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:53:55.970882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b5465-9181-4393-95dc-21d7156b56bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:55.972959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02093d0b-87a4-4fb6-99fd-20d65d69d7b9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:55.974414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5465-9181-4393-95dc-21d7156b56bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:55.982051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5465-9181-4393-95dc-21d7156b56bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:55.983096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5465-9181-4393-95dc-21d7156b56bf admin admin] PATH_INFO: `/v3/users/ddb7b29dc8464329abda9836b09b907f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:55.985294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5465-9181-4393-95dc-21d7156b56bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddb7b29dc8464329abda9836b09b907f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:55.987584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5465-9181-4393-95dc-21d7156b56bf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:56.000276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02093d0b-87a4-4fb6-99fd-20d65d69d7b9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:56.018052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d2532-2b7a-43ce-9425-be5b9a4b2e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:56.018979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d2532-2b7a-43ce-9425-be5b9a4b2e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:56.020501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d2532-2b7a-43ce-9425-be5b9a4b2e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:56.021274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d2532-2b7a-43ce-9425-be5b9a4b2e6f admin admin] PATH_INFO: `/v3/projects/5b635bf0f5f24567b72a5f02ea6a57a8/users/761d28842ab9456fb877864cac93fbbc/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:56.059074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d2532-2b7a-43ce-9425-be5b9a4b2e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b635bf0f5f24567b72a5f02ea6a57a8, user_id=761d28842ab9456fb877864cac93fbbc, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:56.065852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d2532-2b7a-43ce-9425-be5b9a4b2e6f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:56.111969 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1406/2773] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:53:55 2022] PUT /identity/v3/projects/5b635bf0f5f24567b72a5f02ea6a57a8/users/761d28842ab9456fb877864cac93fbbc/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:53:56.114724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f3a37-cd6e-482f-bd33-f9ef03264d54 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:56.141606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f3a37-cd6e-482f-bd33-f9ef03264d54 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:56.143052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f3a37-cd6e-482f-bd33-f9ef03264d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:56.147121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-780b5465-9181-4393-95dc-21d7156b56bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddb7b29dc8464329abda9836b09b907f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:56.178617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-780b5465-9181-4393-95dc-21d7156b56bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddb7b29dc8464329abda9836b09b907f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:56.180106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-780b5465-9181-4393-95dc-21d7156b56bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddb7b29dc8464329abda9836b09b907f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:56.222864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-780b5465-9181-4393-95dc-21d7156b56bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddb7b29dc8464329abda9836b09b907f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:56.227276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-629f3a37-cd6e-482f-bd33-f9ef03264d54 None None] MFA Rules not processed for user `761d28842ab9456fb877864cac93fbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:56.228679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-629f3a37-cd6e-482f-bd33-f9ef03264d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:56.243979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-780b5465-9181-4393-95dc-21d7156b56bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddb7b29dc8464329abda9836b09b907f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:56.265268 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1368/2774] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:53:55 2022] DELETE /identity/v3/users/ddb7b29dc8464329abda9836b09b907f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:53:56.267201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59818-fcce-4dd0-acb6-78815e046914 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:56.269279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59818-fcce-4dd0-acb6-78815e046914 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:56.270570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59818-fcce-4dd0-acb6-78815e046914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:56.286314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:56.331640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a7b59818-fcce-4dd0-acb6-78815e046914 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:56.333666 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1407/2775] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:56 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:56.334325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a7b59818-fcce-4dd0-acb6-78815e046914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:56.335494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d939ef-bcfd-43a4-a08e-2537742c8be7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:56.335840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d939ef-bcfd-43a4-a08e-2537742c8be7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:56.336121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d939ef-bcfd-43a4-a08e-2537742c8be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:56.370086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-30d939ef-bcfd-43a4-a08e-2537742c8be7 None None] MFA Rules not processed for user `761d28842ab9456fb877864cac93fbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:56.373347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-30d939ef-bcfd-43a4-a08e-2537742c8be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:56.463882 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1369/2776] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:56.470640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59818-fcce-4dd0-acb6-78815e046914 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:56.477537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a7b59818-fcce-4dd0-acb6-78815e046914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:56.534306 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1408/2777] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:56 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:56.540427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:56.596017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59818-fcce-4dd0-acb6-78815e046914 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:56.600272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7113f3e-811f-452a-b031-b484486419a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:56.601533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7113f3e-811f-452a-b031-b484486419a2 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:56.601824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7113f3e-811f-452a-b031-b484486419a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:56.602105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7113f3e-811f-452a-b031-b484486419a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:56.613237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7113f3e-811f-452a-b031-b484486419a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:56.615104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7113f3e-811f-452a-b031-b484486419a2 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:56.627685 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1370/2778] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:53:56.699477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d939ef-bcfd-43a4-a08e-2537742c8be7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:56.701199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-30d939ef-bcfd-43a4-a08e-2537742c8be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:56.724852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7113f3e-811f-452a-b031-b484486419a2 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:56.728539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c7113f3e-811f-452a-b031-b484486419a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:56.766310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7113f3e-811f-452a-b031-b484486419a2 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:56.795782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffe300-f776-4919-9052-a53f243ec867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:56.802854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffe300-f776-4919-9052-a53f243ec867 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:56.803803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffe300-f776-4919-9052-a53f243ec867 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:56.805528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffe300-f776-4919-9052-a53f243ec867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:56.813383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9fffe300-f776-4919-9052-a53f243ec867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:56.831000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffe300-f776-4919-9052-a53f243ec867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:56.834798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffe300-f776-4919-9052-a53f243ec867 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:56.852009 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1371/2779] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:53:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:53:56.908972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d939ef-bcfd-43a4-a08e-2537742c8be7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:56.915378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb961d5-d31f-401f-b6fa-0c949170e286 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:56.916200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb961d5-d31f-401f-b6fa-0c949170e286 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:56.916642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb961d5-d31f-401f-b6fa-0c949170e286 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:56.917101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb961d5-d31f-401f-b6fa-0c949170e286 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:56.931944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb961d5-d31f-401f-b6fa-0c949170e286 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:56.933970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb961d5-d31f-401f-b6fa-0c949170e286 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:56.957773 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1409/2780] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:53:56 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:53:57.043988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffe300-f776-4919-9052-a53f243ec867 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:57.044677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9fffe300-f776-4919-9052-a53f243ec867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:57.060485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb048fa0-fea3-445c-800f-01cfa417fa38 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:57.060805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb048fa0-fea3-445c-800f-01cfa417fa38 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:57.061122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb048fa0-fea3-445c-800f-01cfa417fa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:57.114065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffe300-f776-4919-9052-a53f243ec867 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:57.161834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-eb048fa0-fea3-445c-800f-01cfa417fa38 None None] MFA Rules not processed for user `761d28842ab9456fb877864cac93fbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:57.162059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:57.162059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:57.162059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:57.162302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-eb048fa0-fea3-445c-800f-01cfa417fa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:57.162413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27 admin admin] PATH_INFO: `/v3/projects/44e122acac834fc2843f725beb5daeff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:57.165115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44e122acac834fc2843f725beb5daeff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:57.172361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:57.192674 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1410/2781] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:57 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:57.272181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f5aca-44ff-4f9d-9235-a0722372b3ab None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:57.272553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f5aca-44ff-4f9d-9235-a0722372b3ab None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:57.272816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f5aca-44ff-4f9d-9235-a0722372b3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:57.336592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4c1f5aca-44ff-4f9d-9235-a0722372b3ab None None] MFA Rules not processed for user `761d28842ab9456fb877864cac93fbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:57.338328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4c1f5aca-44ff-4f9d-9235-a0722372b3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:57.353223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44e122acac834fc2843f725beb5daeff', 'request_id': 'req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:57.375240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44e122acac834fc2843f725beb5daeff', 'request_id': 'req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:57.388498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44e122acac834fc2843f725beb5daeff', 'request_id': 'req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:57.419820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27 admin admin] The token cache is being invalidate because project 44e122acac834fc2843f725beb5daeff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:53:57.420114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:57.422729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-49f2a6c4-31f4-496b-82cf-d1b4864bfa27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:57.430141 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1372/2782] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:53:57 2022] DELETE /identity/v3/projects/44e122acac834fc2843f725beb5daeff => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:53:57.432004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75472f-7143-4362-ae80-53f2ad809923 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:57.432294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75472f-7143-4362-ae80-53f2ad809923 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:57.432571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75472f-7143-4362-ae80-53f2ad809923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:57.488783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ae75472f-7143-4362-ae80-53f2ad809923 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:57.493913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ae75472f-7143-4362-ae80-53f2ad809923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:57.505507 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1411/2783] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:57 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:57.542468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983df2c8-d479-41a3-bf43-fe7ae92c43b6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:57.542748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983df2c8-d479-41a3-bf43-fe7ae92c43b6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:57.543027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983df2c8-d479-41a3-bf43-fe7ae92c43b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:57.575571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-983df2c8-d479-41a3-bf43-fe7ae92c43b6 None None] MFA Rules not processed for user `761d28842ab9456fb877864cac93fbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:57.577405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-983df2c8-d479-41a3-bf43-fe7ae92c43b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:57.610607 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1412/2784] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:57 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:57.628335 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1373/2785] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:53:57.636257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058472a-b3a2-4151-9ef1-ea60e451d413 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:57.636566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058472a-b3a2-4151-9ef1-ea60e451d413 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:57.636819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058472a-b3a2-4151-9ef1-ea60e451d413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:57.650930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:57.668679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d499f-ba26-483d-8025-9d17fd04e3f1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:57.669084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d499f-ba26-483d-8025-9d17fd04e3f1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:57.669455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d499f-ba26-483d-8025-9d17fd04e3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:57.693981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3058472a-b3a2-4151-9ef1-ea60e451d413 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:57.694727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-802d499f-ba26-483d-8025-9d17fd04e3f1 None None] MFA Rules not processed for user `761d28842ab9456fb877864cac93fbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:57.695358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3058472a-b3a2-4151-9ef1-ea60e451d413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:57.696161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-802d499f-ba26-483d-8025-9d17fd04e3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:57.708271 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1374/2786] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:57 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:57.734435 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1413/2787] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:57.737620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d499f-ba26-483d-8025-9d17fd04e3f1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:57.752205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058472a-b3a2-4151-9ef1-ea60e451d413 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:57.752395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3058472a-b3a2-4151-9ef1-ea60e451d413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:57.771222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d499f-ba26-483d-8025-9d17fd04e3f1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:57.772257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058472a-b3a2-4151-9ef1-ea60e451d413 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:57.784954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a2542-6cf9-4ffd-89f0-f5e2ed3d676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:57.785642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a2542-6cf9-4ffd-89f0-f5e2ed3d676a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:57.788069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712684c0-66fd-4235-80e0-dd65cc8ce5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:57.789057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a2542-6cf9-4ffd-89f0-f5e2ed3d676a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:57.789291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712684c0-66fd-4235-80e0-dd65cc8ce5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:57.789405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a2542-6cf9-4ffd-89f0-f5e2ed3d676a admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:57.789571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712684c0-66fd-4235-80e0-dd65cc8ce5da admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:57.789827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712684c0-66fd-4235-80e0-dd65cc8ce5da admin admin] PATH_INFO: `/v3/users/761d28842ab9456fb877864cac93fbbc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:57.790995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712684c0-66fd-4235-80e0-dd65cc8ce5da admin admin] RBAC: Authorizing `identity:delete_user(user_id=761d28842ab9456fb877864cac93fbbc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:57.791183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a2542-6cf9-4ffd-89f0-f5e2ed3d676a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:53:57.791462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a2542-6cf9-4ffd-89f0-f5e2ed3d676a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:57.792950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712684c0-66fd-4235-80e0-dd65cc8ce5da admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:57.798768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a2542-6cf9-4ffd-89f0-f5e2ed3d676a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:57.837937 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1375/2788] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:53:57.839558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a2542-6cf9-4ffd-89f0-f5e2ed3d676a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:57.870879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a2542-6cf9-4ffd-89f0-f5e2ed3d676a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:57.880118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-712684c0-66fd-4235-80e0-dd65cc8ce5da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '761d28842ab9456fb877864cac93fbbc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:57.883613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3164abb-a53e-4679-87c0-b83bbef43be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:57.884241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3164abb-a53e-4679-87c0-b83bbef43be9 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:57.884523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3164abb-a53e-4679-87c0-b83bbef43be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:57.884800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3164abb-a53e-4679-87c0-b83bbef43be9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:57.885470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3164abb-a53e-4679-87c0-b83bbef43be9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:57.887380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3164abb-a53e-4679-87c0-b83bbef43be9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:57.918381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-712684c0-66fd-4235-80e0-dd65cc8ce5da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '761d28842ab9456fb877864cac93fbbc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:57.930259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-712684c0-66fd-4235-80e0-dd65cc8ce5da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '761d28842ab9456fb877864cac93fbbc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:57.931029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-712684c0-66fd-4235-80e0-dd65cc8ce5da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '761d28842ab9456fb877864cac93fbbc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:57.932041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-712684c0-66fd-4235-80e0-dd65cc8ce5da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '761d28842ab9456fb877864cac93fbbc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:57.940202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:57.965743 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1414/2789] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:53:57 2022] DELETE /identity/v3/users/761d28842ab9456fb877864cac93fbbc => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:53:57.970973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762259af-a3d9-40c1-a5f3-921d787aa2c2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:57.971356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762259af-a3d9-40c1-a5f3-921d787aa2c2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:57.971702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762259af-a3d9-40c1-a5f3-921d787aa2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:57.979881 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1376/2790] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:53:57 2022] POST /identity/v3/projects => generated 397 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:53:57.982116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3164abb-a53e-4679-87c0-b83bbef43be9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:57.989276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c3164abb-a53e-4679-87c0-b83bbef43be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:58.007187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-762259af-a3d9-40c1-a5f3-921d787aa2c2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:58.008862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-762259af-a3d9-40c1-a5f3-921d787aa2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:58.115846 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1415/2791] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:58.148323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3164abb-a53e-4679-87c0-b83bbef43be9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:58.156350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306836e-d456-434a-9c95-eb397fd0c922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:58.156821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306836e-d456-434a-9c95-eb397fd0c922 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:58.157153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306836e-d456-434a-9c95-eb397fd0c922 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:58.157434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306836e-d456-434a-9c95-eb397fd0c922 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:58.158087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306836e-d456-434a-9c95-eb397fd0c922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:58.160369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306836e-d456-434a-9c95-eb397fd0c922 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:58.166516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762259af-a3d9-40c1-a5f3-921d787aa2c2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:58.168012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-762259af-a3d9-40c1-a5f3-921d787aa2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:58.233425 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1377/2792] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:53:57 2022] POST /identity/v3/users => generated 299 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:53:58.236440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306836e-d456-434a-9c95-eb397fd0c922 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:58.255643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306836e-d456-434a-9c95-eb397fd0c922 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:58.291298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218126a5-027d-46c7-9d92-266b9e77a9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:58.291950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218126a5-027d-46c7-9d92-266b9e77a9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:58.292741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218126a5-027d-46c7-9d92-266b9e77a9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:58.293213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218126a5-027d-46c7-9d92-266b9e77a9d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:58.293894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218126a5-027d-46c7-9d92-266b9e77a9d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:58.296773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218126a5-027d-46c7-9d92-266b9e77a9d9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:58.351754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762259af-a3d9-40c1-a5f3-921d787aa2c2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:58.356518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38e4e-37f5-4985-bbff-9df5d1f093d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:58.361775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38e4e-37f5-4985-bbff-9df5d1f093d6 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:58.361775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38e4e-37f5-4985-bbff-9df5d1f093d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:58.361775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38e4e-37f5-4985-bbff-9df5d1f093d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:58.366236 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1378/2793] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:53:58.369834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218126a5-027d-46c7-9d92-266b9e77a9d9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:58.385897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed38e4e-37f5-4985-bbff-9df5d1f093d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:58.387373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed38e4e-37f5-4985-bbff-9df5d1f093d6 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:58.393757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218126a5-027d-46c7-9d92-266b9e77a9d9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:58.407247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7796f-227b-4fba-a9ee-63aab7620e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:58.407889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7796f-227b-4fba-a9ee-63aab7620e1e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:58.408182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7796f-227b-4fba-a9ee-63aab7620e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:58.408463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7796f-227b-4fba-a9ee-63aab7620e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:58.409145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7796f-227b-4fba-a9ee-63aab7620e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:58.410761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7796f-227b-4fba-a9ee-63aab7620e1e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:58.425863 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1416/2794] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:53:58.440450 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1379/2795] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:53:58.445328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38e4e-37f5-4985-bbff-9df5d1f093d6 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:58.469760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38e4e-37f5-4985-bbff-9df5d1f093d6 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:58.487146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c340b4-2e37-47b5-8362-0de62b58cf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:58.488174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c340b4-2e37-47b5-8362-0de62b58cf86 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:58.488453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c340b4-2e37-47b5-8362-0de62b58cf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:58.488732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c340b4-2e37-47b5-8362-0de62b58cf86 admin admin] PATH_INFO: `/v3/projects/dad6ee8212ab418cba7f718ce4bb4726/users/27be6827d30e48acac9a59786021f7bb/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:58.512949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:58.531462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c340b4-2e37-47b5-8362-0de62b58cf86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad6ee8212ab418cba7f718ce4bb4726, user_id=27be6827d30e48acac9a59786021f7bb, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:58.540249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c340b4-2e37-47b5-8362-0de62b58cf86 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:58.590032 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1417/2796] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:53:58 2022] PUT /identity/v3/projects/dad6ee8212ab418cba7f718ce4bb4726/users/27be6827d30e48acac9a59786021f7bb/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:53:58.594632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b38a69-1d79-46f9-9438-653581696843 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:58.595668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b38a69-1d79-46f9-9438-653581696843 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:58.595959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b38a69-1d79-46f9-9438-653581696843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:58.625733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e1b38a69-1d79-46f9-9438-653581696843 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:58.627413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e1b38a69-1d79-46f9-9438-653581696843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:58.778682 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1380/2797] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:58.797423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c340b4-2e37-47b5-8362-0de62b58cf86 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:58.803185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-98c340b4-2e37-47b5-8362-0de62b58cf86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:58.894171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c340b4-2e37-47b5-8362-0de62b58cf86 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:58.899504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d236978-2042-4373-b05c-5aaf5a874e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:58.900128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d236978-2042-4373-b05c-5aaf5a874e4f service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:58.900417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d236978-2042-4373-b05c-5aaf5a874e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:58.902766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d236978-2042-4373-b05c-5aaf5a874e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:58.917647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d236978-2042-4373-b05c-5aaf5a874e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:58.921266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d236978-2042-4373-b05c-5aaf5a874e4f service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:58.950590 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1418/2798] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:53:59.245208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b38a69-1d79-46f9-9438-653581696843 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:59.247312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e1b38a69-1d79-46f9-9438-653581696843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:59.268274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b38a69-1d79-46f9-9438-653581696843 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:59.306352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dbf35-d10e-4940-b3aa-924576e67281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:59.309746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dbf35-d10e-4940-b3aa-924576e67281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:59.312051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dbf35-d10e-4940-b3aa-924576e67281 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:59.313193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dbf35-d10e-4940-b3aa-924576e67281 admin admin] PATH_INFO: `/v3/projects/5b635bf0f5f24567b72a5f02ea6a57a8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:59.317543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882dbf35-d10e-4940-b3aa-924576e67281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b635bf0f5f24567b72a5f02ea6a57a8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:59.324063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882dbf35-d10e-4940-b3aa-924576e67281 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:59.453295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-882dbf35-d10e-4940-b3aa-924576e67281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b635bf0f5f24567b72a5f02ea6a57a8', 'request_id': 'req-882dbf35-d10e-4940-b3aa-924576e67281'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:59.475837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-882dbf35-d10e-4940-b3aa-924576e67281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b635bf0f5f24567b72a5f02ea6a57a8', 'request_id': 'req-882dbf35-d10e-4940-b3aa-924576e67281'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:59.479802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=5,keystone:SELECT=90,keystone:DELETE=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:53:59.495020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-882dbf35-d10e-4940-b3aa-924576e67281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b635bf0f5f24567b72a5f02ea6a57a8', 'request_id': 'req-882dbf35-d10e-4940-b3aa-924576e67281'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:59.515592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-882dbf35-d10e-4940-b3aa-924576e67281 admin admin] The token cache is being invalidate because project 5b635bf0f5f24567b72a5f02ea6a57a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:53:59.516376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-882dbf35-d10e-4940-b3aa-924576e67281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:59.518168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-882dbf35-d10e-4940-b3aa-924576e67281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:53:59.520565 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1381/2799] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:53:59 2022] DELETE /identity/v3/projects/5b635bf0f5f24567b72a5f02ea6a57a8 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:53:59.525807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8c417-ed07-4bc3-a68c-4230670adf0c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:59.526281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8c417-ed07-4bc3-a68c-4230670adf0c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:59.526368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8c417-ed07-4bc3-a68c-4230670adf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:59.570621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2db8c417-ed07-4bc3-a68c-4230670adf0c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:59.571873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2db8c417-ed07-4bc3-a68c-4230670adf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:59.704304 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1419/2800] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:59.710818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90751234-8743-4a9c-b418-7f03a8efd12a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:59.711150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90751234-8743-4a9c-b418-7f03a8efd12a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:59.711816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90751234-8743-4a9c-b418-7f03a8efd12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:59.747128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-90751234-8743-4a9c-b418-7f03a8efd12a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:53:59.748586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-90751234-8743-4a9c-b418-7f03a8efd12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:53:59.761634 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1382/2801] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:53:59.766080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8c417-ed07-4bc3-a68c-4230670adf0c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:59.779698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8c417-ed07-4bc3-a68c-4230670adf0c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:59.789824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193e8d3-bf4d-4686-aec6-c7a1505c657c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:59.790824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e8d3-bf4d-4686-aec6-c7a1505c657c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:59.797104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e8d3-bf4d-4686-aec6-c7a1505c657c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:59.797489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e8d3-bf4d-4686-aec6-c7a1505c657c admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:59.798267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193e8d3-bf4d-4686-aec6-c7a1505c657c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:53:59.799416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193e8d3-bf4d-4686-aec6-c7a1505c657c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:59.806959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193e8d3-bf4d-4686-aec6-c7a1505c657c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:59.847300 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1420/2802] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:53:59.851153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90751234-8743-4a9c-b418-7f03a8efd12a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:59.864839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90751234-8743-4a9c-b418-7f03a8efd12a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:53:59.900931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d8dc8-f09b-4690-9b07-1059fe42afb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:53:59.900931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d8dc8-f09b-4690-9b07-1059fe42afb5 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:53:59.900931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d8dc8-f09b-4690-9b07-1059fe42afb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:53:59.900931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d8dc8-f09b-4690-9b07-1059fe42afb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:53:59.901609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d8dc8-f09b-4690-9b07-1059fe42afb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:53:59.901609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d8dc8-f09b-4690-9b07-1059fe42afb5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:53:59.948728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193e8d3-bf4d-4686-aec6-c7a1505c657c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:53:59.951064 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1383/2803] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:53:59 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:53:59.967464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193e8d3-bf4d-4686-aec6-c7a1505c657c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:00.059355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7215e8-5c26-4ee5-9e3b-a30d104bc301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:00.059903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7215e8-5c26-4ee5-9e3b-a30d104bc301 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:00.060337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7215e8-5c26-4ee5-9e3b-a30d104bc301 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:00.060619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7215e8-5c26-4ee5-9e3b-a30d104bc301 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:00.061317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7215e8-5c26-4ee5-9e3b-a30d104bc301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:00.063266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7215e8-5c26-4ee5-9e3b-a30d104bc301 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:00.089875 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1421/2804] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:53:59 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:54:00.093967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d8dc8-f09b-4690-9b07-1059fe42afb5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:00.109917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d8dc8-f09b-4690-9b07-1059fe42afb5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:00.126801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f579d0-c002-4768-9771-a4f3e16245c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:00.127497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f579d0-c002-4768-9771-a4f3e16245c9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:00.128034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f579d0-c002-4768-9771-a4f3e16245c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:00.133034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f579d0-c002-4768-9771-a4f3e16245c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:00.133034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f579d0-c002-4768-9771-a4f3e16245c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:00.133034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f579d0-c002-4768-9771-a4f3e16245c9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:00.163747 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1384/2805] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:00.165874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7215e8-5c26-4ee5-9e3b-a30d104bc301 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:00.181390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7215e8-5c26-4ee5-9e3b-a30d104bc301 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:00.190247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265b1e3-91ce-4894-9eef-7a1ee3190fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:00.191224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265b1e3-91ce-4894-9eef-7a1ee3190fec admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:00.191649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265b1e3-91ce-4894-9eef-7a1ee3190fec admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:00.192032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265b1e3-91ce-4894-9eef-7a1ee3190fec admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:00.192924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265b1e3-91ce-4894-9eef-7a1ee3190fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:00.194755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265b1e3-91ce-4894-9eef-7a1ee3190fec admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:00.197577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:00.212731 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1422/2806] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:00.216264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f579d0-c002-4768-9771-a4f3e16245c9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:00.234708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f579d0-c002-4768-9771-a4f3e16245c9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:00.247227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1a557-4f8b-4417-a75a-afb3994c1e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:00.247918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1a557-4f8b-4417-a75a-afb3994c1e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:00.248203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1a557-4f8b-4417-a75a-afb3994c1e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:00.248482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1a557-4f8b-4417-a75a-afb3994c1e34 admin admin] PATH_INFO: `/v3/projects/5e1442bbd96148d196c957d881a7449e/users/18b23f4ce1d44682b0c187e8aac1a7df/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:00.268194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1a557-4f8b-4417-a75a-afb3994c1e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1442bbd96148d196c957d881a7449e, user_id=18b23f4ce1d44682b0c187e8aac1a7df, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:00.271916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1a557-4f8b-4417-a75a-afb3994c1e34 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:00.302786 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1385/2807] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:00 2022] PUT /identity/v3/projects/5e1442bbd96148d196c957d881a7449e/users/18b23f4ce1d44682b0c187e8aac1a7df/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:54:00.304591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec664cb-9c4c-435e-9b3f-20f25b321857 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:00.304889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec664cb-9c4c-435e-9b3f-20f25b321857 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:00.305251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec664cb-9c4c-435e-9b3f-20f25b321857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:00.338406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0ec664cb-9c4c-435e-9b3f-20f25b321857 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:00.339484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0ec664cb-9c4c-435e-9b3f-20f25b321857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:00.438257 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1423/2808] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:00.445833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1a557-4f8b-4417-a75a-afb3994c1e34 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:00.448515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-aee1a557-4f8b-4417-a75a-afb3994c1e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:00.548691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1a557-4f8b-4417-a75a-afb3994c1e34 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:00.567217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36779121-cca6-4c02-aa6f-03f8d8b38e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:00.567217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36779121-cca6-4c02-aa6f-03f8d8b38e05 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:00.567217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36779121-cca6-4c02-aa6f-03f8d8b38e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:00.567217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36779121-cca6-4c02-aa6f-03f8d8b38e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:00.588595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36779121-cca6-4c02-aa6f-03f8d8b38e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:00.590308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36779121-cca6-4c02-aa6f-03f8d8b38e05 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:00.609428 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1386/2809] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:04.467280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec664cb-9c4c-435e-9b3f-20f25b321857 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:04.469194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0ec664cb-9c4c-435e-9b3f-20f25b321857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:04.489684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec664cb-9c4c-435e-9b3f-20f25b321857 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:04.517797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7600b-a8c3-43b0-9b15-40559c836347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:04.518460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7600b-a8c3-43b0-9b15-40559c836347 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:04.518750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7600b-a8c3-43b0-9b15-40559c836347 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:04.520477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7600b-a8c3-43b0-9b15-40559c836347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:04.525801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d2e7600b-a8c3-43b0-9b15-40559c836347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:04.550480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7600b-a8c3-43b0-9b15-40559c836347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:04.552054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7600b-a8c3-43b0-9b15-40559c836347 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:04.573779 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1424/2810] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:10.386306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36779121-cca6-4c02-aa6f-03f8d8b38e05 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:10.388306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-36779121-cca6-4c02-aa6f-03f8d8b38e05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:10.421564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36779121-cca6-4c02-aa6f-03f8d8b38e05 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:10.444074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:10.446877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:10.448602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:10.449323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] PATH_INFO: `/v3/users/ee0fdf107484448d862ac3216df0d9dc` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:10.453687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee0fdf107484448d862ac3216df0d9dc)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:10.455971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:10.547042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee0fdf107484448d862ac3216df0d9dc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:10.562711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee0fdf107484448d862ac3216df0d9dc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:10.564158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee0fdf107484448d862ac3216df0d9dc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:10.618269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee0fdf107484448d862ac3216df0d9dc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:10.631983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:10.637359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee0fdf107484448d862ac3216df0d9dc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:10.653688 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1387/2811] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:54:10 2022] DELETE /identity/v3/users/ee0fdf107484448d862ac3216df0d9dc => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:11.422272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7600b-a8c3-43b0-9b15-40559c836347 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:11.479200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7600b-a8c3-43b0-9b15-40559c836347 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:11.800609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:11.801299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:11.801591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:11.801884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357 admin admin] PATH_INFO: `/v3/projects/68cdc132e95244478bf01490de3e0941` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:11.803401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68cdc132e95244478bf01490de3e0941)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:11.808340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:12.008392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68cdc132e95244478bf01490de3e0941', 'request_id': 'req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:12.046077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68cdc132e95244478bf01490de3e0941', 'request_id': 'req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:12.066444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68cdc132e95244478bf01490de3e0941', 'request_id': 'req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:12.115380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357 admin admin] The token cache is being invalidate because project 68cdc132e95244478bf01490de3e0941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:54:12.115704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:12.117079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ab754fc4-07b0-4abf-bf4e-e1dd9dec9357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:12.119009 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1425/2812] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:54:11 2022] DELETE /identity/v3/projects/68cdc132e95244478bf01490de3e0941 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:12.123389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:12.126047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:12.169252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1563dd3-760c-4d4e-a52d-ca1031906ab4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:12.285759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e9b01-be0c-462b-a2fd-be31427c7d0e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:12.286111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e9b01-be0c-462b-a2fd-be31427c7d0e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:12.286646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e9b01-be0c-462b-a2fd-be31427c7d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:12.381706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:12.382744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:12.385495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:12.385703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:12.386280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] PATH_INFO: `/v3/users/30ebd40778e3492199eef13eb0ab3310` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:12.387958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ebd40778e3492199eef13eb0ab3310)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:12.391723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:12.401656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-931e9b01-be0c-462b-a2fd-be31427c7d0e None None] MFA Rules not processed for user `27be6827d30e48acac9a59786021f7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:12.403235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-931e9b01-be0c-462b-a2fd-be31427c7d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:12.534255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ebd40778e3492199eef13eb0ab3310'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:12.589238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ebd40778e3492199eef13eb0ab3310'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:12.595256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ebd40778e3492199eef13eb0ab3310'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:12.616929 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1426/2813] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:54:12.690175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ebd40778e3492199eef13eb0ab3310'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:12.703457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c373f5-3a19-4414-b22e-b401fa34502e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:12.704096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c373f5-3a19-4414-b22e-b401fa34502e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:12.704573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c373f5-3a19-4414-b22e-b401fa34502e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:12.719432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ebd40778e3492199eef13eb0ab3310'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:12.735737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-09c373f5-3a19-4414-b22e-b401fa34502e None None] MFA Rules not processed for user `27be6827d30e48acac9a59786021f7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:12.740259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-09c373f5-3a19-4414-b22e-b401fa34502e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:12.740545 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1388/2814] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:54:12 2022] DELETE /identity/v3/users/30ebd40778e3492199eef13eb0ab3310 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:12.907672 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1427/2815] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:54:12.921954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:12.924097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:13.181037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d3455-8d5e-4ada-8587-ac61b19250bc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:13.190840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b870e9f-9f00-47fd-9965-e9fc67406b66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:13.191641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b870e9f-9f00-47fd-9965-e9fc67406b66 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:13.192118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b870e9f-9f00-47fd-9965-e9fc67406b66 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:13.192531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b870e9f-9f00-47fd-9965-e9fc67406b66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:13.195423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4b870e9f-9f00-47fd-9965-e9fc67406b66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:13.219995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b870e9f-9f00-47fd-9965-e9fc67406b66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:13.222284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b870e9f-9f00-47fd-9965-e9fc67406b66 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:13.229324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:13.254560 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1389/2816] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:12 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:14.020422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf19d4-1ed1-4de1-8283-36457e05063b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:14.021019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf19d4-1ed1-4de1-8283-36457e05063b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:14.021419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf19d4-1ed1-4de1-8283-36457e05063b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:14.156706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-58bf19d4-1ed1-4de1-8283-36457e05063b None None] MFA Rules not processed for user `18b23f4ce1d44682b0c187e8aac1a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:14.159453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-58bf19d4-1ed1-4de1-8283-36457e05063b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:14.365263 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1428/2817] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:14.717490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b870e9f-9f00-47fd-9965-e9fc67406b66 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:14.717490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4b870e9f-9f00-47fd-9965-e9fc67406b66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:14.852669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b870e9f-9f00-47fd-9965-e9fc67406b66 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:14.870416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:14.871341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:14.871795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:14.872243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39 admin admin] PATH_INFO: `/v3/projects/ae38955142824be086f297cc4310991d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:14.873626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae38955142824be086f297cc4310991d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:14.875972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:14.897784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf19d4-1ed1-4de1-8283-36457e05063b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:14.899607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-58bf19d4-1ed1-4de1-8283-36457e05063b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:14.956531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf19d4-1ed1-4de1-8283-36457e05063b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:14.982351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86388138-285b-4f6b-ac90-464881c3bf10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:14.983124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86388138-285b-4f6b-ac90-464881c3bf10 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:14.985614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86388138-285b-4f6b-ac90-464881c3bf10 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:14.986582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86388138-285b-4f6b-ac90-464881c3bf10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:14.990390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-86388138-285b-4f6b-ac90-464881c3bf10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:15.004533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae38955142824be086f297cc4310991d', 'request_id': 'req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:15.015566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae38955142824be086f297cc4310991d', 'request_id': 'req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:15.015895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86388138-285b-4f6b-ac90-464881c3bf10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:15.021120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86388138-285b-4f6b-ac90-464881c3bf10 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:15.026477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae38955142824be086f297cc4310991d', 'request_id': 'req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:15.059910 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1429/2818] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:54:15.061251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86388138-285b-4f6b-ac90-464881c3bf10 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:15.061984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39 admin admin] The token cache is being invalidate because project ae38955142824be086f297cc4310991d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:54:15.062277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:15.063103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f0fbcc40-e386-4e18-ab2b-7a12ef193c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:15.064506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-86388138-285b-4f6b-ac90-464881c3bf10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:15.068393 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1390/2819] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:54:14 2022] DELETE /identity/v3/projects/ae38955142824be086f297cc4310991d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:15.073323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459be31-4b85-46fd-9d9c-d6610f0defb5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:15.073976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459be31-4b85-46fd-9d9c-d6610f0defb5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:15.074415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459be31-4b85-46fd-9d9c-d6610f0defb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:15.142991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d459be31-4b85-46fd-9d9c-d6610f0defb5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:15.147969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d459be31-4b85-46fd-9d9c-d6610f0defb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:15.181863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:15.329009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86388138-285b-4f6b-ac90-464881c3bf10 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:15.370788 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1391/2820] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:54:15.379141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459be31-4b85-46fd-9d9c-d6610f0defb5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:15.385912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d459be31-4b85-46fd-9d9c-d6610f0defb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:15.484603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178886ce-e150-4ccb-b0cf-c887661cb79e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:15.485754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178886ce-e150-4ccb-b0cf-c887661cb79e service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:15.486400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178886ce-e150-4ccb-b0cf-c887661cb79e service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:15.486825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178886ce-e150-4ccb-b0cf-c887661cb79e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:15.490296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-178886ce-e150-4ccb-b0cf-c887661cb79e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:15.515441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:15.521808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178886ce-e150-4ccb-b0cf-c887661cb79e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:15.525422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178886ce-e150-4ccb-b0cf-c887661cb79e service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:15.554068 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1430/2821] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:15.557227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f92411-519d-4167-a3e8-70c581c45b26 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:15.557522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f92411-519d-4167-a3e8-70c581c45b26 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:15.557800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f92411-519d-4167-a3e8-70c581c45b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:15.590557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459be31-4b85-46fd-9d9c-d6610f0defb5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:15.600076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59349173-a2eb-47b7-9609-68a06d9d401e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:15.601121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59349173-a2eb-47b7-9609-68a06d9d401e service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:15.601605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59349173-a2eb-47b7-9609-68a06d9d401e service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:15.602045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59349173-a2eb-47b7-9609-68a06d9d401e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:15.604379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-59349173-a2eb-47b7-9609-68a06d9d401e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:15.610384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f9f92411-519d-4167-a3e8-70c581c45b26 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:15.611253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f9f92411-519d-4167-a3e8-70c581c45b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:15.626434 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1431/2822] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:15.645555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f92411-519d-4167-a3e8-70c581c45b26 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:15.684187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f92411-519d-4167-a3e8-70c581c45b26 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:15.732028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1d7bd-af7e-4e9e-b767-1cd0989bf45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:15.735773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1d7bd-af7e-4e9e-b767-1cd0989bf45d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:15.735773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1d7bd-af7e-4e9e-b767-1cd0989bf45d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:15.737127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1d7bd-af7e-4e9e-b767-1cd0989bf45d admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:15.739317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1d7bd-af7e-4e9e-b767-1cd0989bf45d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:54:15.742544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1d7bd-af7e-4e9e-b767-1cd0989bf45d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:15.745592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1d7bd-af7e-4e9e-b767-1cd0989bf45d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:15.786580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59349173-a2eb-47b7-9609-68a06d9d401e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:15.787961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59349173-a2eb-47b7-9609-68a06d9d401e service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:15.800853 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1432/2823] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:54:15.801836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1d7bd-af7e-4e9e-b767-1cd0989bf45d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:15.803552 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1392/2824] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:15.828276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1d7bd-af7e-4e9e-b767-1cd0989bf45d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:15.841872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01e50e-c0ab-45d8-a327-18e3e802ac9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:15.842606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01e50e-c0ab-45d8-a327-18e3e802ac9c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:15.843154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01e50e-c0ab-45d8-a327-18e3e802ac9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:15.843424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01e50e-c0ab-45d8-a327-18e3e802ac9c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:15.844068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01e50e-c0ab-45d8-a327-18e3e802ac9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:15.852078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01e50e-c0ab-45d8-a327-18e3e802ac9c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:15.936558 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1433/2825] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:54:15 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:15.940848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59349173-a2eb-47b7-9609-68a06d9d401e service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:15.977749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59349173-a2eb-47b7-9609-68a06d9d401e service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:16.191717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f493d6-ac3f-461c-b65a-f68ffd396e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:16.192337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f493d6-ac3f-461c-b65a-f68ffd396e73 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:16.192628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f493d6-ac3f-461c-b65a-f68ffd396e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:16.192928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f493d6-ac3f-461c-b65a-f68ffd396e73 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:16.193754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f493d6-ac3f-461c-b65a-f68ffd396e73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:16.196912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f493d6-ac3f-461c-b65a-f68ffd396e73 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:16.271316 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1393/2826] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:54:15 2022] POST /identity/v3/users => generated 290 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:16.272816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01e50e-c0ab-45d8-a327-18e3e802ac9c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:16.289506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01e50e-c0ab-45d8-a327-18e3e802ac9c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:16.314110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe5835-e59a-4117-9285-f8953bb7ca0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:16.315778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe5835-e59a-4117-9285-f8953bb7ca0c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:16.317936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe5835-e59a-4117-9285-f8953bb7ca0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:16.318461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe5835-e59a-4117-9285-f8953bb7ca0c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:16.321332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe5835-e59a-4117-9285-f8953bb7ca0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:16.326459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe5835-e59a-4117-9285-f8953bb7ca0c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:16.352820 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1434/2827] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:16.357601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f493d6-ac3f-461c-b65a-f68ffd396e73 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:16.378547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f493d6-ac3f-461c-b65a-f68ffd396e73 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:16.395978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc620e07-fafb-45cd-b318-b024fad0dee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:16.397692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc620e07-fafb-45cd-b318-b024fad0dee7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:16.397995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc620e07-fafb-45cd-b318-b024fad0dee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:16.398276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc620e07-fafb-45cd-b318-b024fad0dee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:16.400958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc620e07-fafb-45cd-b318-b024fad0dee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:16.439857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc620e07-fafb-45cd-b318-b024fad0dee7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:16.470925 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1394/2828] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:16.471378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe5835-e59a-4117-9285-f8953bb7ca0c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:16.486350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe5835-e59a-4117-9285-f8953bb7ca0c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:16.508326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463871f0-e910-42ce-a018-395ab4eba7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:16.509512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463871f0-e910-42ce-a018-395ab4eba7f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:16.509997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463871f0-e910-42ce-a018-395ab4eba7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:16.510486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463871f0-e910-42ce-a018-395ab4eba7f1 admin admin] PATH_INFO: `/v3/projects/ecfb9a5fe42a4c99a294e05cad3e03a3/users/d950b3116e1a48ea8b7b2d0c335622db/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:16.549909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463871f0-e910-42ce-a018-395ab4eba7f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecfb9a5fe42a4c99a294e05cad3e03a3, user_id=d950b3116e1a48ea8b7b2d0c335622db, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:16.555017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463871f0-e910-42ce-a018-395ab4eba7f1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:16.581262 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1435/2829] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:16 2022] PUT /identity/v3/projects/ecfb9a5fe42a4c99a294e05cad3e03a3/users/d950b3116e1a48ea8b7b2d0c335622db/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:54:16.592564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ab8ce-2a89-46d8-8843-d11965b3c25d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:16.593510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ab8ce-2a89-46d8-8843-d11965b3c25d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:16.593794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ab8ce-2a89-46d8-8843-d11965b3c25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:16.652147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-df1ab8ce-2a89-46d8-8843-d11965b3c25d None None] MFA Rules not processed for user `d950b3116e1a48ea8b7b2d0c335622db`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:16.653896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-df1ab8ce-2a89-46d8-8843-d11965b3c25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:16.734528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:16.786556 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1395/2830] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:16.798316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463871f0-e910-42ce-a018-395ab4eba7f1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:16.929588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:16.969578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463871f0-e910-42ce-a018-395ab4eba7f1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:16.979105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf290a-c040-4831-bab0-c2db37d93c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:16.980072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf290a-c040-4831-bab0-c2db37d93c0a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:16.980468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf290a-c040-4831-bab0-c2db37d93c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:16.981311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf290a-c040-4831-bab0-c2db37d93c0a admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:16.982119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf290a-c040-4831-bab0-c2db37d93c0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:16.986559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf290a-c040-4831-bab0-c2db37d93c0a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:17.084598 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1436/2831] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:54:16 2022] POST /identity/v3/projects => generated 379 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:17.089229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ab8ce-2a89-46d8-8843-d11965b3c25d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:17.107192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ab8ce-2a89-46d8-8843-d11965b3c25d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:17.228042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09450c7-8e00-4b5e-bbbe-47ff35a85509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:17.229133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09450c7-8e00-4b5e-bbbe-47ff35a85509 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:17.229585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09450c7-8e00-4b5e-bbbe-47ff35a85509 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:17.230027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09450c7-8e00-4b5e-bbbe-47ff35a85509 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:17.230861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09450c7-8e00-4b5e-bbbe-47ff35a85509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:17.245773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09450c7-8e00-4b5e-bbbe-47ff35a85509 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:17.301245 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1396/2832] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:54:17 2022] POST /identity/v3/users => generated 296 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:17.304460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf290a-c040-4831-bab0-c2db37d93c0a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:17.321236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf290a-c040-4831-bab0-c2db37d93c0a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:17.334764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efb36e-6aa9-4a22-8b36-2dcddb970c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:17.335614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efb36e-6aa9-4a22-8b36-2dcddb970c4e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:17.336096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efb36e-6aa9-4a22-8b36-2dcddb970c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:17.337274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efb36e-6aa9-4a22-8b36-2dcddb970c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:17.338114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efb36e-6aa9-4a22-8b36-2dcddb970c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:17.339938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efb36e-6aa9-4a22-8b36-2dcddb970c4e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:17.365135 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1437/2833] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:17.370242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09450c7-8e00-4b5e-bbbe-47ff35a85509 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:17.386345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09450c7-8e00-4b5e-bbbe-47ff35a85509 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:17.398867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adda4f3-3277-41a5-a8d5-e69e2a7b3ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:17.399842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adda4f3-3277-41a5-a8d5-e69e2a7b3ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:17.400261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adda4f3-3277-41a5-a8d5-e69e2a7b3ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:17.400642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adda4f3-3277-41a5-a8d5-e69e2a7b3ea7 admin admin] PATH_INFO: `/v3/projects/d09f1b78ffac416ebc790368b25b0120/users/cd786a4dbd8e404584752183731a6309/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:17.446933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adda4f3-3277-41a5-a8d5-e69e2a7b3ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09f1b78ffac416ebc790368b25b0120, user_id=cd786a4dbd8e404584752183731a6309, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:17.452523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adda4f3-3277-41a5-a8d5-e69e2a7b3ea7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:17.505039 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1397/2834] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:17 2022] PUT /identity/v3/projects/d09f1b78ffac416ebc790368b25b0120/users/cd786a4dbd8e404584752183731a6309/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:17.509252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efb36e-6aa9-4a22-8b36-2dcddb970c4e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:17.520598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efb36e-6aa9-4a22-8b36-2dcddb970c4e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:17.638363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212b935-847c-4c8b-80ee-75d3a9ff5ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:17.639529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212b935-847c-4c8b-80ee-75d3a9ff5ecf admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:17.640153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212b935-847c-4c8b-80ee-75d3a9ff5ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:17.651027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212b935-847c-4c8b-80ee-75d3a9ff5ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:17.654175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212b935-847c-4c8b-80ee-75d3a9ff5ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:17.656123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212b935-847c-4c8b-80ee-75d3a9ff5ecf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:17.684984 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1438/2835] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:17.686813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adda4f3-3277-41a5-a8d5-e69e2a7b3ea7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:17.702034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adda4f3-3277-41a5-a8d5-e69e2a7b3ea7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:17.710712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c141d9a-27ba-4f9a-b65c-5a6ca011f656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:17.711732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c141d9a-27ba-4f9a-b65c-5a6ca011f656 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:17.712129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c141d9a-27ba-4f9a-b65c-5a6ca011f656 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:17.712514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c141d9a-27ba-4f9a-b65c-5a6ca011f656 admin admin] PATH_INFO: `/v3/projects/d09f1b78ffac416ebc790368b25b0120/users/cd786a4dbd8e404584752183731a6309/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:17.714949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c141d9a-27ba-4f9a-b65c-5a6ca011f656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09f1b78ffac416ebc790368b25b0120, user_id=cd786a4dbd8e404584752183731a6309, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:17.719012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c141d9a-27ba-4f9a-b65c-5a6ca011f656 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:17.733797 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1398/2836] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:17 2022] PUT /identity/v3/projects/d09f1b78ffac416ebc790368b25b0120/users/cd786a4dbd8e404584752183731a6309/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:17.746207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effebe4b-de30-4160-8cec-cd5305b1c5b0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:17.746500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effebe4b-de30-4160-8cec-cd5305b1c5b0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:17.746781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effebe4b-de30-4160-8cec-cd5305b1c5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:17.843357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-effebe4b-de30-4160-8cec-cd5305b1c5b0 None None] MFA Rules not processed for user `cd786a4dbd8e404584752183731a6309`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:17.844528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-effebe4b-de30-4160-8cec-cd5305b1c5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:17.971542 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1439/2837] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:17.986496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c141d9a-27ba-4f9a-b65c-5a6ca011f656 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:17.988994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3c141d9a-27ba-4f9a-b65c-5a6ca011f656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:18.124682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c141d9a-27ba-4f9a-b65c-5a6ca011f656 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:18.141257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6b7d1-aa0d-426e-97de-78892d2f3e3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:18.142162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6b7d1-aa0d-426e-97de-78892d2f3e3e service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:18.142664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6b7d1-aa0d-426e-97de-78892d2f3e3e service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:18.143130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6b7d1-aa0d-426e-97de-78892d2f3e3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:18.155806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6b7d1-aa0d-426e-97de-78892d2f3e3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:18.158410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6b7d1-aa0d-426e-97de-78892d2f3e3e service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:18.175698 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1399/2838] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:18.374099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568facad-b183-4312-92c6-bde49510e506 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:18.375052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568facad-b183-4312-92c6-bde49510e506 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:18.375507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568facad-b183-4312-92c6-bde49510e506 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:18.377765 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1440/2839] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:54:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:54:18.385417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6b7d1-aa0d-426e-97de-78892d2f3e3e service nova] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:54:18.413353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6b7d1-aa0d-426e-97de-78892d2f3e3e service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:18.445274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6b7d1-aa0d-426e-97de-78892d2f3e3e service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:18.457831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b23475-e213-4c78-aaf8-2654d9135afa tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd786a4dbd8e404584752183731a6309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09f1b78ffac416ebc790368b25b0120', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:18.466754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b23475-e213-4c78-aaf8-2654d9135afa tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:18.468829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b23475-e213-4c78-aaf8-2654d9135afa tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:18.469302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b23475-e213-4c78-aaf8-2654d9135afa tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] PATH_INFO: `/v3/projects/ecfb9a5fe42a4c99a294e05cad3e03a3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:18.472565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b23475-e213-4c78-aaf8-2654d9135afa tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] RBAC: Authorizing `identity:get_project(project_id=ecfb9a5fe42a4c99a294e05cad3e03a3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:18.476670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b23475-e213-4c78-aaf8-2654d9135afa tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:18.482181 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1400/2840] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:54:18 2022] GET /identity/v3/projects/ecfb9a5fe42a4c99a294e05cad3e03a3 => generated 379 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:54:18.527015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568facad-b183-4312-92c6-bde49510e506 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:18.545200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568facad-b183-4312-92c6-bde49510e506 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:18.552182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69a15b-5549-44c7-9cd2-273ace3631f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:18.553140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69a15b-5549-44c7-9cd2-273ace3631f8 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:18.554388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69a15b-5549-44c7-9cd2-273ace3631f8 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:18.555111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69a15b-5549-44c7-9cd2-273ace3631f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:18.609892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:18.613665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bfaaf-2faf-44be-8ed3-a943f8a19506 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:18.613959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bfaaf-2faf-44be-8ed3-a943f8a19506 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:18.614242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bfaaf-2faf-44be-8ed3-a943f8a19506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:18.630239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:18.647666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7c8bfaaf-2faf-44be-8ed3-a943f8a19506 None None] MFA Rules not processed for user `18b23f4ce1d44682b0c187e8aac1a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:18.649028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7c8bfaaf-2faf-44be-8ed3-a943f8a19506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:18.651224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69a15b-5549-44c7-9cd2-273ace3631f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:18.653221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69a15b-5549-44c7-9cd2-273ace3631f8 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:18.670409 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1441/2841] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:18.728949 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1401/2842] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:18.736353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e5f79-ea9f-402c-a063-aee07fedb8f2 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:18.736673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e5f79-ea9f-402c-a063-aee07fedb8f2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:18.737516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e5f79-ea9f-402c-a063-aee07fedb8f2 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:18.738891 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1442/2843] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:54:18.749573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bfaaf-2faf-44be-8ed3-a943f8a19506 None None] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:54:18.763069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bfaaf-2faf-44be-8ed3-a943f8a19506 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:18.774267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bfaaf-2faf-44be-8ed3-a943f8a19506 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:18.779551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f823cd-792c-464d-8bbb-7fe319d85b3a tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd786a4dbd8e404584752183731a6309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09f1b78ffac416ebc790368b25b0120', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:18.780193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f823cd-792c-464d-8bbb-7fe319d85b3a tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:18.780478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f823cd-792c-464d-8bbb-7fe319d85b3a tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:18.780760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f823cd-792c-464d-8bbb-7fe319d85b3a tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] PATH_INFO: `/v3/projects/ecfb9a5fe42a4c99a294e05cad3e03a3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:18.782017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f823cd-792c-464d-8bbb-7fe319d85b3a tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] RBAC: Authorizing `identity:get_project(project_id=ecfb9a5fe42a4c99a294e05cad3e03a3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:18.784287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f823cd-792c-464d-8bbb-7fe319d85b3a tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:18.793778 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1402/2844] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:54:18 2022] GET /identity/v3/projects/ecfb9a5fe42a4c99a294e05cad3e03a3 => generated 379 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:54:19.487227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b89ea-b65e-4880-94cb-fd6fbd65400d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:19.487667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b89ea-b65e-4880-94cb-fd6fbd65400d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:19.487777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b89ea-b65e-4880-94cb-fd6fbd65400d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:19.525908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a63b89ea-b65e-4880-94cb-fd6fbd65400d None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:19.527459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a63b89ea-b65e-4880-94cb-fd6fbd65400d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:19.641401 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1443/2845] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:54:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:19.663514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f823cd-792c-464d-8bbb-7fe319d85b3a tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:19.665947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-15f823cd-792c-464d-8bbb-7fe319d85b3a tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:19.783817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f823cd-792c-464d-8bbb-7fe319d85b3a tempest-FlavorsAccessTestJSON-1409941766 tempest-FlavorsAccessTestJSON-1409941766-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:19.790116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a23290-f2cd-4b8a-952a-c82aba543174 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:19.790741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a23290-f2cd-4b8a-952a-c82aba543174 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:19.791787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a23290-f2cd-4b8a-952a-c82aba543174 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:19.792088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a23290-f2cd-4b8a-952a-c82aba543174 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:19.803166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a23290-f2cd-4b8a-952a-c82aba543174 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:19.805553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a23290-f2cd-4b8a-952a-c82aba543174 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:19.816599 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1403/2846] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:54:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:54:20.658523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b89ea-b65e-4880-94cb-fd6fbd65400d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:20.694822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b89ea-b65e-4880-94cb-fd6fbd65400d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:20.743490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9f68c-3525-4a6c-801b-f5b0bb4985b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:20.747788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9f68c-3525-4a6c-801b-f5b0bb4985b5 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:20.748993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9f68c-3525-4a6c-801b-f5b0bb4985b5 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:20.749450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9f68c-3525-4a6c-801b-f5b0bb4985b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:20.757763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-29a9f68c-3525-4a6c-801b-f5b0bb4985b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:20.797431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9f68c-3525-4a6c-801b-f5b0bb4985b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:20.808141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9f68c-3525-4a6c-801b-f5b0bb4985b5 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:20.830677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710291e-b4f9-4d13-9500-fea886882380 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:20.831650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710291e-b4f9-4d13-9500-fea886882380 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:20.832131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710291e-b4f9-4d13-9500-fea886882380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:20.860752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9710291e-b4f9-4d13-9500-fea886882380 None None] MFA Rules not processed for user `90d7e4c2e577467f949be3dc480cc7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:20.862299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9710291e-b4f9-4d13-9500-fea886882380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:20.879548 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1444/2847] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:21.119340 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1404/2848] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:54:22.732796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9f68c-3525-4a6c-801b-f5b0bb4985b5 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:22.872722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9f68c-3525-4a6c-801b-f5b0bb4985b5 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:22.879002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:22.879921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:22.880366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:22.880878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] PATH_INFO: `/v3/users/d950b3116e1a48ea8b7b2d0c335622db` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:22.883121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d950b3116e1a48ea8b7b2d0c335622db)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:22.885346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:23.008630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd950b3116e1a48ea8b7b2d0c335622db'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:23.030027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd950b3116e1a48ea8b7b2d0c335622db'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:23.037013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:23.046428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd950b3116e1a48ea8b7b2d0c335622db'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:23.047550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd950b3116e1a48ea8b7b2d0c335622db'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:23.048848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd950b3116e1a48ea8b7b2d0c335622db'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:23.085818 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1445/2849] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:54:22 2022] DELETE /identity/v3/users/d950b3116e1a48ea8b7b2d0c335622db => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:23.090933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5893d4b-295b-40d1-8492-723663a613cb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:23.091858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5893d4b-295b-40d1-8492-723663a613cb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:23.092509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5893d4b-295b-40d1-8492-723663a613cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:23.133673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b5893d4b-295b-40d1-8492-723663a613cb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:23.135129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b5893d4b-295b-40d1-8492-723663a613cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:23.233360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:23.306676 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1405/2850] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:54:23.417270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:23.418976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:23.601166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a31f-90e3-4f36-9499-598b2f3407ed admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:23.608518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dead8-1729-40ae-94d2-0cbf430da689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:23.609896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dead8-1729-40ae-94d2-0cbf430da689 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:23.610384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dead8-1729-40ae-94d2-0cbf430da689 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:23.610858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dead8-1729-40ae-94d2-0cbf430da689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:23.626051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dead8-1729-40ae-94d2-0cbf430da689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:23.627967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dead8-1729-40ae-94d2-0cbf430da689 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:23.651697 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1446/2851] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:24.500253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5893d4b-295b-40d1-8492-723663a613cb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:24.502671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b5893d4b-295b-40d1-8492-723663a613cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:24.524169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5893d4b-295b-40d1-8492-723663a613cb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:24.535057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:24.535930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:24.536380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:24.536854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] PATH_INFO: `/v3/projects/ecfb9a5fe42a4c99a294e05cad3e03a3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:24.541588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecfb9a5fe42a4c99a294e05cad3e03a3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:24.543875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:24.669425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecfb9a5fe42a4c99a294e05cad3e03a3', 'request_id': 'req-89be3a46-b289-4e14-a3c7-43176f75b0b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:24.689367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecfb9a5fe42a4c99a294e05cad3e03a3', 'request_id': 'req-89be3a46-b289-4e14-a3c7-43176f75b0b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:24.699988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecfb9a5fe42a4c99a294e05cad3e03a3', 'request_id': 'req-89be3a46-b289-4e14-a3c7-43176f75b0b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:24.722173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] The token cache is being invalidate because project ecfb9a5fe42a4c99a294e05cad3e03a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:54:24.722594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:24.723543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:24.726659 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1406/2852] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:54:24 2022] DELETE /identity/v3/projects/ecfb9a5fe42a4c99a294e05cad3e03a3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:24.730699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dead8-1729-40ae-94d2-0cbf430da689 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:24.747688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dead8-1729-40ae-94d2-0cbf430da689 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:24.922611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc42bd1-ef66-4d2f-9a20-4a50e24bbff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:24.923532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc42bd1-ef66-4d2f-9a20-4a50e24bbff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:24.924469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc42bd1-ef66-4d2f-9a20-4a50e24bbff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:24.924959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc42bd1-ef66-4d2f-9a20-4a50e24bbff5 admin admin] PATH_INFO: `/v3/users/cd786a4dbd8e404584752183731a6309` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:24.929654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc42bd1-ef66-4d2f-9a20-4a50e24bbff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd786a4dbd8e404584752183731a6309)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:24.934532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc42bd1-ef66-4d2f-9a20-4a50e24bbff5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:25.038270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7cc42bd1-ef66-4d2f-9a20-4a50e24bbff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd786a4dbd8e404584752183731a6309'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:25.060684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7cc42bd1-ef66-4d2f-9a20-4a50e24bbff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd786a4dbd8e404584752183731a6309'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:25.074423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7cc42bd1-ef66-4d2f-9a20-4a50e24bbff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd786a4dbd8e404584752183731a6309'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:25.075899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7cc42bd1-ef66-4d2f-9a20-4a50e24bbff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd786a4dbd8e404584752183731a6309'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:25.076524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7cc42bd1-ef66-4d2f-9a20-4a50e24bbff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd786a4dbd8e404584752183731a6309'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:25.107713 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1447/2853] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:54:24 2022] DELETE /identity/v3/users/cd786a4dbd8e404584752183731a6309 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:25.704530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:25.707685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:25.744240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be3a46-b289-4e14-a3c7-43176f75b0b2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:25.829449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e06481-ee08-4d95-be5a-a95a327c3988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:25.830621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e06481-ee08-4d95-be5a-a95a327c3988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:25.831083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e06481-ee08-4d95-be5a-a95a327c3988 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:25.831743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e06481-ee08-4d95-be5a-a95a327c3988 admin admin] PATH_INFO: `/v3/projects/d09f1b78ffac416ebc790368b25b0120` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:25.833840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e06481-ee08-4d95-be5a-a95a327c3988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d09f1b78ffac416ebc790368b25b0120)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:25.836704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e06481-ee08-4d95-be5a-a95a327c3988 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:25.943770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6e06481-ee08-4d95-be5a-a95a327c3988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd09f1b78ffac416ebc790368b25b0120', 'request_id': 'req-c6e06481-ee08-4d95-be5a-a95a327c3988'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:25.968600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6e06481-ee08-4d95-be5a-a95a327c3988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd09f1b78ffac416ebc790368b25b0120', 'request_id': 'req-c6e06481-ee08-4d95-be5a-a95a327c3988'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:25.981317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6e06481-ee08-4d95-be5a-a95a327c3988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd09f1b78ffac416ebc790368b25b0120', 'request_id': 'req-c6e06481-ee08-4d95-be5a-a95a327c3988'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:25.985029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:26.025311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6e06481-ee08-4d95-be5a-a95a327c3988 admin admin] The token cache is being invalidate because project d09f1b78ffac416ebc790368b25b0120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:54:26.025625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6e06481-ee08-4d95-be5a-a95a327c3988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:26.026518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6e06481-ee08-4d95-be5a-a95a327c3988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:26.031114 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1407/2854] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:54:25 2022] DELETE /identity/v3/projects/d09f1b78ffac416ebc790368b25b0120 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:26.037919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80398450-8a3e-40d8-8f64-190b0742778e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:26.039338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80398450-8a3e-40d8-8f64-190b0742778e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:26.039814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80398450-8a3e-40d8-8f64-190b0742778e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:26.066809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:26.106383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-80398450-8a3e-40d8-8f64-190b0742778e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:26.108971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-80398450-8a3e-40d8-8f64-190b0742778e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:26.262648 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1448/2855] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:26.273636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74dd82-db5e-441e-b53a-07a99ab8a01e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:26.274260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74dd82-db5e-441e-b53a-07a99ab8a01e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:26.274612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74dd82-db5e-441e-b53a-07a99ab8a01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:26.309871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3a74dd82-db5e-441e-b53a-07a99ab8a01e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:26.312495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3a74dd82-db5e-441e-b53a-07a99ab8a01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:26.335960 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1408/2856] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:26.340556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80398450-8a3e-40d8-8f64-190b0742778e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:26.359753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80398450-8a3e-40d8-8f64-190b0742778e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:26.375376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a17f0-6d85-4a35-aee2-323b561d6f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:26.377244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a17f0-6d85-4a35-aee2-323b561d6f89 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:26.379482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a17f0-6d85-4a35-aee2-323b561d6f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:26.379668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a17f0-6d85-4a35-aee2-323b561d6f89 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:26.380434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a17f0-6d85-4a35-aee2-323b561d6f89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:54:26.382822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a17f0-6d85-4a35-aee2-323b561d6f89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:26.385861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a17f0-6d85-4a35-aee2-323b561d6f89 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:26.422993 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1449/2857] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:54:26.424978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74dd82-db5e-441e-b53a-07a99ab8a01e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:26.440871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74dd82-db5e-441e-b53a-07a99ab8a01e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:26.461324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a9622-fbfd-4435-866e-e73f70394c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:26.462135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a9622-fbfd-4435-866e-e73f70394c71 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:26.462497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a9622-fbfd-4435-866e-e73f70394c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:26.462946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a9622-fbfd-4435-866e-e73f70394c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:26.463690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a9622-fbfd-4435-866e-e73f70394c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:26.466113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a9622-fbfd-4435-866e-e73f70394c71 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:26.552557 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1409/2858] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:54:26 2022] POST /identity/v3/projects => generated 393 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:26.554039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a17f0-6d85-4a35-aee2-323b561d6f89 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:26.569700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a17f0-6d85-4a35-aee2-323b561d6f89 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:26.706309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df9c1e-ae8a-4bd4-be60-f383fb1a5716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:26.707274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df9c1e-ae8a-4bd4-be60-f383fb1a5716 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:26.707809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df9c1e-ae8a-4bd4-be60-f383fb1a5716 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:26.708325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df9c1e-ae8a-4bd4-be60-f383fb1a5716 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:26.710026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df9c1e-ae8a-4bd4-be60-f383fb1a5716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:26.712671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df9c1e-ae8a-4bd4-be60-f383fb1a5716 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:26.746496 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1450/2859] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:54:26 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:26.746881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a9622-fbfd-4435-866e-e73f70394c71 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:26.773557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a9622-fbfd-4435-866e-e73f70394c71 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:26.781929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07e214-9e56-4197-a27a-4a43f6e1fca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:26.782810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07e214-9e56-4197-a27a-4a43f6e1fca9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:26.783262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07e214-9e56-4197-a27a-4a43f6e1fca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:26.783908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07e214-9e56-4197-a27a-4a43f6e1fca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:26.784865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07e214-9e56-4197-a27a-4a43f6e1fca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:26.824978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07e214-9e56-4197-a27a-4a43f6e1fca9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:26.864346 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1410/2860] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:54:26.974453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df9c1e-ae8a-4bd4-be60-f383fb1a5716 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:27.029709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df9c1e-ae8a-4bd4-be60-f383fb1a5716 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:27.051697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90590bdb-2549-43a1-b5d8-de2dad5d5395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:27.052523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90590bdb-2549-43a1-b5d8-de2dad5d5395 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:27.052660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90590bdb-2549-43a1-b5d8-de2dad5d5395 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:27.053034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90590bdb-2549-43a1-b5d8-de2dad5d5395 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:27.053707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90590bdb-2549-43a1-b5d8-de2dad5d5395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:27.055326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90590bdb-2549-43a1-b5d8-de2dad5d5395 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:27.067528 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1451/2861] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:27.072358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07e214-9e56-4197-a27a-4a43f6e1fca9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:27.087302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07e214-9e56-4197-a27a-4a43f6e1fca9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:27.094393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a7d28-4227-4ef2-84e1-6d243d52aa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:27.095158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a7d28-4227-4ef2-84e1-6d243d52aa21 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:27.095570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a7d28-4227-4ef2-84e1-6d243d52aa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:27.095901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a7d28-4227-4ef2-84e1-6d243d52aa21 admin admin] PATH_INFO: `/v3/projects/8d76fd40b44147ac91a2854a73d3c22e/users/d8043dedc11c4788aa1fdff778f60c1a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:27.120087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a7d28-4227-4ef2-84e1-6d243d52aa21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d76fd40b44147ac91a2854a73d3c22e, user_id=d8043dedc11c4788aa1fdff778f60c1a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:27.123341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a7d28-4227-4ef2-84e1-6d243d52aa21 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:27.148943 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1411/2862] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:27 2022] PUT /identity/v3/projects/8d76fd40b44147ac91a2854a73d3c22e/users/d8043dedc11c4788aa1fdff778f60c1a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:27.160529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cf7c1-6bb5-4171-92eb-a8877c9854d1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:27.161131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cf7c1-6bb5-4171-92eb-a8877c9854d1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:27.161508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cf7c1-6bb5-4171-92eb-a8877c9854d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:27.222123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-547cf7c1-6bb5-4171-92eb-a8877c9854d1 None None] MFA Rules not processed for user `d8043dedc11c4788aa1fdff778f60c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:27.223643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-547cf7c1-6bb5-4171-92eb-a8877c9854d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:27.383183 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1452/2863] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:54:27.384320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a7d28-4227-4ef2-84e1-6d243d52aa21 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:27.513071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a7d28-4227-4ef2-84e1-6d243d52aa21 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:27.526766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44166ced-5f7a-4cef-9a31-1d5dcb0db922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:27.529119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44166ced-5f7a-4cef-9a31-1d5dcb0db922 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:27.538321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44166ced-5f7a-4cef-9a31-1d5dcb0db922 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:27.538321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44166ced-5f7a-4cef-9a31-1d5dcb0db922 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:27.538321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44166ced-5f7a-4cef-9a31-1d5dcb0db922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:27.538321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44166ced-5f7a-4cef-9a31-1d5dcb0db922 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:27.574236 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1412/2864] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:54:27 2022] POST /identity/v3/projects => generated 393 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:54:27.577377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cf7c1-6bb5-4171-92eb-a8877c9854d1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:27.599959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:27.609443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cf7c1-6bb5-4171-92eb-a8877c9854d1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:27.762436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb76210-e097-4084-b3a4-ed3b8c48f777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:27.763456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb76210-e097-4084-b3a4-ed3b8c48f777 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:27.764362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb76210-e097-4084-b3a4-ed3b8c48f777 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:27.765116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb76210-e097-4084-b3a4-ed3b8c48f777 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:27.766541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb76210-e097-4084-b3a4-ed3b8c48f777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:27.769987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb76210-e097-4084-b3a4-ed3b8c48f777 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:27.812591 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1453/2865] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:54:27 2022] POST /identity/v3/users => generated 303 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:54:27.816505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44166ced-5f7a-4cef-9a31-1d5dcb0db922 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:27.834649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44166ced-5f7a-4cef-9a31-1d5dcb0db922 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:27.841648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b2199-12b0-4db3-bfb4-fdf984d6ec81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:27.842754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b2199-12b0-4db3-bfb4-fdf984d6ec81 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:27.843498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b2199-12b0-4db3-bfb4-fdf984d6ec81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:27.844017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b2199-12b0-4db3-bfb4-fdf984d6ec81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:27.845139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b2199-12b0-4db3-bfb4-fdf984d6ec81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:27.846979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b2199-12b0-4db3-bfb4-fdf984d6ec81 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:27.862237 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1413/2866] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:27.866634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb76210-e097-4084-b3a4-ed3b8c48f777 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:27.882013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb76210-e097-4084-b3a4-ed3b8c48f777 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:27.888546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38477321-5039-4563-ae6d-109ba0e477ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:27.896602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38477321-5039-4563-ae6d-109ba0e477ab admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:27.897824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38477321-5039-4563-ae6d-109ba0e477ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:27.898122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38477321-5039-4563-ae6d-109ba0e477ab admin admin] PATH_INFO: `/v3/projects/015d3789e700412a86cf0fe652461001/users/bebe8c834cd44bb7afbf8b26258fb854/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:27.919327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38477321-5039-4563-ae6d-109ba0e477ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=015d3789e700412a86cf0fe652461001, user_id=bebe8c834cd44bb7afbf8b26258fb854, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:27.922756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38477321-5039-4563-ae6d-109ba0e477ab admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:27.954850 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1454/2867] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:27 2022] PUT /identity/v3/projects/015d3789e700412a86cf0fe652461001/users/bebe8c834cd44bb7afbf8b26258fb854/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:27.959131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b2199-12b0-4db3-bfb4-fdf984d6ec81 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:27.972866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b2199-12b0-4db3-bfb4-fdf984d6ec81 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:28.080174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efcbbc-f153-4aa3-9737-74270eefdbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:28.081692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efcbbc-f153-4aa3-9737-74270eefdbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:28.082534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efcbbc-f153-4aa3-9737-74270eefdbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:28.083233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efcbbc-f153-4aa3-9737-74270eefdbc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:28.084057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efcbbc-f153-4aa3-9737-74270eefdbc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:28.085745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efcbbc-f153-4aa3-9737-74270eefdbc3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:28.087601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:28.123508 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1414/2868] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:28.123742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38477321-5039-4563-ae6d-109ba0e477ab admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:28.140282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38477321-5039-4563-ae6d-109ba0e477ab admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:28.152167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a37054-2f0c-497d-a0c5-94c2b4c0cfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:28.152854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a37054-2f0c-497d-a0c5-94c2b4c0cfc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:28.153173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a37054-2f0c-497d-a0c5-94c2b4c0cfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:28.153454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a37054-2f0c-497d-a0c5-94c2b4c0cfc2 admin admin] PATH_INFO: `/v3/projects/015d3789e700412a86cf0fe652461001/users/bebe8c834cd44bb7afbf8b26258fb854/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:28.159466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a37054-2f0c-497d-a0c5-94c2b4c0cfc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=015d3789e700412a86cf0fe652461001, user_id=bebe8c834cd44bb7afbf8b26258fb854, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:28.165756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a37054-2f0c-497d-a0c5-94c2b4c0cfc2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:28.199967 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1455/2869] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:28 2022] PUT /identity/v3/projects/015d3789e700412a86cf0fe652461001/users/bebe8c834cd44bb7afbf8b26258fb854/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:28.201460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082d68d-844e-448f-b74b-1722e4fc63c7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:28.201750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082d68d-844e-448f-b74b-1722e4fc63c7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:28.202029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082d68d-844e-448f-b74b-1722e4fc63c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:28.292192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8082d68d-844e-448f-b74b-1722e4fc63c7 None None] MFA Rules not processed for user `bebe8c834cd44bb7afbf8b26258fb854`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:28.292192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8082d68d-844e-448f-b74b-1722e4fc63c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:28.419568 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1415/2870] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:28 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:54:35.335695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a37054-2f0c-497d-a0c5-94c2b4c0cfc2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:35.337455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c5a37054-2f0c-497d-a0c5-94c2b4c0cfc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:35.479241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a37054-2f0c-497d-a0c5-94c2b4c0cfc2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:35.511024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc559cc-0dc1-446d-911d-46146917f923 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:35.511578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc559cc-0dc1-446d-911d-46146917f923 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:35.511861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc559cc-0dc1-446d-911d-46146917f923 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:35.512237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc559cc-0dc1-446d-911d-46146917f923 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:35.517223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ddc559cc-0dc1-446d-911d-46146917f923 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:35.543448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc559cc-0dc1-446d-911d-46146917f923 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:35.544981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc559cc-0dc1-446d-911d-46146917f923 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:35.564361 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1456/2871] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:35.576494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebac7fc-9cca-44f3-816a-5e95acaa6154 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:35.577090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebac7fc-9cca-44f3-816a-5e95acaa6154 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:35.577090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebac7fc-9cca-44f3-816a-5e95acaa6154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:35.600668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7ebac7fc-9cca-44f3-816a-5e95acaa6154 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:35.602146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7ebac7fc-9cca-44f3-816a-5e95acaa6154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:35.681568 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1416/2872] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:54:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:35.698403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc559cc-0dc1-446d-911d-46146917f923 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:35.699880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ddc559cc-0dc1-446d-911d-46146917f923 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:35.782116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:35.804174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebac7fc-9cca-44f3-816a-5e95acaa6154 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:35.859236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc559cc-0dc1-446d-911d-46146917f923 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:35.876119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fa6ab-6e76-4f17-a353-cfa461b311bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:35.876488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fa6ab-6e76-4f17-a353-cfa461b311bd service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:35.876834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fa6ab-6e76-4f17-a353-cfa461b311bd service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:35.877066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fa6ab-6e76-4f17-a353-cfa461b311bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:35.893929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fa6ab-6e76-4f17-a353-cfa461b311bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:35.895726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fa6ab-6e76-4f17-a353-cfa461b311bd service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:35.932454 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1457/2873] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:54:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:54:36.013594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebac7fc-9cca-44f3-816a-5e95acaa6154 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:36.022801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:36.023549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:36.023861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:36.024169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] PATH_INFO: `/v3/users/d8043dedc11c4788aa1fdff778f60c1a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:36.025456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8043dedc11c4788aa1fdff778f60c1a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:36.055932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:36.204111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:36.288569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8043dedc11c4788aa1fdff778f60c1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:36.309495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8043dedc11c4788aa1fdff778f60c1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:36.310919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8043dedc11c4788aa1fdff778f60c1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:36.324701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8043dedc11c4788aa1fdff778f60c1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:36.338569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8043dedc11c4788aa1fdff778f60c1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:36.368321 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1417/2874] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:54:35 2022] DELETE /identity/v3/users/d8043dedc11c4788aa1fdff778f60c1a => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:54:36.370185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ffd9f-f3ae-4a2c-ba32-7819fce369b7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:36.370693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ffd9f-f3ae-4a2c-ba32-7819fce369b7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:36.371140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ffd9f-f3ae-4a2c-ba32-7819fce369b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:36.419746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b46ffd9f-f3ae-4a2c-ba32-7819fce369b7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:36.423135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b46ffd9f-f3ae-4a2c-ba32-7819fce369b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:36.560631 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1458/2875] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:36.580108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:36.582358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:36.733079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbfd74-36bb-45b9-a34e-bd93363afac7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:36.742452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87b1f8-132d-43c7-b13b-4a0ba5eed0fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:36.743283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87b1f8-132d-43c7-b13b-4a0ba5eed0fe service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:36.743404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87b1f8-132d-43c7-b13b-4a0ba5eed0fe service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:36.743684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87b1f8-132d-43c7-b13b-4a0ba5eed0fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:36.753850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87b1f8-132d-43c7-b13b-4a0ba5eed0fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:36.756830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87b1f8-132d-43c7-b13b-4a0ba5eed0fe service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:36.793348 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1418/2876] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:37.266202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ffd9f-f3ae-4a2c-ba32-7819fce369b7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:37.267661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b46ffd9f-f3ae-4a2c-ba32-7819fce369b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:37.293815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ffd9f-f3ae-4a2c-ba32-7819fce369b7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:37.319624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:37.320462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:37.320663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:37.321455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] PATH_INFO: `/v3/projects/8d76fd40b44147ac91a2854a73d3c22e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:37.323088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d76fd40b44147ac91a2854a73d3c22e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:37.325443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:37.400501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d76fd40b44147ac91a2854a73d3c22e', 'request_id': 'req-202edb4b-0182-41ad-b140-e1f8649bd786'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:37.412624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d76fd40b44147ac91a2854a73d3c22e', 'request_id': 'req-202edb4b-0182-41ad-b140-e1f8649bd786'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:37.421896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d76fd40b44147ac91a2854a73d3c22e', 'request_id': 'req-202edb4b-0182-41ad-b140-e1f8649bd786'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:37.438487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] The token cache is being invalidate because project 8d76fd40b44147ac91a2854a73d3c22e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:54:37.439245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:37.440167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:37.442377 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1459/2877] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:54:37 2022] DELETE /identity/v3/projects/8d76fd40b44147ac91a2854a73d3c22e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:37.447037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87b1f8-132d-43c7-b13b-4a0ba5eed0fe service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:37.459763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87b1f8-132d-43c7-b13b-4a0ba5eed0fe service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:37.538373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6412e8-0b7f-467e-b8b8-b0d9d377c153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:37.539180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6412e8-0b7f-467e-b8b8-b0d9d377c153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:37.539565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6412e8-0b7f-467e-b8b8-b0d9d377c153 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:37.539912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6412e8-0b7f-467e-b8b8-b0d9d377c153 admin admin] PATH_INFO: `/v3/users/bebe8c834cd44bb7afbf8b26258fb854` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:37.541178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6412e8-0b7f-467e-b8b8-b0d9d377c153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bebe8c834cd44bb7afbf8b26258fb854)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:37.543267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6412e8-0b7f-467e-b8b8-b0d9d377c153 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:37.620527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bf6412e8-0b7f-467e-b8b8-b0d9d377c153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bebe8c834cd44bb7afbf8b26258fb854'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:37.622960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:37.644809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bf6412e8-0b7f-467e-b8b8-b0d9d377c153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bebe8c834cd44bb7afbf8b26258fb854'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:37.647620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bf6412e8-0b7f-467e-b8b8-b0d9d377c153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bebe8c834cd44bb7afbf8b26258fb854'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:37.664244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bf6412e8-0b7f-467e-b8b8-b0d9d377c153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bebe8c834cd44bb7afbf8b26258fb854'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:37.676976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bf6412e8-0b7f-467e-b8b8-b0d9d377c153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bebe8c834cd44bb7afbf8b26258fb854'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:37.698320 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1419/2878] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:54:37 2022] DELETE /identity/v3/users/bebe8c834cd44bb7afbf8b26258fb854 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:38.094681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:38.096858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:38.158782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202edb4b-0182-41ad-b140-e1f8649bd786 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:38.237534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:38.262646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a1721-ac6a-488b-87d1-5889c1f51193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:38.263614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a1721-ac6a-488b-87d1-5889c1f51193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:38.263912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a1721-ac6a-488b-87d1-5889c1f51193 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:38.264305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a1721-ac6a-488b-87d1-5889c1f51193 admin admin] PATH_INFO: `/v3/projects/015d3789e700412a86cf0fe652461001` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:38.265502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a1721-ac6a-488b-87d1-5889c1f51193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=015d3789e700412a86cf0fe652461001)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:38.267989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a1721-ac6a-488b-87d1-5889c1f51193 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:38.344660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-777a1721-ac6a-488b-87d1-5889c1f51193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '015d3789e700412a86cf0fe652461001', 'request_id': 'req-777a1721-ac6a-488b-87d1-5889c1f51193'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:38.361248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-777a1721-ac6a-488b-87d1-5889c1f51193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '015d3789e700412a86cf0fe652461001', 'request_id': 'req-777a1721-ac6a-488b-87d1-5889c1f51193'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:38.378432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-777a1721-ac6a-488b-87d1-5889c1f51193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '015d3789e700412a86cf0fe652461001', 'request_id': 'req-777a1721-ac6a-488b-87d1-5889c1f51193'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:38.397177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-777a1721-ac6a-488b-87d1-5889c1f51193 admin admin] The token cache is being invalidate because project 015d3789e700412a86cf0fe652461001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:54:38.397536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-777a1721-ac6a-488b-87d1-5889c1f51193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:38.398213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-777a1721-ac6a-488b-87d1-5889c1f51193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:38.400951 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1460/2879] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:54:38 2022] DELETE /identity/v3/projects/015d3789e700412a86cf0fe652461001 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:38.405751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c079f32-a751-4bd5-9590-446414a0b1c7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:38.406068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c079f32-a751-4bd5-9590-446414a0b1c7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:38.406355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c079f32-a751-4bd5-9590-446414a0b1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:38.430997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8c079f32-a751-4bd5-9590-446414a0b1c7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:38.432914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8c079f32-a751-4bd5-9590-446414a0b1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:38.539422 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1420/2880] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:38.546843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213677-ba29-466b-b7ad-2af8895df41f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:38.547138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213677-ba29-466b-b7ad-2af8895df41f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:38.547426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213677-ba29-466b-b7ad-2af8895df41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:38.583855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7f213677-ba29-466b-b7ad-2af8895df41f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:38.585676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7f213677-ba29-466b-b7ad-2af8895df41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:38.596937 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1461/2881] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:38.603951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c079f32-a751-4bd5-9590-446414a0b1c7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:38.617939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c079f32-a751-4bd5-9590-446414a0b1c7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:38.627055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90f7e3-15b9-45fe-a319-a48517cbbf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:38.629185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90f7e3-15b9-45fe-a319-a48517cbbf88 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:38.630086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90f7e3-15b9-45fe-a319-a48517cbbf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:38.630632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90f7e3-15b9-45fe-a319-a48517cbbf88 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:38.632410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90f7e3-15b9-45fe-a319-a48517cbbf88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:54:38.633388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90f7e3-15b9-45fe-a319-a48517cbbf88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:38.638222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90f7e3-15b9-45fe-a319-a48517cbbf88 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:38.656936 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1421/2882] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:54:38.660773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f213677-ba29-466b-b7ad-2af8895df41f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:38.701068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f213677-ba29-466b-b7ad-2af8895df41f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:38.716860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1f03a-2d40-45e2-a6dc-fcd97b66c9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:38.723109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1f03a-2d40-45e2-a6dc-fcd97b66c9ee admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:38.723109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1f03a-2d40-45e2-a6dc-fcd97b66c9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:38.723109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1f03a-2d40-45e2-a6dc-fcd97b66c9ee admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:38.723109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1f03a-2d40-45e2-a6dc-fcd97b66c9ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:38.723109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1f03a-2d40-45e2-a6dc-fcd97b66c9ee admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:38.845300 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1462/2883] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:54:38 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:38.867862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90f7e3-15b9-45fe-a319-a48517cbbf88 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:38.889818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90f7e3-15b9-45fe-a319-a48517cbbf88 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:39.014708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dfe85-308e-4c11-9a85-882666223c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:39.015709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dfe85-308e-4c11-9a85-882666223c5a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:39.016196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dfe85-308e-4c11-9a85-882666223c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:39.016869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dfe85-308e-4c11-9a85-882666223c5a admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:39.017978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dfe85-308e-4c11-9a85-882666223c5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:39.020558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dfe85-308e-4c11-9a85-882666223c5a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:39.058643 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1422/2884] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:54:38 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:39.059860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1f03a-2d40-45e2-a6dc-fcd97b66c9ee admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:39.088185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1f03a-2d40-45e2-a6dc-fcd97b66c9ee admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:39.094037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1f24e-bd70-4f63-80b5-279aec023837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:39.094755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1f24e-bd70-4f63-80b5-279aec023837 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:39.095113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1f24e-bd70-4f63-80b5-279aec023837 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:39.095540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1f24e-bd70-4f63-80b5-279aec023837 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:39.096505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1f24e-bd70-4f63-80b5-279aec023837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:39.098662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1f24e-bd70-4f63-80b5-279aec023837 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:39.114337 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1463/2885] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:54:39.118736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dfe85-308e-4c11-9a85-882666223c5a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:39.134674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dfe85-308e-4c11-9a85-882666223c5a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:39.142652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a5f2d-62c7-402e-bb10-9cf6bd0334a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:39.143284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a5f2d-62c7-402e-bb10-9cf6bd0334a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:39.143835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a5f2d-62c7-402e-bb10-9cf6bd0334a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:39.144129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a5f2d-62c7-402e-bb10-9cf6bd0334a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:39.144778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a5f2d-62c7-402e-bb10-9cf6bd0334a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:39.146807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a5f2d-62c7-402e-bb10-9cf6bd0334a0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:39.168632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1f24e-bd70-4f63-80b5-279aec023837 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:39.170675 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1423/2886] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:39.198180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1f24e-bd70-4f63-80b5-279aec023837 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:39.222976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39b499-8f06-450d-9716-ab4b475fe4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:39.223947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39b499-8f06-450d-9716-ab4b475fe4ab admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:39.224231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39b499-8f06-450d-9716-ab4b475fe4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:39.224516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39b499-8f06-450d-9716-ab4b475fe4ab admin admin] PATH_INFO: `/v3/projects/5e45c33379264eb48e4e356b6d27ea57/users/11898a53e6e74891ab4de4508f583f78/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:39.256330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39b499-8f06-450d-9716-ab4b475fe4ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e45c33379264eb48e4e356b6d27ea57, user_id=11898a53e6e74891ab4de4508f583f78, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:39.260221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39b499-8f06-450d-9716-ab4b475fe4ab admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:39.294852 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1464/2887] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:39 2022] PUT /identity/v3/projects/5e45c33379264eb48e4e356b6d27ea57/users/11898a53e6e74891ab4de4508f583f78/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:39.306082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232d8e1-7fbb-4fea-b9d1-fa7ed6186f7d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:39.306418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232d8e1-7fbb-4fea-b9d1-fa7ed6186f7d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:39.306643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232d8e1-7fbb-4fea-b9d1-fa7ed6186f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:39.346684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e232d8e1-7fbb-4fea-b9d1-fa7ed6186f7d None None] MFA Rules not processed for user `11898a53e6e74891ab4de4508f583f78`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:39.348347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e232d8e1-7fbb-4fea-b9d1-fa7ed6186f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:39.374099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:39.452124 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1424/2888] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:39.453211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39b499-8f06-450d-9716-ab4b475fe4ab admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:39.563732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39b499-8f06-450d-9716-ab4b475fe4ab admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:39.577194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197f3c4-050c-421c-9d87-7fec760878c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:39.579079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197f3c4-050c-421c-9d87-7fec760878c9 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:39.579079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197f3c4-050c-421c-9d87-7fec760878c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:39.579442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197f3c4-050c-421c-9d87-7fec760878c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:39.580208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197f3c4-050c-421c-9d87-7fec760878c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:39.584718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197f3c4-050c-421c-9d87-7fec760878c9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:39.658463 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1465/2889] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:54:39 2022] POST /identity/v3/projects => generated 381 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:39.659983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232d8e1-7fbb-4fea-b9d1-fa7ed6186f7d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:39.672414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232d8e1-7fbb-4fea-b9d1-fa7ed6186f7d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:39.767646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f8ec4-e5f8-4fd2-9eea-315cdb4ae81c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:39.768181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f8ec4-e5f8-4fd2-9eea-315cdb4ae81c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:39.768627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f8ec4-e5f8-4fd2-9eea-315cdb4ae81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:39.781942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:39.800694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e87f8ec4-e5f8-4fd2-9eea-315cdb4ae81c None None] MFA Rules not processed for user `2e11c33c786746e5b3effedd32674582`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:39.809031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e87f8ec4-e5f8-4fd2-9eea-315cdb4ae81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:39.832693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a51d1a-3878-4db4-a3fd-6e7057fd5f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:39.835346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a51d1a-3878-4db4-a3fd-6e7057fd5f1d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:39.836517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a51d1a-3878-4db4-a3fd-6e7057fd5f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:39.836914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a51d1a-3878-4db4-a3fd-6e7057fd5f1d admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:39.838172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a51d1a-3878-4db4-a3fd-6e7057fd5f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:39.845639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a51d1a-3878-4db4-a3fd-6e7057fd5f1d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:39.892297 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1425/2890] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:54:39 2022] POST /identity/v3/users => generated 297 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:39.899542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a51d1a-3878-4db4-a3fd-6e7057fd5f1d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:39.923714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a51d1a-3878-4db4-a3fd-6e7057fd5f1d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:39.954551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1cd36-d73b-4b4d-ab88-a969dd018436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:39.955405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1cd36-d73b-4b4d-ab88-a969dd018436 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:39.955706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1cd36-d73b-4b4d-ab88-a969dd018436 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:39.955976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1cd36-d73b-4b4d-ab88-a969dd018436 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:39.956630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1cd36-d73b-4b4d-ab88-a969dd018436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:39.960805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1cd36-d73b-4b4d-ab88-a969dd018436 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:39.972355 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1466/2891] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:54:39.980997 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1426/2892] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:39.984016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f8ec4-e5f8-4fd2-9eea-315cdb4ae81c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:40.001592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f8ec4-e5f8-4fd2-9eea-315cdb4ae81c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:40.017183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ee666-9add-4ba2-b406-f18456d6c9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:40.018035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ee666-9add-4ba2-b406-f18456d6c9fe admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:40.019127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ee666-9add-4ba2-b406-f18456d6c9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:40.019493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ee666-9add-4ba2-b406-f18456d6c9fe admin admin] PATH_INFO: `/v3/projects/bf3354abfb61426ba2e0f40e105164f1/users/5dc613a8a4c04a6c87359cd43cd18111/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:40.041393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ee666-9add-4ba2-b406-f18456d6c9fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf3354abfb61426ba2e0f40e105164f1, user_id=5dc613a8a4c04a6c87359cd43cd18111, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:40.044758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ee666-9add-4ba2-b406-f18456d6c9fe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:40.076278 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1467/2893] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:39 2022] PUT /identity/v3/projects/bf3354abfb61426ba2e0f40e105164f1/users/5dc613a8a4c04a6c87359cd43cd18111/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:40.081475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1cd36-d73b-4b4d-ab88-a969dd018436 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:40.109315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1cd36-d73b-4b4d-ab88-a969dd018436 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:40.265840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce888748-ad56-4760-b26e-4abf5d72cc3a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:40.266223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce888748-ad56-4760-b26e-4abf5d72cc3a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:40.266580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce888748-ad56-4760-b26e-4abf5d72cc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:40.310301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ce888748-ad56-4760-b26e-4abf5d72cc3a None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:40.315105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ce888748-ad56-4760-b26e-4abf5d72cc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:40.414598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6cf83-43d7-454a-9c37-ee81feb22dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:40.414598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6cf83-43d7-454a-9c37-ee81feb22dbd admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:40.414598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6cf83-43d7-454a-9c37-ee81feb22dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:40.414598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6cf83-43d7-454a-9c37-ee81feb22dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:40.415162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6cf83-43d7-454a-9c37-ee81feb22dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:40.415162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6cf83-43d7-454a-9c37-ee81feb22dbd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:40.451920 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1427/2894] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:40.454324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6cf83-43d7-454a-9c37-ee81feb22dbd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:40.478669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6cf83-43d7-454a-9c37-ee81feb22dbd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:40.493842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ce0cd-6a88-4bb7-80eb-328fc08e90ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:40.495011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ce0cd-6a88-4bb7-80eb-328fc08e90ea admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:40.495618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ce0cd-6a88-4bb7-80eb-328fc08e90ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:40.496104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ce0cd-6a88-4bb7-80eb-328fc08e90ea admin admin] PATH_INFO: `/v3/projects/bf3354abfb61426ba2e0f40e105164f1/users/5dc613a8a4c04a6c87359cd43cd18111/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:40.499541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ce0cd-6a88-4bb7-80eb-328fc08e90ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf3354abfb61426ba2e0f40e105164f1, user_id=5dc613a8a4c04a6c87359cd43cd18111, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:40.502803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ce0cd-6a88-4bb7-80eb-328fc08e90ea admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:40.521388 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1468/2895] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:54:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:54:40.568513 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1428/2896] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:40 2022] PUT /identity/v3/projects/bf3354abfb61426ba2e0f40e105164f1/users/5dc613a8a4c04a6c87359cd43cd18111/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:40.591869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aab255-5cb6-4989-b97c-a3332d501d65 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:40.593580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ce0cd-6a88-4bb7-80eb-328fc08e90ea admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:40.593840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aab255-5cb6-4989-b97c-a3332d501d65 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:40.594467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aab255-5cb6-4989-b97c-a3332d501d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:40.650545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-128ce0cd-6a88-4bb7-80eb-328fc08e90ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:40.650676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f9aab255-5cb6-4989-b97c-a3332d501d65 None None] MFA Rules not processed for user `5dc613a8a4c04a6c87359cd43cd18111`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:40.650676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f9aab255-5cb6-4989-b97c-a3332d501d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:40.658297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:40.745670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ce0cd-6a88-4bb7-80eb-328fc08e90ea admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:40.754977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95058085-2adb-481c-8b91-0f21733ee658 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:40.755833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95058085-2adb-481c-8b91-0f21733ee658 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:40.756568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95058085-2adb-481c-8b91-0f21733ee658 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:40.757071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95058085-2adb-481c-8b91-0f21733ee658 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:40.782567 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1469/2897] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:40.900215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95058085-2adb-481c-8b91-0f21733ee658 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:40.902721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95058085-2adb-481c-8b91-0f21733ee658 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:40.913368 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1429/2898] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:54:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:54:42.066839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aab255-5cb6-4989-b97c-a3332d501d65 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:42.069801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f9aab255-5cb6-4989-b97c-a3332d501d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:42.135701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:42.316665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aab255-5cb6-4989-b97c-a3332d501d65 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:42.330776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ceaed-b7b9-4b57-bd71-308c67372688 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:42.331453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ceaed-b7b9-4b57-bd71-308c67372688 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:42.331755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ceaed-b7b9-4b57-bd71-308c67372688 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:42.332089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ceaed-b7b9-4b57-bd71-308c67372688 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:42.354005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ceaed-b7b9-4b57-bd71-308c67372688 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:42.356474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ceaed-b7b9-4b57-bd71-308c67372688 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:42.382118 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1470/2899] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:42.861791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f2d78-d4ae-4009-a206-f41510752133 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:42.862312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f2d78-d4ae-4009-a206-f41510752133 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:42.862930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f2d78-d4ae-4009-a206-f41510752133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:42.907318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-990f2d78-d4ae-4009-a206-f41510752133 None None] MFA Rules not processed for user `d9688dca94a841cfa29420ad780562cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:42.908811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-990f2d78-d4ae-4009-a206-f41510752133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:43.056019 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1430/2900] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:48.827809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ceaed-b7b9-4b57-bd71-308c67372688 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:49.127930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ceaed-b7b9-4b57-bd71-308c67372688 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:49.138969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:49.140361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:49.142072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:49.142590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] PATH_INFO: `/v3/users/11898a53e6e74891ab4de4508f583f78` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:49.144477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=11898a53e6e74891ab4de4508f583f78)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:49.152118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:49.264137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11898a53e6e74891ab4de4508f583f78'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:49.307241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11898a53e6e74891ab4de4508f583f78'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:49.326697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11898a53e6e74891ab4de4508f583f78'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:49.327381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11898a53e6e74891ab4de4508f583f78'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:49.328136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11898a53e6e74891ab4de4508f583f78'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:49.360738 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1471/2901] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:54:48 2022] DELETE /identity/v3/users/11898a53e6e74891ab4de4508f583f78 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:54:49.361506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb1cc0-1ce8-4661-ae3e-30543b1e2036 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:49.362579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb1cc0-1ce8-4661-ae3e-30543b1e2036 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:49.363094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb1cc0-1ce8-4661-ae3e-30543b1e2036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:49.423276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cecb1cc0-1ce8-4661-ae3e-30543b1e2036 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:49.430197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cecb1cc0-1ce8-4661-ae3e-30543b1e2036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:49.576487 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1431/2902] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:49.618801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:49.620879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:49.672222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:49.788930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066aa36f-9a6c-40dc-9d33-6cbc978114cc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:49.799168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65455a-35b1-41f5-8c3a-af2072c40f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:49.801267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65455a-35b1-41f5-8c3a-af2072c40f24 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:49.801501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65455a-35b1-41f5-8c3a-af2072c40f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:49.801784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65455a-35b1-41f5-8c3a-af2072c40f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:49.862773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65455a-35b1-41f5-8c3a-af2072c40f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:49.864442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65455a-35b1-41f5-8c3a-af2072c40f24 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:49.909164 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1472/2903] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:50.835880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb1cc0-1ce8-4661-ae3e-30543b1e2036 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:50.837526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cecb1cc0-1ce8-4661-ae3e-30543b1e2036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:50.863161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb1cc0-1ce8-4661-ae3e-30543b1e2036 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:50.876582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:50.877279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:50.877567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:50.877850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] PATH_INFO: `/v3/projects/5e45c33379264eb48e4e356b6d27ea57` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:50.882646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e45c33379264eb48e4e356b6d27ea57)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:50.885287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:50.897335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:51.015721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e45c33379264eb48e4e356b6d27ea57', 'request_id': 'req-39f80453-3aaf-45e5-ac7e-6edff214cee8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:51.033100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e45c33379264eb48e4e356b6d27ea57', 'request_id': 'req-39f80453-3aaf-45e5-ac7e-6edff214cee8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:51.045548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e45c33379264eb48e4e356b6d27ea57', 'request_id': 'req-39f80453-3aaf-45e5-ac7e-6edff214cee8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:51.078837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] The token cache is being invalidate because project 5e45c33379264eb48e4e356b6d27ea57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:54:51.079095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:51.079937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:51.084660 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1432/2904] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:54:50 2022] DELETE /identity/v3/projects/5e45c33379264eb48e4e356b6d27ea57 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:51.087980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65455a-35b1-41f5-8c3a-af2072c40f24 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:51.113835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65455a-35b1-41f5-8c3a-af2072c40f24 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:51.320463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db0481-e894-45db-8ca4-6b76a2144162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:51.321144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0481-e894-45db-8ca4-6b76a2144162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:51.321432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0481-e894-45db-8ca4-6b76a2144162 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:51.321765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0481-e894-45db-8ca4-6b76a2144162 admin admin] PATH_INFO: `/v3/users/5dc613a8a4c04a6c87359cd43cd18111` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:51.322938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db0481-e894-45db-8ca4-6b76a2144162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc613a8a4c04a6c87359cd43cd18111)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:51.324996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db0481-e894-45db-8ca4-6b76a2144162 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:51.442513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-27db0481-e894-45db-8ca4-6b76a2144162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc613a8a4c04a6c87359cd43cd18111'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:51.476053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-27db0481-e894-45db-8ca4-6b76a2144162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc613a8a4c04a6c87359cd43cd18111'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:51.490725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-27db0481-e894-45db-8ca4-6b76a2144162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc613a8a4c04a6c87359cd43cd18111'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:51.491661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-27db0481-e894-45db-8ca4-6b76a2144162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc613a8a4c04a6c87359cd43cd18111'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:51.492656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-27db0481-e894-45db-8ca4-6b76a2144162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc613a8a4c04a6c87359cd43cd18111'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:51.540962 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1473/2905] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:54:51 2022] DELETE /identity/v3/users/5dc613a8a4c04a6c87359cd43cd18111 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:52.393651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:52.397786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:52.439955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f80453-3aaf-45e5-ac7e-6edff214cee8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:52.742862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:52.743798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:52.745225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:52.745644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75 admin admin] PATH_INFO: `/v3/projects/bf3354abfb61426ba2e0f40e105164f1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:52.747339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf3354abfb61426ba2e0f40e105164f1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:52.763602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:52.982393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf3354abfb61426ba2e0f40e105164f1', 'request_id': 'req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:52.996569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf3354abfb61426ba2e0f40e105164f1', 'request_id': 'req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:53.039218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf3354abfb61426ba2e0f40e105164f1', 'request_id': 'req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:53.077484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75 admin admin] The token cache is being invalidate because project bf3354abfb61426ba2e0f40e105164f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:54:53.078060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:53.079046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e2fe52e5-059b-46ec-a68b-ab07dd6c6d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:53.084831 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1433/2906] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:54:52 2022] DELETE /identity/v3/projects/bf3354abfb61426ba2e0f40e105164f1 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:53.089846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15dc40-a340-4ff2-a8eb-e2cdd9397edc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:53.090453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15dc40-a340-4ff2-a8eb-e2cdd9397edc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:53.090453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15dc40-a340-4ff2-a8eb-e2cdd9397edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:53.175120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4c15dc40-a340-4ff2-a8eb-e2cdd9397edc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:53.177267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4c15dc40-a340-4ff2-a8eb-e2cdd9397edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:53.307204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:53.368478 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1474/2907] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:53.376761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b6f09-f5ed-44b3-a209-19444b41c606 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:53.377017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b6f09-f5ed-44b3-a209-19444b41c606 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:53.377305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b6f09-f5ed-44b3-a209-19444b41c606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:53.418962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ee6b6f09-f5ed-44b3-a209-19444b41c606 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:53.423938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ee6b6f09-f5ed-44b3-a209-19444b41c606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:53.471347 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1434/2908] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:53.476309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15dc40-a340-4ff2-a8eb-e2cdd9397edc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:53.510031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15dc40-a340-4ff2-a8eb-e2cdd9397edc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:53.598998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384b263-9aec-4d49-9d8e-57ce3047afff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:53.600201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384b263-9aec-4d49-9d8e-57ce3047afff admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:53.600496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384b263-9aec-4d49-9d8e-57ce3047afff admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:53.600768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384b263-9aec-4d49-9d8e-57ce3047afff admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:53.608668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384b263-9aec-4d49-9d8e-57ce3047afff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:54:53.621618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384b263-9aec-4d49-9d8e-57ce3047afff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:53.628084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384b263-9aec-4d49-9d8e-57ce3047afff admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:53.711672 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1475/2909] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:54:53.716247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b6f09-f5ed-44b3-a209-19444b41c606 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:53.755442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b6f09-f5ed-44b3-a209-19444b41c606 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:53.766513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29d589-d55f-400f-ac8a-8e18d5d5870f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:53.767372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29d589-d55f-400f-ac8a-8e18d5d5870f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:53.768174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29d589-d55f-400f-ac8a-8e18d5d5870f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:53.768628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29d589-d55f-400f-ac8a-8e18d5d5870f admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:53.769492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29d589-d55f-400f-ac8a-8e18d5d5870f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:53.773937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29d589-d55f-400f-ac8a-8e18d5d5870f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:53.825346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:53.863753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384b263-9aec-4d49-9d8e-57ce3047afff admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:53.870966 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1435/2910] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:54:53 2022] POST /identity/v3/projects => generated 389 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:53.878975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384b263-9aec-4d49-9d8e-57ce3047afff admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:54.041153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da012850-4af2-4242-9667-4d5d2a53ed9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:54.041781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da012850-4af2-4242-9667-4d5d2a53ed9a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:54.042273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da012850-4af2-4242-9667-4d5d2a53ed9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:54.043300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da012850-4af2-4242-9667-4d5d2a53ed9a admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:54.043982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da012850-4af2-4242-9667-4d5d2a53ed9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:54.047860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da012850-4af2-4242-9667-4d5d2a53ed9a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:54.113945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29d589-d55f-400f-ac8a-8e18d5d5870f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:54.116373 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1476/2911] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:54:53 2022] POST /identity/v3/users => generated 295 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:54.133979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29d589-d55f-400f-ac8a-8e18d5d5870f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:54.150302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0458a58-2bcb-4466-8067-1bba6c0ea817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:54.150951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0458a58-2bcb-4466-8067-1bba6c0ea817 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:54.151241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0458a58-2bcb-4466-8067-1bba6c0ea817 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:54.151525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0458a58-2bcb-4466-8067-1bba6c0ea817 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:54.153666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0458a58-2bcb-4466-8067-1bba6c0ea817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:54.155550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0458a58-2bcb-4466-8067-1bba6c0ea817 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:54.179320 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1436/2912] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:54:54.184024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da012850-4af2-4242-9667-4d5d2a53ed9a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:54.202561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da012850-4af2-4242-9667-4d5d2a53ed9a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:54.212199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f62309-e038-4f1d-ab94-20487b834d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:54.212816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f62309-e038-4f1d-ab94-20487b834d08 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:54.213176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f62309-e038-4f1d-ab94-20487b834d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:54.213751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f62309-e038-4f1d-ab94-20487b834d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:54.214236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f62309-e038-4f1d-ab94-20487b834d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:54.216939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f62309-e038-4f1d-ab94-20487b834d08 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:54.247475 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1477/2913] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:54.248294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0458a58-2bcb-4466-8067-1bba6c0ea817 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:54.259248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0458a58-2bcb-4466-8067-1bba6c0ea817 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:54.283915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e979f9b-bf70-4ee6-90e9-5c9426636307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:54.284598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e979f9b-bf70-4ee6-90e9-5c9426636307 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:54.284884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e979f9b-bf70-4ee6-90e9-5c9426636307 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:54.285201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e979f9b-bf70-4ee6-90e9-5c9426636307 admin admin] PATH_INFO: `/v3/projects/d7a1d0f2ac8e42c9be1fffd847099ac1/users/a98d5becb92443df808c54329bf8f7e9/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:54.339088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e979f9b-bf70-4ee6-90e9-5c9426636307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7a1d0f2ac8e42c9be1fffd847099ac1, user_id=a98d5becb92443df808c54329bf8f7e9, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:54.341977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e979f9b-bf70-4ee6-90e9-5c9426636307 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:54.386072 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1437/2914] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:54 2022] PUT /identity/v3/projects/d7a1d0f2ac8e42c9be1fffd847099ac1/users/a98d5becb92443df808c54329bf8f7e9/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:54.388349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7026263-ce70-4a41-919b-6e36c504caa0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:54.388922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7026263-ce70-4a41-919b-6e36c504caa0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:54.389444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7026263-ce70-4a41-919b-6e36c504caa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:54.475839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c7026263-ce70-4a41-919b-6e36c504caa0 None None] MFA Rules not processed for user `a98d5becb92443df808c54329bf8f7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:54.478716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c7026263-ce70-4a41-919b-6e36c504caa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:54.574770 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1478/2915] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:54:54.580158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e979f9b-bf70-4ee6-90e9-5c9426636307 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:54.721151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e979f9b-bf70-4ee6-90e9-5c9426636307 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:54.726444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ec08b-ce80-4a54-8c3c-161e89632b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:54.727086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ec08b-ce80-4a54-8c3c-161e89632b4f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:54.727376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ec08b-ce80-4a54-8c3c-161e89632b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:54.727657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ec08b-ce80-4a54-8c3c-161e89632b4f admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:54.728306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ec08b-ce80-4a54-8c3c-161e89632b4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:54.730949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ec08b-ce80-4a54-8c3c-161e89632b4f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:54.781780 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1438/2916] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:54:54 2022] POST /identity/v3/projects => generated 391 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:54.782957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7026263-ce70-4a41-919b-6e36c504caa0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:54.795530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7026263-ce70-4a41-919b-6e36c504caa0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:54.837833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:54.874182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175d266-dd7f-4b18-ade2-f1bb4dd9e4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:54.875460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175d266-dd7f-4b18-ade2-f1bb4dd9e4ac admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:54.876043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175d266-dd7f-4b18-ade2-f1bb4dd9e4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:54.876607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175d266-dd7f-4b18-ade2-f1bb4dd9e4ac admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:54.878094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175d266-dd7f-4b18-ade2-f1bb4dd9e4ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:54.881195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175d266-dd7f-4b18-ade2-f1bb4dd9e4ac admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:54.910877 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1479/2917] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:54:54 2022] POST /identity/v3/users => generated 302 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:54:54.914956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ec08b-ce80-4a54-8c3c-161e89632b4f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:54.928088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ec08b-ce80-4a54-8c3c-161e89632b4f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:54.947073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58238611-8340-4bf9-bbb7-db57027ea61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:54.947722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58238611-8340-4bf9-bbb7-db57027ea61e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:54.948000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58238611-8340-4bf9-bbb7-db57027ea61e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:54.948284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58238611-8340-4bf9-bbb7-db57027ea61e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:54.948988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58238611-8340-4bf9-bbb7-db57027ea61e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:54.952314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58238611-8340-4bf9-bbb7-db57027ea61e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:54.992068 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1439/2918] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:54.993094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175d266-dd7f-4b18-ade2-f1bb4dd9e4ac admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:55.033119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175d266-dd7f-4b18-ade2-f1bb4dd9e4ac admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:55.044759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98f9a0-6781-4700-9d4f-70c999707aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:55.046926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98f9a0-6781-4700-9d4f-70c999707aef admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:55.047522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98f9a0-6781-4700-9d4f-70c999707aef admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:55.049525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98f9a0-6781-4700-9d4f-70c999707aef admin admin] PATH_INFO: `/v3/projects/2f08394b65004113b7215f2c9d853bca/users/4c0cb36c93bc4e99a78adbf7f819a5bb/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:55.087699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98f9a0-6781-4700-9d4f-70c999707aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f08394b65004113b7215f2c9d853bca, user_id=4c0cb36c93bc4e99a78adbf7f819a5bb, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:55.092291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98f9a0-6781-4700-9d4f-70c999707aef admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:55.168719 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1480/2919] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:54 2022] PUT /identity/v3/projects/2f08394b65004113b7215f2c9d853bca/users/4c0cb36c93bc4e99a78adbf7f819a5bb/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:54:55.172778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58238611-8340-4bf9-bbb7-db57027ea61e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:55.188441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58238611-8340-4bf9-bbb7-db57027ea61e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:55.347691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565a9ea-9dec-4136-8397-0f2f9638cd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:55.350188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565a9ea-9dec-4136-8397-0f2f9638cd6e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:55.351882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565a9ea-9dec-4136-8397-0f2f9638cd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:55.351882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565a9ea-9dec-4136-8397-0f2f9638cd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:55.354499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565a9ea-9dec-4136-8397-0f2f9638cd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:55.361127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565a9ea-9dec-4136-8397-0f2f9638cd6e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:55.398693 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1440/2920] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:54:55.402771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98f9a0-6781-4700-9d4f-70c999707aef admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:55.422204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98f9a0-6781-4700-9d4f-70c999707aef admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:55.437429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb993f-d3aa-41bd-8f4d-b947e199110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:55.438313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb993f-d3aa-41bd-8f4d-b947e199110c admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:55.438710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb993f-d3aa-41bd-8f4d-b947e199110c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:55.440176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb993f-d3aa-41bd-8f4d-b947e199110c admin admin] PATH_INFO: `/v3/projects/2f08394b65004113b7215f2c9d853bca/users/4c0cb36c93bc4e99a78adbf7f819a5bb/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:55.445535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb993f-d3aa-41bd-8f4d-b947e199110c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f08394b65004113b7215f2c9d853bca, user_id=4c0cb36c93bc4e99a78adbf7f819a5bb, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:55.453052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb993f-d3aa-41bd-8f4d-b947e199110c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:55.480550 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1481/2921] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:54:55 2022] PUT /identity/v3/projects/2f08394b65004113b7215f2c9d853bca/users/4c0cb36c93bc4e99a78adbf7f819a5bb/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:55.567752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94603243-2704-4432-a5a0-3bd726c461bf None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:55.568327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94603243-2704-4432-a5a0-3bd726c461bf None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:55.568813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94603243-2704-4432-a5a0-3bd726c461bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:55.654817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-94603243-2704-4432-a5a0-3bd726c461bf None None] MFA Rules not processed for user `4c0cb36c93bc4e99a78adbf7f819a5bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:55.673612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-94603243-2704-4432-a5a0-3bd726c461bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:55.718565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:55.812824 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1441/2922] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:55 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:55.877612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb993f-d3aa-41bd-8f4d-b947e199110c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:55.879410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6cbb993f-d3aa-41bd-8f4d-b947e199110c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:56.083186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb993f-d3aa-41bd-8f4d-b947e199110c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:56.098681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64f88e-b84e-4b89-a30d-6c904633c6cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:56.099760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64f88e-b84e-4b89-a30d-6c904633c6cc service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:56.100289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64f88e-b84e-4b89-a30d-6c904633c6cc service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:56.100813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64f88e-b84e-4b89-a30d-6c904633c6cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:56.113547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64f88e-b84e-4b89-a30d-6c904633c6cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:56.115656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64f88e-b84e-4b89-a30d-6c904633c6cc service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:56.156495 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1482/2923] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:55 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:56.592545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94603243-2704-4432-a5a0-3bd726c461bf None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:56.698824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94603243-2704-4432-a5a0-3bd726c461bf None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:56.716490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:56.717489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:56.718244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:56.719330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] PATH_INFO: `/v3/users/a98d5becb92443df808c54329bf8f7e9` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:56.721665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a98d5becb92443df808c54329bf8f7e9)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:56.724437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:56.879504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a98d5becb92443df808c54329bf8f7e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:56.907060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a98d5becb92443df808c54329bf8f7e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:56.909580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a98d5becb92443df808c54329bf8f7e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:56.968359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a98d5becb92443df808c54329bf8f7e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:56.984437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a98d5becb92443df808c54329bf8f7e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:56.999703 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1442/2924] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:54:56 2022] DELETE /identity/v3/users/a98d5becb92443df808c54329bf8f7e9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:54:57.001634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc5fdb-afbb-4836-a3ab-9baf8f2b9f00 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:57.002137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc5fdb-afbb-4836-a3ab-9baf8f2b9f00 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:57.002293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc5fdb-afbb-4836-a3ab-9baf8f2b9f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:57.040750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-31cc5fdb-afbb-4836-a3ab-9baf8f2b9f00 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:54:57.042296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-31cc5fdb-afbb-4836-a3ab-9baf8f2b9f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:57.151288 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1483/2925] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:54:57.157738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:57.159444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:57.225298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:57.247689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a1fda-ff7d-4632-9e95-656f2836e86f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:57.251503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e668-1577-47f9-843f-6206eed94d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:57.252126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e668-1577-47f9-843f-6206eed94d93 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:57.252413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e668-1577-47f9-843f-6206eed94d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:57.252694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e668-1577-47f9-843f-6206eed94d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:57.263580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137e668-1577-47f9-843f-6206eed94d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:57.265234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137e668-1577-47f9-843f-6206eed94d93 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:57.281616 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1443/2926] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:54:59.655590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc5fdb-afbb-4836-a3ab-9baf8f2b9f00 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:54:59.658305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-31cc5fdb-afbb-4836-a3ab-9baf8f2b9f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:54:59.683054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc5fdb-afbb-4836-a3ab-9baf8f2b9f00 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:54:59.701240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:54:59.703197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:54:59.703676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:54:59.706205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] PATH_INFO: `/v3/projects/d7a1d0f2ac8e42c9be1fffd847099ac1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:54:59.712433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7a1d0f2ac8e42c9be1fffd847099ac1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:54:59.714841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:54:59.744570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:54:59.885881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7a1d0f2ac8e42c9be1fffd847099ac1', 'request_id': 'req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:59.916296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7a1d0f2ac8e42c9be1fffd847099ac1', 'request_id': 'req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:54:59.949289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7a1d0f2ac8e42c9be1fffd847099ac1', 'request_id': 'req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:00.005282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] The token cache is being invalidate because project d7a1d0f2ac8e42c9be1fffd847099ac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:55:00.016359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:00.020520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:00.024282 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1484/2927] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:54:59 2022] DELETE /identity/v3/projects/d7a1d0f2ac8e42c9be1fffd847099ac1 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:55:00.028328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e668-1577-47f9-843f-6206eed94d93 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:00.046053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e668-1577-47f9-843f-6206eed94d93 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:00.216975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be438d-e3d5-4579-b2e7-48bbce501d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:00.217797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be438d-e3d5-4579-b2e7-48bbce501d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:00.218406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be438d-e3d5-4579-b2e7-48bbce501d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:00.218735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be438d-e3d5-4579-b2e7-48bbce501d00 admin admin] PATH_INFO: `/v3/users/4c0cb36c93bc4e99a78adbf7f819a5bb` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:00.220362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be438d-e3d5-4579-b2e7-48bbce501d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c0cb36c93bc4e99a78adbf7f819a5bb)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:00.222108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be438d-e3d5-4579-b2e7-48bbce501d00 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:00.336374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-40be438d-e3d5-4579-b2e7-48bbce501d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c0cb36c93bc4e99a78adbf7f819a5bb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:00.387669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-40be438d-e3d5-4579-b2e7-48bbce501d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c0cb36c93bc4e99a78adbf7f819a5bb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:00.387669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-40be438d-e3d5-4579-b2e7-48bbce501d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c0cb36c93bc4e99a78adbf7f819a5bb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:00.413061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-40be438d-e3d5-4579-b2e7-48bbce501d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c0cb36c93bc4e99a78adbf7f819a5bb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:00.425516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-40be438d-e3d5-4579-b2e7-48bbce501d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c0cb36c93bc4e99a78adbf7f819a5bb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:00.436208 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1444/2928] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:55:00 2022] DELETE /identity/v3/users/4c0cb36c93bc4e99a78adbf7f819a5bb => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:55:02.873818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:02.875998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:02.892322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551fb6e-9536-4a1c-a9cf-0f8c7c8bc3fe admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:03.001368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4344b3c-35c3-4968-8629-309ed460253d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:03.002672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4344b3c-35c3-4968-8629-309ed460253d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:03.003207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4344b3c-35c3-4968-8629-309ed460253d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:03.003745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4344b3c-35c3-4968-8629-309ed460253d admin admin] PATH_INFO: `/v3/projects/2f08394b65004113b7215f2c9d853bca` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:03.005334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4344b3c-35c3-4968-8629-309ed460253d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f08394b65004113b7215f2c9d853bca)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:03.008297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4344b3c-35c3-4968-8629-309ed460253d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:03.084563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e4344b3c-35c3-4968-8629-309ed460253d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f08394b65004113b7215f2c9d853bca', 'request_id': 'req-e4344b3c-35c3-4968-8629-309ed460253d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:03.188966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e4344b3c-35c3-4968-8629-309ed460253d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f08394b65004113b7215f2c9d853bca', 'request_id': 'req-e4344b3c-35c3-4968-8629-309ed460253d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:03.188966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e4344b3c-35c3-4968-8629-309ed460253d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f08394b65004113b7215f2c9d853bca', 'request_id': 'req-e4344b3c-35c3-4968-8629-309ed460253d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:03.188966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e4344b3c-35c3-4968-8629-309ed460253d admin admin] The token cache is being invalidate because project 2f08394b65004113b7215f2c9d853bca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:55:03.188966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e4344b3c-35c3-4968-8629-309ed460253d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:03.189334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e4344b3c-35c3-4968-8629-309ed460253d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:03.189334 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1485/2929] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:55:02 2022] DELETE /identity/v3/projects/2f08394b65004113b7215f2c9d853bca => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:55:03.189493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eed8f5-893f-4d11-a986-305990d7b9d9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:03.189493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eed8f5-893f-4d11-a986-305990d7b9d9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:03.189493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eed8f5-893f-4d11-a986-305990d7b9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:03.230858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-88eed8f5-893f-4d11-a986-305990d7b9d9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:03.232207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-88eed8f5-893f-4d11-a986-305990d7b9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:03.419913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:03.443584 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1445/2930] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:03.451073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6fd7f-a22f-4072-a410-14f4586a767f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:03.451369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6fd7f-a22f-4072-a410-14f4586a767f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:03.451867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6fd7f-a22f-4072-a410-14f4586a767f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:03.501252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-44f6fd7f-a22f-4072-a410-14f4586a767f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:03.501252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-44f6fd7f-a22f-4072-a410-14f4586a767f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:03.509036 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1486/2931] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:03.514329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eed8f5-893f-4d11-a986-305990d7b9d9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:03.526804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eed8f5-893f-4d11-a986-305990d7b9d9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:03.531601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4079c-fa96-471e-b06b-54b0b180e979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:03.532682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4079c-fa96-471e-b06b-54b0b180e979 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:03.533094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4079c-fa96-471e-b06b-54b0b180e979 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:03.533443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4079c-fa96-471e-b06b-54b0b180e979 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:03.534183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4079c-fa96-471e-b06b-54b0b180e979 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:55:03.534722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4079c-fa96-471e-b06b-54b0b180e979 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:03.536968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4079c-fa96-471e-b06b-54b0b180e979 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:03.555035 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1446/2932] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:55:03.556255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6fd7f-a22f-4072-a410-14f4586a767f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:03.567536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6fd7f-a22f-4072-a410-14f4586a767f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:03.576644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9e7dc-bdd1-416e-a27a-4f9278b1e48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:03.577595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e7dc-bdd1-416e-a27a-4f9278b1e48b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:03.577891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e7dc-bdd1-416e-a27a-4f9278b1e48b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:03.579066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e7dc-bdd1-416e-a27a-4f9278b1e48b admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:03.579592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9e7dc-bdd1-416e-a27a-4f9278b1e48b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:03.584218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9e7dc-bdd1-416e-a27a-4f9278b1e48b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:03.660574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:03.689513 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1487/2933] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:55:03 2022] POST /identity/v3/projects => generated 407 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:55:03.689732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4079c-fa96-471e-b06b-54b0b180e979 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:03.709586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4079c-fa96-471e-b06b-54b0b180e979 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:03.868746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3f96d-e94d-429d-8e37-63097a9a7ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:03.870312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3f96d-e94d-429d-8e37-63097a9a7ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:03.871194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3f96d-e94d-429d-8e37-63097a9a7ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:03.871793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3f96d-e94d-429d-8e37-63097a9a7ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:03.883358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3f96d-e94d-429d-8e37-63097a9a7ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:03.886059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3f96d-e94d-429d-8e37-63097a9a7ec7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:03.920846 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1447/2934] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:55:03 2022] POST /identity/v3/users => generated 304 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:55:03.922131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9e7dc-bdd1-416e-a27a-4f9278b1e48b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:03.947376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9e7dc-bdd1-416e-a27a-4f9278b1e48b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:03.965475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02428b26-7a61-4672-9722-66ae2afb287a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:03.966504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02428b26-7a61-4672-9722-66ae2afb287a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:03.967324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02428b26-7a61-4672-9722-66ae2afb287a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:03.968269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02428b26-7a61-4672-9722-66ae2afb287a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:03.971292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02428b26-7a61-4672-9722-66ae2afb287a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:03.996392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02428b26-7a61-4672-9722-66ae2afb287a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:04.023096 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1488/2935] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:55:04.027310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3f96d-e94d-429d-8e37-63097a9a7ec7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:04.062549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3f96d-e94d-429d-8e37-63097a9a7ec7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:04.084670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37724ec5-1e57-4e5c-802e-e03c587cdf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:04.087328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37724ec5-1e57-4e5c-802e-e03c587cdf13 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:04.089583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37724ec5-1e57-4e5c-802e-e03c587cdf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:04.104103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37724ec5-1e57-4e5c-802e-e03c587cdf13 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:04.108065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37724ec5-1e57-4e5c-802e-e03c587cdf13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:04.112999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37724ec5-1e57-4e5c-802e-e03c587cdf13 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:04.134764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02428b26-7a61-4672-9722-66ae2afb287a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:04.149673 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1448/2936] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:55:04.155802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02428b26-7a61-4672-9722-66ae2afb287a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:04.161855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63566469-b150-4436-bd09-eef2ad4b9ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:04.162871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63566469-b150-4436-bd09-eef2ad4b9ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:04.163176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63566469-b150-4436-bd09-eef2ad4b9ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:04.163520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63566469-b150-4436-bd09-eef2ad4b9ea6 admin admin] PATH_INFO: `/v3/projects/298ecaf75fb648e4b2ead38bd194df07/users/73b170e4f1ce4a31a48c474175cfd634/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:04.186339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63566469-b150-4436-bd09-eef2ad4b9ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=298ecaf75fb648e4b2ead38bd194df07, user_id=73b170e4f1ce4a31a48c474175cfd634, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:04.190344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63566469-b150-4436-bd09-eef2ad4b9ea6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:04.224941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbafbc7-07ab-4231-b5db-53397bcdcbc0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:04.225890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbafbc7-07ab-4231-b5db-53397bcdcbc0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:04.226378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbafbc7-07ab-4231-b5db-53397bcdcbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:04.229915 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1489/2937] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:55:04 2022] PUT /identity/v3/projects/298ecaf75fb648e4b2ead38bd194df07/users/73b170e4f1ce4a31a48c474175cfd634/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:55:04.275893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ccbafbc7-07ab-4231-b5db-53397bcdcbc0 None None] MFA Rules not processed for user `73b170e4f1ce4a31a48c474175cfd634`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:04.277445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ccbafbc7-07ab-4231-b5db-53397bcdcbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:04.402334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63566469-b150-4436-bd09-eef2ad4b9ea6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:04.404586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-63566469-b150-4436-bd09-eef2ad4b9ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:04.441898 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1449/2938] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:04 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:04.444012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2a0e4-b657-4f6e-8a2f-a5b95871b55c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:04.444320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2a0e4-b657-4f6e-8a2f-a5b95871b55c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:04.444590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2a0e4-b657-4f6e-8a2f-a5b95871b55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:04.481956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f9a2a0e4-b657-4f6e-8a2f-a5b95871b55c None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:04.483532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f9a2a0e4-b657-4f6e-8a2f-a5b95871b55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:04.522533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63566469-b150-4436-bd09-eef2ad4b9ea6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:04.528732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931d30b-d89c-4468-9347-5c32e7420b7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:04.529659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931d30b-d89c-4468-9347-5c32e7420b7d service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:04.530031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931d30b-d89c-4468-9347-5c32e7420b7d service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:04.530392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931d30b-d89c-4468-9347-5c32e7420b7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:04.533118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f931d30b-d89c-4468-9347-5c32e7420b7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:04.547323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931d30b-d89c-4468-9347-5c32e7420b7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:04.549275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931d30b-d89c-4468-9347-5c32e7420b7d service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:04.552785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:04.575215 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1490/2939] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:55:04.579429 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1450/2940] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:55:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:04.580471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2a0e4-b657-4f6e-8a2f-a5b95871b55c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:04.582118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f9a2a0e4-b657-4f6e-8a2f-a5b95871b55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:04.590790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931d30b-d89c-4468-9347-5c32e7420b7d service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:04.695565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931d30b-d89c-4468-9347-5c32e7420b7d service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:04.701861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1c72e-adb2-48fb-807f-8c6c9646cffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:04.702496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1c72e-adb2-48fb-807f-8c6c9646cffe admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:04.702780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1c72e-adb2-48fb-807f-8c6c9646cffe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:04.703054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1c72e-adb2-48fb-807f-8c6c9646cffe admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:04.703745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1c72e-adb2-48fb-807f-8c6c9646cffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:04.706751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1c72e-adb2-48fb-807f-8c6c9646cffe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:04.730265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2a0e4-b657-4f6e-8a2f-a5b95871b55c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:04.743452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:04.744785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:04.745114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:04.745393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] PATH_INFO: `/v3/users/18b23f4ce1d44682b0c187e8aac1a7df` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:04.750666 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1491/2941] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:55:04 2022] POST /identity/v3/projects => generated 407 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:55:04.751807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1c72e-adb2-48fb-807f-8c6c9646cffe admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:04.751958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] RBAC: Authorizing `identity:delete_user(user_id=18b23f4ce1d44682b0c187e8aac1a7df)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:04.754016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-40f1c72e-adb2-48fb-807f-8c6c9646cffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:04.756627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:04.823439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:04.858435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18b23f4ce1d44682b0c187e8aac1a7df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:04.877011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18b23f4ce1d44682b0c187e8aac1a7df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:04.877992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18b23f4ce1d44682b0c187e8aac1a7df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:04.888347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18b23f4ce1d44682b0c187e8aac1a7df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:04.891366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1c72e-adb2-48fb-807f-8c6c9646cffe admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:04.902497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18b23f4ce1d44682b0c187e8aac1a7df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:04.908472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69de19c-e578-48c6-be3c-bd2d6845d76b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:04.909276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69de19c-e578-48c6-be3c-bd2d6845d76b service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:04.909648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69de19c-e578-48c6-be3c-bd2d6845d76b service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:04.909982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69de19c-e578-48c6-be3c-bd2d6845d76b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:04.915233 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1451/2942] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:55:04 2022] DELETE /identity/v3/users/18b23f4ce1d44682b0c187e8aac1a7df => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:55:04.915233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:04.915233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:04.915670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d69de19c-e578-48c6-be3c-bd2d6845d76b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:05.007103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453272a-b9cb-427e-ab15-2f504a9b070c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:05.013232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa330fd0-1170-466c-9fe0-f3720f9a0d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:05.013920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa330fd0-1170-466c-9fe0-f3720f9a0d8e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:05.014621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa330fd0-1170-466c-9fe0-f3720f9a0d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:05.015011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa330fd0-1170-466c-9fe0-f3720f9a0d8e admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:05.015607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa330fd0-1170-466c-9fe0-f3720f9a0d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:05.018003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa330fd0-1170-466c-9fe0-f3720f9a0d8e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:05.049755 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1452/2943] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:55:04 2022] POST /identity/v3/users => generated 310 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:55:05.058912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69de19c-e578-48c6-be3c-bd2d6845d76b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:05.059820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa330fd0-1170-466c-9fe0-f3720f9a0d8e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:05.063983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69de19c-e578-48c6-be3c-bd2d6845d76b service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:05.078466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa330fd0-1170-466c-9fe0-f3720f9a0d8e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:05.084836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5969007-4f82-4ab3-9741-c4db732298d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:05.085604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5969007-4f82-4ab3-9741-c4db732298d4 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:05.086409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5969007-4f82-4ab3-9741-c4db732298d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:05.086779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5969007-4f82-4ab3-9741-c4db732298d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:05.087621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5969007-4f82-4ab3-9741-c4db732298d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:05.089644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5969007-4f82-4ab3-9741-c4db732298d4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:05.091103 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1492/2944] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:55:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:55:05.098442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:05.119919 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1453/2945] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:55:05.123957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69de19c-e578-48c6-be3c-bd2d6845d76b service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:05.247545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69de19c-e578-48c6-be3c-bd2d6845d76b service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:05.253855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf77679-9b63-4102-988f-d95e6e5b4089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:05.255554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf77679-9b63-4102-988f-d95e6e5b4089 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:05.255857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf77679-9b63-4102-988f-d95e6e5b4089 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:05.256142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf77679-9b63-4102-988f-d95e6e5b4089 admin admin] PATH_INFO: `/v3/projects/57ad9c02eacc44d0babf538c1e7e0a5b/users/a1bff35191544c3d9dce397df9588963/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:05.294091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf77679-9b63-4102-988f-d95e6e5b4089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ad9c02eacc44d0babf538c1e7e0a5b, user_id=a1bff35191544c3d9dce397df9588963, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:05.296946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf77679-9b63-4102-988f-d95e6e5b4089 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:05.326386 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1493/2946] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:55:05 2022] PUT /identity/v3/projects/57ad9c02eacc44d0babf538c1e7e0a5b/users/a1bff35191544c3d9dce397df9588963/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:55:05.326859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5969007-4f82-4ab3-9741-c4db732298d4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:05.327360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf77679-9b63-4102-988f-d95e6e5b4089 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:05.329430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b5969007-4f82-4ab3-9741-c4db732298d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:05.455755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5969007-4f82-4ab3-9741-c4db732298d4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:05.459643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf77679-9b63-4102-988f-d95e6e5b4089 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:05.472417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273d514-1ba7-4e61-98b2-e52e3939d1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:05.474593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273d514-1ba7-4e61-98b2-e52e3939d1f9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:05.475064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273d514-1ba7-4e61-98b2-e52e3939d1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:05.475522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273d514-1ba7-4e61-98b2-e52e3939d1f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:05.476345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273d514-1ba7-4e61-98b2-e52e3939d1f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:05.478493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273d514-1ba7-4e61-98b2-e52e3939d1f9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:05.485215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:05.491172 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1494/2947] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:55:05.496439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273d514-1ba7-4e61-98b2-e52e3939d1f9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:05.509123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273d514-1ba7-4e61-98b2-e52e3939d1f9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:05.515334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50fb4f-8dcf-4652-93ba-bc30efe2a6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:05.516537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50fb4f-8dcf-4652-93ba-bc30efe2a6e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:05.517110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50fb4f-8dcf-4652-93ba-bc30efe2a6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:05.517634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50fb4f-8dcf-4652-93ba-bc30efe2a6e5 admin admin] PATH_INFO: `/v3/projects/57ad9c02eacc44d0babf538c1e7e0a5b/users/a1bff35191544c3d9dce397df9588963/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:05.519996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50fb4f-8dcf-4652-93ba-bc30efe2a6e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ad9c02eacc44d0babf538c1e7e0a5b, user_id=a1bff35191544c3d9dce397df9588963, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:05.525499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50fb4f-8dcf-4652-93ba-bc30efe2a6e5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:05.543295 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1495/2948] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:55:05 2022] PUT /identity/v3/projects/57ad9c02eacc44d0babf538c1e7e0a5b/users/a1bff35191544c3d9dce397df9588963/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:55:05.555535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75233a3b-7ef6-4962-8564-86e34d7945ba None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:05.555809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75233a3b-7ef6-4962-8564-86e34d7945ba None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:05.556159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75233a3b-7ef6-4962-8564-86e34d7945ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:05.579950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249c632-ca64-43ff-922a-c74e4f0265b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:05.580767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249c632-ca64-43ff-922a-c74e4f0265b5 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:05.581390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249c632-ca64-43ff-922a-c74e4f0265b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:05.585131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249c632-ca64-43ff-922a-c74e4f0265b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:05.586755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9249c632-ca64-43ff-922a-c74e4f0265b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:05.603906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-75233a3b-7ef6-4962-8564-86e34d7945ba None None] MFA Rules not processed for user `a1bff35191544c3d9dce397df9588963`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:05.604189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249c632-ca64-43ff-922a-c74e4f0265b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:05.605811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-75233a3b-7ef6-4962-8564-86e34d7945ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:05.606395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249c632-ca64-43ff-922a-c74e4f0265b5 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:05.616957 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1454/2949] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:55:05.641223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249c632-ca64-43ff-922a-c74e4f0265b5 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:05.643763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9249c632-ca64-43ff-922a-c74e4f0265b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:05.663835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249c632-ca64-43ff-922a-c74e4f0265b5 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:05.765951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:05.766630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:05.766907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:05.767190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1 admin admin] PATH_INFO: `/v3/projects/5e1442bbd96148d196c957d881a7449e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:05.767948 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1496/2950] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:05 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:05.769275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1442bbd96148d196c957d881a7449e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:05.772419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:05.777018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75233a3b-7ef6-4962-8564-86e34d7945ba None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:05.801374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75233a3b-7ef6-4962-8564-86e34d7945ba None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:05.812245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01167128-135d-4493-8380-de21a0d293ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:05.812761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01167128-135d-4493-8380-de21a0d293ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:05.813366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01167128-135d-4493-8380-de21a0d293ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:05.813901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01167128-135d-4493-8380-de21a0d293ec admin admin] PATH_INFO: `/v3/users/73b170e4f1ce4a31a48c474175cfd634` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:05.823853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01167128-135d-4493-8380-de21a0d293ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=73b170e4f1ce4a31a48c474175cfd634)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:05.826180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01167128-135d-4493-8380-de21a0d293ec admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:05.846137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1442bbd96148d196c957d881a7449e', 'request_id': 'req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:05.862561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1442bbd96148d196c957d881a7449e', 'request_id': 'req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:05.864374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:05.889484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1442bbd96148d196c957d881a7449e', 'request_id': 'req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:05.928448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1 admin admin] The token cache is being invalidate because project 5e1442bbd96148d196c957d881a7449e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:55:05.929323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:05.943878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-47e02f8b-8ec3-4a76-8a6e-653da4a994e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:05.946252 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1455/2951] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:55:05 2022] DELETE /identity/v3/projects/5e1442bbd96148d196c957d881a7449e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:55:05.953625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b1f2a-0519-4610-ab0e-454ab81e447e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:05.953944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b1f2a-0519-4610-ab0e-454ab81e447e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:05.954201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b1f2a-0519-4610-ab0e-454ab81e447e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:05.966015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-01167128-135d-4493-8380-de21a0d293ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73b170e4f1ce4a31a48c474175cfd634'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:06.003540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-01167128-135d-4493-8380-de21a0d293ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73b170e4f1ce4a31a48c474175cfd634'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:06.005712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e35b1f2a-0519-4610-ab0e-454ab81e447e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:06.006901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e35b1f2a-0519-4610-ab0e-454ab81e447e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:06.027105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-01167128-135d-4493-8380-de21a0d293ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73b170e4f1ce4a31a48c474175cfd634'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:06.028423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-01167128-135d-4493-8380-de21a0d293ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73b170e4f1ce4a31a48c474175cfd634'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:06.029350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-01167128-135d-4493-8380-de21a0d293ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73b170e4f1ce4a31a48c474175cfd634'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:06.067516 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1497/2952] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:55:05 2022] DELETE /identity/v3/users/73b170e4f1ce4a31a48c474175cfd634 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:55:06.070599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c8af1-a7f0-41e6-9c96-0c1e17167623 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:06.070904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c8af1-a7f0-41e6-9c96-0c1e17167623 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:06.071189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c8af1-a7f0-41e6-9c96-0c1e17167623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:06.147763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-463c8af1-a7f0-41e6-9c96-0c1e17167623 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:06.150737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-463c8af1-a7f0-41e6-9c96-0c1e17167623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:06.152650 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1456/2953] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:06.160378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12ebdc-c6b0-44e4-a103-208a1bb6a255 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:06.160973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12ebdc-c6b0-44e4-a103-208a1bb6a255 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:06.161533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12ebdc-c6b0-44e4-a103-208a1bb6a255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:06.207076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-af12ebdc-c6b0-44e4-a103-208a1bb6a255 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:06.211714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-af12ebdc-c6b0-44e4-a103-208a1bb6a255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:06.252700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:06.294796 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1498/2954] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:06.306648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c8af1-a7f0-41e6-9c96-0c1e17167623 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:06.308329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-463c8af1-a7f0-41e6-9c96-0c1e17167623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:06.349306 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1457/2955] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:06.353695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12ebdc-c6b0-44e4-a103-208a1bb6a255 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:06.371954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12ebdc-c6b0-44e4-a103-208a1bb6a255 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:06.388629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701de23-eea8-4929-8b45-310b4eab0a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:06.389290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701de23-eea8-4929-8b45-310b4eab0a97 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:06.389576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701de23-eea8-4929-8b45-310b4eab0a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:06.390544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701de23-eea8-4929-8b45-310b4eab0a97 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:06.390544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701de23-eea8-4929-8b45-310b4eab0a97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:55:06.392156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701de23-eea8-4929-8b45-310b4eab0a97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:06.393825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701de23-eea8-4929-8b45-310b4eab0a97 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:06.441347 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1458/2956] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:55:06.446063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701de23-eea8-4929-8b45-310b4eab0a97 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:06.460171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c8af1-a7f0-41e6-9c96-0c1e17167623 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:06.467969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701de23-eea8-4929-8b45-310b4eab0a97 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:06.472931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd23d61-4902-4981-9d14-87925d40d566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:06.473842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd23d61-4902-4981-9d14-87925d40d566 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:06.474358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd23d61-4902-4981-9d14-87925d40d566 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:06.475814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd23d61-4902-4981-9d14-87925d40d566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:06.491566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ab2b-9b5b-4a62-8bdb-3f9286882765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:06.492525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ab2b-9b5b-4a62-8bdb-3f9286882765 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:06.493004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ab2b-9b5b-4a62-8bdb-3f9286882765 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:06.493474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ab2b-9b5b-4a62-8bdb-3f9286882765 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:06.494301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578ab2b-9b5b-4a62-8bdb-3f9286882765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:06.498006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578ab2b-9b5b-4a62-8bdb-3f9286882765 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:06.501657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd23d61-4902-4981-9d14-87925d40d566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:06.503311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd23d61-4902-4981-9d14-87925d40d566 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:06.528308 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1499/2957] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:55:06.561654 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1459/2958] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:55:06 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:55:06.566413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd23d61-4902-4981-9d14-87925d40d566 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:06.579614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd23d61-4902-4981-9d14-87925d40d566 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:06.700339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0369160-4de9-4c37-9209-7324c226f034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:06.703546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0369160-4de9-4c37-9209-7324c226f034 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:06.704757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0369160-4de9-4c37-9209-7324c226f034 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:06.706652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0369160-4de9-4c37-9209-7324c226f034 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:06.707340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0369160-4de9-4c37-9209-7324c226f034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:06.716865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0369160-4de9-4c37-9209-7324c226f034 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:06.795098 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1500/2959] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:55:06 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:55:06.799224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ab2b-9b5b-4a62-8bdb-3f9286882765 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:06.813831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ab2b-9b5b-4a62-8bdb-3f9286882765 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:06.830858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc9c81-5256-495b-be20-65c6c70c7314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:06.834974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc9c81-5256-495b-be20-65c6c70c7314 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:06.836699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc9c81-5256-495b-be20-65c6c70c7314 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:06.837837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc9c81-5256-495b-be20-65c6c70c7314 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:06.839811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc9c81-5256-495b-be20-65c6c70c7314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:06.844335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc9c81-5256-495b-be20-65c6c70c7314 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:06.868591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0369160-4de9-4c37-9209-7324c226f034 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:06.899841 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1460/2960] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:55:06.902319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0369160-4de9-4c37-9209-7324c226f034 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:06.910400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d333d5-d1be-4586-ba7d-7d5e168c69d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:06.911212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d333d5-d1be-4586-ba7d-7d5e168c69d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:06.912220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d333d5-d1be-4586-ba7d-7d5e168c69d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:06.912631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d333d5-d1be-4586-ba7d-7d5e168c69d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:06.913632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d333d5-d1be-4586-ba7d-7d5e168c69d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:06.916017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d333d5-d1be-4586-ba7d-7d5e168c69d2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:06.936175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc9c81-5256-495b-be20-65c6c70c7314 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:06.942367 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1501/2961] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:55:06.949481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc9c81-5256-495b-be20-65c6c70c7314 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:06.955975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327073eb-997b-44a1-9e39-3a9de117d743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:06.957445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327073eb-997b-44a1-9e39-3a9de117d743 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:06.959238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327073eb-997b-44a1-9e39-3a9de117d743 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:06.959812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327073eb-997b-44a1-9e39-3a9de117d743 admin admin] PATH_INFO: `/v3/projects/a7715a4d463a44168bc78b9fa7a7718f/users/db307623f73f4532a96342f551ba011c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:06.997674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327073eb-997b-44a1-9e39-3a9de117d743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7715a4d463a44168bc78b9fa7a7718f, user_id=db307623f73f4532a96342f551ba011c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:07.001145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327073eb-997b-44a1-9e39-3a9de117d743 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:07.045247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ee0cc-3fd8-48b1-931c-9d22628a1a20 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:07.045664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ee0cc-3fd8-48b1-931c-9d22628a1a20 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:07.046164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ee0cc-3fd8-48b1-931c-9d22628a1a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:07.049128 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1461/2962] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:55:06 2022] PUT /identity/v3/projects/a7715a4d463a44168bc78b9fa7a7718f/users/db307623f73f4532a96342f551ba011c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:55:07.097221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8e6ee0cc-3fd8-48b1-931c-9d22628a1a20 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:07.101392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8e6ee0cc-3fd8-48b1-931c-9d22628a1a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:07.264751 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1502/2963] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:07.326065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327073eb-997b-44a1-9e39-3a9de117d743 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:07.330290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-327073eb-997b-44a1-9e39-3a9de117d743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:07.367935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ee0cc-3fd8-48b1-931c-9d22628a1a20 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:07.370271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8e6ee0cc-3fd8-48b1-931c-9d22628a1a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:07.390649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:07.408329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ee0cc-3fd8-48b1-931c-9d22628a1a20 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:07.422008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:07.423830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:07.424339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:07.430921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] PATH_INFO: `/v3/projects/298ecaf75fb648e4b2ead38bd194df07` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:07.435984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=298ecaf75fb648e4b2ead38bd194df07)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:07.439765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:07.443484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:07.476612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327073eb-997b-44a1-9e39-3a9de117d743 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:07.480805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91252ec8-0797-41d7-944e-f4832167f405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:07.481599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91252ec8-0797-41d7-944e-f4832167f405 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:07.482143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91252ec8-0797-41d7-944e-f4832167f405 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:07.482563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91252ec8-0797-41d7-944e-f4832167f405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:07.498159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91252ec8-0797-41d7-944e-f4832167f405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:07.500880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91252ec8-0797-41d7-944e-f4832167f405 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:07.535782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '298ecaf75fb648e4b2ead38bd194df07', 'request_id': 'req-d8ccad86-70aa-4ec3-b483-3adbec197c07'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:07.548266 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1462/2964] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:55:07.552473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '298ecaf75fb648e4b2ead38bd194df07', 'request_id': 'req-d8ccad86-70aa-4ec3-b483-3adbec197c07'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:07.568553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '298ecaf75fb648e4b2ead38bd194df07', 'request_id': 'req-d8ccad86-70aa-4ec3-b483-3adbec197c07'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:07.620258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] The token cache is being invalidate because project 298ecaf75fb648e4b2ead38bd194df07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:55:07.621029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:07.621956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:07.625527 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1503/2965] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:55:07 2022] DELETE /identity/v3/projects/298ecaf75fb648e4b2ead38bd194df07 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:55:07.629237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91252ec8-0797-41d7-944e-f4832167f405 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:07.641677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91252ec8-0797-41d7-944e-f4832167f405 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:07.720815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:07.721697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:07.722072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:07.722411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] PATH_INFO: `/v3/users/a1bff35191544c3d9dce397df9588963` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:07.723660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1bff35191544c3d9dce397df9588963)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:07.726005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:07.863506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1bff35191544c3d9dce397df9588963'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:07.908145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1bff35191544c3d9dce397df9588963'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:07.909941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1bff35191544c3d9dce397df9588963'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:07.925575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1bff35191544c3d9dce397df9588963'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:07.944153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1bff35191544c3d9dce397df9588963'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:07.964158 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1463/2966] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:55:07 2022] DELETE /identity/v3/users/a1bff35191544c3d9dce397df9588963 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:55:09.064671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:09.066543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:09.098389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccad86-70aa-4ec3-b483-3adbec197c07 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:09.311858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e14741-1a6a-441c-a098-4adc6c8d0d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:09.339114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e14741-1a6a-441c-a098-4adc6c8d0d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:09.339659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e14741-1a6a-441c-a098-4adc6c8d0d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:09.344969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e14741-1a6a-441c-a098-4adc6c8d0d5d admin admin] PATH_INFO: `/v3/projects/57ad9c02eacc44d0babf538c1e7e0a5b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:09.348464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e14741-1a6a-441c-a098-4adc6c8d0d5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=57ad9c02eacc44d0babf538c1e7e0a5b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:09.353986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e14741-1a6a-441c-a098-4adc6c8d0d5d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:09.514152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93e14741-1a6a-441c-a098-4adc6c8d0d5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57ad9c02eacc44d0babf538c1e7e0a5b', 'request_id': 'req-93e14741-1a6a-441c-a098-4adc6c8d0d5d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:09.534075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:09.537632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:09.538180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93e14741-1a6a-441c-a098-4adc6c8d0d5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57ad9c02eacc44d0babf538c1e7e0a5b', 'request_id': 'req-93e14741-1a6a-441c-a098-4adc6c8d0d5d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:09.561531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93e14741-1a6a-441c-a098-4adc6c8d0d5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57ad9c02eacc44d0babf538c1e7e0a5b', 'request_id': 'req-93e14741-1a6a-441c-a098-4adc6c8d0d5d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:09.606357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93e14741-1a6a-441c-a098-4adc6c8d0d5d admin admin] The token cache is being invalidate because project 57ad9c02eacc44d0babf538c1e7e0a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:55:09.606661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93e14741-1a6a-441c-a098-4adc6c8d0d5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:09.610938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93e14741-1a6a-441c-a098-4adc6c8d0d5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:55:09.617025 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1504/2967] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:55:09 2022] DELETE /identity/v3/projects/57ad9c02eacc44d0babf538c1e7e0a5b => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:55:09.625979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ece63-8e05-42ce-a798-07e6e0e346cd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:09.626269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ece63-8e05-42ce-a798-07e6e0e346cd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:09.626553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ece63-8e05-42ce-a798-07e6e0e346cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:09.664118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:09.700390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ac6ece63-8e05-42ce-a798-07e6e0e346cd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:09.701695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ac6ece63-8e05-42ce-a798-07e6e0e346cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:09.706041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0167610-5927-4d73-8de4-cc21728efe15 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:09.755003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:09.818361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5c958-8e70-46ae-9986-9f4df7dfd39a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:09.819280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5c958-8e70-46ae-9986-9f4df7dfd39a service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:09.819749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5c958-8e70-46ae-9986-9f4df7dfd39a service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:09.820173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5c958-8e70-46ae-9986-9f4df7dfd39a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:09.823156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cff5c958-8e70-46ae-9986-9f4df7dfd39a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:09.852826 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1505/2968] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:09.854778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34104a7a-fb52-4c11-8a5f-e3475c643455 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:09.855298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34104a7a-fb52-4c11-8a5f-e3475c643455 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:09.855820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34104a7a-fb52-4c11-8a5f-e3475c643455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:09.881020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-34104a7a-fb52-4c11-8a5f-e3475c643455 None None] MFA Rules not processed for user `27be6827d30e48acac9a59786021f7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:09.882542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-34104a7a-fb52-4c11-8a5f-e3475c643455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:09.989318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5c958-8e70-46ae-9986-9f4df7dfd39a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:09.991130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5c958-8e70-46ae-9986-9f4df7dfd39a service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:10.026745 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1506/2969] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:09 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:10.028832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3a498-9951-4558-87a2-76772400f3cd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:10.028832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3a498-9951-4558-87a2-76772400f3cd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:10.028832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3a498-9951-4558-87a2-76772400f3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:10.029472 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1464/2970] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:55:10.080023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-edc3a498-9951-4558-87a2-76772400f3cd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:10.082256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-edc3a498-9951-4558-87a2-76772400f3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:10.102168 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1507/2971] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:10.106340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5c958-8e70-46ae-9986-9f4df7dfd39a service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:10.122976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5c958-8e70-46ae-9986-9f4df7dfd39a service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:10.142731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf39164-6dee-4bf5-be34-9a6389316fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:10.143702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf39164-6dee-4bf5-be34-9a6389316fce admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:10.143812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf39164-6dee-4bf5-be34-9a6389316fce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:10.144082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf39164-6dee-4bf5-be34-9a6389316fce admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:10.144857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf39164-6dee-4bf5-be34-9a6389316fce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:55:10.146420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf39164-6dee-4bf5-be34-9a6389316fce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:10.148159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf39164-6dee-4bf5-be34-9a6389316fce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:10.185800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3a498-9951-4558-87a2-76772400f3cd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:10.202971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3a498-9951-4558-87a2-76772400f3cd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:10.203673 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1465/2972] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:55:10.215898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9615b6c-489f-4f73-914e-39759dd8dc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:10.216535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9615b6c-489f-4f73-914e-39759dd8dc2c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:10.216822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9615b6c-489f-4f73-914e-39759dd8dc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:10.217129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9615b6c-489f-4f73-914e-39759dd8dc2c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:10.217770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9615b6c-489f-4f73-914e-39759dd8dc2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:10.225145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9615b6c-489f-4f73-914e-39759dd8dc2c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:10.286053 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1508/2973] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:55:10 2022] POST /identity/v3/projects => generated 397 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:55:10.291305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf39164-6dee-4bf5-be34-9a6389316fce admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:10.307886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf39164-6dee-4bf5-be34-9a6389316fce admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:10.454860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b16c-43e4-434a-9b66-cf4e72dc8185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:10.455487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b16c-43e4-434a-9b66-cf4e72dc8185 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:10.455888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b16c-43e4-434a-9b66-cf4e72dc8185 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:10.456076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b16c-43e4-434a-9b66-cf4e72dc8185 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:10.456678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1b16c-43e4-434a-9b66-cf4e72dc8185 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:10.458809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1b16c-43e4-434a-9b66-cf4e72dc8185 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:10.506636 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1466/2974] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:55:10 2022] POST /identity/v3/users => generated 299 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:55:10.510892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9615b6c-489f-4f73-914e-39759dd8dc2c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:10.523832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9615b6c-489f-4f73-914e-39759dd8dc2c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:10.553602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c145082-d087-4d0c-8090-d35031ad42ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:10.554230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c145082-d087-4d0c-8090-d35031ad42ac admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:10.554519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c145082-d087-4d0c-8090-d35031ad42ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:10.554847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c145082-d087-4d0c-8090-d35031ad42ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:10.555498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c145082-d087-4d0c-8090-d35031ad42ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:10.557256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c145082-d087-4d0c-8090-d35031ad42ac admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:10.580328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b16c-43e4-434a-9b66-cf4e72dc8185 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:10.581138 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1509/2975] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:55:10.591348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b16c-43e4-434a-9b66-cf4e72dc8185 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:10.596804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a891a-62a9-416a-a9d5-70cef6ad8a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:10.597525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a891a-62a9-416a-a9d5-70cef6ad8a6a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:10.597811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a891a-62a9-416a-a9d5-70cef6ad8a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:10.598090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a891a-62a9-416a-a9d5-70cef6ad8a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:10.598747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a891a-62a9-416a-a9d5-70cef6ad8a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:10.601976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a891a-62a9-416a-a9d5-70cef6ad8a6a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:10.620234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c145082-d087-4d0c-8090-d35031ad42ac admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:10.628624 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1467/2976] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:55:10.632725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c145082-d087-4d0c-8090-d35031ad42ac admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:10.637734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d554b-5468-434b-b85c-890018a99f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:10.638402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d554b-5468-434b-b85c-890018a99f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:10.639441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d554b-5468-434b-b85c-890018a99f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:10.639441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d554b-5468-434b-b85c-890018a99f39 admin admin] PATH_INFO: `/v3/projects/a9897f3d9893426f9fba83c8db250799/users/f4d78b7738644ff38eb239daec52d641/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:10.667278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d554b-5468-434b-b85c-890018a99f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9897f3d9893426f9fba83c8db250799, user_id=f4d78b7738644ff38eb239daec52d641, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:10.671792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d554b-5468-434b-b85c-890018a99f39 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:10.706369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c3602-ca78-404e-a687-6ee8a1b24d44 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:10.706660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c3602-ca78-404e-a687-6ee8a1b24d44 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:10.706942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c3602-ca78-404e-a687-6ee8a1b24d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:10.710427 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1510/2977] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:55:10 2022] PUT /identity/v3/projects/a9897f3d9893426f9fba83c8db250799/users/f4d78b7738644ff38eb239daec52d641/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:55:10.750728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-157c3602-ca78-404e-a687-6ee8a1b24d44 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:10.755820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-157c3602-ca78-404e-a687-6ee8a1b24d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:10.778601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:10.883583 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1468/2978] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:10.930348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d554b-5468-434b-b85c-890018a99f39 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:10.938019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-091d554b-5468-434b-b85c-890018a99f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:11.099480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d554b-5468-434b-b85c-890018a99f39 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:11.106140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b8cfc-6f37-4451-8111-d2c15e8c5ed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:11.107407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b8cfc-6f37-4451-8111-d2c15e8c5ed7 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:11.107705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b8cfc-6f37-4451-8111-d2c15e8c5ed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:11.109376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b8cfc-6f37-4451-8111-d2c15e8c5ed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:11.132508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b8cfc-6f37-4451-8111-d2c15e8c5ed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:11.134025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b8cfc-6f37-4451-8111-d2c15e8c5ed7 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:11.161528 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1511/2979] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:55:13.114909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c3602-ca78-404e-a687-6ee8a1b24d44 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:13.119347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-157c3602-ca78-404e-a687-6ee8a1b24d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:13.232974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c3602-ca78-404e-a687-6ee8a1b24d44 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:13.358698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5555b-b830-42fd-ad2b-a895eb3d5f1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:13.359432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5555b-b830-42fd-ad2b-a895eb3d5f1b service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:13.360480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5555b-b830-42fd-ad2b-a895eb3d5f1b service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:13.361379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5555b-b830-42fd-ad2b-a895eb3d5f1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:13.365683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7bc5555b-b830-42fd-ad2b-a895eb3d5f1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:13.435145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5555b-b830-42fd-ad2b-a895eb3d5f1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:13.438832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5555b-b830-42fd-ad2b-a895eb3d5f1b service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:13.476858 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1469/2980] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:55:17.699372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b8cfc-6f37-4451-8111-d2c15e8c5ed7 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:17.700946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0c0b8cfc-6f37-4451-8111-d2c15e8c5ed7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:17.713494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:17.908678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b8cfc-6f37-4451-8111-d2c15e8c5ed7 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:17.941719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91abd7-5bbe-4ad3-918e-db8026042af4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:17.942649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91abd7-5bbe-4ad3-918e-db8026042af4 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:17.943044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91abd7-5bbe-4ad3-918e-db8026042af4 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:17.943432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91abd7-5bbe-4ad3-918e-db8026042af4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:17.945601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3a91abd7-5bbe-4ad3-918e-db8026042af4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:18.101246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91abd7-5bbe-4ad3-918e-db8026042af4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:18.105152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91abd7-5bbe-4ad3-918e-db8026042af4 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:18.135817 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1512/2981] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:55:23.460804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=53 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:25.424119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a52042-3569-4541-92f9-be7e03fcc28f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:25.425258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a52042-3569-4541-92f9-be7e03fcc28f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:25.425777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a52042-3569-4541-92f9-be7e03fcc28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:25.496699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-96a52042-3569-4541-92f9-be7e03fcc28f None None] MFA Rules not processed for user `db307623f73f4532a96342f551ba011c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:25.499238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-96a52042-3569-4541-92f9-be7e03fcc28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:25.635646 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1470/2982] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:55:25.645901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a2be4-0c72-4515-b0bb-92a05921760e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:25.646528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a2be4-0c72-4515-b0bb-92a05921760e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:25.646880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a2be4-0c72-4515-b0bb-92a05921760e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:25.681039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d63a2be4-0c72-4515-b0bb-92a05921760e None None] MFA Rules not processed for user `db307623f73f4532a96342f551ba011c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:25.682598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d63a2be4-0c72-4515-b0bb-92a05921760e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:25.691561 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1513/2983] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:25.702930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a52042-3569-4541-92f9-be7e03fcc28f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:25.704821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-96a52042-3569-4541-92f9-be7e03fcc28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:25.762817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a52042-3569-4541-92f9-be7e03fcc28f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:25.779302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22039fd2-c99e-467a-bdb8-09c1c0938397 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:25.785941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22039fd2-c99e-467a-bdb8-09c1c0938397 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:25.786646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22039fd2-c99e-467a-bdb8-09c1c0938397 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:25.787157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22039fd2-c99e-467a-bdb8-09c1c0938397 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:25.799116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22039fd2-c99e-467a-bdb8-09c1c0938397 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:25.801021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22039fd2-c99e-467a-bdb8-09c1c0938397 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:25.815763 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1471/2984] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:55:26.747454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56ac7b-5f14-482b-8487-8c1b17fcc945 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:26.749366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56ac7b-5f14-482b-8487-8c1b17fcc945 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:26.749950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56ac7b-5f14-482b-8487-8c1b17fcc945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:26.780587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7e56ac7b-5f14-482b-8487-8c1b17fcc945 None None] MFA Rules not processed for user `778561ee5e474accb31816f357985f44`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:26.783473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7e56ac7b-5f14-482b-8487-8c1b17fcc945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:26.964155 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1514/2985] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:26 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:28.673058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3a69d-977f-402b-bd50-70c83f87d6e2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:28.673624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3a69d-977f-402b-bd50-70c83f87d6e2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:28.673624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3a69d-977f-402b-bd50-70c83f87d6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:28.741064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a3c3a69d-977f-402b-bd50-70c83f87d6e2 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:28.744446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a3c3a69d-977f-402b-bd50-70c83f87d6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:28.794130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4136f8-c1ed-43f6-b4f3-f90824fe8651 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:28.795153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4136f8-c1ed-43f6-b4f3-f90824fe8651 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:28.795688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4136f8-c1ed-43f6-b4f3-f90824fe8651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:28.852426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6e4136f8-c1ed-43f6-b4f3-f90824fe8651 None None] MFA Rules not processed for user `db307623f73f4532a96342f551ba011c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:28.857834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6e4136f8-c1ed-43f6-b4f3-f90824fe8651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:28.877657 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1515/2986] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:55:28.982724 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1472/2987] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:55:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:55:28.997024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4136f8-c1ed-43f6-b4f3-f90824fe8651 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:28.999071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6e4136f8-c1ed-43f6-b4f3-f90824fe8651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:29.164122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:29.265412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4136f8-c1ed-43f6-b4f3-f90824fe8651 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:29.281323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76957f88-c83d-44c3-852e-a4c5b58098ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:29.281323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76957f88-c83d-44c3-852e-a4c5b58098ad service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:29.281323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76957f88-c83d-44c3-852e-a4c5b58098ad service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:29.281323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76957f88-c83d-44c3-852e-a4c5b58098ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:29.302749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76957f88-c83d-44c3-852e-a4c5b58098ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:29.309615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76957f88-c83d-44c3-852e-a4c5b58098ad service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:29.341302 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1516/2988] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:55:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:55:30.308657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6c250-41fd-4dec-980d-414ef6426af2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:30.309759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6c250-41fd-4dec-980d-414ef6426af2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:30.310315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6c250-41fd-4dec-980d-414ef6426af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:30.398033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-efe6c250-41fd-4dec-980d-414ef6426af2 None None] MFA Rules not processed for user `f4d78b7738644ff38eb239daec52d641`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:30.399961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-efe6c250-41fd-4dec-980d-414ef6426af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:30.613244 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1473/2989] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:30 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:30.618227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622d889-0840-4098-91e1-f7bf95a7db8a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:30.618669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622d889-0840-4098-91e1-f7bf95a7db8a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:30.618799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622d889-0840-4098-91e1-f7bf95a7db8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:30.734165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f622d889-0840-4098-91e1-f7bf95a7db8a None None] MFA Rules not processed for user `f4d78b7738644ff38eb239daec52d641`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:30.735501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f622d889-0840-4098-91e1-f7bf95a7db8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:30.757930 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1517/2990] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:30 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:32.039849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6c250-41fd-4dec-980d-414ef6426af2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:32.063427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6c250-41fd-4dec-980d-414ef6426af2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:32.090060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e13091-f7c5-4b2d-9beb-ec1469c72b65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:32.091991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e13091-f7c5-4b2d-9beb-ec1469c72b65 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:32.094460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e13091-f7c5-4b2d-9beb-ec1469c72b65 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:32.094961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e13091-f7c5-4b2d-9beb-ec1469c72b65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:32.100378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e0e13091-f7c5-4b2d-9beb-ec1469c72b65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:32.171215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:32.327696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e13091-f7c5-4b2d-9beb-ec1469c72b65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:32.329567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e13091-f7c5-4b2d-9beb-ec1469c72b65 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:32.375554 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1474/2991] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:55:32.734222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622d889-0840-4098-91e1-f7bf95a7db8a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:32.772934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622d889-0840-4098-91e1-f7bf95a7db8a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:32.793983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9aaf5-acf7-4c67-b28a-f9eee6515991 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:32.798644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9aaf5-acf7-4c67-b28a-f9eee6515991 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:32.799409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9aaf5-acf7-4c67-b28a-f9eee6515991 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:32.800260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9aaf5-acf7-4c67-b28a-f9eee6515991 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:32.841159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9aaf5-acf7-4c67-b28a-f9eee6515991 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:32.848175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9aaf5-acf7-4c67-b28a-f9eee6515991 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:32.931116 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1518/2992] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:55:35.610774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e13091-f7c5-4b2d-9beb-ec1469c72b65 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:35.613599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e0e13091-f7c5-4b2d-9beb-ec1469c72b65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:35.635810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e13091-f7c5-4b2d-9beb-ec1469c72b65 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:35.640675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2fb4c-9748-40d3-b29f-eec7dfbaac11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:35.641649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2fb4c-9748-40d3-b29f-eec7dfbaac11 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:35.642289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2fb4c-9748-40d3-b29f-eec7dfbaac11 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:35.642737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2fb4c-9748-40d3-b29f-eec7dfbaac11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:35.650402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bfe2fb4c-9748-40d3-b29f-eec7dfbaac11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:35.662245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2fb4c-9748-40d3-b29f-eec7dfbaac11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:35.663970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2fb4c-9748-40d3-b29f-eec7dfbaac11 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:35.675180 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1475/2993] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:55:36.638545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9aaf5-acf7-4c67-b28a-f9eee6515991 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:36.640415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-73c9aaf5-acf7-4c67-b28a-f9eee6515991 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:36.701144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9aaf5-acf7-4c67-b28a-f9eee6515991 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:36.710718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c3d78-a561-450b-a0d6-b478aa136a1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:36.711641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c3d78-a561-450b-a0d6-b478aa136a1a service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:36.713072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c3d78-a561-450b-a0d6-b478aa136a1a service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:36.713580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c3d78-a561-450b-a0d6-b478aa136a1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:36.718970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-774c3d78-a561-450b-a0d6-b478aa136a1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:36.736223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c3d78-a561-450b-a0d6-b478aa136a1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:36.738581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c3d78-a561-450b-a0d6-b478aa136a1a service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:36.752563 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1519/2994] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:55:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:55:45.670620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=31 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:46.746283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=32 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:55:47.256189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650075c-e7d0-486f-804c-56baf6f31361 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:47.258631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650075c-e7d0-486f-804c-56baf6f31361 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:47.259918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650075c-e7d0-486f-804c-56baf6f31361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:47.292636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6650075c-e7d0-486f-804c-56baf6f31361 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:47.294845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6650075c-e7d0-486f-804c-56baf6f31361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:47.302919 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1476/2995] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:55:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:47.346308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c3d78-a561-450b-a0d6-b478aa136a1a service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:47.366168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c3d78-a561-450b-a0d6-b478aa136a1a service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:47.383987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9337b-71bd-49a4-9158-3e422c2b5cc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:47.385119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9337b-71bd-49a4-9158-3e422c2b5cc1 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:47.386284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9337b-71bd-49a4-9158-3e422c2b5cc1 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:47.386845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9337b-71bd-49a4-9158-3e422c2b5cc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:47.406451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9337b-71bd-49a4-9158-3e422c2b5cc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:47.410803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9337b-71bd-49a4-9158-3e422c2b5cc1 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:47.423451 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1520/2996] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:55:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:55:48.552836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461005f-eb2d-4351-91a0-fa7664710450 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:48.553406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461005f-eb2d-4351-91a0-fa7664710450 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:48.553499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461005f-eb2d-4351-91a0-fa7664710450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:48.581306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6461005f-eb2d-4351-91a0-fa7664710450 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:48.584284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6461005f-eb2d-4351-91a0-fa7664710450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:48.592647 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1477/2997] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:48.613928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9337b-71bd-49a4-9158-3e422c2b5cc1 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:48.626316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9337b-71bd-49a4-9158-3e422c2b5cc1 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:48.635744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7743f85-2144-471c-aada-28389e46d2ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:48.637892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7743f85-2144-471c-aada-28389e46d2ec service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:48.638706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7743f85-2144-471c-aada-28389e46d2ec service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:48.639579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7743f85-2144-471c-aada-28389e46d2ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:48.661842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7743f85-2144-471c-aada-28389e46d2ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:48.663868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7743f85-2144-471c-aada-28389e46d2ec service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:48.689656 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1521/2998] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:55:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:55:50.480468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54297635-2d7e-412e-b5a6-2cb6c6a57e28 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:50.481094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54297635-2d7e-412e-b5a6-2cb6c6a57e28 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:50.481094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54297635-2d7e-412e-b5a6-2cb6c6a57e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:50.515228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-54297635-2d7e-412e-b5a6-2cb6c6a57e28 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:50.516653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-54297635-2d7e-412e-b5a6-2cb6c6a57e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:50.530005 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1478/2999] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:55:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:50.544872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7743f85-2144-471c-aada-28389e46d2ec service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:50.559105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7743f85-2144-471c-aada-28389e46d2ec service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:50.566989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec2fd3-7e13-431c-8a5e-8ec1ba58edcb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:50.567920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec2fd3-7e13-431c-8a5e-8ec1ba58edcb service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:50.569118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec2fd3-7e13-431c-8a5e-8ec1ba58edcb service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:50.569682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec2fd3-7e13-431c-8a5e-8ec1ba58edcb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:50.585551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec2fd3-7e13-431c-8a5e-8ec1ba58edcb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:50.587531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec2fd3-7e13-431c-8a5e-8ec1ba58edcb service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:50.606255 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1522/3000] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:55:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:55:51.387659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba74b5-d805-48c0-a898-85f73c1b19f5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:51.388033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba74b5-d805-48c0-a898-85f73c1b19f5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:51.388388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba74b5-d805-48c0-a898-85f73c1b19f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:51.408638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9aba74b5-d805-48c0-a898-85f73c1b19f5 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:51.410466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9aba74b5-d805-48c0-a898-85f73c1b19f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:51.418408 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1479/3001] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:55:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:51.434010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec2fd3-7e13-431c-8a5e-8ec1ba58edcb service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:55:51.447573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec2fd3-7e13-431c-8a5e-8ec1ba58edcb service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:55:51.453747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0d8e4-4ccc-4891-bbbf-c13dfecdf402 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:55:51.454764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0d8e4-4ccc-4891-bbbf-c13dfecdf402 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:51.455226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0d8e4-4ccc-4891-bbbf-c13dfecdf402 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:51.455741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0d8e4-4ccc-4891-bbbf-c13dfecdf402 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:51.467729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0d8e4-4ccc-4891-bbbf-c13dfecdf402 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:55:51.470297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0d8e4-4ccc-4891-bbbf-c13dfecdf402 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:55:51.484119 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1523/3002] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:55:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:55:52.247608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1bc0c-51be-4cbe-8139-b87eb4fa2059 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:52.248232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1bc0c-51be-4cbe-8139-b87eb4fa2059 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:52.248420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1bc0c-51be-4cbe-8139-b87eb4fa2059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:52.268633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-03b1bc0c-51be-4cbe-8139-b87eb4fa2059 None None] MFA Rules not processed for user `d9688dca94a841cfa29420ad780562cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:52.270185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-03b1bc0c-51be-4cbe-8139-b87eb4fa2059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:52.330502 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1480/3003] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:55:57.568621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b36c0-bd1b-424a-9589-ca2284e28423 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:55:57.569217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b36c0-bd1b-424a-9589-ca2284e28423 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:55:57.569315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b36c0-bd1b-424a-9589-ca2284e28423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:55:57.605664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9c1b36c0-bd1b-424a-9589-ca2284e28423 None None] MFA Rules not processed for user `27be6827d30e48acac9a59786021f7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:55:57.606936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9c1b36c0-bd1b-424a-9589-ca2284e28423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:55:57.703492 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1524/3004] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:55:57 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:01.201272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2cc6d-8891-4b9a-9b2d-cac2ec262414 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:01.201750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2cc6d-8891-4b9a-9b2d-cac2ec262414 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:01.201750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2cc6d-8891-4b9a-9b2d-cac2ec262414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:01.268530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f7c2cc6d-8891-4b9a-9b2d-cac2ec262414 None None] MFA Rules not processed for user `f4d78b7738644ff38eb239daec52d641`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:01.269955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f7c2cc6d-8891-4b9a-9b2d-cac2ec262414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:01.293505 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1481/3005] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:56:07.009192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b36c0-bd1b-424a-9589-ca2284e28423 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:07.024179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b36c0-bd1b-424a-9589-ca2284e28423 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:07.030566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fdd4e1-9289-4cc2-a606-757a07254a93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:07.031258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdd4e1-9289-4cc2-a606-757a07254a93 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:07.031546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdd4e1-9289-4cc2-a606-757a07254a93 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:07.031824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdd4e1-9289-4cc2-a606-757a07254a93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:07.034016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d2fdd4e1-9289-4cc2-a606-757a07254a93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:07.045748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fdd4e1-9289-4cc2-a606-757a07254a93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:07.047969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fdd4e1-9289-4cc2-a606-757a07254a93 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:07.058071 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1525/3006] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:11.190142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2cc6d-8891-4b9a-9b2d-cac2ec262414 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:11.203223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2cc6d-8891-4b9a-9b2d-cac2ec262414 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:11.208724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b7696-ec0e-4465-8406-0cda8b21951d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:11.209367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b7696-ec0e-4465-8406-0cda8b21951d service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:11.209653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b7696-ec0e-4465-8406-0cda8b21951d service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:11.209931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b7696-ec0e-4465-8406-0cda8b21951d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:11.211671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-628b7696-ec0e-4465-8406-0cda8b21951d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:11.274214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b7696-ec0e-4465-8406-0cda8b21951d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:11.276216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b7696-ec0e-4465-8406-0cda8b21951d service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:11.286504 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1482/3007] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:56:11.473377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4f3c9-ad80-47b6-b4b5-dadb8921b77c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:11.473852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4f3c9-ad80-47b6-b4b5-dadb8921b77c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:11.473970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4f3c9-ad80-47b6-b4b5-dadb8921b77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:11.492669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5ac4f3c9-ad80-47b6-b4b5-dadb8921b77c None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:11.493905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5ac4f3c9-ad80-47b6-b4b5-dadb8921b77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:11.501625 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1526/3008] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:56:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:11.513544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b7696-ec0e-4465-8406-0cda8b21951d service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:11.525790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b7696-ec0e-4465-8406-0cda8b21951d service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:11.531296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed12dfa-8e2d-41b1-94bd-892b4f0ee963 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:11.531943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed12dfa-8e2d-41b1-94bd-892b4f0ee963 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:11.532231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed12dfa-8e2d-41b1-94bd-892b4f0ee963 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:11.532510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed12dfa-8e2d-41b1-94bd-892b4f0ee963 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:11.541290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed12dfa-8e2d-41b1-94bd-892b4f0ee963 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:11.542885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed12dfa-8e2d-41b1-94bd-892b4f0ee963 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:11.551582 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1483/3009] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:56:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:56:12.147946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca939264-ed51-4733-96be-0827851fd414 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:12.148242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca939264-ed51-4733-96be-0827851fd414 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:12.148525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca939264-ed51-4733-96be-0827851fd414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:12.167512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ca939264-ed51-4733-96be-0827851fd414 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:12.168714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ca939264-ed51-4733-96be-0827851fd414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:12.176374 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1527/3010] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:56:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:12.188601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed12dfa-8e2d-41b1-94bd-892b4f0ee963 service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:12.200363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed12dfa-8e2d-41b1-94bd-892b4f0ee963 service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:12.205813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dfc164-05bf-4366-b9ac-57e72c58b9a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:12.206421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfc164-05bf-4366-b9ac-57e72c58b9a1 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:12.206735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfc164-05bf-4366-b9ac-57e72c58b9a1 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:12.207015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfc164-05bf-4366-b9ac-57e72c58b9a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:12.215676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dfc164-05bf-4366-b9ac-57e72c58b9a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:12.217322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dfc164-05bf-4366-b9ac-57e72c58b9a1 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:12.226709 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1484/3011] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:56:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:56:13.266766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eace5a4-adac-4fe6-94fe-e1e05721165d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:13.267233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eace5a4-adac-4fe6-94fe-e1e05721165d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:13.267488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eace5a4-adac-4fe6-94fe-e1e05721165d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:13.287844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3eace5a4-adac-4fe6-94fe-e1e05721165d None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:13.289115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3eace5a4-adac-4fe6-94fe-e1e05721165d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:13.296318 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1528/3012] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:13.309302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dfc164-05bf-4366-b9ac-57e72c58b9a1 service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:13.321693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dfc164-05bf-4366-b9ac-57e72c58b9a1 service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:13.327283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5bb92-2d2d-4540-a8fd-a2cfbaa42739 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:13.327889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5bb92-2d2d-4540-a8fd-a2cfbaa42739 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:13.328172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5bb92-2d2d-4540-a8fd-a2cfbaa42739 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:13.328571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5bb92-2d2d-4540-a8fd-a2cfbaa42739 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:13.338494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5bb92-2d2d-4540-a8fd-a2cfbaa42739 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:13.340424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5bb92-2d2d-4540-a8fd-a2cfbaa42739 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:13.349826 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1485/3013] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:56:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:56:13.483294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eace5a4-adac-4fe6-94fe-e1e05721165d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:13.485030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3eace5a4-adac-4fe6-94fe-e1e05721165d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:13.506361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eace5a4-adac-4fe6-94fe-e1e05721165d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:13.512769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:13.513693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:13.514165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:13.514642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] PATH_INFO: `/v3/users/0f8ece445698482795d9b9d5ad5b13aa` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:13.515971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f8ece445698482795d9b9d5ad5b13aa)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:13.518573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:13.584235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f8ece445698482795d9b9d5ad5b13aa'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:13.598461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f8ece445698482795d9b9d5ad5b13aa'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:13.606064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f8ece445698482795d9b9d5ad5b13aa'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:13.606688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f8ece445698482795d9b9d5ad5b13aa'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:13.607318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f8ece445698482795d9b9d5ad5b13aa'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:13.609426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:13.623091 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1529/3014] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:56:13 2022] DELETE /identity/v3/users/0f8ece445698482795d9b9d5ad5b13aa => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:13.627653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c90292-bd22-494d-90d6-8fd85db4a907 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:13.627942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c90292-bd22-494d-90d6-8fd85db4a907 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:13.628221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c90292-bd22-494d-90d6-8fd85db4a907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:13.647614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-88c90292-bd22-494d-90d6-8fd85db4a907 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:13.648836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-88c90292-bd22-494d-90d6-8fd85db4a907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:13.670599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:13.714608 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1486/3015] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:13.727308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:13.729329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:13.793132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ac842-405c-4fbb-9981-64d164b4802b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:13.796973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5059b-9288-40eb-a547-ffb3b56d1a4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:13.797615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5059b-9288-40eb-a547-ffb3b56d1a4e service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:13.797901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5059b-9288-40eb-a547-ffb3b56d1a4e service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:13.798178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5059b-9288-40eb-a547-ffb3b56d1a4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:13.812078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5059b-9288-40eb-a547-ffb3b56d1a4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:13.812078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5059b-9288-40eb-a547-ffb3b56d1a4e service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:13.821370 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1530/3016] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:14.006361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c90292-bd22-494d-90d6-8fd85db4a907 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:14.009396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-88c90292-bd22-494d-90d6-8fd85db4a907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:14.025352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c90292-bd22-494d-90d6-8fd85db4a907 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:14.030885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762b4c3-d327-4e61-8157-b564124b688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:14.031578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762b4c3-d327-4e61-8157-b564124b688e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:14.032170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762b4c3-d327-4e61-8157-b564124b688e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:14.032520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762b4c3-d327-4e61-8157-b564124b688e admin admin] PATH_INFO: `/v3/projects/23ef148ffad044529cb3658ac1ed3107` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:14.033921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762b4c3-d327-4e61-8157-b564124b688e admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ef148ffad044529cb3658ac1ed3107)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:14.036886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762b4c3-d327-4e61-8157-b564124b688e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:14.106084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4762b4c3-d327-4e61-8157-b564124b688e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ef148ffad044529cb3658ac1ed3107', 'request_id': 'req-4762b4c3-d327-4e61-8157-b564124b688e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:14.115742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4762b4c3-d327-4e61-8157-b564124b688e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ef148ffad044529cb3658ac1ed3107', 'request_id': 'req-4762b4c3-d327-4e61-8157-b564124b688e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:14.131496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4762b4c3-d327-4e61-8157-b564124b688e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ef148ffad044529cb3658ac1ed3107', 'request_id': 'req-4762b4c3-d327-4e61-8157-b564124b688e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:14.151763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4762b4c3-d327-4e61-8157-b564124b688e admin admin] The token cache is being invalidate because project 23ef148ffad044529cb3658ac1ed3107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:56:14.152533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4762b4c3-d327-4e61-8157-b564124b688e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:14.153542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4762b4c3-d327-4e61-8157-b564124b688e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:14.155777 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1487/3017] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:56:14 2022] DELETE /identity/v3/projects/23ef148ffad044529cb3658ac1ed3107 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:14.164631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b84a0b-2553-4c44-a991-2148991a71d6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:14.166006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b84a0b-2553-4c44-a991-2148991a71d6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:14.166449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b84a0b-2553-4c44-a991-2148991a71d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:14.214076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f1b84a0b-2553-4c44-a991-2148991a71d6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:14.215917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f1b84a0b-2553-4c44-a991-2148991a71d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:14.321083 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1531/3018] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:14.323954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6be870-bda6-4ef3-b811-a1e87bfbdb46 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:14.324727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6be870-bda6-4ef3-b811-a1e87bfbdb46 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:14.325229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6be870-bda6-4ef3-b811-a1e87bfbdb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:14.346955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fb6be870-bda6-4ef3-b811-a1e87bfbdb46 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:14.348666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fb6be870-bda6-4ef3-b811-a1e87bfbdb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:14.357441 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1488/3019] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:14.361688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b84a0b-2553-4c44-a991-2148991a71d6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:14.372194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b84a0b-2553-4c44-a991-2148991a71d6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:14.377207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb04155-8066-4f29-a4b4-120077421640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:14.377837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb04155-8066-4f29-a4b4-120077421640 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:14.378120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb04155-8066-4f29-a4b4-120077421640 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:14.378504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb04155-8066-4f29-a4b4-120077421640 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:14.379267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb04155-8066-4f29-a4b4-120077421640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:56:14.379462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb04155-8066-4f29-a4b4-120077421640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:14.381600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb04155-8066-4f29-a4b4-120077421640 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:14.397070 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1532/3020] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:56:14.400021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6be870-bda6-4ef3-b811-a1e87bfbdb46 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:14.410988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6be870-bda6-4ef3-b811-a1e87bfbdb46 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:14.416148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d9da2-93e8-4dd0-9f15-55c5c152fd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:14.417191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d9da2-93e8-4dd0-9f15-55c5c152fd7d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:14.417490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d9da2-93e8-4dd0-9f15-55c5c152fd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:14.417767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d9da2-93e8-4dd0-9f15-55c5c152fd7d admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:14.418407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d9da2-93e8-4dd0-9f15-55c5c152fd7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:14.420427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d9da2-93e8-4dd0-9f15-55c5c152fd7d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:14.462565 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1489/3021] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:56:14 2022] POST /identity/v3/projects => generated 401 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:56:14.466735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb04155-8066-4f29-a4b4-120077421640 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:14.478087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb04155-8066-4f29-a4b4-120077421640 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:14.550702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dbdc1-8d0d-4a99-b7e9-acba354af136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:14.551330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dbdc1-8d0d-4a99-b7e9-acba354af136 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:14.551619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dbdc1-8d0d-4a99-b7e9-acba354af136 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:14.551895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dbdc1-8d0d-4a99-b7e9-acba354af136 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:14.552532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dbdc1-8d0d-4a99-b7e9-acba354af136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:14.555470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dbdc1-8d0d-4a99-b7e9-acba354af136 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:14.589773 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1533/3022] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:56:14 2022] POST /identity/v3/users => generated 301 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:56:14.593625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d9da2-93e8-4dd0-9f15-55c5c152fd7d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:14.606106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d9da2-93e8-4dd0-9f15-55c5c152fd7d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:14.611239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132a431-ce20-41d3-a1d5-02338579c25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:14.611852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132a431-ce20-41d3-a1d5-02338579c25e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:14.612131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132a431-ce20-41d3-a1d5-02338579c25e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:14.612406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132a431-ce20-41d3-a1d5-02338579c25e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:14.613591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132a431-ce20-41d3-a1d5-02338579c25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:14.615196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132a431-ce20-41d3-a1d5-02338579c25e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:14.632012 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1490/3023] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:14.634208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dbdc1-8d0d-4a99-b7e9-acba354af136 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:14.656679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dbdc1-8d0d-4a99-b7e9-acba354af136 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:14.663398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310b2b6-361f-4c35-a9c4-e09ab4c2aae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:14.664069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310b2b6-361f-4c35-a9c4-e09ab4c2aae8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:14.664396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310b2b6-361f-4c35-a9c4-e09ab4c2aae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:14.664748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310b2b6-361f-4c35-a9c4-e09ab4c2aae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:14.665617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310b2b6-361f-4c35-a9c4-e09ab4c2aae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:14.667905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310b2b6-361f-4c35-a9c4-e09ab4c2aae8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:14.674603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:14.687425 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1534/3024] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:14.689117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132a431-ce20-41d3-a1d5-02338579c25e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:14.715472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132a431-ce20-41d3-a1d5-02338579c25e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:14.726770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af330776-a264-43e7-b7a9-13d7f261ca39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:14.728741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af330776-a264-43e7-b7a9-13d7f261ca39 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:14.729511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af330776-a264-43e7-b7a9-13d7f261ca39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:14.729804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af330776-a264-43e7-b7a9-13d7f261ca39 admin admin] PATH_INFO: `/v3/projects/a6b699f400a84eed9e606a9a1c366970/users/55530fcf9fea48279b38c2bd7db37122/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:14.773009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af330776-a264-43e7-b7a9-13d7f261ca39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b699f400a84eed9e606a9a1c366970, user_id=55530fcf9fea48279b38c2bd7db37122, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:14.789578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af330776-a264-43e7-b7a9-13d7f261ca39 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:14.827121 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1491/3025] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:56:14 2022] PUT /identity/v3/projects/a6b699f400a84eed9e606a9a1c366970/users/55530fcf9fea48279b38c2bd7db37122/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:14.830688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cb0b0-9d9c-41d1-9d23-3dd327d203ed None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:14.831433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cb0b0-9d9c-41d1-9d23-3dd327d203ed None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:14.832663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cb0b0-9d9c-41d1-9d23-3dd327d203ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:14.839874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af330776-a264-43e7-b7a9-13d7f261ca39 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:14.847317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-af330776-a264-43e7-b7a9-13d7f261ca39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:14.904167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b76cb0b0-9d9c-41d1-9d23-3dd327d203ed None None] MFA Rules not processed for user `55530fcf9fea48279b38c2bd7db37122`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:14.908282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b76cb0b0-9d9c-41d1-9d23-3dd327d203ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:15.004275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:15.016322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af330776-a264-43e7-b7a9-13d7f261ca39 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:15.025065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84776d4-caab-4f1d-b50c-2541181bd8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:15.026237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84776d4-caab-4f1d-b50c-2541181bd8ca service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:15.026998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84776d4-caab-4f1d-b50c-2541181bd8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:15.027798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84776d4-caab-4f1d-b50c-2541181bd8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:15.032945 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1535/3026] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:14 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:15.033123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b84776d4-caab-4f1d-b50c-2541181bd8ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:15.129881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84776d4-caab-4f1d-b50c-2541181bd8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:15.136054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84776d4-caab-4f1d-b50c-2541181bd8ca service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:15.145328 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1492/3027] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:15.165935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cb0b0-9d9c-41d1-9d23-3dd327d203ed None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:15.175541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cb0b0-9d9c-41d1-9d23-3dd327d203ed None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:15.179182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da35a5e-dd88-4a41-a8aa-f120e6c65020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:15.179959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da35a5e-dd88-4a41-a8aa-f120e6c65020 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:15.180246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da35a5e-dd88-4a41-a8aa-f120e6c65020 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:15.180523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da35a5e-dd88-4a41-a8aa-f120e6c65020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:15.182802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9da35a5e-dd88-4a41-a8aa-f120e6c65020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:15.193011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da35a5e-dd88-4a41-a8aa-f120e6c65020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:15.194510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da35a5e-dd88-4a41-a8aa-f120e6c65020 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:15.203561 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1536/3028] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:15.887335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84776d4-caab-4f1d-b50c-2541181bd8ca service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:15.889643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b84776d4-caab-4f1d-b50c-2541181bd8ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:15.920818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84776d4-caab-4f1d-b50c-2541181bd8ca service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:15.929826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8e0f6-d6ee-4cd3-b260-e8111646fe9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:15.930458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8e0f6-d6ee-4cd3-b260-e8111646fe9d service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:15.930982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8e0f6-d6ee-4cd3-b260-e8111646fe9d service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:15.931267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8e0f6-d6ee-4cd3-b260-e8111646fe9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:15.952182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8e0f6-d6ee-4cd3-b260-e8111646fe9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:15.954310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8e0f6-d6ee-4cd3-b260-e8111646fe9d service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:15.974041 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1493/3029] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:15 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:16.031779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da35a5e-dd88-4a41-a8aa-f120e6c65020 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:16.173380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da35a5e-dd88-4a41-a8aa-f120e6c65020 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:16.188149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:16.189009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:16.189524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:16.189833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] PATH_INFO: `/v3/users/55530fcf9fea48279b38c2bd7db37122` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:16.199706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55530fcf9fea48279b38c2bd7db37122)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:16.203765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:16.279184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55530fcf9fea48279b38c2bd7db37122'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:16.297759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55530fcf9fea48279b38c2bd7db37122'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:16.310978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55530fcf9fea48279b38c2bd7db37122'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:16.312418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55530fcf9fea48279b38c2bd7db37122'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:16.314466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55530fcf9fea48279b38c2bd7db37122'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:16.332961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:16.352226 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1537/3030] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:56:16 2022] DELETE /identity/v3/users/55530fcf9fea48279b38c2bd7db37122 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:16.357232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc07ac4-15cf-4f80-bed6-6414c6c4107d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:16.359567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc07ac4-15cf-4f80-bed6-6414c6c4107d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:16.360206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc07ac4-15cf-4f80-bed6-6414c6c4107d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:16.417264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7dc07ac4-15cf-4f80-bed6-6414c6c4107d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:16.419316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7dc07ac4-15cf-4f80-bed6-6414c6c4107d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:16.567362 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1494/3031] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:16.585139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:16.586946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:16.732931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4613e65-caa2-4aa8-8ce6-0de32610ad84 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:16.745195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd757d9-8bfe-47cf-a354-8a707abca704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:16.746296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd757d9-8bfe-47cf-a354-8a707abca704 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:16.746635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd757d9-8bfe-47cf-a354-8a707abca704 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:16.746925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd757d9-8bfe-47cf-a354-8a707abca704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:16.773217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd757d9-8bfe-47cf-a354-8a707abca704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:16.777469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd757d9-8bfe-47cf-a354-8a707abca704 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:16.798972 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1538/3032] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:17.305130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc07ac4-15cf-4f80-bed6-6414c6c4107d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:17.307507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7dc07ac4-15cf-4f80-bed6-6414c6c4107d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:17.326309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc07ac4-15cf-4f80-bed6-6414c6c4107d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:17.331915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d10b9-0599-4714-a178-fb6b64dd49b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:17.332831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d10b9-0599-4714-a178-fb6b64dd49b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:17.333308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d10b9-0599-4714-a178-fb6b64dd49b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:17.334743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d10b9-0599-4714-a178-fb6b64dd49b8 admin admin] PATH_INFO: `/v3/projects/a6b699f400a84eed9e606a9a1c366970` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:17.336309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d10b9-0599-4714-a178-fb6b64dd49b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b699f400a84eed9e606a9a1c366970)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:17.339703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d10b9-0599-4714-a178-fb6b64dd49b8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:17.407995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-419d10b9-0599-4714-a178-fb6b64dd49b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b699f400a84eed9e606a9a1c366970', 'request_id': 'req-419d10b9-0599-4714-a178-fb6b64dd49b8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:17.418291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-419d10b9-0599-4714-a178-fb6b64dd49b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b699f400a84eed9e606a9a1c366970', 'request_id': 'req-419d10b9-0599-4714-a178-fb6b64dd49b8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:17.425715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-419d10b9-0599-4714-a178-fb6b64dd49b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b699f400a84eed9e606a9a1c366970', 'request_id': 'req-419d10b9-0599-4714-a178-fb6b64dd49b8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:17.437660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-419d10b9-0599-4714-a178-fb6b64dd49b8 admin admin] The token cache is being invalidate because project a6b699f400a84eed9e606a9a1c366970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:56:17.437953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-419d10b9-0599-4714-a178-fb6b64dd49b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:17.438814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-419d10b9-0599-4714-a178-fb6b64dd49b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:17.440416 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1495/3033] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:56:17 2022] DELETE /identity/v3/projects/a6b699f400a84eed9e606a9a1c366970 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:17.444124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:17.447816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9a1de-08cc-4376-b916-08f7b6a1ebf7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:17.448453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9a1de-08cc-4376-b916-08f7b6a1ebf7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:17.448575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9a1de-08cc-4376-b916-08f7b6a1ebf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:17.469445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3ce9a1de-08cc-4376-b916-08f7b6a1ebf7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:17.470684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3ce9a1de-08cc-4376-b916-08f7b6a1ebf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:17.539754 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1539/3034] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:17.547727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6000b5-ca01-4b62-b298-517ccef28ee1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:17.548417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6000b5-ca01-4b62-b298-517ccef28ee1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:17.548957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6000b5-ca01-4b62-b298-517ccef28ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:17.577355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3d6000b5-ca01-4b62-b298-517ccef28ee1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:17.577355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3d6000b5-ca01-4b62-b298-517ccef28ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:17.584353 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1496/3035] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:17.588929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9a1de-08cc-4376-b916-08f7b6a1ebf7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:17.599573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9a1de-08cc-4376-b916-08f7b6a1ebf7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:17.607103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd255ded-e79b-424f-b347-eb20a121ea6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:17.607850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd255ded-e79b-424f-b347-eb20a121ea6f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:17.608139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd255ded-e79b-424f-b347-eb20a121ea6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:17.608928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd255ded-e79b-424f-b347-eb20a121ea6f admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:17.609130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd255ded-e79b-424f-b347-eb20a121ea6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:56:17.610177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd255ded-e79b-424f-b347-eb20a121ea6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:17.611087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd255ded-e79b-424f-b347-eb20a121ea6f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:17.624189 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1540/3036] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:56:17.629429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6000b5-ca01-4b62-b298-517ccef28ee1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:17.641110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6000b5-ca01-4b62-b298-517ccef28ee1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:17.646351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d000752-560a-4960-9aa7-92eec756578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:17.646975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d000752-560a-4960-9aa7-92eec756578b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:17.647329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d000752-560a-4960-9aa7-92eec756578b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:17.647675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d000752-560a-4960-9aa7-92eec756578b admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:17.648397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d000752-560a-4960-9aa7-92eec756578b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:17.650422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d000752-560a-4960-9aa7-92eec756578b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:17.681998 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1497/3037] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:56:17 2022] POST /identity/v3/projects => generated 401 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:56:17.686303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd255ded-e79b-424f-b347-eb20a121ea6f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:17.712526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd255ded-e79b-424f-b347-eb20a121ea6f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:17.775930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99b740-0445-4d06-82f0-95f3b7b8218c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:17.776535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99b740-0445-4d06-82f0-95f3b7b8218c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:17.776819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99b740-0445-4d06-82f0-95f3b7b8218c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:17.777201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99b740-0445-4d06-82f0-95f3b7b8218c admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:17.777826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b99b740-0445-4d06-82f0-95f3b7b8218c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:17.779746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b99b740-0445-4d06-82f0-95f3b7b8218c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:17.803097 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1541/3038] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:56:17 2022] POST /identity/v3/users => generated 301 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:56:17.807288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d000752-560a-4960-9aa7-92eec756578b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:17.818211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d000752-560a-4960-9aa7-92eec756578b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:17.823115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bebd8f-b020-40f8-b702-215077698cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:17.823736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bebd8f-b020-40f8-b702-215077698cfd admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:17.824022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bebd8f-b020-40f8-b702-215077698cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:17.824303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bebd8f-b020-40f8-b702-215077698cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:17.824987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bebd8f-b020-40f8-b702-215077698cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:17.826614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bebd8f-b020-40f8-b702-215077698cfd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:17.840486 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1498/3039] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:17.841997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99b740-0445-4d06-82f0-95f3b7b8218c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:17.854840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99b740-0445-4d06-82f0-95f3b7b8218c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:17.859839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243996fd-bdff-4f49-8244-451ff33fa422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:17.860447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243996fd-bdff-4f49-8244-451ff33fa422 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:17.860725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243996fd-bdff-4f49-8244-451ff33fa422 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:17.861588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243996fd-bdff-4f49-8244-451ff33fa422 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:17.862248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243996fd-bdff-4f49-8244-451ff33fa422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:17.864441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243996fd-bdff-4f49-8244-451ff33fa422 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:17.876443 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1542/3040] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:17.880997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bebd8f-b020-40f8-b702-215077698cfd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:17.893146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bebd8f-b020-40f8-b702-215077698cfd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:17.898251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c210a99-e193-47a6-a697-19ed10e2b2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:17.898821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c210a99-e193-47a6-a697-19ed10e2b2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:17.899173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c210a99-e193-47a6-a697-19ed10e2b2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:17.899383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c210a99-e193-47a6-a697-19ed10e2b2c4 admin admin] PATH_INFO: `/v3/projects/725b1b6cb58b449283c1ca003588e248/users/c8a88f53cc78444299e4c462e004edf9/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:17.915421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c210a99-e193-47a6-a697-19ed10e2b2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=725b1b6cb58b449283c1ca003588e248, user_id=c8a88f53cc78444299e4c462e004edf9, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:17.918121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c210a99-e193-47a6-a697-19ed10e2b2c4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:17.933990 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1499/3041] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:56:17 2022] PUT /identity/v3/projects/725b1b6cb58b449283c1ca003588e248/users/c8a88f53cc78444299e4c462e004edf9/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:17.939098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2102456-65f6-48b1-bead-8de8ccb39c9d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:17.939438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2102456-65f6-48b1-bead-8de8ccb39c9d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:17.939744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2102456-65f6-48b1-bead-8de8ccb39c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:17.948520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:17.960429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e2102456-65f6-48b1-bead-8de8ccb39c9d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:17.961679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e2102456-65f6-48b1-bead-8de8ccb39c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:18.024717 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1543/3042] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:18.033867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c210a99-e193-47a6-a697-19ed10e2b2c4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:18.036179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4c210a99-e193-47a6-a697-19ed10e2b2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:18.094530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c210a99-e193-47a6-a697-19ed10e2b2c4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:18.098774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03c58c-b8d1-41ec-9745-791cfa25472c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:18.099381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03c58c-b8d1-41ec-9745-791cfa25472c service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:18.099664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03c58c-b8d1-41ec-9745-791cfa25472c service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:18.099945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03c58c-b8d1-41ec-9745-791cfa25472c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:18.108970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03c58c-b8d1-41ec-9745-791cfa25472c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:18.110505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03c58c-b8d1-41ec-9745-791cfa25472c service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:18.120874 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1500/3043] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:28.016536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=25 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:28.116419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:28.124150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454f07a-d9f8-4951-a8f8-0411264066ee None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:28.124444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454f07a-d9f8-4951-a8f8-0411264066ee None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:28.124712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454f07a-d9f8-4951-a8f8-0411264066ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:28.185220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2454f07a-d9f8-4951-a8f8-0411264066ee None None] MFA Rules not processed for user `c8a88f53cc78444299e4c462e004edf9`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:28.186548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2454f07a-d9f8-4951-a8f8-0411264066ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:28.301576 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1544/3044] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:28 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:28.328002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03c58c-b8d1-41ec-9745-791cfa25472c service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:28.329771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5d03c58c-b8d1-41ec-9745-791cfa25472c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:28.463425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03c58c-b8d1-41ec-9745-791cfa25472c service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:28.477194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8909dc-8d49-41b5-befc-0751f8cd9de2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:28.477971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8909dc-8d49-41b5-befc-0751f8cd9de2 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:28.478329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8909dc-8d49-41b5-befc-0751f8cd9de2 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:28.478694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8909dc-8d49-41b5-befc-0751f8cd9de2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:28.493581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8909dc-8d49-41b5-befc-0751f8cd9de2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:28.495450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8909dc-8d49-41b5-befc-0751f8cd9de2 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:28.521199 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1501/3045] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:28 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:37.358175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454f07a-d9f8-4951-a8f8-0411264066ee None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:37.360332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2454f07a-d9f8-4951-a8f8-0411264066ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:37.572582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454f07a-d9f8-4951-a8f8-0411264066ee None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:37.580470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207eb0e-86ec-48d5-8299-cfc729374116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:37.581988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207eb0e-86ec-48d5-8299-cfc729374116 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:37.588407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207eb0e-86ec-48d5-8299-cfc729374116 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:37.588825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207eb0e-86ec-48d5-8299-cfc729374116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:37.590893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4207eb0e-86ec-48d5-8299-cfc729374116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:37.866976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207eb0e-86ec-48d5-8299-cfc729374116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:37.869009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207eb0e-86ec-48d5-8299-cfc729374116 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:37.883695 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1545/3046] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:37 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:38.199419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8909dc-8d49-41b5-befc-0751f8cd9de2 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:38.202232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8c8909dc-8d49-41b5-befc-0751f8cd9de2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:38.440483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207eb0e-86ec-48d5-8299-cfc729374116 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:38.443130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4207eb0e-86ec-48d5-8299-cfc729374116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:38.471492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207eb0e-86ec-48d5-8299-cfc729374116 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:38.484699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9988b0-d341-46a7-91ba-27ce0fa84d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:38.486848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9988b0-d341-46a7-91ba-27ce0fa84d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:38.487076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9988b0-d341-46a7-91ba-27ce0fa84d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:38.488783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9988b0-d341-46a7-91ba-27ce0fa84d6f admin admin] PATH_INFO: `/v3/users/d9688dca94a841cfa29420ad780562cd` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:38.494730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9988b0-d341-46a7-91ba-27ce0fa84d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9688dca94a841cfa29420ad780562cd)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:38.501370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9988b0-d341-46a7-91ba-27ce0fa84d6f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:38.508437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8909dc-8d49-41b5-befc-0751f8cd9de2 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:38.518198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801046ac-b4fa-4187-9ac9-9f459c88e5ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:38.519758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801046ac-b4fa-4187-9ac9-9f459c88e5ee service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:38.520225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801046ac-b4fa-4187-9ac9-9f459c88e5ee service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:38.520666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801046ac-b4fa-4187-9ac9-9f459c88e5ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:38.528285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-801046ac-b4fa-4187-9ac9-9f459c88e5ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:38.572838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801046ac-b4fa-4187-9ac9-9f459c88e5ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:38.574616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801046ac-b4fa-4187-9ac9-9f459c88e5ee service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:38.592978 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1502/3047] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:38.667831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:38.696318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a9988b0-d341-46a7-91ba-27ce0fa84d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9688dca94a841cfa29420ad780562cd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:38.746419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a9988b0-d341-46a7-91ba-27ce0fa84d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9688dca94a841cfa29420ad780562cd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:38.764604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a9988b0-d341-46a7-91ba-27ce0fa84d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9688dca94a841cfa29420ad780562cd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:38.765315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a9988b0-d341-46a7-91ba-27ce0fa84d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9688dca94a841cfa29420ad780562cd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:38.765927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a9988b0-d341-46a7-91ba-27ce0fa84d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9688dca94a841cfa29420ad780562cd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:38.788600 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1546/3048] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:56:38 2022] DELETE /identity/v3/users/d9688dca94a841cfa29420ad780562cd => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:56:39.176590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801046ac-b4fa-4187-9ac9-9f459c88e5ee service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:39.238550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801046ac-b4fa-4187-9ac9-9f459c88e5ee service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:39.289721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:39.386358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd3c06-9c39-41b9-a065-60e95f446aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:39.387139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd3c06-9c39-41b9-a065-60e95f446aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:39.387655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd3c06-9c39-41b9-a065-60e95f446aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:39.388173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd3c06-9c39-41b9-a065-60e95f446aa7 admin admin] PATH_INFO: `/v3/projects/f17cee6ff91f4292b688b5cc63292039` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:39.392382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd3c06-9c39-41b9-a065-60e95f446aa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f17cee6ff91f4292b688b5cc63292039)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:39.395032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd3c06-9c39-41b9-a065-60e95f446aa7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:39.533015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1cdd3c06-9c39-41b9-a065-60e95f446aa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f17cee6ff91f4292b688b5cc63292039', 'request_id': 'req-1cdd3c06-9c39-41b9-a065-60e95f446aa7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:39.563005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1cdd3c06-9c39-41b9-a065-60e95f446aa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f17cee6ff91f4292b688b5cc63292039', 'request_id': 'req-1cdd3c06-9c39-41b9-a065-60e95f446aa7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:39.582314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1cdd3c06-9c39-41b9-a065-60e95f446aa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f17cee6ff91f4292b688b5cc63292039', 'request_id': 'req-1cdd3c06-9c39-41b9-a065-60e95f446aa7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:39.607769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1cdd3c06-9c39-41b9-a065-60e95f446aa7 admin admin] The token cache is being invalidate because project f17cee6ff91f4292b688b5cc63292039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:56:39.608484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1cdd3c06-9c39-41b9-a065-60e95f446aa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:39.610056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1cdd3c06-9c39-41b9-a065-60e95f446aa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:39.620415 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1503/3049] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:56:39 2022] DELETE /identity/v3/projects/f17cee6ff91f4292b688b5cc63292039 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:39.622842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6aa78-b883-4c4a-ae3b-a7b148c976a1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:39.623786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6aa78-b883-4c4a-ae3b-a7b148c976a1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:39.623786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6aa78-b883-4c4a-ae3b-a7b148c976a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:39.677839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-11e6aa78-b883-4c4a-ae3b-a7b148c976a1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:39.680065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-11e6aa78-b883-4c4a-ae3b-a7b148c976a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:39.784068 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1547/3050] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:56:39.792447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39eabb-47af-4538-9848-3114ebdad14e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:39.792741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39eabb-47af-4538-9848-3114ebdad14e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:39.793054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39eabb-47af-4538-9848-3114ebdad14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:39.823499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ae39eabb-47af-4538-9848-3114ebdad14e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:39.826703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ae39eabb-47af-4538-9848-3114ebdad14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:39.836698 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1504/3051] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:39.840802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6aa78-b883-4c4a-ae3b-a7b148c976a1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:39.859734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6aa78-b883-4c4a-ae3b-a7b148c976a1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:39.871454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85bbe9-a110-44d4-aa91-3a5edd04f39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:39.872107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85bbe9-a110-44d4-aa91-3a5edd04f39d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:39.872386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85bbe9-a110-44d4-aa91-3a5edd04f39d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:39.872725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85bbe9-a110-44d4-aa91-3a5edd04f39d admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:39.873375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85bbe9-a110-44d4-aa91-3a5edd04f39d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:56:39.873691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85bbe9-a110-44d4-aa91-3a5edd04f39d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:39.876087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85bbe9-a110-44d4-aa91-3a5edd04f39d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:39.897761 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1548/3052] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:56:39.898768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39eabb-47af-4538-9848-3114ebdad14e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:39.921974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39eabb-47af-4538-9848-3114ebdad14e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:39.939914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d24b7-399b-46c2-8daa-f205dd42b2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:39.942390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d24b7-399b-46c2-8daa-f205dd42b2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:39.945217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d24b7-399b-46c2-8daa-f205dd42b2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:39.946475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d24b7-399b-46c2-8daa-f205dd42b2a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:39.948292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d24b7-399b-46c2-8daa-f205dd42b2a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:39.955941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d24b7-399b-46c2-8daa-f205dd42b2a8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:40.073355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85bbe9-a110-44d4-aa91-3a5edd04f39d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:40.074298 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1505/3053] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:56:39 2022] POST /identity/v3/projects => generated 391 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:56:40.086299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85bbe9-a110-44d4-aa91-3a5edd04f39d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:40.258260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed47693-1a7d-418c-a24e-a1527b181cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:40.261169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed47693-1a7d-418c-a24e-a1527b181cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:40.262409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed47693-1a7d-418c-a24e-a1527b181cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:40.263053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed47693-1a7d-418c-a24e-a1527b181cf0 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:40.264546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed47693-1a7d-418c-a24e-a1527b181cf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:40.268128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed47693-1a7d-418c-a24e-a1527b181cf0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:40.308373 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1549/3054] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:56:40 2022] POST /identity/v3/users => generated 296 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:56:40.312920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d24b7-399b-46c2-8daa-f205dd42b2a8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:40.326212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d24b7-399b-46c2-8daa-f205dd42b2a8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:40.331151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd376b4-f227-4f8a-b700-4431687f7745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:40.332277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd376b4-f227-4f8a-b700-4431687f7745 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:40.332841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd376b4-f227-4f8a-b700-4431687f7745 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:40.333508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd376b4-f227-4f8a-b700-4431687f7745 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:40.334747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd376b4-f227-4f8a-b700-4431687f7745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:40.336446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd376b4-f227-4f8a-b700-4431687f7745 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:40.353156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed47693-1a7d-418c-a24e-a1527b181cf0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:40.361477 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1506/3055] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:40.367693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed47693-1a7d-418c-a24e-a1527b181cf0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:40.373436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b9657-0aea-4b21-baf5-e81efe0ec310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:40.374914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b9657-0aea-4b21-baf5-e81efe0ec310 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:40.375438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b9657-0aea-4b21-baf5-e81efe0ec310 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:40.376227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b9657-0aea-4b21-baf5-e81efe0ec310 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:40.377178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b9657-0aea-4b21-baf5-e81efe0ec310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:40.379900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b9657-0aea-4b21-baf5-e81efe0ec310 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:40.396036 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1550/3056] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:40.399938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd376b4-f227-4f8a-b700-4431687f7745 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:40.415753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd376b4-f227-4f8a-b700-4431687f7745 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:40.421098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af26b6-e496-4d46-aa9c-fede6623b17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:40.422057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af26b6-e496-4d46-aa9c-fede6623b17d admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:40.422384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af26b6-e496-4d46-aa9c-fede6623b17d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:40.422613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af26b6-e496-4d46-aa9c-fede6623b17d admin admin] PATH_INFO: `/v3/projects/98285399acfb4823a65117e6e499fb9a/users/2d129b2222f7408db35cae47dcb9a60a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:40.449300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af26b6-e496-4d46-aa9c-fede6623b17d admin admin] RBAC: Authorizing `identity:create_grant(project_id=98285399acfb4823a65117e6e499fb9a, user_id=2d129b2222f7408db35cae47dcb9a60a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:40.452299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af26b6-e496-4d46-aa9c-fede6623b17d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:40.482279 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1507/3057] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:56:40 2022] PUT /identity/v3/projects/98285399acfb4823a65117e6e499fb9a/users/2d129b2222f7408db35cae47dcb9a60a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:40.493332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9a92b-7616-425b-b823-06a8e38e79dd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:40.493714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9a92b-7616-425b-b823-06a8e38e79dd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:40.493982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9a92b-7616-425b-b823-06a8e38e79dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:40.507730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:40.562032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8eb9a92b-7616-425b-b823-06a8e38e79dd None None] MFA Rules not processed for user `2d129b2222f7408db35cae47dcb9a60a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:40.563698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8eb9a92b-7616-425b-b823-06a8e38e79dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:40.703281 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1551/3058] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:42.229639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af26b6-e496-4d46-aa9c-fede6623b17d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:42.232619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a7af26b6-e496-4d46-aa9c-fede6623b17d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:42.352843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:42.392014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af26b6-e496-4d46-aa9c-fede6623b17d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:42.400154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd50e6e-3fc7-4080-8ff3-360ad3ecfc53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:42.401214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd50e6e-3fc7-4080-8ff3-360ad3ecfc53 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:42.402101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd50e6e-3fc7-4080-8ff3-360ad3ecfc53 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:42.403432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd50e6e-3fc7-4080-8ff3-360ad3ecfc53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:42.416436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd50e6e-3fc7-4080-8ff3-360ad3ecfc53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:42.418652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd50e6e-3fc7-4080-8ff3-360ad3ecfc53 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:42.431553 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1508/3059] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:42.491944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9a92b-7616-425b-b823-06a8e38e79dd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:42.494541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8eb9a92b-7616-425b-b823-06a8e38e79dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:42.610662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9a92b-7616-425b-b823-06a8e38e79dd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:42.616302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b4a84-5268-47ca-a69b-4f6b6d9eb599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:42.617273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b4a84-5268-47ca-a69b-4f6b6d9eb599 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:42.617789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b4a84-5268-47ca-a69b-4f6b6d9eb599 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:42.620346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b4a84-5268-47ca-a69b-4f6b6d9eb599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:42.624234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-851b4a84-5268-47ca-a69b-4f6b6d9eb599 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:42.753068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b4a84-5268-47ca-a69b-4f6b6d9eb599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:42.767996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b4a84-5268-47ca-a69b-4f6b6d9eb599 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:42.799599 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1552/3060] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:44.454026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd50e6e-3fc7-4080-8ff3-360ad3ecfc53 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:44.455508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ffd50e6e-3fc7-4080-8ff3-360ad3ecfc53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:44.471070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd50e6e-3fc7-4080-8ff3-360ad3ecfc53 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:44.476601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:44.477294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:44.477585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:44.477865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] PATH_INFO: `/v3/users/2e11c33c786746e5b3effedd32674582` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:44.479067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e11c33c786746e5b3effedd32674582)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:44.481233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:44.546791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e11c33c786746e5b3effedd32674582'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:44.568285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e11c33c786746e5b3effedd32674582'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:44.569471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e11c33c786746e5b3effedd32674582'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:44.593943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e11c33c786746e5b3effedd32674582'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:44.609067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e11c33c786746e5b3effedd32674582'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:44.623051 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1509/3061] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:56:44 2022] DELETE /identity/v3/users/2e11c33c786746e5b3effedd32674582 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:44.624219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1fc4c-a7f4-4145-9edc-2de90e93addc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:44.624724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1fc4c-a7f4-4145-9edc-2de90e93addc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:44.624831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1fc4c-a7f4-4145-9edc-2de90e93addc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:44.648703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-66c1fc4c-a7f4-4145-9edc-2de90e93addc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:44.649915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-66c1fc4c-a7f4-4145-9edc-2de90e93addc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:44.702407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:44.723865 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1553/3062] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:56:44.741203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:44.743937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:44.820167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ceb52-6aaf-4fae-95e7-a81b46590267 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:44.826495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdc988-9e70-4c20-83d3-356f4e4a3daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:44.827147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdc988-9e70-4c20-83d3-356f4e4a3daa service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:44.827432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdc988-9e70-4c20-83d3-356f4e4a3daa service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:44.827918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdc988-9e70-4c20-83d3-356f4e4a3daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:44.839273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdc988-9e70-4c20-83d3-356f4e4a3daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:44.840910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdc988-9e70-4c20-83d3-356f4e4a3daa service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:44.853984 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1510/3063] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:45.289335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1fc4c-a7f4-4145-9edc-2de90e93addc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:45.291348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-66c1fc4c-a7f4-4145-9edc-2de90e93addc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:45.308371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1fc4c-a7f4-4145-9edc-2de90e93addc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:45.314048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:45.315001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:45.315573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:45.316092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] PATH_INFO: `/v3/projects/e96ec8c49f384e7d81244fd2476d0496` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:45.317516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e96ec8c49f384e7d81244fd2476d0496)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:45.320103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:45.402584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e96ec8c49f384e7d81244fd2476d0496', 'request_id': 'req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:45.418847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e96ec8c49f384e7d81244fd2476d0496', 'request_id': 'req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:45.427827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e96ec8c49f384e7d81244fd2476d0496', 'request_id': 'req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:45.442972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] The token cache is being invalidate because project e96ec8c49f384e7d81244fd2476d0496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:56:45.443524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:45.444405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:45.449792 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1554/3064] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:56:45 2022] DELETE /identity/v3/projects/e96ec8c49f384e7d81244fd2476d0496 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:45.450820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdc988-9e70-4c20-83d3-356f4e4a3daa service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:45.470335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdc988-9e70-4c20-83d3-356f4e4a3daa service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:45.546418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:45.563622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7271a07-5018-4149-b6ef-279a4a41b543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:45.564716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7271a07-5018-4149-b6ef-279a4a41b543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:45.565214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7271a07-5018-4149-b6ef-279a4a41b543 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:45.565694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7271a07-5018-4149-b6ef-279a4a41b543 admin admin] PATH_INFO: `/v3/users/2f91eb6e45a14d1e8c18c9b94ae1567e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:45.567069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7271a07-5018-4149-b6ef-279a4a41b543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f91eb6e45a14d1e8c18c9b94ae1567e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:45.570081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7271a07-5018-4149-b6ef-279a4a41b543 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:45.644629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e7271a07-5018-4149-b6ef-279a4a41b543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f91eb6e45a14d1e8c18c9b94ae1567e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:45.665391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e7271a07-5018-4149-b6ef-279a4a41b543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f91eb6e45a14d1e8c18c9b94ae1567e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:45.667882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e7271a07-5018-4149-b6ef-279a4a41b543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f91eb6e45a14d1e8c18c9b94ae1567e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:45.677810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e7271a07-5018-4149-b6ef-279a4a41b543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f91eb6e45a14d1e8c18c9b94ae1567e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:45.689835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e7271a07-5018-4149-b6ef-279a4a41b543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f91eb6e45a14d1e8c18c9b94ae1567e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:45.703071 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1511/3065] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:56:45 2022] DELETE /identity/v3/users/2f91eb6e45a14d1e8c18c9b94ae1567e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:46.060951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:46.062548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:46.091011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6cc1d5-34b6-4d9e-b0c5-5c249517abcd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:46.250948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:46.251464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:46.251992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:46.252454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc admin admin] PATH_INFO: `/v3/projects/baf7e7f35c3f42b0a853dc2a0c196753` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:46.255908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=baf7e7f35c3f42b0a853dc2a0c196753)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:46.258902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:46.370302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baf7e7f35c3f42b0a853dc2a0c196753', 'request_id': 'req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:46.383567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baf7e7f35c3f42b0a853dc2a0c196753', 'request_id': 'req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:46.392785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baf7e7f35c3f42b0a853dc2a0c196753', 'request_id': 'req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:46.423417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:46.442156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc admin admin] The token cache is being invalidate because project baf7e7f35c3f42b0a853dc2a0c196753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:56:46.444800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:46.447528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cdc0f3f4-68c9-4b54-8302-411dc9c501fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:46.451236 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1555/3066] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:56:46 2022] DELETE /identity/v3/projects/baf7e7f35c3f42b0a853dc2a0c196753 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:46.457718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eb696-8a68-4124-be12-566f596c39a5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:46.458961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eb696-8a68-4124-be12-566f596c39a5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:46.458961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eb696-8a68-4124-be12-566f596c39a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:46.506565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-972eb696-8a68-4124-be12-566f596c39a5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:46.512178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-972eb696-8a68-4124-be12-566f596c39a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:46.669224 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1512/3067] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:46.678911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed46010-5a99-422c-9b22-a9e116f9654c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:46.679617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed46010-5a99-422c-9b22-a9e116f9654c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:46.680108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed46010-5a99-422c-9b22-a9e116f9654c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:46.723994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6ed46010-5a99-422c-9b22-a9e116f9654c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:46.726370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6ed46010-5a99-422c-9b22-a9e116f9654c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:46.757745 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1556/3068] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:46.760990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eb696-8a68-4124-be12-566f596c39a5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:46.775822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eb696-8a68-4124-be12-566f596c39a5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:46.789109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1618e6-abed-420c-a00d-48a206bd6cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:46.790551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1618e6-abed-420c-a00d-48a206bd6cfd admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:46.792981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1618e6-abed-420c-a00d-48a206bd6cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:46.793454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1618e6-abed-420c-a00d-48a206bd6cfd admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:46.794210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1618e6-abed-420c-a00d-48a206bd6cfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:56:46.794673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1618e6-abed-420c-a00d-48a206bd6cfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:46.799132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1618e6-abed-420c-a00d-48a206bd6cfd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:46.833101 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1513/3069] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:56:46.838404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed46010-5a99-422c-9b22-a9e116f9654c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:46.856073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed46010-5a99-422c-9b22-a9e116f9654c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:46.874662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e4b30-454b-49e7-81bb-caaef5e3634b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:46.876428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e4b30-454b-49e7-81bb-caaef5e3634b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:46.877356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e4b30-454b-49e7-81bb-caaef5e3634b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:46.878533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e4b30-454b-49e7-81bb-caaef5e3634b admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:46.880398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e4b30-454b-49e7-81bb-caaef5e3634b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:46.886625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e4b30-454b-49e7-81bb-caaef5e3634b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:47.021701 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1557/3070] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:56:46 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:56:47.026672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1618e6-abed-420c-a00d-48a206bd6cfd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:47.051370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1618e6-abed-420c-a00d-48a206bd6cfd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:47.098595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e4b30-454b-49e7-81bb-caaef5e3634b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:47.100576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-398e4b30-454b-49e7-81bb-caaef5e3634b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:47.291271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:47.316251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3415176-c668-49cb-9c9b-34ba8e37d0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:47.318401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3415176-c668-49cb-9c9b-34ba8e37d0bd admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:47.319139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3415176-c668-49cb-9c9b-34ba8e37d0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:47.319692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3415176-c668-49cb-9c9b-34ba8e37d0bd admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:47.323578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3415176-c668-49cb-9c9b-34ba8e37d0bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:47.328871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e4b30-454b-49e7-81bb-caaef5e3634b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:47.329342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3415176-c668-49cb-9c9b-34ba8e37d0bd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:47.360023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18132b7d-2678-46d5-bd4e-8bfa8bf20e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:47.363566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18132b7d-2678-46d5-bd4e-8bfa8bf20e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:47.364442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18132b7d-2678-46d5-bd4e-8bfa8bf20e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:47.365276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18132b7d-2678-46d5-bd4e-8bfa8bf20e29 admin admin] PATH_INFO: `/v3/users/2d129b2222f7408db35cae47dcb9a60a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:47.369579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18132b7d-2678-46d5-bd4e-8bfa8bf20e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d129b2222f7408db35cae47dcb9a60a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:47.372806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18132b7d-2678-46d5-bd4e-8bfa8bf20e29 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:47.401394 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1514/3071] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:56:47 2022] POST /identity/v3/users => generated 287 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:56:47.406231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3415176-c668-49cb-9c9b-34ba8e37d0bd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:47.425732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3415176-c668-49cb-9c9b-34ba8e37d0bd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:47.437981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15b64-8895-4c9a-a11f-d989bd23e79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:47.439098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15b64-8895-4c9a-a11f-d989bd23e79f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:47.439663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15b64-8895-4c9a-a11f-d989bd23e79f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:47.440363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15b64-8895-4c9a-a11f-d989bd23e79f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:47.441719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f15b64-8895-4c9a-a11f-d989bd23e79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:47.444102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f15b64-8895-4c9a-a11f-d989bd23e79f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:47.455204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-18132b7d-2678-46d5-bd4e-8bfa8bf20e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d129b2222f7408db35cae47dcb9a60a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:47.467655 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1515/3072] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:47.472771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15b64-8895-4c9a-a11f-d989bd23e79f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:47.488783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15b64-8895-4c9a-a11f-d989bd23e79f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:47.495736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-18132b7d-2678-46d5-bd4e-8bfa8bf20e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d129b2222f7408db35cae47dcb9a60a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:47.507725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e2454-bbd2-45fe-80f7-95e98a2063fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:47.508502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e2454-bbd2-45fe-80f7-95e98a2063fe admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:47.508948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e2454-bbd2-45fe-80f7-95e98a2063fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:47.509442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e2454-bbd2-45fe-80f7-95e98a2063fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:47.510695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e2454-bbd2-45fe-80f7-95e98a2063fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:47.511443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-18132b7d-2678-46d5-bd4e-8bfa8bf20e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d129b2222f7408db35cae47dcb9a60a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:47.512977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e2454-bbd2-45fe-80f7-95e98a2063fe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:47.513254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-18132b7d-2678-46d5-bd4e-8bfa8bf20e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d129b2222f7408db35cae47dcb9a60a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:47.514676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-18132b7d-2678-46d5-bd4e-8bfa8bf20e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d129b2222f7408db35cae47dcb9a60a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:47.528391 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1516/3073] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:56:47.533333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e2454-bbd2-45fe-80f7-95e98a2063fe admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:47.538205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0e7e2454-bbd2-45fe-80f7-95e98a2063fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:47.548439 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1558/3074] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:56:47 2022] DELETE /identity/v3/users/2d129b2222f7408db35cae47dcb9a60a => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:47.560244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab1ad5-08d3-444a-acea-6def0c759f45 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:47.561892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab1ad5-08d3-444a-acea-6def0c759f45 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:47.562374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab1ad5-08d3-444a-acea-6def0c759f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:47.566952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e2454-bbd2-45fe-80f7-95e98a2063fe admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:47.591436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-05ab1ad5-08d3-444a-acea-6def0c759f45 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:47.594567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-05ab1ad5-08d3-444a-acea-6def0c759f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:47.661770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:47.689408 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1559/3075] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:47.696041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a8167-4f44-4bc2-bcdd-0929bdd22e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:47.696719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a8167-4f44-4bc2-bcdd-0929bdd22e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:47.697056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a8167-4f44-4bc2-bcdd-0929bdd22e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:47.697400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a8167-4f44-4bc2-bcdd-0929bdd22e88 admin admin] PATH_INFO: `/v3/projects/9e0bd7ead5d747e7b16f0aeaa5cdbc04/users/e689ea75cc614441b0e4d3944675e7b3/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:47.720216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab1ad5-08d3-444a-acea-6def0c759f45 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:47.721568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a8167-4f44-4bc2-bcdd-0929bdd22e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0bd7ead5d747e7b16f0aeaa5cdbc04, user_id=e689ea75cc614441b0e4d3944675e7b3, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:47.724791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-05ab1ad5-08d3-444a-acea-6def0c759f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:47.725675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a8167-4f44-4bc2-bcdd-0929bdd22e88 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:47.756363 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1517/3076] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:56:47 2022] PUT /identity/v3/projects/9e0bd7ead5d747e7b16f0aeaa5cdbc04/users/e689ea75cc614441b0e4d3944675e7b3/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:47.768436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565dc53-e8d6-4235-8e97-4d2b574f1112 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:47.769588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565dc53-e8d6-4235-8e97-4d2b574f1112 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:47.770070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565dc53-e8d6-4235-8e97-4d2b574f1112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:47.822782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b565dc53-e8d6-4235-8e97-4d2b574f1112 None None] MFA Rules not processed for user `e689ea75cc614441b0e4d3944675e7b3`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:47.824776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b565dc53-e8d6-4235-8e97-4d2b574f1112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:47.859606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab1ad5-08d3-444a-acea-6def0c759f45 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:47.970316 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1518/3077] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:47.970561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3373fa-4606-4f2e-b24d-447c06748007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:47.971042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3373fa-4606-4f2e-b24d-447c06748007 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:47.971330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3373fa-4606-4f2e-b24d-447c06748007 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:47.971648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3373fa-4606-4f2e-b24d-447c06748007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:47.973306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565dc53-e8d6-4235-8e97-4d2b574f1112 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:48.045833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:48.086190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565dc53-e8d6-4235-8e97-4d2b574f1112 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:48.091597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949792c2-7b9a-40ff-b30a-4b559da79b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:48.091967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3373fa-4606-4f2e-b24d-447c06748007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:48.092248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949792c2-7b9a-40ff-b30a-4b559da79b75 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:48.092576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949792c2-7b9a-40ff-b30a-4b559da79b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:48.092931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949792c2-7b9a-40ff-b30a-4b559da79b75 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:48.093574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949792c2-7b9a-40ff-b30a-4b559da79b75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:48.094129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3373fa-4606-4f2e-b24d-447c06748007 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:48.095801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949792c2-7b9a-40ff-b30a-4b559da79b75 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:48.105610 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1560/3078] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:48.144587 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1519/3079] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:56:47 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:56:48.145773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3373fa-4606-4f2e-b24d-447c06748007 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:48.157655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3373fa-4606-4f2e-b24d-447c06748007 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:48.259375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:48.270443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6f782-889e-4b69-a4ad-7477b8fe0c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:48.271577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6f782-889e-4b69-a4ad-7477b8fe0c8f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:48.272039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6f782-889e-4b69-a4ad-7477b8fe0c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:48.272701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6f782-889e-4b69-a4ad-7477b8fe0c8f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:48.273953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6f782-889e-4b69-a4ad-7477b8fe0c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:48.276227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6f782-889e-4b69-a4ad-7477b8fe0c8f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:48.305207 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1561/3080] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:56:48 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:56:48.305859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949792c2-7b9a-40ff-b30a-4b559da79b75 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:48.318039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949792c2-7b9a-40ff-b30a-4b559da79b75 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:48.323317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db960be6-1b7b-4442-a9d0-7c0a9d0712f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:48.324013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db960be6-1b7b-4442-a9d0-7c0a9d0712f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:48.324365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db960be6-1b7b-4442-a9d0-7c0a9d0712f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:48.324710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db960be6-1b7b-4442-a9d0-7c0a9d0712f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:48.325470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db960be6-1b7b-4442-a9d0-7c0a9d0712f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:48.327138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db960be6-1b7b-4442-a9d0-7c0a9d0712f4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:48.341411 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1520/3081] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:48.342931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6f782-889e-4b69-a4ad-7477b8fe0c8f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:48.357722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6f782-889e-4b69-a4ad-7477b8fe0c8f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:48.365641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447ec80-a0c5-4916-a73a-383babe657f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:48.366467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447ec80-a0c5-4916-a73a-383babe657f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:48.366919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447ec80-a0c5-4916-a73a-383babe657f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:48.367221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447ec80-a0c5-4916-a73a-383babe657f4 admin admin] PATH_INFO: `/v3/projects/28fd3d4b280f40da96ac5f21cb33ece7/users/589e6d2ded624d9cb3e8c4437ace56ea/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:48.390056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447ec80-a0c5-4916-a73a-383babe657f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28fd3d4b280f40da96ac5f21cb33ece7, user_id=589e6d2ded624d9cb3e8c4437ace56ea, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:48.392830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447ec80-a0c5-4916-a73a-383babe657f4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:48.416874 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1562/3082] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:56:48 2022] PUT /identity/v3/projects/28fd3d4b280f40da96ac5f21cb33ece7/users/589e6d2ded624d9cb3e8c4437ace56ea/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:48.418017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db960be6-1b7b-4442-a9d0-7c0a9d0712f4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:48.450494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db960be6-1b7b-4442-a9d0-7c0a9d0712f4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:48.548728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50831660-dd51-4aed-b4e9-ec4ae0c19b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:48.550426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50831660-dd51-4aed-b4e9-ec4ae0c19b5f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:48.550426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50831660-dd51-4aed-b4e9-ec4ae0c19b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:48.550642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50831660-dd51-4aed-b4e9-ec4ae0c19b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:48.551653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50831660-dd51-4aed-b4e9-ec4ae0c19b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:48.553677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50831660-dd51-4aed-b4e9-ec4ae0c19b5f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:48.564884 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1521/3083] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:48.569109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447ec80-a0c5-4916-a73a-383babe657f4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:48.582337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447ec80-a0c5-4916-a73a-383babe657f4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:48.587152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09137d20-0e4a-44a3-9232-c3f38c462d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:48.588100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09137d20-0e4a-44a3-9232-c3f38c462d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:48.588525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09137d20-0e4a-44a3-9232-c3f38c462d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:48.588812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09137d20-0e4a-44a3-9232-c3f38c462d1e admin admin] PATH_INFO: `/v3/projects/28fd3d4b280f40da96ac5f21cb33ece7/users/589e6d2ded624d9cb3e8c4437ace56ea/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:48.590870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09137d20-0e4a-44a3-9232-c3f38c462d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=28fd3d4b280f40da96ac5f21cb33ece7, user_id=589e6d2ded624d9cb3e8c4437ace56ea, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:48.593928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09137d20-0e4a-44a3-9232-c3f38c462d1e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:48.614948 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1563/3084] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:56:48 2022] PUT /identity/v3/projects/28fd3d4b280f40da96ac5f21cb33ece7/users/589e6d2ded624d9cb3e8c4437ace56ea/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:56:48.689398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24bcdf-6cfb-4848-a3a4-d05961404104 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:48.689783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24bcdf-6cfb-4848-a3a4-d05961404104 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:48.690132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24bcdf-6cfb-4848-a3a4-d05961404104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:48.731059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4f24bcdf-6cfb-4848-a3a4-d05961404104 None None] MFA Rules not processed for user `589e6d2ded624d9cb3e8c4437ace56ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:48.733077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4f24bcdf-6cfb-4848-a3a4-d05961404104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:48.822298 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1522/3085] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:48.829782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09137d20-0e4a-44a3-9232-c3f38c462d1e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:48.847311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09137d20-0e4a-44a3-9232-c3f38c462d1e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:48.853071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf425a5-a1a6-4871-8b83-1b70b98c069a tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e6d2ded624d9cb3e8c4437ace56ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28fd3d4b280f40da96ac5f21cb33ece7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:48.853704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf425a5-a1a6-4871-8b83-1b70b98c069a tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:48.853993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf425a5-a1a6-4871-8b83-1b70b98c069a tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:48.854274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf425a5-a1a6-4871-8b83-1b70b98c069a tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:48.854952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf425a5-a1a6-4871-8b83-1b70b98c069a tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:56:48.855266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf425a5-a1a6-4871-8b83-1b70b98c069a tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:48.857406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf425a5-a1a6-4871-8b83-1b70b98c069a tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:48.874313 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1564/3086] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:56:48.878466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24bcdf-6cfb-4848-a3a4-d05961404104 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:48.894132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24bcdf-6cfb-4848-a3a4-d05961404104 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:48.903130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f665e-5717-424e-a3f6-c783a0bacff9 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e6d2ded624d9cb3e8c4437ace56ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28fd3d4b280f40da96ac5f21cb33ece7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:48.903874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f665e-5717-424e-a3f6-c783a0bacff9 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:48.904330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f665e-5717-424e-a3f6-c783a0bacff9 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:48.904598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f665e-5717-424e-a3f6-c783a0bacff9 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:48.906133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f665e-5717-424e-a3f6-c783a0bacff9 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:48.910724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f665e-5717-424e-a3f6-c783a0bacff9 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:48.954242 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1523/3087] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:56:48 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:56:48.960092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf425a5-a1a6-4871-8b83-1b70b98c069a tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:48.976871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf425a5-a1a6-4871-8b83-1b70b98c069a tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:49.058231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc5390-dbfd-4e74-b4fd-e524f9fc3a5e tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e6d2ded624d9cb3e8c4437ace56ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28fd3d4b280f40da96ac5f21cb33ece7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:49.058937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc5390-dbfd-4e74-b4fd-e524f9fc3a5e tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:49.059301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc5390-dbfd-4e74-b4fd-e524f9fc3a5e tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:49.059652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc5390-dbfd-4e74-b4fd-e524f9fc3a5e tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:49.060502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc5390-dbfd-4e74-b4fd-e524f9fc3a5e tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:56:49.060986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc5390-dbfd-4e74-b4fd-e524f9fc3a5e tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:49.062881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc5390-dbfd-4e74-b4fd-e524f9fc3a5e tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:49.079948 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1565/3088] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:56:49.830224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f665e-5717-424e-a3f6-c783a0bacff9 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:49.835055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b75f665e-5717-424e-a3f6-c783a0bacff9 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:49.866534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:50.022966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f665e-5717-424e-a3f6-c783a0bacff9 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:50.039201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:50.043328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:50.043773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:50.044155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] PATH_INFO: `/v3/projects/98285399acfb4823a65117e6e499fb9a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:50.048520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98285399acfb4823a65117e6e499fb9a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:50.052255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:50.140794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98285399acfb4823a65117e6e499fb9a', 'request_id': 'req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:50.165113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98285399acfb4823a65117e6e499fb9a', 'request_id': 'req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:50.178004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98285399acfb4823a65117e6e499fb9a', 'request_id': 'req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:50.203090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] The token cache is being invalidate because project 98285399acfb4823a65117e6e499fb9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:56:50.204046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:50.209452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:50.232034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598fb3b-832e-4b0e-808c-ec2584abb6d8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:50.232525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598fb3b-832e-4b0e-808c-ec2584abb6d8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:50.232876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598fb3b-832e-4b0e-808c-ec2584abb6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:50.238904 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1524/3089] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:56:49 2022] DELETE /identity/v3/projects/98285399acfb4823a65117e6e499fb9a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:50.257190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:50.262084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:50.285334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7598fb3b-832e-4b0e-808c-ec2584abb6d8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:50.287478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7598fb3b-832e-4b0e-808c-ec2584abb6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:50.450410 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1566/3090] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:50.452603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c7971-eb59-4e02-8137-1ecb50a31bae None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:50.452915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c7971-eb59-4e02-8137-1ecb50a31bae None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:50.453354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c7971-eb59-4e02-8137-1ecb50a31bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:50.490238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0fe78-3fcd-43a5-b457-c5c85f6a0366 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:50.495446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-377c7971-eb59-4e02-8137-1ecb50a31bae None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:50.496692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-377c7971-eb59-4e02-8137-1ecb50a31bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:50.507464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f954c39-5218-464c-ae95-aea77980d915 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:50.510204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f954c39-5218-464c-ae95-aea77980d915 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:50.510776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f954c39-5218-464c-ae95-aea77980d915 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:50.511336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f954c39-5218-464c-ae95-aea77980d915 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:50.519070 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1567/3091] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:56:50.520164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c7971-eb59-4e02-8137-1ecb50a31bae None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:50.524675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8f954c39-5218-464c-ae95-aea77980d915 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:50.530052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c7971-eb59-4e02-8137-1ecb50a31bae None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:50.548376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577ebbc-906b-4139-b3cf-dd6533451197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:50.549028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577ebbc-906b-4139-b3cf-dd6533451197 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:50.549313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577ebbc-906b-4139-b3cf-dd6533451197 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:50.550117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577ebbc-906b-4139-b3cf-dd6533451197 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:50.550776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577ebbc-906b-4139-b3cf-dd6533451197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:56:50.551072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577ebbc-906b-4139-b3cf-dd6533451197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:50.553103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577ebbc-906b-4139-b3cf-dd6533451197 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:50.584015 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1568/3092] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:56:50.585116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577ebbc-906b-4139-b3cf-dd6533451197 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:50.602254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577ebbc-906b-4139-b3cf-dd6533451197 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:50.619694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3a688-5794-4d2b-ab00-7ac1a47d58f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:50.620294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3a688-5794-4d2b-ab00-7ac1a47d58f4 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:50.620580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3a688-5794-4d2b-ab00-7ac1a47d58f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:50.621089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3a688-5794-4d2b-ab00-7ac1a47d58f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:50.621597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3a688-5794-4d2b-ab00-7ac1a47d58f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:50.623433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3a688-5794-4d2b-ab00-7ac1a47d58f4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:50.637632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:50.645040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:50.691600 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1569/3093] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:56:50 2022] POST /identity/v3/projects => generated 397 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:56:50.703354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3a688-5794-4d2b-ab00-7ac1a47d58f4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:50.712611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f954c39-5218-464c-ae95-aea77980d915 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:50.718501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f954c39-5218-464c-ae95-aea77980d915 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:50.770365 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1525/3094] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:50.805362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78824e-d9f4-4061-bfe0-8db98c7db2f7 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:50.805660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78824e-d9f4-4061-bfe0-8db98c7db2f7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:50.805992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78824e-d9f4-4061-bfe0-8db98c7db2f7 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:50.808161 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1526/3095] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:56:50.817436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78824e-d9f4-4061-bfe0-8db98c7db2f7 None None] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:56:50.829428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3a688-5794-4d2b-ab00-7ac1a47d58f4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:50.853433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9649c3-813c-42c1-9a80-e18e5c6281bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:50.854253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9649c3-813c-42c1-9a80-e18e5c6281bb admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:50.854388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9649c3-813c-42c1-9a80-e18e5c6281bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:50.854745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9649c3-813c-42c1-9a80-e18e5c6281bb admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:50.855385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9649c3-813c-42c1-9a80-e18e5c6281bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:50.864741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9649c3-813c-42c1-9a80-e18e5c6281bb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:50.948776 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1570/3096] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:56:50 2022] POST /identity/v3/users => generated 299 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:56:50.953217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9649c3-813c-42c1-9a80-e18e5c6281bb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:50.983872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9649c3-813c-42c1-9a80-e18e5c6281bb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:50.996873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad827f-5ddb-488d-bfa3-cf23784ad4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:50.997963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad827f-5ddb-488d-bfa3-cf23784ad4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:50.998416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad827f-5ddb-488d-bfa3-cf23784ad4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:50.999480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad827f-5ddb-488d-bfa3-cf23784ad4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:51.000612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad827f-5ddb-488d-bfa3-cf23784ad4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:51.003184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad827f-5ddb-488d-bfa3-cf23784ad4c8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:51.019501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78824e-d9f4-4061-bfe0-8db98c7db2f7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:51.020331 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1571/3097] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:51.023747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad827f-5ddb-488d-bfa3-cf23784ad4c8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:51.038470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad827f-5ddb-488d-bfa3-cf23784ad4c8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:51.048227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70a2dc-2747-4eee-8f2f-b02698778b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:51.050240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70a2dc-2747-4eee-8f2f-b02698778b54 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:51.050652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70a2dc-2747-4eee-8f2f-b02698778b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:51.051359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70a2dc-2747-4eee-8f2f-b02698778b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:51.052003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70a2dc-2747-4eee-8f2f-b02698778b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:51.054098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70a2dc-2747-4eee-8f2f-b02698778b54 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:51.067202 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1572/3098] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:51.071127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70a2dc-2747-4eee-8f2f-b02698778b54 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:51.092042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70a2dc-2747-4eee-8f2f-b02698778b54 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:51.101765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910b0d0-f311-4e5a-b2ac-be693e41e327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:51.102757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910b0d0-f311-4e5a-b2ac-be693e41e327 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:51.102954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910b0d0-f311-4e5a-b2ac-be693e41e327 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:51.103235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910b0d0-f311-4e5a-b2ac-be693e41e327 admin admin] PATH_INFO: `/v3/projects/e033e783340b44719f9f77e6d1671a00/users/bb1def1d544546d198ee70cab711964d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:51.137816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910b0d0-f311-4e5a-b2ac-be693e41e327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e033e783340b44719f9f77e6d1671a00, user_id=bb1def1d544546d198ee70cab711964d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:51.143752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910b0d0-f311-4e5a-b2ac-be693e41e327 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:51.173819 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1573/3099] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:56:51 2022] PUT /identity/v3/projects/e033e783340b44719f9f77e6d1671a00/users/bb1def1d544546d198ee70cab711964d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:51.174217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78824e-d9f4-4061-bfe0-8db98c7db2f7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:51.187201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c595984-ea63-42db-bc1e-1402fc60d70a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:51.187553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c595984-ea63-42db-bc1e-1402fc60d70a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:51.187767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c595984-ea63-42db-bc1e-1402fc60d70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:51.244990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0c595984-ea63-42db-bc1e-1402fc60d70a None None] MFA Rules not processed for user `bb1def1d544546d198ee70cab711964d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:51.246510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0c595984-ea63-42db-bc1e-1402fc60d70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:51.270366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac3d1a-f9d3-463d-af3d-7f6e4ee07054 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e6d2ded624d9cb3e8c4437ace56ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28fd3d4b280f40da96ac5f21cb33ece7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:51.271021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac3d1a-f9d3-463d-af3d-7f6e4ee07054 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:51.271440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac3d1a-f9d3-463d-af3d-7f6e4ee07054 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:51.271717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac3d1a-f9d3-463d-af3d-7f6e4ee07054 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] PATH_INFO: `/v3/projects/aa5ca21ed2ba4455850119dab62e661c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:51.273159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac3d1a-f9d3-463d-af3d-7f6e4ee07054 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorizing `identity:get_project(project_id=aa5ca21ed2ba4455850119dab62e661c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:51.276086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac3d1a-f9d3-463d-af3d-7f6e4ee07054 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:51.279413 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1527/3100] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:56:50 2022] GET /identity/v3/projects/aa5ca21ed2ba4455850119dab62e661c => generated 371 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:56:51.461919 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1574/3101] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:51 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:53.055320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0e684-04c3-421a-8516-59e95f6a6a8b None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:53.055904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0e684-04c3-421a-8516-59e95f6a6a8b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:53.057912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0e684-04c3-421a-8516-59e95f6a6a8b None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:53.060703 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1528/3102] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:56:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:56:53.066061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c595984-ea63-42db-bc1e-1402fc60d70a None None] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:56:53.110817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:53.253951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c595984-ea63-42db-bc1e-1402fc60d70a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:53.269468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c595984-ea63-42db-bc1e-1402fc60d70a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:53.276232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3148d1-b5e3-406a-91d6-e715232033b3 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e6d2ded624d9cb3e8c4437ace56ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28fd3d4b280f40da96ac5f21cb33ece7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:53.277228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3148d1-b5e3-406a-91d6-e715232033b3 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:53.277524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3148d1-b5e3-406a-91d6-e715232033b3 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:53.278370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3148d1-b5e3-406a-91d6-e715232033b3 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] PATH_INFO: `/v3/projects/aa5ca21ed2ba4455850119dab62e661c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:53.279585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3148d1-b5e3-406a-91d6-e715232033b3 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorizing `identity:get_project(project_id=aa5ca21ed2ba4455850119dab62e661c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:53.282036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3148d1-b5e3-406a-91d6-e715232033b3 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:53.298339 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1575/3103] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:56:53 2022] GET /identity/v3/projects/aa5ca21ed2ba4455850119dab62e661c => generated 371 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:56:53.348667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0e684-04c3-421a-8516-59e95f6a6a8b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:53.380462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0e684-04c3-421a-8516-59e95f6a6a8b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:53.388001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5968f4-be3e-4eb7-9dc3-238f5e30c7b2 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e6d2ded624d9cb3e8c4437ace56ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28fd3d4b280f40da96ac5f21cb33ece7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:53.389507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5968f4-be3e-4eb7-9dc3-238f5e30c7b2 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:53.389975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5968f4-be3e-4eb7-9dc3-238f5e30c7b2 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:53.390365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5968f4-be3e-4eb7-9dc3-238f5e30c7b2 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:53.395568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5968f4-be3e-4eb7-9dc3-238f5e30c7b2 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:56:53.396002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5968f4-be3e-4eb7-9dc3-238f5e30c7b2 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:53.397836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5968f4-be3e-4eb7-9dc3-238f5e30c7b2 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:53.418571 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1529/3104] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:56:53.423966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3148d1-b5e3-406a-91d6-e715232033b3 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:53.442535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3148d1-b5e3-406a-91d6-e715232033b3 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:53.449402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5045bb0-86df-4abc-8a1e-c47944b6b632 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e6d2ded624d9cb3e8c4437ace56ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28fd3d4b280f40da96ac5f21cb33ece7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:53.450294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5045bb0-86df-4abc-8a1e-c47944b6b632 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:53.450760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5045bb0-86df-4abc-8a1e-c47944b6b632 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:53.451201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5045bb0-86df-4abc-8a1e-c47944b6b632 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:53.452305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5045bb0-86df-4abc-8a1e-c47944b6b632 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:53.454642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5045bb0-86df-4abc-8a1e-c47944b6b632 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:53.492515 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1576/3105] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:56:53 2022] POST /identity/v3/users => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:56:53.493498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5968f4-be3e-4eb7-9dc3-238f5e30c7b2 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:53.506070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5968f4-be3e-4eb7-9dc3-238f5e30c7b2 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:53.507444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:53.517939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413682aa-1153-4431-a364-54a1c06c7f41 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e6d2ded624d9cb3e8c4437ace56ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28fd3d4b280f40da96ac5f21cb33ece7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:53.518723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413682aa-1153-4431-a364-54a1c06c7f41 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:53.519018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413682aa-1153-4431-a364-54a1c06c7f41 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:53.519304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413682aa-1153-4431-a364-54a1c06c7f41 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:53.520059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413682aa-1153-4431-a364-54a1c06c7f41 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:56:53.520431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413682aa-1153-4431-a364-54a1c06c7f41 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:53.522163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413682aa-1153-4431-a364-54a1c06c7f41 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:53.565031 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1530/3106] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:56:53.588488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5d8f0-6508-486d-812d-3e83951dcb41 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:53.588874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5d8f0-6508-486d-812d-3e83951dcb41 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:53.589939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5d8f0-6508-486d-812d-3e83951dcb41 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:53.595177 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1577/3107] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:56:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:56:53.604308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413682aa-1153-4431-a364-54a1c06c7f41 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:56:53.639446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413682aa-1153-4431-a364-54a1c06c7f41 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:53.652380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413682aa-1153-4431-a364-54a1c06c7f41 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:53.662427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5372e-a800-4bf8-b6c1-117af1223501 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e6d2ded624d9cb3e8c4437ace56ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28fd3d4b280f40da96ac5f21cb33ece7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:53.664283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5372e-a800-4bf8-b6c1-117af1223501 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:53.665616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5372e-a800-4bf8-b6c1-117af1223501 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:53.666106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5372e-a800-4bf8-b6c1-117af1223501 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] PATH_INFO: `/v3/projects/aa5ca21ed2ba4455850119dab62e661c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:53.688393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5372e-a800-4bf8-b6c1-117af1223501 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorizing `identity:get_project(project_id=aa5ca21ed2ba4455850119dab62e661c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:53.691993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5372e-a800-4bf8-b6c1-117af1223501 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:53.704409 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1531/3108] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:56:53 2022] GET /identity/v3/projects/aa5ca21ed2ba4455850119dab62e661c => generated 371 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:56:54.512345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cbe14-a595-4392-99c4-ad7a51260ed5 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:54.512775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cbe14-a595-4392-99c4-ad7a51260ed5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:54.513092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cbe14-a595-4392-99c4-ad7a51260ed5 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:54.514615 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1578/3109] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:56:54.525212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5372e-a800-4bf8-b6c1-117af1223501 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:56:54.548595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5372e-a800-4bf8-b6c1-117af1223501 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:54.561450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5372e-a800-4bf8-b6c1-117af1223501 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:54.569635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f8415-16bf-44e5-91f4-bae7e187a70f tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e6d2ded624d9cb3e8c4437ace56ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28fd3d4b280f40da96ac5f21cb33ece7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:54.571722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f8415-16bf-44e5-91f4-bae7e187a70f tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:54.572366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f8415-16bf-44e5-91f4-bae7e187a70f tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:54.573195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f8415-16bf-44e5-91f4-bae7e187a70f tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] PATH_INFO: `/v3/projects/aa5ca21ed2ba4455850119dab62e661c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:54.578962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f8415-16bf-44e5-91f4-bae7e187a70f tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorizing `identity:get_project(project_id=aa5ca21ed2ba4455850119dab62e661c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:54.590088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f8415-16bf-44e5-91f4-bae7e187a70f tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:54.593331 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1532/3110] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:56:54 2022] GET /identity/v3/projects/aa5ca21ed2ba4455850119dab62e661c => generated 371 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:56:54.678690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cbe14-a595-4392-99c4-ad7a51260ed5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:54.713851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cbe14-a595-4392-99c4-ad7a51260ed5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:54.739552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e6d2ded624d9cb3e8c4437ace56ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28fd3d4b280f40da96ac5f21cb33ece7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:54.740225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:54.740520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:54.746453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] PATH_INFO: `/v3/users/417d593ea3c649f4ae88696e9dc7f74d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:54.785389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorizing `identity:delete_user(user_id=417d593ea3c649f4ae88696e9dc7f74d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:54.790662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:54.908112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '417d593ea3c649f4ae88696e9dc7f74d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:54.929588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '417d593ea3c649f4ae88696e9dc7f74d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:54.976656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '417d593ea3c649f4ae88696e9dc7f74d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:54.977915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '417d593ea3c649f4ae88696e9dc7f74d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:54.978982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '417d593ea3c649f4ae88696e9dc7f74d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:54.999467 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1579/3111] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:56:54 2022] DELETE /identity/v3/users/417d593ea3c649f4ae88696e9dc7f74d => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:55.003735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f8415-16bf-44e5-91f4-bae7e187a70f tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:55.005881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8b7f8415-16bf-44e5-91f4-bae7e187a70f tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:55.023654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f8415-16bf-44e5-91f4-bae7e187a70f tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:55.178603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e6d2ded624d9cb3e8c4437ace56ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28fd3d4b280f40da96ac5f21cb33ece7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:55.179848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:55.180519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:55.181235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] PATH_INFO: `/v3/projects/aa5ca21ed2ba4455850119dab62e661c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:55.182885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorizing `identity:delete_project(project_id=aa5ca21ed2ba4455850119dab62e661c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:55.186136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:55.286649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa5ca21ed2ba4455850119dab62e661c', 'request_id': 'req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:55.308937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa5ca21ed2ba4455850119dab62e661c', 'request_id': 'req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:55.320927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa5ca21ed2ba4455850119dab62e661c', 'request_id': 'req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:55.348179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] The token cache is being invalidate because project aa5ca21ed2ba4455850119dab62e661c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:56:55.349096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:55.350543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:55.354889 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1533/3112] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:56:55 2022] DELETE /identity/v3/projects/aa5ca21ed2ba4455850119dab62e661c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:55.361045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:55.363154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:55.452710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:55.456771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:55.467129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:55.510961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ffdb9-6286-49d3-a38d-8f6f8db44480 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:55.612401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:55.615796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb00a96-3e0c-4859-a5dc-e39f345fa5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:55.616473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb00a96-3e0c-4859-a5dc-e39f345fa5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:55.616764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb00a96-3e0c-4859-a5dc-e39f345fa5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:55.617123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb00a96-3e0c-4859-a5dc-e39f345fa5ab admin admin] PATH_INFO: `/v3/users/e689ea75cc614441b0e4d3944675e7b3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:55.620018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb00a96-3e0c-4859-a5dc-e39f345fa5ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e689ea75cc614441b0e4d3944675e7b3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:55.622057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb00a96-3e0c-4859-a5dc-e39f345fa5ab admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:55.626472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25d2fc-a837-4a50-ae30-cf3cca9ec270 tempest-QuotasAdminTestV257-1335190500 tempest-QuotasAdminTestV257-1335190500-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:55.636850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81742926-7ff8-43ff-96d3-0373bcea03a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:55.638112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81742926-7ff8-43ff-96d3-0373bcea03a3 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:55.638705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81742926-7ff8-43ff-96d3-0373bcea03a3 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:55.639242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81742926-7ff8-43ff-96d3-0373bcea03a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:55.641868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-81742926-7ff8-43ff-96d3-0373bcea03a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:55.719694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbb00a96-3e0c-4859-a5dc-e39f345fa5ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e689ea75cc614441b0e4d3944675e7b3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:55.740525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbb00a96-3e0c-4859-a5dc-e39f345fa5ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e689ea75cc614441b0e4d3944675e7b3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:55.765794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbb00a96-3e0c-4859-a5dc-e39f345fa5ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e689ea75cc614441b0e4d3944675e7b3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:55.766607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbb00a96-3e0c-4859-a5dc-e39f345fa5ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e689ea75cc614441b0e4d3944675e7b3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:55.768222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbb00a96-3e0c-4859-a5dc-e39f345fa5ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e689ea75cc614441b0e4d3944675e7b3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:55.774490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81742926-7ff8-43ff-96d3-0373bcea03a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:55.776012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81742926-7ff8-43ff-96d3-0373bcea03a3 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:55.790011 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1534/3113] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:55 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:55.798687 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1580/3114] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:56:55 2022] DELETE /identity/v3/users/e689ea75cc614441b0e4d3944675e7b3 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:55.803625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3283292-c300-4e69-9103-2f2ba22640be None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:55.803915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3283292-c300-4e69-9103-2f2ba22640be None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:55.804187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3283292-c300-4e69-9103-2f2ba22640be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:55.835083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f3283292-c300-4e69-9103-2f2ba22640be None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:55.837110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f3283292-c300-4e69-9103-2f2ba22640be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:55.923225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edb309-3364-4be4-8728-a8ad3a5624d8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:55.923849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edb309-3364-4be4-8728-a8ad3a5624d8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:55.925010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edb309-3364-4be4-8728-a8ad3a5624d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:55.972440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a6edb309-3364-4be4-8728-a8ad3a5624d8 None None] MFA Rules not processed for user `bb1def1d544546d198ee70cab711964d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:55.972986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a6edb309-3364-4be4-8728-a8ad3a5624d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:55.996519 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1535/3115] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:55.996519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3283292-c300-4e69-9103-2f2ba22640be None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:55.999323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f3283292-c300-4e69-9103-2f2ba22640be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:56.047870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3283292-c300-4e69-9103-2f2ba22640be None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:56.065861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:56.066860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:56.067354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:56.067841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] PATH_INFO: `/v3/users/27be6827d30e48acac9a59786021f7bb` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:56.069566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27be6827d30e48acac9a59786021f7bb)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:56.088932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:56.156182 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1581/3116] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:55 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:56.157304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edb309-3364-4be4-8728-a8ad3a5624d8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:56.175754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a6edb309-3364-4be4-8728-a8ad3a5624d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:56.207239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27be6827d30e48acac9a59786021f7bb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:56.237699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:56.240556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27be6827d30e48acac9a59786021f7bb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:56.241469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27be6827d30e48acac9a59786021f7bb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:56.261038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27be6827d30e48acac9a59786021f7bb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:56.285259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27be6827d30e48acac9a59786021f7bb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:56.303758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edb309-3364-4be4-8728-a8ad3a5624d8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:56.308622 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1536/3117] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:56:55 2022] DELETE /identity/v3/users/27be6827d30e48acac9a59786021f7bb => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:56.310647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fff0f8-cecb-41e1-aa48-e2439395c8fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:56.311900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fff0f8-cecb-41e1-aa48-e2439395c8fd service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:56.313310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fff0f8-cecb-41e1-aa48-e2439395c8fd service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:56.313763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fff0f8-cecb-41e1-aa48-e2439395c8fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:56.317060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-23fff0f8-cecb-41e1-aa48-e2439395c8fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:56.336324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fff0f8-cecb-41e1-aa48-e2439395c8fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:56.355514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fff0f8-cecb-41e1-aa48-e2439395c8fd service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:56.399934 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1582/3118] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:57.167851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:57.170766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:57.189102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:57.383179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc78ca-cc5d-4d85-957c-fac5308c0ed7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:57.393174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:57.396033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:57.396447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:57.396848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] PATH_INFO: `/v3/projects/dad6ee8212ab418cba7f718ce4bb4726` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:57.399275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad6ee8212ab418cba7f718ce4bb4726)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:57.403237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:57.477917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad6ee8212ab418cba7f718ce4bb4726', 'request_id': 'req-355205b3-2d39-47fa-98de-bf796c966f96'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:57.490614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad6ee8212ab418cba7f718ce4bb4726', 'request_id': 'req-355205b3-2d39-47fa-98de-bf796c966f96'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:57.503315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad6ee8212ab418cba7f718ce4bb4726', 'request_id': 'req-355205b3-2d39-47fa-98de-bf796c966f96'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:57.524488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] The token cache is being invalidate because project dad6ee8212ab418cba7f718ce4bb4726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:56:57.524932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:57.525720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:57.532395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa328c-6655-4e7e-acf3-b7a11556f41b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:57.532885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa328c-6655-4e7e-acf3-b7a11556f41b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:57.533198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa328c-6655-4e7e-acf3-b7a11556f41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:57.536755 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1537/3119] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:56:57 2022] DELETE /identity/v3/projects/dad6ee8212ab418cba7f718ce4bb4726 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:57.556975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e1fa328c-6655-4e7e-acf3-b7a11556f41b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:57.558749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e1fa328c-6655-4e7e-acf3-b7a11556f41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:57.593199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:57.595051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:57.770000 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1583/3120] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:57.771109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa328c-6655-4e7e-acf3-b7a11556f41b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:57.774404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e1fa328c-6655-4e7e-acf3-b7a11556f41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:57.785134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355205b3-2d39-47fa-98de-bf796c966f96 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:57.796370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27951e-bc6d-40ee-ae3f-a09cb4d39b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:57.798090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27951e-bc6d-40ee-ae3f-a09cb4d39b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:57.800123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27951e-bc6d-40ee-ae3f-a09cb4d39b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:57.800668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27951e-bc6d-40ee-ae3f-a09cb4d39b1f admin admin] PATH_INFO: `/v3/users/bb1def1d544546d198ee70cab711964d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:57.804414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27951e-bc6d-40ee-ae3f-a09cb4d39b1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb1def1d544546d198ee70cab711964d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:57.810547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27951e-bc6d-40ee-ae3f-a09cb4d39b1f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:57.912155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:57.914172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9c27951e-bc6d-40ee-ae3f-a09cb4d39b1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb1def1d544546d198ee70cab711964d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:57.932338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa328c-6655-4e7e-acf3-b7a11556f41b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:57.939026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528466a4-e1a9-47f9-b328-6ade7180c25f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:57.939989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528466a4-e1a9-47f9-b328-6ade7180c25f service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:57.941278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528466a4-e1a9-47f9-b328-6ade7180c25f service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:57.941734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528466a4-e1a9-47f9-b328-6ade7180c25f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:57.942772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9c27951e-bc6d-40ee-ae3f-a09cb4d39b1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb1def1d544546d198ee70cab711964d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:57.944129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-528466a4-e1a9-47f9-b328-6ade7180c25f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:57.944836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9c27951e-bc6d-40ee-ae3f-a09cb4d39b1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb1def1d544546d198ee70cab711964d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:57.958566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528466a4-e1a9-47f9-b328-6ade7180c25f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:57.960688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528466a4-e1a9-47f9-b328-6ade7180c25f service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:57.967831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9c27951e-bc6d-40ee-ae3f-a09cb4d39b1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb1def1d544546d198ee70cab711964d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:57.973694 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1584/3121] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:57.975496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c704a9-4c88-4219-b2d0-6601b1c16f6e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:57.976074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c704a9-4c88-4219-b2d0-6601b1c16f6e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:57.976548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c704a9-4c88-4219-b2d0-6601b1c16f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:57.982640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9c27951e-bc6d-40ee-ae3f-a09cb4d39b1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb1def1d544546d198ee70cab711964d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:57.995472 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1538/3122] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:56:57 2022] DELETE /identity/v3/users/bb1def1d544546d198ee70cab711964d => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:58.000284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7e7ed-3516-42f5-9819-1d69ea555360 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:58.000666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7e7ed-3516-42f5-9819-1d69ea555360 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:58.001206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-22c704a9-4c88-4219-b2d0-6601b1c16f6e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:58.003657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-22c704a9-4c88-4219-b2d0-6601b1c16f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:58.007442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7e7ed-3516-42f5-9819-1d69ea555360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:58.060848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-86a7e7ed-3516-42f5-9819-1d69ea555360 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:58.062363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-86a7e7ed-3516-42f5-9819-1d69ea555360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:58.066972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:58.078351 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1585/3123] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:58.082962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c704a9-4c88-4219-b2d0-6601b1c16f6e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:58.105550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c704a9-4c88-4219-b2d0-6601b1c16f6e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:58.116633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4e5a6-5777-4771-b0ce-6da5df66ca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:58.117721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4e5a6-5777-4771-b0ce-6da5df66ca03 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:58.118277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4e5a6-5777-4771-b0ce-6da5df66ca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:58.118835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4e5a6-5777-4771-b0ce-6da5df66ca03 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:58.119645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4e5a6-5777-4771-b0ce-6da5df66ca03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:56:58.120066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4e5a6-5777-4771-b0ce-6da5df66ca03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:58.122158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4e5a6-5777-4771-b0ce-6da5df66ca03 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:58.138578 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1586/3124] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:56:58.142417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4e5a6-5777-4771-b0ce-6da5df66ca03 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:58.160449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4e5a6-5777-4771-b0ce-6da5df66ca03 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:58.166881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a00fb4-fa03-48bb-ba8b-8af54400caad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:58.168438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a00fb4-fa03-48bb-ba8b-8af54400caad admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:58.168438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a00fb4-fa03-48bb-ba8b-8af54400caad admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:58.168438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a00fb4-fa03-48bb-ba8b-8af54400caad admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:58.169536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a00fb4-fa03-48bb-ba8b-8af54400caad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:58.173111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a00fb4-fa03-48bb-ba8b-8af54400caad admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:58.216381 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1587/3125] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:56:58 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:56:58.218359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a00fb4-fa03-48bb-ba8b-8af54400caad admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:58.221049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c5a00fb4-fa03-48bb-ba8b-8af54400caad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:58.221331 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1539/3126] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:58.222492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7e7ed-3516-42f5-9819-1d69ea555360 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:58.295754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7e7ed-3516-42f5-9819-1d69ea555360 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:58.345541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a00fb4-fa03-48bb-ba8b-8af54400caad admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:58.354120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:58.355720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:58.356015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:58.356299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] PATH_INFO: `/v3/projects/9e0bd7ead5d747e7b16f0aeaa5cdbc04` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:58.357891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0bd7ead5d747e7b16f0aeaa5cdbc04)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:58.359848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:58.448823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24fdd5-be77-43f3-9ce9-fd8bdbfbd5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:58.450415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0bd7ead5d747e7b16f0aeaa5cdbc04', 'request_id': 'req-b35caf4f-e788-4ace-bed8-4160473da7c5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:58.451006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24fdd5-be77-43f3-9ce9-fd8bdbfbd5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:58.451381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24fdd5-be77-43f3-9ce9-fd8bdbfbd5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:58.452105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24fdd5-be77-43f3-9ce9-fd8bdbfbd5e1 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:58.452841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24fdd5-be77-43f3-9ce9-fd8bdbfbd5e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:58.455871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24fdd5-be77-43f3-9ce9-fd8bdbfbd5e1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:58.463097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0bd7ead5d747e7b16f0aeaa5cdbc04', 'request_id': 'req-b35caf4f-e788-4ace-bed8-4160473da7c5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:58.475079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0bd7ead5d747e7b16f0aeaa5cdbc04', 'request_id': 'req-b35caf4f-e788-4ace-bed8-4160473da7c5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:58.491268 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1540/3127] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:56:58 2022] POST /identity/v3/users => generated 292 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:56:58.491268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24fdd5-be77-43f3-9ce9-fd8bdbfbd5e1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:58.495216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cb24fdd5-be77-43f3-9ce9-fd8bdbfbd5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:58.499596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] The token cache is being invalidate because project 9e0bd7ead5d747e7b16f0aeaa5cdbc04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:56:58.499884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:58.500636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:58.505457 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1588/3128] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:56:58 2022] DELETE /identity/v3/projects/9e0bd7ead5d747e7b16f0aeaa5cdbc04 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:58.505457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:58.505689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:58.644720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:58.649630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24fdd5-be77-43f3-9ce9-fd8bdbfbd5e1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:58.655697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:58.762642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35caf4f-e788-4ace-bed8-4160473da7c5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:58.775486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:58.775486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:58.775486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:58.775486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] PATH_INFO: `/v3/users/c8a88f53cc78444299e4c462e004edf9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:58.776050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a88f53cc78444299e4c462e004edf9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:58.777924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:58.785784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51638bad-ca50-441c-9b5f-011a1fea3388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:58.787401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51638bad-ca50-441c-9b5f-011a1fea3388 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:58.788079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51638bad-ca50-441c-9b5f-011a1fea3388 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:58.790450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51638bad-ca50-441c-9b5f-011a1fea3388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:58.793857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-51638bad-ca50-441c-9b5f-011a1fea3388 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:58.823899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51638bad-ca50-441c-9b5f-011a1fea3388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:58.825605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51638bad-ca50-441c-9b5f-011a1fea3388 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:58.840824 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1541/3129] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:56:58.841990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51638bad-ca50-441c-9b5f-011a1fea3388 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:58.849102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-51638bad-ca50-441c-9b5f-011a1fea3388 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:58.877406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a88f53cc78444299e4c462e004edf9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:58.880758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51638bad-ca50-441c-9b5f-011a1fea3388 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:58.896577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ccde6-e414-4d64-a241-54aa183d2a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:58.897408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ccde6-e414-4d64-a241-54aa183d2a17 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:58.898205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ccde6-e414-4d64-a241-54aa183d2a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:58.898205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ccde6-e414-4d64-a241-54aa183d2a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:58.899115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ccde6-e414-4d64-a241-54aa183d2a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:58.901818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ccde6-e414-4d64-a241-54aa183d2a17 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:58.904506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a88f53cc78444299e4c462e004edf9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:58.918046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a88f53cc78444299e4c462e004edf9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:58.918731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a88f53cc78444299e4c462e004edf9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:58.919102 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1542/3130] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:58.920052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ccde6-e414-4d64-a241-54aa183d2a17 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:58.921380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a88f53cc78444299e4c462e004edf9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:58.922751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-016ccde6-e414-4d64-a241-54aa183d2a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:58.939314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ccde6-e414-4d64-a241-54aa183d2a17 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:58.940844 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1589/3131] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:56:58 2022] DELETE /identity/v3/users/c8a88f53cc78444299e4c462e004edf9 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:58.941794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:58.943533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:59.026177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54eebd-d419-4684-af97-e46f3b19d142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:59.027234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54eebd-d419-4684-af97-e46f3b19d142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:59.027525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54eebd-d419-4684-af97-e46f3b19d142 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:59.027914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54eebd-d419-4684-af97-e46f3b19d142 admin admin] PATH_INFO: `/v3/users/589e6d2ded624d9cb3e8c4437ace56ea` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:59.029220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54eebd-d419-4684-af97-e46f3b19d142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=589e6d2ded624d9cb3e8c4437ace56ea)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:59.031280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54eebd-d419-4684-af97-e46f3b19d142 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:59.103100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc4e23-df34-4292-af3c-0d0124471a56 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:59.111077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b8932-8b59-49d6-80d4-acaf949797e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:59.113208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b8932-8b59-49d6-80d4-acaf949797e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:59.113718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b8932-8b59-49d6-80d4-acaf949797e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:59.114205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b8932-8b59-49d6-80d4-acaf949797e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:59.115481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b8932-8b59-49d6-80d4-acaf949797e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:59.117842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b8932-8b59-49d6-80d4-acaf949797e1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:59.134584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f54eebd-d419-4684-af97-e46f3b19d142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '589e6d2ded624d9cb3e8c4437ace56ea'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:59.135538 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1590/3132] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:56:59.140354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b8932-8b59-49d6-80d4-acaf949797e1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:59.155639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b8932-8b59-49d6-80d4-acaf949797e1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:59.162324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9828dc-329a-4bb4-972f-ac42d26a7f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:59.163294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9828dc-329a-4bb4-972f-ac42d26a7f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:59.163783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9828dc-329a-4bb4-972f-ac42d26a7f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:59.164457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9828dc-329a-4bb4-972f-ac42d26a7f20 admin admin] PATH_INFO: `/v3/projects/bcdfcb6afebd476ca5781f53008fe1c8/users/d4bb698e4a2e4598acdb662511ce9eeb/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:59.181799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f54eebd-d419-4684-af97-e46f3b19d142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '589e6d2ded624d9cb3e8c4437ace56ea'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:59.182620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f54eebd-d419-4684-af97-e46f3b19d142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '589e6d2ded624d9cb3e8c4437ace56ea'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:59.187686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9828dc-329a-4bb4-972f-ac42d26a7f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcdfcb6afebd476ca5781f53008fe1c8, user_id=d4bb698e4a2e4598acdb662511ce9eeb, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:59.192164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f54eebd-d419-4684-af97-e46f3b19d142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '589e6d2ded624d9cb3e8c4437ace56ea'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:59.192727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9828dc-329a-4bb4-972f-ac42d26a7f20 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:59.205496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f54eebd-d419-4684-af97-e46f3b19d142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '589e6d2ded624d9cb3e8c4437ace56ea'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:56:59.223305 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1543/3133] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:56:58 2022] DELETE /identity/v3/users/589e6d2ded624d9cb3e8c4437ace56ea => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:59.225157 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1591/3134] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:56:59 2022] PUT /identity/v3/projects/bcdfcb6afebd476ca5781f53008fe1c8/users/d4bb698e4a2e4598acdb662511ce9eeb/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:56:59.240566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996f1a-cdf7-4ae5-88df-9132e6e293e5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:59.241805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996f1a-cdf7-4ae5-88df-9132e6e293e5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:59.242609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996f1a-cdf7-4ae5-88df-9132e6e293e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:59.289183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-44996f1a-cdf7-4ae5-88df-9132e6e293e5 None None] MFA Rules not processed for user `d4bb698e4a2e4598acdb662511ce9eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:56:59.290466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-44996f1a-cdf7-4ae5-88df-9132e6e293e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:59.302071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:59.423303 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1544/3135] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:56:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:56:59.731544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9828dc-329a-4bb4-972f-ac42d26a7f20 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:59.735904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fa9828dc-329a-4bb4-972f-ac42d26a7f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:59.782906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44996f1a-cdf7-4ae5-88df-9132e6e293e5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:56:59.784957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-44996f1a-cdf7-4ae5-88df-9132e6e293e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:56:59.831025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:56:59.864886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9828dc-329a-4bb4-972f-ac42d26a7f20 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:59.922930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44996f1a-cdf7-4ae5-88df-9132e6e293e5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:56:59.933815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858b172-9b40-47a1-880a-657b6acf1afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:59.936121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b172-9b40-47a1-880a-657b6acf1afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:59.936837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b172-9b40-47a1-880a-657b6acf1afe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:59.938192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b172-9b40-47a1-880a-657b6acf1afe admin admin] PATH_INFO: `/v3/projects/725b1b6cb58b449283c1ca003588e248` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:59.941579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858b172-9b40-47a1-880a-657b6acf1afe admin admin] RBAC: Authorizing `identity:delete_project(project_id=725b1b6cb58b449283c1ca003588e248)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:59.947656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:56:59.948240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:56:59.948530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:56:59.948808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] PATH_INFO: `/v3/projects/e033e783340b44719f9f77e6d1671a00` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:56:59.950056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e033e783340b44719f9f77e6d1671a00)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:56:59.950333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858b172-9b40-47a1-880a-657b6acf1afe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:56:59.952183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.041838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e033e783340b44719f9f77e6d1671a00', 'request_id': 'req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:00.043559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9858b172-9b40-47a1-880a-657b6acf1afe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '725b1b6cb58b449283c1ca003588e248', 'request_id': 'req-9858b172-9b40-47a1-880a-657b6acf1afe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:00.052390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e033e783340b44719f9f77e6d1671a00', 'request_id': 'req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:00.058521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9858b172-9b40-47a1-880a-657b6acf1afe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '725b1b6cb58b449283c1ca003588e248', 'request_id': 'req-9858b172-9b40-47a1-880a-657b6acf1afe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:00.064138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e033e783340b44719f9f77e6d1671a00', 'request_id': 'req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:00.069322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9858b172-9b40-47a1-880a-657b6acf1afe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '725b1b6cb58b449283c1ca003588e248', 'request_id': 'req-9858b172-9b40-47a1-880a-657b6acf1afe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:00.080638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] The token cache is being invalidate because project e033e783340b44719f9f77e6d1671a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:00.081696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:00.082707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:00.084671 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1592/3136] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:56:59 2022] DELETE /identity/v3/projects/e033e783340b44719f9f77e6d1671a00 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:00.086028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:00.096179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:00.098590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9858b172-9b40-47a1-880a-657b6acf1afe admin admin] The token cache is being invalidate because project 725b1b6cb58b449283c1ca003588e248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:00.099040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9858b172-9b40-47a1-880a-657b6acf1afe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:00.099860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9858b172-9b40-47a1-880a-657b6acf1afe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:00.110029 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1545/3137] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:56:59 2022] DELETE /identity/v3/projects/725b1b6cb58b449283c1ca003588e248 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:57:00.111552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad411b-81d6-4797-9c8e-037577ba31cf None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.112511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad411b-81d6-4797-9c8e-037577ba31cf None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.112964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad411b-81d6-4797-9c8e-037577ba31cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.153346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-22ad411b-81d6-4797-9c8e-037577ba31cf None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:00.154829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-22ad411b-81d6-4797-9c8e-037577ba31cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:00.205904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:00.242397 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1546/3138] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:00.244609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cc7eb-ae71-4996-ad0d-6122f45f1ca1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.245234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cc7eb-ae71-4996-ad0d-6122f45f1ca1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.245684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cc7eb-ae71-4996-ad0d-6122f45f1ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.267088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81deb9-18ea-4a0c-9b9b-e44f7eb9793f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:00.273869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934e1de-9c95-42e8-87d8-77691fefc3da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:00.274770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934e1de-9c95-42e8-87d8-77691fefc3da service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.276607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-796cc7eb-ae71-4996-ad0d-6122f45f1ca1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:00.279903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-796cc7eb-ae71-4996-ad0d-6122f45f1ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:00.284390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934e1de-9c95-42e8-87d8-77691fefc3da service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.290531 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1547/3139] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:00.291125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934e1de-9c95-42e8-87d8-77691fefc3da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.292250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96f00c-4dbf-4ad5-99f2-a8661952d988 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.292770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96f00c-4dbf-4ad5-99f2-a8661952d988 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.293234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96f00c-4dbf-4ad5-99f2-a8661952d988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.296885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5934e1de-9c95-42e8-87d8-77691fefc3da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:00.324937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ce96f00c-4dbf-4ad5-99f2-a8661952d988 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:00.327140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ce96f00c-4dbf-4ad5-99f2-a8661952d988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:00.341786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:00.344607 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1548/3140] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:00.346406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f700777-a4b7-4c31-b33c-7708d05bbd8c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.354492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f700777-a4b7-4c31-b33c-7708d05bbd8c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.354937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f700777-a4b7-4c31-b33c-7708d05bbd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.391455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7f700777-a4b7-4c31-b33c-7708d05bbd8c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:00.394166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7f700777-a4b7-4c31-b33c-7708d05bbd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:00.407369 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1549/3141] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:00.408716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f700777-a4b7-4c31-b33c-7708d05bbd8c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:00.425429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f700777-a4b7-4c31-b33c-7708d05bbd8c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:00.432667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbb6ad-02d5-48f7-8e2c-ab296ea0fb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:00.433463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbb6ad-02d5-48f7-8e2c-ab296ea0fb37 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.433855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbb6ad-02d5-48f7-8e2c-ab296ea0fb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.434239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbb6ad-02d5-48f7-8e2c-ab296ea0fb37 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.437197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbb6ad-02d5-48f7-8e2c-ab296ea0fb37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:00.438002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbb6ad-02d5-48f7-8e2c-ab296ea0fb37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:00.442467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbb6ad-02d5-48f7-8e2c-ab296ea0fb37 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.445179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934e1de-9c95-42e8-87d8-77691fefc3da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:00.449053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934e1de-9c95-42e8-87d8-77691fefc3da service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.464497 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1550/3142] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:00.465583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbb6ad-02d5-48f7-8e2c-ab296ea0fb37 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:00.474128 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1593/3143] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:00.484876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbb6ad-02d5-48f7-8e2c-ab296ea0fb37 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:00.489345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934e1de-9c95-42e8-87d8-77691fefc3da service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:00.507936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb88458-a2b3-4d0d-ae6c-f24a3ecd00c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:00.510015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb88458-a2b3-4d0d-ae6c-f24a3ecd00c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.510767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb88458-a2b3-4d0d-ae6c-f24a3ecd00c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.512305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb88458-a2b3-4d0d-ae6c-f24a3ecd00c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.513181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb88458-a2b3-4d0d-ae6c-f24a3ecd00c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:00.515584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934e1de-9c95-42e8-87d8-77691fefc3da service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:00.515968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb88458-a2b3-4d0d-ae6c-f24a3ecd00c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:00.520170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb88458-a2b3-4d0d-ae6c-f24a3ecd00c7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.527022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010efd58-3782-48f2-8c3a-47dd800a82d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:00.527191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010efd58-3782-48f2-8c3a-47dd800a82d8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.527413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010efd58-3782-48f2-8c3a-47dd800a82d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.527723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010efd58-3782-48f2-8c3a-47dd800a82d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.530853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010efd58-3782-48f2-8c3a-47dd800a82d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:00.539605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010efd58-3782-48f2-8c3a-47dd800a82d8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.543079 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1551/3144] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:00.550999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb88458-a2b3-4d0d-ae6c-f24a3ecd00c7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:00.564006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb88458-a2b3-4d0d-ae6c-f24a3ecd00c7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:00.569301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48b571-a146-4054-89bf-3512fd343a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:00.570021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48b571-a146-4054-89bf-3512fd343a5e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.570377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48b571-a146-4054-89bf-3512fd343a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.570728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48b571-a146-4054-89bf-3512fd343a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.572099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48b571-a146-4054-89bf-3512fd343a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:00.573892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48b571-a146-4054-89bf-3512fd343a5e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.592036 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1594/3145] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:00 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:00.597658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010efd58-3782-48f2-8c3a-47dd800a82d8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:00.614492 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1552/3146] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:00 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:00.618682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48b571-a146-4054-89bf-3512fd343a5e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:00.691233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010efd58-3782-48f2-8c3a-47dd800a82d8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:00.729592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48b571-a146-4054-89bf-3512fd343a5e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:00.737553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe00934-ea74-4f8b-95f0-00090bbd1b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:00.738302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe00934-ea74-4f8b-95f0-00090bbd1b68 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.738764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe00934-ea74-4f8b-95f0-00090bbd1b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.739377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe00934-ea74-4f8b-95f0-00090bbd1b68 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.741345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe00934-ea74-4f8b-95f0-00090bbd1b68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:00.745128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe00934-ea74-4f8b-95f0-00090bbd1b68 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.780962 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1553/3147] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:00 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:00.782165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf2218-9b5b-4028-94c2-87bf1accb6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:00.784318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf2218-9b5b-4028-94c2-87bf1accb6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.784651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe00934-ea74-4f8b-95f0-00090bbd1b68 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:00.784988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf2218-9b5b-4028-94c2-87bf1accb6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.785433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf2218-9b5b-4028-94c2-87bf1accb6a0 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.786727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf2218-9b5b-4028-94c2-87bf1accb6a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:00.789812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf2218-9b5b-4028-94c2-87bf1accb6a0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.800144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe00934-ea74-4f8b-95f0-00090bbd1b68 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:00.809905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ec174-3178-473e-a782-971143a6f9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:00.812174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ec174-3178-473e-a782-971143a6f9bc admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.812174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ec174-3178-473e-a782-971143a6f9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.812621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ec174-3178-473e-a782-971143a6f9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.813807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ec174-3178-473e-a782-971143a6f9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:00.816990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ec174-3178-473e-a782-971143a6f9bc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.822068 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1595/3148] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:00 2022] POST /identity/v3/users => generated 295 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:57:00.826570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf2218-9b5b-4028-94c2-87bf1accb6a0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:00.834571 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1554/3149] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:57:00.835621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ec174-3178-473e-a782-971143a6f9bc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:00.867835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf2218-9b5b-4028-94c2-87bf1accb6a0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:00.869928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ec174-3178-473e-a782-971143a6f9bc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:00.875412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d70af-9e10-4ca5-acff-d8bc80e46a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:00.876132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d70af-9e10-4ca5-acff-d8bc80e46a4c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.876600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d70af-9e10-4ca5-acff-d8bc80e46a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.877001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d70af-9e10-4ca5-acff-d8bc80e46a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.877801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d70af-9e10-4ca5-acff-d8bc80e46a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:00.879789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d70af-9e10-4ca5-acff-d8bc80e46a4c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.885214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e9a93-5f47-4031-8a0e-d899b6f27b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:00.885844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e9a93-5f47-4031-8a0e-d899b6f27b49 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.886135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e9a93-5f47-4031-8a0e-d899b6f27b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.886412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e9a93-5f47-4031-8a0e-d899b6f27b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.887068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e9a93-5f47-4031-8a0e-d899b6f27b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:00.891937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e9a93-5f47-4031-8a0e-d899b6f27b49 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.894206 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1596/3150] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:00.901552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d70af-9e10-4ca5-acff-d8bc80e46a4c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:00.911661 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1555/3151] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:57:00.913195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e9a93-5f47-4031-8a0e-d899b6f27b49 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:00.925898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d70af-9e10-4ca5-acff-d8bc80e46a4c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:00.926626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e9a93-5f47-4031-8a0e-d899b6f27b49 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:00.932038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae28286-be68-4c16-8c2a-452191453bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:00.933110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae28286-be68-4c16-8c2a-452191453bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.933496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae28286-be68-4c16-8c2a-452191453bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.933876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae28286-be68-4c16-8c2a-452191453bd1 admin admin] PATH_INFO: `/v3/projects/5500274fc1834f768e8659be97e4004c/users/f40d4376bec042e49ee8356e22baa812/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.938917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c34b6-7fd5-4e3b-9240-28a8001a3889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:00.939735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c34b6-7fd5-4e3b-9240-28a8001a3889 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.943163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c34b6-7fd5-4e3b-9240-28a8001a3889 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.943477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c34b6-7fd5-4e3b-9240-28a8001a3889 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.944159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c34b6-7fd5-4e3b-9240-28a8001a3889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:00.950856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c34b6-7fd5-4e3b-9240-28a8001a3889 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.954295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae28286-be68-4c16-8c2a-452191453bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5500274fc1834f768e8659be97e4004c, user_id=f40d4376bec042e49ee8356e22baa812, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:00.957953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae28286-be68-4c16-8c2a-452191453bd1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:00.975934 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1556/3152] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:00 2022] PUT /identity/v3/projects/5500274fc1834f768e8659be97e4004c/users/f40d4376bec042e49ee8356e22baa812/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:00.981452 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1597/3153] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:57:00.982086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e284f-f900-4602-8a23-7092fe74b6aa None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:00.982385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e284f-f900-4602-8a23-7092fe74b6aa None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:00.984393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e284f-f900-4602-8a23-7092fe74b6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:00.985512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c34b6-7fd5-4e3b-9240-28a8001a3889 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:01.000708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c34b6-7fd5-4e3b-9240-28a8001a3889 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:01.026040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-335e284f-f900-4602-8a23-7092fe74b6aa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:01.026040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-335e284f-f900-4602-8a23-7092fe74b6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:01.037721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:01.039332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:01.148253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e788-4bee-4489-b5c7-942256e71940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:01.148972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e788-4bee-4489-b5c7-942256e71940 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:01.149263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e788-4bee-4489-b5c7-942256e71940 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:01.149544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e788-4bee-4489-b5c7-942256e71940 admin admin] PATH_INFO: `/v3/projects/090fc6c9de624a209a90631c0d22d48f/users/7838d64ab35e4608b668e9f65d594d6e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:01.170901 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1557/3154] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:01.171571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e284f-f900-4602-8a23-7092fe74b6aa None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:01.175495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-335e284f-f900-4602-8a23-7092fe74b6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:01.189114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5e788-4bee-4489-b5c7-942256e71940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=090fc6c9de624a209a90631c0d22d48f, user_id=7838d64ab35e4608b668e9f65d594d6e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:01.192624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5e788-4bee-4489-b5c7-942256e71940 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:01.202190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e284f-f900-4602-8a23-7092fe74b6aa None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:01.218364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c17006-0def-4789-8d0b-0bbef2a52c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:01.219298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c17006-0def-4789-8d0b-0bbef2a52c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:01.219590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c17006-0def-4789-8d0b-0bbef2a52c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:01.219870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c17006-0def-4789-8d0b-0bbef2a52c71 admin admin] PATH_INFO: `/v3/projects/28fd3d4b280f40da96ac5f21cb33ece7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:01.224573 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1598/3155] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:00 2022] PUT /identity/v3/projects/090fc6c9de624a209a90631c0d22d48f/users/7838d64ab35e4608b668e9f65d594d6e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:57:01.225513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e788-4bee-4489-b5c7-942256e71940 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:01.226868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c17006-0def-4789-8d0b-0bbef2a52c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28fd3d4b280f40da96ac5f21cb33ece7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:01.228854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c17006-0def-4789-8d0b-0bbef2a52c71 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:01.232742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b5c5e788-4bee-4489-b5c7-942256e71940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:01.344749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-38c17006-0def-4789-8d0b-0bbef2a52c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28fd3d4b280f40da96ac5f21cb33ece7', 'request_id': 'req-38c17006-0def-4789-8d0b-0bbef2a52c71'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:01.358328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e788-4bee-4489-b5c7-942256e71940 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:01.364378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-38c17006-0def-4789-8d0b-0bbef2a52c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28fd3d4b280f40da96ac5f21cb33ece7', 'request_id': 'req-38c17006-0def-4789-8d0b-0bbef2a52c71'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:01.379949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-38c17006-0def-4789-8d0b-0bbef2a52c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28fd3d4b280f40da96ac5f21cb33ece7', 'request_id': 'req-38c17006-0def-4789-8d0b-0bbef2a52c71'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:01.402290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-38c17006-0def-4789-8d0b-0bbef2a52c71 admin admin] The token cache is being invalidate because project 28fd3d4b280f40da96ac5f21cb33ece7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:01.402593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-38c17006-0def-4789-8d0b-0bbef2a52c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:01.404585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-38c17006-0def-4789-8d0b-0bbef2a52c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:01.407299 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1558/3156] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:01 2022] DELETE /identity/v3/projects/28fd3d4b280f40da96ac5f21cb33ece7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:01.409178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6efc2-7936-439c-a5a7-014c26fe9a6a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:01.409486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6efc2-7936-439c-a5a7-014c26fe9a6a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:01.409761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6efc2-7936-439c-a5a7-014c26fe9a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:01.458953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-23e6efc2-7936-439c-a5a7-014c26fe9a6a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:01.460238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-23e6efc2-7936-439c-a5a7-014c26fe9a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:01.485252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a84542-4430-4e87-83f5-6ed5db8b4fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:01.485969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a84542-4430-4e87-83f5-6ed5db8b4fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:01.489113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a84542-4430-4e87-83f5-6ed5db8b4fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:01.489498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a84542-4430-4e87-83f5-6ed5db8b4fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:01.494392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b1a84542-4430-4e87-83f5-6ed5db8b4fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:01.578900 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1559/3157] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:01.580328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd69d2-1298-4ac6-9d7f-309a56bc48fa None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:01.581092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd69d2-1298-4ac6-9d7f-309a56bc48fa None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:01.581400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd69d2-1298-4ac6-9d7f-309a56bc48fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:01.629433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-52bd69d2-1298-4ac6-9d7f-309a56bc48fa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:01.630704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-52bd69d2-1298-4ac6-9d7f-309a56bc48fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:01.658044 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1560/3158] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:01.660540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8582331-7fcf-491a-950d-5b7c19217580 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:01.660710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8582331-7fcf-491a-950d-5b7c19217580 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:01.660869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8582331-7fcf-491a-950d-5b7c19217580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:01.664693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a84542-4430-4e87-83f5-6ed5db8b4fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:01.666833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a84542-4430-4e87-83f5-6ed5db8b4fc3 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:01.673743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:01.687322 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1599/3159] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:01.688284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a84542-4430-4e87-83f5-6ed5db8b4fc3 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:01.689785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b1a84542-4430-4e87-83f5-6ed5db8b4fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:01.705419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a84542-4430-4e87-83f5-6ed5db8b4fc3 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:01.710401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d97ae8-4010-4141-91b3-73f98352c836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:01.711025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d97ae8-4010-4141-91b3-73f98352c836 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:01.711155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a8582331-7fcf-491a-950d-5b7c19217580 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:01.711378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d97ae8-4010-4141-91b3-73f98352c836 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:01.711585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d97ae8-4010-4141-91b3-73f98352c836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:01.712890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a8582331-7fcf-491a-950d-5b7c19217580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:01.726012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d97ae8-4010-4141-91b3-73f98352c836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:01.728174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d97ae8-4010-4141-91b3-73f98352c836 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:01.738916 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1561/3160] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:01.740153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8582331-7fcf-491a-950d-5b7c19217580 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:01.741341 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1600/3161] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:01.764364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8582331-7fcf-491a-950d-5b7c19217580 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:01.775697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2d91c-f69e-4248-8eda-fedd12e9edb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:01.776335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d91c-f69e-4248-8eda-fedd12e9edb0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:01.776624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d91c-f69e-4248-8eda-fedd12e9edb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:01.777022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d91c-f69e-4248-8eda-fedd12e9edb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:01.777593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2d91c-f69e-4248-8eda-fedd12e9edb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:01.777905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2d91c-f69e-4248-8eda-fedd12e9edb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:01.779535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2d91c-f69e-4248-8eda-fedd12e9edb0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:01.815016 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1562/3162] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:01.819763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d97ae8-4010-4141-91b3-73f98352c836 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:01.837976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d97ae8-4010-4141-91b3-73f98352c836 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:01.852390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f3ef6-a8b3-4c1c-8207-118f1174db4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:01.853324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f3ef6-a8b3-4c1c-8207-118f1174db4b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:01.853647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f3ef6-a8b3-4c1c-8207-118f1174db4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:01.853925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f3ef6-a8b3-4c1c-8207-118f1174db4b admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:01.854884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f3ef6-a8b3-4c1c-8207-118f1174db4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:01.860687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f3ef6-a8b3-4c1c-8207-118f1174db4b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:01.933927 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1601/3163] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:01 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:57:01.934286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2d91c-f69e-4248-8eda-fedd12e9edb0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:01.955000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2d91c-f69e-4248-8eda-fedd12e9edb0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:02.012160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:02.210229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1a11c-2c89-4b6c-8db9-d9b06446ec89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:02.211371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1a11c-2c89-4b6c-8db9-d9b06446ec89 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:02.212325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1a11c-2c89-4b6c-8db9-d9b06446ec89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:02.212736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1a11c-2c89-4b6c-8db9-d9b06446ec89 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:02.215775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1a11c-2c89-4b6c-8db9-d9b06446ec89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:02.233343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1a11c-2c89-4b6c-8db9-d9b06446ec89 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:02.295580 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1563/3164] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:01 2022] POST /identity/v3/users => generated 295 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:57:02.300146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f3ef6-a8b3-4c1c-8207-118f1174db4b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:02.314172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f3ef6-a8b3-4c1c-8207-118f1174db4b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:02.328344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acbae1-ad41-43d2-a53a-7623baf57bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:02.329396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acbae1-ad41-43d2-a53a-7623baf57bae admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:02.329848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acbae1-ad41-43d2-a53a-7623baf57bae admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:02.330323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acbae1-ad41-43d2-a53a-7623baf57bae admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:02.331182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acbae1-ad41-43d2-a53a-7623baf57bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:02.333426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acbae1-ad41-43d2-a53a-7623baf57bae admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:02.369691 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1602/3165] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:02.371137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1a11c-2c89-4b6c-8db9-d9b06446ec89 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:02.384267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1a11c-2c89-4b6c-8db9-d9b06446ec89 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:02.404788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a963e6-c845-4528-82ad-4688f46e33e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:02.405450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a963e6-c845-4528-82ad-4688f46e33e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:02.405734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a963e6-c845-4528-82ad-4688f46e33e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:02.406034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a963e6-c845-4528-82ad-4688f46e33e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:02.406696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a963e6-c845-4528-82ad-4688f46e33e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:02.409508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a963e6-c845-4528-82ad-4688f46e33e7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:02.447407 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1564/3166] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:02.450591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acbae1-ad41-43d2-a53a-7623baf57bae admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:02.467541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acbae1-ad41-43d2-a53a-7623baf57bae admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:02.498308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee14939-b8cc-4400-a9b6-b53df3b50dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:02.499719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee14939-b8cc-4400-a9b6-b53df3b50dae admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:02.502781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee14939-b8cc-4400-a9b6-b53df3b50dae admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:02.504377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee14939-b8cc-4400-a9b6-b53df3b50dae admin admin] PATH_INFO: `/v3/projects/93e3aeb041454256bdf249b6ed6a5214/users/37a9d8dd01d540d398c6112b0091d1ac/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:02.549881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee14939-b8cc-4400-a9b6-b53df3b50dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e3aeb041454256bdf249b6ed6a5214, user_id=37a9d8dd01d540d398c6112b0091d1ac, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:02.556591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee14939-b8cc-4400-a9b6-b53df3b50dae admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:02.596502 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1603/3167] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:02 2022] PUT /identity/v3/projects/93e3aeb041454256bdf249b6ed6a5214/users/37a9d8dd01d540d398c6112b0091d1ac/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:02.596728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607ad60-e019-4869-a9f6-3c96581acf77 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:02.596992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607ad60-e019-4869-a9f6-3c96581acf77 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:02.597220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607ad60-e019-4869-a9f6-3c96581acf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:02.640077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8607ad60-e019-4869-a9f6-3c96581acf77 None None] MFA Rules not processed for user `37a9d8dd01d540d398c6112b0091d1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:02.641507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8607ad60-e019-4869-a9f6-3c96581acf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:02.766998 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1565/3168] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:02.772169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee14939-b8cc-4400-a9b6-b53df3b50dae admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:02.861060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee14939-b8cc-4400-a9b6-b53df3b50dae admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:02.870000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579f46e-3564-4a40-95f6-9331d2c78c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:02.871595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579f46e-3564-4a40-95f6-9331d2c78c33 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:02.872175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579f46e-3564-4a40-95f6-9331d2c78c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:02.873453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579f46e-3564-4a40-95f6-9331d2c78c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:02.874371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579f46e-3564-4a40-95f6-9331d2c78c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:02.878157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579f46e-3564-4a40-95f6-9331d2c78c33 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:02.923282 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1604/3169] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:02 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:02.936020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607ad60-e019-4869-a9f6-3c96581acf77 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:02.976624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607ad60-e019-4869-a9f6-3c96581acf77 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:03.242814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e28d3e-e22c-4f48-ab6b-489ab27097c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:03.243882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e28d3e-e22c-4f48-ab6b-489ab27097c5 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:03.244176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e28d3e-e22c-4f48-ab6b-489ab27097c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:03.244456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e28d3e-e22c-4f48-ab6b-489ab27097c5 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:03.245154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e28d3e-e22c-4f48-ab6b-489ab27097c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:03.247308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e28d3e-e22c-4f48-ab6b-489ab27097c5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:03.309393 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1566/3170] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:02 2022] POST /identity/v3/users => generated 300 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:03.313496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579f46e-3564-4a40-95f6-9331d2c78c33 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:03.336339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579f46e-3564-4a40-95f6-9331d2c78c33 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:03.350439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be97c4-57b6-4823-b29f-eaba3e037015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:03.351080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be97c4-57b6-4823-b29f-eaba3e037015 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:03.351413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be97c4-57b6-4823-b29f-eaba3e037015 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:03.351699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be97c4-57b6-4823-b29f-eaba3e037015 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:03.352362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be97c4-57b6-4823-b29f-eaba3e037015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:03.358119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be97c4-57b6-4823-b29f-eaba3e037015 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:03.376545 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1605/3171] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:03.381321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e28d3e-e22c-4f48-ab6b-489ab27097c5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:03.397659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e28d3e-e22c-4f48-ab6b-489ab27097c5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:03.402138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be97c4-57b6-4823-b29f-eaba3e037015 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:03.403952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a9be97c4-57b6-4823-b29f-eaba3e037015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:03.422187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425fd0a-5055-4e90-8ec3-3677a16b25c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:03.423678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425fd0a-5055-4e90-8ec3-3677a16b25c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:03.423966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425fd0a-5055-4e90-8ec3-3677a16b25c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:03.424792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425fd0a-5055-4e90-8ec3-3677a16b25c0 admin admin] PATH_INFO: `/v3/projects/84409c8ff521450abb0cc252fe4d141f/users/262cef2ff9524fa7bd00e03daf575174/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:03.427656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be97c4-57b6-4823-b29f-eaba3e037015 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:03.441951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:03.449581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f0ec9-a304-435d-956d-c65ecdcd6a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:03.452207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0ec9-a304-435d-956d-c65ecdcd6a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:03.452832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425fd0a-5055-4e90-8ec3-3677a16b25c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84409c8ff521450abb0cc252fe4d141f, user_id=262cef2ff9524fa7bd00e03daf575174, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:03.453069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0ec9-a304-435d-956d-c65ecdcd6a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:03.453377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0ec9-a304-435d-956d-c65ecdcd6a99 admin admin] PATH_INFO: `/v3/users/d4bb698e4a2e4598acdb662511ce9eeb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:03.456738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425fd0a-5055-4e90-8ec3-3677a16b25c0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:03.457768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f0ec9-a304-435d-956d-c65ecdcd6a99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4bb698e4a2e4598acdb662511ce9eeb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:03.460128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f0ec9-a304-435d-956d-c65ecdcd6a99 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:03.493687 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1567/3172] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:03 2022] PUT /identity/v3/projects/84409c8ff521450abb0cc252fe4d141f/users/262cef2ff9524fa7bd00e03daf575174/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:57:03.495465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425fd0a-5055-4e90-8ec3-3677a16b25c0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:03.576866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-640f0ec9-a304-435d-956d-c65ecdcd6a99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4bb698e4a2e4598acdb662511ce9eeb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:03.597343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:03.606714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-640f0ec9-a304-435d-956d-c65ecdcd6a99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4bb698e4a2e4598acdb662511ce9eeb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:03.624975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-640f0ec9-a304-435d-956d-c65ecdcd6a99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4bb698e4a2e4598acdb662511ce9eeb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:03.626871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-640f0ec9-a304-435d-956d-c65ecdcd6a99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4bb698e4a2e4598acdb662511ce9eeb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:03.629393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-640f0ec9-a304-435d-956d-c65ecdcd6a99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4bb698e4a2e4598acdb662511ce9eeb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:03.641030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425fd0a-5055-4e90-8ec3-3677a16b25c0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:03.652629 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1606/3173] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:03 2022] DELETE /identity/v3/users/d4bb698e4a2e4598acdb662511ce9eeb => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:57:03.653110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0485672-8f32-45a5-b9b3-48cb0c204ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:03.653110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0485672-8f32-45a5-b9b3-48cb0c204ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:03.653110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0485672-8f32-45a5-b9b3-48cb0c204ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:03.653110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0485672-8f32-45a5-b9b3-48cb0c204ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:03.653528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0485672-8f32-45a5-b9b3-48cb0c204ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:03.654242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0485672-8f32-45a5-b9b3-48cb0c204ae9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:03.655025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e750c-15f7-4ba2-8012-b849ed93a06d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:03.655430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e750c-15f7-4ba2-8012-b849ed93a06d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:03.655824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e750c-15f7-4ba2-8012-b849ed93a06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:03.683101 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1568/3174] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:03.687521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0485672-8f32-45a5-b9b3-48cb0c204ae9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:03.690727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3f4e750c-15f7-4ba2-8012-b849ed93a06d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:03.691539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d0485672-8f32-45a5-b9b3-48cb0c204ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:03.692269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3f4e750c-15f7-4ba2-8012-b849ed93a06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:03.725315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0485672-8f32-45a5-b9b3-48cb0c204ae9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:03.787536 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1607/3175] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:57:03.825605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad280520-b019-4f03-bbc7-bc12d2156688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:03.828108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad280520-b019-4f03-bbc7-bc12d2156688 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:03.830073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad280520-b019-4f03-bbc7-bc12d2156688 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:03.830402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad280520-b019-4f03-bbc7-bc12d2156688 admin admin] PATH_INFO: `/v3/projects/84409c8ff521450abb0cc252fe4d141f/users/262cef2ff9524fa7bd00e03daf575174/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:03.831724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e750c-15f7-4ba2-8012-b849ed93a06d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:03.833693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3f4e750c-15f7-4ba2-8012-b849ed93a06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:03.845165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad280520-b019-4f03-bbc7-bc12d2156688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84409c8ff521450abb0cc252fe4d141f, user_id=262cef2ff9524fa7bd00e03daf575174, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:03.853132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad280520-b019-4f03-bbc7-bc12d2156688 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:03.881327 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1569/3176] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:03 2022] PUT /identity/v3/projects/84409c8ff521450abb0cc252fe4d141f/users/262cef2ff9524fa7bd00e03daf575174/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:03.892277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb04b0-4558-4cae-8ced-29d7025f00e0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:03.892661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb04b0-4558-4cae-8ced-29d7025f00e0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:03.893645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb04b0-4558-4cae-8ced-29d7025f00e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:03.964521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-78eb04b0-4558-4cae-8ced-29d7025f00e0 None None] MFA Rules not processed for user `262cef2ff9524fa7bd00e03daf575174`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:03.965794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-78eb04b0-4558-4cae-8ced-29d7025f00e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:03.966205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e750c-15f7-4ba2-8012-b849ed93a06d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:04.055293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529ebe0-fbf9-4133-bda8-6de0d51e5aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:04.057164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ebe0-fbf9-4133-bda8-6de0d51e5aab service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:04.058004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ebe0-fbf9-4133-bda8-6de0d51e5aab service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:04.058537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ebe0-fbf9-4133-bda8-6de0d51e5aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:04.087658 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1570/3177] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:04.103238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb04b0-4558-4cae-8ced-29d7025f00e0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:04.105268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-78eb04b0-4558-4cae-8ced-29d7025f00e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:04.112370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:04.170533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:04.195043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529ebe0-fbf9-4133-bda8-6de0d51e5aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:04.197092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529ebe0-fbf9-4133-bda8-6de0d51e5aab service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:04.211212 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1608/3178] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:04.226918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb04b0-4558-4cae-8ced-29d7025f00e0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:04.234220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c9c24-83a4-4221-ae61-880c6e52f8fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:04.235018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c9c24-83a4-4221-ae61-880c6e52f8fd service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:04.235716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c9c24-83a4-4221-ae61-880c6e52f8fd service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:04.235716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c9c24-83a4-4221-ae61-880c6e52f8fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:04.237791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-da5c9c24-83a4-4221-ae61-880c6e52f8fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:04.315334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c9c24-83a4-4221-ae61-880c6e52f8fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:04.317546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c9c24-83a4-4221-ae61-880c6e52f8fd service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:04.328772 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1571/3179] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:06.032795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529ebe0-fbf9-4133-bda8-6de0d51e5aab service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:06.039576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a529ebe0-fbf9-4133-bda8-6de0d51e5aab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:06.065472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529ebe0-fbf9-4133-bda8-6de0d51e5aab service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:06.088405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93be9e-dd29-4bdb-bd54-045986f00b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:06.089540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93be9e-dd29-4bdb-bd54-045986f00b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:06.089889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93be9e-dd29-4bdb-bd54-045986f00b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:06.090168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93be9e-dd29-4bdb-bd54-045986f00b53 admin admin] PATH_INFO: `/v3/projects/bcdfcb6afebd476ca5781f53008fe1c8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:06.097455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93be9e-dd29-4bdb-bd54-045986f00b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcdfcb6afebd476ca5781f53008fe1c8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:06.099400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93be9e-dd29-4bdb-bd54-045986f00b53 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:06.253637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea93be9e-dd29-4bdb-bd54-045986f00b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcdfcb6afebd476ca5781f53008fe1c8', 'request_id': 'req-ea93be9e-dd29-4bdb-bd54-045986f00b53'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:06.273293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea93be9e-dd29-4bdb-bd54-045986f00b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcdfcb6afebd476ca5781f53008fe1c8', 'request_id': 'req-ea93be9e-dd29-4bdb-bd54-045986f00b53'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:06.292334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea93be9e-dd29-4bdb-bd54-045986f00b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcdfcb6afebd476ca5781f53008fe1c8', 'request_id': 'req-ea93be9e-dd29-4bdb-bd54-045986f00b53'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:06.319296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea93be9e-dd29-4bdb-bd54-045986f00b53 admin admin] The token cache is being invalidate because project bcdfcb6afebd476ca5781f53008fe1c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:06.319612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea93be9e-dd29-4bdb-bd54-045986f00b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:06.329026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea93be9e-dd29-4bdb-bd54-045986f00b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:06.354105 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1609/3180] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:06 2022] DELETE /identity/v3/projects/bcdfcb6afebd476ca5781f53008fe1c8 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:06.356048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643635db-1e37-48e4-a517-91671848a07d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:06.358535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643635db-1e37-48e4-a517-91671848a07d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:06.359060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643635db-1e37-48e4-a517-91671848a07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:06.414685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-643635db-1e37-48e4-a517-91671848a07d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:06.416226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-643635db-1e37-48e4-a517-91671848a07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:06.570830 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1572/3181] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:06.576227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adf52a-f19d-4119-af8c-2096b78d1156 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:06.576542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adf52a-f19d-4119-af8c-2096b78d1156 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:06.576828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adf52a-f19d-4119-af8c-2096b78d1156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:06.598888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643635db-1e37-48e4-a517-91671848a07d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:06.600979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-643635db-1e37-48e4-a517-91671848a07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:06.637920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-98adf52a-f19d-4119-af8c-2096b78d1156 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:06.641873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-98adf52a-f19d-4119-af8c-2096b78d1156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:06.677581 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1610/3182] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:06.681564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adf52a-f19d-4119-af8c-2096b78d1156 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:06.693857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:06.713632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adf52a-f19d-4119-af8c-2096b78d1156 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:06.731190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbd153-4f61-4a4d-9533-7b1ccb3e296a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:06.731822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbd153-4f61-4a4d-9533-7b1ccb3e296a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:06.736304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbd153-4f61-4a4d-9533-7b1ccb3e296a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:06.736724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbd153-4f61-4a4d-9533-7b1ccb3e296a admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:06.745097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbd153-4f61-4a4d-9533-7b1ccb3e296a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:06.745424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbd153-4f61-4a4d-9533-7b1ccb3e296a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:06.756954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbd153-4f61-4a4d-9533-7b1ccb3e296a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:06.758019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643635db-1e37-48e4-a517-91671848a07d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:06.787285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b030a61-9a4f-47f4-af45-c301d27c70d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:06.791755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b030a61-9a4f-47f4-af45-c301d27c70d1 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:06.792309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b030a61-9a4f-47f4-af45-c301d27c70d1 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:06.792797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b030a61-9a4f-47f4-af45-c301d27c70d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:06.803992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7b030a61-9a4f-47f4-af45-c301d27c70d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:06.839380 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1611/3183] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:06.840495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbd153-4f61-4a4d-9533-7b1ccb3e296a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:06.873605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b030a61-9a4f-47f4-af45-c301d27c70d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:06.875138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b030a61-9a4f-47f4-af45-c301d27c70d1 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:06.878287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbd153-4f61-4a4d-9533-7b1ccb3e296a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:06.890913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095ea17-cb59-4506-9218-af8347ea13e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:06.891457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095ea17-cb59-4506-9218-af8347ea13e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:06.891743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095ea17-cb59-4506-9218-af8347ea13e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:06.892320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095ea17-cb59-4506-9218-af8347ea13e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:06.894352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095ea17-cb59-4506-9218-af8347ea13e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:06.897324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095ea17-cb59-4506-9218-af8347ea13e1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:06.929517 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1573/3184] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:06.959075 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1612/3185] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:06 2022] POST /identity/v3/projects => generated 389 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:06.962865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b030a61-9a4f-47f4-af45-c301d27c70d1 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:06.984973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b030a61-9a4f-47f4-af45-c301d27c70d1 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:07.132583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7ae53-fbcf-47f3-9474-20f447ce2036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:07.132583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7ae53-fbcf-47f3-9474-20f447ce2036 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:07.132583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7ae53-fbcf-47f3-9474-20f447ce2036 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:07.132583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7ae53-fbcf-47f3-9474-20f447ce2036 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:07.132851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7ae53-fbcf-47f3-9474-20f447ce2036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:07.139362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7ae53-fbcf-47f3-9474-20f447ce2036 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:07.199914 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1574/3186] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:06 2022] POST /identity/v3/users => generated 295 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:07.203822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095ea17-cb59-4506-9218-af8347ea13e1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:07.224907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095ea17-cb59-4506-9218-af8347ea13e1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:07.245673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffbdf2-520c-4210-ae57-330b65d22888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:07.246393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffbdf2-520c-4210-ae57-330b65d22888 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:07.246687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffbdf2-520c-4210-ae57-330b65d22888 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:07.246964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffbdf2-520c-4210-ae57-330b65d22888 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:07.247620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffbdf2-520c-4210-ae57-330b65d22888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:07.249293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffbdf2-520c-4210-ae57-330b65d22888 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:07.298478 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1613/3187] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:07.298650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7ae53-fbcf-47f3-9474-20f447ce2036 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:07.315357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7ae53-fbcf-47f3-9474-20f447ce2036 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:07.330431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb0136-376d-4a0b-97a7-5ac2705c6e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:07.331653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb0136-376d-4a0b-97a7-5ac2705c6e09 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:07.332014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb0136-376d-4a0b-97a7-5ac2705c6e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:07.332368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb0136-376d-4a0b-97a7-5ac2705c6e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:07.333381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb0136-376d-4a0b-97a7-5ac2705c6e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:07.335269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb0136-376d-4a0b-97a7-5ac2705c6e09 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:07.374209 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1575/3188] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:57:07.376542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffbdf2-520c-4210-ae57-330b65d22888 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:07.391745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffbdf2-520c-4210-ae57-330b65d22888 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:07.404809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cad46e-37e5-49ac-9282-8a157a195035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:07.405523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cad46e-37e5-49ac-9282-8a157a195035 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:07.406025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cad46e-37e5-49ac-9282-8a157a195035 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:07.406627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cad46e-37e5-49ac-9282-8a157a195035 admin admin] PATH_INFO: `/v3/projects/42f287c9b2f24f4a94002ca61190c12d/users/3da1337b27b64e008b26a2c464578913/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:07.450368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cad46e-37e5-49ac-9282-8a157a195035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f287c9b2f24f4a94002ca61190c12d, user_id=3da1337b27b64e008b26a2c464578913, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:07.453390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cad46e-37e5-49ac-9282-8a157a195035 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:07.507509 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1614/3189] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:07 2022] PUT /identity/v3/projects/42f287c9b2f24f4a94002ca61190c12d/users/3da1337b27b64e008b26a2c464578913/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:07.512572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda90db-49eb-4779-b1f0-88e87d46bdc6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:07.513098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda90db-49eb-4779-b1f0-88e87d46bdc6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:07.513527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda90db-49eb-4779-b1f0-88e87d46bdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:07.552187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-dbda90db-49eb-4779-b1f0-88e87d46bdc6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:07.553489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dbda90db-49eb-4779-b1f0-88e87d46bdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:07.721275 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1576/3190] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:07.810547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cad46e-37e5-49ac-9282-8a157a195035 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:07.812363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-55cad46e-37e5-49ac-9282-8a157a195035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:07.985458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:07.991342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cad46e-37e5-49ac-9282-8a157a195035 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:07.991458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda90db-49eb-4779-b1f0-88e87d46bdc6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:07.993617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dbda90db-49eb-4779-b1f0-88e87d46bdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:08.007297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebc92a-95ec-42d8-9ed4-b6cf3d1f008b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:08.008181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebc92a-95ec-42d8-9ed4-b6cf3d1f008b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:08.008618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebc92a-95ec-42d8-9ed4-b6cf3d1f008b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:08.009087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebc92a-95ec-42d8-9ed4-b6cf3d1f008b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:08.037261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebc92a-95ec-42d8-9ed4-b6cf3d1f008b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:08.038673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebc92a-95ec-42d8-9ed4-b6cf3d1f008b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:08.069561 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1615/3191] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:08.098269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:08.133038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda90db-49eb-4779-b1f0-88e87d46bdc6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:08.147160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cabff-1fe5-408e-aced-ace849de48b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:08.148644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cabff-1fe5-408e-aced-ace849de48b4 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:08.149197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cabff-1fe5-408e-aced-ace849de48b4 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:08.149667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cabff-1fe5-408e-aced-ace849de48b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:08.151737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5f6cabff-1fe5-408e-aced-ace849de48b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:08.248213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6cabff-1fe5-408e-aced-ace849de48b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:08.250474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6cabff-1fe5-408e-aced-ace849de48b4 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:08.266881 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1577/3192] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:09.032554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebc92a-95ec-42d8-9ed4-b6cf3d1f008b service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:09.034933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-deebc92a-95ec-42d8-9ed4-b6cf3d1f008b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:09.136733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebc92a-95ec-42d8-9ed4-b6cf3d1f008b service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:09.152496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:09.153553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:09.153763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:09.154052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] PATH_INFO: `/v3/users/90d7e4c2e577467f949be3dc480cc7a2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:09.155337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90d7e4c2e577467f949be3dc480cc7a2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:09.159768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:09.301146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90d7e4c2e577467f949be3dc480cc7a2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:09.319269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90d7e4c2e577467f949be3dc480cc7a2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:09.332523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90d7e4c2e577467f949be3dc480cc7a2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:09.333329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90d7e4c2e577467f949be3dc480cc7a2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:09.334236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90d7e4c2e577467f949be3dc480cc7a2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:09.371286 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1616/3193] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:09 2022] DELETE /identity/v3/users/90d7e4c2e577467f949be3dc480cc7a2 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:10.162310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cabff-1fe5-408e-aced-ace849de48b4 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:10.184061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cabff-1fe5-408e-aced-ace849de48b4 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:10.370299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:10.370871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:10.371158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:10.371437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] PATH_INFO: `/v3/projects/6a7e10b9d8594d16a4c1eb63196ea329` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:10.373036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a7e10b9d8594d16a4c1eb63196ea329)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:10.376385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:10.490915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a7e10b9d8594d16a4c1eb63196ea329', 'request_id': 'req-07f592cf-0a36-4496-8d82-9bf4c34617e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:10.505336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a7e10b9d8594d16a4c1eb63196ea329', 'request_id': 'req-07f592cf-0a36-4496-8d82-9bf4c34617e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:10.515051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a7e10b9d8594d16a4c1eb63196ea329', 'request_id': 'req-07f592cf-0a36-4496-8d82-9bf4c34617e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:10.546057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] The token cache is being invalidate because project 6a7e10b9d8594d16a4c1eb63196ea329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:10.546369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:10.547377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:10.569173 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1578/3194] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:10 2022] DELETE /identity/v3/projects/6a7e10b9d8594d16a4c1eb63196ea329 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:10.572921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:10.581393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:10.645126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376061bd-502b-4783-9b17-a9018a484fa0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:10.773092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f86ab-4b01-4896-86ec-bba528f04a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:10.773999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f86ab-4b01-4896-86ec-bba528f04a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:10.774392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f86ab-4b01-4896-86ec-bba528f04a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:10.774635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f86ab-4b01-4896-86ec-bba528f04a67 admin admin] PATH_INFO: `/v3/users/f1b49ae269454b179df6ebdc94a487ba` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:10.776009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f86ab-4b01-4896-86ec-bba528f04a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b49ae269454b179df6ebdc94a487ba)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:10.778814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f86ab-4b01-4896-86ec-bba528f04a67 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:10.879865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:10.892038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e68f86ab-4b01-4896-86ec-bba528f04a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b49ae269454b179df6ebdc94a487ba'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:10.952742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e68f86ab-4b01-4896-86ec-bba528f04a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b49ae269454b179df6ebdc94a487ba'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:10.995182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e68f86ab-4b01-4896-86ec-bba528f04a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b49ae269454b179df6ebdc94a487ba'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:10.997655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e68f86ab-4b01-4896-86ec-bba528f04a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b49ae269454b179df6ebdc94a487ba'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:10.999178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e68f86ab-4b01-4896-86ec-bba528f04a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b49ae269454b179df6ebdc94a487ba'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:11.047911 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1617/3195] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:10 2022] DELETE /identity/v3/users/f1b49ae269454b179df6ebdc94a487ba => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:12.618599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:12.620523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:12.649403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f592cf-0a36-4496-8d82-9bf4c34617e9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:12.692345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:12.947154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd803c-2423-4fd6-897b-37742003980e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:12.948255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd803c-2423-4fd6-897b-37742003980e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:12.949043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd803c-2423-4fd6-897b-37742003980e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:12.950486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd803c-2423-4fd6-897b-37742003980e admin admin] PATH_INFO: `/v3/projects/587a114de1624c6fb97b479e2395ef77` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:12.956501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd803c-2423-4fd6-897b-37742003980e admin admin] RBAC: Authorizing `identity:delete_project(project_id=587a114de1624c6fb97b479e2395ef77)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:12.964082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd803c-2423-4fd6-897b-37742003980e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:13.161534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f7fd803c-2423-4fd6-897b-37742003980e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587a114de1624c6fb97b479e2395ef77', 'request_id': 'req-f7fd803c-2423-4fd6-897b-37742003980e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:13.176650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f7fd803c-2423-4fd6-897b-37742003980e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587a114de1624c6fb97b479e2395ef77', 'request_id': 'req-f7fd803c-2423-4fd6-897b-37742003980e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:13.186678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f7fd803c-2423-4fd6-897b-37742003980e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587a114de1624c6fb97b479e2395ef77', 'request_id': 'req-f7fd803c-2423-4fd6-897b-37742003980e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:13.280990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f7fd803c-2423-4fd6-897b-37742003980e admin admin] The token cache is being invalidate because project 587a114de1624c6fb97b479e2395ef77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:13.281329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f7fd803c-2423-4fd6-897b-37742003980e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:13.282095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f7fd803c-2423-4fd6-897b-37742003980e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:13.284858 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1579/3196] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:12 2022] DELETE /identity/v3/projects/587a114de1624c6fb97b479e2395ef77 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:13.289846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c826796-a1d2-4400-a637-c9f68e12836a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:13.290347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c826796-a1d2-4400-a637-c9f68e12836a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:13.290418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c826796-a1d2-4400-a637-c9f68e12836a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:13.421491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5c826796-a1d2-4400-a637-c9f68e12836a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:13.423078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5c826796-a1d2-4400-a637-c9f68e12836a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:13.559055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfc998-9145-482b-bc1c-c3497ab74386 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:13.559055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfc998-9145-482b-bc1c-c3497ab74386 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:13.559430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfc998-9145-482b-bc1c-c3497ab74386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:13.559995 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1618/3197] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:13.606453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3fdfc998-9145-482b-bc1c-c3497ab74386 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:13.610295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3fdfc998-9145-482b-bc1c-c3497ab74386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:13.636645 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1580/3198] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:57:13.641737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c826796-a1d2-4400-a637-c9f68e12836a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:13.660163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c826796-a1d2-4400-a637-c9f68e12836a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:13.680655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5ac3d-b740-4347-92f7-49a5223979e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:13.681334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5ac3d-b740-4347-92f7-49a5223979e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:13.681630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5ac3d-b740-4347-92f7-49a5223979e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:13.681914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5ac3d-b740-4347-92f7-49a5223979e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:13.682594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5ac3d-b740-4347-92f7-49a5223979e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:13.682991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5ac3d-b740-4347-92f7-49a5223979e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:13.685042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5ac3d-b740-4347-92f7-49a5223979e9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:13.754927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfc998-9145-482b-bc1c-c3497ab74386 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:13.762521 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1619/3199] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:13.777732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfc998-9145-482b-bc1c-c3497ab74386 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:13.793718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86447490-d50a-427e-ae8a-3bfe1da77b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:13.794558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86447490-d50a-427e-ae8a-3bfe1da77b1d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:13.795392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86447490-d50a-427e-ae8a-3bfe1da77b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:13.795920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86447490-d50a-427e-ae8a-3bfe1da77b1d admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:13.799487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86447490-d50a-427e-ae8a-3bfe1da77b1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:13.802628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86447490-d50a-427e-ae8a-3bfe1da77b1d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:13.896849 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1581/3200] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:13 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:13.902873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5ac3d-b740-4347-92f7-49a5223979e9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:13.920012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5ac3d-b740-4347-92f7-49a5223979e9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:14.112012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2249e19-a05d-4066-a62c-a28ffc732e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:14.112738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2249e19-a05d-4066-a62c-a28ffc732e52 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:14.112979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2249e19-a05d-4066-a62c-a28ffc732e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:14.113265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2249e19-a05d-4066-a62c-a28ffc732e52 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:14.113970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2249e19-a05d-4066-a62c-a28ffc732e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:14.125783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2249e19-a05d-4066-a62c-a28ffc732e52 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:14.205544 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1620/3201] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:13 2022] POST /identity/v3/users => generated 288 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:14.209912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86447490-d50a-427e-ae8a-3bfe1da77b1d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:14.226173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86447490-d50a-427e-ae8a-3bfe1da77b1d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:14.251390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d25b2-871c-4a41-a574-0402d87b6932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:14.251844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d25b2-871c-4a41-a574-0402d87b6932 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:14.252665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d25b2-871c-4a41-a574-0402d87b6932 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:14.252980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d25b2-871c-4a41-a574-0402d87b6932 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:14.253651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d25b2-871c-4a41-a574-0402d87b6932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:14.255524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d25b2-871c-4a41-a574-0402d87b6932 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:14.296360 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1582/3202] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:14.301064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2249e19-a05d-4066-a62c-a28ffc732e52 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:14.317784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2249e19-a05d-4066-a62c-a28ffc732e52 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:14.331677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0638e-9aa9-4e0e-9914-1c4daad3eb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:14.332320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0638e-9aa9-4e0e-9914-1c4daad3eb16 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:14.332602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0638e-9aa9-4e0e-9914-1c4daad3eb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:14.332885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0638e-9aa9-4e0e-9914-1c4daad3eb16 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:14.335045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0638e-9aa9-4e0e-9914-1c4daad3eb16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:14.338407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0638e-9aa9-4e0e-9914-1c4daad3eb16 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:14.341050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:14.368030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d25b2-871c-4a41-a574-0402d87b6932 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:14.380574 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1621/3203] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:14.382786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d25b2-871c-4a41-a574-0402d87b6932 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:14.400376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18815ee5-156c-4bc9-8bef-0894eaa89f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:14.401260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18815ee5-156c-4bc9-8bef-0894eaa89f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:14.401489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18815ee5-156c-4bc9-8bef-0894eaa89f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:14.401850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18815ee5-156c-4bc9-8bef-0894eaa89f20 admin admin] PATH_INFO: `/v3/projects/df8d15077d42417ca165bf7400c3b87a/users/f52a9125d09348b8b00997797b49340e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:14.433824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18815ee5-156c-4bc9-8bef-0894eaa89f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8d15077d42417ca165bf7400c3b87a, user_id=f52a9125d09348b8b00997797b49340e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:14.436556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18815ee5-156c-4bc9-8bef-0894eaa89f20 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:14.487573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9f23-ceb2-484d-bf28-5e409503c129 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:14.487870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9f23-ceb2-484d-bf28-5e409503c129 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:14.488152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9f23-ceb2-484d-bf28-5e409503c129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:14.490903 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1583/3204] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:14 2022] PUT /identity/v3/projects/df8d15077d42417ca165bf7400c3b87a/users/f52a9125d09348b8b00997797b49340e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:14.545152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-185d9f23-ceb2-484d-bf28-5e409503c129 None None] MFA Rules not processed for user `f52a9125d09348b8b00997797b49340e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:14.546490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-185d9f23-ceb2-484d-bf28-5e409503c129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:14.686468 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1622/3205] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:14.691879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18815ee5-156c-4bc9-8bef-0894eaa89f20 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:14.765263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:14.820669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18815ee5-156c-4bc9-8bef-0894eaa89f20 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:14.832702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25918049-e92c-4170-941c-86a3b539e090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:14.833696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25918049-e92c-4170-941c-86a3b539e090 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:14.834133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25918049-e92c-4170-941c-86a3b539e090 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:14.834558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25918049-e92c-4170-941c-86a3b539e090 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:14.835346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25918049-e92c-4170-941c-86a3b539e090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:14.837544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25918049-e92c-4170-941c-86a3b539e090 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:14.885372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d9f23-ceb2-484d-bf28-5e409503c129 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:14.889565 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1584/3206] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:14 2022] POST /identity/v3/projects => generated 373 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:14.901671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d9f23-ceb2-484d-bf28-5e409503c129 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:14.999997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f47e7b-695c-4922-9259-3dc2a7823383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:15.001225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f47e7b-695c-4922-9259-3dc2a7823383 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:15.001660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f47e7b-695c-4922-9259-3dc2a7823383 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:15.009359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f47e7b-695c-4922-9259-3dc2a7823383 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:15.010228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f47e7b-695c-4922-9259-3dc2a7823383 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:15.013721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f47e7b-695c-4922-9259-3dc2a7823383 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:15.056038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25918049-e92c-4170-941c-86a3b539e090 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:15.069238 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1623/3207] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:14 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:15.072044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25918049-e92c-4170-941c-86a3b539e090 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:15.084299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bd644-2e3b-448d-9726-dfeebf905b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:15.087198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bd644-2e3b-448d-9726-dfeebf905b5e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:15.087569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bd644-2e3b-448d-9726-dfeebf905b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:15.087929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bd644-2e3b-448d-9726-dfeebf905b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:15.090257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bd644-2e3b-448d-9726-dfeebf905b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:15.092041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bd644-2e3b-448d-9726-dfeebf905b5e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:15.114989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f47e7b-695c-4922-9259-3dc2a7823383 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:15.126343 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1585/3208] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:15.129031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f47e7b-695c-4922-9259-3dc2a7823383 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:15.135465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39b4c3-7a38-4de3-a96e-c05b41ea27bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:15.136275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39b4c3-7a38-4de3-a96e-c05b41ea27bf admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:15.136662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39b4c3-7a38-4de3-a96e-c05b41ea27bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:15.137146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39b4c3-7a38-4de3-a96e-c05b41ea27bf admin admin] PATH_INFO: `/v3/projects/191d3406ce1148958ea336d84b8e4ff9/users/914f4feb06074937aa175ff8ca0b31c5/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:15.156424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39b4c3-7a38-4de3-a96e-c05b41ea27bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=191d3406ce1148958ea336d84b8e4ff9, user_id=914f4feb06074937aa175ff8ca0b31c5, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:15.161189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39b4c3-7a38-4de3-a96e-c05b41ea27bf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:15.182610 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1624/3209] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:15 2022] PUT /identity/v3/projects/191d3406ce1148958ea336d84b8e4ff9/users/914f4feb06074937aa175ff8ca0b31c5/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:15.184046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bd644-2e3b-448d-9726-dfeebf905b5e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:15.199563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bd644-2e3b-448d-9726-dfeebf905b5e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:15.291305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71709b-3d52-4d61-a5c0-1b6270adf789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:15.292419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71709b-3d52-4d61-a5c0-1b6270adf789 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:15.293310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71709b-3d52-4d61-a5c0-1b6270adf789 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:15.293815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71709b-3d52-4d61-a5c0-1b6270adf789 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:15.294997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71709b-3d52-4d61-a5c0-1b6270adf789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:15.300666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71709b-3d52-4d61-a5c0-1b6270adf789 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:15.322699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39b4c3-7a38-4de3-a96e-c05b41ea27bf admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:15.323613 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1586/3210] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:15.351626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39b4c3-7a38-4de3-a96e-c05b41ea27bf admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:15.358639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78767e72-1e38-48d2-9a1d-8dd00c8af239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:15.359699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78767e72-1e38-48d2-9a1d-8dd00c8af239 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:15.360137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78767e72-1e38-48d2-9a1d-8dd00c8af239 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:15.360568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78767e72-1e38-48d2-9a1d-8dd00c8af239 admin admin] PATH_INFO: `/v3/projects/191d3406ce1148958ea336d84b8e4ff9/users/914f4feb06074937aa175ff8ca0b31c5/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:15.365064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78767e72-1e38-48d2-9a1d-8dd00c8af239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=191d3406ce1148958ea336d84b8e4ff9, user_id=914f4feb06074937aa175ff8ca0b31c5, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:15.368560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78767e72-1e38-48d2-9a1d-8dd00c8af239 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:15.397788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e23e09-27f2-445b-9ec8-54966313f17e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:15.401116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e23e09-27f2-445b-9ec8-54966313f17e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:15.402030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e23e09-27f2-445b-9ec8-54966313f17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:15.418263 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1625/3211] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:15 2022] PUT /identity/v3/projects/191d3406ce1148958ea336d84b8e4ff9/users/914f4feb06074937aa175ff8ca0b31c5/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:15.453004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-13e23e09-27f2-445b-9ec8-54966313f17e None None] MFA Rules not processed for user `914f4feb06074937aa175ff8ca0b31c5`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:15.456221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-13e23e09-27f2-445b-9ec8-54966313f17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:15.629922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78767e72-1e38-48d2-9a1d-8dd00c8af239 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:15.640756 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1587/3212] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:15.649977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78767e72-1e38-48d2-9a1d-8dd00c8af239 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:15.670080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59985b-8a23-4d89-baea-097cbf9d5cda tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:15.670781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59985b-8a23-4d89-baea-097cbf9d5cda tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:15.671077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59985b-8a23-4d89-baea-097cbf9d5cda tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:15.671365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59985b-8a23-4d89-baea-097cbf9d5cda tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:15.672108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59985b-8a23-4d89-baea-097cbf9d5cda tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:15.672426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59985b-8a23-4d89-baea-097cbf9d5cda tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:15.677233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59985b-8a23-4d89-baea-097cbf9d5cda tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:15.717106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e23e09-27f2-445b-9ec8-54966313f17e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:15.728153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e23e09-27f2-445b-9ec8-54966313f17e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:15.732205 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1626/3213] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:15.755363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b3fca-19e8-4757-bc01-6c56f794a130 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:15.759465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b3fca-19e8-4757-bc01-6c56f794a130 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:15.759768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b3fca-19e8-4757-bc01-6c56f794a130 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:15.760047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b3fca-19e8-4757-bc01-6c56f794a130 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:15.760727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b3fca-19e8-4757-bc01-6c56f794a130 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:15.768992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b3fca-19e8-4757-bc01-6c56f794a130 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:15.832845 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1588/3214] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:15 2022] POST /identity/v3/projects => generated 369 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:57:15.836411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59985b-8a23-4d89-baea-097cbf9d5cda tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:15.865743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59985b-8a23-4d89-baea-097cbf9d5cda tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:16.031931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:16.204533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eff3ef-7c18-4a69-87cf-9480da7048cd tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:16.210313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eff3ef-7c18-4a69-87cf-9480da7048cd tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:16.210613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eff3ef-7c18-4a69-87cf-9480da7048cd tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:16.210894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eff3ef-7c18-4a69-87cf-9480da7048cd tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:16.219376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eff3ef-7c18-4a69-87cf-9480da7048cd tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:16.219804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eff3ef-7c18-4a69-87cf-9480da7048cd tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:16.231471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eff3ef-7c18-4a69-87cf-9480da7048cd tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:16.269985 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1627/3215] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:16.423114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b3fca-19e8-4757-bc01-6c56f794a130 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:16.428869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1c3b3fca-19e8-4757-bc01-6c56f794a130 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:16.461467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:16.786417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b3fca-19e8-4757-bc01-6c56f794a130 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:16.796854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f29a36-11c3-4b07-b413-51c87cdda179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:16.798423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f29a36-11c3-4b07-b413-51c87cdda179 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:16.799038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f29a36-11c3-4b07-b413-51c87cdda179 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:16.799700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f29a36-11c3-4b07-b413-51c87cdda179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:16.810825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f29a36-11c3-4b07-b413-51c87cdda179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:16.824459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f29a36-11c3-4b07-b413-51c87cdda179 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:16.855843 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1589/3216] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:16.865906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71578c-c5e7-403e-ba7a-742be2d25f80 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:16.866478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71578c-c5e7-403e-ba7a-742be2d25f80 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:16.866762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71578c-c5e7-403e-ba7a-742be2d25f80 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:16.870606 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1628/3217] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:57:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:57:16.876306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f29a36-11c3-4b07-b413-51c87cdda179 service nova] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:57:16.891959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f29a36-11c3-4b07-b413-51c87cdda179 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:16.913872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f29a36-11c3-4b07-b413-51c87cdda179 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:16.922202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fc671-0d5e-413f-820d-f7b92b312ab0 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:16.923201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fc671-0d5e-413f-820d-f7b92b312ab0 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:16.923201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fc671-0d5e-413f-820d-f7b92b312ab0 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:16.923481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fc671-0d5e-413f-820d-f7b92b312ab0 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/f7a9688e86564b39b3b7a800dff9cb11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:16.925762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fc671-0d5e-413f-820d-f7b92b312ab0 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:get_project(project_id=f7a9688e86564b39b3b7a800dff9cb11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:16.927980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fc671-0d5e-413f-820d-f7b92b312ab0 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:16.933461 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1590/3218] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:57:16 2022] GET /identity/v3/projects/f7a9688e86564b39b3b7a800dff9cb11 => generated 369 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:17.095058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f48df-d92b-4f5b-a72c-7e5b4b92b827 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:17.095653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f48df-d92b-4f5b-a72c-7e5b4b92b827 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:17.096748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f48df-d92b-4f5b-a72c-7e5b4b92b827 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:17.098642 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1629/3219] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:57:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:57:17.103371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fc671-0d5e-413f-820d-f7b92b312ab0 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:57:17.126096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fc671-0d5e-413f-820d-f7b92b312ab0 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:17.147856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fc671-0d5e-413f-820d-f7b92b312ab0 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:17.154782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a21e-4ebb-4e76-b57a-f3e02cfffeb7 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:17.155454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a21e-4ebb-4e76-b57a-f3e02cfffeb7 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:17.155749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a21e-4ebb-4e76-b57a-f3e02cfffeb7 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:17.156033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a21e-4ebb-4e76-b57a-f3e02cfffeb7 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/f7a9688e86564b39b3b7a800dff9cb11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:17.165914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528a21e-4ebb-4e76-b57a-f3e02cfffeb7 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:get_project(project_id=f7a9688e86564b39b3b7a800dff9cb11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:17.168041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528a21e-4ebb-4e76-b57a-f3e02cfffeb7 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:17.181561 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1591/3220] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:57:17 2022] GET /identity/v3/projects/f7a9688e86564b39b3b7a800dff9cb11 => generated 369 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:17.350144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce889dd6-be74-420c-8b07-caa923a985a1 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:17.350681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce889dd6-be74-420c-8b07-caa923a985a1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:17.351109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce889dd6-be74-420c-8b07-caa923a985a1 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:17.352707 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1630/3221] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:57:17.357972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a21e-4ebb-4e76-b57a-f3e02cfffeb7 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:57:17.388282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a21e-4ebb-4e76-b57a-f3e02cfffeb7 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:17.411770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a21e-4ebb-4e76-b57a-f3e02cfffeb7 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:17.427606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaa4f4-01e8-45fd-8df9-a946b883707d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:17.428781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa4f4-01e8-45fd-8df9-a946b883707d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:17.429333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa4f4-01e8-45fd-8df9-a946b883707d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:17.429924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa4f4-01e8-45fd-8df9-a946b883707d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/f7a9688e86564b39b3b7a800dff9cb11` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:17.431372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baaa4f4-01e8-45fd-8df9-a946b883707d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:get_project(project_id=f7a9688e86564b39b3b7a800dff9cb11)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:17.433628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baaa4f4-01e8-45fd-8df9-a946b883707d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:17.435830 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1592/3222] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:57:17 2022] GET /identity/v3/projects/f7a9688e86564b39b3b7a800dff9cb11 => generated 369 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:57:17.466281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce889dd6-be74-420c-8b07-caa923a985a1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:17.487711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce889dd6-be74-420c-8b07-caa923a985a1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:17.501186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4728c-8c0c-44f8-86f5-89d412500f61 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:17.502118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4728c-8c0c-44f8-86f5-89d412500f61 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:17.503045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4728c-8c0c-44f8-86f5-89d412500f61 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:17.503475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4728c-8c0c-44f8-86f5-89d412500f61 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/f7a9688e86564b39b3b7a800dff9cb11` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:17.505613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4728c-8c0c-44f8-86f5-89d412500f61 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f7a9688e86564b39b3b7a800dff9cb11)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:17.511554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4728c-8c0c-44f8-86f5-89d412500f61 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:17.622607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09c4728c-8c0c-44f8-86f5-89d412500f61 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a9688e86564b39b3b7a800dff9cb11', 'request_id': 'req-09c4728c-8c0c-44f8-86f5-89d412500f61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:17.632686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09c4728c-8c0c-44f8-86f5-89d412500f61 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a9688e86564b39b3b7a800dff9cb11', 'request_id': 'req-09c4728c-8c0c-44f8-86f5-89d412500f61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:17.642683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaa4f4-01e8-45fd-8df9-a946b883707d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:17.644811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0baaa4f4-01e8-45fd-8df9-a946b883707d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:17.645938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09c4728c-8c0c-44f8-86f5-89d412500f61 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a9688e86564b39b3b7a800dff9cb11', 'request_id': 'req-09c4728c-8c0c-44f8-86f5-89d412500f61'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:17.658614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09c4728c-8c0c-44f8-86f5-89d412500f61 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] The token cache is being invalidate because project f7a9688e86564b39b3b7a800dff9cb11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:17.659131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09c4728c-8c0c-44f8-86f5-89d412500f61 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:17.660003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09c4728c-8c0c-44f8-86f5-89d412500f61 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:17.679101 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1631/3223] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:17 2022] DELETE /identity/v3/projects/f7a9688e86564b39b3b7a800dff9cb11 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:17.682069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a2c72-26ef-4339-a2db-dfc9b12b0211 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:17.682678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a2c72-26ef-4339-a2db-dfc9b12b0211 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:17.683162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a2c72-26ef-4339-a2db-dfc9b12b0211 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:17.689324 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1632/3224] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:57:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:57:17.691523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a2c72-26ef-4339-a2db-dfc9b12b0211 None None] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:57:17.694155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-677a2c72-26ef-4339-a2db-dfc9b12b0211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:17.790986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a2c72-26ef-4339-a2db-dfc9b12b0211 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:17.792508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-677a2c72-26ef-4339-a2db-dfc9b12b0211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:17.841590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:17.844198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaa4f4-01e8-45fd-8df9-a946b883707d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:17.878142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a2c72-26ef-4339-a2db-dfc9b12b0211 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:17.884846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224c74e-dc6a-4409-82c8-1c4403fed6dc tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:17.886169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224c74e-dc6a-4409-82c8-1c4403fed6dc tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:17.886660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224c74e-dc6a-4409-82c8-1c4403fed6dc tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:17.887142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224c74e-dc6a-4409-82c8-1c4403fed6dc tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/df8d15077d42417ca165bf7400c3b87a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:17.888568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224c74e-dc6a-4409-82c8-1c4403fed6dc tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:get_project(project_id=df8d15077d42417ca165bf7400c3b87a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:17.891269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224c74e-dc6a-4409-82c8-1c4403fed6dc tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:17.914155 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1633/3225] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:57:17 2022] GET /identity/v3/projects/df8d15077d42417ca165bf7400c3b87a => generated 375 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:17.915595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224c74e-dc6a-4409-82c8-1c4403fed6dc tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:17.927304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:17.930163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224c74e-dc6a-4409-82c8-1c4403fed6dc tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:17.936786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a78f98-ee9c-4e1a-bde4-44baeb088b3b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:17.937763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a78f98-ee9c-4e1a-bde4-44baeb088b3b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:17.938259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a78f98-ee9c-4e1a-bde4-44baeb088b3b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:17.938744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a78f98-ee9c-4e1a-bde4-44baeb088b3b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:17.940185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a78f98-ee9c-4e1a-bde4-44baeb088b3b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:17.940734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a78f98-ee9c-4e1a-bde4-44baeb088b3b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:17.943552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a78f98-ee9c-4e1a-bde4-44baeb088b3b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:17.953232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dab74b-c9f3-4fd8-ab77-ec0e87673719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:17.954033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dab74b-c9f3-4fd8-ab77-ec0e87673719 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:17.954459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dab74b-c9f3-4fd8-ab77-ec0e87673719 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:17.954878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dab74b-c9f3-4fd8-ab77-ec0e87673719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:17.963926 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1634/3226] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:17.964601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b5dab74b-c9f3-4fd8-ab77-ec0e87673719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:17.965412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a78f98-ee9c-4e1a-bde4-44baeb088b3b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:17.977346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a78f98-ee9c-4e1a-bde4-44baeb088b3b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:17.982929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c731420-1dd3-47af-9573-4260d7780f3d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:17.983812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c731420-1dd3-47af-9573-4260d7780f3d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:17.984385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c731420-1dd3-47af-9573-4260d7780f3d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:17.984871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c731420-1dd3-47af-9573-4260d7780f3d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:17.985780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c731420-1dd3-47af-9573-4260d7780f3d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:17.988029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c731420-1dd3-47af-9573-4260d7780f3d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:18.028002 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1635/3227] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:17 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:18.029322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c731420-1dd3-47af-9573-4260d7780f3d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:18.100124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dab74b-c9f3-4fd8-ab77-ec0e87673719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:18.102180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dab74b-c9f3-4fd8-ab77-ec0e87673719 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:18.120207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c731420-1dd3-47af-9573-4260d7780f3d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:18.124254 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1593/3228] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:18.126080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c049c-519e-4694-a39b-3533cb2db481 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:18.126956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c049c-519e-4694-a39b-3533cb2db481 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:18.127571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c049c-519e-4694-a39b-3533cb2db481 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:18.128125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c049c-519e-4694-a39b-3533cb2db481 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:18.129155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c049c-519e-4694-a39b-3533cb2db481 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:18.129683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c049c-519e-4694-a39b-3533cb2db481 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:18.131723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c049c-519e-4694-a39b-3533cb2db481 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:18.168304 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1636/3229] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:18.169931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d56a6-20fb-42a8-9856-b6476c18f716 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:18.170291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d56a6-20fb-42a8-9856-b6476c18f716 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:18.170575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d56a6-20fb-42a8-9856-b6476c18f716 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:18.171928 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1594/3230] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 20:57:18.178966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c049c-519e-4694-a39b-3533cb2db481 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:57:18.295090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c049c-519e-4694-a39b-3533cb2db481 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:18.320552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c049c-519e-4694-a39b-3533cb2db481 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:18.338637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2474319-2af1-44c7-bc11-d6302e889716 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:18.340420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2474319-2af1-44c7-bc11-d6302e889716 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:18.340627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2474319-2af1-44c7-bc11-d6302e889716 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:18.340965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2474319-2af1-44c7-bc11-d6302e889716 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/682deb5531564b5db5b94ebad74c6d4e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:18.345545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2474319-2af1-44c7-bc11-d6302e889716 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:get_project(project_id=682deb5531564b5db5b94ebad74c6d4e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:18.349822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2474319-2af1-44c7-bc11-d6302e889716 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:18.352400 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1637/3231] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:57:18 2022] GET /identity/v3/projects/682deb5531564b5db5b94ebad74c6d4e => generated 371 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:18.482198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701daad2-baaf-4607-9521-b968892dbd90 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:18.482709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701daad2-baaf-4607-9521-b968892dbd90 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:18.482800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701daad2-baaf-4607-9521-b968892dbd90 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:18.484230 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1595/3232] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:57:18.491065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2474319-2af1-44c7-bc11-d6302e889716 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:57:18.511215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2474319-2af1-44c7-bc11-d6302e889716 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:18.527491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2474319-2af1-44c7-bc11-d6302e889716 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:18.533052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b656552-3151-4224-a45e-0f6017bdf68d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:18.537391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b656552-3151-4224-a45e-0f6017bdf68d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:18.538567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b656552-3151-4224-a45e-0f6017bdf68d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:18.539070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b656552-3151-4224-a45e-0f6017bdf68d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/682deb5531564b5db5b94ebad74c6d4e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:18.541348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b656552-3151-4224-a45e-0f6017bdf68d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:get_project(project_id=682deb5531564b5db5b94ebad74c6d4e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:18.543931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b656552-3151-4224-a45e-0f6017bdf68d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:18.558522 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1638/3233] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:57:18 2022] GET /identity/v3/projects/682deb5531564b5db5b94ebad74c6d4e => generated 371 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:18.609248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701daad2-baaf-4607-9521-b968892dbd90 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:18.628195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701daad2-baaf-4607-9521-b968892dbd90 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:18.635516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d49784-cec5-4be6-b3e6-174f50094910 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:18.636294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d49784-cec5-4be6-b3e6-174f50094910 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:18.636686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d49784-cec5-4be6-b3e6-174f50094910 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:18.637251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d49784-cec5-4be6-b3e6-174f50094910 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:18.638106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d49784-cec5-4be6-b3e6-174f50094910 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:18.638463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d49784-cec5-4be6-b3e6-174f50094910 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:18.640930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d49784-cec5-4be6-b3e6-174f50094910 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:18.673486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b656552-3151-4224-a45e-0f6017bdf68d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:18.685834 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1596/3234] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:18.688700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b656552-3151-4224-a45e-0f6017bdf68d tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:18.696540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9064e97-b63d-4fe8-87f0-47fa1abf99b2 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:18.697504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9064e97-b63d-4fe8-87f0-47fa1abf99b2 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:18.697921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9064e97-b63d-4fe8-87f0-47fa1abf99b2 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:18.698360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9064e97-b63d-4fe8-87f0-47fa1abf99b2 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:18.699545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9064e97-b63d-4fe8-87f0-47fa1abf99b2 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:18.702140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9064e97-b63d-4fe8-87f0-47fa1abf99b2 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:18.753243 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1639/3235] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:18 2022] POST /identity/v3/users => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:18.754584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d49784-cec5-4be6-b3e6-174f50094910 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:18.770463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d49784-cec5-4be6-b3e6-174f50094910 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:18.784560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0505b-6fb1-40da-9391-730363473c17 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:18.785550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0505b-6fb1-40da-9391-730363473c17 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:18.787145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0505b-6fb1-40da-9391-730363473c17 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:18.787592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0505b-6fb1-40da-9391-730363473c17 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:18.790467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0505b-6fb1-40da-9391-730363473c17 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:18.791387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0505b-6fb1-40da-9391-730363473c17 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:18.795070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0505b-6fb1-40da-9391-730363473c17 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:18.849154 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1597/3236] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:18.866574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d916df-c45b-46cc-a9f1-5624d3d19ac1 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:18.867286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d916df-c45b-46cc-a9f1-5624d3d19ac1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:18.867286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d916df-c45b-46cc-a9f1-5624d3d19ac1 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:18.870699 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1640/3237] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:57:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:57:18.878842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0505b-6fb1-40da-9391-730363473c17 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:57:18.880840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-56e0505b-6fb1-40da-9391-730363473c17 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:18.904728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0505b-6fb1-40da-9391-730363473c17 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:18.922463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0505b-6fb1-40da-9391-730363473c17 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:18.940166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5145b4-3438-4ae1-95a8-2624e2eddddb tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:18.941141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5145b4-3438-4ae1-95a8-2624e2eddddb tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:18.941579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5145b4-3438-4ae1-95a8-2624e2eddddb tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:18.942077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5145b4-3438-4ae1-95a8-2624e2eddddb tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/682deb5531564b5db5b94ebad74c6d4e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:18.944166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5145b4-3438-4ae1-95a8-2624e2eddddb tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:get_project(project_id=682deb5531564b5db5b94ebad74c6d4e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:18.949114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5145b4-3438-4ae1-95a8-2624e2eddddb tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:18.965029 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1598/3238] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:57:18 2022] GET /identity/v3/projects/682deb5531564b5db5b94ebad74c6d4e => generated 371 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:19.258510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5219d1-a95f-4d0b-95c8-9298615d8417 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:19.260025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5219d1-a95f-4d0b-95c8-9298615d8417 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:19.260401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5219d1-a95f-4d0b-95c8-9298615d8417 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:19.265415 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1641/3239] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:57:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:57:19.267232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5145b4-3438-4ae1-95a8-2624e2eddddb tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:57:19.309439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5145b4-3438-4ae1-95a8-2624e2eddddb tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:19.325334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5145b4-3438-4ae1-95a8-2624e2eddddb tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:19.332106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1849c8-177f-4a3b-a405-53667113d057 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:19.332879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1849c8-177f-4a3b-a405-53667113d057 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:19.333276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1849c8-177f-4a3b-a405-53667113d057 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:19.333675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1849c8-177f-4a3b-a405-53667113d057 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/682deb5531564b5db5b94ebad74c6d4e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:19.334959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1849c8-177f-4a3b-a405-53667113d057 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:get_project(project_id=682deb5531564b5db5b94ebad74c6d4e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:19.338137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1849c8-177f-4a3b-a405-53667113d057 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:19.343451 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1599/3240] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:57:19 2022] GET /identity/v3/projects/682deb5531564b5db5b94ebad74c6d4e => generated 371 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:19.379985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5219d1-a95f-4d0b-95c8-9298615d8417 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:19.389463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:19.399369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5219d1-a95f-4d0b-95c8-9298615d8417 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:19.406369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaee96-355c-48b7-a717-4c47e5f13b14 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:19.407241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaee96-355c-48b7-a717-4c47e5f13b14 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:19.407660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaee96-355c-48b7-a717-4c47e5f13b14 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:19.408047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaee96-355c-48b7-a717-4c47e5f13b14 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/users/e6d25936a3154576ada571f8e0ca4369` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:19.430590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebaee96-355c-48b7-a717-4c47e5f13b14 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:delete_user(user_id=e6d25936a3154576ada571f8e0ca4369)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:19.433047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebaee96-355c-48b7-a717-4c47e5f13b14 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:19.534740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ebaee96-355c-48b7-a717-4c47e5f13b14 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6d25936a3154576ada571f8e0ca4369'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:19.564483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ebaee96-355c-48b7-a717-4c47e5f13b14 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6d25936a3154576ada571f8e0ca4369'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:19.579330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ebaee96-355c-48b7-a717-4c47e5f13b14 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6d25936a3154576ada571f8e0ca4369'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:19.580025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ebaee96-355c-48b7-a717-4c47e5f13b14 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6d25936a3154576ada571f8e0ca4369'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:19.581419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ebaee96-355c-48b7-a717-4c47e5f13b14 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6d25936a3154576ada571f8e0ca4369'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:19.603224 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1642/3241] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:19 2022] DELETE /identity/v3/users/e6d25936a3154576ada571f8e0ca4369 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:19.607590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1849c8-177f-4a3b-a405-53667113d057 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:19.611119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fb1849c8-177f-4a3b-a405-53667113d057 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:19.629460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1849c8-177f-4a3b-a405-53667113d057 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:19.754503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:19.759046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:19.759736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:19.760197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/682deb5531564b5db5b94ebad74c6d4e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:19.776353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:delete_project(project_id=682deb5531564b5db5b94ebad74c6d4e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:19.780679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:19.875603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '682deb5531564b5db5b94ebad74c6d4e', 'request_id': 'req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:19.878093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:19.887440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '682deb5531564b5db5b94ebad74c6d4e', 'request_id': 'req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:19.897165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '682deb5531564b5db5b94ebad74c6d4e', 'request_id': 'req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:19.924891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] The token cache is being invalidate because project 682deb5531564b5db5b94ebad74c6d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:19.925195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:19.926435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:19.934200 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1600/3242] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:19 2022] DELETE /identity/v3/projects/682deb5531564b5db5b94ebad74c6d4e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:19.946790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3feca0-7407-45a1-96af-6f72b5e3b352 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:19.948865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3feca0-7407-45a1-96af-6f72b5e3b352 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:19.949647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3feca0-7407-45a1-96af-6f72b5e3b352 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:19.951108 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1643/3243] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:57:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:57:19.956786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:57:19.959195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:20.060781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:20.062449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:20.267300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c061a8f-4c66-4c48-9bfa-d8d0a11e6c09 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:20.274668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9757c-d85a-4a8d-987b-d2a27fb5536b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:20.275343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9757c-d85a-4a8d-987b-d2a27fb5536b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:20.275628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9757c-d85a-4a8d-987b-d2a27fb5536b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:20.275912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9757c-d85a-4a8d-987b-d2a27fb5536b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/df8d15077d42417ca165bf7400c3b87a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:20.277123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9757c-d85a-4a8d-987b-d2a27fb5536b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:get_project(project_id=df8d15077d42417ca165bf7400c3b87a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:20.279949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9757c-d85a-4a8d-987b-d2a27fb5536b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:20.284412 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1601/3244] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:57:19 2022] GET /identity/v3/projects/df8d15077d42417ca165bf7400c3b87a => generated 375 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:57:20.318273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cec0c-77be-4c25-a2a5-bb6d2aadd6c3 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:20.319083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cec0c-77be-4c25-a2a5-bb6d2aadd6c3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:20.319504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cec0c-77be-4c25-a2a5-bb6d2aadd6c3 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:20.321326 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1644/3245] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:57:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:57:20.331446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9757c-d85a-4a8d-987b-d2a27fb5536b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:57:20.360207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9757c-d85a-4a8d-987b-d2a27fb5536b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:20.376676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9757c-d85a-4a8d-987b-d2a27fb5536b tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:20.385139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f8a75-06bf-4fb4-b0bd-a982e4daefc9 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:20.385728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f8a75-06bf-4fb4-b0bd-a982e4daefc9 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:20.386016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f8a75-06bf-4fb4-b0bd-a982e4daefc9 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:20.386297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f8a75-06bf-4fb4-b0bd-a982e4daefc9 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/df8d15077d42417ca165bf7400c3b87a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:20.389823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f8a75-06bf-4fb4-b0bd-a982e4daefc9 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:get_project(project_id=df8d15077d42417ca165bf7400c3b87a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:20.392102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f8a75-06bf-4fb4-b0bd-a982e4daefc9 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:20.409146 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1602/3246] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:57:20 2022] GET /identity/v3/projects/df8d15077d42417ca165bf7400c3b87a => generated 375 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:20.681281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410ebf6-be95-44c4-bb9d-8564acaa584f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:20.681563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410ebf6-be95-44c4-bb9d-8564acaa584f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:20.681848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410ebf6-be95-44c4-bb9d-8564acaa584f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:20.770189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2410ebf6-be95-44c4-bb9d-8564acaa584f None None] MFA Rules not processed for user `7838d64ab35e4608b668e9f65d594d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:20.773122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2410ebf6-be95-44c4-bb9d-8564acaa584f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:20.924527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194975f-a6d1-4795-837c-f0048558ed8f None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:20.925080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194975f-a6d1-4795-837c-f0048558ed8f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:20.925925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194975f-a6d1-4795-837c-f0048558ed8f None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:20.927763 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1603/3247] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 20:57:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 20:57:20.932978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194975f-a6d1-4795-837c-f0048558ed8f None None] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 20:57:20.956349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194975f-a6d1-4795-837c-f0048558ed8f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:20.963452 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1645/3248] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:20.967670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410ebf6-be95-44c4-bb9d-8564acaa584f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:20.969356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2410ebf6-be95-44c4-bb9d-8564acaa584f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:21.000088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194975f-a6d1-4795-837c-f0048558ed8f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:21.014951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5c02c-c06a-4784-b4aa-3a7f691dc7b4 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '914f4feb06074937aa175ff8ca0b31c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191d3406ce1148958ea336d84b8e4ff9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:21.016387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5c02c-c06a-4784-b4aa-3a7f691dc7b4 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:21.016847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5c02c-c06a-4784-b4aa-3a7f691dc7b4 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:21.017186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5c02c-c06a-4784-b4aa-3a7f691dc7b4 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] PATH_INFO: `/v3/projects/df8d15077d42417ca165bf7400c3b87a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:21.018916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5c02c-c06a-4784-b4aa-3a7f691dc7b4 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorizing `identity:get_project(project_id=df8d15077d42417ca165bf7400c3b87a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:21.024405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5c02c-c06a-4784-b4aa-3a7f691dc7b4 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:21.038294 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1604/3249] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 20:57:20 2022] GET /identity/v3/projects/df8d15077d42417ca165bf7400c3b87a => generated 375 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:57:21.089873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:21.122241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410ebf6-be95-44c4-bb9d-8564acaa584f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:21.135072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed4929-3a00-4937-984c-c039919c81c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:21.135732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed4929-3a00-4937-984c-c039919c81c5 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:21.136022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed4929-3a00-4937-984c-c039919c81c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:21.136304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed4929-3a00-4937-984c-c039919c81c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:21.137002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed4929-3a00-4937-984c-c039919c81c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:21.138992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed4929-3a00-4937-984c-c039919c81c5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:21.208489 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1646/3250] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:20 2022] POST /identity/v3/projects => generated 387 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:21.209891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5c02c-c06a-4784-b4aa-3a7f691dc7b4 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:21.233764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5c02c-c06a-4784-b4aa-3a7f691dc7b4 tempest-QuotasAdminTestJSON-453852387 tempest-QuotasAdminTestJSON-453852387-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:21.365997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:21.381723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59658b59-3416-4c70-be64-635cca3e87f1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:21.382192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59658b59-3416-4c70-be64-635cca3e87f1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:21.382298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59658b59-3416-4c70-be64-635cca3e87f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:21.391936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036df1b-396b-4c2f-b922-5ceedd37a1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:21.393733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036df1b-396b-4c2f-b922-5ceedd37a1ce admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:21.394026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036df1b-396b-4c2f-b922-5ceedd37a1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:21.394306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036df1b-396b-4c2f-b922-5ceedd37a1ce admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:21.395886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036df1b-396b-4c2f-b922-5ceedd37a1ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:21.403267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036df1b-396b-4c2f-b922-5ceedd37a1ce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:21.434589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-59658b59-3416-4c70-be64-635cca3e87f1 None None] MFA Rules not processed for user `db307623f73f4532a96342f551ba011c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:21.435961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-59658b59-3416-4c70-be64-635cca3e87f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:21.458186 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1605/3251] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:21 2022] POST /identity/v3/users => generated 300 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:21.460117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036df1b-396b-4c2f-b922-5ceedd37a1ce admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:21.507491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036df1b-396b-4c2f-b922-5ceedd37a1ce admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:21.528751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfe3a8-0e95-45b4-9d6b-d70133f48d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:21.529423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfe3a8-0e95-45b4-9d6b-d70133f48d82 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:21.529717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfe3a8-0e95-45b4-9d6b-d70133f48d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:21.530028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfe3a8-0e95-45b4-9d6b-d70133f48d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:21.530748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfe3a8-0e95-45b4-9d6b-d70133f48d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:21.534314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfe3a8-0e95-45b4-9d6b-d70133f48d82 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:21.564512 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1606/3252] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:21.572651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a1330-7b00-409c-9441-31226529ae12 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:21.573885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a1330-7b00-409c-9441-31226529ae12 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:21.574170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a1330-7b00-409c-9441-31226529ae12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:21.623403 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1647/3253] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:21.627480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59658b59-3416-4c70-be64-635cca3e87f1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:21.636081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e58a1330-7b00-409c-9441-31226529ae12 None None] MFA Rules not processed for user `3da1337b27b64e008b26a2c464578913`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:21.639056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e58a1330-7b00-409c-9441-31226529ae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:21.660292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59658b59-3416-4c70-be64-635cca3e87f1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:21.675140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb03967-3e2e-419e-a848-42a9448c3968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:21.676684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb03967-3e2e-419e-a848-42a9448c3968 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:21.677262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb03967-3e2e-419e-a848-42a9448c3968 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:21.686494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb03967-3e2e-419e-a848-42a9448c3968 admin admin] PATH_INFO: `/v3/projects/c9ae9fb6f6f8493abaf3857f81992925/users/94d4c06fa26e400ead26ab08cd86bb43/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:21.718673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb03967-3e2e-419e-a848-42a9448c3968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ae9fb6f6f8493abaf3857f81992925, user_id=94d4c06fa26e400ead26ab08cd86bb43, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:21.723184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb03967-3e2e-419e-a848-42a9448c3968 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:21.761605 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1648/3254] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:21 2022] PUT /identity/v3/projects/c9ae9fb6f6f8493abaf3857f81992925/users/94d4c06fa26e400ead26ab08cd86bb43/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:21.762429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb03967-3e2e-419e-a848-42a9448c3968 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:21.765060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5cb03967-3e2e-419e-a848-42a9448c3968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:21.823351 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1607/3255] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:21.823351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a1330-7b00-409c-9441-31226529ae12 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:21.852168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a1330-7b00-409c-9441-31226529ae12 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:22.000545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb03967-3e2e-419e-a848-42a9448c3968 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:22.025908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:22.030716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:22.031550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:22.032650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] PATH_INFO: `/v3/users/f52a9125d09348b8b00997797b49340e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:22.035237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8c379-29b2-4872-9d02-b831938bdece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:22.036494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f52a9125d09348b8b00997797b49340e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:22.038381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8c379-29b2-4872-9d02-b831938bdece admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:22.054238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8c379-29b2-4872-9d02-b831938bdece admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:22.054585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8c379-29b2-4872-9d02-b831938bdece admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:22.058163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8c379-29b2-4872-9d02-b831938bdece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:22.059679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:22.061436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8c379-29b2-4872-9d02-b831938bdece admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:22.086511 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1608/3256] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:22.087572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8c379-29b2-4872-9d02-b831938bdece admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:22.091190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f8a8c379-29b2-4872-9d02-b831938bdece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:22.110880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8c379-29b2-4872-9d02-b831938bdece admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:22.139083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f52a9125d09348b8b00997797b49340e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:22.141034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:22.141762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:22.143067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:22.143067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] PATH_INFO: `/v3/users/37a9d8dd01d540d398c6112b0091d1ac` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:22.144190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a9d8dd01d540d398c6112b0091d1ac)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:22.155003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:22.155582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:22.158972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f52a9125d09348b8b00997797b49340e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:22.178042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f52a9125d09348b8b00997797b49340e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:22.178879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f52a9125d09348b8b00997797b49340e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:22.179646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f52a9125d09348b8b00997797b49340e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:22.199374 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1649/3257] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:21 2022] DELETE /identity/v3/users/f52a9125d09348b8b00997797b49340e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:22.200427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:22.202297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:22.294454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a9d8dd01d540d398c6112b0091d1ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:22.308078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:22.336987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a9d8dd01d540d398c6112b0091d1ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:22.339739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a9d8dd01d540d398c6112b0091d1ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:22.364513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31864f60-71b5-4c6f-a249-160f3a01ada6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:22.365005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a9d8dd01d540d398c6112b0091d1ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:22.374701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f117fc-c774-4187-8630-11225574fd6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:22.375493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f117fc-c774-4187-8630-11225574fd6b service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:22.375892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f117fc-c774-4187-8630-11225574fd6b service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:22.376458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f117fc-c774-4187-8630-11225574fd6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:22.381420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b2f117fc-c774-4187-8630-11225574fd6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:22.382506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a9d8dd01d540d398c6112b0091d1ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:22.403718 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1609/3258] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:22 2022] DELETE /identity/v3/users/37a9d8dd01d540d398c6112b0091d1ac => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:22.408766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:22.410582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:22.477671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f117fc-c774-4187-8630-11225574fd6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:22.479570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f117fc-c774-4187-8630-11225574fd6b service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:22.507552 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1650/3259] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:22.509492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cf318-7138-4ee8-9b0a-99e3099be80b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:22.510004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cf318-7138-4ee8-9b0a-99e3099be80b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:22.510487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cf318-7138-4ee8-9b0a-99e3099be80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:22.532043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-674cf318-7138-4ee8-9b0a-99e3099be80b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:22.534985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-674cf318-7138-4ee8-9b0a-99e3099be80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:22.583329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58ba15-6bba-4b96-b49b-5be22d73819b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:22.594795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4898a-dd26-42d2-856a-f752e9b5f3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:22.595523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4898a-dd26-42d2-856a-f752e9b5f3f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:22.596370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4898a-dd26-42d2-856a-f752e9b5f3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:22.596724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4898a-dd26-42d2-856a-f752e9b5f3f6 admin admin] PATH_INFO: `/v3/projects/c9ae9fb6f6f8493abaf3857f81992925/users/94d4c06fa26e400ead26ab08cd86bb43/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:22.600728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4898a-dd26-42d2-856a-f752e9b5f3f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ae9fb6f6f8493abaf3857f81992925, user_id=94d4c06fa26e400ead26ab08cd86bb43, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:22.608865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4898a-dd26-42d2-856a-f752e9b5f3f6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:22.618210 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1651/3260] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:22.619833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70864a-9745-4541-9a41-a6b0a1605a27 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:22.620228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70864a-9745-4541-9a41-a6b0a1605a27 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:22.620624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70864a-9745-4541-9a41-a6b0a1605a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:22.633127 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1610/3261] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:22 2022] PUT /identity/v3/projects/c9ae9fb6f6f8493abaf3857f81992925/users/94d4c06fa26e400ead26ab08cd86bb43/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:22.634704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d1990-c341-4268-a470-8f4440cf8ffd None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:22.635193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d1990-c341-4268-a470-8f4440cf8ffd None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:22.635619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d1990-c341-4268-a470-8f4440cf8ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:22.657955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fe70864a-9745-4541-9a41-a6b0a1605a27 None None] MFA Rules not processed for user `f40d4376bec042e49ee8356e22baa812`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:22.659736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fe70864a-9745-4541-9a41-a6b0a1605a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:22.665870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:22.678852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c74d1990-c341-4268-a470-8f4440cf8ffd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:22.682268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c74d1990-c341-4268-a470-8f4440cf8ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:22.845890 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1611/3262] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:22.847031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d1990-c341-4268-a470-8f4440cf8ffd None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:22.851902 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1652/3263] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:22.854230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c74d1990-c341-4268-a470-8f4440cf8ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:22.881548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70864a-9745-4541-9a41-a6b0a1605a27 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:22.883661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fe70864a-9745-4541-9a41-a6b0a1605a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:23.006986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d1990-c341-4268-a470-8f4440cf8ffd None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:23.012813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d42b5-492a-4346-8de7-754663d6c413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:23.013503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d42b5-492a-4346-8de7-754663d6c413 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:23.014105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d42b5-492a-4346-8de7-754663d6c413 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:23.014393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d42b5-492a-4346-8de7-754663d6c413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:23.028182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d42b5-492a-4346-8de7-754663d6c413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:23.029216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70864a-9745-4541-9a41-a6b0a1605a27 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:23.029897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d42b5-492a-4346-8de7-754663d6c413 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:23.034170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed4d4e-d5fd-4040-9e4a-f1384ea0125b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:23.035026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed4d4e-d5fd-4040-9e4a-f1384ea0125b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:23.035462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed4d4e-d5fd-4040-9e4a-f1384ea0125b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:23.035890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed4d4e-d5fd-4040-9e4a-f1384ea0125b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:23.043701 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1612/3264] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:23.044920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d42b5-492a-4346-8de7-754663d6c413 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:23.046671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b51d42b5-492a-4346-8de7-754663d6c413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:23.062682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:23.082053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed4d4e-d5fd-4040-9e4a-f1384ea0125b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:23.083648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed4d4e-d5fd-4040-9e4a-f1384ea0125b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:23.119926 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1653/3265] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:57:23.200719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d42b5-492a-4346-8de7-754663d6c413 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:23.212988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8203f-3c88-4296-80fc-a1b577feebdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:23.213564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8203f-3c88-4296-80fc-a1b577feebdb service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:23.213892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8203f-3c88-4296-80fc-a1b577feebdb service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:23.214193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8203f-3c88-4296-80fc-a1b577feebdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:23.231679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8203f-3c88-4296-80fc-a1b577feebdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:23.237612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8203f-3c88-4296-80fc-a1b577feebdb service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:23.264479 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1613/3266] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:24.260080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed4d4e-d5fd-4040-9e4a-f1384ea0125b service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:24.261759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7aed4d4e-d5fd-4040-9e4a-f1384ea0125b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:24.307565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed4d4e-d5fd-4040-9e4a-f1384ea0125b service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:24.337422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:24.337422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:24.337422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:24.337422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] PATH_INFO: `/v3/projects/df8d15077d42417ca165bf7400c3b87a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:24.337744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df8d15077d42417ca165bf7400c3b87a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:24.339692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:24.424425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8203f-3c88-4296-80fc-a1b577feebdb service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:24.426224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-58a8203f-3c88-4296-80fc-a1b577feebdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:24.449182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8203f-3c88-4296-80fc-a1b577feebdb service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:24.463933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:24.464968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:24.465507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:24.466006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] PATH_INFO: `/v3/projects/93e3aeb041454256bdf249b6ed6a5214` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:24.469359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=93e3aeb041454256bdf249b6ed6a5214)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:24.472246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:24.511195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df8d15077d42417ca165bf7400c3b87a', 'request_id': 'req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.522940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df8d15077d42417ca165bf7400c3b87a', 'request_id': 'req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.532972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df8d15077d42417ca165bf7400c3b87a', 'request_id': 'req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.552487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] The token cache is being invalidate because project df8d15077d42417ca165bf7400c3b87a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:24.552788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.555101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.559330 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1654/3267] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:24 2022] DELETE /identity/v3/projects/df8d15077d42417ca165bf7400c3b87a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:24.563563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:24.566890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:24.587847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93e3aeb041454256bdf249b6ed6a5214', 'request_id': 'req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.606109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93e3aeb041454256bdf249b6ed6a5214', 'request_id': 'req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.619811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93e3aeb041454256bdf249b6ed6a5214', 'request_id': 'req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.644159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] The token cache is being invalidate because project 93e3aeb041454256bdf249b6ed6a5214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:24.647318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.648974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:24.649881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.651681 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1614/3268] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:24 2022] DELETE /identity/v3/projects/93e3aeb041454256bdf249b6ed6a5214 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:24.655629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:24.657251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:24.735303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ab6b9-0d6c-4a3b-8fa3-cae843cc8a26 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:24.756051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eca7f8-b462-4e61-93c3-6d7f825b1b2a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:24.761691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:24.762533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:24.762921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:24.763371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] PATH_INFO: `/v3/users/262cef2ff9524fa7bd00e03daf575174` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:24.764763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=262cef2ff9524fa7bd00e03daf575174)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:24.767128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:24.794593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:24.878786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '262cef2ff9524fa7bd00e03daf575174'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.915555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:24.915887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:24.916184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:24.916469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] PATH_INFO: `/v3/users/914f4feb06074937aa175ff8ca0b31c5` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:24.921025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '262cef2ff9524fa7bd00e03daf575174'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.929609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '262cef2ff9524fa7bd00e03daf575174'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.930092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=914f4feb06074937aa175ff8ca0b31c5)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:24.949066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:24.955965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '262cef2ff9524fa7bd00e03daf575174'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.977393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '262cef2ff9524fa7bd00e03daf575174'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:24.990143 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1615/3269] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:24 2022] DELETE /identity/v3/users/262cef2ff9524fa7bd00e03daf575174 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:25.087950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914f4feb06074937aa175ff8ca0b31c5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:25.163804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914f4feb06074937aa175ff8ca0b31c5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:25.174876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914f4feb06074937aa175ff8ca0b31c5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:25.175758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914f4feb06074937aa175ff8ca0b31c5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:25.176571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914f4feb06074937aa175ff8ca0b31c5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:25.192290 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1655/3270] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:24 2022] DELETE /identity/v3/users/914f4feb06074937aa175ff8ca0b31c5 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:26.458799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:26.460633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:26.598681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769dfde-5a4a-413b-9c4c-4c84f29221bf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:26.687670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81171e3d-969f-4431-8416-c3dcdedf8ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:26.688583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81171e3d-969f-4431-8416-c3dcdedf8ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:26.688880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81171e3d-969f-4431-8416-c3dcdedf8ade admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:26.689255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81171e3d-969f-4431-8416-c3dcdedf8ade admin admin] PATH_INFO: `/v3/projects/84409c8ff521450abb0cc252fe4d141f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:26.690786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81171e3d-969f-4431-8416-c3dcdedf8ade admin admin] RBAC: Authorizing `identity:delete_project(project_id=84409c8ff521450abb0cc252fe4d141f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:26.693178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81171e3d-969f-4431-8416-c3dcdedf8ade admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:26.724027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:26.726936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:26.760224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a510d-1240-4586-a635-a7abde80cd43 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:26.779001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:26.787846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-81171e3d-969f-4431-8416-c3dcdedf8ade admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84409c8ff521450abb0cc252fe4d141f', 'request_id': 'req-81171e3d-969f-4431-8416-c3dcdedf8ade'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:26.804789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-81171e3d-969f-4431-8416-c3dcdedf8ade admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84409c8ff521450abb0cc252fe4d141f', 'request_id': 'req-81171e3d-969f-4431-8416-c3dcdedf8ade'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:26.819969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-81171e3d-969f-4431-8416-c3dcdedf8ade admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84409c8ff521450abb0cc252fe4d141f', 'request_id': 'req-81171e3d-969f-4431-8416-c3dcdedf8ade'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:26.841771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-81171e3d-969f-4431-8416-c3dcdedf8ade admin admin] The token cache is being invalidate because project 84409c8ff521450abb0cc252fe4d141f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:26.842340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-81171e3d-969f-4431-8416-c3dcdedf8ade admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:26.843718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-81171e3d-969f-4431-8416-c3dcdedf8ade admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:26.850538 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1616/3271] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:26 2022] DELETE /identity/v3/projects/84409c8ff521450abb0cc252fe4d141f => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:26.852316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99812928-b037-4a3d-9020-c4e8ab1eab23 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:26.852686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99812928-b037-4a3d-9020-c4e8ab1eab23 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:26.853103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99812928-b037-4a3d-9020-c4e8ab1eab23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:26.888805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-99812928-b037-4a3d-9020-c4e8ab1eab23 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:26.890223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-99812928-b037-4a3d-9020-c4e8ab1eab23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:26.920740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:26.942534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817add9-60ea-4bae-a6e6-c6b6932aee23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:26.943882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817add9-60ea-4bae-a6e6-c6b6932aee23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:26.944181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817add9-60ea-4bae-a6e6-c6b6932aee23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:26.944464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817add9-60ea-4bae-a6e6-c6b6932aee23 admin admin] PATH_INFO: `/v3/projects/191d3406ce1148958ea336d84b8e4ff9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:26.947987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817add9-60ea-4bae-a6e6-c6b6932aee23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=191d3406ce1148958ea336d84b8e4ff9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:26.953936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817add9-60ea-4bae-a6e6-c6b6932aee23 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:26.989963 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1617/3272] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:26.994076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367c00d-83df-4d95-a53c-1125c567d8cc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:26.994637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367c00d-83df-4d95-a53c-1125c567d8cc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:26.995099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367c00d-83df-4d95-a53c-1125c567d8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:27.040352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7367c00d-83df-4d95-a53c-1125c567d8cc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:27.041912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7367c00d-83df-4d95-a53c-1125c567d8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:27.061803 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1618/3273] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:27.064117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367c00d-83df-4d95-a53c-1125c567d8cc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:27.086549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367c00d-83df-4d95-a53c-1125c567d8cc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:27.126133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b817add9-60ea-4bae-a6e6-c6b6932aee23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '191d3406ce1148958ea336d84b8e4ff9', 'request_id': 'req-b817add9-60ea-4bae-a6e6-c6b6932aee23'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:27.146838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b817add9-60ea-4bae-a6e6-c6b6932aee23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '191d3406ce1148958ea336d84b8e4ff9', 'request_id': 'req-b817add9-60ea-4bae-a6e6-c6b6932aee23'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:27.169930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b817add9-60ea-4bae-a6e6-c6b6932aee23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '191d3406ce1148958ea336d84b8e4ff9', 'request_id': 'req-b817add9-60ea-4bae-a6e6-c6b6932aee23'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:27.202376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b817add9-60ea-4bae-a6e6-c6b6932aee23 admin admin] The token cache is being invalidate because project 191d3406ce1148958ea336d84b8e4ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:27.202723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b817add9-60ea-4bae-a6e6-c6b6932aee23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:27.204186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b817add9-60ea-4bae-a6e6-c6b6932aee23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:27.208922 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1656/3274] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:26 2022] DELETE /identity/v3/projects/191d3406ce1148958ea336d84b8e4ff9 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:57:27.216290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b19ab-df5c-40e2-8eff-177d4424626c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:27.216570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b19ab-df5c-40e2-8eff-177d4424626c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:27.217586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b19ab-df5c-40e2-8eff-177d4424626c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:27.253475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea531f32-5844-436c-a22d-1fd9b0eaeaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:27.254117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea531f32-5844-436c-a22d-1fd9b0eaeaa7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:27.254404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea531f32-5844-436c-a22d-1fd9b0eaeaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:27.254684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea531f32-5844-436c-a22d-1fd9b0eaeaa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:27.254815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-886b19ab-df5c-40e2-8eff-177d4424626c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:27.255343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea531f32-5844-436c-a22d-1fd9b0eaeaa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:27.255647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea531f32-5844-436c-a22d-1fd9b0eaeaa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:27.256589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-886b19ab-df5c-40e2-8eff-177d4424626c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:27.257847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea531f32-5844-436c-a22d-1fd9b0eaeaa7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:27.273023 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1657/3275] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:57:27.280280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fe56e-7acc-4981-8cfa-f701a20abf1d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:27.281004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fe56e-7acc-4981-8cfa-f701a20abf1d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:27.281101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fe56e-7acc-4981-8cfa-f701a20abf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:27.292308 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1619/3276] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:27.293288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea531f32-5844-436c-a22d-1fd9b0eaeaa7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:27.301641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ea531f32-5844-436c-a22d-1fd9b0eaeaa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:27.311598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-641fe56e-7acc-4981-8cfa-f701a20abf1d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:27.313970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-641fe56e-7acc-4981-8cfa-f701a20abf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:27.326913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea531f32-5844-436c-a22d-1fd9b0eaeaa7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:27.332319 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1658/3277] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:27.333510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fe56e-7acc-4981-8cfa-f701a20abf1d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:27.336802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf3b3a-3aa2-42ca-a60b-24e1a4e21f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:27.337467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf3b3a-3aa2-42ca-a60b-24e1a4e21f4a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:27.337756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf3b3a-3aa2-42ca-a60b-24e1a4e21f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:27.338037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf3b3a-3aa2-42ca-a60b-24e1a4e21f4a admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:27.338686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf3b3a-3aa2-42ca-a60b-24e1a4e21f4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:27.342537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf3b3a-3aa2-42ca-a60b-24e1a4e21f4a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:27.345444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fe56e-7acc-4981-8cfa-f701a20abf1d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:27.352229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785308a1-320e-42e6-9234-3e461089caca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:27.353179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785308a1-320e-42e6-9234-3e461089caca admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:27.353474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785308a1-320e-42e6-9234-3e461089caca admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:27.353912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785308a1-320e-42e6-9234-3e461089caca admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:27.356398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785308a1-320e-42e6-9234-3e461089caca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:27.356982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785308a1-320e-42e6-9234-3e461089caca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:27.360505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785308a1-320e-42e6-9234-3e461089caca admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:27.398819 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1620/3278] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:27 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:27.399744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf3b3a-3aa2-42ca-a60b-24e1a4e21f4a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:27.418166 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1659/3279] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:27.419207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785308a1-320e-42e6-9234-3e461089caca admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:27.446496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785308a1-320e-42e6-9234-3e461089caca admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:27.494181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:27.534234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf3b3a-3aa2-42ca-a60b-24e1a4e21f4a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:27.540275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3ceda-daa9-410f-87be-4a570e6bd391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:27.540929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3ceda-daa9-410f-87be-4a570e6bd391 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:27.541225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3ceda-daa9-410f-87be-4a570e6bd391 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:27.541504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3ceda-daa9-410f-87be-4a570e6bd391 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:27.542311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3ceda-daa9-410f-87be-4a570e6bd391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:27.544485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3ceda-daa9-410f-87be-4a570e6bd391 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:27.580512 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1621/3280] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:27 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:27.585427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3ceda-daa9-410f-87be-4a570e6bd391 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:27.607802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3ceda-daa9-410f-87be-4a570e6bd391 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:27.621745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c2f6c-0e6c-4c31-bbba-ea413c81142c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:27.622528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c2f6c-0e6c-4c31-bbba-ea413c81142c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:27.622910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c2f6c-0e6c-4c31-bbba-ea413c81142c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:27.623301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c2f6c-0e6c-4c31-bbba-ea413c81142c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:27.625892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c2f6c-0e6c-4c31-bbba-ea413c81142c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:27.630589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c2f6c-0e6c-4c31-bbba-ea413c81142c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:27.654503 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1622/3281] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:27.656057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b1452-1fa0-4ae9-9f15-c6811020effc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:27.656812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b1452-1fa0-4ae9-9f15-c6811020effc admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:27.657932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c2f6c-0e6c-4c31-bbba-ea413c81142c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:27.658354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b1452-1fa0-4ae9-9f15-c6811020effc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:27.658750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b1452-1fa0-4ae9-9f15-c6811020effc admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:27.659501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b1452-1fa0-4ae9-9f15-c6811020effc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:27.661961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b1452-1fa0-4ae9-9f15-c6811020effc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:27.677947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c2f6c-0e6c-4c31-bbba-ea413c81142c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:27.695226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c52d70-751e-4219-b5e5-dcf027525b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:27.696039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c52d70-751e-4219-b5e5-dcf027525b2d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:27.696428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c52d70-751e-4219-b5e5-dcf027525b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:27.696809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c52d70-751e-4219-b5e5-dcf027525b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:27.697682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c52d70-751e-4219-b5e5-dcf027525b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:27.699472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c52d70-751e-4219-b5e5-dcf027525b2d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:27.725185 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1660/3282] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:27 2022] POST /identity/v3/projects => generated 373 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:27.725735 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1623/3283] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:27.727601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b1452-1fa0-4ae9-9f15-c6811020effc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:27.729764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c52d70-751e-4219-b5e5-dcf027525b2d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:27.753075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c52d70-751e-4219-b5e5-dcf027525b2d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:27.832980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:27.905504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b1452-1fa0-4ae9-9f15-c6811020effc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:27.937773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4365172-341d-40d0-8708-f417f4dc1dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:27.940794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4365172-341d-40d0-8708-f417f4dc1dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:27.945197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4365172-341d-40d0-8708-f417f4dc1dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:27.947140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4365172-341d-40d0-8708-f417f4dc1dd4 admin admin] PATH_INFO: `/v3/projects/9bf7c2d797a74562993d8605bda80ae4/users/0c77fca40dbc430db41e4a30639b28b5/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:27.964745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbde758-af2b-4e20-93ba-3ad1f855fe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:27.966764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbde758-af2b-4e20-93ba-3ad1f855fe0c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:27.969706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbde758-af2b-4e20-93ba-3ad1f855fe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:27.973427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbde758-af2b-4e20-93ba-3ad1f855fe0c admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:27.976061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbde758-af2b-4e20-93ba-3ad1f855fe0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:27.984116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbde758-af2b-4e20-93ba-3ad1f855fe0c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:28.025470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4365172-341d-40d0-8708-f417f4dc1dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf7c2d797a74562993d8605bda80ae4, user_id=0c77fca40dbc430db41e4a30639b28b5, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:28.033647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4365172-341d-40d0-8708-f417f4dc1dd4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:28.061291 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1624/3284] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:27 2022] POST /identity/v3/users => generated 287 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:57:28.068409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbde758-af2b-4e20-93ba-3ad1f855fe0c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:28.072428 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1661/3285] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:27 2022] PUT /identity/v3/projects/9bf7c2d797a74562993d8605bda80ae4/users/0c77fca40dbc430db41e4a30639b28b5/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:57:28.085938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf12d8-a242-4f0b-811e-191497dacf18 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:28.086357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf12d8-a242-4f0b-811e-191497dacf18 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:28.086746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf12d8-a242-4f0b-811e-191497dacf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:28.100797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbde758-af2b-4e20-93ba-3ad1f855fe0c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:28.200231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6cbf12d8-a242-4f0b-811e-191497dacf18 None None] MFA Rules not processed for user `0c77fca40dbc430db41e4a30639b28b5`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:28.204367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6cbf12d8-a242-4f0b-811e-191497dacf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:28.317726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0022a59-8d72-4e80-abaa-0b9e116c1bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:28.318419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0022a59-8d72-4e80-abaa-0b9e116c1bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:28.318781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0022a59-8d72-4e80-abaa-0b9e116c1bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:28.319074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0022a59-8d72-4e80-abaa-0b9e116c1bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:28.321434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0022a59-8d72-4e80-abaa-0b9e116c1bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:28.337862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0022a59-8d72-4e80-abaa-0b9e116c1bb6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:28.357964 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1625/3286] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:28.362153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0022a59-8d72-4e80-abaa-0b9e116c1bb6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:28.386511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0022a59-8d72-4e80-abaa-0b9e116c1bb6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:28.410298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e2940-670c-46e0-ac2d-b0976296671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:28.411221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e2940-670c-46e0-ac2d-b0976296671f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:28.411674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e2940-670c-46e0-ac2d-b0976296671f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:28.412624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e2940-670c-46e0-ac2d-b0976296671f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:28.413507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e2940-670c-46e0-ac2d-b0976296671f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:28.416518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e2940-670c-46e0-ac2d-b0976296671f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:28.451939 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1626/3287] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:28.458010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e2940-670c-46e0-ac2d-b0976296671f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:28.484367 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1662/3288] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:57:28.485664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e2940-670c-46e0-ac2d-b0976296671f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:28.488773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf12d8-a242-4f0b-811e-191497dacf18 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:28.503030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcedec-acba-4673-b605-3def0fcf7f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:28.504399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcedec-acba-4673-b605-3def0fcf7f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:28.504862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcedec-acba-4673-b605-3def0fcf7f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:28.505360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcedec-acba-4673-b605-3def0fcf7f0e admin admin] PATH_INFO: `/v3/projects/0c65af796be74447b2bbba4e8b9d4f35/users/9baf371d1c0942eda0b4ea9466b03b79/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:28.513961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf12d8-a242-4f0b-811e-191497dacf18 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:28.534978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2558a65-a600-4115-a507-60954591b0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:28.535804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2558a65-a600-4115-a507-60954591b0de admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:28.536587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2558a65-a600-4115-a507-60954591b0de admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:28.538891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2558a65-a600-4115-a507-60954591b0de admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:28.540125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2558a65-a600-4115-a507-60954591b0de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:28.543707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2558a65-a600-4115-a507-60954591b0de admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:28.556945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcedec-acba-4673-b605-3def0fcf7f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c65af796be74447b2bbba4e8b9d4f35, user_id=9baf371d1c0942eda0b4ea9466b03b79, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:28.572835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcedec-acba-4673-b605-3def0fcf7f0e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:28.593187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:28.607936 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1627/3289] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:28 2022] PUT /identity/v3/projects/0c65af796be74447b2bbba4e8b9d4f35/users/9baf371d1c0942eda0b4ea9466b03b79/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:28.620129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da049f13-a7da-4c2d-8e10-a660a985b994 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:28.620431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da049f13-a7da-4c2d-8e10-a660a985b994 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:28.620719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da049f13-a7da-4c2d-8e10-a660a985b994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:28.707156 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1663/3290] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:28 2022] POST /identity/v3/projects => generated 377 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:28.711256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2558a65-a600-4115-a507-60954591b0de admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:28.727694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-da049f13-a7da-4c2d-8e10-a660a985b994 None None] MFA Rules not processed for user `9baf371d1c0942eda0b4ea9466b03b79`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:28.734222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-da049f13-a7da-4c2d-8e10-a660a985b994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:28.901408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2558a65-a600-4115-a507-60954591b0de admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:28.932194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012c1ce-2aa9-4274-b938-42623cb1b378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:28.932831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012c1ce-2aa9-4274-b938-42623cb1b378 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:28.933144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012c1ce-2aa9-4274-b938-42623cb1b378 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:28.933424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012c1ce-2aa9-4274-b938-42623cb1b378 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:28.934072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012c1ce-2aa9-4274-b938-42623cb1b378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:28.943648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012c1ce-2aa9-4274-b938-42623cb1b378 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:28.950581 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1628/3291] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:28.950581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f9bb3-4b1f-45b2-9654-0c5a24c37d5e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:28.950581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f9bb3-4b1f-45b2-9654-0c5a24c37d5e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:28.950581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f9bb3-4b1f-45b2-9654-0c5a24c37d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:28.990780 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1664/3292] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:28 2022] POST /identity/v3/users => generated 295 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:28.992100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012c1ce-2aa9-4274-b938-42623cb1b378 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:28.995375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1f1f9bb3-4b1f-45b2-9654-0c5a24c37d5e None None] MFA Rules not processed for user `f40d4376bec042e49ee8356e22baa812`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:28.996427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1f1f9bb3-4b1f-45b2-9654-0c5a24c37d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:29.015754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012c1ce-2aa9-4274-b938-42623cb1b378 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:29.029160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314523f4-c682-431e-94f1-634a0b04e506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:29.029755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314523f4-c682-431e-94f1-634a0b04e506 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:29.030042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314523f4-c682-431e-94f1-634a0b04e506 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:29.030316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314523f4-c682-431e-94f1-634a0b04e506 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:29.030960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314523f4-c682-431e-94f1-634a0b04e506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:29.040266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314523f4-c682-431e-94f1-634a0b04e506 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:29.121221 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1665/3293] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:28 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:29.122307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314523f4-c682-431e-94f1-634a0b04e506 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:29.141368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:29.162331 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1629/3294] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:29.163260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f9bb3-4b1f-45b2-9654-0c5a24c37d5e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:29.215760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f9bb3-4b1f-45b2-9654-0c5a24c37d5e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:29.412862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdb9d9-844b-4e47-8b82-099a87a45fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:29.413478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdb9d9-844b-4e47-8b82-099a87a45fcc admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:29.413761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdb9d9-844b-4e47-8b82-099a87a45fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:29.414032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdb9d9-844b-4e47-8b82-099a87a45fcc admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:29.414675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdb9d9-844b-4e47-8b82-099a87a45fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:29.416610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdb9d9-844b-4e47-8b82-099a87a45fcc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:29.422656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314523f4-c682-431e-94f1-634a0b04e506 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:29.440170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfdc65-00f9-4bca-bd83-70ff017568be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:29.444823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfdc65-00f9-4bca-bd83-70ff017568be admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:29.448153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfdc65-00f9-4bca-bd83-70ff017568be admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:29.448153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfdc65-00f9-4bca-bd83-70ff017568be admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:29.448719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfdc65-00f9-4bca-bd83-70ff017568be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:29.454574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfdc65-00f9-4bca-bd83-70ff017568be admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:29.497105 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1630/3295] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:29 2022] POST /identity/v3/users => generated 293 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:29.498457 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1666/3296] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:29.505342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdb9d9-844b-4e47-8b82-099a87a45fcc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:29.506906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfdc65-00f9-4bca-bd83-70ff017568be admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:29.533346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdb9d9-844b-4e47-8b82-099a87a45fcc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:29.551545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd24703-5645-4d19-a0fd-bfe655886573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:29.552232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd24703-5645-4d19-a0fd-bfe655886573 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:29.552517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd24703-5645-4d19-a0fd-bfe655886573 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:29.552796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd24703-5645-4d19-a0fd-bfe655886573 admin admin] PATH_INFO: `/v3/projects/e941437cffc54d9b9a8b2257c0c23bd9/users/7db1b70a90d34e3cb860ca5a9867d5f2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:29.560179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfdc65-00f9-4bca-bd83-70ff017568be admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:29.580979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e473a8-4a4c-4a93-97aa-c9e6f3d88332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:29.582810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e473a8-4a4c-4a93-97aa-c9e6f3d88332 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:29.584264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e473a8-4a4c-4a93-97aa-c9e6f3d88332 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:29.584803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e473a8-4a4c-4a93-97aa-c9e6f3d88332 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:29.585741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e473a8-4a4c-4a93-97aa-c9e6f3d88332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:29.587617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e473a8-4a4c-4a93-97aa-c9e6f3d88332 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:29.599042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd24703-5645-4d19-a0fd-bfe655886573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e941437cffc54d9b9a8b2257c0c23bd9, user_id=7db1b70a90d34e3cb860ca5a9867d5f2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:29.607792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd24703-5645-4d19-a0fd-bfe655886573 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:29.618039 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1667/3297] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:29.622424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e473a8-4a4c-4a93-97aa-c9e6f3d88332 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:29.637404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:29.650116 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1631/3298] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:29 2022] PUT /identity/v3/projects/e941437cffc54d9b9a8b2257c0c23bd9/users/7db1b70a90d34e3cb860ca5a9867d5f2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:29.653992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd24703-5645-4d19-a0fd-bfe655886573 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:29.667459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e473a8-4a4c-4a93-97aa-c9e6f3d88332 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:29.826754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bac38-9d9c-4b0c-be4e-910094f163f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:29.827677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bac38-9d9c-4b0c-be4e-910094f163f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:29.828111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bac38-9d9c-4b0c-be4e-910094f163f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:29.828507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bac38-9d9c-4b0c-be4e-910094f163f8 admin admin] PATH_INFO: `/v3/projects/d2293bd0d1f748b68b7cc271094ecf1e/users/aafa2cc6f37847e78154294a7937f3a4/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:29.832396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd24703-5645-4d19-a0fd-bfe655886573 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:29.847298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e11c6-662a-417d-b946-89ef8a3e1151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:29.850678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e11c6-662a-417d-b946-89ef8a3e1151 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:29.852740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e11c6-662a-417d-b946-89ef8a3e1151 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:29.855801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e11c6-662a-417d-b946-89ef8a3e1151 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:29.858507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e11c6-662a-417d-b946-89ef8a3e1151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:29.861865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e11c6-662a-417d-b946-89ef8a3e1151 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:29.872245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591bac38-9d9c-4b0c-be4e-910094f163f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2293bd0d1f748b68b7cc271094ecf1e, user_id=aafa2cc6f37847e78154294a7937f3a4, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:29.875787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591bac38-9d9c-4b0c-be4e-910094f163f8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:29.896803 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1632/3299] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:29.900888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e11c6-662a-417d-b946-89ef8a3e1151 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:29.919125 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1668/3300] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:29 2022] PUT /identity/v3/projects/d2293bd0d1f748b68b7cc271094ecf1e/users/aafa2cc6f37847e78154294a7937f3a4/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:29.922923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bac38-9d9c-4b0c-be4e-910094f163f8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:29.931830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e11c6-662a-417d-b946-89ef8a3e1151 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:30.031392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bac38-9d9c-4b0c-be4e-910094f163f8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:30.042592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889156c-baa2-4a45-80a9-4c4cf8ae103b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:30.045217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889156c-baa2-4a45-80a9-4c4cf8ae103b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:30.045548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889156c-baa2-4a45-80a9-4c4cf8ae103b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:30.045835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889156c-baa2-4a45-80a9-4c4cf8ae103b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:30.046567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889156c-baa2-4a45-80a9-4c4cf8ae103b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:30.049220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889156c-baa2-4a45-80a9-4c4cf8ae103b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:30.066420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6740c-a1af-4522-882d-f67634d0ff52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:30.067318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6740c-a1af-4522-882d-f67634d0ff52 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:30.067763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6740c-a1af-4522-882d-f67634d0ff52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:30.068635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6740c-a1af-4522-882d-f67634d0ff52 admin admin] PATH_INFO: `/v3/projects/e941437cffc54d9b9a8b2257c0c23bd9/users/7db1b70a90d34e3cb860ca5a9867d5f2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:30.077236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6740c-a1af-4522-882d-f67634d0ff52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e941437cffc54d9b9a8b2257c0c23bd9, user_id=7db1b70a90d34e3cb860ca5a9867d5f2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:30.077372 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1669/3301] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:30.077748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6740c-a1af-4522-882d-f67634d0ff52 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:30.078174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889156c-baa2-4a45-80a9-4c4cf8ae103b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:30.102818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889156c-baa2-4a45-80a9-4c4cf8ae103b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:30.108542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:30.112821 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1633/3302] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:29 2022] PUT /identity/v3/projects/e941437cffc54d9b9a8b2257c0c23bd9/users/7db1b70a90d34e3cb860ca5a9867d5f2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:30.114602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1a887-d9ae-47b2-878d-5096a60d220a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:30.114832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1a887-d9ae-47b2-878d-5096a60d220a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:30.115566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1a887-d9ae-47b2-878d-5096a60d220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:30.247995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ebb1a887-d9ae-47b2-878d-5096a60d220a None None] MFA Rules not processed for user `7db1b70a90d34e3cb860ca5a9867d5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:30.252976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ebb1a887-d9ae-47b2-878d-5096a60d220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:30.305490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f734e8-106a-40d0-b736-e2e48283fc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:30.306849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f734e8-106a-40d0-b736-e2e48283fc5e admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:30.307144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f734e8-106a-40d0-b736-e2e48283fc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:30.307706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f734e8-106a-40d0-b736-e2e48283fc5e admin admin] PATH_INFO: `/v3/projects/d2293bd0d1f748b68b7cc271094ecf1e/users/aafa2cc6f37847e78154294a7937f3a4/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:30.310386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f734e8-106a-40d0-b736-e2e48283fc5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2293bd0d1f748b68b7cc271094ecf1e, user_id=aafa2cc6f37847e78154294a7937f3a4, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:30.313301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f734e8-106a-40d0-b736-e2e48283fc5e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:30.340942 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1670/3303] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:30 2022] PUT /identity/v3/projects/d2293bd0d1f748b68b7cc271094ecf1e/users/aafa2cc6f37847e78154294a7937f3a4/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:30.348116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfada8e-03eb-4cdb-b89c-68774f3cd039 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:30.348645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfada8e-03eb-4cdb-b89c-68774f3cd039 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:30.350356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfada8e-03eb-4cdb-b89c-68774f3cd039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:30.445628 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1634/3304] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:30.447283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da401de2-568f-4dc4-883a-65fcc7786c14 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:30.447689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da401de2-568f-4dc4-883a-65fcc7786c14 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:30.448101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da401de2-568f-4dc4-883a-65fcc7786c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:30.459492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ddfada8e-03eb-4cdb-b89c-68774f3cd039 None None] MFA Rules not processed for user `aafa2cc6f37847e78154294a7937f3a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:30.463985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ddfada8e-03eb-4cdb-b89c-68774f3cd039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:30.491150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-da401de2-568f-4dc4-883a-65fcc7786c14 None None] MFA Rules not processed for user `0c77fca40dbc430db41e4a30639b28b5`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:30.492572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-da401de2-568f-4dc4-883a-65fcc7786c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:30.515090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:30.609475 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1635/3305] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:30.622462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da401de2-568f-4dc4-883a-65fcc7786c14 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:30.625207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-da401de2-568f-4dc4-883a-65fcc7786c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:30.639941 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1671/3306] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:30.645439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa0f56-d20c-4a6e-a970-64cb6c4a71c6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:30.645820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa0f56-d20c-4a6e-a970-64cb6c4a71c6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:30.646191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa0f56-d20c-4a6e-a970-64cb6c4a71c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:30.682589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-89aa0f56-d20c-4a6e-a970-64cb6c4a71c6 None None] MFA Rules not processed for user `9baf371d1c0942eda0b4ea9466b03b79`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:30.684250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-89aa0f56-d20c-4a6e-a970-64cb6c4a71c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:30.819206 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1672/3307] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:30.844585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da401de2-568f-4dc4-883a-65fcc7786c14 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:30.859621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f69dd-a4e0-4c59-b055-530daf78d59e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:30.860263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f69dd-a4e0-4c59-b055-530daf78d59e service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:30.860544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f69dd-a4e0-4c59-b055-530daf78d59e service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:30.860818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f69dd-a4e0-4c59-b055-530daf78d59e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:30.880661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f69dd-a4e0-4c59-b055-530daf78d59e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:30.882399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f69dd-a4e0-4c59-b055-530daf78d59e service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:30.899346 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1636/3308] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:57:31.006663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa0f56-d20c-4a6e-a970-64cb6c4a71c6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:31.008361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-89aa0f56-d20c-4a6e-a970-64cb6c4a71c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:31.012507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f69dd-a4e0-4c59-b055-530daf78d59e service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:31.014636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-685f69dd-a4e0-4c59-b055-530daf78d59e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:31.035689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa0f56-d20c-4a6e-a970-64cb6c4a71c6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:31.046797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ad5b5-b739-4d63-ba89-dded9c5f9fe9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:31.047620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ad5b5-b739-4d63-ba89-dded9c5f9fe9 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:31.048113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ad5b5-b739-4d63-ba89-dded9c5f9fe9 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:31.048648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ad5b5-b739-4d63-ba89-dded9c5f9fe9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:31.077046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ad5b5-b739-4d63-ba89-dded9c5f9fe9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:31.079113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ad5b5-b739-4d63-ba89-dded9c5f9fe9 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:31.109709 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1673/3309] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:31.160407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f69dd-a4e0-4c59-b055-530daf78d59e service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:31.168166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:31.168678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:31.169001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:31.169287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] PATH_INFO: `/v3/users/f4d78b7738644ff38eb239daec52d641` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:31.171406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4d78b7738644ff38eb239daec52d641)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:31.173386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:31.257437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:31.263652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4d78b7738644ff38eb239daec52d641'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:31.286795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4d78b7738644ff38eb239daec52d641'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:31.288830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4d78b7738644ff38eb239daec52d641'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:31.301399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4d78b7738644ff38eb239daec52d641'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:31.315995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4d78b7738644ff38eb239daec52d641'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:31.324928 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1637/3310] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:31 2022] DELETE /identity/v3/users/f4d78b7738644ff38eb239daec52d641 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:57:31.809265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3e449-4e81-42de-ba8b-5dc3ef45c852 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:31.809934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3e449-4e81-42de-ba8b-5dc3ef45c852 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:31.810792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3e449-4e81-42de-ba8b-5dc3ef45c852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:31.902361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6ee3e449-4e81-42de-ba8b-5dc3ef45c852 None None] MFA Rules not processed for user `0c77fca40dbc430db41e4a30639b28b5`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:31.904028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6ee3e449-4e81-42de-ba8b-5dc3ef45c852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:31.943632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:32.172660 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1674/3311] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:57:33.185787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:33.187329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:33.323317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08a1fd-9fed-45d0-8aa1-1c9465b6453f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:33.336215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ce24d-54d5-4776-b0b4-fa53f8b573d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:33.337195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ce24d-54d5-4776-b0b4-fa53f8b573d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:33.337662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ce24d-54d5-4776-b0b4-fa53f8b573d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:33.338125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ce24d-54d5-4776-b0b4-fa53f8b573d9 admin admin] PATH_INFO: `/v3/projects/a9897f3d9893426f9fba83c8db250799` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:33.339485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ce24d-54d5-4776-b0b4-fa53f8b573d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9897f3d9893426f9fba83c8db250799)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:33.341909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ce24d-54d5-4776-b0b4-fa53f8b573d9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:33.453183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-354ce24d-54d5-4776-b0b4-fa53f8b573d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9897f3d9893426f9fba83c8db250799', 'request_id': 'req-354ce24d-54d5-4776-b0b4-fa53f8b573d9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:33.475651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-354ce24d-54d5-4776-b0b4-fa53f8b573d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9897f3d9893426f9fba83c8db250799', 'request_id': 'req-354ce24d-54d5-4776-b0b4-fa53f8b573d9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:33.485650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-354ce24d-54d5-4776-b0b4-fa53f8b573d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9897f3d9893426f9fba83c8db250799', 'request_id': 'req-354ce24d-54d5-4776-b0b4-fa53f8b573d9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:33.499114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-354ce24d-54d5-4776-b0b4-fa53f8b573d9 admin admin] The token cache is being invalidate because project a9897f3d9893426f9fba83c8db250799 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:33.499417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-354ce24d-54d5-4776-b0b4-fa53f8b573d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:33.500117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-354ce24d-54d5-4776-b0b4-fa53f8b573d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:33.502537 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1638/3312] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:33 2022] DELETE /identity/v3/projects/a9897f3d9893426f9fba83c8db250799 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:33.509091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152dde-0461-4642-a624-32bc7ffffdd7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:33.509985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152dde-0461-4642-a624-32bc7ffffdd7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:33.510270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152dde-0461-4642-a624-32bc7ffffdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:33.646095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b7152dde-0461-4642-a624-32bc7ffffdd7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:33.651709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b7152dde-0461-4642-a624-32bc7ffffdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:33.822785 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1675/3313] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:33.829586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d331bcf-c87a-425a-95f1-bea5bd19178f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:33.830183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d331bcf-c87a-425a-95f1-bea5bd19178f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:33.830628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d331bcf-c87a-425a-95f1-bea5bd19178f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:33.903705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1d331bcf-c87a-425a-95f1-bea5bd19178f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:33.905397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1d331bcf-c87a-425a-95f1-bea5bd19178f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:33.939101 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1639/3314] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:33.943098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152dde-0461-4642-a624-32bc7ffffdd7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:33.959000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152dde-0461-4642-a624-32bc7ffffdd7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:33.971866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c6fd-f1f2-433d-852f-fe99b1922d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:33.972479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c6fd-f1f2-433d-852f-fe99b1922d9c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:33.972764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c6fd-f1f2-433d-852f-fe99b1922d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:33.973235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c6fd-f1f2-433d-852f-fe99b1922d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:33.974471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c6fd-f1f2-433d-852f-fe99b1922d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:33.975131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c6fd-f1f2-433d-852f-fe99b1922d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:33.976942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c6fd-f1f2-433d-852f-fe99b1922d9c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:34.037125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d331bcf-c87a-425a-95f1-bea5bd19178f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:34.041401 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1676/3315] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:34.048787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d331bcf-c87a-425a-95f1-bea5bd19178f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:34.062101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fc1ae-ec63-4e95-bcdb-0b2f061a72dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:34.064275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fc1ae-ec63-4e95-bcdb-0b2f061a72dc admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:34.064565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fc1ae-ec63-4e95-bcdb-0b2f061a72dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:34.067546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fc1ae-ec63-4e95-bcdb-0b2f061a72dc admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:34.069520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fc1ae-ec63-4e95-bcdb-0b2f061a72dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:34.076725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fc1ae-ec63-4e95-bcdb-0b2f061a72dc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:34.162291 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1640/3316] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:34 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:57:34.166544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c6fd-f1f2-433d-852f-fe99b1922d9c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:34.181958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c6fd-f1f2-433d-852f-fe99b1922d9c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:34.320702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902b014-1b33-49ba-882c-f0b26555c046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:34.322118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902b014-1b33-49ba-882c-f0b26555c046 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:34.334064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902b014-1b33-49ba-882c-f0b26555c046 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:34.334563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902b014-1b33-49ba-882c-f0b26555c046 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:34.335635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e902b014-1b33-49ba-882c-f0b26555c046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:34.338696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e902b014-1b33-49ba-882c-f0b26555c046 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:34.396486 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1677/3317] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:34 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:34.397078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fc1ae-ec63-4e95-bcdb-0b2f061a72dc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:34.418771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fc1ae-ec63-4e95-bcdb-0b2f061a72dc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:34.439866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f47aaf-86f1-4408-863b-e26a39221554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:34.445018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f47aaf-86f1-4408-863b-e26a39221554 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:34.445528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f47aaf-86f1-4408-863b-e26a39221554 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:34.446026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f47aaf-86f1-4408-863b-e26a39221554 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:34.448072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f47aaf-86f1-4408-863b-e26a39221554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:34.452386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f47aaf-86f1-4408-863b-e26a39221554 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:34.481821 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1641/3318] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:57:34.485533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902b014-1b33-49ba-882c-f0b26555c046 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:34.503404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902b014-1b33-49ba-882c-f0b26555c046 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:34.518023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927dfa48-f750-42c3-8fad-f7de12a9a5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:34.520362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927dfa48-f750-42c3-8fad-f7de12a9a5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:34.520945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927dfa48-f750-42c3-8fad-f7de12a9a5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:34.521828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927dfa48-f750-42c3-8fad-f7de12a9a5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:34.524704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927dfa48-f750-42c3-8fad-f7de12a9a5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:34.527312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927dfa48-f750-42c3-8fad-f7de12a9a5b1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:34.569042 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1678/3319] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:34.574924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f47aaf-86f1-4408-863b-e26a39221554 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:34.595665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f47aaf-86f1-4408-863b-e26a39221554 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:34.619147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fea0b5-51c3-41fc-bf04-e1e65877ca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:34.619998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fea0b5-51c3-41fc-bf04-e1e65877ca79 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:34.620297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fea0b5-51c3-41fc-bf04-e1e65877ca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:34.620583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fea0b5-51c3-41fc-bf04-e1e65877ca79 admin admin] PATH_INFO: `/v3/projects/713d2a60596b4d2d9701779c1d414ed1/users/999db58fc81f4a9ab60bdd21674ec8c2/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:34.663503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:34.680593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fea0b5-51c3-41fc-bf04-e1e65877ca79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=713d2a60596b4d2d9701779c1d414ed1, user_id=999db58fc81f4a9ab60bdd21674ec8c2, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:34.686967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fea0b5-51c3-41fc-bf04-e1e65877ca79 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:34.751369 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1642/3320] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:34 2022] PUT /identity/v3/projects/713d2a60596b4d2d9701779c1d414ed1/users/999db58fc81f4a9ab60bdd21674ec8c2/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:57:34.757112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b827cc-caac-4450-a08a-3633c935dc82 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:34.757408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b827cc-caac-4450-a08a-3633c935dc82 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:34.757750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b827cc-caac-4450-a08a-3633c935dc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:34.794867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-18b827cc-caac-4450-a08a-3633c935dc82 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:34.798103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-18b827cc-caac-4450-a08a-3633c935dc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:34.871956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:35.015844 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1679/3321] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:35.064388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fea0b5-51c3-41fc-bf04-e1e65877ca79 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:35.067796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d9fea0b5-51c3-41fc-bf04-e1e65877ca79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:35.205886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fea0b5-51c3-41fc-bf04-e1e65877ca79 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:35.239395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa983f-e84c-465d-89b9-148bae605046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:35.240365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa983f-e84c-465d-89b9-148bae605046 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:35.241412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa983f-e84c-465d-89b9-148bae605046 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:35.242056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa983f-e84c-465d-89b9-148bae605046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:35.265637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa983f-e84c-465d-89b9-148bae605046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:35.270293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa983f-e84c-465d-89b9-148bae605046 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:35.286657 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1643/3322] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:44.469349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ef29-b0d3-43e7-96c6-4b80d00a885c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:44.469893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ef29-b0d3-43e7-96c6-4b80d00a885c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:44.469893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ef29-b0d3-43e7-96c6-4b80d00a885c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:44.508644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2595ef29-b0d3-43e7-96c6-4b80d00a885c None None] MFA Rules not processed for user `94d4c06fa26e400ead26ab08cd86bb43`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:44.509985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2595ef29-b0d3-43e7-96c6-4b80d00a885c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:44.601091 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1680/3323] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:57:44.615274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa983f-e84c-465d-89b9-148bae605046 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:44.616856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8ffa983f-e84c-465d-89b9-148bae605046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:44.711174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa983f-e84c-465d-89b9-148bae605046 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:44.716515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d3528-35d6-4642-af71-eac52523f53e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:44.717181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d3528-35d6-4642-af71-eac52523f53e service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:44.718516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d3528-35d6-4642-af71-eac52523f53e service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:44.718806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d3528-35d6-4642-af71-eac52523f53e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:44.721707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e53d3528-35d6-4642-af71-eac52523f53e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:44.791774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d3528-35d6-4642-af71-eac52523f53e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:44.793362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d3528-35d6-4642-af71-eac52523f53e service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:44.805491 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1644/3324] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:50.862800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595ef29-b0d3-43e7-96c6-4b80d00a885c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:50.865182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2595ef29-b0d3-43e7-96c6-4b80d00a885c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:50.897001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595ef29-b0d3-43e7-96c6-4b80d00a885c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:50.926268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:50.930037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:50.931490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:50.932317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] PATH_INFO: `/v3/users/778561ee5e474accb31816f357985f44` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:50.935361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=778561ee5e474accb31816f357985f44)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:50.939368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:51.081962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '778561ee5e474accb31816f357985f44'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:51.109963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '778561ee5e474accb31816f357985f44'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:51.132034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '778561ee5e474accb31816f357985f44'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:51.132930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '778561ee5e474accb31816f357985f44'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:51.133732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '778561ee5e474accb31816f357985f44'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:51.179012 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1681/3325] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:50 2022] DELETE /identity/v3/users/778561ee5e474accb31816f357985f44 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:51.583222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66799980-e153-44a9-bbba-afaa71ac781f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:51.584059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66799980-e153-44a9-bbba-afaa71ac781f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:51.584488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66799980-e153-44a9-bbba-afaa71ac781f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:51.636781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-66799980-e153-44a9-bbba-afaa71ac781f None None] MFA Rules not processed for user `999db58fc81f4a9ab60bdd21674ec8c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:51.638343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-66799980-e153-44a9-bbba-afaa71ac781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:51.663288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:51.795317 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1645/3326] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:51.808358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:51.811150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:51.894126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:51.968570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1591cf-85df-4518-a296-720c4e512b56 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:51.975635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b70b2-e9ce-4f0e-b930-89a8497a8581 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:51.976288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b70b2-e9ce-4f0e-b930-89a8497a8581 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:51.976577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b70b2-e9ce-4f0e-b930-89a8497a8581 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:51.976859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b70b2-e9ce-4f0e-b930-89a8497a8581 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:51.988449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b70b2-e9ce-4f0e-b930-89a8497a8581 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:51.990414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b70b2-e9ce-4f0e-b930-89a8497a8581 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:52.022031 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1682/3327] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:52.305790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66799980-e153-44a9-bbba-afaa71ac781f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:52.307328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-66799980-e153-44a9-bbba-afaa71ac781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:52.485268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66799980-e153-44a9-bbba-afaa71ac781f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:52.512096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:52.512791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:52.515965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:52.516260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] PATH_INFO: `/v3/projects/361f907a187547aabda6cb54eeeb3684` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:52.518127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=361f907a187547aabda6cb54eeeb3684)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:52.520726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:52.625760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '361f907a187547aabda6cb54eeeb3684', 'request_id': 'req-d0a3f3d0-130a-4686-a0f3-21742449fe5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:52.638497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '361f907a187547aabda6cb54eeeb3684', 'request_id': 'req-d0a3f3d0-130a-4686-a0f3-21742449fe5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:52.648105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '361f907a187547aabda6cb54eeeb3684', 'request_id': 'req-d0a3f3d0-130a-4686-a0f3-21742449fe5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:52.662223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] The token cache is being invalidate because project 361f907a187547aabda6cb54eeeb3684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:52.663703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:52.663703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:52.669371 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1646/3328] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:52 2022] DELETE /identity/v3/projects/361f907a187547aabda6cb54eeeb3684 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:52.674485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b70b2-e9ce-4f0e-b930-89a8497a8581 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:52.690629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b70b2-e9ce-4f0e-b930-89a8497a8581 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:52.790794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3f479-957e-49af-8f55-d83e5cd1a637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:52.791567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3f479-957e-49af-8f55-d83e5cd1a637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:52.791930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3f479-957e-49af-8f55-d83e5cd1a637 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:52.792285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3f479-957e-49af-8f55-d83e5cd1a637 admin admin] PATH_INFO: `/v3/users/0e1dda23b62b4c369b10b00340c6dcbd` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:52.793576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3f479-957e-49af-8f55-d83e5cd1a637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1dda23b62b4c369b10b00340c6dcbd)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:52.795658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3f479-957e-49af-8f55-d83e5cd1a637 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:52.879568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6ea3f479-957e-49af-8f55-d83e5cd1a637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1dda23b62b4c369b10b00340c6dcbd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:52.904715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6ea3f479-957e-49af-8f55-d83e5cd1a637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1dda23b62b4c369b10b00340c6dcbd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:52.926519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6ea3f479-957e-49af-8f55-d83e5cd1a637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1dda23b62b4c369b10b00340c6dcbd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:52.928186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6ea3f479-957e-49af-8f55-d83e5cd1a637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1dda23b62b4c369b10b00340c6dcbd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:52.930843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6ea3f479-957e-49af-8f55-d83e5cd1a637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1dda23b62b4c369b10b00340c6dcbd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:52.955133 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1683/3329] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:52 2022] DELETE /identity/v3/users/0e1dda23b62b4c369b10b00340c6dcbd => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:53.602299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:53.604416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:53.625474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3f3d0-130a-4686-a0f3-21742449fe5d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:53.811604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:53.868268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:53.870167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:53.870673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:53.871301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d admin admin] PATH_INFO: `/v3/projects/e826490d9b834105834b6e18fa788131` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:53.873932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e826490d9b834105834b6e18fa788131)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:53.876057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:53.984352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e826490d9b834105834b6e18fa788131', 'request_id': 'req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:53.996847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e826490d9b834105834b6e18fa788131', 'request_id': 'req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:54.005179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e826490d9b834105834b6e18fa788131', 'request_id': 'req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:54.029378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d admin admin] The token cache is being invalidate because project e826490d9b834105834b6e18fa788131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:54.030206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:54.031322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d3071b1a-ddbd-4681-b0e1-2b2116d1183d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:54.040331 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1647/3330] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:53 2022] DELETE /identity/v3/projects/e826490d9b834105834b6e18fa788131 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:54.045461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3aa41-03c2-4c3f-98d5-4a85d3fd4933 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:54.046361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3aa41-03c2-4c3f-98d5-4a85d3fd4933 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:54.046791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3aa41-03c2-4c3f-98d5-4a85d3fd4933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:54.098911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f9e3aa41-03c2-4c3f-98d5-4a85d3fd4933 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:54.101744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f9e3aa41-03c2-4c3f-98d5-4a85d3fd4933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:54.168875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:54.232950 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1684/3331] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:54.238455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883026c-793a-4b45-8930-a13f343b59a5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:54.238753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883026c-793a-4b45-8930-a13f343b59a5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:54.239030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883026c-793a-4b45-8930-a13f343b59a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:54.289952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3883026c-793a-4b45-8930-a13f343b59a5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:54.291223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3883026c-793a-4b45-8930-a13f343b59a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:54.336211 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1648/3332] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:54.351007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3aa41-03c2-4c3f-98d5-4a85d3fd4933 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:54.360430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883026c-793a-4b45-8930-a13f343b59a5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:54.364142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3883026c-793a-4b45-8930-a13f343b59a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:54.379466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3aa41-03c2-4c3f-98d5-4a85d3fd4933 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:54.397884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a662ce-7c18-4041-a172-a172c581597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:54.403674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883026c-793a-4b45-8930-a13f343b59a5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:54.404920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a662ce-7c18-4041-a172-a172c581597a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:54.406930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a662ce-7c18-4041-a172-a172c581597a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:54.407233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a662ce-7c18-4041-a172-a172c581597a admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:54.410062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a662ce-7c18-4041-a172-a172c581597a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:54.410062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a662ce-7c18-4041-a172-a172c581597a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:54.412016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a662ce-7c18-4041-a172-a172c581597a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:54.420946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:54.422174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:54.422683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:54.423913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] PATH_INFO: `/v3/users/3da1337b27b64e008b26a2c464578913` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:54.425713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3da1337b27b64e008b26a2c464578913)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:54.429024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:54.458509 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1685/3333] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:54.459323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a662ce-7c18-4041-a172-a172c581597a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:54.484395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a662ce-7c18-4041-a172-a172c581597a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:54.490377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfefba-2ca2-475c-adf2-7807f86cc6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:54.491842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfefba-2ca2-475c-adf2-7807f86cc6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:54.492139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfefba-2ca2-475c-adf2-7807f86cc6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:54.492423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfefba-2ca2-475c-adf2-7807f86cc6c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:54.497425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfefba-2ca2-475c-adf2-7807f86cc6c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:54.497860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfefba-2ca2-475c-adf2-7807f86cc6c7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:54.552803 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1686/3334] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:54 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:54.567275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3da1337b27b64e008b26a2c464578913'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:54.576186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfefba-2ca2-475c-adf2-7807f86cc6c7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:54.588094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3da1337b27b64e008b26a2c464578913'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:54.589648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3da1337b27b64e008b26a2c464578913'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:54.607705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3da1337b27b64e008b26a2c464578913'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:54.619805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3da1337b27b64e008b26a2c464578913'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:54.632398 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1649/3335] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:54 2022] DELETE /identity/v3/users/3da1337b27b64e008b26a2c464578913 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:57:54.731028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfefba-2ca2-475c-adf2-7807f86cc6c7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:54.866859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6529b58-7dad-4480-a274-37b6a085128e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:54.868327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6529b58-7dad-4480-a274-37b6a085128e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:54.870307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6529b58-7dad-4480-a274-37b6a085128e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:54.870685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6529b58-7dad-4480-a274-37b6a085128e admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:54.871428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6529b58-7dad-4480-a274-37b6a085128e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:54.876493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6529b58-7dad-4480-a274-37b6a085128e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:54.929303 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1687/3336] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:54 2022] POST /identity/v3/users => generated 292 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:54.932652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:54.937356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:54.963155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6529b58-7dad-4480-a274-37b6a085128e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:54.965385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a6529b58-7dad-4480-a274-37b6a085128e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:54.980190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f6b77-0853-4c21-89de-21538aa52ca2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:54.991449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d75f2-d0e9-428a-8b31-0d24e5c63c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:54.992601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d75f2-d0e9-428a-8b31-0d24e5c63c2c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:54.992789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d75f2-d0e9-428a-8b31-0d24e5c63c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:54.993098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d75f2-d0e9-428a-8b31-0d24e5c63c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:54.993764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d75f2-d0e9-428a-8b31-0d24e5c63c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:54.996588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d75f2-d0e9-428a-8b31-0d24e5c63c2c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:55.006430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6529b58-7dad-4480-a274-37b6a085128e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:55.025548 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1650/3337] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:55.027987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd601e-1695-4ef9-b6ff-23f2437b9580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:55.029310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd601e-1695-4ef9-b6ff-23f2437b9580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.029533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd601e-1695-4ef9-b6ff-23f2437b9580 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.029823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd601e-1695-4ef9-b6ff-23f2437b9580 admin admin] PATH_INFO: `/v3/projects/42f287c9b2f24f4a94002ca61190c12d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.031230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bd601e-1695-4ef9-b6ff-23f2437b9580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f287c9b2f24f4a94002ca61190c12d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:55.034773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d75f2-d0e9-428a-8b31-0d24e5c63c2c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:55.034911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bd601e-1695-4ef9-b6ff-23f2437b9580 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:55.057300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d75f2-d0e9-428a-8b31-0d24e5c63c2c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:55.072986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d612e66-73e6-43d4-867c-e9a58ca2be9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:55.073611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d612e66-73e6-43d4-867c-e9a58ca2be9c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.073907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d612e66-73e6-43d4-867c-e9a58ca2be9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.074192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d612e66-73e6-43d4-867c-e9a58ca2be9c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.075013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d612e66-73e6-43d4-867c-e9a58ca2be9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:55.076650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d612e66-73e6-43d4-867c-e9a58ca2be9c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:55.102898 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1651/3338] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:55.110856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d612e66-73e6-43d4-867c-e9a58ca2be9c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:55.122967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:55.143507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d612e66-73e6-43d4-867c-e9a58ca2be9c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:55.155070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09bd601e-1695-4ef9-b6ff-23f2437b9580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f287c9b2f24f4a94002ca61190c12d', 'request_id': 'req-09bd601e-1695-4ef9-b6ff-23f2437b9580'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:55.173928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09bd601e-1695-4ef9-b6ff-23f2437b9580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f287c9b2f24f4a94002ca61190c12d', 'request_id': 'req-09bd601e-1695-4ef9-b6ff-23f2437b9580'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:55.176120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:55.187909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09bd601e-1695-4ef9-b6ff-23f2437b9580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f287c9b2f24f4a94002ca61190c12d', 'request_id': 'req-09bd601e-1695-4ef9-b6ff-23f2437b9580'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:55.207241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09bd601e-1695-4ef9-b6ff-23f2437b9580 admin admin] The token cache is being invalidate because project 42f287c9b2f24f4a94002ca61190c12d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:55.208167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09bd601e-1695-4ef9-b6ff-23f2437b9580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:55.208290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09bd601e-1695-4ef9-b6ff-23f2437b9580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:55.226283 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1688/3339] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:54 2022] DELETE /identity/v3/projects/42f287c9b2f24f4a94002ca61190c12d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:55.229735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8828584-0235-4ec3-9912-ad152bbff067 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.230181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8828584-0235-4ec3-9912-ad152bbff067 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.230472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8828584-0235-4ec3-9912-ad152bbff067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.231500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8a0eb-6afa-4bac-8a9c-9e752643deb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:55.232573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8a0eb-6afa-4bac-8a9c-9e752643deb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.232573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8a0eb-6afa-4bac-8a9c-9e752643deb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.232975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8a0eb-6afa-4bac-8a9c-9e752643deb7 admin admin] PATH_INFO: `/v3/projects/a0a0f20e6a344a52afc41bc546f14405/users/764f034ac9b3441fa08a4ebfa7a965fd/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.258584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8a0eb-6afa-4bac-8a9c-9e752643deb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a0f20e6a344a52afc41bc546f14405, user_id=764f034ac9b3441fa08a4ebfa7a965fd, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:55.261518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8a0eb-6afa-4bac-8a9c-9e752643deb7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:55.262511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e8828584-0235-4ec3-9912-ad152bbff067 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:55.263802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e8828584-0235-4ec3-9912-ad152bbff067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:55.280483 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1689/3340] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:55.282292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45486fce-4eb6-4615-a89a-3c336891fcf0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.283115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45486fce-4eb6-4615-a89a-3c336891fcf0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.283399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45486fce-4eb6-4615-a89a-3c336891fcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.287030 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1652/3341] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:55 2022] PUT /identity/v3/projects/a0a0f20e6a344a52afc41bc546f14405/users/764f034ac9b3441fa08a4ebfa7a965fd/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:55.300522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875f38f-85a2-475c-9cbe-2cf15e98b7a6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.301004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875f38f-85a2-475c-9cbe-2cf15e98b7a6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.301294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875f38f-85a2-475c-9cbe-2cf15e98b7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.328950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-45486fce-4eb6-4615-a89a-3c336891fcf0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:55.330440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-45486fce-4eb6-4615-a89a-3c336891fcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:55.353436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0875f38f-85a2-475c-9cbe-2cf15e98b7a6 None None] MFA Rules not processed for user `764f034ac9b3441fa08a4ebfa7a965fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:55.354908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0875f38f-85a2-475c-9cbe-2cf15e98b7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:55.419952 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1690/3342] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:55.424681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45486fce-4eb6-4615-a89a-3c336891fcf0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:55.435421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45486fce-4eb6-4615-a89a-3c336891fcf0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:55.441161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00148de5-4a83-45c4-90ae-3684c5a37e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:55.441927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00148de5-4a83-45c4-90ae-3684c5a37e29 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.442284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00148de5-4a83-45c4-90ae-3684c5a37e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.442801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00148de5-4a83-45c4-90ae-3684c5a37e29 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.444144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00148de5-4a83-45c4-90ae-3684c5a37e29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:55.444537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00148de5-4a83-45c4-90ae-3684c5a37e29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:55.446634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00148de5-4a83-45c4-90ae-3684c5a37e29 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:55.461508 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1691/3343] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:57:55.465856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00148de5-4a83-45c4-90ae-3684c5a37e29 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:55.479782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00148de5-4a83-45c4-90ae-3684c5a37e29 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:55.486583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af7686-40ee-4086-96eb-bb6a72ea9cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:55.487350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af7686-40ee-4086-96eb-bb6a72ea9cea admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.487708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af7686-40ee-4086-96eb-bb6a72ea9cea admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.487991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af7686-40ee-4086-96eb-bb6a72ea9cea admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.488690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af7686-40ee-4086-96eb-bb6a72ea9cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:55.491039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af7686-40ee-4086-96eb-bb6a72ea9cea admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:55.530589 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1653/3344] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:55.535420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875f38f-85a2-475c-9cbe-2cf15e98b7a6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:55.540778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0875f38f-85a2-475c-9cbe-2cf15e98b7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:55.550784 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1692/3345] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:55 2022] POST /identity/v3/projects => generated 359 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:55.552003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af7686-40ee-4086-96eb-bb6a72ea9cea admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:55.569335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875f38f-85a2-475c-9cbe-2cf15e98b7a6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:55.686884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af7686-40ee-4086-96eb-bb6a72ea9cea admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:55.701235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1738fbc-f671-4f9c-be06-e73cf328f7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:55.703786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1738fbc-f671-4f9c-be06-e73cf328f7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.704797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1738fbc-f671-4f9c-be06-e73cf328f7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.705105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1738fbc-f671-4f9c-be06-e73cf328f7a1 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.709887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1738fbc-f671-4f9c-be06-e73cf328f7a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:55.713399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1738fbc-f671-4f9c-be06-e73cf328f7a1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:55.726285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c660a-1bea-4ba6-bf54-bf7af8ad57a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:55.726950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c660a-1bea-4ba6-bf54-bf7af8ad57a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.727299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c660a-1bea-4ba6-bf54-bf7af8ad57a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.727503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c660a-1bea-4ba6-bf54-bf7af8ad57a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.730372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c660a-1bea-4ba6-bf54-bf7af8ad57a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:55.734198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c660a-1bea-4ba6-bf54-bf7af8ad57a9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:55.763122 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1693/3346] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:55 2022] POST /identity/v3/users => generated 280 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:55.764066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1738fbc-f671-4f9c-be06-e73cf328f7a1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:55.779613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1738fbc-f671-4f9c-be06-e73cf328f7a1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:55.786000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2c55f-4911-4de7-9bbf-e8b919cfaf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:55.786620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2c55f-4911-4de7-9bbf-e8b919cfaf60 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.786961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2c55f-4911-4de7-9bbf-e8b919cfaf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.787173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2c55f-4911-4de7-9bbf-e8b919cfaf60 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.787820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2c55f-4911-4de7-9bbf-e8b919cfaf60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:55.789480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2c55f-4911-4de7-9bbf-e8b919cfaf60 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:55.791280 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1654/3347] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:55 2022] POST /identity/v3/projects => generated 385 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:57:55.795289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c660a-1bea-4ba6-bf54-bf7af8ad57a9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:55.797764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:55.815186 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1694/3348] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:55.821549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2c55f-4911-4de7-9bbf-e8b919cfaf60 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:55.826870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:55.838294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2c55f-4911-4de7-9bbf-e8b919cfaf60 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:55.902178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c660a-1bea-4ba6-bf54-bf7af8ad57a9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:55.914160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575590d0-b927-4ac3-9403-b7facfe1d34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:55.914777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575590d0-b927-4ac3-9403-b7facfe1d34f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.915333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575590d0-b927-4ac3-9403-b7facfe1d34f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.915333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575590d0-b927-4ac3-9403-b7facfe1d34f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.915963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575590d0-b927-4ac3-9403-b7facfe1d34f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:55.917937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575590d0-b927-4ac3-9403-b7facfe1d34f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:55.930287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d663a9a-be16-4330-88f4-616349896a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:55.930287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d663a9a-be16-4330-88f4-616349896a27 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.930549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d663a9a-be16-4330-88f4-616349896a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.930819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d663a9a-be16-4330-88f4-616349896a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.931476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d663a9a-be16-4330-88f4-616349896a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:55.933479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d663a9a-be16-4330-88f4-616349896a27 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:55.953278 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1655/3349] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:55 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:57:55.954850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575590d0-b927-4ac3-9403-b7facfe1d34f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:55.965507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575590d0-b927-4ac3-9403-b7facfe1d34f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:55.971325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41926-9794-49ec-aec1-dff521e3581d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:55.972042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41926-9794-49ec-aec1-dff521e3581d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.972543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41926-9794-49ec-aec1-dff521e3581d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.972653 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1695/3350] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:55.973071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41926-9794-49ec-aec1-dff521e3581d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.973563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d663a9a-be16-4330-88f4-616349896a27 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:55.973853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d41926-9794-49ec-aec1-dff521e3581d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:55.975530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d41926-9794-49ec-aec1-dff521e3581d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:55.983672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d663a9a-be16-4330-88f4-616349896a27 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:55.989557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a470a9-1a39-4432-8012-0097fbef0066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:55.990367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a470a9-1a39-4432-8012-0097fbef0066 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:55.990665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a470a9-1a39-4432-8012-0097fbef0066 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:55.990955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a470a9-1a39-4432-8012-0097fbef0066 admin admin] PATH_INFO: `/v3/projects/ab87b0541b3e4c558e246e41600a27fe/users/bed5d20ece834fedb82fa38ac13ba412/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:55.992286 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1656/3351] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:55.993318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41926-9794-49ec-aec1-dff521e3581d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:56.005989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41926-9794-49ec-aec1-dff521e3581d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:56.013834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26e7f7-928f-4554-85f9-0e984b97cd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:56.014314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a470a9-1a39-4432-8012-0097fbef0066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab87b0541b3e4c558e246e41600a27fe, user_id=bed5d20ece834fedb82fa38ac13ba412, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:56.015104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26e7f7-928f-4554-85f9-0e984b97cd1a admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:56.015658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26e7f7-928f-4554-85f9-0e984b97cd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:56.016102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26e7f7-928f-4554-85f9-0e984b97cd1a admin admin] PATH_INFO: `/v3/projects/4b9048a2771540d485cb4e127443168f/users/99bc778b012a4aa3b0ad8855496c480e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:56.017600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a470a9-1a39-4432-8012-0097fbef0066 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:56.034785 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1696/3352] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:55 2022] PUT /identity/v3/projects/ab87b0541b3e4c558e246e41600a27fe/users/bed5d20ece834fedb82fa38ac13ba412/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:56.038116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21b5c8-fa98-4a38-a678-76cfa0378497 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:56.038415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21b5c8-fa98-4a38-a678-76cfa0378497 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:56.038705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21b5c8-fa98-4a38-a678-76cfa0378497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:56.046055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26e7f7-928f-4554-85f9-0e984b97cd1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9048a2771540d485cb4e127443168f, user_id=99bc778b012a4aa3b0ad8855496c480e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:56.049439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26e7f7-928f-4554-85f9-0e984b97cd1a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:56.064043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fe21b5c8-fa98-4a38-a678-76cfa0378497 None None] MFA Rules not processed for user `0c77fca40dbc430db41e4a30639b28b5`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:56.065432 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1657/3353] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:55 2022] PUT /identity/v3/projects/4b9048a2771540d485cb4e127443168f/users/99bc778b012a4aa3b0ad8855496c480e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:56.066845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6194b0-9fb8-4329-8c41-789c780205de None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:56.067212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6194b0-9fb8-4329-8c41-789c780205de None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:56.067564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6194b0-9fb8-4329-8c41-789c780205de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:56.068313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fe21b5c8-fa98-4a38-a678-76cfa0378497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:56.113955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5f6194b0-9fb8-4329-8c41-789c780205de None None] MFA Rules not processed for user `bed5d20ece834fedb82fa38ac13ba412`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:56.115151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5f6194b0-9fb8-4329-8c41-789c780205de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:56.161098 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1697/3354] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:57:56.161939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21b5c8-fa98-4a38-a678-76cfa0378497 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:56.213531 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1658/3355] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:56.240212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21b5c8-fa98-4a38-a678-76cfa0378497 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:56.245257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234dce3-07bd-4de8-b196-c307b8af7dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:56.246088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234dce3-07bd-4de8-b196-c307b8af7dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:56.246378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234dce3-07bd-4de8-b196-c307b8af7dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:56.246661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234dce3-07bd-4de8-b196-c307b8af7dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:56.247317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234dce3-07bd-4de8-b196-c307b8af7dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:56.249033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234dce3-07bd-4de8-b196-c307b8af7dd2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:56.259982 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1698/3356] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:56.263542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6194b0-9fb8-4329-8c41-789c780205de None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:56.274703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6194b0-9fb8-4329-8c41-789c780205de None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:56.279392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280e857-f5c3-4960-a849-33403a1208da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:56.280060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280e857-f5c3-4960-a849-33403a1208da admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:56.280374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280e857-f5c3-4960-a849-33403a1208da admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:56.280805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280e857-f5c3-4960-a849-33403a1208da admin admin] PATH_INFO: `/v3/projects/4b9048a2771540d485cb4e127443168f/users/99bc778b012a4aa3b0ad8855496c480e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:56.283349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280e857-f5c3-4960-a849-33403a1208da admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9048a2771540d485cb4e127443168f, user_id=99bc778b012a4aa3b0ad8855496c480e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:56.286117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280e857-f5c3-4960-a849-33403a1208da admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:56.300969 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1659/3357] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:56 2022] PUT /identity/v3/projects/4b9048a2771540d485cb4e127443168f/users/99bc778b012a4aa3b0ad8855496c480e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:56.367470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdc80a-c7bf-4854-ae40-b9f0cbe6ddc5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:56.367772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdc80a-c7bf-4854-ae40-b9f0cbe6ddc5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:56.367962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdc80a-c7bf-4854-ae40-b9f0cbe6ddc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:56.389450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:56.399926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b9bdc80a-c7bf-4854-ae40-b9f0cbe6ddc5 None None] MFA Rules not processed for user `99bc778b012a4aa3b0ad8855496c480e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:56.402421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b9bdc80a-c7bf-4854-ae40-b9f0cbe6ddc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:56.546995 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1699/3358] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:56.731941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280e857-f5c3-4960-a849-33403a1208da admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:56.733990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8280e857-f5c3-4960-a849-33403a1208da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:56.798567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:56.836160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280e857-f5c3-4960-a849-33403a1208da admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:56.842484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec9547-42ca-4eae-9503-a09dabf4cadd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:56.843107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec9547-42ca-4eae-9503-a09dabf4cadd service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:56.843386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec9547-42ca-4eae-9503-a09dabf4cadd service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:56.843659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec9547-42ca-4eae-9503-a09dabf4cadd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:56.914937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec9547-42ca-4eae-9503-a09dabf4cadd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:56.916538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec9547-42ca-4eae-9503-a09dabf4cadd service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:56.926903 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1660/3359] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:56 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:56.949064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdc80a-c7bf-4854-ae40-b9f0cbe6ddc5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:56.950640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b9bdc80a-c7bf-4854-ae40-b9f0cbe6ddc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:56.968772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdc80a-c7bf-4854-ae40-b9f0cbe6ddc5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:56.975328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb81b8-cdc4-40a8-8b85-47c1aeba170d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:56.976489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb81b8-cdc4-40a8-8b85-47c1aeba170d service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:56.976829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb81b8-cdc4-40a8-8b85-47c1aeba170d service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:56.977296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb81b8-cdc4-40a8-8b85-47c1aeba170d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:56.986559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb81b8-cdc4-40a8-8b85-47c1aeba170d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:56.989113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb81b8-cdc4-40a8-8b85-47c1aeba170d service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:57.007166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec9547-42ca-4eae-9503-a09dabf4cadd service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:57.033105 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1700/3360] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:57.146068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec9547-42ca-4eae-9503-a09dabf4cadd service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:57.151659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:57.152313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:57.152599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:57.152880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] PATH_INFO: `/v3/users/bed5d20ece834fedb82fa38ac13ba412` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:57.154981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bed5d20ece834fedb82fa38ac13ba412)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:57.157201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:57.237261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bed5d20ece834fedb82fa38ac13ba412'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:57.254968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bed5d20ece834fedb82fa38ac13ba412'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:57.255843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bed5d20ece834fedb82fa38ac13ba412'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:57.273221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bed5d20ece834fedb82fa38ac13ba412'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:57.299079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bed5d20ece834fedb82fa38ac13ba412'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:57.310322 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1661/3361] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:57 2022] DELETE /identity/v3/users/bed5d20ece834fedb82fa38ac13ba412 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:57.315233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44712-7dac-43ab-9e87-0053e5e601ba None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:57.315845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44712-7dac-43ab-9e87-0053e5e601ba None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:57.316126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44712-7dac-43ab-9e87-0053e5e601ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:57.383286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-79d44712-7dac-43ab-9e87-0053e5e601ba None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:57.384526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-79d44712-7dac-43ab-9e87-0053e5e601ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:57.506863 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1701/3362] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:57.515468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:57.517671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:57.525842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:57.557587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44712-7dac-43ab-9e87-0053e5e601ba None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:57.562137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-79d44712-7dac-43ab-9e87-0053e5e601ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:57.592085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44712-7dac-43ab-9e87-0053e5e601ba None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:57.610988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:57.611709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:57.612043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:57.612387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] PATH_INFO: `/v3/users/764f034ac9b3441fa08a4ebfa7a965fd` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:57.613705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] RBAC: Authorizing `identity:delete_user(user_id=764f034ac9b3441fa08a4ebfa7a965fd)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:57.616079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:57.634287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c5afc-d74a-4ba5-bc4f-4e409e73fb81 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:57.638335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c9628-3dba-4218-b243-bdae34e0225b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:57.638726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c9628-3dba-4218-b243-bdae34e0225b service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:57.639042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c9628-3dba-4218-b243-bdae34e0225b service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:57.639302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c9628-3dba-4218-b243-bdae34e0225b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:57.652686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c9628-3dba-4218-b243-bdae34e0225b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:57.654730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c9628-3dba-4218-b243-bdae34e0225b service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:57.678008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764f034ac9b3441fa08a4ebfa7a965fd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:57.684057 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1662/3363] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:57.696178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764f034ac9b3441fa08a4ebfa7a965fd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:57.707810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764f034ac9b3441fa08a4ebfa7a965fd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:57.708678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764f034ac9b3441fa08a4ebfa7a965fd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:57.709491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764f034ac9b3441fa08a4ebfa7a965fd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:57.716979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:57.734708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d311c5d-6fb8-4633-9073-d4048e7c8775 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:57.735005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d311c5d-6fb8-4633-9073-d4048e7c8775 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:57.735514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d311c5d-6fb8-4633-9073-d4048e7c8775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:57.738769 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1702/3364] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:57 2022] DELETE /identity/v3/users/764f034ac9b3441fa08a4ebfa7a965fd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:57.760069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7d311c5d-6fb8-4633-9073-d4048e7c8775 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:57.761493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7d311c5d-6fb8-4633-9073-d4048e7c8775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:57.853209 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1663/3365] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:57.861795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:57.863758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:57.956858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e605e7-cd33-4f9f-a39f-af8baf55b69c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:57.961584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da593e7c-6fd1-4ef2-a780-ef576b9d1384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:57.962276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da593e7c-6fd1-4ef2-a780-ef576b9d1384 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:57.962558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da593e7c-6fd1-4ef2-a780-ef576b9d1384 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:57.963234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da593e7c-6fd1-4ef2-a780-ef576b9d1384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:57.975792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da593e7c-6fd1-4ef2-a780-ef576b9d1384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:57.977292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da593e7c-6fd1-4ef2-a780-ef576b9d1384 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:57.988620 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1703/3366] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:57:58.025250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d311c5d-6fb8-4633-9073-d4048e7c8775 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:58.027842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7d311c5d-6fb8-4633-9073-d4048e7c8775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:58.043776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d311c5d-6fb8-4633-9073-d4048e7c8775 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:58.051238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479951e-ea66-4cc4-a2e4-9e639802491e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:58.052007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479951e-ea66-4cc4-a2e4-9e639802491e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:58.052401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479951e-ea66-4cc4-a2e4-9e639802491e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:58.052665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479951e-ea66-4cc4-a2e4-9e639802491e admin admin] PATH_INFO: `/v3/projects/ab87b0541b3e4c558e246e41600a27fe` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:58.053953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479951e-ea66-4cc4-a2e4-9e639802491e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab87b0541b3e4c558e246e41600a27fe)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:58.064717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479951e-ea66-4cc4-a2e4-9e639802491e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:58.152011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8479951e-ea66-4cc4-a2e4-9e639802491e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab87b0541b3e4c558e246e41600a27fe', 'request_id': 'req-8479951e-ea66-4cc4-a2e4-9e639802491e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:58.160342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8479951e-ea66-4cc4-a2e4-9e639802491e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab87b0541b3e4c558e246e41600a27fe', 'request_id': 'req-8479951e-ea66-4cc4-a2e4-9e639802491e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:58.168255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8479951e-ea66-4cc4-a2e4-9e639802491e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab87b0541b3e4c558e246e41600a27fe', 'request_id': 'req-8479951e-ea66-4cc4-a2e4-9e639802491e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:58.182382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8479951e-ea66-4cc4-a2e4-9e639802491e admin admin] The token cache is being invalidate because project ab87b0541b3e4c558e246e41600a27fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:58.182781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8479951e-ea66-4cc4-a2e4-9e639802491e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:58.183549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8479951e-ea66-4cc4-a2e4-9e639802491e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:58.185400 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1664/3367] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:58 2022] DELETE /identity/v3/projects/ab87b0541b3e4c558e246e41600a27fe => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:57:58.192367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29117051-c386-4424-b118-7503dafea77a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:58.192663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29117051-c386-4424-b118-7503dafea77a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:58.193149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29117051-c386-4424-b118-7503dafea77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:58.217146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-29117051-c386-4424-b118-7503dafea77a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:58.218509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-29117051-c386-4424-b118-7503dafea77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:58.301745 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1704/3368] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:58.310842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496ec63-79ad-432b-9572-18421e87bb5d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:58.311158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496ec63-79ad-432b-9572-18421e87bb5d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:58.311435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496ec63-79ad-432b-9572-18421e87bb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:58.359540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b496ec63-79ad-432b-9572-18421e87bb5d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:58.361055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b496ec63-79ad-432b-9572-18421e87bb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:58.370834 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1665/3369] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:57:58.375344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29117051-c386-4424-b118-7503dafea77a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:58.389712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29117051-c386-4424-b118-7503dafea77a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:58.417031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c084b-8fdc-41fc-a874-a68ebd3ab88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:58.417966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c084b-8fdc-41fc-a874-a68ebd3ab88a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:58.418420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c084b-8fdc-41fc-a874-a68ebd3ab88a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:58.418851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c084b-8fdc-41fc-a874-a68ebd3ab88a admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:58.421672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c084b-8fdc-41fc-a874-a68ebd3ab88a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:57:58.423111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c084b-8fdc-41fc-a874-a68ebd3ab88a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:58.431174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c084b-8fdc-41fc-a874-a68ebd3ab88a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:58.452658 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1705/3370] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:57:58.453617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496ec63-79ad-432b-9572-18421e87bb5d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:58.465404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496ec63-79ad-432b-9572-18421e87bb5d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:58.472124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb17595-22b0-49ca-a75c-8ebf99f23cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:58.472887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb17595-22b0-49ca-a75c-8ebf99f23cfe admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:58.473181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb17595-22b0-49ca-a75c-8ebf99f23cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:58.473387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb17595-22b0-49ca-a75c-8ebf99f23cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:58.474093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb17595-22b0-49ca-a75c-8ebf99f23cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:58.476116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb17595-22b0-49ca-a75c-8ebf99f23cfe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:58.516226 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1666/3371] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:57:58 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:58.520424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c084b-8fdc-41fc-a874-a68ebd3ab88a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:58.536500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c084b-8fdc-41fc-a874-a68ebd3ab88a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:58.566575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb17595-22b0-49ca-a75c-8ebf99f23cfe admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:58.569148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cbb17595-22b0-49ca-a75c-8ebf99f23cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:58.602998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:58.676604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de903e1-a326-4abd-887c-24911bc1b96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:58.676604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de903e1-a326-4abd-887c-24911bc1b96c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:58.676604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de903e1-a326-4abd-887c-24911bc1b96c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:58.676604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de903e1-a326-4abd-887c-24911bc1b96c admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:58.676965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de903e1-a326-4abd-887c-24911bc1b96c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:58.679064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de903e1-a326-4abd-887c-24911bc1b96c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:58.706306 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1706/3372] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:57:58 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:57:58.710188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de903e1-a326-4abd-887c-24911bc1b96c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:58.723827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de903e1-a326-4abd-887c-24911bc1b96c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:58.729491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d15f0-e49f-4f3b-89e4-987f899bfae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:58.730193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d15f0-e49f-4f3b-89e4-987f899bfae4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:58.730480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d15f0-e49f-4f3b-89e4-987f899bfae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:58.730815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d15f0-e49f-4f3b-89e4-987f899bfae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:58.731470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d15f0-e49f-4f3b-89e4-987f899bfae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:58.733133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d15f0-e49f-4f3b-89e4-987f899bfae4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:58.742585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb17595-22b0-49ca-a75c-8ebf99f23cfe admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:58.746049 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1707/3373] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:57:58.749863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d15f0-e49f-4f3b-89e4-987f899bfae4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:58.750211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:58.751693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:58.752137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:58.752641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] PATH_INFO: `/v3/projects/a0a0f20e6a344a52afc41bc546f14405` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:58.755079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a0f20e6a344a52afc41bc546f14405)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:58.756470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:58.760595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:58.764549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d15f0-e49f-4f3b-89e4-987f899bfae4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:58.769953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c735b8-78df-4e9a-ad95-973461fee2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:58.770741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c735b8-78df-4e9a-ad95-973461fee2fc admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:58.771725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c735b8-78df-4e9a-ad95-973461fee2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:58.771725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c735b8-78df-4e9a-ad95-973461fee2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:58.772255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c735b8-78df-4e9a-ad95-973461fee2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:58.774010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c735b8-78df-4e9a-ad95-973461fee2fc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:58.785776 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1708/3374] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:57:58.790107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c735b8-78df-4e9a-ad95-973461fee2fc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:58.802358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c735b8-78df-4e9a-ad95-973461fee2fc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:58.809398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f6979-d19b-424d-8ad1-4b01a7c410f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:58.810628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f6979-d19b-424d-8ad1-4b01a7c410f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:58.810951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f6979-d19b-424d-8ad1-4b01a7c410f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:58.811377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f6979-d19b-424d-8ad1-4b01a7c410f5 admin admin] PATH_INFO: `/v3/projects/fb7084c9131a4f77b9688e10cd4c84de/users/322d75aacb244323b79f7f0e2ac464b8/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:58.829738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f6979-d19b-424d-8ad1-4b01a7c410f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb7084c9131a4f77b9688e10cd4c84de, user_id=322d75aacb244323b79f7f0e2ac464b8, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:58.833614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f6979-d19b-424d-8ad1-4b01a7c410f5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:58.837343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a0f20e6a344a52afc41bc546f14405', 'request_id': 'req-387fedc8-a6a9-4cd3-9c68-55f893e1654e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:58.849454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a0f20e6a344a52afc41bc546f14405', 'request_id': 'req-387fedc8-a6a9-4cd3-9c68-55f893e1654e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:58.854245 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1709/3375] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:57:58 2022] PUT /identity/v3/projects/fb7084c9131a4f77b9688e10cd4c84de/users/322d75aacb244323b79f7f0e2ac464b8/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:57:58.859797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a0f20e6a344a52afc41bc546f14405', 'request_id': 'req-387fedc8-a6a9-4cd3-9c68-55f893e1654e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:58.860391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dcf43-cf9a-4569-988b-57c3c917a65b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:58.861374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dcf43-cf9a-4569-988b-57c3c917a65b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:58.861870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dcf43-cf9a-4569-988b-57c3c917a65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:58.875973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] The token cache is being invalidate because project a0a0f20e6a344a52afc41bc546f14405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:57:58.876282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:58.876999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:58.881658 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1667/3376] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:57:58 2022] DELETE /identity/v3/projects/a0a0f20e6a344a52afc41bc546f14405 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:57:58.882833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:58.884350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:58.899480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f77dcf43-cf9a-4569-988b-57c3c917a65b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:57:58.900772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f77dcf43-cf9a-4569-988b-57c3c917a65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:58.964734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fedc8-a6a9-4cd3-9c68-55f893e1654e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:58.978435 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1710/3377] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:57:58.988483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dcf43-cf9a-4569-988b-57c3c917a65b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:57:58.990975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f77dcf43-cf9a-4569-988b-57c3c917a65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:57:59.018467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:57:59.041773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6f4c8-a415-4259-95d9-7a1949cffae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:59.042444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6f4c8-a415-4259-95d9-7a1949cffae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:59.042679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6f4c8-a415-4259-95d9-7a1949cffae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:59.043019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6f4c8-a415-4259-95d9-7a1949cffae7 admin admin] PATH_INFO: `/v3/users/99bc778b012a4aa3b0ad8855496c480e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:59.044168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6f4c8-a415-4259-95d9-7a1949cffae7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99bc778b012a4aa3b0ad8855496c480e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:59.046251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6f4c8-a415-4259-95d9-7a1949cffae7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:59.059955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dcf43-cf9a-4569-988b-57c3c917a65b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:57:59.065912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b29cb-2e42-487e-8672-cab097085bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:57:59.067197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b29cb-2e42-487e-8672-cab097085bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:57:59.067615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b29cb-2e42-487e-8672-cab097085bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:57:59.067906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b29cb-2e42-487e-8672-cab097085bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:57:59.081148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b29cb-2e42-487e-8672-cab097085bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:57:59.082740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b29cb-2e42-487e-8672-cab097085bf2 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:57:59.093978 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1711/3378] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:57:59.105343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-adf6f4c8-a415-4259-95d9-7a1949cffae7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99bc778b012a4aa3b0ad8855496c480e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:59.120677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-adf6f4c8-a415-4259-95d9-7a1949cffae7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99bc778b012a4aa3b0ad8855496c480e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:59.121656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-adf6f4c8-a415-4259-95d9-7a1949cffae7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99bc778b012a4aa3b0ad8855496c480e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:59.131653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-adf6f4c8-a415-4259-95d9-7a1949cffae7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99bc778b012a4aa3b0ad8855496c480e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:59.147534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-adf6f4c8-a415-4259-95d9-7a1949cffae7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99bc778b012a4aa3b0ad8855496c480e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:57:59.158348 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1668/3379] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:57:58 2022] DELETE /identity/v3/users/99bc778b012a4aa3b0ad8855496c480e => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:00.109729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b29cb-2e42-487e-8672-cab097085bf2 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:00.132863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b29cb-2e42-487e-8672-cab097085bf2 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:00.317733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc75dd-db22-427d-83c5-6bd782ae645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:00.319201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc75dd-db22-427d-83c5-6bd782ae645f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:00.319586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc75dd-db22-427d-83c5-6bd782ae645f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:00.320085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc75dd-db22-427d-83c5-6bd782ae645f admin admin] PATH_INFO: `/v3/projects/4b9048a2771540d485cb4e127443168f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:00.322696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc75dd-db22-427d-83c5-6bd782ae645f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b9048a2771540d485cb4e127443168f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:00.331206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc75dd-db22-427d-83c5-6bd782ae645f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:00.360170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:00.419185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ecdc75dd-db22-427d-83c5-6bd782ae645f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b9048a2771540d485cb4e127443168f', 'request_id': 'req-ecdc75dd-db22-427d-83c5-6bd782ae645f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:00.431439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ecdc75dd-db22-427d-83c5-6bd782ae645f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b9048a2771540d485cb4e127443168f', 'request_id': 'req-ecdc75dd-db22-427d-83c5-6bd782ae645f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:00.447074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ecdc75dd-db22-427d-83c5-6bd782ae645f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b9048a2771540d485cb4e127443168f', 'request_id': 'req-ecdc75dd-db22-427d-83c5-6bd782ae645f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:00.460970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ecdc75dd-db22-427d-83c5-6bd782ae645f admin admin] The token cache is being invalidate because project 4b9048a2771540d485cb4e127443168f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:58:00.461253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ecdc75dd-db22-427d-83c5-6bd782ae645f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:00.461898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ecdc75dd-db22-427d-83c5-6bd782ae645f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:00.464739 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1712/3380] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:58:00 2022] DELETE /identity/v3/projects/4b9048a2771540d485cb4e127443168f => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:00.470446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7138e4-cf60-4fcd-973f-b6aedcb6ec66 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:00.470944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7138e4-cf60-4fcd-973f-b6aedcb6ec66 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:00.471212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7138e4-cf60-4fcd-973f-b6aedcb6ec66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:00.504445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1b7138e4-cf60-4fcd-973f-b6aedcb6ec66 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:00.506753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1b7138e4-cf60-4fcd-973f-b6aedcb6ec66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:00.641743 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1669/3381] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:00.644278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f782a3-c738-41b7-8ec8-cbe70af47111 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:00.644647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f782a3-c738-41b7-8ec8-cbe70af47111 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:00.645050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f782a3-c738-41b7-8ec8-cbe70af47111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:00.679740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-27f782a3-c738-41b7-8ec8-cbe70af47111 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:00.682162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-27f782a3-c738-41b7-8ec8-cbe70af47111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:00.699997 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1713/3382] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:00.700875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7138e4-cf60-4fcd-973f-b6aedcb6ec66 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:00.716183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7138e4-cf60-4fcd-973f-b6aedcb6ec66 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:00.729069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fadbb-925e-4288-95d0-a4f02d210ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:00.731157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fadbb-925e-4288-95d0-a4f02d210ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:00.731554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fadbb-925e-4288-95d0-a4f02d210ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:00.732196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fadbb-925e-4288-95d0-a4f02d210ca0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:00.734251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fadbb-925e-4288-95d0-a4f02d210ca0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:58:00.734641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fadbb-925e-4288-95d0-a4f02d210ca0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:00.738755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fadbb-925e-4288-95d0-a4f02d210ca0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:00.771775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f782a3-c738-41b7-8ec8-cbe70af47111 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:00.778908 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1670/3383] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:58:00.782978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f782a3-c738-41b7-8ec8-cbe70af47111 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:00.788271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf1981-a8ea-4b93-8cbd-84b0f8ff44a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:00.789002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf1981-a8ea-4b93-8cbd-84b0f8ff44a1 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:00.789372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf1981-a8ea-4b93-8cbd-84b0f8ff44a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:00.789712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf1981-a8ea-4b93-8cbd-84b0f8ff44a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:00.790606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf1981-a8ea-4b93-8cbd-84b0f8ff44a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:00.792847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf1981-a8ea-4b93-8cbd-84b0f8ff44a1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:00.841519 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1714/3384] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:58:00 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:58:00.843905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fadbb-925e-4288-95d0-a4f02d210ca0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:00.857295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fadbb-925e-4288-95d0-a4f02d210ca0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:00.919404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:00.940688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749d6cc-cf20-45d8-a2c5-59dc1d179d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:00.941447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749d6cc-cf20-45d8-a2c5-59dc1d179d02 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:00.941900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749d6cc-cf20-45d8-a2c5-59dc1d179d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:00.942359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749d6cc-cf20-45d8-a2c5-59dc1d179d02 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:00.943133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749d6cc-cf20-45d8-a2c5-59dc1d179d02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:00.945277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749d6cc-cf20-45d8-a2c5-59dc1d179d02 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:00.976474 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1671/3385] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:58:00 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:58:00.980355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf1981-a8ea-4b93-8cbd-84b0f8ff44a1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:00.993400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf1981-a8ea-4b93-8cbd-84b0f8ff44a1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:00.999809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20709ca9-2b07-40ce-a5e9-310481fbe556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:01.000518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20709ca9-2b07-40ce-a5e9-310481fbe556 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:01.000808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20709ca9-2b07-40ce-a5e9-310481fbe556 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:01.001670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20709ca9-2b07-40ce-a5e9-310481fbe556 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:01.002711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20709ca9-2b07-40ce-a5e9-310481fbe556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:01.005970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20709ca9-2b07-40ce-a5e9-310481fbe556 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:01.032085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749d6cc-cf20-45d8-a2c5-59dc1d179d02 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:01.038767 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1715/3386] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:01.047139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749d6cc-cf20-45d8-a2c5-59dc1d179d02 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:01.060717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4d3dd-b1c2-4d13-9ab4-ded21a52532c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:01.075600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4d3dd-b1c2-4d13-9ab4-ded21a52532c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:01.077736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4d3dd-b1c2-4d13-9ab4-ded21a52532c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:01.078075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4d3dd-b1c2-4d13-9ab4-ded21a52532c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:01.078909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4d3dd-b1c2-4d13-9ab4-ded21a52532c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:01.085715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4d3dd-b1c2-4d13-9ab4-ded21a52532c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:01.124219 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1672/3387] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:58:01.128618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20709ca9-2b07-40ce-a5e9-310481fbe556 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:01.154884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20709ca9-2b07-40ce-a5e9-310481fbe556 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:01.172379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811ecf7-b3fd-46e2-b276-cdf6c08211a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:01.173747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811ecf7-b3fd-46e2-b276-cdf6c08211a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:01.174179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811ecf7-b3fd-46e2-b276-cdf6c08211a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:01.175344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811ecf7-b3fd-46e2-b276-cdf6c08211a7 admin admin] PATH_INFO: `/v3/projects/5a2f3c186f88497181cf6a1d806631b1/users/e6970a72952e4d4a8f26e0f8ee23f364/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:01.201732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811ecf7-b3fd-46e2-b276-cdf6c08211a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a2f3c186f88497181cf6a1d806631b1, user_id=e6970a72952e4d4a8f26e0f8ee23f364, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:01.208166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811ecf7-b3fd-46e2-b276-cdf6c08211a7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:01.239666 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1716/3388] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:58:01 2022] PUT /identity/v3/projects/5a2f3c186f88497181cf6a1d806631b1/users/e6970a72952e4d4a8f26e0f8ee23f364/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:01.240189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235e87-f0b7-484a-bd62-9eece6037574 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:01.240488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235e87-f0b7-484a-bd62-9eece6037574 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:01.240772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235e87-f0b7-484a-bd62-9eece6037574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:01.279805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-93235e87-f0b7-484a-bd62-9eece6037574 None None] MFA Rules not processed for user `e6970a72952e4d4a8f26e0f8ee23f364`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:01.281999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-93235e87-f0b7-484a-bd62-9eece6037574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:01.439469 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1673/3389] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:58:01.441252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811ecf7-b3fd-46e2-b276-cdf6c08211a7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:01.593420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811ecf7-b3fd-46e2-b276-cdf6c08211a7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:01.608794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1721e39-0316-4c78-b4b9-bde125326ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:01.609506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1721e39-0316-4c78-b4b9-bde125326ffe admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:01.609794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1721e39-0316-4c78-b4b9-bde125326ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:01.610068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1721e39-0316-4c78-b4b9-bde125326ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:01.611345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1721e39-0316-4c78-b4b9-bde125326ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:01.615552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1721e39-0316-4c78-b4b9-bde125326ffe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:01.684626 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1717/3390] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:58:01 2022] POST /identity/v3/projects => generated 383 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:58:01.688949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93235e87-f0b7-484a-bd62-9eece6037574 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:01.706516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93235e87-f0b7-484a-bd62-9eece6037574 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:01.834521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dad2f-6cd9-4db9-87b3-a0d09bac1ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:01.837181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dad2f-6cd9-4db9-87b3-a0d09bac1ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:01.837922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dad2f-6cd9-4db9-87b3-a0d09bac1ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:01.838531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dad2f-6cd9-4db9-87b3-a0d09bac1ba8 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:01.840169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581dad2f-6cd9-4db9-87b3-a0d09bac1ba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:01.843864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581dad2f-6cd9-4db9-87b3-a0d09bac1ba8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:01.892846 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1674/3391] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:58:01 2022] POST /identity/v3/users => generated 298 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:58:01.898893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1721e39-0316-4c78-b4b9-bde125326ffe admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:01.920836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1721e39-0316-4c78-b4b9-bde125326ffe admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:01.951912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8f6b5-3dab-49ab-ada7-32514fe8e607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:01.953432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8f6b5-3dab-49ab-ada7-32514fe8e607 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:01.954969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8f6b5-3dab-49ab-ada7-32514fe8e607 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:01.955448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8f6b5-3dab-49ab-ada7-32514fe8e607 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:01.956934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8f6b5-3dab-49ab-ada7-32514fe8e607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:01.958754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8f6b5-3dab-49ab-ada7-32514fe8e607 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:01.982051 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1718/3392] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:01.987991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dad2f-6cd9-4db9-87b3-a0d09bac1ba8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:02.004590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dad2f-6cd9-4db9-87b3-a0d09bac1ba8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:02.018472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f18ab-93ea-4f47-9318-d254567444a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:02.020641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f18ab-93ea-4f47-9318-d254567444a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:02.021171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f18ab-93ea-4f47-9318-d254567444a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:02.021742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f18ab-93ea-4f47-9318-d254567444a0 admin admin] PATH_INFO: `/v3/projects/9643215ef92149eeb8c0f3124b8b1bac/users/93af5a1eb0204db2aacd3373cd1decf7/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:02.073655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f18ab-93ea-4f47-9318-d254567444a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9643215ef92149eeb8c0f3124b8b1bac, user_id=93af5a1eb0204db2aacd3373cd1decf7, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:02.076629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f18ab-93ea-4f47-9318-d254567444a0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:02.117068 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1675/3393] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:58:01 2022] PUT /identity/v3/projects/9643215ef92149eeb8c0f3124b8b1bac/users/93af5a1eb0204db2aacd3373cd1decf7/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:02.119230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8f6b5-3dab-49ab-ada7-32514fe8e607 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:02.144352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8f6b5-3dab-49ab-ada7-32514fe8e607 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:02.163736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:02.274854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cb5e4-d220-4f53-bdce-69b2217e33b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:02.277922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cb5e4-d220-4f53-bdce-69b2217e33b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:02.278486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cb5e4-d220-4f53-bdce-69b2217e33b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:02.279316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cb5e4-d220-4f53-bdce-69b2217e33b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:02.284217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cb5e4-d220-4f53-bdce-69b2217e33b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:02.285075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cb5e4-d220-4f53-bdce-69b2217e33b4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:02.311063 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1719/3394] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:02.317164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f18ab-93ea-4f47-9318-d254567444a0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:02.331559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f18ab-93ea-4f47-9318-d254567444a0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:02.337976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6900f7-3ed6-4518-862d-0abcadc0e0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:02.338839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6900f7-3ed6-4518-862d-0abcadc0e0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:02.339133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6900f7-3ed6-4518-862d-0abcadc0e0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:02.339419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6900f7-3ed6-4518-862d-0abcadc0e0c1 admin admin] PATH_INFO: `/v3/projects/9643215ef92149eeb8c0f3124b8b1bac/users/93af5a1eb0204db2aacd3373cd1decf7/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:02.341697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6900f7-3ed6-4518-862d-0abcadc0e0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9643215ef92149eeb8c0f3124b8b1bac, user_id=93af5a1eb0204db2aacd3373cd1decf7, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:02.350960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6900f7-3ed6-4518-862d-0abcadc0e0c1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:02.396450 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1676/3395] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:58:02 2022] PUT /identity/v3/projects/9643215ef92149eeb8c0f3124b8b1bac/users/93af5a1eb0204db2aacd3373cd1decf7/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:02.406767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184784e-4fb7-4c18-bde3-cd13406b45bf None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:02.407063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184784e-4fb7-4c18-bde3-cd13406b45bf None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:02.407904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184784e-4fb7-4c18-bde3-cd13406b45bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:02.482421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3184784e-4fb7-4c18-bde3-cd13406b45bf None None] MFA Rules not processed for user `93af5a1eb0204db2aacd3373cd1decf7`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:02.484439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3184784e-4fb7-4c18-bde3-cd13406b45bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:02.690787 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1720/3396] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:02.692747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4782b-88b8-4837-a606-c8a7c88586f2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:02.693415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4782b-88b8-4837-a606-c8a7c88586f2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:02.693830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4782b-88b8-4837-a606-c8a7c88586f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:02.711761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:02.754127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-03f4782b-88b8-4837-a606-c8a7c88586f2 None None] MFA Rules not processed for user `e6970a72952e4d4a8f26e0f8ee23f364`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:02.761319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-03f4782b-88b8-4837-a606-c8a7c88586f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:02.924327 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1677/3397] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:02.934699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184784e-4fb7-4c18-bde3-cd13406b45bf None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:02.940460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3184784e-4fb7-4c18-bde3-cd13406b45bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:03.185776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184784e-4fb7-4c18-bde3-cd13406b45bf None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:03.206041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06499cc6-50df-4dba-aa0f-c6532ea6902b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:03.208389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06499cc6-50df-4dba-aa0f-c6532ea6902b service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:03.208389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06499cc6-50df-4dba-aa0f-c6532ea6902b service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:03.208389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06499cc6-50df-4dba-aa0f-c6532ea6902b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:03.242358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06499cc6-50df-4dba-aa0f-c6532ea6902b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:03.245293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06499cc6-50df-4dba-aa0f-c6532ea6902b service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:03.258210 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1721/3398] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:09.305450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51d7bf-5839-4000-9b1d-ec0a7729d2cd None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:09.306043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51d7bf-5839-4000-9b1d-ec0a7729d2cd None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:09.306229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51d7bf-5839-4000-9b1d-ec0a7729d2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:09.321603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06499cc6-50df-4dba-aa0f-c6532ea6902b service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:09.345708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06499cc6-50df-4dba-aa0f-c6532ea6902b service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:09.396476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b4f89-d965-4007-9bfe-15a75865a950 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:09.400559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b4f89-d965-4007-9bfe-15a75865a950 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:09.401715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b4f89-d965-4007-9bfe-15a75865a950 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:09.404765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b4f89-d965-4007-9bfe-15a75865a950 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:09.408269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e26b4f89-d965-4007-9bfe-15a75865a950 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:09.427576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6b51d7bf-5839-4000-9b1d-ec0a7729d2cd None None] MFA Rules not processed for user `322d75aacb244323b79f7f0e2ac464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:09.432213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6b51d7bf-5839-4000-9b1d-ec0a7729d2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:09.450813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:09.565178 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1678/3399] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:09.570163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51d7bf-5839-4000-9b1d-ec0a7729d2cd None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:09.572328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6b51d7bf-5839-4000-9b1d-ec0a7729d2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:09.585840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b4f89-d965-4007-9bfe-15a75865a950 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:09.587487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b4f89-d965-4007-9bfe-15a75865a950 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:09.599316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51d7bf-5839-4000-9b1d-ec0a7729d2cd None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:09.606965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e63e29-d74f-455c-b94a-408e6d9189c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:09.607640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e63e29-d74f-455c-b94a-408e6d9189c4 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:09.608571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e63e29-d74f-455c-b94a-408e6d9189c4 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:09.609243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e63e29-d74f-455c-b94a-408e6d9189c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:09.615890 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1722/3400] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:09.621926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e63e29-d74f-455c-b94a-408e6d9189c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:09.626494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e63e29-d74f-455c-b94a-408e6d9189c4 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:09.642605 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1679/3401] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:12.675991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b4f89-d965-4007-9bfe-15a75865a950 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:12.695985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b4f89-d965-4007-9bfe-15a75865a950 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:12.706411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f0d9b-016c-445c-9c0c-18bbda228022 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:12.707059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f0d9b-016c-445c-9c0c-18bbda228022 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:12.707346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f0d9b-016c-445c-9c0c-18bbda228022 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:12.707627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f0d9b-016c-445c-9c0c-18bbda228022 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:12.713143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-600f0d9b-016c-445c-9c0c-18bbda228022 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:12.839398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f0d9b-016c-445c-9c0c-18bbda228022 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:12.841398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f0d9b-016c-445c-9c0c-18bbda228022 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:12.853771 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1723/3402] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:12.988132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e63e29-d74f-455c-b94a-408e6d9189c4 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:12.990370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-76e63e29-d74f-455c-b94a-408e6d9189c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:13.158925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e63e29-d74f-455c-b94a-408e6d9189c4 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:13.171803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45183e-f53e-4838-a6f7-81cadf81908d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:13.173745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45183e-f53e-4838-a6f7-81cadf81908d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:13.174101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45183e-f53e-4838-a6f7-81cadf81908d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:13.174842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45183e-f53e-4838-a6f7-81cadf81908d admin admin] PATH_INFO: `/v3/users/999db58fc81f4a9ab60bdd21674ec8c2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:13.178720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45183e-f53e-4838-a6f7-81cadf81908d admin admin] RBAC: Authorizing `identity:delete_user(user_id=999db58fc81f4a9ab60bdd21674ec8c2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:13.182438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45183e-f53e-4838-a6f7-81cadf81908d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:13.252808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:13.291732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e45183e-f53e-4838-a6f7-81cadf81908d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '999db58fc81f4a9ab60bdd21674ec8c2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:13.326723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e45183e-f53e-4838-a6f7-81cadf81908d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '999db58fc81f4a9ab60bdd21674ec8c2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:13.328225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e45183e-f53e-4838-a6f7-81cadf81908d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '999db58fc81f4a9ab60bdd21674ec8c2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:13.356876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e45183e-f53e-4838-a6f7-81cadf81908d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '999db58fc81f4a9ab60bdd21674ec8c2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:13.376440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e45183e-f53e-4838-a6f7-81cadf81908d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '999db58fc81f4a9ab60bdd21674ec8c2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:13.392493 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1680/3403] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:58:12 2022] DELETE /identity/v3/users/999db58fc81f4a9ab60bdd21674ec8c2 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:14.295101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f0d9b-016c-445c-9c0c-18bbda228022 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:14.308607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f0d9b-016c-445c-9c0c-18bbda228022 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:14.423440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:14.424184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:14.424476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:14.424814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901 admin admin] PATH_INFO: `/v3/projects/713d2a60596b4d2d9701779c1d414ed1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:14.426114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=713d2a60596b4d2d9701779c1d414ed1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:14.428359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:14.544699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:14.546329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '713d2a60596b4d2d9701779c1d414ed1', 'request_id': 'req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:14.574801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '713d2a60596b4d2d9701779c1d414ed1', 'request_id': 'req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:14.584480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '713d2a60596b4d2d9701779c1d414ed1', 'request_id': 'req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:14.603645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901 admin admin] The token cache is being invalidate because project 713d2a60596b4d2d9701779c1d414ed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:58:14.604248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:14.605352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9fc97e3b-ee5d-455e-86fe-3ce9efa89901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:14.607592 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1724/3404] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:58:14 2022] DELETE /identity/v3/projects/713d2a60596b4d2d9701779c1d414ed1 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:14.614989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b884eb-7da3-493b-b12c-0b9274d8b513 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:14.618219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b884eb-7da3-493b-b12c-0b9274d8b513 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:14.618806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b884eb-7da3-493b-b12c-0b9274d8b513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:14.680411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d6b884eb-7da3-493b-b12c-0b9274d8b513 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:14.682224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d6b884eb-7da3-493b-b12c-0b9274d8b513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:14.840198 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1681/3405] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:14.849624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709127e-043f-4334-b0a4-546a30f47d89 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:14.850150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709127e-043f-4334-b0a4-546a30f47d89 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:14.850612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709127e-043f-4334-b0a4-546a30f47d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:14.882086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b709127e-043f-4334-b0a4-546a30f47d89 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:14.883641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b709127e-043f-4334-b0a4-546a30f47d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:14.897613 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1725/3406] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:14.901661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b884eb-7da3-493b-b12c-0b9274d8b513 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:14.931461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b884eb-7da3-493b-b12c-0b9274d8b513 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:14.942661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94646ddf-76d9-4ab7-8c47-9ce673a7c25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:14.944805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94646ddf-76d9-4ab7-8c47-9ce673a7c25b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:14.946208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94646ddf-76d9-4ab7-8c47-9ce673a7c25b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:14.946806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94646ddf-76d9-4ab7-8c47-9ce673a7c25b admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:14.949090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94646ddf-76d9-4ab7-8c47-9ce673a7c25b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:58:14.949483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94646ddf-76d9-4ab7-8c47-9ce673a7c25b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:14.955244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94646ddf-76d9-4ab7-8c47-9ce673a7c25b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:14.989971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709127e-043f-4334-b0a4-546a30f47d89 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:15.001555 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1682/3407] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:58:15.001743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709127e-043f-4334-b0a4-546a30f47d89 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:15.014664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0988e88-e50f-4a13-9390-393c80984842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:15.015369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988e88-e50f-4a13-9390-393c80984842 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:15.016157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988e88-e50f-4a13-9390-393c80984842 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:15.016157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988e88-e50f-4a13-9390-393c80984842 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:15.024009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0988e88-e50f-4a13-9390-393c80984842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:15.026126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0988e88-e50f-4a13-9390-393c80984842 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:15.066197 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1726/3408] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:58:14 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:58:15.070541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94646ddf-76d9-4ab7-8c47-9ce673a7c25b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:15.082584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94646ddf-76d9-4ab7-8c47-9ce673a7c25b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:15.175415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181b6bd-a708-4f5b-abee-996b7aa10953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:15.176343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181b6bd-a708-4f5b-abee-996b7aa10953 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:15.176710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181b6bd-a708-4f5b-abee-996b7aa10953 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:15.177274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181b6bd-a708-4f5b-abee-996b7aa10953 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:15.178325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181b6bd-a708-4f5b-abee-996b7aa10953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:15.180498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181b6bd-a708-4f5b-abee-996b7aa10953 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:15.213909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58599b-faf5-4b56-b6c8-35095e9a4fb5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:15.214439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58599b-faf5-4b56-b6c8-35095e9a4fb5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:15.214761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58599b-faf5-4b56-b6c8-35095e9a4fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:15.220087 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1683/3409] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:58:15 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:58:15.221643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181b6bd-a708-4f5b-abee-996b7aa10953 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:15.246265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fd58599b-faf5-4b56-b6c8-35095e9a4fb5 None None] MFA Rules not processed for user `322d75aacb244323b79f7f0e2ac464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:15.248759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fd58599b-faf5-4b56-b6c8-35095e9a4fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:15.249261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181b6bd-a708-4f5b-abee-996b7aa10953 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:15.262768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da713c66-36aa-43ba-abe4-ee57f79dd21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:15.263407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da713c66-36aa-43ba-abe4-ee57f79dd21c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:15.265022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da713c66-36aa-43ba-abe4-ee57f79dd21c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:15.265305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da713c66-36aa-43ba-abe4-ee57f79dd21c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:15.265970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da713c66-36aa-43ba-abe4-ee57f79dd21c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:15.272810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da713c66-36aa-43ba-abe4-ee57f79dd21c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:15.299557 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1684/3410] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:15.306032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da713c66-36aa-43ba-abe4-ee57f79dd21c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:15.328645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da713c66-36aa-43ba-abe4-ee57f79dd21c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:15.334826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e675a-df22-42a0-97af-a8fdedef20f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:15.335457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e675a-df22-42a0-97af-a8fdedef20f9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:15.335746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e675a-df22-42a0-97af-a8fdedef20f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:15.336015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e675a-df22-42a0-97af-a8fdedef20f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:15.336665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e675a-df22-42a0-97af-a8fdedef20f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:15.342008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e675a-df22-42a0-97af-a8fdedef20f9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:15.363166 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1685/3411] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:15.364603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e675a-df22-42a0-97af-a8fdedef20f9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:15.365721 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1727/3412] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:15.370308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:15.377621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e675a-df22-42a0-97af-a8fdedef20f9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:15.384094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6b32d-1729-4605-9828-6451085dafc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:15.384804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6b32d-1729-4605-9828-6451085dafc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:15.385792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6b32d-1729-4605-9828-6451085dafc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:15.389853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6b32d-1729-4605-9828-6451085dafc8 admin admin] PATH_INFO: `/v3/projects/db2b0543b40d486dac0aaa8426afd3d6/users/fb89866055274ae1b0855f89e7b520cc/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:15.413266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6b32d-1729-4605-9828-6451085dafc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db2b0543b40d486dac0aaa8426afd3d6, user_id=fb89866055274ae1b0855f89e7b520cc, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:15.416326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6b32d-1729-4605-9828-6451085dafc8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:15.453939 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1686/3413] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:58:15 2022] PUT /identity/v3/projects/db2b0543b40d486dac0aaa8426afd3d6/users/fb89866055274ae1b0855f89e7b520cc/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:15.462111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd49cc-cfcc-486b-997f-ef6e7e41415d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:15.462685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd49cc-cfcc-486b-997f-ef6e7e41415d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:15.462964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd49cc-cfcc-486b-997f-ef6e7e41415d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:15.508501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-66bd49cc-cfcc-486b-997f-ef6e7e41415d None None] MFA Rules not processed for user `fb89866055274ae1b0855f89e7b520cc`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:15.511979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-66bd49cc-cfcc-486b-997f-ef6e7e41415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:15.637029 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1728/3414] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:15.652253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6b32d-1729-4605-9828-6451085dafc8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:15.654225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-58d6b32d-1729-4605-9828-6451085dafc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:15.751745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6b32d-1729-4605-9828-6451085dafc8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:15.756265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce62880-155a-42a0-97f8-7214ff0701c6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:15.757184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce62880-155a-42a0-97f8-7214ff0701c6 service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:15.757627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce62880-155a-42a0-97f8-7214ff0701c6 service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:15.758122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce62880-155a-42a0-97f8-7214ff0701c6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:15.768607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce62880-155a-42a0-97f8-7214ff0701c6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:15.770628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce62880-155a-42a0-97f8-7214ff0701c6 service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:15.782956 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1687/3415] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:16.072120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e202026-bb4b-4e9b-bd3a-f4fe46a0bb50 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:16.072421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e202026-bb4b-4e9b-bd3a-f4fe46a0bb50 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:16.072701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e202026-bb4b-4e9b-bd3a-f4fe46a0bb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:16.132116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6e202026-bb4b-4e9b-bd3a-f4fe46a0bb50 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:16.132306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6e202026-bb4b-4e9b-bd3a-f4fe46a0bb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:16.195307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:16.248701 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1729/3416] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 20:58:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:16.298381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce62880-155a-42a0-97f8-7214ff0701c6 service glance] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:16.299965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2ce62880-155a-42a0-97f8-7214ff0701c6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:16.433517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce62880-155a-42a0-97f8-7214ff0701c6 service glance] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:16.441056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6009c8-8c22-459f-a366-c5e3888d0ac7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:16.442370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6009c8-8c22-459f-a366-c5e3888d0ac7 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:16.442868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6009c8-8c22-459f-a366-c5e3888d0ac7 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:16.444212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6009c8-8c22-459f-a366-c5e3888d0ac7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:16.473968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e202026-bb4b-4e9b-bd3a-f4fe46a0bb50 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:16.481474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6009c8-8c22-459f-a366-c5e3888d0ac7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:16.482731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6e202026-bb4b-4e9b-bd3a-f4fe46a0bb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:16.486304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6009c8-8c22-459f-a366-c5e3888d0ac7 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:16.502869 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1688/3417] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:58:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 20:58:16.635668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e202026-bb4b-4e9b-bd3a-f4fe46a0bb50 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:16.649722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b9314-dcac-4ad2-a7b7-4477b2a55459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:16.650479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b9314-dcac-4ad2-a7b7-4477b2a55459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:16.653239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b9314-dcac-4ad2-a7b7-4477b2a55459 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:16.653631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b9314-dcac-4ad2-a7b7-4477b2a55459 admin admin] PATH_INFO: `/v3/users/f40d4376bec042e49ee8356e22baa812` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:16.655581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b9314-dcac-4ad2-a7b7-4477b2a55459 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f40d4376bec042e49ee8356e22baa812)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:16.664356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b9314-dcac-4ad2-a7b7-4477b2a55459 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:16.776646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-af1b9314-dcac-4ad2-a7b7-4477b2a55459 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f40d4376bec042e49ee8356e22baa812'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:16.793180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-af1b9314-dcac-4ad2-a7b7-4477b2a55459 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f40d4376bec042e49ee8356e22baa812'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:16.804720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-af1b9314-dcac-4ad2-a7b7-4477b2a55459 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f40d4376bec042e49ee8356e22baa812'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:16.805538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-af1b9314-dcac-4ad2-a7b7-4477b2a55459 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f40d4376bec042e49ee8356e22baa812'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:16.806300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-af1b9314-dcac-4ad2-a7b7-4477b2a55459 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f40d4376bec042e49ee8356e22baa812'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:16.831829 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1730/3418] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:58:16 2022] DELETE /identity/v3/users/f40d4376bec042e49ee8356e22baa812 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:17.108959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6009c8-8c22-459f-a366-c5e3888d0ac7 service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:17.126931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6009c8-8c22-459f-a366-c5e3888d0ac7 service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:17.364732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934cc45-acc4-4ff3-af07-dd135b782670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:17.366993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934cc45-acc4-4ff3-af07-dd135b782670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:17.368450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934cc45-acc4-4ff3-af07-dd135b782670 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:17.368945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934cc45-acc4-4ff3-af07-dd135b782670 admin admin] PATH_INFO: `/v3/projects/5500274fc1834f768e8659be97e4004c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:17.372982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934cc45-acc4-4ff3-af07-dd135b782670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5500274fc1834f768e8659be97e4004c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:17.376020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934cc45-acc4-4ff3-af07-dd135b782670 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:17.409623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:17.473388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7934cc45-acc4-4ff3-af07-dd135b782670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5500274fc1834f768e8659be97e4004c', 'request_id': 'req-7934cc45-acc4-4ff3-af07-dd135b782670'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:17.506422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7934cc45-acc4-4ff3-af07-dd135b782670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5500274fc1834f768e8659be97e4004c', 'request_id': 'req-7934cc45-acc4-4ff3-af07-dd135b782670'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:17.518346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7934cc45-acc4-4ff3-af07-dd135b782670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5500274fc1834f768e8659be97e4004c', 'request_id': 'req-7934cc45-acc4-4ff3-af07-dd135b782670'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:17.558094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7934cc45-acc4-4ff3-af07-dd135b782670 admin admin] The token cache is being invalidate because project 5500274fc1834f768e8659be97e4004c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:58:17.560143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7934cc45-acc4-4ff3-af07-dd135b782670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:17.560726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7934cc45-acc4-4ff3-af07-dd135b782670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:17.563699 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1689/3419] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:58:17 2022] DELETE /identity/v3/projects/5500274fc1834f768e8659be97e4004c => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:17.590532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f2b61-d56c-4922-ba29-ac900954a162 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:17.592105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f2b61-d56c-4922-ba29-ac900954a162 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:17.592566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f2b61-d56c-4922-ba29-ac900954a162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:17.636735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2e7f2b61-d56c-4922-ba29-ac900954a162 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:17.638010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2e7f2b61-d56c-4922-ba29-ac900954a162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:17.725007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:17.740852 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1731/3420] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:17.750155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091658d3-ac25-47c3-a9fc-c6ee2c371fe8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:17.751151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091658d3-ac25-47c3-a9fc-c6ee2c371fe8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:17.751437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091658d3-ac25-47c3-a9fc-c6ee2c371fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:17.778610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-091658d3-ac25-47c3-a9fc-c6ee2c371fe8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:17.781161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-091658d3-ac25-47c3-a9fc-c6ee2c371fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:17.795646 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1690/3421] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:17.797393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f2b61-d56c-4922-ba29-ac900954a162 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:17.815225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f2b61-d56c-4922-ba29-ac900954a162 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:17.828008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec57fa-6860-4a93-a111-2e3d47069e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:17.830367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec57fa-6860-4a93-a111-2e3d47069e4c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:17.832085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec57fa-6860-4a93-a111-2e3d47069e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:17.834674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec57fa-6860-4a93-a111-2e3d47069e4c admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:17.836348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec57fa-6860-4a93-a111-2e3d47069e4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:58:17.837082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec57fa-6860-4a93-a111-2e3d47069e4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:17.842833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec57fa-6860-4a93-a111-2e3d47069e4c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:17.881937 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1732/3422] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:58:17.885722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091658d3-ac25-47c3-a9fc-c6ee2c371fe8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:17.902679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091658d3-ac25-47c3-a9fc-c6ee2c371fe8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:17.909649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5e9fe-d385-4f77-b4e1-f13a5b9a15ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:17.910471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5e9fe-d385-4f77-b4e1-f13a5b9a15ef admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:17.910834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5e9fe-d385-4f77-b4e1-f13a5b9a15ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:17.911203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5e9fe-d385-4f77-b4e1-f13a5b9a15ef admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:17.912156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5e9fe-d385-4f77-b4e1-f13a5b9a15ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:17.914888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5e9fe-d385-4f77-b4e1-f13a5b9a15ef admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:17.965512 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1691/3423] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:58:17 2022] POST /identity/v3/projects => generated 391 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:58:17.969735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec57fa-6860-4a93-a111-2e3d47069e4c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:17.984241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec57fa-6860-4a93-a111-2e3d47069e4c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:18.119540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b5dc6-eaf6-450c-830a-70d1ae532985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:18.120397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b5dc6-eaf6-450c-830a-70d1ae532985 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:18.120826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b5dc6-eaf6-450c-830a-70d1ae532985 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:18.121292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b5dc6-eaf6-450c-830a-70d1ae532985 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:18.122109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b5dc6-eaf6-450c-830a-70d1ae532985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:18.124230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b5dc6-eaf6-450c-830a-70d1ae532985 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:18.159242 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1733/3424] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:58:17 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:58:18.164492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5e9fe-d385-4f77-b4e1-f13a5b9a15ef admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:18.193560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5e9fe-d385-4f77-b4e1-f13a5b9a15ef admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:18.206045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034df14-72d3-42a5-87fe-a3463124abd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:18.206838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034df14-72d3-42a5-87fe-a3463124abd6 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:18.207218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034df14-72d3-42a5-87fe-a3463124abd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:18.207615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034df14-72d3-42a5-87fe-a3463124abd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:18.208434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034df14-72d3-42a5-87fe-a3463124abd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:18.211258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034df14-72d3-42a5-87fe-a3463124abd6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:18.239130 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1692/3425] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:18.245679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b5dc6-eaf6-450c-830a-70d1ae532985 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:18.257540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b5dc6-eaf6-450c-830a-70d1ae532985 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:18.263535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a5e5d-ba7b-486b-9023-c2207e0a58a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:18.264784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a5e5d-ba7b-486b-9023-c2207e0a58a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:18.265806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a5e5d-ba7b-486b-9023-c2207e0a58a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:18.266327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a5e5d-ba7b-486b-9023-c2207e0a58a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:18.267214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a5e5d-ba7b-486b-9023-c2207e0a58a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:18.269258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a5e5d-ba7b-486b-9023-c2207e0a58a1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:18.288053 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1734/3426] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:18.290075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034df14-72d3-42a5-87fe-a3463124abd6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:18.305147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034df14-72d3-42a5-87fe-a3463124abd6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:18.312070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c387b-aa6c-481f-b37a-001a35f1a05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:18.314107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c387b-aa6c-481f-b37a-001a35f1a05c admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:18.314654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c387b-aa6c-481f-b37a-001a35f1a05c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:18.315190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c387b-aa6c-481f-b37a-001a35f1a05c admin admin] PATH_INFO: `/v3/projects/7262d6f38eee4c1184a6b471cf69b008/users/336a280254b44d5e9c99abbce6fb88d5/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:18.337676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c387b-aa6c-481f-b37a-001a35f1a05c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7262d6f38eee4c1184a6b471cf69b008, user_id=336a280254b44d5e9c99abbce6fb88d5, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:18.340984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c387b-aa6c-481f-b37a-001a35f1a05c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:18.358835 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1693/3427] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:58:18 2022] PUT /identity/v3/projects/7262d6f38eee4c1184a6b471cf69b008/users/336a280254b44d5e9c99abbce6fb88d5/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:58:18.364571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ba8d4-a95d-4478-bbe6-7003cd580488 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:18.364861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ba8d4-a95d-4478-bbe6-7003cd580488 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:18.365167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ba8d4-a95d-4478-bbe6-7003cd580488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:18.386219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-404ba8d4-a95d-4478-bbe6-7003cd580488 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:18.387466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-404ba8d4-a95d-4478-bbe6-7003cd580488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:18.460229 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1735/3428] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:18.476972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c387b-aa6c-481f-b37a-001a35f1a05c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:18.478863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7c5c387b-aa6c-481f-b37a-001a35f1a05c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:18.609120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c387b-aa6c-481f-b37a-001a35f1a05c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:18.615232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ead54a-e6eb-407e-ac25-7c1950001d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:18.615854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ead54a-e6eb-407e-ac25-7c1950001d23 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:18.616138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ead54a-e6eb-407e-ac25-7c1950001d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:18.616415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ead54a-e6eb-407e-ac25-7c1950001d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:18.628479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ead54a-e6eb-407e-ac25-7c1950001d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:18.630054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ead54a-e6eb-407e-ac25-7c1950001d23 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:18.641542 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1694/3429] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:58:23.185688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ba8d4-a95d-4478-bbe6-7003cd580488 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:23.187852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-404ba8d4-a95d-4478-bbe6-7003cd580488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:23.214134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ead54a-e6eb-407e-ac25-7c1950001d23 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:23.216498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-09ead54a-e6eb-407e-ac25-7c1950001d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:23.251661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ead54a-e6eb-407e-ac25-7c1950001d23 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:23.260192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:23.260848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:23.261159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:23.261442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] PATH_INFO: `/v3/users/7838d64ab35e4608b668e9f65d594d6e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:23.265643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7838d64ab35e4608b668e9f65d594d6e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:23.268230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:23.323355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:23.332849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:23.335080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ba8d4-a95d-4478-bbe6-7003cd580488 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:23.347323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9ed22-d45a-4cae-bc5f-36adbc18f4a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:23.348067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9ed22-d45a-4cae-bc5f-36adbc18f4a7 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:23.348513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9ed22-d45a-4cae-bc5f-36adbc18f4a7 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:23.349074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9ed22-d45a-4cae-bc5f-36adbc18f4a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:23.352524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6ea9ed22-d45a-4cae-bc5f-36adbc18f4a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:23.352800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7838d64ab35e4608b668e9f65d594d6e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:23.368874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7838d64ab35e4608b668e9f65d594d6e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:23.369859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7838d64ab35e4608b668e9f65d594d6e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:23.378585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7838d64ab35e4608b668e9f65d594d6e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:23.389407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7838d64ab35e4608b668e9f65d594d6e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:23.401871 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1695/3430] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:58:23 2022] DELETE /identity/v3/users/7838d64ab35e4608b668e9f65d594d6e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:23.513728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9ed22-d45a-4cae-bc5f-36adbc18f4a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:23.515493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9ed22-d45a-4cae-bc5f-36adbc18f4a7 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:23.541759 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1736/3431] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:58:23.708084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:23.710766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:23.823634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc3ded-77b6-4384-bb58-ec965caa5ee6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:23.830605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:23.831265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:23.832031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:23.832031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] PATH_INFO: `/v3/projects/090fc6c9de624a209a90631c0d22d48f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:23.833512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=090fc6c9de624a209a90631c0d22d48f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:23.835462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:23.907442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090fc6c9de624a209a90631c0d22d48f', 'request_id': 'req-e8c574ab-5f89-4be1-ae44-c045e9649637'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:23.918138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090fc6c9de624a209a90631c0d22d48f', 'request_id': 'req-e8c574ab-5f89-4be1-ae44-c045e9649637'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:23.926870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090fc6c9de624a209a90631c0d22d48f', 'request_id': 'req-e8c574ab-5f89-4be1-ae44-c045e9649637'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:23.939728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] The token cache is being invalidate because project 090fc6c9de624a209a90631c0d22d48f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:58:23.940147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:23.941104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:23.945464 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1696/3432] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:58:23 2022] DELETE /identity/v3/projects/090fc6c9de624a209a90631c0d22d48f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:23.946679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9ed22-d45a-4cae-bc5f-36adbc18f4a7 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:23.962155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9ed22-d45a-4cae-bc5f-36adbc18f4a7 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:24.074705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb4844-b18c-4722-96c8-2effb333cd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:24.075442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb4844-b18c-4722-96c8-2effb333cd79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:24.075729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb4844-b18c-4722-96c8-2effb333cd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:24.076084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb4844-b18c-4722-96c8-2effb333cd79 admin admin] PATH_INFO: `/v3/users/94d4c06fa26e400ead26ab08cd86bb43` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:24.077475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb4844-b18c-4722-96c8-2effb333cd79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94d4c06fa26e400ead26ab08cd86bb43)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:24.080005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb4844-b18c-4722-96c8-2effb333cd79 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:24.145484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58cb4844-b18c-4722-96c8-2effb333cd79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94d4c06fa26e400ead26ab08cd86bb43'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:24.161771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58cb4844-b18c-4722-96c8-2effb333cd79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94d4c06fa26e400ead26ab08cd86bb43'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:24.170201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58cb4844-b18c-4722-96c8-2effb333cd79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94d4c06fa26e400ead26ab08cd86bb43'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:24.171059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58cb4844-b18c-4722-96c8-2effb333cd79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94d4c06fa26e400ead26ab08cd86bb43'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:24.171828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58cb4844-b18c-4722-96c8-2effb333cd79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94d4c06fa26e400ead26ab08cd86bb43'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:24.189644 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1737/3433] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:58:23 2022] DELETE /identity/v3/users/94d4c06fa26e400ead26ab08cd86bb43 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:24.381666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:24.385981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:24.410130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c574ab-5f89-4be1-ae44-c045e9649637 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:24.494314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:24.529996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50353b9d-9df0-4681-89ba-2b907f780644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:24.530852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50353b9d-9df0-4681-89ba-2b907f780644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:24.531408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50353b9d-9df0-4681-89ba-2b907f780644 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:24.532396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50353b9d-9df0-4681-89ba-2b907f780644 admin admin] PATH_INFO: `/v3/projects/c9ae9fb6f6f8493abaf3857f81992925` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:24.534220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50353b9d-9df0-4681-89ba-2b907f780644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9ae9fb6f6f8493abaf3857f81992925)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:24.537150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50353b9d-9df0-4681-89ba-2b907f780644 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:24.554478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d016780-17ed-4419-89e7-8389eb987f5a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:24.554783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d016780-17ed-4419-89e7-8389eb987f5a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:24.556291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d016780-17ed-4419-89e7-8389eb987f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:24.587505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2d016780-17ed-4419-89e7-8389eb987f5a None None] MFA Rules not processed for user `e6970a72952e4d4a8f26e0f8ee23f364`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:24.613883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2d016780-17ed-4419-89e7-8389eb987f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:24.623094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:24.682884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-50353b9d-9df0-4681-89ba-2b907f780644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9ae9fb6f6f8493abaf3857f81992925', 'request_id': 'req-50353b9d-9df0-4681-89ba-2b907f780644'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:24.690793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-50353b9d-9df0-4681-89ba-2b907f780644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9ae9fb6f6f8493abaf3857f81992925', 'request_id': 'req-50353b9d-9df0-4681-89ba-2b907f780644'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:24.699306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-50353b9d-9df0-4681-89ba-2b907f780644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9ae9fb6f6f8493abaf3857f81992925', 'request_id': 'req-50353b9d-9df0-4681-89ba-2b907f780644'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:24.700973 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1738/3434] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:24.703157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83df7b4-c043-4bd2-8ff6-848fc35008e8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:24.703651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83df7b4-c043-4bd2-8ff6-848fc35008e8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:24.704369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83df7b4-c043-4bd2-8ff6-848fc35008e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:24.712264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-50353b9d-9df0-4681-89ba-2b907f780644 admin admin] The token cache is being invalidate because project c9ae9fb6f6f8493abaf3857f81992925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:58:24.712668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-50353b9d-9df0-4681-89ba-2b907f780644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:24.713476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-50353b9d-9df0-4681-89ba-2b907f780644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:24.721037 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1697/3435] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:58:24 2022] DELETE /identity/v3/projects/c9ae9fb6f6f8493abaf3857f81992925 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:24.722689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39356e-f7bf-496e-8170-842383dc67b1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:24.723150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39356e-f7bf-496e-8170-842383dc67b1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:24.723464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39356e-f7bf-496e-8170-842383dc67b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:24.752506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9c39356e-f7bf-496e-8170-842383dc67b1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:24.757733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9c39356e-f7bf-496e-8170-842383dc67b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:24.758421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f83df7b4-c043-4bd2-8ff6-848fc35008e8 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:24.759837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f83df7b4-c043-4bd2-8ff6-848fc35008e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:24.848540 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1739/3436] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 20:58:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:24.873940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83df7b4-c043-4bd2-8ff6-848fc35008e8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:24.876417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f83df7b4-c043-4bd2-8ff6-848fc35008e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:24.902332 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1698/3437] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:24.911157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2d449-af91-4112-bee5-00059ee9d91d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:24.911444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2d449-af91-4112-bee5-00059ee9d91d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:24.911737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2d449-af91-4112-bee5-00059ee9d91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:24.937217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c4f2d449-af91-4112-bee5-00059ee9d91d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:24.938431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c4f2d449-af91-4112-bee5-00059ee9d91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:24.954317 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1699/3438] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:24.956805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2d449-af91-4112-bee5-00059ee9d91d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:24.980672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2d449-af91-4112-bee5-00059ee9d91d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:24.982026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83df7b4-c043-4bd2-8ff6-848fc35008e8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:24.990573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ff454-dd58-4b6f-8e5a-d1405f990df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:24.990573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ff454-dd58-4b6f-8e5a-d1405f990df9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:24.990833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ff454-dd58-4b6f-8e5a-d1405f990df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:24.991095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ff454-dd58-4b6f-8e5a-d1405f990df9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:24.991874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ff454-dd58-4b6f-8e5a-d1405f990df9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:58:24.992189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ff454-dd58-4b6f-8e5a-d1405f990df9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:24.992663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b030fb4-fa2f-4c9c-931d-0a84ff80c336 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:24.993663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b030fb4-fa2f-4c9c-931d-0a84ff80c336 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:24.993957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ff454-dd58-4b6f-8e5a-d1405f990df9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:24.995208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b030fb4-fa2f-4c9c-931d-0a84ff80c336 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:24.997556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b030fb4-fa2f-4c9c-931d-0a84ff80c336 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:25.013832 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1700/3439] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:58:25.018358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ff454-dd58-4b6f-8e5a-d1405f990df9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:25.033844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b030fb4-fa2f-4c9c-931d-0a84ff80c336 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:25.034428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ff454-dd58-4b6f-8e5a-d1405f990df9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:25.035465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b030fb4-fa2f-4c9c-931d-0a84ff80c336 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:25.041572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc893e-59a7-4c09-b4f4-840ab4f5f0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:25.042211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc893e-59a7-4c09-b4f4-840ab4f5f0a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:25.042494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc893e-59a7-4c09-b4f4-840ab4f5f0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:25.042776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc893e-59a7-4c09-b4f4-840ab4f5f0a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:25.043420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc893e-59a7-4c09-b4f4-840ab4f5f0a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:25.046393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc893e-59a7-4c09-b4f4-840ab4f5f0a9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:25.046899 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1740/3440] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 20:58:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 20:58:25.115047 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1701/3441] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:58:25 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:58:25.124983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b030fb4-fa2f-4c9c-931d-0a84ff80c336 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:25.140144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b030fb4-fa2f-4c9c-931d-0a84ff80c336 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:25.229949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc893e-59a7-4c09-b4f4-840ab4f5f0a9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:25.232519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7dbc893e-59a7-4c09-b4f4-840ab4f5f0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:25.238681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:25.253979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8848f8-d0f6-485e-be9e-a57ad407408c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:25.254799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8848f8-d0f6-485e-be9e-a57ad407408c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:25.255159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8848f8-d0f6-485e-be9e-a57ad407408c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:25.255443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8848f8-d0f6-485e-be9e-a57ad407408c admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:25.256623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8848f8-d0f6-485e-be9e-a57ad407408c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:25.258698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8848f8-d0f6-485e-be9e-a57ad407408c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:25.289145 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1741/3442] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:58:25 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:58:25.290136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8848f8-d0f6-485e-be9e-a57ad407408c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:25.294970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:25.305548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8848f8-d0f6-485e-be9e-a57ad407408c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:25.316345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5e4bd-4f2c-46a4-a97d-5cce71caa524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:25.317174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5e4bd-4f2c-46a4-a97d-5cce71caa524 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:25.318042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5e4bd-4f2c-46a4-a97d-5cce71caa524 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:25.318042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5e4bd-4f2c-46a4-a97d-5cce71caa524 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:25.318507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5e4bd-4f2c-46a4-a97d-5cce71caa524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:25.320146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5e4bd-4f2c-46a4-a97d-5cce71caa524 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:25.338748 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1742/3443] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:25.339836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5e4bd-4f2c-46a4-a97d-5cce71caa524 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:25.352773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5e4bd-4f2c-46a4-a97d-5cce71caa524 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:25.359208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30203c-d410-4fe6-b8f1-2d37eb88d469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:25.360968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30203c-d410-4fe6-b8f1-2d37eb88d469 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:25.361455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30203c-d410-4fe6-b8f1-2d37eb88d469 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:25.362163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30203c-d410-4fe6-b8f1-2d37eb88d469 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:25.362732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc893e-59a7-4c09-b4f4-840ab4f5f0a9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:25.363311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30203c-d410-4fe6-b8f1-2d37eb88d469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:25.364928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30203c-d410-4fe6-b8f1-2d37eb88d469 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:25.369174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712d4a0-8e7b-461f-95fe-95fe30d3cb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:25.370506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712d4a0-8e7b-461f-95fe-95fe30d3cb3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:25.370964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712d4a0-8e7b-461f-95fe-95fe30d3cb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:25.371249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712d4a0-8e7b-461f-95fe-95fe30d3cb3b admin admin] PATH_INFO: `/v3/users/fb89866055274ae1b0855f89e7b520cc` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:25.372869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712d4a0-8e7b-461f-95fe-95fe30d3cb3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb89866055274ae1b0855f89e7b520cc)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:25.376579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712d4a0-8e7b-461f-95fe-95fe30d3cb3b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:25.397045 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1743/3444] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:25.400208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30203c-d410-4fe6-b8f1-2d37eb88d469 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:25.418691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30203c-d410-4fe6-b8f1-2d37eb88d469 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:25.425821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94bc4a-78b8-47c1-a9e5-19b212f7af98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:25.426629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94bc4a-78b8-47c1-a9e5-19b212f7af98 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:25.427351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94bc4a-78b8-47c1-a9e5-19b212f7af98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:25.427351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94bc4a-78b8-47c1-a9e5-19b212f7af98 admin admin] PATH_INFO: `/v3/projects/18ae2766f0674492909ede79ed1832d7/users/859a4503b4d84a4e8c3213e2de02492d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:25.445053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94bc4a-78b8-47c1-a9e5-19b212f7af98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ae2766f0674492909ede79ed1832d7, user_id=859a4503b4d84a4e8c3213e2de02492d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:25.447997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94bc4a-78b8-47c1-a9e5-19b212f7af98 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:25.451721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c712d4a0-8e7b-461f-95fe-95fe30d3cb3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb89866055274ae1b0855f89e7b520cc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:25.470174 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1744/3445] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:58:25 2022] PUT /identity/v3/projects/18ae2766f0674492909ede79ed1832d7/users/859a4503b4d84a4e8c3213e2de02492d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:25.471501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c712d4a0-8e7b-461f-95fe-95fe30d3cb3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb89866055274ae1b0855f89e7b520cc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:25.472028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f9048-20d7-4cfd-b72d-c0016243c54a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:25.472424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f9048-20d7-4cfd-b72d-c0016243c54a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:25.472811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f9048-20d7-4cfd-b72d-c0016243c54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:25.473407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c712d4a0-8e7b-461f-95fe-95fe30d3cb3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb89866055274ae1b0855f89e7b520cc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:25.492125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c712d4a0-8e7b-461f-95fe-95fe30d3cb3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb89866055274ae1b0855f89e7b520cc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:25.503114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c712d4a0-8e7b-461f-95fe-95fe30d3cb3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb89866055274ae1b0855f89e7b520cc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:25.521138 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1702/3446] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:58:25 2022] DELETE /identity/v3/users/fb89866055274ae1b0855f89e7b520cc => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:25.521138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5308f32-5940-47c2-b9e3-e4b4d845669e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:25.521138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5308f32-5940-47c2-b9e3-e4b4d845669e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:25.521138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5308f32-5940-47c2-b9e3-e4b4d845669e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:25.527295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-257f9048-20d7-4cfd-b72d-c0016243c54a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:25.529708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-257f9048-20d7-4cfd-b72d-c0016243c54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:25.555553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a5308f32-5940-47c2-b9e3-e4b4d845669e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:25.556759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a5308f32-5940-47c2-b9e3-e4b4d845669e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:25.635617 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1703/3447] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:25.644060 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1745/3448] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:25.661221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5308f32-5940-47c2-b9e3-e4b4d845669e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:25.664212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a5308f32-5940-47c2-b9e3-e4b4d845669e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:25.667174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f9048-20d7-4cfd-b72d-c0016243c54a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:25.674447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-257f9048-20d7-4cfd-b72d-c0016243c54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:25.835974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f9048-20d7-4cfd-b72d-c0016243c54a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:25.838611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5308f32-5940-47c2-b9e3-e4b4d845669e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:25.846610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082baaa9-039e-4462-86ac-b2531b9de7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:25.847239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082baaa9-039e-4462-86ac-b2531b9de7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:25.848255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082baaa9-039e-4462-86ac-b2531b9de7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:25.848545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082baaa9-039e-4462-86ac-b2531b9de7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:25.850065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587d8cb-7128-4129-b25b-66e238bdd0d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:25.850512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587d8cb-7128-4129-b25b-66e238bdd0d6 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:25.850938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587d8cb-7128-4129-b25b-66e238bdd0d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:25.851512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587d8cb-7128-4129-b25b-66e238bdd0d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:25.864575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:25.874267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082baaa9-039e-4462-86ac-b2531b9de7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:25.874267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082baaa9-039e-4462-86ac-b2531b9de7c9 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:25.876577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587d8cb-7128-4129-b25b-66e238bdd0d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:25.879262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587d8cb-7128-4129-b25b-66e238bdd0d6 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:25.894149 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1746/3449] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:25.897155 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1704/3450] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:26.530851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082baaa9-039e-4462-86ac-b2531b9de7c9 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:26.533088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-082baaa9-039e-4462-86ac-b2531b9de7c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:26.558069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082baaa9-039e-4462-86ac-b2531b9de7c9 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:26.570273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226294e-0c3f-4ab1-a01f-a317fc20dba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:26.570941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226294e-0c3f-4ab1-a01f-a317fc20dba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:26.571230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226294e-0c3f-4ab1-a01f-a317fc20dba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:26.571514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226294e-0c3f-4ab1-a01f-a317fc20dba0 admin admin] PATH_INFO: `/v3/projects/db2b0543b40d486dac0aaa8426afd3d6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:26.573634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b226294e-0c3f-4ab1-a01f-a317fc20dba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db2b0543b40d486dac0aaa8426afd3d6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:26.576214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b226294e-0c3f-4ab1-a01f-a317fc20dba0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:26.673855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:26.694542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b226294e-0c3f-4ab1-a01f-a317fc20dba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db2b0543b40d486dac0aaa8426afd3d6', 'request_id': 'req-b226294e-0c3f-4ab1-a01f-a317fc20dba0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:26.706849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b226294e-0c3f-4ab1-a01f-a317fc20dba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db2b0543b40d486dac0aaa8426afd3d6', 'request_id': 'req-b226294e-0c3f-4ab1-a01f-a317fc20dba0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:26.718207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b226294e-0c3f-4ab1-a01f-a317fc20dba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db2b0543b40d486dac0aaa8426afd3d6', 'request_id': 'req-b226294e-0c3f-4ab1-a01f-a317fc20dba0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:26.734972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b226294e-0c3f-4ab1-a01f-a317fc20dba0 admin admin] The token cache is being invalidate because project db2b0543b40d486dac0aaa8426afd3d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:58:26.735278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b226294e-0c3f-4ab1-a01f-a317fc20dba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:26.736262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b226294e-0c3f-4ab1-a01f-a317fc20dba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:26.745785 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1747/3451] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:58:26 2022] DELETE /identity/v3/projects/db2b0543b40d486dac0aaa8426afd3d6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:26.751350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963796b-1741-4eff-b3fb-610beeca71fc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:26.751800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963796b-1741-4eff-b3fb-610beeca71fc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:26.751958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963796b-1741-4eff-b3fb-610beeca71fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:26.793252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4963796b-1741-4eff-b3fb-610beeca71fc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:26.797517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4963796b-1741-4eff-b3fb-610beeca71fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:26.909105 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1705/3452] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:26.923395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e4425-c56f-48fe-b982-57a5d8e8c97d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:26.923688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e4425-c56f-48fe-b982-57a5d8e8c97d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:26.923971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e4425-c56f-48fe-b982-57a5d8e8c97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:26.948996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-180e4425-c56f-48fe-b982-57a5d8e8c97d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:26.950278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-180e4425-c56f-48fe-b982-57a5d8e8c97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:26.961024 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1748/3453] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:26.965351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963796b-1741-4eff-b3fb-610beeca71fc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:26.977301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963796b-1741-4eff-b3fb-610beeca71fc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:26.982528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69cf95-827a-4835-86eb-3ed68702ba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:26.983135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69cf95-827a-4835-86eb-3ed68702ba50 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:26.983425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69cf95-827a-4835-86eb-3ed68702ba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:26.983994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69cf95-827a-4835-86eb-3ed68702ba50 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:26.984657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69cf95-827a-4835-86eb-3ed68702ba50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:58:26.985372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69cf95-827a-4835-86eb-3ed68702ba50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:26.987245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69cf95-827a-4835-86eb-3ed68702ba50 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:27.002658 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1706/3454] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:58:27.006946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e4425-c56f-48fe-b982-57a5d8e8c97d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:27.019443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e4425-c56f-48fe-b982-57a5d8e8c97d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:27.025270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31397f-a350-43b0-9ee5-d04b385b36d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:27.026014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31397f-a350-43b0-9ee5-d04b385b36d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:27.026366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31397f-a350-43b0-9ee5-d04b385b36d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:27.026710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31397f-a350-43b0-9ee5-d04b385b36d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:27.028641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31397f-a350-43b0-9ee5-d04b385b36d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:27.031163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31397f-a350-43b0-9ee5-d04b385b36d6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:27.095890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69cf95-827a-4835-86eb-3ed68702ba50 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:27.096070 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1749/3455] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:58:27 2022] POST /identity/v3/projects => generated 381 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:58:27.107189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69cf95-827a-4835-86eb-3ed68702ba50 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:27.248841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753e5f4-d451-46d8-8038-06310c9fa883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:27.250307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753e5f4-d451-46d8-8038-06310c9fa883 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:27.251214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753e5f4-d451-46d8-8038-06310c9fa883 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:27.252165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753e5f4-d451-46d8-8038-06310c9fa883 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:27.252948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753e5f4-d451-46d8-8038-06310c9fa883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:27.254955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753e5f4-d451-46d8-8038-06310c9fa883 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:27.298868 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1707/3456] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:58:27 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:58:27.306124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31397f-a350-43b0-9ee5-d04b385b36d6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:27.320092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31397f-a350-43b0-9ee5-d04b385b36d6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:27.336833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf20a92-927b-41a7-9a22-402c90b9d1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:27.337541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf20a92-927b-41a7-9a22-402c90b9d1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:27.337881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf20a92-927b-41a7-9a22-402c90b9d1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:27.338169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf20a92-927b-41a7-9a22-402c90b9d1b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:27.340384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf20a92-927b-41a7-9a22-402c90b9d1b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:27.344928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf20a92-927b-41a7-9a22-402c90b9d1b2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:27.371310 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1750/3457] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:27.375258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753e5f4-d451-46d8-8038-06310c9fa883 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:27.391924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753e5f4-d451-46d8-8038-06310c9fa883 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:27.399424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73f2bf-f15f-4506-ae20-ed5f81dba8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:27.400805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73f2bf-f15f-4506-ae20-ed5f81dba8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:27.401401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73f2bf-f15f-4506-ae20-ed5f81dba8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:27.402061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73f2bf-f15f-4506-ae20-ed5f81dba8f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:27.403904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73f2bf-f15f-4506-ae20-ed5f81dba8f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:27.407008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73f2bf-f15f-4506-ae20-ed5f81dba8f1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:27.429596 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1708/3458] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:27.434539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf20a92-927b-41a7-9a22-402c90b9d1b2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:27.451858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf20a92-927b-41a7-9a22-402c90b9d1b2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:27.467122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0a02d-1f2d-425a-afac-bec3194f5820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:27.482922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0a02d-1f2d-425a-afac-bec3194f5820 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:27.483288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0a02d-1f2d-425a-afac-bec3194f5820 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:27.483591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0a02d-1f2d-425a-afac-bec3194f5820 admin admin] PATH_INFO: `/v3/projects/6047f3a5056d43549f602b273ad8eab3/users/fba464c031484398bf568f091e5018a4/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:27.521747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0a02d-1f2d-425a-afac-bec3194f5820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6047f3a5056d43549f602b273ad8eab3, user_id=fba464c031484398bf568f091e5018a4, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:27.526994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0a02d-1f2d-425a-afac-bec3194f5820 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:27.548308 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1751/3459] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:58:27 2022] PUT /identity/v3/projects/6047f3a5056d43549f602b273ad8eab3/users/fba464c031484398bf568f091e5018a4/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:58:27.556321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96346aa2-69a8-4bbb-80d1-8e97487e1cef None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:27.556721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96346aa2-69a8-4bbb-80d1-8e97487e1cef None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:27.557122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96346aa2-69a8-4bbb-80d1-8e97487e1cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:27.619372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-96346aa2-69a8-4bbb-80d1-8e97487e1cef None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:27.621386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-96346aa2-69a8-4bbb-80d1-8e97487e1cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:27.708349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:27.720506 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1709/3460] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:27.736190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0a02d-1f2d-425a-afac-bec3194f5820 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:27.738130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6ce0a02d-1f2d-425a-afac-bec3194f5820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:27.829991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0a02d-1f2d-425a-afac-bec3194f5820 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:27.839373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3fdc7-a2f1-4931-b097-ed9910fd8283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:27.841929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3fdc7-a2f1-4931-b097-ed9910fd8283 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:27.842604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3fdc7-a2f1-4931-b097-ed9910fd8283 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:27.842895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3fdc7-a2f1-4931-b097-ed9910fd8283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:27.867402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3fdc7-a2f1-4931-b097-ed9910fd8283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:27.869227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3fdc7-a2f1-4931-b097-ed9910fd8283 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:27.880460 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1752/3461] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:30.172940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0650be-845d-4d19-927e-03b351054cb4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:30.173466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0650be-845d-4d19-927e-03b351054cb4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:30.173466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0650be-845d-4d19-927e-03b351054cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:30.231987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4d0650be-845d-4d19-927e-03b351054cb4 None None] MFA Rules not processed for user `336a280254b44d5e9c99abbce6fb88d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:30.234122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4d0650be-845d-4d19-927e-03b351054cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:30.379634 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1710/3462] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:30.391404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759762a-b443-4624-9180-84781367036e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:30.391798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759762a-b443-4624-9180-84781367036e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:30.391966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759762a-b443-4624-9180-84781367036e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:30.421668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7759762a-b443-4624-9180-84781367036e None None] MFA Rules not processed for user `336a280254b44d5e9c99abbce6fb88d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:30.424288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7759762a-b443-4624-9180-84781367036e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:30.447336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0650be-845d-4d19-927e-03b351054cb4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:30.449010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4d0650be-845d-4d19-927e-03b351054cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:30.453153 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1753/3463] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:30.627051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0650be-845d-4d19-927e-03b351054cb4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:30.642647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdee95-7148-4476-8cc2-7a0e45b27501 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:30.647946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdee95-7148-4476-8cc2-7a0e45b27501 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:30.648304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdee95-7148-4476-8cc2-7a0e45b27501 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:30.648594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdee95-7148-4476-8cc2-7a0e45b27501 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:30.673187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdee95-7148-4476-8cc2-7a0e45b27501 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:30.675103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdee95-7148-4476-8cc2-7a0e45b27501 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:30.724874 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1711/3464] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:58:40.408758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=67,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:40.687685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=66 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:42.449971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2931e9-ba95-4f69-97e0-6defb5508837 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:42.450433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2931e9-ba95-4f69-97e0-6defb5508837 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:42.450708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2931e9-ba95-4f69-97e0-6defb5508837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:42.564176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ae2931e9-ba95-4f69-97e0-6defb5508837 None None] MFA Rules not processed for user `859a4503b4d84a4e8c3213e2de02492d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:42.566041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ae2931e9-ba95-4f69-97e0-6defb5508837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:42.733633 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1754/3465] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:42.744726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba862d-b402-460d-887d-d47b1d4094ce None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:42.745304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba862d-b402-460d-887d-d47b1d4094ce None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:42.745400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba862d-b402-460d-887d-d47b1d4094ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:42.791477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d7ba862d-b402-460d-887d-d47b1d4094ce None None] MFA Rules not processed for user `859a4503b4d84a4e8c3213e2de02492d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:42.792662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d7ba862d-b402-460d-887d-d47b1d4094ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:42.803128 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1712/3466] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:42.897173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2931e9-ba95-4f69-97e0-6defb5508837 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:42.900730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ae2931e9-ba95-4f69-97e0-6defb5508837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:42.936234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2931e9-ba95-4f69-97e0-6defb5508837 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:42.948154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cf5a5-19b9-4c93-9e0e-01676b0a16df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:42.950245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cf5a5-19b9-4c93-9e0e-01676b0a16df service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:42.950751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cf5a5-19b9-4c93-9e0e-01676b0a16df service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:42.951256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cf5a5-19b9-4c93-9e0e-01676b0a16df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:42.963832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cf5a5-19b9-4c93-9e0e-01676b0a16df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:42.966910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cf5a5-19b9-4c93-9e0e-01676b0a16df service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:42.995215 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1755/3467] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:46.509572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aea04-0053-40a8-88b2-12e3ab99a7c2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:46.510726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aea04-0053-40a8-88b2-12e3ab99a7c2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:46.511236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aea04-0053-40a8-88b2-12e3ab99a7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:46.568679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cc0aea04-0053-40a8-88b2-12e3ab99a7c2 None None] MFA Rules not processed for user `fba464c031484398bf568f091e5018a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:46.570243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cc0aea04-0053-40a8-88b2-12e3ab99a7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:46.729537 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1713/3468] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:46.741660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3381-2ff3-4e7f-8d80-7189fa156d2e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:46.742194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3381-2ff3-4e7f-8d80-7189fa156d2e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:46.742616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3381-2ff3-4e7f-8d80-7189fa156d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:46.783893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e1dd3381-2ff3-4e7f-8d80-7189fa156d2e None None] MFA Rules not processed for user `fba464c031484398bf568f091e5018a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:46.786695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e1dd3381-2ff3-4e7f-8d80-7189fa156d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:46.804691 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1756/3469] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:58:46.945644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aea04-0053-40a8-88b2-12e3ab99a7c2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:46.973482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aea04-0053-40a8-88b2-12e3ab99a7c2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:46.979669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363f836-3eb6-45e7-9e31-9738a6c10704 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:46.980418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363f836-3eb6-45e7-9e31-9738a6c10704 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:46.980769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363f836-3eb6-45e7-9e31-9738a6c10704 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:46.981873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363f836-3eb6-45e7-9e31-9738a6c10704 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:46.991766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363f836-3eb6-45e7-9e31-9738a6c10704 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:46.994130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363f836-3eb6-45e7-9e31-9738a6c10704 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:47.006822 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1714/3470] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:47.402398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3381-2ff3-4e7f-8d80-7189fa156d2e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:47.406370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e1dd3381-2ff3-4e7f-8d80-7189fa156d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:47.422671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3381-2ff3-4e7f-8d80-7189fa156d2e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:47.431451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84bd56-0658-419f-a9e0-2be144e73ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:47.432317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84bd56-0658-419f-a9e0-2be144e73ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:47.432623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84bd56-0658-419f-a9e0-2be144e73ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:47.433203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84bd56-0658-419f-a9e0-2be144e73ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:47.440099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2d84bd56-0658-419f-a9e0-2be144e73ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:47.461739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84bd56-0658-419f-a9e0-2be144e73ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:47.465643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84bd56-0658-419f-a9e0-2be144e73ba6 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:47.519634 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1757/3471] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:47.932621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363f836-3eb6-45e7-9e31-9738a6c10704 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:47.949653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363f836-3eb6-45e7-9e31-9738a6c10704 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:47.954144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82316c54-02e6-4b21-a8e6-940a1155f033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:47.955054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82316c54-02e6-4b21-a8e6-940a1155f033 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:47.955566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82316c54-02e6-4b21-a8e6-940a1155f033 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:47.956090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82316c54-02e6-4b21-a8e6-940a1155f033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:47.958525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-82316c54-02e6-4b21-a8e6-940a1155f033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:47.976939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82316c54-02e6-4b21-a8e6-940a1155f033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:47.979966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82316c54-02e6-4b21-a8e6-940a1155f033 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:47.997321 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1715/3472] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:53.660097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84bd56-0658-419f-a9e0-2be144e73ba6 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:53.660097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2d84bd56-0658-419f-a9e0-2be144e73ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:53.682725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84bd56-0658-419f-a9e0-2be144e73ba6 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:53.689952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:53.690654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:53.690946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:53.691230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] PATH_INFO: `/v3/users/9baf371d1c0942eda0b4ea9466b03b79` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:53.692566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] RBAC: Authorizing `identity:delete_user(user_id=9baf371d1c0942eda0b4ea9466b03b79)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:53.694745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:53.751104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9baf371d1c0942eda0b4ea9466b03b79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:53.766585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9baf371d1c0942eda0b4ea9466b03b79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:53.783137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9baf371d1c0942eda0b4ea9466b03b79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:53.784226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9baf371d1c0942eda0b4ea9466b03b79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:53.784870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9baf371d1c0942eda0b4ea9466b03b79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:53.805307 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1758/3473] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:58:53 2022] DELETE /identity/v3/users/9baf371d1c0942eda0b4ea9466b03b79 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:53.810161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77c031-f49c-497a-b17e-c7cd4534cd7b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:53.810663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77c031-f49c-497a-b17e-c7cd4534cd7b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:53.811039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77c031-f49c-497a-b17e-c7cd4534cd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:53.834133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0f77c031-f49c-497a-b17e-c7cd4534cd7b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:53.836289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0f77c031-f49c-497a-b17e-c7cd4534cd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:53.927418 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1716/3474] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:53.939400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:53.941282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:53.994448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:54.044814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec32a5-3f4e-49d9-8ed6-9d731df416af admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:54.054522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d38a3d-4ce8-4d49-b03c-3e596e2bc4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:54.055380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d38a3d-4ce8-4d49-b03c-3e596e2bc4aa service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:54.055715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d38a3d-4ce8-4d49-b03c-3e596e2bc4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:54.056017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d38a3d-4ce8-4d49-b03c-3e596e2bc4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:54.068104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d38a3d-4ce8-4d49-b03c-3e596e2bc4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:54.070284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d38a3d-4ce8-4d49-b03c-3e596e2bc4aa service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:54.082118 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1759/3475] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:54.514158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77c031-f49c-497a-b17e-c7cd4534cd7b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:54.516705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0f77c031-f49c-497a-b17e-c7cd4534cd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:54.540806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77c031-f49c-497a-b17e-c7cd4534cd7b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:54.553733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:54.554402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:54.554689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:54.554967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] PATH_INFO: `/v3/projects/0c65af796be74447b2bbba4e8b9d4f35` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:54.556359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c65af796be74447b2bbba4e8b9d4f35)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:54.558404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:54.622176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c65af796be74447b2bbba4e8b9d4f35', 'request_id': 'req-ecc147d4-5219-433b-9160-8d3fc003d85b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:54.626294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:54.636972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c65af796be74447b2bbba4e8b9d4f35', 'request_id': 'req-ecc147d4-5219-433b-9160-8d3fc003d85b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:54.645207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c65af796be74447b2bbba4e8b9d4f35', 'request_id': 'req-ecc147d4-5219-433b-9160-8d3fc003d85b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:54.657502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] The token cache is being invalidate because project 0c65af796be74447b2bbba4e8b9d4f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:58:54.657976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:54.658668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:54.660822 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1717/3476] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:58:54 2022] DELETE /identity/v3/projects/0c65af796be74447b2bbba4e8b9d4f35 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:54.664633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d38a3d-4ce8-4d49-b03c-3e596e2bc4aa service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:54.676461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d38a3d-4ce8-4d49-b03c-3e596e2bc4aa service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:54.777413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4e883-4cf1-433b-add1-927ae7633579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:54.778448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4e883-4cf1-433b-add1-927ae7633579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:54.778986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4e883-4cf1-433b-add1-927ae7633579 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:54.779470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4e883-4cf1-433b-add1-927ae7633579 admin admin] PATH_INFO: `/v3/users/aafa2cc6f37847e78154294a7937f3a4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:54.781291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4e883-4cf1-433b-add1-927ae7633579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aafa2cc6f37847e78154294a7937f3a4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:54.786656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4e883-4cf1-433b-add1-927ae7633579 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:54.852696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26c4e883-4cf1-433b-add1-927ae7633579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aafa2cc6f37847e78154294a7937f3a4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:54.869586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26c4e883-4cf1-433b-add1-927ae7633579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aafa2cc6f37847e78154294a7937f3a4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:54.879528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26c4e883-4cf1-433b-add1-927ae7633579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aafa2cc6f37847e78154294a7937f3a4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:54.880249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26c4e883-4cf1-433b-add1-927ae7633579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aafa2cc6f37847e78154294a7937f3a4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:54.881143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26c4e883-4cf1-433b-add1-927ae7633579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aafa2cc6f37847e78154294a7937f3a4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:54.897178 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1760/3477] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:58:54 2022] DELETE /identity/v3/users/aafa2cc6f37847e78154294a7937f3a4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:55.697698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:55.699464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:55.722754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc147d4-5219-433b-9160-8d3fc003d85b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:55.820629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac96841-8327-4ef7-bf15-fc7fac8f1893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:55.821736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac96841-8327-4ef7-bf15-fc7fac8f1893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:55.822404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac96841-8327-4ef7-bf15-fc7fac8f1893 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:55.822898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac96841-8327-4ef7-bf15-fc7fac8f1893 admin admin] PATH_INFO: `/v3/projects/d2293bd0d1f748b68b7cc271094ecf1e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:55.824930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac96841-8327-4ef7-bf15-fc7fac8f1893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2293bd0d1f748b68b7cc271094ecf1e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:55.827365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac96841-8327-4ef7-bf15-fc7fac8f1893 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:55.916428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1ac96841-8327-4ef7-bf15-fc7fac8f1893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2293bd0d1f748b68b7cc271094ecf1e', 'request_id': 'req-1ac96841-8327-4ef7-bf15-fc7fac8f1893'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:55.930064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1ac96841-8327-4ef7-bf15-fc7fac8f1893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2293bd0d1f748b68b7cc271094ecf1e', 'request_id': 'req-1ac96841-8327-4ef7-bf15-fc7fac8f1893'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:55.940968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1ac96841-8327-4ef7-bf15-fc7fac8f1893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2293bd0d1f748b68b7cc271094ecf1e', 'request_id': 'req-1ac96841-8327-4ef7-bf15-fc7fac8f1893'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:55.955269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1ac96841-8327-4ef7-bf15-fc7fac8f1893 admin admin] The token cache is being invalidate because project d2293bd0d1f748b68b7cc271094ecf1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:58:55.955579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1ac96841-8327-4ef7-bf15-fc7fac8f1893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:55.956800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1ac96841-8327-4ef7-bf15-fc7fac8f1893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:55.961789 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1718/3478] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:58:55 2022] DELETE /identity/v3/projects/d2293bd0d1f748b68b7cc271094ecf1e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:55.963756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc63775-4ed1-4368-aab4-56a03f53eb6e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:55.964224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc63775-4ed1-4368-aab4-56a03f53eb6e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:55.964341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc63775-4ed1-4368-aab4-56a03f53eb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:55.989332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2fc63775-4ed1-4368-aab4-56a03f53eb6e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:55.991241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2fc63775-4ed1-4368-aab4-56a03f53eb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:56.043495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:56.066545 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1761/3479] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:56.074718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb9f77-c3a2-4929-888d-5f0bfa963871 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:56.075261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb9f77-c3a2-4929-888d-5f0bfa963871 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:56.075761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb9f77-c3a2-4929-888d-5f0bfa963871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:56.105895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2cfb9f77-c3a2-4929-888d-5f0bfa963871 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:56.107372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2cfb9f77-c3a2-4929-888d-5f0bfa963871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:56.117795 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1719/3480] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:58:56.122056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc63775-4ed1-4368-aab4-56a03f53eb6e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:56.134955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc63775-4ed1-4368-aab4-56a03f53eb6e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:56.141042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e1d37-e0b4-4e79-90fd-d0ffbadcda0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:56.142223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e1d37-e0b4-4e79-90fd-d0ffbadcda0a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:56.142936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e1d37-e0b4-4e79-90fd-d0ffbadcda0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:56.143469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e1d37-e0b4-4e79-90fd-d0ffbadcda0a admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:56.144294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e1d37-e0b4-4e79-90fd-d0ffbadcda0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:58:56.144673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e1d37-e0b4-4e79-90fd-d0ffbadcda0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:56.146499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e1d37-e0b4-4e79-90fd-d0ffbadcda0a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:56.163697 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1762/3481] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:58:56.167892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb9f77-c3a2-4929-888d-5f0bfa963871 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:56.179767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb9f77-c3a2-4929-888d-5f0bfa963871 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:56.186886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685a93d-00a8-489c-bda4-1fc696ac212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:56.188500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685a93d-00a8-489c-bda4-1fc696ac212e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:56.188787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685a93d-00a8-489c-bda4-1fc696ac212e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:56.189166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685a93d-00a8-489c-bda4-1fc696ac212e admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:56.189809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685a93d-00a8-489c-bda4-1fc696ac212e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:56.200966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685a93d-00a8-489c-bda4-1fc696ac212e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:56.264972 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1720/3482] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:58:56 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:58:56.269513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e1d37-e0b4-4e79-90fd-d0ffbadcda0a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:56.280765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e1d37-e0b4-4e79-90fd-d0ffbadcda0a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:56.401632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516bafa-e14e-4686-948a-7159abd32191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:56.402507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516bafa-e14e-4686-948a-7159abd32191 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:56.402962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516bafa-e14e-4686-948a-7159abd32191 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:56.403247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516bafa-e14e-4686-948a-7159abd32191 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:56.403888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516bafa-e14e-4686-948a-7159abd32191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:56.406049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516bafa-e14e-4686-948a-7159abd32191 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:56.450408 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1763/3483] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:58:56 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:58:56.453960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685a93d-00a8-489c-bda4-1fc696ac212e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:56.468341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685a93d-00a8-489c-bda4-1fc696ac212e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:56.483504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13d3ce-1b74-4c9a-9830-7c237f089999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:56.485490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13d3ce-1b74-4c9a-9830-7c237f089999 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:56.485490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13d3ce-1b74-4c9a-9830-7c237f089999 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:56.485490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13d3ce-1b74-4c9a-9830-7c237f089999 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:56.485906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13d3ce-1b74-4c9a-9830-7c237f089999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:56.487485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13d3ce-1b74-4c9a-9830-7c237f089999 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:56.507355 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1721/3484] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:56.509490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516bafa-e14e-4686-948a-7159abd32191 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:56.522508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516bafa-e14e-4686-948a-7159abd32191 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:56.531177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13525ce-fc4c-43e7-8c11-0b4a5f60713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:56.532050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13525ce-fc4c-43e7-8c11-0b4a5f60713c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:56.532462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13525ce-fc4c-43e7-8c11-0b4a5f60713c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:56.532889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13525ce-fc4c-43e7-8c11-0b4a5f60713c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:56.533723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13525ce-fc4c-43e7-8c11-0b4a5f60713c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:56.535518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13525ce-fc4c-43e7-8c11-0b4a5f60713c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:56.554286 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1764/3485] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:56.558523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13d3ce-1b74-4c9a-9830-7c237f089999 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:56.575111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13d3ce-1b74-4c9a-9830-7c237f089999 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:56.597386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a6418-c418-4495-87ba-e35f2fe8aec8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:56.597611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a6418-c418-4495-87ba-e35f2fe8aec8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:56.597913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a6418-c418-4495-87ba-e35f2fe8aec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:56.598023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ebbba-cf6d-4aa1-a6d1-022f90fad713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:56.598766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ebbba-cf6d-4aa1-a6d1-022f90fad713 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:56.599146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ebbba-cf6d-4aa1-a6d1-022f90fad713 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:56.599602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ebbba-cf6d-4aa1-a6d1-022f90fad713 admin admin] PATH_INFO: `/v3/projects/aa0d61268a4f4ea58d07ce52ad0fd477/users/de9c44038e5449a1a231de449134264e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:56.620453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5d8a6418-c418-4495-87ba-e35f2fe8aec8 None None] MFA Rules not processed for user `e6970a72952e4d4a8f26e0f8ee23f364`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:56.621765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5d8a6418-c418-4495-87ba-e35f2fe8aec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:56.629173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ebbba-cf6d-4aa1-a6d1-022f90fad713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0d61268a4f4ea58d07ce52ad0fd477, user_id=de9c44038e5449a1a231de449134264e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:56.632413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ebbba-cf6d-4aa1-a6d1-022f90fad713 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:56.651176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:56.663364 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1722/3486] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:58:56 2022] PUT /identity/v3/projects/aa0d61268a4f4ea58d07ce52ad0fd477/users/de9c44038e5449a1a231de449134264e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:56.674179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded9b91-1f0b-425d-88f8-690f78d7569e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:56.674840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded9b91-1f0b-425d-88f8-690f78d7569e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:56.675325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded9b91-1f0b-425d-88f8-690f78d7569e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:56.711057 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1765/3487] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:58:56.740752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7ded9b91-1f0b-425d-88f8-690f78d7569e None None] MFA Rules not processed for user `de9c44038e5449a1a231de449134264e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:56.742365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7ded9b91-1f0b-425d-88f8-690f78d7569e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:56.838036 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1723/3488] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:56.842821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a6418-c418-4495-87ba-e35f2fe8aec8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:56.857258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a6418-c418-4495-87ba-e35f2fe8aec8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:56.932394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194cdd1-701f-4f51-a39a-b90b618d82c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:56.933177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194cdd1-701f-4f51-a39a-b90b618d82c6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:56.933536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194cdd1-701f-4f51-a39a-b90b618d82c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:56.933888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194cdd1-701f-4f51-a39a-b90b618d82c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:56.934804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5194cdd1-701f-4f51-a39a-b90b618d82c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:56.936983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5194cdd1-701f-4f51-a39a-b90b618d82c6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:56.943273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:56.959298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded9b91-1f0b-425d-88f8-690f78d7569e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:56.961130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7ded9b91-1f0b-425d-88f8-690f78d7569e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:56.983182 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1766/3489] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:58:56 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:58:56.987765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194cdd1-701f-4f51-a39a-b90b618d82c6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:57.038364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded9b91-1f0b-425d-88f8-690f78d7569e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:57.089647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194cdd1-701f-4f51-a39a-b90b618d82c6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:57.096175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819df1b-7089-4f4d-9fb5-d83a8d187136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:57.098183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819df1b-7089-4f4d-9fb5-d83a8d187136 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:57.098577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819df1b-7089-4f4d-9fb5-d83a8d187136 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:57.098959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819df1b-7089-4f4d-9fb5-d83a8d187136 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:57.099784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819df1b-7089-4f4d-9fb5-d83a8d187136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:57.102657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819df1b-7089-4f4d-9fb5-d83a8d187136 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:57.118976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029baf8c-08c8-4463-bae7-8370ad1afd05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:57.119970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029baf8c-08c8-4463-bae7-8370ad1afd05 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:57.120560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029baf8c-08c8-4463-bae7-8370ad1afd05 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:57.121059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029baf8c-08c8-4463-bae7-8370ad1afd05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:57.125320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-029baf8c-08c8-4463-bae7-8370ad1afd05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:57.147546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029baf8c-08c8-4463-bae7-8370ad1afd05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:57.149702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029baf8c-08c8-4463-bae7-8370ad1afd05 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:57.164807 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1724/3490] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:57.165491 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1767/3491] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:58:56 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:58:57.169533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029baf8c-08c8-4463-bae7-8370ad1afd05 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:57.185275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029baf8c-08c8-4463-bae7-8370ad1afd05 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:57.196448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53c220-9fbe-4c18-a5b6-45c49f7646d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:57.198915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53c220-9fbe-4c18-a5b6-45c49f7646d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:57.199361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53c220-9fbe-4c18-a5b6-45c49f7646d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:57.199807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53c220-9fbe-4c18-a5b6-45c49f7646d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:57.200538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53c220-9fbe-4c18-a5b6-45c49f7646d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:57.204051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53c220-9fbe-4c18-a5b6-45c49f7646d0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:57.217176 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1725/3492] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:58:57.221838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819df1b-7089-4f4d-9fb5-d83a8d187136 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:57.234123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819df1b-7089-4f4d-9fb5-d83a8d187136 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:57.247458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0574c-d1a6-4286-a533-d48468e9c899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:57.248253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0574c-d1a6-4286-a533-d48468e9c899 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:57.248615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0574c-d1a6-4286-a533-d48468e9c899 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:57.248920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0574c-d1a6-4286-a533-d48468e9c899 admin admin] PATH_INFO: `/v3/projects/5247d3af30d74b339d6986bf6e731e6e/users/ca42f2606e8446579f92f9a538460bd2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:57.268301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0574c-d1a6-4286-a533-d48468e9c899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5247d3af30d74b339d6986bf6e731e6e, user_id=ca42f2606e8446579f92f9a538460bd2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:57.272337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0574c-d1a6-4286-a533-d48468e9c899 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:57.290754 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1768/3493] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:58:57 2022] PUT /identity/v3/projects/5247d3af30d74b339d6986bf6e731e6e/users/ca42f2606e8446579f92f9a538460bd2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:57.294408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53c220-9fbe-4c18-a5b6-45c49f7646d0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:57.307377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53c220-9fbe-4c18-a5b6-45c49f7646d0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:57.349119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:57.382412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315009c2-eb0f-4879-be34-3e1d97f2915c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:57.383488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315009c2-eb0f-4879-be34-3e1d97f2915c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:57.383788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315009c2-eb0f-4879-be34-3e1d97f2915c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:57.384209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315009c2-eb0f-4879-be34-3e1d97f2915c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:57.384911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315009c2-eb0f-4879-be34-3e1d97f2915c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:57.387206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315009c2-eb0f-4879-be34-3e1d97f2915c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:57.401939 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1726/3494] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:58:57.405837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0574c-d1a6-4286-a533-d48468e9c899 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:57.417709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0574c-d1a6-4286-a533-d48468e9c899 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:57.422685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d434277-5442-4b40-bc84-e5b8d4dc22fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:57.423345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d434277-5442-4b40-bc84-e5b8d4dc22fd admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:57.423629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d434277-5442-4b40-bc84-e5b8d4dc22fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:57.424348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d434277-5442-4b40-bc84-e5b8d4dc22fd admin admin] PATH_INFO: `/v3/projects/5247d3af30d74b339d6986bf6e731e6e/users/ca42f2606e8446579f92f9a538460bd2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:57.426002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d434277-5442-4b40-bc84-e5b8d4dc22fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5247d3af30d74b339d6986bf6e731e6e, user_id=ca42f2606e8446579f92f9a538460bd2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:57.428701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d434277-5442-4b40-bc84-e5b8d4dc22fd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:57.439577 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1769/3495] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:58:57 2022] PUT /identity/v3/projects/5247d3af30d74b339d6986bf6e731e6e/users/ca42f2606e8446579f92f9a538460bd2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:57.450823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03339c-09bf-480a-9694-e4454cda8399 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:57.451358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03339c-09bf-480a-9694-e4454cda8399 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:57.451744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03339c-09bf-480a-9694-e4454cda8399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:57.483934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1f03339c-09bf-480a-9694-e4454cda8399 None None] MFA Rules not processed for user `ca42f2606e8446579f92f9a538460bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:57.485146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1f03339c-09bf-480a-9694-e4454cda8399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:57.585994 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1727/3496] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:57.676179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d434277-5442-4b40-bc84-e5b8d4dc22fd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:57.682934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7d434277-5442-4b40-bc84-e5b8d4dc22fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:57.772809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d434277-5442-4b40-bc84-e5b8d4dc22fd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:57.779208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebd8f7-7381-4824-9b5f-8bd664f10bb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:57.780242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd8f7-7381-4824-9b5f-8bd664f10bb7 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:57.780524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd8f7-7381-4824-9b5f-8bd664f10bb7 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:57.780831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd8f7-7381-4824-9b5f-8bd664f10bb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:57.791379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebd8f7-7381-4824-9b5f-8bd664f10bb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:57.793855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebd8f7-7381-4824-9b5f-8bd664f10bb7 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:57.809612 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1770/3497] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:58.134753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f03339c-09bf-480a-9694-e4454cda8399 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:58.223560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f03339c-09bf-480a-9694-e4454cda8399 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:58.228463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:58.229135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:58.229550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:58.229843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] PATH_INFO: `/v3/users/de9c44038e5449a1a231de449134264e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:58.231041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de9c44038e5449a1a231de449134264e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:58.233038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:58.295390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de9c44038e5449a1a231de449134264e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:58.304210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:58.310839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de9c44038e5449a1a231de449134264e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:58.312735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de9c44038e5449a1a231de449134264e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:58.321453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de9c44038e5449a1a231de449134264e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:58.333608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de9c44038e5449a1a231de449134264e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:58.341766 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1728/3498] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:58:58 2022] DELETE /identity/v3/users/de9c44038e5449a1a231de449134264e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:58.346908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec71b8-f492-4da9-a880-b70e195230c9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:58.347746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec71b8-f492-4da9-a880-b70e195230c9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:58.348218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec71b8-f492-4da9-a880-b70e195230c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:58.371172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-44ec71b8-f492-4da9-a880-b70e195230c9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:58.372485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-44ec71b8-f492-4da9-a880-b70e195230c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:58.393492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:58.436307 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1771/3499] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:58:58.446918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:58.448354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:58.509908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b87a9d-9f0d-4b8a-9376-a7903ea67f52 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:58.513706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a971a71-ab7e-4638-8c80-0163c7351a97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:58.514313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a971a71-ab7e-4638-8c80-0163c7351a97 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:58.514608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a971a71-ab7e-4638-8c80-0163c7351a97 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:58.514872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a971a71-ab7e-4638-8c80-0163c7351a97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:58.525798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a971a71-ab7e-4638-8c80-0163c7351a97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:58.527279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a971a71-ab7e-4638-8c80-0163c7351a97 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:58.537552 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1729/3500] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:58:58.905692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec71b8-f492-4da9-a880-b70e195230c9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:58.907820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-44ec71b8-f492-4da9-a880-b70e195230c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:58.919799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec71b8-f492-4da9-a880-b70e195230c9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:58.924619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:58.925468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:58.925857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:58.926336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] PATH_INFO: `/v3/projects/aa0d61268a4f4ea58d07ce52ad0fd477` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:58.927983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa0d61268a4f4ea58d07ce52ad0fd477)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:58.930261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:58.980344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa0d61268a4f4ea58d07ce52ad0fd477', 'request_id': 'req-60ca0010-d7af-475f-a5c9-b0b651bce96b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:58.988347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa0d61268a4f4ea58d07ce52ad0fd477', 'request_id': 'req-60ca0010-d7af-475f-a5c9-b0b651bce96b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:58.994864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa0d61268a4f4ea58d07ce52ad0fd477', 'request_id': 'req-60ca0010-d7af-475f-a5c9-b0b651bce96b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.005895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] The token cache is being invalidate because project aa0d61268a4f4ea58d07ce52ad0fd477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:58:59.006188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.006836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.008329 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1772/3501] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:58:58 2022] DELETE /identity/v3/projects/aa0d61268a4f4ea58d07ce52ad0fd477 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:59.012320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a971a71-ab7e-4638-8c80-0163c7351a97 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:59.023217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a971a71-ab7e-4638-8c80-0163c7351a97 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:59.100910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17f777-d5cb-4d89-9773-fb9f1ea32ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:59.102143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17f777-d5cb-4d89-9773-fb9f1ea32ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:59.102260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17f777-d5cb-4d89-9773-fb9f1ea32ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:59.102767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17f777-d5cb-4d89-9773-fb9f1ea32ce3 admin admin] PATH_INFO: `/v3/users/ca42f2606e8446579f92f9a538460bd2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:59.103953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17f777-d5cb-4d89-9773-fb9f1ea32ce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca42f2606e8446579f92f9a538460bd2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:59.105942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17f777-d5cb-4d89-9773-fb9f1ea32ce3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:59.192945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-de17f777-d5cb-4d89-9773-fb9f1ea32ce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca42f2606e8446579f92f9a538460bd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.213478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-de17f777-d5cb-4d89-9773-fb9f1ea32ce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca42f2606e8446579f92f9a538460bd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.214477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-de17f777-d5cb-4d89-9773-fb9f1ea32ce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca42f2606e8446579f92f9a538460bd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.223501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-de17f777-d5cb-4d89-9773-fb9f1ea32ce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca42f2606e8446579f92f9a538460bd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.231593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-de17f777-d5cb-4d89-9773-fb9f1ea32ce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca42f2606e8446579f92f9a538460bd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.233934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:59.240452 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1730/3502] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:58:59 2022] DELETE /identity/v3/users/ca42f2606e8446579f92f9a538460bd2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:59.725762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:58:59.727975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:58:59.742842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca0010-d7af-475f-a5c9-b0b651bce96b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:58:59.834814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:58:59.835635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:59.835984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:59.836323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33 admin admin] PATH_INFO: `/v3/projects/5247d3af30d74b339d6986bf6e731e6e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:59.837712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5247d3af30d74b339d6986bf6e731e6e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:58:59.839691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:58:59.898813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5247d3af30d74b339d6986bf6e731e6e', 'request_id': 'req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.900948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:58:59.913708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5247d3af30d74b339d6986bf6e731e6e', 'request_id': 'req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.921528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5247d3af30d74b339d6986bf6e731e6e', 'request_id': 'req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.940820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33 admin admin] The token cache is being invalidate because project 5247d3af30d74b339d6986bf6e731e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:58:59.941171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.942136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-154db5f9-dbeb-4bc7-a2c9-958bfa4c6b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:58:59.947626 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1773/3503] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:58:59 2022] DELETE /identity/v3/projects/5247d3af30d74b339d6986bf6e731e6e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:58:59.950317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49df059-a673-4b01-a1fc-773e0c7ac299 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:58:59.950612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49df059-a673-4b01-a1fc-773e0c7ac299 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:58:59.950892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49df059-a673-4b01-a1fc-773e0c7ac299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:58:59.978508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e49df059-a673-4b01-a1fc-773e0c7ac299 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:58:59.979820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e49df059-a673-4b01-a1fc-773e0c7ac299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:00.130104 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1731/3504] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:00.138043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be6a90-39f8-40a7-a5f3-3f449605fcbe None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:00.138331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be6a90-39f8-40a7-a5f3-3f449605fcbe None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:00.138607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be6a90-39f8-40a7-a5f3-3f449605fcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:00.164516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-39be6a90-39f8-40a7-a5f3-3f449605fcbe None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:00.165838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-39be6a90-39f8-40a7-a5f3-3f449605fcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:00.178082 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1774/3505] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:00.182035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49df059-a673-4b01-a1fc-773e0c7ac299 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:00.199554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49df059-a673-4b01-a1fc-773e0c7ac299 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:00.211095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de0669-19f4-4e27-baf6-d696b31bcf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:00.216425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de0669-19f4-4e27-baf6-d696b31bcf57 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:00.217315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de0669-19f4-4e27-baf6-d696b31bcf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:00.218014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de0669-19f4-4e27-baf6-d696b31bcf57 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:00.222044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de0669-19f4-4e27-baf6-d696b31bcf57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:59:00.223278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de0669-19f4-4e27-baf6-d696b31bcf57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:00.227210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de0669-19f4-4e27-baf6-d696b31bcf57 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:00.260017 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1732/3506] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:59:00.264697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be6a90-39f8-40a7-a5f3-3f449605fcbe None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:00.294675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be6a90-39f8-40a7-a5f3-3f449605fcbe None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:00.305536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34384d9-9161-4de9-aa1c-3fa804164413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:00.305861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34384d9-9161-4de9-aa1c-3fa804164413 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:00.306155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34384d9-9161-4de9-aa1c-3fa804164413 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:00.306434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34384d9-9161-4de9-aa1c-3fa804164413 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:00.307902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34384d9-9161-4de9-aa1c-3fa804164413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:00.313269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34384d9-9161-4de9-aa1c-3fa804164413 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:00.378716 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1775/3507] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:59:00 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:59:00.381295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de0669-19f4-4e27-baf6-d696b31bcf57 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:00.394813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de0669-19f4-4e27-baf6-d696b31bcf57 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:00.550161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2a085-cb44-4296-a90e-c38e27e10fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:00.551690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2a085-cb44-4296-a90e-c38e27e10fed admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:00.552415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2a085-cb44-4296-a90e-c38e27e10fed admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:00.552415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2a085-cb44-4296-a90e-c38e27e10fed admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:00.553980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2a085-cb44-4296-a90e-c38e27e10fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:00.559539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2a085-cb44-4296-a90e-c38e27e10fed admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:00.602360 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1733/3508] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:59:00 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:59:00.606379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34384d9-9161-4de9-aa1c-3fa804164413 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:00.618694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34384d9-9161-4de9-aa1c-3fa804164413 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:00.623377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cb038-0a55-43dd-84d8-46f4d15c8522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:00.624066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cb038-0a55-43dd-84d8-46f4d15c8522 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:00.624352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cb038-0a55-43dd-84d8-46f4d15c8522 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:00.624647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cb038-0a55-43dd-84d8-46f4d15c8522 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:00.625341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cb038-0a55-43dd-84d8-46f4d15c8522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:00.626933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cb038-0a55-43dd-84d8-46f4d15c8522 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:00.640384 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1776/3509] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:59:00.641666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2a085-cb44-4296-a90e-c38e27e10fed admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:00.658463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2a085-cb44-4296-a90e-c38e27e10fed admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:00.665749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74609c62-8d89-4d89-8b4e-994db2c98a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:00.667048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74609c62-8d89-4d89-8b4e-994db2c98a89 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:00.667346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74609c62-8d89-4d89-8b4e-994db2c98a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:00.667750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74609c62-8d89-4d89-8b4e-994db2c98a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:00.668412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74609c62-8d89-4d89-8b4e-994db2c98a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:00.670057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74609c62-8d89-4d89-8b4e-994db2c98a89 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:00.686200 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1734/3510] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:59:00.691057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cb038-0a55-43dd-84d8-46f4d15c8522 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:00.703959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cb038-0a55-43dd-84d8-46f4d15c8522 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:00.708504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eb94b-3ef5-4b18-b95c-d40781851211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:00.709187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eb94b-3ef5-4b18-b95c-d40781851211 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:00.710044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eb94b-3ef5-4b18-b95c-d40781851211 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:00.710044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eb94b-3ef5-4b18-b95c-d40781851211 admin admin] PATH_INFO: `/v3/projects/b0fbc0c573fc45b58137b43959b57034/users/50bba3c790c44425a9786b8d1404df28/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:00.727424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691eb94b-3ef5-4b18-b95c-d40781851211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0fbc0c573fc45b58137b43959b57034, user_id=50bba3c790c44425a9786b8d1404df28, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:00.731953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691eb94b-3ef5-4b18-b95c-d40781851211 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:00.746824 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1777/3511] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:59:00 2022] PUT /identity/v3/projects/b0fbc0c573fc45b58137b43959b57034/users/50bba3c790c44425a9786b8d1404df28/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 20:59:00.752805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceea34c-440e-4756-88a1-38d04dace41b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:00.753149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceea34c-440e-4756-88a1-38d04dace41b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:00.753456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceea34c-440e-4756-88a1-38d04dace41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:00.776940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9ceea34c-440e-4756-88a1-38d04dace41b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:00.778178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9ceea34c-440e-4756-88a1-38d04dace41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:00.854982 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1735/3512] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:00.877833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eb94b-3ef5-4b18-b95c-d40781851211 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:00.880692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-691eb94b-3ef5-4b18-b95c-d40781851211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:00.969301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eb94b-3ef5-4b18-b95c-d40781851211 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:00.974194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ae5b8-9c80-47b6-9ab1-844e0654805a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:00.974944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ae5b8-9c80-47b6-9ab1-844e0654805a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:00.975524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ae5b8-9c80-47b6-9ab1-844e0654805a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:00.975881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ae5b8-9c80-47b6-9ab1-844e0654805a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:00.989480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ae5b8-9c80-47b6-9ab1-844e0654805a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:00.991090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ae5b8-9c80-47b6-9ab1-844e0654805a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:01.003205 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1778/3513] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:59:04.753787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceea34c-440e-4756-88a1-38d04dace41b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:04.755569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9ceea34c-440e-4756-88a1-38d04dace41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:04.765071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:04.770459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceea34c-440e-4756-88a1-38d04dace41b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:04.775265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d8357-7e51-446e-887f-008648ff9871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:04.776265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d8357-7e51-446e-887f-008648ff9871 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:04.776766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d8357-7e51-446e-887f-008648ff9871 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:04.777091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d8357-7e51-446e-887f-008648ff9871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:04.779245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5d5d8357-7e51-446e-887f-008648ff9871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:04.793206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d8357-7e51-446e-887f-008648ff9871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:04.795704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d8357-7e51-446e-887f-008648ff9871 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:04.806884 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1736/3514] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:59:08.918584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed436c-7ffd-4bd4-998d-e7bdaa29eefc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:08.920128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed436c-7ffd-4bd4-998d-e7bdaa29eefc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:08.920649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed436c-7ffd-4bd4-998d-e7bdaa29eefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:08.964010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e9ed436c-7ffd-4bd4-998d-e7bdaa29eefc None None] MFA Rules not processed for user `50bba3c790c44425a9786b8d1404df28`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:08.966195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e9ed436c-7ffd-4bd4-998d-e7bdaa29eefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:09.066424 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1779/3515] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:09.067152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d8357-7e51-446e-887f-008648ff9871 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:09.081217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d8357-7e51-446e-887f-008648ff9871 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:09.110065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fa8be-096c-41ec-adce-69d25247fde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:09.110709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fa8be-096c-41ec-adce-69d25247fde8 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:09.110994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fa8be-096c-41ec-adce-69d25247fde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:09.111278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fa8be-096c-41ec-adce-69d25247fde8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:09.112425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125fa8be-096c-41ec-adce-69d25247fde8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:09.113845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125fa8be-096c-41ec-adce-69d25247fde8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:09.206828 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1737/3516] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:59:09 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:59:09.211150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed436c-7ffd-4bd4-998d-e7bdaa29eefc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:09.224430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed436c-7ffd-4bd4-998d-e7bdaa29eefc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:09.230087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:09.302857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27643c4-9a2a-4da0-a9da-d96c1eb57022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:09.303582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27643c4-9a2a-4da0-a9da-d96c1eb57022 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:09.303872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27643c4-9a2a-4da0-a9da-d96c1eb57022 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:09.304151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27643c4-9a2a-4da0-a9da-d96c1eb57022 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:09.306576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27643c4-9a2a-4da0-a9da-d96c1eb57022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:09.322906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27643c4-9a2a-4da0-a9da-d96c1eb57022 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:09.376969 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1780/3517] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:59:09 2022] POST /identity/v3/users => generated 299 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:59:09.378958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fa8be-096c-41ec-adce-69d25247fde8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:09.392046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fa8be-096c-41ec-adce-69d25247fde8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:09.398332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21230c1-6b76-4170-af09-1288caa0597f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:09.399336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21230c1-6b76-4170-af09-1288caa0597f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:09.399943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21230c1-6b76-4170-af09-1288caa0597f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:09.400397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21230c1-6b76-4170-af09-1288caa0597f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:09.401688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21230c1-6b76-4170-af09-1288caa0597f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:09.403782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21230c1-6b76-4170-af09-1288caa0597f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:09.416646 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1738/3518] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:59:09.423514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27643c4-9a2a-4da0-a9da-d96c1eb57022 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:09.445430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27643c4-9a2a-4da0-a9da-d96c1eb57022 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:09.451268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce6020-ed7c-4b3a-86a3-abe08615b759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:09.451694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce6020-ed7c-4b3a-86a3-abe08615b759 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:09.452051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce6020-ed7c-4b3a-86a3-abe08615b759 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:09.452432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce6020-ed7c-4b3a-86a3-abe08615b759 admin admin] PATH_INFO: `/v3/projects/c6a7ecf606c54105947f66a1434670a8/users/e7f444821052471a8ae828275402b8ec/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:09.477622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce6020-ed7c-4b3a-86a3-abe08615b759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a7ecf606c54105947f66a1434670a8, user_id=e7f444821052471a8ae828275402b8ec, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:09.480481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce6020-ed7c-4b3a-86a3-abe08615b759 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:09.500957 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1781/3519] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:59:09 2022] PUT /identity/v3/projects/c6a7ecf606c54105947f66a1434670a8/users/e7f444821052471a8ae828275402b8ec/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:59:09.503597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21230c1-6b76-4170-af09-1288caa0597f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:09.516375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21230c1-6b76-4170-af09-1288caa0597f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:09.596305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd06ce-fc95-4e27-96eb-a18f004334d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:09.597623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd06ce-fc95-4e27-96eb-a18f004334d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:09.597917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd06ce-fc95-4e27-96eb-a18f004334d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:09.598201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd06ce-fc95-4e27-96eb-a18f004334d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:09.599525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd06ce-fc95-4e27-96eb-a18f004334d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:09.613534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd06ce-fc95-4e27-96eb-a18f004334d3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:09.637348 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1739/3520] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:59:09.650638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce6020-ed7c-4b3a-86a3-abe08615b759 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:09.667456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce6020-ed7c-4b3a-86a3-abe08615b759 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:09.689010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549d292-ecb4-4a9e-9a9c-8c2989cd703e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:09.690729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549d292-ecb4-4a9e-9a9c-8c2989cd703e admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:09.691169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549d292-ecb4-4a9e-9a9c-8c2989cd703e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:09.691485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549d292-ecb4-4a9e-9a9c-8c2989cd703e admin admin] PATH_INFO: `/v3/projects/c6a7ecf606c54105947f66a1434670a8/users/e7f444821052471a8ae828275402b8ec/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:09.694206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549d292-ecb4-4a9e-9a9c-8c2989cd703e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a7ecf606c54105947f66a1434670a8, user_id=e7f444821052471a8ae828275402b8ec, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:09.698778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549d292-ecb4-4a9e-9a9c-8c2989cd703e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:09.719850 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1782/3521] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:59:09 2022] PUT /identity/v3/projects/c6a7ecf606c54105947f66a1434670a8/users/e7f444821052471a8ae828275402b8ec/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:59:14.759191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd06ce-fc95-4e27-96eb-a18f004334d3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:14.761308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9fcd06ce-fc95-4e27-96eb-a18f004334d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:15.031376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd06ce-fc95-4e27-96eb-a18f004334d3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:15.227668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:15.315004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb1008-d23b-44fe-839e-d7c9e7f86e57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:15.315509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb1008-d23b-44fe-839e-d7c9e7f86e57 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:15.315791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb1008-d23b-44fe-839e-d7c9e7f86e57 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:15.316081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb1008-d23b-44fe-839e-d7c9e7f86e57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:15.325277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-eefb1008-d23b-44fe-839e-d7c9e7f86e57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:15.361944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb1008-d23b-44fe-839e-d7c9e7f86e57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:15.376298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb1008-d23b-44fe-839e-d7c9e7f86e57 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:15.411058 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1740/3522] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:59:17.944272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a38c16-d5b9-4066-9204-33ac4c56766c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:17.944845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a38c16-d5b9-4066-9204-33ac4c56766c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:17.944845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a38c16-d5b9-4066-9204-33ac4c56766c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:17.990448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-14a38c16-d5b9-4066-9204-33ac4c56766c None None] MFA Rules not processed for user `e7f444821052471a8ae828275402b8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:17.993464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-14a38c16-d5b9-4066-9204-33ac4c56766c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:18.103138 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1783/3523] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:18.117131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb1008-d23b-44fe-839e-d7c9e7f86e57 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:18.119529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-eefb1008-d23b-44fe-839e-d7c9e7f86e57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:18.144849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb1008-d23b-44fe-839e-d7c9e7f86e57 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:18.151480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f46fb9-9d62-4a7e-883d-303cd0b7e76b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:18.152381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f46fb9-9d62-4a7e-883d-303cd0b7e76b service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:18.152837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f46fb9-9d62-4a7e-883d-303cd0b7e76b service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:18.153298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f46fb9-9d62-4a7e-883d-303cd0b7e76b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:18.242295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f46fb9-9d62-4a7e-883d-303cd0b7e76b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:18.243961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f46fb9-9d62-4a7e-883d-303cd0b7e76b service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:18.262443 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1741/3524] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:59:21.305776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755b0de-f6e6-48b6-a8f8-3bdc1119cce8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:21.306311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755b0de-f6e6-48b6-a8f8-3bdc1119cce8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:21.306431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755b0de-f6e6-48b6-a8f8-3bdc1119cce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:21.329132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8755b0de-f6e6-48b6-a8f8-3bdc1119cce8 None None] MFA Rules not processed for user `e6970a72952e4d4a8f26e0f8ee23f364`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:21.330387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8755b0de-f6e6-48b6-a8f8-3bdc1119cce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:21.388090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:21.449673 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1784/3525] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:22.101228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8baf3-3fb4-4477-8f7c-f8199dd4f4cb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:22.101761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8baf3-3fb4-4477-8f7c-f8199dd4f4cb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:22.102035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8baf3-3fb4-4477-8f7c-f8199dd4f4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:22.149645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8bf8baf3-3fb4-4477-8f7c-f8199dd4f4cb None None] MFA Rules not processed for user `50bba3c790c44425a9786b8d1404df28`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:22.151199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8bf8baf3-3fb4-4477-8f7c-f8199dd4f4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:22.159907 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1742/3526] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:22.590778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755b0de-f6e6-48b6-a8f8-3bdc1119cce8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:22.593511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8755b0de-f6e6-48b6-a8f8-3bdc1119cce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:22.807485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755b0de-f6e6-48b6-a8f8-3bdc1119cce8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:22.821720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1afebe-3929-4f5f-b966-790d883ce331 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:22.822940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1afebe-3929-4f5f-b966-790d883ce331 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:22.825970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1afebe-3929-4f5f-b966-790d883ce331 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:22.826285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1afebe-3929-4f5f-b966-790d883ce331 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:22.829928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ab1afebe-3929-4f5f-b966-790d883ce331 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:22.871765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1afebe-3929-4f5f-b966-790d883ce331 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:22.873738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1afebe-3929-4f5f-b966-790d883ce331 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:22.903908 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1785/3527] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:59:23.933355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8baf3-3fb4-4477-8f7c-f8199dd4f4cb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:23.935654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8bf8baf3-3fb4-4477-8f7c-f8199dd4f4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:24.104348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8baf3-3fb4-4477-8f7c-f8199dd4f4cb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:24.123957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:24.125377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:24.126185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:24.126874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] PATH_INFO: `/v3/users/fba464c031484398bf568f091e5018a4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:24.128433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba464c031484398bf568f091e5018a4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:24.130579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:24.140272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1afebe-3929-4f5f-b966-790d883ce331 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:24.143105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ab1afebe-3929-4f5f-b966-790d883ce331 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:24.170320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1afebe-3929-4f5f-b966-790d883ce331 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:24.184280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:24.185277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:24.186090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:24.186524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] PATH_INFO: `/v3/users/336a280254b44d5e9c99abbce6fb88d5` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:24.192522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=336a280254b44d5e9c99abbce6fb88d5)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:24.196651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:24.226364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba464c031484398bf568f091e5018a4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:24.246989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba464c031484398bf568f091e5018a4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:24.247874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba464c031484398bf568f091e5018a4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:24.258971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:24.273422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba464c031484398bf568f091e5018a4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:24.284545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba464c031484398bf568f091e5018a4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:24.302063 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1743/3528] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:59:23 2022] DELETE /identity/v3/users/fba464c031484398bf568f091e5018a4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:59:24.317245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336a280254b44d5e9c99abbce6fb88d5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:24.355925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336a280254b44d5e9c99abbce6fb88d5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:24.367408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336a280254b44d5e9c99abbce6fb88d5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:24.369624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336a280254b44d5e9c99abbce6fb88d5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:24.373372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336a280254b44d5e9c99abbce6fb88d5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:24.402537 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1786/3529] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:59:24 2022] DELETE /identity/v3/users/336a280254b44d5e9c99abbce6fb88d5 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:59:24.748556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:24.751137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:24.800143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:24.802452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:24.920515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:24.922969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96515dc6-3953-4b5f-994c-a33fcada15bd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:24.937545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ceff6-1bca-49e6-b291-afb49bec7e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:24.938898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ceff6-1bca-49e6-b291-afb49bec7e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:24.939352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ceff6-1bca-49e6-b291-afb49bec7e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:24.939792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ceff6-1bca-49e6-b291-afb49bec7e34 admin admin] PATH_INFO: `/v3/projects/6047f3a5056d43549f602b273ad8eab3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:24.941370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ceff6-1bca-49e6-b291-afb49bec7e34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6047f3a5056d43549f602b273ad8eab3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:24.943403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c75ec6-1872-4d41-a78c-b63a93b6d6ee admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:24.943922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ceff6-1bca-49e6-b291-afb49bec7e34 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:24.965547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52ad24-8cc8-413a-9224-9aea74cdee85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:24.966353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52ad24-8cc8-413a-9224-9aea74cdee85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:24.967075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52ad24-8cc8-413a-9224-9aea74cdee85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:24.967403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52ad24-8cc8-413a-9224-9aea74cdee85 admin admin] PATH_INFO: `/v3/projects/7262d6f38eee4c1184a6b471cf69b008` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:24.969532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52ad24-8cc8-413a-9224-9aea74cdee85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7262d6f38eee4c1184a6b471cf69b008)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:24.972140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52ad24-8cc8-413a-9224-9aea74cdee85 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:25.052040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e52ad24-8cc8-413a-9224-9aea74cdee85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7262d6f38eee4c1184a6b471cf69b008', 'request_id': 'req-5e52ad24-8cc8-413a-9224-9aea74cdee85'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:25.062193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e52ad24-8cc8-413a-9224-9aea74cdee85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7262d6f38eee4c1184a6b471cf69b008', 'request_id': 'req-5e52ad24-8cc8-413a-9224-9aea74cdee85'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:25.073316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e52ad24-8cc8-413a-9224-9aea74cdee85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7262d6f38eee4c1184a6b471cf69b008', 'request_id': 'req-5e52ad24-8cc8-413a-9224-9aea74cdee85'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:25.090989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e52ad24-8cc8-413a-9224-9aea74cdee85 admin admin] The token cache is being invalidate because project 7262d6f38eee4c1184a6b471cf69b008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:59:25.091528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e52ad24-8cc8-413a-9224-9aea74cdee85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:25.093551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e52ad24-8cc8-413a-9224-9aea74cdee85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:25.094522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d61ceff6-1bca-49e6-b291-afb49bec7e34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6047f3a5056d43549f602b273ad8eab3', 'request_id': 'req-d61ceff6-1bca-49e6-b291-afb49bec7e34'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:25.100722 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1787/3530] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:59:24 2022] DELETE /identity/v3/projects/7262d6f38eee4c1184a6b471cf69b008 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:59:25.102526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8825e88-6939-4a16-96e5-e1c6d450b5bf None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.102867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8825e88-6939-4a16-96e5-e1c6d450b5bf None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.103851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8825e88-6939-4a16-96e5-e1c6d450b5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.106577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d61ceff6-1bca-49e6-b291-afb49bec7e34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6047f3a5056d43549f602b273ad8eab3', 'request_id': 'req-d61ceff6-1bca-49e6-b291-afb49bec7e34'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:25.122524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d61ceff6-1bca-49e6-b291-afb49bec7e34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6047f3a5056d43549f602b273ad8eab3', 'request_id': 'req-d61ceff6-1bca-49e6-b291-afb49bec7e34'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:25.136983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d61ceff6-1bca-49e6-b291-afb49bec7e34 admin admin] The token cache is being invalidate because project 6047f3a5056d43549f602b273ad8eab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:59:25.137382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d61ceff6-1bca-49e6-b291-afb49bec7e34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:25.138147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d61ceff6-1bca-49e6-b291-afb49bec7e34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:25.139656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b8825e88-6939-4a16-96e5-e1c6d450b5bf None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:25.143102 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1744/3531] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:59:24 2022] DELETE /identity/v3/projects/6047f3a5056d43549f602b273ad8eab3 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:59:25.143252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b8825e88-6939-4a16-96e5-e1c6d450b5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:25.145268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6d9c7-fbee-4174-ba39-eab74f8dadf5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.145565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6d9c7-fbee-4174-ba39-eab74f8dadf5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.145850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6d9c7-fbee-4174-ba39-eab74f8dadf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.170452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a8c6d9c7-fbee-4174-ba39-eab74f8dadf5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:25.173041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a8c6d9c7-fbee-4174-ba39-eab74f8dadf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:25.242143 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1788/3532] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:25.251588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a85729-bddf-4b43-b0f7-15ef32cae0cc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.252407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a85729-bddf-4b43-b0f7-15ef32cae0cc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.253043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a85729-bddf-4b43-b0f7-15ef32cae0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.266747 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1745/3533] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:25.275377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee470ad7-5fce-4a1f-b482-3000eaab8fb3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.275791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee470ad7-5fce-4a1f-b482-3000eaab8fb3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.276507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee470ad7-5fce-4a1f-b482-3000eaab8fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.291646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d5a85729-bddf-4b43-b0f7-15ef32cae0cc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:25.293729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d5a85729-bddf-4b43-b0f7-15ef32cae0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:25.306889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ee470ad7-5fce-4a1f-b482-3000eaab8fb3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:25.308885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ee470ad7-5fce-4a1f-b482-3000eaab8fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:25.315532 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1789/3534] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:25.318784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a85729-bddf-4b43-b0f7-15ef32cae0cc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:25.323788 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1746/3535] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 20:59:25.328052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee470ad7-5fce-4a1f-b482-3000eaab8fb3 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:25.332723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a85729-bddf-4b43-b0f7-15ef32cae0cc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:25.347723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fe0a0-962d-45dc-b9b0-1aba2ec0f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:25.348585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fe0a0-962d-45dc-b9b0-1aba2ec0f1ad admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.348876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fe0a0-962d-45dc-b9b0-1aba2ec0f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.349235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fe0a0-962d-45dc-b9b0-1aba2ec0f1ad admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.350734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fe0a0-962d-45dc-b9b0-1aba2ec0f1ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:59:25.351049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fe0a0-962d-45dc-b9b0-1aba2ec0f1ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:25.352485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee470ad7-5fce-4a1f-b482-3000eaab8fb3 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:25.353000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fe0a0-962d-45dc-b9b0-1aba2ec0f1ad admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:25.364092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63eaeb-f878-4555-9746-da3030ffa528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:25.364941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63eaeb-f878-4555-9746-da3030ffa528 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.365737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63eaeb-f878-4555-9746-da3030ffa528 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.366128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63eaeb-f878-4555-9746-da3030ffa528 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.367116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63eaeb-f878-4555-9746-da3030ffa528 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:59:25.367483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63eaeb-f878-4555-9746-da3030ffa528 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:25.372268 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1790/3536] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 20:59:25.372442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63eaeb-f878-4555-9746-da3030ffa528 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:25.373413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fe0a0-962d-45dc-b9b0-1aba2ec0f1ad admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:25.381921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:25.386462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fe0a0-962d-45dc-b9b0-1aba2ec0f1ad admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:25.387049 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1747/3537] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:59:25.402919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d1ea2-9b9b-4406-80c9-7207909f3944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:25.404037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d1ea2-9b9b-4406-80c9-7207909f3944 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.404164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63eaeb-f878-4555-9746-da3030ffa528 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:25.404592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d1ea2-9b9b-4406-80c9-7207909f3944 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.404885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d1ea2-9b9b-4406-80c9-7207909f3944 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.406089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d1ea2-9b9b-4406-80c9-7207909f3944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:25.411087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d1ea2-9b9b-4406-80c9-7207909f3944 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:25.427110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63eaeb-f878-4555-9746-da3030ffa528 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:25.433641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526fe72-a1f3-4eec-b550-deb92be44604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:25.434705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526fe72-a1f3-4eec-b550-deb92be44604 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.435006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526fe72-a1f3-4eec-b550-deb92be44604 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.435292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526fe72-a1f3-4eec-b550-deb92be44604 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.436481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526fe72-a1f3-4eec-b550-deb92be44604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:25.438620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526fe72-a1f3-4eec-b550-deb92be44604 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:25.460831 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1791/3538] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:59:25 2022] POST /identity/v3/projects => generated 397 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:59:25.466217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d1ea2-9b9b-4406-80c9-7207909f3944 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:25.502835 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1748/3539] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:59:25 2022] POST /identity/v3/projects => generated 395 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:59:25.513294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526fe72-a1f3-4eec-b550-deb92be44604 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:25.569308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:25.629697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d1ea2-9b9b-4406-80c9-7207909f3944 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:25.679073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526fe72-a1f3-4eec-b550-deb92be44604 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:25.695477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f2437-0154-4f0a-8ce2-f1d23681d875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:25.696348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2437-0154-4f0a-8ce2-f1d23681d875 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.696796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2437-0154-4f0a-8ce2-f1d23681d875 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.697258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2437-0154-4f0a-8ce2-f1d23681d875 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.698082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f2437-0154-4f0a-8ce2-f1d23681d875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:25.700047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f2437-0154-4f0a-8ce2-f1d23681d875 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:25.731476 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1749/3540] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:59:25 2022] POST /identity/v3/users => generated 298 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:59:25.735717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f2437-0154-4f0a-8ce2-f1d23681d875 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:25.763486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f2437-0154-4f0a-8ce2-f1d23681d875 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:25.769784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3308c5-6ba0-4f0e-9259-348ff850c709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:25.770539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3308c5-6ba0-4f0e-9259-348ff850c709 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.770931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3308c5-6ba0-4f0e-9259-348ff850c709 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.771311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3308c5-6ba0-4f0e-9259-348ff850c709 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.772059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3308c5-6ba0-4f0e-9259-348ff850c709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:25.774157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3308c5-6ba0-4f0e-9259-348ff850c709 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:25.787924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed27dc-ed2e-47dd-b1c1-d79c63012ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:25.788817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed27dc-ed2e-47dd-b1c1-d79c63012ebe admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.789147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed27dc-ed2e-47dd-b1c1-d79c63012ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.789427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed27dc-ed2e-47dd-b1c1-d79c63012ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.790088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed27dc-ed2e-47dd-b1c1-d79c63012ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:25.791994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed27dc-ed2e-47dd-b1c1-d79c63012ebe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:25.808379 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1750/3541] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:59:25.812841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed27dc-ed2e-47dd-b1c1-d79c63012ebe admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:25.816816 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1792/3542] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:59:25 2022] POST /identity/v3/users => generated 299 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:59:25.820722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3308c5-6ba0-4f0e-9259-348ff850c709 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:25.827196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed27dc-ed2e-47dd-b1c1-d79c63012ebe admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:25.833817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9cf9e-0c9b-45f7-b987-289415f3f2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:25.835270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9cf9e-0c9b-45f7-b987-289415f3f2bf admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.835894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9cf9e-0c9b-45f7-b987-289415f3f2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.838782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3308c5-6ba0-4f0e-9259-348ff850c709 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:25.841586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9cf9e-0c9b-45f7-b987-289415f3f2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.843835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9cf9e-0c9b-45f7-b987-289415f3f2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:25.845395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e0fa2-4807-4efb-b9b6-dc0138427e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:25.846181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e0fa2-4807-4efb-b9b6-dc0138427e43 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.846339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9cf9e-0c9b-45f7-b987-289415f3f2bf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:25.846752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e0fa2-4807-4efb-b9b6-dc0138427e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.847375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e0fa2-4807-4efb-b9b6-dc0138427e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.848613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e0fa2-4807-4efb-b9b6-dc0138427e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:25.852769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e0fa2-4807-4efb-b9b6-dc0138427e43 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:25.860974 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1751/3543] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:59:25.864464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9cf9e-0c9b-45f7-b987-289415f3f2bf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:25.871895 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1793/3544] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:59:25.873647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e0fa2-4807-4efb-b9b6-dc0138427e43 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:25.891280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e0fa2-4807-4efb-b9b6-dc0138427e43 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:25.893549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9cf9e-0c9b-45f7-b987-289415f3f2bf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:25.899196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ba669-a90c-4929-9c52-b20e17772423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:25.899828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84d173-6b9c-49d7-8416-39f26e3cfd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:25.900089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ba669-a90c-4929-9c52-b20e17772423 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.900429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ba669-a90c-4929-9c52-b20e17772423 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.900723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ba669-a90c-4929-9c52-b20e17772423 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.900975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84d173-6b9c-49d7-8416-39f26e3cfd20 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.901394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ba669-a90c-4929-9c52-b20e17772423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:25.901612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84d173-6b9c-49d7-8416-39f26e3cfd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.902444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84d173-6b9c-49d7-8416-39f26e3cfd20 admin admin] PATH_INFO: `/v3/projects/cf527a7e5cbd41a1bf27d9e8cb663900/users/31880a51f3464381bf28535f2e52d75e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.903063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ba669-a90c-4929-9c52-b20e17772423 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:25.918682 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1794/3545] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:59:25.923988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ba669-a90c-4929-9c52-b20e17772423 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:25.929041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84d173-6b9c-49d7-8416-39f26e3cfd20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf527a7e5cbd41a1bf27d9e8cb663900, user_id=31880a51f3464381bf28535f2e52d75e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:25.946402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ba669-a90c-4929-9c52-b20e17772423 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:25.951860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84d173-6b9c-49d7-8416-39f26e3cfd20 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:25.955560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10876824-c3c7-4ca6-97d7-d78bd0b3bb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:25.956242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10876824-c3c7-4ca6-97d7-d78bd0b3bb3e admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.956529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10876824-c3c7-4ca6-97d7-d78bd0b3bb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.956809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10876824-c3c7-4ca6-97d7-d78bd0b3bb3e admin admin] PATH_INFO: `/v3/projects/f5cdab592b0646aaa02deff957842bc0/users/6b25b05106404b64adc5476924f2e69d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.977754 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1752/3546] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:59:25 2022] PUT /identity/v3/projects/cf527a7e5cbd41a1bf27d9e8cb663900/users/31880a51f3464381bf28535f2e52d75e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:59:25.981989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10876824-c3c7-4ca6-97d7-d78bd0b3bb3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5cdab592b0646aaa02deff957842bc0, user_id=6b25b05106404b64adc5476924f2e69d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:25.983248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69abe73-851e-4260-a7ec-3dbac634cea3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:25.984133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69abe73-851e-4260-a7ec-3dbac634cea3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:25.985946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69abe73-851e-4260-a7ec-3dbac634cea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:25.986841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10876824-c3c7-4ca6-97d7-d78bd0b3bb3e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:26.009593 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1795/3547] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:59:25 2022] PUT /identity/v3/projects/f5cdab592b0646aaa02deff957842bc0/users/6b25b05106404b64adc5476924f2e69d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:59:26.024883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d12a0-a7f4-47c8-a0ef-a502bf4dd86f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:26.025238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d12a0-a7f4-47c8-a0ef-a502bf4dd86f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:26.027121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d12a0-a7f4-47c8-a0ef-a502bf4dd86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:26.038388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c69abe73-851e-4260-a7ec-3dbac634cea3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:26.040281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c69abe73-851e-4260-a7ec-3dbac634cea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:26.074705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-010d12a0-a7f4-47c8-a0ef-a502bf4dd86f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:26.075958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-010d12a0-a7f4-47c8-a0ef-a502bf4dd86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:26.166384 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1796/3548] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:26.183717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d12a0-a7f4-47c8-a0ef-a502bf4dd86f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:26.186796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-010d12a0-a7f4-47c8-a0ef-a502bf4dd86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:26.236724 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1753/3549] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:26.241113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:26.261429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69abe73-851e-4260-a7ec-3dbac634cea3 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:26.263126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c69abe73-851e-4260-a7ec-3dbac634cea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:26.269859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d12a0-a7f4-47c8-a0ef-a502bf4dd86f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:26.273757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6375b39-8957-400e-bc69-1602615958ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:26.274491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6375b39-8957-400e-bc69-1602615958ff service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:26.274780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6375b39-8957-400e-bc69-1602615958ff service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:26.275060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6375b39-8957-400e-bc69-1602615958ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:26.282475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69abe73-851e-4260-a7ec-3dbac634cea3 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:26.285327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6375b39-8957-400e-bc69-1602615958ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:26.286778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6375b39-8957-400e-bc69-1602615958ff service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:26.288103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336adcf-1d3f-406c-9836-82c4ec123740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:26.289768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336adcf-1d3f-406c-9836-82c4ec123740 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:26.290057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336adcf-1d3f-406c-9836-82c4ec123740 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:26.290336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336adcf-1d3f-406c-9836-82c4ec123740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:26.301655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336adcf-1d3f-406c-9836-82c4ec123740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:26.302979 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1797/3550] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:59:26.303859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336adcf-1d3f-406c-9836-82c4ec123740 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:26.314614 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1754/3551] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:59:32.980116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6375b39-8957-400e-bc69-1602615958ff service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:32.985403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d6375b39-8957-400e-bc69-1602615958ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:33.187350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6375b39-8957-400e-bc69-1602615958ff service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:33.205561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f988b-3210-43af-bbc0-12cd03d17fcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:33.206203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f988b-3210-43af-bbc0-12cd03d17fcc service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:33.206492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f988b-3210-43af-bbc0-12cd03d17fcc service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:33.206931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f988b-3210-43af-bbc0-12cd03d17fcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:33.210713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d27f988b-3210-43af-bbc0-12cd03d17fcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:33.258299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f988b-3210-43af-bbc0-12cd03d17fcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:33.259955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f988b-3210-43af-bbc0-12cd03d17fcc service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:33.285024 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1798/3552] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:59:35.713791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6105ad-927e-4948-afe2-85ecdea88ab5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:35.714596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6105ad-927e-4948-afe2-85ecdea88ab5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:35.714596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6105ad-927e-4948-afe2-85ecdea88ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:35.765558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:35.788210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6f6105ad-927e-4948-afe2-85ecdea88ab5 None None] MFA Rules not processed for user `6b25b05106404b64adc5476924f2e69d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:35.791002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6f6105ad-927e-4948-afe2-85ecdea88ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:35.931524 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1755/3553] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:35 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:35.943758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed63c3-b5cd-4e7e-9008-6f52d9f7ef2f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:35.944256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed63c3-b5cd-4e7e-9008-6f52d9f7ef2f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:35.944256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed63c3-b5cd-4e7e-9008-6f52d9f7ef2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:35.986551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-11ed63c3-b5cd-4e7e-9008-6f52d9f7ef2f None None] MFA Rules not processed for user `6b25b05106404b64adc5476924f2e69d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:35.987801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-11ed63c3-b5cd-4e7e-9008-6f52d9f7ef2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:36.003087 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1799/3554] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:35 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:36.011755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6105ad-927e-4948-afe2-85ecdea88ab5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:36.044533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6105ad-927e-4948-afe2-85ecdea88ab5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:36.063261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd1881-6241-4d8c-9456-4a34e379f3e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:36.063912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd1881-6241-4d8c-9456-4a34e379f3e0 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:36.064196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd1881-6241-4d8c-9456-4a34e379f3e0 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:36.064473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd1881-6241-4d8c-9456-4a34e379f3e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:36.102419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd1881-6241-4d8c-9456-4a34e379f3e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:36.104396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd1881-6241-4d8c-9456-4a34e379f3e0 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:36.142892 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1756/3555] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:36 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:59:36.966253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9353e90-ef9e-42d7-8b49-5da8cd604b6d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:36.968328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9353e90-ef9e-42d7-8b49-5da8cd604b6d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:36.968860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9353e90-ef9e-42d7-8b49-5da8cd604b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:37.033701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f9353e90-ef9e-42d7-8b49-5da8cd604b6d None None] MFA Rules not processed for user `31880a51f3464381bf28535f2e52d75e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:37.034974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f9353e90-ef9e-42d7-8b49-5da8cd604b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:37.182382 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1800/3556] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:36 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:37.646264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd1881-6241-4d8c-9456-4a34e379f3e0 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:37.649788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d7fd1881-6241-4d8c-9456-4a34e379f3e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:37.688873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd1881-6241-4d8c-9456-4a34e379f3e0 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:37.699874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8f921-8147-4ec1-917b-46d92a8b4469 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:37.700621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8f921-8147-4ec1-917b-46d92a8b4469 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:37.701022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8f921-8147-4ec1-917b-46d92a8b4469 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:37.701524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8f921-8147-4ec1-917b-46d92a8b4469 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:37.722824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8f921-8147-4ec1-917b-46d92a8b4469 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:37.725747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8f921-8147-4ec1-917b-46d92a8b4469 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:37.742137 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1757/3557] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:37 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:59:44.425181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9353e90-ef9e-42d7-8b49-5da8cd604b6d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:44.443394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9353e90-ef9e-42d7-8b49-5da8cd604b6d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:44.454965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb64c9-3446-4f43-ae0f-8506f6b2bdbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:44.462864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb64c9-3446-4f43-ae0f-8506f6b2bdbc service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:44.463237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb64c9-3446-4f43-ae0f-8506f6b2bdbc service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:44.463590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb64c9-3446-4f43-ae0f-8506f6b2bdbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:44.466368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-31cb64c9-3446-4f43-ae0f-8506f6b2bdbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:44.486736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb64c9-3446-4f43-ae0f-8506f6b2bdbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:44.488997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb64c9-3446-4f43-ae0f-8506f6b2bdbc service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:44.504500 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1801/3558] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:59:44.752260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8f921-8147-4ec1-917b-46d92a8b4469 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:44.765688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8f921-8147-4ec1-917b-46d92a8b4469 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:44.771096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:44.771808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:44.772195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:44.772570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] PATH_INFO: `/v3/users/31880a51f3464381bf28535f2e52d75e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:44.775649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31880a51f3464381bf28535f2e52d75e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:44.778235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:44.862882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31880a51f3464381bf28535f2e52d75e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:44.892068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31880a51f3464381bf28535f2e52d75e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:44.893300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31880a51f3464381bf28535f2e52d75e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:44.906177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31880a51f3464381bf28535f2e52d75e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:44.916192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31880a51f3464381bf28535f2e52d75e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:44.936781 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1758/3559] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:59:44 2022] DELETE /identity/v3/users/31880a51f3464381bf28535f2e52d75e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:59:44.982883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1492b-ac54-40c2-bf07-c4aad74b07da None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:44.983450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1492b-ac54-40c2-bf07-c4aad74b07da None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:44.983739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1492b-ac54-40c2-bf07-c4aad74b07da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:45.024335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-65d1492b-ac54-40c2-bf07-c4aad74b07da None None] MFA Rules not processed for user `e6970a72952e4d4a8f26e0f8ee23f364`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:45.026478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-65d1492b-ac54-40c2-bf07-c4aad74b07da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:45.081095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:45.139475 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1802/3560] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:45.345373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:45.347927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:45.412197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:45.439783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8c0a8-3d7d-459a-80d7-b200147d88a8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:45.452203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b1433-3416-4bdd-a3f6-9daec04c7fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:45.453022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1433-3416-4bdd-a3f6-9daec04c7fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:45.453202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1433-3416-4bdd-a3f6-9daec04c7fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:45.453471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1433-3416-4bdd-a3f6-9daec04c7fa6 admin admin] PATH_INFO: `/v3/projects/cf527a7e5cbd41a1bf27d9e8cb663900` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:45.454639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b1433-3416-4bdd-a3f6-9daec04c7fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf527a7e5cbd41a1bf27d9e8cb663900)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:45.456714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b1433-3416-4bdd-a3f6-9daec04c7fa6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:45.525000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-326b1433-3416-4bdd-a3f6-9daec04c7fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf527a7e5cbd41a1bf27d9e8cb663900', 'request_id': 'req-326b1433-3416-4bdd-a3f6-9daec04c7fa6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:45.534663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-326b1433-3416-4bdd-a3f6-9daec04c7fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf527a7e5cbd41a1bf27d9e8cb663900', 'request_id': 'req-326b1433-3416-4bdd-a3f6-9daec04c7fa6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:45.544500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-326b1433-3416-4bdd-a3f6-9daec04c7fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf527a7e5cbd41a1bf27d9e8cb663900', 'request_id': 'req-326b1433-3416-4bdd-a3f6-9daec04c7fa6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:45.558580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-326b1433-3416-4bdd-a3f6-9daec04c7fa6 admin admin] The token cache is being invalidate because project cf527a7e5cbd41a1bf27d9e8cb663900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 20:59:45.558879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-326b1433-3416-4bdd-a3f6-9daec04c7fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:45.559972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-326b1433-3416-4bdd-a3f6-9daec04c7fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:45.562061 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1759/3561] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 20:59:45 2022] DELETE /identity/v3/projects/cf527a7e5cbd41a1bf27d9e8cb663900 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:59:45.570398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1435a9f-4a48-4aaa-942f-ca3e6fed0e03 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:45.571372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1435a9f-4a48-4aaa-942f-ca3e6fed0e03 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:45.571872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1435a9f-4a48-4aaa-942f-ca3e6fed0e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:45.599281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b1435a9f-4a48-4aaa-942f-ca3e6fed0e03 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:45.601839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b1435a9f-4a48-4aaa-942f-ca3e6fed0e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:45.696485 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1803/3562] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:45.704880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05eea1-6e8e-4bb9-b34f-4b07a5a47baa None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:45.705206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05eea1-6e8e-4bb9-b34f-4b07a5a47baa None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:45.705491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05eea1-6e8e-4bb9-b34f-4b07a5a47baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:45.731123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ba05eea1-6e8e-4bb9-b34f-4b07a5a47baa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:45.732363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ba05eea1-6e8e-4bb9-b34f-4b07a5a47baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:45.757170 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1760/3563] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:45.759950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1435a9f-4a48-4aaa-942f-ca3e6fed0e03 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:45.771434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1435a9f-4a48-4aaa-942f-ca3e6fed0e03 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:45.785104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132454a-d25d-429e-aa6f-7573da0a8478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:45.785747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132454a-d25d-429e-aa6f-7573da0a8478 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:45.786032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132454a-d25d-429e-aa6f-7573da0a8478 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:45.786319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132454a-d25d-429e-aa6f-7573da0a8478 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:45.786969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132454a-d25d-429e-aa6f-7573da0a8478 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 20:59:45.787277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132454a-d25d-429e-aa6f-7573da0a8478 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:45.789421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132454a-d25d-429e-aa6f-7573da0a8478 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:45.808396 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1804/3564] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 20:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 20:59:45.812569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05eea1-6e8e-4bb9-b34f-4b07a5a47baa None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:45.823002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05eea1-6e8e-4bb9-b34f-4b07a5a47baa None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:45.828228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17015e-89ed-47e9-a21e-a95adabddaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:45.828997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17015e-89ed-47e9-a21e-a95adabddaf8 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:45.829404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17015e-89ed-47e9-a21e-a95adabddaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:45.829789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17015e-89ed-47e9-a21e-a95adabddaf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:45.830727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17015e-89ed-47e9-a21e-a95adabddaf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:45.833615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17015e-89ed-47e9-a21e-a95adabddaf8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:45.888288 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1761/3565] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 20:59:45 2022] POST /identity/v3/projects => generated 395 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 20:59:45.893066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132454a-d25d-429e-aa6f-7573da0a8478 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:45.904158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132454a-d25d-429e-aa6f-7573da0a8478 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:45.976501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a306052-055a-4384-9ad0-bb0774d4020a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:45.977245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a306052-055a-4384-9ad0-bb0774d4020a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:45.977586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a306052-055a-4384-9ad0-bb0774d4020a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:45.977930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a306052-055a-4384-9ad0-bb0774d4020a admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:45.978810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a306052-055a-4384-9ad0-bb0774d4020a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:45.981054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a306052-055a-4384-9ad0-bb0774d4020a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:46.007811 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1805/3566] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 20:59:45 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 20:59:46.011648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17015e-89ed-47e9-a21e-a95adabddaf8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:46.026394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17015e-89ed-47e9-a21e-a95adabddaf8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:46.032929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bc28f-8985-43bd-a3e2-fb4442618acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:46.033710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bc28f-8985-43bd-a3e2-fb4442618acb admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:46.035154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bc28f-8985-43bd-a3e2-fb4442618acb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:46.035453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bc28f-8985-43bd-a3e2-fb4442618acb admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:46.036273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bc28f-8985-43bd-a3e2-fb4442618acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:46.038505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bc28f-8985-43bd-a3e2-fb4442618acb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:46.057166 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1762/3567] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 20:59:46.062927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a306052-055a-4384-9ad0-bb0774d4020a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:46.079768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a306052-055a-4384-9ad0-bb0774d4020a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:46.087194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b8d44-fde1-4bb0-a987-6ec8b4de1129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:46.088000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b8d44-fde1-4bb0-a987-6ec8b4de1129 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:46.088307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b8d44-fde1-4bb0-a987-6ec8b4de1129 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:46.089012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b8d44-fde1-4bb0-a987-6ec8b4de1129 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:46.089838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b8d44-fde1-4bb0-a987-6ec8b4de1129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:46.092226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b8d44-fde1-4bb0-a987-6ec8b4de1129 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:46.113233 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1806/3568] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 20:59:46.114977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bc28f-8985-43bd-a3e2-fb4442618acb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:46.150672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bc28f-8985-43bd-a3e2-fb4442618acb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:46.156231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717c5a2-8da7-4877-913d-f98cd922b8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:46.157516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717c5a2-8da7-4877-913d-f98cd922b8de admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:46.157889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717c5a2-8da7-4877-913d-f98cd922b8de admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:46.158992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717c5a2-8da7-4877-913d-f98cd922b8de admin admin] PATH_INFO: `/v3/projects/630205311fc04912910e56076899f18d/users/fa304e40e3894910be1a5e1e97a34b1a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:46.179182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717c5a2-8da7-4877-913d-f98cd922b8de admin admin] RBAC: Authorizing `identity:create_grant(project_id=630205311fc04912910e56076899f18d, user_id=fa304e40e3894910be1a5e1e97a34b1a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:46.182586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717c5a2-8da7-4877-913d-f98cd922b8de admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:46.197470 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1763/3569] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 20:59:46 2022] PUT /identity/v3/projects/630205311fc04912910e56076899f18d/users/fa304e40e3894910be1a5e1e97a34b1a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:59:46.202861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6431b0e-3301-4fc8-bcaf-308fe11ea7ab None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:46.203535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6431b0e-3301-4fc8-bcaf-308fe11ea7ab None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:46.204361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6431b0e-3301-4fc8-bcaf-308fe11ea7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:46.236340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f6431b0e-3301-4fc8-bcaf-308fe11ea7ab None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:46.238173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f6431b0e-3301-4fc8-bcaf-308fe11ea7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:46.294217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:46.361498 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1807/3570] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:46.370793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717c5a2-8da7-4877-913d-f98cd922b8de admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:46.372706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8717c5a2-8da7-4877-913d-f98cd922b8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:46.481220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717c5a2-8da7-4877-913d-f98cd922b8de admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:46.487888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f15f32-3f6e-45dd-92a6-0479de5dde31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:46.488518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f15f32-3f6e-45dd-92a6-0479de5dde31 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:46.488805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f15f32-3f6e-45dd-92a6-0479de5dde31 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:46.489152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f15f32-3f6e-45dd-92a6-0479de5dde31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:46.499929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f15f32-3f6e-45dd-92a6-0479de5dde31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:46.501410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f15f32-3f6e-45dd-92a6-0479de5dde31 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:46.511574 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1764/3571] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 20:59:52.592352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08ff65-78dc-453c-ae58-45bd9e8c914e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:52.592790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08ff65-78dc-453c-ae58-45bd9e8c914e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:52.592790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08ff65-78dc-453c-ae58-45bd9e8c914e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:52.629230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7d08ff65-78dc-453c-ae58-45bd9e8c914e None None] MFA Rules not processed for user `859a4503b4d84a4e8c3213e2de02492d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:52.630559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7d08ff65-78dc-453c-ae58-45bd9e8c914e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:52.733979 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1808/3572] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:54.165592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3ea9e-61ff-42ce-bd0c-3bd9117e76e7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:54.166069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3ea9e-61ff-42ce-bd0c-3bd9117e76e7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:54.166232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3ea9e-61ff-42ce-bd0c-3bd9117e76e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:54.198816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bde3ea9e-61ff-42ce-bd0c-3bd9117e76e7 None None] MFA Rules not processed for user `6b25b05106404b64adc5476924f2e69d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:54.200087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bde3ea9e-61ff-42ce-bd0c-3bd9117e76e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:54.225316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:54.306117 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1765/3573] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:54.395882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fa45c-9384-4254-b5d3-34f36e6b18ee None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:54.396578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fa45c-9384-4254-b5d3-34f36e6b18ee None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:54.396861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fa45c-9384-4254-b5d3-34f36e6b18ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:54.429945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7b0fa45c-9384-4254-b5d3-34f36e6b18ee None None] MFA Rules not processed for user `6b25b05106404b64adc5476924f2e69d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:54.432419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7b0fa45c-9384-4254-b5d3-34f36e6b18ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:54.445634 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1809/3574] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:54.499294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04320521-ebc3-4fc5-b20a-5cbf87a170c8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:54.499682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04320521-ebc3-4fc5-b20a-5cbf87a170c8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:54.500758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04320521-ebc3-4fc5-b20a-5cbf87a170c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:54.544662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-04320521-ebc3-4fc5-b20a-5cbf87a170c8 None None] MFA Rules not processed for user `6b25b05106404b64adc5476924f2e69d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:54.547833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-04320521-ebc3-4fc5-b20a-5cbf87a170c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:54.565251 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1766/3575] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:54.592577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c3210-eb36-4b70-bb3f-d57262a73f78 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:54.592915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c3210-eb36-4b70-bb3f-d57262a73f78 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:54.593229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c3210-eb36-4b70-bb3f-d57262a73f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:54.614141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9b5c3210-eb36-4b70-bb3f-d57262a73f78 None None] MFA Rules not processed for user `6b25b05106404b64adc5476924f2e69d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:54.615570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9b5c3210-eb36-4b70-bb3f-d57262a73f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:54.632381 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1810/3576] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:54.762538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a626c6-5bee-4738-bb96-9eb0c879dd07 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:54.763196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a626c6-5bee-4738-bb96-9eb0c879dd07 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:54.763696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a626c6-5bee-4738-bb96-9eb0c879dd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:54.805272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-39a626c6-5bee-4738-bb96-9eb0c879dd07 None None] MFA Rules not processed for user `6b25b05106404b64adc5476924f2e69d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:54.807979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-39a626c6-5bee-4738-bb96-9eb0c879dd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:54.830410 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1767/3577] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:55.168597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433485f-d1ea-4e88-ac7d-07ae848a3304 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:55.169177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433485f-d1ea-4e88-ac7d-07ae848a3304 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:55.169649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433485f-d1ea-4e88-ac7d-07ae848a3304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:55.230655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9433485f-d1ea-4e88-ac7d-07ae848a3304 None None] MFA Rules not processed for user `fa304e40e3894910be1a5e1e97a34b1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:55.232039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9433485f-d1ea-4e88-ac7d-07ae848a3304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:55.337737 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1811/3578] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:55 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:55.348994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde64831-5770-48c8-bc16-6b61e57e31a6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:55.349480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde64831-5770-48c8-bc16-6b61e57e31a6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:55.349660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde64831-5770-48c8-bc16-6b61e57e31a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:55.371801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cde64831-5770-48c8-bc16-6b61e57e31a6 None None] MFA Rules not processed for user `fa304e40e3894910be1a5e1e97a34b1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:55.373963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cde64831-5770-48c8-bc16-6b61e57e31a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:55.383409 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1768/3579] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:55 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:55.439245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433485f-d1ea-4e88-ac7d-07ae848a3304 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:55.440870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9433485f-d1ea-4e88-ac7d-07ae848a3304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:55.510835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 20:59:55.527595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433485f-d1ea-4e88-ac7d-07ae848a3304 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:55.533082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64730dbb-653c-4ac3-a0d0-9c4ccba74f26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:55.533712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64730dbb-653c-4ac3-a0d0-9c4ccba74f26 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:55.534000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64730dbb-653c-4ac3-a0d0-9c4ccba74f26 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:55.534274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64730dbb-653c-4ac3-a0d0-9c4ccba74f26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:55.541849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64730dbb-653c-4ac3-a0d0-9c4ccba74f26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:55.543393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64730dbb-653c-4ac3-a0d0-9c4ccba74f26 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:55.552064 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1812/3580] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:55 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 20:59:58.886065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde64831-5770-48c8-bc16-6b61e57e31a6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:58.889398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cde64831-5770-48c8-bc16-6b61e57e31a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:58.929049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde64831-5770-48c8-bc16-6b61e57e31a6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:58.940254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:58.941104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:58.941401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:58.941907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] PATH_INFO: `/v3/users/0c77fca40dbc430db41e4a30639b28b5` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:58.943342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c77fca40dbc430db41e4a30639b28b5)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:58.962421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:59.110278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c77fca40dbc430db41e4a30639b28b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:59.151845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c77fca40dbc430db41e4a30639b28b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:59.152718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c77fca40dbc430db41e4a30639b28b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:59.179768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c77fca40dbc430db41e4a30639b28b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:59.190196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c77fca40dbc430db41e4a30639b28b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 20:59:59.201128 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1769/3581] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 20:59:58 2022] DELETE /identity/v3/users/0c77fca40dbc430db41e4a30639b28b5 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 20:59:59.203102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c269c54-f604-474a-9a66-60ebf5a53f5c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:59.203607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c269c54-f604-474a-9a66-60ebf5a53f5c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:59.203915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c269c54-f604-474a-9a66-60ebf5a53f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:59.241927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4c269c54-f604-474a-9a66-60ebf5a53f5c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 20:59:59.244115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4c269c54-f604-474a-9a66-60ebf5a53f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:59.415204 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1813/3582] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 20:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 20:59:59.453217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 20:59:59.454885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 20:59:59.683421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22284a-5f35-4ba2-a6f4-de240da68a7f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 20:59:59.691998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5802b7-aa24-4295-b522-9d718e21ee7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 20:59:59.692736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5802b7-aa24-4295-b522-9d718e21ee7b service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 20:59:59.693125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5802b7-aa24-4295-b522-9d718e21ee7b service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 20:59:59.693486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5802b7-aa24-4295-b522-9d718e21ee7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 20:59:59.725661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5802b7-aa24-4295-b522-9d718e21ee7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 20:59:59.727695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5802b7-aa24-4295-b522-9d718e21ee7b service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 20:59:59.748152 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1770/3583] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 20:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:00:00.211514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c269c54-f604-474a-9a66-60ebf5a53f5c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:00.213934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4c269c54-f604-474a-9a66-60ebf5a53f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:00.236776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c269c54-f604-474a-9a66-60ebf5a53f5c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:00.241808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:00.242569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:00.243002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:00.243290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] PATH_INFO: `/v3/projects/9bf7c2d797a74562993d8605bda80ae4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:00.244475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf7c2d797a74562993d8605bda80ae4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:00.246711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:00.306796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf7c2d797a74562993d8605bda80ae4', 'request_id': 'req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:00.314763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf7c2d797a74562993d8605bda80ae4', 'request_id': 'req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:00.322344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf7c2d797a74562993d8605bda80ae4', 'request_id': 'req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:00.341021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] The token cache is being invalidate because project 9bf7c2d797a74562993d8605bda80ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:00:00.341623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:00.342731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:00.344514 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1814/3584] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:00:00 2022] DELETE /identity/v3/projects/9bf7c2d797a74562993d8605bda80ae4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:00:00.350458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5802b7-aa24-4295-b522-9d718e21ee7b service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:00.364138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5802b7-aa24-4295-b522-9d718e21ee7b service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:00.379520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:00.445122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202479f-96a8-479b-b7e8-a90fa3615929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:00.446128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202479f-96a8-479b-b7e8-a90fa3615929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:00.446389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202479f-96a8-479b-b7e8-a90fa3615929 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:00.446675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202479f-96a8-479b-b7e8-a90fa3615929 admin admin] PATH_INFO: `/v3/users/7db1b70a90d34e3cb860ca5a9867d5f2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:00.447855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202479f-96a8-479b-b7e8-a90fa3615929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db1b70a90d34e3cb860ca5a9867d5f2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:00.449852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202479f-96a8-479b-b7e8-a90fa3615929 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:00.519754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e202479f-96a8-479b-b7e8-a90fa3615929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db1b70a90d34e3cb860ca5a9867d5f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:00.561766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e202479f-96a8-479b-b7e8-a90fa3615929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db1b70a90d34e3cb860ca5a9867d5f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:00.562580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e202479f-96a8-479b-b7e8-a90fa3615929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db1b70a90d34e3cb860ca5a9867d5f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:00.593638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e202479f-96a8-479b-b7e8-a90fa3615929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db1b70a90d34e3cb860ca5a9867d5f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:00.609151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e202479f-96a8-479b-b7e8-a90fa3615929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db1b70a90d34e3cb860ca5a9867d5f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:00.616870 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1771/3585] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:00:00 2022] DELETE /identity/v3/users/7db1b70a90d34e3cb860ca5a9867d5f2 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:00:01.279148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:01.281306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:01.294816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef1a2d-ce23-4de9-a501-a1bf9d98e504 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:01.370655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa41ae-64b1-47d5-963b-6e0e6ef97509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:01.371308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa41ae-64b1-47d5-963b-6e0e6ef97509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:01.371596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa41ae-64b1-47d5-963b-6e0e6ef97509 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:01.372067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa41ae-64b1-47d5-963b-6e0e6ef97509 admin admin] PATH_INFO: `/v3/projects/e941437cffc54d9b9a8b2257c0c23bd9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:01.374278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa41ae-64b1-47d5-963b-6e0e6ef97509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e941437cffc54d9b9a8b2257c0c23bd9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:01.376549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa41ae-64b1-47d5-963b-6e0e6ef97509 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:01.393418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:01.442681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-97aa41ae-64b1-47d5-963b-6e0e6ef97509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e941437cffc54d9b9a8b2257c0c23bd9', 'request_id': 'req-97aa41ae-64b1-47d5-963b-6e0e6ef97509'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:01.451407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-97aa41ae-64b1-47d5-963b-6e0e6ef97509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e941437cffc54d9b9a8b2257c0c23bd9', 'request_id': 'req-97aa41ae-64b1-47d5-963b-6e0e6ef97509'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:01.459129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-97aa41ae-64b1-47d5-963b-6e0e6ef97509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e941437cffc54d9b9a8b2257c0c23bd9', 'request_id': 'req-97aa41ae-64b1-47d5-963b-6e0e6ef97509'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:01.475813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-97aa41ae-64b1-47d5-963b-6e0e6ef97509 admin admin] The token cache is being invalidate because project e941437cffc54d9b9a8b2257c0c23bd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:00:01.476119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-97aa41ae-64b1-47d5-963b-6e0e6ef97509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:01.477217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-97aa41ae-64b1-47d5-963b-6e0e6ef97509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:01.480334 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1815/3586] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:00:01 2022] DELETE /identity/v3/projects/e941437cffc54d9b9a8b2257c0c23bd9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:00:01.488020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e275f51-e865-4e06-80f2-12603c1b448d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:01.488587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e275f51-e865-4e06-80f2-12603c1b448d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:01.488798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e275f51-e865-4e06-80f2-12603c1b448d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:01.512457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5e275f51-e865-4e06-80f2-12603c1b448d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:01.514084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5e275f51-e865-4e06-80f2-12603c1b448d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:01.596253 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1772/3587] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:01.604608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30804c4-6a4f-4ff4-ae45-32143b06fc49 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:01.604932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30804c4-6a4f-4ff4-ae45-32143b06fc49 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:01.605211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30804c4-6a4f-4ff4-ae45-32143b06fc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:01.625770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b30804c4-6a4f-4ff4-ae45-32143b06fc49 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:01.627229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b30804c4-6a4f-4ff4-ae45-32143b06fc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:01.637550 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1816/3588] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:01.642635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e275f51-e865-4e06-80f2-12603c1b448d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:01.657272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e275f51-e865-4e06-80f2-12603c1b448d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:01.664092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88de2d8-1e3f-46c5-bed0-3146d3952d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:01.665044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88de2d8-1e3f-46c5-bed0-3146d3952d76 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:01.665336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88de2d8-1e3f-46c5-bed0-3146d3952d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:01.665844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88de2d8-1e3f-46c5-bed0-3146d3952d76 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:01.666664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88de2d8-1e3f-46c5-bed0-3146d3952d76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:00:01.667319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88de2d8-1e3f-46c5-bed0-3146d3952d76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:01.669499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88de2d8-1e3f-46c5-bed0-3146d3952d76 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:01.686630 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1773/3589] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:00:01.691219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30804c4-6a4f-4ff4-ae45-32143b06fc49 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:01.701142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30804c4-6a4f-4ff4-ae45-32143b06fc49 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:01.706628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c9d53-cdcf-452e-a7c1-479743cc86dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:01.707370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c9d53-cdcf-452e-a7c1-479743cc86dd admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:01.707754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c9d53-cdcf-452e-a7c1-479743cc86dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:01.708142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c9d53-cdcf-452e-a7c1-479743cc86dd admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:01.708873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c9d53-cdcf-452e-a7c1-479743cc86dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:01.710919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c9d53-cdcf-452e-a7c1-479743cc86dd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:01.740267 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1817/3590] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:00:01 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:00:01.744395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88de2d8-1e3f-46c5-bed0-3146d3952d76 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:01.755630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88de2d8-1e3f-46c5-bed0-3146d3952d76 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:01.776456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:01.829513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fd64a-a3ae-4ad2-8710-730d6c826816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:01.830160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fd64a-a3ae-4ad2-8710-730d6c826816 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:01.830421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fd64a-a3ae-4ad2-8710-730d6c826816 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:01.830703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fd64a-a3ae-4ad2-8710-730d6c826816 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:01.831345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fd64a-a3ae-4ad2-8710-730d6c826816 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:01.833313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fd64a-a3ae-4ad2-8710-730d6c826816 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:01.857877 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1774/3591] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:00:01 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:00:01.862282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c9d53-cdcf-452e-a7c1-479743cc86dd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:01.874780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c9d53-cdcf-452e-a7c1-479743cc86dd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:01.879976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8bbfa-b0ad-4674-9ca1-c54872a1221e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:01.881280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8bbfa-b0ad-4674-9ca1-c54872a1221e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:01.881655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8bbfa-b0ad-4674-9ca1-c54872a1221e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:01.881965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8bbfa-b0ad-4674-9ca1-c54872a1221e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:01.882921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8bbfa-b0ad-4674-9ca1-c54872a1221e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:01.885022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8bbfa-b0ad-4674-9ca1-c54872a1221e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:01.898493 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1818/3592] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:00:01.902538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fd64a-a3ae-4ad2-8710-730d6c826816 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:01.918021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fd64a-a3ae-4ad2-8710-730d6c826816 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:01.926181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03d12a-5a58-4f6e-9d20-87795ee5c517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:01.927089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d12a-5a58-4f6e-9d20-87795ee5c517 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:01.927581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d12a-5a58-4f6e-9d20-87795ee5c517 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:01.928022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d12a-5a58-4f6e-9d20-87795ee5c517 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:01.929309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03d12a-5a58-4f6e-9d20-87795ee5c517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:01.931430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03d12a-5a58-4f6e-9d20-87795ee5c517 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:01.953822 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1775/3593] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:00:01.957984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8bbfa-b0ad-4674-9ca1-c54872a1221e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:01.969124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8bbfa-b0ad-4674-9ca1-c54872a1221e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:01.974362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1df2ff-53ed-4b96-9cdd-f948fbb6edc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:01.975324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1df2ff-53ed-4b96-9cdd-f948fbb6edc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:01.975593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1df2ff-53ed-4b96-9cdd-f948fbb6edc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:01.976046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1df2ff-53ed-4b96-9cdd-f948fbb6edc1 admin admin] PATH_INFO: `/v3/projects/f9175f0f4d204936aed9f62d91c141f0/users/a1e643d2a39441cda1035502d08d909a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:01.990784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1df2ff-53ed-4b96-9cdd-f948fbb6edc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9175f0f4d204936aed9f62d91c141f0, user_id=a1e643d2a39441cda1035502d08d909a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:01.993554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1df2ff-53ed-4b96-9cdd-f948fbb6edc1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:02.006829 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1819/3594] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:00:01 2022] PUT /identity/v3/projects/f9175f0f4d204936aed9f62d91c141f0/users/a1e643d2a39441cda1035502d08d909a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:00:02.019554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e13ee8-a4a9-4474-9bf2-69c538263c3b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:02.019931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e13ee8-a4a9-4474-9bf2-69c538263c3b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:02.020283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e13ee8-a4a9-4474-9bf2-69c538263c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:02.048052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-89e13ee8-a4a9-4474-9bf2-69c538263c3b None None] MFA Rules not processed for user `a1e643d2a39441cda1035502d08d909a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:02.049344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-89e13ee8-a4a9-4474-9bf2-69c538263c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:02.133772 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1776/3595] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:02.138313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1df2ff-53ed-4b96-9cdd-f948fbb6edc1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:02.205238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1df2ff-53ed-4b96-9cdd-f948fbb6edc1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:02.210173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f6330-9299-4961-8730-c41abe220452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:02.210923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f6330-9299-4961-8730-c41abe220452 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:02.211289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f6330-9299-4961-8730-c41abe220452 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:02.211636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f6330-9299-4961-8730-c41abe220452 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:02.212348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f6330-9299-4961-8730-c41abe220452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:02.214529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f6330-9299-4961-8730-c41abe220452 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:02.249990 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1820/3596] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:00:02 2022] POST /identity/v3/projects => generated 393 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:00:02.257220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e13ee8-a4a9-4474-9bf2-69c538263c3b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:02.272583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e13ee8-a4a9-4474-9bf2-69c538263c3b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:02.362493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a690c5d-b22a-44b9-8c81-4589fedef08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:02.363123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a690c5d-b22a-44b9-8c81-4589fedef08a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:02.363411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a690c5d-b22a-44b9-8c81-4589fedef08a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:02.363714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a690c5d-b22a-44b9-8c81-4589fedef08a admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:02.364477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a690c5d-b22a-44b9-8c81-4589fedef08a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:02.367099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a690c5d-b22a-44b9-8c81-4589fedef08a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:02.404940 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1777/3597] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:00:02 2022] POST /identity/v3/users => generated 303 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:00:02.407686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f6330-9299-4961-8730-c41abe220452 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:02.421449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f6330-9299-4961-8730-c41abe220452 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:02.427414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c779d8a-3865-4f4c-8cb0-58544475aff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:02.428138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c779d8a-3865-4f4c-8cb0-58544475aff2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:02.428531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c779d8a-3865-4f4c-8cb0-58544475aff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:02.428938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c779d8a-3865-4f4c-8cb0-58544475aff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:02.429698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c779d8a-3865-4f4c-8cb0-58544475aff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:02.431427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c779d8a-3865-4f4c-8cb0-58544475aff2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:02.446440 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1821/3598] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:00:02.451017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a690c5d-b22a-44b9-8c81-4589fedef08a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:02.468703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a690c5d-b22a-44b9-8c81-4589fedef08a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:02.478806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d750fd2-566a-4076-9d6a-8aeddd14a77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:02.481246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d750fd2-566a-4076-9d6a-8aeddd14a77f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:02.482221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d750fd2-566a-4076-9d6a-8aeddd14a77f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:02.483796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d750fd2-566a-4076-9d6a-8aeddd14a77f admin admin] PATH_INFO: `/v3/projects/0d05514342ed4dc383c025be27b5808b/users/538736eb093546e1a81e2b4c3e537166/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:02.509362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d750fd2-566a-4076-9d6a-8aeddd14a77f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d05514342ed4dc383c025be27b5808b, user_id=538736eb093546e1a81e2b4c3e537166, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:02.512005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d750fd2-566a-4076-9d6a-8aeddd14a77f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:02.515345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:02.533352 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1778/3599] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:00:02 2022] PUT /identity/v3/projects/0d05514342ed4dc383c025be27b5808b/users/538736eb093546e1a81e2b4c3e537166/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:00:02.536926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c779d8a-3865-4f4c-8cb0-58544475aff2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:02.549323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c779d8a-3865-4f4c-8cb0-58544475aff2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:02.586979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:02.651791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a013f94-f4ab-4110-bdc2-6600554681e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:02.652871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a013f94-f4ab-4110-bdc2-6600554681e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:02.653284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a013f94-f4ab-4110-bdc2-6600554681e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:02.653635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a013f94-f4ab-4110-bdc2-6600554681e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:02.654404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a013f94-f4ab-4110-bdc2-6600554681e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:02.657455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a013f94-f4ab-4110-bdc2-6600554681e7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:02.671844 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1822/3600] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:00:02.676204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d750fd2-566a-4076-9d6a-8aeddd14a77f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:02.688423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d750fd2-566a-4076-9d6a-8aeddd14a77f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:02.694551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf28d80-b3ba-4956-9350-82a1be39d558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:02.695467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf28d80-b3ba-4956-9350-82a1be39d558 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:02.695822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf28d80-b3ba-4956-9350-82a1be39d558 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:02.696180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf28d80-b3ba-4956-9350-82a1be39d558 admin admin] PATH_INFO: `/v3/projects/0d05514342ed4dc383c025be27b5808b/users/538736eb093546e1a81e2b4c3e537166/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:02.698539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf28d80-b3ba-4956-9350-82a1be39d558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d05514342ed4dc383c025be27b5808b, user_id=538736eb093546e1a81e2b4c3e537166, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:02.701120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf28d80-b3ba-4956-9350-82a1be39d558 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:02.721470 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1779/3601] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:00:02 2022] PUT /identity/v3/projects/0d05514342ed4dc383c025be27b5808b/users/538736eb093546e1a81e2b4c3e537166/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:00:02.732039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b23a21-b537-4b74-99d1-0b75a8a7c24d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:02.732354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b23a21-b537-4b74-99d1-0b75a8a7c24d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:02.732761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b23a21-b537-4b74-99d1-0b75a8a7c24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:02.771992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a2b23a21-b537-4b74-99d1-0b75a8a7c24d None None] MFA Rules not processed for user `538736eb093546e1a81e2b4c3e537166`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:02.774577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a2b23a21-b537-4b74-99d1-0b75a8a7c24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:02.861347 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1823/3602] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:02.874954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda083ce-cd1e-47a8-88ed-a89106faa321 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:02.875485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda083ce-cd1e-47a8-88ed-a89106faa321 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:02.875766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda083ce-cd1e-47a8-88ed-a89106faa321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:02.897453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-eda083ce-cd1e-47a8-88ed-a89106faa321 None None] MFA Rules not processed for user `a1e643d2a39441cda1035502d08d909a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:02.898976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-eda083ce-cd1e-47a8-88ed-a89106faa321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:02.958889 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1780/3603] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:02.971389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b23a21-b537-4b74-99d1-0b75a8a7c24d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:02.974260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a2b23a21-b537-4b74-99d1-0b75a8a7c24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:03.084613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b23a21-b537-4b74-99d1-0b75a8a7c24d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:03.094474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8227f-c4f3-4e59-850c-6b80e9328488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:03.095314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8227f-c4f3-4e59-850c-6b80e9328488 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:03.095887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8227f-c4f3-4e59-850c-6b80e9328488 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:03.096164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8227f-c4f3-4e59-850c-6b80e9328488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:03.108376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8227f-c4f3-4e59-850c-6b80e9328488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:03.111761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8227f-c4f3-4e59-850c-6b80e9328488 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:03.130900 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1824/3604] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:00:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:00:08.020777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa0d7c-9795-430e-a2e2-1a168ad60548 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:08.021278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa0d7c-9795-430e-a2e2-1a168ad60548 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:08.021767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa0d7c-9795-430e-a2e2-1a168ad60548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:08.027159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8227f-c4f3-4e59-850c-6b80e9328488 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:08.029270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ecd8227f-c4f3-4e59-850c-6b80e9328488 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:08.057149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:08.076619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3afa0d7c-9795-430e-a2e2-1a168ad60548 None None] MFA Rules not processed for user `e6970a72952e4d4a8f26e0f8ee23f364`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:08.078384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3afa0d7c-9795-430e-a2e2-1a168ad60548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:08.166984 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1781/3605] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:00:08.167695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa0d7c-9795-430e-a2e2-1a168ad60548 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:08.169189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3afa0d7c-9795-430e-a2e2-1a168ad60548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:08.179292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8227f-c4f3-4e59-850c-6b80e9328488 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:08.185735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e12fb-19f3-4a30-9ac2-206cc7ed0b80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:08.186987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e12fb-19f3-4a30-9ac2-206cc7ed0b80 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:08.186987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e12fb-19f3-4a30-9ac2-206cc7ed0b80 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:08.187213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e12fb-19f3-4a30-9ac2-206cc7ed0b80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:08.189074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-408e12fb-19f3-4a30-9ac2-206cc7ed0b80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:08.210789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e12fb-19f3-4a30-9ac2-206cc7ed0b80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:08.222387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e12fb-19f3-4a30-9ac2-206cc7ed0b80 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:08.241298 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1825/3606] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:00:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:00:08.297797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa0d7c-9795-430e-a2e2-1a168ad60548 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:08.302217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44fa2c-a9cb-46bc-a83a-dab9fefe4a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:08.303006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44fa2c-a9cb-46bc-a83a-dab9fefe4a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:08.303296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44fa2c-a9cb-46bc-a83a-dab9fefe4a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:08.303583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44fa2c-a9cb-46bc-a83a-dab9fefe4a79 admin admin] PATH_INFO: `/v3/users/6b25b05106404b64adc5476924f2e69d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:08.305048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44fa2c-a9cb-46bc-a83a-dab9fefe4a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b25b05106404b64adc5476924f2e69d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:08.307091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44fa2c-a9cb-46bc-a83a-dab9fefe4a79 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:08.350764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:08.371492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d44fa2c-a9cb-46bc-a83a-dab9fefe4a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b25b05106404b64adc5476924f2e69d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:08.395037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d44fa2c-a9cb-46bc-a83a-dab9fefe4a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b25b05106404b64adc5476924f2e69d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:08.395863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d44fa2c-a9cb-46bc-a83a-dab9fefe4a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b25b05106404b64adc5476924f2e69d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:08.407665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d44fa2c-a9cb-46bc-a83a-dab9fefe4a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b25b05106404b64adc5476924f2e69d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:08.425866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d44fa2c-a9cb-46bc-a83a-dab9fefe4a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b25b05106404b64adc5476924f2e69d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:08.443857 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1782/3607] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:00:08 2022] DELETE /identity/v3/users/6b25b05106404b64adc5476924f2e69d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:00:08.917818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e12fb-19f3-4a30-9ac2-206cc7ed0b80 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:08.930314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e12fb-19f3-4a30-9ac2-206cc7ed0b80 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:09.034768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903244f-ed78-4f47-9e55-93c392840261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:09.036316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903244f-ed78-4f47-9e55-93c392840261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:09.036869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903244f-ed78-4f47-9e55-93c392840261 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:09.037402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903244f-ed78-4f47-9e55-93c392840261 admin admin] PATH_INFO: `/v3/projects/f5cdab592b0646aaa02deff957842bc0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:09.040765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903244f-ed78-4f47-9e55-93c392840261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5cdab592b0646aaa02deff957842bc0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:09.044608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903244f-ed78-4f47-9e55-93c392840261 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:09.127205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a903244f-ed78-4f47-9e55-93c392840261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5cdab592b0646aaa02deff957842bc0', 'request_id': 'req-a903244f-ed78-4f47-9e55-93c392840261'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:09.193956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a903244f-ed78-4f47-9e55-93c392840261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5cdab592b0646aaa02deff957842bc0', 'request_id': 'req-a903244f-ed78-4f47-9e55-93c392840261'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:09.210006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a903244f-ed78-4f47-9e55-93c392840261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5cdab592b0646aaa02deff957842bc0', 'request_id': 'req-a903244f-ed78-4f47-9e55-93c392840261'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:09.237212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a903244f-ed78-4f47-9e55-93c392840261 admin admin] The token cache is being invalidate because project f5cdab592b0646aaa02deff957842bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:00:09.237808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a903244f-ed78-4f47-9e55-93c392840261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:09.238655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a903244f-ed78-4f47-9e55-93c392840261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:09.246855 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1826/3608] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:00:08 2022] DELETE /identity/v3/projects/f5cdab592b0646aaa02deff957842bc0 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:00:09.247794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef72fcf-3554-4a07-b238-7683e37c82c5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:09.248321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef72fcf-3554-4a07-b238-7683e37c82c5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:09.248413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef72fcf-3554-4a07-b238-7683e37c82c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:09.298086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-aef72fcf-3554-4a07-b238-7683e37c82c5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:09.299354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-aef72fcf-3554-4a07-b238-7683e37c82c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:09.408442 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1783/3609] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:09.416373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f96a2-d996-4182-8c22-3aa1814ac235 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:09.416879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f96a2-d996-4182-8c22-3aa1814ac235 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:09.417140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f96a2-d996-4182-8c22-3aa1814ac235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:09.438613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-052f96a2-d996-4182-8c22-3aa1814ac235 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:09.441937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-052f96a2-d996-4182-8c22-3aa1814ac235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:09.454304 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1827/3610] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:09.455523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef72fcf-3554-4a07-b238-7683e37c82c5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:09.465965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef72fcf-3554-4a07-b238-7683e37c82c5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:09.476104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be38a64-aaa3-41e7-8065-e3a10788be2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:09.476778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be38a64-aaa3-41e7-8065-e3a10788be2e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:09.477133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be38a64-aaa3-41e7-8065-e3a10788be2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:09.477453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be38a64-aaa3-41e7-8065-e3a10788be2e admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:09.478166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be38a64-aaa3-41e7-8065-e3a10788be2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:00:09.478497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be38a64-aaa3-41e7-8065-e3a10788be2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:09.480209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be38a64-aaa3-41e7-8065-e3a10788be2e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:09.497277 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1784/3611] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:00:09.501765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f96a2-d996-4182-8c22-3aa1814ac235 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:09.512981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f96a2-d996-4182-8c22-3aa1814ac235 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:09.519172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c96e7f-8bcb-4548-bf14-79ce2001f580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:09.519849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c96e7f-8bcb-4548-bf14-79ce2001f580 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:09.520241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c96e7f-8bcb-4548-bf14-79ce2001f580 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:09.520634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c96e7f-8bcb-4548-bf14-79ce2001f580 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:09.521473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c96e7f-8bcb-4548-bf14-79ce2001f580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:09.523738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c96e7f-8bcb-4548-bf14-79ce2001f580 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:09.560962 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1828/3612] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:00:09 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:00:09.563228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be38a64-aaa3-41e7-8065-e3a10788be2e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:09.576102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be38a64-aaa3-41e7-8065-e3a10788be2e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:09.638549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bf13d-b1f1-4ba1-b8f5-3d7b31a90d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:09.639849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bf13d-b1f1-4ba1-b8f5-3d7b31a90d66 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:09.640135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bf13d-b1f1-4ba1-b8f5-3d7b31a90d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:09.640410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bf13d-b1f1-4ba1-b8f5-3d7b31a90d66 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:09.641084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bf13d-b1f1-4ba1-b8f5-3d7b31a90d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:09.643088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bf13d-b1f1-4ba1-b8f5-3d7b31a90d66 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:09.667671 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1785/3613] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:00:09 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:00:09.669188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c96e7f-8bcb-4548-bf14-79ce2001f580 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:09.680846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c96e7f-8bcb-4548-bf14-79ce2001f580 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:09.688990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e51fa-9945-4605-83fe-611b7d9e26d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:09.690514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e51fa-9945-4605-83fe-611b7d9e26d1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:09.690909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e51fa-9945-4605-83fe-611b7d9e26d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:09.691400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e51fa-9945-4605-83fe-611b7d9e26d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:09.692318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e51fa-9945-4605-83fe-611b7d9e26d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:09.694305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e51fa-9945-4605-83fe-611b7d9e26d1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:09.697093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:09.709789 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1829/3614] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:00:09.710964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bf13d-b1f1-4ba1-b8f5-3d7b31a90d66 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:09.721589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bf13d-b1f1-4ba1-b8f5-3d7b31a90d66 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:09.726703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42b8e7-b769-4049-ab77-609c0a8ba590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:09.727343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42b8e7-b769-4049-ab77-609c0a8ba590 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:09.727627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42b8e7-b769-4049-ab77-609c0a8ba590 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:09.727906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42b8e7-b769-4049-ab77-609c0a8ba590 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:09.728586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42b8e7-b769-4049-ab77-609c0a8ba590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:09.730385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42b8e7-b769-4049-ab77-609c0a8ba590 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:09.741585 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1786/3615] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:00:09.746020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e51fa-9945-4605-83fe-611b7d9e26d1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:09.757961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e51fa-9945-4605-83fe-611b7d9e26d1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:09.762813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7135d38-a556-4d24-9d54-08f462b49cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:09.763483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7135d38-a556-4d24-9d54-08f462b49cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:09.763836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7135d38-a556-4d24-9d54-08f462b49cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:09.764680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7135d38-a556-4d24-9d54-08f462b49cb4 admin admin] PATH_INFO: `/v3/projects/7eef34f8ef5e4e73802caff2f4e5a023/users/3db58915ddb1452490f90c8f2a583220/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:09.779796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7135d38-a556-4d24-9d54-08f462b49cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eef34f8ef5e4e73802caff2f4e5a023, user_id=3db58915ddb1452490f90c8f2a583220, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:09.782434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7135d38-a556-4d24-9d54-08f462b49cb4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:09.800565 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1830/3616] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:00:09 2022] PUT /identity/v3/projects/7eef34f8ef5e4e73802caff2f4e5a023/users/3db58915ddb1452490f90c8f2a583220/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:00:09.811535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e28bf-9ccc-43b9-9ef9-b10d8d08c2ba None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:09.811852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e28bf-9ccc-43b9-9ef9-b10d8d08c2ba None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:09.812104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e28bf-9ccc-43b9-9ef9-b10d8d08c2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:09.850036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c75e28bf-9ccc-43b9-9ef9-b10d8d08c2ba None None] MFA Rules not processed for user `3db58915ddb1452490f90c8f2a583220`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:09.851460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c75e28bf-9ccc-43b9-9ef9-b10d8d08c2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:09.858724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:09.939161 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1787/3617] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:09.950021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38af4f-05bb-480f-8022-c6ffda08f4a5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:09.950323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38af4f-05bb-480f-8022-c6ffda08f4a5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:09.950608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38af4f-05bb-480f-8022-c6ffda08f4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:09.971773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8b38af4f-05bb-480f-8022-c6ffda08f4a5 None None] MFA Rules not processed for user `3db58915ddb1452490f90c8f2a583220`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:09.973117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8b38af4f-05bb-480f-8022-c6ffda08f4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:09.980646 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1831/3618] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:09.991615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e28bf-9ccc-43b9-9ef9-b10d8d08c2ba None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:09.993124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c75e28bf-9ccc-43b9-9ef9-b10d8d08c2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:10.070770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e28bf-9ccc-43b9-9ef9-b10d8d08c2ba None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:10.077195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5a76c-97a1-4d62-bba8-8b208f01a95b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:10.077799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5a76c-97a1-4d62-bba8-8b208f01a95b service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:10.078077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5a76c-97a1-4d62-bba8-8b208f01a95b service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:10.078348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5a76c-97a1-4d62-bba8-8b208f01a95b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:10.086316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5a76c-97a1-4d62-bba8-8b208f01a95b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:10.087941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5a76c-97a1-4d62-bba8-8b208f01a95b service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:10.097877 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1788/3619] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:00:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:00:10.377158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da3929-d217-4752-ac41-ea5d2137657f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:10.377614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da3929-d217-4752-ac41-ea5d2137657f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:10.377911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da3929-d217-4752-ac41-ea5d2137657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:10.397844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-57da3929-d217-4752-ac41-ea5d2137657f None None] MFA Rules not processed for user `3db58915ddb1452490f90c8f2a583220`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:10.399069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-57da3929-d217-4752-ac41-ea5d2137657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:10.412382 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1832/3620] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:20.093150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=59 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:20.392108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=17,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:24.460747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5a76c-97a1-4d62-bba8-8b208f01a95b service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:24.463379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-84d5a76c-97a1-4d62-bba8-8b208f01a95b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:24.495132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5a76c-97a1-4d62-bba8-8b208f01a95b service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:24.501680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb0aaa-3de8-4812-abea-d675365037b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:24.502328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb0aaa-3de8-4812-abea-d675365037b3 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:24.502609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb0aaa-3de8-4812-abea-d675365037b3 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:24.502902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb0aaa-3de8-4812-abea-d675365037b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:24.504638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3bbb0aaa-3de8-4812-abea-d675365037b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:24.588581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb0aaa-3de8-4812-abea-d675365037b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:24.590249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb0aaa-3de8-4812-abea-d675365037b3 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:24.602306 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1789/3621] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:00:24 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:00:24.630024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f98f9-e66b-461e-97c0-338ad5b0007d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:24.630553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f98f9-e66b-461e-97c0-338ad5b0007d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:24.630851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f98f9-e66b-461e-97c0-338ad5b0007d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:24.652152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a38f98f9-e66b-461e-97c0-338ad5b0007d None None] MFA Rules not processed for user `a1e643d2a39441cda1035502d08d909a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:24.653421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a38f98f9-e66b-461e-97c0-338ad5b0007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:24.717710 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1833/3622] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:25.280400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb0aaa-3de8-4812-abea-d675365037b3 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:25.281848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3bbb0aaa-3de8-4812-abea-d675365037b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:25.364031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb0aaa-3de8-4812-abea-d675365037b3 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:25.369203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351b131-2023-4a2d-a271-4cae65572f43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:25.369831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351b131-2023-4a2d-a271-4cae65572f43 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:25.370119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351b131-2023-4a2d-a271-4cae65572f43 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:25.370492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351b131-2023-4a2d-a271-4cae65572f43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:25.372425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e351b131-2023-4a2d-a271-4cae65572f43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:25.450589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351b131-2023-4a2d-a271-4cae65572f43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:25.452377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351b131-2023-4a2d-a271-4cae65572f43 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:25.464196 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1790/3623] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:00:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:00:29.827906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f98f9-e66b-461e-97c0-338ad5b0007d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:29.829514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a38f98f9-e66b-461e-97c0-338ad5b0007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:29.906618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f98f9-e66b-461e-97c0-338ad5b0007d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:29.911402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804e51d-46a2-43cc-89aa-304b1d466755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:29.912039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804e51d-46a2-43cc-89aa-304b1d466755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:29.912320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804e51d-46a2-43cc-89aa-304b1d466755 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:29.912599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804e51d-46a2-43cc-89aa-304b1d466755 admin admin] PATH_INFO: `/v3/users/322d75aacb244323b79f7f0e2ac464b8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:29.913743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804e51d-46a2-43cc-89aa-304b1d466755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=322d75aacb244323b79f7f0e2ac464b8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:29.915717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804e51d-46a2-43cc-89aa-304b1d466755 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:29.965418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a804e51d-46a2-43cc-89aa-304b1d466755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '322d75aacb244323b79f7f0e2ac464b8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:29.978114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a804e51d-46a2-43cc-89aa-304b1d466755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '322d75aacb244323b79f7f0e2ac464b8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:29.986716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a804e51d-46a2-43cc-89aa-304b1d466755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '322d75aacb244323b79f7f0e2ac464b8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:29.987507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a804e51d-46a2-43cc-89aa-304b1d466755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '322d75aacb244323b79f7f0e2ac464b8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:29.988148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a804e51d-46a2-43cc-89aa-304b1d466755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '322d75aacb244323b79f7f0e2ac464b8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:30.001574 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1834/3624] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:00:29 2022] DELETE /identity/v3/users/322d75aacb244323b79f7f0e2ac464b8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:00:30.287504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351b131-2023-4a2d-a271-4cae65572f43 service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:30.301752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351b131-2023-4a2d-a271-4cae65572f43 service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:30.350117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:30.363299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:30.363941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:30.365147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:30.365607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d admin admin] PATH_INFO: `/v3/projects/fb7084c9131a4f77b9688e10cd4c84de` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:30.367311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb7084c9131a4f77b9688e10cd4c84de)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:30.370462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:30.452645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb7084c9131a4f77b9688e10cd4c84de', 'request_id': 'req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:30.464364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb7084c9131a4f77b9688e10cd4c84de', 'request_id': 'req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:30.483983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb7084c9131a4f77b9688e10cd4c84de', 'request_id': 'req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:30.497580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d admin admin] The token cache is being invalidate because project fb7084c9131a4f77b9688e10cd4c84de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:00:30.498418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:30.499113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0503da3b-6ecc-45b7-aaa2-0b9c83b0187d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:30.500651 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1791/3625] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:00:30 2022] DELETE /identity/v3/projects/fb7084c9131a4f77b9688e10cd4c84de => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:00:30.506015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e6e31-abbe-4625-ac51-6fe6fb3c483d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:30.506171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e6e31-abbe-4625-ac51-6fe6fb3c483d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:30.506357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e6e31-abbe-4625-ac51-6fe6fb3c483d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:30.548387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-583e6e31-abbe-4625-ac51-6fe6fb3c483d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:30.549841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-583e6e31-abbe-4625-ac51-6fe6fb3c483d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:30.572971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:30.618249 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1835/3626] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:30.626531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78896d8-927d-4acb-86ad-7ed46acb0855 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:30.626850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78896d8-927d-4acb-86ad-7ed46acb0855 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:30.627135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78896d8-927d-4acb-86ad-7ed46acb0855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:30.648425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a78896d8-927d-4acb-86ad-7ed46acb0855 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:30.650418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a78896d8-927d-4acb-86ad-7ed46acb0855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:30.663303 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1792/3627] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:00:30.667325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e6e31-abbe-4625-ac51-6fe6fb3c483d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:30.681966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e6e31-abbe-4625-ac51-6fe6fb3c483d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:30.686794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fba50-cfcd-40b4-ba75-ea10dca9060e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:30.687486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fba50-cfcd-40b4-ba75-ea10dca9060e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:30.687844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fba50-cfcd-40b4-ba75-ea10dca9060e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:30.688195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fba50-cfcd-40b4-ba75-ea10dca9060e admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:30.688953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fba50-cfcd-40b4-ba75-ea10dca9060e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:00:30.689337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fba50-cfcd-40b4-ba75-ea10dca9060e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:30.691795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fba50-cfcd-40b4-ba75-ea10dca9060e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:30.706265 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1836/3628] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:00:30.710515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78896d8-927d-4acb-86ad-7ed46acb0855 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:30.720037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78896d8-927d-4acb-86ad-7ed46acb0855 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:30.725453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f943a-2d4d-47f7-8331-1c0cb18b35ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:30.726313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f943a-2d4d-47f7-8331-1c0cb18b35ad admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:30.726733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f943a-2d4d-47f7-8331-1c0cb18b35ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:30.727117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f943a-2d4d-47f7-8331-1c0cb18b35ad admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:30.727873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f943a-2d4d-47f7-8331-1c0cb18b35ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:30.730102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f943a-2d4d-47f7-8331-1c0cb18b35ad admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:30.760332 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1793/3629] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:00:30 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:00:30.765248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fba50-cfcd-40b4-ba75-ea10dca9060e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:30.775653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fba50-cfcd-40b4-ba75-ea10dca9060e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:30.842648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e071d-084f-42a3-b09d-350e67453f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:30.843590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e071d-084f-42a3-b09d-350e67453f36 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:30.843886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e071d-084f-42a3-b09d-350e67453f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:30.844173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e071d-084f-42a3-b09d-350e67453f36 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:30.844816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e071d-084f-42a3-b09d-350e67453f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:30.847390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e071d-084f-42a3-b09d-350e67453f36 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:30.873816 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1837/3630] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:00:30 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:00:30.877871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f943a-2d4d-47f7-8331-1c0cb18b35ad admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:30.889390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f943a-2d4d-47f7-8331-1c0cb18b35ad admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:30.894536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06de992-1949-4f0a-8f6c-ddc887728244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:30.895234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06de992-1949-4f0a-8f6c-ddc887728244 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:30.895529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06de992-1949-4f0a-8f6c-ddc887728244 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:30.895814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06de992-1949-4f0a-8f6c-ddc887728244 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:30.896520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06de992-1949-4f0a-8f6c-ddc887728244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:30.898201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06de992-1949-4f0a-8f6c-ddc887728244 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:30.924548 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1794/3631] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:00:30.929427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e071d-084f-42a3-b09d-350e67453f36 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:30.941839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e071d-084f-42a3-b09d-350e67453f36 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:30.946910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ceebf-be87-458b-bf0a-91d81a302e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:30.947590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ceebf-be87-458b-bf0a-91d81a302e53 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:30.947948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ceebf-be87-458b-bf0a-91d81a302e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:30.948355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ceebf-be87-458b-bf0a-91d81a302e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:30.949132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ceebf-be87-458b-bf0a-91d81a302e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:30.951076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ceebf-be87-458b-bf0a-91d81a302e53 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:30.975351 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1838/3632] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:00:30.976532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06de992-1949-4f0a-8f6c-ddc887728244 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:30.987466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06de992-1949-4f0a-8f6c-ddc887728244 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:30.992276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac33d7f-4a9a-4992-b302-776f04cabc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:30.992995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac33d7f-4a9a-4992-b302-776f04cabc16 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:30.993289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac33d7f-4a9a-4992-b302-776f04cabc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:30.993570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac33d7f-4a9a-4992-b302-776f04cabc16 admin admin] PATH_INFO: `/v3/projects/146fb657f35e47de8fbad6cacc39f501/users/8923b1af5fd54c31ac0af85564cfdd92/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:31.009306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac33d7f-4a9a-4992-b302-776f04cabc16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=146fb657f35e47de8fbad6cacc39f501, user_id=8923b1af5fd54c31ac0af85564cfdd92, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:31.011985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac33d7f-4a9a-4992-b302-776f04cabc16 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:31.025414 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1795/3633] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:00:30 2022] PUT /identity/v3/projects/146fb657f35e47de8fbad6cacc39f501/users/8923b1af5fd54c31ac0af85564cfdd92/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:00:31.030560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25b906-e889-45d4-8d3c-929e5c0ff3dc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:31.030853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25b906-e889-45d4-8d3c-929e5c0ff3dc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:31.031137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25b906-e889-45d4-8d3c-929e5c0ff3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:31.050567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0f25b906-e889-45d4-8d3c-929e5c0ff3dc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:31.051771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0f25b906-e889-45d4-8d3c-929e5c0ff3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:31.113340 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1839/3634] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:00:31.123817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac33d7f-4a9a-4992-b302-776f04cabc16 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:31.126338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4ac33d7f-4a9a-4992-b302-776f04cabc16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:31.187078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac33d7f-4a9a-4992-b302-776f04cabc16 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:31.191205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09297e70-f9f1-4aeb-aeaf-90434519eaa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:31.192183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09297e70-f9f1-4aeb-aeaf-90434519eaa6 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:31.192183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09297e70-f9f1-4aeb-aeaf-90434519eaa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:31.192402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09297e70-f9f1-4aeb-aeaf-90434519eaa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:31.202066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09297e70-f9f1-4aeb-aeaf-90434519eaa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:31.203507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09297e70-f9f1-4aeb-aeaf-90434519eaa6 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:31.214022 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1796/3635] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:00:37.426872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc1a2a-5949-48ff-ac35-dd772b61adb9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:37.427941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc1a2a-5949-48ff-ac35-dd772b61adb9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:37.428447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc1a2a-5949-48ff-ac35-dd772b61adb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:37.464444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a8cc1a2a-5949-48ff-ac35-dd772b61adb9 None None] MFA Rules not processed for user `8923b1af5fd54c31ac0af85564cfdd92`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:37.465975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a8cc1a2a-5949-48ff-ac35-dd772b61adb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:37.500401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:37.581048 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1840/3636] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:37.590599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09297e70-f9f1-4aeb-aeaf-90434519eaa6 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:37.592684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-09297e70-f9f1-4aeb-aeaf-90434519eaa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:37.614270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:37.671584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09297e70-f9f1-4aeb-aeaf-90434519eaa6 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:37.677122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8646a-77d6-40fa-b08d-65324f03f4ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:37.677731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8646a-77d6-40fa-b08d-65324f03f4ad service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:37.678016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8646a-77d6-40fa-b08d-65324f03f4ad service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:37.678295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8646a-77d6-40fa-b08d-65324f03f4ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:37.686046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8646a-77d6-40fa-b08d-65324f03f4ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:37.687616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8646a-77d6-40fa-b08d-65324f03f4ad service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:37.696327 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1797/3637] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:00:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:00:39.394320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02398b-fd4f-4605-bd20-8464c419e31a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:39.394923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02398b-fd4f-4605-bd20-8464c419e31a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:39.395022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02398b-fd4f-4605-bd20-8464c419e31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:39.425171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4f02398b-fd4f-4605-bd20-8464c419e31a None None] MFA Rules not processed for user `8923b1af5fd54c31ac0af85564cfdd92`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:00:39.426667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4f02398b-fd4f-4605-bd20-8464c419e31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:39.453201 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1841/3638] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:00:39.837566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8646a-77d6-40fa-b08d-65324f03f4ad service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:39.851041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8646a-77d6-40fa-b08d-65324f03f4ad service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:39.855789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289ecd5-7b46-4449-a35a-139c9dd7e2d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:39.856730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289ecd5-7b46-4449-a35a-139c9dd7e2d4 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:39.857700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289ecd5-7b46-4449-a35a-139c9dd7e2d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:39.858126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289ecd5-7b46-4449-a35a-139c9dd7e2d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:39.862523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7289ecd5-7b46-4449-a35a-139c9dd7e2d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:39.875351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289ecd5-7b46-4449-a35a-139c9dd7e2d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:39.876875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289ecd5-7b46-4449-a35a-139c9dd7e2d4 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:39.887280 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1798/3639] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:00:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:00:40.763749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02398b-fd4f-4605-bd20-8464c419e31a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:40.765571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4f02398b-fd4f-4605-bd20-8464c419e31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:40.798166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02398b-fd4f-4605-bd20-8464c419e31a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:40.804077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40615b58-9e0e-46ee-9d03-49a631822b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:40.804704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40615b58-9e0e-46ee-9d03-49a631822b88 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:40.805139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40615b58-9e0e-46ee-9d03-49a631822b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:40.805424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40615b58-9e0e-46ee-9d03-49a631822b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:40.807210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-40615b58-9e0e-46ee-9d03-49a631822b88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:40.840383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40615b58-9e0e-46ee-9d03-49a631822b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:40.843827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40615b58-9e0e-46ee-9d03-49a631822b88 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:40.889854 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1842/3640] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:00:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:00:49.883889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=32 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:50.868510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=27 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:00:59.816745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289ecd5-7b46-4449-a35a-139c9dd7e2d4 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:00:59.818374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7289ecd5-7b46-4449-a35a-139c9dd7e2d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:00:59.835862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289ecd5-7b46-4449-a35a-139c9dd7e2d4 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:00:59.841302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:00:59.842324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:59.842716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:59.843096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] PATH_INFO: `/v3/users/a1e643d2a39441cda1035502d08d909a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:00:59.844357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1e643d2a39441cda1035502d08d909a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:00:59.846732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:00:59.909266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1e643d2a39441cda1035502d08d909a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:59.927124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1e643d2a39441cda1035502d08d909a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:59.927881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1e643d2a39441cda1035502d08d909a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:59.938011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1e643d2a39441cda1035502d08d909a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:59.946809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1e643d2a39441cda1035502d08d909a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:00:59.966786 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1799/3641] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:00:59 2022] DELETE /identity/v3/users/a1e643d2a39441cda1035502d08d909a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:00:59.971827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38bdf9-2297-4037-9bc4-1a94a0953f14 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:00:59.972417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38bdf9-2297-4037-9bc4-1a94a0953f14 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:00:59.973339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38bdf9-2297-4037-9bc4-1a94a0953f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:00.003895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4d38bdf9-2297-4037-9bc4-1a94a0953f14 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:00.005791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4d38bdf9-2297-4037-9bc4-1a94a0953f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:00.080519 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1843/3642] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:00.512050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:00.515971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:00.597417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd86867-61c4-45d5-9f12-659ba8d8753f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:00.616805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7f826-48dd-474a-9d29-793d28f4585e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:00.617985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f826-48dd-474a-9d29-793d28f4585e service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:00.618283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f826-48dd-474a-9d29-793d28f4585e service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:00.618568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f826-48dd-474a-9d29-793d28f4585e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:00.647760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7f826-48dd-474a-9d29-793d28f4585e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:00.653996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7f826-48dd-474a-9d29-793d28f4585e service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:00.679980 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1800/3643] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:01.079301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38bdf9-2297-4037-9bc4-1a94a0953f14 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:01.080780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4d38bdf9-2297-4037-9bc4-1a94a0953f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:01.109481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38bdf9-2297-4037-9bc4-1a94a0953f14 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:01.116924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:01.117731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:01.118073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:01.118360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] PATH_INFO: `/v3/projects/f9175f0f4d204936aed9f62d91c141f0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:01.126090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9175f0f4d204936aed9f62d91c141f0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:01.128841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:01.214029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9175f0f4d204936aed9f62d91c141f0', 'request_id': 'req-66e3212a-d9ee-4072-9d8c-8019752eaf7b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.226909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9175f0f4d204936aed9f62d91c141f0', 'request_id': 'req-66e3212a-d9ee-4072-9d8c-8019752eaf7b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.236330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9175f0f4d204936aed9f62d91c141f0', 'request_id': 'req-66e3212a-d9ee-4072-9d8c-8019752eaf7b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.260440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] The token cache is being invalidate because project f9175f0f4d204936aed9f62d91c141f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:01:01.260744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.261504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.263695 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1844/3644] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:01:01 2022] DELETE /identity/v3/projects/f9175f0f4d204936aed9f62d91c141f0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:01.268112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7f826-48dd-474a-9d29-793d28f4585e service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:01.279099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7f826-48dd-474a-9d29-793d28f4585e service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:01.343972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb467e8-31ee-458c-b6ff-7d2633926c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:01.344687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb467e8-31ee-458c-b6ff-7d2633926c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:01.345127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb467e8-31ee-458c-b6ff-7d2633926c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:01.345733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb467e8-31ee-458c-b6ff-7d2633926c58 admin admin] PATH_INFO: `/v3/users/538736eb093546e1a81e2b4c3e537166` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:01.347051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb467e8-31ee-458c-b6ff-7d2633926c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=538736eb093546e1a81e2b4c3e537166)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:01.349044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb467e8-31ee-458c-b6ff-7d2633926c58 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:01.373018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:01.407306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cb467e8-31ee-458c-b6ff-7d2633926c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '538736eb093546e1a81e2b4c3e537166'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.420061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cb467e8-31ee-458c-b6ff-7d2633926c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '538736eb093546e1a81e2b4c3e537166'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.420785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cb467e8-31ee-458c-b6ff-7d2633926c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '538736eb093546e1a81e2b4c3e537166'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.427870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cb467e8-31ee-458c-b6ff-7d2633926c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '538736eb093546e1a81e2b4c3e537166'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.435932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cb467e8-31ee-458c-b6ff-7d2633926c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '538736eb093546e1a81e2b4c3e537166'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.442240 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1801/3645] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:01:01 2022] DELETE /identity/v3/users/538736eb093546e1a81e2b4c3e537166 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:01.649228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:01.651467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:01.669565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3212a-d9ee-4072-9d8c-8019752eaf7b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:01.790665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890f39a-f880-4d7d-85ff-ac057c86481b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:01.791320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890f39a-f880-4d7d-85ff-ac057c86481b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:01.791635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890f39a-f880-4d7d-85ff-ac057c86481b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:01.792150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890f39a-f880-4d7d-85ff-ac057c86481b admin admin] PATH_INFO: `/v3/projects/0d05514342ed4dc383c025be27b5808b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:01.794580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890f39a-f880-4d7d-85ff-ac057c86481b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d05514342ed4dc383c025be27b5808b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:01.797440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890f39a-f880-4d7d-85ff-ac057c86481b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:01.845309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:01.891898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0890f39a-f880-4d7d-85ff-ac057c86481b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d05514342ed4dc383c025be27b5808b', 'request_id': 'req-0890f39a-f880-4d7d-85ff-ac057c86481b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.905243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0890f39a-f880-4d7d-85ff-ac057c86481b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d05514342ed4dc383c025be27b5808b', 'request_id': 'req-0890f39a-f880-4d7d-85ff-ac057c86481b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.913424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0890f39a-f880-4d7d-85ff-ac057c86481b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d05514342ed4dc383c025be27b5808b', 'request_id': 'req-0890f39a-f880-4d7d-85ff-ac057c86481b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.935683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0890f39a-f880-4d7d-85ff-ac057c86481b admin admin] The token cache is being invalidate because project 0d05514342ed4dc383c025be27b5808b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:01:01.936267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0890f39a-f880-4d7d-85ff-ac057c86481b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.937362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0890f39a-f880-4d7d-85ff-ac057c86481b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:01.944351 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1845/3646] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:01:01 2022] DELETE /identity/v3/projects/0d05514342ed4dc383c025be27b5808b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:01.946225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25f6c5-a763-4ad7-adc4-c8750cf14ced None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:01.946797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25f6c5-a763-4ad7-adc4-c8750cf14ced None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:01.947127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25f6c5-a763-4ad7-adc4-c8750cf14ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:01.976601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9b25f6c5-a763-4ad7-adc4-c8750cf14ced None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:01.978474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9b25f6c5-a763-4ad7-adc4-c8750cf14ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:02.046594 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1802/3647] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:02.055643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae977c2-43a4-40e5-8035-22b4ae048df9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:02.056017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae977c2-43a4-40e5-8035-22b4ae048df9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:02.056311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae977c2-43a4-40e5-8035-22b4ae048df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:02.079397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4ae977c2-43a4-40e5-8035-22b4ae048df9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:02.080631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4ae977c2-43a4-40e5-8035-22b4ae048df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:02.092144 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1846/3648] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:02.093742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25f6c5-a763-4ad7-adc4-c8750cf14ced None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:02.104463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25f6c5-a763-4ad7-adc4-c8750cf14ced None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:02.110439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3f9de-3442-4d1f-9f8d-bea1b76c1d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:02.111651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3f9de-3442-4d1f-9f8d-bea1b76c1d38 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:02.112149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3f9de-3442-4d1f-9f8d-bea1b76c1d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:02.112505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3f9de-3442-4d1f-9f8d-bea1b76c1d38 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:02.113283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3f9de-3442-4d1f-9f8d-bea1b76c1d38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:01:02.113837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3f9de-3442-4d1f-9f8d-bea1b76c1d38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:02.116876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3f9de-3442-4d1f-9f8d-bea1b76c1d38 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:02.144240 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1803/3649] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:01:02.145229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae977c2-43a4-40e5-8035-22b4ae048df9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:02.161623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae977c2-43a4-40e5-8035-22b4ae048df9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:02.172678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5e72a-900b-4162-94a2-9ca08b33a523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:02.177700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5e72a-900b-4162-94a2-9ca08b33a523 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:02.178094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5e72a-900b-4162-94a2-9ca08b33a523 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:02.178406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5e72a-900b-4162-94a2-9ca08b33a523 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:02.179408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5e72a-900b-4162-94a2-9ca08b33a523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:02.188624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5e72a-900b-4162-94a2-9ca08b33a523 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:02.229795 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1847/3650] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:01:02 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:02.234393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3f9de-3442-4d1f-9f8d-bea1b76c1d38 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:02.259476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3f9de-3442-4d1f-9f8d-bea1b76c1d38 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:02.391860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c84a6-3931-424b-b4d3-1d47b753e11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:02.392516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c84a6-3931-424b-b4d3-1d47b753e11d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:02.393055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c84a6-3931-424b-b4d3-1d47b753e11d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:02.393408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c84a6-3931-424b-b4d3-1d47b753e11d admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:02.394118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c84a6-3931-424b-b4d3-1d47b753e11d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:02.396277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c84a6-3931-424b-b4d3-1d47b753e11d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:02.431748 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1804/3651] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:01:02 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:02.436512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5e72a-900b-4162-94a2-9ca08b33a523 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:02.457153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5e72a-900b-4162-94a2-9ca08b33a523 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:02.464328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf254c68-dd02-4b24-b9c0-4f3b78bbc08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:02.465843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf254c68-dd02-4b24-b9c0-4f3b78bbc08c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:02.466167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf254c68-dd02-4b24-b9c0-4f3b78bbc08c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:02.466423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf254c68-dd02-4b24-b9c0-4f3b78bbc08c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:02.467091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf254c68-dd02-4b24-b9c0-4f3b78bbc08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:02.468886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf254c68-dd02-4b24-b9c0-4f3b78bbc08c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:02.484936 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1848/3652] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:01:02.486528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c84a6-3931-424b-b4d3-1d47b753e11d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:02.499900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c84a6-3931-424b-b4d3-1d47b753e11d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:02.506799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0e18f-7508-4750-b6fc-92fb8d687498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:02.507459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0e18f-7508-4750-b6fc-92fb8d687498 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:02.507981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0e18f-7508-4750-b6fc-92fb8d687498 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:02.508432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0e18f-7508-4750-b6fc-92fb8d687498 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:02.509247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0e18f-7508-4750-b6fc-92fb8d687498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:02.511112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0e18f-7508-4750-b6fc-92fb8d687498 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:02.518131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:02.529756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf254c68-dd02-4b24-b9c0-4f3b78bbc08c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:02.530789 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1805/3653] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:02.540563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf254c68-dd02-4b24-b9c0-4f3b78bbc08c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:02.547459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cdaa9-90fa-403e-a509-c8fa3d1805c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:02.548350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cdaa9-90fa-403e-a509-c8fa3d1805c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:02.548853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cdaa9-90fa-403e-a509-c8fa3d1805c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:02.549452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cdaa9-90fa-403e-a509-c8fa3d1805c8 admin admin] PATH_INFO: `/v3/projects/35119286edb94a8fb711a37863321fbe/users/9884fe87dedd42bba0442aca48ebdc8d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:02.567938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816cdaa9-90fa-403e-a509-c8fa3d1805c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35119286edb94a8fb711a37863321fbe, user_id=9884fe87dedd42bba0442aca48ebdc8d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:02.570741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816cdaa9-90fa-403e-a509-c8fa3d1805c8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:02.592680 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1849/3654] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:01:02 2022] PUT /identity/v3/projects/35119286edb94a8fb711a37863321fbe/users/9884fe87dedd42bba0442aca48ebdc8d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:02.594976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadeabde-aacf-4226-b2ae-633ab4c81b33 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:02.595736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadeabde-aacf-4226-b2ae-633ab4c81b33 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:02.596343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadeabde-aacf-4226-b2ae-633ab4c81b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:02.640274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-aadeabde-aacf-4226-b2ae-633ab4c81b33 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:02.641965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-aadeabde-aacf-4226-b2ae-633ab4c81b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:02.746363 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1806/3655] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:02.762154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cdaa9-90fa-403e-a509-c8fa3d1805c8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:02.765004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-816cdaa9-90fa-403e-a509-c8fa3d1805c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:02.811184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadeabde-aacf-4226-b2ae-633ab4c81b33 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:02.812947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-aadeabde-aacf-4226-b2ae-633ab4c81b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:02.825802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadeabde-aacf-4226-b2ae-633ab4c81b33 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:02.836978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:02.838246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:02.838714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:02.839154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] PATH_INFO: `/v3/users/3db58915ddb1452490f90c8f2a583220` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:02.841525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3db58915ddb1452490f90c8f2a583220)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:02.846796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:02.855874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cdaa9-90fa-403e-a509-c8fa3d1805c8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:02.861291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b7a4e-ca6c-4b6c-ba4f-ee273a1ab344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:02.861962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b7a4e-ca6c-4b6c-ba4f-ee273a1ab344 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:02.862384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b7a4e-ca6c-4b6c-ba4f-ee273a1ab344 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:02.862818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b7a4e-ca6c-4b6c-ba4f-ee273a1ab344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:02.886781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b7a4e-ca6c-4b6c-ba4f-ee273a1ab344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:02.888715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b7a4e-ca6c-4b6c-ba4f-ee273a1ab344 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:02.900023 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1850/3656] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:01:02.942956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3db58915ddb1452490f90c8f2a583220'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:02.961842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3db58915ddb1452490f90c8f2a583220'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:02.962414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3db58915ddb1452490f90c8f2a583220'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:02.972025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3db58915ddb1452490f90c8f2a583220'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:02.982458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3db58915ddb1452490f90c8f2a583220'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:02.990511 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1807/3657] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:01:02 2022] DELETE /identity/v3/users/3db58915ddb1452490f90c8f2a583220 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:01:02.995122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c26aa-9e3e-417a-9f4a-0b94e755a87e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:02.995531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c26aa-9e3e-417a-9f4a-0b94e755a87e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:02.995751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c26aa-9e3e-417a-9f4a-0b94e755a87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:03.028825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-566c26aa-9e3e-417a-9f4a-0b94e755a87e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:03.043335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-566c26aa-9e3e-417a-9f4a-0b94e755a87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:03.104932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:03.130185 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1851/3658] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:03.165304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:03.167447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:03.244419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb596a82-20cf-4bb5-ac0b-6031baee7427 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:03.248519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543d754-3a1a-4d32-bde1-3f9989f4975c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:03.249554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543d754-3a1a-4d32-bde1-3f9989f4975c service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:03.249675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543d754-3a1a-4d32-bde1-3f9989f4975c service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:03.249952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543d754-3a1a-4d32-bde1-3f9989f4975c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:03.260680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543d754-3a1a-4d32-bde1-3f9989f4975c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:03.262425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543d754-3a1a-4d32-bde1-3f9989f4975c service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:03.273415 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1808/3659] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:01:05.031264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c26aa-9e3e-417a-9f4a-0b94e755a87e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:05.033688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-566c26aa-9e3e-417a-9f4a-0b94e755a87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:05.051681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c26aa-9e3e-417a-9f4a-0b94e755a87e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:05.066079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d79c9-7328-429a-a440-bd2346099b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:05.066989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d79c9-7328-429a-a440-bd2346099b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:05.069800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d79c9-7328-429a-a440-bd2346099b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:05.070257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d79c9-7328-429a-a440-bd2346099b67 admin admin] PATH_INFO: `/v3/projects/7eef34f8ef5e4e73802caff2f4e5a023` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:05.073825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d79c9-7328-429a-a440-bd2346099b67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eef34f8ef5e4e73802caff2f4e5a023)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:05.076660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d79c9-7328-429a-a440-bd2346099b67 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:05.238015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-176d79c9-7328-429a-a440-bd2346099b67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eef34f8ef5e4e73802caff2f4e5a023', 'request_id': 'req-176d79c9-7328-429a-a440-bd2346099b67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:05.265898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-176d79c9-7328-429a-a440-bd2346099b67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eef34f8ef5e4e73802caff2f4e5a023', 'request_id': 'req-176d79c9-7328-429a-a440-bd2346099b67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:05.285270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-176d79c9-7328-429a-a440-bd2346099b67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eef34f8ef5e4e73802caff2f4e5a023', 'request_id': 'req-176d79c9-7328-429a-a440-bd2346099b67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:05.343556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-176d79c9-7328-429a-a440-bd2346099b67 admin admin] The token cache is being invalidate because project 7eef34f8ef5e4e73802caff2f4e5a023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:01:05.343865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-176d79c9-7328-429a-a440-bd2346099b67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:05.345873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-176d79c9-7328-429a-a440-bd2346099b67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:05.351391 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1852/3660] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:01:05 2022] DELETE /identity/v3/projects/7eef34f8ef5e4e73802caff2f4e5a023 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:05.353461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a20db-ea8b-46ba-94fc-6dfc0ecfc9d7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:05.353948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a20db-ea8b-46ba-94fc-6dfc0ecfc9d7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:05.354046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a20db-ea8b-46ba-94fc-6dfc0ecfc9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:05.396342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a37a20db-ea8b-46ba-94fc-6dfc0ecfc9d7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:05.420698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a37a20db-ea8b-46ba-94fc-6dfc0ecfc9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:05.456009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:05.563927 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1809/3661] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:05.571809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb231476-439c-4929-ade5-b0909ab325aa None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:05.572103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb231476-439c-4929-ade5-b0909ab325aa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:05.572386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb231476-439c-4929-ade5-b0909ab325aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:05.614333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bb231476-439c-4929-ade5-b0909ab325aa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:05.615873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bb231476-439c-4929-ade5-b0909ab325aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:05.630238 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1853/3662] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:05.630928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a20db-ea8b-46ba-94fc-6dfc0ecfc9d7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:05.644452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a20db-ea8b-46ba-94fc-6dfc0ecfc9d7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:05.651651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6525a-3fac-4a5e-a261-275c867d6e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:05.652991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6525a-3fac-4a5e-a261-275c867d6e40 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:05.653521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6525a-3fac-4a5e-a261-275c867d6e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:05.653896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6525a-3fac-4a5e-a261-275c867d6e40 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:05.655194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e6525a-3fac-4a5e-a261-275c867d6e40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:01:05.655605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e6525a-3fac-4a5e-a261-275c867d6e40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:05.657718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e6525a-3fac-4a5e-a261-275c867d6e40 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:05.674800 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1810/3663] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:01:05.678909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb231476-439c-4929-ade5-b0909ab325aa None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:05.690018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb231476-439c-4929-ade5-b0909ab325aa None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:05.696585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9b45d-7342-4c02-a846-5c16fb78294a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:05.697488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9b45d-7342-4c02-a846-5c16fb78294a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:05.697932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9b45d-7342-4c02-a846-5c16fb78294a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:05.698598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9b45d-7342-4c02-a846-5c16fb78294a admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:05.699251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9b45d-7342-4c02-a846-5c16fb78294a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:05.701265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9b45d-7342-4c02-a846-5c16fb78294a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:05.748327 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1854/3664] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:01:05 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:05.749775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6525a-3fac-4a5e-a261-275c867d6e40 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:05.763769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6525a-3fac-4a5e-a261-275c867d6e40 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:05.854555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46ac05-d6bd-4a59-854e-1f88ae9cccd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:05.855654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46ac05-d6bd-4a59-854e-1f88ae9cccd0 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:05.856115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46ac05-d6bd-4a59-854e-1f88ae9cccd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:05.857069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46ac05-d6bd-4a59-854e-1f88ae9cccd0 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:05.857867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46ac05-d6bd-4a59-854e-1f88ae9cccd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:05.861001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46ac05-d6bd-4a59-854e-1f88ae9cccd0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:05.924459 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1811/3665] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:01:05 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:05.928359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9b45d-7342-4c02-a846-5c16fb78294a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:05.945265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9b45d-7342-4c02-a846-5c16fb78294a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:05.962057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f094ae-ab19-4fa2-bc40-343ca0194dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:05.962928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f094ae-ab19-4fa2-bc40-343ca0194dac admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:05.963218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f094ae-ab19-4fa2-bc40-343ca0194dac admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:05.964632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f094ae-ab19-4fa2-bc40-343ca0194dac admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:05.966640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f094ae-ab19-4fa2-bc40-343ca0194dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:05.968807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f094ae-ab19-4fa2-bc40-343ca0194dac admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:05.984966 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1855/3666] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:05.989599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46ac05-d6bd-4a59-854e-1f88ae9cccd0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:06.015029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46ac05-d6bd-4a59-854e-1f88ae9cccd0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:06.027983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cba83-f4b2-4b8c-a2ce-cbe39e8f1ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:06.029416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cba83-f4b2-4b8c-a2ce-cbe39e8f1ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:06.029809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cba83-f4b2-4b8c-a2ce-cbe39e8f1ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:06.030191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cba83-f4b2-4b8c-a2ce-cbe39e8f1ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:06.031779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cba83-f4b2-4b8c-a2ce-cbe39e8f1ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:06.033553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cba83-f4b2-4b8c-a2ce-cbe39e8f1ce8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:06.050783 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1812/3667] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:06.051827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f094ae-ab19-4fa2-bc40-343ca0194dac admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:06.063229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f094ae-ab19-4fa2-bc40-343ca0194dac admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:06.069331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d2b1a-cd7e-470c-8004-e1c9c70d5c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:06.070002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d2b1a-cd7e-470c-8004-e1c9c70d5c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:06.070291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d2b1a-cd7e-470c-8004-e1c9c70d5c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:06.070582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d2b1a-cd7e-470c-8004-e1c9c70d5c21 admin admin] PATH_INFO: `/v3/projects/b9b0f161c27a4b83865404bc57c99eca/users/7d240c36223b49cbb24e51bcb061a040/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:06.087186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d2b1a-cd7e-470c-8004-e1c9c70d5c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9b0f161c27a4b83865404bc57c99eca, user_id=7d240c36223b49cbb24e51bcb061a040, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:06.089990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d2b1a-cd7e-470c-8004-e1c9c70d5c21 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:06.106036 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1856/3668] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:01:06 2022] PUT /identity/v3/projects/b9b0f161c27a4b83865404bc57c99eca/users/7d240c36223b49cbb24e51bcb061a040/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:01:06.120434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8784b-7a53-4695-a86e-b443d5265ed2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:06.120434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8784b-7a53-4695-a86e-b443d5265ed2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:06.120434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8784b-7a53-4695-a86e-b443d5265ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:06.159633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2ef8784b-7a53-4695-a86e-b443d5265ed2 None None] MFA Rules not processed for user `7d240c36223b49cbb24e51bcb061a040`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:06.162147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2ef8784b-7a53-4695-a86e-b443d5265ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:06.318907 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1813/3669] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:06.332090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882fbe-88d9-4804-b763-d8dd14a1da67 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:06.332392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882fbe-88d9-4804-b763-d8dd14a1da67 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:06.332678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882fbe-88d9-4804-b763-d8dd14a1da67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:06.358197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5b882fbe-88d9-4804-b763-d8dd14a1da67 None None] MFA Rules not processed for user `7d240c36223b49cbb24e51bcb061a040`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:06.359720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5b882fbe-88d9-4804-b763-d8dd14a1da67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:06.367895 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1857/3670] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:06.424027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8784b-7a53-4695-a86e-b443d5265ed2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:06.428453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2ef8784b-7a53-4695-a86e-b443d5265ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:06.459485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:06.638756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8784b-7a53-4695-a86e-b443d5265ed2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:06.650035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2a96e-1678-496a-b41c-f0c2715830ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:06.651059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2a96e-1678-496a-b41c-f0c2715830ec service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:06.651563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2a96e-1678-496a-b41c-f0c2715830ec service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:06.652189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2a96e-1678-496a-b41c-f0c2715830ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:06.680138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2a96e-1678-496a-b41c-f0c2715830ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:06.683649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2a96e-1678-496a-b41c-f0c2715830ec service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:06.722257 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1814/3671] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:07.296108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1091e5-01c2-4300-a7fe-4d532d8265e4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:07.296602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1091e5-01c2-4300-a7fe-4d532d8265e4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:07.296714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1091e5-01c2-4300-a7fe-4d532d8265e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:07.349896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2b1091e5-01c2-4300-a7fe-4d532d8265e4 None None] MFA Rules not processed for user `859a4503b4d84a4e8c3213e2de02492d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:07.351161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b1091e5-01c2-4300-a7fe-4d532d8265e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:07.623436 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1858/3672] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:10.705720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c1da7-9b88-445a-8340-614c1bfa815b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:10.706798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c1da7-9b88-445a-8340-614c1bfa815b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:10.707093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c1da7-9b88-445a-8340-614c1bfa815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:10.752537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-948c1da7-9b88-445a-8340-614c1bfa815b None None] MFA Rules not processed for user `9884fe87dedd42bba0442aca48ebdc8d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:10.753885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-948c1da7-9b88-445a-8340-614c1bfa815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:10.840066 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1815/3673] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:10.844392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1091e5-01c2-4300-a7fe-4d532d8265e4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:10.846355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b1091e5-01c2-4300-a7fe-4d532d8265e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:10.862579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:10.966842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1091e5-01c2-4300-a7fe-4d532d8265e4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:10.972289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18a090-510b-418f-88a8-59a41f380f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:10.973744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18a090-510b-418f-88a8-59a41f380f15 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:10.974227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18a090-510b-418f-88a8-59a41f380f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:10.974658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18a090-510b-418f-88a8-59a41f380f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:10.975607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18a090-510b-418f-88a8-59a41f380f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:10.978470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18a090-510b-418f-88a8-59a41f380f15 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:11.032046 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1859/3674] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:01:10 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:01:11.035847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c1da7-9b88-445a-8340-614c1bfa815b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:11.052269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c1da7-9b88-445a-8340-614c1bfa815b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:11.147545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e48a3-3c27-4399-8369-fdb3d3fc9155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:11.148669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e48a3-3c27-4399-8369-fdb3d3fc9155 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:11.148878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e48a3-3c27-4399-8369-fdb3d3fc9155 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:11.149132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e48a3-3c27-4399-8369-fdb3d3fc9155 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:11.149769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e48a3-3c27-4399-8369-fdb3d3fc9155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:11.152315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e48a3-3c27-4399-8369-fdb3d3fc9155 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:11.187934 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1816/3675] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:01:11 2022] POST /identity/v3/users => generated 299 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:01:11.192309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18a090-510b-418f-88a8-59a41f380f15 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:11.208773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18a090-510b-418f-88a8-59a41f380f15 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:11.234224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db91a09-9cf3-4e24-89e0-dce0ff86c9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:11.235144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db91a09-9cf3-4e24-89e0-dce0ff86c9ce admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:11.235819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db91a09-9cf3-4e24-89e0-dce0ff86c9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:11.236270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db91a09-9cf3-4e24-89e0-dce0ff86c9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:11.237171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db91a09-9cf3-4e24-89e0-dce0ff86c9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:11.247490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db91a09-9cf3-4e24-89e0-dce0ff86c9ce admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:11.277658 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1860/3676] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:11.281282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e48a3-3c27-4399-8369-fdb3d3fc9155 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:11.293491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e48a3-3c27-4399-8369-fdb3d3fc9155 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:11.312722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb0d6a-ce49-4439-9d8b-d8be7bce4d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:11.314347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb0d6a-ce49-4439-9d8b-d8be7bce4d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:11.314651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb0d6a-ce49-4439-9d8b-d8be7bce4d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:11.314946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb0d6a-ce49-4439-9d8b-d8be7bce4d19 admin admin] PATH_INFO: `/v3/projects/0303211069184f75b9e8c6d075999295/users/5d1dda56f7cb4adda980038b707d34de/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:11.381548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:11.393186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb0d6a-ce49-4439-9d8b-d8be7bce4d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0303211069184f75b9e8c6d075999295, user_id=5d1dda56f7cb4adda980038b707d34de, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:11.396156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb0d6a-ce49-4439-9d8b-d8be7bce4d19 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:11.452462 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1817/3677] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:01:11 2022] PUT /identity/v3/projects/0303211069184f75b9e8c6d075999295/users/5d1dda56f7cb4adda980038b707d34de/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:11.453697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db91a09-9cf3-4e24-89e0-dce0ff86c9ce admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:11.471143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db91a09-9cf3-4e24-89e0-dce0ff86c9ce admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:11.686196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d99390-ed9a-4606-a3c1-72e2f2ca50b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:11.686906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d99390-ed9a-4606-a3c1-72e2f2ca50b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:11.687267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d99390-ed9a-4606-a3c1-72e2f2ca50b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:11.687930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d99390-ed9a-4606-a3c1-72e2f2ca50b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:11.688401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d99390-ed9a-4606-a3c1-72e2f2ca50b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:11.693692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d99390-ed9a-4606-a3c1-72e2f2ca50b8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:11.696736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb0d6a-ce49-4439-9d8b-d8be7bce4d19 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:11.699092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0afb0d6a-ce49-4439-9d8b-d8be7bce4d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:11.718122 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1861/3678] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:11.726904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d99390-ed9a-4606-a3c1-72e2f2ca50b8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:11.731743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb0d6a-ce49-4439-9d8b-d8be7bce4d19 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:11.747549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb403d-f8e5-4e84-ba69-a5b4aa94caa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:11.749088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb403d-f8e5-4e84-ba69-a5b4aa94caa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:11.749812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb403d-f8e5-4e84-ba69-a5b4aa94caa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:11.750499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb403d-f8e5-4e84-ba69-a5b4aa94caa8 admin admin] PATH_INFO: `/v3/users/fa304e40e3894910be1a5e1e97a34b1a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:11.750995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d99390-ed9a-4606-a3c1-72e2f2ca50b8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:11.759877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb403d-f8e5-4e84-ba69-a5b4aa94caa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa304e40e3894910be1a5e1e97a34b1a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:11.763990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb403d-f8e5-4e84-ba69-a5b4aa94caa8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:11.765908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bebc2f-ff04-4d6d-b972-96fc571d66d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:11.766680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bebc2f-ff04-4d6d-b972-96fc571d66d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:11.767037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bebc2f-ff04-4d6d-b972-96fc571d66d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:11.767391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bebc2f-ff04-4d6d-b972-96fc571d66d8 admin admin] PATH_INFO: `/v3/projects/0303211069184f75b9e8c6d075999295/users/5d1dda56f7cb4adda980038b707d34de/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:11.773103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bebc2f-ff04-4d6d-b972-96fc571d66d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0303211069184f75b9e8c6d075999295, user_id=5d1dda56f7cb4adda980038b707d34de, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:11.777049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bebc2f-ff04-4d6d-b972-96fc571d66d8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:11.811763 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1862/3679] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:01:11 2022] PUT /identity/v3/projects/0303211069184f75b9e8c6d075999295/users/5d1dda56f7cb4adda980038b707d34de/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:11.912257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3fdb403d-f8e5-4e84-ba69-a5b4aa94caa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa304e40e3894910be1a5e1e97a34b1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:11.938756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3fdb403d-f8e5-4e84-ba69-a5b4aa94caa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa304e40e3894910be1a5e1e97a34b1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:11.939649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3fdb403d-f8e5-4e84-ba69-a5b4aa94caa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa304e40e3894910be1a5e1e97a34b1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:11.955685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3fdb403d-f8e5-4e84-ba69-a5b4aa94caa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa304e40e3894910be1a5e1e97a34b1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:11.970046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3fdb403d-f8e5-4e84-ba69-a5b4aa94caa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa304e40e3894910be1a5e1e97a34b1a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:11.987483 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1818/3680] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:01:11 2022] DELETE /identity/v3/users/fa304e40e3894910be1a5e1e97a34b1a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:12.672616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bebc2f-ff04-4d6d-b972-96fc571d66d8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:12.675239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-33bebc2f-ff04-4d6d-b972-96fc571d66d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:12.851530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bebc2f-ff04-4d6d-b972-96fc571d66d8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:12.882884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:13.003282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518a252-55ea-4dcb-a564-1f6347090e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:13.003886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518a252-55ea-4dcb-a564-1f6347090e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:13.004298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518a252-55ea-4dcb-a564-1f6347090e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:13.004593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518a252-55ea-4dcb-a564-1f6347090e5f admin admin] PATH_INFO: `/v3/projects/630205311fc04912910e56076899f18d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:13.006464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518a252-55ea-4dcb-a564-1f6347090e5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=630205311fc04912910e56076899f18d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:13.008655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518a252-55ea-4dcb-a564-1f6347090e5f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:13.189075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b518a252-55ea-4dcb-a564-1f6347090e5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '630205311fc04912910e56076899f18d', 'request_id': 'req-b518a252-55ea-4dcb-a564-1f6347090e5f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:13.205459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b518a252-55ea-4dcb-a564-1f6347090e5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '630205311fc04912910e56076899f18d', 'request_id': 'req-b518a252-55ea-4dcb-a564-1f6347090e5f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:13.227971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b518a252-55ea-4dcb-a564-1f6347090e5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '630205311fc04912910e56076899f18d', 'request_id': 'req-b518a252-55ea-4dcb-a564-1f6347090e5f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:13.263037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b518a252-55ea-4dcb-a564-1f6347090e5f admin admin] The token cache is being invalidate because project 630205311fc04912910e56076899f18d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:01:13.263659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b518a252-55ea-4dcb-a564-1f6347090e5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:13.267723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b518a252-55ea-4dcb-a564-1f6347090e5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:13.275674 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1863/3681] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:01:12 2022] DELETE /identity/v3/projects/630205311fc04912910e56076899f18d => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:13.277421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c6efc-9cd9-41bf-8c0a-df010a1647f9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:13.278337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c6efc-9cd9-41bf-8c0a-df010a1647f9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:13.278780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c6efc-9cd9-41bf-8c0a-df010a1647f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:13.363221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-009c6efc-9cd9-41bf-8c0a-df010a1647f9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:13.369201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-009c6efc-9cd9-41bf-8c0a-df010a1647f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:13.579955 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1819/3682] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:13.588298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5d3dc-cfba-4cda-9e23-cd9cd5cad647 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:13.588855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5d3dc-cfba-4cda-9e23-cd9cd5cad647 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:13.589330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5d3dc-cfba-4cda-9e23-cd9cd5cad647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:13.620365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f8b5d3dc-cfba-4cda-9e23-cd9cd5cad647 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:13.621618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f8b5d3dc-cfba-4cda-9e23-cd9cd5cad647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:13.671167 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1864/3683] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:13.675682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c6efc-9cd9-41bf-8c0a-df010a1647f9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:13.706130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c6efc-9cd9-41bf-8c0a-df010a1647f9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:13.725774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4ad39-8ff9-4057-a05c-38073bd57405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:13.726535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4ad39-8ff9-4057-a05c-38073bd57405 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:13.726677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4ad39-8ff9-4057-a05c-38073bd57405 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:13.728289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4ad39-8ff9-4057-a05c-38073bd57405 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:13.743751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4ad39-8ff9-4057-a05c-38073bd57405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:01:13.744624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4ad39-8ff9-4057-a05c-38073bd57405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:13.751486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4ad39-8ff9-4057-a05c-38073bd57405 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:13.791069 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1820/3684] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:01:13.795909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5d3dc-cfba-4cda-9e23-cd9cd5cad647 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:13.812203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5d3dc-cfba-4cda-9e23-cd9cd5cad647 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:13.826659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226467e-cfcc-4c51-852a-9e929be4f8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:13.827376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226467e-cfcc-4c51-852a-9e929be4f8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:13.843459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226467e-cfcc-4c51-852a-9e929be4f8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:13.843775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226467e-cfcc-4c51-852a-9e929be4f8e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:13.847754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226467e-cfcc-4c51-852a-9e929be4f8e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:13.851379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226467e-cfcc-4c51-852a-9e929be4f8e8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:13.922046 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1865/3685] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:01:13 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:13.926069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4ad39-8ff9-4057-a05c-38073bd57405 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:13.944597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4ad39-8ff9-4057-a05c-38073bd57405 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:14.071668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:14.087971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b85bc-d1bd-4dbd-9f86-fa4d8e06d039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:14.089665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b85bc-d1bd-4dbd-9f86-fa4d8e06d039 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:14.090163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b85bc-d1bd-4dbd-9f86-fa4d8e06d039 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:14.091116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b85bc-d1bd-4dbd-9f86-fa4d8e06d039 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:14.093118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b85bc-d1bd-4dbd-9f86-fa4d8e06d039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:14.098255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b85bc-d1bd-4dbd-9f86-fa4d8e06d039 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:14.157998 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1821/3686] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:01:13 2022] POST /identity/v3/users => generated 287 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:14.161667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226467e-cfcc-4c51-852a-9e929be4f8e8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:14.174508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226467e-cfcc-4c51-852a-9e929be4f8e8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:14.193571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b85bc-d1bd-4dbd-9f86-fa4d8e06d039 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:14.195467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-337b85bc-d1bd-4dbd-9f86-fa4d8e06d039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:14.203802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe1aca-8380-4a1a-bb05-87952a815876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:14.204432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe1aca-8380-4a1a-bb05-87952a815876 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:14.204718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe1aca-8380-4a1a-bb05-87952a815876 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:14.205223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe1aca-8380-4a1a-bb05-87952a815876 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:14.206053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe1aca-8380-4a1a-bb05-87952a815876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:14.207825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe1aca-8380-4a1a-bb05-87952a815876 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:14.217508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b85bc-d1bd-4dbd-9f86-fa4d8e06d039 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:14.230215 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1866/3687] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:14.234028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2a3b8-b4bb-427b-b79e-f016146151e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:14.234916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2a3b8-b4bb-427b-b79e-f016146151e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:14.235367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2a3b8-b4bb-427b-b79e-f016146151e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:14.235822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2a3b8-b4bb-427b-b79e-f016146151e0 admin admin] PATH_INFO: `/v3/users/e6970a72952e4d4a8f26e0f8ee23f364` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:14.237789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe1aca-8380-4a1a-bb05-87952a815876 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:14.238739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2a3b8-b4bb-427b-b79e-f016146151e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6970a72952e4d4a8f26e0f8ee23f364)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:14.240872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2a3b8-b4bb-427b-b79e-f016146151e0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:14.262760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe1aca-8380-4a1a-bb05-87952a815876 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:14.278754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b78d3-fed1-4dce-8b2e-b9809dd27158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:14.279635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b78d3-fed1-4dce-8b2e-b9809dd27158 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:14.279854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b78d3-fed1-4dce-8b2e-b9809dd27158 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:14.280700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b78d3-fed1-4dce-8b2e-b9809dd27158 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:14.280828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b78d3-fed1-4dce-8b2e-b9809dd27158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:14.282499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b78d3-fed1-4dce-8b2e-b9809dd27158 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:14.308966 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1867/3688] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:14.317036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b78d3-fed1-4dce-8b2e-b9809dd27158 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:14.339138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-44a2a3b8-b4bb-427b-b79e-f016146151e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6970a72952e4d4a8f26e0f8ee23f364'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:14.347832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b78d3-fed1-4dce-8b2e-b9809dd27158 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:14.356235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e496a-d925-4d0d-b8f4-87f15fcf6e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:14.357547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e496a-d925-4d0d-b8f4-87f15fcf6e62 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:14.358024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e496a-d925-4d0d-b8f4-87f15fcf6e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:14.358470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e496a-d925-4d0d-b8f4-87f15fcf6e62 admin admin] PATH_INFO: `/v3/projects/75a128c413284fcb96ecbdb84d33c862/users/9abfaac1da5f41f7a522b009f8073ac8/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:14.368482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-44a2a3b8-b4bb-427b-b79e-f016146151e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6970a72952e4d4a8f26e0f8ee23f364'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:14.369302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-44a2a3b8-b4bb-427b-b79e-f016146151e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6970a72952e4d4a8f26e0f8ee23f364'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:14.383297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e496a-d925-4d0d-b8f4-87f15fcf6e62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75a128c413284fcb96ecbdb84d33c862, user_id=9abfaac1da5f41f7a522b009f8073ac8, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:14.387198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e496a-d925-4d0d-b8f4-87f15fcf6e62 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:14.388956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-44a2a3b8-b4bb-427b-b79e-f016146151e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6970a72952e4d4a8f26e0f8ee23f364'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:14.399324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-44a2a3b8-b4bb-427b-b79e-f016146151e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6970a72952e4d4a8f26e0f8ee23f364'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:14.419864 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1868/3689] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:01:14 2022] PUT /identity/v3/projects/75a128c413284fcb96ecbdb84d33c862/users/9abfaac1da5f41f7a522b009f8073ac8/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:14.420234 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1822/3690] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:01:14 2022] DELETE /identity/v3/users/e6970a72952e4d4a8f26e0f8ee23f364 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:14.422275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1d3ee-6ff0-40a7-ac25-e1da0cd0ee53 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:14.422451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570fecc-6c56-42aa-8ba2-8dec895ffccf None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:14.422559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1d3ee-6ff0-40a7-ac25-e1da0cd0ee53 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:14.422842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1d3ee-6ff0-40a7-ac25-e1da0cd0ee53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:14.422956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570fecc-6c56-42aa-8ba2-8dec895ffccf None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:14.423240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570fecc-6c56-42aa-8ba2-8dec895ffccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:14.460146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9570fecc-6c56-42aa-8ba2-8dec895ffccf None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:14.462686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9570fecc-6c56-42aa-8ba2-8dec895ffccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:14.488021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b8e1d3ee-6ff0-40a7-ac25-e1da0cd0ee53 None None] MFA Rules not processed for user `9abfaac1da5f41f7a522b009f8073ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:14.489306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b8e1d3ee-6ff0-40a7-ac25-e1da0cd0ee53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:14.505108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:14.620410 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1869/3691] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:01:14.629619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570fecc-6c56-42aa-8ba2-8dec895ffccf None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:14.631988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9570fecc-6c56-42aa-8ba2-8dec895ffccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:14.667780 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1823/3692] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:01:14.682943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd7379-684b-4347-999f-26eb8bdbc68f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:14.683246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd7379-684b-4347-999f-26eb8bdbc68f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:14.683559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd7379-684b-4347-999f-26eb8bdbc68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:14.721501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570fecc-6c56-42aa-8ba2-8dec895ffccf None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:14.727438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ea45c-aeb2-47ea-8fb4-7fddd93c8f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:14.728321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ea45c-aeb2-47ea-8fb4-7fddd93c8f77 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:14.728611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ea45c-aeb2-47ea-8fb4-7fddd93c8f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:14.728918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ea45c-aeb2-47ea-8fb4-7fddd93c8f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:14.733821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-addd7379-684b-4347-999f-26eb8bdbc68f None None] MFA Rules not processed for user `9abfaac1da5f41f7a522b009f8073ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:14.736458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-addd7379-684b-4347-999f-26eb8bdbc68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:14.745482 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1824/3693] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:01:14.757431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd7379-684b-4347-999f-26eb8bdbc68f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:14.758991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-addd7379-684b-4347-999f-26eb8bdbc68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:14.759114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ea45c-aeb2-47ea-8fb4-7fddd93c8f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:14.796122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ea45c-aeb2-47ea-8fb4-7fddd93c8f77 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:14.812601 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1870/3694] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:14.881916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:14.950047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd7379-684b-4347-999f-26eb8bdbc68f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:14.959275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da60d86-b500-46a6-93e9-a88e7818d3d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:14.960281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da60d86-b500-46a6-93e9-a88e7818d3d0 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:14.960747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da60d86-b500-46a6-93e9-a88e7818d3d0 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:14.961304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da60d86-b500-46a6-93e9-a88e7818d3d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:14.975775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da60d86-b500-46a6-93e9-a88e7818d3d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:14.979880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da60d86-b500-46a6-93e9-a88e7818d3d0 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:15.000495 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1825/3695] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:01:15.707449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ea45c-aeb2-47ea-8fb4-7fddd93c8f77 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:15.709323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-778ea45c-aeb2-47ea-8fb4-7fddd93c8f77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:15.742471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ea45c-aeb2-47ea-8fb4-7fddd93c8f77 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:15.747983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:15.748540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:15.748828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:15.749263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] PATH_INFO: `/v3/projects/5a2f3c186f88497181cf6a1d806631b1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:15.751687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a2f3c186f88497181cf6a1d806631b1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:15.755527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:15.823694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a2f3c186f88497181cf6a1d806631b1', 'request_id': 'req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:15.835730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a2f3c186f88497181cf6a1d806631b1', 'request_id': 'req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:15.844331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a2f3c186f88497181cf6a1d806631b1', 'request_id': 'req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:15.868856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] The token cache is being invalidate because project 5a2f3c186f88497181cf6a1d806631b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:01:15.869449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:15.869698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:15.875903 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1871/3696] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:01:15 2022] DELETE /identity/v3/projects/5a2f3c186f88497181cf6a1d806631b1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:15.880036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da60d86-b500-46a6-93e9-a88e7818d3d0 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:15.900880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da60d86-b500-46a6-93e9-a88e7818d3d0 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:16.029391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fbe49-90f9-4703-8262-186cc30d95a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:16.030242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fbe49-90f9-4703-8262-186cc30d95a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:16.030821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fbe49-90f9-4703-8262-186cc30d95a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:16.031475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fbe49-90f9-4703-8262-186cc30d95a3 admin admin] PATH_INFO: `/v3/users/93af5a1eb0204db2aacd3373cd1decf7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:16.035953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fbe49-90f9-4703-8262-186cc30d95a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93af5a1eb0204db2aacd3373cd1decf7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:16.040369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fbe49-90f9-4703-8262-186cc30d95a3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:16.175778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-184fbe49-90f9-4703-8262-186cc30d95a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93af5a1eb0204db2aacd3373cd1decf7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:16.195909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-184fbe49-90f9-4703-8262-186cc30d95a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93af5a1eb0204db2aacd3373cd1decf7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:16.196944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-184fbe49-90f9-4703-8262-186cc30d95a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93af5a1eb0204db2aacd3373cd1decf7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:16.207384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-184fbe49-90f9-4703-8262-186cc30d95a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93af5a1eb0204db2aacd3373cd1decf7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:16.234763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-184fbe49-90f9-4703-8262-186cc30d95a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93af5a1eb0204db2aacd3373cd1decf7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:16.263062 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1826/3697] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:01:15 2022] DELETE /identity/v3/users/93af5a1eb0204db2aacd3373cd1decf7 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:16.629370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:16.631326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:16.658659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8ebe0-b5ec-422e-8ae9-e7c47d22c579 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:16.817936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:16.826498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:16.827199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:16.827712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:16.828756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9 admin admin] PATH_INFO: `/v3/projects/9643215ef92149eeb8c0f3124b8b1bac` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:16.833213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9643215ef92149eeb8c0f3124b8b1bac)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:16.834702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:16.895482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9643215ef92149eeb8c0f3124b8b1bac', 'request_id': 'req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:16.906044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9643215ef92149eeb8c0f3124b8b1bac', 'request_id': 'req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:16.916665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9643215ef92149eeb8c0f3124b8b1bac', 'request_id': 'req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:16.935410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9 admin admin] The token cache is being invalidate because project 9643215ef92149eeb8c0f3124b8b1bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:01:16.936146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:16.937135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b6cbb14a-bcf6-47c9-a1fe-0912a87bc7c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:16.939358 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1872/3698] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:01:16 2022] DELETE /identity/v3/projects/9643215ef92149eeb8c0f3124b8b1bac => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:16.944417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbce9f-da0e-4947-9433-29fbf0d9c5dd None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:16.945530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbce9f-da0e-4947-9433-29fbf0d9c5dd None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:16.949642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbce9f-da0e-4947-9433-29fbf0d9c5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:16.989739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0abbce9f-da0e-4947-9433-29fbf0d9c5dd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:17.003912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0abbce9f-da0e-4947-9433-29fbf0d9c5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:17.094171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:17.227974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d567b40-80c8-4079-ab5a-54d7f2cbf301 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:17.228236 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1827/3699] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:17.229272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d567b40-80c8-4079-ab5a-54d7f2cbf301 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:17.229564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d567b40-80c8-4079-ab5a-54d7f2cbf301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:17.264360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5d567b40-80c8-4079-ab5a-54d7f2cbf301 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:17.265970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5d567b40-80c8-4079-ab5a-54d7f2cbf301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:17.288246 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1873/3700] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:01:17.294624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbce9f-da0e-4947-9433-29fbf0d9c5dd None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:17.311489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbce9f-da0e-4947-9433-29fbf0d9c5dd None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:17.325503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58bf93-c648-4ea6-a77e-5bc396857fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:17.326479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58bf93-c648-4ea6-a77e-5bc396857fde admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:17.326815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58bf93-c648-4ea6-a77e-5bc396857fde admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:17.327295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58bf93-c648-4ea6-a77e-5bc396857fde admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:17.330978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58bf93-c648-4ea6-a77e-5bc396857fde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:01:17.331302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58bf93-c648-4ea6-a77e-5bc396857fde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:17.332995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58bf93-c648-4ea6-a77e-5bc396857fde admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:17.370401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d567b40-80c8-4079-ab5a-54d7f2cbf301 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:17.372696 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1828/3701] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:01:17.400297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d567b40-80c8-4079-ab5a-54d7f2cbf301 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:17.417634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75183e-29ef-4ed4-8137-7a01da70098c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:17.418278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75183e-29ef-4ed4-8137-7a01da70098c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:17.419119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75183e-29ef-4ed4-8137-7a01da70098c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:17.419401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75183e-29ef-4ed4-8137-7a01da70098c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:17.420629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75183e-29ef-4ed4-8137-7a01da70098c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:17.422609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75183e-29ef-4ed4-8137-7a01da70098c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:17.490881 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1874/3702] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:01:17 2022] POST /identity/v3/projects => generated 393 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:17.496338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58bf93-c648-4ea6-a77e-5bc396857fde admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:17.508807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58bf93-c648-4ea6-a77e-5bc396857fde admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:17.622618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb226e92-f47b-4b1c-ade5-c18121abaefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:17.623275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb226e92-f47b-4b1c-ade5-c18121abaefa admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:17.623632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb226e92-f47b-4b1c-ade5-c18121abaefa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:17.623829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb226e92-f47b-4b1c-ade5-c18121abaefa admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:17.624487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb226e92-f47b-4b1c-ade5-c18121abaefa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:17.626493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb226e92-f47b-4b1c-ade5-c18121abaefa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:17.653419 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1829/3703] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:01:17 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:17.657606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75183e-29ef-4ed4-8137-7a01da70098c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:17.668969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75183e-29ef-4ed4-8137-7a01da70098c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:17.675288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a63f5-9722-46cb-b758-098288e9ed9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:17.676206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a63f5-9722-46cb-b758-098288e9ed9b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:17.676650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a63f5-9722-46cb-b758-098288e9ed9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:17.677162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a63f5-9722-46cb-b758-098288e9ed9b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:17.678069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a63f5-9722-46cb-b758-098288e9ed9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:17.679930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a63f5-9722-46cb-b758-098288e9ed9b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:17.695115 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1875/3704] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:17.707326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb226e92-f47b-4b1c-ade5-c18121abaefa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:17.723535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb226e92-f47b-4b1c-ade5-c18121abaefa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:17.732046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60934eb2-bd8c-43bc-8b88-9cf7268ef3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:17.732732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60934eb2-bd8c-43bc-8b88-9cf7268ef3c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:17.733075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60934eb2-bd8c-43bc-8b88-9cf7268ef3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:17.733354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60934eb2-bd8c-43bc-8b88-9cf7268ef3c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:17.734219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60934eb2-bd8c-43bc-8b88-9cf7268ef3c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:17.736017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60934eb2-bd8c-43bc-8b88-9cf7268ef3c5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:17.763156 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1830/3705] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:17.764237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a63f5-9722-46cb-b758-098288e9ed9b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:17.779923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a63f5-9722-46cb-b758-098288e9ed9b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:17.786635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb26c-907b-4c59-9869-5c0fa7ede8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:17.788150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb26c-907b-4c59-9869-5c0fa7ede8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:17.789971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb26c-907b-4c59-9869-5c0fa7ede8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:17.790639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb26c-907b-4c59-9869-5c0fa7ede8bd admin admin] PATH_INFO: `/v3/projects/7d15750c4924457ca34c30658ecaa9a4/users/a106e1a727b1489eb4ff5678c512b481/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:17.821511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efb26c-907b-4c59-9869-5c0fa7ede8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d15750c4924457ca34c30658ecaa9a4, user_id=a106e1a727b1489eb4ff5678c512b481, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:17.824985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efb26c-907b-4c59-9869-5c0fa7ede8bd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:17.861291 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1876/3706] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:01:17 2022] PUT /identity/v3/projects/7d15750c4924457ca34c30658ecaa9a4/users/a106e1a727b1489eb4ff5678c512b481/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:17.866836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c89cb9-df91-49b9-bd48-f59ba9c47849 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:17.868038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c89cb9-df91-49b9-bd48-f59ba9c47849 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:17.868479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c89cb9-df91-49b9-bd48-f59ba9c47849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:17.909304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-32c89cb9-df91-49b9-bd48-f59ba9c47849 None None] MFA Rules not processed for user `a106e1a727b1489eb4ff5678c512b481`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:17.910815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-32c89cb9-df91-49b9-bd48-f59ba9c47849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:17.988594 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1831/3707] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:17.993270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb26c-907b-4c59-9869-5c0fa7ede8bd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:18.085918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb26c-907b-4c59-9869-5c0fa7ede8bd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:18.091830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c7747-5b87-4d6f-b168-b53e1b9aa394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:18.092546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c7747-5b87-4d6f-b168-b53e1b9aa394 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:18.092828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c7747-5b87-4d6f-b168-b53e1b9aa394 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:18.093127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c7747-5b87-4d6f-b168-b53e1b9aa394 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:18.093756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c7747-5b87-4d6f-b168-b53e1b9aa394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:18.096808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c7747-5b87-4d6f-b168-b53e1b9aa394 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:18.148890 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1877/3708] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:01:17 2022] POST /identity/v3/projects => generated 393 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:01:18.149796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c89cb9-df91-49b9-bd48-f59ba9c47849 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:18.162233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c89cb9-df91-49b9-bd48-f59ba9c47849 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:18.222240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:18.253329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad896933-41eb-42be-92fc-af3ae3c954d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:18.254776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad896933-41eb-42be-92fc-af3ae3c954d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:18.255069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad896933-41eb-42be-92fc-af3ae3c954d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:18.255478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad896933-41eb-42be-92fc-af3ae3c954d4 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:18.256318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad896933-41eb-42be-92fc-af3ae3c954d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:18.258307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad896933-41eb-42be-92fc-af3ae3c954d4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:18.288670 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1832/3709] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:01:18 2022] POST /identity/v3/users => generated 303 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:01:18.289711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c7747-5b87-4d6f-b168-b53e1b9aa394 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:18.304928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c7747-5b87-4d6f-b168-b53e1b9aa394 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:18.322343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc600170-eed4-43c2-9edc-ae7964d3711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:18.323167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc600170-eed4-43c2-9edc-ae7964d3711e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:18.323559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc600170-eed4-43c2-9edc-ae7964d3711e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:18.323939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc600170-eed4-43c2-9edc-ae7964d3711e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:18.325637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc600170-eed4-43c2-9edc-ae7964d3711e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:18.327612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc600170-eed4-43c2-9edc-ae7964d3711e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:18.330356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:18.353684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad896933-41eb-42be-92fc-af3ae3c954d4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:18.365174 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1878/3710] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:01:18.367102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad896933-41eb-42be-92fc-af3ae3c954d4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:18.374747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6901f2a-111e-4a6f-95bd-4b022e0ede77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:18.376931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6901f2a-111e-4a6f-95bd-4b022e0ede77 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:18.377561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6901f2a-111e-4a6f-95bd-4b022e0ede77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:18.378129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6901f2a-111e-4a6f-95bd-4b022e0ede77 admin admin] PATH_INFO: `/v3/projects/94aa25977bd241e69033438537070864/users/95183bf477a248a489ec81056e6f25ec/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:18.397419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6901f2a-111e-4a6f-95bd-4b022e0ede77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94aa25977bd241e69033438537070864, user_id=95183bf477a248a489ec81056e6f25ec, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:18.400745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6901f2a-111e-4a6f-95bd-4b022e0ede77 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:18.421699 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1833/3711] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:01:18 2022] PUT /identity/v3/projects/94aa25977bd241e69033438537070864/users/95183bf477a248a489ec81056e6f25ec/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:18.425532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc600170-eed4-43c2-9edc-ae7964d3711e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:18.442187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc600170-eed4-43c2-9edc-ae7964d3711e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:18.574041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a97ee3-933f-4b69-9bf6-2305cc232d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:18.574701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a97ee3-933f-4b69-9bf6-2305cc232d8a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:18.574993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a97ee3-933f-4b69-9bf6-2305cc232d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:18.575272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a97ee3-933f-4b69-9bf6-2305cc232d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:18.575932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a97ee3-933f-4b69-9bf6-2305cc232d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:18.577612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a97ee3-933f-4b69-9bf6-2305cc232d8a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:18.589183 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1879/3712] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:18.593531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6901f2a-111e-4a6f-95bd-4b022e0ede77 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:18.606742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6901f2a-111e-4a6f-95bd-4b022e0ede77 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:18.620173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28fce7-b34a-4311-aa3a-db6ed777b318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:18.621052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28fce7-b34a-4311-aa3a-db6ed777b318 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:18.621877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28fce7-b34a-4311-aa3a-db6ed777b318 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:18.622259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28fce7-b34a-4311-aa3a-db6ed777b318 admin admin] PATH_INFO: `/v3/projects/94aa25977bd241e69033438537070864/users/95183bf477a248a489ec81056e6f25ec/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:18.624931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28fce7-b34a-4311-aa3a-db6ed777b318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94aa25977bd241e69033438537070864, user_id=95183bf477a248a489ec81056e6f25ec, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:18.627952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28fce7-b34a-4311-aa3a-db6ed777b318 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:18.640006 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1834/3713] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:01:18 2022] PUT /identity/v3/projects/94aa25977bd241e69033438537070864/users/95183bf477a248a489ec81056e6f25ec/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:01:18.650117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdc1ad-2bc5-4d2c-b4c0-30ec3bdbb14a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:18.650961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdc1ad-2bc5-4d2c-b4c0-30ec3bdbb14a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:18.650961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdc1ad-2bc5-4d2c-b4c0-30ec3bdbb14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:18.684919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-66cdc1ad-2bc5-4d2c-b4c0-30ec3bdbb14a None None] MFA Rules not processed for user `95183bf477a248a489ec81056e6f25ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:18.686449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-66cdc1ad-2bc5-4d2c-b4c0-30ec3bdbb14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:18.770101 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1880/3714] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:18 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:18.781741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28fce7-b34a-4311-aa3a-db6ed777b318 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:18.783556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cf28fce7-b34a-4311-aa3a-db6ed777b318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:18.880460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28fce7-b34a-4311-aa3a-db6ed777b318 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:18.888676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac040f7-78c8-42c5-b1ed-be60f276fc06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:18.889872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac040f7-78c8-42c5-b1ed-be60f276fc06 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:18.890265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac040f7-78c8-42c5-b1ed-be60f276fc06 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:18.891228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac040f7-78c8-42c5-b1ed-be60f276fc06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:18.913162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac040f7-78c8-42c5-b1ed-be60f276fc06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:18.916776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac040f7-78c8-42c5-b1ed-be60f276fc06 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:18.939472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fa035-9eed-4c44-9868-b1b638dc2488 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:18.940455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fa035-9eed-4c44-9868-b1b638dc2488 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:18.941409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fa035-9eed-4c44-9868-b1b638dc2488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:18.957942 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1835/3715] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:18 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:18.975245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ed2fa035-9eed-4c44-9868-b1b638dc2488 None None] MFA Rules not processed for user `50bba3c790c44425a9786b8d1404df28`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:18.977003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ed2fa035-9eed-4c44-9868-b1b638dc2488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:19.004130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac040f7-78c8-42c5-b1ed-be60f276fc06 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:19.025569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac040f7-78c8-42c5-b1ed-be60f276fc06 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:19.040579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2b792-d885-434a-82bc-83a83bfbc871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:19.041752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2b792-d885-434a-82bc-83a83bfbc871 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:19.042144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2b792-d885-434a-82bc-83a83bfbc871 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:19.043003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2b792-d885-434a-82bc-83a83bfbc871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:19.076719 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1881/3716] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:19.134967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2b792-d885-434a-82bc-83a83bfbc871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:19.138781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2b792-d885-434a-82bc-83a83bfbc871 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:19.182905 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1836/3717] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:19 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:19.183944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fa035-9eed-4c44-9868-b1b638dc2488 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:19.243941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:19.318487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fa035-9eed-4c44-9868-b1b638dc2488 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:19.332863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:19.334097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:19.334097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:19.334419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] PATH_INFO: `/v3/users/a106e1a727b1489eb4ff5678c512b481` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:19.335646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a106e1a727b1489eb4ff5678c512b481)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:19.338084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:19.456440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a106e1a727b1489eb4ff5678c512b481'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:19.478103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a106e1a727b1489eb4ff5678c512b481'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:19.499147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a106e1a727b1489eb4ff5678c512b481'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:19.499780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a106e1a727b1489eb4ff5678c512b481'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:19.500429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a106e1a727b1489eb4ff5678c512b481'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:19.526163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8e8d3-76b5-4eb2-bd31-5e898745c95e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:19.526501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8e8d3-76b5-4eb2-bd31-5e898745c95e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:19.526726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8e8d3-76b5-4eb2-bd31-5e898745c95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:19.530377 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1882/3718] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:01:19 2022] DELETE /identity/v3/users/a106e1a727b1489eb4ff5678c512b481 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:19.564803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-27d8e8d3-76b5-4eb2-bd31-5e898745c95e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:19.568479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-27d8e8d3-76b5-4eb2-bd31-5e898745c95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:19.576047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:19.707416 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1837/3719] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:19.724668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:19.726475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:19.836430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e852d-f9f7-429e-8ad6-712be8699062 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:19.843826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1ebe2-05aa-4277-98ea-ee0a6fa21ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:19.844779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1ebe2-05aa-4277-98ea-ee0a6fa21ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:19.845284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1ebe2-05aa-4277-98ea-ee0a6fa21ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:19.845910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1ebe2-05aa-4277-98ea-ee0a6fa21ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:19.862141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1ebe2-05aa-4277-98ea-ee0a6fa21ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:19.864726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1ebe2-05aa-4277-98ea-ee0a6fa21ab2 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:19.896881 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1883/3720] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:20.735359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8e8d3-76b5-4eb2-bd31-5e898745c95e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:20.737310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-27d8e8d3-76b5-4eb2-bd31-5e898745c95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:20.753716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8e8d3-76b5-4eb2-bd31-5e898745c95e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:20.761169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721ebe4-cc3f-4bfe-8af7-75144b33d445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:20.762066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721ebe4-cc3f-4bfe-8af7-75144b33d445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:20.762853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721ebe4-cc3f-4bfe-8af7-75144b33d445 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:20.763330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721ebe4-cc3f-4bfe-8af7-75144b33d445 admin admin] PATH_INFO: `/v3/projects/7d15750c4924457ca34c30658ecaa9a4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:20.766614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721ebe4-cc3f-4bfe-8af7-75144b33d445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d15750c4924457ca34c30658ecaa9a4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:20.769391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721ebe4-cc3f-4bfe-8af7-75144b33d445 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:20.836840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e721ebe4-cc3f-4bfe-8af7-75144b33d445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d15750c4924457ca34c30658ecaa9a4', 'request_id': 'req-e721ebe4-cc3f-4bfe-8af7-75144b33d445'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:20.853305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e721ebe4-cc3f-4bfe-8af7-75144b33d445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d15750c4924457ca34c30658ecaa9a4', 'request_id': 'req-e721ebe4-cc3f-4bfe-8af7-75144b33d445'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:20.863311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e721ebe4-cc3f-4bfe-8af7-75144b33d445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d15750c4924457ca34c30658ecaa9a4', 'request_id': 'req-e721ebe4-cc3f-4bfe-8af7-75144b33d445'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:20.886805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e721ebe4-cc3f-4bfe-8af7-75144b33d445 admin admin] The token cache is being invalidate because project 7d15750c4924457ca34c30658ecaa9a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:01:20.887196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e721ebe4-cc3f-4bfe-8af7-75144b33d445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:20.888051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e721ebe4-cc3f-4bfe-8af7-75144b33d445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:20.890742 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1838/3721] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:01:20 2022] DELETE /identity/v3/projects/7d15750c4924457ca34c30658ecaa9a4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:20.894786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1ebe2-05aa-4277-98ea-ee0a6fa21ab2 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:20.909904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1ebe2-05aa-4277-98ea-ee0a6fa21ab2 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:20.992371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:21.010895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:21.011438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:21.011799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:21.012829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] PATH_INFO: `/v3/users/95183bf477a248a489ec81056e6f25ec` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:21.014264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] RBAC: Authorizing `identity:delete_user(user_id=95183bf477a248a489ec81056e6f25ec)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:21.016414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:21.091509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95183bf477a248a489ec81056e6f25ec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:21.168666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7169f-0062-436c-9057-de56238bcee1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:21.169225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7169f-0062-436c-9057-de56238bcee1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:21.169670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7169f-0062-436c-9057-de56238bcee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:21.169839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95183bf477a248a489ec81056e6f25ec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:21.183245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95183bf477a248a489ec81056e6f25ec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:21.187418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95183bf477a248a489ec81056e6f25ec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:21.189113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95183bf477a248a489ec81056e6f25ec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:21.244761 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1884/3722] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:01:20 2022] DELETE /identity/v3/users/95183bf477a248a489ec81056e6f25ec => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:21.294814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-46e7169f-0062-436c-9057-de56238bcee1 None None] MFA Rules not processed for user `5d1dda56f7cb4adda980038b707d34de`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:21.296092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-46e7169f-0062-436c-9057-de56238bcee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:21.422320 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1839/3723] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:21.437213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:21.439697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:21.676768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11dd5e-f2a6-4325-aa43-d5bb30ab24df admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:21.681591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d9823-b261-46c8-9849-f7bca9c7d0a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:21.683210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d9823-b261-46c8-9849-f7bca9c7d0a4 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:21.683660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d9823-b261-46c8-9849-f7bca9c7d0a4 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:21.684252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d9823-b261-46c8-9849-f7bca9c7d0a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:21.728632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d9823-b261-46c8-9849-f7bca9c7d0a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:21.731685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d9823-b261-46c8-9849-f7bca9c7d0a4 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:21.747363 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1885/3724] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:21.945323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7169f-0062-436c-9057-de56238bcee1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:21.947080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-46e7169f-0062-436c-9057-de56238bcee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:21.995093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:22.054198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7169f-0062-436c-9057-de56238bcee1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:22.063981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55785e0f-a402-4e2a-99b8-c76488d9e964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:22.065143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55785e0f-a402-4e2a-99b8-c76488d9e964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:22.065824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55785e0f-a402-4e2a-99b8-c76488d9e964 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:22.066327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55785e0f-a402-4e2a-99b8-c76488d9e964 admin admin] PATH_INFO: `/v3/projects/94aa25977bd241e69033438537070864` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:22.067728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55785e0f-a402-4e2a-99b8-c76488d9e964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94aa25977bd241e69033438537070864)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:22.073995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55785e0f-a402-4e2a-99b8-c76488d9e964 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:22.217872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55785e0f-a402-4e2a-99b8-c76488d9e964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94aa25977bd241e69033438537070864', 'request_id': 'req-55785e0f-a402-4e2a-99b8-c76488d9e964'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:22.231634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55785e0f-a402-4e2a-99b8-c76488d9e964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94aa25977bd241e69033438537070864', 'request_id': 'req-55785e0f-a402-4e2a-99b8-c76488d9e964'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:22.248866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55785e0f-a402-4e2a-99b8-c76488d9e964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94aa25977bd241e69033438537070864', 'request_id': 'req-55785e0f-a402-4e2a-99b8-c76488d9e964'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:22.278492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55785e0f-a402-4e2a-99b8-c76488d9e964 admin admin] The token cache is being invalidate because project 94aa25977bd241e69033438537070864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:01:22.279079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55785e0f-a402-4e2a-99b8-c76488d9e964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:22.284712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55785e0f-a402-4e2a-99b8-c76488d9e964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:22.286623 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1840/3725] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:01:21 2022] DELETE /identity/v3/projects/94aa25977bd241e69033438537070864 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:22.293616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dccbb-ea2e-4019-9c5e-f197e0e91895 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:22.294197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dccbb-ea2e-4019-9c5e-f197e0e91895 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:22.294919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dccbb-ea2e-4019-9c5e-f197e0e91895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:22.373321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1a0dccbb-ea2e-4019-9c5e-f197e0e91895 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:22.379127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1a0dccbb-ea2e-4019-9c5e-f197e0e91895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:22.527671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:22.575259 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1886/3726] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:22.584879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd9824-ea17-411c-9804-5e1c1e5f5ecf None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:22.585442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd9824-ea17-411c-9804-5e1c1e5f5ecf None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:22.585950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd9824-ea17-411c-9804-5e1c1e5f5ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:22.633108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3acd9824-ea17-411c-9804-5e1c1e5f5ecf None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:22.636068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3acd9824-ea17-411c-9804-5e1c1e5f5ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:22.668940 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1841/3727] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:22.669188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dccbb-ea2e-4019-9c5e-f197e0e91895 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:22.696835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dccbb-ea2e-4019-9c5e-f197e0e91895 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:22.711381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae4f19-cc87-4fca-8cf5-ba2c93bb85dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:22.712041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae4f19-cc87-4fca-8cf5-ba2c93bb85dd admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:22.712338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae4f19-cc87-4fca-8cf5-ba2c93bb85dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:22.712614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae4f19-cc87-4fca-8cf5-ba2c93bb85dd admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:22.713373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae4f19-cc87-4fca-8cf5-ba2c93bb85dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:01:22.713697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae4f19-cc87-4fca-8cf5-ba2c93bb85dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:22.715380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae4f19-cc87-4fca-8cf5-ba2c93bb85dd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:22.759216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd9824-ea17-411c-9804-5e1c1e5f5ecf None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:22.775553 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1887/3728] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:01:22.786086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd9824-ea17-411c-9804-5e1c1e5f5ecf None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:22.826861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf8269-fbc4-4ec3-ad72-a4698077072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:22.827548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf8269-fbc4-4ec3-ad72-a4698077072b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:22.827913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf8269-fbc4-4ec3-ad72-a4698077072b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:22.828256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf8269-fbc4-4ec3-ad72-a4698077072b admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:22.829018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf8269-fbc4-4ec3-ad72-a4698077072b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:22.840082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf8269-fbc4-4ec3-ad72-a4698077072b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:22.908542 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1842/3729] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:01:22 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:22.910388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae4f19-cc87-4fca-8cf5-ba2c93bb85dd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:22.928216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae4f19-cc87-4fca-8cf5-ba2c93bb85dd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:23.090910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dce4de-0630-4586-b86a-9501a66d044f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:23.092165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dce4de-0630-4586-b86a-9501a66d044f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:23.092545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dce4de-0630-4586-b86a-9501a66d044f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:23.092938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dce4de-0630-4586-b86a-9501a66d044f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:23.093992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dce4de-0630-4586-b86a-9501a66d044f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:23.097116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dce4de-0630-4586-b86a-9501a66d044f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:23.162578 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1888/3730] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:01:22 2022] POST /identity/v3/users => generated 290 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:23.178318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf8269-fbc4-4ec3-ad72-a4698077072b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:23.194227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf8269-fbc4-4ec3-ad72-a4698077072b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:23.200765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d9a7f-213a-4bc9-b1c9-730fab810922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:23.201532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d9a7f-213a-4bc9-b1c9-730fab810922 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:23.201882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d9a7f-213a-4bc9-b1c9-730fab810922 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:23.202277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d9a7f-213a-4bc9-b1c9-730fab810922 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:23.203001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d9a7f-213a-4bc9-b1c9-730fab810922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:23.204762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d9a7f-213a-4bc9-b1c9-730fab810922 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:23.222125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dce4de-0630-4586-b86a-9501a66d044f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:23.227750 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1843/3731] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:23.235997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dce4de-0630-4586-b86a-9501a66d044f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:23.243856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72bd87-e560-4ff3-8e64-48227f8f0f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:23.244798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72bd87-e560-4ff3-8e64-48227f8f0f65 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:23.260979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72bd87-e560-4ff3-8e64-48227f8f0f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:23.260979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72bd87-e560-4ff3-8e64-48227f8f0f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:23.260979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72bd87-e560-4ff3-8e64-48227f8f0f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:23.260979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72bd87-e560-4ff3-8e64-48227f8f0f65 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:23.277242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d9a7f-213a-4bc9-b1c9-730fab810922 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:23.290628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d9a7f-213a-4bc9-b1c9-730fab810922 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:23.291494 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1889/3732] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:23.302583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790817f-32c7-404c-9ca3-5ac9b91208d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:23.303347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790817f-32c7-404c-9ca3-5ac9b91208d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:23.303717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790817f-32c7-404c-9ca3-5ac9b91208d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:23.304071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790817f-32c7-404c-9ca3-5ac9b91208d7 admin admin] PATH_INFO: `/v3/projects/30d39cd627084dda8cb00e993f5d19a1/users/3a394ae937c54b2a82c89628d4feeb72/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:23.351137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790817f-32c7-404c-9ca3-5ac9b91208d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30d39cd627084dda8cb00e993f5d19a1, user_id=3a394ae937c54b2a82c89628d4feeb72, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:23.357073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790817f-32c7-404c-9ca3-5ac9b91208d7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:23.381925 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1844/3733] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:01:23 2022] PUT /identity/v3/projects/30d39cd627084dda8cb00e993f5d19a1/users/3a394ae937c54b2a82c89628d4feeb72/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:23.386941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d189d4-7c78-4e21-9db6-c1e317273b0c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:23.387994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d189d4-7c78-4e21-9db6-c1e317273b0c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:23.388099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d189d4-7c78-4e21-9db6-c1e317273b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:23.435627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-93d189d4-7c78-4e21-9db6-c1e317273b0c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:23.439371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-93d189d4-7c78-4e21-9db6-c1e317273b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:23.615281 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1890/3734] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:23.630644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790817f-32c7-404c-9ca3-5ac9b91208d7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:23.632857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2790817f-32c7-404c-9ca3-5ac9b91208d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:23.863431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:23.885138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790817f-32c7-404c-9ca3-5ac9b91208d7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:23.903279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76b346-1132-4cd9-ae6a-4e3d7a749be0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:23.903921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76b346-1132-4cd9-ae6a-4e3d7a749be0 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:23.904791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76b346-1132-4cd9-ae6a-4e3d7a749be0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:23.905095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76b346-1132-4cd9-ae6a-4e3d7a749be0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:23.947839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a76b346-1132-4cd9-ae6a-4e3d7a749be0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:23.950420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a76b346-1132-4cd9-ae6a-4e3d7a749be0 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:23.965814 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1845/3735] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:33.592938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:33.660028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c8616-fa72-4ebf-baa5-32252a6da936 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:33.660028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c8616-fa72-4ebf-baa5-32252a6da936 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:33.660028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c8616-fa72-4ebf-baa5-32252a6da936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:33.702581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cd5c8616-fa72-4ebf-baa5-32252a6da936 None None] MFA Rules not processed for user `3a394ae937c54b2a82c89628d4feeb72`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:33.703953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cd5c8616-fa72-4ebf-baa5-32252a6da936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:33.799542 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1891/3736] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:01:33.818262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76b346-1132-4cd9-ae6a-4e3d7a749be0 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:33.819729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8a76b346-1132-4cd9-ae6a-4e3d7a749be0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:33.954666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76b346-1132-4cd9-ae6a-4e3d7a749be0 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:33.961404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74112131-0d01-4652-abbb-0eed3b33c340 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:33.962045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74112131-0d01-4652-abbb-0eed3b33c340 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:33.962333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74112131-0d01-4652-abbb-0eed3b33c340 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:33.962614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74112131-0d01-4652-abbb-0eed3b33c340 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:33.973529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74112131-0d01-4652-abbb-0eed3b33c340 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:33.975221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74112131-0d01-4652-abbb-0eed3b33c340 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:34.005052 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1846/3737] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:34.812344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f06979-8f39-4e88-8466-cc7b29f102f0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:34.812909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f06979-8f39-4e88-8466-cc7b29f102f0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:34.812909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f06979-8f39-4e88-8466-cc7b29f102f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:34.838465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f1f06979-8f39-4e88-8466-cc7b29f102f0 None None] MFA Rules not processed for user `3a394ae937c54b2a82c89628d4feeb72`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:34.841067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f1f06979-8f39-4e88-8466-cc7b29f102f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:34.851172 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1892/3738] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:01:34.918583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce362c1-7a6e-4a89-b0bc-2487ef586552 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:34.919077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce362c1-7a6e-4a89-b0bc-2487ef586552 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:34.919169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce362c1-7a6e-4a89-b0bc-2487ef586552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:34.984269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5ce362c1-7a6e-4a89-b0bc-2487ef586552 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:34.985940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5ce362c1-7a6e-4a89-b0bc-2487ef586552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:35.075810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1847/3739] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:01:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:35.155505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f06979-8f39-4e88-8466-cc7b29f102f0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:35.161620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f1f06979-8f39-4e88-8466-cc7b29f102f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:35.319593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f06979-8f39-4e88-8466-cc7b29f102f0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:35.326676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7537f-94d5-4319-8ecd-45b47dfa9b1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:35.327562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7537f-94d5-4319-8ecd-45b47dfa9b1c service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:35.328027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7537f-94d5-4319-8ecd-45b47dfa9b1c service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:35.328480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7537f-94d5-4319-8ecd-45b47dfa9b1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:35.339573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7537f-94d5-4319-8ecd-45b47dfa9b1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:35.342555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7537f-94d5-4319-8ecd-45b47dfa9b1c service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:35.355185 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1893/3740] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:01:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:01:44.124324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce362c1-7a6e-4a89-b0bc-2487ef586552 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:44.149698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5ce362c1-7a6e-4a89-b0bc-2487ef586552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:44.267955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:44.303873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce362c1-7a6e-4a89-b0bc-2487ef586552 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:44.310334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969087a-41da-4866-be91-4ac826c91435 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:44.311509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969087a-41da-4866-be91-4ac826c91435 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:44.312093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969087a-41da-4866-be91-4ac826c91435 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:44.312538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969087a-41da-4866-be91-4ac826c91435 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:44.314686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2969087a-41da-4866-be91-4ac826c91435 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:44.334899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969087a-41da-4866-be91-4ac826c91435 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:44.336884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969087a-41da-4866-be91-4ac826c91435 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:44.371236 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1848/3741] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:44.453142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7537f-94d5-4319-8ecd-45b47dfa9b1c service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:44.454642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d2f7537f-94d5-4319-8ecd-45b47dfa9b1c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:44.493118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7537f-94d5-4319-8ecd-45b47dfa9b1c service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:44.514805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731c93b-e372-4eea-8a9b-2d5a33ff2612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:44.515812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731c93b-e372-4eea-8a9b-2d5a33ff2612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:44.516266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731c93b-e372-4eea-8a9b-2d5a33ff2612 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:44.516702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731c93b-e372-4eea-8a9b-2d5a33ff2612 admin admin] PATH_INFO: `/v3/users/859a4503b4d84a4e8c3213e2de02492d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:44.518446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731c93b-e372-4eea-8a9b-2d5a33ff2612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=859a4503b4d84a4e8c3213e2de02492d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:44.522133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731c93b-e372-4eea-8a9b-2d5a33ff2612 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:44.608515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2731c93b-e372-4eea-8a9b-2d5a33ff2612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '859a4503b4d84a4e8c3213e2de02492d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:44.610123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:44.639205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2731c93b-e372-4eea-8a9b-2d5a33ff2612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '859a4503b4d84a4e8c3213e2de02492d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:44.661825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2731c93b-e372-4eea-8a9b-2d5a33ff2612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '859a4503b4d84a4e8c3213e2de02492d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:44.662740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2731c93b-e372-4eea-8a9b-2d5a33ff2612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '859a4503b4d84a4e8c3213e2de02492d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:44.663427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2731c93b-e372-4eea-8a9b-2d5a33ff2612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '859a4503b4d84a4e8c3213e2de02492d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:44.685082 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1894/3742] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:01:44 2022] DELETE /identity/v3/users/859a4503b4d84a4e8c3213e2de02492d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:44.967742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969087a-41da-4866-be91-4ac826c91435 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:44.983059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969087a-41da-4866-be91-4ac826c91435 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:45.051288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f23b34-7218-4138-a6af-81244452c6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:45.052215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f23b34-7218-4138-a6af-81244452c6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:45.052694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f23b34-7218-4138-a6af-81244452c6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:45.053186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f23b34-7218-4138-a6af-81244452c6a6 admin admin] PATH_INFO: `/v3/projects/18ae2766f0674492909ede79ed1832d7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:45.054339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f23b34-7218-4138-a6af-81244452c6a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18ae2766f0674492909ede79ed1832d7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:45.056353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f23b34-7218-4138-a6af-81244452c6a6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:45.115585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-86f23b34-7218-4138-a6af-81244452c6a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18ae2766f0674492909ede79ed1832d7', 'request_id': 'req-86f23b34-7218-4138-a6af-81244452c6a6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:45.132695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-86f23b34-7218-4138-a6af-81244452c6a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18ae2766f0674492909ede79ed1832d7', 'request_id': 'req-86f23b34-7218-4138-a6af-81244452c6a6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:45.142961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-86f23b34-7218-4138-a6af-81244452c6a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18ae2766f0674492909ede79ed1832d7', 'request_id': 'req-86f23b34-7218-4138-a6af-81244452c6a6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:45.158620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-86f23b34-7218-4138-a6af-81244452c6a6 admin admin] The token cache is being invalidate because project 18ae2766f0674492909ede79ed1832d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:01:45.159094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-86f23b34-7218-4138-a6af-81244452c6a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:45.159888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-86f23b34-7218-4138-a6af-81244452c6a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:45.161841 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1849/3743] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:01:44 2022] DELETE /identity/v3/projects/18ae2766f0674492909ede79ed1832d7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:45.166776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ca8d8-7a4c-4ba1-83fa-1df90d5c19f4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:45.167245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ca8d8-7a4c-4ba1-83fa-1df90d5c19f4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:45.167708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ca8d8-7a4c-4ba1-83fa-1df90d5c19f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:45.189435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-026ca8d8-7a4c-4ba1-83fa-1df90d5c19f4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:45.190960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-026ca8d8-7a4c-4ba1-83fa-1df90d5c19f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:45.260849 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1895/3744] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:45.271079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15085975-2775-495c-8898-ced9499b0d8e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:45.275068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15085975-2775-495c-8898-ced9499b0d8e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:45.275373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15085975-2775-495c-8898-ced9499b0d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:45.309571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-15085975-2775-495c-8898-ced9499b0d8e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:45.310985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-15085975-2775-495c-8898-ced9499b0d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:45.321534 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1850/3745] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:45.325733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ca8d8-7a4c-4ba1-83fa-1df90d5c19f4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:45.337242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ca8d8-7a4c-4ba1-83fa-1df90d5c19f4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:45.342178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d2237-f160-4461-ac56-cb060be62dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:45.342821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2237-f160-4461-ac56-cb060be62dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:45.344130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2237-f160-4461-ac56-cb060be62dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:45.344848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2237-f160-4461-ac56-cb060be62dc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:45.348969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d2237-f160-4461-ac56-cb060be62dc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:01:45.349310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d2237-f160-4461-ac56-cb060be62dc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:45.350964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d2237-f160-4461-ac56-cb060be62dc4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:45.369665 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1896/3746] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:01:45.374466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15085975-2775-495c-8898-ced9499b0d8e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:45.384301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15085975-2775-495c-8898-ced9499b0d8e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:45.391528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43c0a0-9d77-4d14-924e-d02c827b2bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:45.392151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43c0a0-9d77-4d14-924e-d02c827b2bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:45.392444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43c0a0-9d77-4d14-924e-d02c827b2bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:45.392726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43c0a0-9d77-4d14-924e-d02c827b2bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:45.393476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43c0a0-9d77-4d14-924e-d02c827b2bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:45.396259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43c0a0-9d77-4d14-924e-d02c827b2bd2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:45.468961 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1851/3747] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:01:45 2022] POST /identity/v3/projects => generated 387 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:45.470131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d2237-f160-4461-ac56-cb060be62dc4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:45.481877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d2237-f160-4461-ac56-cb060be62dc4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:45.632163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12dbdd-63a7-488c-9cbc-26a04807c2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:45.634746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12dbdd-63a7-488c-9cbc-26a04807c2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:45.635470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12dbdd-63a7-488c-9cbc-26a04807c2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:45.635932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12dbdd-63a7-488c-9cbc-26a04807c2c6 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:45.637250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12dbdd-63a7-488c-9cbc-26a04807c2c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:45.640385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12dbdd-63a7-488c-9cbc-26a04807c2c6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:45.682211 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1897/3748] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:01:45 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:01:45.685368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43c0a0-9d77-4d14-924e-d02c827b2bd2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:45.722649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43c0a0-9d77-4d14-924e-d02c827b2bd2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:45.755967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b1c86-6441-4760-af78-4c96a6678ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:45.756776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b1c86-6441-4760-af78-4c96a6678ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:45.757165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b1c86-6441-4760-af78-4c96a6678ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:45.757771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b1c86-6441-4760-af78-4c96a6678ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:45.760201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b1c86-6441-4760-af78-4c96a6678ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:45.761982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b1c86-6441-4760-af78-4c96a6678ca8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:45.774448 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1852/3749] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:45.778728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12dbdd-63a7-488c-9cbc-26a04807c2c6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:45.812977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12dbdd-63a7-488c-9cbc-26a04807c2c6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:45.829195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb3897-8b86-4d96-a4ad-68e7c4a323c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:45.829409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3897-8b86-4d96-a4ad-68e7c4a323c9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:45.829651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3897-8b86-4d96-a4ad-68e7c4a323c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:45.829906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3897-8b86-4d96-a4ad-68e7c4a323c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:45.830571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb3897-8b86-4d96-a4ad-68e7c4a323c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:45.832324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb3897-8b86-4d96-a4ad-68e7c4a323c9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:45.870501 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1898/3750] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:01:45.874778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b1c86-6441-4760-af78-4c96a6678ca8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:45.891988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b1c86-6441-4760-af78-4c96a6678ca8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:45.904777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e87ba-154a-4f89-9104-b3c820653ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:45.906284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e87ba-154a-4f89-9104-b3c820653ca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:45.906830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e87ba-154a-4f89-9104-b3c820653ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:45.907345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e87ba-154a-4f89-9104-b3c820653ca9 admin admin] PATH_INFO: `/v3/projects/250efefcd9e84269a090b0a7ec433c87/users/f7e2c822238a45d9b4d01ecf2cebc29a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:45.939926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e87ba-154a-4f89-9104-b3c820653ca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250efefcd9e84269a090b0a7ec433c87, user_id=f7e2c822238a45d9b4d01ecf2cebc29a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:45.954903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e87ba-154a-4f89-9104-b3c820653ca9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:45.978969 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1853/3751] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:01:45 2022] PUT /identity/v3/projects/250efefcd9e84269a090b0a7ec433c87/users/f7e2c822238a45d9b4d01ecf2cebc29a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:45.984357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9005a0f-f409-466c-befe-aa3dc85a43ec None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:45.984956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9005a0f-f409-466c-befe-aa3dc85a43ec None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:45.985415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9005a0f-f409-466c-befe-aa3dc85a43ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:46.009261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b9005a0f-f409-466c-befe-aa3dc85a43ec None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:46.010692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b9005a0f-f409-466c-befe-aa3dc85a43ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:46.078335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:46.135396 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1899/3752] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:46.175977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e87ba-154a-4f89-9104-b3c820653ca9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:46.181394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-167e87ba-154a-4f89-9104-b3c820653ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:46.195521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:46.300244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e87ba-154a-4f89-9104-b3c820653ca9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:46.305761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c936e1-b142-4bed-8d52-2094c5655bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:46.306399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c936e1-b142-4bed-8d52-2094c5655bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:46.306685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c936e1-b142-4bed-8d52-2094c5655bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:46.306972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c936e1-b142-4bed-8d52-2094c5655bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:46.321658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c936e1-b142-4bed-8d52-2094c5655bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:46.323206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c936e1-b142-4bed-8d52-2094c5655bb3 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:46.340398 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1854/3753] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:46.754575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9005a0f-f409-466c-befe-aa3dc85a43ec None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:46.756700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b9005a0f-f409-466c-befe-aa3dc85a43ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:46.771622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9005a0f-f409-466c-befe-aa3dc85a43ec None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:46.777959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:46.778992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:46.779525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:46.780021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] PATH_INFO: `/v3/users/7d240c36223b49cbb24e51bcb061a040` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:46.781585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d240c36223b49cbb24e51bcb061a040)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:46.784459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:46.866019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d240c36223b49cbb24e51bcb061a040'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:46.891231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d240c36223b49cbb24e51bcb061a040'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:46.906798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d240c36223b49cbb24e51bcb061a040'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:46.911901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d240c36223b49cbb24e51bcb061a040'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:46.912765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d240c36223b49cbb24e51bcb061a040'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:46.935442 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1900/3754] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:01:46 2022] DELETE /identity/v3/users/7d240c36223b49cbb24e51bcb061a040 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:46.939907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cb809-1325-491b-82dc-0c3bd7fd0936 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:46.940407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cb809-1325-491b-82dc-0c3bd7fd0936 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:46.941426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cb809-1325-491b-82dc-0c3bd7fd0936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:46.972058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c32cb809-1325-491b-82dc-0c3bd7fd0936 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:46.973509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c32cb809-1325-491b-82dc-0c3bd7fd0936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:47.055204 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1855/3755] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:47.068944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:47.070863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:47.162344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468170c-f89b-4f32-96f7-8bda1eee78c9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:47.166486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b419787-e16c-4dce-a6f8-cf1e5185def4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:47.167190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b419787-e16c-4dce-a6f8-cf1e5185def4 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:47.167544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b419787-e16c-4dce-a6f8-cf1e5185def4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:47.167933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b419787-e16c-4dce-a6f8-cf1e5185def4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:47.182665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b419787-e16c-4dce-a6f8-cf1e5185def4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:47.184887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b419787-e16c-4dce-a6f8-cf1e5185def4 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:47.210791 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1901/3756] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:47.666924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13984b-3eda-42e7-9c3c-d48e2f81ecd1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:47.668783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13984b-3eda-42e7-9c3c-d48e2f81ecd1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:47.669746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13984b-3eda-42e7-9c3c-d48e2f81ecd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:47.716330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b419787-e16c-4dce-a6f8-cf1e5185def4 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:47.718733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8b419787-e16c-4dce-a6f8-cf1e5185def4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:47.748295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ea13984b-3eda-42e7-9c3c-d48e2f81ecd1 None None] MFA Rules not processed for user `f7e2c822238a45d9b4d01ecf2cebc29a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:47.748643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ea13984b-3eda-42e7-9c3c-d48e2f81ecd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:47.759255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b419787-e16c-4dce-a6f8-cf1e5185def4 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:47.769507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c7594-a85e-47d9-8bbf-e67263557ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:47.770234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c7594-a85e-47d9-8bbf-e67263557ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:47.770531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c7594-a85e-47d9-8bbf-e67263557ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:47.770944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c7594-a85e-47d9-8bbf-e67263557ca0 admin admin] PATH_INFO: `/v3/projects/b9b0f161c27a4b83865404bc57c99eca` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:47.772131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c7594-a85e-47d9-8bbf-e67263557ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9b0f161c27a4b83865404bc57c99eca)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:47.775083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c7594-a85e-47d9-8bbf-e67263557ca0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:47.825071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:47.835156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-600c7594-a85e-47d9-8bbf-e67263557ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9b0f161c27a4b83865404bc57c99eca', 'request_id': 'req-600c7594-a85e-47d9-8bbf-e67263557ca0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:47.846359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-600c7594-a85e-47d9-8bbf-e67263557ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9b0f161c27a4b83865404bc57c99eca', 'request_id': 'req-600c7594-a85e-47d9-8bbf-e67263557ca0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:47.854122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-600c7594-a85e-47d9-8bbf-e67263557ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9b0f161c27a4b83865404bc57c99eca', 'request_id': 'req-600c7594-a85e-47d9-8bbf-e67263557ca0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:47.871560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-600c7594-a85e-47d9-8bbf-e67263557ca0 admin admin] The token cache is being invalidate because project b9b0f161c27a4b83865404bc57c99eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:01:47.871560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-600c7594-a85e-47d9-8bbf-e67263557ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:47.872956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-600c7594-a85e-47d9-8bbf-e67263557ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:47.874862 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1902/3757] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:01:47 2022] DELETE /identity/v3/projects/b9b0f161c27a4b83865404bc57c99eca => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:47.879687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca019ce-8e14-4ad7-b1ae-05278c1c0e64 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:47.879984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca019ce-8e14-4ad7-b1ae-05278c1c0e64 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:47.880272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca019ce-8e14-4ad7-b1ae-05278c1c0e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:47.904528 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1856/3758] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:47.916339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aa470-1f77-4766-afe5-eaf4caa6df96 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:47.916708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aa470-1f77-4766-afe5-eaf4caa6df96 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:47.917099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aa470-1f77-4766-afe5-eaf4caa6df96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:47.934661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9ca019ce-8e14-4ad7-b1ae-05278c1c0e64 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:47.940542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9ca019ce-8e14-4ad7-b1ae-05278c1c0e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:47.958126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a31aa470-1f77-4766-afe5-eaf4caa6df96 None None] MFA Rules not processed for user `f7e2c822238a45d9b4d01ecf2cebc29a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:47.963200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a31aa470-1f77-4766-afe5-eaf4caa6df96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:48.024368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:48.037519 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1903/3759] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:48.046553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f134e2b-7bc6-4471-99bc-39f801b9d29a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:48.046951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f134e2b-7bc6-4471-99bc-39f801b9d29a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:48.047144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f134e2b-7bc6-4471-99bc-39f801b9d29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:48.063195 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1857/3760] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:01:48.073180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6f134e2b-7bc6-4471-99bc-39f801b9d29a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:48.073695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657174e-3d30-4d0a-8dbc-036fafd90cc6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:48.074030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657174e-3d30-4d0a-8dbc-036fafd90cc6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:48.074373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657174e-3d30-4d0a-8dbc-036fafd90cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:48.074519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6f134e2b-7bc6-4471-99bc-39f801b9d29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:48.084099 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1904/3761] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:48.088147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f134e2b-7bc6-4471-99bc-39f801b9d29a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:48.102026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6657174e-3d30-4d0a-8dbc-036fafd90cc6 None None] MFA Rules not processed for user `f7e2c822238a45d9b4d01ecf2cebc29a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:48.103409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6657174e-3d30-4d0a-8dbc-036fafd90cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:48.108096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f134e2b-7bc6-4471-99bc-39f801b9d29a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:48.118942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d631ca-e09b-487f-a3af-fe5c8f2c3c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:48.119727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d631ca-e09b-487f-a3af-fe5c8f2c3c6f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:48.120010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d631ca-e09b-487f-a3af-fe5c8f2c3c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:48.120286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d631ca-e09b-487f-a3af-fe5c8f2c3c6f admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:48.121512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d631ca-e09b-487f-a3af-fe5c8f2c3c6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:01:48.121512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d631ca-e09b-487f-a3af-fe5c8f2c3c6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:48.122789 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1858/3762] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:48.123089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d631ca-e09b-487f-a3af-fe5c8f2c3c6f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:48.127494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657174e-3d30-4d0a-8dbc-036fafd90cc6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:48.129277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6657174e-3d30-4d0a-8dbc-036fafd90cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:48.178919 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1905/3763] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:01:48.183566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d631ca-e09b-487f-a3af-fe5c8f2c3c6f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:48.205839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d631ca-e09b-487f-a3af-fe5c8f2c3c6f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:48.220488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a743bce-7404-43ff-8e46-81967f78e4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:48.222642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a743bce-7404-43ff-8e46-81967f78e4ef admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:48.224208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a743bce-7404-43ff-8e46-81967f78e4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:48.224734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a743bce-7404-43ff-8e46-81967f78e4ef admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:48.225245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657174e-3d30-4d0a-8dbc-036fafd90cc6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:48.226305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a743bce-7404-43ff-8e46-81967f78e4ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:48.231525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61c643-8ead-40d7-9cfe-cf8d5ebe7103 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:48.232151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61c643-8ead-40d7-9cfe-cf8d5ebe7103 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:48.232437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61c643-8ead-40d7-9cfe-cf8d5ebe7103 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:48.232849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61c643-8ead-40d7-9cfe-cf8d5ebe7103 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:48.234131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a743bce-7404-43ff-8e46-81967f78e4ef admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:48.242297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61c643-8ead-40d7-9cfe-cf8d5ebe7103 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:48.244046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61c643-8ead-40d7-9cfe-cf8d5ebe7103 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:48.268868 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1859/3764] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:48.317838 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1906/3765] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:01:48 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:48.322273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61c643-8ead-40d7-9cfe-cf8d5ebe7103 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:48.339308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61c643-8ead-40d7-9cfe-cf8d5ebe7103 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:48.420433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f91ae-6bf5-4124-9fb7-9816338e53c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:48.421170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f91ae-6bf5-4124-9fb7-9816338e53c1 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:48.421456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f91ae-6bf5-4124-9fb7-9816338e53c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:48.421722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f91ae-6bf5-4124-9fb7-9816338e53c1 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:48.422373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f91ae-6bf5-4124-9fb7-9816338e53c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:48.424670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f91ae-6bf5-4124-9fb7-9816338e53c1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:48.457709 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1860/3766] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:01:48 2022] POST /identity/v3/users => generated 282 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:48.460778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a743bce-7404-43ff-8e46-81967f78e4ef admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:48.484029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a743bce-7404-43ff-8e46-81967f78e4ef admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:48.492576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fa5ff-f87f-4d0e-af8a-aef8a14663e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:48.493240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fa5ff-f87f-4d0e-af8a-aef8a14663e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:48.493525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fa5ff-f87f-4d0e-af8a-aef8a14663e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:48.493802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fa5ff-f87f-4d0e-af8a-aef8a14663e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:48.494449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fa5ff-f87f-4d0e-af8a-aef8a14663e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:48.497860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fa5ff-f87f-4d0e-af8a-aef8a14663e5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:48.516385 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1907/3767] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:48.520195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f91ae-6bf5-4124-9fb7-9816338e53c1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:48.534996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f91ae-6bf5-4124-9fb7-9816338e53c1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:48.540335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00afd12-798f-4d49-ab4f-cf62f5c5165b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:48.541127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00afd12-798f-4d49-ab4f-cf62f5c5165b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:48.541416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00afd12-798f-4d49-ab4f-cf62f5c5165b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:48.542322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00afd12-798f-4d49-ab4f-cf62f5c5165b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:48.543056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00afd12-798f-4d49-ab4f-cf62f5c5165b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:48.544888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00afd12-798f-4d49-ab4f-cf62f5c5165b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:48.561517 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1861/3768] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:48.562710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fa5ff-f87f-4d0e-af8a-aef8a14663e5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:48.579682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fa5ff-f87f-4d0e-af8a-aef8a14663e5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:48.588333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659ba32-bc44-4995-adea-e3e8378ce574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:48.591062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659ba32-bc44-4995-adea-e3e8378ce574 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:48.591519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659ba32-bc44-4995-adea-e3e8378ce574 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:48.591991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659ba32-bc44-4995-adea-e3e8378ce574 admin admin] PATH_INFO: `/v3/projects/eec5b0b9923849be853b15fc46e80c6c/users/9ed3befd359548c28e1cb5287d802e79/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:48.621778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659ba32-bc44-4995-adea-e3e8378ce574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec5b0b9923849be853b15fc46e80c6c, user_id=9ed3befd359548c28e1cb5287d802e79, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:48.625487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659ba32-bc44-4995-adea-e3e8378ce574 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:48.668490 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1908/3769] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:01:48 2022] PUT /identity/v3/projects/eec5b0b9923849be853b15fc46e80c6c/users/9ed3befd359548c28e1cb5287d802e79/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:48.669863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee5bcb-44cf-4513-8bbb-a80dcdff967b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:48.671212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee5bcb-44cf-4513-8bbb-a80dcdff967b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:48.671656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee5bcb-44cf-4513-8bbb-a80dcdff967b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:48.747793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b7ee5bcb-44cf-4513-8bbb-a80dcdff967b None None] MFA Rules not processed for user `9ed3befd359548c28e1cb5287d802e79`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:48.749098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b7ee5bcb-44cf-4513-8bbb-a80dcdff967b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:48.847845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:48.953783 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1862/3770] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:49.474190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f32c4c-c883-4770-92aa-5df24217b3ce None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:49.474730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f32c4c-c883-4770-92aa-5df24217b3ce None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:49.474975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f32c4c-c883-4770-92aa-5df24217b3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:49.492775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee5bcb-44cf-4513-8bbb-a80dcdff967b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:49.546736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-33f32c4c-c883-4770-92aa-5df24217b3ce None None] MFA Rules not processed for user `f7e2c822238a45d9b4d01ecf2cebc29a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:49.548223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-33f32c4c-c883-4770-92aa-5df24217b3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:49.679058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:49.683503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee5bcb-44cf-4513-8bbb-a80dcdff967b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:49.695050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3888f-3d40-4b3d-8e5f-d3b8dceeba92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:49.696102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3888f-3d40-4b3d-8e5f-d3b8dceeba92 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:49.696803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3888f-3d40-4b3d-8e5f-d3b8dceeba92 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:49.697996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3888f-3d40-4b3d-8e5f-d3b8dceeba92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:49.703085 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1909/3771] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:49.719420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3888f-3d40-4b3d-8e5f-d3b8dceeba92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:49.721484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3888f-3d40-4b3d-8e5f-d3b8dceeba92 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:49.751905 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1863/3772] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:50.080541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f32c4c-c883-4770-92aa-5df24217b3ce None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:50.253089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f32c4c-c883-4770-92aa-5df24217b3ce None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:50.266257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:50.281374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:50.281815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:50.286997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] PATH_INFO: `/v3/users/9ed3befd359548c28e1cb5287d802e79` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:50.292138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ed3befd359548c28e1cb5287d802e79)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:50.300245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:50.415207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ed3befd359548c28e1cb5287d802e79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:50.457473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ed3befd359548c28e1cb5287d802e79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:50.478426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ed3befd359548c28e1cb5287d802e79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:50.485679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ed3befd359548c28e1cb5287d802e79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:50.487159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ed3befd359548c28e1cb5287d802e79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:50.535309 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1910/3773] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:01:50 2022] DELETE /identity/v3/users/9ed3befd359548c28e1cb5287d802e79 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:50.537475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb0ddc-2e33-4dfc-9a83-95457e9011d2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:50.537957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb0ddc-2e33-4dfc-9a83-95457e9011d2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:50.538500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb0ddc-2e33-4dfc-9a83-95457e9011d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:50.579922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-76eb0ddc-2e33-4dfc-9a83-95457e9011d2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:50.584378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-76eb0ddc-2e33-4dfc-9a83-95457e9011d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:50.864933 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1864/3774] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:01:50.878973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:50.880780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:51.031725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e0b4d-c2cf-4377-b8f7-65e6c4a9ac6d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:51.041744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a823a-cbdf-4170-970a-bfd486640eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:51.042381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a823a-cbdf-4170-970a-bfd486640eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:51.043607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a823a-cbdf-4170-970a-bfd486640eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:51.043892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a823a-cbdf-4170-970a-bfd486640eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:51.081156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a823a-cbdf-4170-970a-bfd486640eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:51.082638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a823a-cbdf-4170-970a-bfd486640eb4 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:51.110573 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1911/3775] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:51.417004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedfaba-2089-4c74-b0b0-3d4250772c35 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:51.417771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedfaba-2089-4c74-b0b0-3d4250772c35 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:51.418275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedfaba-2089-4c74-b0b0-3d4250772c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:51.454139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-aaedfaba-2089-4c74-b0b0-3d4250772c35 None None] MFA Rules not processed for user `f7e2c822238a45d9b4d01ecf2cebc29a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:51.455647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-aaedfaba-2089-4c74-b0b0-3d4250772c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:51.602937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:51.622935 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1865/3776] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:52.769315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a823a-cbdf-4170-970a-bfd486640eb4 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:52.771129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d38a823a-cbdf-4170-970a-bfd486640eb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:52.797388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a823a-cbdf-4170-970a-bfd486640eb4 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:52.818430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec709d-ccc9-4511-b889-93b2f01ed42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:52.819061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec709d-ccc9-4511-b889-93b2f01ed42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:52.819351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec709d-ccc9-4511-b889-93b2f01ed42a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:52.819624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec709d-ccc9-4511-b889-93b2f01ed42a admin admin] PATH_INFO: `/v3/projects/eec5b0b9923849be853b15fc46e80c6c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:52.821242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec709d-ccc9-4511-b889-93b2f01ed42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec5b0b9923849be853b15fc46e80c6c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:52.823702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec709d-ccc9-4511-b889-93b2f01ed42a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:52.898349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:52.950099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ecec709d-ccc9-4511-b889-93b2f01ed42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec5b0b9923849be853b15fc46e80c6c', 'request_id': 'req-ecec709d-ccc9-4511-b889-93b2f01ed42a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:52.981328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ecec709d-ccc9-4511-b889-93b2f01ed42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec5b0b9923849be853b15fc46e80c6c', 'request_id': 'req-ecec709d-ccc9-4511-b889-93b2f01ed42a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:52.995486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ecec709d-ccc9-4511-b889-93b2f01ed42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec5b0b9923849be853b15fc46e80c6c', 'request_id': 'req-ecec709d-ccc9-4511-b889-93b2f01ed42a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:53.039682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ecec709d-ccc9-4511-b889-93b2f01ed42a admin admin] The token cache is being invalidate because project eec5b0b9923849be853b15fc46e80c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:01:53.040053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ecec709d-ccc9-4511-b889-93b2f01ed42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:53.040804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ecec709d-ccc9-4511-b889-93b2f01ed42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:53.046329 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1912/3777] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:01:52 2022] DELETE /identity/v3/projects/eec5b0b9923849be853b15fc46e80c6c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:53.048335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11759a-e221-41cf-b465-19582d319e4b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:53.048829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11759a-e221-41cf-b465-19582d319e4b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:53.048976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11759a-e221-41cf-b465-19582d319e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:53.088054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ee11759a-e221-41cf-b465-19582d319e4b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:53.089336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ee11759a-e221-41cf-b465-19582d319e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:53.237553 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1866/3778] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:53.246633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59ba81-3716-4af5-b4f6-b5dfde666eb3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:53.247062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59ba81-3716-4af5-b4f6-b5dfde666eb3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:53.247352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59ba81-3716-4af5-b4f6-b5dfde666eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:53.306270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6b59ba81-3716-4af5-b4f6-b5dfde666eb3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:53.307540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6b59ba81-3716-4af5-b4f6-b5dfde666eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:53.330587 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1913/3779] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:53.334811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11759a-e221-41cf-b465-19582d319e4b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:53.352824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11759a-e221-41cf-b465-19582d319e4b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:53.367457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad06e1-e41e-4db4-84c2-0f2c3c3f0fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:53.368026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad06e1-e41e-4db4-84c2-0f2c3c3f0fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:53.368351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad06e1-e41e-4db4-84c2-0f2c3c3f0fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:53.368663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad06e1-e41e-4db4-84c2-0f2c3c3f0fa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:53.369405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad06e1-e41e-4db4-84c2-0f2c3c3f0fa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:01:53.375703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad06e1-e41e-4db4-84c2-0f2c3c3f0fa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:53.377889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad06e1-e41e-4db4-84c2-0f2c3c3f0fa0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:53.465747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59ba81-3716-4af5-b4f6-b5dfde666eb3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:53.477408 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1867/3780] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:01:53.481463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59ba81-3716-4af5-b4f6-b5dfde666eb3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:53.497163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145277e-6ade-4c6d-ab76-3a9e4fa18e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:53.497806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145277e-6ade-4c6d-ab76-3a9e4fa18e09 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:53.498191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145277e-6ade-4c6d-ab76-3a9e4fa18e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:53.498474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145277e-6ade-4c6d-ab76-3a9e4fa18e09 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:53.499246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145277e-6ade-4c6d-ab76-3a9e4fa18e09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:53.507186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145277e-6ade-4c6d-ab76-3a9e4fa18e09 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:53.616405 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1914/3781] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:01:53 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:53.618388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad06e1-e41e-4db4-84c2-0f2c3c3f0fa0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:53.650417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad06e1-e41e-4db4-84c2-0f2c3c3f0fa0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:53.825863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e81977-5484-4327-bd4c-89332d52a3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:53.826992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e81977-5484-4327-bd4c-89332d52a3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:53.827281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e81977-5484-4327-bd4c-89332d52a3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:53.827565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e81977-5484-4327-bd4c-89332d52a3a5 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:53.828359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e81977-5484-4327-bd4c-89332d52a3a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:53.831414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e81977-5484-4327-bd4c-89332d52a3a5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:53.883882 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1868/3782] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:01:53 2022] POST /identity/v3/users => generated 288 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:01:53.886305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145277e-6ade-4c6d-ab76-3a9e4fa18e09 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:53.903848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145277e-6ade-4c6d-ab76-3a9e4fa18e09 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:53.937308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad1ff9-7efd-49a6-95dd-123e40017457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:53.938755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad1ff9-7efd-49a6-95dd-123e40017457 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:53.939435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad1ff9-7efd-49a6-95dd-123e40017457 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:53.941866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad1ff9-7efd-49a6-95dd-123e40017457 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:53.943772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad1ff9-7efd-49a6-95dd-123e40017457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:53.947921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad1ff9-7efd-49a6-95dd-123e40017457 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:53.967914 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1915/3783] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:53.972108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e81977-5484-4327-bd4c-89332d52a3a5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:54.008298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e81977-5484-4327-bd4c-89332d52a3a5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:54.026393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf61e5-cd46-4f0a-9d81-8955246d88ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:54.027689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf61e5-cd46-4f0a-9d81-8955246d88ab admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:54.027996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf61e5-cd46-4f0a-9d81-8955246d88ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:54.028283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf61e5-cd46-4f0a-9d81-8955246d88ab admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:54.028985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf61e5-cd46-4f0a-9d81-8955246d88ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:54.030636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf61e5-cd46-4f0a-9d81-8955246d88ab admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:54.048307 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1869/3784] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:01:54.052247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad1ff9-7efd-49a6-95dd-123e40017457 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:54.067033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad1ff9-7efd-49a6-95dd-123e40017457 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:54.078161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479d903-5da1-4b78-871b-4da9176eaaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:54.079004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479d903-5da1-4b78-871b-4da9176eaaff admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:54.079298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479d903-5da1-4b78-871b-4da9176eaaff admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:54.079580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479d903-5da1-4b78-871b-4da9176eaaff admin admin] PATH_INFO: `/v3/projects/a5a5c2699a4b42f5a60fd808e824296b/users/1b2dd96d938b41d4ba40007aa072288b/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:54.097153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479d903-5da1-4b78-871b-4da9176eaaff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a5c2699a4b42f5a60fd808e824296b, user_id=1b2dd96d938b41d4ba40007aa072288b, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:54.111325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479d903-5da1-4b78-871b-4da9176eaaff admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:54.147318 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1916/3785] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:01:54 2022] PUT /identity/v3/projects/a5a5c2699a4b42f5a60fd808e824296b/users/1b2dd96d938b41d4ba40007aa072288b/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:54.153417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daeb00b-3399-4746-bb72-44309a855b1e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:54.154292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daeb00b-3399-4746-bb72-44309a855b1e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:54.154748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daeb00b-3399-4746-bb72-44309a855b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:54.180692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8daeb00b-3399-4746-bb72-44309a855b1e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:54.182367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8daeb00b-3399-4746-bb72-44309a855b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:54.263101 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1870/3786] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:54.278342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479d903-5da1-4b78-871b-4da9176eaaff admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:54.280819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3479d903-5da1-4b78-871b-4da9176eaaff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:54.377762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479d903-5da1-4b78-871b-4da9176eaaff admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:54.384713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a14eb-cd39-4ed3-9dd9-c19622c8f439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:54.385451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a14eb-cd39-4ed3-9dd9-c19622c8f439 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:54.385740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a14eb-cd39-4ed3-9dd9-c19622c8f439 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:54.386023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a14eb-cd39-4ed3-9dd9-c19622c8f439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:54.406748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a14eb-cd39-4ed3-9dd9-c19622c8f439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:54.408534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a14eb-cd39-4ed3-9dd9-c19622c8f439 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:54.437424 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1917/3787] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:54.906804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daeb00b-3399-4746-bb72-44309a855b1e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:54.908290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8daeb00b-3399-4746-bb72-44309a855b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:54.923716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:55.086750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daeb00b-3399-4746-bb72-44309a855b1e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:55.097276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b7dac-154e-4982-b824-5668e437f4b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:55.098305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b7dac-154e-4982-b824-5668e437f4b6 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:55.098648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b7dac-154e-4982-b824-5668e437f4b6 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:55.098938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b7dac-154e-4982-b824-5668e437f4b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:55.101472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9c6b7dac-154e-4982-b824-5668e437f4b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:55.211896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b7dac-154e-4982-b824-5668e437f4b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:55.216208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b7dac-154e-4982-b824-5668e437f4b6 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:55.230935 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1871/3788] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:01:59.342372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a14eb-cd39-4ed3-9dd9-c19622c8f439 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:59.344699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-767a14eb-cd39-4ed3-9dd9-c19622c8f439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:59.362609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a14eb-cd39-4ed3-9dd9-c19622c8f439 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:59.377774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:59.378827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:59.379374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:59.379918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] PATH_INFO: `/v3/users/9abfaac1da5f41f7a522b009f8073ac8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:59.381838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9abfaac1da5f41f7a522b009f8073ac8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:59.384400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:59.453762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:01:59.469545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9abfaac1da5f41f7a522b009f8073ac8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:59.494988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9abfaac1da5f41f7a522b009f8073ac8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:59.519403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9abfaac1da5f41f7a522b009f8073ac8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:59.520495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9abfaac1da5f41f7a522b009f8073ac8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:59.521256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9abfaac1da5f41f7a522b009f8073ac8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:01:59.543578 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1918/3789] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:01:59 2022] DELETE /identity/v3/users/9abfaac1da5f41f7a522b009f8073ac8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:01:59.547920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dc298-a564-4cda-b8f2-03a251d978b0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:59.548443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dc298-a564-4cda-b8f2-03a251d978b0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:59.548531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dc298-a564-4cda-b8f2-03a251d978b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:59.577629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-491dc298-a564-4cda-b8f2-03a251d978b0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:01:59.579095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-491dc298-a564-4cda-b8f2-03a251d978b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:59.666417 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1872/3790] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:01:59.684655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:01:59.686812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:01:59.809166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6019cf-5136-41af-b99b-10e2e4c9f299 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:01:59.814016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3edd14-0f24-462e-8bfa-41358c65322f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:01:59.815241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3edd14-0f24-462e-8bfa-41358c65322f service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:01:59.815782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3edd14-0f24-462e-8bfa-41358c65322f service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:01:59.816287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3edd14-0f24-462e-8bfa-41358c65322f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:01:59.830312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3edd14-0f24-462e-8bfa-41358c65322f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:01:59.834368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3edd14-0f24-462e-8bfa-41358c65322f service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:01:59.852544 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1919/3791] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:00.789456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491dc298-a564-4cda-b8f2-03a251d978b0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:00.790994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-491dc298-a564-4cda-b8f2-03a251d978b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:00.804351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491dc298-a564-4cda-b8f2-03a251d978b0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:00.809254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c56576-32ab-4e69-adeb-ba8b53d83fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:00.810052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c56576-32ab-4e69-adeb-ba8b53d83fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:00.810584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c56576-32ab-4e69-adeb-ba8b53d83fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:00.810868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c56576-32ab-4e69-adeb-ba8b53d83fe6 admin admin] PATH_INFO: `/v3/projects/75a128c413284fcb96ecbdb84d33c862` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:00.812015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c56576-32ab-4e69-adeb-ba8b53d83fe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75a128c413284fcb96ecbdb84d33c862)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:00.814350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c56576-32ab-4e69-adeb-ba8b53d83fe6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:00.873057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:00.890622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-75c56576-32ab-4e69-adeb-ba8b53d83fe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75a128c413284fcb96ecbdb84d33c862', 'request_id': 'req-75c56576-32ab-4e69-adeb-ba8b53d83fe6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:00.901587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-75c56576-32ab-4e69-adeb-ba8b53d83fe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75a128c413284fcb96ecbdb84d33c862', 'request_id': 'req-75c56576-32ab-4e69-adeb-ba8b53d83fe6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:00.910715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-75c56576-32ab-4e69-adeb-ba8b53d83fe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75a128c413284fcb96ecbdb84d33c862', 'request_id': 'req-75c56576-32ab-4e69-adeb-ba8b53d83fe6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:00.927793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-75c56576-32ab-4e69-adeb-ba8b53d83fe6 admin admin] The token cache is being invalidate because project 75a128c413284fcb96ecbdb84d33c862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:00.928230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-75c56576-32ab-4e69-adeb-ba8b53d83fe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:00.928997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-75c56576-32ab-4e69-adeb-ba8b53d83fe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:00.935530 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1873/3792] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:00 2022] DELETE /identity/v3/projects/75a128c413284fcb96ecbdb84d33c862 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:00.937737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ce969-fcf4-4cb2-9013-adf02ee8bf48 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:00.938241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ce969-fcf4-4cb2-9013-adf02ee8bf48 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:00.938420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ce969-fcf4-4cb2-9013-adf02ee8bf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:00.968326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d93ce969-fcf4-4cb2-9013-adf02ee8bf48 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:00.970012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d93ce969-fcf4-4cb2-9013-adf02ee8bf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:01.071684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39521d7-ed48-4a02-8adb-93c676736ec8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:01.071974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39521d7-ed48-4a02-8adb-93c676736ec8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:01.072252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39521d7-ed48-4a02-8adb-93c676736ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:01.084260 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1920/3793] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:01.101091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c39521d7-ed48-4a02-8adb-93c676736ec8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:01.102413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c39521d7-ed48-4a02-8adb-93c676736ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:01.118964 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1874/3794] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:01.124304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ce969-fcf4-4cb2-9013-adf02ee8bf48 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:01.149308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ce969-fcf4-4cb2-9013-adf02ee8bf48 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:01.163391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449fe1-1a13-4270-897b-dde710cc8400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:01.164496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449fe1-1a13-4270-897b-dde710cc8400 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:01.164496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449fe1-1a13-4270-897b-dde710cc8400 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:01.164712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449fe1-1a13-4270-897b-dde710cc8400 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:01.165262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06449fe1-1a13-4270-897b-dde710cc8400 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:01.165759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06449fe1-1a13-4270-897b-dde710cc8400 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:01.167374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06449fe1-1a13-4270-897b-dde710cc8400 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:01.188014 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1921/3795] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:01.189942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39521d7-ed48-4a02-8adb-93c676736ec8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:01.204834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39521d7-ed48-4a02-8adb-93c676736ec8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:01.227831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d71256-4c48-47d1-8bcf-ea5ea2c5ae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:01.228481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d71256-4c48-47d1-8bcf-ea5ea2c5ae39 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:01.228768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d71256-4c48-47d1-8bcf-ea5ea2c5ae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:01.229395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d71256-4c48-47d1-8bcf-ea5ea2c5ae39 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:01.230104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d71256-4c48-47d1-8bcf-ea5ea2c5ae39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:01.232081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d71256-4c48-47d1-8bcf-ea5ea2c5ae39 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:01.340885 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1875/3796] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:01 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:01.341061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449fe1-1a13-4270-897b-dde710cc8400 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:01.363104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449fe1-1a13-4270-897b-dde710cc8400 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:01.471418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:01.495441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b4b6-4b85-48c8-89ec-ac600195af59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:01.496810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b4b6-4b85-48c8-89ec-ac600195af59 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:01.497844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b4b6-4b85-48c8-89ec-ac600195af59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:01.499754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b4b6-4b85-48c8-89ec-ac600195af59 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:01.500609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9b4b6-4b85-48c8-89ec-ac600195af59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:01.507093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9b4b6-4b85-48c8-89ec-ac600195af59 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:01.553342 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1922/3797] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:01 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:01.557701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d71256-4c48-47d1-8bcf-ea5ea2c5ae39 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:01.578901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d71256-4c48-47d1-8bcf-ea5ea2c5ae39 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:01.594531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2650360-8620-4217-92e1-71d3a0aa8993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:01.595186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2650360-8620-4217-92e1-71d3a0aa8993 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:01.595477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2650360-8620-4217-92e1-71d3a0aa8993 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:01.596427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2650360-8620-4217-92e1-71d3a0aa8993 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:01.597122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2650360-8620-4217-92e1-71d3a0aa8993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:01.599291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2650360-8620-4217-92e1-71d3a0aa8993 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:01.632281 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1876/3798] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:01.636965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b4b6-4b85-48c8-89ec-ac600195af59 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:01.655039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b4b6-4b85-48c8-89ec-ac600195af59 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:01.676361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42145c76-9fec-4547-963e-75ca83040fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:01.677441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42145c76-9fec-4547-963e-75ca83040fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:01.678092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42145c76-9fec-4547-963e-75ca83040fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:01.678898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42145c76-9fec-4547-963e-75ca83040fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:01.682424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42145c76-9fec-4547-963e-75ca83040fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:01.687686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42145c76-9fec-4547-963e-75ca83040fd2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:01.719217 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1923/3799] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:01.720764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2650360-8620-4217-92e1-71d3a0aa8993 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:01.735844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2650360-8620-4217-92e1-71d3a0aa8993 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:01.748521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16186441-455c-4b26-a815-531fcebd2f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:01.749225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16186441-455c-4b26-a815-531fcebd2f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:01.749520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16186441-455c-4b26-a815-531fcebd2f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:01.750234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16186441-455c-4b26-a815-531fcebd2f19 admin admin] PATH_INFO: `/v3/projects/0d8fde784a4a45308d5ae744f89b52c3/users/93c44a51d8254cfdaed58ef0fd9e0145/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:01.770245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16186441-455c-4b26-a815-531fcebd2f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8fde784a4a45308d5ae744f89b52c3, user_id=93c44a51d8254cfdaed58ef0fd9e0145, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:01.773018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16186441-455c-4b26-a815-531fcebd2f19 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:01.814191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5cae5-2ce0-41d5-9458-faffa5346d9d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:01.817106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5cae5-2ce0-41d5-9458-faffa5346d9d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:01.817880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5cae5-2ce0-41d5-9458-faffa5346d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:01.827469 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1877/3800] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:01 2022] PUT /identity/v3/projects/0d8fde784a4a45308d5ae744f89b52c3/users/93c44a51d8254cfdaed58ef0fd9e0145/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:01.871808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-18b5cae5-2ce0-41d5-9458-faffa5346d9d None None] MFA Rules not processed for user `93c44a51d8254cfdaed58ef0fd9e0145`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:01.873346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-18b5cae5-2ce0-41d5-9458-faffa5346d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:01.980944 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1924/3801] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:01.990968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16186441-455c-4b26-a815-531fcebd2f19 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:01.993476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-16186441-455c-4b26-a815-531fcebd2f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:02.083463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16186441-455c-4b26-a815-531fcebd2f19 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:02.089998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58d614-23f5-4ab3-bb41-186cc92e2b40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:02.090638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58d614-23f5-4ab3-bb41-186cc92e2b40 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:02.090924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58d614-23f5-4ab3-bb41-186cc92e2b40 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:02.091206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58d614-23f5-4ab3-bb41-186cc92e2b40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:02.100930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58d614-23f5-4ab3-bb41-186cc92e2b40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:02.102632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58d614-23f5-4ab3-bb41-186cc92e2b40 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:02.115138 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1878/3802] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:02.501954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5cae5-2ce0-41d5-9458-faffa5346d9d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:02.677582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5cae5-2ce0-41d5-9458-faffa5346d9d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:02.695770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dae052-75a9-4ef0-8ddc-046f4fee3b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:02.696441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dae052-75a9-4ef0-8ddc-046f4fee3b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:02.696722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dae052-75a9-4ef0-8ddc-046f4fee3b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:02.697215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dae052-75a9-4ef0-8ddc-046f4fee3b5c admin admin] PATH_INFO: `/v3/users/93c44a51d8254cfdaed58ef0fd9e0145` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:02.700289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dae052-75a9-4ef0-8ddc-046f4fee3b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c44a51d8254cfdaed58ef0fd9e0145)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:02.702623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dae052-75a9-4ef0-8ddc-046f4fee3b5c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:02.819611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58d614-23f5-4ab3-bb41-186cc92e2b40 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:02.821368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7b58d614-23f5-4ab3-bb41-186cc92e2b40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:02.847068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e5dae052-75a9-4ef0-8ddc-046f4fee3b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c44a51d8254cfdaed58ef0fd9e0145'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:02.854045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:02.868005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e5dae052-75a9-4ef0-8ddc-046f4fee3b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c44a51d8254cfdaed58ef0fd9e0145'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:02.884354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e5dae052-75a9-4ef0-8ddc-046f4fee3b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c44a51d8254cfdaed58ef0fd9e0145'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:02.885847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e5dae052-75a9-4ef0-8ddc-046f4fee3b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c44a51d8254cfdaed58ef0fd9e0145'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:02.888056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e5dae052-75a9-4ef0-8ddc-046f4fee3b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c44a51d8254cfdaed58ef0fd9e0145'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:02.935516 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1925/3803] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:02 2022] DELETE /identity/v3/users/93c44a51d8254cfdaed58ef0fd9e0145 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:02.943827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1da8e-8c69-46b3-b97d-5381a46a3705 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:02.944231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1da8e-8c69-46b3-b97d-5381a46a3705 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:02.944631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1da8e-8c69-46b3-b97d-5381a46a3705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:02.953828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:03.012277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3fb1da8e-8c69-46b3-b97d-5381a46a3705 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:03.013700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3fb1da8e-8c69-46b3-b97d-5381a46a3705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:03.024278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58d614-23f5-4ab3-bb41-186cc92e2b40 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:03.102553 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1926/3804] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:03.103922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29409c7c-a5ef-4583-8b10-f532bc4c3666 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:03.104341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29409c7c-a5ef-4583-8b10-f532bc4c3666 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:03.104634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29409c7c-a5ef-4583-8b10-f532bc4c3666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:03.132948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-29409c7c-a5ef-4583-8b10-f532bc4c3666 None None] MFA Rules not processed for user `9884fe87dedd42bba0442aca48ebdc8d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:03.132948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-29409c7c-a5ef-4583-8b10-f532bc4c3666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:03.194284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e5e9d-8fcd-4d6f-831d-ebfa00594b8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:03.196305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5e9d-8fcd-4d6f-831d-ebfa00594b8c service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:03.197257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5e9d-8fcd-4d6f-831d-ebfa00594b8c service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:03.198187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5e9d-8fcd-4d6f-831d-ebfa00594b8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:03.204342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c73e5e9d-8fcd-4d6f-831d-ebfa00594b8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:03.232219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e5e9d-8fcd-4d6f-831d-ebfa00594b8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:03.235228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e5e9d-8fcd-4d6f-831d-ebfa00594b8c service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:03.249786 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1879/3805] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:02:03.251026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e5e9d-8fcd-4d6f-831d-ebfa00594b8c service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:03.252921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c73e5e9d-8fcd-4d6f-831d-ebfa00594b8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:03.257260 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1927/3806] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:03.347386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e5e9d-8fcd-4d6f-831d-ebfa00594b8c service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:03.352771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374463b-3d47-42c8-a3fd-7515611ad5db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:03.354012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374463b-3d47-42c8-a3fd-7515611ad5db service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:03.354806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374463b-3d47-42c8-a3fd-7515611ad5db service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:03.355935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374463b-3d47-42c8-a3fd-7515611ad5db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:03.377528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374463b-3d47-42c8-a3fd-7515611ad5db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:03.380084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374463b-3d47-42c8-a3fd-7515611ad5db service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:03.409052 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1880/3807] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:05.313606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29409c7c-a5ef-4583-8b10-f532bc4c3666 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:05.314178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-29409c7c-a5ef-4583-8b10-f532bc4c3666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:05.362011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29409c7c-a5ef-4583-8b10-f532bc4c3666 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:05.384812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8a79a-2072-42f7-9266-9a4f38153647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:05.385580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8a79a-2072-42f7-9266-9a4f38153647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:05.385876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8a79a-2072-42f7-9266-9a4f38153647 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:05.386668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8a79a-2072-42f7-9266-9a4f38153647 admin admin] PATH_INFO: `/v3/projects/0d8fde784a4a45308d5ae744f89b52c3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:05.403509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8a79a-2072-42f7-9266-9a4f38153647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8fde784a4a45308d5ae744f89b52c3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:05.408097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8a79a-2072-42f7-9266-9a4f38153647 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:05.554747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fdf8a79a-2072-42f7-9266-9a4f38153647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8fde784a4a45308d5ae744f89b52c3', 'request_id': 'req-fdf8a79a-2072-42f7-9266-9a4f38153647'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:05.602719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fdf8a79a-2072-42f7-9266-9a4f38153647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8fde784a4a45308d5ae744f89b52c3', 'request_id': 'req-fdf8a79a-2072-42f7-9266-9a4f38153647'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:05.616767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fdf8a79a-2072-42f7-9266-9a4f38153647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8fde784a4a45308d5ae744f89b52c3', 'request_id': 'req-fdf8a79a-2072-42f7-9266-9a4f38153647'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:05.644034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:05.658413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fdf8a79a-2072-42f7-9266-9a4f38153647 admin admin] The token cache is being invalidate because project 0d8fde784a4a45308d5ae744f89b52c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:05.658648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fdf8a79a-2072-42f7-9266-9a4f38153647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:05.665412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fdf8a79a-2072-42f7-9266-9a4f38153647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:05.667961 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1928/3808] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:05 2022] DELETE /identity/v3/projects/0d8fde784a4a45308d5ae744f89b52c3 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:05.676624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7beba1-f623-4b58-b7b3-44c05c179202 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:05.677217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7beba1-f623-4b58-b7b3-44c05c179202 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:05.677217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7beba1-f623-4b58-b7b3-44c05c179202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:05.727479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2b7beba1-f623-4b58-b7b3-44c05c179202 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:05.729365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2b7beba1-f623-4b58-b7b3-44c05c179202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:05.885482 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1881/3809] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:05.887758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66ddc8-5aed-43e3-b428-dfeb0032fc19 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:05.888058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66ddc8-5aed-43e3-b428-dfeb0032fc19 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:05.888342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66ddc8-5aed-43e3-b428-dfeb0032fc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:05.913178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0d66ddc8-5aed-43e3-b428-dfeb0032fc19 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:05.914689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0d66ddc8-5aed-43e3-b428-dfeb0032fc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:05.935007 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1929/3810] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:05.941000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7beba1-f623-4b58-b7b3-44c05c179202 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:05.965272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7beba1-f623-4b58-b7b3-44c05c179202 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:05.973550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc56c4-9525-4aa1-886b-402f435677e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:05.974736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc56c4-9525-4aa1-886b-402f435677e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:05.975025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc56c4-9525-4aa1-886b-402f435677e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:05.975305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc56c4-9525-4aa1-886b-402f435677e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:05.975964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc56c4-9525-4aa1-886b-402f435677e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:05.976276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc56c4-9525-4aa1-886b-402f435677e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:05.977999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc56c4-9525-4aa1-886b-402f435677e1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:06.020696 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1882/3811] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:06.023710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66ddc8-5aed-43e3-b428-dfeb0032fc19 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:06.051521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66ddc8-5aed-43e3-b428-dfeb0032fc19 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:06.078079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595b311-e633-44c0-b245-4a3c8a4baf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:06.078716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595b311-e633-44c0-b245-4a3c8a4baf01 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:06.079268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595b311-e633-44c0-b245-4a3c8a4baf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:06.079687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595b311-e633-44c0-b245-4a3c8a4baf01 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:06.080206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595b311-e633-44c0-b245-4a3c8a4baf01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:06.085424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595b311-e633-44c0-b245-4a3c8a4baf01 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:06.189915 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1930/3812] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:06 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:06.194155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc56c4-9525-4aa1-886b-402f435677e1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:06.210702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc56c4-9525-4aa1-886b-402f435677e1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:06.227050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:06.337754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4b482-80ae-4023-aa97-b3c35e46d480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:06.338538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4b482-80ae-4023-aa97-b3c35e46d480 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:06.338831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4b482-80ae-4023-aa97-b3c35e46d480 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:06.339113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4b482-80ae-4023-aa97-b3c35e46d480 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:06.340517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4b482-80ae-4023-aa97-b3c35e46d480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:06.342908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4b482-80ae-4023-aa97-b3c35e46d480 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:06.382877 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1883/3813] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:06 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:06.387359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595b311-e633-44c0-b245-4a3c8a4baf01 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:06.404518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595b311-e633-44c0-b245-4a3c8a4baf01 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:06.417761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35573a-3b4a-4d19-80cd-6a5ed569cfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:06.418467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35573a-3b4a-4d19-80cd-6a5ed569cfbb admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:06.418854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35573a-3b4a-4d19-80cd-6a5ed569cfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:06.419977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35573a-3b4a-4d19-80cd-6a5ed569cfbb admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:06.420654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35573a-3b4a-4d19-80cd-6a5ed569cfbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:06.422332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35573a-3b4a-4d19-80cd-6a5ed569cfbb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:06.477608 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1931/3814] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:06.481604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4b482-80ae-4023-aa97-b3c35e46d480 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:06.496447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4b482-80ae-4023-aa97-b3c35e46d480 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:06.509485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e41968-d2eb-4122-8b66-27fe864e5060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:06.509485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e41968-d2eb-4122-8b66-27fe864e5060 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:06.509485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e41968-d2eb-4122-8b66-27fe864e5060 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:06.509485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e41968-d2eb-4122-8b66-27fe864e5060 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:06.511054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e41968-d2eb-4122-8b66-27fe864e5060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:06.514560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e41968-d2eb-4122-8b66-27fe864e5060 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:06.535773 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1884/3815] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:06.539744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35573a-3b4a-4d19-80cd-6a5ed569cfbb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:06.552609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35573a-3b4a-4d19-80cd-6a5ed569cfbb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:06.559127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25313c9-b32f-403d-a052-ca498a720dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:06.559740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25313c9-b32f-403d-a052-ca498a720dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:06.560029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25313c9-b32f-403d-a052-ca498a720dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:06.560314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25313c9-b32f-403d-a052-ca498a720dc8 admin admin] PATH_INFO: `/v3/projects/c59b2c80975d499baf2115b959a9fa4f/users/18faf58f4b7e49aca012a16043d85079/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:06.578703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25313c9-b32f-403d-a052-ca498a720dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c59b2c80975d499baf2115b959a9fa4f, user_id=18faf58f4b7e49aca012a16043d85079, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:06.582605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25313c9-b32f-403d-a052-ca498a720dc8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:06.603066 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1932/3816] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:06 2022] PUT /identity/v3/projects/c59b2c80975d499baf2115b959a9fa4f/users/18faf58f4b7e49aca012a16043d85079/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:06.613797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38c45b-6f07-4926-96b4-5c008ce2f1ba None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:06.614207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38c45b-6f07-4926-96b4-5c008ce2f1ba None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:06.614591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38c45b-6f07-4926-96b4-5c008ce2f1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:06.683989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5b38c45b-6f07-4926-96b4-5c008ce2f1ba None None] MFA Rules not processed for user `18faf58f4b7e49aca012a16043d85079`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:06.686128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5b38c45b-6f07-4926-96b4-5c008ce2f1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:06.976056 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1885/3817] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:06.980828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25313c9-b32f-403d-a052-ca498a720dc8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:07.111412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25313c9-b32f-403d-a052-ca498a720dc8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:07.116680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb781d-02d3-4c7d-ba99-e1a00f68cc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:07.117719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb781d-02d3-4c7d-ba99-e1a00f68cc4d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:07.118068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb781d-02d3-4c7d-ba99-e1a00f68cc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:07.118414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb781d-02d3-4c7d-ba99-e1a00f68cc4d admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:07.119154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb781d-02d3-4c7d-ba99-e1a00f68cc4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:07.123810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb781d-02d3-4c7d-ba99-e1a00f68cc4d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:07.186500 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1933/3818] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:06 2022] POST /identity/v3/projects => generated 367 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:07.188582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38c45b-6f07-4926-96b4-5c008ce2f1ba None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:07.212047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38c45b-6f07-4926-96b4-5c008ce2f1ba None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:07.326680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12f120-95aa-40a9-9e10-0a8407bf514f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:07.327320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12f120-95aa-40a9-9e10-0a8407bf514f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:07.327602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12f120-95aa-40a9-9e10-0a8407bf514f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:07.327876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12f120-95aa-40a9-9e10-0a8407bf514f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:07.329307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12f120-95aa-40a9-9e10-0a8407bf514f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:07.331887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12f120-95aa-40a9-9e10-0a8407bf514f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:07.369301 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1886/3819] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:07 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:07.373138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb781d-02d3-4c7d-ba99-e1a00f68cc4d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:07.388342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb781d-02d3-4c7d-ba99-e1a00f68cc4d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:07.404019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a73533-522a-4203-8daa-d6494861751f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:07.405150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a73533-522a-4203-8daa-d6494861751f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:07.405660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a73533-522a-4203-8daa-d6494861751f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:07.406144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a73533-522a-4203-8daa-d6494861751f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:07.406831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a73533-522a-4203-8daa-d6494861751f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:07.408472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a73533-522a-4203-8daa-d6494861751f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:07.428782 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1934/3820] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:07.433012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12f120-95aa-40a9-9e10-0a8407bf514f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:07.453128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12f120-95aa-40a9-9e10-0a8407bf514f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:07.467110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fabfa-9bb7-41fc-94d6-bbee89857d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:07.467804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fabfa-9bb7-41fc-94d6-bbee89857d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:07.468090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fabfa-9bb7-41fc-94d6-bbee89857d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:07.468388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fabfa-9bb7-41fc-94d6-bbee89857d32 admin admin] PATH_INFO: `/v3/projects/5a73c67cd3884404b77e671881ad136b/users/b32e17cc8320480ca87c01ab3fdffa2c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:07.474647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:07.506697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fabfa-9bb7-41fc-94d6-bbee89857d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a73c67cd3884404b77e671881ad136b, user_id=b32e17cc8320480ca87c01ab3fdffa2c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:07.509608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fabfa-9bb7-41fc-94d6-bbee89857d32 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:07.534699 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1887/3821] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:07 2022] PUT /identity/v3/projects/5a73c67cd3884404b77e671881ad136b/users/b32e17cc8320480ca87c01ab3fdffa2c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:07.538857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a73533-522a-4203-8daa-d6494861751f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:07.552537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a73533-522a-4203-8daa-d6494861751f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:07.676253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f6764-7c87-4b1e-ad00-52bb7b567a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:07.679710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f6764-7c87-4b1e-ad00-52bb7b567a34 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:07.680078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f6764-7c87-4b1e-ad00-52bb7b567a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:07.681256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f6764-7c87-4b1e-ad00-52bb7b567a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:07.682066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f6764-7c87-4b1e-ad00-52bb7b567a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:07.689640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f6764-7c87-4b1e-ad00-52bb7b567a34 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:07.764721 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1935/3822] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:07.767720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fabfa-9bb7-41fc-94d6-bbee89857d32 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:07.786884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fabfa-9bb7-41fc-94d6-bbee89857d32 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:07.796153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956971e3-332e-4db0-8c97-d5969ad4d2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:07.797588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956971e3-332e-4db0-8c97-d5969ad4d2d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:07.798051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956971e3-332e-4db0-8c97-d5969ad4d2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:07.798533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956971e3-332e-4db0-8c97-d5969ad4d2d9 admin admin] PATH_INFO: `/v3/projects/5a73c67cd3884404b77e671881ad136b/users/b32e17cc8320480ca87c01ab3fdffa2c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:07.805427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956971e3-332e-4db0-8c97-d5969ad4d2d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a73c67cd3884404b77e671881ad136b, user_id=b32e17cc8320480ca87c01ab3fdffa2c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:07.813549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956971e3-332e-4db0-8c97-d5969ad4d2d9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:07.829952 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1888/3823] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:07 2022] PUT /identity/v3/projects/5a73c67cd3884404b77e671881ad136b/users/b32e17cc8320480ca87c01ab3fdffa2c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:07.841851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff0502-ce12-418b-824a-e3644789b7f0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:07.842127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff0502-ce12-418b-824a-e3644789b7f0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:07.842419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff0502-ce12-418b-824a-e3644789b7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:07.861853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:07.916768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-75ff0502-ce12-418b-824a-e3644789b7f0 None None] MFA Rules not processed for user `b32e17cc8320480ca87c01ab3fdffa2c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:07.918176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-75ff0502-ce12-418b-824a-e3644789b7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:08.051868 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1936/3824] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:08.056225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956971e3-332e-4db0-8c97-d5969ad4d2d9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:08.080225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956971e3-332e-4db0-8c97-d5969ad4d2d9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:08.097679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb6239-c01b-4d06-9913-4be5c5519c79 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:08.098878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb6239-c01b-4d06-9913-4be5c5519c79 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:08.099160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb6239-c01b-4d06-9913-4be5c5519c79 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:08.099443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb6239-c01b-4d06-9913-4be5c5519c79 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:08.100756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb6239-c01b-4d06-9913-4be5c5519c79 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:08.101825 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-86bb6239-c01b-4d06-9913-4be5c5519c79 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:08.101825 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:08.101825 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:08.105158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb6239-c01b-4d06-9913-4be5c5519c79 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:08.160226 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1889/3825] 10.222.0.16 () {60 vars in 1150 bytes} [Sat Sep 3 21:02:08 2022] POST /identity/v3/domains => generated 276 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:08.164103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff0502-ce12-418b-824a-e3644789b7f0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:08.178484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff0502-ce12-418b-824a-e3644789b7f0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:08.316368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47668a8f-1f62-4a02-b8bc-a0eef1f311d0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:08.317131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47668a8f-1f62-4a02-b8bc-a0eef1f311d0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:08.317449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47668a8f-1f62-4a02-b8bc-a0eef1f311d0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:08.317727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47668a8f-1f62-4a02-b8bc-a0eef1f311d0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:08.318437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47668a8f-1f62-4a02-b8bc-a0eef1f311d0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:08.319478 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-47668a8f-1f62-4a02-b8bc-a0eef1f311d0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:08.319478 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:08.319478 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:08.320373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47668a8f-1f62-4a02-b8bc-a0eef1f311d0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:08.357290 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1937/3826] 10.222.0.16 () {60 vars in 1150 bytes} [Sat Sep 3 21:02:08 2022] POST /identity/v3/domains => generated 277 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:08.361030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb6239-c01b-4d06-9913-4be5c5519c79 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:08.377083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb6239-c01b-4d06-9913-4be5c5519c79 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:08.460363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c587a-8ae0-48c0-8c99-b4146032bc7b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:08.461369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c587a-8ae0-48c0-8c99-b4146032bc7b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:08.461859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c587a-8ae0-48c0-8c99-b4146032bc7b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:08.462686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c587a-8ae0-48c0-8c99-b4146032bc7b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:08.463535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c587a-8ae0-48c0-8c99-b4146032bc7b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:08.466025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c587a-8ae0-48c0-8c99-b4146032bc7b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:08.499963 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1890/3827] 10.222.0.16 () {60 vars in 1150 bytes} [Sat Sep 3 21:02:08 2022] POST /identity/v3/domains => generated 277 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:08.504703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47668a8f-1f62-4a02-b8bc-a0eef1f311d0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:08.522282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47668a8f-1f62-4a02-b8bc-a0eef1f311d0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:08.636723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32501f-700e-4050-880c-15cd4041970f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:08.637609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32501f-700e-4050-880c-15cd4041970f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:08.638005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32501f-700e-4050-880c-15cd4041970f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:08.638928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32501f-700e-4050-880c-15cd4041970f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:08.639755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32501f-700e-4050-880c-15cd4041970f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:08.641790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32501f-700e-4050-880c-15cd4041970f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:08.667085 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1938/3828] 10.222.0.16 () {60 vars in 1150 bytes} [Sat Sep 3 21:02:08 2022] POST /identity/v3/domains => generated 278 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:08.672347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c587a-8ae0-48c0-8c99-b4146032bc7b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:08.693854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c587a-8ae0-48c0-8c99-b4146032bc7b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:08.832375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:08.833908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:08.834118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:08.834454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/49c57303e53e4c658dd515a46d951db6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:08.848734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=49c57303e53e4c658dd515a46d951db6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:08.850191 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:08.850191 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:08.850191 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:08.851463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:08.902727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '49c57303e53e4c658dd515a46d951db6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:08.936348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '49c57303e53e4c658dd515a46d951db6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:08.985451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:09.031279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49c57303e53e4c658dd515a46d951db6', 'request_id': 'req-7fa90d23-c92f-48d4-a408-67829839bb73'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.055615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49c57303e53e4c658dd515a46d951db6', 'request_id': 'req-7fa90d23-c92f-48d4-a408-67829839bb73'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.080248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49c57303e53e4c658dd515a46d951db6', 'request_id': 'req-7fa90d23-c92f-48d4-a408-67829839bb73'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.099109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] The token cache is being invalidate because project 49c57303e53e4c658dd515a46d951db6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:09.099484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.100308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.120959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '49c57303e53e4c658dd515a46d951db6', 'request_id': 'req-7fa90d23-c92f-48d4-a408-67829839bb73'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.125731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '49c57303e53e4c658dd515a46d951db6', 'request_id': 'req-7fa90d23-c92f-48d4-a408-67829839bb73'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.127936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '49c57303e53e4c658dd515a46d951db6', 'request_id': 'req-7fa90d23-c92f-48d4-a408-67829839bb73'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.135012 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1891/3829] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:02:08 2022] DELETE /identity/v3/domains/49c57303e53e4c658dd515a46d951db6 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:09.140446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32501f-700e-4050-880c-15cd4041970f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:09.142327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3b32501f-700e-4050-880c-15cd4041970f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:09.156885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32501f-700e-4050-880c-15cd4041970f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:09.178707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:09.291277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d4862-4d65-439d-8f18-e26d54cc82d5 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:09.291910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d4862-4d65-439d-8f18-e26d54cc82d5 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:09.292255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d4862-4d65-439d-8f18-e26d54cc82d5 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:09.292585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d4862-4d65-439d-8f18-e26d54cc82d5 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:09.293434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d4862-4d65-439d-8f18-e26d54cc82d5 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:09.295716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d4862-4d65-439d-8f18-e26d54cc82d5 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:09.322797 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1939/3830] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:02:09 2022] POST /identity/v3/domains => generated 249 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:09.327742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:09.344186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa90d23-c92f-48d4-a408-67829839bb73 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:09.505675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe9123-e3d5-4b9f-874c-8b694a44c502 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:09.506717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe9123-e3d5-4b9f-874c-8b694a44c502 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:09.507008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe9123-e3d5-4b9f-874c-8b694a44c502 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:09.507397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe9123-e3d5-4b9f-874c-8b694a44c502 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/951bd1fb1b6d4a9a80d72216a7915659` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:09.543978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe9123-e3d5-4b9f-874c-8b694a44c502 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=951bd1fb1b6d4a9a80d72216a7915659)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:09.548030 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-5efe9123-e3d5-4b9f-874c-8b694a44c502 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:09.548030 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:09.548030 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:09.552216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe9123-e3d5-4b9f-874c-8b694a44c502 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:09.608029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5efe9123-e3d5-4b9f-874c-8b694a44c502 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '951bd1fb1b6d4a9a80d72216a7915659'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.609548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5efe9123-e3d5-4b9f-874c-8b694a44c502 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '951bd1fb1b6d4a9a80d72216a7915659'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.646744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5efe9123-e3d5-4b9f-874c-8b694a44c502 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '951bd1fb1b6d4a9a80d72216a7915659'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.648195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5efe9123-e3d5-4b9f-874c-8b694a44c502 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '951bd1fb1b6d4a9a80d72216a7915659'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.654472 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1892/3831] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:02:09 2022] PATCH /identity/v3/domains/951bd1fb1b6d4a9a80d72216a7915659 => generated 250 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:09.656061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d4862-4d65-439d-8f18-e26d54cc82d5 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:09.658624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d20d4862-4d65-439d-8f18-e26d54cc82d5 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:09.675127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d4862-4d65-439d-8f18-e26d54cc82d5 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:09.820401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:09.821375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:09.821916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:09.822356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/951bd1fb1b6d4a9a80d72216a7915659` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:09.838304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=951bd1fb1b6d4a9a80d72216a7915659)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:09.841475 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:09.841475 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:09.841475 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:09.844053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:09.847586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb384ba-d525-460d-b538-8fddc9b3a294 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:09.848073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb384ba-d525-460d-b538-8fddc9b3a294 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:09.848191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb384ba-d525-460d-b538-8fddc9b3a294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:09.879352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0cb384ba-d525-460d-b538-8fddc9b3a294 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:09.880607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0cb384ba-d525-460d-b538-8fddc9b3a294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:09.886630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '951bd1fb1b6d4a9a80d72216a7915659'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.905582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '951bd1fb1b6d4a9a80d72216a7915659'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:09.935326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:09.962296 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1893/3832] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:02:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:02:09.977522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb384ba-d525-460d-b538-8fddc9b3a294 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:09.980155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0cb384ba-d525-460d-b538-8fddc9b3a294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:10.024483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '951bd1fb1b6d4a9a80d72216a7915659', 'request_id': 'req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:10.041317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '951bd1fb1b6d4a9a80d72216a7915659', 'request_id': 'req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:10.051673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '951bd1fb1b6d4a9a80d72216a7915659', 'request_id': 'req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:10.078264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] The token cache is being invalidate because project 951bd1fb1b6d4a9a80d72216a7915659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:10.078785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:10.081708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:10.098122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb384ba-d525-460d-b538-8fddc9b3a294 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:10.127926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:10.137575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '951bd1fb1b6d4a9a80d72216a7915659', 'request_id': 'req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:10.155900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '951bd1fb1b6d4a9a80d72216a7915659', 'request_id': 'req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:10.158547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '951bd1fb1b6d4a9a80d72216a7915659', 'request_id': 'req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:10.180656 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1940/3833] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:02:09 2022] DELETE /identity/v3/domains/951bd1fb1b6d4a9a80d72216a7915659 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:10.182694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6d820-e3af-4822-b345-6ae44bfee4e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:10.183749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6d820-e3af-4822-b345-6ae44bfee4e7 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:10.184043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6d820-e3af-4822-b345-6ae44bfee4e7 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:10.184321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6d820-e3af-4822-b345-6ae44bfee4e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:10.188088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-81e6d820-e3af-4822-b345-6ae44bfee4e7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:10.195065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:10.200256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:10.302521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6d820-e3af-4822-b345-6ae44bfee4e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:10.304382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6d820-e3af-4822-b345-6ae44bfee4e7 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:10.315375 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1894/3834] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:02:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 21:02:10.335953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f479c6-3bc3-4eb2-aab8-dbd60387fd9f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:10.341109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62c056-9858-4459-b73e-31467c2ebdd8 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:10.341869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62c056-9858-4459-b73e-31467c2ebdd8 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:10.342321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62c056-9858-4459-b73e-31467c2ebdd8 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:10.342719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62c056-9858-4459-b73e-31467c2ebdd8 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:10.343982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62c056-9858-4459-b73e-31467c2ebdd8 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:10.345869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62c056-9858-4459-b73e-31467c2ebdd8 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:10.380834 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1941/3835] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:02:10 2022] POST /identity/v3/domains => generated 277 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:10.382235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6d820-e3af-4822-b345-6ae44bfee4e7 service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:10.400189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6d820-e3af-4822-b345-6ae44bfee4e7 service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:10.523531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:10.629958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:10.634070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:10.634372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:10.634664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/580c3ac7662e43288df5d0a6e13d1d41` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:10.671506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba978cf-1e61-4889-8965-49ba9e8bd8b5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:10.672145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba978cf-1e61-4889-8965-49ba9e8bd8b5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:10.672145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba978cf-1e61-4889-8965-49ba9e8bd8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:10.674063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=580c3ac7662e43288df5d0a6e13d1d41)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:10.675951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:10.723697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '580c3ac7662e43288df5d0a6e13d1d41'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:10.725654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '580c3ac7662e43288df5d0a6e13d1d41'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:10.777837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8ba978cf-1e61-4889-8965-49ba9e8bd8b5 None None] MFA Rules not processed for user `1b2dd96d938b41d4ba40007aa072288b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:10.778576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '580c3ac7662e43288df5d0a6e13d1d41'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:10.780043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8ba978cf-1e61-4889-8965-49ba9e8bd8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:10.781401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '580c3ac7662e43288df5d0a6e13d1d41'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:10.790490 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1895/3836] 10.222.0.16 () {60 vars in 1250 bytes} [Sat Sep 3 21:02:10 2022] PATCH /identity/v3/domains/580c3ac7662e43288df5d0a6e13d1d41 => generated 278 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:10.793860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:10.799625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:10.961842 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1942/3837] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:02:10.994090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2499a04-fad1-4774-845e-965563094152 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:11.009468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201710d-1831-4793-a714-497f254bc397 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:11.010723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201710d-1831-4793-a714-497f254bc397 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:11.011265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201710d-1831-4793-a714-497f254bc397 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:11.011808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201710d-1831-4793-a714-497f254bc397 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/580c3ac7662e43288df5d0a6e13d1d41` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:11.030109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201710d-1831-4793-a714-497f254bc397 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=580c3ac7662e43288df5d0a6e13d1d41)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:11.033321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201710d-1831-4793-a714-497f254bc397 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:11.039365 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1896/3838] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:02:10 2022] GET /identity/v3/domains/580c3ac7662e43288df5d0a6e13d1d41 => generated 278 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:11.040429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba978cf-1e61-4889-8965-49ba9e8bd8b5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:11.062083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba978cf-1e61-4889-8965-49ba9e8bd8b5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:11.078893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:11.079697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:11.080010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:11.080287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/580c3ac7662e43288df5d0a6e13d1d41` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:11.089912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=580c3ac7662e43288df5d0a6e13d1d41)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:11.091751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:11.168009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '580c3ac7662e43288df5d0a6e13d1d41'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:11.180210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '580c3ac7662e43288df5d0a6e13d1d41'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:11.234005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201710d-1831-4793-a714-497f254bc397 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:11.236068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c201710d-1831-4793-a714-497f254bc397 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:11.312080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:11.315908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '580c3ac7662e43288df5d0a6e13d1d41', 'request_id': 'req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:11.343157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '580c3ac7662e43288df5d0a6e13d1d41', 'request_id': 'req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:11.359473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '580c3ac7662e43288df5d0a6e13d1d41', 'request_id': 'req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:11.378077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] The token cache is being invalidate because project 580c3ac7662e43288df5d0a6e13d1d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:11.380080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:11.382044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:11.425420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '580c3ac7662e43288df5d0a6e13d1d41', 'request_id': 'req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:11.450673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '580c3ac7662e43288df5d0a6e13d1d41', 'request_id': 'req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:11.453820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '580c3ac7662e43288df5d0a6e13d1d41', 'request_id': 'req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:11.478944 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1943/3839] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:02:11 2022] DELETE /identity/v3/domains/580c3ac7662e43288df5d0a6e13d1d41 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:11.483265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:11.493480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:11.563828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201710d-1831-4793-a714-497f254bc397 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:11.672037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe1cdc-c5a2-4599-beec-d5c43ae025b2 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:11.675909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:11.689647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc7c79-c5c5-47d7-b749-e50e00f0c6fe tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:11.690679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc7c79-c5c5-47d7-b749-e50e00f0c6fe tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:11.691441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc7c79-c5c5-47d7-b749-e50e00f0c6fe tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:11.691944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc7c79-c5c5-47d7-b749-e50e00f0c6fe tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:11.692950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc7c79-c5c5-47d7-b749-e50e00f0c6fe tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:11.694876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc7c79-c5c5-47d7-b749-e50e00f0c6fe tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:11.761496 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1944/3840] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Sep 3 21:02:11 2022] GET /identity/v3/domains => generated 1348 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:11.765788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc7c79-c5c5-47d7-b749-e50e00f0c6fe tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:11.793600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc7c79-c5c5-47d7-b749-e50e00f0c6fe tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:11.808766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b375146-3705-48d6-a67d-eece6c218506 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:11.810460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b375146-3705-48d6-a67d-eece6c218506 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:11.811588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b375146-3705-48d6-a67d-eece6c218506 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:11.811744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b375146-3705-48d6-a67d-eece6c218506 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/580c3ac7662e43288df5d0a6e13d1d41` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:11.823986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471878a-8bca-4f81-adac-3e41a42cf6f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:11.825087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471878a-8bca-4f81-adac-3e41a42cf6f4 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:11.825481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471878a-8bca-4f81-adac-3e41a42cf6f4 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:11.825865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471878a-8bca-4f81-adac-3e41a42cf6f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:11.827783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b375146-3705-48d6-a67d-eece6c218506 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:02:11.830497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1471878a-8bca-4f81-adac-3e41a42cf6f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-8b375146-3705-48d6-a67d-eece6c218506 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Could not find domain: 580c3ac7662e43288df5d0a6e13d1d41.: keystone.exception.DomainNotFound: Could not find domain: 580c3ac7662e43288df5d0a6e13d1d41. Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 580c3ac7662e43288df5d0a6e13d1d41. Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:11.910825 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:02:11.912598 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 580c3ac7662e43288df5d0a6e13d1d41. Sep 03 21:02:11.914164 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:11.916449 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1945/3841] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:02:11 2022] PATCH /identity/v3/domains/580c3ac7662e43288df5d0a6e13d1d41 => generated 112 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:02:11.921376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b375146-3705-48d6-a67d-eece6c218506 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:11.937893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b375146-3705-48d6-a67d-eece6c218506 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:11.953802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a28048-bd2f-4e16-a346-df89da7eaff1 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:11.954928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a28048-bd2f-4e16-a346-df89da7eaff1 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:11.956208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a28048-bd2f-4e16-a346-df89da7eaff1 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:11.958220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a28048-bd2f-4e16-a346-df89da7eaff1 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:11.963406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a28048-bd2f-4e16-a346-df89da7eaff1 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:11.973912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a28048-bd2f-4e16-a346-df89da7eaff1 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:12.023916 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1946/3842] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:02:11 2022] POST /identity/v3/domains => generated 276 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:12.030380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a28048-bd2f-4e16-a346-df89da7eaff1 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:12.035593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471878a-8bca-4f81-adac-3e41a42cf6f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:12.051207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471878a-8bca-4f81-adac-3e41a42cf6f4 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:12.091775 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1897/3843] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 858 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:02:12.174580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a28048-bd2f-4e16-a346-df89da7eaff1 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:12.180271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c08cb-7c9c-4e46-81c5-41f1a9e41c8e tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:12.180925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c08cb-7c9c-4e46-81c5-41f1a9e41c8e tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:12.181301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c08cb-7c9c-4e46-81c5-41f1a9e41c8e tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:12.181589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c08cb-7c9c-4e46-81c5-41f1a9e41c8e tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:12.182246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c08cb-7c9c-4e46-81c5-41f1a9e41c8e tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:12.184352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c08cb-7c9c-4e46-81c5-41f1a9e41c8e tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:12.267825 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1947/3844] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:12 2022] POST /identity/v3/users => generated 346 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:12.270805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471878a-8bca-4f81-adac-3e41a42cf6f4 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:12.288528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471878a-8bca-4f81-adac-3e41a42cf6f4 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:12.312434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136374c-69fc-42da-961c-6158b507e677 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:12.313167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136374c-69fc-42da-961c-6158b507e677 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:12.313517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136374c-69fc-42da-961c-6158b507e677 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:12.313808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136374c-69fc-42da-961c-6158b507e677 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:12.314462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136374c-69fc-42da-961c-6158b507e677 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:12.316337 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-7136374c-69fc-42da-961c-6158b507e677 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:12.316337 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:12.316337 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:12.317603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136374c-69fc-42da-961c-6158b507e677 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:12.362288 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1898/3845] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Sep 3 21:02:12 2022] POST /identity/v3/groups => generated 324 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:12.362810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c08cb-7c9c-4e46-81c5-41f1a9e41c8e tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:12.384539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c08cb-7c9c-4e46-81c5-41f1a9e41c8e tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:12.421607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:12.422797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:12.425808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:12.426682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/06776a2dc2194b0aab4a5f4564564f80` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:12.429342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=06776a2dc2194b0aab4a5f4564564f80)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:12.434999 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:12.434999 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:12.434999 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:12.438401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:12.450380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:12.505541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '06776a2dc2194b0aab4a5f4564564f80'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:12.506939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '06776a2dc2194b0aab4a5f4564564f80'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:12.587925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '06776a2dc2194b0aab4a5f4564564f80'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:12.590921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '06776a2dc2194b0aab4a5f4564564f80'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:12.598546 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1948/3846] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:02:12 2022] PATCH /identity/v3/domains/06776a2dc2194b0aab4a5f4564564f80 => generated 277 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:12.600717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136374c-69fc-42da-961c-6158b507e677 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:12.606213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7136374c-69fc-42da-961c-6158b507e677 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:12.634426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136374c-69fc-42da-961c-6158b507e677 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:12.835684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:12.836623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:12.837170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:12.837615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/06776a2dc2194b0aab4a5f4564564f80` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:12.856120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=06776a2dc2194b0aab4a5f4564564f80)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:12.858143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:12.921138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '06776a2dc2194b0aab4a5f4564564f80'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:12.926321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '06776a2dc2194b0aab4a5f4564564f80'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:12.974559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:13.367650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff53b3dc376b4ad58c02c5c360778e34'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.403461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff53b3dc376b4ad58c02c5c360778e34'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.404327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff53b3dc376b4ad58c02c5c360778e34'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.417123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff53b3dc376b4ad58c02c5c360778e34'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.458087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff53b3dc376b4ad58c02c5c360778e34'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.613629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06776a2dc2194b0aab4a5f4564564f80', 'request_id': 'req-78f3ab9b-0ec8-4eda-b566-05401996476a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.637962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06776a2dc2194b0aab4a5f4564564f80', 'request_id': 'req-78f3ab9b-0ec8-4eda-b566-05401996476a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.650831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06776a2dc2194b0aab4a5f4564564f80', 'request_id': 'req-78f3ab9b-0ec8-4eda-b566-05401996476a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.674719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] The token cache is being invalidate because project 06776a2dc2194b0aab4a5f4564564f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:13.675131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.676081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.688709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '06776a2dc2194b0aab4a5f4564564f80', 'request_id': 'req-78f3ab9b-0ec8-4eda-b566-05401996476a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.695367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '06776a2dc2194b0aab4a5f4564564f80', 'request_id': 'req-78f3ab9b-0ec8-4eda-b566-05401996476a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.696265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '06776a2dc2194b0aab4a5f4564564f80', 'request_id': 'req-78f3ab9b-0ec8-4eda-b566-05401996476a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:13.716933 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1899/3847] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:02:12 2022] DELETE /identity/v3/domains/06776a2dc2194b0aab4a5f4564564f80 => generated 0 bytes in 1104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:13.814142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:13.827380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86294c8b-eb9d-4492-9aad-18081eb818e7 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:14.025010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869cfa6-e19e-4375-ab9f-767ecc3aefdc tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:14.025636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869cfa6-e19e-4375-ab9f-767ecc3aefdc tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:14.025929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869cfa6-e19e-4375-ab9f-767ecc3aefdc tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:14.026267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869cfa6-e19e-4375-ab9f-767ecc3aefdc tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/06776a2dc2194b0aab4a5f4564564f80` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:14.052564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869cfa6-e19e-4375-ab9f-767ecc3aefdc tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=06776a2dc2194b0aab4a5f4564564f80)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:14.054715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869cfa6-e19e-4375-ab9f-767ecc3aefdc tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-8869cfa6-e19e-4375-ab9f-767ecc3aefdc tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Could not find domain: 06776a2dc2194b0aab4a5f4564564f80.: keystone.exception.DomainNotFound: Could not find domain: 06776a2dc2194b0aab4a5f4564564f80. Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 06776a2dc2194b0aab4a5f4564564f80. Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:14.085765 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:02:14.086941 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 06776a2dc2194b0aab4a5f4564564f80. Sep 03 21:02:14.088040 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:14.091375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:14.093156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:14.100920 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1949/3848] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:02:13 2022] GET /identity/v3/domains/06776a2dc2194b0aab4a5f4564564f80 => generated 112 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:02:14.104954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=73,keystone:DELETE=25,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:14.130384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3ab9b-0ec8-4eda-b566-05401996476a tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:14.152780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562be409-fd76-451f-b43c-451bb495284b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:14.152780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562be409-fd76-451f-b43c-451bb495284b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:14.152780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562be409-fd76-451f-b43c-451bb495284b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:14.154740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562be409-fd76-451f-b43c-451bb495284b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/users/ff53b3dc376b4ad58c02c5c360778e34` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:14.184326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562be409-fd76-451f-b43c-451bb495284b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:get_user(user_id=ff53b3dc376b4ad58c02c5c360778e34)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:14.186638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562be409-fd76-451f-b43c-451bb495284b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-562be409-fd76-451f-b43c-451bb495284b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Could not find user: ff53b3dc376b4ad58c02c5c360778e34.: keystone.exception.UserNotFound: Could not find user: ff53b3dc376b4ad58c02c5c360778e34. Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 21:02:14.237911 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:02:14.239074 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: ff53b3dc376b4ad58c02c5c360778e34. Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:14.240598 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1900/3849] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Sep 3 21:02:14 2022] GET /identity/v3/users/ff53b3dc376b4ad58c02c5c360778e34 => generated 110 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:02:14.243544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869cfa6-e19e-4375-ab9f-767ecc3aefdc tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:14.276952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869cfa6-e19e-4375-ab9f-767ecc3aefdc tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:14.295868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da814808-3655-47e5-b421-0dd24256d9ee tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:14.297634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da814808-3655-47e5-b421-0dd24256d9ee tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:14.298108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da814808-3655-47e5-b421-0dd24256d9ee tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:14.298536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da814808-3655-47e5-b421-0dd24256d9ee tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/groups/7f8dce2c0d35452a8e5c3952bb98125c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:14.330612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da814808-3655-47e5-b421-0dd24256d9ee tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:get_group(group_id=7f8dce2c0d35452a8e5c3952bb98125c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:14.332233 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-da814808-3655-47e5-b421-0dd24256d9ee tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:14.332233 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:14.332233 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:14.333885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da814808-3655-47e5-b421-0dd24256d9ee tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-da814808-3655-47e5-b421-0dd24256d9ee tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Could not find group: 7f8dce2c0d35452a8e5c3952bb98125c.: keystone.exception.GroupNotFound: Could not find group: 7f8dce2c0d35452a8e5c3952bb98125c. Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 58, in get Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_group(group_id) Sep 03 21:02:14.363145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 70, in _get_group Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.identity_api.get_group(group_id)) Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:02:14.364181 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: 7f8dce2c0d35452a8e5c3952bb98125c. Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:14.382168 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1950/3850] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:14 2022] GET /identity/v3/groups/7f8dce2c0d35452a8e5c3952bb98125c => generated 111 bytes in 121 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:02:14.382996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562be409-fd76-451f-b43c-451bb495284b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:14.402526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562be409-fd76-451f-b43c-451bb495284b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:14.416693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19798720-c5e8-44e8-a9a7-d2e42317a88d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:14.418292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19798720-c5e8-44e8-a9a7-d2e42317a88d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:14.418601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19798720-c5e8-44e8-a9a7-d2e42317a88d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:14.418866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19798720-c5e8-44e8-a9a7-d2e42317a88d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/groups/7f8dce2c0d35452a8e5c3952bb98125c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:14.435242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19798720-c5e8-44e8-a9a7-d2e42317a88d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-19798720-c5e8-44e8-a9a7-d2e42317a88d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Could not find group: 7f8dce2c0d35452a8e5c3952bb98125c.: keystone.exception.GroupNotFound: Could not find group: 7f8dce2c0d35452a8e5c3952bb98125c. Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 133, in delete Sep 03 21:02:14.452943 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_group') Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 21:02:14.455152 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: 7f8dce2c0d35452a8e5c3952bb98125c. Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:14.457333 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1901/3851] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:02:14 2022] DELETE /identity/v3/groups/7f8dce2c0d35452a8e5c3952bb98125c => generated 111 bytes in 89 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:02:14.460624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da814808-3655-47e5-b421-0dd24256d9ee tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:14.473823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da814808-3655-47e5-b421-0dd24256d9ee tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:14.501644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe88db1-39fc-47c0-aa73-f121cb16e77d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:14.502356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe88db1-39fc-47c0-aa73-f121cb16e77d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:14.503048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe88db1-39fc-47c0-aa73-f121cb16e77d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:14.503311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe88db1-39fc-47c0-aa73-f121cb16e77d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/users/ff53b3dc376b4ad58c02c5c360778e34` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:14.537887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe88db1-39fc-47c0-aa73-f121cb16e77d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:delete_user(user_id=ff53b3dc376b4ad58c02c5c360778e34)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:14.539829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe88db1-39fc-47c0-aa73-f121cb16e77d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-7fe88db1-39fc-47c0-aa73-f121cb16e77d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Could not find user: ff53b3dc376b4ad58c02c5c360778e34.: keystone.exception.UserNotFound: Could not find user: ff53b3dc376b4ad58c02c5c360778e34. Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 275, in delete Sep 03 21:02:14.608988 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application PROVIDERS.identity_api.delete_user(user_id) Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1207, in delete_user Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application user_old = self.get_user(user_id) Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:02:14.610014 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: ff53b3dc376b4ad58c02c5c360778e34. Sep 03 21:02:14.611150 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:14.616425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19798720-c5e8-44e8-a9a7-d2e42317a88d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:14.633786 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1951/3852] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:14 2022] DELETE /identity/v3/users/ff53b3dc376b4ad58c02c5c360778e34 => generated 110 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:02:14.647477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19798720-c5e8-44e8-a9a7-d2e42317a88d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:14.658908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbd82b-511b-4cdb-b890-fe442bcf3ef3 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:14.660599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbd82b-511b-4cdb-b890-fe442bcf3ef3 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:14.661332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbd82b-511b-4cdb-b890-fe442bcf3ef3 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:14.666323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbd82b-511b-4cdb-b890-fe442bcf3ef3 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/06776a2dc2194b0aab4a5f4564564f80` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:14.700944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abbd82b-511b-4cdb-b890-fe442bcf3ef3 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-7abbd82b-511b-4cdb-b890-fe442bcf3ef3 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Could not find domain: 06776a2dc2194b0aab4a5f4564564f80.: keystone.exception.DomainNotFound: Could not find domain: 06776a2dc2194b0aab4a5f4564564f80. Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 06776a2dc2194b0aab4a5f4564564f80. Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:14.710933 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:02:14.712296 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 06776a2dc2194b0aab4a5f4564564f80. Sep 03 21:02:14.713502 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:14.714823 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1902/3853] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:02:14 2022] PATCH /identity/v3/domains/06776a2dc2194b0aab4a5f4564564f80 => generated 112 bytes in 99 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:02:14.720535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe88db1-39fc-47c0-aa73-f121cb16e77d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:14.747602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe88db1-39fc-47c0-aa73-f121cb16e77d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:14.779007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ff244-e284-4005-9b35-c55b7b07134d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:14.779655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ff244-e284-4005-9b35-c55b7b07134d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:14.779944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ff244-e284-4005-9b35-c55b7b07134d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:14.780253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ff244-e284-4005-9b35-c55b7b07134d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:14.780916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ff244-e284-4005-9b35-c55b7b07134d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:14.783567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ff244-e284-4005-9b35-c55b7b07134d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:14.809145 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1952/3854] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Sep 3 21:02:14 2022] GET /identity/v3/domains => generated 1348 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:14.810197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbd82b-511b-4cdb-b890-fe442bcf3ef3 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:14.820947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbd82b-511b-4cdb-b890-fe442bcf3ef3 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:14.834364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635a08f-b047-4e76-9fed-ec82b9a9987f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:14.835352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635a08f-b047-4e76-9fed-ec82b9a9987f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:14.835847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635a08f-b047-4e76-9fed-ec82b9a9987f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:14.836381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635a08f-b047-4e76-9fed-ec82b9a9987f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:14.837320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635a08f-b047-4e76-9fed-ec82b9a9987f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Adding query filter params (enabled=True) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:14.837859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635a08f-b047-4e76-9fed-ec82b9a9987f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:14.840717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635a08f-b047-4e76-9fed-ec82b9a9987f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:14.872343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ff244-e284-4005-9b35-c55b7b07134d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:14.883615 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1903/3855] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:02:14 2022] GET /identity/v3/domains?enabled=True => generated 1095 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:02:14.885833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ff244-e284-4005-9b35-c55b7b07134d tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:14.891360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90fd74-96fc-4041-9dd0-abf120303ee0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:14.892348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90fd74-96fc-4041-9dd0-abf120303ee0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:14.892853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90fd74-96fc-4041-9dd0-abf120303ee0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:14.893676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90fd74-96fc-4041-9dd0-abf120303ee0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:14.895236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90fd74-96fc-4041-9dd0-abf120303ee0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Adding query filter params (name=tempest-test_domain-582834975) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:14.895834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90fd74-96fc-4041-9dd0-abf120303ee0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:14.899072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90fd74-96fc-4041-9dd0-abf120303ee0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:14.922223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635a08f-b047-4e76-9fed-ec82b9a9987f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:14.933543 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1953/3856] 10.222.0.16 () {58 vars in 1197 bytes} [Sat Sep 3 21:02:14 2022] GET /identity/v3/domains?name=tempest-test_domain-582834975 => generated 407 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:14.934622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635a08f-b047-4e76-9fed-ec82b9a9987f tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:14.940609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf6572-9822-4ec1-97a7-1789a6eaa190 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:14.941494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf6572-9822-4ec1-97a7-1789a6eaa190 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:14.942106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf6572-9822-4ec1-97a7-1789a6eaa190 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:14.942517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf6572-9822-4ec1-97a7-1789a6eaa190 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/06776a2dc2194b0aab4a5f4564564f80` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:14.973388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf6572-9822-4ec1-97a7-1789a6eaa190 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-7ddf6572-9822-4ec1-97a7-1789a6eaa190 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Could not find domain: 06776a2dc2194b0aab4a5f4564564f80.: keystone.exception.DomainNotFound: Could not find domain: 06776a2dc2194b0aab4a5f4564564f80. Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 06776a2dc2194b0aab4a5f4564564f80. Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:14.979439 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:02:14.982642 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 06776a2dc2194b0aab4a5f4564564f80. Sep 03 21:02:14.985182 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:14.986242 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1904/3857] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:02:14 2022] PATCH /identity/v3/domains/06776a2dc2194b0aab4a5f4564564f80 => generated 112 bytes in 59 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 03 21:02:14.986320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90fd74-96fc-4041-9dd0-abf120303ee0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:14.997247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90fd74-96fc-4041-9dd0-abf120303ee0 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:15.026500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967941e-f891-42f3-b50f-df07780ebe3b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:15.027710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967941e-f891-42f3-b50f-df07780ebe3b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:15.028000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967941e-f891-42f3-b50f-df07780ebe3b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:15.028308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967941e-f891-42f3-b50f-df07780ebe3b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/1328d760a33a4587b266a357043a5b9d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:15.080919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967941e-f891-42f3-b50f-df07780ebe3b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=1328d760a33a4587b266a357043a5b9d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:15.082660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967941e-f891-42f3-b50f-df07780ebe3b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:15.256963 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1954/3858] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:02:14 2022] PATCH /identity/v3/domains/1328d760a33a4587b266a357043a5b9d => generated 277 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:15.257614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf6572-9822-4ec1-97a7-1789a6eaa190 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:15.274947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf6572-9822-4ec1-97a7-1789a6eaa190 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:15.289373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:15.290805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:15.291650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:15.292195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/1328d760a33a4587b266a357043a5b9d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:15.333522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=1328d760a33a4587b266a357043a5b9d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:15.337191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:15.480915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '1328d760a33a4587b266a357043a5b9d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:15.490310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '1328d760a33a4587b266a357043a5b9d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:15.636976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1328d760a33a4587b266a357043a5b9d', 'request_id': 'req-55971b8e-45f7-4271-989b-b70c1b61aa07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:15.651913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1328d760a33a4587b266a357043a5b9d', 'request_id': 'req-55971b8e-45f7-4271-989b-b70c1b61aa07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:15.668050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1328d760a33a4587b266a357043a5b9d', 'request_id': 'req-55971b8e-45f7-4271-989b-b70c1b61aa07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:15.695521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] The token cache is being invalidate because project 1328d760a33a4587b266a357043a5b9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:15.696100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:15.699714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:15.710676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '1328d760a33a4587b266a357043a5b9d', 'request_id': 'req-55971b8e-45f7-4271-989b-b70c1b61aa07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:15.715499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '1328d760a33a4587b266a357043a5b9d', 'request_id': 'req-55971b8e-45f7-4271-989b-b70c1b61aa07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:15.716303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '1328d760a33a4587b266a357043a5b9d', 'request_id': 'req-55971b8e-45f7-4271-989b-b70c1b61aa07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:15.723623 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1905/3859] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:02:15 2022] DELETE /identity/v3/domains/1328d760a33a4587b266a357043a5b9d => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:15.727458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967941e-f891-42f3-b50f-df07780ebe3b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:15.731528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d967941e-f891-42f3-b50f-df07780ebe3b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:15.764731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967941e-f891-42f3-b50f-df07780ebe3b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:15.815051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:15.949242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:15.951818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:15.952363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:15.953540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/84a4736b194b4bb9810ca14efcc3b2a3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:15.972612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=84a4736b194b4bb9810ca14efcc3b2a3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:15.974898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:16.015430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '84a4736b194b4bb9810ca14efcc3b2a3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.017527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '84a4736b194b4bb9810ca14efcc3b2a3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.085671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '84a4736b194b4bb9810ca14efcc3b2a3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.090148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '84a4736b194b4bb9810ca14efcc3b2a3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.102371 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1955/3860] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:02:15 2022] PATCH /identity/v3/domains/84a4736b194b4bb9810ca14efcc3b2a3 => generated 278 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:16.103711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:16.141329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55971b8e-45f7-4271-989b-b70c1b61aa07 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:16.231348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:16.292876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:16.293558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:16.293838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:16.294120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/84a4736b194b4bb9810ca14efcc3b2a3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:16.321463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=84a4736b194b4bb9810ca14efcc3b2a3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:16.323197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:16.439861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '84a4736b194b4bb9810ca14efcc3b2a3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.455064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '84a4736b194b4bb9810ca14efcc3b2a3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.625265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a4736b194b4bb9810ca14efcc3b2a3', 'request_id': 'req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.646061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a4736b194b4bb9810ca14efcc3b2a3', 'request_id': 'req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.665715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a4736b194b4bb9810ca14efcc3b2a3', 'request_id': 'req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.708773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] The token cache is being invalidate because project 84a4736b194b4bb9810ca14efcc3b2a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:16.709233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.715483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.746581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '84a4736b194b4bb9810ca14efcc3b2a3', 'request_id': 'req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.762801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '84a4736b194b4bb9810ca14efcc3b2a3', 'request_id': 'req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.764087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '84a4736b194b4bb9810ca14efcc3b2a3', 'request_id': 'req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:16.774885 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1906/3861] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:02:16 2022] DELETE /identity/v3/domains/84a4736b194b4bb9810ca14efcc3b2a3 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:16.779588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:16.781988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:16.821013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624446f-afe7-4154-a332-af4c2e30964b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:17.135740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8b779-0363-4fcf-9f80-6a99a0e0331b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:17.138300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8b779-0363-4fcf-9f80-6a99a0e0331b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:17.139402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8b779-0363-4fcf-9f80-6a99a0e0331b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:17.139692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8b779-0363-4fcf-9f80-6a99a0e0331b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/026cdb95ae1a42ee9f3176eb89c7b908` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:17.200601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c8b779-0363-4fcf-9f80-6a99a0e0331b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=026cdb95ae1a42ee9f3176eb89c7b908)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:17.210344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c8b779-0363-4fcf-9f80-6a99a0e0331b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:17.309675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7c8b779-0363-4fcf-9f80-6a99a0e0331b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '026cdb95ae1a42ee9f3176eb89c7b908'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:17.320614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7c8b779-0363-4fcf-9f80-6a99a0e0331b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '026cdb95ae1a42ee9f3176eb89c7b908'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:17.417970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7c8b779-0363-4fcf-9f80-6a99a0e0331b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '026cdb95ae1a42ee9f3176eb89c7b908'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:17.425408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7c8b779-0363-4fcf-9f80-6a99a0e0331b tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '026cdb95ae1a42ee9f3176eb89c7b908'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:17.433401 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1956/3862] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:02:16 2022] PATCH /identity/v3/domains/026cdb95ae1a42ee9f3176eb89c7b908 => generated 277 bytes in 652 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:17.434756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:17.437491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:17.469288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607efa3e-0b1b-4b5a-9e2b-8a12bfc6f693 tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:17.647596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32e17cc8320480ca87c01ab3fdffa2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a73c67cd3884404b77e671881ad136b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:17.649272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:17.652009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:17.652603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] PATH_INFO: `/v3/domains/026cdb95ae1a42ee9f3176eb89c7b908` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:17.686873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=026cdb95ae1a42ee9f3176eb89c7b908)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:17.689543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:17.728673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:17.749782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaeabd-0c1f-4fea-a3f4-9dbc66725059 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:17.750098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaeabd-0c1f-4fea-a3f4-9dbc66725059 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:17.750594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaeabd-0c1f-4fea-a3f4-9dbc66725059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:17.771334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '026cdb95ae1a42ee9f3176eb89c7b908'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:17.778695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '026cdb95ae1a42ee9f3176eb89c7b908'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:17.816682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7abaeabd-0c1f-4fea-a3f4-9dbc66725059 None None] MFA Rules not processed for user `3a394ae937c54b2a82c89628d4feeb72`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:17.818058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7abaeabd-0c1f-4fea-a3f4-9dbc66725059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:17.890585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:UPDATE=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:17.898855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '026cdb95ae1a42ee9f3176eb89c7b908', 'request_id': 'req-fb7700e8-f000-4578-8174-8e3c16b7c4fa'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:17.911737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '026cdb95ae1a42ee9f3176eb89c7b908', 'request_id': 'req-fb7700e8-f000-4578-8174-8e3c16b7c4fa'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:17.922586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '026cdb95ae1a42ee9f3176eb89c7b908', 'request_id': 'req-fb7700e8-f000-4578-8174-8e3c16b7c4fa'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:17.951183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] The token cache is being invalidate because project 026cdb95ae1a42ee9f3176eb89c7b908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:17.951641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:17.952923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:17.981946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '026cdb95ae1a42ee9f3176eb89c7b908', 'request_id': 'req-fb7700e8-f000-4578-8174-8e3c16b7c4fa'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:17.993198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '026cdb95ae1a42ee9f3176eb89c7b908', 'request_id': 'req-fb7700e8-f000-4578-8174-8e3c16b7c4fa'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:18.000869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '026cdb95ae1a42ee9f3176eb89c7b908', 'request_id': 'req-fb7700e8-f000-4578-8174-8e3c16b7c4fa'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:18.022394 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1907/3863] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:02:17 2022] DELETE /identity/v3/domains/026cdb95ae1a42ee9f3176eb89c7b908 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:18.026716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:18.038983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:18.093977 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1957/3864] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 03 21:02:18.275735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7700e8-f000-4578-8174-8e3c16b7c4fa tempest-DomainsTestJSON-1486899628 tempest-DomainsTestJSON-1486899628-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:18.294305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:18.294981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:18.295268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:18.295570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] PATH_INFO: `/v3/users/18faf58f4b7e49aca012a16043d85079` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:18.305896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18faf58f4b7e49aca012a16043d85079)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:18.308110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:18.482533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18faf58f4b7e49aca012a16043d85079'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:18.509427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:18.530283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18faf58f4b7e49aca012a16043d85079'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:18.531143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18faf58f4b7e49aca012a16043d85079'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:18.551034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18faf58f4b7e49aca012a16043d85079'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:18.570735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18faf58f4b7e49aca012a16043d85079'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:18.585405 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1908/3865] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:18 2022] DELETE /identity/v3/users/18faf58f4b7e49aca012a16043d85079 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:18.589964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb95c0-d6e2-4f38-ac7d-679b91f2af44 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:18.590441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb95c0-d6e2-4f38-ac7d-679b91f2af44 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:18.590666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb95c0-d6e2-4f38-ac7d-679b91f2af44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:18.645445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bdfb95c0-d6e2-4f38-ac7d-679b91f2af44 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:18.646977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bdfb95c0-d6e2-4f38-ac7d-679b91f2af44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:18.782565 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1958/3866] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:18.927191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:18.930237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:18.970050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb95c0-d6e2-4f38-ac7d-679b91f2af44 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:18.973579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bdfb95c0-d6e2-4f38-ac7d-679b91f2af44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:19.025767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb95c0-d6e2-4f38-ac7d-679b91f2af44 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:19.035226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:19.049241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:19.049607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:19.049826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] PATH_INFO: `/v3/users/50bba3c790c44425a9786b8d1404df28` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:19.055219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50bba3c790c44425a9786b8d1404df28)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:19.063510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:19.161072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdfa23-3704-403a-bb0e-a5e78a493f09 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:19.182457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a0df1-954b-46c8-bb52-04be9482b8f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:19.183142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a0df1-954b-46c8-bb52-04be9482b8f2 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:19.183634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a0df1-954b-46c8-bb52-04be9482b8f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:19.183770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a0df1-954b-46c8-bb52-04be9482b8f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:19.186240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50bba3c790c44425a9786b8d1404df28'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:19.210035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a0df1-954b-46c8-bb52-04be9482b8f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:19.218112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a0df1-954b-46c8-bb52-04be9482b8f2 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:19.224673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50bba3c790c44425a9786b8d1404df28'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:19.239340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50bba3c790c44425a9786b8d1404df28'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:19.241063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50bba3c790c44425a9786b8d1404df28'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:19.242106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50bba3c790c44425a9786b8d1404df28'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:19.295674 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1909/3867] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:19.295780 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1959/3868] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:18 2022] DELETE /identity/v3/users/50bba3c790c44425a9786b8d1404df28 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:20.326294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a0df1-954b-46c8-bb52-04be9482b8f2 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:20.347989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a0df1-954b-46c8-bb52-04be9482b8f2 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:20.534850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:20.536216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:20.536968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:20.538257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] PATH_INFO: `/v3/projects/b0fbc0c573fc45b58137b43959b57034` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:20.539820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0fbc0c573fc45b58137b43959b57034)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:20.542938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:20.649067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:20.651238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:20.651707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0fbc0c573fc45b58137b43959b57034', 'request_id': 'req-6223c113-9e53-4d4e-aa92-ad13907b6130'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:20.674456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0fbc0c573fc45b58137b43959b57034', 'request_id': 'req-6223c113-9e53-4d4e-aa92-ad13907b6130'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:20.675607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c02794-e3b2-4f32-a10f-325a7d34afb4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:20.699970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0fbc0c573fc45b58137b43959b57034', 'request_id': 'req-6223c113-9e53-4d4e-aa92-ad13907b6130'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:20.723664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] The token cache is being invalidate because project b0fbc0c573fc45b58137b43959b57034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:20.723982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:20.726114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:20.729233 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1910/3869] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:20 2022] DELETE /identity/v3/projects/b0fbc0c573fc45b58137b43959b57034 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:20.734347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:20.736166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:20.751614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:20.814695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:20.843010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88371e62-e4ad-4f0f-95b3-93219f49978b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:20.843898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88371e62-e4ad-4f0f-95b3-93219f49978b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:20.844187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88371e62-e4ad-4f0f-95b3-93219f49978b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:20.844469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88371e62-e4ad-4f0f-95b3-93219f49978b admin admin] PATH_INFO: `/v3/projects/c59b2c80975d499baf2115b959a9fa4f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:20.846688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88371e62-e4ad-4f0f-95b3-93219f49978b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c59b2c80975d499baf2115b959a9fa4f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:20.848754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88371e62-e4ad-4f0f-95b3-93219f49978b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:20.865791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223c113-9e53-4d4e-aa92-ad13907b6130 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:20.871382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:20.872027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:20.872534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:20.872659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] PATH_INFO: `/v3/users/e7f444821052471a8ae828275402b8ec` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:20.873999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f444821052471a8ae828275402b8ec)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:20.876625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:20.942876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-88371e62-e4ad-4f0f-95b3-93219f49978b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c59b2c80975d499baf2115b959a9fa4f', 'request_id': 'req-88371e62-e4ad-4f0f-95b3-93219f49978b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:20.950513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-88371e62-e4ad-4f0f-95b3-93219f49978b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c59b2c80975d499baf2115b959a9fa4f', 'request_id': 'req-88371e62-e4ad-4f0f-95b3-93219f49978b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:20.951615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f444821052471a8ae828275402b8ec'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:20.967220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-88371e62-e4ad-4f0f-95b3-93219f49978b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c59b2c80975d499baf2115b959a9fa4f', 'request_id': 'req-88371e62-e4ad-4f0f-95b3-93219f49978b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:20.999570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f444821052471a8ae828275402b8ec'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.000704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f444821052471a8ae828275402b8ec'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.007340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-88371e62-e4ad-4f0f-95b3-93219f49978b admin admin] The token cache is being invalidate because project c59b2c80975d499baf2115b959a9fa4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:21.007637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-88371e62-e4ad-4f0f-95b3-93219f49978b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.008439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-88371e62-e4ad-4f0f-95b3-93219f49978b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.011588 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1960/3870] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:20 2022] DELETE /identity/v3/projects/c59b2c80975d499baf2115b959a9fa4f => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:21.013213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f444821052471a8ae828275402b8ec'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.014768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2524eb-dbb4-405e-8526-5cfb3966b4c0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:21.014980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2524eb-dbb4-405e-8526-5cfb3966b4c0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:21.016637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2524eb-dbb4-405e-8526-5cfb3966b4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:21.025030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f444821052471a8ae828275402b8ec'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.036342 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1911/3871] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:20 2022] DELETE /identity/v3/users/e7f444821052471a8ae828275402b8ec => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:21.037984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:21.052787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:21.063554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cc2524eb-dbb4-405e-8526-5cfb3966b4c0 None None] MFA Rules not processed for user `1b2dd96d938b41d4ba40007aa072288b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:21.064879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cc2524eb-dbb4-405e-8526-5cfb3966b4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:21.170852 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1961/3872] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:21.216063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb40d48-b7cd-43d8-baa4-fc9f22b45ed0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:21.223561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:21.225061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:21.225641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:21.226297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] PATH_INFO: `/v3/users/b32e17cc8320480ca87c01ab3fdffa2c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:21.228042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b32e17cc8320480ca87c01ab3fdffa2c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:21.231321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:21.240126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2524eb-dbb4-405e-8526-5cfb3966b4c0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:21.242025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cc2524eb-dbb4-405e-8526-5cfb3966b4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:21.290480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:21.320396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b32e17cc8320480ca87c01ab3fdffa2c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.341713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b32e17cc8320480ca87c01ab3fdffa2c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.343046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b32e17cc8320480ca87c01ab3fdffa2c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.352665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b32e17cc8320480ca87c01ab3fdffa2c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.364680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b32e17cc8320480ca87c01ab3fdffa2c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.373323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2524eb-dbb4-405e-8526-5cfb3966b4c0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:21.380020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116608d8-439e-4ef0-b5d3-eb4070460d74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:21.380723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116608d8-439e-4ef0-b5d3-eb4070460d74 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:21.381171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116608d8-439e-4ef0-b5d3-eb4070460d74 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:21.381515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116608d8-439e-4ef0-b5d3-eb4070460d74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:21.383531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-116608d8-439e-4ef0-b5d3-eb4070460d74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:21.388640 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1912/3873] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:21 2022] DELETE /identity/v3/users/b32e17cc8320480ca87c01ab3fdffa2c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:21.411585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116608d8-439e-4ef0-b5d3-eb4070460d74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:21.413995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116608d8-439e-4ef0-b5d3-eb4070460d74 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:21.441903 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1962/3874] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:21.467750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:21.469489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:21.637296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ae16f-9ada-4fb7-945e-3b24d9f7f399 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:21.660667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdbd02-1536-4584-8f23-b97e40492c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:21.661774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdbd02-1536-4584-8f23-b97e40492c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:21.662474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdbd02-1536-4584-8f23-b97e40492c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:21.662981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdbd02-1536-4584-8f23-b97e40492c76 admin admin] PATH_INFO: `/v3/projects/c6a7ecf606c54105947f66a1434670a8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:21.664474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdbd02-1536-4584-8f23-b97e40492c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a7ecf606c54105947f66a1434670a8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:21.666685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdbd02-1536-4584-8f23-b97e40492c76 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:21.771749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-59bdbd02-1536-4584-8f23-b97e40492c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a7ecf606c54105947f66a1434670a8', 'request_id': 'req-59bdbd02-1536-4584-8f23-b97e40492c76'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.784711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-59bdbd02-1536-4584-8f23-b97e40492c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a7ecf606c54105947f66a1434670a8', 'request_id': 'req-59bdbd02-1536-4584-8f23-b97e40492c76'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.796228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-59bdbd02-1536-4584-8f23-b97e40492c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a7ecf606c54105947f66a1434670a8', 'request_id': 'req-59bdbd02-1536-4584-8f23-b97e40492c76'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.844014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-59bdbd02-1536-4584-8f23-b97e40492c76 admin admin] The token cache is being invalidate because project c6a7ecf606c54105947f66a1434670a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:21.844462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-59bdbd02-1536-4584-8f23-b97e40492c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.846886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-59bdbd02-1536-4584-8f23-b97e40492c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:21.854191 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1913/3875] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:21 2022] DELETE /identity/v3/projects/c6a7ecf606c54105947f66a1434670a8 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:21.856234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e656-95a7-4d3f-ae9c-1ce0f00b639b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:21.856695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e656-95a7-4d3f-ae9c-1ce0f00b639b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:21.857146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e656-95a7-4d3f-ae9c-1ce0f00b639b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:21.927184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-44b0e656-95a7-4d3f-ae9c-1ce0f00b639b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:21.928869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-44b0e656-95a7-4d3f-ae9c-1ce0f00b639b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:21.957662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:22.116770 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1963/3876] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:22.130778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0e656-95a7-4d3f-ae9c-1ce0f00b639b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:22.132483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-44b0e656-95a7-4d3f-ae9c-1ce0f00b639b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:22.140147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad65c9e-4752-453c-8c99-d15912b2a930 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:22.140465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad65c9e-4752-453c-8c99-d15912b2a930 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:22.140698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad65c9e-4752-453c-8c99-d15912b2a930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:22.162666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0e656-95a7-4d3f-ae9c-1ce0f00b639b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:22.179391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ed72f-9bf6-414e-ab75-615e80db0264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:22.181738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3ad65c9e-4752-453c-8c99-d15912b2a930 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:22.183285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ed72f-9bf6-414e-ab75-615e80db0264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:22.184310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3ad65c9e-4752-453c-8c99-d15912b2a930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:22.184948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ed72f-9bf6-414e-ab75-615e80db0264 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:22.185550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ed72f-9bf6-414e-ab75-615e80db0264 admin admin] PATH_INFO: `/v3/projects/5a73c67cd3884404b77e671881ad136b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:22.191027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ed72f-9bf6-414e-ab75-615e80db0264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a73c67cd3884404b77e671881ad136b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:22.205152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ed72f-9bf6-414e-ab75-615e80db0264 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:22.218045 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1914/3877] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:22.219280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad65c9e-4752-453c-8c99-d15912b2a930 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:22.231795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad65c9e-4752-453c-8c99-d15912b2a930 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:22.241128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1706b-f8e3-4e2b-acd8-ad4a8dfede06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:22.242093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1706b-f8e3-4e2b-acd8-ad4a8dfede06 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:22.242579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1706b-f8e3-4e2b-acd8-ad4a8dfede06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:22.243053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1706b-f8e3-4e2b-acd8-ad4a8dfede06 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:22.243906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1706b-f8e3-4e2b-acd8-ad4a8dfede06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:22.244428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1706b-f8e3-4e2b-acd8-ad4a8dfede06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:22.247012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1706b-f8e3-4e2b-acd8-ad4a8dfede06 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:22.272976 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1915/3878] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:22.273469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1706b-f8e3-4e2b-acd8-ad4a8dfede06 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:22.304619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1706b-f8e3-4e2b-acd8-ad4a8dfede06 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:22.307011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d1ed72f-9bf6-414e-ab75-615e80db0264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a73c67cd3884404b77e671881ad136b', 'request_id': 'req-9d1ed72f-9bf6-414e-ab75-615e80db0264'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:22.319910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d1ed72f-9bf6-414e-ab75-615e80db0264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a73c67cd3884404b77e671881ad136b', 'request_id': 'req-9d1ed72f-9bf6-414e-ab75-615e80db0264'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:22.334582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d1ed72f-9bf6-414e-ab75-615e80db0264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a73c67cd3884404b77e671881ad136b', 'request_id': 'req-9d1ed72f-9bf6-414e-ab75-615e80db0264'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:22.334891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:22.371501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d1ed72f-9bf6-414e-ab75-615e80db0264 admin admin] The token cache is being invalidate because project 5a73c67cd3884404b77e671881ad136b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:22.371956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d1ed72f-9bf6-414e-ab75-615e80db0264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:22.374237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9d1ed72f-9bf6-414e-ab75-615e80db0264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:22.389365 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1964/3879] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:22 2022] DELETE /identity/v3/projects/5a73c67cd3884404b77e671881ad136b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:22.396786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7c1b2-035b-4df9-a8dd-c2d1446b8c03 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:22.422431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7c1b2-035b-4df9-a8dd-c2d1446b8c03 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:22.422966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7c1b2-035b-4df9-a8dd-c2d1446b8c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:22.502948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b1d7c1b2-035b-4df9-a8dd-c2d1446b8c03 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:22.504570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b1d7c1b2-035b-4df9-a8dd-c2d1446b8c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:22.525653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5a770-bf89-437c-8d9d-6538004d2885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:22.527436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5a770-bf89-437c-8d9d-6538004d2885 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:22.528296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5a770-bf89-437c-8d9d-6538004d2885 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:22.529247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5a770-bf89-437c-8d9d-6538004d2885 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:22.530042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5a770-bf89-437c-8d9d-6538004d2885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:22.535015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5a770-bf89-437c-8d9d-6538004d2885 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:22.600338 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1916/3880] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:22 2022] POST /identity/v3/projects => generated 397 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:22.610848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5a770-bf89-437c-8d9d-6538004d2885 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:22.619192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8ab5a770-bf89-437c-8d9d-6538004d2885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:22.637852 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1965/3881] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:22.648119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19beb20-cbaa-4217-878d-b81849b36a53 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:22.648407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19beb20-cbaa-4217-878d-b81849b36a53 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:22.648833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19beb20-cbaa-4217-878d-b81849b36a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:22.671386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c19beb20-cbaa-4217-878d-b81849b36a53 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:22.672717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c19beb20-cbaa-4217-878d-b81849b36a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:22.731450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:22.785778 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1966/3882] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:22.787203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19beb20-cbaa-4217-878d-b81849b36a53 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:22.821556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19beb20-cbaa-4217-878d-b81849b36a53 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:22.836008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d356d-9686-4ed1-a1bb-c62e40e36f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:22.837511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d356d-9686-4ed1-a1bb-c62e40e36f3f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:22.839420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d356d-9686-4ed1-a1bb-c62e40e36f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:22.839882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d356d-9686-4ed1-a1bb-c62e40e36f3f admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:22.841158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5a770-bf89-437c-8d9d-6538004d2885 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:22.841433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d356d-9686-4ed1-a1bb-c62e40e36f3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:22.842858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d356d-9686-4ed1-a1bb-c62e40e36f3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:22.847918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d356d-9686-4ed1-a1bb-c62e40e36f3f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:22.853324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e11c11-043e-43e8-8099-5a1c6297ed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:22.853969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e11c11-043e-43e8-8099-5a1c6297ed1a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:22.856095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e11c11-043e-43e8-8099-5a1c6297ed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:22.856392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e11c11-043e-43e8-8099-5a1c6297ed1a admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:22.859076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e11c11-043e-43e8-8099-5a1c6297ed1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:22.863137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e11c11-043e-43e8-8099-5a1c6297ed1a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:22.902821 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1967/3883] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:22.905978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d356d-9686-4ed1-a1bb-c62e40e36f3f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:22.923253 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1917/3884] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:22 2022] POST /identity/v3/users => generated 299 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:22.927979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e11c11-043e-43e8-8099-5a1c6297ed1a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:22.957302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d356d-9686-4ed1-a1bb-c62e40e36f3f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:22.963187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e11c11-043e-43e8-8099-5a1c6297ed1a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:22.982166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7998d-39c0-444d-816e-4cce3e0ce528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:22.983738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc4575-0123-4727-b6ba-b26da4c64757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:22.987539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7998d-39c0-444d-816e-4cce3e0ce528 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:22.988313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc4575-0123-4727-b6ba-b26da4c64757 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:22.988825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7998d-39c0-444d-816e-4cce3e0ce528 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:22.989149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7998d-39c0-444d-816e-4cce3e0ce528 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:22.990799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc4575-0123-4727-b6ba-b26da4c64757 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:22.992085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7998d-39c0-444d-816e-4cce3e0ce528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:22.992518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc4575-0123-4727-b6ba-b26da4c64757 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:22.997519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc4575-0123-4727-b6ba-b26da4c64757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:23.000211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7998d-39c0-444d-816e-4cce3e0ce528 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:23.007075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc4575-0123-4727-b6ba-b26da4c64757 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:23.039078 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1918/3885] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:23.039078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7998d-39c0-444d-816e-4cce3e0ce528 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:23.060398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7998d-39c0-444d-816e-4cce3e0ce528 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:23.074446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe78e5-c90b-472a-8e4b-8f25510a4ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:23.075926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe78e5-c90b-472a-8e4b-8f25510a4ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:23.075926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe78e5-c90b-472a-8e4b-8f25510a4ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:23.075926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe78e5-c90b-472a-8e4b-8f25510a4ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:23.079315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe78e5-c90b-472a-8e4b-8f25510a4ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:23.083303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe78e5-c90b-472a-8e4b-8f25510a4ba1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:23.107674 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1968/3886] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:22 2022] POST /identity/v3/projects => generated 371 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:23.110369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc4575-0123-4727-b6ba-b26da4c64757 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:23.114536 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1919/3887] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:23.118941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe78e5-c90b-472a-8e4b-8f25510a4ba1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:23.141938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe78e5-c90b-472a-8e4b-8f25510a4ba1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:23.286618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e2118-8a5a-4864-b89c-97a322aba6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:23.287979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e2118-8a5a-4864-b89c-97a322aba6ec admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:23.291142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e2118-8a5a-4864-b89c-97a322aba6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:23.291507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e2118-8a5a-4864-b89c-97a322aba6ec admin admin] PATH_INFO: `/v3/projects/58e50b6481544a30bce4e39f80dd6eae/users/2a51b34b13cf44d99b9d385a5f29ad1c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:23.299530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc4575-0123-4727-b6ba-b26da4c64757 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:23.313019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7811208-e5e9-4a93-8291-03f457de9bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:23.313904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7811208-e5e9-4a93-8291-03f457de9bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:23.315593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7811208-e5e9-4a93-8291-03f457de9bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:23.315956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7811208-e5e9-4a93-8291-03f457de9bd7 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:23.320645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7811208-e5e9-4a93-8291-03f457de9bd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:23.321829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:23.329311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7811208-e5e9-4a93-8291-03f457de9bd7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:23.334034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e2118-8a5a-4864-b89c-97a322aba6ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=58e50b6481544a30bce4e39f80dd6eae, user_id=2a51b34b13cf44d99b9d385a5f29ad1c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:23.343130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e2118-8a5a-4864-b89c-97a322aba6ec admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:23.382343 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1920/3888] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:23 2022] PUT /identity/v3/projects/58e50b6481544a30bce4e39f80dd6eae/users/2a51b34b13cf44d99b9d385a5f29ad1c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:23.385362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667af80-69ff-4f50-bc0b-76a22e9417f3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:23.386699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667af80-69ff-4f50-bc0b-76a22e9417f3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:23.387211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667af80-69ff-4f50-bc0b-76a22e9417f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:23.392798 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1969/3889] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:23 2022] POST /identity/v3/users => generated 286 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:23.395480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7811208-e5e9-4a93-8291-03f457de9bd7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:23.423078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7811208-e5e9-4a93-8291-03f457de9bd7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:23.458141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4667af80-69ff-4f50-bc0b-76a22e9417f3 None None] MFA Rules not processed for user `2a51b34b13cf44d99b9d385a5f29ad1c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:23.463487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4667af80-69ff-4f50-bc0b-76a22e9417f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:23.598886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc125b-08aa-4e73-86d5-67b9c253a82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:23.599524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc125b-08aa-4e73-86d5-67b9c253a82a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:23.599815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc125b-08aa-4e73-86d5-67b9c253a82a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:23.600098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc125b-08aa-4e73-86d5-67b9c253a82a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:23.600759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc125b-08aa-4e73-86d5-67b9c253a82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:23.607318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc125b-08aa-4e73-86d5-67b9c253a82a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:23.630465 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1970/3890] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:23.635582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc125b-08aa-4e73-86d5-67b9c253a82a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:23.654805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc125b-08aa-4e73-86d5-67b9c253a82a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:23.669722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128900e-9399-4dc1-9391-834de432a4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:23.670359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128900e-9399-4dc1-9391-834de432a4cf admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:23.670645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128900e-9399-4dc1-9391-834de432a4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:23.670924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128900e-9399-4dc1-9391-834de432a4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:23.671581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128900e-9399-4dc1-9391-834de432a4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:23.675498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128900e-9399-4dc1-9391-834de432a4cf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:23.709874 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1971/3891] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:02:23.710766 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1921/3892] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:23 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:23.711117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128900e-9399-4dc1-9391-834de432a4cf admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:23.727453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128900e-9399-4dc1-9391-834de432a4cf admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:23.742342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54d20d-e666-495f-a737-96af83e25425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:23.744317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54d20d-e666-495f-a737-96af83e25425 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:23.746154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54d20d-e666-495f-a737-96af83e25425 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:23.746578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54d20d-e666-495f-a737-96af83e25425 admin admin] PATH_INFO: `/v3/projects/40b96e7c9c8f4e6f9e165819a7c06b5e/users/e97308e8274c4a35b0359bfcfea84b93/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:23.795852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54d20d-e666-495f-a737-96af83e25425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40b96e7c9c8f4e6f9e165819a7c06b5e, user_id=e97308e8274c4a35b0359bfcfea84b93, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:23.804779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54d20d-e666-495f-a737-96af83e25425 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:23.819923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:23.858119 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1972/3893] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:23 2022] PUT /identity/v3/projects/40b96e7c9c8f4e6f9e165819a7c06b5e/users/e97308e8274c4a35b0359bfcfea84b93/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:23.859913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3e202-464a-4715-ad90-dbe900e66d4c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:23.860216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3e202-464a-4715-ad90-dbe900e66d4c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:23.860504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3e202-464a-4715-ad90-dbe900e66d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:23.945186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-80a3e202-464a-4715-ad90-dbe900e66d4c None None] MFA Rules not processed for user `e97308e8274c4a35b0359bfcfea84b93`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:23.946504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-80a3e202-464a-4715-ad90-dbe900e66d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:24.156864 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1922/3894] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:02:24.161624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54d20d-e666-495f-a737-96af83e25425 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:24.326985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54d20d-e666-495f-a737-96af83e25425 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:24.333431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd2c1d-668f-4fef-9dac-022c5e26755d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:24.334363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd2c1d-668f-4fef-9dac-022c5e26755d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:24.334865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd2c1d-668f-4fef-9dac-022c5e26755d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:24.335392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd2c1d-668f-4fef-9dac-022c5e26755d admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:24.336238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd2c1d-668f-4fef-9dac-022c5e26755d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:24.342062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd2c1d-668f-4fef-9dac-022c5e26755d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:24.352717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3e202-464a-4715-ad90-dbe900e66d4c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:24.354930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-80a3e202-464a-4715-ad90-dbe900e66d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:24.412976 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1973/3895] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:24 2022] POST /identity/v3/projects => generated 373 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:24.417759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd2c1d-668f-4fef-9dac-022c5e26755d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:24.526881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3e202-464a-4715-ad90-dbe900e66d4c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:24.534284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:24.551369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd2c1d-668f-4fef-9dac-022c5e26755d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:24.567029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de67d0-13a5-4fb7-a262-566fd2d67357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:24.573024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de67d0-13a5-4fb7-a262-566fd2d67357 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:24.574196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de67d0-13a5-4fb7-a262-566fd2d67357 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:24.574621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de67d0-13a5-4fb7-a262-566fd2d67357 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:24.575509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de67d0-13a5-4fb7-a262-566fd2d67357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:24.577658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de67d0-13a5-4fb7-a262-566fd2d67357 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:24.618073 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1974/3896] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:24 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:24.619053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de67d0-13a5-4fb7-a262-566fd2d67357 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:24.630585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd97c8-a3d4-4921-aeb1-3ec602d8a4bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:24.632468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd97c8-a3d4-4921-aeb1-3ec602d8a4bb service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:24.633024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd97c8-a3d4-4921-aeb1-3ec602d8a4bb service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:24.633663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd97c8-a3d4-4921-aeb1-3ec602d8a4bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:24.634757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de67d0-13a5-4fb7-a262-566fd2d67357 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:24.641110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df837f-e3f1-4c75-a785-75d53589efb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:24.641749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df837f-e3f1-4c75-a785-75d53589efb6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:24.642043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df837f-e3f1-4c75-a785-75d53589efb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:24.642325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df837f-e3f1-4c75-a785-75d53589efb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:24.642986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df837f-e3f1-4c75-a785-75d53589efb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:24.645062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df837f-e3f1-4c75-a785-75d53589efb6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:24.662355 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1975/3897] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:24.663290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df837f-e3f1-4c75-a785-75d53589efb6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:24.675452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df837f-e3f1-4c75-a785-75d53589efb6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:24.681966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d013dc-3984-4f1e-bbe7-10f1430af654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:24.683188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d013dc-3984-4f1e-bbe7-10f1430af654 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:24.683500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d013dc-3984-4f1e-bbe7-10f1430af654 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:24.683768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d013dc-3984-4f1e-bbe7-10f1430af654 admin admin] PATH_INFO: `/v3/projects/96311ac1a0b849a48e4e123e784f8e36/users/3dc61cae01c44ef2b8be0dfe8fe8ec5b/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:24.705446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d013dc-3984-4f1e-bbe7-10f1430af654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96311ac1a0b849a48e4e123e784f8e36, user_id=3dc61cae01c44ef2b8be0dfe8fe8ec5b, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:24.708637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d013dc-3984-4f1e-bbe7-10f1430af654 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:24.712084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd97c8-a3d4-4921-aeb1-3ec602d8a4bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:24.715851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd97c8-a3d4-4921-aeb1-3ec602d8a4bb service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:24.732185 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1976/3898] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:24 2022] PUT /identity/v3/projects/96311ac1a0b849a48e4e123e784f8e36/users/3dc61cae01c44ef2b8be0dfe8fe8ec5b/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:24.733373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d013dc-3984-4f1e-bbe7-10f1430af654 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:24.747947 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1923/3899] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:24 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:24.779823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:24.843037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d013dc-3984-4f1e-bbe7-10f1430af654 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:24.857300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fac9ea-74e9-486a-b05a-fb7dea1ea490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:24.857926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fac9ea-74e9-486a-b05a-fb7dea1ea490 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:24.858229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fac9ea-74e9-486a-b05a-fb7dea1ea490 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:24.858517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fac9ea-74e9-486a-b05a-fb7dea1ea490 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:24.859218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fac9ea-74e9-486a-b05a-fb7dea1ea490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:24.860854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fac9ea-74e9-486a-b05a-fb7dea1ea490 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:24.887754 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1977/3900] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:24.898251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd97c8-a3d4-4921-aeb1-3ec602d8a4bb service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:24.927386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd97c8-a3d4-4921-aeb1-3ec602d8a4bb service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:24.938416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1443d46-106a-442b-84cd-3adeb3585219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:24.939111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1443d46-106a-442b-84cd-3adeb3585219 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:24.939407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1443d46-106a-442b-84cd-3adeb3585219 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:24.939684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1443d46-106a-442b-84cd-3adeb3585219 admin admin] PATH_INFO: `/v3/projects/96311ac1a0b849a48e4e123e784f8e36/users/3dc61cae01c44ef2b8be0dfe8fe8ec5b/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:24.941792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1443d46-106a-442b-84cd-3adeb3585219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96311ac1a0b849a48e4e123e784f8e36, user_id=3dc61cae01c44ef2b8be0dfe8fe8ec5b, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:24.945185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1443d46-106a-442b-84cd-3adeb3585219 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:24.990023 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1924/3901] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:24 2022] PUT /identity/v3/projects/96311ac1a0b849a48e4e123e784f8e36/users/3dc61cae01c44ef2b8be0dfe8fe8ec5b/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:25.001862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e97ffa-d849-43da-8773-b08637666071 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:25.002307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e97ffa-d849-43da-8773-b08637666071 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:25.002707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e97ffa-d849-43da-8773-b08637666071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:25.075033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-29e97ffa-d849-43da-8773-b08637666071 None None] MFA Rules not processed for user `3dc61cae01c44ef2b8be0dfe8fe8ec5b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:25.076302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-29e97ffa-d849-43da-8773-b08637666071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:25.238498 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1978/3902] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:25.240225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1443d46-106a-442b-84cd-3adeb3585219 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:25.257633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1443d46-106a-442b-84cd-3adeb3585219 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:25.272613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9857b-5575-4c66-8bfd-58b235c4dde9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:25.273429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9857b-5575-4c66-8bfd-58b235c4dde9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:25.278281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9857b-5575-4c66-8bfd-58b235c4dde9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:25.278663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9857b-5575-4c66-8bfd-58b235c4dde9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:25.279291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9857b-5575-4c66-8bfd-58b235c4dde9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:25.280746 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-e0c9857b-5575-4c66-8bfd-58b235c4dde9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:25.280746 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:25.280746 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:25.283057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9857b-5575-4c66-8bfd-58b235c4dde9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:25.360302 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1925/3903] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:25 2022] POST /identity/v3/services => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:25.360839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e97ffa-d849-43da-8773-b08637666071 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:25.376968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e97ffa-d849-43da-8773-b08637666071 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:25.429511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561dae05-4c40-4fc7-a446-1b5b6e51c358 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:25.429511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dae05-4c40-4fc7-a446-1b5b6e51c358 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:25.429511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dae05-4c40-4fc7-a446-1b5b6e51c358 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:25.429869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dae05-4c40-4fc7-a446-1b5b6e51c358 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:25.429869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561dae05-4c40-4fc7-a446-1b5b6e51c358 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:25.429869 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-561dae05-4c40-4fc7-a446-1b5b6e51c358 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:25.429869 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:25.429869 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:25.429869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561dae05-4c40-4fc7-a446-1b5b6e51c358 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:25.469215 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1979/3904] 10.222.0.16 () {60 vars in 1213 bytes} [Sat Sep 3 21:02:25 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 334 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:25.471898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9857b-5575-4c66-8bfd-58b235c4dde9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:25.498523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9857b-5575-4c66-8bfd-58b235c4dde9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:25.505333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2418d-2389-4ff1-89a9-d9cd1e654029 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:25.506078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2418d-2389-4ff1-89a9-d9cd1e654029 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:25.506472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2418d-2389-4ff1-89a9-d9cd1e654029 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:25.506861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2418d-2389-4ff1-89a9-d9cd1e654029 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:25.507847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2418d-2389-4ff1-89a9-d9cd1e654029 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:25.519691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2418d-2389-4ff1-89a9-d9cd1e654029 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:25.549735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561dae05-4c40-4fc7-a446-1b5b6e51c358 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:25.552446 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1926/3905] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:25 2022] POST /identity/v3/services => generated 289 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:25.565811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561dae05-4c40-4fc7-a446-1b5b6e51c358 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:25.626909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8037a59-df79-45c4-a09c-6cc516f7c37a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:25.627554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8037a59-df79-45c4-a09c-6cc516f7c37a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:25.629550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8037a59-df79-45c4-a09c-6cc516f7c37a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:25.629859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8037a59-df79-45c4-a09c-6cc516f7c37a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:25.633351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8037a59-df79-45c4-a09c-6cc516f7c37a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:25.639016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8037a59-df79-45c4-a09c-6cc516f7c37a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:25.658326 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1980/3906] 10.222.0.16 () {60 vars in 1213 bytes} [Sat Sep 3 21:02:25 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 332 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:25.658981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2418d-2389-4ff1-89a9-d9cd1e654029 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:25.680559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2418d-2389-4ff1-89a9-d9cd1e654029 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:25.705834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65330875-bd6b-4579-9571-09bd7febfcaa tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:25.708351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65330875-bd6b-4579-9571-09bd7febfcaa tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:25.709129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65330875-bd6b-4579-9571-09bd7febfcaa tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:25.709590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65330875-bd6b-4579-9571-09bd7febfcaa tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:25.711176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65330875-bd6b-4579-9571-09bd7febfcaa tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:25.717493 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-65330875-bd6b-4579-9571-09bd7febfcaa tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoint_groups": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:25.717493 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:25.717493 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:25.719895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65330875-bd6b-4579-9571-09bd7febfcaa tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:25.737175 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1927/3907] 10.222.0.16 () {58 vars in 1191 bytes} [Sat Sep 3 21:02:25 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 764 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:25.741352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8037a59-df79-45c4-a09c-6cc516f7c37a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:25.755504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8037a59-df79-45c4-a09c-6cc516f7c37a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:25.767613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04b118-2232-4017-96cd-4df823a40c9a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:25.768277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04b118-2232-4017-96cd-4df823a40c9a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:25.768563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04b118-2232-4017-96cd-4df823a40c9a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:25.768845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04b118-2232-4017-96cd-4df823a40c9a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/6f3bcf60cd58466280a9fe1748f7e9bf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:25.769522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04b118-2232-4017-96cd-4df823a40c9a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=6f3bcf60cd58466280a9fe1748f7e9bf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:25.773473 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-ec04b118-2232-4017-96cd-4df823a40c9a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:25.773473 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:25.773473 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:25.776210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04b118-2232-4017-96cd-4df823a40c9a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:25.793813 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1981/3908] 10.222.0.16 () {58 vars in 1290 bytes} [Sat Sep 3 21:02:25 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups/6f3bcf60cd58466280a9fe1748f7e9bf => generated 332 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:02:25.797887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65330875-bd6b-4579-9571-09bd7febfcaa tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:25.815581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65330875-bd6b-4579-9571-09bd7febfcaa tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:25.831356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb271a4-ff1c-4195-80d7-8322be3afe4b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:25.832036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb271a4-ff1c-4195-80d7-8322be3afe4b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:25.832327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb271a4-ff1c-4195-80d7-8322be3afe4b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:25.833564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb271a4-ff1c-4195-80d7-8322be3afe4b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/6f3bcf60cd58466280a9fe1748f7e9bf` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:25.833564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb271a4-ff1c-4195-80d7-8322be3afe4b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=6f3bcf60cd58466280a9fe1748f7e9bf)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:25.834815 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-cdb271a4-ff1c-4195-80d7-8322be3afe4b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:25.834815 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:25.834815 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:25.835725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb271a4-ff1c-4195-80d7-8322be3afe4b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:25.852874 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1928/3909] 10.222.0.16 () {58 vars in 1291 bytes} [Sat Sep 3 21:02:25 2022] HEAD /identity/v3/OS-EP-FILTER/endpoint_groups/6f3bcf60cd58466280a9fe1748f7e9bf => generated 0 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 03 21:02:25.857362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04b118-2232-4017-96cd-4df823a40c9a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:25.869594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04b118-2232-4017-96cd-4df823a40c9a tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:25.888403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7bd86-8242-4a76-a5c1-e1cce0abe3a9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:25.888934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7bd86-8242-4a76-a5c1-e1cce0abe3a9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:25.890263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7bd86-8242-4a76-a5c1-e1cce0abe3a9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:25.891157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7bd86-8242-4a76-a5c1-e1cce0abe3a9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/6f3bcf60cd58466280a9fe1748f7e9bf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:25.894471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7bd86-8242-4a76-a5c1-e1cce0abe3a9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=6f3bcf60cd58466280a9fe1748f7e9bf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:25.895626 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-06a7bd86-8242-4a76-a5c1-e1cce0abe3a9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:25.895626 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:25.895626 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:25.896954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7bd86-8242-4a76-a5c1-e1cce0abe3a9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:25.928767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb271a4-ff1c-4195-80d7-8322be3afe4b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:25.939203 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1982/3910] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Sep 3 21:02:25 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/6f3bcf60cd58466280a9fe1748f7e9bf => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:25.940295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7bd86-8242-4a76-a5c1-e1cce0abe3a9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:25.964742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7bd86-8242-4a76-a5c1-e1cce0abe3a9 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:25.981488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0668a-f97a-4062-86a6-d3e7855a393b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:25.982347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0668a-f97a-4062-86a6-d3e7855a393b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:25.983894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0668a-f97a-4062-86a6-d3e7855a393b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:25.984288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0668a-f97a-4062-86a6-d3e7855a393b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:25.985964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0668a-f97a-4062-86a6-d3e7855a393b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:25.991014 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-76e0668a-f97a-4062-86a6-d3e7855a393b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoint_groups": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:25.991014 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:25.991014 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:25.994915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0668a-f97a-4062-86a6-d3e7855a393b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.024657 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1983/3911] 10.222.0.16 () {58 vars in 1191 bytes} [Sat Sep 3 21:02:25 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 451 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:26.029201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0668a-f97a-4062-86a6-d3e7855a393b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:26.051025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0668a-f97a-4062-86a6-d3e7855a393b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.070565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58b0ab-873d-4386-9c62-c114c973c2e3 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.071304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58b0ab-873d-4386-9c62-c114c973c2e3 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.072173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58b0ab-873d-4386-9c62-c114c973c2e3 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.072522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58b0ab-873d-4386-9c62-c114c973c2e3 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/6f3bcf60cd58466280a9fe1748f7e9bf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.073278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f58b0ab-873d-4386-9c62-c114c973c2e3 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=6f3bcf60cd58466280a9fe1748f7e9bf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.077209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f58b0ab-873d-4386-9c62-c114c973c2e3 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.087858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-9f58b0ab-873d-4386-9c62-c114c973c2e3 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Could not find Endpoint Group: 6f3bcf60cd58466280a9fe1748f7e9bf.: keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: 6f3bcf60cd58466280a9fe1748f7e9bf. Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_ep_filter.py", line 107, in delete Sep 03 21:02:26.105911 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return (PROVIDERS.catalog_api.delete_endpoint_group(endpoint_group_id), Sep 03 21:02:26.107063 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 519, in delete_endpoint_group Sep 03 21:02:26.107063 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application endpoint_group_ref = self._get_endpoint_group(session, Sep 03 21:02:26.107063 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 495, in _get_endpoint_group Sep 03 21:02:26.107063 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.EndpointGroupNotFound( Sep 03 21:02:26.107063 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: 6f3bcf60cd58466280a9fe1748f7e9bf. Sep 03 21:02:26.107063 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:26.107691 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1984/3912] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Sep 3 21:02:26 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/6f3bcf60cd58466280a9fe1748f7e9bf => generated 120 bytes in 79 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:02:26.112144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58b0ab-873d-4386-9c62-c114c973c2e3 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:26.131430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58b0ab-873d-4386-9c62-c114c973c2e3 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.138947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb271a4-ff1c-4195-80d7-8322be3afe4b tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.143944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4e851-028f-4d38-825c-292b82b9af02 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.145907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4e851-028f-4d38-825c-292b82b9af02 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.146192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4e851-028f-4d38-825c-292b82b9af02 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.146477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4e851-028f-4d38-825c-292b82b9af02 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/services/8ce756ba3c444493a0b3eaf9f63074b5` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.148120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4e851-028f-4d38-825c-292b82b9af02 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:delete_service(service_id=8ce756ba3c444493a0b3eaf9f63074b5)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.149171 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-08f4e851-028f-4d38-825c-292b82b9af02 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:26.149171 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:26.149171 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:26.150080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4e851-028f-4d38-825c-292b82b9af02 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.154771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ff84d-decb-46e5-94c1-2adf70dd26ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.155563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ff84d-decb-46e5-94c1-2adf70dd26ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.155853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ff84d-decb-46e5-94c1-2adf70dd26ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.156130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ff84d-decb-46e5-94c1-2adf70dd26ae admin admin] PATH_INFO: `/v3/users/2a51b34b13cf44d99b9d385a5f29ad1c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.160860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ff84d-decb-46e5-94c1-2adf70dd26ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a51b34b13cf44d99b9d385a5f29ad1c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.162867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ff84d-decb-46e5-94c1-2adf70dd26ae admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.187547 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1985/3913] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:26 2022] DELETE /identity/v3/services/8ce756ba3c444493a0b3eaf9f63074b5 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:26.193082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4e851-028f-4d38-825c-292b82b9af02 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:26.222668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4e851-028f-4d38-825c-292b82b9af02 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.228573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590d841-9b9d-4f3f-94d5-310792931618 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.229230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590d841-9b9d-4f3f-94d5-310792931618 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.230723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590d841-9b9d-4f3f-94d5-310792931618 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.231030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590d841-9b9d-4f3f-94d5-310792931618 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.231698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590d841-9b9d-4f3f-94d5-310792931618 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.233592 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-2590d841-9b9d-4f3f-94d5-310792931618 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:26.233592 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:26.233592 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:26.234512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590d841-9b9d-4f3f-94d5-310792931618 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.239532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:26.259028 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1986/3914] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:26 2022] POST /identity/v3/services => generated 288 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:26.260530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590d841-9b9d-4f3f-94d5-310792931618 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:26.289628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590d841-9b9d-4f3f-94d5-310792931618 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.300671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d8191-35a6-4679-af34-b4661d69fe3c tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.301331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d8191-35a6-4679-af34-b4661d69fe3c tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.301629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d8191-35a6-4679-af34-b4661d69fe3c tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.301903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d8191-35a6-4679-af34-b4661d69fe3c tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.302576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d8191-35a6-4679-af34-b4661d69fe3c tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.305609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-594ff84d-decb-46e5-94c1-2adf70dd26ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a51b34b13cf44d99b9d385a5f29ad1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:26.308698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d8191-35a6-4679-af34-b4661d69fe3c tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.333094 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1987/3915] 10.222.0.16 () {60 vars in 1213 bytes} [Sat Sep 3 21:02:26 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 332 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:26.335794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d8191-35a6-4679-af34-b4661d69fe3c tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:26.336862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-594ff84d-decb-46e5-94c1-2adf70dd26ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a51b34b13cf44d99b9d385a5f29ad1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:26.337714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-594ff84d-decb-46e5-94c1-2adf70dd26ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a51b34b13cf44d99b9d385a5f29ad1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:26.346095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-594ff84d-decb-46e5-94c1-2adf70dd26ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a51b34b13cf44d99b9d385a5f29ad1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:26.357466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-594ff84d-decb-46e5-94c1-2adf70dd26ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a51b34b13cf44d99b9d385a5f29ad1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:26.358064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d8191-35a6-4679-af34-b4661d69fe3c tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.365738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862380c4-ab5e-470f-b8a8-ade38bd753df tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.366437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862380c4-ab5e-470f-b8a8-ade38bd753df tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.366804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862380c4-ab5e-470f-b8a8-ade38bd753df tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.367161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862380c4-ab5e-470f-b8a8-ade38bd753df tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.367962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862380c4-ab5e-470f-b8a8-ade38bd753df tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.368296 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1929/3916] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:25 2022] DELETE /identity/v3/users/2a51b34b13cf44d99b9d385a5f29ad1c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:26.370127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862380c4-ab5e-470f-b8a8-ade38bd753df tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.373332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64549d88-530c-40ce-9c70-436e9a1a4a4e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.373722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64549d88-530c-40ce-9c70-436e9a1a4a4e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.374074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64549d88-530c-40ce-9c70-436e9a1a4a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.398343 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1988/3917] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:26 2022] POST /identity/v3/services => generated 289 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:26.404370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862380c4-ab5e-470f-b8a8-ade38bd753df tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:26.408556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-862380c4-ab5e-470f-b8a8-ade38bd753df tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:26.413285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-64549d88-530c-40ce-9c70-436e9a1a4a4e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:26.418316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-64549d88-530c-40ce-9c70-436e9a1a4a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:26.466073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862380c4-ab5e-470f-b8a8-ade38bd753df tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.575772 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1930/3918] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:26 2022] POST /identity/v3/auth/tokens => generated 3641 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:26.577733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd017e09-57f0-49d1-8f92-fc1a944423ed tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.578389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd017e09-57f0-49d1-8f92-fc1a944423ed tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.578676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd017e09-57f0-49d1-8f92-fc1a944423ed tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.578960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd017e09-57f0-49d1-8f92-fc1a944423ed tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/fe0ba2f0c2304ac4a1919e6ef438bd6f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.579653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd017e09-57f0-49d1-8f92-fc1a944423ed tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:update_endpoint_group(endpoint_group_id=fe0ba2f0c2304ac4a1919e6ef438bd6f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.580753 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-bd017e09-57f0-49d1-8f92-fc1a944423ed tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:26.580753 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:26.580753 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:26.581950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd017e09-57f0-49d1-8f92-fc1a944423ed tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.586960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64549d88-530c-40ce-9c70-436e9a1a4a4e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:26.601133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-64549d88-530c-40ce-9c70-436e9a1a4a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:26.611687 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1989/3919] 10.222.0.16 () {60 vars in 1313 bytes} [Sat Sep 3 21:02:26 2022] PATCH /identity/v3/OS-EP-FILTER/endpoint_groups/fe0ba2f0c2304ac4a1919e6ef438bd6f => generated 335 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:26.615358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd017e09-57f0-49d1-8f92-fc1a944423ed tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:26.632092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd017e09-57f0-49d1-8f92-fc1a944423ed tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.648412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb59f0b-d65a-4f58-9fd7-2ac55b706261 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.650342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb59f0b-d65a-4f58-9fd7-2ac55b706261 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.650654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb59f0b-d65a-4f58-9fd7-2ac55b706261 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.650951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb59f0b-d65a-4f58-9fd7-2ac55b706261 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/services/fa5688aa407241dbbe99d2edfc791ba3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.652931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb59f0b-d65a-4f58-9fd7-2ac55b706261 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:delete_service(service_id=fa5688aa407241dbbe99d2edfc791ba3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.654713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb59f0b-d65a-4f58-9fd7-2ac55b706261 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.688259 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1990/3920] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:26 2022] DELETE /identity/v3/services/fa5688aa407241dbbe99d2edfc791ba3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:26.693075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb59f0b-d65a-4f58-9fd7-2ac55b706261 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:26.715997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:26.743042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb59f0b-d65a-4f58-9fd7-2ac55b706261 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.746687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64549d88-530c-40ce-9c70-436e9a1a4a4e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.754374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbc852-ee8e-45d1-9150-c039522fd917 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.755039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbc852-ee8e-45d1-9150-c039522fd917 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.755449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbc852-ee8e-45d1-9150-c039522fd917 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.756437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbc852-ee8e-45d1-9150-c039522fd917 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/fe0ba2f0c2304ac4a1919e6ef438bd6f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.757081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbc852-ee8e-45d1-9150-c039522fd917 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=fe0ba2f0c2304ac4a1919e6ef438bd6f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.761336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbc852-ee8e-45d1-9150-c039522fd917 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.790277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187f828-b33f-4ff2-838d-48a2d98834a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.792107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187f828-b33f-4ff2-838d-48a2d98834a4 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.793321 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1991/3921] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Sep 3 21:02:26 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/fe0ba2f0c2304ac4a1919e6ef438bd6f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:26.797462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187f828-b33f-4ff2-838d-48a2d98834a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.798337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187f828-b33f-4ff2-838d-48a2d98834a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.798634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbc852-ee8e-45d1-9150-c039522fd917 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:26.815510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbc852-ee8e-45d1-9150-c039522fd917 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.821036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92182a28-2fb9-49d1-b464-c7d8250f6a64 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.822112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92182a28-2fb9-49d1-b464-c7d8250f6a64 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.822409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92182a28-2fb9-49d1-b464-c7d8250f6a64 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.822693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92182a28-2fb9-49d1-b464-c7d8250f6a64 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/services/bde665bdd39f4db5a09dad385685b92e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.823820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92182a28-2fb9-49d1-b464-c7d8250f6a64 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:delete_service(service_id=bde665bdd39f4db5a09dad385685b92e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.824318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187f828-b33f-4ff2-838d-48a2d98834a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.825743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92182a28-2fb9-49d1-b464-c7d8250f6a64 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.826424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187f828-b33f-4ff2-838d-48a2d98834a4 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.873360 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1931/3922] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:26 2022] GET /identity/v3/auth/tokens => generated 3373 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:02:26.873945 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1992/3923] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:26 2022] DELETE /identity/v3/services/bde665bdd39f4db5a09dad385685b92e => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:26.879495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187f828-b33f-4ff2-838d-48a2d98834a4 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:26.893093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187f828-b33f-4ff2-838d-48a2d98834a4 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.914846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e808787-a2a9-495d-9cc3-93865eb15590 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.915856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e808787-a2a9-495d-9cc3-93865eb15590 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.916348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e808787-a2a9-495d-9cc3-93865eb15590 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.917087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e808787-a2a9-495d-9cc3-93865eb15590 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/ac7b370f7a0c4ac3b163dc373d5e59c8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.917906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e808787-a2a9-495d-9cc3-93865eb15590 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=ac7b370f7a0c4ac3b163dc373d5e59c8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.919569 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-4e808787-a2a9-495d-9cc3-93865eb15590 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:26.919569 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:26.919569 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:26.921919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e808787-a2a9-495d-9cc3-93865eb15590 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:26.941621 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1932/3924] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Sep 3 21:02:26 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/ac7b370f7a0c4ac3b163dc373d5e59c8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:26.945979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92182a28-2fb9-49d1-b464-c7d8250f6a64 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:26.962281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92182a28-2fb9-49d1-b464-c7d8250f6a64 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:26.973586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e55262-99e2-4544-a4e3-ad7742d52595 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc61cae01c44ef2b8be0dfe8fe8ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96311ac1a0b849a48e4e123e784f8e36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:26.974047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e55262-99e2-4544-a4e3-ad7742d52595 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:26.974656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e55262-99e2-4544-a4e3-ad7742d52595 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:26.975405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e55262-99e2-4544-a4e3-ad7742d52595 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] PATH_INFO: `/v3/services/3b1176e06c9a4dc1a9161de2623f4a7d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:26.976068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e55262-99e2-4544-a4e3-ad7742d52595 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorizing `identity:delete_service(service_id=3b1176e06c9a4dc1a9161de2623f4a7d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:26.980531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e55262-99e2-4544-a4e3-ad7742d52595 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:27.028543 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1993/3925] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:26 2022] DELETE /identity/v3/services/3b1176e06c9a4dc1a9161de2623f4a7d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:27.032643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e808787-a2a9-495d-9cc3-93865eb15590 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:27.034960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4e808787-a2a9-495d-9cc3-93865eb15590 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:27.081335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e808787-a2a9-495d-9cc3-93865eb15590 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:27.117681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b7e1f-b39a-428b-b155-ca49ee3a5fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:27.119924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b7e1f-b39a-428b-b155-ca49ee3a5fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:27.120331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b7e1f-b39a-428b-b155-ca49ee3a5fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:27.121879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b7e1f-b39a-428b-b155-ca49ee3a5fc1 admin admin] PATH_INFO: `/v3/users/e97308e8274c4a35b0359bfcfea84b93` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:27.123355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b7e1f-b39a-428b-b155-ca49ee3a5fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e97308e8274c4a35b0359bfcfea84b93)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:27.125530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b7e1f-b39a-428b-b155-ca49ee3a5fc1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:27.328489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d9b7e1f-b39a-428b-b155-ca49ee3a5fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e97308e8274c4a35b0359bfcfea84b93'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:27.360818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e55262-99e2-4544-a4e3-ad7742d52595 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:27.366730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b6e55262-99e2-4544-a4e3-ad7742d52595 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:27.377603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d9b7e1f-b39a-428b-b155-ca49ee3a5fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e97308e8274c4a35b0359bfcfea84b93'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:27.378364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d9b7e1f-b39a-428b-b155-ca49ee3a5fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e97308e8274c4a35b0359bfcfea84b93'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:27.389410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d9b7e1f-b39a-428b-b155-ca49ee3a5fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e97308e8274c4a35b0359bfcfea84b93'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:27.391226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e55262-99e2-4544-a4e3-ad7742d52595 tempest-EndPointGroupsTest-1970745959 tempest-EndPointGroupsTest-1970745959-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:27.397992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:27.399943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:27.400482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:27.400785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1 admin admin] PATH_INFO: `/v3/projects/58e50b6481544a30bce4e39f80dd6eae` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:27.401792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d9b7e1f-b39a-428b-b155-ca49ee3a5fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e97308e8274c4a35b0359bfcfea84b93'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:27.402729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58e50b6481544a30bce4e39f80dd6eae)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:27.410055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:27.418285 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1933/3926] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:27 2022] DELETE /identity/v3/users/e97308e8274c4a35b0359bfcfea84b93 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:27.422097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd72ea2-967e-4383-9870-c702c141e3c8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:27.422394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd72ea2-967e-4383-9870-c702c141e3c8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:27.422683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd72ea2-967e-4383-9870-c702c141e3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:27.448582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:UPDATE=1,keystone:DELETE=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:27.490099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8fd72ea2-967e-4383-9870-c702c141e3c8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:27.491364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8fd72ea2-967e-4383-9870-c702c141e3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:27.540368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58e50b6481544a30bce4e39f80dd6eae', 'request_id': 'req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:27.551815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58e50b6481544a30bce4e39f80dd6eae', 'request_id': 'req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:27.572428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58e50b6481544a30bce4e39f80dd6eae', 'request_id': 'req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:27.620566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1 admin admin] The token cache is being invalidate because project 58e50b6481544a30bce4e39f80dd6eae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:27.620963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:27.626379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6934d932-5ccc-4cb9-a9e5-fae9afbda5c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:27.631986 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1994/3927] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:27 2022] DELETE /identity/v3/projects/58e50b6481544a30bce4e39f80dd6eae => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:27.636909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:27.638122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80200571-7eca-457c-8e25-cde459d9c33c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:27.638701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80200571-7eca-457c-8e25-cde459d9c33c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:27.639190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80200571-7eca-457c-8e25-cde459d9c33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:27.664614 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1934/3928] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:27.675963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-80200571-7eca-457c-8e25-cde459d9c33c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:27.678726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-80200571-7eca-457c-8e25-cde459d9c33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:27.700275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd72ea2-967e-4383-9870-c702c141e3c8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:27.702811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8fd72ea2-967e-4383-9870-c702c141e3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:27.765604 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1995/3929] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:27.782947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779abd2-0d84-473f-aeb6-e5c960496f22 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:27.784824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779abd2-0d84-473f-aeb6-e5c960496f22 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:27.785226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779abd2-0d84-473f-aeb6-e5c960496f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:27.810376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2779abd2-0d84-473f-aeb6-e5c960496f22 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:27.811615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2779abd2-0d84-473f-aeb6-e5c960496f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:27.829001 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1996/3930] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:27.833535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779abd2-0d84-473f-aeb6-e5c960496f22 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:27.839530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd72ea2-967e-4383-9870-c702c141e3c8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:27.850604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779abd2-0d84-473f-aeb6-e5c960496f22 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:27.857210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cf3f9-cbe3-438a-9a76-1102432964a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:27.858064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cf3f9-cbe3-438a-9a76-1102432964a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:27.858701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cf3f9-cbe3-438a-9a76-1102432964a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:27.859302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cf3f9-cbe3-438a-9a76-1102432964a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:27.860149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cf3f9-cbe3-438a-9a76-1102432964a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:27.860774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cf3f9-cbe3-438a-9a76-1102432964a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:27.862719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cf3f9-cbe3-438a-9a76-1102432964a6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:27.879140 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1997/3931] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:27.893404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cf3f9-cbe3-438a-9a76-1102432964a6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:27.906656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cf3f9-cbe3-438a-9a76-1102432964a6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:27.911386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba93f2-b92d-4aaa-b176-6fbcf42a816f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:27.911999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba93f2-b92d-4aaa-b176-6fbcf42a816f service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:27.912275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba93f2-b92d-4aaa-b176-6fbcf42a816f service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:27.912550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba93f2-b92d-4aaa-b176-6fbcf42a816f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:27.913117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9d1dd-31d7-488d-a8ff-3244bc68b9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:27.913681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9d1dd-31d7-488d-a8ff-3244bc68b9df admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:27.913978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9d1dd-31d7-488d-a8ff-3244bc68b9df admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:27.914358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9d1dd-31d7-488d-a8ff-3244bc68b9df admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:27.914936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd9d1dd-31d7-488d-a8ff-3244bc68b9df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:27.918298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd9d1dd-31d7-488d-a8ff-3244bc68b9df admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:27.923239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba93f2-b92d-4aaa-b176-6fbcf42a816f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:27.925330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba93f2-b92d-4aaa-b176-6fbcf42a816f service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:27.937526 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1935/3932] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:27.956944 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1998/3933] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:27 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:27.958495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba93f2-b92d-4aaa-b176-6fbcf42a816f service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:27.968524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba93f2-b92d-4aaa-b176-6fbcf42a816f service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:28.029970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62387e4-5590-4ddb-bc58-d867c474f6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:28.030584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62387e4-5590-4ddb-bc58-d867c474f6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:28.030895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62387e4-5590-4ddb-bc58-d867c474f6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:28.031170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62387e4-5590-4ddb-bc58-d867c474f6f7 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:28.031830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62387e4-5590-4ddb-bc58-d867c474f6f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:28.033755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62387e4-5590-4ddb-bc58-d867c474f6f7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:28.055557 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1936/3934] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:27 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:28.060268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9d1dd-31d7-488d-a8ff-3244bc68b9df admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:28.071666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9d1dd-31d7-488d-a8ff-3244bc68b9df admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:28.076673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f500c7-5117-440a-b931-1e25e6913d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:28.077446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f500c7-5117-440a-b931-1e25e6913d4a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:28.078110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f500c7-5117-440a-b931-1e25e6913d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:28.078390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f500c7-5117-440a-b931-1e25e6913d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:28.079070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f500c7-5117-440a-b931-1e25e6913d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:28.081012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f500c7-5117-440a-b931-1e25e6913d4a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:28.095984 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 1999/3935] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:02:28.097618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62387e4-5590-4ddb-bc58-d867c474f6f7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:28.112182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62387e4-5590-4ddb-bc58-d867c474f6f7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:28.117260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5846d-7277-4bb6-a132-fd4d33652255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:28.117917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5846d-7277-4bb6-a132-fd4d33652255 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:28.118236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5846d-7277-4bb6-a132-fd4d33652255 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:28.118533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5846d-7277-4bb6-a132-fd4d33652255 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:28.119213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5846d-7277-4bb6-a132-fd4d33652255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:28.121109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5846d-7277-4bb6-a132-fd4d33652255 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:28.155484 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1937/3936] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:02:28.160489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f500c7-5117-440a-b931-1e25e6913d4a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:28.180955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f500c7-5117-440a-b931-1e25e6913d4a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:28.184059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b19936-ccab-461e-abd5-20c8167acb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:28.184851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b19936-ccab-461e-abd5-20c8167acb9f admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:28.185635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b19936-ccab-461e-abd5-20c8167acb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:28.186014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b19936-ccab-461e-abd5-20c8167acb9f admin admin] PATH_INFO: `/v3/projects/fde6874e10ae4e12ad874c65942f9d59/users/c3187514ca644ff68c992526f38783c7/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:28.206175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b19936-ccab-461e-abd5-20c8167acb9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde6874e10ae4e12ad874c65942f9d59, user_id=c3187514ca644ff68c992526f38783c7, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:28.209042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b19936-ccab-461e-abd5-20c8167acb9f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:28.224695 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2000/3937] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:28 2022] PUT /identity/v3/projects/fde6874e10ae4e12ad874c65942f9d59/users/c3187514ca644ff68c992526f38783c7/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:28.230255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b0e92-b398-495b-b233-e9c20045e2e9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:28.230791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b0e92-b398-495b-b233-e9c20045e2e9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:28.231078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b0e92-b398-495b-b233-e9c20045e2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:28.251155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d77b0e92-b398-495b-b233-e9c20045e2e9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:28.252135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d77b0e92-b398-495b-b233-e9c20045e2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:28.277762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:28.297696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b19936-ccab-461e-abd5-20c8167acb9f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:28.299513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-95b19936-ccab-461e-abd5-20c8167acb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:28.327433 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1938/3938] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:02:28.330911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:28.336447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b0e92-b398-495b-b233-e9c20045e2e9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:28.337792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d77b0e92-b398-495b-b233-e9c20045e2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:28.376457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b19936-ccab-461e-abd5-20c8167acb9f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:28.381739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:28.382487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:28.382849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:28.383203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] PATH_INFO: `/v3/projects/40b96e7c9c8f4e6f9e165819a7c06b5e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:28.384617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40b96e7c9c8f4e6f9e165819a7c06b5e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:28.386833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:28.400470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b0e92-b398-495b-b233-e9c20045e2e9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:28.405032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fecd6-f548-469c-8e12-6173798592e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:28.405796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fecd6-f548-469c-8e12-6173798592e3 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:28.406718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fecd6-f548-469c-8e12-6173798592e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:28.407013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fecd6-f548-469c-8e12-6173798592e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:28.418173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0fecd6-f548-469c-8e12-6173798592e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:28.419747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0fecd6-f548-469c-8e12-6173798592e3 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:28.430978 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1939/3939] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:02:28.439320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40b96e7c9c8f4e6f9e165819a7c06b5e', 'request_id': 'req-a235071c-bc00-4aff-bc31-421e92590e04'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:28.447123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40b96e7c9c8f4e6f9e165819a7c06b5e', 'request_id': 'req-a235071c-bc00-4aff-bc31-421e92590e04'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:28.453703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40b96e7c9c8f4e6f9e165819a7c06b5e', 'request_id': 'req-a235071c-bc00-4aff-bc31-421e92590e04'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:28.466362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] The token cache is being invalidate because project 40b96e7c9c8f4e6f9e165819a7c06b5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:28.466821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:28.467554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:28.469208 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2001/3940] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:28 2022] DELETE /identity/v3/projects/40b96e7c9c8f4e6f9e165819a7c06b5e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:28.473434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fecd6-f548-469c-8e12-6173798592e3 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:28.485390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fecd6-f548-469c-8e12-6173798592e3 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:28.544045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72810ed4-50c1-4d96-a2cb-7bc0652b4555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:28.544686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72810ed4-50c1-4d96-a2cb-7bc0652b4555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:28.544995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72810ed4-50c1-4d96-a2cb-7bc0652b4555 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:28.545273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72810ed4-50c1-4d96-a2cb-7bc0652b4555 admin admin] PATH_INFO: `/v3/users/3dc61cae01c44ef2b8be0dfe8fe8ec5b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:28.546412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72810ed4-50c1-4d96-a2cb-7bc0652b4555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dc61cae01c44ef2b8be0dfe8fe8ec5b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:28.548401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72810ed4-50c1-4d96-a2cb-7bc0652b4555 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:28.609272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-72810ed4-50c1-4d96-a2cb-7bc0652b4555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dc61cae01c44ef2b8be0dfe8fe8ec5b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:28.624339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-72810ed4-50c1-4d96-a2cb-7bc0652b4555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dc61cae01c44ef2b8be0dfe8fe8ec5b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:28.625199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-72810ed4-50c1-4d96-a2cb-7bc0652b4555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dc61cae01c44ef2b8be0dfe8fe8ec5b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:28.633542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-72810ed4-50c1-4d96-a2cb-7bc0652b4555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dc61cae01c44ef2b8be0dfe8fe8ec5b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:28.635488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:28.644142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-72810ed4-50c1-4d96-a2cb-7bc0652b4555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dc61cae01c44ef2b8be0dfe8fe8ec5b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:28.654037 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1940/3941] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:28 2022] DELETE /identity/v3/users/3dc61cae01c44ef2b8be0dfe8fe8ec5b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:29.328556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:29.330244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:29.345842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235071c-bc00-4aff-bc31-421e92590e04 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:29.410636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:29.411318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:29.411529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:29.411815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d admin admin] PATH_INFO: `/v3/projects/96311ac1a0b849a48e4e123e784f8e36` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:29.412982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d admin admin] RBAC: Authorizing `identity:delete_project(project_id=96311ac1a0b849a48e4e123e784f8e36)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:29.414925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:29.463502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96311ac1a0b849a48e4e123e784f8e36', 'request_id': 'req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:29.468486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:29.477872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96311ac1a0b849a48e4e123e784f8e36', 'request_id': 'req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:29.485988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96311ac1a0b849a48e4e123e784f8e36', 'request_id': 'req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:29.497607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d admin admin] The token cache is being invalidate because project 96311ac1a0b849a48e4e123e784f8e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:29.497898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:29.498569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-acad6ae4-bacf-4d1b-bb2b-6c7c8d22010d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:29.500254 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2002/3942] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:29 2022] DELETE /identity/v3/projects/96311ac1a0b849a48e4e123e784f8e36 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:29.504936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c4538-f0fe-45aa-bae0-c16fc996f1d8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:29.505427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c4538-f0fe-45aa-bae0-c16fc996f1d8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:29.505680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c4538-f0fe-45aa-bae0-c16fc996f1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:29.525790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1d1c4538-f0fe-45aa-bae0-c16fc996f1d8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:29.527097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1d1c4538-f0fe-45aa-bae0-c16fc996f1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:29.594944 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1941/3943] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:29.596689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8423d-71ce-45bc-8a06-facc7b17ddc3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:29.597013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8423d-71ce-45bc-8a06-facc7b17ddc3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:29.597298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8423d-71ce-45bc-8a06-facc7b17ddc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:29.630065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-96e8423d-71ce-45bc-8a06-facc7b17ddc3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:29.631311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-96e8423d-71ce-45bc-8a06-facc7b17ddc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:29.640671 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2003/3944] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:29.645216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c4538-f0fe-45aa-bae0-c16fc996f1d8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:29.655480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c4538-f0fe-45aa-bae0-c16fc996f1d8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:29.660462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20932e-c86f-4f59-99e3-700837d8544e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:29.661198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20932e-c86f-4f59-99e3-700837d8544e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:29.661429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20932e-c86f-4f59-99e3-700837d8544e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:29.661710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20932e-c86f-4f59-99e3-700837d8544e admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:29.662448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20932e-c86f-4f59-99e3-700837d8544e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:29.662790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20932e-c86f-4f59-99e3-700837d8544e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:29.664401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20932e-c86f-4f59-99e3-700837d8544e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:29.683241 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1942/3945] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:29.686792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8423d-71ce-45bc-8a06-facc7b17ddc3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:29.696546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8423d-71ce-45bc-8a06-facc7b17ddc3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:29.701274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac9599-ac75-4be3-a66a-1d4dd87c3c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:29.701897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac9599-ac75-4be3-a66a-1d4dd87c3c2f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:29.702238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac9599-ac75-4be3-a66a-1d4dd87c3c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:29.702511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac9599-ac75-4be3-a66a-1d4dd87c3c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:29.703151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac9599-ac75-4be3-a66a-1d4dd87c3c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:29.705160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac9599-ac75-4be3-a66a-1d4dd87c3c2f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:29.743103 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2004/3946] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:29 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:29.744007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20932e-c86f-4f59-99e3-700837d8544e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:29.761002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20932e-c86f-4f59-99e3-700837d8544e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:29.861735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162eb90e-fd65-46bb-b43a-7f0f229b9bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:29.863768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162eb90e-fd65-46bb-b43a-7f0f229b9bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:29.864360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162eb90e-fd65-46bb-b43a-7f0f229b9bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:29.864877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162eb90e-fd65-46bb-b43a-7f0f229b9bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:29.865824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162eb90e-fd65-46bb-b43a-7f0f229b9bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:29.867938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162eb90e-fd65-46bb-b43a-7f0f229b9bf0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:29.894842 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1943/3947] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:29 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:29.897719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac9599-ac75-4be3-a66a-1d4dd87c3c2f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:29.911895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac9599-ac75-4be3-a66a-1d4dd87c3c2f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:29.920339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7999283-988d-4434-8304-6b9928560eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:29.920995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7999283-988d-4434-8304-6b9928560eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:29.921716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7999283-988d-4434-8304-6b9928560eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:29.922494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7999283-988d-4434-8304-6b9928560eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:29.924183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7999283-988d-4434-8304-6b9928560eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:29.933084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7999283-988d-4434-8304-6b9928560eb6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:29.961228 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2005/3948] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:29.965789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162eb90e-fd65-46bb-b43a-7f0f229b9bf0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:29.977052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162eb90e-fd65-46bb-b43a-7f0f229b9bf0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:29.982773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c326885-d9a5-4c81-901c-9563b2ee5e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:29.983393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c326885-d9a5-4c81-901c-9563b2ee5e7c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:29.983680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c326885-d9a5-4c81-901c-9563b2ee5e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:29.984487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c326885-d9a5-4c81-901c-9563b2ee5e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:29.985179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c326885-d9a5-4c81-901c-9563b2ee5e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:29.987057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c326885-d9a5-4c81-901c-9563b2ee5e7c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:30.006358 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1944/3949] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:30.009691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7999283-988d-4434-8304-6b9928560eb6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:30.022890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7999283-988d-4434-8304-6b9928560eb6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:30.034281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cda8cf-19bc-491c-b49e-f744c619bfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:30.035183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cda8cf-19bc-491c-b49e-f744c619bfcd admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:30.036185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cda8cf-19bc-491c-b49e-f744c619bfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:30.036666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cda8cf-19bc-491c-b49e-f744c619bfcd admin admin] PATH_INFO: `/v3/projects/a9910f909654434e8a6b6997b2df88ba/users/e2ff212e966948d68fa104e937a0be41/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:30.058316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cda8cf-19bc-491c-b49e-f744c619bfcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9910f909654434e8a6b6997b2df88ba, user_id=e2ff212e966948d68fa104e937a0be41, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:30.062057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cda8cf-19bc-491c-b49e-f744c619bfcd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:30.080369 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2006/3950] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:30 2022] PUT /identity/v3/projects/a9910f909654434e8a6b6997b2df88ba/users/e2ff212e966948d68fa104e937a0be41/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:30.092001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d0f41-44f1-491a-9d0c-e43e11f20bab None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:30.092377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d0f41-44f1-491a-9d0c-e43e11f20bab None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:30.092774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d0f41-44f1-491a-9d0c-e43e11f20bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:30.159319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-989d0f41-44f1-491a-9d0c-e43e11f20bab None None] MFA Rules not processed for user `e2ff212e966948d68fa104e937a0be41`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:30.160610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-989d0f41-44f1-491a-9d0c-e43e11f20bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:30.219648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:30.247045 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1945/3951] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:30.248194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cda8cf-19bc-491c-b49e-f744c619bfcd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:30.352691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cda8cf-19bc-491c-b49e-f744c619bfcd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:30.359088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995afbd-59bc-43cd-9eb6-2212a0aace77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:30.359787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995afbd-59bc-43cd-9eb6-2212a0aace77 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:30.360144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995afbd-59bc-43cd-9eb6-2212a0aace77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:30.360491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995afbd-59bc-43cd-9eb6-2212a0aace77 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:30.361245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995afbd-59bc-43cd-9eb6-2212a0aace77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:30.363428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995afbd-59bc-43cd-9eb6-2212a0aace77 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:30.408629 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2007/3952] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:30 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:30.411122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d0f41-44f1-491a-9d0c-e43e11f20bab None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:30.425299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d0f41-44f1-491a-9d0c-e43e11f20bab None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:30.497596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae0b23-64d4-4e5c-a6b5-caae61667908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:30.498332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae0b23-64d4-4e5c-a6b5-caae61667908 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:30.498724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae0b23-64d4-4e5c-a6b5-caae61667908 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:30.499188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae0b23-64d4-4e5c-a6b5-caae61667908 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:30.499895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae0b23-64d4-4e5c-a6b5-caae61667908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:30.502102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae0b23-64d4-4e5c-a6b5-caae61667908 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:30.531990 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1946/3953] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:30 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:30.535945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995afbd-59bc-43cd-9eb6-2212a0aace77 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:30.560316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995afbd-59bc-43cd-9eb6-2212a0aace77 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:30.566095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6c465-c4f2-4faa-8f9e-dfb629368c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:30.567150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6c465-c4f2-4faa-8f9e-dfb629368c91 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:30.567150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6c465-c4f2-4faa-8f9e-dfb629368c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:30.567937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6c465-c4f2-4faa-8f9e-dfb629368c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:30.568299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6c465-c4f2-4faa-8f9e-dfb629368c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:30.571417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6c465-c4f2-4faa-8f9e-dfb629368c91 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:30.583609 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2008/3954] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:02:30.588292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae0b23-64d4-4e5c-a6b5-caae61667908 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:30.604628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae0b23-64d4-4e5c-a6b5-caae61667908 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:30.614019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b267338-3389-401c-ba84-9006620e77ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:30.614763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b267338-3389-401c-ba84-9006620e77ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:30.615054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b267338-3389-401c-ba84-9006620e77ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:30.616310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b267338-3389-401c-ba84-9006620e77ee admin admin] PATH_INFO: `/v3/projects/ed4c98ad20104dc0bee099240ee1e79a/users/f506a640b1d34910ad6bd049bf89debe/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:30.639202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b267338-3389-401c-ba84-9006620e77ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4c98ad20104dc0bee099240ee1e79a, user_id=f506a640b1d34910ad6bd049bf89debe, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:30.642643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b267338-3389-401c-ba84-9006620e77ee admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:30.664589 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1947/3955] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:30 2022] PUT /identity/v3/projects/ed4c98ad20104dc0bee099240ee1e79a/users/f506a640b1d34910ad6bd049bf89debe/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:30.667156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6c465-c4f2-4faa-8f9e-dfb629368c91 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:30.683594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6c465-c4f2-4faa-8f9e-dfb629368c91 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:30.782927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:30.801707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0f430-9fd0-4258-8961-f16ca7ca1834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:30.802360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0f430-9fd0-4258-8961-f16ca7ca1834 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:30.802644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0f430-9fd0-4258-8961-f16ca7ca1834 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:30.803400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0f430-9fd0-4258-8961-f16ca7ca1834 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:30.804056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0f430-9fd0-4258-8961-f16ca7ca1834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:30.806137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0f430-9fd0-4258-8961-f16ca7ca1834 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:30.820330 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2009/3956] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:02:30.824973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b267338-3389-401c-ba84-9006620e77ee admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:30.842552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b267338-3389-401c-ba84-9006620e77ee admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:30.848221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7830b0-0d53-435f-9cd1-c054aceb47c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:30.849084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7830b0-0d53-435f-9cd1-c054aceb47c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:30.849508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7830b0-0d53-435f-9cd1-c054aceb47c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:30.850058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7830b0-0d53-435f-9cd1-c054aceb47c0 admin admin] PATH_INFO: `/v3/projects/ed4c98ad20104dc0bee099240ee1e79a/users/f506a640b1d34910ad6bd049bf89debe/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:30.852730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7830b0-0d53-435f-9cd1-c054aceb47c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4c98ad20104dc0bee099240ee1e79a, user_id=f506a640b1d34910ad6bd049bf89debe, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:30.857824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7830b0-0d53-435f-9cd1-c054aceb47c0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:30.876998 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1948/3957] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:30 2022] PUT /identity/v3/projects/ed4c98ad20104dc0bee099240ee1e79a/users/f506a640b1d34910ad6bd049bf89debe/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:30.888594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9c449-3f3e-4f95-bbc1-99c4fd087ea4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:30.888882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9c449-3f3e-4f95-bbc1-99c4fd087ea4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:30.889234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9c449-3f3e-4f95-bbc1-99c4fd087ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:30.932576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0da9c449-3f3e-4f95-bbc1-99c4fd087ea4 None None] MFA Rules not processed for user `f506a640b1d34910ad6bd049bf89debe`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:30.935112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0da9c449-3f3e-4f95-bbc1-99c4fd087ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:31.052554 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2010/3958] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:31.058111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7830b0-0d53-435f-9cd1-c054aceb47c0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:31.074245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7830b0-0d53-435f-9cd1-c054aceb47c0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:31.081512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e628c-f99c-42b9-a73f-00d7a374dceb tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:31.082719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e628c-f99c-42b9-a73f-00d7a374dceb tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:31.083025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e628c-f99c-42b9-a73f-00d7a374dceb tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:31.083314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e628c-f99c-42b9-a73f-00d7a374dceb tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:31.083991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e628c-f99c-42b9-a73f-00d7a374dceb tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:31.086068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e628c-f99c-42b9-a73f-00d7a374dceb tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:31.172494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9c449-3f3e-4f95-bbc1-99c4fd087ea4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:31.173205 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1949/3959] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:31 2022] POST /identity/v3/services => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:31.185833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9c449-3f3e-4f95-bbc1-99c4fd087ea4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:31.226917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1711253-a0f8-48b4-b728-d605aef94969 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:31.227668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1711253-a0f8-48b4-b728-d605aef94969 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:31.228089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1711253-a0f8-48b4-b728-d605aef94969 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:31.228363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1711253-a0f8-48b4-b728-d605aef94969 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:31.229098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1711253-a0f8-48b4-b728-d605aef94969 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:31.231064 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-b1711253-a0f8-48b4-b728-d605aef94969 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:31.231064 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:31.231064 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:31.231795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1711253-a0f8-48b4-b728-d605aef94969 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-b1711253-a0f8-48b4-b728-d605aef94969 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: On instance['enabled']: Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: On instance['enabled']: Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: 'False' Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise error Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application On instance['enabled']: Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application 'False' Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.254889 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 03 21:02:31.256773 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 03 21:02:31.257983 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 03 21:02:31.257983 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 03 21:02:31.257983 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.257983 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:31.257983 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:31.257983 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.257983 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application On instance['enabled']: Sep 03 21:02:31.257983 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application 'False' Sep 03 21:02:31.257983 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.257983 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2011/3960] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Sep 3 21:02:31 2022] POST /identity/v3/endpoints => generated 283 bytes in 84 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 03 21:02:31.263030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e628c-f99c-42b9-a73f-00d7a374dceb tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:31.287224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e628c-f99c-42b9-a73f-00d7a374dceb tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:31.300563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ad701-8b2e-4ced-939e-3b6a596377b5 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:31.301095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ad701-8b2e-4ced-939e-3b6a596377b5 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:31.301385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ad701-8b2e-4ced-939e-3b6a596377b5 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:31.301667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ad701-8b2e-4ced-939e-3b6a596377b5 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:31.302351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ad701-8b2e-4ced-939e-3b6a596377b5 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:31.303380 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-996ad701-8b2e-4ced-939e-3b6a596377b5 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:31.303380 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:31.303380 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:31.304276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ad701-8b2e-4ced-939e-3b6a596377b5 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-996ad701-8b2e-4ced-939e-3b6a596377b5 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: On instance['enabled']: Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: On instance['enabled']: Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: 'True' Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise error Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application On instance['enabled']: Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application 'True' Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:31.333833 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 03 21:02:31.335433 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 03 21:02:31.336519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 03 21:02:31.336519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 03 21:02:31.336519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:31.336519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:31.336519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:31.336519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:31.336519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application On instance['enabled']: Sep 03 21:02:31.336519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application 'True' Sep 03 21:02:31.336519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:31.336519 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1950/3961] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Sep 3 21:02:31 2022] POST /identity/v3/endpoints => generated 281 bytes in 73 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 03 21:02:31.342575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1711253-a0f8-48b4-b728-d605aef94969 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:31.363705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1711253-a0f8-48b4-b728-d605aef94969 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:31.380723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2d0b1-03d3-46e9-8ff9-5608fb481e85 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:31.381308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2d0b1-03d3-46e9-8ff9-5608fb481e85 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:31.381599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2d0b1-03d3-46e9-8ff9-5608fb481e85 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:31.381883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2d0b1-03d3-46e9-8ff9-5608fb481e85 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:31.382568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2d0b1-03d3-46e9-8ff9-5608fb481e85 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:31.384407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2d0b1-03d3-46e9-8ff9-5608fb481e85 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:31.459392 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-ccf2d0b1-03d3-46e9-8ff9-5608fb481e85 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1842475425. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:02:31.459392 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:02:31.459392 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:31.525393 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2012/3962] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Sep 3 21:02:31 2022] POST /identity/v3/endpoints => generated 361 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:31.530601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ad701-8b2e-4ced-939e-3b6a596377b5 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:31.545169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ad701-8b2e-4ced-939e-3b6a596377b5 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:31.562839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581fcd3-bf17-444b-bb9e-13a868fffafc tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:31.563684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581fcd3-bf17-444b-bb9e-13a868fffafc tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:31.563975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581fcd3-bf17-444b-bb9e-13a868fffafc tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:31.564253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581fcd3-bf17-444b-bb9e-13a868fffafc tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/regions/tempest-region-1842475425` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:31.564841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581fcd3-bf17-444b-bb9e-13a868fffafc tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1842475425)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:31.566050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581fcd3-bf17-444b-bb9e-13a868fffafc tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:31.568268 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1951/3963] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Sep 3 21:02:31 2022] GET /identity/v3/regions/tempest-region-1842475425 => generated 180 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:31.572364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2d0b1-03d3-46e9-8ff9-5608fb481e85 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:31.589468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2d0b1-03d3-46e9-8ff9-5608fb481e85 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:31.595398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99257321-3e36-49c6-be08-e5a56857e3f6 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:31.596049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99257321-3e36-49c6-be08-e5a56857e3f6 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:31.596768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99257321-3e36-49c6-be08-e5a56857e3f6 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:31.598512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99257321-3e36-49c6-be08-e5a56857e3f6 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/endpoints/5f5769eebac245ab881e700265eec816` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:31.609801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99257321-3e36-49c6-be08-e5a56857e3f6 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=5f5769eebac245ab881e700265eec816)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:31.612810 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-99257321-3e36-49c6-be08-e5a56857e3f6 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:31.612810 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:31.612810 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:31.613977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99257321-3e36-49c6-be08-e5a56857e3f6 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-99257321-3e36-49c6-be08-e5a56857e3f6 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: On instance['enabled']: Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: On instance['enabled']: Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: 'False' Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise error Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application On instance['enabled']: Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application 'False' Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.622526 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 03 21:02:31.624162 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 03 21:02:31.625266 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 03 21:02:31.625266 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 03 21:02:31.625266 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.625266 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:31.625266 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:31.625266 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.625266 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application On instance['enabled']: Sep 03 21:02:31.625266 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application 'False' Sep 03 21:02:31.625266 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:31.625266 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2013/3964] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Sep 3 21:02:31 2022] PATCH /identity/v3/endpoints/5f5769eebac245ab881e700265eec816 => generated 283 bytes in 52 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 03 21:02:31.633974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581fcd3-bf17-444b-bb9e-13a868fffafc tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:31.645978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581fcd3-bf17-444b-bb9e-13a868fffafc tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:31.651144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5945d7-23f4-4b38-a7c1-0c1c92bfb569 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:31.652824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5945d7-23f4-4b38-a7c1-0c1c92bfb569 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:31.653149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5945d7-23f4-4b38-a7c1-0c1c92bfb569 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:31.653473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5945d7-23f4-4b38-a7c1-0c1c92bfb569 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/endpoints/5f5769eebac245ab881e700265eec816` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:31.655356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5945d7-23f4-4b38-a7c1-0c1c92bfb569 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=5f5769eebac245ab881e700265eec816)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:31.656444 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-db5945d7-23f4-4b38-a7c1-0c1c92bfb569 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:31.656444 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:31.656444 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:31.657383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5945d7-23f4-4b38-a7c1-0c1c92bfb569 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:31.670111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db5945d7-23f4-4b38-a7c1-0c1c92bfb569 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '5f5769eebac245ab881e700265eec816', 'request_id': 'req-db5945d7-23f4-4b38-a7c1-0c1c92bfb569'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:31.688487 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1952/3965] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Sep 3 21:02:31 2022] DELETE /identity/v3/endpoints/5f5769eebac245ab881e700265eec816 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:31.688998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99257321-3e36-49c6-be08-e5a56857e3f6 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:31.702843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99257321-3e36-49c6-be08-e5a56857e3f6 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:31.726020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03147bbb-2701-47f9-b533-5486e678081f tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:31.726576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03147bbb-2701-47f9-b533-5486e678081f tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:31.726872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03147bbb-2701-47f9-b533-5486e678081f tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:31.727153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03147bbb-2701-47f9-b533-5486e678081f tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/regions/tempest-region-1842475425` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:31.727741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03147bbb-2701-47f9-b533-5486e678081f tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1842475425)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:31.728758 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-03147bbb-2701-47f9-b533-5486e678081f tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:31.728758 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:31.728758 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:31.730327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03147bbb-2701-47f9-b533-5486e678081f tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:31.776887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5945d7-23f4-4b38-a7c1-0c1c92bfb569 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:31.778003 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2014/3966] 10.222.0.16 () {58 vars in 1209 bytes} [Sat Sep 3 21:02:31 2022] DELETE /identity/v3/regions/tempest-region-1842475425 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:31.795711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5945d7-23f4-4b38-a7c1-0c1c92bfb569 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:31.857833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664760e4-4078-405c-a959-5eb7aafc2b3b tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:31.858462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664760e4-4078-405c-a959-5eb7aafc2b3b tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:31.858789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664760e4-4078-405c-a959-5eb7aafc2b3b tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:31.859111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664760e4-4078-405c-a959-5eb7aafc2b3b tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:31.860843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664760e4-4078-405c-a959-5eb7aafc2b3b tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:31.863332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664760e4-4078-405c-a959-5eb7aafc2b3b tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:31.890470 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-664760e4-4078-405c-a959-5eb7aafc2b3b tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1063145997. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:02:31.890470 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:02:31.890470 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:31.963481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03147bbb-2701-47f9-b533-5486e678081f tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:31.977702 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1953/3967] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Sep 3 21:02:31 2022] POST /identity/v3/endpoints => generated 360 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:31.979081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03147bbb-2701-47f9-b533-5486e678081f tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:32.000087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdfbf2-7f02-404b-ba36-e0e3ef9c5fea tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:32.000881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdfbf2-7f02-404b-ba36-e0e3ef9c5fea tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:32.001200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdfbf2-7f02-404b-ba36-e0e3ef9c5fea tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:32.001478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdfbf2-7f02-404b-ba36-e0e3ef9c5fea tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/regions/tempest-region-1063145997` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:32.002162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdfbf2-7f02-404b-ba36-e0e3ef9c5fea tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1063145997)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:32.003256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdfbf2-7f02-404b-ba36-e0e3ef9c5fea tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:32.011675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664760e4-4078-405c-a959-5eb7aafc2b3b tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:32.019438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:INSERT=8,keystone:DELETE=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:32.024126 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2015/3968] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Sep 3 21:02:31 2022] GET /identity/v3/regions/tempest-region-1063145997 => generated 180 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:32.031131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664760e4-4078-405c-a959-5eb7aafc2b3b tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:32.037153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293b131-3beb-4814-bdce-b38cf0e4e106 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:32.037821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293b131-3beb-4814-bdce-b38cf0e4e106 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:32.038900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293b131-3beb-4814-bdce-b38cf0e4e106 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:32.039128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293b131-3beb-4814-bdce-b38cf0e4e106 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/endpoints/8312aadda9af4592ab66ff23473ca274` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:32.045861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293b131-3beb-4814-bdce-b38cf0e4e106 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=8312aadda9af4592ab66ff23473ca274)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:32.047402 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-2293b131-3beb-4814-bdce-b38cf0e4e106 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:32.047402 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:32.047402 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:32.048514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293b131-3beb-4814-bdce-b38cf0e4e106 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-2293b131-3beb-4814-bdce-b38cf0e4e106 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: On instance['enabled']: Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: On instance['enabled']: Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: 'True' Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise error Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application On instance['enabled']: Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application 'True' Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:32.057025 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 03 21:02:32.059256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 03 21:02:32.060305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 03 21:02:32.060305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 03 21:02:32.060305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:32.060305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 03 21:02:32.060305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 03 21:02:32.060305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:32.060305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application On instance['enabled']: Sep 03 21:02:32.060305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application 'True' Sep 03 21:02:32.060305 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:32.060305 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1954/3969] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Sep 3 21:02:32 2022] PATCH /identity/v3/endpoints/8312aadda9af4592ab66ff23473ca274 => generated 281 bytes in 48 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 03 21:02:32.063542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdfbf2-7f02-404b-ba36-e0e3ef9c5fea tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:32.105079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdfbf2-7f02-404b-ba36-e0e3ef9c5fea tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:32.123457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93391f22-37bb-48f3-a2db-76844b7437d7 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:32.124286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93391f22-37bb-48f3-a2db-76844b7437d7 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:32.124703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93391f22-37bb-48f3-a2db-76844b7437d7 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:32.125171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93391f22-37bb-48f3-a2db-76844b7437d7 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/endpoints/8312aadda9af4592ab66ff23473ca274` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:32.132728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93391f22-37bb-48f3-a2db-76844b7437d7 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=8312aadda9af4592ab66ff23473ca274)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:32.132728 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-93391f22-37bb-48f3-a2db-76844b7437d7 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:32.132728 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:32.132728 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:32.132728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93391f22-37bb-48f3-a2db-76844b7437d7 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:32.147177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93391f22-37bb-48f3-a2db-76844b7437d7 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '8312aadda9af4592ab66ff23473ca274', 'request_id': 'req-93391f22-37bb-48f3-a2db-76844b7437d7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:32.180672 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2016/3970] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Sep 3 21:02:32 2022] DELETE /identity/v3/endpoints/8312aadda9af4592ab66ff23473ca274 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:32.185196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293b131-3beb-4814-bdce-b38cf0e4e106 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:32.200689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293b131-3beb-4814-bdce-b38cf0e4e106 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:32.240798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e1248-9b69-48bf-8147-5ced659d4762 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:32.242422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e1248-9b69-48bf-8147-5ced659d4762 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:32.242715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e1248-9b69-48bf-8147-5ced659d4762 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:32.242996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e1248-9b69-48bf-8147-5ced659d4762 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/regions/tempest-region-1063145997` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:32.247487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e1248-9b69-48bf-8147-5ced659d4762 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1063145997)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:32.248537 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-425e1248-9b69-48bf-8147-5ced659d4762 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:32.248537 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:32.248537 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:32.249498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e1248-9b69-48bf-8147-5ced659d4762 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:32.291388 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1955/3971] 10.222.0.16 () {58 vars in 1209 bytes} [Sat Sep 3 21:02:32 2022] DELETE /identity/v3/regions/tempest-region-1063145997 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:32.297218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93391f22-37bb-48f3-a2db-76844b7437d7 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:32.310037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93391f22-37bb-48f3-a2db-76844b7437d7 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:32.318440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:32.332952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac66da-0396-4adc-bcd6-92d7299fede2 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f506a640b1d34910ad6bd049bf89debe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed4c98ad20104dc0bee099240ee1e79a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:32.334220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac66da-0396-4adc-bcd6-92d7299fede2 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:32.334822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac66da-0396-4adc-bcd6-92d7299fede2 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:32.335326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac66da-0396-4adc-bcd6-92d7299fede2 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] PATH_INFO: `/v3/services/59c18bf94e72429fb2fc51cb949cc669` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:32.336145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac66da-0396-4adc-bcd6-92d7299fede2 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorizing `identity:delete_service(service_id=59c18bf94e72429fb2fc51cb949cc669)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:32.338063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac66da-0396-4adc-bcd6-92d7299fede2 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:32.374238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e1248-9b69-48bf-8147-5ced659d4762 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:32.374997 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2017/3972] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:32 2022] DELETE /identity/v3/services/59c18bf94e72429fb2fc51cb949cc669 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:32.580380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e1248-9b69-48bf-8147-5ced659d4762 tempest-EndpointsNegativeTestJSON-246791775 tempest-EndpointsNegativeTestJSON-246791775-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:32.619149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:32.619845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:32.620237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:32.620516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] PATH_INFO: `/v3/users/e2ff212e966948d68fa104e937a0be41` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:32.621903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ff212e966948d68fa104e937a0be41)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:32.625108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:32.905373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ff212e966948d68fa104e937a0be41'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:32.923684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ff212e966948d68fa104e937a0be41'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:32.924873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ff212e966948d68fa104e937a0be41'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:32.938586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ff212e966948d68fa104e937a0be41'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:32.950815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ff212e966948d68fa104e937a0be41'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:32.967048 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1956/3973] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:32 2022] DELETE /identity/v3/users/e2ff212e966948d68fa104e937a0be41 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:32.970662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cb219-c028-4644-9457-3c4df8e8352d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:32.971862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cb219-c028-4644-9457-3c4df8e8352d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:32.972418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cb219-c028-4644-9457-3c4df8e8352d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:32.996776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f11cb219-c028-4644-9457-3c4df8e8352d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:32.998470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f11cb219-c028-4644-9457-3c4df8e8352d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:33.126789 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2018/3974] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:33.147407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:33.147407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:33.213695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:33.302512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190ebaf-46f2-477a-8f83-3cc51d5f29df admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:33.326538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e4533-f23c-4855-be8d-90aeb3183da8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:33.327982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e4533-f23c-4855-be8d-90aeb3183da8 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:33.329720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e4533-f23c-4855-be8d-90aeb3183da8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:33.330694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e4533-f23c-4855-be8d-90aeb3183da8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:33.360421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e4533-f23c-4855-be8d-90aeb3183da8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:33.361961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e4533-f23c-4855-be8d-90aeb3183da8 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:33.375987 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1957/3975] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:34.583917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cb219-c028-4644-9457-3c4df8e8352d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:34.585573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f11cb219-c028-4644-9457-3c4df8e8352d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:34.605084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cb219-c028-4644-9457-3c4df8e8352d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:34.615560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:34.616569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:34.617045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:34.617962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] PATH_INFO: `/v3/projects/a9910f909654434e8a6b6997b2df88ba` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:34.620161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9910f909654434e8a6b6997b2df88ba)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:34.625676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:34.763031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9910f909654434e8a6b6997b2df88ba', 'request_id': 'req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:34.775197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9910f909654434e8a6b6997b2df88ba', 'request_id': 'req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:34.786853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9910f909654434e8a6b6997b2df88ba', 'request_id': 'req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:34.804364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] The token cache is being invalidate because project a9910f909654434e8a6b6997b2df88ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:34.805153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:34.806210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:34.808599 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2019/3976] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:34 2022] DELETE /identity/v3/projects/a9910f909654434e8a6b6997b2df88ba => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:34.813317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e4533-f23c-4855-be8d-90aeb3183da8 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:34.825836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e4533-f23c-4855-be8d-90aeb3183da8 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:34.895875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe9794-2643-4951-a321-f05822693d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:34.896405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9794-2643-4951-a321-f05822693d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:34.896683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9794-2643-4951-a321-f05822693d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:34.896980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9794-2643-4951-a321-f05822693d53 admin admin] PATH_INFO: `/v3/users/f506a640b1d34910ad6bd049bf89debe` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:34.899244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe9794-2643-4951-a321-f05822693d53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f506a640b1d34910ad6bd049bf89debe)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:34.900810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe9794-2643-4951-a321-f05822693d53 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:34.986117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-71fe9794-2643-4951-a321-f05822693d53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f506a640b1d34910ad6bd049bf89debe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:35.004744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-71fe9794-2643-4951-a321-f05822693d53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f506a640b1d34910ad6bd049bf89debe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:35.007315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-71fe9794-2643-4951-a321-f05822693d53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f506a640b1d34910ad6bd049bf89debe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:35.014155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-71fe9794-2643-4951-a321-f05822693d53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f506a640b1d34910ad6bd049bf89debe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:35.023553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-71fe9794-2643-4951-a321-f05822693d53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f506a640b1d34910ad6bd049bf89debe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:35.037715 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1958/3977] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:34 2022] DELETE /identity/v3/users/f506a640b1d34910ad6bd049bf89debe => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:36.562412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:36.565929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:36.579627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c9331-8dd2-438b-ab1d-6095ee3fc2d2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:36.702855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1dad4-ec47-41de-99a8-fb5ac400b16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:36.703496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1dad4-ec47-41de-99a8-fb5ac400b16c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:36.704039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1dad4-ec47-41de-99a8-fb5ac400b16c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:36.704627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1dad4-ec47-41de-99a8-fb5ac400b16c admin admin] PATH_INFO: `/v3/projects/ed4c98ad20104dc0bee099240ee1e79a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:36.705863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1dad4-ec47-41de-99a8-fb5ac400b16c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed4c98ad20104dc0bee099240ee1e79a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:36.708561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1dad4-ec47-41de-99a8-fb5ac400b16c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:36.711253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:36.781051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02d1dad4-ec47-41de-99a8-fb5ac400b16c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed4c98ad20104dc0bee099240ee1e79a', 'request_id': 'req-02d1dad4-ec47-41de-99a8-fb5ac400b16c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:36.801500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02d1dad4-ec47-41de-99a8-fb5ac400b16c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed4c98ad20104dc0bee099240ee1e79a', 'request_id': 'req-02d1dad4-ec47-41de-99a8-fb5ac400b16c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:36.814518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02d1dad4-ec47-41de-99a8-fb5ac400b16c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed4c98ad20104dc0bee099240ee1e79a', 'request_id': 'req-02d1dad4-ec47-41de-99a8-fb5ac400b16c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:36.836061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02d1dad4-ec47-41de-99a8-fb5ac400b16c admin admin] The token cache is being invalidate because project ed4c98ad20104dc0bee099240ee1e79a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:36.836439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02d1dad4-ec47-41de-99a8-fb5ac400b16c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:36.837546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02d1dad4-ec47-41de-99a8-fb5ac400b16c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:36.839653 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2020/3978] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:36 2022] DELETE /identity/v3/projects/ed4c98ad20104dc0bee099240ee1e79a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:36.846111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5c302-c99e-462c-828c-a8ad9b908a3f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:36.849291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5c302-c99e-462c-828c-a8ad9b908a3f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:36.849985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5c302-c99e-462c-828c-a8ad9b908a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:36.912128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f2f5c302-c99e-462c-828c-a8ad9b908a3f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:36.914829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f2f5c302-c99e-462c-828c-a8ad9b908a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:36.981294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:37.027700 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1959/3979] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:37.035176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86942d97-e292-4f21-8e32-6789741d7cc9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:37.035537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86942d97-e292-4f21-8e32-6789741d7cc9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:37.035820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86942d97-e292-4f21-8e32-6789741d7cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:37.057349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-86942d97-e292-4f21-8e32-6789741d7cc9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:37.059247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-86942d97-e292-4f21-8e32-6789741d7cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:37.074797 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2021/3980] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:02:37.079107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5c302-c99e-462c-828c-a8ad9b908a3f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:37.089898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5c302-c99e-462c-828c-a8ad9b908a3f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:37.095414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623ad40-77b2-4423-8c3f-36a75b23395b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:37.096264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623ad40-77b2-4423-8c3f-36a75b23395b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:37.096809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623ad40-77b2-4423-8c3f-36a75b23395b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:37.097117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623ad40-77b2-4423-8c3f-36a75b23395b admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:37.097767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623ad40-77b2-4423-8c3f-36a75b23395b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:37.098672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623ad40-77b2-4423-8c3f-36a75b23395b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:37.100342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623ad40-77b2-4423-8c3f-36a75b23395b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:37.119544 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1960/3981] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:37.120772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86942d97-e292-4f21-8e32-6789741d7cc9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:37.137288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86942d97-e292-4f21-8e32-6789741d7cc9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:37.154513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1433cc-e762-4964-831d-c0223bcc9a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:37.155161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1433cc-e762-4964-831d-c0223bcc9a20 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:37.155452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1433cc-e762-4964-831d-c0223bcc9a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:37.155728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1433cc-e762-4964-831d-c0223bcc9a20 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:37.156390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1433cc-e762-4964-831d-c0223bcc9a20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:37.158355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1433cc-e762-4964-831d-c0223bcc9a20 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:37.214568 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2022/3982] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:37 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:37.218610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623ad40-77b2-4423-8c3f-36a75b23395b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:37.233955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623ad40-77b2-4423-8c3f-36a75b23395b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:37.427953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59895cc1-6a49-4b6e-8ea2-5f6da9aae0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:37.428889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59895cc1-6a49-4b6e-8ea2-5f6da9aae0aa admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:37.429159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59895cc1-6a49-4b6e-8ea2-5f6da9aae0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:37.429402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59895cc1-6a49-4b6e-8ea2-5f6da9aae0aa admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:37.430078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59895cc1-6a49-4b6e-8ea2-5f6da9aae0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:37.432017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59895cc1-6a49-4b6e-8ea2-5f6da9aae0aa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:37.502920 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1961/3983] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:37 2022] POST /identity/v3/users => generated 294 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:37.507021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1433cc-e762-4964-831d-c0223bcc9a20 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:37.532429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1433cc-e762-4964-831d-c0223bcc9a20 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:37.538293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e3cec-4ce9-4483-96ef-87a96d5bb098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:37.538957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e3cec-4ce9-4483-96ef-87a96d5bb098 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:37.539246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e3cec-4ce9-4483-96ef-87a96d5bb098 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:37.539520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e3cec-4ce9-4483-96ef-87a96d5bb098 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:37.540217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e3cec-4ce9-4483-96ef-87a96d5bb098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:37.542800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e3cec-4ce9-4483-96ef-87a96d5bb098 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:37.559423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59895cc1-6a49-4b6e-8ea2-5f6da9aae0aa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:37.559848 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2023/3984] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:37.572068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59895cc1-6a49-4b6e-8ea2-5f6da9aae0aa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:37.582604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b1341-84ce-4be9-92e7-bd089cccf139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:37.585408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b1341-84ce-4be9-92e7-bd089cccf139 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:37.585703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b1341-84ce-4be9-92e7-bd089cccf139 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:37.585984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b1341-84ce-4be9-92e7-bd089cccf139 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:37.586831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b1341-84ce-4be9-92e7-bd089cccf139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:37.589262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b1341-84ce-4be9-92e7-bd089cccf139 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:37.609554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e3cec-4ce9-4483-96ef-87a96d5bb098 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:37.610992 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1962/3985] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:37.626255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e3cec-4ce9-4483-96ef-87a96d5bb098 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:37.631459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d46b0-8877-4de2-bd36-fb8d62d72e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:37.632327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d46b0-8877-4de2-bd36-fb8d62d72e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:37.632890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d46b0-8877-4de2-bd36-fb8d62d72e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:37.633388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d46b0-8877-4de2-bd36-fb8d62d72e49 admin admin] PATH_INFO: `/v3/projects/998fc2f469564d1db12ec5512bac177c/users/49764b8f3d024b568c0e4b495d6c45d5/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:37.655412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d46b0-8877-4de2-bd36-fb8d62d72e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=998fc2f469564d1db12ec5512bac177c, user_id=49764b8f3d024b568c0e4b495d6c45d5, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:37.661054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d46b0-8877-4de2-bd36-fb8d62d72e49 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:37.685141 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2024/3986] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:37 2022] PUT /identity/v3/projects/998fc2f469564d1db12ec5512bac177c/users/49764b8f3d024b568c0e4b495d6c45d5/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:37.696963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097ff6f-6155-4141-981f-d652f64e01d6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:37.697263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097ff6f-6155-4141-981f-d652f64e01d6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:37.697540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097ff6f-6155-4141-981f-d652f64e01d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:37.734591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8097ff6f-6155-4141-981f-d652f64e01d6 None None] MFA Rules not processed for user `49764b8f3d024b568c0e4b495d6c45d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:37.737186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8097ff6f-6155-4141-981f-d652f64e01d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:37.887147 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1963/3987] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:37.891784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d46b0-8877-4de2-bd36-fb8d62d72e49 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:37.974303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d46b0-8877-4de2-bd36-fb8d62d72e49 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:37.979081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc92fb4-9c2e-4b23-9da0-1cd0bba99a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:37.979902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc92fb4-9c2e-4b23-9da0-1cd0bba99a24 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:37.980182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc92fb4-9c2e-4b23-9da0-1cd0bba99a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:37.980627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc92fb4-9c2e-4b23-9da0-1cd0bba99a24 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:37.981511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc92fb4-9c2e-4b23-9da0-1cd0bba99a24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:37.984595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc92fb4-9c2e-4b23-9da0-1cd0bba99a24 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:38.042079 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2025/3988] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:37 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:38.050549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097ff6f-6155-4141-981f-d652f64e01d6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:38.070357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097ff6f-6155-4141-981f-d652f64e01d6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:38.138751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:38.177848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9ff05-9600-46a5-b005-7f2675738492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:38.178738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9ff05-9600-46a5-b005-7f2675738492 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:38.179256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9ff05-9600-46a5-b005-7f2675738492 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:38.179591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9ff05-9600-46a5-b005-7f2675738492 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:38.180315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9ff05-9600-46a5-b005-7f2675738492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:38.183112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9ff05-9600-46a5-b005-7f2675738492 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:38.223896 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1964/3989] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:38 2022] POST /identity/v3/users => generated 300 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:38.224923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc92fb4-9c2e-4b23-9da0-1cd0bba99a24 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:38.238602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc92fb4-9c2e-4b23-9da0-1cd0bba99a24 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:38.244784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f2ebc-d7f2-4e4a-96dd-717e86e8f322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:38.245976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f2ebc-d7f2-4e4a-96dd-717e86e8f322 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:38.247169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f2ebc-d7f2-4e4a-96dd-717e86e8f322 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:38.247462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f2ebc-d7f2-4e4a-96dd-717e86e8f322 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:38.248340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f2ebc-d7f2-4e4a-96dd-717e86e8f322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:38.251022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f2ebc-d7f2-4e4a-96dd-717e86e8f322 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:38.258544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:38.273514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9ff05-9600-46a5-b005-7f2675738492 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:38.286257 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2026/3990] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:38.305273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9ff05-9600-46a5-b005-7f2675738492 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:38.321313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95038ae6-92e2-4aed-a9be-58d6cbd68b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:38.321874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95038ae6-92e2-4aed-a9be-58d6cbd68b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:38.322181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95038ae6-92e2-4aed-a9be-58d6cbd68b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:38.322468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95038ae6-92e2-4aed-a9be-58d6cbd68b27 admin admin] PATH_INFO: `/v3/projects/c22679bd26804f75bba0da34241257f2/users/85ef99fd3fce409f9457f8da1fe2894a/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:38.394232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95038ae6-92e2-4aed-a9be-58d6cbd68b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22679bd26804f75bba0da34241257f2, user_id=85ef99fd3fce409f9457f8da1fe2894a, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:38.398772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95038ae6-92e2-4aed-a9be-58d6cbd68b27 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:38.421948 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1965/3991] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:38 2022] PUT /identity/v3/projects/c22679bd26804f75bba0da34241257f2/users/85ef99fd3fce409f9457f8da1fe2894a/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:38.426322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f2ebc-d7f2-4e4a-96dd-717e86e8f322 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:38.445207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f2ebc-d7f2-4e4a-96dd-717e86e8f322 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:38.485888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df859b-0cbc-4a8a-994d-436025c5c274 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:38.486186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df859b-0cbc-4a8a-994d-436025c5c274 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:38.486468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df859b-0cbc-4a8a-994d-436025c5c274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:38.567392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-71df859b-0cbc-4a8a-994d-436025c5c274 None None] MFA Rules not processed for user `c3187514ca644ff68c992526f38783c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:38.568624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-71df859b-0cbc-4a8a-994d-436025c5c274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:38.574340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f750ec0-8dff-43d3-b2bb-cf75191331e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:38.576841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f750ec0-8dff-43d3-b2bb-cf75191331e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:38.577187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f750ec0-8dff-43d3-b2bb-cf75191331e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:38.577466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f750ec0-8dff-43d3-b2bb-cf75191331e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:38.578132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f750ec0-8dff-43d3-b2bb-cf75191331e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:38.579756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f750ec0-8dff-43d3-b2bb-cf75191331e1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:38.606341 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2027/3992] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:38.608525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f750ec0-8dff-43d3-b2bb-cf75191331e1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:38.628843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f750ec0-8dff-43d3-b2bb-cf75191331e1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:38.635548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5bd20-65da-49da-8963-ea1d3fe525a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:38.636873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5bd20-65da-49da-8963-ea1d3fe525a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:38.637349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5bd20-65da-49da-8963-ea1d3fe525a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:38.638077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5bd20-65da-49da-8963-ea1d3fe525a7 admin admin] PATH_INFO: `/v3/projects/c22679bd26804f75bba0da34241257f2/users/85ef99fd3fce409f9457f8da1fe2894a/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:38.640461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5bd20-65da-49da-8963-ea1d3fe525a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22679bd26804f75bba0da34241257f2, user_id=85ef99fd3fce409f9457f8da1fe2894a, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:38.643887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5bd20-65da-49da-8963-ea1d3fe525a7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:38.658748 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2028/3993] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:38 2022] PUT /identity/v3/projects/c22679bd26804f75bba0da34241257f2/users/85ef99fd3fce409f9457f8da1fe2894a/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:38.670455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad0b18-d805-4e47-ad26-ef1bf7d23d5a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:38.671028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad0b18-d805-4e47-ad26-ef1bf7d23d5a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:38.671510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad0b18-d805-4e47-ad26-ef1bf7d23d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:38.701051 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1966/3994] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:38.713634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f06cb3-ca4b-4fcc-b450-cb7093300df8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:38.713923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f06cb3-ca4b-4fcc-b450-cb7093300df8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:38.714196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f06cb3-ca4b-4fcc-b450-cb7093300df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:38.741705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a0ad0b18-d805-4e47-ad26-ef1bf7d23d5a None None] MFA Rules not processed for user `85ef99fd3fce409f9457f8da1fe2894a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:38.743574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a0ad0b18-d805-4e47-ad26-ef1bf7d23d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:38.761965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b4f06cb3-ca4b-4fcc-b450-cb7093300df8 None None] MFA Rules not processed for user `c3187514ca644ff68c992526f38783c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:38.766615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b4f06cb3-ca4b-4fcc-b450-cb7093300df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:38.897625 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1967/3995] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:38.913658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f06cb3-ca4b-4fcc-b450-cb7093300df8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:38.913658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b4f06cb3-ca4b-4fcc-b450-cb7093300df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:38.917635 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2029/3996] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:38.919361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad0b18-d805-4e47-ad26-ef1bf7d23d5a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:38.950946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad0b18-d805-4e47-ad26-ef1bf7d23d5a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:38.967693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a4ee5-cca9-4c30-b070-f8276c58abea tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ef99fd3fce409f9457f8da1fe2894a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c22679bd26804f75bba0da34241257f2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:38.969006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a4ee5-cca9-4c30-b070-f8276c58abea tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:38.969454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a4ee5-cca9-4c30-b070-f8276c58abea tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:38.969888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a4ee5-cca9-4c30-b070-f8276c58abea tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] PATH_INFO: `/v3/projects/998fc2f469564d1db12ec5512bac177c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:38.972319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a4ee5-cca9-4c30-b070-f8276c58abea tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorizing `identity:get_project(project_id=998fc2f469564d1db12ec5512bac177c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:38.977887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a4ee5-cca9-4c30-b070-f8276c58abea tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:38.988404 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2030/3997] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:02:38 2022] GET /identity/v3/projects/998fc2f469564d1db12ec5512bac177c => generated 387 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:38.990647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a4ee5-cca9-4c30-b070-f8276c58abea tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:39.004000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:39.019819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a4ee5-cca9-4c30-b070-f8276c58abea tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:39.026048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d656c-e8d2-452f-996b-2004ff8dba28 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ef99fd3fce409f9457f8da1fe2894a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c22679bd26804f75bba0da34241257f2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:39.028429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d656c-e8d2-452f-996b-2004ff8dba28 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:39.029016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d656c-e8d2-452f-996b-2004ff8dba28 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:39.029414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d656c-e8d2-452f-996b-2004ff8dba28 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:39.030053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d656c-e8d2-452f-996b-2004ff8dba28 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:39.037743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d656c-e8d2-452f-996b-2004ff8dba28 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:39.126335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f06cb3-ca4b-4fcc-b450-cb7093300df8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:39.126563 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2031/3998] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:02:38 2022] POST /identity/v3/projects => generated 339 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:39.129245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d656c-e8d2-452f-996b-2004ff8dba28 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:39.272914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:39.286305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98dce9d-8dca-4bb6-821e-c6fcfa536ad4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:39.289185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dce9d-8dca-4bb6-821e-c6fcfa536ad4 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:39.289185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dce9d-8dca-4bb6-821e-c6fcfa536ad4 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:39.289185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dce9d-8dca-4bb6-821e-c6fcfa536ad4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:39.353284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d656c-e8d2-452f-996b-2004ff8dba28 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:39.377151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df28b81-5d8a-48c3-a639-7cf455dd46b2 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ef99fd3fce409f9457f8da1fe2894a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c22679bd26804f75bba0da34241257f2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:39.377793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df28b81-5d8a-48c3-a639-7cf455dd46b2 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:39.378087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df28b81-5d8a-48c3-a639-7cf455dd46b2 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:39.379567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df28b81-5d8a-48c3-a639-7cf455dd46b2 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:39.379567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df28b81-5d8a-48c3-a639-7cf455dd46b2 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:39.380055 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-7df28b81-5d8a-48c3-a639-7cf455dd46b2 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:39.380055 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:39.380055 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:39.381252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df28b81-5d8a-48c3-a639-7cf455dd46b2 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:39.403613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98dce9d-8dca-4bb6-821e-c6fcfa536ad4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:39.406419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98dce9d-8dca-4bb6-821e-c6fcfa536ad4 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:39.429491 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2032/3999] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Sep 3 21:02:39 2022] GET /identity/v3/projects => generated 6596 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:39.434958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df28b81-5d8a-48c3-a639-7cf455dd46b2 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:39.454514 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1968/4000] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:39.460305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df28b81-5d8a-48c3-a639-7cf455dd46b2 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:39.472504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3d8fd-73d1-4382-8a25-1118d6dae98d tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ef99fd3fce409f9457f8da1fe2894a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c22679bd26804f75bba0da34241257f2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:39.473209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3d8fd-73d1-4382-8a25-1118d6dae98d tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:39.473499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3d8fd-73d1-4382-8a25-1118d6dae98d tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:39.473788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3d8fd-73d1-4382-8a25-1118d6dae98d tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] PATH_INFO: `/v3/projects/998fc2f469564d1db12ec5512bac177c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:39.478351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3d8fd-73d1-4382-8a25-1118d6dae98d tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorizing `identity:get_project(project_id=998fc2f469564d1db12ec5512bac177c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:39.482160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3d8fd-73d1-4382-8a25-1118d6dae98d tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:39.488313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98dce9d-8dca-4bb6-821e-c6fcfa536ad4 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:39.488534 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2033/4001] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:02:39 2022] GET /identity/v3/projects/998fc2f469564d1db12ec5512bac177c => generated 387 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:39.516103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98dce9d-8dca-4bb6-821e-c6fcfa536ad4 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:39.539183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850bc6e-f12d-4ec3-ad96-b8a8d0d8b42f tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ef99fd3fce409f9457f8da1fe2894a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c22679bd26804f75bba0da34241257f2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:39.540452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850bc6e-f12d-4ec3-ad96-b8a8d0d8b42f tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:39.540931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850bc6e-f12d-4ec3-ad96-b8a8d0d8b42f tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:39.541394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850bc6e-f12d-4ec3-ad96-b8a8d0d8b42f tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] PATH_INFO: `/v3/projects/227f7f6b82f0471388dc39f3b05af072` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:39.543406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850bc6e-f12d-4ec3-ad96-b8a8d0d8b42f tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorizing `identity:get_project(project_id=227f7f6b82f0471388dc39f3b05af072)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:39.557801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850bc6e-f12d-4ec3-ad96-b8a8d0d8b42f tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:39.560358 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1969/4002] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:02:39 2022] GET /identity/v3/projects/227f7f6b82f0471388dc39f3b05af072 => generated 339 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:39.567511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3d8fd-73d1-4382-8a25-1118d6dae98d tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:39.582785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3d8fd-73d1-4382-8a25-1118d6dae98d tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:39.596711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db36fa9-baa4-49b1-bb50-ff084435fccc tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ef99fd3fce409f9457f8da1fe2894a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c22679bd26804f75bba0da34241257f2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:39.597672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db36fa9-baa4-49b1-bb50-ff084435fccc tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:39.598113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db36fa9-baa4-49b1-bb50-ff084435fccc tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:39.598660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db36fa9-baa4-49b1-bb50-ff084435fccc tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:39.599574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db36fa9-baa4-49b1-bb50-ff084435fccc tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:39.601710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db36fa9-baa4-49b1-bb50-ff084435fccc tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:39.631815 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2034/4003] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Sep 3 21:02:39 2022] GET /identity/v3/projects => generated 6596 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:39.636510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850bc6e-f12d-4ec3-ad96-b8a8d0d8b42f tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:39.649217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850bc6e-f12d-4ec3-ad96-b8a8d0d8b42f tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:39.655877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33f51e-e963-4b4f-a51e-7c3ab66b2759 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ef99fd3fce409f9457f8da1fe2894a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c22679bd26804f75bba0da34241257f2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:39.656495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33f51e-e963-4b4f-a51e-7c3ab66b2759 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:39.657345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33f51e-e963-4b4f-a51e-7c3ab66b2759 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:39.657650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33f51e-e963-4b4f-a51e-7c3ab66b2759 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:39.660608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33f51e-e963-4b4f-a51e-7c3ab66b2759 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:39.660608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33f51e-e963-4b4f-a51e-7c3ab66b2759 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:39.660846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33f51e-e963-4b4f-a51e-7c3ab66b2759 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:39.700212 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1970/4004] 10.222.0.16 () {58 vars in 1166 bytes} [Sat Sep 3 21:02:39 2022] GET /identity/v3/projects?domain_id=default => generated 6265 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:39.700804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db36fa9-baa4-49b1-bb50-ff084435fccc tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:39.715377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db36fa9-baa4-49b1-bb50-ff084435fccc tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:39.720530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e27109-667c-4922-b0eb-3aeb5fea8df1 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ef99fd3fce409f9457f8da1fe2894a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c22679bd26804f75bba0da34241257f2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:39.721650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27109-667c-4922-b0eb-3aeb5fea8df1 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:39.722198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27109-667c-4922-b0eb-3aeb5fea8df1 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:39.722567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27109-667c-4922-b0eb-3aeb5fea8df1 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:39.723299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e27109-667c-4922-b0eb-3aeb5fea8df1 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:39.726910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e27109-667c-4922-b0eb-3aeb5fea8df1 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:39.769463 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2035/4005] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Sep 3 21:02:39 2022] GET /identity/v3/projects => generated 6596 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:39.770688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33f51e-e963-4b4f-a51e-7c3ab66b2759 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:39.791449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33f51e-e963-4b4f-a51e-7c3ab66b2759 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:39.797765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffb83d-026d-4bd0-aeb4-23f94e0030f8 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ef99fd3fce409f9457f8da1fe2894a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c22679bd26804f75bba0da34241257f2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:39.798630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffb83d-026d-4bd0-aeb4-23f94e0030f8 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:39.799105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffb83d-026d-4bd0-aeb4-23f94e0030f8 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:39.799551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffb83d-026d-4bd0-aeb4-23f94e0030f8 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:39.800969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffb83d-026d-4bd0-aeb4-23f94e0030f8 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:39.801450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffb83d-026d-4bd0-aeb4-23f94e0030f8 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:39.804169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffb83d-026d-4bd0-aeb4-23f94e0030f8 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:39.827622 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1971/4006] 10.222.0.16 () {58 vars in 1166 bytes} [Sat Sep 3 21:02:39 2022] GET /identity/v3/projects?domain_id=default => generated 6265 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:39.835332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e27109-667c-4922-b0eb-3aeb5fea8df1 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:39.849459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e27109-667c-4922-b0eb-3aeb5fea8df1 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:39.855567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb4293-3f79-480d-bc5c-f1bc00775aaf tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ef99fd3fce409f9457f8da1fe2894a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c22679bd26804f75bba0da34241257f2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:39.856296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb4293-3f79-480d-bc5c-f1bc00775aaf tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:39.856650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb4293-3f79-480d-bc5c-f1bc00775aaf tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:39.857039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb4293-3f79-480d-bc5c-f1bc00775aaf tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:39.858393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb4293-3f79-480d-bc5c-f1bc00775aaf tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:39.859965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb4293-3f79-480d-bc5c-f1bc00775aaf tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:39.899868 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2036/4007] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Sep 3 21:02:39 2022] GET /identity/v3/projects => generated 6596 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:39.900516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffb83d-026d-4bd0-aeb4-23f94e0030f8 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:39.923500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffb83d-026d-4bd0-aeb4-23f94e0030f8 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:39.940062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82618cc3-a8f9-48bf-bf0b-63f1cc113e41 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ef99fd3fce409f9457f8da1fe2894a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c22679bd26804f75bba0da34241257f2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:39.941216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82618cc3-a8f9-48bf-bf0b-63f1cc113e41 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:39.941709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82618cc3-a8f9-48bf-bf0b-63f1cc113e41 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:39.942606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82618cc3-a8f9-48bf-bf0b-63f1cc113e41 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:39.943635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82618cc3-a8f9-48bf-bf0b-63f1cc113e41 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Adding query filter params (name=tempest-ListProjectsStaticTestJSON-218903041) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:39.944127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82618cc3-a8f9-48bf-bf0b-63f1cc113e41 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:39.947330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82618cc3-a8f9-48bf-bf0b-63f1cc113e41 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:39.994107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb4293-3f79-480d-bc5c-f1bc00775aaf tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:40.008974 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1972/4008] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:02:39 2022] GET /identity/v3/projects?name=tempest-ListProjectsStaticTestJSON-218903041 => generated 534 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:40.017022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb4293-3f79-480d-bc5c-f1bc00775aaf tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:40.028751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a77d57-2c64-463e-b275-a42f0c832419 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ef99fd3fce409f9457f8da1fe2894a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c22679bd26804f75bba0da34241257f2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:40.029843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a77d57-2c64-463e-b275-a42f0c832419 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:40.031248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a77d57-2c64-463e-b275-a42f0c832419 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:40.032404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a77d57-2c64-463e-b275-a42f0c832419 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] PATH_INFO: `/v3/projects/227f7f6b82f0471388dc39f3b05af072` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:40.038321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a77d57-2c64-463e-b275-a42f0c832419 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorizing `identity:delete_project(project_id=227f7f6b82f0471388dc39f3b05af072)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:40.040700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a77d57-2c64-463e-b275-a42f0c832419 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:40.122920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d1a77d57-2c64-463e-b275-a42f0c832419 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '227f7f6b82f0471388dc39f3b05af072', 'request_id': 'req-d1a77d57-2c64-463e-b275-a42f0c832419'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:40.132152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d1a77d57-2c64-463e-b275-a42f0c832419 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '227f7f6b82f0471388dc39f3b05af072', 'request_id': 'req-d1a77d57-2c64-463e-b275-a42f0c832419'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:40.142197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d1a77d57-2c64-463e-b275-a42f0c832419 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '227f7f6b82f0471388dc39f3b05af072', 'request_id': 'req-d1a77d57-2c64-463e-b275-a42f0c832419'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:40.159507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d1a77d57-2c64-463e-b275-a42f0c832419 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] The token cache is being invalidate because project 227f7f6b82f0471388dc39f3b05af072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:40.159988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d1a77d57-2c64-463e-b275-a42f0c832419 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:40.160979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d1a77d57-2c64-463e-b275-a42f0c832419 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:40.162946 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2037/4009] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:39 2022] DELETE /identity/v3/projects/227f7f6b82f0471388dc39f3b05af072 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:40.166877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82618cc3-a8f9-48bf-bf0b-63f1cc113e41 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:40.168808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-82618cc3-a8f9-48bf-bf0b-63f1cc113e41 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:40.214214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:40.337665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82618cc3-a8f9-48bf-bf0b-63f1cc113e41 tempest-ListProjectsStaticTestJSON-281337831 tempest-ListProjectsStaticTestJSON-281337831-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:40.443098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:40.445541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:40.447780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:40.448977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] PATH_INFO: `/v3/users/49764b8f3d024b568c0e4b495d6c45d5` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:40.450585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49764b8f3d024b568c0e4b495d6c45d5)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:40.461532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:40.572187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49764b8f3d024b568c0e4b495d6c45d5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:40.586606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49764b8f3d024b568c0e4b495d6c45d5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:40.587450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49764b8f3d024b568c0e4b495d6c45d5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:40.607605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49764b8f3d024b568c0e4b495d6c45d5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:40.619168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49764b8f3d024b568c0e4b495d6c45d5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:40.631658 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1973/4010] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:40 2022] DELETE /identity/v3/users/49764b8f3d024b568c0e4b495d6c45d5 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:40.632425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668358d7-ce32-4bb3-a455-6bafaba1c510 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:40.632934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668358d7-ce32-4bb3-a455-6bafaba1c510 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:40.633161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668358d7-ce32-4bb3-a455-6bafaba1c510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:40.681562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-668358d7-ce32-4bb3-a455-6bafaba1c510 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:40.682558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-668358d7-ce32-4bb3-a455-6bafaba1c510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:40.857555 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2038/4011] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:02:40.875619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:40.877356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:40.965828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:40.986627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f66166-503c-40c4-b127-20b7c2a7d3b7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:40.999958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765743ea-fa5c-4209-ad92-a8f71e930d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:41.000604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765743ea-fa5c-4209-ad92-a8f71e930d69 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:41.000885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765743ea-fa5c-4209-ad92-a8f71e930d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:41.001781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765743ea-fa5c-4209-ad92-a8f71e930d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:41.024774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765743ea-fa5c-4209-ad92-a8f71e930d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:41.026461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765743ea-fa5c-4209-ad92-a8f71e930d69 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:41.045664 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1974/4012] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:42.193794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668358d7-ce32-4bb3-a455-6bafaba1c510 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:42.195346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-668358d7-ce32-4bb3-a455-6bafaba1c510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:42.211475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668358d7-ce32-4bb3-a455-6bafaba1c510 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:42.216423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:42.217565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:42.218010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:42.218595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] PATH_INFO: `/v3/projects/998fc2f469564d1db12ec5512bac177c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:42.219905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] RBAC: Authorizing `identity:delete_project(project_id=998fc2f469564d1db12ec5512bac177c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:42.222279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:42.239675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:42.304459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '998fc2f469564d1db12ec5512bac177c', 'request_id': 'req-15ede802-4d0f-492d-8a9d-ecf70362934d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:42.315082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '998fc2f469564d1db12ec5512bac177c', 'request_id': 'req-15ede802-4d0f-492d-8a9d-ecf70362934d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:42.326735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '998fc2f469564d1db12ec5512bac177c', 'request_id': 'req-15ede802-4d0f-492d-8a9d-ecf70362934d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:42.339657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] The token cache is being invalidate because project 998fc2f469564d1db12ec5512bac177c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:42.340053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:42.341262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:42.342969 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2039/4013] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:42 2022] DELETE /identity/v3/projects/998fc2f469564d1db12ec5512bac177c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:42.350766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765743ea-fa5c-4209-ad92-a8f71e930d69 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:42.366239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765743ea-fa5c-4209-ad92-a8f71e930d69 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:42.509962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:42.511245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:42.511772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:42.512530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] PATH_INFO: `/v3/users/85ef99fd3fce409f9457f8da1fe2894a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:42.529753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85ef99fd3fce409f9457f8da1fe2894a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:42.535555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:42.656992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85ef99fd3fce409f9457f8da1fe2894a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:42.682963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85ef99fd3fce409f9457f8da1fe2894a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:42.684983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85ef99fd3fce409f9457f8da1fe2894a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:42.693601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85ef99fd3fce409f9457f8da1fe2894a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:42.724317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85ef99fd3fce409f9457f8da1fe2894a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:42.757001 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1975/4014] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:42 2022] DELETE /identity/v3/users/85ef99fd3fce409f9457f8da1fe2894a => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:43.551222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:43.553869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:43.620311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ede802-4d0f-492d-8a9d-ecf70362934d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:43.711300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ebb0d-e4a3-4eb0-a09c-e7388fadf4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:43.715528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ebb0d-e4a3-4eb0-a09c-e7388fadf4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:43.715910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ebb0d-e4a3-4eb0-a09c-e7388fadf4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:43.717504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ebb0d-e4a3-4eb0-a09c-e7388fadf4c0 admin admin] PATH_INFO: `/v3/users/8923b1af5fd54c31ac0af85564cfdd92` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:43.719162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ebb0d-e4a3-4eb0-a09c-e7388fadf4c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8923b1af5fd54c31ac0af85564cfdd92)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:43.723864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ebb0d-e4a3-4eb0-a09c-e7388fadf4c0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:43.807209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:43.810327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:43.837921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2e1ebb0d-e4a3-4eb0-a09c-e7388fadf4c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8923b1af5fd54c31ac0af85564cfdd92'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:43.858442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe87d62-63d8-4b7a-92da-f39133a5e499 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:43.873813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2e1ebb0d-e4a3-4eb0-a09c-e7388fadf4c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8923b1af5fd54c31ac0af85564cfdd92'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:43.874996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:43.876138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:43.877545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:43.878457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79 admin admin] PATH_INFO: `/v3/projects/c22679bd26804f75bba0da34241257f2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:43.882460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c22679bd26804f75bba0da34241257f2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:43.887452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2e1ebb0d-e4a3-4eb0-a09c-e7388fadf4c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8923b1af5fd54c31ac0af85564cfdd92'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:43.888479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2e1ebb0d-e4a3-4eb0-a09c-e7388fadf4c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8923b1af5fd54c31ac0af85564cfdd92'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:43.889365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2e1ebb0d-e4a3-4eb0-a09c-e7388fadf4c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8923b1af5fd54c31ac0af85564cfdd92'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:43.900043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:43.915641 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2040/4015] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:43 2022] DELETE /identity/v3/users/8923b1af5fd54c31ac0af85564cfdd92 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:44.016570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c22679bd26804f75bba0da34241257f2', 'request_id': 'req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:44.026913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c22679bd26804f75bba0da34241257f2', 'request_id': 'req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:44.041062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c22679bd26804f75bba0da34241257f2', 'request_id': 'req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:44.046606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:44.060877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79 admin admin] The token cache is being invalidate because project c22679bd26804f75bba0da34241257f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:44.062427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:44.063634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-328747e2-ae6c-4eb5-97f8-ffc5f1463d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:44.070372 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1976/4016] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:43 2022] DELETE /identity/v3/projects/c22679bd26804f75bba0da34241257f2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:44.074257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f144994-4d23-4692-9007-935dd1fe90d3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.074748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f144994-4d23-4692-9007-935dd1fe90d3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.075191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f144994-4d23-4692-9007-935dd1fe90d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.123623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8f144994-4d23-4692-9007-935dd1fe90d3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:44.127398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8f144994-4d23-4692-9007-935dd1fe90d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:44.170501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:44.280741 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2041/4017] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:44.281808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e04d89-db96-4738-a62b-062607ed76b2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.282105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e04d89-db96-4738-a62b-062607ed76b2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.282385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e04d89-db96-4738-a62b-062607ed76b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.305883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-48e04d89-db96-4738-a62b-062607ed76b2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:44.307141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-48e04d89-db96-4738-a62b-062607ed76b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:44.321751 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1977/4018] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:44.326819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f144994-4d23-4692-9007-935dd1fe90d3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:44.337142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f144994-4d23-4692-9007-935dd1fe90d3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:44.342337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f235c-7afc-4f8c-a77f-b21cb4461a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:44.343112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f235c-7afc-4f8c-a77f-b21cb4461a4c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.343252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f235c-7afc-4f8c-a77f-b21cb4461a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.344100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f235c-7afc-4f8c-a77f-b21cb4461a4c admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.344252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f235c-7afc-4f8c-a77f-b21cb4461a4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:44.344490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f235c-7afc-4f8c-a77f-b21cb4461a4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:44.346166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f235c-7afc-4f8c-a77f-b21cb4461a4c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:44.365889 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2042/4019] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:02:44.367210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e04d89-db96-4738-a62b-062607ed76b2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:44.379893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f235c-7afc-4f8c-a77f-b21cb4461a4c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:44.387297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b07f235c-7afc-4f8c-a77f-b21cb4461a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:44.405015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e04d89-db96-4738-a62b-062607ed76b2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:44.411120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767e3c0-d3ea-49fe-90f4-58bc6c533584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:44.412055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767e3c0-d3ea-49fe-90f4-58bc6c533584 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.412555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767e3c0-d3ea-49fe-90f4-58bc6c533584 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.413082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767e3c0-d3ea-49fe-90f4-58bc6c533584 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.413925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767e3c0-d3ea-49fe-90f4-58bc6c533584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:44.415052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f235c-7afc-4f8c-a77f-b21cb4461a4c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:44.417906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767e3c0-d3ea-49fe-90f4-58bc6c533584 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:44.423562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d8c82-5884-4bc1-8c0c-96b439978a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:44.424952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d8c82-5884-4bc1-8c0c-96b439978a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.425253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d8c82-5884-4bc1-8c0c-96b439978a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.425535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d8c82-5884-4bc1-8c0c-96b439978a1b admin admin] PATH_INFO: `/v3/projects/146fb657f35e47de8fbad6cacc39f501` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.428023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d8c82-5884-4bc1-8c0c-96b439978a1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=146fb657f35e47de8fbad6cacc39f501)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:44.430834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d8c82-5884-4bc1-8c0c-96b439978a1b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:44.458546 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1978/4020] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:44 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:44.462344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767e3c0-d3ea-49fe-90f4-58bc6c533584 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:44.496590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-099d8c82-5884-4bc1-8c0c-96b439978a1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '146fb657f35e47de8fbad6cacc39f501', 'request_id': 'req-099d8c82-5884-4bc1-8c0c-96b439978a1b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:44.506171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-099d8c82-5884-4bc1-8c0c-96b439978a1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '146fb657f35e47de8fbad6cacc39f501', 'request_id': 'req-099d8c82-5884-4bc1-8c0c-96b439978a1b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:44.513903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-099d8c82-5884-4bc1-8c0c-96b439978a1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '146fb657f35e47de8fbad6cacc39f501', 'request_id': 'req-099d8c82-5884-4bc1-8c0c-96b439978a1b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:44.526075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-099d8c82-5884-4bc1-8c0c-96b439978a1b admin admin] The token cache is being invalidate because project 146fb657f35e47de8fbad6cacc39f501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:44.526075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-099d8c82-5884-4bc1-8c0c-96b439978a1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:44.526800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-099d8c82-5884-4bc1-8c0c-96b439978a1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:44.528805 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2043/4021] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:44 2022] DELETE /identity/v3/projects/146fb657f35e47de8fbad6cacc39f501 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:44.534427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e378e94-a22f-4440-a621-f3564f177bf8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.534920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e378e94-a22f-4440-a621-f3564f177bf8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.535870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e378e94-a22f-4440-a621-f3564f177bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.562637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767e3c0-d3ea-49fe-90f4-58bc6c533584 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:44.565836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9e378e94-a22f-4440-a621-f3564f177bf8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:44.567176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9e378e94-a22f-4440-a621-f3564f177bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:44.680038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e075c-0436-4c48-8a17-bdbadd9db779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:44.680679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e075c-0436-4c48-8a17-bdbadd9db779 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.682052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e075c-0436-4c48-8a17-bdbadd9db779 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.683038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e075c-0436-4c48-8a17-bdbadd9db779 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.685678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e075c-0436-4c48-8a17-bdbadd9db779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:44.687817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e075c-0436-4c48-8a17-bdbadd9db779 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:44.701184 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2044/4022] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:44.710436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9264df4-435f-4bd8-baba-0b1c7d9864aa None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.711224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9264df4-435f-4bd8-baba-0b1c7d9864aa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.712142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9264df4-435f-4bd8-baba-0b1c7d9864aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.724678 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1979/4023] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:44 2022] POST /identity/v3/users => generated 287 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:44.729260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e075c-0436-4c48-8a17-bdbadd9db779 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:44.731992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fa1e075c-0436-4c48-8a17-bdbadd9db779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:44.746740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e075c-0436-4c48-8a17-bdbadd9db779 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:44.759601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32587f-2937-4f5d-acc5-031c4bd0368b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:44.759830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32587f-2937-4f5d-acc5-031c4bd0368b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.761697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32587f-2937-4f5d-acc5-031c4bd0368b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.762423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32587f-2937-4f5d-acc5-031c4bd0368b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.763774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32587f-2937-4f5d-acc5-031c4bd0368b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:44.767401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32587f-2937-4f5d-acc5-031c4bd0368b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:44.771084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f9264df4-435f-4bd8-baba-0b1c7d9864aa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:44.772627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f9264df4-435f-4bd8-baba-0b1c7d9864aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:44.784373 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2045/4024] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:02:44.788635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9264df4-435f-4bd8-baba-0b1c7d9864aa None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:44.790278 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1980/4025] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:02:44.793865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32587f-2937-4f5d-acc5-031c4bd0368b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:44.799774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9264df4-435f-4bd8-baba-0b1c7d9864aa None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:44.806436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32587f-2937-4f5d-acc5-031c4bd0368b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:44.808343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66ba53-d69c-4105-80f3-1a1499ba4bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:44.809011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66ba53-d69c-4105-80f3-1a1499ba4bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.809299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66ba53-d69c-4105-80f3-1a1499ba4bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.809754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66ba53-d69c-4105-80f3-1a1499ba4bd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.810343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66ba53-d69c-4105-80f3-1a1499ba4bd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:44.810654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66ba53-d69c-4105-80f3-1a1499ba4bd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:44.814577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6070ee-5731-4a0e-b087-54aa2764304a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:44.815404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6070ee-5731-4a0e-b087-54aa2764304a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.815915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6070ee-5731-4a0e-b087-54aa2764304a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.816523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6070ee-5731-4a0e-b087-54aa2764304a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.817104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66ba53-d69c-4105-80f3-1a1499ba4bd3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:44.817611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6070ee-5731-4a0e-b087-54aa2764304a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:44.819610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6070ee-5731-4a0e-b087-54aa2764304a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:44.835910 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1981/4026] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:02:44.838500 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2046/4027] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:02:44.844118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6070ee-5731-4a0e-b087-54aa2764304a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:44.844295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66ba53-d69c-4105-80f3-1a1499ba4bd3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:44.855364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6070ee-5731-4a0e-b087-54aa2764304a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:44.861622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66ba53-d69c-4105-80f3-1a1499ba4bd3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:44.861838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b2096-0e79-4fb8-bbec-b7d7fab703f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:44.862563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b2096-0e79-4fb8-bbec-b7d7fab703f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.863022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b2096-0e79-4fb8-bbec-b7d7fab703f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.863395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b2096-0e79-4fb8-bbec-b7d7fab703f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.864125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b2096-0e79-4fb8-bbec-b7d7fab703f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:44.866174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b2096-0e79-4fb8-bbec-b7d7fab703f7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:44.869111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ada89-fb2f-450e-8170-598087bf2bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:44.870554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ada89-fb2f-450e-8170-598087bf2bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.871030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ada89-fb2f-450e-8170-598087bf2bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.871474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ada89-fb2f-450e-8170-598087bf2bd2 admin admin] PATH_INFO: `/v3/projects/1d98c4cc6666426188573a9ac45c51ba/users/82f898e9c45e4b509f250c2b95e97e26/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:44.876372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:44.896816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ada89-fb2f-450e-8170-598087bf2bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d98c4cc6666426188573a9ac45c51ba, user_id=82f898e9c45e4b509f250c2b95e97e26, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:44.903488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ada89-fb2f-450e-8170-598087bf2bd2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:44.910574 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1982/4028] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:44 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:44.912354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b2096-0e79-4fb8-bbec-b7d7fab703f7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:44.924873 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2047/4029] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:44 2022] PUT /identity/v3/projects/1d98c4cc6666426188573a9ac45c51ba/users/82f898e9c45e4b509f250c2b95e97e26/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:44.943330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d997d-ee82-4caa-a0cc-2f4c074242ee None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:44.951655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:44.961364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d997d-ee82-4caa-a0cc-2f4c074242ee None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:44.961917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d997d-ee82-4caa-a0cc-2f4c074242ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:45.034269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5e5d997d-ee82-4caa-a0cc-2f4c074242ee None None] MFA Rules not processed for user `82f898e9c45e4b509f250c2b95e97e26`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:45.035532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5e5d997d-ee82-4caa-a0cc-2f4c074242ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:45.061809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b2096-0e79-4fb8-bbec-b7d7fab703f7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:45.156585 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2048/4030] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:45.159629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d997d-ee82-4caa-a0cc-2f4c074242ee None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:45.168707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4638e-91e2-4e48-8d64-2ef128f8a084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:45.170829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4638e-91e2-4e48-8d64-2ef128f8a084 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:45.172167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4638e-91e2-4e48-8d64-2ef128f8a084 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:45.172768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4638e-91e2-4e48-8d64-2ef128f8a084 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:45.174254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4638e-91e2-4e48-8d64-2ef128f8a084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:45.179832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4638e-91e2-4e48-8d64-2ef128f8a084 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:45.235065 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1983/4031] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:44 2022] POST /identity/v3/users => generated 285 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:45.239662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4638e-91e2-4e48-8d64-2ef128f8a084 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:45.262628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4638e-91e2-4e48-8d64-2ef128f8a084 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:45.275060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e1ae8-6781-4e28-9b7e-22072e435ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:45.275692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e1ae8-6781-4e28-9b7e-22072e435ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:45.277183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e1ae8-6781-4e28-9b7e-22072e435ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:45.277489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e1ae8-6781-4e28-9b7e-22072e435ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:45.278151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e1ae8-6781-4e28-9b7e-22072e435ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:45.280277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e1ae8-6781-4e28-9b7e-22072e435ec2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:45.301905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d997d-ee82-4caa-a0cc-2f4c074242ee None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:45.306241 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1984/4032] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:45.316635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e1ae8-6781-4e28-9b7e-22072e435ec2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:45.333546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44180f97-b2b3-44ba-8019-415957c386dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:45.336807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44180f97-b2b3-44ba-8019-415957c386dc admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:45.337034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44180f97-b2b3-44ba-8019-415957c386dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:45.337303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44180f97-b2b3-44ba-8019-415957c386dc admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:45.338161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44180f97-b2b3-44ba-8019-415957c386dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:45.340473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44180f97-b2b3-44ba-8019-415957c386dc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:45.353036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e1ae8-6781-4e28-9b7e-22072e435ec2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:45.366190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fd8c7-8246-4cd6-b5ae-66d8f96bb954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:45.366888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fd8c7-8246-4cd6-b5ae-66d8f96bb954 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:45.367178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fd8c7-8246-4cd6-b5ae-66d8f96bb954 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:45.367456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fd8c7-8246-4cd6-b5ae-66d8f96bb954 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:45.368119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fd8c7-8246-4cd6-b5ae-66d8f96bb954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:45.370432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fd8c7-8246-4cd6-b5ae-66d8f96bb954 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:45.382962 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2049/4033] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:45 2022] POST /identity/v3/projects => generated 375 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:45.384007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44180f97-b2b3-44ba-8019-415957c386dc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:45.396124 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1985/4034] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:45.400155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fd8c7-8246-4cd6-b5ae-66d8f96bb954 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:45.418715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fd8c7-8246-4cd6-b5ae-66d8f96bb954 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:45.530248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44180f97-b2b3-44ba-8019-415957c386dc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:45.546959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae547768-2cdf-4229-ae2f-458e50fd833a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:45.549071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae547768-2cdf-4229-ae2f-458e50fd833a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:45.549511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae547768-2cdf-4229-ae2f-458e50fd833a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:45.549969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae547768-2cdf-4229-ae2f-458e50fd833a admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:45.551185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae547768-2cdf-4229-ae2f-458e50fd833a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:45.556752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae547768-2cdf-4229-ae2f-458e50fd833a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:45.581072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:45.581737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982410f-3dc3-47e8-b4d9-f48041814470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:45.582216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982410f-3dc3-47e8-b4d9-f48041814470 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:45.582623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982410f-3dc3-47e8-b4d9-f48041814470 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:45.582989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982410f-3dc3-47e8-b4d9-f48041814470 admin admin] PATH_INFO: `/v3/projects/26e12c7419f641b4a7e56274a34826ee/users/77edcf09f5f84e51a4fb08dbaf763a67/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:45.621420 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2050/4035] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:45 2022] POST /identity/v3/users => generated 294 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:45.625314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982410f-3dc3-47e8-b4d9-f48041814470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e12c7419f641b4a7e56274a34826ee, user_id=77edcf09f5f84e51a4fb08dbaf763a67, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:45.626203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae547768-2cdf-4229-ae2f-458e50fd833a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:45.628641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982410f-3dc3-47e8-b4d9-f48041814470 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:45.648090 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1986/4036] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:45 2022] PUT /identity/v3/projects/26e12c7419f641b4a7e56274a34826ee/users/77edcf09f5f84e51a4fb08dbaf763a67/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:45.652827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae547768-2cdf-4229-ae2f-458e50fd833a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:45.653435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cea9f3-406d-47a4-9757-eb58c54756be None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:45.653884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cea9f3-406d-47a4-9757-eb58c54756be None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:45.654258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cea9f3-406d-47a4-9757-eb58c54756be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:45.679496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-34cea9f3-406d-47a4-9757-eb58c54756be None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:45.680888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-34cea9f3-406d-47a4-9757-eb58c54756be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:45.686812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:45.757024 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1987/4037] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:45.761194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83301237-f717-4c89-b460-d5cee0d28cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:45.762644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83301237-f717-4c89-b460-d5cee0d28cdc admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:45.763409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83301237-f717-4c89-b460-d5cee0d28cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:45.765338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83301237-f717-4c89-b460-d5cee0d28cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:45.766616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83301237-f717-4c89-b460-d5cee0d28cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:45.771691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83301237-f717-4c89-b460-d5cee0d28cdc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:45.785442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cea9f3-406d-47a4-9757-eb58c54756be None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:45.787189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-34cea9f3-406d-47a4-9757-eb58c54756be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:45.796853 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2051/4038] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:45.813273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83301237-f717-4c89-b460-d5cee0d28cdc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:45.849007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83301237-f717-4c89-b460-d5cee0d28cdc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:45.856442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638be5f-88de-435f-8634-351505b6bc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:45.857424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638be5f-88de-435f-8634-351505b6bc9a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:45.857962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638be5f-88de-435f-8634-351505b6bc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:45.859310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638be5f-88de-435f-8634-351505b6bc9a admin admin] PATH_INFO: `/v3/projects/93c9e6370bb042109401b5e56c4d8e4e/users/7524192bcd7e4cc581b1fa43e7edf370/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:45.880101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638be5f-88de-435f-8634-351505b6bc9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c9e6370bb042109401b5e56c4d8e4e, user_id=7524192bcd7e4cc581b1fa43e7edf370, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:45.885000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638be5f-88de-435f-8634-351505b6bc9a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:45.909550 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2052/4039] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:45 2022] PUT /identity/v3/projects/93c9e6370bb042109401b5e56c4d8e4e/users/7524192bcd7e4cc581b1fa43e7edf370/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:45.911951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638be5f-88de-435f-8634-351505b6bc9a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:45.919338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cea9f3-406d-47a4-9757-eb58c54756be None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:46.002697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cc491-074d-4dfa-b663-7570b32841d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:46.003347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cc491-074d-4dfa-b663-7570b32841d7 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.003639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cc491-074d-4dfa-b663-7570b32841d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.004592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cc491-074d-4dfa-b663-7570b32841d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.018473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cc491-074d-4dfa-b663-7570b32841d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:46.019276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638be5f-88de-435f-8634-351505b6bc9a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:46.021590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cc491-074d-4dfa-b663-7570b32841d7 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:46.024854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08a243-ba43-4802-a3b8-f877d86f1358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:46.025699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08a243-ba43-4802-a3b8-f877d86f1358 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.026797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08a243-ba43-4802-a3b8-f877d86f1358 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.027191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08a243-ba43-4802-a3b8-f877d86f1358 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.027954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08a243-ba43-4802-a3b8-f877d86f1358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:46.030606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08a243-ba43-4802-a3b8-f877d86f1358 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:46.035585 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1988/4040] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:46.043946 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2053/4041] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:02:46.048808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cc491-074d-4dfa-b663-7570b32841d7 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:46.059826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cc491-074d-4dfa-b663-7570b32841d7 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:46.065372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b05c36-0c9b-41cb-a81f-1a07a603fa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:46.066440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b05c36-0c9b-41cb-a81f-1a07a603fa8f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.066861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b05c36-0c9b-41cb-a81f-1a07a603fa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.067206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b05c36-0c9b-41cb-a81f-1a07a603fa8f admin admin] PATH_INFO: `/v3/projects/93c9e6370bb042109401b5e56c4d8e4e/users/7524192bcd7e4cc581b1fa43e7edf370/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.069464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b05c36-0c9b-41cb-a81f-1a07a603fa8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c9e6370bb042109401b5e56c4d8e4e, user_id=7524192bcd7e4cc581b1fa43e7edf370, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:46.072382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b05c36-0c9b-41cb-a81f-1a07a603fa8f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:46.089054 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1989/4042] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:46 2022] PUT /identity/v3/projects/93c9e6370bb042109401b5e56c4d8e4e/users/7524192bcd7e4cc581b1fa43e7edf370/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:46.098797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8d759-da2d-4705-a00d-c9f07d9683a8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.099343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8d759-da2d-4705-a00d-c9f07d9683a8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.099871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8d759-da2d-4705-a00d-c9f07d9683a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.161361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-80f8d759-da2d-4705-a00d-c9f07d9683a8 None None] MFA Rules not processed for user `7524192bcd7e4cc581b1fa43e7edf370`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:46.164318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-80f8d759-da2d-4705-a00d-c9f07d9683a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:46.236781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:46.268954 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2054/4043] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:46.274194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b05c36-0c9b-41cb-a81f-1a07a603fa8f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:46.289581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b05c36-0c9b-41cb-a81f-1a07a603fa8f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:46.299295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f84e3a-a1c6-40e8-9e6f-fbbefcfa3613 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:46.299943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f84e3a-a1c6-40e8-9e6f-fbbefcfa3613 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.300235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f84e3a-a1c6-40e8-9e6f-fbbefcfa3613 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.300522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f84e3a-a1c6-40e8-9e6f-fbbefcfa3613 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.301244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f84e3a-a1c6-40e8-9e6f-fbbefcfa3613 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:46.302264 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-78f84e3a-a1c6-40e8-9e6f-fbbefcfa3613 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:46.302264 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:46.302264 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:46.303169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f84e3a-a1c6-40e8-9e6f-fbbefcfa3613 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:46.324959 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1990/4044] 10.222.0.16 () {60 vars in 1185 bytes} [Sat Sep 3 21:02:46 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 265 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:46.327115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8d759-da2d-4705-a00d-c9f07d9683a8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:46.338881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8d759-da2d-4705-a00d-c9f07d9683a8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:46.344122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0533bc-ae7a-438b-88dc-85b1b59e82a8 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:46.344931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0533bc-ae7a-438b-88dc-85b1b59e82a8 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.345311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0533bc-ae7a-438b-88dc-85b1b59e82a8 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.346058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0533bc-ae7a-438b-88dc-85b1b59e82a8 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/4ad265f9b3074f3983a96e425d8fc493` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.346754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0533bc-ae7a-438b-88dc-85b1b59e82a8 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=4ad265f9b3074f3983a96e425d8fc493)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:46.347946 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-7a0533bc-ae7a-438b-88dc-85b1b59e82a8 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:46.347946 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:46.347946 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:46.349207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0533bc-ae7a-438b-88dc-85b1b59e82a8 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:46.361519 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2055/4045] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Sep 3 21:02:46 2022] GET /identity/v3/OS-OAUTH1/consumers/4ad265f9b3074f3983a96e425d8fc493 => generated 219 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:46.362165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f84e3a-a1c6-40e8-9e6f-fbbefcfa3613 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:46.382158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f84e3a-a1c6-40e8-9e6f-fbbefcfa3613 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:46.400165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:46.400838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.403507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.403819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/4ad265f9b3074f3983a96e425d8fc493` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.404826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=4ad265f9b3074f3983a96e425d8fc493)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:46.406297 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:46.406297 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:46.406297 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:46.407273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:46.407577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invalidating token cache because consumer 4ad265f9b3074f3983a96e425d8fc493 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:46.407917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:46.408679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:46.431193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '4ad265f9b3074f3983a96e425d8fc493', 'request_id': 'req-a449276d-c8b1-4014-9041-cf5707308511'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:46.453219 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1991/4046] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Sep 3 21:02:46 2022] DELETE /identity/v3/OS-OAUTH1/consumers/4ad265f9b3074f3983a96e425d8fc493 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:46.454804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0533bc-ae7a-438b-88dc-85b1b59e82a8 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:46.457835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7a0533bc-ae7a-438b-88dc-85b1b59e82a8 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:46.486065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0533bc-ae7a-438b-88dc-85b1b59e82a8 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:46.493249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bbf30-024c-40e7-8f14-ec0c311eba54 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:46.494176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bbf30-024c-40e7-8f14-ec0c311eba54 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.495629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bbf30-024c-40e7-8f14-ec0c311eba54 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.495957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bbf30-024c-40e7-8f14-ec0c311eba54 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.496619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bbf30-024c-40e7-8f14-ec0c311eba54 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:46.497703 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-779bbf30-024c-40e7-8f14-ec0c311eba54 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:46.497703 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:46.497703 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:46.499425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bbf30-024c-40e7-8f14-ec0c311eba54 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:46.543955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:46.545892 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2056/4047] 10.222.0.16 () {60 vars in 1185 bytes} [Sat Sep 3 21:02:46 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:46.578302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449276d-c8b1-4014-9041-cf5707308511 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:46.592785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea3811-1eb3-47a5-ac2c-4596c465c44a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:46.593490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea3811-1eb3-47a5-ac2c-4596c465c44a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.593777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea3811-1eb3-47a5-ac2c-4596c465c44a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.594058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea3811-1eb3-47a5-ac2c-4596c465c44a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/9092f098c1884909b505c15973dc130e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.595071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea3811-1eb3-47a5-ac2c-4596c465c44a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=9092f098c1884909b505c15973dc130e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:46.603176 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-ceea3811-1eb3-47a5-ac2c-4596c465c44a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:46.603176 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:46.603176 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:46.605210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea3811-1eb3-47a5-ac2c-4596c465c44a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:46.619346 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1992/4048] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Sep 3 21:02:46 2022] GET /identity/v3/OS-OAUTH1/consumers/9092f098c1884909b505c15973dc130e => generated 220 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:46.624846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bbf30-024c-40e7-8f14-ec0c311eba54 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:46.643395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bbf30-024c-40e7-8f14-ec0c311eba54 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:46.649391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:46.650209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.650621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.651688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/9092f098c1884909b505c15973dc130e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.652747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=9092f098c1884909b505c15973dc130e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:46.658664 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:46.658664 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:02:46.658664 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:02:46.659630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:46.659911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invalidating token cache because consumer 9092f098c1884909b505c15973dc130e has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:46.660234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:46.661728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:46.681721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '9092f098c1884909b505c15973dc130e', 'request_id': 'req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:46.705658 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2057/4049] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Sep 3 21:02:46 2022] DELETE /identity/v3/OS-OAUTH1/consumers/9092f098c1884909b505c15973dc130e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:46.709720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea3811-1eb3-47a5-ac2c-4596c465c44a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:46.715210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ceea3811-1eb3-47a5-ac2c-4596c465c44a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:46.747302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea3811-1eb3-47a5-ac2c-4596c465c44a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:46.753879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fe219-73f6-42ab-b25a-e284cc1913c6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:46.754560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fe219-73f6-42ab-b25a-e284cc1913c6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.754850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fe219-73f6-42ab-b25a-e284cc1913c6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.755134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fe219-73f6-42ab-b25a-e284cc1913c6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/9092f098c1884909b505c15973dc130e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.755734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fe219-73f6-42ab-b25a-e284cc1913c6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=9092f098c1884909b505c15973dc130e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:46.757389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fe219-73f6-42ab-b25a-e284cc1913c6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:46.761045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-024fe219-73f6-42ab-b25a-e284cc1913c6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 100, in get Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_consumer(consumer_id) Sep 03 21:02:46.784988 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 95, in _get_consumer Sep 03 21:02:46.786206 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.oauth_api.get_consumer(consumer_id)) Sep 03 21:02:46.786206 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 101, in get_consumer Sep 03 21:02:46.786206 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.get_consumer_with_secret(consumer_id)) Sep 03 21:02:46.786206 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 96, in get_consumer_with_secret Sep 03 21:02:46.786206 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Sep 03 21:02:46.786206 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Sep 03 21:02:46.786206 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Sep 03 21:02:46.786206 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Sep 03 21:02:46.786206 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:46.788766 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1993/4050] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Sep 3 21:02:46 2022] GET /identity/v3/OS-OAUTH1/consumers/9092f098c1884909b505c15973dc130e => generated 74 bytes in 80 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 21:02:46.793909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:46.807392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccb7c0-cbb3-4af3-a24d-f6bb11e3747a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:46.814133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d638f-2041-40f2-88da-83f986913f98 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:46.814692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d638f-2041-40f2-88da-83f986913f98 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.815194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d638f-2041-40f2-88da-83f986913f98 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.817544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d638f-2041-40f2-88da-83f986913f98 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/9092f098c1884909b505c15973dc130e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.817809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d638f-2041-40f2-88da-83f986913f98 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=9092f098c1884909b505c15973dc130e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:46.820196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d638f-2041-40f2-88da-83f986913f98 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:46.820196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-630d638f-2041-40f2-88da-83f986913f98 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invalidating token cache because consumer 9092f098c1884909b505c15973dc130e has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:46.820938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-630d638f-2041-40f2-88da-83f986913f98 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:46.821507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-630d638f-2041-40f2-88da-83f986913f98 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-630d638f-2041-40f2-88da-83f986913f98 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 123, in delete Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application PROVIDERS.oauth_api.delete_consumer( Sep 03 21:02:46.845508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:02:46.847130 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:02:46.847130 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/core.py", line 139, in delete_consumer Sep 03 21:02:46.847130 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ret = self.driver.delete_consumer(consumer_id) Sep 03 21:02:46.847130 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 133, in delete_consumer Sep 03 21:02:46.847130 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._delete_consumer(session, consumer_id) Sep 03 21:02:46.847130 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 110, in _delete_consumer Sep 03 21:02:46.847130 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Sep 03 21:02:46.847130 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Sep 03 21:02:46.847130 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Sep 03 21:02:46.847130 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Sep 03 21:02:46.847130 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:02:46.859672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fe219-73f6-42ab-b25a-e284cc1913c6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:46.861681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-024fe219-73f6-42ab-b25a-e284cc1913c6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:46.877133 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2058/4051] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Sep 3 21:02:46 2022] DELETE /identity/v3/OS-OAUTH1/consumers/9092f098c1884909b505c15973dc130e => generated 74 bytes in 66 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 21:02:46.907090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fe219-73f6-42ab-b25a-e284cc1913c6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:46.921028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f28df0-0c10-4e92-83bb-d697239e0fd6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:46.923579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f28df0-0c10-4e92-83bb-d697239e0fd6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:46.924747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f28df0-0c10-4e92-83bb-d697239e0fd6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:46.925509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f28df0-0c10-4e92-83bb-d697239e0fd6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:46.928422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f28df0-0c10-4e92-83bb-d697239e0fd6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:46.934271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f28df0-0c10-4e92-83bb-d697239e0fd6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:46.973844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d638f-2041-40f2-88da-83f986913f98 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:46.991134 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1994/4052] 10.222.0.16 () {60 vars in 1185 bytes} [Sat Sep 3 21:02:46 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:46.997743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d638f-2041-40f2-88da-83f986913f98 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:47.005223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbbe4a-d924-4bfb-bd10-4cf47af0e4d0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:47.006432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbbe4a-d924-4bfb-bd10-4cf47af0e4d0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:47.007046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbbe4a-d924-4bfb-bd10-4cf47af0e4d0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:47.007505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbbe4a-d924-4bfb-bd10-4cf47af0e4d0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:47.008682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbbe4a-d924-4bfb-bd10-4cf47af0e4d0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:47.010759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbbe4a-d924-4bfb-bd10-4cf47af0e4d0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:47.042649 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2059/4053] 10.222.0.16 () {60 vars in 1185 bytes} [Sat Sep 3 21:02:46 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:47.046229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f28df0-0c10-4e92-83bb-d697239e0fd6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:47.059980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f28df0-0c10-4e92-83bb-d697239e0fd6 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:47.064829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afb2cc-c2a2-4501-9efd-dab3d382a94a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:47.065502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afb2cc-c2a2-4501-9efd-dab3d382a94a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:47.065961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afb2cc-c2a2-4501-9efd-dab3d382a94a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:47.066398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afb2cc-c2a2-4501-9efd-dab3d382a94a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:47.066988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afb2cc-c2a2-4501-9efd-dab3d382a94a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:list_consumers()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:47.068402 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-24afb2cc-c2a2-4501-9efd-dab3d382a94a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_consumers": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:47.068402 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:47.068402 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:47.069932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afb2cc-c2a2-4501-9efd-dab3d382a94a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:47.081106 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1995/4054] 10.222.0.16 () {58 vars in 1164 bytes} [Sat Sep 3 21:02:47 2022] GET /identity/v3/OS-OAUTH1/consumers => generated 536 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:47.085845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbbe4a-d924-4bfb-bd10-4cf47af0e4d0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:47.099410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbbe4a-d924-4bfb-bd10-4cf47af0e4d0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:47.106550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1143b85-0aa5-4650-9012-7ae8946fd36c tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:47.107211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1143b85-0aa5-4650-9012-7ae8946fd36c tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:47.107502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1143b85-0aa5-4650-9012-7ae8946fd36c tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:47.107785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1143b85-0aa5-4650-9012-7ae8946fd36c tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/fc9b3c6f9a6f46e7a566bdbf46bc3c2c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:47.109181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1143b85-0aa5-4650-9012-7ae8946fd36c tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=fc9b3c6f9a6f46e7a566bdbf46bc3c2c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:47.111396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1143b85-0aa5-4650-9012-7ae8946fd36c tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:47.111655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b1143b85-0aa5-4650-9012-7ae8946fd36c tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invalidating token cache because consumer fc9b3c6f9a6f46e7a566bdbf46bc3c2c has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:47.112181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b1143b85-0aa5-4650-9012-7ae8946fd36c tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.113746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b1143b85-0aa5-4650-9012-7ae8946fd36c tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.144219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b1143b85-0aa5-4650-9012-7ae8946fd36c tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'fc9b3c6f9a6f46e7a566bdbf46bc3c2c', 'request_id': 'req-b1143b85-0aa5-4650-9012-7ae8946fd36c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.164747 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2060/4055] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Sep 3 21:02:47 2022] DELETE /identity/v3/OS-OAUTH1/consumers/fc9b3c6f9a6f46e7a566bdbf46bc3c2c => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:47.170069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afb2cc-c2a2-4501-9efd-dab3d382a94a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:47.171907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-24afb2cc-c2a2-4501-9efd-dab3d382a94a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:47.189180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afb2cc-c2a2-4501-9efd-dab3d382a94a tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:47.195135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:47.196222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:47.196222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:47.196449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/2a1d6265581c41f19a7f015d48330488` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:47.196961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=2a1d6265581c41f19a7f015d48330488)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:47.199001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:47.199001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invalidating token cache because consumer 2a1d6265581c41f19a7f015d48330488 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:47.199223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.199877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.216634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '2a1d6265581c41f19a7f015d48330488', 'request_id': 'req-858ae0d0-9d8b-4393-ac76-5655021f27b4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.240351 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1996/4056] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Sep 3 21:02:47 2022] DELETE /identity/v3/OS-OAUTH1/consumers/2a1d6265581c41f19a7f015d48330488 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:47.246058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1143b85-0aa5-4650-9012-7ae8946fd36c tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:47.256943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1143b85-0aa5-4650-9012-7ae8946fd36c tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:47.261942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532327b6-3b00-4953-8260-ded1a0289b24 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:47.264856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532327b6-3b00-4953-8260-ded1a0289b24 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:47.265445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532327b6-3b00-4953-8260-ded1a0289b24 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:47.265770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532327b6-3b00-4953-8260-ded1a0289b24 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:47.267534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532327b6-3b00-4953-8260-ded1a0289b24 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:47.269262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532327b6-3b00-4953-8260-ded1a0289b24 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:47.295413 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2061/4057] 10.222.0.16 () {60 vars in 1185 bytes} [Sat Sep 3 21:02:47 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:47.295698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:47.297619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:47.313975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ae0d0-9d8b-4393-ac76-5655021f27b4 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:47.319809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4872c8f-300d-4421-8fef-984be557c1a0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:47.320522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4872c8f-300d-4421-8fef-984be557c1a0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:47.320818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4872c8f-300d-4421-8fef-984be557c1a0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:47.321146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4872c8f-300d-4421-8fef-984be557c1a0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/43117152e61746dfba656dcd650ebd40` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:47.321882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4872c8f-300d-4421-8fef-984be557c1a0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:update_consumer(consumer_id=43117152e61746dfba656dcd650ebd40)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:47.323099 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-a4872c8f-300d-4421-8fef-984be557c1a0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:02:47.323099 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:02:47.323099 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:02:47.324044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4872c8f-300d-4421-8fef-984be557c1a0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:47.349849 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1997/4058] 10.222.0.16 () {60 vars in 1285 bytes} [Sat Sep 3 21:02:47 2022] PATCH /identity/v3/OS-OAUTH1/consumers/43117152e61746dfba656dcd650ebd40 => generated 220 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:02:47.351946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532327b6-3b00-4953-8260-ded1a0289b24 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:47.367124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532327b6-3b00-4953-8260-ded1a0289b24 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:47.374326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94e370-7171-492d-b241-9d2233323566 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:47.375336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94e370-7171-492d-b241-9d2233323566 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:47.376193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94e370-7171-492d-b241-9d2233323566 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:47.377375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94e370-7171-492d-b241-9d2233323566 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/43117152e61746dfba656dcd650ebd40` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:47.378209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94e370-7171-492d-b241-9d2233323566 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=43117152e61746dfba656dcd650ebd40)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:47.380026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94e370-7171-492d-b241-9d2233323566 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:47.394316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4872c8f-300d-4421-8fef-984be557c1a0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:47.395096 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2062/4059] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Sep 3 21:02:47 2022] GET /identity/v3/OS-OAUTH1/consumers/43117152e61746dfba656dcd650ebd40 => generated 220 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:47.408709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4872c8f-300d-4421-8fef-984be557c1a0 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:47.423170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47f083-8111-410d-9d97-17b72125664f tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7524192bcd7e4cc581b1fa43e7edf370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93c9e6370bb042109401b5e56c4d8e4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:47.424141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47f083-8111-410d-9d97-17b72125664f tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:47.424607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47f083-8111-410d-9d97-17b72125664f tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:47.425084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47f083-8111-410d-9d97-17b72125664f tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/43117152e61746dfba656dcd650ebd40` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:47.425881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47f083-8111-410d-9d97-17b72125664f tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=43117152e61746dfba656dcd650ebd40)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:47.428249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47f083-8111-410d-9d97-17b72125664f tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:47.428813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-aa47f083-8111-410d-9d97-17b72125664f tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invalidating token cache because consumer 43117152e61746dfba656dcd650ebd40 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:47.429228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-aa47f083-8111-410d-9d97-17b72125664f tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.429839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-aa47f083-8111-410d-9d97-17b72125664f tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.442645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-aa47f083-8111-410d-9d97-17b72125664f tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '43117152e61746dfba656dcd650ebd40', 'request_id': 'req-aa47f083-8111-410d-9d97-17b72125664f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.468858 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1998/4060] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Sep 3 21:02:47 2022] DELETE /identity/v3/OS-OAUTH1/consumers/43117152e61746dfba656dcd650ebd40 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:47.474710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94e370-7171-492d-b241-9d2233323566 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:47.478710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5e94e370-7171-492d-b241-9d2233323566 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:47.590005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94e370-7171-492d-b241-9d2233323566 tempest-OAUTHConsumersV3Test-730928802 tempest-OAUTHConsumersV3Test-730928802-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:47.598166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:47.601193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:47.601723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:47.602287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] PATH_INFO: `/v3/users/82f898e9c45e4b509f250c2b95e97e26` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:47.605488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82f898e9c45e4b509f250c2b95e97e26)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:47.612460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:47.664174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:47.700817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82f898e9c45e4b509f250c2b95e97e26'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.737855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82f898e9c45e4b509f250c2b95e97e26'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.749184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82f898e9c45e4b509f250c2b95e97e26'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.750134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82f898e9c45e4b509f250c2b95e97e26'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.751521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82f898e9c45e4b509f250c2b95e97e26'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:47.768577 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2063/4061] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:47 2022] DELETE /identity/v3/users/82f898e9c45e4b509f250c2b95e97e26 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:47.772935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b011aaf-ec1d-444a-9f2e-e33cc452cb56 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:47.773228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b011aaf-ec1d-444a-9f2e-e33cc452cb56 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:47.773506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b011aaf-ec1d-444a-9f2e-e33cc452cb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:47.851614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7b011aaf-ec1d-444a-9f2e-e33cc452cb56 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:47.852773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7b011aaf-ec1d-444a-9f2e-e33cc452cb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:48.005189 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 1999/4062] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:48.018591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:48.020942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:48.178346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f2186-d837-4800-bac8-72886b464a57 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:48.186065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7808f08-e691-4ad4-b5d2-f9dd54bd5ec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:48.187129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7808f08-e691-4ad4-b5d2-f9dd54bd5ec4 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:48.187479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7808f08-e691-4ad4-b5d2-f9dd54bd5ec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:48.187950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7808f08-e691-4ad4-b5d2-f9dd54bd5ec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:48.200829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7808f08-e691-4ad4-b5d2-f9dd54bd5ec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:48.202603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7808f08-e691-4ad4-b5d2-f9dd54bd5ec4 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:48.218346 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2064/4063] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:48.911538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b011aaf-ec1d-444a-9f2e-e33cc452cb56 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:48.913023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7b011aaf-ec1d-444a-9f2e-e33cc452cb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:48.933234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b011aaf-ec1d-444a-9f2e-e33cc452cb56 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:48.945056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:48.946223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:48.946734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:48.947319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] PATH_INFO: `/v3/projects/1d98c4cc6666426188573a9ac45c51ba` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:48.962044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d98c4cc6666426188573a9ac45c51ba)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:48.983262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:49.076386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d98c4cc6666426188573a9ac45c51ba', 'request_id': 'req-016603b5-16f4-47a6-ad89-624de28dafa6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:49.092101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d98c4cc6666426188573a9ac45c51ba', 'request_id': 'req-016603b5-16f4-47a6-ad89-624de28dafa6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:49.109916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d98c4cc6666426188573a9ac45c51ba', 'request_id': 'req-016603b5-16f4-47a6-ad89-624de28dafa6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:49.115632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=10,keystone:UPDATE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:49.161390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] The token cache is being invalidate because project 1d98c4cc6666426188573a9ac45c51ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:49.162170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:49.163379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:49.167522 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2000/4064] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:48 2022] DELETE /identity/v3/projects/1d98c4cc6666426188573a9ac45c51ba => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:49.171539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7808f08-e691-4ad4-b5d2-f9dd54bd5ec4 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:49.188798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7808f08-e691-4ad4-b5d2-f9dd54bd5ec4 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:49.306666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c2302-eba9-4b72-bb6c-1a2985f5747b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:49.307683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c2302-eba9-4b72-bb6c-1a2985f5747b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:49.308028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c2302-eba9-4b72-bb6c-1a2985f5747b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:49.308387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c2302-eba9-4b72-bb6c-1a2985f5747b admin admin] PATH_INFO: `/v3/users/7524192bcd7e4cc581b1fa43e7edf370` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:49.309613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c2302-eba9-4b72-bb6c-1a2985f5747b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7524192bcd7e4cc581b1fa43e7edf370)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:49.312551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c2302-eba9-4b72-bb6c-1a2985f5747b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:49.385584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:49.390302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a84c2302-eba9-4b72-bb6c-1a2985f5747b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7524192bcd7e4cc581b1fa43e7edf370'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:49.409596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a84c2302-eba9-4b72-bb6c-1a2985f5747b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7524192bcd7e4cc581b1fa43e7edf370'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:49.425322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a84c2302-eba9-4b72-bb6c-1a2985f5747b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7524192bcd7e4cc581b1fa43e7edf370'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:49.426118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a84c2302-eba9-4b72-bb6c-1a2985f5747b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7524192bcd7e4cc581b1fa43e7edf370'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:49.427307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a84c2302-eba9-4b72-bb6c-1a2985f5747b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7524192bcd7e4cc581b1fa43e7edf370'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:49.448942 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2065/4065] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:49 2022] DELETE /identity/v3/users/7524192bcd7e4cc581b1fa43e7edf370 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:51.289940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:51.291444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:51.308720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016603b5-16f4-47a6-ad89-624de28dafa6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:51.372864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df46ef9-8314-47fc-8919-4be702b709d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:51.375420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df46ef9-8314-47fc-8919-4be702b709d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:51.376912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df46ef9-8314-47fc-8919-4be702b709d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:51.377431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df46ef9-8314-47fc-8919-4be702b709d3 admin admin] PATH_INFO: `/v3/projects/93c9e6370bb042109401b5e56c4d8e4e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:51.378882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df46ef9-8314-47fc-8919-4be702b709d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c9e6370bb042109401b5e56c4d8e4e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:51.381051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df46ef9-8314-47fc-8919-4be702b709d3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:51.437367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2df46ef9-8314-47fc-8919-4be702b709d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c9e6370bb042109401b5e56c4d8e4e', 'request_id': 'req-2df46ef9-8314-47fc-8919-4be702b709d3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:51.446206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2df46ef9-8314-47fc-8919-4be702b709d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c9e6370bb042109401b5e56c4d8e4e', 'request_id': 'req-2df46ef9-8314-47fc-8919-4be702b709d3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:51.455806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2df46ef9-8314-47fc-8919-4be702b709d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c9e6370bb042109401b5e56c4d8e4e', 'request_id': 'req-2df46ef9-8314-47fc-8919-4be702b709d3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:51.467085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2df46ef9-8314-47fc-8919-4be702b709d3 admin admin] The token cache is being invalidate because project 93c9e6370bb042109401b5e56c4d8e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:51.467457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2df46ef9-8314-47fc-8919-4be702b709d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:51.468200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2df46ef9-8314-47fc-8919-4be702b709d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:51.484325 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2001/4066] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:51 2022] DELETE /identity/v3/projects/93c9e6370bb042109401b5e56c4d8e4e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:51.487117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e2190-3fe5-4c1b-8c88-152546ac26be None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:51.488112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e2190-3fe5-4c1b-8c88-152546ac26be None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:51.488700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e2190-3fe5-4c1b-8c88-152546ac26be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:51.519043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-774e2190-3fe5-4c1b-8c88-152546ac26be None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:51.521117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-774e2190-3fe5-4c1b-8c88-152546ac26be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:51.699305 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2066/4067] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:02:51.707941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb1fb4-3cc2-4ce7-bca2-c4bf59a9cae4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:51.708531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb1fb4-3cc2-4ce7-bca2-c4bf59a9cae4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:51.709264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb1fb4-3cc2-4ce7-bca2-c4bf59a9cae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:51.750017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d3eb1fb4-3cc2-4ce7-bca2-c4bf59a9cae4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:51.753796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d3eb1fb4-3cc2-4ce7-bca2-c4bf59a9cae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:51.777958 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2002/4068] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:51.779720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e2190-3fe5-4c1b-8c88-152546ac26be None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:51.829452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e2190-3fe5-4c1b-8c88-152546ac26be None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:51.844999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3155f-50fc-4e01-8e65-7aa33ab0ad78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:51.846196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3155f-50fc-4e01-8e65-7aa33ab0ad78 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:51.846912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3155f-50fc-4e01-8e65-7aa33ab0ad78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:51.850181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3155f-50fc-4e01-8e65-7aa33ab0ad78 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:51.862537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3155f-50fc-4e01-8e65-7aa33ab0ad78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:51.863732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3155f-50fc-4e01-8e65-7aa33ab0ad78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:51.868570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3155f-50fc-4e01-8e65-7aa33ab0ad78 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:51.914497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb1fb4-3cc2-4ce7-bca2-c4bf59a9cae4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:51.928998 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2067/4069] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:51.931066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb1fb4-3cc2-4ce7-bca2-c4bf59a9cae4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:51.939789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e2933-5990-42d1-a023-4adeda145385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:51.940803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e2933-5990-42d1-a023-4adeda145385 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:51.941319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e2933-5990-42d1-a023-4adeda145385 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:51.941749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e2933-5990-42d1-a023-4adeda145385 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:51.942831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e2933-5990-42d1-a023-4adeda145385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:51.945394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e2933-5990-42d1-a023-4adeda145385 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:51.987023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3155f-50fc-4e01-8e65-7aa33ab0ad78 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:51.993790 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2003/4070] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:51 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:52.003533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3155f-50fc-4e01-8e65-7aa33ab0ad78 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:52.170766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb4b3e-7758-414a-868f-f81eb42934cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:52.171381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb4b3e-7758-414a-868f-f81eb42934cd admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:52.171813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb4b3e-7758-414a-868f-f81eb42934cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:52.172066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb4b3e-7758-414a-868f-f81eb42934cd admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:52.176808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb4b3e-7758-414a-868f-f81eb42934cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:52.183644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb4b3e-7758-414a-868f-f81eb42934cd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:52.257617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e2933-5990-42d1-a023-4adeda145385 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:52.266054 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2068/4071] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:51 2022] POST /identity/v3/users => generated 292 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:52.270329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e2933-5990-42d1-a023-4adeda145385 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:52.277462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4339e7-fac7-4d61-bfa2-8be2de16a816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:52.278312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4339e7-fac7-4d61-bfa2-8be2de16a816 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:52.278706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4339e7-fac7-4d61-bfa2-8be2de16a816 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:52.279248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4339e7-fac7-4d61-bfa2-8be2de16a816 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:52.279995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4339e7-fac7-4d61-bfa2-8be2de16a816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:52.282267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4339e7-fac7-4d61-bfa2-8be2de16a816 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:52.319730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb4b3e-7758-414a-868f-f81eb42934cd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:52.333334 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2004/4072] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:52.337929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb4b3e-7758-414a-868f-f81eb42934cd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:52.352847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c43f0-fb98-4581-bb9e-f1dc43d82c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:52.354618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c43f0-fb98-4581-bb9e-f1dc43d82c39 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:52.355163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c43f0-fb98-4581-bb9e-f1dc43d82c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:52.356874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c43f0-fb98-4581-bb9e-f1dc43d82c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:52.360422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c43f0-fb98-4581-bb9e-f1dc43d82c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:52.368221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c43f0-fb98-4581-bb9e-f1dc43d82c39 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:52.389807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4339e7-fac7-4d61-bfa2-8be2de16a816 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:52.390846 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2069/4073] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:52.414053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4339e7-fac7-4d61-bfa2-8be2de16a816 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:52.424076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898b288-ac04-4839-a031-78346a6ae5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:52.424973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898b288-ac04-4839-a031-78346a6ae5e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:52.425372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898b288-ac04-4839-a031-78346a6ae5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:52.425893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898b288-ac04-4839-a031-78346a6ae5e2 admin admin] PATH_INFO: `/v3/projects/68d55139c0794917875dc6f553f70c20/users/42c2943d3ab742afa429ba779150bb23/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:52.457281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898b288-ac04-4839-a031-78346a6ae5e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d55139c0794917875dc6f553f70c20, user_id=42c2943d3ab742afa429ba779150bb23, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:52.460719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898b288-ac04-4839-a031-78346a6ae5e2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:52.500092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa85af-c7e4-4107-8c88-f9926285e32d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:52.500595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa85af-c7e4-4107-8c88-f9926285e32d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:52.501012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa85af-c7e4-4107-8c88-f9926285e32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:52.510864 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2005/4074] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:52 2022] PUT /identity/v3/projects/68d55139c0794917875dc6f553f70c20/users/42c2943d3ab742afa429ba779150bb23/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:52.590939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-29aa85af-c7e4-4107-8c88-f9926285e32d None None] MFA Rules not processed for user `42c2943d3ab742afa429ba779150bb23`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:52.595367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-29aa85af-c7e4-4107-8c88-f9926285e32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:52.638458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:52.757541 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2070/4075] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:52.762410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898b288-ac04-4839-a031-78346a6ae5e2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:52.823304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:52.896190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898b288-ac04-4839-a031-78346a6ae5e2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:52.901839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c4f52-58a2-461b-8bfe-9c542a963b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:52.902472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c4f52-58a2-461b-8bfe-9c542a963b81 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:52.902761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c4f52-58a2-461b-8bfe-9c542a963b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:52.903044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c4f52-58a2-461b-8bfe-9c542a963b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:52.903689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c4f52-58a2-461b-8bfe-9c542a963b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:52.906963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c4f52-58a2-461b-8bfe-9c542a963b81 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:52.957757 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2006/4076] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:52 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:52.961531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa85af-c7e4-4107-8c88-f9926285e32d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:52.987808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa85af-c7e4-4107-8c88-f9926285e32d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:53.101495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a62a36-b2b0-413a-9fe7-ab1e3ae97b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:53.102803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a62a36-b2b0-413a-9fe7-ab1e3ae97b1f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:53.103100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a62a36-b2b0-413a-9fe7-ab1e3ae97b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:53.103385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a62a36-b2b0-413a-9fe7-ab1e3ae97b1f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:53.104247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a62a36-b2b0-413a-9fe7-ab1e3ae97b1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:53.107535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a62a36-b2b0-413a-9fe7-ab1e3ae97b1f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:53.174886 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2071/4077] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:52 2022] POST /identity/v3/users => generated 299 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:53.176449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c4f52-58a2-461b-8bfe-9c542a963b81 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:53.192591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c4f52-58a2-461b-8bfe-9c542a963b81 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:53.203593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9faf8-d05d-40d3-911a-73ce6380ca60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:53.204848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9faf8-d05d-40d3-911a-73ce6380ca60 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:53.205372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9faf8-d05d-40d3-911a-73ce6380ca60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:53.206465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9faf8-d05d-40d3-911a-73ce6380ca60 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:53.206465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9faf8-d05d-40d3-911a-73ce6380ca60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:53.208618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9faf8-d05d-40d3-911a-73ce6380ca60 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:53.226606 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2007/4078] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:53.227266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a62a36-b2b0-413a-9fe7-ab1e3ae97b1f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:53.253487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a62a36-b2b0-413a-9fe7-ab1e3ae97b1f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:53.291367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c9238-1e33-46b6-9eff-1e93d7b22421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:53.292509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c9238-1e33-46b6-9eff-1e93d7b22421 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:53.293243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c9238-1e33-46b6-9eff-1e93d7b22421 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:53.293888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c9238-1e33-46b6-9eff-1e93d7b22421 admin admin] PATH_INFO: `/v3/projects/33046660bb1146348f704884dd3256e5/users/8af1cdee3b3b4a829a5dd6698a660812/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:53.346201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c9238-1e33-46b6-9eff-1e93d7b22421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33046660bb1146348f704884dd3256e5, user_id=8af1cdee3b3b4a829a5dd6698a660812, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:53.352139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c9238-1e33-46b6-9eff-1e93d7b22421 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:53.373788 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2072/4079] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:53 2022] PUT /identity/v3/projects/33046660bb1146348f704884dd3256e5/users/8af1cdee3b3b4a829a5dd6698a660812/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:53.377525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9faf8-d05d-40d3-911a-73ce6380ca60 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:53.395688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9faf8-d05d-40d3-911a-73ce6380ca60 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:53.556523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3fc77-b1d6-4ec4-844f-ade5e887c8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:53.559650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3fc77-b1d6-4ec4-844f-ade5e887c8aa admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:53.560819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3fc77-b1d6-4ec4-844f-ade5e887c8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:53.561311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3fc77-b1d6-4ec4-844f-ade5e887c8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:53.564192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3fc77-b1d6-4ec4-844f-ade5e887c8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:53.569726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3fc77-b1d6-4ec4-844f-ade5e887c8aa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:53.606132 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2008/4080] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:53.606454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c9238-1e33-46b6-9eff-1e93d7b22421 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:53.622883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c9238-1e33-46b6-9eff-1e93d7b22421 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:53.628401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80ffde-63a2-4f51-8f54-c11809427c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:53.629398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80ffde-63a2-4f51-8f54-c11809427c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:53.629691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80ffde-63a2-4f51-8f54-c11809427c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:53.629980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80ffde-63a2-4f51-8f54-c11809427c1b admin admin] PATH_INFO: `/v3/projects/33046660bb1146348f704884dd3256e5/users/8af1cdee3b3b4a829a5dd6698a660812/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:53.632530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80ffde-63a2-4f51-8f54-c11809427c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=33046660bb1146348f704884dd3256e5, user_id=8af1cdee3b3b4a829a5dd6698a660812, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:53.635883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80ffde-63a2-4f51-8f54-c11809427c1b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:53.659956 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2073/4081] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:53 2022] PUT /identity/v3/projects/33046660bb1146348f704884dd3256e5/users/8af1cdee3b3b4a829a5dd6698a660812/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:53.671007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d9e6d-6072-42a4-9ad8-2d3cddfe88bb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:53.672195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d9e6d-6072-42a4-9ad8-2d3cddfe88bb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:53.672730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d9e6d-6072-42a4-9ad8-2d3cddfe88bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:53.737856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ff4d9e6d-6072-42a4-9ad8-2d3cddfe88bb None None] MFA Rules not processed for user `8af1cdee3b3b4a829a5dd6698a660812`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:53.742220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ff4d9e6d-6072-42a4-9ad8-2d3cddfe88bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:53.879365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80ffde-63a2-4f51-8f54-c11809427c1b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:53.879802 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2009/4082] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:53.894986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80ffde-63a2-4f51-8f54-c11809427c1b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:53.902872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb63c0-b1fe-409c-ba23-8c7b6a4e15c0 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af1cdee3b3b4a829a5dd6698a660812', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33046660bb1146348f704884dd3256e5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:53.902872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb63c0-b1fe-409c-ba23-8c7b6a4e15c0 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:53.903318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb63c0-b1fe-409c-ba23-8c7b6a4e15c0 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:53.903714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb63c0-b1fe-409c-ba23-8c7b6a4e15c0 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:53.904427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb63c0-b1fe-409c-ba23-8c7b6a4e15c0 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:53.906981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb63c0-b1fe-409c-ba23-8c7b6a4e15c0 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:54.011431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d9e6d-6072-42a4-9ad8-2d3cddfe88bb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:54.035268 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2074/4083] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:53 2022] POST /identity/v3/projects => generated 360 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:54.053660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:54.164008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d9e6d-6072-42a4-9ad8-2d3cddfe88bb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:54.318017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d411ec-c00c-45b5-822c-6a48d47271fa tempest-ProjectsNegativeTestJSON-173771028 tempest-ProjectsNegativeTestJSON-173771028-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2943d3ab742afa429ba779150bb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d55139c0794917875dc6f553f70c20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:54.320930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d411ec-c00c-45b5-822c-6a48d47271fa tempest-ProjectsNegativeTestJSON-173771028 tempest-ProjectsNegativeTestJSON-173771028-project] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:54.320930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d411ec-c00c-45b5-822c-6a48d47271fa tempest-ProjectsNegativeTestJSON-173771028 tempest-ProjectsNegativeTestJSON-173771028-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:54.320930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d411ec-c00c-45b5-822c-6a48d47271fa tempest-ProjectsNegativeTestJSON-173771028 tempest-ProjectsNegativeTestJSON-173771028-project] PATH_INFO: `/v3/projects/49678ecb72394224be50ba1d38146469` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:54.323524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d411ec-c00c-45b5-822c-6a48d47271fa tempest-ProjectsNegativeTestJSON-173771028 tempest-ProjectsNegativeTestJSON-173771028-project] RBAC: Authorizing `identity:delete_project(project_id=49678ecb72394224be50ba1d38146469)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-86d411ec-c00c-45b5-822c-6a48d47271fa tempest-ProjectsNegativeTestJSON-173771028 tempest-ProjectsNegativeTestJSON-173771028-project] You are not authorized to perform the requested action: identity:delete_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 207, in delete Sep 03 21:02:54.339217 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 03 21:02:54.340444 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 03 21:02:54.340444 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 03 21:02:54.340444 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 03 21:02:54.340444 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 03 21:02:54.340444 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 03 21:02:54.340444 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 03 21:02:54.340444 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Sep 03 21:02:54.340444 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:02:54.340444 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2010/4084] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:54 2022] DELETE /identity/v3/projects/49678ecb72394224be50ba1d38146469 => generated 136 bytes in 331 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:02:54.346996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb63c0-b1fe-409c-ba23-8c7b6a4e15c0 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:54.446235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:54.478705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb63c0-b1fe-409c-ba23-8c7b6a4e15c0 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:54.485286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b8fb1-647c-496b-a989-9cba25437480 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af1cdee3b3b4a829a5dd6698a660812', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33046660bb1146348f704884dd3256e5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:54.486188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b8fb1-647c-496b-a989-9cba25437480 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:54.486524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b8fb1-647c-496b-a989-9cba25437480 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:54.486884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b8fb1-647c-496b-a989-9cba25437480 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] PATH_INFO: `/v3/projects/49678ecb72394224be50ba1d38146469` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:54.488477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b8fb1-647c-496b-a989-9cba25437480 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] RBAC: Authorizing `identity:delete_project(project_id=49678ecb72394224be50ba1d38146469)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:54.491521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b8fb1-647c-496b-a989-9cba25437480 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:54.553160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-988b8fb1-647c-496b-a989-9cba25437480 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49678ecb72394224be50ba1d38146469', 'request_id': 'req-988b8fb1-647c-496b-a989-9cba25437480'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:54.563137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-988b8fb1-647c-496b-a989-9cba25437480 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49678ecb72394224be50ba1d38146469', 'request_id': 'req-988b8fb1-647c-496b-a989-9cba25437480'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:54.576575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-988b8fb1-647c-496b-a989-9cba25437480 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49678ecb72394224be50ba1d38146469', 'request_id': 'req-988b8fb1-647c-496b-a989-9cba25437480'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:54.592797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-988b8fb1-647c-496b-a989-9cba25437480 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] The token cache is being invalidate because project 49678ecb72394224be50ba1d38146469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:54.593145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-988b8fb1-647c-496b-a989-9cba25437480 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:54.594293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-988b8fb1-647c-496b-a989-9cba25437480 tempest-ProjectsNegativeTestJSON-1879895511 tempest-ProjectsNegativeTestJSON-1879895511-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:54.596282 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2075/4085] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:54 2022] DELETE /identity/v3/projects/49678ecb72394224be50ba1d38146469 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:54.602138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d411ec-c00c-45b5-822c-6a48d47271fa tempest-ProjectsNegativeTestJSON-173771028 tempest-ProjectsNegativeTestJSON-173771028-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:54.603983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-86d411ec-c00c-45b5-822c-6a48d47271fa tempest-ProjectsNegativeTestJSON-173771028 tempest-ProjectsNegativeTestJSON-173771028-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:54.714995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d411ec-c00c-45b5-822c-6a48d47271fa tempest-ProjectsNegativeTestJSON-173771028 tempest-ProjectsNegativeTestJSON-173771028-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:54.820670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:54.821616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:54.822090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:54.822376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] PATH_INFO: `/v3/users/42c2943d3ab742afa429ba779150bb23` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:54.823984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c2943d3ab742afa429ba779150bb23)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:54.826351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:54.917545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:54.960459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c2943d3ab742afa429ba779150bb23'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:54.977064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c2943d3ab742afa429ba779150bb23'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:54.978058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c2943d3ab742afa429ba779150bb23'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:54.987406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c2943d3ab742afa429ba779150bb23'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:54.996412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c2943d3ab742afa429ba779150bb23'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:55.009151 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2011/4086] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:54 2022] DELETE /identity/v3/users/42c2943d3ab742afa429ba779150bb23 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:55.011383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e84956-e3ce-4058-9fde-4a3e63852782 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:55.011853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e84956-e3ce-4058-9fde-4a3e63852782 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:55.012074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e84956-e3ce-4058-9fde-4a3e63852782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:55.051138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d7e84956-e3ce-4058-9fde-4a3e63852782 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:55.053079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d7e84956-e3ce-4058-9fde-4a3e63852782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:55.170315 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2076/4087] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:02:55.192129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:55.194553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:55.371531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4e557-c418-4190-b87a-fc42467ec8ed admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:55.376258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5743b3-e3d5-4ce2-b772-f32c84886e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:55.377121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5743b3-e3d5-4ce2-b772-f32c84886e83 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:55.377415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5743b3-e3d5-4ce2-b772-f32c84886e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:55.377696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5743b3-e3d5-4ce2-b772-f32c84886e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:55.415628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5743b3-e3d5-4ce2-b772-f32c84886e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:55.417596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5743b3-e3d5-4ce2-b772-f32c84886e83 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:55.435767 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2012/4088] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:02:55.967399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e84956-e3ce-4058-9fde-4a3e63852782 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:55.968912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d7e84956-e3ce-4058-9fde-4a3e63852782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:55.992691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e84956-e3ce-4058-9fde-4a3e63852782 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:56.007598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e10396-9bd1-4748-9545-7fe1ff812b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:56.009768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e10396-9bd1-4748-9545-7fe1ff812b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:56.012064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e10396-9bd1-4748-9545-7fe1ff812b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:56.013397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e10396-9bd1-4748-9545-7fe1ff812b9a admin admin] PATH_INFO: `/v3/projects/68d55139c0794917875dc6f553f70c20` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:56.015677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e10396-9bd1-4748-9545-7fe1ff812b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d55139c0794917875dc6f553f70c20)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:56.026300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e10396-9bd1-4748-9545-7fe1ff812b9a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:56.159211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9e10396-9bd1-4748-9545-7fe1ff812b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d55139c0794917875dc6f553f70c20', 'request_id': 'req-e9e10396-9bd1-4748-9545-7fe1ff812b9a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:56.180230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9e10396-9bd1-4748-9545-7fe1ff812b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d55139c0794917875dc6f553f70c20', 'request_id': 'req-e9e10396-9bd1-4748-9545-7fe1ff812b9a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:56.188371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9e10396-9bd1-4748-9545-7fe1ff812b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d55139c0794917875dc6f553f70c20', 'request_id': 'req-e9e10396-9bd1-4748-9545-7fe1ff812b9a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:56.196929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:56.204847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9e10396-9bd1-4748-9545-7fe1ff812b9a admin admin] The token cache is being invalidate because project 68d55139c0794917875dc6f553f70c20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:56.206510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9e10396-9bd1-4748-9545-7fe1ff812b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:56.207420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9e10396-9bd1-4748-9545-7fe1ff812b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:56.213118 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2077/4089] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:55 2022] DELETE /identity/v3/projects/68d55139c0794917875dc6f553f70c20 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:56.217471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5743b3-e3d5-4ce2-b772-f32c84886e83 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:56.231903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5743b3-e3d5-4ce2-b772-f32c84886e83 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:56.330374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:56.333313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:56.333589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:56.333871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] PATH_INFO: `/v3/users/8af1cdee3b3b4a829a5dd6698a660812` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:56.335493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af1cdee3b3b4a829a5dd6698a660812)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:56.342508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:56.437667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:56.442946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af1cdee3b3b4a829a5dd6698a660812'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:56.469345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af1cdee3b3b4a829a5dd6698a660812'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:56.481447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af1cdee3b3b4a829a5dd6698a660812'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:56.504016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda92e73-5d30-4674-9797-f27c7a2a93cc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:56.508562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda92e73-5d30-4674-9797-f27c7a2a93cc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:56.508562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda92e73-5d30-4674-9797-f27c7a2a93cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:56.508786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af1cdee3b3b4a829a5dd6698a660812'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:56.518506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af1cdee3b3b4a829a5dd6698a660812'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:56.528297 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2013/4090] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:02:56 2022] DELETE /identity/v3/users/8af1cdee3b3b4a829a5dd6698a660812 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:56.544136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-dda92e73-5d30-4674-9797-f27c7a2a93cc None None] MFA Rules not processed for user `77edcf09f5f84e51a4fb08dbaf763a67`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:56.545474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dda92e73-5d30-4674-9797-f27c7a2a93cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:56.685431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:56.692802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:56.693193 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2078/4091] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:56.818335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11032c10-19b7-4b42-a78d-e23b3e028309 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:56.824114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b530d-9049-466d-847e-04c04287a581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:56.824114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b530d-9049-466d-847e-04c04287a581 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:56.824114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b530d-9049-466d-847e-04c04287a581 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:56.824114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b530d-9049-466d-847e-04c04287a581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:56.842518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b530d-9049-466d-847e-04c04287a581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:56.844307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b530d-9049-466d-847e-04c04287a581 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:56.857381 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2014/4092] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:02:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:02:57.231526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda92e73-5d30-4674-9797-f27c7a2a93cc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:57.233725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dda92e73-5d30-4674-9797-f27c7a2a93cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:57.415352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda92e73-5d30-4674-9797-f27c7a2a93cc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:57.423485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:57.424400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:57.424920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:57.425481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2 admin admin] PATH_INFO: `/v3/projects/33046660bb1146348f704884dd3256e5` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:57.427293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33046660bb1146348f704884dd3256e5)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:57.430273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:57.538031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33046660bb1146348f704884dd3256e5', 'request_id': 'req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:57.554274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33046660bb1146348f704884dd3256e5', 'request_id': 'req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:57.569569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33046660bb1146348f704884dd3256e5', 'request_id': 'req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:57.605746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2 admin admin] The token cache is being invalidate because project 33046660bb1146348f704884dd3256e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:02:57.606229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:57.608734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-91052fa5-b50f-4e63-9c0a-0f2c4368b4b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:02:57.616758 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2079/4093] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:02:57 2022] DELETE /identity/v3/projects/33046660bb1146348f704884dd3256e5 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:02:57.622949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0f1fd-c19f-420d-9b30-b5c28467dfca None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:57.623489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0f1fd-c19f-420d-9b30-b5c28467dfca None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:57.623747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0f1fd-c19f-420d-9b30-b5c28467dfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:57.665726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-90f0f1fd-c19f-420d-9b30-b5c28467dfca None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:57.667274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-90f0f1fd-c19f-420d-9b30-b5c28467dfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:57.800283 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2015/4094] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:57.821176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce09814-014c-4970-adbf-8c45b9cdf7e9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:57.821176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce09814-014c-4970-adbf-8c45b9cdf7e9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:57.821176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce09814-014c-4970-adbf-8c45b9cdf7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:57.863258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5ce09814-014c-4970-adbf-8c45b9cdf7e9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:57.864602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5ce09814-014c-4970-adbf-8c45b9cdf7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:57.876776 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2080/4095] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:57.878075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0f1fd-c19f-420d-9b30-b5c28467dfca None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:57.891791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0f1fd-c19f-420d-9b30-b5c28467dfca None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:57.905725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f6074-0e7a-4b4e-a367-eb1580fdc853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:57.906781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f6074-0e7a-4b4e-a367-eb1580fdc853 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:57.908547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f6074-0e7a-4b4e-a367-eb1580fdc853 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:57.908928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f6074-0e7a-4b4e-a367-eb1580fdc853 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:57.909709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f6074-0e7a-4b4e-a367-eb1580fdc853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:02:57.910086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f6074-0e7a-4b4e-a367-eb1580fdc853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:57.911844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f6074-0e7a-4b4e-a367-eb1580fdc853 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:57.927179 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2016/4096] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:02:57.931355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce09814-014c-4970-adbf-8c45b9cdf7e9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:57.949836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce09814-014c-4970-adbf-8c45b9cdf7e9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:57.955959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e33bd3-b953-41e2-9549-3140f999e184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:57.956595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e33bd3-b953-41e2-9549-3140f999e184 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:57.956889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e33bd3-b953-41e2-9549-3140f999e184 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:57.957427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e33bd3-b953-41e2-9549-3140f999e184 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:57.958244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e33bd3-b953-41e2-9549-3140f999e184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:57.961325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e33bd3-b953-41e2-9549-3140f999e184 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:57.976099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:58.000958 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2081/4097] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:57 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:58.003537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f6074-0e7a-4b4e-a367-eb1580fdc853 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:58.016873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f6074-0e7a-4b4e-a367-eb1580fdc853 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:58.085662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:58.108494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe91128-4f57-4b23-9fa6-0e3c825dc298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:58.109492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe91128-4f57-4b23-9fa6-0e3c825dc298 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:58.109700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe91128-4f57-4b23-9fa6-0e3c825dc298 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:58.109966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe91128-4f57-4b23-9fa6-0e3c825dc298 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:58.110713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe91128-4f57-4b23-9fa6-0e3c825dc298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:58.113186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe91128-4f57-4b23-9fa6-0e3c825dc298 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:58.148789 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2017/4098] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:58 2022] POST /identity/v3/users => generated 284 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:02:58.152741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e33bd3-b953-41e2-9549-3140f999e184 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:58.168104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e33bd3-b953-41e2-9549-3140f999e184 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:58.178380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263c6d9-6b25-403b-99c1-d88a3bcbeb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:58.179149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263c6d9-6b25-403b-99c1-d88a3bcbeb02 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:58.180431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263c6d9-6b25-403b-99c1-d88a3bcbeb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:58.181174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263c6d9-6b25-403b-99c1-d88a3bcbeb02 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:58.181796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263c6d9-6b25-403b-99c1-d88a3bcbeb02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:58.184705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263c6d9-6b25-403b-99c1-d88a3bcbeb02 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:58.207208 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2082/4099] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:02:58.212692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe91128-4f57-4b23-9fa6-0e3c825dc298 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:58.228353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe91128-4f57-4b23-9fa6-0e3c825dc298 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:58.251398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49253f99-fae0-49bf-bf8b-656e7709deb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:58.252043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49253f99-fae0-49bf-bf8b-656e7709deb4 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:58.252792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49253f99-fae0-49bf-bf8b-656e7709deb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:58.253112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49253f99-fae0-49bf-bf8b-656e7709deb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:58.253777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49253f99-fae0-49bf-bf8b-656e7709deb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:58.255407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49253f99-fae0-49bf-bf8b-656e7709deb4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:58.273187 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2018/4100] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:02:58.279514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263c6d9-6b25-403b-99c1-d88a3bcbeb02 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:58.295316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263c6d9-6b25-403b-99c1-d88a3bcbeb02 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:58.302144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd8114-ff35-49c6-8ece-35cfd9434ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:58.303124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd8114-ff35-49c6-8ece-35cfd9434ec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:58.303815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd8114-ff35-49c6-8ece-35cfd9434ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:58.304374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd8114-ff35-49c6-8ece-35cfd9434ec1 admin admin] PATH_INFO: `/v3/projects/8cab7ff6b7984684919b31814648971b/users/a9db5e3de11245e5964d198d6da88c46/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:58.323463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd8114-ff35-49c6-8ece-35cfd9434ec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cab7ff6b7984684919b31814648971b, user_id=a9db5e3de11245e5964d198d6da88c46, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:58.326755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd8114-ff35-49c6-8ece-35cfd9434ec1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:58.351636 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2083/4101] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:58 2022] PUT /identity/v3/projects/8cab7ff6b7984684919b31814648971b/users/a9db5e3de11245e5964d198d6da88c46/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:58.363506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60913e-3e47-4bb6-959f-973aaa63f4e6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:58.363867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60913e-3e47-4bb6-959f-973aaa63f4e6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:58.364150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60913e-3e47-4bb6-959f-973aaa63f4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:58.412921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fd60913e-3e47-4bb6-959f-973aaa63f4e6 None None] MFA Rules not processed for user `a9db5e3de11245e5964d198d6da88c46`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:58.414675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fd60913e-3e47-4bb6-959f-973aaa63f4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:58.523765 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2019/4102] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:58.529995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd8114-ff35-49c6-8ece-35cfd9434ec1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:58.730410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd8114-ff35-49c6-8ece-35cfd9434ec1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:58.753257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b6b55-04bf-4a4b-a698-49d1e3afe5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:58.753902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b6b55-04bf-4a4b-a698-49d1e3afe5c7 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:58.754847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b6b55-04bf-4a4b-a698-49d1e3afe5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:58.755749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b6b55-04bf-4a4b-a698-49d1e3afe5c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:58.758321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b6b55-04bf-4a4b-a698-49d1e3afe5c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:58.762636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b6b55-04bf-4a4b-a698-49d1e3afe5c7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:58.858931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60913e-3e47-4bb6-959f-973aaa63f4e6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:58.880298 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2084/4103] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:02:58 2022] POST /identity/v3/projects => generated 369 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:58.887533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60913e-3e47-4bb6-959f-973aaa63f4e6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:59.028518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5fafd-7142-4279-a1a3-941a7fc596a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:59.029268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5fafd-7142-4279-a1a3-941a7fc596a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:59.029635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5fafd-7142-4279-a1a3-941a7fc596a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:59.030093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5fafd-7142-4279-a1a3-941a7fc596a0 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:59.031810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5fafd-7142-4279-a1a3-941a7fc596a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:59.034529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5fafd-7142-4279-a1a3-941a7fc596a0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:59.090704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b6b55-04bf-4a4b-a698-49d1e3afe5c7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:59.104805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b6b55-04bf-4a4b-a698-49d1e3afe5c7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:59.105971 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2020/4104] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:58 2022] POST /identity/v3/users => generated 291 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:59.143004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ce6e5-cba7-4a2a-b989-ee06587c1c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:59.143629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ce6e5-cba7-4a2a-b989-ee06587c1c50 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:59.143752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ce6e5-cba7-4a2a-b989-ee06587c1c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:59.144024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ce6e5-cba7-4a2a-b989-ee06587c1c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:59.144711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ce6e5-cba7-4a2a-b989-ee06587c1c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:59.146382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ce6e5-cba7-4a2a-b989-ee06587c1c50 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:59.168937 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2085/4105] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:02:59.172830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5fafd-7142-4279-a1a3-941a7fc596a0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:59.187858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5fafd-7142-4279-a1a3-941a7fc596a0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:59.195551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd75d4f-09f3-4614-a4f0-cadd40ede478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:59.197024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd75d4f-09f3-4614-a4f0-cadd40ede478 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:59.197417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd75d4f-09f3-4614-a4f0-cadd40ede478 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:59.198363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd75d4f-09f3-4614-a4f0-cadd40ede478 admin admin] PATH_INFO: `/v3/projects/b9222872f37347c49d3974931081bcd4/users/6b868ff8d3a74c988cf1f1bf73e5efb6/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:59.219103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd75d4f-09f3-4614-a4f0-cadd40ede478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9222872f37347c49d3974931081bcd4, user_id=6b868ff8d3a74c988cf1f1bf73e5efb6, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:59.224008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd75d4f-09f3-4614-a4f0-cadd40ede478 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:59.265020 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2021/4106] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:59 2022] PUT /identity/v3/projects/b9222872f37347c49d3974931081bcd4/users/6b868ff8d3a74c988cf1f1bf73e5efb6/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:59.265368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ce6e5-cba7-4a2a-b989-ee06587c1c50 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:59.280659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ce6e5-cba7-4a2a-b989-ee06587c1c50 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:59.457839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735ec9e-874b-438e-a9a1-dc897366724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:59.458493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735ec9e-874b-438e-a9a1-dc897366724d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:59.458779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735ec9e-874b-438e-a9a1-dc897366724d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:59.459057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735ec9e-874b-438e-a9a1-dc897366724d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:59.461054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735ec9e-874b-438e-a9a1-dc897366724d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:59.466339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735ec9e-874b-438e-a9a1-dc897366724d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:59.495256 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2086/4107] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:02:59.499986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd75d4f-09f3-4614-a4f0-cadd40ede478 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:59.514693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd75d4f-09f3-4614-a4f0-cadd40ede478 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:59.527025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d36bf-6acd-40c0-a6a4-c240ac2d5f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:59.527754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d36bf-6acd-40c0-a6a4-c240ac2d5f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:59.528036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d36bf-6acd-40c0-a6a4-c240ac2d5f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:59.528318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d36bf-6acd-40c0-a6a4-c240ac2d5f8b admin admin] PATH_INFO: `/v3/projects/b9222872f37347c49d3974931081bcd4/users/6b868ff8d3a74c988cf1f1bf73e5efb6/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:59.535642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d36bf-6acd-40c0-a6a4-c240ac2d5f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9222872f37347c49d3974931081bcd4, user_id=6b868ff8d3a74c988cf1f1bf73e5efb6, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:59.542913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d36bf-6acd-40c0-a6a4-c240ac2d5f8b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:59.581911 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2022/4108] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:02:59 2022] PUT /identity/v3/projects/b9222872f37347c49d3974931081bcd4/users/6b868ff8d3a74c988cf1f1bf73e5efb6/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:02:59.583701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf5ab8-4178-48eb-b6ca-5ffe9b1eebec None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:59.584221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf5ab8-4178-48eb-b6ca-5ffe9b1eebec None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:59.584687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf5ab8-4178-48eb-b6ca-5ffe9b1eebec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:59.645555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4dcf5ab8-4178-48eb-b6ca-5ffe9b1eebec None None] MFA Rules not processed for user `6b868ff8d3a74c988cf1f1bf73e5efb6`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:02:59.647072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4dcf5ab8-4178-48eb-b6ca-5ffe9b1eebec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:02:59.673131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:59.829439 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2087/4109] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:02:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:02:59.837206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d36bf-6acd-40c0-a6a4-c240ac2d5f8b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:59.854766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d36bf-6acd-40c0-a6a4-c240ac2d5f8b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:02:59.870185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc0157-ef24-4882-8f04-52aefdca140e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:02:59.871502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc0157-ef24-4882-8f04-52aefdca140e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:02:59.873305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc0157-ef24-4882-8f04-52aefdca140e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:02:59.873847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc0157-ef24-4882-8f04-52aefdca140e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:02:59.874764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc0157-ef24-4882-8f04-52aefdca140e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:02:59.880080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc0157-ef24-4882-8f04-52aefdca140e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:02:59.913412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:02:59.947903 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2023/4110] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:02:59 2022] POST /identity/v3/projects => generated 328 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:02:59.954493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf5ab8-4178-48eb-b6ca-5ffe9b1eebec None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:02:59.975919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf5ab8-4178-48eb-b6ca-5ffe9b1eebec None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:00.061145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41484f58-b5c0-4fdc-be03-b43943e2b1c4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:00.061419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41484f58-b5c0-4fdc-be03-b43943e2b1c4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:00.061881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41484f58-b5c0-4fdc-be03-b43943e2b1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:00.104937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-41484f58-b5c0-4fdc-be03-b43943e2b1c4 None None] MFA Rules not processed for user `77edcf09f5f84e51a4fb08dbaf763a67`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:00.106232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-41484f58-b5c0-4fdc-be03-b43943e2b1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:00.141088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e925b6-104e-4d67-aa9e-48626f0b73f6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:00.144671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e925b6-104e-4d67-aa9e-48626f0b73f6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:00.145010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e925b6-104e-4d67-aa9e-48626f0b73f6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:00.145296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e925b6-104e-4d67-aa9e-48626f0b73f6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:00.145981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e925b6-104e-4d67-aa9e-48626f0b73f6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:00.149366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e925b6-104e-4d67-aa9e-48626f0b73f6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:00.202540 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2088/4111] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:02:59 2022] POST /identity/v3/users => generated 416 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:00.206433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e925b6-104e-4d67-aa9e-48626f0b73f6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:00.238350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e925b6-104e-4d67-aa9e-48626f0b73f6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:00.245263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e24df-7b3a-457a-804e-e869272fcd23 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:00.246095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e24df-7b3a-457a-804e-e869272fcd23 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:00.247867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e24df-7b3a-457a-804e-e869272fcd23 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:00.248815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e24df-7b3a-457a-804e-e869272fcd23 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:00.250231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e24df-7b3a-457a-804e-e869272fcd23 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:00.265098 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-307e24df-7b3a-457a-804e-e869272fcd23 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:03:00.265098 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:03:00.265098 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:03:00.265098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e24df-7b3a-457a-804e-e869272fcd23 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:00.287801 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2089/4112] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:03:00 2022] POST /identity/v3/roles => generated 244 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:00.292548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e24df-7b3a-457a-804e-e869272fcd23 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:00.302463 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2024/4113] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:00.334455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e24df-7b3a-457a-804e-e869272fcd23 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:00.344953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd71c1-239e-4548-9b60-0ab8bc3ba83d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:00.350904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd71c1-239e-4548-9b60-0ab8bc3ba83d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:00.351956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd71c1-239e-4548-9b60-0ab8bc3ba83d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:00.352489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd71c1-239e-4548-9b60-0ab8bc3ba83d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:00.356148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd71c1-239e-4548-9b60-0ab8bc3ba83d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:00.360002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd71c1-239e-4548-9b60-0ab8bc3ba83d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:00.401421 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2090/4114] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:03:00 2022] POST /identity/v3/roles => generated 246 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:00.405366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41484f58-b5c0-4fdc-be03-b43943e2b1c4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:00.422604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41484f58-b5c0-4fdc-be03-b43943e2b1c4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:00.428757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ed0ec-2be2-4650-a936-fd69ddcf2384 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:00.430873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ed0ec-2be2-4650-a936-fd69ddcf2384 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:00.431533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ed0ec-2be2-4650-a936-fd69ddcf2384 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:00.432857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ed0ec-2be2-4650-a936-fd69ddcf2384 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/72341ade11d44ba1bff2cbf122d67e2b/users/dc075ba4a3b74fdf83facb248d09ca4d/roles/edaf062d3e3e4307b525bb4e824c13ca` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:00.497169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ed0ec-2be2-4650-a936-fd69ddcf2384 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_grant(project_id=72341ade11d44ba1bff2cbf122d67e2b, user_id=dc075ba4a3b74fdf83facb248d09ca4d, role_id=edaf062d3e3e4307b525bb4e824c13ca)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:00.500474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ed0ec-2be2-4650-a936-fd69ddcf2384 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:00.533423 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2025/4115] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:00 2022] PUT /identity/v3/projects/72341ade11d44ba1bff2cbf122d67e2b/users/dc075ba4a3b74fdf83facb248d09ca4d/roles/edaf062d3e3e4307b525bb4e824c13ca => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:00.535147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd71c1-239e-4548-9b60-0ab8bc3ba83d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:00.557999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd71c1-239e-4548-9b60-0ab8bc3ba83d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:00.726440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fa4b4-e008-408f-abd8-5e2891055e2a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:00.727232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fa4b4-e008-408f-abd8-5e2891055e2a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:00.727586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fa4b4-e008-408f-abd8-5e2891055e2a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:00.727935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fa4b4-e008-408f-abd8-5e2891055e2a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/72341ade11d44ba1bff2cbf122d67e2b/users/dc075ba4a3b74fdf83facb248d09ca4d/roles/630068fbfbfa45d0a6cfa991e22a80f3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:00.735216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fa4b4-e008-408f-abd8-5e2891055e2a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_grant(project_id=72341ade11d44ba1bff2cbf122d67e2b, user_id=dc075ba4a3b74fdf83facb248d09ca4d, role_id=630068fbfbfa45d0a6cfa991e22a80f3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:00.738082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fa4b4-e008-408f-abd8-5e2891055e2a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:00.767372 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2091/4116] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:00 2022] PUT /identity/v3/projects/72341ade11d44ba1bff2cbf122d67e2b/users/dc075ba4a3b74fdf83facb248d09ca4d/roles/630068fbfbfa45d0a6cfa991e22a80f3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:00.778017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ed0ec-2be2-4650-a936-fd69ddcf2384 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:00.799854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ed0ec-2be2-4650-a936-fd69ddcf2384 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:00.974246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613e23a-08d2-4f7b-8b63-d04d52f1db5c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:00.975044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613e23a-08d2-4f7b-8b63-d04d52f1db5c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:00.975700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613e23a-08d2-4f7b-8b63-d04d52f1db5c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:00.976157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613e23a-08d2-4f7b-8b63-d04d52f1db5c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:00.977147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613e23a-08d2-4f7b-8b63-d04d52f1db5c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:00.980565 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-9613e23a-08d2-4f7b-8b63-d04d52f1db5c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:03:00.980565 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:03:00.980565 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:03:00.986274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613e23a-08d2-4f7b-8b63-d04d52f1db5c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:01.082079 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2026/4117] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:00 2022] GET /identity/v3/users => generated 8345 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:01.085155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8c0b3-3648-43bb-a036-e5a8d618368c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:01.085450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8c0b3-3648-43bb-a036-e5a8d618368c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:01.086722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8c0b3-3648-43bb-a036-e5a8d618368c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:01.152467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:01.286021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ee269-968b-4c9f-97f4-5c9e8c002041 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:01.286566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ee269-968b-4c9f-97f4-5c9e8c002041 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:01.287003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ee269-968b-4c9f-97f4-5c9e8c002041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:01.310154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7ea8c0b3-3648-43bb-a036-e5a8d618368c None None] MFA Rules not processed for user `dc075ba4a3b74fdf83facb248d09ca4d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:01.311438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7ea8c0b3-3648-43bb-a036-e5a8d618368c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:01.360245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cc3ee269-968b-4c9f-97f4-5c9e8c002041 None None] MFA Rules not processed for user `c3187514ca644ff68c992526f38783c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:01.362169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cc3ee269-968b-4c9f-97f4-5c9e8c002041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:01.542696 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2092/4118] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:03:01.552320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3c34f-dc4a-4d4e-a502-ca474aa16e3b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:01.557614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3c34f-dc4a-4d4e-a502-ca474aa16e3b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:01.557974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3c34f-dc4a-4d4e-a502-ca474aa16e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:01.596977 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2027/4119] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:01.608952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-67a3c34f-dc4a-4d4e-a502-ca474aa16e3b None None] MFA Rules not processed for user `dc075ba4a3b74fdf83facb248d09ca4d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:01.609947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-67a3c34f-dc4a-4d4e-a502-ca474aa16e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:01.626016 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2093/4120] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:01.630360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ee269-968b-4c9f-97f4-5c9e8c002041 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:01.643909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ee269-968b-4c9f-97f4-5c9e8c002041 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:01.650763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286acdd-ff89-4d2b-b1d8-ba405b7b321b tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc075ba4a3b74fdf83facb248d09ca4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72341ade11d44ba1bff2cbf122d67e2b', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-313482767', 'tempest-DelegatedRole-2058626442'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:01.651661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286acdd-ff89-4d2b-b1d8-ba405b7b321b tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:01.651960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286acdd-ff89-4d2b-b1d8-ba405b7b321b tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:01.652239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286acdd-ff89-4d2b-b1d8-ba405b7b321b tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:01.653178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286acdd-ff89-4d2b-b1d8-ba405b7b321b tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] RBAC: Authorizing `identity:create_trust()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:01.655337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286acdd-ff89-4d2b-b1d8-ba405b7b321b tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:01.681604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:01.840515 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2028/4121] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Sep 3 21:03:01 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:01.840763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3c34f-dc4a-4d4e-a502-ca474aa16e3b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:01.858897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3c34f-dc4a-4d4e-a502-ca474aa16e3b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:01.870811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed82fc-d61d-4090-90d0-6441d707f788 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:01.871203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed82fc-d61d-4090-90d0-6441d707f788 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:01.887037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed82fc-d61d-4090-90d0-6441d707f788 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:01.887599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed82fc-d61d-4090-90d0-6441d707f788 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:01.888586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed82fc-d61d-4090-90d0-6441d707f788 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:list_trusts()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:01.890317 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-bbed82fc-d61d-4090-90d0-6441d707f788 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_trusts": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:03:01.890317 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:03:01.890317 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:03:01.891663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed82fc-d61d-4090-90d0-6441d707f788 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:01.908049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286acdd-ff89-4d2b-b1d8-ba405b7b321b tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:01.921932 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2094/4122] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Sep 3 21:03:01 2022] GET /identity/v3/OS-TRUST/trusts/ => generated 562 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:03:01.924131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286acdd-ff89-4d2b-b1d8-ba405b7b321b tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:01.929269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0 tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc075ba4a3b74fdf83facb248d09ca4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72341ade11d44ba1bff2cbf122d67e2b', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-313482767', 'tempest-DelegatedRole-2058626442'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:01.930379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0 tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:01.930963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0 tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:01.931481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0 tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] PATH_INFO: `/v3/OS-TRUST/trusts/2b94e67a21404e47a19b811d93d19691` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:01.948491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0 tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] RBAC: Authorizing `identity:delete_trust(trust_id=2b94e67a21404e47a19b811d93d19691)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:01.953163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0 tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:01.985927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0 tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '2b94e67a21404e47a19b811d93d19691', 'request_id': 'req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:01.997198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0 tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '2b94e67a21404e47a19b811d93d19691', 'request_id': 'req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:01.998145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0 tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '2b94e67a21404e47a19b811d93d19691', 'request_id': 'req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:02.009636 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2029/4123] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Sep 3 21:03:01 2022] DELETE /identity/v3/OS-TRUST/trusts/2b94e67a21404e47a19b811d93d19691 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:02.014092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed82fc-d61d-4090-90d0-6441d707f788 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:02.016012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bbed82fc-d61d-4090-90d0-6441d707f788 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:02.040148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed82fc-d61d-4090-90d0-6441d707f788 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:02.046543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a66330-959e-4599-9cdd-6c6707b1e9fd tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc075ba4a3b74fdf83facb248d09ca4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72341ade11d44ba1bff2cbf122d67e2b', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-313482767', 'tempest-DelegatedRole-2058626442'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:02.048058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a66330-959e-4599-9cdd-6c6707b1e9fd tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:02.048058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a66330-959e-4599-9cdd-6c6707b1e9fd tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:02.048446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a66330-959e-4599-9cdd-6c6707b1e9fd tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] PATH_INFO: `/v3/OS-TRUST/trusts/2b94e67a21404e47a19b811d93d19691` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-33a66330-959e-4599-9cdd-6c6707b1e9fd tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] Could not find trust: 2b94e67a21404e47a19b811d93d19691.: keystone.exception.TrustNotFound: Could not find trust: 2b94e67a21404e47a19b811d93d19691. Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 03 21:03:02.112759 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 2b94e67a21404e47a19b811d93d19691. Sep 03 21:03:02.114184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:03:02.123611 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2095/4124] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:03:02 2022] GET /identity/v3/OS-TRUST/trusts/2b94e67a21404e47a19b811d93d19691 => generated 111 bytes in 105 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:03:02.123910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0 tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:02.123910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0 tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:02.143630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42513e7c-eb9b-46e1-96e6-b7a0308a76e0 tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:02.158114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e7187-cd01-481e-8c20-1e19960219e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:02.158779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e7187-cd01-481e-8c20-1e19960219e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:02.159067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e7187-cd01-481e-8c20-1e19960219e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:02.159349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e7187-cd01-481e-8c20-1e19960219e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles/630068fbfbfa45d0a6cfa991e22a80f3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:02.161166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e7187-cd01-481e-8c20-1e19960219e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_role(role_id=630068fbfbfa45d0a6cfa991e22a80f3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:02.166056 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-e40e7187-cd01-481e-8c20-1e19960219e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:03:02.166056 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:03:02.166056 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:03:02.167726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e7187-cd01-481e-8c20-1e19960219e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:02.251957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e40e7187-cd01-481e-8c20-1e19960219e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invalidating the token cache because role 630068fbfbfa45d0a6cfa991e22a80f3 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:02.252260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e40e7187-cd01-481e-8c20-1e19960219e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:02.252988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e40e7187-cd01-481e-8c20-1e19960219e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:02.255129 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2030/4125] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:02 2022] DELETE /identity/v3/roles/630068fbfbfa45d0a6cfa991e22a80f3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:02.260210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a66330-959e-4599-9cdd-6c6707b1e9fd tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:02.262440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-33a66330-959e-4599-9cdd-6c6707b1e9fd tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:02.293061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a66330-959e-4599-9cdd-6c6707b1e9fd tempest-TrustsV3TestJSON-883558761 tempest-user-1386221800] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:02.460652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:02.464209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:02.466492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:02.466900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles/edaf062d3e3e4307b525bb4e824c13ca` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:02.470770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_role(role_id=edaf062d3e3e4307b525bb4e824c13ca)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:02.474401 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:03:02.474401 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:03:02.474401 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:03:02.476694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:02.580751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invalidating the token cache because role edaf062d3e3e4307b525bb4e824c13ca has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:02.581452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:02.582430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:02.586108 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2096/4126] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:02 2022] DELETE /identity/v3/roles/edaf062d3e3e4307b525bb4e824c13ca => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:02.590725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e7187-cd01-481e-8c20-1e19960219e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:02.624829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e7187-cd01-481e-8c20-1e19960219e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:02.793460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:02.794345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:02.795116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:02.795602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users/dc075ba4a3b74fdf83facb248d09ca4d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:02.798092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_user(user_id=dc075ba4a3b74fdf83facb248d09ca4d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:02.800337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:02.878432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:02.970705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc075ba4a3b74fdf83facb248d09ca4d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:03.038597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc075ba4a3b74fdf83facb248d09ca4d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:03.040180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc075ba4a3b74fdf83facb248d09ca4d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:03.054676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc075ba4a3b74fdf83facb248d09ca4d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:03.070653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc075ba4a3b74fdf83facb248d09ca4d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:03.089177 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2031/4127] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:02 2022] DELETE /identity/v3/users/dc075ba4a3b74fdf83facb248d09ca4d => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:03.090547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:03.094117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:03.117085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5e2e7-db83-460e-b4f4-603637303cb7 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:03.157979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:03.195952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:03.197156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:03.198341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:03.198788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/72341ade11d44ba1bff2cbf122d67e2b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:03.200428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_project(project_id=72341ade11d44ba1bff2cbf122d67e2b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:03.205605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:03.289509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72341ade11d44ba1bff2cbf122d67e2b', 'request_id': 'req-4b0b0f70-d890-41fc-951e-714ba3aabb90'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:03.299609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72341ade11d44ba1bff2cbf122d67e2b', 'request_id': 'req-4b0b0f70-d890-41fc-951e-714ba3aabb90'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:03.309636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72341ade11d44ba1bff2cbf122d67e2b', 'request_id': 'req-4b0b0f70-d890-41fc-951e-714ba3aabb90'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:03.353286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] The token cache is being invalidate because project 72341ade11d44ba1bff2cbf122d67e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:03.353582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:03.354810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:03.368050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:03.369984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:03.374051 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2097/4128] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:03 2022] DELETE /identity/v3/projects/72341ade11d44ba1bff2cbf122d67e2b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:03.391715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8bd1a-5e11-4bef-a949-54696ffb5128 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:03.518101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca66a3-fd6c-46f5-975f-53fbff17c35f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:03.518961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca66a3-fd6c-46f5-975f-53fbff17c35f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:03.520593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca66a3-fd6c-46f5-975f-53fbff17c35f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:03.521645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca66a3-fd6c-46f5-975f-53fbff17c35f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:03.524313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca66a3-fd6c-46f5-975f-53fbff17c35f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:03.527489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca66a3-fd6c-46f5-975f-53fbff17c35f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:03.607056 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2032/4129] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:03:03 2022] POST /identity/v3/projects => generated 329 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:03:03.613251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:03.614659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:03.634944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b0f70-d890-41fc-951e-714ba3aabb90 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:03.778910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d09868-6165-4330-b22d-c9a9f1416c30 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:03.781925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d09868-6165-4330-b22d-c9a9f1416c30 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:03.783796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d09868-6165-4330-b22d-c9a9f1416c30 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:03.784417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d09868-6165-4330-b22d-c9a9f1416c30 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:03.786261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d09868-6165-4330-b22d-c9a9f1416c30 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:03.792910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d09868-6165-4330-b22d-c9a9f1416c30 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:03.861050 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2098/4130] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:03 2022] POST /identity/v3/users => generated 413 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:03.862250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca66a3-fd6c-46f5-975f-53fbff17c35f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:03.882215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca66a3-fd6c-46f5-975f-53fbff17c35f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:03.892923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da70d8-174b-44a9-b7f2-9ff4b77524c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:03.892923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da70d8-174b-44a9-b7f2-9ff4b77524c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:03.893277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da70d8-174b-44a9-b7f2-9ff4b77524c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:03.893363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da70d8-174b-44a9-b7f2-9ff4b77524c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:03.894659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da70d8-174b-44a9-b7f2-9ff4b77524c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:03.897666 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-77da70d8-174b-44a9-b7f2-9ff4b77524c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:03:03.897666 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:03:03.897666 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:03:03.898744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da70d8-174b-44a9-b7f2-9ff4b77524c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:03.931798 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2033/4131] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:03:03 2022] POST /identity/v3/roles => generated 243 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:03.935362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d09868-6165-4330-b22d-c9a9f1416c30 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:03.970398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d09868-6165-4330-b22d-c9a9f1416c30 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:03.991092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31721f-9945-4142-9479-0b370f02b6a1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:03.996554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31721f-9945-4142-9479-0b370f02b6a1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:03.998039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31721f-9945-4142-9479-0b370f02b6a1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:04.000599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31721f-9945-4142-9479-0b370f02b6a1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:04.002042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31721f-9945-4142-9479-0b370f02b6a1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:04.008785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31721f-9945-4142-9479-0b370f02b6a1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:04.058239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da70d8-174b-44a9-b7f2-9ff4b77524c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:04.058663 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2099/4132] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:03:03 2022] POST /identity/v3/roles => generated 247 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:04.078102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da70d8-174b-44a9-b7f2-9ff4b77524c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:04.102080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f6286-6066-4ceb-948b-0f7a980b99dc tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:04.102804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f6286-6066-4ceb-948b-0f7a980b99dc tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:04.103055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f6286-6066-4ceb-948b-0f7a980b99dc tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:04.103349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f6286-6066-4ceb-948b-0f7a980b99dc tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/3c9b465eba0d45ebb5affd8271815e4b/users/32a7b9744ee741418829ebf83e0ec81d/roles/1b96695774514fdf8fe0a9e008292a1b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:04.158794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f6286-6066-4ceb-948b-0f7a980b99dc tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_grant(project_id=3c9b465eba0d45ebb5affd8271815e4b, user_id=32a7b9744ee741418829ebf83e0ec81d, role_id=1b96695774514fdf8fe0a9e008292a1b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:04.161543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f6286-6066-4ceb-948b-0f7a980b99dc tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:04.203614 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2034/4133] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:04 2022] PUT /identity/v3/projects/3c9b465eba0d45ebb5affd8271815e4b/users/32a7b9744ee741418829ebf83e0ec81d/roles/1b96695774514fdf8fe0a9e008292a1b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:04.209359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31721f-9945-4142-9479-0b370f02b6a1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:04.226930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31721f-9945-4142-9479-0b370f02b6a1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:04.356316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:04.423994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65484fbc-b121-49dc-91cf-9e9fd11ede76 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:04.425852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65484fbc-b121-49dc-91cf-9e9fd11ede76 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:04.426149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65484fbc-b121-49dc-91cf-9e9fd11ede76 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:04.426436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65484fbc-b121-49dc-91cf-9e9fd11ede76 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/3c9b465eba0d45ebb5affd8271815e4b/users/32a7b9744ee741418829ebf83e0ec81d/roles/e89e6557c7064f06818b1b4c6926e04a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:04.435684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65484fbc-b121-49dc-91cf-9e9fd11ede76 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_grant(project_id=3c9b465eba0d45ebb5affd8271815e4b, user_id=32a7b9744ee741418829ebf83e0ec81d, role_id=e89e6557c7064f06818b1b4c6926e04a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:04.439079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65484fbc-b121-49dc-91cf-9e9fd11ede76 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:04.458424 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2100/4134] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:04 2022] PUT /identity/v3/projects/3c9b465eba0d45ebb5affd8271815e4b/users/32a7b9744ee741418829ebf83e0ec81d/roles/e89e6557c7064f06818b1b4c6926e04a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:04.462290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f6286-6066-4ceb-948b-0f7a980b99dc tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:04.481180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f6286-6066-4ceb-948b-0f7a980b99dc tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:04.557903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:04.578035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46d933-b411-4aa0-bcb4-324ae11f662c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:04.578809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46d933-b411-4aa0-bcb4-324ae11f662c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:04.579192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46d933-b411-4aa0-bcb4-324ae11f662c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:04.579807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46d933-b411-4aa0-bcb4-324ae11f662c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:04.580716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46d933-b411-4aa0-bcb4-324ae11f662c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:04.582739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46d933-b411-4aa0-bcb4-324ae11f662c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:04.655188 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2035/4135] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:04 2022] GET /identity/v3/users => generated 8342 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:04.656965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260af290-d948-4d4c-a7c2-1baff9f7be9b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:04.657427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260af290-d948-4d4c-a7c2-1baff9f7be9b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:04.657805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260af290-d948-4d4c-a7c2-1baff9f7be9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:04.766581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-260af290-d948-4d4c-a7c2-1baff9f7be9b None None] MFA Rules not processed for user `32a7b9744ee741418829ebf83e0ec81d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:04.768123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-260af290-d948-4d4c-a7c2-1baff9f7be9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:04.862053 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2101/4136] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:04 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:03:04.872775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e88ce4-9d20-4577-8378-b80fac11049d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:04.873157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e88ce4-9d20-4577-8378-b80fac11049d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:04.873446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e88ce4-9d20-4577-8378-b80fac11049d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:04.903697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a2e88ce4-9d20-4577-8378-b80fac11049d None None] MFA Rules not processed for user `32a7b9744ee741418829ebf83e0ec81d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:04.905893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a2e88ce4-9d20-4577-8378-b80fac11049d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:04.929816 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2036/4137] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:04 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:04.935781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260af290-d948-4d4c-a7c2-1baff9f7be9b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:04.952545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260af290-d948-4d4c-a7c2-1baff9f7be9b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:04.961880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b0e56-4851-431c-8eec-ddb9b02daeb5 tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a7b9744ee741418829ebf83e0ec81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c9b465eba0d45ebb5affd8271815e4b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-590036292', 'tempest-NotDelegatedRole-2055167497'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:04.963013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b0e56-4851-431c-8eec-ddb9b02daeb5 tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:04.963626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b0e56-4851-431c-8eec-ddb9b02daeb5 tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:04.964160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b0e56-4851-431c-8eec-ddb9b02daeb5 tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:04.965182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b0e56-4851-431c-8eec-ddb9b02daeb5 tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] RBAC: Authorizing `identity:create_trust()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:04.967150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b0e56-4851-431c-8eec-ddb9b02daeb5 tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:05.173722 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2102/4138] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Sep 3 21:03:04 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:05.175871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e88ce4-9d20-4577-8378-b80fac11049d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:05.190312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e88ce4-9d20-4577-8378-b80fac11049d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:05.195499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32896768-aee4-4fc5-800e-418e6c6872dd tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a7b9744ee741418829ebf83e0ec81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c9b465eba0d45ebb5affd8271815e4b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-590036292', 'tempest-NotDelegatedRole-2055167497'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:05.196132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32896768-aee4-4fc5-800e-418e6c6872dd tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:05.196421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32896768-aee4-4fc5-800e-418e6c6872dd tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:05.196700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32896768-aee4-4fc5-800e-418e6c6872dd tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:05.199211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32896768-aee4-4fc5-800e-418e6c6872dd tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] RBAC: Authorizing `identity:list_trusts_for_trustor()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:05.204772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32896768-aee4-4fc5-800e-418e6c6872dd tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:05.224180 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2037/4139] 10.222.0.16 () {58 vars in 1252 bytes} [Sat Sep 3 21:03:05 2022] GET /identity/v3/OS-TRUST/trusts/?trustor_user_id=32a7b9744ee741418829ebf83e0ec81d => generated 611 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:05.228372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b0e56-4851-431c-8eec-ddb9b02daeb5 tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:05.239817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b0e56-4851-431c-8eec-ddb9b02daeb5 tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:05.244451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16368a5-280b-451f-9db4-285ecf41ab8d tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a7b9744ee741418829ebf83e0ec81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c9b465eba0d45ebb5affd8271815e4b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-590036292', 'tempest-NotDelegatedRole-2055167497'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:05.245139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16368a5-280b-451f-9db4-285ecf41ab8d tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:05.245427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16368a5-280b-451f-9db4-285ecf41ab8d tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:05.245710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16368a5-280b-451f-9db4-285ecf41ab8d tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] PATH_INFO: `/v3/OS-TRUST/trusts/112a946517524828bc640727e1c57684` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:05.255480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16368a5-280b-451f-9db4-285ecf41ab8d tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] RBAC: Authorizing `identity:delete_trust(trust_id=112a946517524828bc640727e1c57684)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:05.258079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16368a5-280b-451f-9db4-285ecf41ab8d tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:05.386342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d16368a5-280b-451f-9db4-285ecf41ab8d tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '112a946517524828bc640727e1c57684', 'request_id': 'req-d16368a5-280b-451f-9db4-285ecf41ab8d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:05.387703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d16368a5-280b-451f-9db4-285ecf41ab8d tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '112a946517524828bc640727e1c57684', 'request_id': 'req-d16368a5-280b-451f-9db4-285ecf41ab8d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:05.413834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d16368a5-280b-451f-9db4-285ecf41ab8d tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '112a946517524828bc640727e1c57684', 'request_id': 'req-d16368a5-280b-451f-9db4-285ecf41ab8d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:05.472654 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2103/4140] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Sep 3 21:03:05 2022] DELETE /identity/v3/OS-TRUST/trusts/112a946517524828bc640727e1c57684 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:05.474484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32896768-aee4-4fc5-800e-418e6c6872dd tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:05.481778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-32896768-aee4-4fc5-800e-418e6c6872dd tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:05.595533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32896768-aee4-4fc5-800e-418e6c6872dd tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:05.630515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafa96e-31e1-4930-8af7-d59d2193b62f tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a7b9744ee741418829ebf83e0ec81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c9b465eba0d45ebb5affd8271815e4b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-590036292', 'tempest-NotDelegatedRole-2055167497'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:05.634014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafa96e-31e1-4930-8af7-d59d2193b62f tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:05.634792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafa96e-31e1-4930-8af7-d59d2193b62f tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:05.638512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafa96e-31e1-4930-8af7-d59d2193b62f tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] PATH_INFO: `/v3/OS-TRUST/trusts/112a946517524828bc640727e1c57684` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-8dafa96e-31e1-4930-8af7-d59d2193b62f tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Could not find trust: 112a946517524828bc640727e1c57684.: keystone.exception.TrustNotFound: Could not find trust: 112a946517524828bc640727e1c57684. Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 03 21:03:05.677530 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 112a946517524828bc640727e1c57684. Sep 03 21:03:05.679794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:03:05.691511 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2038/4141] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:03:05 2022] GET /identity/v3/OS-TRUST/trusts/112a946517524828bc640727e1c57684 => generated 111 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:03:05.697173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16368a5-280b-451f-9db4-285ecf41ab8d tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:05.702623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d16368a5-280b-451f-9db4-285ecf41ab8d tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:05.748535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:05.799039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16368a5-280b-451f-9db4-285ecf41ab8d tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:05.834624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a05353-646a-4198-b945-37c716e171c6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:05.835892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a05353-646a-4198-b945-37c716e171c6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:05.836401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a05353-646a-4198-b945-37c716e171c6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:05.837677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a05353-646a-4198-b945-37c716e171c6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles/e89e6557c7064f06818b1b4c6926e04a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:05.860164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a05353-646a-4198-b945-37c716e171c6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_role(role_id=e89e6557c7064f06818b1b4c6926e04a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:05.862229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a05353-646a-4198-b945-37c716e171c6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:05.945968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-84a05353-646a-4198-b945-37c716e171c6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invalidating the token cache because role e89e6557c7064f06818b1b4c6926e04a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:05.946282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-84a05353-646a-4198-b945-37c716e171c6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:05.948727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-84a05353-646a-4198-b945-37c716e171c6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:05.955221 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2104/4142] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:05 2022] DELETE /identity/v3/roles/e89e6557c7064f06818b1b4c6926e04a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:05.958561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafa96e-31e1-4930-8af7-d59d2193b62f tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:05.965164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8dafa96e-31e1-4930-8af7-d59d2193b62f tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:06.022931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafa96e-31e1-4930-8af7-d59d2193b62f tempest-TrustsV3TestJSON-1049892290 tempest-user-990450105] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:06.115741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b1fde-235e-407f-aa8f-1d7ab70ccd2d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:06.116997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b1fde-235e-407f-aa8f-1d7ab70ccd2d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:06.117535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b1fde-235e-407f-aa8f-1d7ab70ccd2d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:06.118045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b1fde-235e-407f-aa8f-1d7ab70ccd2d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles/1b96695774514fdf8fe0a9e008292a1b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:06.121062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b1fde-235e-407f-aa8f-1d7ab70ccd2d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_role(role_id=1b96695774514fdf8fe0a9e008292a1b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:06.122982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b1fde-235e-407f-aa8f-1d7ab70ccd2d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:06.199310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ad0b1fde-235e-407f-aa8f-1d7ab70ccd2d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invalidating the token cache because role 1b96695774514fdf8fe0a9e008292a1b has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:06.201410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ad0b1fde-235e-407f-aa8f-1d7ab70ccd2d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:06.202394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ad0b1fde-235e-407f-aa8f-1d7ab70ccd2d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:06.205881 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2039/4143] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:05 2022] DELETE /identity/v3/roles/1b96695774514fdf8fe0a9e008292a1b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:06.209842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a05353-646a-4198-b945-37c716e171c6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:06.221454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a05353-646a-4198-b945-37c716e171c6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:06.293739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:06.294446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:06.294740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:06.295018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users/32a7b9744ee741418829ebf83e0ec81d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:06.296257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_user(user_id=32a7b9744ee741418829ebf83e0ec81d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:06.298400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:06.389883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a7b9744ee741418829ebf83e0ec81d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:06.404970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a7b9744ee741418829ebf83e0ec81d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:06.413796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a7b9744ee741418829ebf83e0ec81d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:06.414421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a7b9744ee741418829ebf83e0ec81d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:06.415733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a7b9744ee741418829ebf83e0ec81d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:06.430765 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2105/4144] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:06 2022] DELETE /identity/v3/users/32a7b9744ee741418829ebf83e0ec81d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:06.436041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b1fde-235e-407f-aa8f-1d7ab70ccd2d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:06.437586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ad0b1fde-235e-407f-aa8f-1d7ab70ccd2d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:06.475706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b1fde-235e-407f-aa8f-1d7ab70ccd2d tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:06.585901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:06.586563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:06.586852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:06.587132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/3c9b465eba0d45ebb5affd8271815e4b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:06.591832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3c9b465eba0d45ebb5affd8271815e4b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:06.591832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:06.631809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:06.653686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9b465eba0d45ebb5affd8271815e4b', 'request_id': 'req-346bf860-ef04-48d2-8ee5-3b9497d9e69b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:06.662522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9b465eba0d45ebb5affd8271815e4b', 'request_id': 'req-346bf860-ef04-48d2-8ee5-3b9497d9e69b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:06.670179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9b465eba0d45ebb5affd8271815e4b', 'request_id': 'req-346bf860-ef04-48d2-8ee5-3b9497d9e69b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:06.682009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] The token cache is being invalidate because project 3c9b465eba0d45ebb5affd8271815e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:06.682375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:06.683194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:06.684960 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2040/4145] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:06 2022] DELETE /identity/v3/projects/3c9b465eba0d45ebb5affd8271815e4b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:06.690408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:06.692078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:06.718147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24631b-a775-4f5a-9454-30e33ad3922e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:06.783119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:06.810314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09666-215d-4419-bafc-e21897c2431a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:06.811179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09666-215d-4419-bafc-e21897c2431a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:06.811479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09666-215d-4419-bafc-e21897c2431a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:06.811923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09666-215d-4419-bafc-e21897c2431a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:06.812580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf09666-215d-4419-bafc-e21897c2431a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:06.814672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf09666-215d-4419-bafc-e21897c2431a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:06.852423 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2106/4146] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:03:06 2022] POST /identity/v3/projects => generated 328 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:06.857205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:06.859230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:06.873976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bf860-ef04-48d2-8ee5-3b9497d9e69b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:07.007659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bd4af-262a-4ad4-a993-22b313e239c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:07.008408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bd4af-262a-4ad4-a993-22b313e239c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:07.008789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bd4af-262a-4ad4-a993-22b313e239c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:07.010132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bd4af-262a-4ad4-a993-22b313e239c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:07.011294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0bd4af-262a-4ad4-a993-22b313e239c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:07.013984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0bd4af-262a-4ad4-a993-22b313e239c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:07.047045 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2041/4147] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:06 2022] POST /identity/v3/users => generated 413 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:07.050898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09666-215d-4419-bafc-e21897c2431a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:07.084545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09666-215d-4419-bafc-e21897c2431a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:07.106011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91f399-7670-45eb-8417-08202b78c26f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:07.107047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91f399-7670-45eb-8417-08202b78c26f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:07.108609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91f399-7670-45eb-8417-08202b78c26f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:07.109151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91f399-7670-45eb-8417-08202b78c26f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:07.111882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91f399-7670-45eb-8417-08202b78c26f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:07.115553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91f399-7670-45eb-8417-08202b78c26f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:07.154811 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2107/4148] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:03:07 2022] POST /identity/v3/roles => generated 244 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:07.158483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bd4af-262a-4ad4-a993-22b313e239c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:07.176522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bd4af-262a-4ad4-a993-22b313e239c9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:07.190062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acd050-acc1-4496-8b55-33e285f785e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:07.190701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acd050-acc1-4496-8b55-33e285f785e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:07.190983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acd050-acc1-4496-8b55-33e285f785e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:07.191267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acd050-acc1-4496-8b55-33e285f785e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:07.192030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acd050-acc1-4496-8b55-33e285f785e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:07.193590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acd050-acc1-4496-8b55-33e285f785e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:07.226207 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2042/4149] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:03:07 2022] POST /identity/v3/roles => generated 247 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:07.228165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91f399-7670-45eb-8417-08202b78c26f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:07.254702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91f399-7670-45eb-8417-08202b78c26f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:07.260440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9e244-b190-4f7a-9f8c-08f583495fc4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:07.261495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9e244-b190-4f7a-9f8c-08f583495fc4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:07.262834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9e244-b190-4f7a-9f8c-08f583495fc4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:07.262834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9e244-b190-4f7a-9f8c-08f583495fc4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/f976a811716548358dc02426c97bf2cc/users/8d9092ad6ac9470095bc021b3374ff67/roles/69de9145cd8a49ce8b9f0fa727d9a7b6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:07.321777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9e244-b190-4f7a-9f8c-08f583495fc4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f976a811716548358dc02426c97bf2cc, user_id=8d9092ad6ac9470095bc021b3374ff67, role_id=69de9145cd8a49ce8b9f0fa727d9a7b6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:07.335344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9e244-b190-4f7a-9f8c-08f583495fc4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:07.366032 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2108/4150] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:07 2022] PUT /identity/v3/projects/f976a811716548358dc02426c97bf2cc/users/8d9092ad6ac9470095bc021b3374ff67/roles/69de9145cd8a49ce8b9f0fa727d9a7b6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:07.367249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acd050-acc1-4496-8b55-33e285f785e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:07.385354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acd050-acc1-4496-8b55-33e285f785e2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:07.509013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4458e4e-e9a3-41b2-904c-910782d4e81e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:07.509704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4458e4e-e9a3-41b2-904c-910782d4e81e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:07.510031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4458e4e-e9a3-41b2-904c-910782d4e81e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:07.510317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4458e4e-e9a3-41b2-904c-910782d4e81e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/f976a811716548358dc02426c97bf2cc/users/8d9092ad6ac9470095bc021b3374ff67/roles/04ad1528d1fe4c14b557cd5ccdd61cdb` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:07.512862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4458e4e-e9a3-41b2-904c-910782d4e81e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f976a811716548358dc02426c97bf2cc, user_id=8d9092ad6ac9470095bc021b3374ff67, role_id=04ad1528d1fe4c14b557cd5ccdd61cdb)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:07.516056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4458e4e-e9a3-41b2-904c-910782d4e81e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:07.538783 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2043/4151] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:07 2022] PUT /identity/v3/projects/f976a811716548358dc02426c97bf2cc/users/8d9092ad6ac9470095bc021b3374ff67/roles/04ad1528d1fe4c14b557cd5ccdd61cdb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:07.543711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9e244-b190-4f7a-9f8c-08f583495fc4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:07.560116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9e244-b190-4f7a-9f8c-08f583495fc4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:07.657362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbce1d6-49cb-47cc-b9df-6f378414e86e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:07.658219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbce1d6-49cb-47cc-b9df-6f378414e86e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:07.658543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbce1d6-49cb-47cc-b9df-6f378414e86e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:07.658890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbce1d6-49cb-47cc-b9df-6f378414e86e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:07.659581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbce1d6-49cb-47cc-b9df-6f378414e86e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:07.661011 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-6fbce1d6-49cb-47cc-b9df-6f378414e86e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:03:07.661011 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:03:07.661011 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:03:07.665153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbce1d6-49cb-47cc-b9df-6f378414e86e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:07.876944 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2109/4152] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:07 2022] GET /identity/v3/users => generated 8342 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:07.879493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1e0b4-af0d-4136-987e-897c57225a15 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:07.879786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1e0b4-af0d-4136-987e-897c57225a15 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:07.880304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1e0b4-af0d-4136-987e-897c57225a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:07.927292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5be1e0b4-af0d-4136-987e-897c57225a15 None None] MFA Rules not processed for user `8d9092ad6ac9470095bc021b3374ff67`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:07.928719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5be1e0b4-af0d-4136-987e-897c57225a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:07.936051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=5,keystone:UPDATE=1,keystone:INSERT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:08.023438 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2044/4153] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:08.024934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f320c3-c999-47c3-84ab-a8a3ec8504e5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:08.025316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f320c3-c999-47c3-84ab-a8a3ec8504e5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:08.025656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f320c3-c999-47c3-84ab-a8a3ec8504e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:08.049558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c8f320c3-c999-47c3-84ab-a8a3ec8504e5 None None] MFA Rules not processed for user `8d9092ad6ac9470095bc021b3374ff67`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:08.050993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c8f320c3-c999-47c3-84ab-a8a3ec8504e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:08.064945 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2110/4154] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:08 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:08.065505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1e0b4-af0d-4136-987e-897c57225a15 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:08.076971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1e0b4-af0d-4136-987e-897c57225a15 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:08.082022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2de1d8-b10c-4613-9b6c-5a910f233635 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d9092ad6ac9470095bc021b3374ff67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f976a811716548358dc02426c97bf2cc', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1558940055', 'tempest-NotDelegatedRole-1313793859'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:08.083146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2de1d8-b10c-4613-9b6c-5a910f233635 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:08.083452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2de1d8-b10c-4613-9b6c-5a910f233635 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:08.083740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2de1d8-b10c-4613-9b6c-5a910f233635 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:08.084679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2de1d8-b10c-4613-9b6c-5a910f233635 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorizing `identity:create_trust()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:08.085895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2de1d8-b10c-4613-9b6c-5a910f233635 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:08.196808 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2045/4155] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Sep 3 21:03:08 2022] POST /identity/v3/OS-TRUST/trusts => generated 859 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:03:08.201385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f320c3-c999-47c3-84ab-a8a3ec8504e5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:08.210458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f320c3-c999-47c3-84ab-a8a3ec8504e5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:08.214773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c025cc-9c1f-4e00-967a-08047ba6bac6 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d9092ad6ac9470095bc021b3374ff67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f976a811716548358dc02426c97bf2cc', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1558940055', 'tempest-NotDelegatedRole-1313793859'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:08.215430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c025cc-9c1f-4e00-967a-08047ba6bac6 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:08.215714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c025cc-9c1f-4e00-967a-08047ba6bac6 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:08.215990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c025cc-9c1f-4e00-967a-08047ba6bac6 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] PATH_INFO: `/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:08.232772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c025cc-9c1f-4e00-967a-08047ba6bac6 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorizing `identity:get_trust(trust_id=7ec01fea40fd476ab3a5a43cf6e98a6a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:08.235843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c025cc-9c1f-4e00-967a-08047ba6bac6 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:08.250773 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2111/4156] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:03:08 2022] GET /identity/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a => generated 859 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:08.251973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2de1d8-b10c-4613-9b6c-5a910f233635 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:08.267799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2de1d8-b10c-4613-9b6c-5a910f233635 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:08.276533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a68c3-564d-4c16-8ef1-a7a3bba3361f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d9092ad6ac9470095bc021b3374ff67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f976a811716548358dc02426c97bf2cc', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1558940055', 'tempest-NotDelegatedRole-1313793859'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:08.277272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a68c3-564d-4c16-8ef1-a7a3bba3361f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:08.277570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a68c3-564d-4c16-8ef1-a7a3bba3361f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:08.278193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a68c3-564d-4c16-8ef1-a7a3bba3361f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] PATH_INFO: `/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:08.290868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a68c3-564d-4c16-8ef1-a7a3bba3361f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=7ec01fea40fd476ab3a5a43cf6e98a6a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:08.292425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a68c3-564d-4c16-8ef1-a7a3bba3361f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:08.302672 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2046/4157] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Sep 3 21:03:08 2022] GET /identity/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a/roles => generated 371 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:08.307441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c025cc-9c1f-4e00-967a-08047ba6bac6 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:08.316725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c025cc-9c1f-4e00-967a-08047ba6bac6 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:08.324675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400675a5-9cbf-48ad-9ed1-5b4cf976c942 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d9092ad6ac9470095bc021b3374ff67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f976a811716548358dc02426c97bf2cc', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1558940055', 'tempest-NotDelegatedRole-1313793859'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:08.325764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400675a5-9cbf-48ad-9ed1-5b4cf976c942 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:08.326277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400675a5-9cbf-48ad-9ed1-5b4cf976c942 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:08.326622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400675a5-9cbf-48ad-9ed1-5b4cf976c942 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] PATH_INFO: `/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a/roles/69de9145cd8a49ce8b9f0fa727d9a7b6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:08.333818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400675a5-9cbf-48ad-9ed1-5b4cf976c942 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorizing `identity:get_role_for_trust(trust_id=7ec01fea40fd476ab3a5a43cf6e98a6a, role_id=69de9145cd8a49ce8b9f0fa727d9a7b6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:08.335578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400675a5-9cbf-48ad-9ed1-5b4cf976c942 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:08.348504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a68c3-564d-4c16-8ef1-a7a3bba3361f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:08.349169 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2112/4158] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Sep 3 21:03:08 2022] GET /identity/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a/roles/69de9145cd8a49ce8b9f0fa727d9a7b6 => generated 244 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:08.361793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a68c3-564d-4c16-8ef1-a7a3bba3361f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:08.366831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3330b8-485e-4ec3-ad96-6b32794a6a7f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d9092ad6ac9470095bc021b3374ff67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f976a811716548358dc02426c97bf2cc', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1558940055', 'tempest-NotDelegatedRole-1313793859'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:08.367742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3330b8-485e-4ec3-ad96-6b32794a6a7f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] REQUEST_METHOD: `HEAD` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:08.368124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3330b8-485e-4ec3-ad96-6b32794a6a7f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:08.368510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3330b8-485e-4ec3-ad96-6b32794a6a7f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] PATH_INFO: `/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a/roles/69de9145cd8a49ce8b9f0fa727d9a7b6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:08.383242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3330b8-485e-4ec3-ad96-6b32794a6a7f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorizing `identity:get_role_for_trust(trust_id=7ec01fea40fd476ab3a5a43cf6e98a6a, role_id=69de9145cd8a49ce8b9f0fa727d9a7b6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:08.386970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3330b8-485e-4ec3-ad96-6b32794a6a7f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:08.406857 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2047/4159] 10.222.0.16 () {58 vars in 1369 bytes} [Sat Sep 3 21:03:08 2022] HEAD /identity/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a/roles/69de9145cd8a49ce8b9f0fa727d9a7b6 => generated 0 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:08.411602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400675a5-9cbf-48ad-9ed1-5b4cf976c942 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:08.422356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400675a5-9cbf-48ad-9ed1-5b4cf976c942 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:08.427122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a815135-13d7-492e-aeb0-ee045614bcb3 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d9092ad6ac9470095bc021b3374ff67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f976a811716548358dc02426c97bf2cc', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1558940055', 'tempest-NotDelegatedRole-1313793859'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:08.428137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a815135-13d7-492e-aeb0-ee045614bcb3 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:08.428487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a815135-13d7-492e-aeb0-ee045614bcb3 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:08.428834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a815135-13d7-492e-aeb0-ee045614bcb3 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] PATH_INFO: `/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a/roles/04ad1528d1fe4c14b557cd5ccdd61cdb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:08.436597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a815135-13d7-492e-aeb0-ee045614bcb3 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorizing `identity:get_role_for_trust(trust_id=7ec01fea40fd476ab3a5a43cf6e98a6a, role_id=04ad1528d1fe4c14b557cd5ccdd61cdb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:08.438192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a815135-13d7-492e-aeb0-ee045614bcb3 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-5a815135-13d7-492e-aeb0-ee045614bcb3 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Could not find role: 04ad1528d1fe4c14b557cd5ccdd61cdb.: keystone.exception.RoleNotFound: Could not find role: 04ad1528d1fe4c14b557cd5ccdd61cdb. Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 21:03:08.449706 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 04ad1528d1fe4c14b557cd5ccdd61cdb. Sep 03 21:03:08.451355 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:03:08.455575 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2113/4160] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Sep 3 21:03:08 2022] GET /identity/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a/roles/04ad1528d1fe4c14b557cd5ccdd61cdb => generated 110 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:03:08.456617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3330b8-485e-4ec3-ad96-6b32794a6a7f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:08.469105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3330b8-485e-4ec3-ad96-6b32794a6a7f tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:08.475880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e797e7b-b4c9-4425-bfdc-4bafea161717 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d9092ad6ac9470095bc021b3374ff67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f976a811716548358dc02426c97bf2cc', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1558940055', 'tempest-NotDelegatedRole-1313793859'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:08.489179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e797e7b-b4c9-4425-bfdc-4bafea161717 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] REQUEST_METHOD: `HEAD` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:08.489479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e797e7b-b4c9-4425-bfdc-4bafea161717 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:08.489769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e797e7b-b4c9-4425-bfdc-4bafea161717 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] PATH_INFO: `/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a/roles/04ad1528d1fe4c14b557cd5ccdd61cdb` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:08.504364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e797e7b-b4c9-4425-bfdc-4bafea161717 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorizing `identity:get_role_for_trust(trust_id=7ec01fea40fd476ab3a5a43cf6e98a6a, role_id=04ad1528d1fe4c14b557cd5ccdd61cdb)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:08.506535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e797e7b-b4c9-4425-bfdc-4bafea161717 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-5e797e7b-b4c9-4425-bfdc-4bafea161717 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Could not find role: 04ad1528d1fe4c14b557cd5ccdd61cdb.: keystone.exception.RoleNotFound: Could not find role: 04ad1528d1fe4c14b557cd5ccdd61cdb. Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 21:03:08.522759 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 04ad1528d1fe4c14b557cd5ccdd61cdb. Sep 03 21:03:08.523963 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:03:08.527301 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2048/4161] 10.222.0.16 () {58 vars in 1369 bytes} [Sat Sep 3 21:03:08 2022] HEAD /identity/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a/roles/04ad1528d1fe4c14b557cd5ccdd61cdb => generated 0 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Sep 03 21:03:08.528930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a815135-13d7-492e-aeb0-ee045614bcb3 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:08.541671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a815135-13d7-492e-aeb0-ee045614bcb3 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:08.548273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d9092ad6ac9470095bc021b3374ff67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f976a811716548358dc02426c97bf2cc', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1558940055', 'tempest-NotDelegatedRole-1313793859'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:08.549674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:08.550105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:08.550573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] PATH_INFO: `/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:08.564387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorizing `identity:delete_trust(trust_id=7ec01fea40fd476ab3a5a43cf6e98a6a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:08.566432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:08.581429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:08.600946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '7ec01fea40fd476ab3a5a43cf6e98a6a', 'request_id': 'req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:08.602633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '7ec01fea40fd476ab3a5a43cf6e98a6a', 'request_id': 'req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:08.615909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '7ec01fea40fd476ab3a5a43cf6e98a6a', 'request_id': 'req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:08.628943 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2114/4162] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Sep 3 21:03:08 2022] DELETE /identity/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:08.631684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e797e7b-b4c9-4425-bfdc-4bafea161717 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:08.633526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5e797e7b-b4c9-4425-bfdc-4bafea161717 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:08.667500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e797e7b-b4c9-4425-bfdc-4bafea161717 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:08.677193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed9ebe-4349-4935-aabe-8d243bbd28fb tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d9092ad6ac9470095bc021b3374ff67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f976a811716548358dc02426c97bf2cc', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1558940055', 'tempest-NotDelegatedRole-1313793859'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:08.697023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed9ebe-4349-4935-aabe-8d243bbd28fb tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:08.697681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed9ebe-4349-4935-aabe-8d243bbd28fb tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:08.706095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed9ebe-4349-4935-aabe-8d243bbd28fb tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] PATH_INFO: `/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-5eed9ebe-4349-4935-aabe-8d243bbd28fb tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Could not find trust: 7ec01fea40fd476ab3a5a43cf6e98a6a.: keystone.exception.TrustNotFound: Could not find trust: 7ec01fea40fd476ab3a5a43cf6e98a6a. Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 03 21:03:08.740323 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 7ec01fea40fd476ab3a5a43cf6e98a6a. Sep 03 21:03:08.741794 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:03:08.745792 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2049/4163] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:03:08 2022] GET /identity/v3/OS-TRUST/trusts/7ec01fea40fd476ab3a5a43cf6e98a6a => generated 111 bytes in 114 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:03:08.750478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:08.752094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:08.797447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aa4a8-bd91-4cbc-82d1-f22e81d76d06 tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:08.807664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f23e5-c927-4f6f-9747-3b48ffe838c4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:08.808486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f23e5-c927-4f6f-9747-3b48ffe838c4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:08.808846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f23e5-c927-4f6f-9747-3b48ffe838c4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:08.809228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f23e5-c927-4f6f-9747-3b48ffe838c4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles/04ad1528d1fe4c14b557cd5ccdd61cdb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:08.811431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f23e5-c927-4f6f-9747-3b48ffe838c4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_role(role_id=04ad1528d1fe4c14b557cd5ccdd61cdb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:08.813877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f23e5-c927-4f6f-9747-3b48ffe838c4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:08.898648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a90f23e5-c927-4f6f-9747-3b48ffe838c4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invalidating the token cache because role 04ad1528d1fe4c14b557cd5ccdd61cdb has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:08.899312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a90f23e5-c927-4f6f-9747-3b48ffe838c4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:08.899898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a90f23e5-c927-4f6f-9747-3b48ffe838c4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:08.914489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed9ebe-4349-4935-aabe-8d243bbd28fb tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:08.915782 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2115/4164] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:08 2022] DELETE /identity/v3/roles/04ad1528d1fe4c14b557cd5ccdd61cdb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:08.918367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5eed9ebe-4349-4935-aabe-8d243bbd28fb tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:08.936215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed9ebe-4349-4935-aabe-8d243bbd28fb tempest-TrustsV3TestJSON-880515595 tempest-user-357588088] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:09.033046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:09.085139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f56fc-7e29-4080-92fa-90fe4c98cbd1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:09.085139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f56fc-7e29-4080-92fa-90fe4c98cbd1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:09.085139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f56fc-7e29-4080-92fa-90fe4c98cbd1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:09.085661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f56fc-7e29-4080-92fa-90fe4c98cbd1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles/69de9145cd8a49ce8b9f0fa727d9a7b6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:09.091273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f56fc-7e29-4080-92fa-90fe4c98cbd1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_role(role_id=69de9145cd8a49ce8b9f0fa727d9a7b6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:09.094888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f56fc-7e29-4080-92fa-90fe4c98cbd1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:09.184788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-331f56fc-7e29-4080-92fa-90fe4c98cbd1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invalidating the token cache because role 69de9145cd8a49ce8b9f0fa727d9a7b6 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:09.185372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-331f56fc-7e29-4080-92fa-90fe4c98cbd1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:09.186289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-331f56fc-7e29-4080-92fa-90fe4c98cbd1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:09.188505 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2050/4165] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:08 2022] DELETE /identity/v3/roles/69de9145cd8a49ce8b9f0fa727d9a7b6 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:09.193076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f23e5-c927-4f6f-9747-3b48ffe838c4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:09.205232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f23e5-c927-4f6f-9747-3b48ffe838c4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:09.309048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:09.309048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:09.309048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:09.309452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users/8d9092ad6ac9470095bc021b3374ff67` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:09.309452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_user(user_id=8d9092ad6ac9470095bc021b3374ff67)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:09.309452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:09.405813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9092ad6ac9470095bc021b3374ff67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:09.429826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9092ad6ac9470095bc021b3374ff67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:09.444087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9092ad6ac9470095bc021b3374ff67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:09.444791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9092ad6ac9470095bc021b3374ff67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:09.449172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9092ad6ac9470095bc021b3374ff67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:09.475929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f56fc-7e29-4080-92fa-90fe4c98cbd1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:09.476493 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2116/4166] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:09 2022] DELETE /identity/v3/users/8d9092ad6ac9470095bc021b3374ff67 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:09.483948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-331f56fc-7e29-4080-92fa-90fe4c98cbd1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:09.503620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f56fc-7e29-4080-92fa-90fe4c98cbd1 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:09.594883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:09.595659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:09.596017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:09.596374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/f976a811716548358dc02426c97bf2cc` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:09.598022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f976a811716548358dc02426c97bf2cc)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:09.600133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:09.664253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f976a811716548358dc02426c97bf2cc', 'request_id': 'req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:09.673499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f976a811716548358dc02426c97bf2cc', 'request_id': 'req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:09.683461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f976a811716548358dc02426c97bf2cc', 'request_id': 'req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:09.697102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] The token cache is being invalidate because project f976a811716548358dc02426c97bf2cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:09.698408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:09.699295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:09.701032 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2051/4167] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:09 2022] DELETE /identity/v3/projects/f976a811716548358dc02426c97bf2cc => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:09.706888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:09.708220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:09.724623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fceba-9d1a-488f-99ce-f68cc840f3f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:09.748158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:UPDATE=1,keystone:SELECT=83,keystone:INSERT=2,keystone:DELETE=14 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:09.796655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d77fe-090f-404b-8e97-3e902a9d0133 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:09.797321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d77fe-090f-404b-8e97-3e902a9d0133 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:09.797611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d77fe-090f-404b-8e97-3e902a9d0133 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:09.798009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d77fe-090f-404b-8e97-3e902a9d0133 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:09.798649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d77fe-090f-404b-8e97-3e902a9d0133 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:09.800710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d77fe-090f-404b-8e97-3e902a9d0133 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:09.840415 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2117/4168] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:03:09 2022] POST /identity/v3/projects => generated 329 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:09.844328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:09.846121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:09.869032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83d630-9ecc-4e4c-b8e1-1a62e12c3fd4 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:09.917306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:09.977102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a7d8c-6c43-4b21-9524-66b49d779866 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:09.978343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a7d8c-6c43-4b21-9524-66b49d779866 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:09.978792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a7d8c-6c43-4b21-9524-66b49d779866 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:09.979175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a7d8c-6c43-4b21-9524-66b49d779866 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:09.980711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a7d8c-6c43-4b21-9524-66b49d779866 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:09.982837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a7d8c-6c43-4b21-9524-66b49d779866 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:10.008155 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2052/4169] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:09 2022] POST /identity/v3/users => generated 413 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:10.012455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d77fe-090f-404b-8e97-3e902a9d0133 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:10.031685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d77fe-090f-404b-8e97-3e902a9d0133 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:10.038931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbcbfb-0cf6-46bc-a1aa-76e7b866ea6c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:10.039681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbcbfb-0cf6-46bc-a1aa-76e7b866ea6c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:10.039970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbcbfb-0cf6-46bc-a1aa-76e7b866ea6c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:10.040250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbcbfb-0cf6-46bc-a1aa-76e7b866ea6c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:10.041006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbcbfb-0cf6-46bc-a1aa-76e7b866ea6c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:10.042649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbcbfb-0cf6-46bc-a1aa-76e7b866ea6c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:10.074917 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2118/4170] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:03:10 2022] POST /identity/v3/roles => generated 242 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:03:10.076064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a7d8c-6c43-4b21-9524-66b49d779866 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:10.112825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a7d8c-6c43-4b21-9524-66b49d779866 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:10.147701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40f40c-06c7-4b13-ac8e-f9e58acec931 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:10.148344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40f40c-06c7-4b13-ac8e-f9e58acec931 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:10.148633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40f40c-06c7-4b13-ac8e-f9e58acec931 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:10.149023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40f40c-06c7-4b13-ac8e-f9e58acec931 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:10.149623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40f40c-06c7-4b13-ac8e-f9e58acec931 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:10.151819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40f40c-06c7-4b13-ac8e-f9e58acec931 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:10.179769 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2053/4171] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:03:10 2022] POST /identity/v3/roles => generated 246 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:10.183844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbcbfb-0cf6-46bc-a1aa-76e7b866ea6c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:10.212816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbcbfb-0cf6-46bc-a1aa-76e7b866ea6c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:10.237323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29dd76f-0b38-4903-bff2-a8854f887dd6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:10.238025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dd76f-0b38-4903-bff2-a8854f887dd6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:10.238897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dd76f-0b38-4903-bff2-a8854f887dd6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:10.239273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dd76f-0b38-4903-bff2-a8854f887dd6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/329f6b529b864969aad8ebf4beffa4c9/users/9b31f094a84a4347a2d17664040338d4/roles/949c2831b8f34e3ea6247e0f878e19e3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:10.282491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29dd76f-0b38-4903-bff2-a8854f887dd6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_grant(project_id=329f6b529b864969aad8ebf4beffa4c9, user_id=9b31f094a84a4347a2d17664040338d4, role_id=949c2831b8f34e3ea6247e0f878e19e3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:10.289155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29dd76f-0b38-4903-bff2-a8854f887dd6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:10.344515 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2119/4172] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:10 2022] PUT /identity/v3/projects/329f6b529b864969aad8ebf4beffa4c9/users/9b31f094a84a4347a2d17664040338d4/roles/949c2831b8f34e3ea6247e0f878e19e3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:10.345772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40f40c-06c7-4b13-ac8e-f9e58acec931 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:10.364370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40f40c-06c7-4b13-ac8e-f9e58acec931 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:10.496275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da862e-c682-4add-ba7f-55c5aad794f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:10.498051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da862e-c682-4add-ba7f-55c5aad794f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:10.498357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da862e-c682-4add-ba7f-55c5aad794f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:10.498665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da862e-c682-4add-ba7f-55c5aad794f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/329f6b529b864969aad8ebf4beffa4c9/users/9b31f094a84a4347a2d17664040338d4/roles/84692c59133546a3b27967cf14b53bd7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:10.502726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da862e-c682-4add-ba7f-55c5aad794f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_grant(project_id=329f6b529b864969aad8ebf4beffa4c9, user_id=9b31f094a84a4347a2d17664040338d4, role_id=84692c59133546a3b27967cf14b53bd7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:10.506433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da862e-c682-4add-ba7f-55c5aad794f9 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:10.534141 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2054/4173] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:10 2022] PUT /identity/v3/projects/329f6b529b864969aad8ebf4beffa4c9/users/9b31f094a84a4347a2d17664040338d4/roles/84692c59133546a3b27967cf14b53bd7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:10.536316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29dd76f-0b38-4903-bff2-a8854f887dd6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:10.555537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29dd76f-0b38-4903-bff2-a8854f887dd6 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:10.685477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b2fa1-7aae-426d-9f99-39a80d09761c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:10.686732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b2fa1-7aae-426d-9f99-39a80d09761c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:10.688072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b2fa1-7aae-426d-9f99-39a80d09761c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:10.688645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b2fa1-7aae-426d-9f99-39a80d09761c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:10.689734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b2fa1-7aae-426d-9f99-39a80d09761c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:10.691990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b2fa1-7aae-426d-9f99-39a80d09761c tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:10.745802 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2120/4174] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:10 2022] GET /identity/v3/users => generated 8342 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:10.747487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8c73c-fc75-487b-9a3b-83a198d855c9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:10.747788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8c73c-fc75-487b-9a3b-83a198d855c9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:10.748074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8c73c-fc75-487b-9a3b-83a198d855c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:10.836916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5fc8c73c-fc75-487b-9a3b-83a198d855c9 None None] MFA Rules not processed for user `9b31f094a84a4347a2d17664040338d4`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:10.838302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5fc8c73c-fc75-487b-9a3b-83a198d855c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:10.989208 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2055/4175] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:10 2022] POST /identity/v3/auth/tokens => generated 3277 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:10.990763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a65624-a3fa-40c5-9aac-857b58424297 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:10.990763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a65624-a3fa-40c5-9aac-857b58424297 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:10.990763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a65624-a3fa-40c5-9aac-857b58424297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:11.027348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-58a65624-a3fa-40c5-9aac-857b58424297 None None] MFA Rules not processed for user `9b31f094a84a4347a2d17664040338d4`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:11.028733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-58a65624-a3fa-40c5-9aac-857b58424297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:11.043825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8c73c-fc75-487b-9a3b-83a198d855c9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:11.046785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5fc8c73c-fc75-487b-9a3b-83a198d855c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:11.049160 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2121/4176] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:10 2022] POST /identity/v3/auth/tokens => generated 3277 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:03:11.053016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a65624-a3fa-40c5-9aac-857b58424297 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:11.071548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a65624-a3fa-40c5-9aac-857b58424297 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:11.076080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17e39b-b5ff-4737-8a29-23a61f79567c tempest-TrustsV3TestJSON-1902861609 tempest-user-342872341] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b31f094a84a4347a2d17664040338d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '329f6b529b864969aad8ebf4beffa4c9', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-47717733', 'tempest-NotDelegatedRole-491660966'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:11.076709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17e39b-b5ff-4737-8a29-23a61f79567c tempest-TrustsV3TestJSON-1902861609 tempest-user-342872341] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:11.077019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17e39b-b5ff-4737-8a29-23a61f79567c tempest-TrustsV3TestJSON-1902861609 tempest-user-342872341] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:11.077333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17e39b-b5ff-4737-8a29-23a61f79567c tempest-TrustsV3TestJSON-1902861609 tempest-user-342872341] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:11.077973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17e39b-b5ff-4737-8a29-23a61f79567c tempest-TrustsV3TestJSON-1902861609 tempest-user-342872341] RBAC: Authorizing `identity:create_trust()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:11.079149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17e39b-b5ff-4737-8a29-23a61f79567c tempest-TrustsV3TestJSON-1902861609 tempest-user-342872341] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:11.106819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:11.204794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8c73c-fc75-487b-9a3b-83a198d855c9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:11.211828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:11.213535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:11.214018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:11.214500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] PATH_INFO: `/v3/users/3a394ae937c54b2a82c89628d4feeb72` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:11.216178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a394ae937c54b2a82c89628d4feeb72)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:11.218418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-5b17e39b-b5ff-4737-8a29-23a61f79567c tempest-TrustsV3TestJSON-1902861609 tempest-user-342872341] Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 44, in parse_isotime Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return iso8601.parse_date(timestr) Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/iso8601/iso8601.py", line 127, in parse_date Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise ParseError(f"Unable to parse date string {datestring!r}") Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application iso8601.iso8601.ParseError: Unable to parse date string 'bad.123Z' Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 346, in parse_expiration_date Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application expiration_time = timeutils.parse_isotime(expiration_date) Sep 03 21:03:11.289156 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 46, in parse_isotime Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise ValueError(str(e)) Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ValueError: Unable to parse date string 'bad.123Z' Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 296, in post Sep 03 21:03:11.290262 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application trust['expires_at'] = self._parse_expiration_date( Sep 03 21:03:11.296566 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 127, in _parse_expiration_date Sep 03 21:03:11.296566 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return utils.parse_expiration_date(expiration_date) Sep 03 21:03:11.296566 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 348, in parse_expiration_date Sep 03 21:03:11.296566 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ValidationTimeStampError() Sep 03 21:03:11.296566 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Sep 03 21:03:11.296566 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:03:11.296566 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2122/4177] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Sep 3 21:03:11 2022] POST /identity/v3/OS-TRUST/trusts => generated 227 bytes in 238 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 03 21:03:11.296566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17e39b-b5ff-4737-8a29-23a61f79567c tempest-TrustsV3TestJSON-1902861609 tempest-user-342872341] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:11.303310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:11.328922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17e39b-b5ff-4737-8a29-23a61f79567c tempest-TrustsV3TestJSON-1902861609 tempest-user-342872341] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:11.340983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56ae08-a438-4611-b9bd-e25bdbcf327f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:11.341847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56ae08-a438-4611-b9bd-e25bdbcf327f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:11.346084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56ae08-a438-4611-b9bd-e25bdbcf327f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:11.347378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56ae08-a438-4611-b9bd-e25bdbcf327f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles/84692c59133546a3b27967cf14b53bd7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:11.354174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56ae08-a438-4611-b9bd-e25bdbcf327f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_role(role_id=84692c59133546a3b27967cf14b53bd7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:11.358861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56ae08-a438-4611-b9bd-e25bdbcf327f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:11.383529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a394ae937c54b2a82c89628d4feeb72'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:11.428756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a394ae937c54b2a82c89628d4feeb72'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:11.430817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a394ae937c54b2a82c89628d4feeb72'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:11.446677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a394ae937c54b2a82c89628d4feeb72'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:11.458503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a394ae937c54b2a82c89628d4feeb72'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:11.461668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c56ae08-a438-4611-b9bd-e25bdbcf327f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invalidating the token cache because role 84692c59133546a3b27967cf14b53bd7 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:11.462723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c56ae08-a438-4611-b9bd-e25bdbcf327f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:11.464395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c56ae08-a438-4611-b9bd-e25bdbcf327f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:11.468955 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2056/4179] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:11 2022] DELETE /identity/v3/users/3a394ae937c54b2a82c89628d4feeb72 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:11.469461 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2123/4179] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:11 2022] DELETE /identity/v3/roles/84692c59133546a3b27967cf14b53bd7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:11.472182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:11.474178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:11.569670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c59e1-7ffe-4414-8262-59aa6a9e1192 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:11.574978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf55e5-5756-4f30-9c70-cdbea6c8b2a5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:11.575766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf55e5-5756-4f30-9c70-cdbea6c8b2a5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:11.576065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf55e5-5756-4f30-9c70-cdbea6c8b2a5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:11.576348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf55e5-5756-4f30-9c70-cdbea6c8b2a5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles/949c2831b8f34e3ea6247e0f878e19e3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:11.577993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf55e5-5756-4f30-9c70-cdbea6c8b2a5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_role(role_id=949c2831b8f34e3ea6247e0f878e19e3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:11.579751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf55e5-5756-4f30-9c70-cdbea6c8b2a5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:11.637314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a2bf55e5-5756-4f30-9c70-cdbea6c8b2a5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invalidating the token cache because role 949c2831b8f34e3ea6247e0f878e19e3 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:11.637787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a2bf55e5-5756-4f30-9c70-cdbea6c8b2a5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:11.638475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a2bf55e5-5756-4f30-9c70-cdbea6c8b2a5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:11.640585 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2057/4180] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:11 2022] DELETE /identity/v3/roles/949c2831b8f34e3ea6247e0f878e19e3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:11.647307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56ae08-a438-4611-b9bd-e25bdbcf327f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:11.652854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9c56ae08-a438-4611-b9bd-e25bdbcf327f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:11.723863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf55e5-5756-4f30-9c70-cdbea6c8b2a5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:11.729324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a2bf55e5-5756-4f30-9c70-cdbea6c8b2a5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:11.750844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:11.814194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56ae08-a438-4611-b9bd-e25bdbcf327f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:11.920552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf55e5-5756-4f30-9c70-cdbea6c8b2a5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:11.932192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:11.933256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:11.935215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:11.936374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] PATH_INFO: `/v3/projects/30d39cd627084dda8cb00e993f5d19a1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:11.940024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=30d39cd627084dda8cb00e993f5d19a1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:11.944955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7009a9-9a95-4799-9c56-233d1ced5876 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:11.945571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:11.945873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7009a9-9a95-4799-9c56-233d1ced5876 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:11.946169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7009a9-9a95-4799-9c56-233d1ced5876 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:11.946823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7009a9-9a95-4799-9c56-233d1ced5876 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users/9b31f094a84a4347a2d17664040338d4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:11.949392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7009a9-9a95-4799-9c56-233d1ced5876 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_user(user_id=9b31f094a84a4347a2d17664040338d4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:11.955097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7009a9-9a95-4799-9c56-233d1ced5876 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:12.041963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0a7009a9-9a95-4799-9c56-233d1ced5876 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b31f094a84a4347a2d17664040338d4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.061793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30d39cd627084dda8cb00e993f5d19a1', 'request_id': 'req-c3c4b26a-bdcd-4561-a44a-462377d15f4c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.078816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0a7009a9-9a95-4799-9c56-233d1ced5876 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b31f094a84a4347a2d17664040338d4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.086099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:12.094095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0a7009a9-9a95-4799-9c56-233d1ced5876 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b31f094a84a4347a2d17664040338d4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.094946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0a7009a9-9a95-4799-9c56-233d1ced5876 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b31f094a84a4347a2d17664040338d4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.099182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30d39cd627084dda8cb00e993f5d19a1', 'request_id': 'req-c3c4b26a-bdcd-4561-a44a-462377d15f4c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.099973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0a7009a9-9a95-4799-9c56-233d1ced5876 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b31f094a84a4347a2d17664040338d4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.113081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30d39cd627084dda8cb00e993f5d19a1', 'request_id': 'req-c3c4b26a-bdcd-4561-a44a-462377d15f4c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.146411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] The token cache is being invalidate because project 30d39cd627084dda8cb00e993f5d19a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:12.147044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.147988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.150219 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2058/4182] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:11 2022] DELETE /identity/v3/projects/30d39cd627084dda8cb00e993f5d19a1 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:12.151606 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2124/4182] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:11 2022] DELETE /identity/v3/users/9b31f094a84a4347a2d17664040338d4 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:12.152545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:12.155210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ab4fd-5e07-4e80-8189-7920c13da803 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:12.155486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ab4fd-5e07-4e80-8189-7920c13da803 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:12.156421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ab4fd-5e07-4e80-8189-7920c13da803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:12.173493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:12.203541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-788ab4fd-5e07-4e80-8189-7920c13da803 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:12.205640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-788ab4fd-5e07-4e80-8189-7920c13da803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:12.327642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4b26a-bdcd-4561-a44a-462377d15f4c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:12.330228 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2125/4183] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:12.338902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22ad50-849d-49ba-9325-c44dbb5ea227 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:12.339526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22ad50-849d-49ba-9325-c44dbb5ea227 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:12.340030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22ad50-849d-49ba-9325-c44dbb5ea227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:12.365504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6e22ad50-849d-49ba-9325-c44dbb5ea227 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:12.367716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6e22ad50-849d-49ba-9325-c44dbb5ea227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:12.387049 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2126/4184] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:03:12.392681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22ad50-849d-49ba-9325-c44dbb5ea227 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:12.418523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:12.419521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22ad50-849d-49ba-9325-c44dbb5ea227 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:12.430109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dae672-7446-4e38-9d06-685eebc0a756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:12.430818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dae672-7446-4e38-9d06-685eebc0a756 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:12.431096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dae672-7446-4e38-9d06-685eebc0a756 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:12.431375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dae672-7446-4e38-9d06-685eebc0a756 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:12.435973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dae672-7446-4e38-9d06-685eebc0a756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:03:12.436403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dae672-7446-4e38-9d06-685eebc0a756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:12.440796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dae672-7446-4e38-9d06-685eebc0a756 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:12.442405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:12.443276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:12.443637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:12.443990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/329f6b529b864969aad8ebf4beffa4c9` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:12.445593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_project(project_id=329f6b529b864969aad8ebf4beffa4c9)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:12.448926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:12.473075 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2127/4185] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:12.475338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dae672-7446-4e38-9d06-685eebc0a756 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:12.494642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dae672-7446-4e38-9d06-685eebc0a756 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:12.502398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02a3e2-9293-4b30-9373-3f2c75c59056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:12.502398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02a3e2-9293-4b30-9373-3f2c75c59056 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:12.502877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02a3e2-9293-4b30-9373-3f2c75c59056 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:12.503443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02a3e2-9293-4b30-9373-3f2c75c59056 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:12.504086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02a3e2-9293-4b30-9373-3f2c75c59056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:12.506310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02a3e2-9293-4b30-9373-3f2c75c59056 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:12.537091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329f6b529b864969aad8ebf4beffa4c9', 'request_id': 'req-5abb718f-80a2-41cc-89d8-44cf443acc67'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.549456 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2128/4186] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:03:12 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:12.550730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02a3e2-9293-4b30-9373-3f2c75c59056 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:12.553980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329f6b529b864969aad8ebf4beffa4c9', 'request_id': 'req-5abb718f-80a2-41cc-89d8-44cf443acc67'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.565106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329f6b529b864969aad8ebf4beffa4c9', 'request_id': 'req-5abb718f-80a2-41cc-89d8-44cf443acc67'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.579392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] The token cache is being invalidate because project 329f6b529b864969aad8ebf4beffa4c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:12.580047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.581047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:12.583094 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2059/4187] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:12 2022] DELETE /identity/v3/projects/329f6b529b864969aad8ebf4beffa4c9 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:12.589633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:12.593438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:12.626942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02a3e2-9293-4b30-9373-3f2c75c59056 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:12.632277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4028a-9dd2-4cfe-96fa-b6e611adb56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:12.632933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4028a-9dd2-4cfe-96fa-b6e611adb56b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:12.633212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4028a-9dd2-4cfe-96fa-b6e611adb56b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:12.633498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4028a-9dd2-4cfe-96fa-b6e611adb56b admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:12.634144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4028a-9dd2-4cfe-96fa-b6e611adb56b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:12.637171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4028a-9dd2-4cfe-96fa-b6e611adb56b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:12.664071 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2129/4188] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:12 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:03:12.665064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4028a-9dd2-4cfe-96fa-b6e611adb56b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:12.666872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-57e4028a-9dd2-4cfe-96fa-b6e611adb56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:12.677556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb718f-80a2-41cc-89d8-44cf443acc67 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:12.679261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4028a-9dd2-4cfe-96fa-b6e611adb56b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:12.684687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ea193-0c7e-4d7c-bd1b-f34ecf7ff8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:12.685335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ea193-0c7e-4d7c-bd1b-f34ecf7ff8ed admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:12.685626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ea193-0c7e-4d7c-bd1b-f34ecf7ff8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:12.685909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ea193-0c7e-4d7c-bd1b-f34ecf7ff8ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:12.686561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ea193-0c7e-4d7c-bd1b-f34ecf7ff8ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:12.688172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ea193-0c7e-4d7c-bd1b-f34ecf7ff8ed admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:12.698729 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2130/4189] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:03:12.702718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ea193-0c7e-4d7c-bd1b-f34ecf7ff8ed admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:12.712588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ea193-0c7e-4d7c-bd1b-f34ecf7ff8ed admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:12.718155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c740a-a7a3-46bb-a781-14e8f934a687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:12.718743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c740a-a7a3-46bb-a781-14e8f934a687 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:12.719032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c740a-a7a3-46bb-a781-14e8f934a687 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:12.719313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c740a-a7a3-46bb-a781-14e8f934a687 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:12.719953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c740a-a7a3-46bb-a781-14e8f934a687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:12.721899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c740a-a7a3-46bb-a781-14e8f934a687 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:12.733883 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2131/4190] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:03:12.737939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c740a-a7a3-46bb-a781-14e8f934a687 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:12.742089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34e6aa-76c3-4880-84ca-304d63cffce3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:12.742704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34e6aa-76c3-4880-84ca-304d63cffce3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:12.742993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34e6aa-76c3-4880-84ca-304d63cffce3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:12.743276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34e6aa-76c3-4880-84ca-304d63cffce3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:12.743897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34e6aa-76c3-4880-84ca-304d63cffce3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:12.746343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34e6aa-76c3-4880-84ca-304d63cffce3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:12.748653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c740a-a7a3-46bb-a781-14e8f934a687 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:12.753423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8926c9-21e3-43d7-8fb5-92d94dec23aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:12.754904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8926c9-21e3-43d7-8fb5-92d94dec23aa admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:12.755195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8926c9-21e3-43d7-8fb5-92d94dec23aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:12.755482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8926c9-21e3-43d7-8fb5-92d94dec23aa admin admin] PATH_INFO: `/v3/projects/6c96d2a581dd4fc299fc5020c93efe00/users/cb94dfca0ec44bf4853be254984519ca/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:12.769336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:12.774862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8926c9-21e3-43d7-8fb5-92d94dec23aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c96d2a581dd4fc299fc5020c93efe00, user_id=cb94dfca0ec44bf4853be254984519ca, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:12.778661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8926c9-21e3-43d7-8fb5-92d94dec23aa admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:12.783121 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2060/4191] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:03:12 2022] POST /identity/v3/projects => generated 328 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:03:12.787427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34e6aa-76c3-4880-84ca-304d63cffce3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:12.801320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:12.801502 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2132/4192] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:12 2022] PUT /identity/v3/projects/6c96d2a581dd4fc299fc5020c93efe00/users/cb94dfca0ec44bf4853be254984519ca/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:12.807276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacad3b-629c-48f0-be67-f0db6d0fea0d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:12.807512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacad3b-629c-48f0-be67-f0db6d0fea0d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:12.807844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacad3b-629c-48f0-be67-f0db6d0fea0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:12.829379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5bacad3b-629c-48f0-be67-f0db6d0fea0d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:12.830613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5bacad3b-629c-48f0-be67-f0db6d0fea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:12.895191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34e6aa-76c3-4880-84ca-304d63cffce3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:12.914887 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2133/4193] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:12.923353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacad3b-629c-48f0-be67-f0db6d0fea0d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:12.927359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5bacad3b-629c-48f0-be67-f0db6d0fea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:12.966155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac7824-84b6-4a7c-8573-7490eaaf601e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:12.966775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac7824-84b6-4a7c-8573-7490eaaf601e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:12.967063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac7824-84b6-4a7c-8573-7490eaaf601e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:12.967343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac7824-84b6-4a7c-8573-7490eaaf601e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:12.967997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac7824-84b6-4a7c-8573-7490eaaf601e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:12.969963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac7824-84b6-4a7c-8573-7490eaaf601e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:12.993709 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2061/4194] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:12 2022] POST /identity/v3/users => generated 410 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:12.997892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac7824-84b6-4a7c-8573-7490eaaf601e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:12.998629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacad3b-629c-48f0-be67-f0db6d0fea0d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:13.002657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd6bd3-82f7-4348-b632-310aec6d20c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:13.003273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd6bd3-82f7-4348-b632-310aec6d20c3 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.003690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd6bd3-82f7-4348-b632-310aec6d20c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.004037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd6bd3-82f7-4348-b632-310aec6d20c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.010890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac7824-84b6-4a7c-8573-7490eaaf601e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:13.016227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a223da-b539-43dc-8289-03af72238f4e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:13.017191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a223da-b539-43dc-8289-03af72238f4e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.017820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a223da-b539-43dc-8289-03af72238f4e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.018110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a223da-b539-43dc-8289-03af72238f4e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.018802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a223da-b539-43dc-8289-03af72238f4e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:13.019396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd6bd3-82f7-4348-b632-310aec6d20c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:13.020556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a223da-b539-43dc-8289-03af72238f4e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:13.021009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd6bd3-82f7-4348-b632-310aec6d20c3 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:13.034032 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2134/4195] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:13.044457 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2062/4196] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:03:12 2022] POST /identity/v3/roles => generated 244 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:03:13.045948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd6bd3-82f7-4348-b632-310aec6d20c3 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:13.058567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd6bd3-82f7-4348-b632-310aec6d20c3 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:13.063795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33af57-6fee-4566-b9f8-48daaa910cb5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:13.064409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33af57-6fee-4566-b9f8-48daaa910cb5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.064726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33af57-6fee-4566-b9f8-48daaa910cb5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.065128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33af57-6fee-4566-b9f8-48daaa910cb5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.065902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33af57-6fee-4566-b9f8-48daaa910cb5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:13.067667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33af57-6fee-4566-b9f8-48daaa910cb5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:13.086897 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2135/4197] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:03:13 2022] POST /identity/v3/roles => generated 246 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:13.088336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a223da-b539-43dc-8289-03af72238f4e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:13.100878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a223da-b539-43dc-8289-03af72238f4e tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:13.106355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d700661-594f-42fa-9ede-5f61ac441005 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:13.107039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d700661-594f-42fa-9ede-5f61ac441005 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.107332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d700661-594f-42fa-9ede-5f61ac441005 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.107617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d700661-594f-42fa-9ede-5f61ac441005 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/55092a2e8e114fba9cfedca271bd5aa8/users/b3013820e4204186a1738934e6090756/roles/c45a4d35dd634f459e8d16ffc03d4e3a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.131368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d700661-594f-42fa-9ede-5f61ac441005 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_grant(project_id=55092a2e8e114fba9cfedca271bd5aa8, user_id=b3013820e4204186a1738934e6090756, role_id=c45a4d35dd634f459e8d16ffc03d4e3a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:13.134623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d700661-594f-42fa-9ede-5f61ac441005 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:13.153898 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2063/4198] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:13 2022] PUT /identity/v3/projects/55092a2e8e114fba9cfedca271bd5aa8/users/b3013820e4204186a1738934e6090756/roles/c45a4d35dd634f459e8d16ffc03d4e3a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:13.158106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33af57-6fee-4566-b9f8-48daaa910cb5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:13.173385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33af57-6fee-4566-b9f8-48daaa910cb5 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:13.288828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58767cdd-3c43-494a-a490-dccede64ae2a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:13.290249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58767cdd-3c43-494a-a490-dccede64ae2a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.292111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58767cdd-3c43-494a-a490-dccede64ae2a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.292247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58767cdd-3c43-494a-a490-dccede64ae2a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/55092a2e8e114fba9cfedca271bd5aa8/users/b3013820e4204186a1738934e6090756/roles/2496f8cb16e94ce5b9d9232b22a41962` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.297480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58767cdd-3c43-494a-a490-dccede64ae2a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_grant(project_id=55092a2e8e114fba9cfedca271bd5aa8, user_id=b3013820e4204186a1738934e6090756, role_id=2496f8cb16e94ce5b9d9232b22a41962)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:13.302973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58767cdd-3c43-494a-a490-dccede64ae2a tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:13.325425 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2136/4199] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:13 2022] PUT /identity/v3/projects/55092a2e8e114fba9cfedca271bd5aa8/users/b3013820e4204186a1738934e6090756/roles/2496f8cb16e94ce5b9d9232b22a41962 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:13.329469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d700661-594f-42fa-9ede-5f61ac441005 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:13.345431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d700661-594f-42fa-9ede-5f61ac441005 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:13.422594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0362987-a4c3-4756-b1bd-703207d2a12f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:13.423256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0362987-a4c3-4756-b1bd-703207d2a12f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.423624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0362987-a4c3-4756-b1bd-703207d2a12f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.423870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0362987-a4c3-4756-b1bd-703207d2a12f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.424625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0362987-a4c3-4756-b1bd-703207d2a12f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:13.426635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0362987-a4c3-4756-b1bd-703207d2a12f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:13.462705 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2064/4200] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:13 2022] GET /identity/v3/users => generated 8339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:13.467461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dfbde-b5a8-40e7-b55a-7f23b043d6a9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.467761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dfbde-b5a8-40e7-b55a-7f23b043d6a9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.468037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dfbde-b5a8-40e7-b55a-7f23b043d6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.521842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:13.572675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-875dfbde-b5a8-40e7-b55a-7f23b043d6a9 None None] MFA Rules not processed for user `b3013820e4204186a1738934e6090756`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:13.573959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-875dfbde-b5a8-40e7-b55a-7f23b043d6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:13.676282 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2137/4201] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:13 2022] POST /identity/v3/auth/tokens => generated 3277 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:13.686104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc6fa3-cf20-4c9c-921d-afa4fb63cf46 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.686654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc6fa3-cf20-4c9c-921d-afa4fb63cf46 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.686654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc6fa3-cf20-4c9c-921d-afa4fb63cf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.696292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:13.713228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0bcc6fa3-cf20-4c9c-921d-afa4fb63cf46 None None] MFA Rules not processed for user `b3013820e4204186a1738934e6090756`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:13.714785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0bcc6fa3-cf20-4c9c-921d-afa4fb63cf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:13.725659 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2065/4202] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:13 2022] POST /identity/v3/auth/tokens => generated 3277 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:13.734261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dfbde-b5a8-40e7-b55a-7f23b043d6a9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:13.748841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dfbde-b5a8-40e7-b55a-7f23b043d6a9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:13.753225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4c456-065b-4883-aad4-e176a3dd6ed8 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3013820e4204186a1738934e6090756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55092a2e8e114fba9cfedca271bd5aa8', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1936206497', 'tempest-NotDelegatedRole-932097662'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:13.753930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4c456-065b-4883-aad4-e176a3dd6ed8 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.754332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4c456-065b-4883-aad4-e176a3dd6ed8 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.754676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4c456-065b-4883-aad4-e176a3dd6ed8 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.755582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4c456-065b-4883-aad4-e176a3dd6ed8 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorizing `identity:create_trust()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:13.756832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4c456-065b-4883-aad4-e176a3dd6ed8 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:13.831756 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2138/4203] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Sep 3 21:03:13 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:13.836005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc6fa3-cf20-4c9c-921d-afa4fb63cf46 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:13.848519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc6fa3-cf20-4c9c-921d-afa4fb63cf46 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:13.853110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca619c-8bf8-4f24-b32d-0f464e5b5dcb tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3013820e4204186a1738934e6090756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55092a2e8e114fba9cfedca271bd5aa8', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1936206497', 'tempest-NotDelegatedRole-932097662'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:13.854603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca619c-8bf8-4f24-b32d-0f464e5b5dcb tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.854603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca619c-8bf8-4f24-b32d-0f464e5b5dcb tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.854809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca619c-8bf8-4f24-b32d-0f464e5b5dcb tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] PATH_INFO: `/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.863404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca619c-8bf8-4f24-b32d-0f464e5b5dcb tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorizing `identity:get_trust(trust_id=b49bd7686d9e4d219831400acd73008d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:13.864928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca619c-8bf8-4f24-b32d-0f464e5b5dcb tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:13.873859 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2066/4204] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:03:13 2022] GET /identity/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d => generated 834 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:03:13.877947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4c456-065b-4883-aad4-e176a3dd6ed8 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:13.887905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4c456-065b-4883-aad4-e176a3dd6ed8 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:13.892252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341826c-f99e-4126-8738-ec7b383ae577 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3013820e4204186a1738934e6090756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55092a2e8e114fba9cfedca271bd5aa8', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1936206497', 'tempest-NotDelegatedRole-932097662'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:13.892885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341826c-f99e-4126-8738-ec7b383ae577 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.893196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341826c-f99e-4126-8738-ec7b383ae577 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.893504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341826c-f99e-4126-8738-ec7b383ae577 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] PATH_INFO: `/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.899681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341826c-f99e-4126-8738-ec7b383ae577 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=b49bd7686d9e4d219831400acd73008d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:13.901159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341826c-f99e-4126-8738-ec7b383ae577 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:13.912014 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2139/4205] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Sep 3 21:03:13 2022] GET /identity/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d/roles => generated 371 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:13.913589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca619c-8bf8-4f24-b32d-0f464e5b5dcb tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:13.924269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca619c-8bf8-4f24-b32d-0f464e5b5dcb tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:13.929970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad06ca-a906-4a08-8e59-522169494f93 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3013820e4204186a1738934e6090756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55092a2e8e114fba9cfedca271bd5aa8', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1936206497', 'tempest-NotDelegatedRole-932097662'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:13.930670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad06ca-a906-4a08-8e59-522169494f93 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.931421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad06ca-a906-4a08-8e59-522169494f93 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.931421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad06ca-a906-4a08-8e59-522169494f93 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] PATH_INFO: `/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d/roles/c45a4d35dd634f459e8d16ffc03d4e3a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.945033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad06ca-a906-4a08-8e59-522169494f93 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorizing `identity:get_role_for_trust(trust_id=b49bd7686d9e4d219831400acd73008d, role_id=c45a4d35dd634f459e8d16ffc03d4e3a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:13.945033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad06ca-a906-4a08-8e59-522169494f93 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:13.948683 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2067/4206] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Sep 3 21:03:13 2022] GET /identity/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d/roles/c45a4d35dd634f459e8d16ffc03d4e3a => generated 244 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:03:13.953321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341826c-f99e-4126-8738-ec7b383ae577 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:13.965369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341826c-f99e-4126-8738-ec7b383ae577 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:13.970132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79156b11-e235-43aa-a63a-4831d4e281bf tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3013820e4204186a1738934e6090756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55092a2e8e114fba9cfedca271bd5aa8', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1936206497', 'tempest-NotDelegatedRole-932097662'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:13.970869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79156b11-e235-43aa-a63a-4831d4e281bf tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] REQUEST_METHOD: `HEAD` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:13.971160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79156b11-e235-43aa-a63a-4831d4e281bf tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:13.971445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79156b11-e235-43aa-a63a-4831d4e281bf tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] PATH_INFO: `/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d/roles/c45a4d35dd634f459e8d16ffc03d4e3a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:13.980479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79156b11-e235-43aa-a63a-4831d4e281bf tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorizing `identity:get_role_for_trust(trust_id=b49bd7686d9e4d219831400acd73008d, role_id=c45a4d35dd634f459e8d16ffc03d4e3a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:13.982333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79156b11-e235-43aa-a63a-4831d4e281bf tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:13.992460 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2140/4207] 10.222.0.16 () {58 vars in 1369 bytes} [Sat Sep 3 21:03:13 2022] HEAD /identity/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d/roles/c45a4d35dd634f459e8d16ffc03d4e3a => generated 0 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 03 21:03:13.997749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad06ca-a906-4a08-8e59-522169494f93 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:14.008035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad06ca-a906-4a08-8e59-522169494f93 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:14.012921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d1ed0-9867-4607-a3df-bdda8778c8e0 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3013820e4204186a1738934e6090756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55092a2e8e114fba9cfedca271bd5aa8', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1936206497', 'tempest-NotDelegatedRole-932097662'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:14.013948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d1ed0-9867-4607-a3df-bdda8778c8e0 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:14.015068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d1ed0-9867-4607-a3df-bdda8778c8e0 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:14.015389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d1ed0-9867-4607-a3df-bdda8778c8e0 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] PATH_INFO: `/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d/roles/2496f8cb16e94ce5b9d9232b22a41962` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:14.030427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d1ed0-9867-4607-a3df-bdda8778c8e0 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorizing `identity:get_role_for_trust(trust_id=b49bd7686d9e4d219831400acd73008d, role_id=2496f8cb16e94ce5b9d9232b22a41962)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:14.032176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d1ed0-9867-4607-a3df-bdda8778c8e0 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-e95d1ed0-9867-4607-a3df-bdda8778c8e0 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Could not find role: 2496f8cb16e94ce5b9d9232b22a41962.: keystone.exception.RoleNotFound: Could not find role: 2496f8cb16e94ce5b9d9232b22a41962. Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 21:03:14.046640 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 2496f8cb16e94ce5b9d9232b22a41962. Sep 03 21:03:14.048022 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:03:14.048022 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2068/4208] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Sep 3 21:03:13 2022] GET /identity/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d/roles/2496f8cb16e94ce5b9d9232b22a41962 => generated 110 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:03:14.052107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79156b11-e235-43aa-a63a-4831d4e281bf tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:14.061791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79156b11-e235-43aa-a63a-4831d4e281bf tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:14.066697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fae40-3977-4c76-9ef5-afc2573bf21b tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3013820e4204186a1738934e6090756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55092a2e8e114fba9cfedca271bd5aa8', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1936206497', 'tempest-NotDelegatedRole-932097662'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:14.067540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fae40-3977-4c76-9ef5-afc2573bf21b tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] REQUEST_METHOD: `HEAD` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:14.067858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fae40-3977-4c76-9ef5-afc2573bf21b tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:14.068178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fae40-3977-4c76-9ef5-afc2573bf21b tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] PATH_INFO: `/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d/roles/2496f8cb16e94ce5b9d9232b22a41962` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:14.075634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fae40-3977-4c76-9ef5-afc2573bf21b tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorizing `identity:get_role_for_trust(trust_id=b49bd7686d9e4d219831400acd73008d, role_id=2496f8cb16e94ce5b9d9232b22a41962)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:14.077186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fae40-3977-4c76-9ef5-afc2573bf21b tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-e85fae40-3977-4c76-9ef5-afc2573bf21b tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Could not find role: 2496f8cb16e94ce5b9d9232b22a41962.: keystone.exception.RoleNotFound: Could not find role: 2496f8cb16e94ce5b9d9232b22a41962. Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 21:03:14.089456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 2496f8cb16e94ce5b9d9232b22a41962. Sep 03 21:03:14.093908 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:03:14.093908 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2141/4209] 10.222.0.16 () {58 vars in 1369 bytes} [Sat Sep 3 21:03:14 2022] HEAD /identity/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d/roles/2496f8cb16e94ce5b9d9232b22a41962 => generated 0 bytes in 39 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:03:14.094868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d1ed0-9867-4607-a3df-bdda8778c8e0 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:14.107033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d1ed0-9867-4607-a3df-bdda8778c8e0 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:14.112342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3013820e4204186a1738934e6090756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55092a2e8e114fba9cfedca271bd5aa8', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1936206497', 'tempest-NotDelegatedRole-932097662'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:14.113026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:14.113454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:14.113740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] PATH_INFO: `/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:14.121302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorizing `identity:delete_trust(trust_id=b49bd7686d9e4d219831400acd73008d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:14.123057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:14.143329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'b49bd7686d9e4d219831400acd73008d', 'request_id': 'req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:14.151308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'b49bd7686d9e4d219831400acd73008d', 'request_id': 'req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:14.152015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'b49bd7686d9e4d219831400acd73008d', 'request_id': 'req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:14.159024 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2069/4210] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Sep 3 21:03:14 2022] DELETE /identity/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:14.163336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fae40-3977-4c76-9ef5-afc2573bf21b tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:14.165375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e85fae40-3977-4c76-9ef5-afc2573bf21b tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:14.186526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fae40-3977-4c76-9ef5-afc2573bf21b tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:14.191399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e351eb3-9cd3-4ebb-895c-0fedf549ef87 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3013820e4204186a1738934e6090756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55092a2e8e114fba9cfedca271bd5aa8', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1936206497', 'tempest-NotDelegatedRole-932097662'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:14.192079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e351eb3-9cd3-4ebb-895c-0fedf549ef87 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:14.192364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e351eb3-9cd3-4ebb-895c-0fedf549ef87 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:14.193268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e351eb3-9cd3-4ebb-895c-0fedf549ef87 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] PATH_INFO: `/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-3e351eb3-9cd3-4ebb-895c-0fedf549ef87 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Could not find trust: b49bd7686d9e4d219831400acd73008d.: keystone.exception.TrustNotFound: Could not find trust: b49bd7686d9e4d219831400acd73008d. Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 03 21:03:14.203163 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: b49bd7686d9e4d219831400acd73008d. Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:03:14.205173 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2142/4211] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:03:14 2022] GET /identity/v3/OS-TRUST/trusts/b49bd7686d9e4d219831400acd73008d => generated 111 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:03:14.208837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:14.210943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:14.229980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40ae77-70b2-45c5-bdac-931f84d8bf8f tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:14.235493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab21de-77a0-4061-9ce9-61e6aaced86b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:14.236344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab21de-77a0-4061-9ce9-61e6aaced86b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:14.236733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab21de-77a0-4061-9ce9-61e6aaced86b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:14.237148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab21de-77a0-4061-9ce9-61e6aaced86b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles/2496f8cb16e94ce5b9d9232b22a41962` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:14.238724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab21de-77a0-4061-9ce9-61e6aaced86b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_role(role_id=2496f8cb16e94ce5b9d9232b22a41962)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:14.240498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab21de-77a0-4061-9ce9-61e6aaced86b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:14.309213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-91ab21de-77a0-4061-9ce9-61e6aaced86b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invalidating the token cache because role 2496f8cb16e94ce5b9d9232b22a41962 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:14.309732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-91ab21de-77a0-4061-9ce9-61e6aaced86b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:14.310520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-91ab21de-77a0-4061-9ce9-61e6aaced86b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:14.312885 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2070/4212] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:14 2022] DELETE /identity/v3/roles/2496f8cb16e94ce5b9d9232b22a41962 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:14.316946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e351eb3-9cd3-4ebb-895c-0fedf549ef87 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:14.318819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3e351eb3-9cd3-4ebb-895c-0fedf549ef87 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:14.346352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e351eb3-9cd3-4ebb-895c-0fedf549ef87 tempest-TrustsV3TestJSON-797064660 tempest-user-11019754] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:14.393605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:14.452140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6e99c-b3c9-4a69-b30e-23903a65632f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:14.453253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6e99c-b3c9-4a69-b30e-23903a65632f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:14.453554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6e99c-b3c9-4a69-b30e-23903a65632f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:14.453971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6e99c-b3c9-4a69-b30e-23903a65632f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles/c45a4d35dd634f459e8d16ffc03d4e3a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:14.455445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6e99c-b3c9-4a69-b30e-23903a65632f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_role(role_id=c45a4d35dd634f459e8d16ffc03d4e3a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:14.457183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6e99c-b3c9-4a69-b30e-23903a65632f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:14.526417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9fd6e99c-b3c9-4a69-b30e-23903a65632f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invalidating the token cache because role c45a4d35dd634f459e8d16ffc03d4e3a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:14.527813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9fd6e99c-b3c9-4a69-b30e-23903a65632f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:14.528815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9fd6e99c-b3c9-4a69-b30e-23903a65632f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:14.536223 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2143/4213] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:14 2022] DELETE /identity/v3/roles/c45a4d35dd634f459e8d16ffc03d4e3a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:14.540167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab21de-77a0-4061-9ce9-61e6aaced86b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:14.554075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab21de-77a0-4061-9ce9-61e6aaced86b tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:14.687808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:14.689346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:14.690072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:14.690572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users/b3013820e4204186a1738934e6090756` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:14.692050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_user(user_id=b3013820e4204186a1738934e6090756)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:14.694343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:14.736692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:14.787855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3013820e4204186a1738934e6090756'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:14.815371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3013820e4204186a1738934e6090756'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:14.816294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3013820e4204186a1738934e6090756'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:14.830881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3013820e4204186a1738934e6090756'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:14.843638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3013820e4204186a1738934e6090756'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:14.852129 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2071/4214] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:14 2022] DELETE /identity/v3/users/b3013820e4204186a1738934e6090756 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:14.856557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6e99c-b3c9-4a69-b30e-23903a65632f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:14.858348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9fd6e99c-b3c9-4a69-b30e-23903a65632f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:14.878661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6e99c-b3c9-4a69-b30e-23903a65632f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:14.984974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:14.985849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:14.986150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:14.986436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/55092a2e8e114fba9cfedca271bd5aa8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:14.988309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_project(project_id=55092a2e8e114fba9cfedca271bd5aa8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:14.991019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:15.096248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55092a2e8e114fba9cfedca271bd5aa8', 'request_id': 'req-7ccf8915-0a11-4d4a-9150-3cba2a218337'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:15.113668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55092a2e8e114fba9cfedca271bd5aa8', 'request_id': 'req-7ccf8915-0a11-4d4a-9150-3cba2a218337'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:15.136025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55092a2e8e114fba9cfedca271bd5aa8', 'request_id': 'req-7ccf8915-0a11-4d4a-9150-3cba2a218337'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:15.164291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] The token cache is being invalidate because project 55092a2e8e114fba9cfedca271bd5aa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:15.164990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:15.166396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:15.180007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:15.180275 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2144/4215] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:14 2022] DELETE /identity/v3/projects/55092a2e8e114fba9cfedca271bd5aa8 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:15.182326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:15.212053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c3c43-b45d-496e-baf4-b95eff9520cb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:15.328164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f9f88-bccc-456c-aa17-1ee10d61e3c8 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:15.329055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f9f88-bccc-456c-aa17-1ee10d61e3c8 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:15.329181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f9f88-bccc-456c-aa17-1ee10d61e3c8 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:15.330036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f9f88-bccc-456c-aa17-1ee10d61e3c8 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:15.330755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f9f88-bccc-456c-aa17-1ee10d61e3c8 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:15.333006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f9f88-bccc-456c-aa17-1ee10d61e3c8 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:15.375580 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2072/4216] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:03:15 2022] POST /identity/v3/projects => generated 329 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:15.381080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:15.386159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:15.406585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf8915-0a11-4d4a-9150-3cba2a218337 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:15.439875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:15.541590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c8f91-17e2-4914-8f11-34268dee33ae tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:15.542354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c8f91-17e2-4914-8f11-34268dee33ae tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:15.542647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c8f91-17e2-4914-8f11-34268dee33ae tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:15.542932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c8f91-17e2-4914-8f11-34268dee33ae tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:15.543598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c8f91-17e2-4914-8f11-34268dee33ae tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:15.545682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c8f91-17e2-4914-8f11-34268dee33ae tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:15.586772 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2145/4217] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:15 2022] POST /identity/v3/users => generated 416 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:15.591250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f9f88-bccc-456c-aa17-1ee10d61e3c8 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:15.604452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f9f88-bccc-456c-aa17-1ee10d61e3c8 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:15.609667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ace922-86f0-4838-aef8-75f62e442606 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:15.610299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ace922-86f0-4838-aef8-75f62e442606 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:15.610674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ace922-86f0-4838-aef8-75f62e442606 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:15.610887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ace922-86f0-4838-aef8-75f62e442606 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:15.611614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ace922-86f0-4838-aef8-75f62e442606 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:15.613325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ace922-86f0-4838-aef8-75f62e442606 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:15.641356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c8f91-17e2-4914-8f11-34268dee33ae tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:15.648832 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2073/4218] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:03:15 2022] POST /identity/v3/roles => generated 244 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:15.657717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c8f91-17e2-4914-8f11-34268dee33ae tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:15.663252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4fbbd-abdd-4c37-be70-ba15367d958f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:15.664065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4fbbd-abdd-4c37-be70-ba15367d958f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:15.664367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4fbbd-abdd-4c37-be70-ba15367d958f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:15.664851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4fbbd-abdd-4c37-be70-ba15367d958f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:15.665602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4fbbd-abdd-4c37-be70-ba15367d958f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:15.667479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4fbbd-abdd-4c37-be70-ba15367d958f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:15.715106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ace922-86f0-4838-aef8-75f62e442606 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:15.723666 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2146/4219] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:03:15 2022] POST /identity/v3/roles => generated 247 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:03:15.730736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ace922-86f0-4838-aef8-75f62e442606 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:15.739797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881d82d-1062-4d9b-b16d-d2b2d93ab927 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:15.743798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881d82d-1062-4d9b-b16d-d2b2d93ab927 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:15.744692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881d82d-1062-4d9b-b16d-d2b2d93ab927 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:15.745018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881d82d-1062-4d9b-b16d-d2b2d93ab927 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/7a91e7a2f55249eda78d1ff5d993a0e9/users/9559b4b487c14f0bbc90d762eedc9374/roles/21d16725315f4870b94e20cf58607cbc` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:15.776597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881d82d-1062-4d9b-b16d-d2b2d93ab927 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7a91e7a2f55249eda78d1ff5d993a0e9, user_id=9559b4b487c14f0bbc90d762eedc9374, role_id=21d16725315f4870b94e20cf58607cbc)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:15.787836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881d82d-1062-4d9b-b16d-d2b2d93ab927 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:15.815999 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2074/4220] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:15 2022] PUT /identity/v3/projects/7a91e7a2f55249eda78d1ff5d993a0e9/users/9559b4b487c14f0bbc90d762eedc9374/roles/21d16725315f4870b94e20cf58607cbc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:15.821028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4fbbd-abdd-4c37-be70-ba15367d958f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:15.840432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4fbbd-abdd-4c37-be70-ba15367d958f tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:15.970792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2318a-1427-42e9-8be6-60e4e1103038 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:15.972124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2318a-1427-42e9-8be6-60e4e1103038 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:15.972686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2318a-1427-42e9-8be6-60e4e1103038 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:15.975405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2318a-1427-42e9-8be6-60e4e1103038 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/7a91e7a2f55249eda78d1ff5d993a0e9/users/9559b4b487c14f0bbc90d762eedc9374/roles/5656b4ebf87c49d785f44d38089f4c62` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:15.980826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2318a-1427-42e9-8be6-60e4e1103038 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7a91e7a2f55249eda78d1ff5d993a0e9, user_id=9559b4b487c14f0bbc90d762eedc9374, role_id=5656b4ebf87c49d785f44d38089f4c62)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:15.983933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2318a-1427-42e9-8be6-60e4e1103038 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:16.007398 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2147/4221] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:15 2022] PUT /identity/v3/projects/7a91e7a2f55249eda78d1ff5d993a0e9/users/9559b4b487c14f0bbc90d762eedc9374/roles/5656b4ebf87c49d785f44d38089f4c62 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:16.008067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881d82d-1062-4d9b-b16d-d2b2d93ab927 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:16.025167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881d82d-1062-4d9b-b16d-d2b2d93ab927 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:16.182960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:16.206878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e9988-641e-4c8c-8466-997e21a4ccf3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:16.216492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e9988-641e-4c8c-8466-997e21a4ccf3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:16.216799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e9988-641e-4c8c-8466-997e21a4ccf3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:16.217814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e9988-641e-4c8c-8466-997e21a4ccf3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:16.218044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e9988-641e-4c8c-8466-997e21a4ccf3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:16.223373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e9988-641e-4c8c-8466-997e21a4ccf3 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:16.323057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cbe8d-c18a-4416-a3ad-e585522fa056 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:16.323627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cbe8d-c18a-4416-a3ad-e585522fa056 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:16.323941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cbe8d-c18a-4416-a3ad-e585522fa056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:16.328301 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2075/4222] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:16 2022] GET /identity/v3/users => generated 8345 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:03:16.384436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-007cbe8d-c18a-4416-a3ad-e585522fa056 None None] MFA Rules not processed for user `9559b4b487c14f0bbc90d762eedc9374`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:16.387479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-007cbe8d-c18a-4416-a3ad-e585522fa056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:16.493042 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2148/4223] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:16 2022] POST /identity/v3/auth/tokens => generated 3281 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:03:16.503276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8830990-f6fc-4236-8085-342ebde1e44c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:16.508731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8830990-f6fc-4236-8085-342ebde1e44c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:16.509177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8830990-f6fc-4236-8085-342ebde1e44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:16.537550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e8830990-f6fc-4236-8085-342ebde1e44c None None] MFA Rules not processed for user `9559b4b487c14f0bbc90d762eedc9374`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:16.539191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e8830990-f6fc-4236-8085-342ebde1e44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:16.551846 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2076/4224] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:16 2022] POST /identity/v3/auth/tokens => generated 3281 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:16.555923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cbe8d-c18a-4416-a3ad-e585522fa056 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:16.570596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cbe8d-c18a-4416-a3ad-e585522fa056 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:16.576855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621d956-2fae-4737-ab22-2289a839038b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9559b4b487c14f0bbc90d762eedc9374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a91e7a2f55249eda78d1ff5d993a0e9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1760533107', 'tempest-DelegatedRole-1619777161'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:16.577609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621d956-2fae-4737-ab22-2289a839038b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:16.577960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621d956-2fae-4737-ab22-2289a839038b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:16.578312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621d956-2fae-4737-ab22-2289a839038b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:16.579413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621d956-2fae-4737-ab22-2289a839038b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorizing `identity:create_trust()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:16.598444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621d956-2fae-4737-ab22-2289a839038b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:16.659364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:16.915632 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2149/4225] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Sep 3 21:03:16 2022] POST /identity/v3/OS-TRUST/trusts => generated 835 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:16.917049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8830990-f6fc-4236-8085-342ebde1e44c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:16.939229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8830990-f6fc-4236-8085-342ebde1e44c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:16.945276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a9e6f-bced-49b6-adfe-6ad6a992abab tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9559b4b487c14f0bbc90d762eedc9374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a91e7a2f55249eda78d1ff5d993a0e9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1760533107', 'tempest-DelegatedRole-1619777161'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:16.946075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a9e6f-bced-49b6-adfe-6ad6a992abab tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:16.947775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a9e6f-bced-49b6-adfe-6ad6a992abab tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:16.947775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a9e6f-bced-49b6-adfe-6ad6a992abab tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] PATH_INFO: `/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:16.959608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a9e6f-bced-49b6-adfe-6ad6a992abab tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorizing `identity:get_trust(trust_id=f296f443397b4c5ea4072a439a0cc9e4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:16.965121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a9e6f-bced-49b6-adfe-6ad6a992abab tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:16.974427 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2077/4226] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:03:16 2022] GET /identity/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4 => generated 835 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:03:16.978875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621d956-2fae-4737-ab22-2289a839038b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:16.995804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621d956-2fae-4737-ab22-2289a839038b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:17.005892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25c71e-3794-4534-b824-6c5a7166cce4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9559b4b487c14f0bbc90d762eedc9374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a91e7a2f55249eda78d1ff5d993a0e9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1760533107', 'tempest-DelegatedRole-1619777161'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:17.009458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25c71e-3794-4534-b824-6c5a7166cce4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:17.010740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25c71e-3794-4534-b824-6c5a7166cce4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:17.011826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25c71e-3794-4534-b824-6c5a7166cce4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] PATH_INFO: `/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:17.029368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25c71e-3794-4534-b824-6c5a7166cce4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=f296f443397b4c5ea4072a439a0cc9e4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:17.030885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25c71e-3794-4534-b824-6c5a7166cce4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:17.050663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a9e6f-bced-49b6-adfe-6ad6a992abab tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:17.051906 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2150/4227] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Sep 3 21:03:16 2022] GET /identity/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4/roles => generated 371 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:17.063833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a9e6f-bced-49b6-adfe-6ad6a992abab tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:17.079894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9fc07-167c-45e2-8dc5-61e55a5a55cd tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9559b4b487c14f0bbc90d762eedc9374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a91e7a2f55249eda78d1ff5d993a0e9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1760533107', 'tempest-DelegatedRole-1619777161'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:17.080584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9fc07-167c-45e2-8dc5-61e55a5a55cd tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:17.080883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9fc07-167c-45e2-8dc5-61e55a5a55cd tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:17.081196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9fc07-167c-45e2-8dc5-61e55a5a55cd tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] PATH_INFO: `/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4/roles/21d16725315f4870b94e20cf58607cbc` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:17.088775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9fc07-167c-45e2-8dc5-61e55a5a55cd tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorizing `identity:get_role_for_trust(trust_id=f296f443397b4c5ea4072a439a0cc9e4, role_id=21d16725315f4870b94e20cf58607cbc)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:17.090516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9fc07-167c-45e2-8dc5-61e55a5a55cd tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:17.106192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25c71e-3794-4534-b824-6c5a7166cce4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:17.117502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25c71e-3794-4534-b824-6c5a7166cce4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:17.118257 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2078/4228] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Sep 3 21:03:17 2022] GET /identity/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4/roles/21d16725315f4870b94e20cf58607cbc => generated 244 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:17.124801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf60335-1377-4223-9293-2b6f2748629b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9559b4b487c14f0bbc90d762eedc9374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a91e7a2f55249eda78d1ff5d993a0e9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1760533107', 'tempest-DelegatedRole-1619777161'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:17.125514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf60335-1377-4223-9293-2b6f2748629b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] REQUEST_METHOD: `HEAD` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:17.125808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf60335-1377-4223-9293-2b6f2748629b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:17.126098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf60335-1377-4223-9293-2b6f2748629b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] PATH_INFO: `/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4/roles/21d16725315f4870b94e20cf58607cbc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:17.133938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf60335-1377-4223-9293-2b6f2748629b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorizing `identity:get_role_for_trust(trust_id=f296f443397b4c5ea4072a439a0cc9e4, role_id=21d16725315f4870b94e20cf58607cbc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:17.136806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf60335-1377-4223-9293-2b6f2748629b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:17.149339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9fc07-167c-45e2-8dc5-61e55a5a55cd tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:17.154418 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2151/4229] 10.222.0.16 () {58 vars in 1369 bytes} [Sat Sep 3 21:03:17 2022] HEAD /identity/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4/roles/21d16725315f4870b94e20cf58607cbc => generated 0 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:17.159891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9fc07-167c-45e2-8dc5-61e55a5a55cd tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:17.164504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742a2d5-803c-495c-b06e-321f64428f67 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9559b4b487c14f0bbc90d762eedc9374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a91e7a2f55249eda78d1ff5d993a0e9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1760533107', 'tempest-DelegatedRole-1619777161'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:17.165211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742a2d5-803c-495c-b06e-321f64428f67 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:17.165504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742a2d5-803c-495c-b06e-321f64428f67 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:17.165771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742a2d5-803c-495c-b06e-321f64428f67 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] PATH_INFO: `/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4/roles/5656b4ebf87c49d785f44d38089f4c62` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:17.176324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742a2d5-803c-495c-b06e-321f64428f67 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorizing `identity:get_role_for_trust(trust_id=f296f443397b4c5ea4072a439a0cc9e4, role_id=5656b4ebf87c49d785f44d38089f4c62)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:17.178679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742a2d5-803c-495c-b06e-321f64428f67 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-7742a2d5-803c-495c-b06e-321f64428f67 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Could not find role: 5656b4ebf87c49d785f44d38089f4c62.: keystone.exception.RoleNotFound: Could not find role: 5656b4ebf87c49d785f44d38089f4c62. Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 21:03:17.193256 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 5656b4ebf87c49d785f44d38089f4c62. Sep 03 21:03:17.194372 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:03:17.199520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf60335-1377-4223-9293-2b6f2748629b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:17.209797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf60335-1377-4223-9293-2b6f2748629b tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:17.210562 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2079/4230] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Sep 3 21:03:17 2022] GET /identity/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4/roles/5656b4ebf87c49d785f44d38089f4c62 => generated 110 bytes in 61 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 03 21:03:17.215484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5745e-756e-4a63-b6a7-d3329653fee4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9559b4b487c14f0bbc90d762eedc9374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a91e7a2f55249eda78d1ff5d993a0e9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1760533107', 'tempest-DelegatedRole-1619777161'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:17.215891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5745e-756e-4a63-b6a7-d3329653fee4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] REQUEST_METHOD: `HEAD` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:17.216184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5745e-756e-4a63-b6a7-d3329653fee4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:17.216471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5745e-756e-4a63-b6a7-d3329653fee4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] PATH_INFO: `/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4/roles/5656b4ebf87c49d785f44d38089f4c62` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:17.224002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5745e-756e-4a63-b6a7-d3329653fee4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorizing `identity:get_role_for_trust(trust_id=f296f443397b4c5ea4072a439a0cc9e4, role_id=5656b4ebf87c49d785f44d38089f4c62)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:17.225626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5745e-756e-4a63-b6a7-d3329653fee4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-dac5745e-756e-4a63-b6a7-d3329653fee4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Could not find role: 5656b4ebf87c49d785f44d38089f4c62.: keystone.exception.RoleNotFound: Could not find role: 5656b4ebf87c49d785f44d38089f4c62. Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 21:03:17.239104 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 5656b4ebf87c49d785f44d38089f4c62. Sep 03 21:03:17.240314 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:03:17.246280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742a2d5-803c-495c-b06e-321f64428f67 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:17.255425 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2152/4231] 10.222.0.16 () {58 vars in 1369 bytes} [Sat Sep 3 21:03:17 2022] HEAD /identity/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4/roles/5656b4ebf87c49d785f44d38089f4c62 => generated 0 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:03:17.257639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742a2d5-803c-495c-b06e-321f64428f67 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:17.262818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0992a0-691a-41c1-9478-30776b67782a tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9559b4b487c14f0bbc90d762eedc9374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a91e7a2f55249eda78d1ff5d993a0e9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1760533107', 'tempest-DelegatedRole-1619777161'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:17.263476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0992a0-691a-41c1-9478-30776b67782a tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:17.263766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0992a0-691a-41c1-9478-30776b67782a tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:17.264051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0992a0-691a-41c1-9478-30776b67782a tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] PATH_INFO: `/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:17.271725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0992a0-691a-41c1-9478-30776b67782a tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorizing `identity:delete_trust(trust_id=f296f443397b4c5ea4072a439a0cc9e4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:17.273499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0992a0-691a-41c1-9478-30776b67782a tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:17.298081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5d0992a0-691a-41c1-9478-30776b67782a tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'f296f443397b4c5ea4072a439a0cc9e4', 'request_id': 'req-5d0992a0-691a-41c1-9478-30776b67782a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:17.329437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5d0992a0-691a-41c1-9478-30776b67782a tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'f296f443397b4c5ea4072a439a0cc9e4', 'request_id': 'req-5d0992a0-691a-41c1-9478-30776b67782a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:17.331304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5d0992a0-691a-41c1-9478-30776b67782a tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'f296f443397b4c5ea4072a439a0cc9e4', 'request_id': 'req-5d0992a0-691a-41c1-9478-30776b67782a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:17.342503 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2080/4232] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Sep 3 21:03:17 2022] DELETE /identity/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:17.346952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5745e-756e-4a63-b6a7-d3329653fee4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:17.348729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dac5745e-756e-4a63-b6a7-d3329653fee4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:17.375190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5745e-756e-4a63-b6a7-d3329653fee4 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:17.401816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec735fff-8808-4ac0-a418-9932cdc44c28 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9559b4b487c14f0bbc90d762eedc9374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a91e7a2f55249eda78d1ff5d993a0e9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1760533107', 'tempest-DelegatedRole-1619777161'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:17.402494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735fff-8808-4ac0-a418-9932cdc44c28 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:17.402784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735fff-8808-4ac0-a418-9932cdc44c28 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:17.403063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735fff-8808-4ac0-a418-9932cdc44c28 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] PATH_INFO: `/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-ec735fff-8808-4ac0-a418-9932cdc44c28 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Could not find trust: f296f443397b4c5ea4072a439a0cc9e4.: keystone.exception.TrustNotFound: Could not find trust: f296f443397b4c5ea4072a439a0cc9e4. Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 03 21:03:17.414004 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: f296f443397b4c5ea4072a439a0cc9e4. Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:03:17.415456 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2153/4233] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:03:17 2022] GET /identity/v3/OS-TRUST/trusts/f296f443397b4c5ea4072a439a0cc9e4 => generated 111 bytes in 69 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:03:17.420646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0992a0-691a-41c1-9478-30776b67782a tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:17.422257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5d0992a0-691a-41c1-9478-30776b67782a tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:17.442464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0992a0-691a-41c1-9478-30776b67782a tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:17.448797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e825d6-8854-4a03-8367-dc5a5fc22e81 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:17.449477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e825d6-8854-4a03-8367-dc5a5fc22e81 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:17.450138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e825d6-8854-4a03-8367-dc5a5fc22e81 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:17.450548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e825d6-8854-4a03-8367-dc5a5fc22e81 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles/5656b4ebf87c49d785f44d38089f4c62` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:17.452273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e825d6-8854-4a03-8367-dc5a5fc22e81 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_role(role_id=5656b4ebf87c49d785f44d38089f4c62)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:17.454158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e825d6-8854-4a03-8367-dc5a5fc22e81 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:17.507885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-62e825d6-8854-4a03-8367-dc5a5fc22e81 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invalidating the token cache because role 5656b4ebf87c49d785f44d38089f4c62 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:17.508367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-62e825d6-8854-4a03-8367-dc5a5fc22e81 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:17.509232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-62e825d6-8854-4a03-8367-dc5a5fc22e81 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:17.515819 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2081/4234] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:17 2022] DELETE /identity/v3/roles/5656b4ebf87c49d785f44d38089f4c62 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:17.516573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec735fff-8808-4ac0-a418-9932cdc44c28 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:17.518611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ec735fff-8808-4ac0-a418-9932cdc44c28 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:17.545280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec735fff-8808-4ac0-a418-9932cdc44c28 tempest-TrustsV3TestJSON-1282649458 tempest-user-1569468768] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:17.636052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b030964-2a9c-4106-910b-2acd293826d2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:17.636710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b030964-2a9c-4106-910b-2acd293826d2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:17.637036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b030964-2a9c-4106-910b-2acd293826d2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:17.637320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b030964-2a9c-4106-910b-2acd293826d2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/roles/21d16725315f4870b94e20cf58607cbc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:17.638778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b030964-2a9c-4106-910b-2acd293826d2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_role(role_id=21d16725315f4870b94e20cf58607cbc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:17.640410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b030964-2a9c-4106-910b-2acd293826d2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:17.723394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b030964-2a9c-4106-910b-2acd293826d2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invalidating the token cache because role 21d16725315f4870b94e20cf58607cbc has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:17.723875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b030964-2a9c-4106-910b-2acd293826d2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:17.724596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b030964-2a9c-4106-910b-2acd293826d2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:17.726962 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2154/4235] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:17 2022] DELETE /identity/v3/roles/21d16725315f4870b94e20cf58607cbc => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:17.731591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e825d6-8854-4a03-8367-dc5a5fc22e81 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:17.758457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e825d6-8854-4a03-8367-dc5a5fc22e81 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:17.906186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:17.907475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:17.907790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:17.908105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/users/9559b4b487c14f0bbc90d762eedc9374` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:17.919057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_user(user_id=9559b4b487c14f0bbc90d762eedc9374)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:17.921311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:17.929220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:18.042640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9559b4b487c14f0bbc90d762eedc9374'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.068760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9559b4b487c14f0bbc90d762eedc9374'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.070260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9559b4b487c14f0bbc90d762eedc9374'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.094225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9559b4b487c14f0bbc90d762eedc9374'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.142722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9559b4b487c14f0bbc90d762eedc9374'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.151407 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2082/4236] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:17 2022] DELETE /identity/v3/users/9559b4b487c14f0bbc90d762eedc9374 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:18.171122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b030964-2a9c-4106-910b-2acd293826d2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:18.173059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b030964-2a9c-4106-910b-2acd293826d2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:18.189961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:18.205360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b030964-2a9c-4106-910b-2acd293826d2 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:18.382951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b82c3-71ad-492f-a0e3-7af3084c1270 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b868ff8d3a74c988cf1f1bf73e5efb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9222872f37347c49d3974931081bcd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:18.382951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b82c3-71ad-492f-a0e3-7af3084c1270 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:18.383694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b82c3-71ad-492f-a0e3-7af3084c1270 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:18.383694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b82c3-71ad-492f-a0e3-7af3084c1270 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] PATH_INFO: `/v3/projects/7a91e7a2f55249eda78d1ff5d993a0e9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:18.393331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b82c3-71ad-492f-a0e3-7af3084c1270 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7a91e7a2f55249eda78d1ff5d993a0e9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:18.395574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b82c3-71ad-492f-a0e3-7af3084c1270 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:18.478395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7e7b82c3-71ad-492f-a0e3-7af3084c1270 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a91e7a2f55249eda78d1ff5d993a0e9', 'request_id': 'req-7e7b82c3-71ad-492f-a0e3-7af3084c1270'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.492337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7e7b82c3-71ad-492f-a0e3-7af3084c1270 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a91e7a2f55249eda78d1ff5d993a0e9', 'request_id': 'req-7e7b82c3-71ad-492f-a0e3-7af3084c1270'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.501099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7e7b82c3-71ad-492f-a0e3-7af3084c1270 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a91e7a2f55249eda78d1ff5d993a0e9', 'request_id': 'req-7e7b82c3-71ad-492f-a0e3-7af3084c1270'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.515854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7e7b82c3-71ad-492f-a0e3-7af3084c1270 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] The token cache is being invalidate because project 7a91e7a2f55249eda78d1ff5d993a0e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:18.516528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7e7b82c3-71ad-492f-a0e3-7af3084c1270 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.517015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7e7b82c3-71ad-492f-a0e3-7af3084c1270 tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.518889 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2155/4237] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:18 2022] DELETE /identity/v3/projects/7a91e7a2f55249eda78d1ff5d993a0e9 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:18.524262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:18.526456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:18.606574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58c078-c866-4fc4-a3af-33f1f54537fb tempest-TrustsV3TestJSON-1542427021 tempest-TrustsV3TestJSON-1542427021-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:18.692935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:18.692935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:18.692935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:18.692935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] PATH_INFO: `/v3/users/a9db5e3de11245e5964d198d6da88c46` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:18.693486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9db5e3de11245e5964d198d6da88c46)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:18.697218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:18.747918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:18.783098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9db5e3de11245e5964d198d6da88c46'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.804392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9db5e3de11245e5964d198d6da88c46'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.805275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9db5e3de11245e5964d198d6da88c46'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.825820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9db5e3de11245e5964d198d6da88c46'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.842665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9db5e3de11245e5964d198d6da88c46'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:18.852280 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2083/4238] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:18 2022] DELETE /identity/v3/users/a9db5e3de11245e5964d198d6da88c46 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:18.857240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6dfc1-d50f-4a20-8ca7-d47e7e543633 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:18.857549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6dfc1-d50f-4a20-8ca7-d47e7e543633 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:18.857910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6dfc1-d50f-4a20-8ca7-d47e7e543633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:18.888958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fef6dfc1-d50f-4a20-8ca7-d47e7e543633 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:18.890516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fef6dfc1-d50f-4a20-8ca7-d47e7e543633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:19.012580 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2156/4239] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:19.026624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:19.031653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:19.262080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23c892-44d6-41db-98c4-61e361530f94 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:19.275693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13102934-3069-406a-955c-17560f92ac13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:19.276556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13102934-3069-406a-955c-17560f92ac13 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:19.276840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13102934-3069-406a-955c-17560f92ac13 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:19.277205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13102934-3069-406a-955c-17560f92ac13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:19.322342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13102934-3069-406a-955c-17560f92ac13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:19.324016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13102934-3069-406a-955c-17560f92ac13 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:19.359892 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2084/4240] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:19.938845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6dfc1-d50f-4a20-8ca7-d47e7e543633 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:19.940349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fef6dfc1-d50f-4a20-8ca7-d47e7e543633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:19.958985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6dfc1-d50f-4a20-8ca7-d47e7e543633 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:19.971943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:19.972762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:19.973435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:19.974078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] PATH_INFO: `/v3/users/f7e2c822238a45d9b4d01ecf2cebc29a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:19.975836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7e2c822238a45d9b4d01ecf2cebc29a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:19.979003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:20.024142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:20.050280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7e2c822238a45d9b4d01ecf2cebc29a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.069364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7e2c822238a45d9b4d01ecf2cebc29a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.079150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7e2c822238a45d9b4d01ecf2cebc29a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.080133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7e2c822238a45d9b4d01ecf2cebc29a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.081067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7e2c822238a45d9b4d01ecf2cebc29a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.104591 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2157/4241] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:19 2022] DELETE /identity/v3/users/f7e2c822238a45d9b4d01ecf2cebc29a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:20.279384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13102934-3069-406a-955c-17560f92ac13 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:20.280374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-13102934-3069-406a-955c-17560f92ac13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:20.305722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13102934-3069-406a-955c-17560f92ac13 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:20.464683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:20.465464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:20.465827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:20.466180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] PATH_INFO: `/v3/projects/8cab7ff6b7984684919b31814648971b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:20.471019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cab7ff6b7984684919b31814648971b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:20.473656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:20.568803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:20.570728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:20.604062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cab7ff6b7984684919b31814648971b', 'request_id': 'req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.612098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:20.620888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cab7ff6b7984684919b31814648971b', 'request_id': 'req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.633168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249ea46-1b5b-43b2-8d7c-a9208396ec3d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:20.638376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cab7ff6b7984684919b31814648971b', 'request_id': 'req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.652756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] The token cache is being invalidate because project 8cab7ff6b7984684919b31814648971b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:20.653153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.654023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.657226 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2085/4242] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:20 2022] DELETE /identity/v3/projects/8cab7ff6b7984684919b31814648971b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:20.661477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:20.665081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:20.790409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:20.791621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:20.791913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:20.792367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0 admin admin] PATH_INFO: `/v3/projects/250efefcd9e84269a090b0a7ec433c87` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:20.793663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=250efefcd9e84269a090b0a7ec433c87)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:20.796447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:20.813747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160eb55c-5ec3-4c2a-b8a4-6353af3166b5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:20.819616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aea035-e056-4f51-9d47-ab51d4cd0266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:20.820437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aea035-e056-4f51-9d47-ab51d4cd0266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:20.820671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aea035-e056-4f51-9d47-ab51d4cd0266 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:20.821102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aea035-e056-4f51-9d47-ab51d4cd0266 admin admin] PATH_INFO: `/v3/users/6b868ff8d3a74c988cf1f1bf73e5efb6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:20.822733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aea035-e056-4f51-9d47-ab51d4cd0266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b868ff8d3a74c988cf1f1bf73e5efb6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:20.826971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aea035-e056-4f51-9d47-ab51d4cd0266 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:20.890487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250efefcd9e84269a090b0a7ec433c87', 'request_id': 'req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.903494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e2aea035-e056-4f51-9d47-ab51d4cd0266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b868ff8d3a74c988cf1f1bf73e5efb6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.913041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250efefcd9e84269a090b0a7ec433c87', 'request_id': 'req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.926678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e2aea035-e056-4f51-9d47-ab51d4cd0266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b868ff8d3a74c988cf1f1bf73e5efb6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.928604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250efefcd9e84269a090b0a7ec433c87', 'request_id': 'req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.929439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e2aea035-e056-4f51-9d47-ab51d4cd0266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b868ff8d3a74c988cf1f1bf73e5efb6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.946403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e2aea035-e056-4f51-9d47-ab51d4cd0266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b868ff8d3a74c988cf1f1bf73e5efb6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.954346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0 admin admin] The token cache is being invalidate because project 250efefcd9e84269a090b0a7ec433c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:20.954833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.956162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3388fdea-23dc-4f2f-b1cd-ac365dcd7dd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.959468 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2158/4243] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:20 2022] DELETE /identity/v3/projects/250efefcd9e84269a090b0a7ec433c87 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:20.965715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b95b63-63af-4cf7-898e-690ddf4dd370 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:20.966231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b95b63-63af-4cf7-898e-690ddf4dd370 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:20.966752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b95b63-63af-4cf7-898e-690ddf4dd370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:20.969481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e2aea035-e056-4f51-9d47-ab51d4cd0266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b868ff8d3a74c988cf1f1bf73e5efb6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:20.978464 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2086/4244] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:20 2022] DELETE /identity/v3/users/6b868ff8d3a74c988cf1f1bf73e5efb6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:21.006452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b1b95b63-63af-4cf7-898e-690ddf4dd370 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:21.010179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b1b95b63-63af-4cf7-898e-690ddf4dd370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:21.065511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:21.106194 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2159/4245] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:21.107454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46e999-22bb-47be-afd2-ebad17c0262b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:21.107756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46e999-22bb-47be-afd2-ebad17c0262b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:21.108030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46e999-22bb-47be-afd2-ebad17c0262b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:21.136686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4a46e999-22bb-47be-afd2-ebad17c0262b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:21.138087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4a46e999-22bb-47be-afd2-ebad17c0262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:21.151519 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2087/4246] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:21.153082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b95b63-63af-4cf7-898e-690ddf4dd370 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:21.181109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b95b63-63af-4cf7-898e-690ddf4dd370 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:21.193974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5343164-9bbe-4456-8a59-4e3c1cfe9e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:21.196845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5343164-9bbe-4456-8a59-4e3c1cfe9e2a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:21.197363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5343164-9bbe-4456-8a59-4e3c1cfe9e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:21.197540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5343164-9bbe-4456-8a59-4e3c1cfe9e2a admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:21.200368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5343164-9bbe-4456-8a59-4e3c1cfe9e2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:03:21.201202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5343164-9bbe-4456-8a59-4e3c1cfe9e2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:21.207550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5343164-9bbe-4456-8a59-4e3c1cfe9e2a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:21.251017 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2160/4247] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:21.252445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46e999-22bb-47be-afd2-ebad17c0262b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:21.277635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46e999-22bb-47be-afd2-ebad17c0262b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:21.291409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4921df-58f1-40fa-bb1d-b51ea0605b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:21.292301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4921df-58f1-40fa-bb1d-b51ea0605b23 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:21.292301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4921df-58f1-40fa-bb1d-b51ea0605b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:21.292754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4921df-58f1-40fa-bb1d-b51ea0605b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:21.293460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4921df-58f1-40fa-bb1d-b51ea0605b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:21.295254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4921df-58f1-40fa-bb1d-b51ea0605b23 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:21.362806 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2088/4248] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:03:21 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:21.370732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5343164-9bbe-4456-8a59-4e3c1cfe9e2a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:21.390294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5343164-9bbe-4456-8a59-4e3c1cfe9e2a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:21.502909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4921df-58f1-40fa-bb1d-b51ea0605b23 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:21.504685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8f4921df-58f1-40fa-bb1d-b51ea0605b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:21.548086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fac28-9be5-46bc-b457-41f67fcd1f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:21.548870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fac28-9be5-46bc-b457-41f67fcd1f45 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:21.549183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fac28-9be5-46bc-b457-41f67fcd1f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:21.549464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fac28-9be5-46bc-b457-41f67fcd1f45 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:21.550127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fac28-9be5-46bc-b457-41f67fcd1f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:21.552292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fac28-9be5-46bc-b457-41f67fcd1f45 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:21.581662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:21.583771 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2161/4249] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:21 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:21.589413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fac28-9be5-46bc-b457-41f67fcd1f45 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:21.603555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fac28-9be5-46bc-b457-41f67fcd1f45 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:21.609166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c5a6e-1ed6-4d23-aa5a-0f6ce0dc167f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:21.609800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c5a6e-1ed6-4d23-aa5a-0f6ce0dc167f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:21.610164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c5a6e-1ed6-4d23-aa5a-0f6ce0dc167f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:21.610486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c5a6e-1ed6-4d23-aa5a-0f6ce0dc167f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:21.611146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c5a6e-1ed6-4d23-aa5a-0f6ce0dc167f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:21.612877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c5a6e-1ed6-4d23-aa5a-0f6ce0dc167f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:21.630011 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2162/4250] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:03:21.636506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4921df-58f1-40fa-bb1d-b51ea0605b23 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:21.645883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c5a6e-1ed6-4d23-aa5a-0f6ce0dc167f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:21.656464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:21.657226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:21.657516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:21.657811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8 admin admin] PATH_INFO: `/v3/projects/b9222872f37347c49d3974931081bcd4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:21.659109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9222872f37347c49d3974931081bcd4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:21.661291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:21.671870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c5a6e-1ed6-4d23-aa5a-0f6ce0dc167f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:21.687818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb02758-aa6d-410a-bd88-4767e7f513f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:21.695952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb02758-aa6d-410a-bd88-4767e7f513f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:21.696529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb02758-aa6d-410a-bd88-4767e7f513f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:21.697155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb02758-aa6d-410a-bd88-4767e7f513f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:21.698061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb02758-aa6d-410a-bd88-4767e7f513f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:21.699694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb02758-aa6d-410a-bd88-4767e7f513f4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:21.733033 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2163/4251] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:21.738512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb02758-aa6d-410a-bd88-4767e7f513f4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:21.752843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb02758-aa6d-410a-bd88-4767e7f513f4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:21.795858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9222872f37347c49d3974931081bcd4', 'request_id': 'req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:21.807964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9222872f37347c49d3974931081bcd4', 'request_id': 'req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:21.817960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9222872f37347c49d3974931081bcd4', 'request_id': 'req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:21.831733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8 admin admin] The token cache is being invalidate because project b9222872f37347c49d3974931081bcd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:21.832041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:21.832808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6cd0f274-74f5-4aa0-96ff-fb0d363935f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:21.834528 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2089/4252] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:21 2022] DELETE /identity/v3/projects/b9222872f37347c49d3974931081bcd4 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:21.842094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5a499-5502-4c47-b1d3-fc65acbda000 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:21.842511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5a499-5502-4c47-b1d3-fc65acbda000 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:21.842901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5a499-5502-4c47-b1d3-fc65acbda000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:21.863224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b1746-182d-4b29-b13e-2e72e695d7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:21.864208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b1746-182d-4b29-b13e-2e72e695d7b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:21.864618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b1746-182d-4b29-b13e-2e72e695d7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:21.865004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b1746-182d-4b29-b13e-2e72e695d7b5 admin admin] PATH_INFO: `/v3/projects/8915b5da4f224da89e9699a7599350a1/users/d485394bc49e4380b152d32d09440281/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:21.871755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-efa5a499-5502-4c47-b1d3-fc65acbda000 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:21.873885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-efa5a499-5502-4c47-b1d3-fc65acbda000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:21.889644 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2090/4253] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:03:21.891081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b1746-182d-4b29-b13e-2e72e695d7b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8915b5da4f224da89e9699a7599350a1, user_id=d485394bc49e4380b152d32d09440281, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:21.894849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b1746-182d-4b29-b13e-2e72e695d7b5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:21.898999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4402b6-bb13-45da-af86-db142cf64757 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:21.899420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4402b6-bb13-45da-af86-db142cf64757 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:21.901965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4402b6-bb13-45da-af86-db142cf64757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:21.926247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1b4402b6-bb13-45da-af86-db142cf64757 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:21.928092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1b4402b6-bb13-45da-af86-db142cf64757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:21.930074 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2164/4254] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:21 2022] PUT /identity/v3/projects/8915b5da4f224da89e9699a7599350a1/users/d485394bc49e4380b152d32d09440281/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:21.936485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29047c7-505b-478e-a8e0-e7af88d2c459 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:21.936923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29047c7-505b-478e-a8e0-e7af88d2c459 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:21.937434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29047c7-505b-478e-a8e0-e7af88d2c459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:21.965609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a29047c7-505b-478e-a8e0-e7af88d2c459 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:21.968814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a29047c7-505b-478e-a8e0-e7af88d2c459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:21.982054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:22.032445 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2091/4255] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:03:22.034029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4402b6-bb13-45da-af86-db142cf64757 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:22.053343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4402b6-bb13-45da-af86-db142cf64757 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:22.061978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd4ac7-d095-4962-b32e-2f38b9f4174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:22.062843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd4ac7-d095-4962-b32e-2f38b9f4174c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.063303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd4ac7-d095-4962-b32e-2f38b9f4174c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.063655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd4ac7-d095-4962-b32e-2f38b9f4174c admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.065008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd4ac7-d095-4962-b32e-2f38b9f4174c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:03:22.065400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd4ac7-d095-4962-b32e-2f38b9f4174c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:22.067078 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2165/4256] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:22.070646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd4ac7-d095-4962-b32e-2f38b9f4174c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:22.078463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29047c7-505b-478e-a8e0-e7af88d2c459 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:22.082078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a29047c7-505b-478e-a8e0-e7af88d2c459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:22.089046 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2092/4257] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:03:22.093626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd4ac7-d095-4962-b32e-2f38b9f4174c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:22.109329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd4ac7-d095-4962-b32e-2f38b9f4174c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:22.115816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50aae2-9c68-44d1-82d7-dffb144568cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:22.116972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50aae2-9c68-44d1-82d7-dffb144568cf admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.118110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50aae2-9c68-44d1-82d7-dffb144568cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.118718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50aae2-9c68-44d1-82d7-dffb144568cf admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.120099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50aae2-9c68-44d1-82d7-dffb144568cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:22.122614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50aae2-9c68-44d1-82d7-dffb144568cf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:22.162134 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2093/4258] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:03:22 2022] POST /identity/v3/projects => generated 359 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:03:22.167133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50aae2-9c68-44d1-82d7-dffb144568cf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:22.192390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29047c7-505b-478e-a8e0-e7af88d2c459 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:22.199493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:22.253021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50aae2-9c68-44d1-82d7-dffb144568cf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:22.261643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a4138-0a70-443d-83ba-0aef3266ceaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:22.262554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a4138-0a70-443d-83ba-0aef3266ceaf admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.263392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a4138-0a70-443d-83ba-0aef3266ceaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.263933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a4138-0a70-443d-83ba-0aef3266ceaf admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.265473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a4138-0a70-443d-83ba-0aef3266ceaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:22.268574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a4138-0a70-443d-83ba-0aef3266ceaf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:22.273249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace72bda-1e74-43ba-97d7-8da739b98ce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:22.274104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace72bda-1e74-43ba-97d7-8da739b98ce1 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.274488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace72bda-1e74-43ba-97d7-8da739b98ce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.274946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace72bda-1e74-43ba-97d7-8da739b98ce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.291317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace72bda-1e74-43ba-97d7-8da739b98ce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:22.302216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace72bda-1e74-43ba-97d7-8da739b98ce1 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:22.320824 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2166/4259] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:03:22.323611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ecc3f-ce8b-4ab4-a940-ea1a92c7a7d3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.323979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ecc3f-ce8b-4ab4-a940-ea1a92c7a7d3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.324484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ecc3f-ce8b-4ab4-a940-ea1a92c7a7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.331074 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2094/4260] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:22 2022] POST /identity/v3/users => generated 280 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:03:22.332048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a4138-0a70-443d-83ba-0aef3266ceaf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:22.345370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a4138-0a70-443d-83ba-0aef3266ceaf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:22.351518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f0789-ede8-47dc-965a-f8f4b7aad0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:22.352199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f0789-ede8-47dc-965a-f8f4b7aad0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.352491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f0789-ede8-47dc-965a-f8f4b7aad0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.352770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f0789-ede8-47dc-965a-f8f4b7aad0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.353687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f0789-ede8-47dc-965a-f8f4b7aad0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:22.355420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f0789-ede8-47dc-965a-f8f4b7aad0c6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:22.368011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1a8ecc3f-ce8b-4ab4-a940-ea1a92c7a7d3 None None] MFA Rules not processed for user `cb94dfca0ec44bf4853be254984519ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:22.371394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1a8ecc3f-ce8b-4ab4-a940-ea1a92c7a7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:22.373523 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2095/4261] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:22.375124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f0789-ede8-47dc-965a-f8f4b7aad0c6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:22.390148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f0789-ede8-47dc-965a-f8f4b7aad0c6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:22.397201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c62a3a-05f7-4d55-942e-20cd344ec05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:22.397924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c62a3a-05f7-4d55-942e-20cd344ec05e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.398975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c62a3a-05f7-4d55-942e-20cd344ec05e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.399379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c62a3a-05f7-4d55-942e-20cd344ec05e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.400429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c62a3a-05f7-4d55-942e-20cd344ec05e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:22.402777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c62a3a-05f7-4d55-942e-20cd344ec05e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:22.421403 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2096/4262] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:22.422997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c62a3a-05f7-4d55-942e-20cd344ec05e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:22.441604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c62a3a-05f7-4d55-942e-20cd344ec05e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:22.449594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291f7d8-6747-4ee1-b052-d802e1ca4019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:22.452074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291f7d8-6747-4ee1-b052-d802e1ca4019 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.453926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291f7d8-6747-4ee1-b052-d802e1ca4019 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.454866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291f7d8-6747-4ee1-b052-d802e1ca4019 admin admin] PATH_INFO: `/v3/projects/30e821b139de40c39b1bc4099206e1f9/users/b184cedb7e534128a9acb1d3a4aaaa07/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.478426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291f7d8-6747-4ee1-b052-d802e1ca4019 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30e821b139de40c39b1bc4099206e1f9, user_id=b184cedb7e534128a9acb1d3a4aaaa07, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:22.481763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291f7d8-6747-4ee1-b052-d802e1ca4019 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:22.501010 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2167/4263] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:22.505087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ecc3f-ce8b-4ab4-a940-ea1a92c7a7d3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:22.506980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1a8ecc3f-ce8b-4ab4-a940-ea1a92c7a7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:22.517790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:22.519772 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2097/4264] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:22 2022] PUT /identity/v3/projects/30e821b139de40c39b1bc4099206e1f9/users/b184cedb7e534128a9acb1d3a4aaaa07/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:22.530663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939ad0d-553f-44b3-95f6-4616bec24969 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.531007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939ad0d-553f-44b3-95f6-4616bec24969 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.531382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939ad0d-553f-44b3-95f6-4616bec24969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.535238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ecc3f-ce8b-4ab4-a940-ea1a92c7a7d3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:22.584007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f939ad0d-553f-44b3-95f6-4616bec24969 None None] MFA Rules not processed for user `b184cedb7e534128a9acb1d3a4aaaa07`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:22.587766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f939ad0d-553f-44b3-95f6-4616bec24969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:22.620724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe663232-1cea-4a45-9f60-4cb83cd468e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:22.621456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe663232-1cea-4a45-9f60-4cb83cd468e4 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.621753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe663232-1cea-4a45-9f60-4cb83cd468e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.622035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe663232-1cea-4a45-9f60-4cb83cd468e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.622691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe663232-1cea-4a45-9f60-4cb83cd468e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:22.624679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe663232-1cea-4a45-9f60-4cb83cd468e4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:22.668483 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2168/4265] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:03:22 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:22.672770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe663232-1cea-4a45-9f60-4cb83cd468e4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:22.726343 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2098/4266] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:22.737934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939ad0d-553f-44b3-95f6-4616bec24969 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:22.756877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939ad0d-553f-44b3-95f6-4616bec24969 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:22.792527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe663232-1cea-4a45-9f60-4cb83cd468e4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:22.810324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb2431-3ae8-4286-948d-14f037e9cdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:22.811082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb2431-3ae8-4286-948d-14f037e9cdb8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.811467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb2431-3ae8-4286-948d-14f037e9cdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.811888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb2431-3ae8-4286-948d-14f037e9cdb8 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.813400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb2431-3ae8-4286-948d-14f037e9cdb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:22.819322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb2431-3ae8-4286-948d-14f037e9cdb8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:22.868722 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2169/4267] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:22 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:22.873167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb2431-3ae8-4286-948d-14f037e9cdb8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:22.897600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:22.910634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb2431-3ae8-4286-948d-14f037e9cdb8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:22.918583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:22.919385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.920337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.920337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] PATH_INFO: `/v3/users/b184cedb7e534128a9acb1d3a4aaaa07` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.921369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b184cedb7e534128a9acb1d3a4aaaa07)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:22.923182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c406443-37c7-4c11-b9ec-e76447884432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:22.923385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:22.924206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c406443-37c7-4c11-b9ec-e76447884432 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.925480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c406443-37c7-4c11-b9ec-e76447884432 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.925987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c406443-37c7-4c11-b9ec-e76447884432 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.927149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c406443-37c7-4c11-b9ec-e76447884432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:22.929794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c406443-37c7-4c11-b9ec-e76447884432 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:22.948720 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2170/4268] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:22.950644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0d56f-bad5-4454-9154-7d4f9e380742 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:22.951127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0d56f-bad5-4454-9154-7d4f9e380742 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:22.952284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0d56f-bad5-4454-9154-7d4f9e380742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:22.991024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b184cedb7e534128a9acb1d3a4aaaa07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:23.001130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cee0d56f-bad5-4454-9154-7d4f9e380742 None None] MFA Rules not processed for user `c3187514ca644ff68c992526f38783c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:23.003942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cee0d56f-bad5-4454-9154-7d4f9e380742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:23.006239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b184cedb7e534128a9acb1d3a4aaaa07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:23.007275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b184cedb7e534128a9acb1d3a4aaaa07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:23.018315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b184cedb7e534128a9acb1d3a4aaaa07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:23.030402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b184cedb7e534128a9acb1d3a4aaaa07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:23.051133 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2099/4269] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:22 2022] DELETE /identity/v3/users/b184cedb7e534128a9acb1d3a4aaaa07 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:23.052476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:23.055356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:23.129548 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2171/4270] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:23.131153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fce99e-0fd9-4380-b055-bbf358552f9c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:23.131454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fce99e-0fd9-4380-b055-bbf358552f9c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:23.131740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fce99e-0fd9-4380-b055-bbf358552f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:23.179901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b2fce99e-0fd9-4380-b055-bbf358552f9c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:23.183380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b2fce99e-0fd9-4380-b055-bbf358552f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:23.188177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2135eb-7ece-48b3-8564-f745b13445aa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:23.215549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed4c08-f538-4635-b6c5-50d074b4205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:23.216954 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2172/4271] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:23.217198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed4c08-f538-4635-b6c5-50d074b4205f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:23.217705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed4c08-f538-4635-b6c5-50d074b4205f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:23.217911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed4c08-f538-4635-b6c5-50d074b4205f admin admin] PATH_INFO: `/v3/projects/60ba47cefe86402abcf32330758da25f/users/b40eb82cf0d64b05b2260c6ea0afaae2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:23.240803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed4c08-f538-4635-b6c5-50d074b4205f admin admin] RBAC: Authorizing `identity:create_grant(project_id=60ba47cefe86402abcf32330758da25f, user_id=b40eb82cf0d64b05b2260c6ea0afaae2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:23.243689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fce99e-0fd9-4380-b055-bbf358552f9c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:23.245726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b2fce99e-0fd9-4380-b055-bbf358552f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:23.253260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:23.260154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed4c08-f538-4635-b6c5-50d074b4205f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:23.309214 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2100/4272] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:23 2022] PUT /identity/v3/projects/60ba47cefe86402abcf32330758da25f/users/b40eb82cf0d64b05b2260c6ea0afaae2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:23.310206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed4c08-f538-4635-b6c5-50d074b4205f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:23.441730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fce99e-0fd9-4380-b055-bbf358552f9c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:23.450883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:23.462036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed4c08-f538-4635-b6c5-50d074b4205f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:23.491732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcc889-72f6-489b-b401-e411f54d942f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:23.493527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcc889-72f6-489b-b401-e411f54d942f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:23.495680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcc889-72f6-489b-b401-e411f54d942f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:23.496884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcc889-72f6-489b-b401-e411f54d942f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:23.500273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcc889-72f6-489b-b401-e411f54d942f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:23.505236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcc889-72f6-489b-b401-e411f54d942f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:23.518893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd159c-0f49-4de4-8982-cf1305894ef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:23.519525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd159c-0f49-4de4-8982-cf1305894ef8 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:23.519885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd159c-0f49-4de4-8982-cf1305894ef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:23.520244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd159c-0f49-4de4-8982-cf1305894ef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:23.531779 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2101/4273] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:23.536063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcc889-72f6-489b-b401-e411f54d942f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:23.537478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd159c-0f49-4de4-8982-cf1305894ef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:23.539275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd159c-0f49-4de4-8982-cf1305894ef8 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:23.554268 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2173/4274] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:03:23.555594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcc889-72f6-489b-b401-e411f54d942f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:23.572489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab907cb6-ce10-4539-91e7-2d4575d63a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:23.575121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab907cb6-ce10-4539-91e7-2d4575d63a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:23.575460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab907cb6-ce10-4539-91e7-2d4575d63a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:23.575854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab907cb6-ce10-4539-91e7-2d4575d63a5d admin admin] PATH_INFO: `/v3/projects/60ba47cefe86402abcf32330758da25f/users/b40eb82cf0d64b05b2260c6ea0afaae2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:23.578845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab907cb6-ce10-4539-91e7-2d4575d63a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=60ba47cefe86402abcf32330758da25f, user_id=b40eb82cf0d64b05b2260c6ea0afaae2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:23.582047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab907cb6-ce10-4539-91e7-2d4575d63a5d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:23.601430 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2102/4275] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:23 2022] PUT /identity/v3/projects/60ba47cefe86402abcf32330758da25f/users/b40eb82cf0d64b05b2260c6ea0afaae2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:24.514631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd159c-0f49-4de4-8982-cf1305894ef8 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:24.516201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d4cd159c-0f49-4de4-8982-cf1305894ef8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:24.723156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd159c-0f49-4de4-8982-cf1305894ef8 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:24.726068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab907cb6-ce10-4539-91e7-2d4575d63a5d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:24.728423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ab907cb6-ce10-4539-91e7-2d4575d63a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:24.829774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab907cb6-ce10-4539-91e7-2d4575d63a5d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:24.835638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf66a28-ed00-4e38-b5a8-b4fe83b9a969 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:24.836423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf66a28-ed00-4e38-b5a8-b4fe83b9a969 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:24.837358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf66a28-ed00-4e38-b5a8-b4fe83b9a969 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:24.837358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf66a28-ed00-4e38-b5a8-b4fe83b9a969 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:24.840716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:24.841913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fbf66a28-ed00-4e38-b5a8-b4fe83b9a969 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:24.864015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702cc283-0254-4139-8842-61f5cb560737 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:24.864652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cc283-0254-4139-8842-61f5cb560737 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:24.864964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cc283-0254-4139-8842-61f5cb560737 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:24.865247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cc283-0254-4139-8842-61f5cb560737 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:24.867236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-702cc283-0254-4139-8842-61f5cb560737 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:24.870940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf66a28-ed00-4e38-b5a8-b4fe83b9a969 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:24.874891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf66a28-ed00-4e38-b5a8-b4fe83b9a969 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:24.896003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702cc283-0254-4139-8842-61f5cb560737 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:24.897340 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2103/4276] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:03:24.897989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702cc283-0254-4139-8842-61f5cb560737 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:24.900393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf66a28-ed00-4e38-b5a8-b4fe83b9a969 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:24.902349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fbf66a28-ed00-4e38-b5a8-b4fe83b9a969 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:24.921179 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2174/4277] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:25.018544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf66a28-ed00-4e38-b5a8-b4fe83b9a969 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:25.031386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f30248-537e-4510-a574-da7e57229160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:25.032544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f30248-537e-4510-a574-da7e57229160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:25.033127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f30248-537e-4510-a574-da7e57229160 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:25.033646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f30248-537e-4510-a574-da7e57229160 admin admin] PATH_INFO: `/v3/projects/30e821b139de40c39b1bc4099206e1f9` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:25.035324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f30248-537e-4510-a574-da7e57229160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30e821b139de40c39b1bc4099206e1f9)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:25.044444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f30248-537e-4510-a574-da7e57229160 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:25.064845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c25b3-1721-464f-93ae-d678774a1f24 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:25.065160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c25b3-1721-464f-93ae-d678774a1f24 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:25.065443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c25b3-1721-464f-93ae-d678774a1f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:25.114078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1c5c25b3-1721-464f-93ae-d678774a1f24 None None] MFA Rules not processed for user `c3187514ca644ff68c992526f38783c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:25.119818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1c5c25b3-1721-464f-93ae-d678774a1f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:25.169365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-19f30248-537e-4510-a574-da7e57229160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30e821b139de40c39b1bc4099206e1f9', 'request_id': 'req-19f30248-537e-4510-a574-da7e57229160'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:25.182453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-19f30248-537e-4510-a574-da7e57229160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30e821b139de40c39b1bc4099206e1f9', 'request_id': 'req-19f30248-537e-4510-a574-da7e57229160'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:25.190498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-19f30248-537e-4510-a574-da7e57229160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30e821b139de40c39b1bc4099206e1f9', 'request_id': 'req-19f30248-537e-4510-a574-da7e57229160'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:25.212432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:25.220865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-19f30248-537e-4510-a574-da7e57229160 admin admin] The token cache is being invalidate because project 30e821b139de40c39b1bc4099206e1f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:25.222051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-19f30248-537e-4510-a574-da7e57229160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:25.222957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-19f30248-537e-4510-a574-da7e57229160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:25.234375 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2104/4278] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:24 2022] DELETE /identity/v3/projects/30e821b139de40c39b1bc4099206e1f9 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:25.235983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebae9d-33b1-4212-b916-b84fc3d2012b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:25.236137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebae9d-33b1-4212-b916-b84fc3d2012b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:25.236420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebae9d-33b1-4212-b916-b84fc3d2012b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:25.266384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c3ebae9d-33b1-4212-b916-b84fc3d2012b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:25.267814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c3ebae9d-33b1-4212-b916-b84fc3d2012b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:25.285128 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2175/4279] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:03:25.396473 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2105/4280] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:25.405106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388e85a-5af7-41fa-a161-e6f9e7a56b84 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:25.405405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388e85a-5af7-41fa-a161-e6f9e7a56b84 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:25.405675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388e85a-5af7-41fa-a161-e6f9e7a56b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:25.426692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c388e85a-5af7-41fa-a161-e6f9e7a56b84 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:25.427843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c388e85a-5af7-41fa-a161-e6f9e7a56b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:25.437431 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2176/4281] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:25.441837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebae9d-33b1-4212-b916-b84fc3d2012b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:25.452669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebae9d-33b1-4212-b916-b84fc3d2012b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:25.458264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7282e37-80b0-4df5-b519-2a88fe69202f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:25.459900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7282e37-80b0-4df5-b519-2a88fe69202f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:25.460189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7282e37-80b0-4df5-b519-2a88fe69202f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:25.460464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7282e37-80b0-4df5-b519-2a88fe69202f admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:25.461168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7282e37-80b0-4df5-b519-2a88fe69202f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:03:25.461475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7282e37-80b0-4df5-b519-2a88fe69202f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:25.463829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7282e37-80b0-4df5-b519-2a88fe69202f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:25.476557 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2106/4282] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:25.481093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388e85a-5af7-41fa-a161-e6f9e7a56b84 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:25.491115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388e85a-5af7-41fa-a161-e6f9e7a56b84 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:25.497794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702331ed-475b-4116-980a-9a26bf89e2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:25.498288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702331ed-475b-4116-980a-9a26bf89e2bf admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:25.498627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702331ed-475b-4116-980a-9a26bf89e2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:25.498834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702331ed-475b-4116-980a-9a26bf89e2bf admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:25.499411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702331ed-475b-4116-980a-9a26bf89e2bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:25.501398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702331ed-475b-4116-980a-9a26bf89e2bf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:25.537358 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2177/4283] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:03:25 2022] POST /identity/v3/projects => generated 407 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:03:25.538690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7282e37-80b0-4df5-b519-2a88fe69202f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:25.550296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7282e37-80b0-4df5-b519-2a88fe69202f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:25.632603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4fd89-a7c9-4511-8ec9-4b4948b68797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:25.632603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4fd89-a7c9-4511-8ec9-4b4948b68797 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:25.632973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4fd89-a7c9-4511-8ec9-4b4948b68797 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:25.633350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4fd89-a7c9-4511-8ec9-4b4948b68797 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:25.634098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4fd89-a7c9-4511-8ec9-4b4948b68797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:25.638560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4fd89-a7c9-4511-8ec9-4b4948b68797 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:25.676315 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2107/4284] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:25 2022] POST /identity/v3/users => generated 304 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:25.677927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702331ed-475b-4116-980a-9a26bf89e2bf admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:25.690940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702331ed-475b-4116-980a-9a26bf89e2bf admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:25.695870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c27efd-6554-44c3-9963-1debbeaec6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:25.696474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c27efd-6554-44c3-9963-1debbeaec6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:25.696857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c27efd-6554-44c3-9963-1debbeaec6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:25.697812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c27efd-6554-44c3-9963-1debbeaec6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:25.698469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c27efd-6554-44c3-9963-1debbeaec6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:25.700137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c27efd-6554-44c3-9963-1debbeaec6a0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:25.715250 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2178/4285] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:25.716851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4fd89-a7c9-4511-8ec9-4b4948b68797 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:25.730314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4fd89-a7c9-4511-8ec9-4b4948b68797 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:25.735211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40b6ef-896a-4b6f-8d0e-57815bcc5780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:25.735921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40b6ef-896a-4b6f-8d0e-57815bcc5780 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:25.736275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40b6ef-896a-4b6f-8d0e-57815bcc5780 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:25.736650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40b6ef-896a-4b6f-8d0e-57815bcc5780 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:25.737451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40b6ef-896a-4b6f-8d0e-57815bcc5780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:25.739277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40b6ef-896a-4b6f-8d0e-57815bcc5780 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:25.755246 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2108/4286] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:25.756526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c27efd-6554-44c3-9963-1debbeaec6a0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:25.767407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c27efd-6554-44c3-9963-1debbeaec6a0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:25.773485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0842d64-ad97-4737-9010-c4f1763a3190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:25.774178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0842d64-ad97-4737-9010-c4f1763a3190 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:25.774462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0842d64-ad97-4737-9010-c4f1763a3190 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:25.774808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0842d64-ad97-4737-9010-c4f1763a3190 admin admin] PATH_INFO: `/v3/projects/0422f8af03dc49d0b6313beec40e8919/users/151825024598499ca06d8b4bc0683970/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:25.798230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0842d64-ad97-4737-9010-c4f1763a3190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0422f8af03dc49d0b6313beec40e8919, user_id=151825024598499ca06d8b4bc0683970, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:25.801088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0842d64-ad97-4737-9010-c4f1763a3190 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:25.819465 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2179/4287] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:25 2022] PUT /identity/v3/projects/0422f8af03dc49d0b6313beec40e8919/users/151825024598499ca06d8b4bc0683970/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:25.830617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38ee3b-f365-4d43-ad29-22def21fc49a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:25.830897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38ee3b-f365-4d43-ad29-22def21fc49a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:25.831179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38ee3b-f365-4d43-ad29-22def21fc49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:25.867053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0f38ee3b-f365-4d43-ad29-22def21fc49a None None] MFA Rules not processed for user `151825024598499ca06d8b4bc0683970`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:25.868358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0f38ee3b-f365-4d43-ad29-22def21fc49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:25.941302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:25.960543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0842d64-ad97-4737-9010-c4f1763a3190 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:25.961167 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2109/4288] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:25 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:03:26.052405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0842d64-ad97-4737-9010-c4f1763a3190 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:26.059563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6c314-f1b1-4fd2-8a07-6d02af4b51de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:26.060189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6c314-f1b1-4fd2-8a07-6d02af4b51de admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:26.060476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6c314-f1b1-4fd2-8a07-6d02af4b51de admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:26.060757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6c314-f1b1-4fd2-8a07-6d02af4b51de admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:26.061460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6c314-f1b1-4fd2-8a07-6d02af4b51de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:26.063498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6c314-f1b1-4fd2-8a07-6d02af4b51de admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:26.099266 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2180/4289] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:03:25 2022] POST /identity/v3/projects => generated 409 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:26.105068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38ee3b-f365-4d43-ad29-22def21fc49a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:26.133933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38ee3b-f365-4d43-ad29-22def21fc49a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:26.260795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73badb65-996e-4e85-8e52-06726383ef1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:26.261589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73badb65-996e-4e85-8e52-06726383ef1f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:26.262037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73badb65-996e-4e85-8e52-06726383ef1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:26.262609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73badb65-996e-4e85-8e52-06726383ef1f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:26.263537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73badb65-996e-4e85-8e52-06726383ef1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:26.272230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73badb65-996e-4e85-8e52-06726383ef1f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:26.322658 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2110/4290] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:26 2022] POST /identity/v3/users => generated 311 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:26.324820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6c314-f1b1-4fd2-8a07-6d02af4b51de admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:26.347717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6c314-f1b1-4fd2-8a07-6d02af4b51de admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:26.358778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099e05d-e67c-47fb-8ce3-d4192948fa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:26.359636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099e05d-e67c-47fb-8ce3-d4192948fa4a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:26.360130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099e05d-e67c-47fb-8ce3-d4192948fa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:26.360601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099e05d-e67c-47fb-8ce3-d4192948fa4a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:26.361480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099e05d-e67c-47fb-8ce3-d4192948fa4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:26.363141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099e05d-e67c-47fb-8ce3-d4192948fa4a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:26.384310 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2181/4291] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:26.388286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73badb65-996e-4e85-8e52-06726383ef1f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:26.403254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73badb65-996e-4e85-8e52-06726383ef1f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:26.412299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8d1bf-8460-4b74-ad6e-f89281358e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:26.413487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8d1bf-8460-4b74-ad6e-f89281358e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:26.413487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8d1bf-8460-4b74-ad6e-f89281358e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:26.413824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8d1bf-8460-4b74-ad6e-f89281358e30 admin admin] PATH_INFO: `/v3/projects/c45e27a79480475ba20255a3ecddcaf1/users/a941fdc988f04424b0519a67fc8bd4f7/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:26.439387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8d1bf-8460-4b74-ad6e-f89281358e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45e27a79480475ba20255a3ecddcaf1, user_id=a941fdc988f04424b0519a67fc8bd4f7, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:26.442471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8d1bf-8460-4b74-ad6e-f89281358e30 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:26.468314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099e05d-e67c-47fb-8ce3-d4192948fa4a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:26.468647 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2111/4292] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:26 2022] PUT /identity/v3/projects/c45e27a79480475ba20255a3ecddcaf1/users/a941fdc988f04424b0519a67fc8bd4f7/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:26.474256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:26.491497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099e05d-e67c-47fb-8ce3-d4192948fa4a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:26.572943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b1608-28eb-4ad4-9733-3bcf6bfb323b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:26.573601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b1608-28eb-4ad4-9733-3bcf6bfb323b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:26.573738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b1608-28eb-4ad4-9733-3bcf6bfb323b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:26.573991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b1608-28eb-4ad4-9733-3bcf6bfb323b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:26.576350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b1608-28eb-4ad4-9733-3bcf6bfb323b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:26.577054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b1608-28eb-4ad4-9733-3bcf6bfb323b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:26.593053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8d1bf-8460-4b74-ad6e-f89281358e30 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:26.605113 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2182/4293] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:26.606273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8d1bf-8460-4b74-ad6e-f89281358e30 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:26.611333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d0302-e415-4358-831b-95113e90743c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:26.612013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0302-e415-4358-831b-95113e90743c admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:26.612330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0302-e415-4358-831b-95113e90743c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:26.612657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0302-e415-4358-831b-95113e90743c admin admin] PATH_INFO: `/v3/projects/c45e27a79480475ba20255a3ecddcaf1/users/a941fdc988f04424b0519a67fc8bd4f7/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:26.614887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d0302-e415-4358-831b-95113e90743c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45e27a79480475ba20255a3ecddcaf1, user_id=a941fdc988f04424b0519a67fc8bd4f7, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:26.617606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d0302-e415-4358-831b-95113e90743c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:26.640112 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2112/4294] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:26 2022] PUT /identity/v3/projects/c45e27a79480475ba20255a3ecddcaf1/users/a941fdc988f04424b0519a67fc8bd4f7/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:26.642314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebabc9-4977-41e1-a907-6f8a5ddf01d2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:26.643105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebabc9-4977-41e1-a907-6f8a5ddf01d2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:26.643552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebabc9-4977-41e1-a907-6f8a5ddf01d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:26.734181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c7ebabc9-4977-41e1-a907-6f8a5ddf01d2 None None] MFA Rules not processed for user `a941fdc988f04424b0519a67fc8bd4f7`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:26.749674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c7ebabc9-4977-41e1-a907-6f8a5ddf01d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:26.870010 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2183/4295] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:26 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:26.887934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d0302-e415-4358-831b-95113e90743c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:26.890646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-772d0302-e415-4358-831b-95113e90743c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:26.991678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d0302-e415-4358-831b-95113e90743c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:26.996662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e5630-3e87-4390-b607-6a17d60e6778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:26.997389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5630-3e87-4390-b607-6a17d60e6778 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:26.998101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5630-3e87-4390-b607-6a17d60e6778 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:26.998415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5630-3e87-4390-b607-6a17d60e6778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:27.010845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e5630-3e87-4390-b607-6a17d60e6778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:27.012379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e5630-3e87-4390-b607-6a17d60e6778 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:27.026671 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2113/4296] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:26 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:32.074204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb8e17-1b42-43dc-8398-b3a5d88c8ff7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:32.074704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb8e17-1b42-43dc-8398-b3a5d88c8ff7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:32.074826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb8e17-1b42-43dc-8398-b3a5d88c8ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:32.154877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a4eb8e17-1b42-43dc-8398-b3a5d88c8ff7 None None] MFA Rules not processed for user `d485394bc49e4380b152d32d09440281`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:32.156503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a4eb8e17-1b42-43dc-8398-b3a5d88c8ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:32.273450 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2184/4297] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:32.289058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e5630-3e87-4390-b607-6a17d60e6778 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:32.291731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-535e5630-3e87-4390-b607-6a17d60e6778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:32.364378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:32.397936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e5630-3e87-4390-b607-6a17d60e6778 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:32.406863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a0809-b6a4-4707-bfc0-af3dd844d27e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:32.407991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a0809-b6a4-4707-bfc0-af3dd844d27e service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:32.408619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a0809-b6a4-4707-bfc0-af3dd844d27e service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:32.409234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a0809-b6a4-4707-bfc0-af3dd844d27e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:32.421708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a0809-b6a4-4707-bfc0-af3dd844d27e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:32.424203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a0809-b6a4-4707-bfc0-af3dd844d27e service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:32.438507 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2114/4298] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:03:33.044343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb8e17-1b42-43dc-8398-b3a5d88c8ff7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:33.072155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:33.202837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb8e17-1b42-43dc-8398-b3a5d88c8ff7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:33.209023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:33.211400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:33.212076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:33.213195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] PATH_INFO: `/v3/users/151825024598499ca06d8b4bc0683970` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:33.214658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=151825024598499ca06d8b4bc0683970)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:33.222301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:33.324559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '151825024598499ca06d8b4bc0683970'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:33.360484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '151825024598499ca06d8b4bc0683970'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:33.371688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '151825024598499ca06d8b4bc0683970'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:33.372995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '151825024598499ca06d8b4bc0683970'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:33.373806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '151825024598499ca06d8b4bc0683970'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:33.390320 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2185/4299] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:33 2022] DELETE /identity/v3/users/151825024598499ca06d8b4bc0683970 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:33.394577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e27f8d-825b-4ace-95c3-87bc606cd227 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:33.395079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e27f8d-825b-4ace-95c3-87bc606cd227 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:33.395205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e27f8d-825b-4ace-95c3-87bc606cd227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:33.415015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-28e27f8d-825b-4ace-95c3-87bc606cd227 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:33.416787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-28e27f8d-825b-4ace-95c3-87bc606cd227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:33.495109 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2115/4300] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:33.550049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:33.551505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:33.697495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b560f-d440-4d34-a566-c4561ed50762 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:33.704240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf715c-b62c-40ba-8cf8-5e9102fb1b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:33.704860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf715c-b62c-40ba-8cf8-5e9102fb1b44 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:33.705249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf715c-b62c-40ba-8cf8-5e9102fb1b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:33.705458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf715c-b62c-40ba-8cf8-5e9102fb1b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:33.729418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf715c-b62c-40ba-8cf8-5e9102fb1b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:33.730883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf715c-b62c-40ba-8cf8-5e9102fb1b44 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:33.760470 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2186/4301] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:34.736540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02c85a-b028-4b92-8a1b-f36e79eea4f4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:34.737432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02c85a-b028-4b92-8a1b-f36e79eea4f4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:34.737919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02c85a-b028-4b92-8a1b-f36e79eea4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:34.770665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fc02c85a-b028-4b92-8a1b-f36e79eea4f4 None None] MFA Rules not processed for user `d485394bc49e4380b152d32d09440281`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:34.772962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fc02c85a-b028-4b92-8a1b-f36e79eea4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:34.895270 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2116/4302] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:35.016322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf715c-b62c-40ba-8cf8-5e9102fb1b44 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:35.018336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8baf715c-b62c-40ba-8cf8-5e9102fb1b44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:35.054168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf715c-b62c-40ba-8cf8-5e9102fb1b44 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:35.083898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:35.084649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:35.085077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:35.085450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] PATH_INFO: `/v3/projects/0422f8af03dc49d0b6313beec40e8919` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:35.088337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0422f8af03dc49d0b6313beec40e8919)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:35.090470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:35.198604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:35.224295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0422f8af03dc49d0b6313beec40e8919', 'request_id': 'req-f429cf38-b523-4232-aa4a-2ffbe2acad56'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:35.234913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0422f8af03dc49d0b6313beec40e8919', 'request_id': 'req-f429cf38-b523-4232-aa4a-2ffbe2acad56'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:35.243139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0422f8af03dc49d0b6313beec40e8919', 'request_id': 'req-f429cf38-b523-4232-aa4a-2ffbe2acad56'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:35.260069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] The token cache is being invalidate because project 0422f8af03dc49d0b6313beec40e8919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:35.260595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:35.261398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:35.263422 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2187/4303] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:35 2022] DELETE /identity/v3/projects/0422f8af03dc49d0b6313beec40e8919 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:35.267045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02c85a-b028-4b92-8a1b-f36e79eea4f4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:35.281314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02c85a-b028-4b92-8a1b-f36e79eea4f4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:35.394913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e3435-6f70-45f6-abd6-b4a8a59c9e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:35.395968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e3435-6f70-45f6-abd6-b4a8a59c9e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:35.396668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e3435-6f70-45f6-abd6-b4a8a59c9e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:35.397348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e3435-6f70-45f6-abd6-b4a8a59c9e01 admin admin] PATH_INFO: `/v3/users/a941fdc988f04424b0519a67fc8bd4f7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:35.399132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e3435-6f70-45f6-abd6-b4a8a59c9e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a941fdc988f04424b0519a67fc8bd4f7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:35.401869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e3435-6f70-45f6-abd6-b4a8a59c9e01 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:35.450104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:35.513285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-760e3435-6f70-45f6-abd6-b4a8a59c9e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a941fdc988f04424b0519a67fc8bd4f7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:35.533107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-760e3435-6f70-45f6-abd6-b4a8a59c9e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a941fdc988f04424b0519a67fc8bd4f7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:35.539885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-760e3435-6f70-45f6-abd6-b4a8a59c9e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a941fdc988f04424b0519a67fc8bd4f7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:35.550069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-760e3435-6f70-45f6-abd6-b4a8a59c9e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a941fdc988f04424b0519a67fc8bd4f7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:35.564842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-760e3435-6f70-45f6-abd6-b4a8a59c9e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a941fdc988f04424b0519a67fc8bd4f7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:35.573763 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2117/4304] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:35 2022] DELETE /identity/v3/users/a941fdc988f04424b0519a67fc8bd4f7 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:35.905368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:35.907137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:35.934053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429cf38-b523-4232-aa4a-2ffbe2acad56 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:36.038065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b533481-191e-4f3e-bd54-200b453a4bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:36.038748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b533481-191e-4f3e-bd54-200b453a4bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:36.039248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b533481-191e-4f3e-bd54-200b453a4bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:36.039718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b533481-191e-4f3e-bd54-200b453a4bd0 admin admin] PATH_INFO: `/v3/projects/c45e27a79480475ba20255a3ecddcaf1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:36.042987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b533481-191e-4f3e-bd54-200b453a4bd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c45e27a79480475ba20255a3ecddcaf1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:36.045435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b533481-191e-4f3e-bd54-200b453a4bd0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:36.169368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0b533481-191e-4f3e-bd54-200b453a4bd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c45e27a79480475ba20255a3ecddcaf1', 'request_id': 'req-0b533481-191e-4f3e-bd54-200b453a4bd0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:36.183766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0b533481-191e-4f3e-bd54-200b453a4bd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c45e27a79480475ba20255a3ecddcaf1', 'request_id': 'req-0b533481-191e-4f3e-bd54-200b453a4bd0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:36.202742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0b533481-191e-4f3e-bd54-200b453a4bd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c45e27a79480475ba20255a3ecddcaf1', 'request_id': 'req-0b533481-191e-4f3e-bd54-200b453a4bd0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:36.231481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0b533481-191e-4f3e-bd54-200b453a4bd0 admin admin] The token cache is being invalidate because project c45e27a79480475ba20255a3ecddcaf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:36.231779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0b533481-191e-4f3e-bd54-200b453a4bd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:36.236632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0b533481-191e-4f3e-bd54-200b453a4bd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:36.238673 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2188/4305] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:35 2022] DELETE /identity/v3/projects/c45e27a79480475ba20255a3ecddcaf1 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:36.244162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6c371-164f-4225-acc9-e6c773437916 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:36.246450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6c371-164f-4225-acc9-e6c773437916 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:36.246903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6c371-164f-4225-acc9-e6c773437916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:36.279513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f3b6c371-164f-4225-acc9-e6c773437916 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:36.281153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f3b6c371-164f-4225-acc9-e6c773437916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:36.381124 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2118/4306] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:36.385666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01901396-b227-4473-86d9-affd994226e7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:36.386565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01901396-b227-4473-86d9-affd994226e7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:36.387036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01901396-b227-4473-86d9-affd994226e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:36.415246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-01901396-b227-4473-86d9-affd994226e7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:36.417779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-01901396-b227-4473-86d9-affd994226e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:36.430604 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2189/4307] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:36.435088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6c371-164f-4225-acc9-e6c773437916 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:36.446337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6c371-164f-4225-acc9-e6c773437916 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:36.453913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b2301-5679-4c8b-b493-886282b16c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:36.455421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b2301-5679-4c8b-b493-886282b16c78 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:36.455973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b2301-5679-4c8b-b493-886282b16c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:36.456462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b2301-5679-4c8b-b493-886282b16c78 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:36.458902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b2301-5679-4c8b-b493-886282b16c78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:03:36.459465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b2301-5679-4c8b-b493-886282b16c78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:36.462025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b2301-5679-4c8b-b493-886282b16c78 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:36.485235 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2119/4308] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:03:36.488608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01901396-b227-4473-86d9-affd994226e7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:36.502471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01901396-b227-4473-86d9-affd994226e7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:36.508333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8f683-f7ed-43d3-ab61-0b587a4f5b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:36.509137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8f683-f7ed-43d3-ab61-0b587a4f5b37 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:36.509543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8f683-f7ed-43d3-ab61-0b587a4f5b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:36.510070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8f683-f7ed-43d3-ab61-0b587a4f5b37 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:36.511033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8f683-f7ed-43d3-ab61-0b587a4f5b37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:36.513247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8f683-f7ed-43d3-ab61-0b587a4f5b37 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:36.558641 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2190/4309] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:03:36 2022] POST /identity/v3/projects => generated 393 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:36.563290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b2301-5679-4c8b-b493-886282b16c78 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:36.575136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b2301-5679-4c8b-b493-886282b16c78 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:36.726947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb2d50-1ddc-4668-9f8f-d840d02945f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:36.728042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb2d50-1ddc-4668-9f8f-d840d02945f9 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:36.728504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb2d50-1ddc-4668-9f8f-d840d02945f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:36.728797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb2d50-1ddc-4668-9f8f-d840d02945f9 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:36.729465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb2d50-1ddc-4668-9f8f-d840d02945f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:36.731370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb2d50-1ddc-4668-9f8f-d840d02945f9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:36.789876 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2120/4310] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:36 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:03:36.790884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8f683-f7ed-43d3-ab61-0b587a4f5b37 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:36.813686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8f683-f7ed-43d3-ab61-0b587a4f5b37 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:36.831929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d7644-39e8-406e-ae7f-ebf192412a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:36.832570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d7644-39e8-406e-ae7f-ebf192412a9f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:36.832859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d7644-39e8-406e-ae7f-ebf192412a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:36.833189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d7644-39e8-406e-ae7f-ebf192412a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:36.833926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d7644-39e8-406e-ae7f-ebf192412a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:36.835587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d7644-39e8-406e-ae7f-ebf192412a9f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:36.879800 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2191/4311] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:36.883968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb2d50-1ddc-4668-9f8f-d840d02945f9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:36.900842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb2d50-1ddc-4668-9f8f-d840d02945f9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:36.914146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803ce6d-cf2f-47be-b7b0-965e474388f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:36.914793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803ce6d-cf2f-47be-b7b0-965e474388f9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:36.918153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803ce6d-cf2f-47be-b7b0-965e474388f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:36.918512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803ce6d-cf2f-47be-b7b0-965e474388f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:36.919390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803ce6d-cf2f-47be-b7b0-965e474388f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:36.921094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803ce6d-cf2f-47be-b7b0-965e474388f9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:36.962779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d7644-39e8-406e-ae7f-ebf192412a9f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:36.975339 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2121/4312] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:36.987200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d7644-39e8-406e-ae7f-ebf192412a9f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:37.008044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64db8dc-c65b-4b00-a29e-8b695a1490b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:37.008752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64db8dc-c65b-4b00-a29e-8b695a1490b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:37.010170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64db8dc-c65b-4b00-a29e-8b695a1490b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:37.010617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64db8dc-c65b-4b00-a29e-8b695a1490b7 admin admin] PATH_INFO: `/v3/projects/03467039d25647d29e6254e8cb9a86dc/users/fe32e8b9bf234d6eabafc847e54eaf00/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:37.018482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:37.050611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64db8dc-c65b-4b00-a29e-8b695a1490b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03467039d25647d29e6254e8cb9a86dc, user_id=fe32e8b9bf234d6eabafc847e54eaf00, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:37.053252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64db8dc-c65b-4b00-a29e-8b695a1490b7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:37.091389 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2192/4313] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:36 2022] PUT /identity/v3/projects/03467039d25647d29e6254e8cb9a86dc/users/fe32e8b9bf234d6eabafc847e54eaf00/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:37.197363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0810cdb-e82d-4b63-8e12-16faaa4cbe49 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:37.197620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0810cdb-e82d-4b63-8e12-16faaa4cbe49 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:37.197855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0810cdb-e82d-4b63-8e12-16faaa4cbe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:37.215031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:37.244724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e0810cdb-e82d-4b63-8e12-16faaa4cbe49 None None] MFA Rules not processed for user `fe32e8b9bf234d6eabafc847e54eaf00`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:37.247712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e0810cdb-e82d-4b63-8e12-16faaa4cbe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:37.400789 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2122/4314] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:37.401829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64db8dc-c65b-4b00-a29e-8b695a1490b7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:37.504041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc860c4-a6ee-4f41-87e1-d4528524fdaf None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:37.504335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc860c4-a6ee-4f41-87e1-d4528524fdaf None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:37.504636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc860c4-a6ee-4f41-87e1-d4528524fdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:37.527315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64db8dc-c65b-4b00-a29e-8b695a1490b7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:37.536403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9d94a-2a7a-4ac9-b3f1-f5bb205b134d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:37.540286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9d94a-2a7a-4ac9-b3f1-f5bb205b134d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:37.541632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9d94a-2a7a-4ac9-b3f1-f5bb205b134d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:37.542464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9d94a-2a7a-4ac9-b3f1-f5bb205b134d admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:37.545457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9d94a-2a7a-4ac9-b3f1-f5bb205b134d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:37.549128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9d94a-2a7a-4ac9-b3f1-f5bb205b134d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:37.552027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5bc860c4-a6ee-4f41-87e1-d4528524fdaf None None] MFA Rules not processed for user `b40eb82cf0d64b05b2260c6ea0afaae2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:37.555818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5bc860c4-a6ee-4f41-87e1-d4528524fdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:37.675980 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2193/4315] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:03:37 2022] POST /identity/v3/projects => generated 393 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:37.680337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9d94a-2a7a-4ac9-b3f1-f5bb205b134d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:37.684344 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2123/4316] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:37.699780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc860c4-a6ee-4f41-87e1-d4528524fdaf None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:37.702371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5bc860c4-a6ee-4f41-87e1-d4528524fdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:37.813795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc860c4-a6ee-4f41-87e1-d4528524fdaf None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:37.839196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9d94a-2a7a-4ac9-b3f1-f5bb205b134d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:37.845714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec4dca-faa7-4846-b67f-ef78e2c6c185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:37.846544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec4dca-faa7-4846-b67f-ef78e2c6c185 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:37.847150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec4dca-faa7-4846-b67f-ef78e2c6c185 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:37.847527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec4dca-faa7-4846-b67f-ef78e2c6c185 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:37.848175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec4dca-faa7-4846-b67f-ef78e2c6c185 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:37.850282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec4dca-faa7-4846-b67f-ef78e2c6c185 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:37.880652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:37.904751 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2194/4317] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:37 2022] POST /identity/v3/users => generated 303 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:03:37.905783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec4dca-faa7-4846-b67f-ef78e2c6c185 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:37.917128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e14819-59e6-4ad2-ab2e-caa7c5c7cdaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:37.917749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec4dca-faa7-4846-b67f-ef78e2c6c185 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:37.919993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e14819-59e6-4ad2-ab2e-caa7c5c7cdaf service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:37.920520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e14819-59e6-4ad2-ab2e-caa7c5c7cdaf service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:37.921337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e14819-59e6-4ad2-ab2e-caa7c5c7cdaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:37.923164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036238c2-fcb3-446e-9897-be7d331ab1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:37.923905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036238c2-fcb3-446e-9897-be7d331ab1a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:37.924231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036238c2-fcb3-446e-9897-be7d331ab1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:37.924571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036238c2-fcb3-446e-9897-be7d331ab1a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:37.925439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-93e14819-59e6-4ad2-ab2e-caa7c5c7cdaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:37.925888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036238c2-fcb3-446e-9897-be7d331ab1a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:37.927808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036238c2-fcb3-446e-9897-be7d331ab1a1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:37.948219 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2195/4318] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:37.953252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036238c2-fcb3-446e-9897-be7d331ab1a1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:37.992706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036238c2-fcb3-446e-9897-be7d331ab1a1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:38.008557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cdede-b562-4170-911e-579e5e97a377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:38.009823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cdede-b562-4170-911e-579e5e97a377 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:38.010114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cdede-b562-4170-911e-579e5e97a377 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:38.010457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cdede-b562-4170-911e-579e5e97a377 admin admin] PATH_INFO: `/v3/projects/71de713bc6964b95a4eba8678f534470/users/6b5b66a79292456aa14f0de8510ebc86/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:38.070223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e14819-59e6-4ad2-ab2e-caa7c5c7cdaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:38.073293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e14819-59e6-4ad2-ab2e-caa7c5c7cdaf service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:38.092304 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2124/4319] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:38.093389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cdede-b562-4170-911e-579e5e97a377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71de713bc6964b95a4eba8678f534470, user_id=6b5b66a79292456aa14f0de8510ebc86, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:38.101312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cdede-b562-4170-911e-579e5e97a377 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:38.154778 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2196/4320] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:37 2022] PUT /identity/v3/projects/71de713bc6964b95a4eba8678f534470/users/6b5b66a79292456aa14f0de8510ebc86/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:38.155109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e14819-59e6-4ad2-ab2e-caa7c5c7cdaf service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:38.172385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e14819-59e6-4ad2-ab2e-caa7c5c7cdaf service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:38.403377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f27dcc-580e-45b6-ae06-6db32977b519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:38.405809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f27dcc-580e-45b6-ae06-6db32977b519 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:38.406402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f27dcc-580e-45b6-ae06-6db32977b519 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:38.406857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f27dcc-580e-45b6-ae06-6db32977b519 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:38.407704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f27dcc-580e-45b6-ae06-6db32977b519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:38.409531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f27dcc-580e-45b6-ae06-6db32977b519 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:38.423255 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2125/4321] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:38.427267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cdede-b562-4170-911e-579e5e97a377 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:38.450745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cdede-b562-4170-911e-579e5e97a377 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:38.456142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76499f12-645a-4615-8d95-00625426e78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:38.457193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76499f12-645a-4615-8d95-00625426e78a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:38.458230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76499f12-645a-4615-8d95-00625426e78a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:38.459029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76499f12-645a-4615-8d95-00625426e78a admin admin] PATH_INFO: `/v3/projects/71de713bc6964b95a4eba8678f534470/users/6b5b66a79292456aa14f0de8510ebc86/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:38.468449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76499f12-645a-4615-8d95-00625426e78a admin admin] RBAC: Authorizing `identity:create_grant(project_id=71de713bc6964b95a4eba8678f534470, user_id=6b5b66a79292456aa14f0de8510ebc86, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:38.471710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76499f12-645a-4615-8d95-00625426e78a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:38.517359 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2197/4322] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:38 2022] PUT /identity/v3/projects/71de713bc6964b95a4eba8678f534470/users/6b5b66a79292456aa14f0de8510ebc86/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:38.518686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e9a48-59d6-40c4-b6c4-3fd49d4109ee None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:38.518978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e9a48-59d6-40c4-b6c4-3fd49d4109ee None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:38.519260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e9a48-59d6-40c4-b6c4-3fd49d4109ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:38.591089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bc8e9a48-59d6-40c4-b6c4-3fd49d4109ee None None] MFA Rules not processed for user `6b5b66a79292456aa14f0de8510ebc86`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:38.592851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bc8e9a48-59d6-40c4-b6c4-3fd49d4109ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:38.802281 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2126/4323] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:38.838399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76499f12-645a-4615-8d95-00625426e78a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:38.841321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-76499f12-645a-4615-8d95-00625426e78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:38.876378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:39.028023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76499f12-645a-4615-8d95-00625426e78a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:39.033292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84130278-670f-4154-8b42-33e325c828d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:39.035259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84130278-670f-4154-8b42-33e325c828d9 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:39.035728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84130278-670f-4154-8b42-33e325c828d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:39.036170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84130278-670f-4154-8b42-33e325c828d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:39.048609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84130278-670f-4154-8b42-33e325c828d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:39.050578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84130278-670f-4154-8b42-33e325c828d9 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:39.063348 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2198/4324] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:38 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:41.998662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e9a48-59d6-40c4-b6c4-3fd49d4109ee None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:42.004686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:42.160800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e9a48-59d6-40c4-b6c4-3fd49d4109ee None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:42.168057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:42.169138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:42.169678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:42.170318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] PATH_INFO: `/v3/users/fe32e8b9bf234d6eabafc847e54eaf00` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:42.176694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe32e8b9bf234d6eabafc847e54eaf00)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:42.179415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:42.287547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe32e8b9bf234d6eabafc847e54eaf00'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:42.319898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe32e8b9bf234d6eabafc847e54eaf00'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:42.322927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe32e8b9bf234d6eabafc847e54eaf00'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:42.335089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe32e8b9bf234d6eabafc847e54eaf00'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:42.358567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe32e8b9bf234d6eabafc847e54eaf00'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:42.370255 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2127/4325] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:41 2022] DELETE /identity/v3/users/fe32e8b9bf234d6eabafc847e54eaf00 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:42.375173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d3b66-17da-4f08-80a8-bd71449521de None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:42.375742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d3b66-17da-4f08-80a8-bd71449521de None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:42.375841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d3b66-17da-4f08-80a8-bd71449521de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:42.405028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d39d3b66-17da-4f08-80a8-bd71449521de None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:42.406542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d39d3b66-17da-4f08-80a8-bd71449521de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:42.557600 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2199/4326] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:42.569432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:42.571235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:42.696687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e508ca-52b3-43ac-94b1-759a7b87efcb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:42.706217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d97d6-8551-4d10-9ff0-fc9cb38f957e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:42.708234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d97d6-8551-4d10-9ff0-fc9cb38f957e service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:42.708820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d97d6-8551-4d10-9ff0-fc9cb38f957e service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:42.710149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d97d6-8551-4d10-9ff0-fc9cb38f957e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:42.734843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d97d6-8551-4d10-9ff0-fc9cb38f957e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:42.736530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d97d6-8551-4d10-9ff0-fc9cb38f957e service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:42.766434 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2128/4327] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:43.979784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d3b66-17da-4f08-80a8-bd71449521de None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:43.981398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d39d3b66-17da-4f08-80a8-bd71449521de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:43.993282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d3b66-17da-4f08-80a8-bd71449521de None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:43.999011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:43.999656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:43.999991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:44.000217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] PATH_INFO: `/v3/projects/03467039d25647d29e6254e8cb9a86dc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:44.001403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03467039d25647d29e6254e8cb9a86dc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:44.003339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:44.061073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03467039d25647d29e6254e8cb9a86dc', 'request_id': 'req-56f7d85f-3e28-4be0-9910-7b9685beb044'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.071280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03467039d25647d29e6254e8cb9a86dc', 'request_id': 'req-56f7d85f-3e28-4be0-9910-7b9685beb044'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.079530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03467039d25647d29e6254e8cb9a86dc', 'request_id': 'req-56f7d85f-3e28-4be0-9910-7b9685beb044'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.093915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] The token cache is being invalidate because project 03467039d25647d29e6254e8cb9a86dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:44.094218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.094902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.096618 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2200/4328] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:43 2022] DELETE /identity/v3/projects/03467039d25647d29e6254e8cb9a86dc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:44.102470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d97d6-8551-4d10-9ff0-fc9cb38f957e service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:44.125139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d97d6-8551-4d10-9ff0-fc9cb38f957e service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:44.153190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:44.197548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c35fc-c8b1-460f-86da-836d1c00d327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:44.198309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c35fc-c8b1-460f-86da-836d1c00d327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:44.198595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c35fc-c8b1-460f-86da-836d1c00d327 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:44.198874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c35fc-c8b1-460f-86da-836d1c00d327 admin admin] PATH_INFO: `/v3/users/6b5b66a79292456aa14f0de8510ebc86` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:44.200003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c35fc-c8b1-460f-86da-836d1c00d327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b5b66a79292456aa14f0de8510ebc86)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:44.202033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c35fc-c8b1-460f-86da-836d1c00d327 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:44.265353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-060c35fc-c8b1-460f-86da-836d1c00d327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b5b66a79292456aa14f0de8510ebc86'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.295425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-060c35fc-c8b1-460f-86da-836d1c00d327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b5b66a79292456aa14f0de8510ebc86'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.296412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-060c35fc-c8b1-460f-86da-836d1c00d327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b5b66a79292456aa14f0de8510ebc86'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.305717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-060c35fc-c8b1-460f-86da-836d1c00d327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b5b66a79292456aa14f0de8510ebc86'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.316204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-060c35fc-c8b1-460f-86da-836d1c00d327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b5b66a79292456aa14f0de8510ebc86'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.326804 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2129/4329] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:44 2022] DELETE /identity/v3/users/6b5b66a79292456aa14f0de8510ebc86 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:44.710339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:44.711838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:44.728088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7d85f-3e28-4be0-9910-7b9685beb044 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:44.770806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:44.799973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:44.800652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:44.800969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:44.801272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c admin admin] PATH_INFO: `/v3/projects/71de713bc6964b95a4eba8678f534470` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:44.802647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c admin admin] RBAC: Authorizing `identity:delete_project(project_id=71de713bc6964b95a4eba8678f534470)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:44.804608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:44.866386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71de713bc6964b95a4eba8678f534470', 'request_id': 'req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.874225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71de713bc6964b95a4eba8678f534470', 'request_id': 'req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.881338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71de713bc6964b95a4eba8678f534470', 'request_id': 'req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.894003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c admin admin] The token cache is being invalidate because project 71de713bc6964b95a4eba8678f534470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:44.894384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.895066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e5e61e3-7fbb-40cd-a5c1-dfa93663029c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:44.896486 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2201/4330] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:44 2022] DELETE /identity/v3/projects/71de713bc6964b95a4eba8678f534470 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:44.901786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346af56-89cb-4eb8-a8e1-1a96efb00c3e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:44.902084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346af56-89cb-4eb8-a8e1-1a96efb00c3e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:44.902370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346af56-89cb-4eb8-a8e1-1a96efb00c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:44.922592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e346af56-89cb-4eb8-a8e1-1a96efb00c3e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:44.924477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e346af56-89cb-4eb8-a8e1-1a96efb00c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:45.015930 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2130/4331] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:03:45.017544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7244a-46e3-49e8-acd2-15905b58e551 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:45.018111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7244a-46e3-49e8-acd2-15905b58e551 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:45.018111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7244a-46e3-49e8-acd2-15905b58e551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:45.073192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ecd7244a-46e3-49e8-acd2-15905b58e551 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:45.074866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ecd7244a-46e3-49e8-acd2-15905b58e551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:45.086431 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2202/4332] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:03:45.088980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346af56-89cb-4eb8-a8e1-1a96efb00c3e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:45.115655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346af56-89cb-4eb8-a8e1-1a96efb00c3e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:45.122527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca515472-9744-4806-9018-0baf0d8c3728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:45.123546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca515472-9744-4806-9018-0baf0d8c3728 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:45.124063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca515472-9744-4806-9018-0baf0d8c3728 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:45.124654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca515472-9744-4806-9018-0baf0d8c3728 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:45.125598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca515472-9744-4806-9018-0baf0d8c3728 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:03:45.126160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca515472-9744-4806-9018-0baf0d8c3728 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:45.128030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca515472-9744-4806-9018-0baf0d8c3728 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:45.144708 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2131/4333] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:03:45.149289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7244a-46e3-49e8-acd2-15905b58e551 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:45.167570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7244a-46e3-49e8-acd2-15905b58e551 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:45.174149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dab1d-8d5e-438e-a6e2-3d4947471b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:45.175158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dab1d-8d5e-438e-a6e2-3d4947471b3f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:45.175528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dab1d-8d5e-438e-a6e2-3d4947471b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:45.175753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dab1d-8d5e-438e-a6e2-3d4947471b3f admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:45.177571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5dab1d-8d5e-438e-a6e2-3d4947471b3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:45.179551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5dab1d-8d5e-438e-a6e2-3d4947471b3f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:45.222479 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2203/4334] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:03:45 2022] POST /identity/v3/projects => generated 389 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:45.224285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca515472-9744-4806-9018-0baf0d8c3728 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:45.250367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca515472-9744-4806-9018-0baf0d8c3728 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:45.296399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:45.339396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c751c7-f809-454b-a240-4aa764ce68ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:45.340227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c751c7-f809-454b-a240-4aa764ce68ab admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:45.340683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c751c7-f809-454b-a240-4aa764ce68ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:45.341123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c751c7-f809-454b-a240-4aa764ce68ab admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:45.343285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c751c7-f809-454b-a240-4aa764ce68ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:45.346601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c751c7-f809-454b-a240-4aa764ce68ab admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:45.382126 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2132/4335] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:45 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:45.383039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dab1d-8d5e-438e-a6e2-3d4947471b3f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:45.401241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dab1d-8d5e-438e-a6e2-3d4947471b3f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:45.419986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e128f-f92a-4741-8129-557b25989216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:45.420642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e128f-f92a-4741-8129-557b25989216 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:45.420964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e128f-f92a-4741-8129-557b25989216 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:45.421447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e128f-f92a-4741-8129-557b25989216 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:45.422306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e128f-f92a-4741-8129-557b25989216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:45.424246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e128f-f92a-4741-8129-557b25989216 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:45.442780 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2204/4336] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:45.453298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c751c7-f809-454b-a240-4aa764ce68ab admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:45.475601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c751c7-f809-454b-a240-4aa764ce68ab admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:45.484271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb782ff-982b-4690-ad22-b4ea1b9b0416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:45.485684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb782ff-982b-4690-ad22-b4ea1b9b0416 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:45.486040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb782ff-982b-4690-ad22-b4ea1b9b0416 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:45.486405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb782ff-982b-4690-ad22-b4ea1b9b0416 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:45.487558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb782ff-982b-4690-ad22-b4ea1b9b0416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:45.489269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb782ff-982b-4690-ad22-b4ea1b9b0416 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:45.505543 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2133/4337] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:45.506330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e128f-f92a-4741-8129-557b25989216 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:45.520243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e128f-f92a-4741-8129-557b25989216 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:45.528074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d221f-7e91-4a6a-a24b-aaaa07f941ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:45.528918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d221f-7e91-4a6a-a24b-aaaa07f941ca admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:45.529377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d221f-7e91-4a6a-a24b-aaaa07f941ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:45.529764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d221f-7e91-4a6a-a24b-aaaa07f941ca admin admin] PATH_INFO: `/v3/projects/978e9b2208da49f9b206829b62fd49af/users/ec13e5aa744c49b8831d09c8806a02fc/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:45.551207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d221f-7e91-4a6a-a24b-aaaa07f941ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=978e9b2208da49f9b206829b62fd49af, user_id=ec13e5aa744c49b8831d09c8806a02fc, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:45.554317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d221f-7e91-4a6a-a24b-aaaa07f941ca admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:45.583126 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2205/4338] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:45 2022] PUT /identity/v3/projects/978e9b2208da49f9b206829b62fd49af/users/ec13e5aa744c49b8831d09c8806a02fc/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:45.586848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e37f1d-74ea-4c18-b94d-5d914c99f719 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:45.587938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e37f1d-74ea-4c18-b94d-5d914c99f719 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:45.588490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e37f1d-74ea-4c18-b94d-5d914c99f719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:45.640974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-08e37f1d-74ea-4c18-b94d-5d914c99f719 None None] MFA Rules not processed for user `ec13e5aa744c49b8831d09c8806a02fc`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:45.642253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-08e37f1d-74ea-4c18-b94d-5d914c99f719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:45.724057 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2134/4339] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:45.733649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d221f-7e91-4a6a-a24b-aaaa07f941ca admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:45.735740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bb8d221f-7e91-4a6a-a24b-aaaa07f941ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:45.842750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d221f-7e91-4a6a-a24b-aaaa07f941ca admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:45.847980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca80765-a333-4a79-ad7f-51c787cadf59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:45.848875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca80765-a333-4a79-ad7f-51c787cadf59 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:45.849507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca80765-a333-4a79-ad7f-51c787cadf59 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:45.849892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca80765-a333-4a79-ad7f-51c787cadf59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:45.861017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca80765-a333-4a79-ad7f-51c787cadf59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:45.862983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca80765-a333-4a79-ad7f-51c787cadf59 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:45.874268 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2206/4340] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:45.974471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e37f1d-74ea-4c18-b94d-5d914c99f719 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:46.112748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e37f1d-74ea-4c18-b94d-5d914c99f719 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:46.122979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:46.122979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:46.122979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:46.122979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] PATH_INFO: `/v3/users/ec13e5aa744c49b8831d09c8806a02fc` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:46.124708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec13e5aa744c49b8831d09c8806a02fc)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:46.127071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:46.219714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:46.223378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec13e5aa744c49b8831d09c8806a02fc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:46.249789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec13e5aa744c49b8831d09c8806a02fc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:46.252329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec13e5aa744c49b8831d09c8806a02fc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:46.275713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec13e5aa744c49b8831d09c8806a02fc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:46.287619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec13e5aa744c49b8831d09c8806a02fc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:46.298794 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2135/4341] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:45 2022] DELETE /identity/v3/users/ec13e5aa744c49b8831d09c8806a02fc => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:46.300389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b35820-d646-4ba2-aea1-65900f8a18bd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:46.300877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b35820-d646-4ba2-aea1-65900f8a18bd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:46.301026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b35820-d646-4ba2-aea1-65900f8a18bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:46.319233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:46.328608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-12b35820-d646-4ba2-aea1-65900f8a18bd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:46.330222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-12b35820-d646-4ba2-aea1-65900f8a18bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:46.521859 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2207/4342] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:46.543342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:46.544682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:46.636992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf42f7-e232-4b89-b8a0-d5686bc160bd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:46.656318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ac2d4-c896-4608-b98e-5e95a1a26c47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:46.659335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ac2d4-c896-4608-b98e-5e95a1a26c47 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:46.659644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ac2d4-c896-4608-b98e-5e95a1a26c47 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:46.659923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ac2d4-c896-4608-b98e-5e95a1a26c47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:46.690683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ac2d4-c896-4608-b98e-5e95a1a26c47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:46.692461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ac2d4-c896-4608-b98e-5e95a1a26c47 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:46.714694 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2136/4343] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:47.624439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b35820-d646-4ba2-aea1-65900f8a18bd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:47.626037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-12b35820-d646-4ba2-aea1-65900f8a18bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:47.650510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b35820-d646-4ba2-aea1-65900f8a18bd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:47.661641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8ab94-2159-46ff-9481-26380df8eeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:47.664134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8ab94-2159-46ff-9481-26380df8eeac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:47.664676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8ab94-2159-46ff-9481-26380df8eeac admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:47.665685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8ab94-2159-46ff-9481-26380df8eeac admin admin] PATH_INFO: `/v3/projects/978e9b2208da49f9b206829b62fd49af` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:47.667275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8ab94-2159-46ff-9481-26380df8eeac admin admin] RBAC: Authorizing `identity:delete_project(project_id=978e9b2208da49f9b206829b62fd49af)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:47.669947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8ab94-2159-46ff-9481-26380df8eeac admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:47.753695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8a8ab94-2159-46ff-9481-26380df8eeac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '978e9b2208da49f9b206829b62fd49af', 'request_id': 'req-a8a8ab94-2159-46ff-9481-26380df8eeac'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:47.771160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8a8ab94-2159-46ff-9481-26380df8eeac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '978e9b2208da49f9b206829b62fd49af', 'request_id': 'req-a8a8ab94-2159-46ff-9481-26380df8eeac'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:47.787880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8a8ab94-2159-46ff-9481-26380df8eeac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '978e9b2208da49f9b206829b62fd49af', 'request_id': 'req-a8a8ab94-2159-46ff-9481-26380df8eeac'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:47.800935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8a8ab94-2159-46ff-9481-26380df8eeac admin admin] The token cache is being invalidate because project 978e9b2208da49f9b206829b62fd49af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:47.801351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8a8ab94-2159-46ff-9481-26380df8eeac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:47.802168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8a8ab94-2159-46ff-9481-26380df8eeac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:47.804349 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2208/4344] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:47 2022] DELETE /identity/v3/projects/978e9b2208da49f9b206829b62fd49af => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:47.808947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3606a9-cc78-41f4-aa1e-baa6b3872f5b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:47.809608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3606a9-cc78-41f4-aa1e-baa6b3872f5b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:47.809802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3606a9-cc78-41f4-aa1e-baa6b3872f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:47.843266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-dc3606a9-cc78-41f4-aa1e-baa6b3872f5b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:47.844880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dc3606a9-cc78-41f4-aa1e-baa6b3872f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:48.017251 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2137/4345] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:48.025359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f37cfa-c64a-4907-ad65-cc7a45fc608a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:48.026030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f37cfa-c64a-4907-ad65-cc7a45fc608a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:48.026234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f37cfa-c64a-4907-ad65-cc7a45fc608a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:48.058503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a6f37cfa-c64a-4907-ad65-cc7a45fc608a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:48.060128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a6f37cfa-c64a-4907-ad65-cc7a45fc608a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:48.071592 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2209/4346] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:48.076870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3606a9-cc78-41f4-aa1e-baa6b3872f5b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:48.101755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3606a9-cc78-41f4-aa1e-baa6b3872f5b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:48.105723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f37cfa-c64a-4907-ad65-cc7a45fc608a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:48.107740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a6f37cfa-c64a-4907-ad65-cc7a45fc608a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:48.138319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e284310-34a0-4f22-ade8-0ad1397cdb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:48.140460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e284310-34a0-4f22-ade8-0ad1397cdb18 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:48.142690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e284310-34a0-4f22-ade8-0ad1397cdb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:48.143414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e284310-34a0-4f22-ade8-0ad1397cdb18 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:48.146334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e284310-34a0-4f22-ade8-0ad1397cdb18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:03:48.146763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e284310-34a0-4f22-ade8-0ad1397cdb18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:48.151408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e284310-34a0-4f22-ade8-0ad1397cdb18 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:48.203581 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2138/4347] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:48.205042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e284310-34a0-4f22-ade8-0ad1397cdb18 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:48.231567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e284310-34a0-4f22-ade8-0ad1397cdb18 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:48.246067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88d652-f028-4004-8000-5860b65b8176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:48.248056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88d652-f028-4004-8000-5860b65b8176 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:48.248989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88d652-f028-4004-8000-5860b65b8176 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:48.249680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88d652-f028-4004-8000-5860b65b8176 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:48.252361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88d652-f028-4004-8000-5860b65b8176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:48.256212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88d652-f028-4004-8000-5860b65b8176 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:48.267397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f37cfa-c64a-4907-ad65-cc7a45fc608a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:48.273147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d195a9-1280-476e-bce4-08e2c60c0afa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:48.275411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d195a9-1280-476e-bce4-08e2c60c0afa service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:48.275962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d195a9-1280-476e-bce4-08e2c60c0afa service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:48.276419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d195a9-1280-476e-bce4-08e2c60c0afa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:48.278925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-15d195a9-1280-476e-bce4-08e2c60c0afa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:48.292578 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2139/4348] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:03:48 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:48.297266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88d652-f028-4004-8000-5860b65b8176 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:48.336323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:48.352071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:48.399247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d195a9-1280-476e-bce4-08e2c60c0afa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:48.401807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d195a9-1280-476e-bce4-08e2c60c0afa service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:48.407418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88d652-f028-4004-8000-5860b65b8176 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:48.417373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3eb51-eec6-4bba-9cf2-18d0bfffaca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:48.418172 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2210/4349] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:48.419063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d195a9-1280-476e-bce4-08e2c60c0afa service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:48.420481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-15d195a9-1280-476e-bce4-08e2c60c0afa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:48.430707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3eb51-eec6-4bba-9cf2-18d0bfffaca8 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:48.431071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3eb51-eec6-4bba-9cf2-18d0bfffaca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:48.431363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3eb51-eec6-4bba-9cf2-18d0bfffaca8 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:48.432666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3eb51-eec6-4bba-9cf2-18d0bfffaca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:48.434679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3eb51-eec6-4bba-9cf2-18d0bfffaca8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:48.437467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d195a9-1280-476e-bce4-08e2c60c0afa service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:48.481952 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2140/4350] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:48 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:48.482982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3eb51-eec6-4bba-9cf2-18d0bfffaca8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:48.504241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3eb51-eec6-4bba-9cf2-18d0bfffaca8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:48.515956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb80174-4e70-4229-b983-8d589f04c529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:48.516591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb80174-4e70-4229-b983-8d589f04c529 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:48.516880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb80174-4e70-4229-b983-8d589f04c529 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:48.517187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb80174-4e70-4229-b983-8d589f04c529 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:48.518169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb80174-4e70-4229-b983-8d589f04c529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:48.519729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb80174-4e70-4229-b983-8d589f04c529 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:48.525675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddb139-06d5-4978-abae-3817d97dac7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:48.526628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddb139-06d5-4978-abae-3817d97dac7b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:48.526628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddb139-06d5-4978-abae-3817d97dac7b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:48.526873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddb139-06d5-4978-abae-3817d97dac7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:48.532849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c6ddb139-06d5-4978-abae-3817d97dac7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:48.545015 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2141/4351] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:48.545915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb80174-4e70-4229-b983-8d589f04c529 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:48.564039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb80174-4e70-4229-b983-8d589f04c529 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:48.574553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b679a-96ae-4e7e-9cf3-e36fe89eb250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:48.575345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b679a-96ae-4e7e-9cf3-e36fe89eb250 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:48.575739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b679a-96ae-4e7e-9cf3-e36fe89eb250 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:48.576123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b679a-96ae-4e7e-9cf3-e36fe89eb250 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:48.576873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b679a-96ae-4e7e-9cf3-e36fe89eb250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:48.582843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b679a-96ae-4e7e-9cf3-e36fe89eb250 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:48.606521 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2142/4352] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:48.610960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b679a-96ae-4e7e-9cf3-e36fe89eb250 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:48.628812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b679a-96ae-4e7e-9cf3-e36fe89eb250 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:48.636063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efef11-807c-42a6-b095-6036555584ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:48.636886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efef11-807c-42a6-b095-6036555584ac admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:48.637224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efef11-807c-42a6-b095-6036555584ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:48.637509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efef11-807c-42a6-b095-6036555584ac admin admin] PATH_INFO: `/v3/projects/1cb77ea78cc046f7aa8bf5d5f6cf7b2f/users/17b75a15889c4dd38315591ae8fc1b6a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:48.661020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddb139-06d5-4978-abae-3817d97dac7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:48.662742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddb139-06d5-4978-abae-3817d97dac7b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:48.662873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efef11-807c-42a6-b095-6036555584ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cb77ea78cc046f7aa8bf5d5f6cf7b2f, user_id=17b75a15889c4dd38315591ae8fc1b6a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:48.666588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efef11-807c-42a6-b095-6036555584ac admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:48.678967 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2211/4353] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:48.684368 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2143/4354] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:48 2022] PUT /identity/v3/projects/1cb77ea78cc046f7aa8bf5d5f6cf7b2f/users/17b75a15889c4dd38315591ae8fc1b6a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:48.696167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92c72c-6d46-4af8-be79-2d3b37e3090b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:48.696536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92c72c-6d46-4af8-be79-2d3b37e3090b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:48.696828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92c72c-6d46-4af8-be79-2d3b37e3090b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:48.737320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2b92c72c-6d46-4af8-be79-2d3b37e3090b None None] MFA Rules not processed for user `17b75a15889c4dd38315591ae8fc1b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:48.738653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b92c72c-6d46-4af8-be79-2d3b37e3090b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:48.834868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:48.855447 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2212/4355] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:48.865327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efef11-807c-42a6-b095-6036555584ac admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:48.936605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efef11-807c-42a6-b095-6036555584ac admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:48.941351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e4267-e422-48d0-bd6b-75182ae7ebe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:48.942066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e4267-e422-48d0-bd6b-75182ae7ebe2 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:48.942419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e4267-e422-48d0-bd6b-75182ae7ebe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:48.942770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e4267-e422-48d0-bd6b-75182ae7ebe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:48.956455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e4267-e422-48d0-bd6b-75182ae7ebe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:48.958084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e4267-e422-48d0-bd6b-75182ae7ebe2 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:48.968363 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2144/4356] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:50.667458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36903f-9f12-4600-b72f-54c67cd927bc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:50.667947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36903f-9f12-4600-b72f-54c67cd927bc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:50.668206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36903f-9f12-4600-b72f-54c67cd927bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:50.690502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-dd36903f-9f12-4600-b72f-54c67cd927bc None None] MFA Rules not processed for user `77edcf09f5f84e51a4fb08dbaf763a67`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:50.692421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dd36903f-9f12-4600-b72f-54c67cd927bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:50.782034 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2213/4357] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:54.093819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e4267-e422-48d0-bd6b-75182ae7ebe2 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:54.095372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8e8e4267-e422-48d0-bd6b-75182ae7ebe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:54.217773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbae045-910a-495e-b3e5-ce5f70373f1a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:54.218303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbae045-910a-495e-b3e5-ce5f70373f1a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:54.218410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbae045-910a-495e-b3e5-ce5f70373f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:54.243724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ecbae045-910a-495e-b3e5-ce5f70373f1a None None] MFA Rules not processed for user `c3187514ca644ff68c992526f38783c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:54.246107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ecbae045-910a-495e-b3e5-ce5f70373f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:54.331035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e4267-e422-48d0-bd6b-75182ae7ebe2 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:54.341710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:54.342372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:54.342656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:54.342936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] PATH_INFO: `/v3/users/cb94dfca0ec44bf4853be254984519ca` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:54.351649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb94dfca0ec44bf4853be254984519ca)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:54.357717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:54.369209 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2214/4358] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:54.392780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:54.505139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb94dfca0ec44bf4853be254984519ca'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:54.540804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb94dfca0ec44bf4853be254984519ca'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:54.541887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb94dfca0ec44bf4853be254984519ca'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:54.551933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb94dfca0ec44bf4853be254984519ca'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:54.562923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb94dfca0ec44bf4853be254984519ca'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:54.578764 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2145/4359] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:54 2022] DELETE /identity/v3/users/cb94dfca0ec44bf4853be254984519ca => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:55.658232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbae045-910a-495e-b3e5-ce5f70373f1a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:55.681926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbae045-910a-495e-b3e5-ce5f70373f1a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:55.830477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:55.831758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:55.832241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:55.832997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] PATH_INFO: `/v3/projects/6c96d2a581dd4fc299fc5020c93efe00` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:55.834425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c96d2a581dd4fc299fc5020c93efe00)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:55.836619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:55.932222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:55.946796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c96d2a581dd4fc299fc5020c93efe00', 'request_id': 'req-09aa5105-63f7-421a-aae7-af8129e51037'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:55.984224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c96d2a581dd4fc299fc5020c93efe00', 'request_id': 'req-09aa5105-63f7-421a-aae7-af8129e51037'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:55.995652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c96d2a581dd4fc299fc5020c93efe00', 'request_id': 'req-09aa5105-63f7-421a-aae7-af8129e51037'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:56.017925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] The token cache is being invalidate because project 6c96d2a581dd4fc299fc5020c93efe00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:56.018223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:56.019537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:56.021732 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2215/4360] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:55 2022] DELETE /identity/v3/projects/6c96d2a581dd4fc299fc5020c93efe00 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:56.026678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:56.029452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:56.059180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c939341-3fb8-4740-b4d1-24abf9ea2e2d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:56.247618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:56.250623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:56.251125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:56.251583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] PATH_INFO: `/v3/users/b40eb82cf0d64b05b2260c6ea0afaae2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:56.254497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b40eb82cf0d64b05b2260c6ea0afaae2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:56.263793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:56.395911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b40eb82cf0d64b05b2260c6ea0afaae2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:56.439928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b40eb82cf0d64b05b2260c6ea0afaae2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:56.441354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b40eb82cf0d64b05b2260c6ea0afaae2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:56.458669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b40eb82cf0d64b05b2260c6ea0afaae2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:56.469166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b40eb82cf0d64b05b2260c6ea0afaae2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:56.479652 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2146/4361] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:03:56 2022] DELETE /identity/v3/users/b40eb82cf0d64b05b2260c6ea0afaae2 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:56.896947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:56.898770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:57.113225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa5105-63f7-421a-aae7-af8129e51037 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:57.283233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57ab3b-9641-473a-92f3-e8300630f159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:57.285274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57ab3b-9641-473a-92f3-e8300630f159 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:57.286215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57ab3b-9641-473a-92f3-e8300630f159 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:57.287096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57ab3b-9641-473a-92f3-e8300630f159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:57.292531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7d57ab3b-9641-473a-92f3-e8300630f159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:57.323445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57ab3b-9641-473a-92f3-e8300630f159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:57.325661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57ab3b-9641-473a-92f3-e8300630f159 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:57.369180 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2216/4362] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:03:57.486616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:57.489716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:57.542410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:57.716246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f91f85-6541-4a1b-ae4a-d72b141ae9f6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:57.725816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed6684-b101-484c-a60f-015949a7659f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:57.726925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed6684-b101-484c-a60f-015949a7659f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:57.727500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed6684-b101-484c-a60f-015949a7659f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:57.728460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed6684-b101-484c-a60f-015949a7659f admin admin] PATH_INFO: `/v3/projects/60ba47cefe86402abcf32330758da25f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:57.736597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed6684-b101-484c-a60f-015949a7659f admin admin] RBAC: Authorizing `identity:delete_project(project_id=60ba47cefe86402abcf32330758da25f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:57.740096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed6684-b101-484c-a60f-015949a7659f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:57.928179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-85ed6684-b101-484c-a60f-015949a7659f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60ba47cefe86402abcf32330758da25f', 'request_id': 'req-85ed6684-b101-484c-a60f-015949a7659f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:57.945720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-85ed6684-b101-484c-a60f-015949a7659f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60ba47cefe86402abcf32330758da25f', 'request_id': 'req-85ed6684-b101-484c-a60f-015949a7659f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:57.955409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-85ed6684-b101-484c-a60f-015949a7659f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60ba47cefe86402abcf32330758da25f', 'request_id': 'req-85ed6684-b101-484c-a60f-015949a7659f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:57.990000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-85ed6684-b101-484c-a60f-015949a7659f admin admin] The token cache is being invalidate because project 60ba47cefe86402abcf32330758da25f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:03:57.990312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-85ed6684-b101-484c-a60f-015949a7659f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:57.992911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-85ed6684-b101-484c-a60f-015949a7659f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:03:58.001688 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2147/4363] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:03:57 2022] DELETE /identity/v3/projects/60ba47cefe86402abcf32330758da25f => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:03:58.006704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d7d8c-1a31-4a44-8a90-f52209864ff7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:58.010554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d7d8c-1a31-4a44-8a90-f52209864ff7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:58.011116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d7d8c-1a31-4a44-8a90-f52209864ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:58.088448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8f2d7d8c-1a31-4a44-8a90-f52209864ff7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:58.090512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8f2d7d8c-1a31-4a44-8a90-f52209864ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:58.200080 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2217/4364] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:58.210703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340a363-c7b5-4fa9-a0e2-e3983a0a0678 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:58.211271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340a363-c7b5-4fa9-a0e2-e3983a0a0678 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:58.212100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340a363-c7b5-4fa9-a0e2-e3983a0a0678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:58.251887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9340a363-c7b5-4fa9-a0e2-e3983a0a0678 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:58.253242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9340a363-c7b5-4fa9-a0e2-e3983a0a0678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:58.264308 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2148/4365] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:03:58.287772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d7d8c-1a31-4a44-8a90-f52209864ff7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:58.306021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d7d8c-1a31-4a44-8a90-f52209864ff7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:58.315417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a4b80-935c-4de7-9e93-075d6d60e50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:58.316262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a4b80-935c-4de7-9e93-075d6d60e50b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:58.316631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a4b80-935c-4de7-9e93-075d6d60e50b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:58.318484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a4b80-935c-4de7-9e93-075d6d60e50b admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:58.319561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a4b80-935c-4de7-9e93-075d6d60e50b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:03:58.319952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a4b80-935c-4de7-9e93-075d6d60e50b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:58.322679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a4b80-935c-4de7-9e93-075d6d60e50b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:58.379885 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2218/4366] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:03:58.384434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340a363-c7b5-4fa9-a0e2-e3983a0a0678 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:58.397423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340a363-c7b5-4fa9-a0e2-e3983a0a0678 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:58.405196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f6254-bcf2-4d05-ba63-2d2997e45363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:58.406131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f6254-bcf2-4d05-ba63-2d2997e45363 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:58.406769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f6254-bcf2-4d05-ba63-2d2997e45363 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:58.407057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f6254-bcf2-4d05-ba63-2d2997e45363 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:58.407725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f6254-bcf2-4d05-ba63-2d2997e45363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:58.409928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f6254-bcf2-4d05-ba63-2d2997e45363 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:58.483328 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2149/4367] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:03:58 2022] POST /identity/v3/projects => generated 389 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:58.484528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a4b80-935c-4de7-9e93-075d6d60e50b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:58.514908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a4b80-935c-4de7-9e93-075d6d60e50b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:58.578928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:58.661477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5efc5-8957-4b92-b4eb-37b5ac08f277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:58.662984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5efc5-8957-4b92-b4eb-37b5ac08f277 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:58.663363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5efc5-8957-4b92-b4eb-37b5ac08f277 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:58.663752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5efc5-8957-4b92-b4eb-37b5ac08f277 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:58.664514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5efc5-8957-4b92-b4eb-37b5ac08f277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:58.669316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5efc5-8957-4b92-b4eb-37b5ac08f277 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:58.714440 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2219/4368] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:03:58 2022] POST /identity/v3/users => generated 295 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:03:58.718097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f6254-bcf2-4d05-ba63-2d2997e45363 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:58.734923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f6254-bcf2-4d05-ba63-2d2997e45363 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:58.750092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983e71f-0b18-4308-b4db-09bf12405f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:58.751002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983e71f-0b18-4308-b4db-09bf12405f8f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:58.751300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983e71f-0b18-4308-b4db-09bf12405f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:58.751735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983e71f-0b18-4308-b4db-09bf12405f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:58.752823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983e71f-0b18-4308-b4db-09bf12405f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:58.754545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983e71f-0b18-4308-b4db-09bf12405f8f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:58.781443 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2150/4369] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:58.782291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5efc5-8957-4b92-b4eb-37b5ac08f277 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:58.794128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5efc5-8957-4b92-b4eb-37b5ac08f277 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:58.805270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4f1f6-46ef-4228-994b-1097b18bfae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:58.805897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4f1f6-46ef-4228-994b-1097b18bfae6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:58.806049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4f1f6-46ef-4228-994b-1097b18bfae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:58.806219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4f1f6-46ef-4228-994b-1097b18bfae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:58.808308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4f1f6-46ef-4228-994b-1097b18bfae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:58.813338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4f1f6-46ef-4228-994b-1097b18bfae6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:58.843079 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2220/4370] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:03:58.847885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983e71f-0b18-4308-b4db-09bf12405f8f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:58.871044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983e71f-0b18-4308-b4db-09bf12405f8f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:58.880099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc59f2-df8d-418c-80e8-1cf9f06dad36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:58.881094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc59f2-df8d-418c-80e8-1cf9f06dad36 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:58.881726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc59f2-df8d-418c-80e8-1cf9f06dad36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:58.882188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc59f2-df8d-418c-80e8-1cf9f06dad36 admin admin] PATH_INFO: `/v3/projects/484597f6a0554dd2af819e55e8e7b7d2/users/c897f757440d479790743e6a8c1dfd91/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:58.932484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc59f2-df8d-418c-80e8-1cf9f06dad36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=484597f6a0554dd2af819e55e8e7b7d2, user_id=c897f757440d479790743e6a8c1dfd91, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:58.940221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc59f2-df8d-418c-80e8-1cf9f06dad36 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:58.973379 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2151/4371] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:03:58 2022] PUT /identity/v3/projects/484597f6a0554dd2af819e55e8e7b7d2/users/c897f757440d479790743e6a8c1dfd91/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:03:58.976372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d58513-d6cc-4cf4-bd06-2c57c1d2a2d0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:58.976372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d58513-d6cc-4cf4-bd06-2c57c1d2a2d0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:58.976775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d58513-d6cc-4cf4-bd06-2c57c1d2a2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:59.003525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-26d58513-d6cc-4cf4-bd06-2c57c1d2a2d0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:03:59.005805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-26d58513-d6cc-4cf4-bd06-2c57c1d2a2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:59.102970 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2221/4372] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:03:59.176245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc59f2-df8d-418c-80e8-1cf9f06dad36 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:03:59.179448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c5dc59f2-df8d-418c-80e8-1cf9f06dad36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:03:59.239155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:03:59.269358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc59f2-df8d-418c-80e8-1cf9f06dad36 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:03:59.274760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509343f0-beb5-4757-a700-212c8782775b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:03:59.275023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509343f0-beb5-4757-a700-212c8782775b service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:03:59.275307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509343f0-beb5-4757-a700-212c8782775b service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:03:59.275732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509343f0-beb5-4757-a700-212c8782775b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:03:59.296765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509343f0-beb5-4757-a700-212c8782775b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:03:59.300729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509343f0-beb5-4757-a700-212c8782775b service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:03:59.324382 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2152/4373] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:01.900051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d58513-d6cc-4cf4-bd06-2c57c1d2a2d0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:01.901685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-26d58513-d6cc-4cf4-bd06-2c57c1d2a2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:01.923763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d58513-d6cc-4cf4-bd06-2c57c1d2a2d0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:01.932939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:01.934238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:01.934718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:01.935167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] PATH_INFO: `/v3/users/17b75a15889c4dd38315591ae8fc1b6a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:01.937417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=17b75a15889c4dd38315591ae8fc1b6a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:01.943299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:02.070786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17b75a15889c4dd38315591ae8fc1b6a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:02.110883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17b75a15889c4dd38315591ae8fc1b6a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:02.140382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17b75a15889c4dd38315591ae8fc1b6a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:02.141736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17b75a15889c4dd38315591ae8fc1b6a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:02.142365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17b75a15889c4dd38315591ae8fc1b6a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:02.174951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a2a24-7868-4ecd-9b4e-409acee3ea52 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:02.175512 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2222/4374] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:04:01 2022] DELETE /identity/v3/users/17b75a15889c4dd38315591ae8fc1b6a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:02.176089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a2a24-7868-4ecd-9b4e-409acee3ea52 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:02.176385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a2a24-7868-4ecd-9b4e-409acee3ea52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:02.209019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d20a2a24-7868-4ecd-9b4e-409acee3ea52 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:02.211208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d20a2a24-7868-4ecd-9b4e-409acee3ea52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:02.329898 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2153/4375] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:02.355416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:02.358083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:02.387831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:02.499849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcc80e-7ba0-405e-afe0-fbd91f367b6f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:02.503940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2070b0-88ef-498c-8df4-069313a9b879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:02.504654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2070b0-88ef-498c-8df4-069313a9b879 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:02.504845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2070b0-88ef-498c-8df4-069313a9b879 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:02.505225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2070b0-88ef-498c-8df4-069313a9b879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:02.517275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2070b0-88ef-498c-8df4-069313a9b879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:02.518852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2070b0-88ef-498c-8df4-069313a9b879 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:02.539177 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2223/4376] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:03.861363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a2a24-7868-4ecd-9b4e-409acee3ea52 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:03.863359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d20a2a24-7868-4ecd-9b4e-409acee3ea52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:03.890143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a2a24-7868-4ecd-9b4e-409acee3ea52 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:03.905833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32743e59-b265-4fac-94a0-1b8ef68d87af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:03.906496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32743e59-b265-4fac-94a0-1b8ef68d87af admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:03.907596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32743e59-b265-4fac-94a0-1b8ef68d87af admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:03.907596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32743e59-b265-4fac-94a0-1b8ef68d87af admin admin] PATH_INFO: `/v3/projects/1cb77ea78cc046f7aa8bf5d5f6cf7b2f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:03.908761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32743e59-b265-4fac-94a0-1b8ef68d87af admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cb77ea78cc046f7aa8bf5d5f6cf7b2f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:03.911188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32743e59-b265-4fac-94a0-1b8ef68d87af admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:04.018012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-32743e59-b265-4fac-94a0-1b8ef68d87af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cb77ea78cc046f7aa8bf5d5f6cf7b2f', 'request_id': 'req-32743e59-b265-4fac-94a0-1b8ef68d87af'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:04.029686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-32743e59-b265-4fac-94a0-1b8ef68d87af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cb77ea78cc046f7aa8bf5d5f6cf7b2f', 'request_id': 'req-32743e59-b265-4fac-94a0-1b8ef68d87af'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:04.039353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-32743e59-b265-4fac-94a0-1b8ef68d87af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cb77ea78cc046f7aa8bf5d5f6cf7b2f', 'request_id': 'req-32743e59-b265-4fac-94a0-1b8ef68d87af'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:04.054931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-32743e59-b265-4fac-94a0-1b8ef68d87af admin admin] The token cache is being invalidate because project 1cb77ea78cc046f7aa8bf5d5f6cf7b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:04:04.055398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-32743e59-b265-4fac-94a0-1b8ef68d87af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:04.057048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-32743e59-b265-4fac-94a0-1b8ef68d87af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:04.061875 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2154/4377] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:04:03 2022] DELETE /identity/v3/projects/1cb77ea78cc046f7aa8bf5d5f6cf7b2f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:04.064090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394b06-d4d6-4fbf-abd0-162dbe9a8199 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:04.064736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394b06-d4d6-4fbf-abd0-162dbe9a8199 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:04.064736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394b06-d4d6-4fbf-abd0-162dbe9a8199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:04.098125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1d394b06-d4d6-4fbf-abd0-162dbe9a8199 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:04.099421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1d394b06-d4d6-4fbf-abd0-162dbe9a8199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:04.222635 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2224/4378] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:04.230833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ffe6c-e3e6-431e-8eca-c0bdf973453a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:04.231229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ffe6c-e3e6-431e-8eca-c0bdf973453a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:04.231613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ffe6c-e3e6-431e-8eca-c0bdf973453a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:04.270251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-982ffe6c-e3e6-431e-8eca-c0bdf973453a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:04.271568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-982ffe6c-e3e6-431e-8eca-c0bdf973453a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:04.282757 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2155/4379] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:04:04.288041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394b06-d4d6-4fbf-abd0-162dbe9a8199 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:04.300108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394b06-d4d6-4fbf-abd0-162dbe9a8199 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:04.327401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3f0fb-f4ef-4817-8da3-6d037340f14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:04.328051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3f0fb-f4ef-4817-8da3-6d037340f14c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:04.333064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3f0fb-f4ef-4817-8da3-6d037340f14c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:04.333439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3f0fb-f4ef-4817-8da3-6d037340f14c admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:04.334688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3f0fb-f4ef-4817-8da3-6d037340f14c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:04:04.335016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3f0fb-f4ef-4817-8da3-6d037340f14c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:04.339627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3f0fb-f4ef-4817-8da3-6d037340f14c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:04.367524 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2225/4380] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:04:04.372532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ffe6c-e3e6-431e-8eca-c0bdf973453a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:04.384759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ffe6c-e3e6-431e-8eca-c0bdf973453a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:04.389970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969508e-85eb-4d04-b07c-6fcf42fce2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:04.390914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969508e-85eb-4d04-b07c-6fcf42fce2bd admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:04.391308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969508e-85eb-4d04-b07c-6fcf42fce2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:04.391692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969508e-85eb-4d04-b07c-6fcf42fce2bd admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:04.392445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969508e-85eb-4d04-b07c-6fcf42fce2bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:04.394715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969508e-85eb-4d04-b07c-6fcf42fce2bd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:04.436844 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2156/4381] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:04:04 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:04:04.441179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3f0fb-f4ef-4817-8da3-6d037340f14c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:04.454676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3f0fb-f4ef-4817-8da3-6d037340f14c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:04.567587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dab28d-f284-4595-8342-8b887513d753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:04.568234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dab28d-f284-4595-8342-8b887513d753 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:04.568527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dab28d-f284-4595-8342-8b887513d753 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:04.568810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dab28d-f284-4595-8342-8b887513d753 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:04.569690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dab28d-f284-4595-8342-8b887513d753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:04.572537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dab28d-f284-4595-8342-8b887513d753 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:04.612075 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2226/4382] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:04:04 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:04:04.616416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969508e-85eb-4d04-b07c-6fcf42fce2bd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:04.648382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969508e-85eb-4d04-b07c-6fcf42fce2bd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:04.659271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1972d0-de1f-47cf-9e59-91c347dbbbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:04.661672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1972d0-de1f-47cf-9e59-91c347dbbbca admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:04.662141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1972d0-de1f-47cf-9e59-91c347dbbbca admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:04.663015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1972d0-de1f-47cf-9e59-91c347dbbbca admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:04.665478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1972d0-de1f-47cf-9e59-91c347dbbbca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:04.690474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1972d0-de1f-47cf-9e59-91c347dbbbca admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:04.729917 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2157/4383] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:04.730888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dab28d-f284-4595-8342-8b887513d753 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:04.748876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dab28d-f284-4595-8342-8b887513d753 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:04.764574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59263279-7b2f-422f-9c3c-d2b4014e6533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:04.765520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59263279-7b2f-422f-9c3c-d2b4014e6533 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:04.766153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59263279-7b2f-422f-9c3c-d2b4014e6533 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:04.766643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59263279-7b2f-422f-9c3c-d2b4014e6533 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:04.767511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59263279-7b2f-422f-9c3c-d2b4014e6533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:04.769399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59263279-7b2f-422f-9c3c-d2b4014e6533 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:04.787293 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2227/4384] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:04.791545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1972d0-de1f-47cf-9e59-91c347dbbbca admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:04.803595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1972d0-de1f-47cf-9e59-91c347dbbbca admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:04.810754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7c2bf-c639-4720-b488-8c49d6f2ba10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:04.811271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7c2bf-c639-4720-b488-8c49d6f2ba10 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:04.811963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7c2bf-c639-4720-b488-8c49d6f2ba10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:04.811963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7c2bf-c639-4720-b488-8c49d6f2ba10 admin admin] PATH_INFO: `/v3/projects/8da3c32d1cc843458f7744cc038f8be1/users/c53c59095759426887a885de3c7f02e6/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:04.829983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7c2bf-c639-4720-b488-8c49d6f2ba10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da3c32d1cc843458f7744cc038f8be1, user_id=c53c59095759426887a885de3c7f02e6, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:04.832598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7c2bf-c639-4720-b488-8c49d6f2ba10 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:04.873953 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2158/4385] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:04:04 2022] PUT /identity/v3/projects/8da3c32d1cc843458f7744cc038f8be1/users/c53c59095759426887a885de3c7f02e6/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:04.875342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea665b06-4267-493c-98e5-75e140987fa2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:04.875651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea665b06-4267-493c-98e5-75e140987fa2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:04.876115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea665b06-4267-493c-98e5-75e140987fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:04.921273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ea665b06-4267-493c-98e5-75e140987fa2 None None] MFA Rules not processed for user `c53c59095759426887a885de3c7f02e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:04.922519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ea665b06-4267-493c-98e5-75e140987fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:04.942485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:05.057418 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2228/4386] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:05.062938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7c2bf-c639-4720-b488-8c49d6f2ba10 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:05.065435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-eca7c2bf-c639-4720-b488-8c49d6f2ba10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:05.073905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:05.171164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10445fb6-1636-442e-b4fe-db84c4ec3ac5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:05.171667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10445fb6-1636-442e-b4fe-db84c4ec3ac5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:05.171777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10445fb6-1636-442e-b4fe-db84c4ec3ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:05.198591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7c2bf-c639-4720-b488-8c49d6f2ba10 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:05.203753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f1c78-8af8-4929-a54a-a3aada54e819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:05.204700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f1c78-8af8-4929-a54a-a3aada54e819 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:05.205217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f1c78-8af8-4929-a54a-a3aada54e819 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:05.205729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f1c78-8af8-4929-a54a-a3aada54e819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:05.223192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f1c78-8af8-4929-a54a-a3aada54e819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:05.225006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f1c78-8af8-4929-a54a-a3aada54e819 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:05.225145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-10445fb6-1636-442e-b4fe-db84c4ec3ac5 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:05.226358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-10445fb6-1636-442e-b4fe-db84c4ec3ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:05.239216 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2159/4387] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:05.339056 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2229/4388] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:04:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:04:05.351662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f1c78-8af8-4929-a54a-a3aada54e819 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:05.354306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-052f1c78-8af8-4929-a54a-a3aada54e819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:05.398480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f1c78-8af8-4929-a54a-a3aada54e819 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:05.423584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360a8d5-f552-47c7-bee3-08aa1392dc67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:05.424219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360a8d5-f552-47c7-bee3-08aa1392dc67 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:05.424527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360a8d5-f552-47c7-bee3-08aa1392dc67 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:05.425344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360a8d5-f552-47c7-bee3-08aa1392dc67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:05.426764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10445fb6-1636-442e-b4fe-db84c4ec3ac5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:05.428082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4360a8d5-f552-47c7-bee3-08aa1392dc67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:05.429694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-10445fb6-1636-442e-b4fe-db84c4ec3ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:05.455650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360a8d5-f552-47c7-bee3-08aa1392dc67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:05.458032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360a8d5-f552-47c7-bee3-08aa1392dc67 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:05.482684 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2160/4389] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:05.621075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10445fb6-1636-442e-b4fe-db84c4ec3ac5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:05.630526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a0176-95f6-410c-886d-53553fd3841f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:05.633059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a0176-95f6-410c-886d-53553fd3841f service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:05.633435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a0176-95f6-410c-886d-53553fd3841f service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:05.634321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a0176-95f6-410c-886d-53553fd3841f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:05.651415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a0176-95f6-410c-886d-53553fd3841f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:05.653455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a0176-95f6-410c-886d-53553fd3841f service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:05.663829 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2230/4390] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:04:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:04:10.778925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4094ee9-114b-46d2-b8f3-273d2c14c67e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:10.779630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4094ee9-114b-46d2-b8f3-273d2c14c67e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:10.780148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4094ee9-114b-46d2-b8f3-273d2c14c67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:10.859820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a4094ee9-114b-46d2-b8f3-273d2c14c67e None None] MFA Rules not processed for user `c897f757440d479790743e6a8c1dfd91`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:10.861758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a4094ee9-114b-46d2-b8f3-273d2c14c67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:10.991438 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2161/4391] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:10.995848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729366ad-0d7a-456f-881f-d48bde9eddbc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:10.996370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729366ad-0d7a-456f-881f-d48bde9eddbc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:10.996485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729366ad-0d7a-456f-881f-d48bde9eddbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:11.018129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-729366ad-0d7a-456f-881f-d48bde9eddbc None None] MFA Rules not processed for user `c897f757440d479790743e6a8c1dfd91`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:11.019628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-729366ad-0d7a-456f-881f-d48bde9eddbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:11.031647 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2231/4392] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:11.056783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4094ee9-114b-46d2-b8f3-273d2c14c67e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:11.059896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a4094ee9-114b-46d2-b8f3-273d2c14c67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:11.252097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4094ee9-114b-46d2-b8f3-273d2c14c67e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:11.259383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd3584-4410-42c2-ba55-22ca22177e2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:11.260073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd3584-4410-42c2-ba55-22ca22177e2c service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:11.260368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd3584-4410-42c2-ba55-22ca22177e2c service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:11.260649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd3584-4410-42c2-ba55-22ca22177e2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:11.281616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd3584-4410-42c2-ba55-22ca22177e2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:11.287027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd3584-4410-42c2-ba55-22ca22177e2c service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:11.312194 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2162/4393] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:16.763333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729366ad-0d7a-456f-881f-d48bde9eddbc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:16.765405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-729366ad-0d7a-456f-881f-d48bde9eddbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:16.844724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729366ad-0d7a-456f-881f-d48bde9eddbc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:16.893196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36c51e-1fae-4f5b-b35d-7887e0fe257e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:16.894110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36c51e-1fae-4f5b-b35d-7887e0fe257e service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:16.894594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36c51e-1fae-4f5b-b35d-7887e0fe257e service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:16.895044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36c51e-1fae-4f5b-b35d-7887e0fe257e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:16.898101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8c36c51e-1fae-4f5b-b35d-7887e0fe257e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:16.946771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36c51e-1fae-4f5b-b35d-7887e0fe257e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:16.951139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36c51e-1fae-4f5b-b35d-7887e0fe257e service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:16.983067 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2232/4394] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:21.298413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:22.609218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e85fd-ca5a-4ddf-b7f6-1b1954929a26 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:22.609828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e85fd-ca5a-4ddf-b7f6-1b1954929a26 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:22.610109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e85fd-ca5a-4ddf-b7f6-1b1954929a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:22.638059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d06e85fd-ca5a-4ddf-b7f6-1b1954929a26 None None] MFA Rules not processed for user `1b2dd96d938b41d4ba40007aa072288b`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:22.639417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d06e85fd-ca5a-4ddf-b7f6-1b1954929a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:22.718439 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2163/4395] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:24.229255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36c51e-1fae-4f5b-b35d-7887e0fe257e service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:24.231735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8c36c51e-1fae-4f5b-b35d-7887e0fe257e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:24.269319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:24.345201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36c51e-1fae-4f5b-b35d-7887e0fe257e service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:24.351599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad5073-6a20-41c6-b8d6-1895a3803c2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:24.353113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad5073-6a20-41c6-b8d6-1895a3803c2e service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:24.353571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad5073-6a20-41c6-b8d6-1895a3803c2e service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:24.354179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad5073-6a20-41c6-b8d6-1895a3803c2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:24.356520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9bad5073-6a20-41c6-b8d6-1895a3803c2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:24.375422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad5073-6a20-41c6-b8d6-1895a3803c2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:24.377571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad5073-6a20-41c6-b8d6-1895a3803c2e service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:24.394454 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2233/4396] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:27.059178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e85fd-ca5a-4ddf-b7f6-1b1954929a26 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:27.062201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d06e85fd-ca5a-4ddf-b7f6-1b1954929a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:27.188643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e85fd-ca5a-4ddf-b7f6-1b1954929a26 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:27.194606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:27.195870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:27.196352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:27.196810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] PATH_INFO: `/v3/users/9884fe87dedd42bba0442aca48ebdc8d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:27.198698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9884fe87dedd42bba0442aca48ebdc8d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:27.201177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:27.278974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9884fe87dedd42bba0442aca48ebdc8d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:27.294534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9884fe87dedd42bba0442aca48ebdc8d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:27.295549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9884fe87dedd42bba0442aca48ebdc8d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:27.304740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9884fe87dedd42bba0442aca48ebdc8d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:27.316124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9884fe87dedd42bba0442aca48ebdc8d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:27.331179 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2164/4397] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:04:27 2022] DELETE /identity/v3/users/9884fe87dedd42bba0442aca48ebdc8d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:28.347560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad5073-6a20-41c6-b8d6-1895a3803c2e service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:28.371433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad5073-6a20-41c6-b8d6-1895a3803c2e service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:28.452637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:28.453466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:28.453881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:28.454252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] PATH_INFO: `/v3/projects/35119286edb94a8fb711a37863321fbe` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:28.455476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35119286edb94a8fb711a37863321fbe)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:28.457656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:28.518156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35119286edb94a8fb711a37863321fbe', 'request_id': 'req-d9568192-2ff6-4170-b603-ec7c93239e26'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:28.526892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35119286edb94a8fb711a37863321fbe', 'request_id': 'req-d9568192-2ff6-4170-b603-ec7c93239e26'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:28.534181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35119286edb94a8fb711a37863321fbe', 'request_id': 'req-d9568192-2ff6-4170-b603-ec7c93239e26'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:28.547923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] The token cache is being invalidate because project 35119286edb94a8fb711a37863321fbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:04:28.548280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:28.553398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:28.555392 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2234/4398] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:04:28 2022] DELETE /identity/v3/projects/35119286edb94a8fb711a37863321fbe => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:28.559472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:28.561174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:28.577035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff123ca7-e575-4026-b774-7519fa5fb8b8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:28.589682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:28.670047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e57dee-82a9-46bd-b6f2-01a467e269b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:28.672063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e57dee-82a9-46bd-b6f2-01a467e269b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:28.672601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e57dee-82a9-46bd-b6f2-01a467e269b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:28.673578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e57dee-82a9-46bd-b6f2-01a467e269b7 admin admin] PATH_INFO: `/v3/users/5d1dda56f7cb4adda980038b707d34de` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:28.675428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e57dee-82a9-46bd-b6f2-01a467e269b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1dda56f7cb4adda980038b707d34de)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:28.678237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e57dee-82a9-46bd-b6f2-01a467e269b7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:28.748519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c2e57dee-82a9-46bd-b6f2-01a467e269b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1dda56f7cb4adda980038b707d34de'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:28.771777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c2e57dee-82a9-46bd-b6f2-01a467e269b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1dda56f7cb4adda980038b707d34de'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:28.772565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c2e57dee-82a9-46bd-b6f2-01a467e269b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1dda56f7cb4adda980038b707d34de'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:28.783851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c2e57dee-82a9-46bd-b6f2-01a467e269b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1dda56f7cb4adda980038b707d34de'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:28.792562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c2e57dee-82a9-46bd-b6f2-01a467e269b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1dda56f7cb4adda980038b707d34de'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:28.800866 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2165/4399] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:04:28 2022] DELETE /identity/v3/users/5d1dda56f7cb4adda980038b707d34de => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:29.401146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:29.402972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:29.422972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9568192-2ff6-4170-b603-ec7c93239e26 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:29.457747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:29.494062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b528cfb-3f76-4f80-b726-46ede41a7e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:29.494711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b528cfb-3f76-4f80-b726-46ede41a7e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:29.495002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b528cfb-3f76-4f80-b726-46ede41a7e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:29.495283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b528cfb-3f76-4f80-b726-46ede41a7e96 admin admin] PATH_INFO: `/v3/projects/0303211069184f75b9e8c6d075999295` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:29.496544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b528cfb-3f76-4f80-b726-46ede41a7e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0303211069184f75b9e8c6d075999295)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:29.498567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b528cfb-3f76-4f80-b726-46ede41a7e96 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:29.551221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6b528cfb-3f76-4f80-b726-46ede41a7e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0303211069184f75b9e8c6d075999295', 'request_id': 'req-6b528cfb-3f76-4f80-b726-46ede41a7e96'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:29.560280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6b528cfb-3f76-4f80-b726-46ede41a7e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0303211069184f75b9e8c6d075999295', 'request_id': 'req-6b528cfb-3f76-4f80-b726-46ede41a7e96'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:29.572606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6b528cfb-3f76-4f80-b726-46ede41a7e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0303211069184f75b9e8c6d075999295', 'request_id': 'req-6b528cfb-3f76-4f80-b726-46ede41a7e96'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:29.587741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6b528cfb-3f76-4f80-b726-46ede41a7e96 admin admin] The token cache is being invalidate because project 0303211069184f75b9e8c6d075999295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:04:29.588329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6b528cfb-3f76-4f80-b726-46ede41a7e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:29.589652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6b528cfb-3f76-4f80-b726-46ede41a7e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:29.591540 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2235/4400] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:04:29 2022] DELETE /identity/v3/projects/0303211069184f75b9e8c6d075999295 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:29.601641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68c87c-4ca7-419a-bf72-7615c7ca4aee None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:29.602576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68c87c-4ca7-419a-bf72-7615c7ca4aee None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:29.603018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68c87c-4ca7-419a-bf72-7615c7ca4aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:29.623850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bb68c87c-4ca7-419a-bf72-7615c7ca4aee None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:29.625435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bb68c87c-4ca7-419a-bf72-7615c7ca4aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:29.732030 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2166/4401] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:29.740776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208cb3-0191-4ece-bb46-403b48dae45d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:29.740776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208cb3-0191-4ece-bb46-403b48dae45d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:29.740776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208cb3-0191-4ece-bb46-403b48dae45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:29.831369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6d208cb3-0191-4ece-bb46-403b48dae45d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:29.833086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6d208cb3-0191-4ece-bb46-403b48dae45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:29.847163 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2236/4402] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:29.852543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68c87c-4ca7-419a-bf72-7615c7ca4aee None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:29.878676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68c87c-4ca7-419a-bf72-7615c7ca4aee None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:29.892118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646de629-293c-4b9e-bb3b-4ee5440a0713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:29.893338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646de629-293c-4b9e-bb3b-4ee5440a0713 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:29.893823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646de629-293c-4b9e-bb3b-4ee5440a0713 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:29.894301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646de629-293c-4b9e-bb3b-4ee5440a0713 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:29.895181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646de629-293c-4b9e-bb3b-4ee5440a0713 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:04:29.895675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646de629-293c-4b9e-bb3b-4ee5440a0713 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:29.898884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646de629-293c-4b9e-bb3b-4ee5440a0713 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:29.914234 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2167/4403] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:04:29.918368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208cb3-0191-4ece-bb46-403b48dae45d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:29.933377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208cb3-0191-4ece-bb46-403b48dae45d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:29.952315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d7152-2768-4a91-9635-41be23c1a57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:29.953381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d7152-2768-4a91-9635-41be23c1a57b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:29.953799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d7152-2768-4a91-9635-41be23c1a57b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:29.954293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d7152-2768-4a91-9635-41be23c1a57b admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:29.955498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d7152-2768-4a91-9635-41be23c1a57b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:29.963895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d7152-2768-4a91-9635-41be23c1a57b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:30.006088 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2237/4404] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:04:29 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:04:30.010951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646de629-293c-4b9e-bb3b-4ee5440a0713 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:30.030201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646de629-293c-4b9e-bb3b-4ee5440a0713 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:30.064719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:30.138941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be96f5a-9b46-4ed4-bfae-d8f313aa7713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:30.139583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96f5a-9b46-4ed4-bfae-d8f313aa7713 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:30.139893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96f5a-9b46-4ed4-bfae-d8f313aa7713 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:30.140179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96f5a-9b46-4ed4-bfae-d8f313aa7713 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:30.140928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be96f5a-9b46-4ed4-bfae-d8f313aa7713 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:30.143216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be96f5a-9b46-4ed4-bfae-d8f313aa7713 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:30.166391 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2168/4405] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:04:30 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:04:30.170444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d7152-2768-4a91-9635-41be23c1a57b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:30.183401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d7152-2768-4a91-9635-41be23c1a57b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:30.190685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126672ad-15e6-4b7d-94e5-5c1abb663cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:30.191310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126672ad-15e6-4b7d-94e5-5c1abb663cac admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:30.191605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126672ad-15e6-4b7d-94e5-5c1abb663cac admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:30.191886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126672ad-15e6-4b7d-94e5-5c1abb663cac admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:30.192540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126672ad-15e6-4b7d-94e5-5c1abb663cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:30.194196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126672ad-15e6-4b7d-94e5-5c1abb663cac admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:30.211075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be96f5a-9b46-4ed4-bfae-d8f313aa7713 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:30.218029 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2238/4406] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:30.222214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be96f5a-9b46-4ed4-bfae-d8f313aa7713 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:30.229054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e6dea-73c2-4d59-99ab-d26df84782c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:30.229705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e6dea-73c2-4d59-99ab-d26df84782c0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:30.229990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e6dea-73c2-4d59-99ab-d26df84782c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:30.230278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e6dea-73c2-4d59-99ab-d26df84782c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:30.231101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e6dea-73c2-4d59-99ab-d26df84782c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:30.232758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e6dea-73c2-4d59-99ab-d26df84782c0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:30.251669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126672ad-15e6-4b7d-94e5-5c1abb663cac admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:30.252026 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2169/4407] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:30.263209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126672ad-15e6-4b7d-94e5-5c1abb663cac admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:30.269178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3a9e2-2170-4d39-a502-9f3272ad0436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:30.269996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3a9e2-2170-4d39-a502-9f3272ad0436 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:30.270355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3a9e2-2170-4d39-a502-9f3272ad0436 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:30.270706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3a9e2-2170-4d39-a502-9f3272ad0436 admin admin] PATH_INFO: `/v3/projects/536376886c2c4f3080f06afffcbb9563/users/d849e4bde6d747ccba99fd5ac15edd17/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:30.304689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3a9e2-2170-4d39-a502-9f3272ad0436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=536376886c2c4f3080f06afffcbb9563, user_id=d849e4bde6d747ccba99fd5ac15edd17, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:30.308785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3a9e2-2170-4d39-a502-9f3272ad0436 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:30.350892 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2239/4408] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:04:30 2022] PUT /identity/v3/projects/536376886c2c4f3080f06afffcbb9563/users/d849e4bde6d747ccba99fd5ac15edd17/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:30.354250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537253d-f3a7-418d-a298-54e869dada1e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:30.354539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537253d-f3a7-418d-a298-54e869dada1e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:30.354816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537253d-f3a7-418d-a298-54e869dada1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:30.433074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f537253d-f3a7-418d-a298-54e869dada1e None None] MFA Rules not processed for user `d849e4bde6d747ccba99fd5ac15edd17`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:30.434312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f537253d-f3a7-418d-a298-54e869dada1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:30.549052 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2170/4409] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:30.559663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3a9e2-2170-4d39-a502-9f3272ad0436 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:30.561486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-71b3a9e2-2170-4d39-a502-9f3272ad0436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:30.675893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:30.704502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3a9e2-2170-4d39-a502-9f3272ad0436 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:30.712297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8a2e3-022b-49ab-8277-e38de9cffd05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:30.713243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8a2e3-022b-49ab-8277-e38de9cffd05 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:30.713598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8a2e3-022b-49ab-8277-e38de9cffd05 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:30.713948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8a2e3-022b-49ab-8277-e38de9cffd05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:30.723463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8a2e3-022b-49ab-8277-e38de9cffd05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:30.725150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8a2e3-022b-49ab-8277-e38de9cffd05 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:30.736094 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2240/4410] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:31.819706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537253d-f3a7-418d-a298-54e869dada1e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:31.945058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537253d-f3a7-418d-a298-54e869dada1e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:31.956530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:31.960404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:31.962492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:31.962953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] PATH_INFO: `/v3/users/d849e4bde6d747ccba99fd5ac15edd17` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:31.964519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d849e4bde6d747ccba99fd5ac15edd17)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:31.967945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:32.046410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:32.096011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd849e4bde6d747ccba99fd5ac15edd17'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:32.165487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd849e4bde6d747ccba99fd5ac15edd17'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:32.166340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd849e4bde6d747ccba99fd5ac15edd17'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:32.188343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd849e4bde6d747ccba99fd5ac15edd17'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:32.203091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd849e4bde6d747ccba99fd5ac15edd17'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:32.215872 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2171/4411] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:04:31 2022] DELETE /identity/v3/users/d849e4bde6d747ccba99fd5ac15edd17 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:32.217374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2cd4b-a944-4f19-b59f-a32b6475ff54 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:32.217855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2cd4b-a944-4f19-b59f-a32b6475ff54 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:32.217958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2cd4b-a944-4f19-b59f-a32b6475ff54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:32.247929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3fa2cd4b-a944-4f19-b59f-a32b6475ff54 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:32.249230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3fa2cd4b-a944-4f19-b59f-a32b6475ff54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:32.370669 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2241/4412] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:04:32.387242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:32.389353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:32.468979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b17932-af41-43d9-8986-40cae21d1e8c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:32.475120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7c800-476a-4909-88b7-57037db45e0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:32.475870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7c800-476a-4909-88b7-57037db45e0e service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:32.476301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7c800-476a-4909-88b7-57037db45e0e service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:32.477491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7c800-476a-4909-88b7-57037db45e0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:32.488265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7c800-476a-4909-88b7-57037db45e0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:32.489746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7c800-476a-4909-88b7-57037db45e0e service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:32.501155 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2172/4413] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:32.956556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2cd4b-a944-4f19-b59f-a32b6475ff54 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:32.960804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3fa2cd4b-a944-4f19-b59f-a32b6475ff54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:32.974945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2cd4b-a944-4f19-b59f-a32b6475ff54 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:32.981620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:32.982421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:32.982795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:32.983304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8 admin admin] PATH_INFO: `/v3/projects/536376886c2c4f3080f06afffcbb9563` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:32.984720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=536376886c2c4f3080f06afffcbb9563)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:32.987072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:33.053151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '536376886c2c4f3080f06afffcbb9563', 'request_id': 'req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:33.068273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '536376886c2c4f3080f06afffcbb9563', 'request_id': 'req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:33.076220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '536376886c2c4f3080f06afffcbb9563', 'request_id': 'req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:33.098599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8 admin admin] The token cache is being invalidate because project 536376886c2c4f3080f06afffcbb9563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:04:33.098977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:33.099920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8163e07-9fd7-4f73-bfdd-3679acd7cfc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:33.102586 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2242/4414] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:04:32 2022] DELETE /identity/v3/projects/536376886c2c4f3080f06afffcbb9563 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:33.107629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed757e-9517-4a29-bd75-d0033a383937 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:33.108551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed757e-9517-4a29-bd75-d0033a383937 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:33.109669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed757e-9517-4a29-bd75-d0033a383937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:33.134894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-17ed757e-9517-4a29-bd75-d0033a383937 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:33.136754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-17ed757e-9517-4a29-bd75-d0033a383937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:33.215895 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2173/4415] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:33.224004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f71c0e-8882-4132-b120-4c4dc052ff93 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:33.225019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f71c0e-8882-4132-b120-4c4dc052ff93 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:33.225303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f71c0e-8882-4132-b120-4c4dc052ff93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:33.253433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-35f71c0e-8882-4132-b120-4c4dc052ff93 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:33.255923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-35f71c0e-8882-4132-b120-4c4dc052ff93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:33.269756 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2243/4416] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:33.275604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed757e-9517-4a29-bd75-d0033a383937 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:33.289957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed757e-9517-4a29-bd75-d0033a383937 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:33.296274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198851d-2ace-45c2-9453-3ab718788004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:33.297502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198851d-2ace-45c2-9453-3ab718788004 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:33.297924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198851d-2ace-45c2-9453-3ab718788004 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:33.298240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198851d-2ace-45c2-9453-3ab718788004 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:33.299529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198851d-2ace-45c2-9453-3ab718788004 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:04:33.299850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198851d-2ace-45c2-9453-3ab718788004 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:33.302058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198851d-2ace-45c2-9453-3ab718788004 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:33.324739 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2174/4417] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:04:33.328053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f71c0e-8882-4132-b120-4c4dc052ff93 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:33.342656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f71c0e-8882-4132-b120-4c4dc052ff93 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:33.352710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ddb4a-a2e8-4572-8f1f-1244dd867fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:33.353468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ddb4a-a2e8-4572-8f1f-1244dd867fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:33.353996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ddb4a-a2e8-4572-8f1f-1244dd867fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:33.354329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ddb4a-a2e8-4572-8f1f-1244dd867fd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:33.355003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ddb4a-a2e8-4572-8f1f-1244dd867fd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:33.356949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ddb4a-a2e8-4572-8f1f-1244dd867fd7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:33.418429 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2244/4418] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:04:33 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:04:33.422139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198851d-2ace-45c2-9453-3ab718788004 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:33.435115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198851d-2ace-45c2-9453-3ab718788004 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:33.498020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:33.544593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957649c-18e9-4391-a6ca-77e933618996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:33.545507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957649c-18e9-4391-a6ca-77e933618996 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:33.545800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957649c-18e9-4391-a6ca-77e933618996 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:33.546075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957649c-18e9-4391-a6ca-77e933618996 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:33.546862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957649c-18e9-4391-a6ca-77e933618996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:33.561128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957649c-18e9-4391-a6ca-77e933618996 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:33.611792 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2175/4419] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:04:33 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:04:33.615570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ddb4a-a2e8-4572-8f1f-1244dd867fd7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:33.626809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ddb4a-a2e8-4572-8f1f-1244dd867fd7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:33.632424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af86fa3-6b2b-4953-9720-8ed5b176de12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:33.633861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af86fa3-6b2b-4953-9720-8ed5b176de12 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:33.634435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af86fa3-6b2b-4953-9720-8ed5b176de12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:33.634903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af86fa3-6b2b-4953-9720-8ed5b176de12 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:33.635711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af86fa3-6b2b-4953-9720-8ed5b176de12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:33.637575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af86fa3-6b2b-4953-9720-8ed5b176de12 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:33.652928 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2245/4420] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:33.656662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957649c-18e9-4391-a6ca-77e933618996 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:33.669048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957649c-18e9-4391-a6ca-77e933618996 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:33.675756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156eb78-049b-43e7-adfd-4e3ef2534034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:33.676468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156eb78-049b-43e7-adfd-4e3ef2534034 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:33.676857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156eb78-049b-43e7-adfd-4e3ef2534034 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:33.677396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156eb78-049b-43e7-adfd-4e3ef2534034 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:33.678152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156eb78-049b-43e7-adfd-4e3ef2534034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:33.680548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156eb78-049b-43e7-adfd-4e3ef2534034 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:33.692278 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2176/4421] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:33.697324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af86fa3-6b2b-4953-9720-8ed5b176de12 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:33.709838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af86fa3-6b2b-4953-9720-8ed5b176de12 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:33.731463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb9506-8359-4216-8bf7-776d43fe531d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:33.732210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb9506-8359-4216-8bf7-776d43fe531d admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:33.732498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb9506-8359-4216-8bf7-776d43fe531d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:33.733080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb9506-8359-4216-8bf7-776d43fe531d admin admin] PATH_INFO: `/v3/projects/e18243f9806149518377c15c3a8c58a1/users/0710b1b0185e41a2a2c68b5ed22fe4f6/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:33.759160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb9506-8359-4216-8bf7-776d43fe531d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18243f9806149518377c15c3a8c58a1, user_id=0710b1b0185e41a2a2c68b5ed22fe4f6, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:33.763088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb9506-8359-4216-8bf7-776d43fe531d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:33.777899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:33.793991 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2246/4422] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:04:33 2022] PUT /identity/v3/projects/e18243f9806149518377c15c3a8c58a1/users/0710b1b0185e41a2a2c68b5ed22fe4f6/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:33.806422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6d16-d7e3-4bd4-9e44-50c33432d96e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:33.806808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6d16-d7e3-4bd4-9e44-50c33432d96e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:33.807017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6d16-d7e3-4bd4-9e44-50c33432d96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:33.836756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ad6e6d16-d7e3-4bd4-9e44-50c33432d96e None None] MFA Rules not processed for user `0710b1b0185e41a2a2c68b5ed22fe4f6`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:33.838487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ad6e6d16-d7e3-4bd4-9e44-50c33432d96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:33.931367 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2177/4423] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:33.943941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb9506-8359-4216-8bf7-776d43fe531d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:33.945833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dffb9506-8359-4216-8bf7-776d43fe531d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:34.091933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb9506-8359-4216-8bf7-776d43fe531d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:34.098059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3baede-3eaf-4751-b130-675276489ad7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:34.098794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3baede-3eaf-4751-b130-675276489ad7 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:34.099273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3baede-3eaf-4751-b130-675276489ad7 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:34.099648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3baede-3eaf-4751-b130-675276489ad7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:34.120139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3baede-3eaf-4751-b130-675276489ad7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:34.122570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3baede-3eaf-4751-b130-675276489ad7 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:34.145305 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2247/4424] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:04:34.199720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6d16-d7e3-4bd4-9e44-50c33432d96e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:34.275943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6d16-d7e3-4bd4-9e44-50c33432d96e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:34.280871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:34.281631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:34.282092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:34.282467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] PATH_INFO: `/v3/users/0710b1b0185e41a2a2c68b5ed22fe4f6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:34.284047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0710b1b0185e41a2a2c68b5ed22fe4f6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:34.286478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:34.366281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0710b1b0185e41a2a2c68b5ed22fe4f6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:34.369368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:34.390362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0710b1b0185e41a2a2c68b5ed22fe4f6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:34.391346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0710b1b0185e41a2a2c68b5ed22fe4f6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:34.401760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0710b1b0185e41a2a2c68b5ed22fe4f6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:34.411561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0710b1b0185e41a2a2c68b5ed22fe4f6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:34.426408 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2178/4425] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:04:34 2022] DELETE /identity/v3/users/0710b1b0185e41a2a2c68b5ed22fe4f6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:34.427872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923ae71-8bc4-4c8c-8f6e-d091755ff84d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:34.428671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923ae71-8bc4-4c8c-8f6e-d091755ff84d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:34.429202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923ae71-8bc4-4c8c-8f6e-d091755ff84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:34.457968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1923ae71-8bc4-4c8c-8f6e-d091755ff84d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:34.460473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1923ae71-8bc4-4c8c-8f6e-d091755ff84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:34.567719 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2248/4426] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:34.582571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:34.584119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:34.707162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e2757-aa83-48d1-8ab5-7cba55dcc227 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:34.718243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b2bd6-1554-4675-961f-1106e5310426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:34.719485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b2bd6-1554-4675-961f-1106e5310426 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:34.719785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b2bd6-1554-4675-961f-1106e5310426 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:34.720073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b2bd6-1554-4675-961f-1106e5310426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:34.733641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b2bd6-1554-4675-961f-1106e5310426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:34.735115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b2bd6-1554-4675-961f-1106e5310426 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:34.750448 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2179/4427] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:35.366734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923ae71-8bc4-4c8c-8f6e-d091755ff84d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:35.370886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1923ae71-8bc4-4c8c-8f6e-d091755ff84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:35.403739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923ae71-8bc4-4c8c-8f6e-d091755ff84d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:35.410686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c673d7e-7342-4c49-a7bf-f2f04769037d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:35.411802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c673d7e-7342-4c49-a7bf-f2f04769037d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:35.412324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c673d7e-7342-4c49-a7bf-f2f04769037d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:35.413388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c673d7e-7342-4c49-a7bf-f2f04769037d admin admin] PATH_INFO: `/v3/projects/e18243f9806149518377c15c3a8c58a1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:35.415954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c673d7e-7342-4c49-a7bf-f2f04769037d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18243f9806149518377c15c3a8c58a1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:35.419730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c673d7e-7342-4c49-a7bf-f2f04769037d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:35.499570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c673d7e-7342-4c49-a7bf-f2f04769037d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18243f9806149518377c15c3a8c58a1', 'request_id': 'req-8c673d7e-7342-4c49-a7bf-f2f04769037d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:35.518237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c673d7e-7342-4c49-a7bf-f2f04769037d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18243f9806149518377c15c3a8c58a1', 'request_id': 'req-8c673d7e-7342-4c49-a7bf-f2f04769037d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:35.534764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c673d7e-7342-4c49-a7bf-f2f04769037d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18243f9806149518377c15c3a8c58a1', 'request_id': 'req-8c673d7e-7342-4c49-a7bf-f2f04769037d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:35.579721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c673d7e-7342-4c49-a7bf-f2f04769037d admin admin] The token cache is being invalidate because project e18243f9806149518377c15c3a8c58a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:04:35.610764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c673d7e-7342-4c49-a7bf-f2f04769037d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:35.612005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c673d7e-7342-4c49-a7bf-f2f04769037d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:35.619282 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2249/4428] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:04:35 2022] DELETE /identity/v3/projects/e18243f9806149518377c15c3a8c58a1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:35.625096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed54d34-275a-4597-9cbb-81b8cd904615 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:35.625625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed54d34-275a-4597-9cbb-81b8cd904615 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:35.625726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed54d34-275a-4597-9cbb-81b8cd904615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:35.705295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6ed54d34-275a-4597-9cbb-81b8cd904615 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:35.706915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6ed54d34-275a-4597-9cbb-81b8cd904615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:35.834813 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2180/4429] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:35.843486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93eeec-a014-4334-8657-e8b5764ab7a9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:35.843829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93eeec-a014-4334-8657-e8b5764ab7a9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:35.844049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93eeec-a014-4334-8657-e8b5764ab7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:35.868834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed54d34-275a-4597-9cbb-81b8cd904615 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:35.870593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6ed54d34-275a-4597-9cbb-81b8cd904615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:35.901408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ca93eeec-a014-4334-8657-e8b5764ab7a9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:35.903849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ca93eeec-a014-4334-8657-e8b5764ab7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:35.930257 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2250/4430] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:35.934784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93eeec-a014-4334-8657-e8b5764ab7a9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:35.944378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:35.958027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93eeec-a014-4334-8657-e8b5764ab7a9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:35.985035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f510b-a823-4e26-bcb9-70c6ea096dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:35.986770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f510b-a823-4e26-bcb9-70c6ea096dfd admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:35.987395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f510b-a823-4e26-bcb9-70c6ea096dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:35.989271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f510b-a823-4e26-bcb9-70c6ea096dfd admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:35.990537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f510b-a823-4e26-bcb9-70c6ea096dfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:04:35.991774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f510b-a823-4e26-bcb9-70c6ea096dfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:35.998056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f510b-a823-4e26-bcb9-70c6ea096dfd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:36.025893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed54d34-275a-4597-9cbb-81b8cd904615 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:36.033320 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2251/4431] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:04:36.035865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f510b-a823-4e26-bcb9-70c6ea096dfd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:36.036840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369150c-f424-4916-a01d-88728ca72abe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:36.037511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369150c-f424-4916-a01d-88728ca72abe service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:36.037796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369150c-f424-4916-a01d-88728ca72abe service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:36.038076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369150c-f424-4916-a01d-88728ca72abe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:36.040493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b369150c-f424-4916-a01d-88728ca72abe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:36.050133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f510b-a823-4e26-bcb9-70c6ea096dfd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:36.054460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:36.060046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369150c-f424-4916-a01d-88728ca72abe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:36.061870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369150c-f424-4916-a01d-88728ca72abe service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:36.062188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05cf78-c497-49a3-9a63-618106781590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:36.063067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05cf78-c497-49a3-9a63-618106781590 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:36.063579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05cf78-c497-49a3-9a63-618106781590 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:36.064025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05cf78-c497-49a3-9a63-618106781590 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:36.065028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05cf78-c497-49a3-9a63-618106781590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:36.068304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05cf78-c497-49a3-9a63-618106781590 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:36.082297 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2181/4432] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:36.114236 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2252/4433] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:04:36 2022] POST /identity/v3/projects => generated 365 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:04:36.122919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369150c-f424-4916-a01d-88728ca72abe service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:36.134819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369150c-f424-4916-a01d-88728ca72abe service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:36.200190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82733c47-bdf4-4052-80ee-d84711d10c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:36.200843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82733c47-bdf4-4052-80ee-d84711d10c26 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:36.201165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82733c47-bdf4-4052-80ee-d84711d10c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:36.201470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82733c47-bdf4-4052-80ee-d84711d10c26 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:36.202277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82733c47-bdf4-4052-80ee-d84711d10c26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:36.204451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82733c47-bdf4-4052-80ee-d84711d10c26 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:36.240997 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2182/4434] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:04:36 2022] POST /identity/v3/users => generated 283 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:04:36.245960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05cf78-c497-49a3-9a63-618106781590 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:36.280545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05cf78-c497-49a3-9a63-618106781590 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:36.280545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a5d42-1b7e-45ca-9575-8e76191711d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:36.281035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a5d42-1b7e-45ca-9575-8e76191711d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:36.281496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a5d42-1b7e-45ca-9575-8e76191711d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:36.281742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a5d42-1b7e-45ca-9575-8e76191711d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:36.282488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a5d42-1b7e-45ca-9575-8e76191711d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:36.284051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a5d42-1b7e-45ca-9575-8e76191711d6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:36.296963 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2253/4435] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:36.300760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82733c47-bdf4-4052-80ee-d84711d10c26 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:36.321188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82733c47-bdf4-4052-80ee-d84711d10c26 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:36.328035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7271c-b62a-4f29-8aff-b1bd033ac9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:36.329192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7271c-b62a-4f29-8aff-b1bd033ac9af admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:36.329566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7271c-b62a-4f29-8aff-b1bd033ac9af admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:36.329937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7271c-b62a-4f29-8aff-b1bd033ac9af admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:36.330588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7271c-b62a-4f29-8aff-b1bd033ac9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:36.332751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7271c-b62a-4f29-8aff-b1bd033ac9af admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:36.360403 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2183/4436] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:36.361345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a5d42-1b7e-45ca-9575-8e76191711d6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:36.388761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a5d42-1b7e-45ca-9575-8e76191711d6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:36.394296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cf41e-f8f0-4e25-a90f-cf8966a16d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:36.395084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cf41e-f8f0-4e25-a90f-cf8966a16d20 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:36.395375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cf41e-f8f0-4e25-a90f-cf8966a16d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:36.395662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cf41e-f8f0-4e25-a90f-cf8966a16d20 admin admin] PATH_INFO: `/v3/projects/a6bda3914676415c8c173972749ef368/users/ac6eac3e9b65428ba9430970e9cbccaf/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:36.422983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3cf41e-f8f0-4e25-a90f-cf8966a16d20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6bda3914676415c8c173972749ef368, user_id=ac6eac3e9b65428ba9430970e9cbccaf, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:36.427528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3cf41e-f8f0-4e25-a90f-cf8966a16d20 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:36.451544 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2254/4437] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:04:36 2022] PUT /identity/v3/projects/a6bda3914676415c8c173972749ef368/users/ac6eac3e9b65428ba9430970e9cbccaf/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:04:36.456775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014ba1-a3ee-44c6-9588-0868e25c31ee None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:36.457249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014ba1-a3ee-44c6-9588-0868e25c31ee None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:36.457563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014ba1-a3ee-44c6-9588-0868e25c31ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:36.485684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-65014ba1-a3ee-44c6-9588-0868e25c31ee None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:36.486924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-65014ba1-a3ee-44c6-9588-0868e25c31ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:36.573829 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2184/4438] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:36.595169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cf41e-f8f0-4e25-a90f-cf8966a16d20 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:36.599066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9a3cf41e-f8f0-4e25-a90f-cf8966a16d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:36.699593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cf41e-f8f0-4e25-a90f-cf8966a16d20 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:36.704213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0a967-f9a3-4476-8863-8bb0e28a150a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:36.705255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0a967-f9a3-4476-8863-8bb0e28a150a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:36.705731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0a967-f9a3-4476-8863-8bb0e28a150a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:36.706179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0a967-f9a3-4476-8863-8bb0e28a150a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:36.719715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0a967-f9a3-4476-8863-8bb0e28a150a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:36.721737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0a967-f9a3-4476-8863-8bb0e28a150a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:36.749439 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2255/4439] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:44.526051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f36236-50e8-4079-8b3c-e1522a0c9122 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:44.526596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f36236-50e8-4079-8b3c-e1522a0c9122 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:44.526596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f36236-50e8-4079-8b3c-e1522a0c9122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:44.573821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-48f36236-50e8-4079-8b3c-e1522a0c9122 None None] MFA Rules not processed for user `ac6eac3e9b65428ba9430970e9cbccaf`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:44.575149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-48f36236-50e8-4079-8b3c-e1522a0c9122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:44.731957 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2185/4440] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:44.746877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0a967-f9a3-4476-8863-8bb0e28a150a service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:44.748445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fbd0a967-f9a3-4476-8863-8bb0e28a150a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:44.909246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0a967-f9a3-4476-8863-8bb0e28a150a service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:44.915890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d019d-8455-4a1e-b5ae-baf8e3857d75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:44.916534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d019d-8455-4a1e-b5ae-baf8e3857d75 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:44.916956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d019d-8455-4a1e-b5ae-baf8e3857d75 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:44.917229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d019d-8455-4a1e-b5ae-baf8e3857d75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:44.927892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d019d-8455-4a1e-b5ae-baf8e3857d75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:44.930580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d019d-8455-4a1e-b5ae-baf8e3857d75 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:44.946494 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2256/4441] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:45.282904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784e10-40e8-4feb-b45f-b193a4d78018 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:45.283270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784e10-40e8-4feb-b45f-b193a4d78018 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:45.290369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784e10-40e8-4feb-b45f-b193a4d78018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:45.391371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:45.437496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-21784e10-40e8-4feb-b45f-b193a4d78018 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:45.438770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-21784e10-40e8-4feb-b45f-b193a4d78018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:45.576250 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2186/4442] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:04:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:45.587549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d019d-8455-4a1e-b5ae-baf8e3857d75 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:45.589343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6a5d019d-8455-4a1e-b5ae-baf8e3857d75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:45.680007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:45.888561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d019d-8455-4a1e-b5ae-baf8e3857d75 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:45.894758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f39eaf-6757-43fa-a7e5-1ffa9365a8ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:45.895970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f39eaf-6757-43fa-a7e5-1ffa9365a8ce service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:45.902386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f39eaf-6757-43fa-a7e5-1ffa9365a8ce service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:45.902901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f39eaf-6757-43fa-a7e5-1ffa9365a8ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:45.926545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f39eaf-6757-43fa-a7e5-1ffa9365a8ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:45.932246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f39eaf-6757-43fa-a7e5-1ffa9365a8ce service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:45.971233 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2257/4443] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:04:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:04:46.795318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21784e10-40e8-4feb-b45f-b193a4d78018 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:46.796773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-21784e10-40e8-4feb-b45f-b193a4d78018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:46.837609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21784e10-40e8-4feb-b45f-b193a4d78018 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:46.857111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:46.858118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:46.858558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:46.858840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] PATH_INFO: `/v3/users/c53c59095759426887a885de3c7f02e6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:46.861458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c53c59095759426887a885de3c7f02e6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:46.863339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:46.987755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c53c59095759426887a885de3c7f02e6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:47.024441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c53c59095759426887a885de3c7f02e6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:47.025668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c53c59095759426887a885de3c7f02e6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:47.035680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c53c59095759426887a885de3c7f02e6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:47.059480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c53c59095759426887a885de3c7f02e6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:47.069788 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2187/4444] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:04:46 2022] DELETE /identity/v3/users/c53c59095759426887a885de3c7f02e6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:47.075665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558e26f-a34c-4e1d-8629-5ab42bba2e27 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:47.076192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558e26f-a34c-4e1d-8629-5ab42bba2e27 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:47.076455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558e26f-a34c-4e1d-8629-5ab42bba2e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:47.123182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4558e26f-a34c-4e1d-8629-5ab42bba2e27 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:47.124519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4558e26f-a34c-4e1d-8629-5ab42bba2e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:47.273974 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2258/4445] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:47.306541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:47.308024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:47.321718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558e26f-a34c-4e1d-8629-5ab42bba2e27 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:47.323167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4558e26f-a34c-4e1d-8629-5ab42bba2e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:47.359915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558e26f-a34c-4e1d-8629-5ab42bba2e27 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:47.380738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9ba4e-e6e6-4aed-b677-c9dd91c18fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:47.381544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9ba4e-e6e6-4aed-b677-c9dd91c18fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:47.381935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9ba4e-e6e6-4aed-b677-c9dd91c18fed admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:47.382334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9ba4e-e6e6-4aed-b677-c9dd91c18fed admin admin] PATH_INFO: `/v3/users/77edcf09f5f84e51a4fb08dbaf763a67` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:47.388118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9ba4e-e6e6-4aed-b677-c9dd91c18fed admin admin] RBAC: Authorizing `identity:delete_user(user_id=77edcf09f5f84e51a4fb08dbaf763a67)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:47.390874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9ba4e-e6e6-4aed-b677-c9dd91c18fed admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:47.478876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b4d24-2a67-4672-b896-31a78387bba5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:47.483479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a96492-e570-4fa0-bf1d-e87a3641a831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:47.484118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96492-e570-4fa0-bf1d-e87a3641a831 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:47.484425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96492-e570-4fa0-bf1d-e87a3641a831 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:47.484773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96492-e570-4fa0-bf1d-e87a3641a831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:47.498801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a96492-e570-4fa0-bf1d-e87a3641a831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:47.501862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a96492-e570-4fa0-bf1d-e87a3641a831 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:47.509843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e8f9ba4e-e6e6-4aed-b677-c9dd91c18fed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77edcf09f5f84e51a4fb08dbaf763a67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:47.517583 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2188/4446] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:04:47.538332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e8f9ba4e-e6e6-4aed-b677-c9dd91c18fed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77edcf09f5f84e51a4fb08dbaf763a67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:47.549235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e8f9ba4e-e6e6-4aed-b677-c9dd91c18fed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77edcf09f5f84e51a4fb08dbaf763a67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:47.550034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e8f9ba4e-e6e6-4aed-b677-c9dd91c18fed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77edcf09f5f84e51a4fb08dbaf763a67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:47.550714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e8f9ba4e-e6e6-4aed-b677-c9dd91c18fed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77edcf09f5f84e51a4fb08dbaf763a67'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:47.610143 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2259/4447] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:04:47 2022] DELETE /identity/v3/users/77edcf09f5f84e51a4fb08dbaf763a67 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:04:47.981905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a96492-e570-4fa0-bf1d-e87a3641a831 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:48.003416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a96492-e570-4fa0-bf1d-e87a3641a831 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:48.104820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:48.195501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:48.196502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:48.197010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:48.197457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] PATH_INFO: `/v3/projects/26e12c7419f641b4a7e56274a34826ee` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:48.198827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e12c7419f641b4a7e56274a34826ee)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:48.201264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:48.353508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e12c7419f641b4a7e56274a34826ee', 'request_id': 'req-7a13061b-0e3c-4150-aff4-0b1fcbfab563'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:48.391090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e12c7419f641b4a7e56274a34826ee', 'request_id': 'req-7a13061b-0e3c-4150-aff4-0b1fcbfab563'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:48.437599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e12c7419f641b4a7e56274a34826ee', 'request_id': 'req-7a13061b-0e3c-4150-aff4-0b1fcbfab563'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:48.460678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] The token cache is being invalidate because project 26e12c7419f641b4a7e56274a34826ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:04:48.460927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:48.461958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:48.464583 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2189/4448] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:04:47 2022] DELETE /identity/v3/projects/26e12c7419f641b4a7e56274a34826ee => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:48.469536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046a113-456d-4eb0-83a0-73449f74d695 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:48.470020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046a113-456d-4eb0-83a0-73449f74d695 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:48.470247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046a113-456d-4eb0-83a0-73449f74d695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:48.531154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:48.533584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:48.569828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a046a113-456d-4eb0-83a0-73449f74d695 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:48.580969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a046a113-456d-4eb0-83a0-73449f74d695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:48.662878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:48.724069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13061b-0e3c-4150-aff4-0b1fcbfab563 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:48.739237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926f23a-fa66-4e25-a640-c906149239f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:48.740051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926f23a-fa66-4e25-a640-c906149239f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:48.740345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926f23a-fa66-4e25-a640-c906149239f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:48.740625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926f23a-fa66-4e25-a640-c906149239f9 admin admin] PATH_INFO: `/v3/projects/8da3c32d1cc843458f7744cc038f8be1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:48.741955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926f23a-fa66-4e25-a640-c906149239f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8da3c32d1cc843458f7744cc038f8be1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:48.746166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926f23a-fa66-4e25-a640-c906149239f9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:48.835442 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2260/4449] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:48.845247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27920cf3-7a11-464f-8466-931c06d1d6f0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:48.845628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27920cf3-7a11-464f-8466-931c06d1d6f0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:48.845975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27920cf3-7a11-464f-8466-931c06d1d6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:48.883185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a926f23a-fa66-4e25-a640-c906149239f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8da3c32d1cc843458f7744cc038f8be1', 'request_id': 'req-a926f23a-fa66-4e25-a640-c906149239f9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:48.907159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a926f23a-fa66-4e25-a640-c906149239f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8da3c32d1cc843458f7744cc038f8be1', 'request_id': 'req-a926f23a-fa66-4e25-a640-c906149239f9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:48.909173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:48.911930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-27920cf3-7a11-464f-8466-931c06d1d6f0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:48.913989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-27920cf3-7a11-464f-8466-931c06d1d6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:48.925213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a926f23a-fa66-4e25-a640-c906149239f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8da3c32d1cc843458f7744cc038f8be1', 'request_id': 'req-a926f23a-fa66-4e25-a640-c906149239f9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:48.978345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a926f23a-fa66-4e25-a640-c906149239f9 admin admin] The token cache is being invalidate because project 8da3c32d1cc843458f7744cc038f8be1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:04:48.978951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a926f23a-fa66-4e25-a640-c906149239f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:48.982148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a926f23a-fa66-4e25-a640-c906149239f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:48.985668 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2190/4450] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:04:48 2022] DELETE /identity/v3/projects/8da3c32d1cc843458f7744cc038f8be1 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:48.987174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3964-3123-4e7a-8940-16ad61d66772 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:48.987646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3964-3123-4e7a-8940-16ad61d66772 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:48.988103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3964-3123-4e7a-8940-16ad61d66772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.017315 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2261/4451] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:49.021464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da58e50-139e-4f76-92cf-087a23d4c42a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:49.022383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da58e50-139e-4f76-92cf-087a23d4c42a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:49.022928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da58e50-139e-4f76-92cf-087a23d4c42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.028889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f5fd3964-3123-4e7a-8940-16ad61d66772 None None] MFA Rules not processed for user `ac6eac3e9b65428ba9430970e9cbccaf`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:49.030431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f5fd3964-3123-4e7a-8940-16ad61d66772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:49.065919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4da58e50-139e-4f76-92cf-087a23d4c42a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:49.067266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4da58e50-139e-4f76-92cf-087a23d4c42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:49.089486 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2262/4452] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:49.089486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da58e50-139e-4f76-92cf-087a23d4c42a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:49.116210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da58e50-139e-4f76-92cf-087a23d4c42a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:49.144202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ad41a-7364-4957-8215-8cf031ba3588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:49.144851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ad41a-7364-4957-8215-8cf031ba3588 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:49.145172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ad41a-7364-4957-8215-8cf031ba3588 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:49.145458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ad41a-7364-4957-8215-8cf031ba3588 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.146136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ad41a-7364-4957-8215-8cf031ba3588 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:04:49.147986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ad41a-7364-4957-8215-8cf031ba3588 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:49.154797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ad41a-7364-4957-8215-8cf031ba3588 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:49.171700 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2191/4453] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:49.176431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae615b-f7ca-4c71-b54a-30c97fdf92e9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:49.176431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae615b-f7ca-4c71-b54a-30c97fdf92e9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:49.176431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae615b-f7ca-4c71-b54a-30c97fdf92e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.187395 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2263/4454] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:04:49.192274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ad41a-7364-4957-8215-8cf031ba3588 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:49.206782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ad41a-7364-4957-8215-8cf031ba3588 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:49.217561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0514a3-9dff-483d-9906-121af1e58423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:49.223955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8fae615b-f7ca-4c71-b54a-30c97fdf92e9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:49.225276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8fae615b-f7ca-4c71-b54a-30c97fdf92e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:49.227170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0514a3-9dff-483d-9906-121af1e58423 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:49.227484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0514a3-9dff-483d-9906-121af1e58423 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:49.227774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0514a3-9dff-483d-9906-121af1e58423 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.228568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0514a3-9dff-483d-9906-121af1e58423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:49.243579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0514a3-9dff-483d-9906-121af1e58423 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:49.249446 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2192/4455] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:49.253741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae615b-f7ca-4c71-b54a-30c97fdf92e9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:49.294338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae615b-f7ca-4c71-b54a-30c97fdf92e9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:49.323689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3ba78-1d03-4e7c-9470-a15f6f846541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:49.323689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3ba78-1d03-4e7c-9470-a15f6f846541 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:49.324035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3ba78-1d03-4e7c-9470-a15f6f846541 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:49.324323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3ba78-1d03-4e7c-9470-a15f6f846541 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.324933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3ba78-1d03-4e7c-9470-a15f6f846541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:04:49.325238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3ba78-1d03-4e7c-9470-a15f6f846541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:49.326867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3ba78-1d03-4e7c-9470-a15f6f846541 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:49.330885 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2264/4456] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:04:49 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:04:49.335027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0514a3-9dff-483d-9906-121af1e58423 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:49.396367 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2193/4457] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:04:49.400496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3ba78-1d03-4e7c-9470-a15f6f846541 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:49.449263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3ba78-1d03-4e7c-9470-a15f6f846541 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:49.472987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec480398-a4fd-4267-aa8b-19efa2f5c264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:49.473700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec480398-a4fd-4267-aa8b-19efa2f5c264 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:49.474053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec480398-a4fd-4267-aa8b-19efa2f5c264 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:49.475193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec480398-a4fd-4267-aa8b-19efa2f5c264 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.476283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec480398-a4fd-4267-aa8b-19efa2f5c264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:49.480824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec480398-a4fd-4267-aa8b-19efa2f5c264 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:49.481724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0514a3-9dff-483d-9906-121af1e58423 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:49.497104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304b6a3-0baf-483a-b25e-8b7a2cddaa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:49.497808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304b6a3-0baf-483a-b25e-8b7a2cddaa6f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:49.498095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304b6a3-0baf-483a-b25e-8b7a2cddaa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:49.498375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304b6a3-0baf-483a-b25e-8b7a2cddaa6f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.500328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304b6a3-0baf-483a-b25e-8b7a2cddaa6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:49.507319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304b6a3-0baf-483a-b25e-8b7a2cddaa6f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:49.578926 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2194/4458] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:04:49 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:04:49.579794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec480398-a4fd-4267-aa8b-19efa2f5c264 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:49.587415 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2265/4459] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:04:49 2022] POST /identity/v3/users => generated 290 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:04:49.588366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304b6a3-0baf-483a-b25e-8b7a2cddaa6f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:49.631539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304b6a3-0baf-483a-b25e-8b7a2cddaa6f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:49.654535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:49.797691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f2fa7-08f9-4a56-bdf0-3df233182cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:49.798340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f2fa7-08f9-4a56-bdf0-3df233182cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:49.798630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f2fa7-08f9-4a56-bdf0-3df233182cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:49.798914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f2fa7-08f9-4a56-bdf0-3df233182cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.799579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f2fa7-08f9-4a56-bdf0-3df233182cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:49.801237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f2fa7-08f9-4a56-bdf0-3df233182cd6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:49.827126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec480398-a4fd-4267-aa8b-19efa2f5c264 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:49.829280 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2266/4460] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:49.834721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f2fa7-08f9-4a56-bdf0-3df233182cd6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:49.846987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3c156-e419-428b-ab61-d98e4a897cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:49.848177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3c156-e419-428b-ab61-d98e4a897cbd admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:49.848705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3c156-e419-428b-ab61-d98e4a897cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:49.849264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3c156-e419-428b-ab61-d98e4a897cbd admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.852505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3c156-e419-428b-ab61-d98e4a897cbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:49.858784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3c156-e419-428b-ab61-d98e4a897cbd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:49.865745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f2fa7-08f9-4a56-bdf0-3df233182cd6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:49.886590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a31e8-ad02-4a31-aa4d-66429055a919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:49.887323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a31e8-ad02-4a31-aa4d-66429055a919 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:49.887555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a31e8-ad02-4a31-aa4d-66429055a919 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:49.887840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a31e8-ad02-4a31-aa4d-66429055a919 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.888442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a31e8-ad02-4a31-aa4d-66429055a919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:49.890164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a31e8-ad02-4a31-aa4d-66429055a919 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:49.923967 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2267/4461] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:49.924954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a31e8-ad02-4a31-aa4d-66429055a919 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:49.946906 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2195/4462] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:04:49 2022] POST /identity/v3/users => generated 293 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:04:49.952859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a31e8-ad02-4a31-aa4d-66429055a919 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:49.953796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3c156-e419-428b-ab61-d98e4a897cbd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:49.965754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8413c06-7c08-477b-9864-67ffcf421f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:49.966607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8413c06-7c08-477b-9864-67ffcf421f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:49.966870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8413c06-7c08-477b-9864-67ffcf421f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:49.967220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8413c06-7c08-477b-9864-67ffcf421f37 admin admin] PATH_INFO: `/v3/projects/30db100f9ded4cd48da8e5f7b67fe09c/users/52bf5760e9ef43379e60159d3d2ed744/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.974881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3c156-e419-428b-ab61-d98e4a897cbd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:49.983389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c13bb7-c648-4095-a5d5-f343e0bc342e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:49.984929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c13bb7-c648-4095-a5d5-f343e0bc342e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:49.985450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c13bb7-c648-4095-a5d5-f343e0bc342e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:49.985978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c13bb7-c648-4095-a5d5-f343e0bc342e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:49.987275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c13bb7-c648-4095-a5d5-f343e0bc342e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:49.989529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c13bb7-c648-4095-a5d5-f343e0bc342e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:49.999365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8413c06-7c08-477b-9864-67ffcf421f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30db100f9ded4cd48da8e5f7b67fe09c, user_id=52bf5760e9ef43379e60159d3d2ed744, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:50.002111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8413c06-7c08-477b-9864-67ffcf421f37 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:50.012375 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2196/4463] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:04:50.021061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c13bb7-c648-4095-a5d5-f343e0bc342e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:50.032799 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2268/4464] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:04:49 2022] PUT /identity/v3/projects/30db100f9ded4cd48da8e5f7b67fe09c/users/52bf5760e9ef43379e60159d3d2ed744/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:50.038525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c13bb7-c648-4095-a5d5-f343e0bc342e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:50.043663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274111e-4db8-43c5-9862-49b3ae058704 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:50.043952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274111e-4db8-43c5-9862-49b3ae058704 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:50.044240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274111e-4db8-43c5-9862-49b3ae058704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:50.079383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:50.110626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3274111e-4db8-43c5-9862-49b3ae058704 None None] MFA Rules not processed for user `52bf5760e9ef43379e60159d3d2ed744`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:50.112226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3274111e-4db8-43c5-9862-49b3ae058704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:50.192574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a186ea0-d9a4-447f-869f-3c50b2415a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:50.193404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a186ea0-d9a4-447f-869f-3c50b2415a07 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:50.198770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a186ea0-d9a4-447f-869f-3c50b2415a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:50.199220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a186ea0-d9a4-447f-869f-3c50b2415a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:50.200041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a186ea0-d9a4-447f-869f-3c50b2415a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:50.205097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a186ea0-d9a4-447f-869f-3c50b2415a07 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:50.245529 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2197/4465] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:50.253044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a186ea0-d9a4-447f-869f-3c50b2415a07 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:50.271347 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2269/4466] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:50.272857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106bf86-16f8-4b86-ad0e-b062eb5e9783 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:50.274360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106bf86-16f8-4b86-ad0e-b062eb5e9783 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:50.274659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106bf86-16f8-4b86-ad0e-b062eb5e9783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:50.287418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a186ea0-d9a4-447f-869f-3c50b2415a07 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:50.303723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4520b-efbd-4fb2-a102-b83263efc219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:50.305130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4520b-efbd-4fb2-a102-b83263efc219 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:50.305587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4520b-efbd-4fb2-a102-b83263efc219 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:50.306004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4520b-efbd-4fb2-a102-b83263efc219 admin admin] PATH_INFO: `/v3/projects/87d300f145fd4855a6b3382997d975a9/users/1d03a8c35f2b4da39fac98b75ce9d810/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:50.330264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4520b-efbd-4fb2-a102-b83263efc219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d300f145fd4855a6b3382997d975a9, user_id=1d03a8c35f2b4da39fac98b75ce9d810, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:50.330434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6106bf86-16f8-4b86-ad0e-b062eb5e9783 None None] MFA Rules not processed for user `52bf5760e9ef43379e60159d3d2ed744`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:50.331289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6106bf86-16f8-4b86-ad0e-b062eb5e9783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:50.334362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4520b-efbd-4fb2-a102-b83263efc219 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:50.358163 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2270/4467] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:50.364772 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2198/4468] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:04:50 2022] PUT /identity/v3/projects/87d300f145fd4855a6b3382997d975a9/users/1d03a8c35f2b4da39fac98b75ce9d810/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:50.375824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a22476-863c-423a-975b-0ea687cb0949 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:50.376126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a22476-863c-423a-975b-0ea687cb0949 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:50.376411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a22476-863c-423a-975b-0ea687cb0949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:50.426146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-04a22476-863c-423a-975b-0ea687cb0949 None None] MFA Rules not processed for user `1d03a8c35f2b4da39fac98b75ce9d810`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:50.426146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-04a22476-863c-423a-975b-0ea687cb0949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:50.503649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:50.519477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4520b-efbd-4fb2-a102-b83263efc219 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:50.525065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f3a4520b-efbd-4fb2-a102-b83263efc219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:50.588324 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2271/4469] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:50.618227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a22476-863c-423a-975b-0ea687cb0949 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:50.619674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-04a22476-863c-423a-975b-0ea687cb0949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:50.675072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4520b-efbd-4fb2-a102-b83263efc219 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:50.688068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e9dd9-90c8-472d-8c1b-dabe6dc6e5eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:50.688834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e9dd9-90c8-472d-8c1b-dabe6dc6e5eb service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:50.691438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e9dd9-90c8-472d-8c1b-dabe6dc6e5eb service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:50.691844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e9dd9-90c8-472d-8c1b-dabe6dc6e5eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:50.731039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a22476-863c-423a-975b-0ea687cb0949 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:50.735581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e590a326-648d-43e0-85ed-3f8bcc77aa1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:50.736559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590a326-648d-43e0-85ed-3f8bcc77aa1e service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:50.737071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590a326-648d-43e0-85ed-3f8bcc77aa1e service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:50.737328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590a326-648d-43e0-85ed-3f8bcc77aa1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:50.755368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e590a326-648d-43e0-85ed-3f8bcc77aa1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:50.761001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e590a326-648d-43e0-85ed-3f8bcc77aa1e service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:50.789976 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2272/4470] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:50.846853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e590a326-648d-43e0-85ed-3f8bcc77aa1e service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:50.849356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e590a326-648d-43e0-85ed-3f8bcc77aa1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:50.852969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e9dd9-90c8-472d-8c1b-dabe6dc6e5eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:50.854997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e9dd9-90c8-472d-8c1b-dabe6dc6e5eb service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:50.879367 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2199/4471] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:04:51.054054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e590a326-648d-43e0-85ed-3f8bcc77aa1e service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:51.073790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bffd0-c92e-42bd-97df-9af159513f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:51.074749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bffd0-c92e-42bd-97df-9af159513f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:51.075201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bffd0-c92e-42bd-97df-9af159513f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:51.075581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bffd0-c92e-42bd-97df-9af159513f61 admin admin] PATH_INFO: `/v3/users/c897f757440d479790743e6a8c1dfd91` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:51.077864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068bffd0-c92e-42bd-97df-9af159513f61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c897f757440d479790743e6a8c1dfd91)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:51.080640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068bffd0-c92e-42bd-97df-9af159513f61 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:51.183251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-068bffd0-c92e-42bd-97df-9af159513f61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c897f757440d479790743e6a8c1dfd91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:51.207860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-068bffd0-c92e-42bd-97df-9af159513f61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c897f757440d479790743e6a8c1dfd91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:51.220114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-068bffd0-c92e-42bd-97df-9af159513f61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c897f757440d479790743e6a8c1dfd91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:51.220839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-068bffd0-c92e-42bd-97df-9af159513f61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c897f757440d479790743e6a8c1dfd91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:51.221542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-068bffd0-c92e-42bd-97df-9af159513f61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c897f757440d479790743e6a8c1dfd91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:51.243444 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2273/4472] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:04:50 2022] DELETE /identity/v3/users/c897f757440d479790743e6a8c1dfd91 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:04:51.747577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e9dd9-90c8-472d-8c1b-dabe6dc6e5eb service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:51.763325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e9dd9-90c8-472d-8c1b-dabe6dc6e5eb service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:51.865421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:51.926986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c33469-658e-40b4-9507-4470d17f28eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:51.928249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c33469-658e-40b4-9507-4470d17f28eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:51.928540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c33469-658e-40b4-9507-4470d17f28eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:51.928822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c33469-658e-40b4-9507-4470d17f28eb admin admin] PATH_INFO: `/v3/projects/484597f6a0554dd2af819e55e8e7b7d2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:51.931341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c33469-658e-40b4-9507-4470d17f28eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=484597f6a0554dd2af819e55e8e7b7d2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:51.935329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c33469-658e-40b4-9507-4470d17f28eb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:52.073837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c1c33469-658e-40b4-9507-4470d17f28eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '484597f6a0554dd2af819e55e8e7b7d2', 'request_id': 'req-c1c33469-658e-40b4-9507-4470d17f28eb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:52.092780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c1c33469-658e-40b4-9507-4470d17f28eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '484597f6a0554dd2af819e55e8e7b7d2', 'request_id': 'req-c1c33469-658e-40b4-9507-4470d17f28eb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:52.105484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c1c33469-658e-40b4-9507-4470d17f28eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '484597f6a0554dd2af819e55e8e7b7d2', 'request_id': 'req-c1c33469-658e-40b4-9507-4470d17f28eb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:52.184577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c1c33469-658e-40b4-9507-4470d17f28eb admin admin] The token cache is being invalidate because project 484597f6a0554dd2af819e55e8e7b7d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:04:52.185205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c1c33469-658e-40b4-9507-4470d17f28eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:52.186147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c1c33469-658e-40b4-9507-4470d17f28eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:04:52.189211 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2200/4473] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:04:51 2022] DELETE /identity/v3/projects/484597f6a0554dd2af819e55e8e7b7d2 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:04:52.194148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cda8d6-797e-4d7a-9ea0-6a33a63cf655 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:52.195032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cda8d6-797e-4d7a-9ea0-6a33a63cf655 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:52.196084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cda8d6-797e-4d7a-9ea0-6a33a63cf655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:52.220219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:52.235423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e9cda8d6-797e-4d7a-9ea0-6a33a63cf655 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:52.237065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e9cda8d6-797e-4d7a-9ea0-6a33a63cf655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:52.354767 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2274/4474] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:52.363158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b533d-5645-404b-9dff-bd70ba49438a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:52.363726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b533d-5645-404b-9dff-bd70ba49438a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:52.364145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b533d-5645-404b-9dff-bd70ba49438a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:52.401419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9c6b533d-5645-404b-9dff-bd70ba49438a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:52.402722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9c6b533d-5645-404b-9dff-bd70ba49438a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:52.416557 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2201/4475] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:52.425165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cda8d6-797e-4d7a-9ea0-6a33a63cf655 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:52.447432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cda8d6-797e-4d7a-9ea0-6a33a63cf655 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:52.484655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023d8e5-fe52-434f-8912-e5534f9be8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:52.484655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023d8e5-fe52-434f-8912-e5534f9be8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:52.484655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023d8e5-fe52-434f-8912-e5534f9be8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:52.484655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023d8e5-fe52-434f-8912-e5534f9be8f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:52.485476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023d8e5-fe52-434f-8912-e5534f9be8f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:04:52.485476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023d8e5-fe52-434f-8912-e5534f9be8f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:52.485476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023d8e5-fe52-434f-8912-e5534f9be8f7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:52.523877 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2275/4476] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:04:52.528048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b533d-5645-404b-9dff-bd70ba49438a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:52.552835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b533d-5645-404b-9dff-bd70ba49438a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:52.569442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23349a9c-9943-49c8-be45-d0f240f2ddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:52.570153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23349a9c-9943-49c8-be45-d0f240f2ddfa admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:52.574777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23349a9c-9943-49c8-be45-d0f240f2ddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:52.577353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23349a9c-9943-49c8-be45-d0f240f2ddfa admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:52.580256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23349a9c-9943-49c8-be45-d0f240f2ddfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:52.584801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23349a9c-9943-49c8-be45-d0f240f2ddfa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:52.654732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023d8e5-fe52-434f-8912-e5534f9be8f7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:52.660772 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2202/4477] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:04:52 2022] POST /identity/v3/projects => generated 397 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:04:52.672082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023d8e5-fe52-434f-8912-e5534f9be8f7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:52.786240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f67f3d-e6cf-4994-8147-43a73c09f512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:52.787007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f67f3d-e6cf-4994-8147-43a73c09f512 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:52.787373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f67f3d-e6cf-4994-8147-43a73c09f512 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:52.787727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f67f3d-e6cf-4994-8147-43a73c09f512 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:52.788462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f67f3d-e6cf-4994-8147-43a73c09f512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:52.791608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f67f3d-e6cf-4994-8147-43a73c09f512 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:52.843130 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2276/4478] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:04:52 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:04:52.846164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23349a9c-9943-49c8-be45-d0f240f2ddfa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:52.860374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23349a9c-9943-49c8-be45-d0f240f2ddfa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:52.868142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bd164-1f46-4367-9268-e8e18c87137a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:52.869546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bd164-1f46-4367-9268-e8e18c87137a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:52.870110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bd164-1f46-4367-9268-e8e18c87137a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:52.870645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bd164-1f46-4367-9268-e8e18c87137a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:52.871993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53bd164-1f46-4367-9268-e8e18c87137a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:52.874225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53bd164-1f46-4367-9268-e8e18c87137a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:52.891424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f67f3d-e6cf-4994-8147-43a73c09f512 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:52.899412 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2203/4479] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:52.906911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f67f3d-e6cf-4994-8147-43a73c09f512 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:52.912857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f35fa-3cdb-4faf-abc4-9ff5840101a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:52.914486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f35fa-3cdb-4faf-abc4-9ff5840101a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:52.914991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f35fa-3cdb-4faf-abc4-9ff5840101a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:52.915457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f35fa-3cdb-4faf-abc4-9ff5840101a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:52.916424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f35fa-3cdb-4faf-abc4-9ff5840101a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:52.918614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f35fa-3cdb-4faf-abc4-9ff5840101a0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:52.937258 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2277/4480] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:04:52.938693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bd164-1f46-4367-9268-e8e18c87137a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:52.953122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bd164-1f46-4367-9268-e8e18c87137a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:52.968651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80d2fb-acb9-4684-accf-7eface988b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:52.979796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80d2fb-acb9-4684-accf-7eface988b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:52.982282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80d2fb-acb9-4684-accf-7eface988b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:52.982735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80d2fb-acb9-4684-accf-7eface988b20 admin admin] PATH_INFO: `/v3/projects/7c6b7140c9474bf5b0f1e6d1618a7937/users/9995a4ecc0ec4f97a46ea58fa158ee61/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:53.041103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80d2fb-acb9-4684-accf-7eface988b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c6b7140c9474bf5b0f1e6d1618a7937, user_id=9995a4ecc0ec4f97a46ea58fa158ee61, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:53.041103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80d2fb-acb9-4684-accf-7eface988b20 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:53.063806 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2204/4481] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:04:52 2022] PUT /identity/v3/projects/7c6b7140c9474bf5b0f1e6d1618a7937/users/9995a4ecc0ec4f97a46ea58fa158ee61/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:04:53.074561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50570367-c0d5-47fa-bc1e-600cbf69a628 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:53.074878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50570367-c0d5-47fa-bc1e-600cbf69a628 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:53.075181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50570367-c0d5-47fa-bc1e-600cbf69a628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:53.149592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-50570367-c0d5-47fa-bc1e-600cbf69a628 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:04:53.150941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-50570367-c0d5-47fa-bc1e-600cbf69a628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:53.295276 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2278/4482] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:04:53.346728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80d2fb-acb9-4684-accf-7eface988b20 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:53.349568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9b80d2fb-acb9-4684-accf-7eface988b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:53.444615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80d2fb-acb9-4684-accf-7eface988b20 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:53.448621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f616950-3044-4319-95fd-1e1181d2a851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:53.449288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f616950-3044-4319-95fd-1e1181d2a851 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:53.449567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f616950-3044-4319-95fd-1e1181d2a851 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:53.449834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f616950-3044-4319-95fd-1e1181d2a851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:53.460164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f616950-3044-4319-95fd-1e1181d2a851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:53.462014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f616950-3044-4319-95fd-1e1181d2a851 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:53.473142 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2205/4483] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:04:59.631916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50570367-c0d5-47fa-bc1e-600cbf69a628 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:04:59.634246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-50570367-c0d5-47fa-bc1e-600cbf69a628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:59.651389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:04:59.804684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50570367-c0d5-47fa-bc1e-600cbf69a628 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:04:59.812971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be708a51-c34b-49c9-bf43-6c8afaa3cde2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:04:59.813863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be708a51-c34b-49c9-bf43-6c8afaa3cde2 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:04:59.814322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be708a51-c34b-49c9-bf43-6c8afaa3cde2 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:04:59.814804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be708a51-c34b-49c9-bf43-6c8afaa3cde2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:04:59.817618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-be708a51-c34b-49c9-bf43-6c8afaa3cde2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:04:59.864599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be708a51-c34b-49c9-bf43-6c8afaa3cde2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:04:59.866492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be708a51-c34b-49c9-bf43-6c8afaa3cde2 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:04:59.895956 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2279/4484] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:04:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:03.235064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f616950-3044-4319-95fd-1e1181d2a851 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:03.236659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0f616950-3044-4319-95fd-1e1181d2a851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:03.254123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:03.259857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f616950-3044-4319-95fd-1e1181d2a851 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:03.298484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:03.299516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:03.300134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:03.300643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] PATH_INFO: `/v3/users/1d03a8c35f2b4da39fac98b75ce9d810` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:03.302289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d03a8c35f2b4da39fac98b75ce9d810)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:03.311424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:03.416581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d03a8c35f2b4da39fac98b75ce9d810'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:03.435552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d03a8c35f2b4da39fac98b75ce9d810'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:03.436705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d03a8c35f2b4da39fac98b75ce9d810'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:03.446940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d03a8c35f2b4da39fac98b75ce9d810'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:03.466977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d03a8c35f2b4da39fac98b75ce9d810'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:03.490909 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2206/4485] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:05:03 2022] DELETE /identity/v3/users/1d03a8c35f2b4da39fac98b75ce9d810 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:03.493109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c31e0-8582-4fe0-9044-109119416511 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:03.493608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c31e0-8582-4fe0-9044-109119416511 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:03.493887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c31e0-8582-4fe0-9044-109119416511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:03.541244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2a0c31e0-8582-4fe0-9044-109119416511 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:03.542680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2a0c31e0-8582-4fe0-9044-109119416511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:03.675954 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2280/4486] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:03.742372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:03.745399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:03.877494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57f679-0ffa-42d5-9248-cb67fc2a424c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:03.877494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b572d41-7787-4803-a19f-fe6d000e74d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:03.877494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b572d41-7787-4803-a19f-fe6d000e74d6 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:03.877494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b572d41-7787-4803-a19f-fe6d000e74d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:03.877863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b572d41-7787-4803-a19f-fe6d000e74d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:03.902849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b572d41-7787-4803-a19f-fe6d000e74d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:03.904538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c31e0-8582-4fe0-9044-109119416511 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:03.905160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b572d41-7787-4803-a19f-fe6d000e74d6 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:03.912766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2a0c31e0-8582-4fe0-9044-109119416511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:03.922666 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2207/4487] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:03.934678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c31e0-8582-4fe0-9044-109119416511 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:03.939091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:03.939803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:03.940160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:03.940450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] PATH_INFO: `/v3/users/52bf5760e9ef43379e60159d3d2ed744` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:03.941761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52bf5760e9ef43379e60159d3d2ed744)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:03.943793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:04.018978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52bf5760e9ef43379e60159d3d2ed744'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:04.036524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52bf5760e9ef43379e60159d3d2ed744'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:04.048763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52bf5760e9ef43379e60159d3d2ed744'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:04.049715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52bf5760e9ef43379e60159d3d2ed744'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:04.051525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52bf5760e9ef43379e60159d3d2ed744'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:04.072777 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2281/4488] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:05:03 2022] DELETE /identity/v3/users/52bf5760e9ef43379e60159d3d2ed744 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:04.074601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646cdab-8ac2-43fc-bd10-28da99781867 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:04.074907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646cdab-8ac2-43fc-bd10-28da99781867 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:04.075192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646cdab-8ac2-43fc-bd10-28da99781867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:04.152116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3646cdab-8ac2-43fc-bd10-28da99781867 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:04.153610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3646cdab-8ac2-43fc-bd10-28da99781867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:04.311573 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2208/4489] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:04.325789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:04.328531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:04.358249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99c768-1cf9-43e6-90e3-df1573d2b16e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:04.358987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99c768-1cf9-43e6-90e3-df1573d2b16e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:04.359511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99c768-1cf9-43e6-90e3-df1573d2b16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:04.377652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:04.477960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cd99c768-1cf9-43e6-90e3-df1573d2b16e None None] MFA Rules not processed for user `9995a4ecc0ec4f97a46ea58fa158ee61`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:04.485069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cd99c768-1cf9-43e6-90e3-df1573d2b16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:04.528230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:04.594065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96133cd2-f979-43f5-8c5b-7edbb769e476 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:04.607743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d530ac4-ee6f-4163-989c-eb57e180dad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:04.634606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d530ac4-ee6f-4163-989c-eb57e180dad6 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:04.637166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d530ac4-ee6f-4163-989c-eb57e180dad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:04.638594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d530ac4-ee6f-4163-989c-eb57e180dad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:04.695985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d530ac4-ee6f-4163-989c-eb57e180dad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:04.698075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d530ac4-ee6f-4163-989c-eb57e180dad6 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:04.731193 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2282/4490] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:04.732059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d530ac4-ee6f-4163-989c-eb57e180dad6 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:04.734229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7d530ac4-ee6f-4163-989c-eb57e180dad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:04.755639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d530ac4-ee6f-4163-989c-eb57e180dad6 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:04.761621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7918a4-ea87-420a-8672-14acbeda1264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:04.762713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7918a4-ea87-420a-8672-14acbeda1264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:04.763685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7918a4-ea87-420a-8672-14acbeda1264 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:04.764209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7918a4-ea87-420a-8672-14acbeda1264 admin admin] PATH_INFO: `/v3/projects/87d300f145fd4855a6b3382997d975a9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:04.766161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7918a4-ea87-420a-8672-14acbeda1264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d300f145fd4855a6b3382997d975a9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:04.769789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7918a4-ea87-420a-8672-14acbeda1264 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:04.790556 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2209/4491] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:04.803541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8a217-e597-471a-a014-56919a90bdb0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:04.803727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8a217-e597-471a-a014-56919a90bdb0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:04.810165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8a217-e597-471a-a014-56919a90bdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:04.877186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-31d8a217-e597-471a-a014-56919a90bdb0 None None] MFA Rules not processed for user `9995a4ecc0ec4f97a46ea58fa158ee61`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:04.880993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-31d8a217-e597-471a-a014-56919a90bdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:04.887779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f7918a4-ea87-420a-8672-14acbeda1264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d300f145fd4855a6b3382997d975a9', 'request_id': 'req-2f7918a4-ea87-420a-8672-14acbeda1264'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:04.909927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f7918a4-ea87-420a-8672-14acbeda1264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d300f145fd4855a6b3382997d975a9', 'request_id': 'req-2f7918a4-ea87-420a-8672-14acbeda1264'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:04.923712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f7918a4-ea87-420a-8672-14acbeda1264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d300f145fd4855a6b3382997d975a9', 'request_id': 'req-2f7918a4-ea87-420a-8672-14acbeda1264'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:04.957517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f7918a4-ea87-420a-8672-14acbeda1264 admin admin] The token cache is being invalidate because project 87d300f145fd4855a6b3382997d975a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:05:04.957824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f7918a4-ea87-420a-8672-14acbeda1264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:04.960546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f7918a4-ea87-420a-8672-14acbeda1264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:04.966710 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2283/4492] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:05:04 2022] DELETE /identity/v3/projects/87d300f145fd4855a6b3382997d975a9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:04.978195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64607e67-c982-4f44-83f0-89bce7e19cfa None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:04.978547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64607e67-c982-4f44-83f0-89bce7e19cfa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:04.978777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64607e67-c982-4f44-83f0-89bce7e19cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.040313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-64607e67-c982-4f44-83f0-89bce7e19cfa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:05.040735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-64607e67-c982-4f44-83f0-89bce7e19cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:05.084059 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2210/4493] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:05.104452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8a217-e597-471a-a014-56919a90bdb0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:05.104452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-31d8a217-e597-471a-a014-56919a90bdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:05.200542 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2284/4494] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:05.208214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715f75a-8daf-4bb9-8e4d-555b001db20d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.208786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715f75a-8daf-4bb9-8e4d-555b001db20d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.209183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715f75a-8daf-4bb9-8e4d-555b001db20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.243083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d715f75a-8daf-4bb9-8e4d-555b001db20d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:05.245439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d715f75a-8daf-4bb9-8e4d-555b001db20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:05.251472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8a217-e597-471a-a014-56919a90bdb0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:05.254864 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2285/4495] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:05:05.259216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715f75a-8daf-4bb9-8e4d-555b001db20d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:05.259412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365d8d5-4e45-4ee9-83c4-5a95da45e1a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:05.259977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365d8d5-4e45-4ee9-83c4-5a95da45e1a3 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.260475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365d8d5-4e45-4ee9-83c4-5a95da45e1a3 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.260752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365d8d5-4e45-4ee9-83c4-5a95da45e1a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.262844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1365d8d5-4e45-4ee9-83c4-5a95da45e1a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:05.274455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715f75a-8daf-4bb9-8e4d-555b001db20d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:05.279488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76707972-727e-463b-8073-c5c84bd5ff6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:05.280209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76707972-727e-463b-8073-c5c84bd5ff6d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.280556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76707972-727e-463b-8073-c5c84bd5ff6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.281006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76707972-727e-463b-8073-c5c84bd5ff6d admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.281787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365d8d5-4e45-4ee9-83c4-5a95da45e1a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:05.281933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76707972-727e-463b-8073-c5c84bd5ff6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:05:05.282312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76707972-727e-463b-8073-c5c84bd5ff6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:05.284005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365d8d5-4e45-4ee9-83c4-5a95da45e1a3 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:05.286673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76707972-727e-463b-8073-c5c84bd5ff6d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:05.295456 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2211/4496] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:05 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:05.296739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365d8d5-4e45-4ee9-83c4-5a95da45e1a3 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:05.298443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1365d8d5-4e45-4ee9-83c4-5a95da45e1a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:05.303594 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2286/4497] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:05:05.308186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76707972-727e-463b-8073-c5c84bd5ff6d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:05.314239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365d8d5-4e45-4ee9-83c4-5a95da45e1a3 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:05.321855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76707972-727e-463b-8073-c5c84bd5ff6d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:05.322129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723414d-2b5d-4768-a24a-2cbfe581e7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:05.322959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723414d-2b5d-4768-a24a-2cbfe581e7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.323509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723414d-2b5d-4768-a24a-2cbfe581e7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.323977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723414d-2b5d-4768-a24a-2cbfe581e7ae admin admin] PATH_INFO: `/v3/projects/30db100f9ded4cd48da8e5f7b67fe09c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.325360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723414d-2b5d-4768-a24a-2cbfe581e7ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=30db100f9ded4cd48da8e5f7b67fe09c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:05.327762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723414d-2b5d-4768-a24a-2cbfe581e7ae admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:05.328224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c7a37-f982-44ef-9a47-baf26bd9fb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:05.328990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c7a37-f982-44ef-9a47-baf26bd9fb01 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.329351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c7a37-f982-44ef-9a47-baf26bd9fb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.329701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c7a37-f982-44ef-9a47-baf26bd9fb01 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.330761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c7a37-f982-44ef-9a47-baf26bd9fb01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:05.333306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c7a37-f982-44ef-9a47-baf26bd9fb01 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:05.357013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:05.381099 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2287/4498] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:05:05 2022] POST /identity/v3/projects => generated 403 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:05:05.383781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:05.384456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c7a37-f982-44ef-9a47-baf26bd9fb01 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:05.398588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a723414d-2b5d-4768-a24a-2cbfe581e7ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30db100f9ded4cd48da8e5f7b67fe09c', 'request_id': 'req-a723414d-2b5d-4768-a24a-2cbfe581e7ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:05.409805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a723414d-2b5d-4768-a24a-2cbfe581e7ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30db100f9ded4cd48da8e5f7b67fe09c', 'request_id': 'req-a723414d-2b5d-4768-a24a-2cbfe581e7ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:05.425232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a723414d-2b5d-4768-a24a-2cbfe581e7ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30db100f9ded4cd48da8e5f7b67fe09c', 'request_id': 'req-a723414d-2b5d-4768-a24a-2cbfe581e7ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:05.444109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a723414d-2b5d-4768-a24a-2cbfe581e7ae admin admin] The token cache is being invalidate because project 30db100f9ded4cd48da8e5f7b67fe09c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:05:05.445316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a723414d-2b5d-4768-a24a-2cbfe581e7ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:05.446583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a723414d-2b5d-4768-a24a-2cbfe581e7ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:05.448690 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2212/4499] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:05:05 2022] DELETE /identity/v3/projects/30db100f9ded4cd48da8e5f7b67fe09c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:05.453864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53c217-9589-48a4-985f-bc9a0f6d9db4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.455049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53c217-9589-48a4-985f-bc9a0f6d9db4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.455527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53c217-9589-48a4-985f-bc9a0f6d9db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.476973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c7a37-f982-44ef-9a47-baf26bd9fb01 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:05.486594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cf53c217-9589-48a4-985f-bc9a0f6d9db4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:05.489033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cf53c217-9589-48a4-985f-bc9a0f6d9db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:05.564426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63cabb-95ae-4da3-bf8a-2cdca9293513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:05.566142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63cabb-95ae-4da3-bf8a-2cdca9293513 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.567577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63cabb-95ae-4da3-bf8a-2cdca9293513 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.568362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63cabb-95ae-4da3-bf8a-2cdca9293513 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.571707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63cabb-95ae-4da3-bf8a-2cdca9293513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:05.575299 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2213/4500] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:05:05.576650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63cabb-95ae-4da3-bf8a-2cdca9293513 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:05.582632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ee304-a092-4e37-9463-9da04263a7c9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.583288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ee304-a092-4e37-9463-9da04263a7c9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.583823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ee304-a092-4e37-9463-9da04263a7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.615354 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2288/4501] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:05:05 2022] POST /identity/v3/users => generated 302 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:05:05.616366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-142ee304-a092-4e37-9463-9da04263a7c9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:05.618009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-142ee304-a092-4e37-9463-9da04263a7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:05.619895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63cabb-95ae-4da3-bf8a-2cdca9293513 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:05.621720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cd63cabb-95ae-4da3-bf8a-2cdca9293513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:05.630293 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2214/4502] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:05.634661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ee304-a092-4e37-9463-9da04263a7c9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:05.644170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ee304-a092-4e37-9463-9da04263a7c9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:05.647720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63cabb-95ae-4da3-bf8a-2cdca9293513 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:05.649837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a615a-6848-4525-a804-7ed9d3a3d1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:05.650453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a615a-6848-4525-a804-7ed9d3a3d1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.650736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a615a-6848-4525-a804-7ed9d3a3d1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.651029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a615a-6848-4525-a804-7ed9d3a3d1d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.651658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a615a-6848-4525-a804-7ed9d3a3d1d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:05:05.651962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a615a-6848-4525-a804-7ed9d3a3d1d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:05.653621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a615a-6848-4525-a804-7ed9d3a3d1d2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:05.657579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf2fcd-e4ea-44c3-841c-eacbacd86c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:05.660725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf2fcd-e4ea-44c3-841c-eacbacd86c93 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.661263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf2fcd-e4ea-44c3-841c-eacbacd86c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.661897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf2fcd-e4ea-44c3-841c-eacbacd86c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.663489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf2fcd-e4ea-44c3-841c-eacbacd86c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:05.665195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf2fcd-e4ea-44c3-841c-eacbacd86c93 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:05.672236 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2215/4503] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:05:05.676160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a615a-6848-4525-a804-7ed9d3a3d1d2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:05.678747 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2289/4504] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:05.682653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf2fcd-e4ea-44c3-841c-eacbacd86c93 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:05.692383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a615a-6848-4525-a804-7ed9d3a3d1d2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:05.693328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf2fcd-e4ea-44c3-841c-eacbacd86c93 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:05.700074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ab14c-b1ed-42cf-809b-beb10e43e416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:05.700694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ab14c-b1ed-42cf-809b-beb10e43e416 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.701014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ab14c-b1ed-42cf-809b-beb10e43e416 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.701291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ab14c-b1ed-42cf-809b-beb10e43e416 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.701928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ab14c-b1ed-42cf-809b-beb10e43e416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:05.703847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ab14c-b1ed-42cf-809b-beb10e43e416 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:05.708632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc39c34-9e9b-41a6-a4e5-bca98da4fd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:05.712930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc39c34-9e9b-41a6-a4e5-bca98da4fd87 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.713217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc39c34-9e9b-41a6-a4e5-bca98da4fd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.713677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc39c34-9e9b-41a6-a4e5-bca98da4fd87 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.715276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc39c34-9e9b-41a6-a4e5-bca98da4fd87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:05.716717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc39c34-9e9b-41a6-a4e5-bca98da4fd87 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:05.747876 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2290/4505] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:05.752047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc39c34-9e9b-41a6-a4e5-bca98da4fd87 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:05.763050 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2216/4506] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:05:05 2022] POST /identity/v3/projects => generated 397 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:05:05.763050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ab14c-b1ed-42cf-809b-beb10e43e416 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:05.765860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc39c34-9e9b-41a6-a4e5-bca98da4fd87 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:05.863033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ab14c-b1ed-42cf-809b-beb10e43e416 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:05.877154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e127be-000e-4f9e-9787-85b1a6699e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:05.877663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8df56b-c97f-4f9e-b989-14b1d94ed398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:05.878526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e127be-000e-4f9e-9787-85b1a6699e07 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.878681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8df56b-c97f-4f9e-b989-14b1d94ed398 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.878681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8df56b-c97f-4f9e-b989-14b1d94ed398 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.878914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8df56b-c97f-4f9e-b989-14b1d94ed398 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.879588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e127be-000e-4f9e-9787-85b1a6699e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.879726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8df56b-c97f-4f9e-b989-14b1d94ed398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:05.880773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e127be-000e-4f9e-9787-85b1a6699e07 admin admin] PATH_INFO: `/v3/projects/05840665f0c344969819bbb69e666cba/users/01504da61db5499cbafc22fb3ebd448b/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:05.883063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8df56b-c97f-4f9e-b989-14b1d94ed398 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:05.901543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e127be-000e-4f9e-9787-85b1a6699e07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05840665f0c344969819bbb69e666cba, user_id=01504da61db5499cbafc22fb3ebd448b, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:05.905465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e127be-000e-4f9e-9787-85b1a6699e07 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:05.908689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:05.930326 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2217/4507] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:05:05 2022] POST /identity/v3/users => generated 299 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:05.934847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8df56b-c97f-4f9e-b989-14b1d94ed398 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:05.946392 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2291/4508] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:05 2022] PUT /identity/v3/projects/05840665f0c344969819bbb69e666cba/users/01504da61db5499cbafc22fb3ebd448b/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:05:05.956715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8df56b-c97f-4f9e-b989-14b1d94ed398 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:05.965727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866aae8-9878-46d8-8a56-d2bb1ffcaff0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:05.966438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:05.967709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866aae8-9878-46d8-8a56-d2bb1ffcaff0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:05.968765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866aae8-9878-46d8-8a56-d2bb1ffcaff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:06.039678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33233c7b-291a-4036-a296-684f78af370f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:06.040231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33233c7b-291a-4036-a296-684f78af370f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:06.040521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33233c7b-291a-4036-a296-684f78af370f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:06.040804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33233c7b-291a-4036-a296-684f78af370f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:06.041485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33233c7b-291a-4036-a296-684f78af370f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:06.044467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33233c7b-291a-4036-a296-684f78af370f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:06.053478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e866aae8-9878-46d8-8a56-d2bb1ffcaff0 None None] MFA Rules not processed for user `01504da61db5499cbafc22fb3ebd448b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:06.060819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e866aae8-9878-46d8-8a56-d2bb1ffcaff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:06.065547 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2218/4509] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:06.067601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33233c7b-291a-4036-a296-684f78af370f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:06.089040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33233c7b-291a-4036-a296-684f78af370f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:06.094923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf54551-c8ea-4973-9aad-39184ee09294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:06.095616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf54551-c8ea-4973-9aad-39184ee09294 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:06.095942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf54551-c8ea-4973-9aad-39184ee09294 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:06.096232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf54551-c8ea-4973-9aad-39184ee09294 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:06.096886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf54551-c8ea-4973-9aad-39184ee09294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:06.099209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf54551-c8ea-4973-9aad-39184ee09294 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:06.118117 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2219/4510] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:06.119805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf54551-c8ea-4973-9aad-39184ee09294 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:06.146423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf54551-c8ea-4973-9aad-39184ee09294 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:06.161154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc36fcc-5cfe-42a7-9ade-f3a6ff9ca8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:06.161845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc36fcc-5cfe-42a7-9ade-f3a6ff9ca8be admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:06.162137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc36fcc-5cfe-42a7-9ade-f3a6ff9ca8be admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:06.162421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc36fcc-5cfe-42a7-9ade-f3a6ff9ca8be admin admin] PATH_INFO: `/v3/projects/94e570d13568401c8fe5191706d80a75/users/43d940c1853e4d4aa86f6f4c61d95109/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:06.224039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc36fcc-5cfe-42a7-9ade-f3a6ff9ca8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e570d13568401c8fe5191706d80a75, user_id=43d940c1853e4d4aa86f6f4c61d95109, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:06.229473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc36fcc-5cfe-42a7-9ade-f3a6ff9ca8be admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:06.253170 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2292/4511] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:06.257650 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2220/4512] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:06 2022] PUT /identity/v3/projects/94e570d13568401c8fe5191706d80a75/users/43d940c1853e4d4aa86f6f4c61d95109/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:06.259623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866aae8-9878-46d8-8a56-d2bb1ffcaff0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:06.270789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a77cc-178f-4b8c-8f69-4c87ea352fbb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:06.272165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a77cc-178f-4b8c-8f69-4c87ea352fbb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:06.288220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866aae8-9878-46d8-8a56-d2bb1ffcaff0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:06.297646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a77cc-178f-4b8c-8f69-4c87ea352fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:06.369801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-305a77cc-178f-4b8c-8f69-4c87ea352fbb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:06.371343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-305a77cc-178f-4b8c-8f69-4c87ea352fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:06.428746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9194b-f9cf-48ee-bd37-6b3c6e18b6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:06.430188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9194b-f9cf-48ee-bd37-6b3c6e18b6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:06.430535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9194b-f9cf-48ee-bd37-6b3c6e18b6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:06.430824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9194b-f9cf-48ee-bd37-6b3c6e18b6e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:06.431471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9194b-f9cf-48ee-bd37-6b3c6e18b6e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:06.433766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9194b-f9cf-48ee-bd37-6b3c6e18b6e6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:06.519893 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2293/4513] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:05:06 2022] POST /identity/v3/projects => generated 403 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:06.520783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9194b-f9cf-48ee-bd37-6b3c6e18b6e6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:06.565630 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2221/4514] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:06.588348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a77cc-178f-4b8c-8f69-4c87ea352fbb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:06.590195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-305a77cc-178f-4b8c-8f69-4c87ea352fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:06.624039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9194b-f9cf-48ee-bd37-6b3c6e18b6e6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:06.630397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc3557-e095-4d28-b313-b02252f07628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:06.631024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc3557-e095-4d28-b313-b02252f07628 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:06.631308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc3557-e095-4d28-b313-b02252f07628 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:06.631581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc3557-e095-4d28-b313-b02252f07628 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:06.632426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc3557-e095-4d28-b313-b02252f07628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:06.634345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc3557-e095-4d28-b313-b02252f07628 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:06.666531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:06.678904 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2294/4515] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:05:06 2022] POST /identity/v3/users => generated 308 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:06.681354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc3557-e095-4d28-b313-b02252f07628 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:06.694686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc3557-e095-4d28-b313-b02252f07628 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:06.701114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031584f4-1cec-4d61-817d-9819631975ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:06.701955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031584f4-1cec-4d61-817d-9819631975ea admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:06.702543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031584f4-1cec-4d61-817d-9819631975ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:06.703017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031584f4-1cec-4d61-817d-9819631975ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:06.703843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031584f4-1cec-4d61-817d-9819631975ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:06.706127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031584f4-1cec-4d61-817d-9819631975ea admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:06.719870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a77cc-178f-4b8c-8f69-4c87ea352fbb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:06.740834 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2295/4516] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:06.745667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031584f4-1cec-4d61-817d-9819631975ea admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:06.764757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031584f4-1cec-4d61-817d-9819631975ea admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:06.780247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b5514-aa2b-4634-9814-1f6cf900226e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:06.780986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b5514-aa2b-4634-9814-1f6cf900226e admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:06.781280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b5514-aa2b-4634-9814-1f6cf900226e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:06.781564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b5514-aa2b-4634-9814-1f6cf900226e admin admin] PATH_INFO: `/v3/projects/46ef16f8d91e464a9d6cab4755edc684/users/8d78ff4b62364dafb8140a94757e7fa5/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:06.783264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:06.808447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b5514-aa2b-4634-9814-1f6cf900226e admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ef16f8d91e464a9d6cab4755edc684, user_id=8d78ff4b62364dafb8140a94757e7fa5, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:06.810823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b5514-aa2b-4634-9814-1f6cf900226e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:06.821391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12686d-bf2d-4db7-94b5-218d328efbf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:06.824390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12686d-bf2d-4db7-94b5-218d328efbf6 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:06.825138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12686d-bf2d-4db7-94b5-218d328efbf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:06.825875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12686d-bf2d-4db7-94b5-218d328efbf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:06.832117 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2296/4517] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:06 2022] PUT /identity/v3/projects/46ef16f8d91e464a9d6cab4755edc684/users/8d78ff4b62364dafb8140a94757e7fa5/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:05:06.836608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b5514-aa2b-4634-9814-1f6cf900226e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:06.843945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12686d-bf2d-4db7-94b5-218d328efbf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:06.846916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12686d-bf2d-4db7-94b5-218d328efbf6 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:06.879002 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2222/4518] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:06.914286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b5514-aa2b-4634-9814-1f6cf900226e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:06.927739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bceff-7998-4733-92a0-a4372adad5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:06.931307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bceff-7998-4733-92a0-a4372adad5fe admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:06.931683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bceff-7998-4733-92a0-a4372adad5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:06.932061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bceff-7998-4733-92a0-a4372adad5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:06.933004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bceff-7998-4733-92a0-a4372adad5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:06.934831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bceff-7998-4733-92a0-a4372adad5fe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:06.949614 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2297/4519] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:06.950826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12686d-bf2d-4db7-94b5-218d328efbf6 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:06.965400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12686d-bf2d-4db7-94b5-218d328efbf6 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:06.973671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c6278-ed3f-4e98-b869-39fb4c1adba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:06.974332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c6278-ed3f-4e98-b869-39fb4c1adba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:06.974606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c6278-ed3f-4e98-b869-39fb4c1adba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:06.975117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c6278-ed3f-4e98-b869-39fb4c1adba8 admin admin] PATH_INFO: `/v3/projects/46ef16f8d91e464a9d6cab4755edc684/users/8d78ff4b62364dafb8140a94757e7fa5/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:06.977187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c6278-ed3f-4e98-b869-39fb4c1adba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ef16f8d91e464a9d6cab4755edc684, user_id=8d78ff4b62364dafb8140a94757e7fa5, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:06.979870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c6278-ed3f-4e98-b869-39fb4c1adba8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:07.003519 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2223/4520] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:06 2022] PUT /identity/v3/projects/46ef16f8d91e464a9d6cab4755edc684/users/8d78ff4b62364dafb8140a94757e7fa5/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:07.006992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc86c2a-8f3a-42ca-897e-e315788cc261 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:07.007867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc86c2a-8f3a-42ca-897e-e315788cc261 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:07.008220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc86c2a-8f3a-42ca-897e-e315788cc261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:07.052330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ddc86c2a-8f3a-42ca-897e-e315788cc261 None None] MFA Rules not processed for user `8d78ff4b62364dafb8140a94757e7fa5`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:07.053858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ddc86c2a-8f3a-42ca-897e-e315788cc261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:07.159004 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2298/4521] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:07.172785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c6278-ed3f-4e98-b869-39fb4c1adba8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:07.175408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a02c6278-ed3f-4e98-b869-39fb4c1adba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:07.268979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c6278-ed3f-4e98-b869-39fb4c1adba8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:07.277633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce173b16-f954-4ec8-8685-f5ec84635716 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:07.279207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce173b16-f954-4ec8-8685-f5ec84635716 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:07.279207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce173b16-f954-4ec8-8685-f5ec84635716 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:07.279455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce173b16-f954-4ec8-8685-f5ec84635716 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:07.298109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce173b16-f954-4ec8-8685-f5ec84635716 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:07.299804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce173b16-f954-4ec8-8685-f5ec84635716 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:07.311992 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2224/4522] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:07 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:07.425443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc86c2a-8f3a-42ca-897e-e315788cc261 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:07.544566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:07.605706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc86c2a-8f3a-42ca-897e-e315788cc261 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:07.611190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:07.613405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:07.613795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:07.614175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] PATH_INFO: `/v3/users/01504da61db5499cbafc22fb3ebd448b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:07.616262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01504da61db5499cbafc22fb3ebd448b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:07.620581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:07.743655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01504da61db5499cbafc22fb3ebd448b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:07.759811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01504da61db5499cbafc22fb3ebd448b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:07.771158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01504da61db5499cbafc22fb3ebd448b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:07.772525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01504da61db5499cbafc22fb3ebd448b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:07.773597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01504da61db5499cbafc22fb3ebd448b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:07.804989 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2299/4523] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:05:07 2022] DELETE /identity/v3/users/01504da61db5499cbafc22fb3ebd448b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:07.809349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec975a-4ac1-43bf-92f4-6e9e74855a95 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:07.809907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec975a-4ac1-43bf-92f4-6e9e74855a95 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:07.809907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec975a-4ac1-43bf-92f4-6e9e74855a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:07.836444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-91ec975a-4ac1-43bf-92f4-6e9e74855a95 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:07.837735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-91ec975a-4ac1-43bf-92f4-6e9e74855a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:07.939550 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2225/4524] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:07.963992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:07.966123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:08.045518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5b485-e8c9-4ac0-8d75-ca2e6c42e4c0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:08.049965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29e084-4673-443f-ac64-8823bc50fe51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:08.050774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29e084-4673-443f-ac64-8823bc50fe51 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:08.051037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29e084-4673-443f-ac64-8823bc50fe51 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:08.051399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29e084-4673-443f-ac64-8823bc50fe51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:08.064300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29e084-4673-443f-ac64-8823bc50fe51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:08.065979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29e084-4673-443f-ac64-8823bc50fe51 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:08.077557 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2300/4525] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:09.465752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec975a-4ac1-43bf-92f4-6e9e74855a95 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:09.467313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-91ec975a-4ac1-43bf-92f4-6e9e74855a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:09.487511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec975a-4ac1-43bf-92f4-6e9e74855a95 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:09.518164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:09.519086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:09.519574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:09.520104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204 admin admin] PATH_INFO: `/v3/projects/05840665f0c344969819bbb69e666cba` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:09.522483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05840665f0c344969819bbb69e666cba)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:09.529096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:09.660218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05840665f0c344969819bbb69e666cba', 'request_id': 'req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:09.673784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05840665f0c344969819bbb69e666cba', 'request_id': 'req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:09.677644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:09.688919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05840665f0c344969819bbb69e666cba', 'request_id': 'req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:09.718389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204 admin admin] The token cache is being invalidate because project 05840665f0c344969819bbb69e666cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:05:09.718749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:09.719863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa3f3bfb-7679-4acb-8cd2-f1a4ed7aa204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:09.721650 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2226/4526] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:05:09 2022] DELETE /identity/v3/projects/05840665f0c344969819bbb69e666cba => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:09.737408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29e084-4673-443f-ac64-8823bc50fe51 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:09.759053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29e084-4673-443f-ac64-8823bc50fe51 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:09.881717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:09.882556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:09.882984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:09.883399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] PATH_INFO: `/v3/users/8d78ff4b62364dafb8140a94757e7fa5` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:09.884966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d78ff4b62364dafb8140a94757e7fa5)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:09.886900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:09.893413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:09.967342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d78ff4b62364dafb8140a94757e7fa5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:09.984757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d78ff4b62364dafb8140a94757e7fa5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:09.999950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d78ff4b62364dafb8140a94757e7fa5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:10.001471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d78ff4b62364dafb8140a94757e7fa5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:10.002903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d78ff4b62364dafb8140a94757e7fa5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:10.023058 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2301/4527] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:05:09 2022] DELETE /identity/v3/users/8d78ff4b62364dafb8140a94757e7fa5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:10.349226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21277a-3cc6-4dbc-9a25-ea42086b8568 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:10.349806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21277a-3cc6-4dbc-9a25-ea42086b8568 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:10.350090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21277a-3cc6-4dbc-9a25-ea42086b8568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:10.395817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1a21277a-3cc6-4dbc-9a25-ea42086b8568 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:10.397122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1a21277a-3cc6-4dbc-9a25-ea42086b8568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:10.519474 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2227/4528] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:10.540969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:10.540969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:10.644933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41840021-38c4-47c6-b906-409950c743e1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:10.651776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005f8dc-e6a6-45da-af9e-c59ca0c9eac4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:10.652686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005f8dc-e6a6-45da-af9e-c59ca0c9eac4 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:10.653119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005f8dc-e6a6-45da-af9e-c59ca0c9eac4 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:10.654638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005f8dc-e6a6-45da-af9e-c59ca0c9eac4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:10.667558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005f8dc-e6a6-45da-af9e-c59ca0c9eac4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:10.669668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005f8dc-e6a6-45da-af9e-c59ca0c9eac4 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:10.683881 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2302/4529] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:05:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:05:11.478613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21277a-3cc6-4dbc-9a25-ea42086b8568 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:11.480069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1a21277a-3cc6-4dbc-9a25-ea42086b8568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:11.579186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21277a-3cc6-4dbc-9a25-ea42086b8568 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:11.587947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:11.589373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:11.590232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:11.591092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0 admin admin] PATH_INFO: `/v3/projects/46ef16f8d91e464a9d6cab4755edc684` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:11.594468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46ef16f8d91e464a9d6cab4755edc684)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:11.597070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:11.689573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46ef16f8d91e464a9d6cab4755edc684', 'request_id': 'req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:11.700172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46ef16f8d91e464a9d6cab4755edc684', 'request_id': 'req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:11.708290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46ef16f8d91e464a9d6cab4755edc684', 'request_id': 'req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:11.725502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0 admin admin] The token cache is being invalidate because project 46ef16f8d91e464a9d6cab4755edc684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:05:11.726438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:11.728470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-746bc36e-abc1-4254-bd5a-c1c6cd8eddf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:11.731562 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2228/4530] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:05:11 2022] DELETE /identity/v3/projects/46ef16f8d91e464a9d6cab4755edc684 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:11.738041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d713ae4-b2e1-43e5-9b88-68623d04364c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:11.739838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d713ae4-b2e1-43e5-9b88-68623d04364c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:11.740361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d713ae4-b2e1-43e5-9b88-68623d04364c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:11.771871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3d713ae4-b2e1-43e5-9b88-68623d04364c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:11.774794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3d713ae4-b2e1-43e5-9b88-68623d04364c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:11.865356 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2303/4531] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:11.866214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a8dba-475b-4c72-9473-4114b3a208e7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:11.867035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a8dba-475b-4c72-9473-4114b3a208e7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:11.867810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a8dba-475b-4c72-9473-4114b3a208e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:11.895427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-212a8dba-475b-4c72-9473-4114b3a208e7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:11.896978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-212a8dba-475b-4c72-9473-4114b3a208e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:11.906869 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2229/4532] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:11.910890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d713ae4-b2e1-43e5-9b88-68623d04364c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:11.934774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d713ae4-b2e1-43e5-9b88-68623d04364c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:11.941696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222fccf-2d21-483e-85ae-37e282a283b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:11.942757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222fccf-2d21-483e-85ae-37e282a283b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:11.943155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222fccf-2d21-483e-85ae-37e282a283b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:11.943576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222fccf-2d21-483e-85ae-37e282a283b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:11.944344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222fccf-2d21-483e-85ae-37e282a283b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:05:11.944862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222fccf-2d21-483e-85ae-37e282a283b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:11.947549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222fccf-2d21-483e-85ae-37e282a283b7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:11.960914 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2304/4533] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:05:11.965806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a8dba-475b-4c72-9473-4114b3a208e7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:11.975536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a8dba-475b-4c72-9473-4114b3a208e7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:11.981016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b379ccb-76f9-42dd-9770-fe6b387f915a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:11.981670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b379ccb-76f9-42dd-9770-fe6b387f915a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:11.981954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b379ccb-76f9-42dd-9770-fe6b387f915a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:11.982228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b379ccb-76f9-42dd-9770-fe6b387f915a admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:11.982875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b379ccb-76f9-42dd-9770-fe6b387f915a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:11.984787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b379ccb-76f9-42dd-9770-fe6b387f915a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:11.999340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:12.024942 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2230/4534] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:05:11 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:12.027698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222fccf-2d21-483e-85ae-37e282a283b7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:12.035420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:12.045218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222fccf-2d21-483e-85ae-37e282a283b7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:12.141038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cda9e-bda1-4460-ae6c-dd2a9e6fb996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:12.142142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cda9e-bda1-4460-ae6c-dd2a9e6fb996 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:12.142142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cda9e-bda1-4460-ae6c-dd2a9e6fb996 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:12.147895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cda9e-bda1-4460-ae6c-dd2a9e6fb996 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:12.154106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157cda9e-bda1-4460-ae6c-dd2a9e6fb996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:12.156429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157cda9e-bda1-4460-ae6c-dd2a9e6fb996 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:12.195527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b379ccb-76f9-42dd-9770-fe6b387f915a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:12.196736 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2305/4535] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:05:12 2022] POST /identity/v3/users => generated 283 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:12.223247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b379ccb-76f9-42dd-9770-fe6b387f915a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:12.244825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a553184-5c7a-41db-a993-9d72e7310ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:12.245719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a553184-5c7a-41db-a993-9d72e7310ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:12.246112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a553184-5c7a-41db-a993-9d72e7310ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:12.246493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a553184-5c7a-41db-a993-9d72e7310ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:12.248275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a553184-5c7a-41db-a993-9d72e7310ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:12.251564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a553184-5c7a-41db-a993-9d72e7310ae0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:12.296428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cda9e-bda1-4460-ae6c-dd2a9e6fb996 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:12.298223 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2231/4536] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:05:12.322956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cda9e-bda1-4460-ae6c-dd2a9e6fb996 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:12.343587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcadb0-8731-4a4a-a48a-cf27496a86f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:12.346369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcadb0-8731-4a4a-a48a-cf27496a86f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:12.346652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcadb0-8731-4a4a-a48a-cf27496a86f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:12.346925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcadb0-8731-4a4a-a48a-cf27496a86f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:12.347664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcadb0-8731-4a4a-a48a-cf27496a86f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:12.352156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcadb0-8731-4a4a-a48a-cf27496a86f8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:12.392541 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2306/4537] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:12.393913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a553184-5c7a-41db-a993-9d72e7310ae0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:12.417421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a553184-5c7a-41db-a993-9d72e7310ae0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:12.431921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a831d51-952e-4a7c-9d44-6dfca12af2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:12.432955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831d51-952e-4a7c-9d44-6dfca12af2bf admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:12.433536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831d51-952e-4a7c-9d44-6dfca12af2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:12.433994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831d51-952e-4a7c-9d44-6dfca12af2bf admin admin] PATH_INFO: `/v3/projects/5bd308cd63fd4e32b287b9b238f7ef0c/users/dc73808524f5489aa10dfbac50ae0426/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:12.470164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a831d51-952e-4a7c-9d44-6dfca12af2bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd308cd63fd4e32b287b9b238f7ef0c, user_id=dc73808524f5489aa10dfbac50ae0426, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:12.474487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a831d51-952e-4a7c-9d44-6dfca12af2bf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:12.516778 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2232/4538] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:12 2022] PUT /identity/v3/projects/5bd308cd63fd4e32b287b9b238f7ef0c/users/dc73808524f5489aa10dfbac50ae0426/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:12.519380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b97cf2-6a33-4209-bd48-cb7342db5e6f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:12.519560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b97cf2-6a33-4209-bd48-cb7342db5e6f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:12.520155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b97cf2-6a33-4209-bd48-cb7342db5e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:12.590268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-64b97cf2-6a33-4209-bd48-cb7342db5e6f None None] MFA Rules not processed for user `dc73808524f5489aa10dfbac50ae0426`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:12.593536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-64b97cf2-6a33-4209-bd48-cb7342db5e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:12.717099 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2307/4539] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:12.720931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a831d51-952e-4a7c-9d44-6dfca12af2bf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:12.805097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a831d51-952e-4a7c-9d44-6dfca12af2bf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:12.810729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58318a69-2b98-4f61-a548-a818bd2d3533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:12.811339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58318a69-2b98-4f61-a548-a818bd2d3533 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:12.812480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58318a69-2b98-4f61-a548-a818bd2d3533 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:12.812777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58318a69-2b98-4f61-a548-a818bd2d3533 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:12.814599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58318a69-2b98-4f61-a548-a818bd2d3533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:12.816810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58318a69-2b98-4f61-a548-a818bd2d3533 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:12.883084 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2233/4540] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:05:12 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:05:12.887238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b97cf2-6a33-4209-bd48-cb7342db5e6f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:12.906217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b97cf2-6a33-4209-bd48-cb7342db5e6f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:13.036769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad554dff-bbaa-43a4-b890-5d67cc747c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:13.038786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad554dff-bbaa-43a4-b890-5d67cc747c49 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:13.039384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad554dff-bbaa-43a4-b890-5d67cc747c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:13.040257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad554dff-bbaa-43a4-b890-5d67cc747c49 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:13.041365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad554dff-bbaa-43a4-b890-5d67cc747c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:13.044733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad554dff-bbaa-43a4-b890-5d67cc747c49 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:13.070985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:13.081161 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2308/4541] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:05:12 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:13.085075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58318a69-2b98-4f61-a548-a818bd2d3533 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:13.113593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58318a69-2b98-4f61-a548-a818bd2d3533 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:13.128565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbaf888-4592-454a-a4a6-012df1bfe673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:13.129793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaf888-4592-454a-a4a6-012df1bfe673 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:13.130672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaf888-4592-454a-a4a6-012df1bfe673 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:13.131303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaf888-4592-454a-a4a6-012df1bfe673 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:13.139240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbaf888-4592-454a-a4a6-012df1bfe673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:13.141446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbaf888-4592-454a-a4a6-012df1bfe673 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:13.156967 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2234/4542] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:05:13.160559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad554dff-bbaa-43a4-b890-5d67cc747c49 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:13.177640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad554dff-bbaa-43a4-b890-5d67cc747c49 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:13.217357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b824525-d840-4bc7-b76f-61d1b0d15ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:13.217357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b824525-d840-4bc7-b76f-61d1b0d15ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:13.217357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b824525-d840-4bc7-b76f-61d1b0d15ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:13.217357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b824525-d840-4bc7-b76f-61d1b0d15ce3 admin admin] PATH_INFO: `/v3/projects/b3ee3943796e49b29730e78e8523d998/users/4fb2e9371c824193b9cc382ba2a70ec2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:13.238361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2ba02-aafa-421e-a8aa-bd126a549ed0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:13.238661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2ba02-aafa-421e-a8aa-bd126a549ed0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:13.238959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2ba02-aafa-421e-a8aa-bd126a549ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:13.265281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b824525-d840-4bc7-b76f-61d1b0d15ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ee3943796e49b29730e78e8523d998, user_id=4fb2e9371c824193b9cc382ba2a70ec2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:13.268006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b824525-d840-4bc7-b76f-61d1b0d15ce3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:13.274884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-acb2ba02-aafa-421e-a8aa-bd126a549ed0 None None] MFA Rules not processed for user `c3187514ca644ff68c992526f38783c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:13.276150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-acb2ba02-aafa-421e-a8aa-bd126a549ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:13.296638 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2309/4543] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:13 2022] PUT /identity/v3/projects/b3ee3943796e49b29730e78e8523d998/users/4fb2e9371c824193b9cc382ba2a70ec2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:13.297859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b824525-d840-4bc7-b76f-61d1b0d15ce3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:13.407836 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2235/4544] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:05:13.480186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b824525-d840-4bc7-b76f-61d1b0d15ce3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:13.501617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf11b90-4a27-495f-8270-103205d03bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:13.502835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf11b90-4a27-495f-8270-103205d03bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:13.503127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf11b90-4a27-495f-8270-103205d03bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:13.504655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf11b90-4a27-495f-8270-103205d03bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:13.505949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf11b90-4a27-495f-8270-103205d03bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:13.508170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf11b90-4a27-495f-8270-103205d03bf3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:13.532284 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2310/4545] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:13.537568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2ba02-aafa-421e-a8aa-bd126a549ed0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:13.563113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2ba02-aafa-421e-a8aa-bd126a549ed0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:13.582842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86ae9b-59a4-4794-b79c-9054cc2c624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:13.584208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86ae9b-59a4-4794-b79c-9054cc2c624f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:13.584498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86ae9b-59a4-4794-b79c-9054cc2c624f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:13.584787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86ae9b-59a4-4794-b79c-9054cc2c624f admin admin] PATH_INFO: `/v3/projects/b3ee3943796e49b29730e78e8523d998/users/4fb2e9371c824193b9cc382ba2a70ec2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:13.587562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86ae9b-59a4-4794-b79c-9054cc2c624f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ee3943796e49b29730e78e8523d998, user_id=4fb2e9371c824193b9cc382ba2a70ec2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:13.591783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86ae9b-59a4-4794-b79c-9054cc2c624f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:13.614273 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2236/4546] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:13 2022] PUT /identity/v3/projects/b3ee3943796e49b29730e78e8523d998/users/4fb2e9371c824193b9cc382ba2a70ec2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:13.625103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c58ffc-fd56-41e3-a8e4-61aa08cef319 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:13.625425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c58ffc-fd56-41e3-a8e4-61aa08cef319 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:13.625712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c58ffc-fd56-41e3-a8e4-61aa08cef319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:13.661947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f4c58ffc-fd56-41e3-a8e4-61aa08cef319 None None] MFA Rules not processed for user `4fb2e9371c824193b9cc382ba2a70ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:13.665972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f4c58ffc-fd56-41e3-a8e4-61aa08cef319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:13.794956 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2311/4547] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:13.814469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86ae9b-59a4-4794-b79c-9054cc2c624f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:13.817494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8b86ae9b-59a4-4794-b79c-9054cc2c624f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:13.897528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:13.985088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86ae9b-59a4-4794-b79c-9054cc2c624f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:13.989278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb027e-5913-414f-b715-335a6f6765a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:13.989986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb027e-5913-414f-b715-335a6f6765a0 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:13.990381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb027e-5913-414f-b715-335a6f6765a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:13.990767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb027e-5913-414f-b715-335a6f6765a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:14.008403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb027e-5913-414f-b715-335a6f6765a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:14.009942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb027e-5913-414f-b715-335a6f6765a0 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:14.030285 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2237/4548] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:14.779440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c58ffc-fd56-41e3-a8e4-61aa08cef319 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:14.883469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:14.893208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c58ffc-fd56-41e3-a8e4-61aa08cef319 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:14.904145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ac853-7644-4523-94df-a3288701623f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:14.906989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ac853-7644-4523-94df-a3288701623f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:14.906989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ac853-7644-4523-94df-a3288701623f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:14.906989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ac853-7644-4523-94df-a3288701623f admin admin] PATH_INFO: `/v3/users/dc73808524f5489aa10dfbac50ae0426` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:14.906989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ac853-7644-4523-94df-a3288701623f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc73808524f5489aa10dfbac50ae0426)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:14.909770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ac853-7644-4523-94df-a3288701623f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:14.977361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-412ac853-7644-4523-94df-a3288701623f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc73808524f5489aa10dfbac50ae0426'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:14.990812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-412ac853-7644-4523-94df-a3288701623f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc73808524f5489aa10dfbac50ae0426'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:15.001148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-412ac853-7644-4523-94df-a3288701623f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc73808524f5489aa10dfbac50ae0426'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:15.002047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-412ac853-7644-4523-94df-a3288701623f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc73808524f5489aa10dfbac50ae0426'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:15.003128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-412ac853-7644-4523-94df-a3288701623f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc73808524f5489aa10dfbac50ae0426'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:15.020144 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2312/4549] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:05:14 2022] DELETE /identity/v3/users/dc73808524f5489aa10dfbac50ae0426 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:15.024777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6bc78-5e40-4a19-8d07-6e5d81995b6a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:15.025882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6bc78-5e40-4a19-8d07-6e5d81995b6a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:15.026167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6bc78-5e40-4a19-8d07-6e5d81995b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:15.051076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1ad6bc78-5e40-4a19-8d07-6e5d81995b6a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:15.052932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1ad6bc78-5e40-4a19-8d07-6e5d81995b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:15.186736 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2238/4550] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:15.186921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ac853-7644-4523-94df-a3288701623f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:15.189622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-412ac853-7644-4523-94df-a3288701623f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:15.284271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ac853-7644-4523-94df-a3288701623f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:15.288386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff132923-fbb6-49ad-a963-b2438194b428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:15.289556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff132923-fbb6-49ad-a963-b2438194b428 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:15.289948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff132923-fbb6-49ad-a963-b2438194b428 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:15.290412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff132923-fbb6-49ad-a963-b2438194b428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:15.300638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff132923-fbb6-49ad-a963-b2438194b428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:15.302225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff132923-fbb6-49ad-a963-b2438194b428 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:15.315537 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2313/4551] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:15.746481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6bc78-5e40-4a19-8d07-6e5d81995b6a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:15.748206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1ad6bc78-5e40-4a19-8d07-6e5d81995b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:15.765273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6bc78-5e40-4a19-8d07-6e5d81995b6a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:15.774084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:15.775072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:15.775547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:15.776624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] PATH_INFO: `/v3/projects/5bd308cd63fd4e32b287b9b238f7ef0c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:15.779333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bd308cd63fd4e32b287b9b238f7ef0c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:15.781836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:15.864777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bd308cd63fd4e32b287b9b238f7ef0c', 'request_id': 'req-6a4d562c-a996-4d57-b69c-345bc75f88c5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:15.876095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bd308cd63fd4e32b287b9b238f7ef0c', 'request_id': 'req-6a4d562c-a996-4d57-b69c-345bc75f88c5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:15.886501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bd308cd63fd4e32b287b9b238f7ef0c', 'request_id': 'req-6a4d562c-a996-4d57-b69c-345bc75f88c5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:15.901631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] The token cache is being invalidate because project 5bd308cd63fd4e32b287b9b238f7ef0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:05:15.901966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:15.902798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:15.907730 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2239/4552] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:05:15 2022] DELETE /identity/v3/projects/5bd308cd63fd4e32b287b9b238f7ef0c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:15.908792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff132923-fbb6-49ad-a963-b2438194b428 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:15.931426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff132923-fbb6-49ad-a963-b2438194b428 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:16.025887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c838e-811e-48cf-a459-7e21472984a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:16.026924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c838e-811e-48cf-a459-7e21472984a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:16.027221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c838e-811e-48cf-a459-7e21472984a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:16.027674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c838e-811e-48cf-a459-7e21472984a6 admin admin] PATH_INFO: `/v3/users/4fb2e9371c824193b9cc382ba2a70ec2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:16.029303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c838e-811e-48cf-a459-7e21472984a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb2e9371c824193b9cc382ba2a70ec2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:16.031349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c838e-811e-48cf-a459-7e21472984a6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:16.070382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:16.124454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-318c838e-811e-48cf-a459-7e21472984a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb2e9371c824193b9cc382ba2a70ec2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:16.154990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-318c838e-811e-48cf-a459-7e21472984a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb2e9371c824193b9cc382ba2a70ec2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:16.171022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-318c838e-811e-48cf-a459-7e21472984a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb2e9371c824193b9cc382ba2a70ec2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:16.175738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-318c838e-811e-48cf-a459-7e21472984a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb2e9371c824193b9cc382ba2a70ec2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:16.176842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-318c838e-811e-48cf-a459-7e21472984a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb2e9371c824193b9cc382ba2a70ec2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:16.210423 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2314/4553] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:05:15 2022] DELETE /identity/v3/users/4fb2e9371c824193b9cc382ba2a70ec2 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:17.054368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:17.057889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:17.085088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d562c-a996-4d57-b69c-345bc75f88c5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:17.283878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:17.318638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b666643-8d30-477f-a875-b60ac29f9f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:17.320981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b666643-8d30-477f-a875-b60ac29f9f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:17.324462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b666643-8d30-477f-a875-b60ac29f9f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:17.325471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b666643-8d30-477f-a875-b60ac29f9f0a admin admin] PATH_INFO: `/v3/projects/b3ee3943796e49b29730e78e8523d998` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:17.328841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b666643-8d30-477f-a875-b60ac29f9f0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ee3943796e49b29730e78e8523d998)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:17.334439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b666643-8d30-477f-a875-b60ac29f9f0a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:17.540096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3b666643-8d30-477f-a875-b60ac29f9f0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ee3943796e49b29730e78e8523d998', 'request_id': 'req-3b666643-8d30-477f-a875-b60ac29f9f0a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:17.564323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3b666643-8d30-477f-a875-b60ac29f9f0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ee3943796e49b29730e78e8523d998', 'request_id': 'req-3b666643-8d30-477f-a875-b60ac29f9f0a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:17.577573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3b666643-8d30-477f-a875-b60ac29f9f0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ee3943796e49b29730e78e8523d998', 'request_id': 'req-3b666643-8d30-477f-a875-b60ac29f9f0a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:17.615025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3b666643-8d30-477f-a875-b60ac29f9f0a admin admin] The token cache is being invalidate because project b3ee3943796e49b29730e78e8523d998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:05:17.615455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3b666643-8d30-477f-a875-b60ac29f9f0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:17.617922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3b666643-8d30-477f-a875-b60ac29f9f0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:17.628367 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2240/4554] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:05:17 2022] DELETE /identity/v3/projects/b3ee3943796e49b29730e78e8523d998 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:17.630530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6fc4e-fbed-4f9f-a91f-4ad4920dc12b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:17.630972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6fc4e-fbed-4f9f-a91f-4ad4920dc12b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:17.631107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6fc4e-fbed-4f9f-a91f-4ad4920dc12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:17.674567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-04d6fc4e-fbed-4f9f-a91f-4ad4920dc12b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:17.676168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-04d6fc4e-fbed-4f9f-a91f-4ad4920dc12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:17.791934 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2315/4555] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:17.799958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99790208-d55a-4469-993d-51e9b9659cbe None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:17.800255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99790208-d55a-4469-993d-51e9b9659cbe None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:17.800536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99790208-d55a-4469-993d-51e9b9659cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:17.861835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-99790208-d55a-4469-993d-51e9b9659cbe None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:17.863447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-99790208-d55a-4469-993d-51e9b9659cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:17.887041 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2241/4556] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:17.891707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6fc4e-fbed-4f9f-a91f-4ad4920dc12b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:17.905304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6fc4e-fbed-4f9f-a91f-4ad4920dc12b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:17.911639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd75986-7f88-4647-9fbe-90fdff9759f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:17.912854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd75986-7f88-4647-9fbe-90fdff9759f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:17.913437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd75986-7f88-4647-9fbe-90fdff9759f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:17.913950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd75986-7f88-4647-9fbe-90fdff9759f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:17.915140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd75986-7f88-4647-9fbe-90fdff9759f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:05:17.915671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd75986-7f88-4647-9fbe-90fdff9759f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:17.917909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd75986-7f88-4647-9fbe-90fdff9759f4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:17.942053 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2316/4557] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:05:17.943143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99790208-d55a-4469-993d-51e9b9659cbe None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:17.956353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99790208-d55a-4469-993d-51e9b9659cbe None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:17.968464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb9484-20b6-41b2-8488-83007d3e89e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:17.970259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb9484-20b6-41b2-8488-83007d3e89e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:17.970618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb9484-20b6-41b2-8488-83007d3e89e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:17.971093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb9484-20b6-41b2-8488-83007d3e89e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:17.973562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb9484-20b6-41b2-8488-83007d3e89e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:17.976022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb9484-20b6-41b2-8488-83007d3e89e3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:18.021424 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2242/4558] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:05:17 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:05:18.022867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd75986-7f88-4647-9fbe-90fdff9759f4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:18.034739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd75986-7f88-4647-9fbe-90fdff9759f4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:18.114641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de444d2-b33e-448b-b4b9-a191181ef299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:18.115609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de444d2-b33e-448b-b4b9-a191181ef299 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:18.116044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de444d2-b33e-448b-b4b9-a191181ef299 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:18.116538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de444d2-b33e-448b-b4b9-a191181ef299 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:18.118161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de444d2-b33e-448b-b4b9-a191181ef299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:18.120449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de444d2-b33e-448b-b4b9-a191181ef299 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:18.155306 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2317/4559] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:05:18 2022] POST /identity/v3/users => generated 283 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:05:18.159838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb9484-20b6-41b2-8488-83007d3e89e3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:18.182385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb9484-20b6-41b2-8488-83007d3e89e3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:18.189462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d64d4-3a71-479e-aecb-bbe9e1dec082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:18.190425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d64d4-3a71-479e-aecb-bbe9e1dec082 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:18.190813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d64d4-3a71-479e-aecb-bbe9e1dec082 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:18.191220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d64d4-3a71-479e-aecb-bbe9e1dec082 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:18.192337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d64d4-3a71-479e-aecb-bbe9e1dec082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:18.194603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d64d4-3a71-479e-aecb-bbe9e1dec082 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:18.212617 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2243/4560] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:18.214864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de444d2-b33e-448b-b4b9-a191181ef299 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:18.229792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de444d2-b33e-448b-b4b9-a191181ef299 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:18.236662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26236eb-aedd-46b5-9bed-c5199bcdbd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:18.237337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26236eb-aedd-46b5-9bed-c5199bcdbd4f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:18.237629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26236eb-aedd-46b5-9bed-c5199bcdbd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:18.237915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26236eb-aedd-46b5-9bed-c5199bcdbd4f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:18.238573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26236eb-aedd-46b5-9bed-c5199bcdbd4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:18.240411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26236eb-aedd-46b5-9bed-c5199bcdbd4f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:18.259433 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2318/4561] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:18.259785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d64d4-3a71-479e-aecb-bbe9e1dec082 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:18.271588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d64d4-3a71-479e-aecb-bbe9e1dec082 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:18.302493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aecd3c-d268-4b9c-91f4-a1cd1882a181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:18.303697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aecd3c-d268-4b9c-91f4-a1cd1882a181 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:18.304686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aecd3c-d268-4b9c-91f4-a1cd1882a181 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:18.305793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aecd3c-d268-4b9c-91f4-a1cd1882a181 admin admin] PATH_INFO: `/v3/projects/79e6078ae59e4181bfd916aa02bf3b4b/users/9b095329658d4c0e9e51bc6c4e4e10c1/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:18.326868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aecd3c-d268-4b9c-91f4-a1cd1882a181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e6078ae59e4181bfd916aa02bf3b4b, user_id=9b095329658d4c0e9e51bc6c4e4e10c1, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:18.329659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aecd3c-d268-4b9c-91f4-a1cd1882a181 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:18.355987 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2244/4562] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:18 2022] PUT /identity/v3/projects/79e6078ae59e4181bfd916aa02bf3b4b/users/9b095329658d4c0e9e51bc6c4e4e10c1/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:05:18.368333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238a855-54b4-45d9-8aaf-40bc691127c5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:18.368633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238a855-54b4-45d9-8aaf-40bc691127c5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:18.368959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238a855-54b4-45d9-8aaf-40bc691127c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:18.381381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:18.443989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d238a855-54b4-45d9-8aaf-40bc691127c5 None None] MFA Rules not processed for user `9b095329658d4c0e9e51bc6c4e4e10c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:18.445301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d238a855-54b4-45d9-8aaf-40bc691127c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:18.526905 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2319/4563] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:18.530705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aecd3c-d268-4b9c-91f4-a1cd1882a181 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:18.633758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aecd3c-d268-4b9c-91f4-a1cd1882a181 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:18.639086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b158828-536e-43d8-b9ca-d9d2bfb9667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:18.639702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b158828-536e-43d8-b9ca-d9d2bfb9667c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:18.640053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b158828-536e-43d8-b9ca-d9d2bfb9667c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:18.640336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b158828-536e-43d8-b9ca-d9d2bfb9667c admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:18.641074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b158828-536e-43d8-b9ca-d9d2bfb9667c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:18.642990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b158828-536e-43d8-b9ca-d9d2bfb9667c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:18.670612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:18.685249 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2245/4564] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:05:18 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:18.689508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238a855-54b4-45d9-8aaf-40bc691127c5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:18.700854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238a855-54b4-45d9-8aaf-40bc691127c5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:18.782308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800e5c5-c130-401d-ab9c-53117d9dfdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:18.782831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800e5c5-c130-401d-ab9c-53117d9dfdf3 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:18.783119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800e5c5-c130-401d-ab9c-53117d9dfdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:18.783397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800e5c5-c130-401d-ab9c-53117d9dfdf3 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:18.784032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800e5c5-c130-401d-ab9c-53117d9dfdf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:18.786069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800e5c5-c130-401d-ab9c-53117d9dfdf3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:18.807180 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2320/4565] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:05:18 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:18.811282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b158828-536e-43d8-b9ca-d9d2bfb9667c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:18.821559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b158828-536e-43d8-b9ca-d9d2bfb9667c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:18.826177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37864d2b-626e-4691-a8fd-2238396187e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:18.826772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37864d2b-626e-4691-a8fd-2238396187e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:18.827055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37864d2b-626e-4691-a8fd-2238396187e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:18.827330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37864d2b-626e-4691-a8fd-2238396187e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:18.827973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37864d2b-626e-4691-a8fd-2238396187e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:18.829615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37864d2b-626e-4691-a8fd-2238396187e5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:18.843883 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2246/4566] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:18.845191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800e5c5-c130-401d-ab9c-53117d9dfdf3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:18.857049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800e5c5-c130-401d-ab9c-53117d9dfdf3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:18.861359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33ac95-ca81-48c3-aa0e-d254a033f305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:18.862027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ac95-ca81-48c3-aa0e-d254a033f305 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:18.862321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ac95-ca81-48c3-aa0e-d254a033f305 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:18.862606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ac95-ca81-48c3-aa0e-d254a033f305 admin admin] PATH_INFO: `/v3/projects/46e129fa91494718af1be3f70dec2e09/users/6d87455754584bb599616309703be994/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:18.879632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33ac95-ca81-48c3-aa0e-d254a033f305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e129fa91494718af1be3f70dec2e09, user_id=6d87455754584bb599616309703be994, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:18.883026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33ac95-ca81-48c3-aa0e-d254a033f305 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:18.897550 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2321/4567] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:18 2022] PUT /identity/v3/projects/46e129fa91494718af1be3f70dec2e09/users/6d87455754584bb599616309703be994/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:18.901778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37864d2b-626e-4691-a8fd-2238396187e5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:18.913133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37864d2b-626e-4691-a8fd-2238396187e5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:19.008563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd6148-fa2d-479b-b6c4-bc0b50792e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:19.009221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd6148-fa2d-479b-b6c4-bc0b50792e18 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:19.009511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd6148-fa2d-479b-b6c4-bc0b50792e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:19.009791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd6148-fa2d-479b-b6c4-bc0b50792e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:19.010438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd6148-fa2d-479b-b6c4-bc0b50792e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:19.012410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd6148-fa2d-479b-b6c4-bc0b50792e18 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:19.025722 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2247/4568] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:19.028879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33ac95-ca81-48c3-aa0e-d254a033f305 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:19.042711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33ac95-ca81-48c3-aa0e-d254a033f305 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:19.048573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a53a6-4649-4a29-b066-c509612f4166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:19.049699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a53a6-4649-4a29-b066-c509612f4166 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:19.049988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a53a6-4649-4a29-b066-c509612f4166 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:19.050265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a53a6-4649-4a29-b066-c509612f4166 admin admin] PATH_INFO: `/v3/projects/46e129fa91494718af1be3f70dec2e09/users/6d87455754584bb599616309703be994/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:19.052203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a53a6-4649-4a29-b066-c509612f4166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e129fa91494718af1be3f70dec2e09, user_id=6d87455754584bb599616309703be994, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:19.055074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a53a6-4649-4a29-b066-c509612f4166 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:19.075806 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2322/4569] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:19 2022] PUT /identity/v3/projects/46e129fa91494718af1be3f70dec2e09/users/6d87455754584bb599616309703be994/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:19.086143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5a0f-a9ed-49c1-a397-f529ab9a383f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:19.086432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5a0f-a9ed-49c1-a397-f529ab9a383f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:19.086715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5a0f-a9ed-49c1-a397-f529ab9a383f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:19.127810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0d5a5a0f-a9ed-49c1-a397-f529ab9a383f None None] MFA Rules not processed for user `6d87455754584bb599616309703be994`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:19.132962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0d5a5a0f-a9ed-49c1-a397-f529ab9a383f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:19.233112 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2248/4570] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:19.245474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a53a6-4649-4a29-b066-c509612f4166 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:19.248017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f68a53a6-4649-4a29-b066-c509612f4166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:19.311724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:19.321580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a53a6-4649-4a29-b066-c509612f4166 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:19.325680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd453557-eb46-4c36-a4eb-33354f38011a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:19.326305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd453557-eb46-4c36-a4eb-33354f38011a service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:19.326574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd453557-eb46-4c36-a4eb-33354f38011a service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:19.326929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd453557-eb46-4c36-a4eb-33354f38011a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:19.341559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd453557-eb46-4c36-a4eb-33354f38011a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:19.343597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd453557-eb46-4c36-a4eb-33354f38011a service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:19.356617 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2323/4571] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:19.474916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a5a0f-a9ed-49c1-a397-f529ab9a383f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:19.490028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a5a0f-a9ed-49c1-a397-f529ab9a383f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:19.495903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d322f3f-d8a3-41fd-8cf2-4de0668919ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:19.497468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d322f3f-d8a3-41fd-8cf2-4de0668919ed service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:19.498336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d322f3f-d8a3-41fd-8cf2-4de0668919ed service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:19.498929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d322f3f-d8a3-41fd-8cf2-4de0668919ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:19.578814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d322f3f-d8a3-41fd-8cf2-4de0668919ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:19.581956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d322f3f-d8a3-41fd-8cf2-4de0668919ed service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:19.596306 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2249/4572] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:20.443783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfc2c1-e970-4bf2-8cdd-cf9c841dd554 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:20.444324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfc2c1-e970-4bf2-8cdd-cf9c841dd554 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:20.444660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfc2c1-e970-4bf2-8cdd-cf9c841dd554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:20.476244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fddfc2c1-e970-4bf2-8cdd-cf9c841dd554 None None] MFA Rules not processed for user `43d940c1853e4d4aa86f6f4c61d95109`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:20.477602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fddfc2c1-e970-4bf2-8cdd-cf9c841dd554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:20.565696 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2324/4573] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:20 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:20.575518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899ba2c-1e46-4bdc-a4fa-3d493fcf220c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:20.576006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899ba2c-1e46-4bdc-a4fa-3d493fcf220c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:20.576084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899ba2c-1e46-4bdc-a4fa-3d493fcf220c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:20.602468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a899ba2c-1e46-4bdc-a4fa-3d493fcf220c None None] MFA Rules not processed for user `43d940c1853e4d4aa86f6f4c61d95109`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:20.604717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a899ba2c-1e46-4bdc-a4fa-3d493fcf220c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:20.612860 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2250/4574] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:20 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:20.630957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfc2c1-e970-4bf2-8cdd-cf9c841dd554 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:20.632628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fddfc2c1-e970-4bf2-8cdd-cf9c841dd554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:20.784011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfc2c1-e970-4bf2-8cdd-cf9c841dd554 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:20.802838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743e902-1c8b-4952-9a7b-9de1760887a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:20.804292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743e902-1c8b-4952-9a7b-9de1760887a8 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:20.804587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743e902-1c8b-4952-9a7b-9de1760887a8 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:20.804867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743e902-1c8b-4952-9a7b-9de1760887a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:20.830941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743e902-1c8b-4952-9a7b-9de1760887a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:20.832859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743e902-1c8b-4952-9a7b-9de1760887a8 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:20.850556 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2325/4575] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:20 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:21.517813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899ba2c-1e46-4bdc-a4fa-3d493fcf220c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:21.523128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:21.633233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899ba2c-1e46-4bdc-a4fa-3d493fcf220c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:21.639733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:21.640406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:21.640696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:21.641012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] PATH_INFO: `/v3/users/9b095329658d4c0e9e51bc6c4e4e10c1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:21.642549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b095329658d4c0e9e51bc6c4e4e10c1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:21.645345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:21.751506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b095329658d4c0e9e51bc6c4e4e10c1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:21.772491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b095329658d4c0e9e51bc6c4e4e10c1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:21.774042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b095329658d4c0e9e51bc6c4e4e10c1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:21.808353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b095329658d4c0e9e51bc6c4e4e10c1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:21.832154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b095329658d4c0e9e51bc6c4e4e10c1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:21.843743 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2251/4576] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:05:21 2022] DELETE /identity/v3/users/9b095329658d4c0e9e51bc6c4e4e10c1 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:21.848793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4aed78-e8b2-48b9-94a9-f522cbb91348 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:21.849666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4aed78-e8b2-48b9-94a9-f522cbb91348 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:21.850091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4aed78-e8b2-48b9-94a9-f522cbb91348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:21.878948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ec4aed78-e8b2-48b9-94a9-f522cbb91348 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:21.880235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ec4aed78-e8b2-48b9-94a9-f522cbb91348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:21.981823 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2326/4577] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:21.989703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:21.994296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:22.124782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d649ed-85a1-4b53-aced-987d4c5ab0ca admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:22.128794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e6e60-f247-4321-bde6-832429e82bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:22.129764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e6e60-f247-4321-bde6-832429e82bfb service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:22.130054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e6e60-f247-4321-bde6-832429e82bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:22.130505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e6e60-f247-4321-bde6-832429e82bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:22.151921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e6e60-f247-4321-bde6-832429e82bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:22.154119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e6e60-f247-4321-bde6-832429e82bfb service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:22.189144 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2252/4578] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:22.526717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4aed78-e8b2-48b9-94a9-f522cbb91348 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:22.528272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ec4aed78-e8b2-48b9-94a9-f522cbb91348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:22.552916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:22.563293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4aed78-e8b2-48b9-94a9-f522cbb91348 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:22.570461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:22.571119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:22.571431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:22.571755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] PATH_INFO: `/v3/projects/79e6078ae59e4181bfd916aa02bf3b4b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:22.572974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e6078ae59e4181bfd916aa02bf3b4b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:22.575736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:22.655581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e6078ae59e4181bfd916aa02bf3b4b', 'request_id': 'req-64733aad-e126-46ed-87ab-fb4a9af2f712'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:22.666614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e6078ae59e4181bfd916aa02bf3b4b', 'request_id': 'req-64733aad-e126-46ed-87ab-fb4a9af2f712'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:22.676486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e6078ae59e4181bfd916aa02bf3b4b', 'request_id': 'req-64733aad-e126-46ed-87ab-fb4a9af2f712'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:22.694338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] The token cache is being invalidate because project 79e6078ae59e4181bfd916aa02bf3b4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:05:22.696309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:22.697038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:22.698855 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2327/4579] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:05:22 2022] DELETE /identity/v3/projects/79e6078ae59e4181bfd916aa02bf3b4b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:22.702943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e6e60-f247-4321-bde6-832429e82bfb service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:22.714178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e6e60-f247-4321-bde6-832429e82bfb service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:22.744742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:22.786982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504f736-e443-46ca-bbc5-10ebb1e61c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:22.787636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504f736-e443-46ca-bbc5-10ebb1e61c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:22.787933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504f736-e443-46ca-bbc5-10ebb1e61c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:22.788274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504f736-e443-46ca-bbc5-10ebb1e61c13 admin admin] PATH_INFO: `/v3/users/6d87455754584bb599616309703be994` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:22.789498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504f736-e443-46ca-bbc5-10ebb1e61c13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d87455754584bb599616309703be994)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:22.791446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504f736-e443-46ca-bbc5-10ebb1e61c13 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:22.857431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0504f736-e443-46ca-bbc5-10ebb1e61c13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d87455754584bb599616309703be994'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:22.875727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0504f736-e443-46ca-bbc5-10ebb1e61c13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d87455754584bb599616309703be994'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:22.901066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0504f736-e443-46ca-bbc5-10ebb1e61c13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d87455754584bb599616309703be994'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:22.923684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0504f736-e443-46ca-bbc5-10ebb1e61c13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d87455754584bb599616309703be994'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:22.933185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0504f736-e443-46ca-bbc5-10ebb1e61c13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d87455754584bb599616309703be994'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:22.949437 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2253/4580] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:05:22 2022] DELETE /identity/v3/users/6d87455754584bb599616309703be994 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:23.448037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:23.452435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:23.524638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64733aad-e126-46ed-87ab-fb4a9af2f712 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:23.638227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6ab96-287f-4fff-8e8a-57719199fdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:23.638888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6ab96-287f-4fff-8e8a-57719199fdc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:23.639280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6ab96-287f-4fff-8e8a-57719199fdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:23.639804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6ab96-287f-4fff-8e8a-57719199fdc6 admin admin] PATH_INFO: `/v3/projects/46e129fa91494718af1be3f70dec2e09` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:23.641170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6ab96-287f-4fff-8e8a-57719199fdc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e129fa91494718af1be3f70dec2e09)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:23.643287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6ab96-287f-4fff-8e8a-57719199fdc6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:23.708771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f2d6ab96-287f-4fff-8e8a-57719199fdc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e129fa91494718af1be3f70dec2e09', 'request_id': 'req-f2d6ab96-287f-4fff-8e8a-57719199fdc6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:23.721187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f2d6ab96-287f-4fff-8e8a-57719199fdc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e129fa91494718af1be3f70dec2e09', 'request_id': 'req-f2d6ab96-287f-4fff-8e8a-57719199fdc6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:23.743562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f2d6ab96-287f-4fff-8e8a-57719199fdc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e129fa91494718af1be3f70dec2e09', 'request_id': 'req-f2d6ab96-287f-4fff-8e8a-57719199fdc6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:23.779128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f2d6ab96-287f-4fff-8e8a-57719199fdc6 admin admin] The token cache is being invalidate because project 46e129fa91494718af1be3f70dec2e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:05:23.779565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f2d6ab96-287f-4fff-8e8a-57719199fdc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:23.781706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f2d6ab96-287f-4fff-8e8a-57719199fdc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:23.782337 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2328/4581] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:05:23 2022] DELETE /identity/v3/projects/46e129fa91494718af1be3f70dec2e09 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:23.787570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de7fe2-29cc-4e21-89ac-4c602ee1a827 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:23.787954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de7fe2-29cc-4e21-89ac-4c602ee1a827 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:23.788946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de7fe2-29cc-4e21-89ac-4c602ee1a827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:23.850817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f6de7fe2-29cc-4e21-89ac-4c602ee1a827 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:23.877021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f6de7fe2-29cc-4e21-89ac-4c602ee1a827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:24.010892 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2254/4582] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:24.011342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73176d01-611b-4d4a-9997-2ce3e452c2e3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:24.011726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73176d01-611b-4d4a-9997-2ce3e452c2e3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:24.012090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73176d01-611b-4d4a-9997-2ce3e452c2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:24.060778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-73176d01-611b-4d4a-9997-2ce3e452c2e3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:24.062359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-73176d01-611b-4d4a-9997-2ce3e452c2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:24.082799 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2329/4583] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:24.087536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de7fe2-29cc-4e21-89ac-4c602ee1a827 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:24.104494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de7fe2-29cc-4e21-89ac-4c602ee1a827 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:24.111454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573ceb0-74f0-47da-9516-2ed880e4d9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:24.112494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573ceb0-74f0-47da-9516-2ed880e4d9ac admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:24.113116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573ceb0-74f0-47da-9516-2ed880e4d9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:24.113675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573ceb0-74f0-47da-9516-2ed880e4d9ac admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:24.114736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573ceb0-74f0-47da-9516-2ed880e4d9ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:05:24.115242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573ceb0-74f0-47da-9516-2ed880e4d9ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:24.117456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573ceb0-74f0-47da-9516-2ed880e4d9ac admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:24.153601 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2255/4584] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:05:24.157978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73176d01-611b-4d4a-9997-2ce3e452c2e3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:24.170603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73176d01-611b-4d4a-9997-2ce3e452c2e3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:24.177727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92849ad2-d114-44e2-aa8b-dd8661b47b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:24.179290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92849ad2-d114-44e2-aa8b-dd8661b47b43 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:24.179828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92849ad2-d114-44e2-aa8b-dd8661b47b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:24.180322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92849ad2-d114-44e2-aa8b-dd8661b47b43 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:24.181434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92849ad2-d114-44e2-aa8b-dd8661b47b43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:24.183707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92849ad2-d114-44e2-aa8b-dd8661b47b43 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:24.199601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:24.250930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573ceb0-74f0-47da-9516-2ed880e4d9ac admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:24.252394 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2330/4585] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:05:24 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:24.266787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573ceb0-74f0-47da-9516-2ed880e4d9ac admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:24.349415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:24.455634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c8b83-83b2-460d-89b5-34920a3e90da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:24.460759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c8b83-83b2-460d-89b5-34920a3e90da admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:24.461296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c8b83-83b2-460d-89b5-34920a3e90da admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:24.461734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c8b83-83b2-460d-89b5-34920a3e90da admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:24.463024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c8b83-83b2-460d-89b5-34920a3e90da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:24.475565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c8b83-83b2-460d-89b5-34920a3e90da admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:24.517131 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2256/4586] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:05:24 2022] POST /identity/v3/users => generated 287 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:05:24.521036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92849ad2-d114-44e2-aa8b-dd8661b47b43 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:24.537560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92849ad2-d114-44e2-aa8b-dd8661b47b43 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:24.549611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6bfc5f-ca12-4cb6-b23d-944e971bd875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:24.550251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6bfc5f-ca12-4cb6-b23d-944e971bd875 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:24.550542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6bfc5f-ca12-4cb6-b23d-944e971bd875 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:24.550822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6bfc5f-ca12-4cb6-b23d-944e971bd875 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:24.551485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6bfc5f-ca12-4cb6-b23d-944e971bd875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:24.553153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6bfc5f-ca12-4cb6-b23d-944e971bd875 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:24.595553 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2331/4587] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:24.599134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c8b83-83b2-460d-89b5-34920a3e90da admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:24.612233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c8b83-83b2-460d-89b5-34920a3e90da admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:24.618909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9d02d-31ff-4c07-aa4b-0e9a2f011471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:24.619755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9d02d-31ff-4c07-aa4b-0e9a2f011471 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:24.620052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9d02d-31ff-4c07-aa4b-0e9a2f011471 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:24.620339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9d02d-31ff-4c07-aa4b-0e9a2f011471 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:24.621075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9d02d-31ff-4c07-aa4b-0e9a2f011471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:24.622695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9d02d-31ff-4c07-aa4b-0e9a2f011471 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:24.641281 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2257/4588] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:24.646185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6bfc5f-ca12-4cb6-b23d-944e971bd875 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:24.675673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6bfc5f-ca12-4cb6-b23d-944e971bd875 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:24.695773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b0245-5b86-4f03-8734-24cf7e48ccf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:24.716270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b0245-5b86-4f03-8734-24cf7e48ccf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:24.716876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b0245-5b86-4f03-8734-24cf7e48ccf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:24.717330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b0245-5b86-4f03-8734-24cf7e48ccf0 admin admin] PATH_INFO: `/v3/projects/312be4c9cb134ff08513dc95f5acf30f/users/50a47947e44b4e8f8562231df4ada9ae/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:24.747504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b0245-5b86-4f03-8734-24cf7e48ccf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=312be4c9cb134ff08513dc95f5acf30f, user_id=50a47947e44b4e8f8562231df4ada9ae, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:24.751271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b0245-5b86-4f03-8734-24cf7e48ccf0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:24.786393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2c825-04be-4bee-ac09-2ccaced8b07b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:24.786704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2c825-04be-4bee-ac09-2ccaced8b07b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:24.786967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2c825-04be-4bee-ac09-2ccaced8b07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:24.790753 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2332/4589] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:24 2022] PUT /identity/v3/projects/312be4c9cb134ff08513dc95f5acf30f/users/50a47947e44b4e8f8562231df4ada9ae/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:24.814100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a7e2c825-04be-4bee-ac09-2ccaced8b07b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:24.815448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a7e2c825-04be-4bee-ac09-2ccaced8b07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:24.902599 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2258/4590] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:24.911963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b0245-5b86-4f03-8734-24cf7e48ccf0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:24.914456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-649b0245-5b86-4f03-8734-24cf7e48ccf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:24.996096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b0245-5b86-4f03-8734-24cf7e48ccf0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:25.000468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85aba98-1e6b-4c6e-bcfb-cfe54ad37cca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:25.001771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85aba98-1e6b-4c6e-bcfb-cfe54ad37cca service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:25.002289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85aba98-1e6b-4c6e-bcfb-cfe54ad37cca service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:25.002767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85aba98-1e6b-4c6e-bcfb-cfe54ad37cca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:25.022379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85aba98-1e6b-4c6e-bcfb-cfe54ad37cca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:25.025107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85aba98-1e6b-4c6e-bcfb-cfe54ad37cca service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:25.037030 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2333/4591] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:25.088070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e49f28-cbf2-4ce6-9ba2-6c1d4ef1e06e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:25.088447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e49f28-cbf2-4ce6-9ba2-6c1d4ef1e06e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:25.088848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e49f28-cbf2-4ce6-9ba2-6c1d4ef1e06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:25.138088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-75e49f28-cbf2-4ce6-9ba2-6c1d4ef1e06e None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:25.139410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-75e49f28-cbf2-4ce6-9ba2-6c1d4ef1e06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:25.150373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85aba98-1e6b-4c6e-bcfb-cfe54ad37cca service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:25.155098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c85aba98-1e6b-4c6e-bcfb-cfe54ad37cca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:25.295722 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2259/4592] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:05:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:25.308201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e49f28-cbf2-4ce6-9ba2-6c1d4ef1e06e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:25.311766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-75e49f28-cbf2-4ce6-9ba2-6c1d4ef1e06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:25.375279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85aba98-1e6b-4c6e-bcfb-cfe54ad37cca service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:25.399205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1e478-c26a-4fd7-9070-aadb9b97d8f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:25.405157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1e478-c26a-4fd7-9070-aadb9b97d8f0 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:25.405844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1e478-c26a-4fd7-9070-aadb9b97d8f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:25.410618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1e478-c26a-4fd7-9070-aadb9b97d8f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:25.416534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dce1e478-c26a-4fd7-9070-aadb9b97d8f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:25.458980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:25.547916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e49f28-cbf2-4ce6-9ba2-6c1d4ef1e06e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:25.564466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34ded5-6597-446e-b700-f52204dd99bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:25.565508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34ded5-6597-446e-b700-f52204dd99bc service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:25.566007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34ded5-6597-446e-b700-f52204dd99bc service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:25.566390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34ded5-6597-446e-b700-f52204dd99bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:25.597270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34ded5-6597-446e-b700-f52204dd99bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:25.612693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:25.624329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34ded5-6597-446e-b700-f52204dd99bc service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:25.631525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1e478-c26a-4fd7-9070-aadb9b97d8f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:25.633777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1e478-c26a-4fd7-9070-aadb9b97d8f0 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:25.657317 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2260/4593] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:05:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:05:25.703120 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2334/4594] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:26.502502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d573346-2795-4342-b739-08ffd069306f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:26.503124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d573346-2795-4342-b739-08ffd069306f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:26.503124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d573346-2795-4342-b739-08ffd069306f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:26.567731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7d573346-2795-4342-b739-08ffd069306f None None] MFA Rules not processed for user `50a47947e44b4e8f8562231df4ada9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:26.569034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7d573346-2795-4342-b739-08ffd069306f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:26.679824 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2261/4595] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:26.680545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1e478-c26a-4fd7-9070-aadb9b97d8f0 service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:26.694670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1e478-c26a-4fd7-9070-aadb9b97d8f0 service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:26.711522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edd13c-e494-4783-8eb9-d8265b3a3d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:26.712195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edd13c-e494-4783-8eb9-d8265b3a3d78 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:26.712651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edd13c-e494-4783-8eb9-d8265b3a3d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:26.713181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edd13c-e494-4783-8eb9-d8265b3a3d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:26.713988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edd13c-e494-4783-8eb9-d8265b3a3d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:26.715924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edd13c-e494-4783-8eb9-d8265b3a3d78 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:26.785726 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2335/4596] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:05:26 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:26.787557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d573346-2795-4342-b739-08ffd069306f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:26.804976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d573346-2795-4342-b739-08ffd069306f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:26.960553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48992d-c97f-41ac-8b68-0a0f5f13548d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:26.961301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48992d-c97f-41ac-8b68-0a0f5f13548d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:26.961975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48992d-c97f-41ac-8b68-0a0f5f13548d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:26.962518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48992d-c97f-41ac-8b68-0a0f5f13548d admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:26.963294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48992d-c97f-41ac-8b68-0a0f5f13548d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:26.965363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48992d-c97f-41ac-8b68-0a0f5f13548d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:27.021271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edd13c-e494-4783-8eb9-d8265b3a3d78 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:27.029029 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2262/4597] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:05:26 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:27.034118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edd13c-e494-4783-8eb9-d8265b3a3d78 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:27.039786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fc330-ac90-4d73-817e-a94b5ecd0e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:27.041033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fc330-ac90-4d73-817e-a94b5ecd0e87 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:27.041033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fc330-ac90-4d73-817e-a94b5ecd0e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:27.041402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fc330-ac90-4d73-817e-a94b5ecd0e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:27.042356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2fc330-ac90-4d73-817e-a94b5ecd0e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:27.044217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2fc330-ac90-4d73-817e-a94b5ecd0e87 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:27.075521 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2336/4598] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:27.079708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48992d-c97f-41ac-8b68-0a0f5f13548d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:27.092470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48992d-c97f-41ac-8b68-0a0f5f13548d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:27.098636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf751b6-007e-4a38-9122-95b75e17a827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:27.099454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf751b6-007e-4a38-9122-95b75e17a827 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:27.099806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf751b6-007e-4a38-9122-95b75e17a827 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:27.100176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf751b6-007e-4a38-9122-95b75e17a827 admin admin] PATH_INFO: `/v3/projects/928e4d9d6dc5404ab891ae0a62360a82/users/423181450f3e486794a3d6c3611f33ae/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:27.124597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf751b6-007e-4a38-9122-95b75e17a827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=928e4d9d6dc5404ab891ae0a62360a82, user_id=423181450f3e486794a3d6c3611f33ae, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:27.126138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf751b6-007e-4a38-9122-95b75e17a827 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:27.152950 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2263/4599] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:27 2022] PUT /identity/v3/projects/928e4d9d6dc5404ab891ae0a62360a82/users/423181450f3e486794a3d6c3611f33ae/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:27.156082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fc330-ac90-4d73-817e-a94b5ecd0e87 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:27.167412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fc330-ac90-4d73-817e-a94b5ecd0e87 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:27.236191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add2fdc-b300-42be-85ed-1e5c73200526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:27.236829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add2fdc-b300-42be-85ed-1e5c73200526 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:27.237153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add2fdc-b300-42be-85ed-1e5c73200526 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:27.237477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add2fdc-b300-42be-85ed-1e5c73200526 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:27.238135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add2fdc-b300-42be-85ed-1e5c73200526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:27.241820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add2fdc-b300-42be-85ed-1e5c73200526 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:27.255138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf751b6-007e-4a38-9122-95b75e17a827 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:27.257811 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2337/4600] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:27.267332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf751b6-007e-4a38-9122-95b75e17a827 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:27.273720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f9b22-a0a2-4e1d-a85b-7d1790bd7551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:27.273720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f9b22-a0a2-4e1d-a85b-7d1790bd7551 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:27.273720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f9b22-a0a2-4e1d-a85b-7d1790bd7551 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:27.274166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f9b22-a0a2-4e1d-a85b-7d1790bd7551 admin admin] PATH_INFO: `/v3/projects/928e4d9d6dc5404ab891ae0a62360a82/users/423181450f3e486794a3d6c3611f33ae/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:27.276424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f9b22-a0a2-4e1d-a85b-7d1790bd7551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=928e4d9d6dc5404ab891ae0a62360a82, user_id=423181450f3e486794a3d6c3611f33ae, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:27.279068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f9b22-a0a2-4e1d-a85b-7d1790bd7551 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:27.291348 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2264/4601] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:27 2022] PUT /identity/v3/projects/928e4d9d6dc5404ab891ae0a62360a82/users/423181450f3e486794a3d6c3611f33ae/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:28.645409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d340b4-bbbd-4320-aca5-6836e67dd1ab None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:28.645924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d340b4-bbbd-4320-aca5-6836e67dd1ab None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:28.645996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d340b4-bbbd-4320-aca5-6836e67dd1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:28.713201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d9d340b4-bbbd-4320-aca5-6836e67dd1ab None None] MFA Rules not processed for user `423181450f3e486794a3d6c3611f33ae`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:28.714463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d9d340b4-bbbd-4320-aca5-6836e67dd1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:28.830862 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2338/4602] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:28.843342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11627e8e-af70-45a5-92f7-373c382e6815 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:28.843750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11627e8e-af70-45a5-92f7-373c382e6815 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:28.843902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11627e8e-af70-45a5-92f7-373c382e6815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:28.866807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-11627e8e-af70-45a5-92f7-373c382e6815 None None] MFA Rules not processed for user `50a47947e44b4e8f8562231df4ada9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:28.868201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-11627e8e-af70-45a5-92f7-373c382e6815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:28.891745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:28.940203 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2265/4603] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:28.948374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d340b4-bbbd-4320-aca5-6836e67dd1ab None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:28.952667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d9d340b4-bbbd-4320-aca5-6836e67dd1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:29.050726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:29.149360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d340b4-bbbd-4320-aca5-6836e67dd1ab None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:29.165387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e5356-ae55-4011-9c94-d17120a1c0d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:29.166370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e5356-ae55-4011-9c94-d17120a1c0d6 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:29.167030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e5356-ae55-4011-9c94-d17120a1c0d6 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:29.167528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e5356-ae55-4011-9c94-d17120a1c0d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:29.195206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e5356-ae55-4011-9c94-d17120a1c0d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:29.198282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e5356-ae55-4011-9c94-d17120a1c0d6 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:29.216170 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2339/4604] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:05:37.957421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11627e8e-af70-45a5-92f7-373c382e6815 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:37.958985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-11627e8e-af70-45a5-92f7-373c382e6815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:38.069866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11627e8e-af70-45a5-92f7-373c382e6815 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:38.075262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19e952-bbb3-4b73-a519-851bfa466e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:38.075921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19e952-bbb3-4b73-a519-851bfa466e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:38.076207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19e952-bbb3-4b73-a519-851bfa466e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:38.076495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19e952-bbb3-4b73-a519-851bfa466e42 admin admin] PATH_INFO: `/v3/users/9995a4ecc0ec4f97a46ea58fa158ee61` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:38.077667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19e952-bbb3-4b73-a519-851bfa466e42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9995a4ecc0ec4f97a46ea58fa158ee61)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:38.080951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19e952-bbb3-4b73-a519-851bfa466e42 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:38.174988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7c19e952-bbb3-4b73-a519-851bfa466e42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9995a4ecc0ec4f97a46ea58fa158ee61'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:38.191326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7c19e952-bbb3-4b73-a519-851bfa466e42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9995a4ecc0ec4f97a46ea58fa158ee61'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:38.195003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7c19e952-bbb3-4b73-a519-851bfa466e42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9995a4ecc0ec4f97a46ea58fa158ee61'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:38.204577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7c19e952-bbb3-4b73-a519-851bfa466e42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9995a4ecc0ec4f97a46ea58fa158ee61'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:38.214533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7c19e952-bbb3-4b73-a519-851bfa466e42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9995a4ecc0ec4f97a46ea58fa158ee61'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:38.222426 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2266/4605] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:05:37 2022] DELETE /identity/v3/users/9995a4ecc0ec4f97a46ea58fa158ee61 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:38.515973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e5356-ae55-4011-9c94-d17120a1c0d6 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:38.547268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e5356-ae55-4011-9c94-d17120a1c0d6 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:38.646072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163c516-01b2-4188-9029-e44bc7ea512e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:38.646923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163c516-01b2-4188-9029-e44bc7ea512e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:38.648093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163c516-01b2-4188-9029-e44bc7ea512e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:38.648411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163c516-01b2-4188-9029-e44bc7ea512e admin admin] PATH_INFO: `/v3/projects/7c6b7140c9474bf5b0f1e6d1618a7937` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:38.649707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163c516-01b2-4188-9029-e44bc7ea512e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c6b7140c9474bf5b0f1e6d1618a7937)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:38.651786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163c516-01b2-4188-9029-e44bc7ea512e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:38.728750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c163c516-01b2-4188-9029-e44bc7ea512e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c6b7140c9474bf5b0f1e6d1618a7937', 'request_id': 'req-c163c516-01b2-4188-9029-e44bc7ea512e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:38.741832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c163c516-01b2-4188-9029-e44bc7ea512e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c6b7140c9474bf5b0f1e6d1618a7937', 'request_id': 'req-c163c516-01b2-4188-9029-e44bc7ea512e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:38.752339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c163c516-01b2-4188-9029-e44bc7ea512e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c6b7140c9474bf5b0f1e6d1618a7937', 'request_id': 'req-c163c516-01b2-4188-9029-e44bc7ea512e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:38.771188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c163c516-01b2-4188-9029-e44bc7ea512e admin admin] The token cache is being invalidate because project 7c6b7140c9474bf5b0f1e6d1618a7937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:05:38.771489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c163c516-01b2-4188-9029-e44bc7ea512e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:38.773967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c163c516-01b2-4188-9029-e44bc7ea512e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:38.777282 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2340/4606] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:05:38 2022] DELETE /identity/v3/projects/7c6b7140c9474bf5b0f1e6d1618a7937 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:38.782792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab58925-e37d-46fc-8005-f58401b98d74 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:38.783290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab58925-e37d-46fc-8005-f58401b98d74 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:38.783665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab58925-e37d-46fc-8005-f58401b98d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:38.839357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2ab58925-e37d-46fc-8005-f58401b98d74 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:38.857685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2ab58925-e37d-46fc-8005-f58401b98d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:38.939780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:38.976929 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2267/4607] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:39.063717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d08ff-74fb-4400-9ec0-1283eee7fe01 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:39.064132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d08ff-74fb-4400-9ec0-1283eee7fe01 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:39.064397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d08ff-74fb-4400-9ec0-1283eee7fe01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:39.088222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c34d08ff-74fb-4400-9ec0-1283eee7fe01 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:39.089780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c34d08ff-74fb-4400-9ec0-1283eee7fe01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:39.108611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab58925-e37d-46fc-8005-f58401b98d74 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:39.130911 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2341/4608] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:39.131401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab58925-e37d-46fc-8005-f58401b98d74 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:39.149611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac395a76-32f3-4a7d-b46a-5cc2198116b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:39.150585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac395a76-32f3-4a7d-b46a-5cc2198116b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:39.151014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac395a76-32f3-4a7d-b46a-5cc2198116b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:39.153856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac395a76-32f3-4a7d-b46a-5cc2198116b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:39.154569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac395a76-32f3-4a7d-b46a-5cc2198116b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:05:39.154885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac395a76-32f3-4a7d-b46a-5cc2198116b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:39.156504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac395a76-32f3-4a7d-b46a-5cc2198116b5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:39.176611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d08ff-74fb-4400-9ec0-1283eee7fe01 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:39.177080 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2268/4609] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:05:39.189032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d08ff-74fb-4400-9ec0-1283eee7fe01 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:39.195512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5815d-8b46-4c82-b478-8c4ea09ff7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:39.196077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5815d-8b46-4c82-b478-8c4ea09ff7d3 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:39.196358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5815d-8b46-4c82-b478-8c4ea09ff7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:39.196634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5815d-8b46-4c82-b478-8c4ea09ff7d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:39.197342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5815d-8b46-4c82-b478-8c4ea09ff7d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:39.202281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5815d-8b46-4c82-b478-8c4ea09ff7d3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:39.246594 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2342/4610] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:05:39 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:39.247032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac395a76-32f3-4a7d-b46a-5cc2198116b5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:39.259909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac395a76-32f3-4a7d-b46a-5cc2198116b5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:39.353192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eeaa8a-2758-456c-b562-055994c12ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:39.354071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eeaa8a-2758-456c-b562-055994c12ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:39.354501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eeaa8a-2758-456c-b562-055994c12ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:39.354947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eeaa8a-2758-456c-b562-055994c12ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:39.355739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eeaa8a-2758-456c-b562-055994c12ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:39.357875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eeaa8a-2758-456c-b562-055994c12ca6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:39.387486 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2269/4611] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:05:39 2022] POST /identity/v3/users => generated 288 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:39.388697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5815d-8b46-4c82-b478-8c4ea09ff7d3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:39.402417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5815d-8b46-4c82-b478-8c4ea09ff7d3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:39.407789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9e5b9-c7c6-4b00-92ec-07eaf45a3195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:39.408410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9e5b9-c7c6-4b00-92ec-07eaf45a3195 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:39.408697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9e5b9-c7c6-4b00-92ec-07eaf45a3195 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:39.409015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9e5b9-c7c6-4b00-92ec-07eaf45a3195 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:39.409674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9e5b9-c7c6-4b00-92ec-07eaf45a3195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:39.411512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9e5b9-c7c6-4b00-92ec-07eaf45a3195 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:39.425268 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2343/4612] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:05:39.429820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eeaa8a-2758-456c-b562-055994c12ca6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:39.443375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eeaa8a-2758-456c-b562-055994c12ca6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:39.452617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50cb715-3b89-4f85-9155-18d9db8137c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:39.453335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50cb715-3b89-4f85-9155-18d9db8137c9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:39.453639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50cb715-3b89-4f85-9155-18d9db8137c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:39.453975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50cb715-3b89-4f85-9155-18d9db8137c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:39.454633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50cb715-3b89-4f85-9155-18d9db8137c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:39.456888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50cb715-3b89-4f85-9155-18d9db8137c9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:39.472540 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2270/4613] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:39.474099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9e5b9-c7c6-4b00-92ec-07eaf45a3195 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:39.485864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9e5b9-c7c6-4b00-92ec-07eaf45a3195 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:39.492962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd2871-7cb2-45dd-8ee6-024042048b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:39.493721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd2871-7cb2-45dd-8ee6-024042048b76 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:39.494128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd2871-7cb2-45dd-8ee6-024042048b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:39.494486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd2871-7cb2-45dd-8ee6-024042048b76 admin admin] PATH_INFO: `/v3/projects/79959bbc9a594e1f81d7e60e5d175e2e/users/fb144426dd4a46caa03a9ebe77e95c4c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:39.497501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:39.514124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd2871-7cb2-45dd-8ee6-024042048b76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79959bbc9a594e1f81d7e60e5d175e2e, user_id=fb144426dd4a46caa03a9ebe77e95c4c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:39.517054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd2871-7cb2-45dd-8ee6-024042048b76 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:39.532162 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2344/4614] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:39 2022] PUT /identity/v3/projects/79959bbc9a594e1f81d7e60e5d175e2e/users/fb144426dd4a46caa03a9ebe77e95c4c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:39.537472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2828a-47b0-4628-a263-0e0bbc26505c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:39.538520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2828a-47b0-4628-a263-0e0bbc26505c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:39.539042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2828a-47b0-4628-a263-0e0bbc26505c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:39.561810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-80f2828a-47b0-4628-a263-0e0bbc26505c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:39.563190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-80f2828a-47b0-4628-a263-0e0bbc26505c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:39.633392 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2271/4615] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:05:39.643354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd2871-7cb2-45dd-8ee6-024042048b76 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:39.645225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3bbd2871-7cb2-45dd-8ee6-024042048b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:39.706097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd2871-7cb2-45dd-8ee6-024042048b76 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:39.710605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85a2e6-1c2b-406b-bf1b-1729ed1dfcee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:39.711304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85a2e6-1c2b-406b-bf1b-1729ed1dfcee service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:39.711667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85a2e6-1c2b-406b-bf1b-1729ed1dfcee service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:39.712155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85a2e6-1c2b-406b-bf1b-1729ed1dfcee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:39.727546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85a2e6-1c2b-406b-bf1b-1729ed1dfcee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:39.729024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85a2e6-1c2b-406b-bf1b-1729ed1dfcee service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:39.738523 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2345/4616] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:44.714532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4183d-5bbe-42ed-8121-5d5655874144 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:44.715009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4183d-5bbe-42ed-8121-5d5655874144 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:44.715114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4183d-5bbe-42ed-8121-5d5655874144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:44.789935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4bb4183d-5bbe-42ed-8121-5d5655874144 None None] MFA Rules not processed for user `fb144426dd4a46caa03a9ebe77e95c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:44.791559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4bb4183d-5bbe-42ed-8121-5d5655874144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:44.893487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:44.921156 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2272/4617] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:44.932250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b328d61-6109-4fbf-b90a-5f8707ec1284 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:44.932750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b328d61-6109-4fbf-b90a-5f8707ec1284 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:44.933617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b328d61-6109-4fbf-b90a-5f8707ec1284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:44.955542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7b328d61-6109-4fbf-b90a-5f8707ec1284 None None] MFA Rules not processed for user `fb144426dd4a46caa03a9ebe77e95c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:44.956775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7b328d61-6109-4fbf-b90a-5f8707ec1284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:44.964802 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2346/4618] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:44.979238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4183d-5bbe-42ed-8121-5d5655874144 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:44.983844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4bb4183d-5bbe-42ed-8121-5d5655874144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:45.076273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4183d-5bbe-42ed-8121-5d5655874144 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:45.082920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc607c37-29f2-40ad-914f-3bbef4474939 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:45.083642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc607c37-29f2-40ad-914f-3bbef4474939 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:45.084031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc607c37-29f2-40ad-914f-3bbef4474939 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:45.084314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc607c37-29f2-40ad-914f-3bbef4474939 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:45.093756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc607c37-29f2-40ad-914f-3bbef4474939 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:45.095387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc607c37-29f2-40ad-914f-3bbef4474939 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:45.106812 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2273/4619] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:45.385366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b328d61-6109-4fbf-b90a-5f8707ec1284 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:45.396945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b328d61-6109-4fbf-b90a-5f8707ec1284 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:45.401450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc0d54-0094-46fc-a253-817161866063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:45.402076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc0d54-0094-46fc-a253-817161866063 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:45.402368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc0d54-0094-46fc-a253-817161866063 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:45.402650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc0d54-0094-46fc-a253-817161866063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:45.405044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c2bc0d54-0094-46fc-a253-817161866063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:45.422043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc0d54-0094-46fc-a253-817161866063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:45.423907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc0d54-0094-46fc-a253-817161866063 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:45.434686 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2347/4620] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:48.338779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc607c37-29f2-40ad-914f-3bbef4474939 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:48.340326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bc607c37-29f2-40ad-914f-3bbef4474939 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:48.364287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc607c37-29f2-40ad-914f-3bbef4474939 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:48.369648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1c1c5-56d4-436d-b9d9-6d6a55d59858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:48.370873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1c1c5-56d4-436d-b9d9-6d6a55d59858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:48.371164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1c1c5-56d4-436d-b9d9-6d6a55d59858 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:48.371648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1c1c5-56d4-436d-b9d9-6d6a55d59858 admin admin] PATH_INFO: `/v3/users/43d940c1853e4d4aa86f6f4c61d95109` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:48.372881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1c1c5-56d4-436d-b9d9-6d6a55d59858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d940c1853e4d4aa86f6f4c61d95109)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:48.378647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1c1c5-56d4-436d-b9d9-6d6a55d59858 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:48.495161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3ec1c1c5-56d4-436d-b9d9-6d6a55d59858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d940c1853e4d4aa86f6f4c61d95109'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:48.510487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3ec1c1c5-56d4-436d-b9d9-6d6a55d59858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d940c1853e4d4aa86f6f4c61d95109'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:48.511176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3ec1c1c5-56d4-436d-b9d9-6d6a55d59858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d940c1853e4d4aa86f6f4c61d95109'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:48.518956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3ec1c1c5-56d4-436d-b9d9-6d6a55d59858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d940c1853e4d4aa86f6f4c61d95109'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:48.528977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3ec1c1c5-56d4-436d-b9d9-6d6a55d59858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d940c1853e4d4aa86f6f4c61d95109'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:48.539372 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2274/4621] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:05:48 2022] DELETE /identity/v3/users/43d940c1853e4d4aa86f6f4c61d95109 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:48.764561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc0d54-0094-46fc-a253-817161866063 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:48.782460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc0d54-0094-46fc-a253-817161866063 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:48.909587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cea55a-97d3-4fbe-9590-04ff55e56f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:48.910451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cea55a-97d3-4fbe-9590-04ff55e56f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:48.910893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cea55a-97d3-4fbe-9590-04ff55e56f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:48.911500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cea55a-97d3-4fbe-9590-04ff55e56f78 admin admin] PATH_INFO: `/v3/projects/94e570d13568401c8fe5191706d80a75` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:48.913302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cea55a-97d3-4fbe-9590-04ff55e56f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94e570d13568401c8fe5191706d80a75)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:48.915239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cea55a-97d3-4fbe-9590-04ff55e56f78 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:48.972994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9cea55a-97d3-4fbe-9590-04ff55e56f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94e570d13568401c8fe5191706d80a75', 'request_id': 'req-c9cea55a-97d3-4fbe-9590-04ff55e56f78'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:48.981727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9cea55a-97d3-4fbe-9590-04ff55e56f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94e570d13568401c8fe5191706d80a75', 'request_id': 'req-c9cea55a-97d3-4fbe-9590-04ff55e56f78'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:48.988143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9cea55a-97d3-4fbe-9590-04ff55e56f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94e570d13568401c8fe5191706d80a75', 'request_id': 'req-c9cea55a-97d3-4fbe-9590-04ff55e56f78'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:48.993790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:49.000780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9cea55a-97d3-4fbe-9590-04ff55e56f78 admin admin] The token cache is being invalidate because project 94e570d13568401c8fe5191706d80a75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:05:49.001363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9cea55a-97d3-4fbe-9590-04ff55e56f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:49.002655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9cea55a-97d3-4fbe-9590-04ff55e56f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:05:49.004622 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2348/4622] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:05:48 2022] DELETE /identity/v3/projects/94e570d13568401c8fe5191706d80a75 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:05:49.009528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1f8a0-0b9b-4e38-a297-cb7dcdc83495 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:49.009918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1f8a0-0b9b-4e38-a297-cb7dcdc83495 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:49.010268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1f8a0-0b9b-4e38-a297-cb7dcdc83495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:49.031563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b1c1f8a0-0b9b-4e38-a297-cb7dcdc83495 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:49.032828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b1c1f8a0-0b9b-4e38-a297-cb7dcdc83495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:49.086990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:49.118463 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2275/4623] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:05:49.126511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945168a-d6f7-44ca-a662-aaecde0814c0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:49.128711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945168a-d6f7-44ca-a662-aaecde0814c0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:49.128822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945168a-d6f7-44ca-a662-aaecde0814c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:49.196350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3945168a-d6f7-44ca-a662-aaecde0814c0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:49.203821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3945168a-d6f7-44ca-a662-aaecde0814c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:49.224584 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2349/4624] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:49.228739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1f8a0-0b9b-4e38-a297-cb7dcdc83495 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:49.248391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1f8a0-0b9b-4e38-a297-cb7dcdc83495 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:49.267122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f3856-10af-420d-a57e-d546e1a5c06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:49.268123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f3856-10af-420d-a57e-d546e1a5c06c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:49.268702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f3856-10af-420d-a57e-d546e1a5c06c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:49.269327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f3856-10af-420d-a57e-d546e1a5c06c admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:49.270218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f3856-10af-420d-a57e-d546e1a5c06c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:05:49.270822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f3856-10af-420d-a57e-d546e1a5c06c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:49.274685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f3856-10af-420d-a57e-d546e1a5c06c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:49.296000 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2276/4625] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:05:49.300385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945168a-d6f7-44ca-a662-aaecde0814c0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:49.316234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945168a-d6f7-44ca-a662-aaecde0814c0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:49.321465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759cf1b-244c-4ea6-8d3c-cfa51bca609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:49.322280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759cf1b-244c-4ea6-8d3c-cfa51bca609e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:49.322744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759cf1b-244c-4ea6-8d3c-cfa51bca609e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:49.323182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759cf1b-244c-4ea6-8d3c-cfa51bca609e admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:49.324700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759cf1b-244c-4ea6-8d3c-cfa51bca609e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:49.327116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759cf1b-244c-4ea6-8d3c-cfa51bca609e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:49.370806 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2350/4626] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:05:49 2022] POST /identity/v3/projects => generated 367 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:49.375070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f3856-10af-420d-a57e-d546e1a5c06c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:49.386135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f3856-10af-420d-a57e-d546e1a5c06c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:49.470452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55027662-ef14-4e3e-bc04-73bbfd9a8833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:49.471089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55027662-ef14-4e3e-bc04-73bbfd9a8833 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:49.471500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55027662-ef14-4e3e-bc04-73bbfd9a8833 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:49.472778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55027662-ef14-4e3e-bc04-73bbfd9a8833 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:49.478190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55027662-ef14-4e3e-bc04-73bbfd9a8833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:49.478190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55027662-ef14-4e3e-bc04-73bbfd9a8833 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:49.544956 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2277/4627] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:05:49 2022] POST /identity/v3/users => generated 284 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:05:49.549359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759cf1b-244c-4ea6-8d3c-cfa51bca609e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:49.561526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759cf1b-244c-4ea6-8d3c-cfa51bca609e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:49.576344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c13a1-7280-483a-acf7-31956607d9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:49.577192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c13a1-7280-483a-acf7-31956607d9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:49.577581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c13a1-7280-483a-acf7-31956607d9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:49.577983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c13a1-7280-483a-acf7-31956607d9d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:49.579005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c13a1-7280-483a-acf7-31956607d9d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:49.583419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c13a1-7280-483a-acf7-31956607d9d2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:49.605360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55027662-ef14-4e3e-bc04-73bbfd9a8833 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:49.606644 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2351/4628] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:49.627529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55027662-ef14-4e3e-bc04-73bbfd9a8833 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:49.634782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4c8e6-ce2c-4cde-8153-6dc1001e2d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:49.635556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4c8e6-ce2c-4cde-8153-6dc1001e2d0b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:49.635955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4c8e6-ce2c-4cde-8153-6dc1001e2d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:49.636337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4c8e6-ce2c-4cde-8153-6dc1001e2d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:49.637170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4c8e6-ce2c-4cde-8153-6dc1001e2d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:49.639453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4c8e6-ce2c-4cde-8153-6dc1001e2d0b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:49.661127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c13a1-7280-483a-acf7-31956607d9d2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:49.668712 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2278/4629] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:05:49.675180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c13a1-7280-483a-acf7-31956607d9d2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:49.680799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf896692-c73f-4ab0-b18f-69de39a72c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:49.681877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf896692-c73f-4ab0-b18f-69de39a72c39 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:49.682526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf896692-c73f-4ab0-b18f-69de39a72c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:49.682921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf896692-c73f-4ab0-b18f-69de39a72c39 admin admin] PATH_INFO: `/v3/projects/0de6b6fc412742beb94162e078268849/users/9e686d314d164930bbcfc9905a8b45ce/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:49.716938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf896692-c73f-4ab0-b18f-69de39a72c39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de6b6fc412742beb94162e078268849, user_id=9e686d314d164930bbcfc9905a8b45ce, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:49.730093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf896692-c73f-4ab0-b18f-69de39a72c39 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:49.776197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d06845-89bf-4152-8606-f6debe2238a2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:49.778000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d06845-89bf-4152-8606-f6debe2238a2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:49.778526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d06845-89bf-4152-8606-f6debe2238a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:49.781560 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2352/4630] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:05:49 2022] PUT /identity/v3/projects/0de6b6fc412742beb94162e078268849/users/9e686d314d164930bbcfc9905a8b45ce/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:05:49.806181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-10d06845-89bf-4152-8606-f6debe2238a2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:49.808779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-10d06845-89bf-4152-8606-f6debe2238a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:50.020381 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2279/4631] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:05:50.027846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf896692-c73f-4ab0-b18f-69de39a72c39 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:50.030210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bf896692-c73f-4ab0-b18f-69de39a72c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:50.114121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf896692-c73f-4ab0-b18f-69de39a72c39 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:50.118919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ab2ba-21df-4d6c-a1d2-c9ea7d9e8df2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:50.119555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab2ba-21df-4d6c-a1d2-c9ea7d9e8df2 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:50.119842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab2ba-21df-4d6c-a1d2-c9ea7d9e8df2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:50.120124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab2ba-21df-4d6c-a1d2-c9ea7d9e8df2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:50.130499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ab2ba-21df-4d6c-a1d2-c9ea7d9e8df2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:50.132879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ab2ba-21df-4d6c-a1d2-c9ea7d9e8df2 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:50.144723 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2353/4632] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:05:51.390820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24977c57-a8bd-4870-a9c2-338e131940e3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:51.391367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24977c57-a8bd-4870-a9c2-338e131940e3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:51.391367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24977c57-a8bd-4870-a9c2-338e131940e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:51.414772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-24977c57-a8bd-4870-a9c2-338e131940e3 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:05:51.415810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-24977c57-a8bd-4870-a9c2-338e131940e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:51.490061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:05:51.507431 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2280/4633] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:05:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:05:51.526876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ab2ba-21df-4d6c-a1d2-c9ea7d9e8df2 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:51.528364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-542ab2ba-21df-4d6c-a1d2-c9ea7d9e8df2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:51.640652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ab2ba-21df-4d6c-a1d2-c9ea7d9e8df2 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:51.653815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f02641-63e1-487c-8921-21ed73d02cb1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:51.654456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f02641-63e1-487c-8921-21ed73d02cb1 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:51.654744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f02641-63e1-487c-8921-21ed73d02cb1 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:51.657885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f02641-63e1-487c-8921-21ed73d02cb1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:51.688049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f02641-63e1-487c-8921-21ed73d02cb1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:51.691329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f02641-63e1-487c-8921-21ed73d02cb1 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:51.725352 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2354/4634] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:05:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:05:59.428970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24977c57-a8bd-4870-a9c2-338e131940e3 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:05:59.430435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-24977c57-a8bd-4870-a9c2-338e131940e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:59.679924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24977c57-a8bd-4870-a9c2-338e131940e3 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:05:59.698194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a55fe-77c3-4dd4-8686-f33126b5de4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:05:59.699421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a55fe-77c3-4dd4-8686-f33126b5de4a service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:05:59.699991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a55fe-77c3-4dd4-8686-f33126b5de4a service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:05:59.704090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a55fe-77c3-4dd4-8686-f33126b5de4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:05:59.708507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9f6a55fe-77c3-4dd4-8686-f33126b5de4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:05:59.748268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a55fe-77c3-4dd4-8686-f33126b5de4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:05:59.751140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a55fe-77c3-4dd4-8686-f33126b5de4a service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:05:59.778072 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2281/4635] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:05:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:06:00.167151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0819cf-3107-4236-a6f0-5dd69b48c327 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:00.168388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0819cf-3107-4236-a6f0-5dd69b48c327 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:00.168840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0819cf-3107-4236-a6f0-5dd69b48c327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:00.210859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ed0819cf-3107-4236-a6f0-5dd69b48c327 None None] MFA Rules not processed for user `9e686d314d164930bbcfc9905a8b45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:00.212344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ed0819cf-3107-4236-a6f0-5dd69b48c327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:00.243247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:00.421512 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2355/4636] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:00.431326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a06454-5826-463d-ac02-0791099b64d2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:00.431868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a06454-5826-463d-ac02-0791099b64d2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:00.431948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a06454-5826-463d-ac02-0791099b64d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:00.499644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-69a06454-5826-463d-ac02-0791099b64d2 None None] MFA Rules not processed for user `9e686d314d164930bbcfc9905a8b45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:00.518384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-69a06454-5826-463d-ac02-0791099b64d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:00.529707 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2282/4637] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:00.686882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0819cf-3107-4236-a6f0-5dd69b48c327 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:00.688744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ed0819cf-3107-4236-a6f0-5dd69b48c327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:00.938935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0819cf-3107-4236-a6f0-5dd69b48c327 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:00.963246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400d4af-7f26-438e-92e4-5046c981ab56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:00.963891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400d4af-7f26-438e-92e4-5046c981ab56 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:00.964225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400d4af-7f26-438e-92e4-5046c981ab56 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:00.964454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400d4af-7f26-438e-92e4-5046c981ab56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:01.008373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400d4af-7f26-438e-92e4-5046c981ab56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:01.014596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400d4af-7f26-438e-92e4-5046c981ab56 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:01.071238 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2356/4638] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:06:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:06:02.379573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a06454-5826-463d-ac02-0791099b64d2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:02.384803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-69a06454-5826-463d-ac02-0791099b64d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:02.448685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a06454-5826-463d-ac02-0791099b64d2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:02.466701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:02.467750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:02.468464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:02.468640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] PATH_INFO: `/v3/users/50a47947e44b4e8f8562231df4ada9ae` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:02.470910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50a47947e44b4e8f8562231df4ada9ae)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:02.475173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:02.662047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50a47947e44b4e8f8562231df4ada9ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:02.713706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50a47947e44b4e8f8562231df4ada9ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:02.718201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50a47947e44b4e8f8562231df4ada9ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:02.735768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50a47947e44b4e8f8562231df4ada9ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:02.747975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50a47947e44b4e8f8562231df4ada9ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:02.769543 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2283/4639] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:06:02 2022] DELETE /identity/v3/users/50a47947e44b4e8f8562231df4ada9ae => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:03.902181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400d4af-7f26-438e-92e4-5046c981ab56 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:03.926230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400d4af-7f26-438e-92e4-5046c981ab56 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:04.072284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:04.073114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:04.073962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:04.074400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] PATH_INFO: `/v3/projects/312be4c9cb134ff08513dc95f5acf30f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:04.076066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=312be4c9cb134ff08513dc95f5acf30f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:04.086323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:04.259107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:04.279115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '312be4c9cb134ff08513dc95f5acf30f', 'request_id': 'req-d8816e1e-3768-4b02-9978-f832b56dc0ca'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:04.308537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '312be4c9cb134ff08513dc95f5acf30f', 'request_id': 'req-d8816e1e-3768-4b02-9978-f832b56dc0ca'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:04.321699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '312be4c9cb134ff08513dc95f5acf30f', 'request_id': 'req-d8816e1e-3768-4b02-9978-f832b56dc0ca'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:04.343984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] The token cache is being invalidate because project 312be4c9cb134ff08513dc95f5acf30f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:04.344285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:04.345096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:04.349719 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2357/4640] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:03 2022] DELETE /identity/v3/projects/312be4c9cb134ff08513dc95f5acf30f => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:04.351674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:04.353518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:04.377005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458220a-bef8-4d06-9d3f-9972cb4d6f58 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:04.557024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:04.588094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013812a-79a7-4b2e-a072-c6adbd4abc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:04.588761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013812a-79a7-4b2e-a072-c6adbd4abc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:04.592284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013812a-79a7-4b2e-a072-c6adbd4abc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:04.592591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013812a-79a7-4b2e-a072-c6adbd4abc71 admin admin] PATH_INFO: `/v3/users/423181450f3e486794a3d6c3611f33ae` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:04.594021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013812a-79a7-4b2e-a072-c6adbd4abc71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=423181450f3e486794a3d6c3611f33ae)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:04.596222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013812a-79a7-4b2e-a072-c6adbd4abc71 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:04.724506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1013812a-79a7-4b2e-a072-c6adbd4abc71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '423181450f3e486794a3d6c3611f33ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:04.752981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1013812a-79a7-4b2e-a072-c6adbd4abc71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '423181450f3e486794a3d6c3611f33ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:04.755450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1013812a-79a7-4b2e-a072-c6adbd4abc71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '423181450f3e486794a3d6c3611f33ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:04.773403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1013812a-79a7-4b2e-a072-c6adbd4abc71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '423181450f3e486794a3d6c3611f33ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:04.795047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1013812a-79a7-4b2e-a072-c6adbd4abc71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '423181450f3e486794a3d6c3611f33ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:04.804927 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2284/4641] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:06:04 2022] DELETE /identity/v3/users/423181450f3e486794a3d6c3611f33ae => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:06.418145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:06.420215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:06.463243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8816e1e-3768-4b02-9978-f832b56dc0ca admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:06.608214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4226d5-b127-4ec6-b399-7a3aa655faa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:06.608924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4226d5-b127-4ec6-b399-7a3aa655faa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:06.609270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4226d5-b127-4ec6-b399-7a3aa655faa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:06.609572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4226d5-b127-4ec6-b399-7a3aa655faa1 admin admin] PATH_INFO: `/v3/projects/928e4d9d6dc5404ab891ae0a62360a82` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:06.610951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4226d5-b127-4ec6-b399-7a3aa655faa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=928e4d9d6dc5404ab891ae0a62360a82)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:06.613379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4226d5-b127-4ec6-b399-7a3aa655faa1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:06.771330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee4226d5-b127-4ec6-b399-7a3aa655faa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '928e4d9d6dc5404ab891ae0a62360a82', 'request_id': 'req-ee4226d5-b127-4ec6-b399-7a3aa655faa1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:06.793210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee4226d5-b127-4ec6-b399-7a3aa655faa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '928e4d9d6dc5404ab891ae0a62360a82', 'request_id': 'req-ee4226d5-b127-4ec6-b399-7a3aa655faa1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:06.803455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee4226d5-b127-4ec6-b399-7a3aa655faa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '928e4d9d6dc5404ab891ae0a62360a82', 'request_id': 'req-ee4226d5-b127-4ec6-b399-7a3aa655faa1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:06.827635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee4226d5-b127-4ec6-b399-7a3aa655faa1 admin admin] The token cache is being invalidate because project 928e4d9d6dc5404ab891ae0a62360a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:06.827937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee4226d5-b127-4ec6-b399-7a3aa655faa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:06.828635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee4226d5-b127-4ec6-b399-7a3aa655faa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:06.836449 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2358/4642] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:06 2022] DELETE /identity/v3/projects/928e4d9d6dc5404ab891ae0a62360a82 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:06.838000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d8b76-ff66-4fda-ba66-f9ca26382d49 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:06.838807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d8b76-ff66-4fda-ba66-f9ca26382d49 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:06.839231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d8b76-ff66-4fda-ba66-f9ca26382d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:06.910106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1c0d8b76-ff66-4fda-ba66-f9ca26382d49 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:06.911380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1c0d8b76-ff66-4fda-ba66-f9ca26382d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:07.065370 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2285/4643] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:07.066160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f5e53-e8d2-44e1-9954-07c1fc328313 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:07.067108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f5e53-e8d2-44e1-9954-07c1fc328313 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:07.067396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f5e53-e8d2-44e1-9954-07c1fc328313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:07.108255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-122f5e53-e8d2-44e1-9954-07c1fc328313 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:07.109713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-122f5e53-e8d2-44e1-9954-07c1fc328313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:07.160954 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2359/4644] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:07.166200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d8b76-ff66-4fda-ba66-f9ca26382d49 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:07.180357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d8b76-ff66-4fda-ba66-f9ca26382d49 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:07.207684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d8dc2-d3d3-46af-9b83-c4da04d80daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:07.211579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d8dc2-d3d3-46af-9b83-c4da04d80daf admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:07.211882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d8dc2-d3d3-46af-9b83-c4da04d80daf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:07.214106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d8dc2-d3d3-46af-9b83-c4da04d80daf admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:07.215969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d8dc2-d3d3-46af-9b83-c4da04d80daf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:06:07.216295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d8dc2-d3d3-46af-9b83-c4da04d80daf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:07.228462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d8dc2-d3d3-46af-9b83-c4da04d80daf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:07.259726 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2286/4645] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:06:07.264192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f5e53-e8d2-44e1-9954-07c1fc328313 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:07.279758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f5e53-e8d2-44e1-9954-07c1fc328313 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:07.300513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7bb4c-11d4-4c8c-8bc8-ea42f3af292c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:07.303917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7bb4c-11d4-4c8c-8bc8-ea42f3af292c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:07.304223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7bb4c-11d4-4c8c-8bc8-ea42f3af292c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:07.304511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7bb4c-11d4-4c8c-8bc8-ea42f3af292c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:07.305236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7bb4c-11d4-4c8c-8bc8-ea42f3af292c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:07.307201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7bb4c-11d4-4c8c-8bc8-ea42f3af292c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:07.370367 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2360/4646] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:06:07 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:07.374717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d8dc2-d3d3-46af-9b83-c4da04d80daf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:07.422828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d8dc2-d3d3-46af-9b83-c4da04d80daf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:07.558269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:07.564732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7e54d-cc8e-4c15-a48b-3884d76da59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:07.566612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7e54d-cc8e-4c15-a48b-3884d76da59d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:07.567530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7e54d-cc8e-4c15-a48b-3884d76da59d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:07.567813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7e54d-cc8e-4c15-a48b-3884d76da59d admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:07.568837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7e54d-cc8e-4c15-a48b-3884d76da59d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:07.570846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7e54d-cc8e-4c15-a48b-3884d76da59d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:07.618988 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2287/4647] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:06:07 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:07.623394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7bb4c-11d4-4c8c-8bc8-ea42f3af292c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:07.638388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7bb4c-11d4-4c8c-8bc8-ea42f3af292c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:07.643876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d6ab4-0214-49ba-9c5a-4fe302c96a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:07.645076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d6ab4-0214-49ba-9c5a-4fe302c96a09 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:07.645623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d6ab4-0214-49ba-9c5a-4fe302c96a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:07.646198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d6ab4-0214-49ba-9c5a-4fe302c96a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:07.647646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d6ab4-0214-49ba-9c5a-4fe302c96a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:07.649899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d6ab4-0214-49ba-9c5a-4fe302c96a09 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:07.675999 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2361/4648] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:07.676179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7e54d-cc8e-4c15-a48b-3884d76da59d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:07.686656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7e54d-cc8e-4c15-a48b-3884d76da59d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:07.701355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19592a25-e1f4-4ca0-ad7b-93d7eb65d31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:07.701994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19592a25-e1f4-4ca0-ad7b-93d7eb65d31c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:07.702302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19592a25-e1f4-4ca0-ad7b-93d7eb65d31c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:07.702564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19592a25-e1f4-4ca0-ad7b-93d7eb65d31c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:07.715751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19592a25-e1f4-4ca0-ad7b-93d7eb65d31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:07.726463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19592a25-e1f4-4ca0-ad7b-93d7eb65d31c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:07.763999 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2288/4649] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:07.768759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d6ab4-0214-49ba-9c5a-4fe302c96a09 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:07.789519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d6ab4-0214-49ba-9c5a-4fe302c96a09 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:07.799086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3e931-ee46-494a-a64d-8a0acd98208b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:07.799796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3e931-ee46-494a-a64d-8a0acd98208b admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:07.800088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3e931-ee46-494a-a64d-8a0acd98208b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:07.800374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3e931-ee46-494a-a64d-8a0acd98208b admin admin] PATH_INFO: `/v3/projects/e2571b47f5984b8786e35e81653e2b78/users/227adbc59c4244c3a7dafbd2eebab48c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:07.855472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3e931-ee46-494a-a64d-8a0acd98208b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2571b47f5984b8786e35e81653e2b78, user_id=227adbc59c4244c3a7dafbd2eebab48c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:07.862713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3e931-ee46-494a-a64d-8a0acd98208b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:07.871965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:07.939331 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2362/4650] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:06:07 2022] PUT /identity/v3/projects/e2571b47f5984b8786e35e81653e2b78/users/227adbc59c4244c3a7dafbd2eebab48c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:07.952595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365a525-2d09-4e2d-87d3-5d82cd33493d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:07.953594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365a525-2d09-4e2d-87d3-5d82cd33493d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:07.954097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365a525-2d09-4e2d-87d3-5d82cd33493d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:07.995225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1365a525-2d09-4e2d-87d3-5d82cd33493d None None] MFA Rules not processed for user `227adbc59c4244c3a7dafbd2eebab48c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:07.997312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1365a525-2d09-4e2d-87d3-5d82cd33493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:08.187726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3e931-ee46-494a-a64d-8a0acd98208b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:08.207697 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2289/4651] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:08.353006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3e931-ee46-494a-a64d-8a0acd98208b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:08.360220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b3b4c-c849-4c9d-9138-7c0a15eda762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:08.360220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b3b4c-c849-4c9d-9138-7c0a15eda762 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:08.360220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b3b4c-c849-4c9d-9138-7c0a15eda762 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:08.360220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b3b4c-c849-4c9d-9138-7c0a15eda762 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:08.360486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b3b4c-c849-4c9d-9138-7c0a15eda762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:08.363108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b3b4c-c849-4c9d-9138-7c0a15eda762 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:08.391741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce163ad-36b0-4a7f-ab20-2c4a7c40a472 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:08.391741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce163ad-36b0-4a7f-ab20-2c4a7c40a472 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:08.393740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce163ad-36b0-4a7f-ab20-2c4a7c40a472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:08.431624 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2363/4652] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:06:08 2022] POST /identity/v3/projects => generated 381 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:08.435039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b3b4c-c849-4c9d-9138-7c0a15eda762 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:08.470400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8ce163ad-36b0-4a7f-ab20-2c4a7c40a472 None None] MFA Rules not processed for user `c3187514ca644ff68c992526f38783c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:08.475966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8ce163ad-36b0-4a7f-ab20-2c4a7c40a472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:08.589881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b3b4c-c849-4c9d-9138-7c0a15eda762 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:08.601691 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2290/4653] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:06:08.611621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f57a38-bf4f-4867-bae9-5faee6846a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:08.611621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f57a38-bf4f-4867-bae9-5faee6846a77 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:08.611621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f57a38-bf4f-4867-bae9-5faee6846a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:08.611621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f57a38-bf4f-4867-bae9-5faee6846a77 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:08.612532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f57a38-bf4f-4867-bae9-5faee6846a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:08.614809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f57a38-bf4f-4867-bae9-5faee6846a77 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:08.665115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce163ad-36b0-4a7f-ab20-2c4a7c40a472 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:08.667436 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2364/4654] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:06:08 2022] POST /identity/v3/users => generated 297 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:08.681281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce163ad-36b0-4a7f-ab20-2c4a7c40a472 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:08.687658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffadaa-a586-46a5-8f18-8221b1daeefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:08.688480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffadaa-a586-46a5-8f18-8221b1daeefc admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:08.688944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffadaa-a586-46a5-8f18-8221b1daeefc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:08.689538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffadaa-a586-46a5-8f18-8221b1daeefc admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:08.691178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ffadaa-a586-46a5-8f18-8221b1daeefc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:08.693061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ffadaa-a586-46a5-8f18-8221b1daeefc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:08.712681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f57a38-bf4f-4867-bae9-5faee6846a77 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:08.724999 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2291/4655] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:08.730421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f57a38-bf4f-4867-bae9-5faee6846a77 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:08.737508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96123c-7baa-4185-a4ce-ebd8585d548a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:08.738344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96123c-7baa-4185-a4ce-ebd8585d548a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:08.740934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96123c-7baa-4185-a4ce-ebd8585d548a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:08.741481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96123c-7baa-4185-a4ce-ebd8585d548a admin admin] PATH_INFO: `/v3/projects/b62306f975b74ace932b3fff910174a6/users/30ce854add81498d8b13e3152f3e9eef/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:08.782315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96123c-7baa-4185-a4ce-ebd8585d548a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62306f975b74ace932b3fff910174a6, user_id=30ce854add81498d8b13e3152f3e9eef, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:08.785215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96123c-7baa-4185-a4ce-ebd8585d548a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:08.813986 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2365/4656] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:06:08 2022] PUT /identity/v3/projects/b62306f975b74ace932b3fff910174a6/users/30ce854add81498d8b13e3152f3e9eef/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:08.818395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffadaa-a586-46a5-8f18-8221b1daeefc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:08.853322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffadaa-a586-46a5-8f18-8221b1daeefc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:09.024875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946c959-36d0-4707-bfdc-a65af45b77ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:09.027675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946c959-36d0-4707-bfdc-a65af45b77ac admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:09.028090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946c959-36d0-4707-bfdc-a65af45b77ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:09.028487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946c959-36d0-4707-bfdc-a65af45b77ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:09.029294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946c959-36d0-4707-bfdc-a65af45b77ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:09.040347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946c959-36d0-4707-bfdc-a65af45b77ac admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:09.043436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:09.072027 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2292/4657] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:09.077348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96123c-7baa-4185-a4ce-ebd8585d548a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:09.126804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96123c-7baa-4185-a4ce-ebd8585d548a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:09.158600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b2de0-f65b-4d78-b79b-c89fd7c13a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:09.159296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b2de0-f65b-4d78-b79b-c89fd7c13a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:09.159584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b2de0-f65b-4d78-b79b-c89fd7c13a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:09.159863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b2de0-f65b-4d78-b79b-c89fd7c13a7b admin admin] PATH_INFO: `/v3/projects/b62306f975b74ace932b3fff910174a6/users/30ce854add81498d8b13e3152f3e9eef/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:09.162036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b2de0-f65b-4d78-b79b-c89fd7c13a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62306f975b74ace932b3fff910174a6, user_id=30ce854add81498d8b13e3152f3e9eef, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:09.165283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b2de0-f65b-4d78-b79b-c89fd7c13a7b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:09.208381 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2366/4658] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:06:09 2022] PUT /identity/v3/projects/b62306f975b74ace932b3fff910174a6/users/30ce854add81498d8b13e3152f3e9eef/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:09.210820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa435d2-db19-4b1d-a240-8563ba6588e0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:09.211402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa435d2-db19-4b1d-a240-8563ba6588e0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:09.211859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa435d2-db19-4b1d-a240-8563ba6588e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:09.314615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8fa435d2-db19-4b1d-a240-8563ba6588e0 None None] MFA Rules not processed for user `30ce854add81498d8b13e3152f3e9eef`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:09.330275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8fa435d2-db19-4b1d-a240-8563ba6588e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:09.460142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b2de0-f65b-4d78-b79b-c89fd7c13a7b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:09.462215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-763b2de0-f65b-4d78-b79b-c89fd7c13a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:09.519412 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2293/4659] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:09.529785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa435d2-db19-4b1d-a240-8563ba6588e0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:09.531692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8fa435d2-db19-4b1d-a240-8563ba6588e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:09.600353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:09.625070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa435d2-db19-4b1d-a240-8563ba6588e0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:09.630594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47b233-c201-4e00-869f-a5fffda0b8fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:09.632072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47b233-c201-4e00-869f-a5fffda0b8fc service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:09.632461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47b233-c201-4e00-869f-a5fffda0b8fc service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:09.632848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47b233-c201-4e00-869f-a5fffda0b8fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:09.690187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b2de0-f65b-4d78-b79b-c89fd7c13a7b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:09.707053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c25a3-cba2-4c45-9fc0-398a0b8a244a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:09.708463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c25a3-cba2-4c45-9fc0-398a0b8a244a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:09.708929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c25a3-cba2-4c45-9fc0-398a0b8a244a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:09.709386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c25a3-cba2-4c45-9fc0-398a0b8a244a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:09.715118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d80c25a3-cba2-4c45-9fc0-398a0b8a244a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:09.734012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47b233-c201-4e00-869f-a5fffda0b8fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:09.735858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47b233-c201-4e00-869f-a5fffda0b8fc service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:09.745559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c25a3-cba2-4c45-9fc0-398a0b8a244a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:09.749076 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2294/4660] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:06:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:06:09.755156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c25a3-cba2-4c45-9fc0-398a0b8a244a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:09.776610 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2367/4661] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:06:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:06:13.955296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47b233-c201-4e00-869f-a5fffda0b8fc service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:14.050773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:14.077425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47b233-c201-4e00-869f-a5fffda0b8fc service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:14.082404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:14.083266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:14.083645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:14.084591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] PATH_INFO: `/v3/users/227adbc59c4244c3a7dafbd2eebab48c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:14.085942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=227adbc59c4244c3a7dafbd2eebab48c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:14.087997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:14.234816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '227adbc59c4244c3a7dafbd2eebab48c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:14.270079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '227adbc59c4244c3a7dafbd2eebab48c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:14.273710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '227adbc59c4244c3a7dafbd2eebab48c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:14.293175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '227adbc59c4244c3a7dafbd2eebab48c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:14.307299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '227adbc59c4244c3a7dafbd2eebab48c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:14.317232 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2295/4662] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:06:13 2022] DELETE /identity/v3/users/227adbc59c4244c3a7dafbd2eebab48c => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:14.321514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca7ff2-c73a-4007-ad59-f63942d12d49 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:14.322350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca7ff2-c73a-4007-ad59-f63942d12d49 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:14.322864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca7ff2-c73a-4007-ad59-f63942d12d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:14.370852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-31ca7ff2-c73a-4007-ad59-f63942d12d49 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:14.372641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-31ca7ff2-c73a-4007-ad59-f63942d12d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:14.514314 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2368/4663] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:14.573059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:14.573059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:14.721719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27371018-1621-43b6-93d9-55e162b75c3a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:14.742408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0a3a5-89df-44bd-8fb0-5be4a77f6904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:14.745970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0a3a5-89df-44bd-8fb0-5be4a77f6904 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:14.746462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0a3a5-89df-44bd-8fb0-5be4a77f6904 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:14.749127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0a3a5-89df-44bd-8fb0-5be4a77f6904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:14.782182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0a3a5-89df-44bd-8fb0-5be4a77f6904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:14.789233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0a3a5-89df-44bd-8fb0-5be4a77f6904 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:14.821309 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2296/4664] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:06:15.532977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca7ff2-c73a-4007-ad59-f63942d12d49 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:15.537145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-31ca7ff2-c73a-4007-ad59-f63942d12d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:15.563500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca7ff2-c73a-4007-ad59-f63942d12d49 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:15.572274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:15.573451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:15.573891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:15.574317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] PATH_INFO: `/v3/users/ac6eac3e9b65428ba9430970e9cbccaf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:15.576039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac6eac3e9b65428ba9430970e9cbccaf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:15.578182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:15.650023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac6eac3e9b65428ba9430970e9cbccaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:15.665008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac6eac3e9b65428ba9430970e9cbccaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:15.675294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac6eac3e9b65428ba9430970e9cbccaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:15.676356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac6eac3e9b65428ba9430970e9cbccaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:15.677271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac6eac3e9b65428ba9430970e9cbccaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:15.712250 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2369/4665] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:06:15 2022] DELETE /identity/v3/users/ac6eac3e9b65428ba9430970e9cbccaf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:16.315227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0a3a5-89df-44bd-8fb0-5be4a77f6904 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:16.317060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f8c0a3a5-89df-44bd-8fb0-5be4a77f6904 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:16.332704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0a3a5-89df-44bd-8fb0-5be4a77f6904 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:16.450508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:16.457282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:16.458355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:16.459431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:16.459876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] PATH_INFO: `/v3/projects/e2571b47f5984b8786e35e81653e2b78` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:16.461787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2571b47f5984b8786e35e81653e2b78)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:16.464579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:16.550409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2571b47f5984b8786e35e81653e2b78', 'request_id': 'req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:16.570256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2571b47f5984b8786e35e81653e2b78', 'request_id': 'req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:16.571468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:16.574291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:16.580199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2571b47f5984b8786e35e81653e2b78', 'request_id': 'req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:16.603850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c18fe-5f43-4dc8-972c-e80472f7d7b1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:16.613402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] The token cache is being invalidate because project e2571b47f5984b8786e35e81653e2b78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:16.614005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:16.615596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:16.620276 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2297/4666] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:16 2022] DELETE /identity/v3/projects/e2571b47f5984b8786e35e81653e2b78 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:16.621402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:16.624579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:16.670897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:16.701431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed4eda-2044-4b52-a680-bae9b9a8663d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:16.703257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed4eda-2044-4b52-a680-bae9b9a8663d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:16.703942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed4eda-2044-4b52-a680-bae9b9a8663d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:16.704432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed4eda-2044-4b52-a680-bae9b9a8663d admin admin] PATH_INFO: `/v3/projects/a6bda3914676415c8c173972749ef368` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:16.706417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed4eda-2044-4b52-a680-bae9b9a8663d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6bda3914676415c8c173972749ef368)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:16.710643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed4eda-2044-4b52-a680-bae9b9a8663d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:16.789082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac87d0-c5db-4030-b2cb-cbaefbd4972c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:16.813360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-39ed4eda-2044-4b52-a680-bae9b9a8663d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6bda3914676415c8c173972749ef368', 'request_id': 'req-39ed4eda-2044-4b52-a680-bae9b9a8663d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:16.829064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-39ed4eda-2044-4b52-a680-bae9b9a8663d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6bda3914676415c8c173972749ef368', 'request_id': 'req-39ed4eda-2044-4b52-a680-bae9b9a8663d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:16.850499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-39ed4eda-2044-4b52-a680-bae9b9a8663d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6bda3914676415c8c173972749ef368', 'request_id': 'req-39ed4eda-2044-4b52-a680-bae9b9a8663d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:16.887977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-39ed4eda-2044-4b52-a680-bae9b9a8663d admin admin] The token cache is being invalidate because project a6bda3914676415c8c173972749ef368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:16.889929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-39ed4eda-2044-4b52-a680-bae9b9a8663d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:16.890916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-39ed4eda-2044-4b52-a680-bae9b9a8663d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:16.905079 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2370/4667] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:16 2022] DELETE /identity/v3/projects/a6bda3914676415c8c173972749ef368 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:16.907494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f3d51-bda1-495b-86bc-754f8153e5a0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:16.907793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f3d51-bda1-495b-86bc-754f8153e5a0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:16.908080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f3d51-bda1-495b-86bc-754f8153e5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:16.976512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:16.977442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:16.977915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:16.978263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] PATH_INFO: `/v3/users/30ce854add81498d8b13e3152f3e9eef` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:16.981049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ce854add81498d8b13e3152f3e9eef)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:16.987723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:17.007163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5d0f3d51-bda1-495b-86bc-754f8153e5a0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:17.009354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5d0f3d51-bda1-495b-86bc-754f8153e5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:17.053689 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2371/4668] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:17.055346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dddd1d-89d0-463d-896b-0a7c35fa0bfa None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:17.055644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dddd1d-89d0-463d-896b-0a7c35fa0bfa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:17.055919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dddd1d-89d0-463d-896b-0a7c35fa0bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:17.127378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:17.143092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ce854add81498d8b13e3152f3e9eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:17.146959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-30dddd1d-89d0-463d-896b-0a7c35fa0bfa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:17.150906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-30dddd1d-89d0-463d-896b-0a7c35fa0bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:17.186036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ce854add81498d8b13e3152f3e9eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:17.187138 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2372/4669] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:17.187487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ce854add81498d8b13e3152f3e9eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:17.191483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dddd1d-89d0-463d-896b-0a7c35fa0bfa None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:17.212673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ce854add81498d8b13e3152f3e9eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:17.224872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dddd1d-89d0-463d-896b-0a7c35fa0bfa None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:17.237783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ce854add81498d8b13e3152f3e9eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:17.245876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3615d-de4a-478d-aa4e-4a4a37789b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:17.246507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3615d-de4a-478d-aa4e-4a4a37789b40 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:17.246794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3615d-de4a-478d-aa4e-4a4a37789b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:17.247064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3615d-de4a-478d-aa4e-4a4a37789b40 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:17.247732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3615d-de4a-478d-aa4e-4a4a37789b40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:06:17.248255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3615d-de4a-478d-aa4e-4a4a37789b40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:17.251285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3615d-de4a-478d-aa4e-4a4a37789b40 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:17.259626 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2298/4670] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:06:16 2022] DELETE /identity/v3/users/30ce854add81498d8b13e3152f3e9eef => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:06:17.274092 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2373/4671] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:06:17.278830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:17.280507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:17.357784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798a866-0f7c-4845-910b-3e55992a7f44 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:17.364944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba2d33-309b-4f8f-ac4f-d9baa1d124c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:17.366329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba2d33-309b-4f8f-ac4f-d9baa1d124c0 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:17.366626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba2d33-309b-4f8f-ac4f-d9baa1d124c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:17.366967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba2d33-309b-4f8f-ac4f-d9baa1d124c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:17.367614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba2d33-309b-4f8f-ac4f-d9baa1d124c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:17.369760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba2d33-309b-4f8f-ac4f-d9baa1d124c0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:17.435489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3615d-de4a-478d-aa4e-4a4a37789b40 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:17.448843 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2299/4672] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:06:17 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:17.449597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3615d-de4a-478d-aa4e-4a4a37789b40 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:17.552325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72656fe-f801-44b0-8eb4-1b31828cc493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:17.553585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72656fe-f801-44b0-8eb4-1b31828cc493 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:17.553955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72656fe-f801-44b0-8eb4-1b31828cc493 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:17.554272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72656fe-f801-44b0-8eb4-1b31828cc493 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:17.554926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72656fe-f801-44b0-8eb4-1b31828cc493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:17.556869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72656fe-f801-44b0-8eb4-1b31828cc493 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:17.596096 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2374/4673] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:06:17 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:17.601522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba2d33-309b-4f8f-ac4f-d9baa1d124c0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:17.613852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba2d33-309b-4f8f-ac4f-d9baa1d124c0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:17.620999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15626ffb-7ccc-4dc1-a8ad-ff4550ec27fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:17.621731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15626ffb-7ccc-4dc1-a8ad-ff4550ec27fe admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:17.622065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15626ffb-7ccc-4dc1-a8ad-ff4550ec27fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:17.622560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15626ffb-7ccc-4dc1-a8ad-ff4550ec27fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:17.623243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15626ffb-7ccc-4dc1-a8ad-ff4550ec27fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:17.624960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15626ffb-7ccc-4dc1-a8ad-ff4550ec27fe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:17.640510 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2300/4674] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:17.642297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72656fe-f801-44b0-8eb4-1b31828cc493 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:17.657866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72656fe-f801-44b0-8eb4-1b31828cc493 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:17.665499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458aa24b-c2c6-4389-917b-70777a71fbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:17.666247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aa24b-c2c6-4389-917b-70777a71fbaf admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:17.666702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aa24b-c2c6-4389-917b-70777a71fbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:17.667159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aa24b-c2c6-4389-917b-70777a71fbaf admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:17.668342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aa24b-c2c6-4389-917b-70777a71fbaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:17.670747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aa24b-c2c6-4389-917b-70777a71fbaf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:17.687271 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2375/4675] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:17.691825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15626ffb-7ccc-4dc1-a8ad-ff4550ec27fe admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:17.715696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15626ffb-7ccc-4dc1-a8ad-ff4550ec27fe admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:17.721439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fea412-c6ec-4f46-8af8-fa726a9ebe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:17.721599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fea412-c6ec-4f46-8af8-fa726a9ebe38 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:17.721860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fea412-c6ec-4f46-8af8-fa726a9ebe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:17.722142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fea412-c6ec-4f46-8af8-fa726a9ebe38 admin admin] PATH_INFO: `/v3/projects/8db106e653804866a09e333bddc1cf3d/users/27989e2a2d014e52913ca77e0448fc63/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:17.756733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fea412-c6ec-4f46-8af8-fa726a9ebe38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db106e653804866a09e333bddc1cf3d, user_id=27989e2a2d014e52913ca77e0448fc63, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:17.760610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fea412-c6ec-4f46-8af8-fa726a9ebe38 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:17.788019 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2301/4676] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:06:17 2022] PUT /identity/v3/projects/8db106e653804866a09e333bddc1cf3d/users/27989e2a2d014e52913ca77e0448fc63/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:17.790471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3899a-1d30-4e34-8918-5fb353944e15 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:17.791100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3899a-1d30-4e34-8918-5fb353944e15 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:17.791398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3899a-1d30-4e34-8918-5fb353944e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:17.831247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e3a3899a-1d30-4e34-8918-5fb353944e15 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:17.833770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e3a3899a-1d30-4e34-8918-5fb353944e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:17.875044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:17.929282 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2376/4677] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:17.973037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fea412-c6ec-4f46-8af8-fa726a9ebe38 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:17.975866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c1fea412-c6ec-4f46-8af8-fa726a9ebe38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:18.134952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fea412-c6ec-4f46-8af8-fa726a9ebe38 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:18.145267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef8bdd-7e48-4047-ad94-ef41b1e3a621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:18.146127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef8bdd-7e48-4047-ad94-ef41b1e3a621 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:18.148646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef8bdd-7e48-4047-ad94-ef41b1e3a621 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:18.149065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef8bdd-7e48-4047-ad94-ef41b1e3a621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:18.174999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef8bdd-7e48-4047-ad94-ef41b1e3a621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:18.176933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef8bdd-7e48-4047-ad94-ef41b1e3a621 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:18.189049 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2302/4678] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:06:18.205869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3899a-1d30-4e34-8918-5fb353944e15 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:18.207659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e3a3899a-1d30-4e34-8918-5fb353944e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:18.231694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3899a-1d30-4e34-8918-5fb353944e15 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:18.242413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:18.243753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:18.244404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:18.244843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] PATH_INFO: `/v3/projects/b62306f975b74ace932b3fff910174a6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:18.246611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b62306f975b74ace932b3fff910174a6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:18.256630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:18.319978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b62306f975b74ace932b3fff910174a6', 'request_id': 'req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:18.330683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b62306f975b74ace932b3fff910174a6', 'request_id': 'req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:18.341304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b62306f975b74ace932b3fff910174a6', 'request_id': 'req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:18.358959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] The token cache is being invalidate because project b62306f975b74ace932b3fff910174a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:18.360293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:18.360816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:18.363118 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2377/4679] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:18 2022] DELETE /identity/v3/projects/b62306f975b74ace932b3fff910174a6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:19.950550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac5aa-1cf5-456b-b7de-54262285eb7e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:19.951133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac5aa-1cf5-456b-b7de-54262285eb7e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:19.951396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac5aa-1cf5-456b-b7de-54262285eb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:19.968978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:19.981640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-435ac5aa-1cf5-456b-b7de-54262285eb7e None None] MFA Rules not processed for user `9e686d314d164930bbcfc9905a8b45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:19.984311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-435ac5aa-1cf5-456b-b7de-54262285eb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:20.077138 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2303/4680] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:21.401542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:21.403066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:21.535232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e6339-1cec-4c56-8891-f08e4fbf8a9c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:21.544431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858ecab-169a-450e-9730-0055a3e39563 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:21.544878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858ecab-169a-450e-9730-0055a3e39563 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:21.545288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858ecab-169a-450e-9730-0055a3e39563 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:21.545719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858ecab-169a-450e-9730-0055a3e39563 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:21.549172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9858ecab-169a-450e-9730-0055a3e39563 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:21.564363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858ecab-169a-450e-9730-0055a3e39563 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:21.566630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858ecab-169a-450e-9730-0055a3e39563 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:21.586406 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2378/4681] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:06:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:06:21.695602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d54fdc-9604-417c-9f81-593f3285eb2d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:21.696111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d54fdc-9604-417c-9f81-593f3285eb2d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:21.696213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d54fdc-9604-417c-9f81-593f3285eb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:21.730554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-98d54fdc-9604-417c-9f81-593f3285eb2d None None] MFA Rules not processed for user `9e686d314d164930bbcfc9905a8b45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:21.732068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-98d54fdc-9604-417c-9f81-593f3285eb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:21.744301 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2304/4682] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:22.289103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858ecab-169a-450e-9730-0055a3e39563 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:22.291329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9858ecab-169a-450e-9730-0055a3e39563 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:22.417101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:22.466242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858ecab-169a-450e-9730-0055a3e39563 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:22.478645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95621503-56c0-4977-8d59-e3f69887c5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:22.479323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95621503-56c0-4977-8d59-e3f69887c5a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:22.479610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95621503-56c0-4977-8d59-e3f69887c5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:22.479891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95621503-56c0-4977-8d59-e3f69887c5a0 admin admin] PATH_INFO: `/v3/users/1b2dd96d938b41d4ba40007aa072288b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:22.481573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95621503-56c0-4977-8d59-e3f69887c5a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b2dd96d938b41d4ba40007aa072288b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:22.488174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95621503-56c0-4977-8d59-e3f69887c5a0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:22.566783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-95621503-56c0-4977-8d59-e3f69887c5a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b2dd96d938b41d4ba40007aa072288b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:22.588804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-95621503-56c0-4977-8d59-e3f69887c5a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b2dd96d938b41d4ba40007aa072288b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:22.599066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-95621503-56c0-4977-8d59-e3f69887c5a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b2dd96d938b41d4ba40007aa072288b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:22.600368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-95621503-56c0-4977-8d59-e3f69887c5a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b2dd96d938b41d4ba40007aa072288b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:22.601746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-95621503-56c0-4977-8d59-e3f69887c5a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b2dd96d938b41d4ba40007aa072288b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:22.620945 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2379/4683] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:06:22 2022] DELETE /identity/v3/users/1b2dd96d938b41d4ba40007aa072288b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:23.229038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d54fdc-9604-417c-9f81-593f3285eb2d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:23.251543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d54fdc-9604-417c-9f81-593f3285eb2d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:23.416964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677fe34-f627-47a5-95b8-926fb1df58c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:23.418340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677fe34-f627-47a5-95b8-926fb1df58c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:23.419412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677fe34-f627-47a5-95b8-926fb1df58c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:23.420270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677fe34-f627-47a5-95b8-926fb1df58c0 admin admin] PATH_INFO: `/v3/projects/a5a5c2699a4b42f5a60fd808e824296b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:23.421518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677fe34-f627-47a5-95b8-926fb1df58c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a5c2699a4b42f5a60fd808e824296b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:23.435469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677fe34-f627-47a5-95b8-926fb1df58c0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:23.551995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6677fe34-f627-47a5-95b8-926fb1df58c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a5c2699a4b42f5a60fd808e824296b', 'request_id': 'req-6677fe34-f627-47a5-95b8-926fb1df58c0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:23.566027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6677fe34-f627-47a5-95b8-926fb1df58c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a5c2699a4b42f5a60fd808e824296b', 'request_id': 'req-6677fe34-f627-47a5-95b8-926fb1df58c0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:23.582045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6677fe34-f627-47a5-95b8-926fb1df58c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a5c2699a4b42f5a60fd808e824296b', 'request_id': 'req-6677fe34-f627-47a5-95b8-926fb1df58c0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:23.598933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6677fe34-f627-47a5-95b8-926fb1df58c0 admin admin] The token cache is being invalidate because project a5a5c2699a4b42f5a60fd808e824296b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:23.599724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6677fe34-f627-47a5-95b8-926fb1df58c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:23.600777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6677fe34-f627-47a5-95b8-926fb1df58c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:23.602896 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2305/4684] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:23 2022] DELETE /identity/v3/projects/a5a5c2699a4b42f5a60fd808e824296b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:23.608793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f7af0-7783-4e70-8d29-d146e564bbe6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:23.609740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f7af0-7783-4e70-8d29-d146e564bbe6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:23.611188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f7af0-7783-4e70-8d29-d146e564bbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:23.637008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cd8f7af0-7783-4e70-8d29-d146e564bbe6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:23.638538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cd8f7af0-7783-4e70-8d29-d146e564bbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:23.740838 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2380/4685] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:23.749025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810cf8b-ec99-4457-a15f-db68099ac3df None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:23.749584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810cf8b-ec99-4457-a15f-db68099ac3df None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:23.750073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810cf8b-ec99-4457-a15f-db68099ac3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:23.775110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2810cf8b-ec99-4457-a15f-db68099ac3df None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:23.776738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2810cf8b-ec99-4457-a15f-db68099ac3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:23.787113 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2306/4686] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:23.791410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f7af0-7783-4e70-8d29-d146e564bbe6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:23.810106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f7af0-7783-4e70-8d29-d146e564bbe6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:23.817298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85505728-74b5-46fb-9d29-d9c3d08cbfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:23.817997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85505728-74b5-46fb-9d29-d9c3d08cbfd3 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:23.818849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85505728-74b5-46fb-9d29-d9c3d08cbfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:23.819179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85505728-74b5-46fb-9d29-d9c3d08cbfd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:23.819848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85505728-74b5-46fb-9d29-d9c3d08cbfd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:06:23.820302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85505728-74b5-46fb-9d29-d9c3d08cbfd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:23.822648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85505728-74b5-46fb-9d29-d9c3d08cbfd3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:23.863414 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2381/4687] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:06:23.868367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810cf8b-ec99-4457-a15f-db68099ac3df None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:23.880380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810cf8b-ec99-4457-a15f-db68099ac3df None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:23.886762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd85507-5b80-4cea-a517-ff38c600546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:23.887507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd85507-5b80-4cea-a517-ff38c600546a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:23.887955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd85507-5b80-4cea-a517-ff38c600546a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:23.888379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd85507-5b80-4cea-a517-ff38c600546a admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:23.889111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd85507-5b80-4cea-a517-ff38c600546a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:23.891210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd85507-5b80-4cea-a517-ff38c600546a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:23.964890 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2307/4688] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:06:23 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:23.969573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85505728-74b5-46fb-9d29-d9c3d08cbfd3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:23.999956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85505728-74b5-46fb-9d29-d9c3d08cbfd3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:24.113005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:24.123572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523c728-911c-4ac0-9e2e-032af30809ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:24.123572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523c728-911c-4ac0-9e2e-032af30809ea admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:24.123572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523c728-911c-4ac0-9e2e-032af30809ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:24.123572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523c728-911c-4ac0-9e2e-032af30809ea admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:24.123913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523c728-911c-4ac0-9e2e-032af30809ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:24.124616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523c728-911c-4ac0-9e2e-032af30809ea admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:24.171368 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2382/4689] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:06:23 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:24.175180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd85507-5b80-4cea-a517-ff38c600546a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:24.197205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd85507-5b80-4cea-a517-ff38c600546a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:24.204178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857e716-6225-4450-86bd-4cd5bec80396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:24.205065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857e716-6225-4450-86bd-4cd5bec80396 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:24.205525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857e716-6225-4450-86bd-4cd5bec80396 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:24.205968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857e716-6225-4450-86bd-4cd5bec80396 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:24.206918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857e716-6225-4450-86bd-4cd5bec80396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:24.211285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857e716-6225-4450-86bd-4cd5bec80396 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:24.218937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:24.246111 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2308/4690] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:24.249710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523c728-911c-4ac0-9e2e-032af30809ea admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:24.265140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523c728-911c-4ac0-9e2e-032af30809ea admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:24.288138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b378191-1b2b-4a41-834a-d7f528caa7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:24.288885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b378191-1b2b-4a41-834a-d7f528caa7bc admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:24.289205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b378191-1b2b-4a41-834a-d7f528caa7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:24.289491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b378191-1b2b-4a41-834a-d7f528caa7bc admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:24.290170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b378191-1b2b-4a41-834a-d7f528caa7bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:24.295864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b378191-1b2b-4a41-834a-d7f528caa7bc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:24.326108 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2383/4691] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:24.330420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857e716-6225-4450-86bd-4cd5bec80396 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:24.351693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857e716-6225-4450-86bd-4cd5bec80396 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:24.378269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f6b8a-1935-4333-aec9-bde3e8b77df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:24.379921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f6b8a-1935-4333-aec9-bde3e8b77df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:24.380364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f6b8a-1935-4333-aec9-bde3e8b77df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:24.382156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f6b8a-1935-4333-aec9-bde3e8b77df1 admin admin] PATH_INFO: `/v3/projects/7cff63264b3b45278b784a4ed9359dd1/users/087a19dd2160472e9dedf9dbee1ca933/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:24.420382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f6b8a-1935-4333-aec9-bde3e8b77df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cff63264b3b45278b784a4ed9359dd1, user_id=087a19dd2160472e9dedf9dbee1ca933, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:24.425639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f6b8a-1935-4333-aec9-bde3e8b77df1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:24.447128 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2309/4692] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:06:24 2022] PUT /identity/v3/projects/7cff63264b3b45278b784a4ed9359dd1/users/087a19dd2160472e9dedf9dbee1ca933/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:24.457546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b5572-0490-47e9-8f5c-0fdba96e7817 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:24.460123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b5572-0490-47e9-8f5c-0fdba96e7817 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:24.460624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b5572-0490-47e9-8f5c-0fdba96e7817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:24.528585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-082b5572-0490-47e9-8f5c-0fdba96e7817 None None] MFA Rules not processed for user `087a19dd2160472e9dedf9dbee1ca933`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:24.529903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-082b5572-0490-47e9-8f5c-0fdba96e7817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:24.634741 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2384/4693] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:24.635887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f6b8a-1935-4333-aec9-bde3e8b77df1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:24.776085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f6b8a-1935-4333-aec9-bde3e8b77df1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:24.781220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c19c27-dacc-4b7d-95e7-2c5c9d885228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:24.781837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c19c27-dacc-4b7d-95e7-2c5c9d885228 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:24.782121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c19c27-dacc-4b7d-95e7-2c5c9d885228 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:24.782550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c19c27-dacc-4b7d-95e7-2c5c9d885228 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:24.783187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c19c27-dacc-4b7d-95e7-2c5c9d885228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:24.785148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c19c27-dacc-4b7d-95e7-2c5c9d885228 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:24.817777 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2310/4694] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:06:24 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:24.821645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b5572-0490-47e9-8f5c-0fdba96e7817 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:24.836761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b5572-0490-47e9-8f5c-0fdba96e7817 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:24.911119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cc47f-517d-41ce-9194-3048d25c9d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:24.911917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cc47f-517d-41ce-9194-3048d25c9d6a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:24.912231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cc47f-517d-41ce-9194-3048d25c9d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:24.912512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cc47f-517d-41ce-9194-3048d25c9d6a admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:24.913204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cc47f-517d-41ce-9194-3048d25c9d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:24.915246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cc47f-517d-41ce-9194-3048d25c9d6a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:24.938450 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2385/4695] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:06:24 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:24.942095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c19c27-dacc-4b7d-95e7-2c5c9d885228 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:24.961964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c19c27-dacc-4b7d-95e7-2c5c9d885228 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:24.967380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d83973-65c1-43af-a515-a649f1b3c1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:24.968300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d83973-65c1-43af-a515-a649f1b3c1f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:24.968413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d83973-65c1-43af-a515-a649f1b3c1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:24.968611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d83973-65c1-43af-a515-a649f1b3c1f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:24.969320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d83973-65c1-43af-a515-a649f1b3c1f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:24.970934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d83973-65c1-43af-a515-a649f1b3c1f2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:24.984958 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2311/4696] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:24.989496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cc47f-517d-41ce-9194-3048d25c9d6a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:25.003350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cc47f-517d-41ce-9194-3048d25c9d6a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:25.009448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371240c2-1a87-4f62-b933-2e8e03256198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:25.010172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371240c2-1a87-4f62-b933-2e8e03256198 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:25.011096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371240c2-1a87-4f62-b933-2e8e03256198 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:25.011393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371240c2-1a87-4f62-b933-2e8e03256198 admin admin] PATH_INFO: `/v3/projects/813253e72655448d88bbfbf32713785e/users/ad9eb2c2e19144d798f7e77ba91a1e8b/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:25.041428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371240c2-1a87-4f62-b933-2e8e03256198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=813253e72655448d88bbfbf32713785e, user_id=ad9eb2c2e19144d798f7e77ba91a1e8b, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:25.046801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371240c2-1a87-4f62-b933-2e8e03256198 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:25.087522 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2386/4697] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:06:24 2022] PUT /identity/v3/projects/813253e72655448d88bbfbf32713785e/users/ad9eb2c2e19144d798f7e77ba91a1e8b/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:25.091598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d83973-65c1-43af-a515-a649f1b3c1f2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:25.107746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d83973-65c1-43af-a515-a649f1b3c1f2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:25.278005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede1adb-1b3a-4dcf-8520-6ac75501dcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:25.279357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede1adb-1b3a-4dcf-8520-6ac75501dcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:25.279970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede1adb-1b3a-4dcf-8520-6ac75501dcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:25.280402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede1adb-1b3a-4dcf-8520-6ac75501dcf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:25.281236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede1adb-1b3a-4dcf-8520-6ac75501dcf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:25.290951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede1adb-1b3a-4dcf-8520-6ac75501dcf8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:25.326854 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2312/4698] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:25.331769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371240c2-1a87-4f62-b933-2e8e03256198 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:25.351680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371240c2-1a87-4f62-b933-2e8e03256198 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:25.358102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cef6f-1dec-4e54-82cd-e40d422d6f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:25.358102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cef6f-1dec-4e54-82cd-e40d422d6f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:25.358102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cef6f-1dec-4e54-82cd-e40d422d6f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:25.358102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cef6f-1dec-4e54-82cd-e40d422d6f4b admin admin] PATH_INFO: `/v3/projects/813253e72655448d88bbfbf32713785e/users/ad9eb2c2e19144d798f7e77ba91a1e8b/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:25.363273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cef6f-1dec-4e54-82cd-e40d422d6f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=813253e72655448d88bbfbf32713785e, user_id=ad9eb2c2e19144d798f7e77ba91a1e8b, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:25.367715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cef6f-1dec-4e54-82cd-e40d422d6f4b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:25.379840 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2387/4699] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:06:25 2022] PUT /identity/v3/projects/813253e72655448d88bbfbf32713785e/users/ad9eb2c2e19144d798f7e77ba91a1e8b/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:25.394397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca068c0d-1e0d-424e-af9e-21af70ff6165 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:25.397646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca068c0d-1e0d-424e-af9e-21af70ff6165 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:25.398101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca068c0d-1e0d-424e-af9e-21af70ff6165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:25.448466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ca068c0d-1e0d-424e-af9e-21af70ff6165 None None] MFA Rules not processed for user `ad9eb2c2e19144d798f7e77ba91a1e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:25.450078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ca068c0d-1e0d-424e-af9e-21af70ff6165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:25.494882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:25.559475 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2313/4700] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:25.561152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cef6f-1dec-4e54-82cd-e40d422d6f4b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:25.578143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cef6f-1dec-4e54-82cd-e40d422d6f4b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:25.582971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2817255-558b-4ac0-94a3-9984f4417147 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:25.583649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2817255-558b-4ac0-94a3-9984f4417147 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:25.584478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2817255-558b-4ac0-94a3-9984f4417147 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:25.584801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2817255-558b-4ac0-94a3-9984f4417147 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:25.586274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2817255-558b-4ac0-94a3-9984f4417147 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:25.588166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2817255-558b-4ac0-94a3-9984f4417147 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:25.602532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:25.639320 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2388/4701] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:06:25 2022] POST /identity/v3/projects => generated 351 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:25.644335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca068c0d-1e0d-424e-af9e-21af70ff6165 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:25.656490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca068c0d-1e0d-424e-af9e-21af70ff6165 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:25.745787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15b92a-8328-4747-b84e-d4cf03549749 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:25.746433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15b92a-8328-4747-b84e-d4cf03549749 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:25.746809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15b92a-8328-4747-b84e-d4cf03549749 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:25.747097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15b92a-8328-4747-b84e-d4cf03549749 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:25.747793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15b92a-8328-4747-b84e-d4cf03549749 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:25.749717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15b92a-8328-4747-b84e-d4cf03549749 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:25.790476 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2314/4702] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:06:25 2022] POST /identity/v3/projects => generated 349 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:06:25.795088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2817255-558b-4ac0-94a3-9984f4417147 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:25.807426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2817255-558b-4ac0-94a3-9984f4417147 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:25.880107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0637103-e3d7-47df-b7c8-fea11b8051e0 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:25.881205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0637103-e3d7-47df-b7c8-fea11b8051e0 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:25.881498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0637103-e3d7-47df-b7c8-fea11b8051e0 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:25.881774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0637103-e3d7-47df-b7c8-fea11b8051e0 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/users/087a19dd2160472e9dedf9dbee1ca933` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:25.883146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0637103-e3d7-47df-b7c8-fea11b8051e0 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:get_user(user_id=087a19dd2160472e9dedf9dbee1ca933)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:25.885497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0637103-e3d7-47df-b7c8-fea11b8051e0 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:25.888942 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2389/4703] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Sep 3 21:06:25 2022] GET /identity/v3/users/087a19dd2160472e9dedf9dbee1ca933 => generated 288 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:06:25.892810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15b92a-8328-4747-b84e-d4cf03549749 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:25.906681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15b92a-8328-4747-b84e-d4cf03549749 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:25.911132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02762065-e207-499b-8847-99dbc41057ae tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:25.911815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02762065-e207-499b-8847-99dbc41057ae tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:25.912193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02762065-e207-499b-8847-99dbc41057ae tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:25.912698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02762065-e207-499b-8847-99dbc41057ae tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:25.913547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02762065-e207-499b-8847-99dbc41057ae tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:25.915540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02762065-e207-499b-8847-99dbc41057ae tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:25.969090 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2315/4704] 10.222.0.16 () {60 vars in 1162 bytes} [Sat Sep 3 21:06:25 2022] POST /identity/v3/credentials => generated 429 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:25.973891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0637103-e3d7-47df-b7c8-fea11b8051e0 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:25.985466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0637103-e3d7-47df-b7c8-fea11b8051e0 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:25.992766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdac91c-6367-4dc8-b8ee-15132d325215 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:25.993959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdac91c-6367-4dc8-b8ee-15132d325215 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:25.994256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdac91c-6367-4dc8-b8ee-15132d325215 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:25.994545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdac91c-6367-4dc8-b8ee-15132d325215 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/credentials/444751fb25fe9c99942c39cdaa4f61613db06464932e820c150bb1ecc250b005` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:25.997006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdac91c-6367-4dc8-b8ee-15132d325215 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=444751fb25fe9c99942c39cdaa4f61613db06464932e820c150bb1ecc250b005)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:25.998842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdac91c-6367-4dc8-b8ee-15132d325215 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.002243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdac91c-6367-4dc8-b8ee-15132d325215 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=444751fb25fe9c99942c39cdaa4f61613db06464932e820c150bb1ecc250b005)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:26.004091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdac91c-6367-4dc8-b8ee-15132d325215 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.021190 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2390/4705] 10.222.0.16 () {60 vars in 1358 bytes} [Sat Sep 3 21:06:25 2022] PATCH /identity/v3/credentials/444751fb25fe9c99942c39cdaa4f61613db06464932e820c150bb1ecc250b005 => generated 436 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:06:26.025791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02762065-e207-499b-8847-99dbc41057ae tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:26.037356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02762065-e207-499b-8847-99dbc41057ae tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:26.042570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cce950-78b9-4e38-bfab-582a8e6707bb tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:26.043215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cce950-78b9-4e38-bfab-582a8e6707bb tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:26.043499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cce950-78b9-4e38-bfab-582a8e6707bb tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:26.043778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cce950-78b9-4e38-bfab-582a8e6707bb tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/credentials/444751fb25fe9c99942c39cdaa4f61613db06464932e820c150bb1ecc250b005` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:26.045837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cce950-78b9-4e38-bfab-582a8e6707bb tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:get_credential(credential_id=444751fb25fe9c99942c39cdaa4f61613db06464932e820c150bb1ecc250b005)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:26.047835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cce950-78b9-4e38-bfab-582a8e6707bb tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.051186 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2316/4706] 10.222.0.16 () {58 vars in 1335 bytes} [Sat Sep 3 21:06:26 2022] GET /identity/v3/credentials/444751fb25fe9c99942c39cdaa4f61613db06464932e820c150bb1ecc250b005 => generated 436 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:06:26.054987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdac91c-6367-4dc8-b8ee-15132d325215 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:26.065855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdac91c-6367-4dc8-b8ee-15132d325215 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:26.070944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98503ca4-3fc9-4ea0-afea-b17a7cc45e31 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:26.071582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98503ca4-3fc9-4ea0-afea-b17a7cc45e31 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:26.071880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98503ca4-3fc9-4ea0-afea-b17a7cc45e31 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:26.072166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98503ca4-3fc9-4ea0-afea-b17a7cc45e31 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/credentials/444751fb25fe9c99942c39cdaa4f61613db06464932e820c150bb1ecc250b005` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:26.074376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98503ca4-3fc9-4ea0-afea-b17a7cc45e31 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=444751fb25fe9c99942c39cdaa4f61613db06464932e820c150bb1ecc250b005)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:26.076209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98503ca4-3fc9-4ea0-afea-b17a7cc45e31 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.105108 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2391/4707] 10.222.0.16 () {58 vars in 1338 bytes} [Sat Sep 3 21:06:26 2022] DELETE /identity/v3/credentials/444751fb25fe9c99942c39cdaa4f61613db06464932e820c150bb1ecc250b005 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:06:26.105864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cce950-78b9-4e38-bfab-582a8e6707bb tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:26.119898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cce950-78b9-4e38-bfab-582a8e6707bb tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:26.130033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474882e-20e5-4e1e-ac7a-9390f25f7831 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:26.131812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474882e-20e5-4e1e-ac7a-9390f25f7831 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:26.132481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474882e-20e5-4e1e-ac7a-9390f25f7831 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:26.133184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474882e-20e5-4e1e-ac7a-9390f25f7831 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:26.134072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474882e-20e5-4e1e-ac7a-9390f25f7831 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:26.138514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474882e-20e5-4e1e-ac7a-9390f25f7831 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.158739 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2317/4708] 10.222.0.16 () {60 vars in 1162 bytes} [Sat Sep 3 21:06:26 2022] POST /identity/v3/credentials => generated 430 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:06:26.162744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98503ca4-3fc9-4ea0-afea-b17a7cc45e31 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:26.178123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98503ca4-3fc9-4ea0-afea-b17a7cc45e31 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:26.187700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f49216-e024-454d-aafd-42072f65dc4e tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:26.188597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f49216-e024-454d-aafd-42072f65dc4e tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:26.189110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f49216-e024-454d-aafd-42072f65dc4e tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:26.189594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f49216-e024-454d-aafd-42072f65dc4e tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:26.190438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f49216-e024-454d-aafd-42072f65dc4e tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:26.194667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f49216-e024-454d-aafd-42072f65dc4e tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.220932 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2392/4709] 10.222.0.16 () {60 vars in 1162 bytes} [Sat Sep 3 21:06:26 2022] POST /identity/v3/credentials => generated 431 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:26.225151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474882e-20e5-4e1e-ac7a-9390f25f7831 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:26.236253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474882e-20e5-4e1e-ac7a-9390f25f7831 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:26.243528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa94bb0-b9b5-49a6-be4f-f54beef75a4c tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:26.244305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa94bb0-b9b5-49a6-be4f-f54beef75a4c tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:26.244670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa94bb0-b9b5-49a6-be4f-f54beef75a4c tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:26.244998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa94bb0-b9b5-49a6-be4f-f54beef75a4c tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:26.245625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa94bb0-b9b5-49a6-be4f-f54beef75a4c tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:list_credentials()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:26.247219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa94bb0-b9b5-49a6-be4f-f54beef75a4c tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.254914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa94bb0-b9b5-49a6-be4f-f54beef75a4c tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:26.256627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa94bb0-b9b5-49a6-be4f-f54beef75a4c tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.258222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa94bb0-b9b5-49a6-be4f-f54beef75a4c tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:26.259816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa94bb0-b9b5-49a6-be4f-f54beef75a4c tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.261952 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2318/4710] 10.222.0.16 () {58 vars in 1140 bytes} [Sat Sep 3 21:06:26 2022] GET /identity/v3/credentials => generated 946 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:06:26.266158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f49216-e024-454d-aafd-42072f65dc4e tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:26.280883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f49216-e024-454d-aafd-42072f65dc4e tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:26.285959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0c795-311e-4805-8193-a2fab022bb7a tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:26.286669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0c795-311e-4805-8193-a2fab022bb7a tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:26.287571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0c795-311e-4805-8193-a2fab022bb7a tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:26.287918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0c795-311e-4805-8193-a2fab022bb7a tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/credentials/40f7fae3ef6976a8fc55165a2fec5962bef69facc065d4d8eaebdf9f1f48d291` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:26.290012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0c795-311e-4805-8193-a2fab022bb7a tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=40f7fae3ef6976a8fc55165a2fec5962bef69facc065d4d8eaebdf9f1f48d291)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:26.291864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0c795-311e-4805-8193-a2fab022bb7a tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.308804 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2393/4711] 10.222.0.16 () {58 vars in 1338 bytes} [Sat Sep 3 21:06:26 2022] DELETE /identity/v3/credentials/40f7fae3ef6976a8fc55165a2fec5962bef69facc065d4d8eaebdf9f1f48d291 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:26.312548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa94bb0-b9b5-49a6-be4f-f54beef75a4c tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:26.341029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa94bb0-b9b5-49a6-be4f-f54beef75a4c tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:26.350123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359ad46-6481-48bc-8c40-4174c9ce86db tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:26.351249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359ad46-6481-48bc-8c40-4174c9ce86db tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:26.352052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359ad46-6481-48bc-8c40-4174c9ce86db tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:26.352344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359ad46-6481-48bc-8c40-4174c9ce86db tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/credentials/3a2d02b065e89bd0c95672640d2d614fbe370dc72a9eae854a710408555b9609` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:26.355768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359ad46-6481-48bc-8c40-4174c9ce86db tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=3a2d02b065e89bd0c95672640d2d614fbe370dc72a9eae854a710408555b9609)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:26.358286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359ad46-6481-48bc-8c40-4174c9ce86db tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.421556 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2319/4712] 10.222.0.16 () {58 vars in 1338 bytes} [Sat Sep 3 21:06:26 2022] DELETE /identity/v3/credentials/3a2d02b065e89bd0c95672640d2d614fbe370dc72a9eae854a710408555b9609 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:06:26.427282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0c795-311e-4805-8193-a2fab022bb7a tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:26.448223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0c795-311e-4805-8193-a2fab022bb7a tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:26.481609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:26.482268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:26.482558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:26.482841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/projects/b31138916cca4d57bb893fff5203b445` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:26.484527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b31138916cca4d57bb893fff5203b445)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:26.486599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.602645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31138916cca4d57bb893fff5203b445', 'request_id': 'req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:26.626270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31138916cca4d57bb893fff5203b445', 'request_id': 'req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:26.635881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31138916cca4d57bb893fff5203b445', 'request_id': 'req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:26.648363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] The token cache is being invalidate because project b31138916cca4d57bb893fff5203b445 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:26.649366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:26.650644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:26.655376 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2394/4713] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:26 2022] DELETE /identity/v3/projects/b31138916cca4d57bb893fff5203b445 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:26.658098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359ad46-6481-48bc-8c40-4174c9ce86db tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:26.661229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f359ad46-6481-48bc-8c40-4174c9ce86db tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:26.679493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359ad46-6481-48bc-8c40-4174c9ce86db tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:26.860281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:26.877584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52016d52-a757-4dd4-b1c0-41cb3bd09aea tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9eb2c2e19144d798f7e77ba91a1e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813253e72655448d88bbfbf32713785e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:26.878881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52016d52-a757-4dd4-b1c0-41cb3bd09aea tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:26.880144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52016d52-a757-4dd4-b1c0-41cb3bd09aea tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:26.880707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52016d52-a757-4dd4-b1c0-41cb3bd09aea tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] PATH_INFO: `/v3/projects/14fa8ffa464e4abc971b5188bd96c3ea` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:26.885973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52016d52-a757-4dd4-b1c0-41cb3bd09aea tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorizing `identity:delete_project(project_id=14fa8ffa464e4abc971b5188bd96c3ea)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:26.890447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52016d52-a757-4dd4-b1c0-41cb3bd09aea tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:26.982013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-52016d52-a757-4dd4-b1c0-41cb3bd09aea tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14fa8ffa464e4abc971b5188bd96c3ea', 'request_id': 'req-52016d52-a757-4dd4-b1c0-41cb3bd09aea'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:26.994335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-52016d52-a757-4dd4-b1c0-41cb3bd09aea tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14fa8ffa464e4abc971b5188bd96c3ea', 'request_id': 'req-52016d52-a757-4dd4-b1c0-41cb3bd09aea'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:27.027804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-52016d52-a757-4dd4-b1c0-41cb3bd09aea tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14fa8ffa464e4abc971b5188bd96c3ea', 'request_id': 'req-52016d52-a757-4dd4-b1c0-41cb3bd09aea'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:27.053736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-52016d52-a757-4dd4-b1c0-41cb3bd09aea tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] The token cache is being invalidate because project 14fa8ffa464e4abc971b5188bd96c3ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:27.054232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-52016d52-a757-4dd4-b1c0-41cb3bd09aea tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:27.055485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-52016d52-a757-4dd4-b1c0-41cb3bd09aea tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:27.058464 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2320/4714] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:26 2022] DELETE /identity/v3/projects/14fa8ffa464e4abc971b5188bd96c3ea => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:27.062042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:27.064329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:27.138799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=9 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:27.222110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6625c0-70f3-41ae-a5f5-eb6c75c5c6f5 tempest-CredentialsTestJSON-448491526 tempest-CredentialsTestJSON-448491526-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:27.298350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a9fb8-d916-4e1f-a18a-219670ede25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:27.298350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a9fb8-d916-4e1f-a18a-219670ede25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:27.298350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a9fb8-d916-4e1f-a18a-219670ede25f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:27.298350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a9fb8-d916-4e1f-a18a-219670ede25f admin admin] PATH_INFO: `/v3/users/087a19dd2160472e9dedf9dbee1ca933` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:27.300840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a9fb8-d916-4e1f-a18a-219670ede25f admin admin] RBAC: Authorizing `identity:delete_user(user_id=087a19dd2160472e9dedf9dbee1ca933)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:27.303472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a9fb8-d916-4e1f-a18a-219670ede25f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:27.384100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-059a9fb8-d916-4e1f-a18a-219670ede25f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '087a19dd2160472e9dedf9dbee1ca933'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:27.401279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-059a9fb8-d916-4e1f-a18a-219670ede25f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '087a19dd2160472e9dedf9dbee1ca933'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:27.409674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-059a9fb8-d916-4e1f-a18a-219670ede25f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '087a19dd2160472e9dedf9dbee1ca933'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:27.410831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-059a9fb8-d916-4e1f-a18a-219670ede25f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '087a19dd2160472e9dedf9dbee1ca933'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:27.411749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-059a9fb8-d916-4e1f-a18a-219670ede25f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '087a19dd2160472e9dedf9dbee1ca933'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:27.439825 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2395/4715] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:06:27 2022] DELETE /identity/v3/users/087a19dd2160472e9dedf9dbee1ca933 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:27.440847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef74c2-cfa2-4789-bc1e-fe020e7cea04 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:27.441338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef74c2-cfa2-4789-bc1e-fe020e7cea04 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:27.441654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef74c2-cfa2-4789-bc1e-fe020e7cea04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:27.469635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-48ef74c2-cfa2-4789-bc1e-fe020e7cea04 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:27.470892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-48ef74c2-cfa2-4789-bc1e-fe020e7cea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:27.551056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a3662-b1cb-4af4-ab73-ed6a7cb7d521 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:27.551492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a3662-b1cb-4af4-ab73-ed6a7cb7d521 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:27.551599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a3662-b1cb-4af4-ab73-ed6a7cb7d521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:27.590771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e74a3662-b1cb-4af4-ab73-ed6a7cb7d521 None None] MFA Rules not processed for user `27989e2a2d014e52913ca77e0448fc63`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:27.592039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e74a3662-b1cb-4af4-ab73-ed6a7cb7d521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:27.600949 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2321/4716] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:06:27.610564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef74c2-cfa2-4789-bc1e-fe020e7cea04 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:27.612736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-48ef74c2-cfa2-4789-bc1e-fe020e7cea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:27.647421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:27.713512 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2396/4717] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:06:27.718789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d82f0-c9c3-4af7-9cab-4ff8999a6b24 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:27.719090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d82f0-c9c3-4af7-9cab-4ff8999a6b24 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:27.719371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d82f0-c9c3-4af7-9cab-4ff8999a6b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:27.720404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef74c2-cfa2-4789-bc1e-fe020e7cea04 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:27.725530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950de6e2-bc2d-4ed2-a61b-2c01e3e46fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:27.726168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950de6e2-bc2d-4ed2-a61b-2c01e3e46fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:27.726460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950de6e2-bc2d-4ed2-a61b-2c01e3e46fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:27.726745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950de6e2-bc2d-4ed2-a61b-2c01e3e46fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:27.742397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5c2d82f0-c9c3-4af7-9cab-4ff8999a6b24 None None] MFA Rules not processed for user `27989e2a2d014e52913ca77e0448fc63`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:27.743978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5c2d82f0-c9c3-4af7-9cab-4ff8999a6b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:27.757183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950de6e2-bc2d-4ed2-a61b-2c01e3e46fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:27.758730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950de6e2-bc2d-4ed2-a61b-2c01e3e46fe0 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:27.761424 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2397/4718] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:27.777122 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2322/4719] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:06:27.837254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d82f0-c9c3-4af7-9cab-4ff8999a6b24 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:27.838875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5c2d82f0-c9c3-4af7-9cab-4ff8999a6b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:27.956056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d82f0-c9c3-4af7-9cab-4ff8999a6b24 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:27.962671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2ab75-b200-4764-848f-9f6cf7a74922 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:27.963496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2ab75-b200-4764-848f-9f6cf7a74922 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:27.963900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2ab75-b200-4764-848f-9f6cf7a74922 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:27.964447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2ab75-b200-4764-848f-9f6cf7a74922 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:27.974260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2ab75-b200-4764-848f-9f6cf7a74922 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:27.976721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2ab75-b200-4764-848f-9f6cf7a74922 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:27.988414 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2398/4720] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:06:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:06:28.383821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950de6e2-bc2d-4ed2-a61b-2c01e3e46fe0 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:28.385361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-950de6e2-bc2d-4ed2-a61b-2c01e3e46fe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:28.414116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950de6e2-bc2d-4ed2-a61b-2c01e3e46fe0 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:28.430009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20e884-5223-4cec-872e-4eda9949c20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:28.430676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20e884-5223-4cec-872e-4eda9949c20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:28.430965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20e884-5223-4cec-872e-4eda9949c20b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:28.431312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20e884-5223-4cec-872e-4eda9949c20b admin admin] PATH_INFO: `/v3/projects/7cff63264b3b45278b784a4ed9359dd1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:28.435143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20e884-5223-4cec-872e-4eda9949c20b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cff63264b3b45278b784a4ed9359dd1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:28.437157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20e884-5223-4cec-872e-4eda9949c20b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:28.496322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:28.543423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba20e884-5223-4cec-872e-4eda9949c20b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cff63264b3b45278b784a4ed9359dd1', 'request_id': 'req-ba20e884-5223-4cec-872e-4eda9949c20b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:28.568186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba20e884-5223-4cec-872e-4eda9949c20b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cff63264b3b45278b784a4ed9359dd1', 'request_id': 'req-ba20e884-5223-4cec-872e-4eda9949c20b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:28.580072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba20e884-5223-4cec-872e-4eda9949c20b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cff63264b3b45278b784a4ed9359dd1', 'request_id': 'req-ba20e884-5223-4cec-872e-4eda9949c20b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:28.604158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba20e884-5223-4cec-872e-4eda9949c20b admin admin] The token cache is being invalidate because project 7cff63264b3b45278b784a4ed9359dd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:28.605220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba20e884-5223-4cec-872e-4eda9949c20b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:28.605974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba20e884-5223-4cec-872e-4eda9949c20b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:28.608017 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2323/4721] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:28 2022] DELETE /identity/v3/projects/7cff63264b3b45278b784a4ed9359dd1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:28.611735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2ab75-b200-4764-848f-9f6cf7a74922 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:28.641918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2ab75-b200-4764-848f-9f6cf7a74922 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:28.758953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:28.759627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:28.759793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:28.760079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] PATH_INFO: `/v3/users/ad9eb2c2e19144d798f7e77ba91a1e8b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:28.761657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad9eb2c2e19144d798f7e77ba91a1e8b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:28.763797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:28.838306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad9eb2c2e19144d798f7e77ba91a1e8b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:28.853297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:28.862831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad9eb2c2e19144d798f7e77ba91a1e8b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:28.873169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad9eb2c2e19144d798f7e77ba91a1e8b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:28.873936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad9eb2c2e19144d798f7e77ba91a1e8b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:28.874667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad9eb2c2e19144d798f7e77ba91a1e8b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:28.898628 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2399/4722] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:06:28 2022] DELETE /identity/v3/users/ad9eb2c2e19144d798f7e77ba91a1e8b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:29.153470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995558-8910-42df-9ac8-2d5c8e8da93f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:29.153911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995558-8910-42df-9ac8-2d5c8e8da93f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:29.154218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995558-8910-42df-9ac8-2d5c8e8da93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:29.179022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5c995558-8910-42df-9ac8-2d5c8e8da93f None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:29.180472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5c995558-8910-42df-9ac8-2d5c8e8da93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:29.276022 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2324/4723] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:29.284351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:29.285840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:29.405466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083962b7-5ed7-4373-af72-c1fe09e0fbab admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:29.412550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16542ff-bdce-4c10-8882-936d95f48037 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:29.413426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16542ff-bdce-4c10-8882-936d95f48037 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:29.413774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16542ff-bdce-4c10-8882-936d95f48037 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:29.414058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16542ff-bdce-4c10-8882-936d95f48037 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:29.426125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16542ff-bdce-4c10-8882-936d95f48037 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:29.429124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16542ff-bdce-4c10-8882-936d95f48037 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:29.439830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c995558-8910-42df-9ac8-2d5c8e8da93f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:29.441408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5c995558-8910-42df-9ac8-2d5c8e8da93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:29.453871 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2400/4724] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:06:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:06:29.613842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c995558-8910-42df-9ac8-2d5c8e8da93f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:29.630089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7e33b-04a0-450a-b2bc-60bc50c66814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:29.630798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7e33b-04a0-450a-b2bc-60bc50c66814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:29.631086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7e33b-04a0-450a-b2bc-60bc50c66814 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:29.631500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7e33b-04a0-450a-b2bc-60bc50c66814 admin admin] PATH_INFO: `/v3/projects/813253e72655448d88bbfbf32713785e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:29.634143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7e33b-04a0-450a-b2bc-60bc50c66814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=813253e72655448d88bbfbf32713785e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:29.636041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7e33b-04a0-450a-b2bc-60bc50c66814 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:29.792796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-06c7e33b-04a0-450a-b2bc-60bc50c66814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '813253e72655448d88bbfbf32713785e', 'request_id': 'req-06c7e33b-04a0-450a-b2bc-60bc50c66814'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:29.814872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-06c7e33b-04a0-450a-b2bc-60bc50c66814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '813253e72655448d88bbfbf32713785e', 'request_id': 'req-06c7e33b-04a0-450a-b2bc-60bc50c66814'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:29.834943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-06c7e33b-04a0-450a-b2bc-60bc50c66814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '813253e72655448d88bbfbf32713785e', 'request_id': 'req-06c7e33b-04a0-450a-b2bc-60bc50c66814'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:29.848002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:29.884769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-06c7e33b-04a0-450a-b2bc-60bc50c66814 admin admin] The token cache is being invalidate because project 813253e72655448d88bbfbf32713785e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:29.885517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-06c7e33b-04a0-450a-b2bc-60bc50c66814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:29.886781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-06c7e33b-04a0-450a-b2bc-60bc50c66814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:29.889606 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2325/4725] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:29 2022] DELETE /identity/v3/projects/813253e72655448d88bbfbf32713785e => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:06:29.894095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd9c2b-fdff-473f-b029-292aeda0f11a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:29.894522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd9c2b-fdff-473f-b029-292aeda0f11a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:29.894656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd9c2b-fdff-473f-b029-292aeda0f11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:29.933110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-45bd9c2b-fdff-473f-b029-292aeda0f11a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:29.934597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-45bd9c2b-fdff-473f-b029-292aeda0f11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:30.097888 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2401/4726] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:30.100205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de018ff-deb3-4a07-ab7c-4c265b8635eb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:30.100512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de018ff-deb3-4a07-ab7c-4c265b8635eb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:30.100797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de018ff-deb3-4a07-ab7c-4c265b8635eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:30.196569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0de018ff-deb3-4a07-ab7c-4c265b8635eb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:30.197896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0de018ff-deb3-4a07-ab7c-4c265b8635eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:30.233630 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2326/4727] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:30.239964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd9c2b-fdff-473f-b029-292aeda0f11a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:30.263642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd9c2b-fdff-473f-b029-292aeda0f11a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:30.272332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b83bd-252a-4402-aa14-5c0bcfad028c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:30.273315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b83bd-252a-4402-aa14-5c0bcfad028c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:30.273857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b83bd-252a-4402-aa14-5c0bcfad028c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:30.274311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b83bd-252a-4402-aa14-5c0bcfad028c admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:30.278696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b83bd-252a-4402-aa14-5c0bcfad028c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:06:30.279019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b83bd-252a-4402-aa14-5c0bcfad028c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:30.281000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b83bd-252a-4402-aa14-5c0bcfad028c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:30.302800 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2402/4728] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:06:30.304548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de018ff-deb3-4a07-ab7c-4c265b8635eb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:30.318002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de018ff-deb3-4a07-ab7c-4c265b8635eb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:30.328379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2c516-500d-4a44-a9bd-acb39aace987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:30.329753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2c516-500d-4a44-a9bd-acb39aace987 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:30.333013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2c516-500d-4a44-a9bd-acb39aace987 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:30.333013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2c516-500d-4a44-a9bd-acb39aace987 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:30.333013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2c516-500d-4a44-a9bd-acb39aace987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:30.335133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2c516-500d-4a44-a9bd-acb39aace987 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:30.403398 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2327/4729] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:06:30 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:30.404331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b83bd-252a-4402-aa14-5c0bcfad028c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:30.423124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b83bd-252a-4402-aa14-5c0bcfad028c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:30.573172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:30.597856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df90ae-1db8-43d4-adf8-f6347dea39a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:30.601729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df90ae-1db8-43d4-adf8-f6347dea39a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:30.602529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df90ae-1db8-43d4-adf8-f6347dea39a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:30.602644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df90ae-1db8-43d4-adf8-f6347dea39a0 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:30.613133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df90ae-1db8-43d4-adf8-f6347dea39a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:30.613133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df90ae-1db8-43d4-adf8-f6347dea39a0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:30.657859 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2403/4730] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:06:30 2022] POST /identity/v3/users => generated 287 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:06:30.662097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2c516-500d-4a44-a9bd-acb39aace987 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:30.679008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2c516-500d-4a44-a9bd-acb39aace987 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:30.695083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd1ee2-a609-4702-950e-98dbc5b3d1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:30.695735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd1ee2-a609-4702-950e-98dbc5b3d1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:30.696026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd1ee2-a609-4702-950e-98dbc5b3d1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:30.696309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd1ee2-a609-4702-950e-98dbc5b3d1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:30.697000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd1ee2-a609-4702-950e-98dbc5b3d1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:30.700979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd1ee2-a609-4702-950e-98dbc5b3d1b3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:30.735103 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2328/4731] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:30.736855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df90ae-1db8-43d4-adf8-f6347dea39a0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:30.756847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df90ae-1db8-43d4-adf8-f6347dea39a0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:30.764960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91debca-3d13-4f4a-bb95-ab962c147954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:30.765960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91debca-3d13-4f4a-bb95-ab962c147954 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:30.766452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91debca-3d13-4f4a-bb95-ab962c147954 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:30.766931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91debca-3d13-4f4a-bb95-ab962c147954 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:30.767837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91debca-3d13-4f4a-bb95-ab962c147954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:30.773648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91debca-3d13-4f4a-bb95-ab962c147954 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:30.805008 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2404/4732] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:30.808629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd1ee2-a609-4702-950e-98dbc5b3d1b3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:30.837297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd1ee2-a609-4702-950e-98dbc5b3d1b3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:30.857107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035d69a-c145-4709-a130-86261ebdbc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:30.858069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035d69a-c145-4709-a130-86261ebdbc3b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:30.858904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035d69a-c145-4709-a130-86261ebdbc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:30.859581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035d69a-c145-4709-a130-86261ebdbc3b admin admin] PATH_INFO: `/v3/projects/db14018d67e840288fa31f7bc0d7de1e/users/de945c49b51441deb2e08c121150aa53/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:30.905314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035d69a-c145-4709-a130-86261ebdbc3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db14018d67e840288fa31f7bc0d7de1e, user_id=de945c49b51441deb2e08c121150aa53, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:30.916296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035d69a-c145-4709-a130-86261ebdbc3b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:30.953638 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2329/4733] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:06:30 2022] PUT /identity/v3/projects/db14018d67e840288fa31f7bc0d7de1e/users/de945c49b51441deb2e08c121150aa53/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:30.965483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a804896-c55f-4949-aa20-2fbf9ccf8678 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:30.966025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a804896-c55f-4949-aa20-2fbf9ccf8678 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:30.966121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a804896-c55f-4949-aa20-2fbf9ccf8678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:31.057947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4a804896-c55f-4949-aa20-2fbf9ccf8678 None None] MFA Rules not processed for user `de945c49b51441deb2e08c121150aa53`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:31.059463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4a804896-c55f-4949-aa20-2fbf9ccf8678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:31.243906 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2405/4734] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:31.248981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035d69a-c145-4709-a130-86261ebdbc3b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:31.263087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035d69a-c145-4709-a130-86261ebdbc3b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:31.289229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95426444-3d6d-4d4e-8e3d-8cf12816183b tempest-IdentityCatalogTest-762846857 tempest-IdentityCatalogTest-762846857-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de945c49b51441deb2e08c121150aa53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db14018d67e840288fa31f7bc0d7de1e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:31.289933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95426444-3d6d-4d4e-8e3d-8cf12816183b tempest-IdentityCatalogTest-762846857 tempest-IdentityCatalogTest-762846857-project] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:31.290216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95426444-3d6d-4d4e-8e3d-8cf12816183b tempest-IdentityCatalogTest-762846857 tempest-IdentityCatalogTest-762846857-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:31.290506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95426444-3d6d-4d4e-8e3d-8cf12816183b tempest-IdentityCatalogTest-762846857 tempest-IdentityCatalogTest-762846857-project] PATH_INFO: `/v3/auth/catalog` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:31.291151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95426444-3d6d-4d4e-8e3d-8cf12816183b tempest-IdentityCatalogTest-762846857 tempest-IdentityCatalogTest-762846857-project] RBAC: Authorizing `identity:get_auth_catalog()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:31.303207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95426444-3d6d-4d4e-8e3d-8cf12816183b tempest-IdentityCatalogTest-762846857 tempest-IdentityCatalogTest-762846857-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:31.305181 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2330/4735] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Sep 3 21:06:31 2022] GET /identity/v3/auth/catalog => generated 2658 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:31.310259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a804896-c55f-4949-aa20-2fbf9ccf8678 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:31.616376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a804896-c55f-4949-aa20-2fbf9ccf8678 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:31.622239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:31.635363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:31.637441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:31.637928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] PATH_INFO: `/v3/users/de945c49b51441deb2e08c121150aa53` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:31.641447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] RBAC: Authorizing `identity:delete_user(user_id=de945c49b51441deb2e08c121150aa53)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:31.644208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:31.757464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de945c49b51441deb2e08c121150aa53'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:31.780661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:31.782452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de945c49b51441deb2e08c121150aa53'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:31.801332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de945c49b51441deb2e08c121150aa53'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:31.802174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de945c49b51441deb2e08c121150aa53'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:31.803006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de945c49b51441deb2e08c121150aa53'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:31.824517 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2406/4736] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:06:31 2022] DELETE /identity/v3/users/de945c49b51441deb2e08c121150aa53 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:31.828672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d39972-63a0-48dc-b7ab-e2565f76fdc5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:31.829210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d39972-63a0-48dc-b7ab-e2565f76fdc5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:31.829286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d39972-63a0-48dc-b7ab-e2565f76fdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:31.867781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-59d39972-63a0-48dc-b7ab-e2565f76fdc5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:31.869203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-59d39972-63a0-48dc-b7ab-e2565f76fdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:32.128587 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2331/4737] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:32.138755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:32.140657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:32.264623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12b985-8592-4703-a806-c9a3fb07185e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:32.274820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca241a-5ce8-4f33-a6fb-f25265739261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:32.275481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca241a-5ce8-4f33-a6fb-f25265739261 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:32.276039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca241a-5ce8-4f33-a6fb-f25265739261 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:32.276146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca241a-5ce8-4f33-a6fb-f25265739261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:32.294682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca241a-5ce8-4f33-a6fb-f25265739261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:32.296391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca241a-5ce8-4f33-a6fb-f25265739261 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:32.333634 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2407/4738] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:06:33.231916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d39972-63a0-48dc-b7ab-e2565f76fdc5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:33.234279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-59d39972-63a0-48dc-b7ab-e2565f76fdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:33.280981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d39972-63a0-48dc-b7ab-e2565f76fdc5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:33.290896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e8697-df0a-405a-b995-0568df29254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:33.292113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e8697-df0a-405a-b995-0568df29254e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:33.292379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e8697-df0a-405a-b995-0568df29254e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:33.292663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e8697-df0a-405a-b995-0568df29254e admin admin] PATH_INFO: `/v3/projects/db14018d67e840288fa31f7bc0d7de1e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:33.298120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e8697-df0a-405a-b995-0568df29254e admin admin] RBAC: Authorizing `identity:delete_project(project_id=db14018d67e840288fa31f7bc0d7de1e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:33.301097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e8697-df0a-405a-b995-0568df29254e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:33.447155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e23e8697-df0a-405a-b995-0568df29254e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db14018d67e840288fa31f7bc0d7de1e', 'request_id': 'req-e23e8697-df0a-405a-b995-0568df29254e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:33.461471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e23e8697-df0a-405a-b995-0568df29254e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db14018d67e840288fa31f7bc0d7de1e', 'request_id': 'req-e23e8697-df0a-405a-b995-0568df29254e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:33.469220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e23e8697-df0a-405a-b995-0568df29254e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db14018d67e840288fa31f7bc0d7de1e', 'request_id': 'req-e23e8697-df0a-405a-b995-0568df29254e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:33.482567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e23e8697-df0a-405a-b995-0568df29254e admin admin] The token cache is being invalidate because project db14018d67e840288fa31f7bc0d7de1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:33.483061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e23e8697-df0a-405a-b995-0568df29254e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:33.483553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e23e8697-df0a-405a-b995-0568df29254e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:33.485600 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2332/4739] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:33 2022] DELETE /identity/v3/projects/db14018d67e840288fa31f7bc0d7de1e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:33.491184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251ec33-baf9-4725-ad15-649f2490247f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:33.491674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251ec33-baf9-4725-ad15-649f2490247f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:33.491751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251ec33-baf9-4725-ad15-649f2490247f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:33.543334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0251ec33-baf9-4725-ad15-649f2490247f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:33.544623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0251ec33-baf9-4725-ad15-649f2490247f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:33.742500 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2408/4740] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:33.751744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb18a6-48ee-4549-8dda-88ba4f82bfef None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:33.752060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb18a6-48ee-4549-8dda-88ba4f82bfef None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:33.752358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb18a6-48ee-4549-8dda-88ba4f82bfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:33.770670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:33.790928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f1cb18a6-48ee-4549-8dda-88ba4f82bfef None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:33.792174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f1cb18a6-48ee-4549-8dda-88ba4f82bfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:33.802807 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2333/4741] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:33.806899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251ec33-baf9-4725-ad15-649f2490247f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:33.836642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251ec33-baf9-4725-ad15-649f2490247f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:33.852857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c3188-d3ce-49a4-8365-bfa9a9583ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:33.853949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c3188-d3ce-49a4-8365-bfa9a9583ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:33.854452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c3188-d3ce-49a4-8365-bfa9a9583ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:33.854942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c3188-d3ce-49a4-8365-bfa9a9583ef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:33.855817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c3188-d3ce-49a4-8365-bfa9a9583ef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:06:33.856351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c3188-d3ce-49a4-8365-bfa9a9583ef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:33.861213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c3188-d3ce-49a4-8365-bfa9a9583ef2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:33.884442 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2409/4742] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:06:33.887694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb18a6-48ee-4549-8dda-88ba4f82bfef None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:33.909536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb18a6-48ee-4549-8dda-88ba4f82bfef None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:33.921175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb169e-5507-4c28-9e7b-4943739684f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:33.922264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb169e-5507-4c28-9e7b-4943739684f0 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:33.922755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb169e-5507-4c28-9e7b-4943739684f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:33.923257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb169e-5507-4c28-9e7b-4943739684f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:33.924096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb169e-5507-4c28-9e7b-4943739684f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:33.926269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb169e-5507-4c28-9e7b-4943739684f0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:33.990572 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2334/4743] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:06:33 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:33.993770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c3188-d3ce-49a4-8365-bfa9a9583ef2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:34.015536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c3188-d3ce-49a4-8365-bfa9a9583ef2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:34.157049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61164f9-38c4-46d1-aec6-0c576cc2ca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:34.158086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61164f9-38c4-46d1-aec6-0c576cc2ca32 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:34.158562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61164f9-38c4-46d1-aec6-0c576cc2ca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:34.159076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61164f9-38c4-46d1-aec6-0c576cc2ca32 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:34.159960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61164f9-38c4-46d1-aec6-0c576cc2ca32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:34.162847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61164f9-38c4-46d1-aec6-0c576cc2ca32 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:34.182045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:34.190312 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2410/4744] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:06:33 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:06:34.194483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb169e-5507-4c28-9e7b-4943739684f0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:34.213629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb169e-5507-4c28-9e7b-4943739684f0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:34.233269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb0bd2-2892-46d3-b5d7-1f742bca282d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:34.234303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb0bd2-2892-46d3-b5d7-1f742bca282d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:34.234477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb0bd2-2892-46d3-b5d7-1f742bca282d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:34.234758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb0bd2-2892-46d3-b5d7-1f742bca282d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:34.235710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb0bd2-2892-46d3-b5d7-1f742bca282d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:34.237200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb0bd2-2892-46d3-b5d7-1f742bca282d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:34.261364 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2335/4745] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:34.266025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61164f9-38c4-46d1-aec6-0c576cc2ca32 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:34.291994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61164f9-38c4-46d1-aec6-0c576cc2ca32 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:34.321827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2d274-c4f6-404b-9739-5d414ceda205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:34.322912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2d274-c4f6-404b-9739-5d414ceda205 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:34.323405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2d274-c4f6-404b-9739-5d414ceda205 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:34.323944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2d274-c4f6-404b-9739-5d414ceda205 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:34.326031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2d274-c4f6-404b-9739-5d414ceda205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:34.328254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2d274-c4f6-404b-9739-5d414ceda205 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:34.356475 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2411/4746] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:34.357324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb0bd2-2892-46d3-b5d7-1f742bca282d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:34.370904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb0bd2-2892-46d3-b5d7-1f742bca282d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:34.385445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88a004-72d0-472d-9906-0af1b2205ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:34.386148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88a004-72d0-472d-9906-0af1b2205ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:34.386441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88a004-72d0-472d-9906-0af1b2205ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:34.386885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88a004-72d0-472d-9906-0af1b2205ed2 admin admin] PATH_INFO: `/v3/projects/4d30f29581ec49e1b9db2cd693a0fccc/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:34.420736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88a004-72d0-472d-9906-0af1b2205ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d30f29581ec49e1b9db2cd693a0fccc, user_id=7792c0517b8e4d9aa2c3bc7a0b2df48a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:34.424539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88a004-72d0-472d-9906-0af1b2205ed2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:34.448718 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2336/4747] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:06:34 2022] PUT /identity/v3/projects/4d30f29581ec49e1b9db2cd693a0fccc/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:34.460550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc5fdd-519a-4ad0-858d-458ec3aea7a3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:34.460868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc5fdd-519a-4ad0-858d-458ec3aea7a3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:34.461137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc5fdd-519a-4ad0-858d-458ec3aea7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:34.531202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-15dc5fdd-519a-4ad0-858d-458ec3aea7a3 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:34.532522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-15dc5fdd-519a-4ad0-858d-458ec3aea7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:34.737020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88a004-72d0-472d-9906-0af1b2205ed2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:34.737687 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2412/4748] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:34.756306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88a004-72d0-472d-9906-0af1b2205ed2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:34.779002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e5aa5-0c78-4cbe-9e10-17c835001672 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7792c0517b8e4d9aa2c3bc7a0b2df48a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d30f29581ec49e1b9db2cd693a0fccc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:34.780050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e5aa5-0c78-4cbe-9e10-17c835001672 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:34.780548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e5aa5-0c78-4cbe-9e10-17c835001672 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:34.781090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e5aa5-0c78-4cbe-9e10-17c835001672 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] PATH_INFO: `/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:34.861347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-728e5aa5-0c78-4cbe-9e10-17c835001672 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:34.899767 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2337/4749] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:06:34 2022] POST /identity/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:06:35.914498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dbd0f-12cf-4ad1-aaf6-be83753c11d8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:35.914969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dbd0f-12cf-4ad1-aaf6-be83753c11d8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:35.916403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dbd0f-12cf-4ad1-aaf6-be83753c11d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:35.945497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-573dbd0f-12cf-4ad1-aaf6-be83753c11d8 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:35.947323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-573dbd0f-12cf-4ad1-aaf6-be83753c11d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:35.954645 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2413/4750] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:35 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:06:35.956272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03dfed-9852-4a9f-afdd-41a3f624dbd1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:35.956747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03dfed-9852-4a9f-afdd-41a3f624dbd1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:35.956862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03dfed-9852-4a9f-afdd-41a3f624dbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:35.984368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6a03dfed-9852-4a9f-afdd-41a3f624dbd1 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:35.985682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6a03dfed-9852-4a9f-afdd-41a3f624dbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:36.009950 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2338/4751] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:36.011139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dbd0f-12cf-4ad1-aaf6-be83753c11d8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:36.023563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dbd0f-12cf-4ad1-aaf6-be83753c11d8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:36.027820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcdcf4-0843-495e-b463-3f464b22ee52 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7792c0517b8e4d9aa2c3bc7a0b2df48a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d30f29581ec49e1b9db2cd693a0fccc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:36.028481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcdcf4-0843-495e-b463-3f464b22ee52 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:36.028769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcdcf4-0843-495e-b463-3f464b22ee52 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:36.029085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcdcf4-0843-495e-b463-3f464b22ee52 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] PATH_INFO: `/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-24dcdcf4-0843-495e-b463-3f464b22ee52 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:06:36.091236 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 03 21:06:36.092145 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:06:36.096822 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2414/4752] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:06:36 2022] POST /identity/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password => generated 188 bytes in 83 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 03 21:06:36.099105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03dfed-9852-4a9f-afdd-41a3f624dbd1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:36.113818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03dfed-9852-4a9f-afdd-41a3f624dbd1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:36.121712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eea820-acba-4074-9e8e-ca4691c84c64 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7792c0517b8e4d9aa2c3bc7a0b2df48a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d30f29581ec49e1b9db2cd693a0fccc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:36.128625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eea820-acba-4074-9e8e-ca4691c84c64 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:36.128625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eea820-acba-4074-9e8e-ca4691c84c64 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:36.129062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eea820-acba-4074-9e8e-ca4691c84c64 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] PATH_INFO: `/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-18eea820-acba-4074-9e8e-ca4691c84c64 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:06:36.221973 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 03 21:06:36.222827 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:06:36.223857 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2339/4753] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:06:36 2022] POST /identity/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password => generated 188 bytes in 126 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 03 21:06:36.228717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcdcf4-0843-495e-b463-3f464b22ee52 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:36.269616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcdcf4-0843-495e-b463-3f464b22ee52 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:36.293981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bb32b-97e0-4be9-b00b-15e0150e436b tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7792c0517b8e4d9aa2c3bc7a0b2df48a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d30f29581ec49e1b9db2cd693a0fccc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:36.294651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bb32b-97e0-4be9-b00b-15e0150e436b tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:36.294933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bb32b-97e0-4be9-b00b-15e0150e436b tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:36.295209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bb32b-97e0-4be9-b00b-15e0150e436b tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] PATH_INFO: `/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:36.388428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ed9bb32b-97e0-4be9-b00b-15e0150e436b tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:36.407770 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2415/4754] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:06:36 2022] POST /identity/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:37.411276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddbd3a-af9c-43f9-b2b9-f79334daffae None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:37.412321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddbd3a-af9c-43f9-b2b9-f79334daffae None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:37.413252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddbd3a-af9c-43f9-b2b9-f79334daffae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:37.434078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-07ddbd3a-af9c-43f9-b2b9-f79334daffae None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:37.435547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-07ddbd3a-af9c-43f9-b2b9-f79334daffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:37.443801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968ff26-f204-44bb-bd15-4148296870b8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:37.444332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968ff26-f204-44bb-bd15-4148296870b8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:37.444665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968ff26-f204-44bb-bd15-4148296870b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:37.449048 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2340/4755] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:37 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:06:37.533203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a968ff26-f204-44bb-bd15-4148296870b8 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:37.534764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a968ff26-f204-44bb-bd15-4148296870b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:37.548014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddbd3a-af9c-43f9-b2b9-f79334daffae None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:37.550792 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2416/4756] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:37.560342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddbd3a-af9c-43f9-b2b9-f79334daffae None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:37.565246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac476f-52bb-4287-bf84-89777021a585 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7792c0517b8e4d9aa2c3bc7a0b2df48a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d30f29581ec49e1b9db2cd693a0fccc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:37.566195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac476f-52bb-4287-bf84-89777021a585 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:37.566707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac476f-52bb-4287-bf84-89777021a585 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:37.567191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac476f-52bb-4287-bf84-89777021a585 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] PATH_INFO: `/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:37.649172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dcac476f-52bb-4287-bf84-89777021a585 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:37.680386 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2341/4757] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:06:37 2022] POST /identity/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:38.682113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82638bd9-9a24-43d4-8685-324361eca19a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:38.683013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82638bd9-9a24-43d4-8685-324361eca19a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:38.683511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82638bd9-9a24-43d4-8685-324361eca19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:38.724588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-82638bd9-9a24-43d4-8685-324361eca19a None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:38.726511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-82638bd9-9a24-43d4-8685-324361eca19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:38.746878 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2417/4758] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:38 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:06:38.752803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae81ea-c4a0-4df2-a2cb-b7c3662dc131 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:38.753352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae81ea-c4a0-4df2-a2cb-b7c3662dc131 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:38.753442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae81ea-c4a0-4df2-a2cb-b7c3662dc131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:38.801980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-59ae81ea-c4a0-4df2-a2cb-b7c3662dc131 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:38.803527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-59ae81ea-c4a0-4df2-a2cb-b7c3662dc131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:38.851506 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2342/4759] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:38.855398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82638bd9-9a24-43d4-8685-324361eca19a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:38.873737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82638bd9-9a24-43d4-8685-324361eca19a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:38.880616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6b42a-9491-41b7-8124-27ff29925ce6 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7792c0517b8e4d9aa2c3bc7a0b2df48a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d30f29581ec49e1b9db2cd693a0fccc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:38.881383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6b42a-9491-41b7-8124-27ff29925ce6 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:38.881793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6b42a-9491-41b7-8124-27ff29925ce6 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:38.882166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6b42a-9491-41b7-8124-27ff29925ce6 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] PATH_INFO: `/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:38.937220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c2e6b42a-9491-41b7-8124-27ff29925ce6 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:38.952448 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2418/4760] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:06:38 2022] POST /identity/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:39.900887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9d07c-0fd3-42a3-aa28-2a2c50877c8b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:39.901926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9d07c-0fd3-42a3-aa28-2a2c50877c8b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:39.902444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9d07c-0fd3-42a3-aa28-2a2c50877c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:39.926704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-13a9d07c-0fd3-42a3-aa28-2a2c50877c8b None None] MFA Rules not processed for user `fb144426dd4a46caa03a9ebe77e95c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:39.928102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-13a9d07c-0fd3-42a3-aa28-2a2c50877c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:39.957588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed10751-441f-4a1a-ad12-09d517de35aa None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:39.959612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed10751-441f-4a1a-ad12-09d517de35aa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:39.960077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed10751-441f-4a1a-ad12-09d517de35aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:39.969591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:UPDATE=2,keystone:DELETE=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:39.985574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8ed10751-441f-4a1a-ad12-09d517de35aa None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:39.987027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8ed10751-441f-4a1a-ad12-09d517de35aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:39.994594 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2419/4761] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:39 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 21:06:39.996253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4df031-a19b-4d0b-b8f2-296025711481 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:39.996392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4df031-a19b-4d0b-b8f2-296025711481 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:39.996765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4df031-a19b-4d0b-b8f2-296025711481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:40.008538 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2343/4762] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:40.027722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:UPDATE=2,keystone:INSERT=4,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:40.037601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3d4df031-a19b-4d0b-b8f2-296025711481 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:40.049108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3d4df031-a19b-4d0b-b8f2-296025711481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:40.074691 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2420/4763] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:40.078773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9d07c-0fd3-42a3-aa28-2a2c50877c8b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:40.097196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9d07c-0fd3-42a3-aa28-2a2c50877c8b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:40.104291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a34544-53a7-4733-aca2-44733bd5cdc0 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7792c0517b8e4d9aa2c3bc7a0b2df48a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d30f29581ec49e1b9db2cd693a0fccc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:40.105667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a34544-53a7-4733-aca2-44733bd5cdc0 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:40.108635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a34544-53a7-4733-aca2-44733bd5cdc0 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:40.109114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a34544-53a7-4733-aca2-44733bd5cdc0 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] PATH_INFO: `/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:40.190158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-70a34544-53a7-4733-aca2-44733bd5cdc0 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:40.205794 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2344/4764] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:06:40 2022] POST /identity/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:40.254982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c27fd-0b84-4268-9bc8-a2ea471c1e3f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:40.256287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c27fd-0b84-4268-9bc8-a2ea471c1e3f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:40.257245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c27fd-0b84-4268-9bc8-a2ea471c1e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:40.283035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d05c27fd-0b84-4268-9bc8-a2ea471c1e3f None None] MFA Rules not processed for user `9e686d314d164930bbcfc9905a8b45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:40.284480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d05c27fd-0b84-4268-9bc8-a2ea471c1e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:40.371316 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2421/4765] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:41.212021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a61b5e-45df-4c35-baf6-15b369ccb145 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:41.212533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a61b5e-45df-4c35-baf6-15b369ccb145 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:41.212609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a61b5e-45df-4c35-baf6-15b369ccb145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:41.241217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d3a61b5e-45df-4c35-baf6-15b369ccb145 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:41.242362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d3a61b5e-45df-4c35-baf6-15b369ccb145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:41.247949 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2345/4766] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:41 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:06:42.252189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7f20c-db29-4ce9-8d5e-6d8ceccb79e6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:42.252694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7f20c-db29-4ce9-8d5e-6d8ceccb79e6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:42.252694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7f20c-db29-4ce9-8d5e-6d8ceccb79e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:42.295436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4cb7f20c-db29-4ce9-8d5e-6d8ceccb79e6 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:42.296692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4cb7f20c-db29-4ce9-8d5e-6d8ceccb79e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:42.305764 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2422/4767] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:42.312603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d1474-1246-4b9e-ab7a-b9eefee224ba None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:42.313490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d1474-1246-4b9e-ab7a-b9eefee224ba None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:42.313909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d1474-1246-4b9e-ab7a-b9eefee224ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:42.359838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-091d1474-1246-4b9e-ab7a-b9eefee224ba None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:42.364199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-091d1474-1246-4b9e-ab7a-b9eefee224ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:42.399393 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2346/4768] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:42 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:06:42.405288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d4dbe-04d9-4d25-8fcf-4f8b7ebe58f8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:42.405620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d4dbe-04d9-4d25-8fcf-4f8b7ebe58f8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:42.405962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d4dbe-04d9-4d25-8fcf-4f8b7ebe58f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:42.517515 ncdv-master-1 devstack@keystone.service[99210]: WARNING keystone.server.flask.application [None req-4e7d4dbe-04d9-4d25-8fcf-4f8b7ebe58f8 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.16: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 03 21:06:42.537754 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2423/4769] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:42 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 122 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 03 21:06:42.538257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5bfe6-5098-4c57-a842-09a57ebaed8e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:42.538257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5bfe6-5098-4c57-a842-09a57ebaed8e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:42.538257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5bfe6-5098-4c57-a842-09a57ebaed8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:42.660449 ncdv-master-1 devstack@keystone.service[99211]: WARNING keystone.server.flask.application [None req-18c5bfe6-5098-4c57-a842-09a57ebaed8e None None] Authorization failed. The request you have made requires authentication. from 10.222.0.16: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 03 21:06:42.664202 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2347/4770] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:42 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 137 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 03 21:06:42.671421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca36ca-dce5-4613-bbbb-fdb5273faf4f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:42.671702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca36ca-dce5-4613-bbbb-fdb5273faf4f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:42.672008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca36ca-dce5-4613-bbbb-fdb5273faf4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:42.741384 ncdv-master-1 devstack@keystone.service[99210]: WARNING keystone.server.flask.application [None req-5fca36ca-dce5-4613-bbbb-fdb5273faf4f None None] Authorization failed. The request you have made requires authentication. from 10.222.0.16: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 03 21:06:42.755331 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2424/4771] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:42 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 85 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 03 21:06:46.968337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb9e6d-9771-4e1a-b270-cfe814a6529a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:46.968736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb9e6d-9771-4e1a-b270-cfe814a6529a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:46.968920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb9e6d-9771-4e1a-b270-cfe814a6529a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:46.991723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-dbdb9e6d-9771-4e1a-b270-cfe814a6529a None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:46.993366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dbdb9e6d-9771-4e1a-b270-cfe814a6529a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:47.074141 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2348/4772] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:47.087530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca36ca-dce5-4613-bbbb-fdb5273faf4f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:47.089600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5fca36ca-dce5-4613-bbbb-fdb5273faf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:47.194269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca36ca-dce5-4613-bbbb-fdb5273faf4f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:47.200518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3255c-e35f-4503-b169-f4af00847f6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:47.201236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3255c-e35f-4503-b169-f4af00847f6d service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:47.201559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3255c-e35f-4503-b169-f4af00847f6d service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:47.201885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3255c-e35f-4503-b169-f4af00847f6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:47.211227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3255c-e35f-4503-b169-f4af00847f6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:47.212924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3255c-e35f-4503-b169-f4af00847f6d service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:47.221450 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2425/4773] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:06:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:06:48.777609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21712a-8efa-4cbb-8294-8585e028403f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:48.778095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21712a-8efa-4cbb-8294-8585e028403f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:48.778200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21712a-8efa-4cbb-8294-8585e028403f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:48.826502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ab21712a-8efa-4cbb-8294-8585e028403f None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:48.827752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ab21712a-8efa-4cbb-8294-8585e028403f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:48.842869 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2349/4774] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:06:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:48.853742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3255c-e35f-4503-b169-f4af00847f6d service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:48.867443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3255c-e35f-4503-b169-f4af00847f6d service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:48.872884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47fa4-5000-4fea-836d-23802785109b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:48.873527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47fa4-5000-4fea-836d-23802785109b service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:48.873843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47fa4-5000-4fea-836d-23802785109b service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:48.874125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47fa4-5000-4fea-836d-23802785109b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:48.882911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be47fa4-5000-4fea-836d-23802785109b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:48.884600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be47fa4-5000-4fea-836d-23802785109b service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:48.893649 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2426/4775] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:06:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 21:06:49.284026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c7efb-f60b-4ae2-b1b4-10953d99a00f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:49.284618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c7efb-f60b-4ae2-b1b4-10953d99a00f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:49.285133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c7efb-f60b-4ae2-b1b4-10953d99a00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:49.322281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d18c7efb-f60b-4ae2-b1b4-10953d99a00f None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:49.324035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d18c7efb-f60b-4ae2-b1b4-10953d99a00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:49.332817 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2350/4776] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:06:49.350184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47fa4-5000-4fea-836d-23802785109b service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:49.368768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47fa4-5000-4fea-836d-23802785109b service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:49.374173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7448c64-73bd-46f2-89df-468c8542f355 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:49.374883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7448c64-73bd-46f2-89df-468c8542f355 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:49.375175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7448c64-73bd-46f2-89df-468c8542f355 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:49.375636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7448c64-73bd-46f2-89df-468c8542f355 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:49.393587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7448c64-73bd-46f2-89df-468c8542f355 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:49.395228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7448c64-73bd-46f2-89df-468c8542f355 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:49.405211 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2427/4777] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:06:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:06:53.381136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c7efb-f60b-4ae2-b1b4-10953d99a00f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:53.383419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d18c7efb-f60b-4ae2-b1b4-10953d99a00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:53.449628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:UPDATE=2,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:53.467693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c7efb-f60b-4ae2-b1b4-10953d99a00f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:53.475137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:53.475790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:53.476418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:53.476418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] PATH_INFO: `/v3/users/c3187514ca644ff68c992526f38783c7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:53.477567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3187514ca644ff68c992526f38783c7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:53.479781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:53.555453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3187514ca644ff68c992526f38783c7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:53.586854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3187514ca644ff68c992526f38783c7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:53.587957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3187514ca644ff68c992526f38783c7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:53.597801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3187514ca644ff68c992526f38783c7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:53.610986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3187514ca644ff68c992526f38783c7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:53.619397 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2351/4778] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:06:53 2022] DELETE /identity/v3/users/c3187514ca644ff68c992526f38783c7 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:53.768553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd96c7-cec6-4089-95fe-41188f62fb99 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:53.769523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd96c7-cec6-4089-95fe-41188f62fb99 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:53.770008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd96c7-cec6-4089-95fe-41188f62fb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:53.830037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fdbd96c7-cec6-4089-95fe-41188f62fb99 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:53.833209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fdbd96c7-cec6-4089-95fe-41188f62fb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:53.840248 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2428/4779] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:53 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:06:53.845380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:53.846938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:53.929788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa593aa1-be54-4413-ab81-e7fc4808dc82 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:53.934422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b72ff-d7c6-459c-bd68-e8863b939523 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7792c0517b8e4d9aa2c3bc7a0b2df48a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d30f29581ec49e1b9db2cd693a0fccc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:53.935229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b72ff-d7c6-459c-bd68-e8863b939523 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:53.935489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b72ff-d7c6-459c-bd68-e8863b939523 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:53.935859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b72ff-d7c6-459c-bd68-e8863b939523 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] PATH_INFO: `/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:53.990668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-813b72ff-d7c6-459c-bd68-e8863b939523 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:53.999329 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2352/4780] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:06:53 2022] POST /identity/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:54.122527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd96c7-cec6-4089-95fe-41188f62fb99 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:54.125167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fdbd96c7-cec6-4089-95fe-41188f62fb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:54.135360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:UPDATE=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:54.201653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd96c7-cec6-4089-95fe-41188f62fb99 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:54.208438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e9d31-85ef-4265-a3b6-49183f118496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:54.209343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e9d31-85ef-4265-a3b6-49183f118496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:54.209644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e9d31-85ef-4265-a3b6-49183f118496 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:54.209929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e9d31-85ef-4265-a3b6-49183f118496 admin admin] PATH_INFO: `/v3/projects/fde6874e10ae4e12ad874c65942f9d59` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:54.211115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e9d31-85ef-4265-a3b6-49183f118496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fde6874e10ae4e12ad874c65942f9d59)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:54.213475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e9d31-85ef-4265-a3b6-49183f118496 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:54.276119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b0e9d31-85ef-4265-a3b6-49183f118496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fde6874e10ae4e12ad874c65942f9d59', 'request_id': 'req-2b0e9d31-85ef-4265-a3b6-49183f118496'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:54.285827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b0e9d31-85ef-4265-a3b6-49183f118496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fde6874e10ae4e12ad874c65942f9d59', 'request_id': 'req-2b0e9d31-85ef-4265-a3b6-49183f118496'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:54.296233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b0e9d31-85ef-4265-a3b6-49183f118496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fde6874e10ae4e12ad874c65942f9d59', 'request_id': 'req-2b0e9d31-85ef-4265-a3b6-49183f118496'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:54.310705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b0e9d31-85ef-4265-a3b6-49183f118496 admin admin] The token cache is being invalidate because project fde6874e10ae4e12ad874c65942f9d59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:54.311010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b0e9d31-85ef-4265-a3b6-49183f118496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:54.311698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b0e9d31-85ef-4265-a3b6-49183f118496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:54.313742 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2429/4781] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:54 2022] DELETE /identity/v3/projects/fde6874e10ae4e12ad874c65942f9d59 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:06:54.321042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebc0f7-2214-4ccc-a317-8c8f0a4331d8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:54.321347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebc0f7-2214-4ccc-a317-8c8f0a4331d8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:54.321608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebc0f7-2214-4ccc-a317-8c8f0a4331d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:54.368737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e0ebc0f7-2214-4ccc-a317-8c8f0a4331d8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:54.370077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e0ebc0f7-2214-4ccc-a317-8c8f0a4331d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:54.466724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:54.516978 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2353/4782] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:54.526241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bce71-7698-4775-8906-d06ac6bd3c9b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:54.526848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bce71-7698-4775-8906-d06ac6bd3c9b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:54.527283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bce71-7698-4775-8906-d06ac6bd3c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:54.556876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9b1bce71-7698-4775-8906-d06ac6bd3c9b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:54.558626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9b1bce71-7698-4775-8906-d06ac6bd3c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:54.584938 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2430/4783] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:06:54.589675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebc0f7-2214-4ccc-a317-8c8f0a4331d8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:54.600496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebc0f7-2214-4ccc-a317-8c8f0a4331d8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:54.605911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a184813-5e4a-4ea5-b017-f1443bd09e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:54.606519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a184813-5e4a-4ea5-b017-f1443bd09e0f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:54.606836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a184813-5e4a-4ea5-b017-f1443bd09e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:54.607075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a184813-5e4a-4ea5-b017-f1443bd09e0f admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:54.607721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a184813-5e4a-4ea5-b017-f1443bd09e0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:06:54.608023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a184813-5e4a-4ea5-b017-f1443bd09e0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:54.609661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a184813-5e4a-4ea5-b017-f1443bd09e0f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:54.622630 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2354/4784] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:06:54.626533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bce71-7698-4775-8906-d06ac6bd3c9b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:54.636926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bce71-7698-4775-8906-d06ac6bd3c9b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:54.641749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f122-b82a-44ac-b215-212b7fccb345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:54.642373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f122-b82a-44ac-b215-212b7fccb345 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:54.642667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f122-b82a-44ac-b215-212b7fccb345 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:54.642954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f122-b82a-44ac-b215-212b7fccb345 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:54.643615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8f122-b82a-44ac-b215-212b7fccb345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:54.646143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8f122-b82a-44ac-b215-212b7fccb345 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:54.677856 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2431/4785] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:06:54 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:06:54.681756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a184813-5e4a-4ea5-b017-f1443bd09e0f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:54.692836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a184813-5e4a-4ea5-b017-f1443bd09e0f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:54.781614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e6c09-1661-4f21-82fb-744344a0866c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:54.782245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e6c09-1661-4f21-82fb-744344a0866c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:54.782668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e6c09-1661-4f21-82fb-744344a0866c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:54.782801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e6c09-1661-4f21-82fb-744344a0866c admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:54.783439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e6c09-1661-4f21-82fb-744344a0866c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:54.786075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e6c09-1661-4f21-82fb-744344a0866c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:54.815723 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2355/4786] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:06:54 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:06:54.820226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f122-b82a-44ac-b215-212b7fccb345 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:54.839143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f122-b82a-44ac-b215-212b7fccb345 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:54.843985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d50f45-6c09-4881-b837-dae893274afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:54.844591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d50f45-6c09-4881-b837-dae893274afc admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:54.844876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d50f45-6c09-4881-b837-dae893274afc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:54.845277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d50f45-6c09-4881-b837-dae893274afc admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:54.845923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d50f45-6c09-4881-b837-dae893274afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:54.847769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d50f45-6c09-4881-b837-dae893274afc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:54.860618 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2432/4787] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:06:54.867778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e6c09-1661-4f21-82fb-744344a0866c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:54.882846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e6c09-1661-4f21-82fb-744344a0866c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:54.885587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0afbd1-e60b-49e1-8fa2-34fbf650d4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:54.886297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0afbd1-e60b-49e1-8fa2-34fbf650d4da admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:54.886585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0afbd1-e60b-49e1-8fa2-34fbf650d4da admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:54.886865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0afbd1-e60b-49e1-8fa2-34fbf650d4da admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:54.887512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0afbd1-e60b-49e1-8fa2-34fbf650d4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:54.889165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0afbd1-e60b-49e1-8fa2-34fbf650d4da admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:54.903754 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2356/4788] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:06:54.908537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d50f45-6c09-4881-b837-dae893274afc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:54.919899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d50f45-6c09-4881-b837-dae893274afc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:54.925724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a8b58-a90e-4319-aa8e-6a15b7d3204c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:54.926589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a8b58-a90e-4319-aa8e-6a15b7d3204c admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:54.927079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a8b58-a90e-4319-aa8e-6a15b7d3204c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:54.927518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a8b58-a90e-4319-aa8e-6a15b7d3204c admin admin] PATH_INFO: `/v3/projects/ae91188b2c8b43d4bc6114581be12847/users/5eff5aacf85a4fda97f6eb375da7cf26/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:54.949118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a8b58-a90e-4319-aa8e-6a15b7d3204c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae91188b2c8b43d4bc6114581be12847, user_id=5eff5aacf85a4fda97f6eb375da7cf26, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:54.952285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a8b58-a90e-4319-aa8e-6a15b7d3204c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:54.969155 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2433/4789] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:06:54 2022] PUT /identity/v3/projects/ae91188b2c8b43d4bc6114581be12847/users/5eff5aacf85a4fda97f6eb375da7cf26/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:54.974638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb65dc-bdba-4fa0-975e-4d1474d9ba27 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:54.974934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb65dc-bdba-4fa0-975e-4d1474d9ba27 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:54.975216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb65dc-bdba-4fa0-975e-4d1474d9ba27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:54.997381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-77fb65dc-bdba-4fa0-975e-4d1474d9ba27 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:55.000048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-77fb65dc-bdba-4fa0-975e-4d1474d9ba27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:55.006106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93f2e9-3312-4351-bc84-63af08337567 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:55.006406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93f2e9-3312-4351-bc84-63af08337567 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:55.006745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93f2e9-3312-4351-bc84-63af08337567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:55.055006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3a93f2e9-3312-4351-bc84-63af08337567 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:55.058223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3a93f2e9-3312-4351-bc84-63af08337567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:55.068260 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2434/4790] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:55 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:06:55.072480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356404d-042e-469c-aeb8-ad085e0e90a6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:55.073070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356404d-042e-469c-aeb8-ad085e0e90a6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:55.073567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356404d-042e-469c-aeb8-ad085e0e90a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:55.077869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6356404d-042e-469c-aeb8-ad085e0e90a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:55.123241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:55.154882 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2357/4791] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:06:55.167683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb65dc-bdba-4fa0-975e-4d1474d9ba27 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:55.169837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-77fb65dc-bdba-4fa0-975e-4d1474d9ba27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-6356404d-042e-469c-aeb8-ad085e0e90a6 None None] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Sep 03 21:06:55.232696 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 185, in authenticate_for_token Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application authenticate(auth_info, auth_context) Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 152, in authenticate Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = method.authenticate(auth_info.get_method_data(method_name)) Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 42, in authenticate Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application token = self._get_token_ref(auth_payload) Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 36, in _get_token_ref Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return PROVIDERS.token_provider_api.validate_token(token_id) Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:06:55.234001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:06:55.235167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 03 21:06:55.235167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 03 21:06:55.235167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:06:55.235167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:06:55.235167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 03 21:06:55.235167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 03 21:06:55.235167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:06:55.235167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:06:55.235167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 03 21:06:55.235167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 03 21:06:55.235167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 03 21:06:55.235167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:06:55.237432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:55.238863 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2435/4792] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:55 2022] POST /identity/v3/auth/tokens => generated 80 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 21:06:55.243861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65005584-2dbf-43c6-b3d6-59a32ea15ae8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:55.244159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65005584-2dbf-43c6-b3d6-59a32ea15ae8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:55.244443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65005584-2dbf-43c6-b3d6-59a32ea15ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:55.267911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb65dc-bdba-4fa0-975e-4d1474d9ba27 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:55.273271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87202e7b-065c-4bee-a32b-6856234e6b7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:55.274493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87202e7b-065c-4bee-a32b-6856234e6b7a service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:55.275001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87202e7b-065c-4bee-a32b-6856234e6b7a service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:55.275659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87202e7b-065c-4bee-a32b-6856234e6b7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:55.290034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87202e7b-065c-4bee-a32b-6856234e6b7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:55.292969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87202e7b-065c-4bee-a32b-6856234e6b7a service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:55.311838 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2358/4793] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:06:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:06:55.315383 ncdv-master-1 devstack@keystone.service[99210]: WARNING keystone.server.flask.application [None req-65005584-2dbf-43c6-b3d6-59a32ea15ae8 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.16: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 03 21:06:55.320338 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2436/4794] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:55 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 74 msecs (HTTP/1.1 401) 6 headers in 252 bytes (2 switches on core 0) Sep 03 21:06:55.329814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dea572-7055-44d4-a4f0-0c3cf94e0c8d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:55.330107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dea572-7055-44d4-a4f0-0c3cf94e0c8d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:55.330388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dea572-7055-44d4-a4f0-0c3cf94e0c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:55.375393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e3dea572-7055-44d4-a4f0-0c3cf94e0c8d None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:55.378694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e3dea572-7055-44d4-a4f0-0c3cf94e0c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:55.397399 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2359/4795] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:55.399178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65005584-2dbf-43c6-b3d6-59a32ea15ae8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:55.421722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65005584-2dbf-43c6-b3d6-59a32ea15ae8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:55.432907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82599f3-913a-4a53-b636-832defe3e725 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7792c0517b8e4d9aa2c3bc7a0b2df48a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d30f29581ec49e1b9db2cd693a0fccc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:55.433571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82599f3-913a-4a53-b636-832defe3e725 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:55.433860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82599f3-913a-4a53-b636-832defe3e725 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:55.434604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82599f3-913a-4a53-b636-832defe3e725 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] PATH_INFO: `/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:55.522840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c82599f3-913a-4a53-b636-832defe3e725 tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:55.542790 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2437/4796] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:06:55 2022] POST /identity/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:56.199495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f287b70-4677-4377-aabe-c9cd05e126e6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:56.200079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f287b70-4677-4377-aabe-c9cd05e126e6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:56.200598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f287b70-4677-4377-aabe-c9cd05e126e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:56.226906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4f287b70-4677-4377-aabe-c9cd05e126e6 None None] MFA Rules not processed for user `d485394bc49e4380b152d32d09440281`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:56.228804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4f287b70-4677-4377-aabe-c9cd05e126e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:56.318955 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2360/4797] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:56.547947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cb858-62b3-4b1b-a5e4-13a994a8c94c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:56.548797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cb858-62b3-4b1b-a5e4-13a994a8c94c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:56.549248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cb858-62b3-4b1b-a5e4-13a994a8c94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:56.581956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-140cb858-62b3-4b1b-a5e4-13a994a8c94c None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:56.584279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-140cb858-62b3-4b1b-a5e4-13a994a8c94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:56.588598 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2438/4798] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:56 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:06:56.594141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614c6b1-f3d0-49ef-9ef2-f5ef18e57c0c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:56.594439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614c6b1-f3d0-49ef-9ef2-f5ef18e57c0c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:56.594724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614c6b1-f3d0-49ef-9ef2-f5ef18e57c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:56.626593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e614c6b1-f3d0-49ef-9ef2-f5ef18e57c0c None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:56.627899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e614c6b1-f3d0-49ef-9ef2-f5ef18e57c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:56.644832 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2361/4799] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:06:56.648503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cb858-62b3-4b1b-a5e4-13a994a8c94c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:56.660344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cb858-62b3-4b1b-a5e4-13a994a8c94c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:56.665074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b07960-7e57-4496-ad57-9a552d9728bd tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7792c0517b8e4d9aa2c3bc7a0b2df48a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d30f29581ec49e1b9db2cd693a0fccc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:56.665896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b07960-7e57-4496-ad57-9a552d9728bd tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:56.666339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b07960-7e57-4496-ad57-9a552d9728bd tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:56.666772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b07960-7e57-4496-ad57-9a552d9728bd tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] PATH_INFO: `/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:56.745778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e1b07960-7e57-4496-ad57-9a552d9728bd tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:56.759390 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2439/4800] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:06:56 2022] POST /identity/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:06:57.765305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20594c60-f77f-421b-8e16-2fb7b12fa1b0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:57.765777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20594c60-f77f-421b-8e16-2fb7b12fa1b0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:57.765865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20594c60-f77f-421b-8e16-2fb7b12fa1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:57.786254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-20594c60-f77f-421b-8e16-2fb7b12fa1b0 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:57.787534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-20594c60-f77f-421b-8e16-2fb7b12fa1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:57.792975 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2362/4801] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:57 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:06:57.797375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbe4ac-680c-4f37-94f5-08b28ac7c1b9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:57.806464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbe4ac-680c-4f37-94f5-08b28ac7c1b9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:57.806987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbe4ac-680c-4f37-94f5-08b28ac7c1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:57.851363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9fcbe4ac-680c-4f37-94f5-08b28ac7c1b9 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:57.853037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9fcbe4ac-680c-4f37-94f5-08b28ac7c1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:57.873924 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2440/4802] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:57.879192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20594c60-f77f-421b-8e16-2fb7b12fa1b0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:57.898977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20594c60-f77f-421b-8e16-2fb7b12fa1b0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:57.924178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dae04c-821d-4132-aa71-c894b1111cea tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7792c0517b8e4d9aa2c3bc7a0b2df48a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d30f29581ec49e1b9db2cd693a0fccc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:57.925086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dae04c-821d-4132-aa71-c894b1111cea tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:57.925447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dae04c-821d-4132-aa71-c894b1111cea tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:57.925734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dae04c-821d-4132-aa71-c894b1111cea tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] PATH_INFO: `/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:58.088703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5dae04c-821d-4132-aa71-c894b1111cea tempest-IdentityV3UsersTest-1437354314 tempest-IdentityV3UsersTest-1437354314-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:58.106727 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2363/4803] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:06:57 2022] POST /identity/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a/password => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:58.547420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbe4ac-680c-4f37-94f5-08b28ac7c1b9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:58.549068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9fcbe4ac-680c-4f37-94f5-08b28ac7c1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:58.594169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbe4ac-680c-4f37-94f5-08b28ac7c1b9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:58.612946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:58.613939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:58.614383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:58.614855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] PATH_INFO: `/v3/users/27989e2a2d014e52913ca77e0448fc63` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:58.617748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27989e2a2d014e52913ca77e0448fc63)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:58.619835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:58.709428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27989e2a2d014e52913ca77e0448fc63'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:58.751381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27989e2a2d014e52913ca77e0448fc63'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:58.762590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27989e2a2d014e52913ca77e0448fc63'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:58.763730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27989e2a2d014e52913ca77e0448fc63'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:58.764690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27989e2a2d014e52913ca77e0448fc63'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:58.780702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=78,keystone:UPDATE=3,keystone:INSERT=5,keystone:DELETE=14 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:06:58.803784 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2441/4804] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:06:58 2022] DELETE /identity/v3/users/27989e2a2d014e52913ca77e0448fc63 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:59.111395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322c600-6641-48f8-af77-de510e9ab510 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:59.111767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322c600-6641-48f8-af77-de510e9ab510 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:59.111999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322c600-6641-48f8-af77-de510e9ab510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:59.234515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7322c600-6641-48f8-af77-de510e9ab510 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:59.235782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7322c600-6641-48f8-af77-de510e9ab510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:59.249829 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2364/4805] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:59 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:06:59.531065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:06:59.532606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:59.654187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2d712-3547-4842-919d-414e68de3ac4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:06:59.671358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:06:59.672560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:59.673136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:59.673796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318 admin admin] PATH_INFO: `/v3/projects/8db106e653804866a09e333bddc1cf3d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:59.675205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db106e653804866a09e333bddc1cf3d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:06:59.678018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:06:59.760407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db106e653804866a09e333bddc1cf3d', 'request_id': 'req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:59.773679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db106e653804866a09e333bddc1cf3d', 'request_id': 'req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:59.783995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db106e653804866a09e333bddc1cf3d', 'request_id': 'req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:59.802429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318 admin admin] The token cache is being invalidate because project 8db106e653804866a09e333bddc1cf3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:06:59.803091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:59.803906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee7dd2b0-4b3b-41c3-bbdc-319bf53c9318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:06:59.809891 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2442/4806] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:06:59 2022] DELETE /identity/v3/projects/8db106e653804866a09e333bddc1cf3d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:06:59.811580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab75df1-a1ce-4345-beff-16fd958d3cb1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:59.812783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab75df1-a1ce-4345-beff-16fd958d3cb1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:59.813635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab75df1-a1ce-4345-beff-16fd958d3cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:59.839251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cab75df1-a1ce-4345-beff-16fd958d3cb1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:59.841493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cab75df1-a1ce-4345-beff-16fd958d3cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:06:59.955712 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2365/4807] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:06:59.963660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3f36f-3361-450c-bf78-a1c1f5f17a46 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:06:59.964181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3f36f-3361-450c-bf78-a1c1f5f17a46 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:06:59.964642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3f36f-3361-450c-bf78-a1c1f5f17a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:06:59.994897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-dbf3f36f-3361-450c-bf78-a1c1f5f17a46 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:06:59.997058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dbf3f36f-3361-450c-bf78-a1c1f5f17a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:00.030248 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2443/4808] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:00.034139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab75df1-a1ce-4345-beff-16fd958d3cb1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:00.048076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab75df1-a1ce-4345-beff-16fd958d3cb1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:00.080456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7c302-70bc-4640-bc5f-14262ce2d780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:00.081351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7c302-70bc-4640-bc5f-14262ce2d780 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:00.081823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7c302-70bc-4640-bc5f-14262ce2d780 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:00.083591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7c302-70bc-4640-bc5f-14262ce2d780 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:00.085549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7c302-70bc-4640-bc5f-14262ce2d780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:00.088948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7c302-70bc-4640-bc5f-14262ce2d780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:00.092656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7c302-70bc-4640-bc5f-14262ce2d780 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:00.127514 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2366/4809] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:00.132432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3f36f-3361-450c-bf78-a1c1f5f17a46 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:00.155960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3f36f-3361-450c-bf78-a1c1f5f17a46 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:00.171836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39166bf-b7bc-4632-8f94-42b12060e55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:00.172495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39166bf-b7bc-4632-8f94-42b12060e55a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:00.172750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39166bf-b7bc-4632-8f94-42b12060e55a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:00.173062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39166bf-b7bc-4632-8f94-42b12060e55a admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:00.173711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39166bf-b7bc-4632-8f94-42b12060e55a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:00.175678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39166bf-b7bc-4632-8f94-42b12060e55a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:00.220822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c8257-d689-42b6-b644-a16d3d6259c5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:00.221145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c8257-d689-42b6-b644-a16d3d6259c5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:00.221644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c8257-d689-42b6-b644-a16d3d6259c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:00.232594 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2444/4810] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:00 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:07:00.236570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39166bf-b7bc-4632-8f94-42b12060e55a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:00.278986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7f6c8257-d689-42b6-b644-a16d3d6259c5 None None] MFA Rules not processed for user `9e686d314d164930bbcfc9905a8b45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:00.280245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7f6c8257-d689-42b6-b644-a16d3d6259c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:00.291384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:UPDATE=2,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:00.375245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39166bf-b7bc-4632-8f94-42b12060e55a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:00.389885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e96a6d-b091-44ac-ba8b-b8d92ac42072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:00.390540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e96a6d-b091-44ac-ba8b-b8d92ac42072 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:00.390864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e96a6d-b091-44ac-ba8b-b8d92ac42072 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:00.392029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e96a6d-b091-44ac-ba8b-b8d92ac42072 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:00.392692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e96a6d-b091-44ac-ba8b-b8d92ac42072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:00.394707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e96a6d-b091-44ac-ba8b-b8d92ac42072 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:00.420776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:00.424940 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2367/4811] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:00.426451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208dd41-1582-4aee-9588-9af6805cd940 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:00.426909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208dd41-1582-4aee-9588-9af6805cd940 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:00.427188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208dd41-1582-4aee-9588-9af6805cd940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:00.460066 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2445/4812] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:00 2022] POST /identity/v3/users => generated 303 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:07:00.463901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e96a6d-b091-44ac-ba8b-b8d92ac42072 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:00.483389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e96a6d-b091-44ac-ba8b-b8d92ac42072 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:00.499844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c67e6b-c6a5-48b8-babb-2eb07f2c9fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:00.500774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c67e6b-c6a5-48b8-babb-2eb07f2c9fed admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:00.501729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c67e6b-c6a5-48b8-babb-2eb07f2c9fed admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:00.502031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c67e6b-c6a5-48b8-babb-2eb07f2c9fed admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:00.503261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c67e6b-c6a5-48b8-babb-2eb07f2c9fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:00.508617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c67e6b-c6a5-48b8-babb-2eb07f2c9fed admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:00.516495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2208dd41-1582-4aee-9588-9af6805cd940 None None] MFA Rules not processed for user `7792c0517b8e4d9aa2c3bc7a0b2df48a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:00.517686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2208dd41-1582-4aee-9588-9af6805cd940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:00.532564 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2446/4813] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:00.536468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c67e6b-c6a5-48b8-babb-2eb07f2c9fed admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:00.596278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c67e6b-c6a5-48b8-babb-2eb07f2c9fed admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:00.604479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5bdaa-cac2-4673-b7de-fb48ef11f398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:00.606502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5bdaa-cac2-4673-b7de-fb48ef11f398 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:00.607013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5bdaa-cac2-4673-b7de-fb48ef11f398 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:00.607755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5bdaa-cac2-4673-b7de-fb48ef11f398 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:00.608614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5bdaa-cac2-4673-b7de-fb48ef11f398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:00.611916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5bdaa-cac2-4673-b7de-fb48ef11f398 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:00.634918 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2447/4814] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:00.638980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5bdaa-cac2-4673-b7de-fb48ef11f398 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:00.653805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5bdaa-cac2-4673-b7de-fb48ef11f398 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:00.661901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25d519-bfd9-40cb-9078-92c62a64c96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:00.662914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25d519-bfd9-40cb-9078-92c62a64c96e admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:00.663282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25d519-bfd9-40cb-9078-92c62a64c96e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:00.664205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25d519-bfd9-40cb-9078-92c62a64c96e admin admin] PATH_INFO: `/v3/projects/1646ccd0c1d442bdbab54ad9f44e3aa6/users/1cf3a90cb1034ae39f67fb0fa74366f7/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:00.687596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25d519-bfd9-40cb-9078-92c62a64c96e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1646ccd0c1d442bdbab54ad9f44e3aa6, user_id=1cf3a90cb1034ae39f67fb0fa74366f7, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:00.691183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25d519-bfd9-40cb-9078-92c62a64c96e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:00.703079 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2368/4815] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:00.709267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208dd41-1582-4aee-9588-9af6805cd940 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:00.710706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2208dd41-1582-4aee-9588-9af6805cd940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:00.718487 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2448/4816] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:00 2022] PUT /identity/v3/projects/1646ccd0c1d442bdbab54ad9f44e3aa6/users/1cf3a90cb1034ae39f67fb0fa74366f7/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:00.725595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74041ae5-1e26-4d3b-a3cf-476cb1557a95 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:00.726389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74041ae5-1e26-4d3b-a3cf-476cb1557a95 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:00.726754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74041ae5-1e26-4d3b-a3cf-476cb1557a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:00.739772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208dd41-1582-4aee-9588-9af6805cd940 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:00.760038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-74041ae5-1e26-4d3b-a3cf-476cb1557a95 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:00.761839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-74041ae5-1e26-4d3b-a3cf-476cb1557a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:00.867035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075025de-f97f-4417-becd-2180a55385fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:00.867829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075025de-f97f-4417-becd-2180a55385fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:00.868237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075025de-f97f-4417-becd-2180a55385fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:00.868521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075025de-f97f-4417-becd-2180a55385fc admin admin] PATH_INFO: `/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:00.873509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075025de-f97f-4417-becd-2180a55385fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7792c0517b8e4d9aa2c3bc7a0b2df48a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:00.875509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075025de-f97f-4417-becd-2180a55385fc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:00.891119 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2449/4817] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:00.904464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74041ae5-1e26-4d3b-a3cf-476cb1557a95 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:00.906154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-74041ae5-1e26-4d3b-a3cf-476cb1557a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:00.967964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-075025de-f97f-4417-becd-2180a55385fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:00.985161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:00.991572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-075025de-f97f-4417-becd-2180a55385fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:00.993918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-075025de-f97f-4417-becd-2180a55385fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:01.010086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-075025de-f97f-4417-becd-2180a55385fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:01.023500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-075025de-f97f-4417-becd-2180a55385fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7792c0517b8e4d9aa2c3bc7a0b2df48a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:01.034127 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2369/4818] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:00 2022] DELETE /identity/v3/users/7792c0517b8e4d9aa2c3bc7a0b2df48a => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:01.043007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74041ae5-1e26-4d3b-a3cf-476cb1557a95 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:01.043384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3252b57-98af-4887-bbf9-2da7cb8f29e1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:01.043679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3252b57-98af-4887-bbf9-2da7cb8f29e1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:01.043956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3252b57-98af-4887-bbf9-2da7cb8f29e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:01.084341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b3252b57-98af-4887-bbf9-2da7cb8f29e1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:01.085666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b3252b57-98af-4887-bbf9-2da7cb8f29e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:01.136086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b4772-60ff-4883-8b29-4aeec1e9a7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:01.136772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b4772-60ff-4883-8b29-4aeec1e9a7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:01.137095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b4772-60ff-4883-8b29-4aeec1e9a7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:01.137413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b4772-60ff-4883-8b29-4aeec1e9a7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:01.139377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-989b4772-60ff-4883-8b29-4aeec1e9a7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:01.158754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:01.255438 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2370/4819] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:01.263725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3252b57-98af-4887-bbf9-2da7cb8f29e1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:01.267169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b3252b57-98af-4887-bbf9-2da7cb8f29e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:01.271118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b4772-60ff-4883-8b29-4aeec1e9a7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:01.272950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b4772-60ff-4883-8b29-4aeec1e9a7f5 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:01.286233 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2450/4820] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:07:01.303303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3252b57-98af-4887-bbf9-2da7cb8f29e1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:01.309492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fc3f6-d181-47f2-b034-fddda5953399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:01.311044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fc3f6-d181-47f2-b034-fddda5953399 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:01.311505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fc3f6-d181-47f2-b034-fddda5953399 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:01.312006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fc3f6-d181-47f2-b034-fddda5953399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:01.330499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fc3f6-d181-47f2-b034-fddda5953399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:01.333746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fc3f6-d181-47f2-b034-fddda5953399 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:01.362773 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2371/4821] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:07:02.219837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b4772-60ff-4883-8b29-4aeec1e9a7f5 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:02.222074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-989b4772-60ff-4883-8b29-4aeec1e9a7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:02.243626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b4772-60ff-4883-8b29-4aeec1e9a7f5 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:02.255623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f21e5-e0a5-4d66-b402-0a3e718a2127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:02.258871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f21e5-e0a5-4d66-b402-0a3e718a2127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:02.259682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f21e5-e0a5-4d66-b402-0a3e718a2127 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:02.261154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f21e5-e0a5-4d66-b402-0a3e718a2127 admin admin] PATH_INFO: `/v3/projects/4d30f29581ec49e1b9db2cd693a0fccc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:02.264493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f21e5-e0a5-4d66-b402-0a3e718a2127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d30f29581ec49e1b9db2cd693a0fccc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:02.287377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f21e5-e0a5-4d66-b402-0a3e718a2127 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:02.351351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-159f21e5-e0a5-4d66-b402-0a3e718a2127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d30f29581ec49e1b9db2cd693a0fccc', 'request_id': 'req-159f21e5-e0a5-4d66-b402-0a3e718a2127'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:02.360507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-159f21e5-e0a5-4d66-b402-0a3e718a2127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d30f29581ec49e1b9db2cd693a0fccc', 'request_id': 'req-159f21e5-e0a5-4d66-b402-0a3e718a2127'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:02.373452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-159f21e5-e0a5-4d66-b402-0a3e718a2127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d30f29581ec49e1b9db2cd693a0fccc', 'request_id': 'req-159f21e5-e0a5-4d66-b402-0a3e718a2127'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:02.407629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-159f21e5-e0a5-4d66-b402-0a3e718a2127 admin admin] The token cache is being invalidate because project 4d30f29581ec49e1b9db2cd693a0fccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:02.407629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-159f21e5-e0a5-4d66-b402-0a3e718a2127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:02.414106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-159f21e5-e0a5-4d66-b402-0a3e718a2127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:02.417022 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2451/4822] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:02 2022] DELETE /identity/v3/projects/4d30f29581ec49e1b9db2cd693a0fccc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:02.421921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94841655-66df-4018-bdf9-6a43e871bf59 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:02.421921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94841655-66df-4018-bdf9-6a43e871bf59 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:02.421921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94841655-66df-4018-bdf9-6a43e871bf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:02.469217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-94841655-66df-4018-bdf9-6a43e871bf59 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:02.471476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-94841655-66df-4018-bdf9-6a43e871bf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:02.571932 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2372/4823] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:02.580853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4038c1-0e56-48af-ade6-d2c0cfd5146f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:02.583637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4038c1-0e56-48af-ade6-d2c0cfd5146f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:02.583882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4038c1-0e56-48af-ade6-d2c0cfd5146f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:02.630470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fb4038c1-0e56-48af-ade6-d2c0cfd5146f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:02.634281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fb4038c1-0e56-48af-ade6-d2c0cfd5146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:02.668929 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2452/4824] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:02.670186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94841655-66df-4018-bdf9-6a43e871bf59 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:02.683075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94841655-66df-4018-bdf9-6a43e871bf59 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:02.708395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112ad57-cda1-4d60-b293-f4198dc1fed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:02.709080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112ad57-cda1-4d60-b293-f4198dc1fed3 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:02.709353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112ad57-cda1-4d60-b293-f4198dc1fed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:02.710246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112ad57-cda1-4d60-b293-f4198dc1fed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:02.710957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112ad57-cda1-4d60-b293-f4198dc1fed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:02.711260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112ad57-cda1-4d60-b293-f4198dc1fed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:02.713527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112ad57-cda1-4d60-b293-f4198dc1fed3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:02.734382 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2373/4825] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:02.738425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4038c1-0e56-48af-ade6-d2c0cfd5146f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:02.775346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4038c1-0e56-48af-ade6-d2c0cfd5146f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:02.787882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01794d1a-b578-4c6d-9e28-480d6b4035b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:02.788390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01794d1a-b578-4c6d-9e28-480d6b4035b1 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:02.789869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01794d1a-b578-4c6d-9e28-480d6b4035b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:02.790155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01794d1a-b578-4c6d-9e28-480d6b4035b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:02.790888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01794d1a-b578-4c6d-9e28-480d6b4035b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:02.799887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01794d1a-b578-4c6d-9e28-480d6b4035b1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:02.880727 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2453/4826] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:02 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:02.882797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112ad57-cda1-4d60-b293-f4198dc1fed3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:02.914443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112ad57-cda1-4d60-b293-f4198dc1fed3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:03.029813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:03.057960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddfe65-fe34-458d-b873-254f7a1acc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:03.058718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddfe65-fe34-458d-b873-254f7a1acc24 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:03.059421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddfe65-fe34-458d-b873-254f7a1acc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:03.059421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddfe65-fe34-458d-b873-254f7a1acc24 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:03.060851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddfe65-fe34-458d-b873-254f7a1acc24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:03.065780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddfe65-fe34-458d-b873-254f7a1acc24 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:03.106247 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2374/4827] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:02 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:03.110419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01794d1a-b578-4c6d-9e28-480d6b4035b1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:03.131427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01794d1a-b578-4c6d-9e28-480d6b4035b1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:03.144084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541590b-f829-4046-ae10-8105b9e1e5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:03.160750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541590b-f829-4046-ae10-8105b9e1e5ca admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:03.161548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541590b-f829-4046-ae10-8105b9e1e5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:03.164935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541590b-f829-4046-ae10-8105b9e1e5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:03.165868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541590b-f829-4046-ae10-8105b9e1e5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:03.167701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541590b-f829-4046-ae10-8105b9e1e5ca admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:03.209080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddfe65-fe34-458d-b873-254f7a1acc24 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:03.213157 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2454/4828] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:03.225756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddfe65-fe34-458d-b873-254f7a1acc24 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:03.232667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d2b0b-1013-4085-8160-a3cb77eced3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:03.233182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d2b0b-1013-4085-8160-a3cb77eced3e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:03.233528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d2b0b-1013-4085-8160-a3cb77eced3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:03.233811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d2b0b-1013-4085-8160-a3cb77eced3e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:03.235374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d2b0b-1013-4085-8160-a3cb77eced3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:03.237433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d2b0b-1013-4085-8160-a3cb77eced3e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:03.252950 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2375/4829] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:03.255403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541590b-f829-4046-ae10-8105b9e1e5ca admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:03.267138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541590b-f829-4046-ae10-8105b9e1e5ca admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:03.273466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf487095-4f79-41ae-a1be-09c8a86c7c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:03.274357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf487095-4f79-41ae-a1be-09c8a86c7c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:03.275303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf487095-4f79-41ae-a1be-09c8a86c7c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:03.275937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf487095-4f79-41ae-a1be-09c8a86c7c21 admin admin] PATH_INFO: `/v3/projects/d5ffc10ee5674988a61423746255caf0/users/64bc0fb485724cf882cade1ea07d5b0f/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:03.304659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf487095-4f79-41ae-a1be-09c8a86c7c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ffc10ee5674988a61423746255caf0, user_id=64bc0fb485724cf882cade1ea07d5b0f, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:03.308111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf487095-4f79-41ae-a1be-09c8a86c7c21 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:03.328340 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2455/4830] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:03 2022] PUT /identity/v3/projects/d5ffc10ee5674988a61423746255caf0/users/64bc0fb485724cf882cade1ea07d5b0f/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:03.342048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7dc589-18e9-4484-8912-75a5adffffa0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:03.342355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7dc589-18e9-4484-8912-75a5adffffa0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:03.342933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7dc589-18e9-4484-8912-75a5adffffa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:03.385531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cd7dc589-18e9-4484-8912-75a5adffffa0 None None] MFA Rules not processed for user `64bc0fb485724cf882cade1ea07d5b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:03.386925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cd7dc589-18e9-4484-8912-75a5adffffa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:03.548371 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2376/4831] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:03.554709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf487095-4f79-41ae-a1be-09c8a86c7c21 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:03.557035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cf487095-4f79-41ae-a1be-09c8a86c7c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:03.619210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:03.680761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf487095-4f79-41ae-a1be-09c8a86c7c21 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:03.686752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbec45-0fb1-46fc-b8d7-11a776ca6da9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:03.687439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbec45-0fb1-46fc-b8d7-11a776ca6da9 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:03.687798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbec45-0fb1-46fc-b8d7-11a776ca6da9 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:03.688151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbec45-0fb1-46fc-b8d7-11a776ca6da9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:03.700462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acbec45-0fb1-46fc-b8d7-11a776ca6da9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:03.703891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acbec45-0fb1-46fc-b8d7-11a776ca6da9 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:03.718885 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2456/4832] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:04.017669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56547695-83ae-48b8-a59c-b29a9f492329 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:04.018191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56547695-83ae-48b8-a59c-b29a9f492329 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:04.018654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56547695-83ae-48b8-a59c-b29a9f492329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:04.062664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-56547695-83ae-48b8-a59c-b29a9f492329 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:04.064417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-56547695-83ae-48b8-a59c-b29a9f492329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:04.211309 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2377/4833] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:07:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:04.225538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbec45-0fb1-46fc-b8d7-11a776ca6da9 service glance] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:04.235133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7acbec45-0fb1-46fc-b8d7-11a776ca6da9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:04.516441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56547695-83ae-48b8-a59c-b29a9f492329 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:04.518332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-56547695-83ae-48b8-a59c-b29a9f492329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:04.612068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbec45-0fb1-46fc-b8d7-11a776ca6da9 service glance] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:04.618398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655aaf4-8ef2-4ad6-bfe0-ab951bbeb02a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:04.619469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655aaf4-8ef2-4ad6-bfe0-ab951bbeb02a service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:04.619907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655aaf4-8ef2-4ad6-bfe0-ab951bbeb02a service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:04.620480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655aaf4-8ef2-4ad6-bfe0-ab951bbeb02a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:04.637044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655aaf4-8ef2-4ad6-bfe0-ab951bbeb02a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:04.641151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655aaf4-8ef2-4ad6-bfe0-ab951bbeb02a service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:04.680168 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2457/4834] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:07:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:07:04.736983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:04.775837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56547695-83ae-48b8-a59c-b29a9f492329 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:04.802884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef1782-f91b-4c1b-8b93-98e88d1a7e23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:04.803671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef1782-f91b-4c1b-8b93-98e88d1a7e23 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:04.804147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef1782-f91b-4c1b-8b93-98e88d1a7e23 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:04.804395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef1782-f91b-4c1b-8b93-98e88d1a7e23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:04.810118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-05ef1782-f91b-4c1b-8b93-98e88d1a7e23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:04.842949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef1782-f91b-4c1b-8b93-98e88d1a7e23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:04.844707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef1782-f91b-4c1b-8b93-98e88d1a7e23 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:04.870552 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2378/4835] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:07:05.470827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8b7de-c2fa-4baf-afb3-c6baf86e7144 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:05.471364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8b7de-c2fa-4baf-afb3-c6baf86e7144 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:05.471729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8b7de-c2fa-4baf-afb3-c6baf86e7144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:05.497134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-19a8b7de-c2fa-4baf-afb3-c6baf86e7144 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:05.500036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-19a8b7de-c2fa-4baf-afb3-c6baf86e7144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:05.541210 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2458/4836] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:07:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:05.618246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef1782-f91b-4c1b-8b93-98e88d1a7e23 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:05.639584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef1782-f91b-4c1b-8b93-98e88d1a7e23 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:05.649580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76763d-fee1-41d6-92df-9c3b24d9b143 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:05.651340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76763d-fee1-41d6-92df-9c3b24d9b143 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:05.651620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76763d-fee1-41d6-92df-9c3b24d9b143 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:05.651899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76763d-fee1-41d6-92df-9c3b24d9b143 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:05.671813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76763d-fee1-41d6-92df-9c3b24d9b143 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:05.675464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76763d-fee1-41d6-92df-9c3b24d9b143 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:05.724971 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2379/4837] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:07:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:07:06.201255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d576cc9-cf29-4e34-b85c-814bbb97dfaf None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:06.202185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d576cc9-cf29-4e34-b85c-814bbb97dfaf None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:06.202659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d576cc9-cf29-4e34-b85c-814bbb97dfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:06.296836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8d576cc9-cf29-4e34-b85c-814bbb97dfaf None None] MFA Rules not processed for user `5eff5aacf85a4fda97f6eb375da7cf26`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:06.299298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8d576cc9-cf29-4e34-b85c-814bbb97dfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:06.530764 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2459/4838] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:06.543250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbf79f-a7aa-4aab-a308-ba5fbda6feff None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:06.543899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbf79f-a7aa-4aab-a308-ba5fbda6feff None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:06.543899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbf79f-a7aa-4aab-a308-ba5fbda6feff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:06.606308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d1dbf79f-a7aa-4aab-a308-ba5fbda6feff None None] MFA Rules not processed for user `5eff5aacf85a4fda97f6eb375da7cf26`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:06.611882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d1dbf79f-a7aa-4aab-a308-ba5fbda6feff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:06.619775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e61a4-048c-40ca-a386-b2a8aad2266b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:06.620293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e61a4-048c-40ca-a386-b2a8aad2266b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:06.620766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e61a4-048c-40ca-a386-b2a8aad2266b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:06.656306 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2380/4839] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:06.676049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0a2e61a4-048c-40ca-a386-b2a8aad2266b None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:06.682154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0a2e61a4-048c-40ca-a386-b2a8aad2266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:06.717731 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2460/4840] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:07:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:07:06.727415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbf79f-a7aa-4aab-a308-ba5fbda6feff None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:06.750554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbf79f-a7aa-4aab-a308-ba5fbda6feff None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:06.767010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186d036-8611-4925-b1ed-2b12730f0d92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:06.769395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186d036-8611-4925-b1ed-2b12730f0d92 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:06.772189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186d036-8611-4925-b1ed-2b12730f0d92 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:06.772671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186d036-8611-4925-b1ed-2b12730f0d92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:06.808287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e61a4-048c-40ca-a386-b2a8aad2266b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:06.809783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0a2e61a4-048c-40ca-a386-b2a8aad2266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:06.810027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186d036-8611-4925-b1ed-2b12730f0d92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:06.812753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186d036-8611-4925-b1ed-2b12730f0d92 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:06.852748 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2381/4841] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:07:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:07:06.882689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:07.089496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e61a4-048c-40ca-a386-b2a8aad2266b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:07.096039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b9aa4-47c3-4f15-bb9b-a5c1916b917c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:07.096694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b9aa4-47c3-4f15-bb9b-a5c1916b917c service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:07.097203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b9aa4-47c3-4f15-bb9b-a5c1916b917c service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:07.097489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b9aa4-47c3-4f15-bb9b-a5c1916b917c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:07.110152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b9aa4-47c3-4f15-bb9b-a5c1916b917c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:07.112055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b9aa4-47c3-4f15-bb9b-a5c1916b917c service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:07.152003 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2461/4842] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:07.194701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dfffa-386b-4465-a481-28f004309fb6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:07.195000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dfffa-386b-4465-a481-28f004309fb6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:07.195287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dfffa-386b-4465-a481-28f004309fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:07.234678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c21dfffa-386b-4465-a481-28f004309fb6 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:07.236935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c21dfffa-386b-4465-a481-28f004309fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:07.250508 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2382/4843] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:07:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:07:07.263849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b9aa4-47c3-4f15-bb9b-a5c1916b917c service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:07.281932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b9aa4-47c3-4f15-bb9b-a5c1916b917c service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:07.289054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1620abf-f46f-412b-a491-eff415761d5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:07.290122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1620abf-f46f-412b-a491-eff415761d5c service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:07.291605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1620abf-f46f-412b-a491-eff415761d5c service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:07.292164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1620abf-f46f-412b-a491-eff415761d5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:07.303003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1620abf-f46f-412b-a491-eff415761d5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:07.306096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1620abf-f46f-412b-a491-eff415761d5c service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:07.317920 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2462/4844] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:07:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:07:07.722113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142df00-cf36-496a-a463-b3b4ccd4cef9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:07.723033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142df00-cf36-496a-a463-b3b4ccd4cef9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:07.723501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142df00-cf36-496a-a463-b3b4ccd4cef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:07.759933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a142df00-cf36-496a-a463-b3b4ccd4cef9 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:07.761465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a142df00-cf36-496a-a463-b3b4ccd4cef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:07.783192 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2383/4845] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:07:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:07.800370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1620abf-f46f-412b-a491-eff415761d5c service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:07.834580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1620abf-f46f-412b-a491-eff415761d5c service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:07.847544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ca053-73c3-45ce-a880-566413f08ec3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:07.849319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ca053-73c3-45ce-a880-566413f08ec3 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:07.849969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ca053-73c3-45ce-a880-566413f08ec3 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:07.850350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ca053-73c3-45ce-a880-566413f08ec3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:07.874260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ca053-73c3-45ce-a880-566413f08ec3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:07.876495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ca053-73c3-45ce-a880-566413f08ec3 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:07.890730 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2463/4846] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:07:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:07:08.191941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9237b-11ac-494f-87dd-05c3804aa9c6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:08.192578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9237b-11ac-494f-87dd-05c3804aa9c6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:08.193002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9237b-11ac-494f-87dd-05c3804aa9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:08.232771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d8b9237b-11ac-494f-87dd-05c3804aa9c6 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:08.234471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d8b9237b-11ac-494f-87dd-05c3804aa9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:08.245010 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2384/4847] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:07:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:08.258399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ca053-73c3-45ce-a880-566413f08ec3 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:08.283890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ca053-73c3-45ce-a880-566413f08ec3 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:08.289735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b287a1-109f-479b-bbe4-07578dd90ea3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:08.290356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b287a1-109f-479b-bbe4-07578dd90ea3 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:08.290641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b287a1-109f-479b-bbe4-07578dd90ea3 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:08.290918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b287a1-109f-479b-bbe4-07578dd90ea3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:08.310124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b287a1-109f-479b-bbe4-07578dd90ea3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:08.312756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b287a1-109f-479b-bbe4-07578dd90ea3 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:08.329915 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2464/4848] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:07:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:07:08.551423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9237b-11ac-494f-87dd-05c3804aa9c6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:08.754653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9237b-11ac-494f-87dd-05c3804aa9c6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:08.763498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:08.764195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:08.764486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:08.764765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] PATH_INFO: `/v3/users/64bc0fb485724cf882cade1ea07d5b0f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:08.766607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64bc0fb485724cf882cade1ea07d5b0f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:08.772735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:08.881976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64bc0fb485724cf882cade1ea07d5b0f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:08.900036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64bc0fb485724cf882cade1ea07d5b0f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:08.901461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64bc0fb485724cf882cade1ea07d5b0f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:08.912845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64bc0fb485724cf882cade1ea07d5b0f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:08.925351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64bc0fb485724cf882cade1ea07d5b0f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:08.934653 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2385/4849] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:08 2022] DELETE /identity/v3/users/64bc0fb485724cf882cade1ea07d5b0f => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:07:08.940522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b352e-a86e-47b9-a946-37e64536ba71 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:08.941258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b352e-a86e-47b9-a946-37e64536ba71 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:08.941729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b352e-a86e-47b9-a946-37e64536ba71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:08.967081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3d3b352e-a86e-47b9-a946-37e64536ba71 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:08.968611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3d3b352e-a86e-47b9-a946-37e64536ba71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:09.042166 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2465/4850] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:07:09.059247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:09.061133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:09.089041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:09.181743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe82a11-7040-4307-8ebf-f9e302d5bbb0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:09.191843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc9981-489e-42cb-b667-c64130625b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:09.193614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc9981-489e-42cb-b667-c64130625b55 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:09.194493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc9981-489e-42cb-b667-c64130625b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:09.195410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc9981-489e-42cb-b667-c64130625b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:09.208867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc9981-489e-42cb-b667-c64130625b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:09.213912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc9981-489e-42cb-b667-c64130625b55 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:09.237963 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2386/4851] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:11.323098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b352e-a86e-47b9-a946-37e64536ba71 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:11.324769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3d3b352e-a86e-47b9-a946-37e64536ba71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:11.401228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b352e-a86e-47b9-a946-37e64536ba71 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:11.407495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf6fda-fc8c-4174-b310-7f13bffda055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:11.409176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf6fda-fc8c-4174-b310-7f13bffda055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:11.409622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf6fda-fc8c-4174-b310-7f13bffda055 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:11.410050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf6fda-fc8c-4174-b310-7f13bffda055 admin admin] PATH_INFO: `/v3/projects/d5ffc10ee5674988a61423746255caf0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:11.411978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf6fda-fc8c-4174-b310-7f13bffda055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ffc10ee5674988a61423746255caf0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:11.414980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf6fda-fc8c-4174-b310-7f13bffda055 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:11.556013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31bf6fda-fc8c-4174-b310-7f13bffda055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ffc10ee5674988a61423746255caf0', 'request_id': 'req-31bf6fda-fc8c-4174-b310-7f13bffda055'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:11.560094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:11.570377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31bf6fda-fc8c-4174-b310-7f13bffda055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ffc10ee5674988a61423746255caf0', 'request_id': 'req-31bf6fda-fc8c-4174-b310-7f13bffda055'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:11.626562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31bf6fda-fc8c-4174-b310-7f13bffda055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ffc10ee5674988a61423746255caf0', 'request_id': 'req-31bf6fda-fc8c-4174-b310-7f13bffda055'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:11.649572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31bf6fda-fc8c-4174-b310-7f13bffda055 admin admin] The token cache is being invalidate because project d5ffc10ee5674988a61423746255caf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:11.650428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31bf6fda-fc8c-4174-b310-7f13bffda055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:11.651369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31bf6fda-fc8c-4174-b310-7f13bffda055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:11.653612 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2466/4852] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:11 2022] DELETE /identity/v3/projects/d5ffc10ee5674988a61423746255caf0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:11.657850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd3580-eeb5-4730-9ed0-868661e8755f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:11.658348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd3580-eeb5-4730-9ed0-868661e8755f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:11.658443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd3580-eeb5-4730-9ed0-868661e8755f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:11.724577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-64bd3580-eeb5-4730-9ed0-868661e8755f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:11.727980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-64bd3580-eeb5-4730-9ed0-868661e8755f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:11.865900 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2387/4853] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:11.870671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4104e8f-e267-41fd-817f-40c366898972 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:11.871325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4104e8f-e267-41fd-817f-40c366898972 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:11.871758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4104e8f-e267-41fd-817f-40c366898972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:11.920084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f4104e8f-e267-41fd-817f-40c366898972 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:11.921542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f4104e8f-e267-41fd-817f-40c366898972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:11.948309 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2467/4854] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:11.949203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd3580-eeb5-4730-9ed0-868661e8755f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:11.965722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd3580-eeb5-4730-9ed0-868661e8755f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:11.981884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bdbd2-87a6-4936-b59a-28fd91485725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:11.982566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bdbd2-87a6-4936-b59a-28fd91485725 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:11.982959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bdbd2-87a6-4936-b59a-28fd91485725 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:11.983355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bdbd2-87a6-4936-b59a-28fd91485725 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:11.984215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bdbd2-87a6-4936-b59a-28fd91485725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:11.984693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bdbd2-87a6-4936-b59a-28fd91485725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:11.996002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bdbd2-87a6-4936-b59a-28fd91485725 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:12.025082 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2388/4855] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:07:12.027291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4104e8f-e267-41fd-817f-40c366898972 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:12.050482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4104e8f-e267-41fd-817f-40c366898972 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:12.058515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192543a-f8ee-4e8e-b137-5df4f29e797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:12.059338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192543a-f8ee-4e8e-b137-5df4f29e797a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:12.059725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192543a-f8ee-4e8e-b137-5df4f29e797a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:12.060234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192543a-f8ee-4e8e-b137-5df4f29e797a admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:12.061121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192543a-f8ee-4e8e-b137-5df4f29e797a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:12.068665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192543a-f8ee-4e8e-b137-5df4f29e797a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:12.159246 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2468/4856] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:12 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:12.160393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bdbd2-87a6-4936-b59a-28fd91485725 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:12.173683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bdbd2-87a6-4936-b59a-28fd91485725 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:12.279236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b5c6b-253a-4cc9-bf8f-7b0fa1caf017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:12.280063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b5c6b-253a-4cc9-bf8f-7b0fa1caf017 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:12.280521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b5c6b-253a-4cc9-bf8f-7b0fa1caf017 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:12.281105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b5c6b-253a-4cc9-bf8f-7b0fa1caf017 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:12.282262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b5c6b-253a-4cc9-bf8f-7b0fa1caf017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:12.285150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b5c6b-253a-4cc9-bf8f-7b0fa1caf017 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:12.335015 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2389/4857] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:12 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:12.338927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192543a-f8ee-4e8e-b137-5df4f29e797a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:12.355496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192543a-f8ee-4e8e-b137-5df4f29e797a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:12.361068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027f390-cf47-41d8-9d37-3b9ee67bf6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:12.361861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027f390-cf47-41d8-9d37-3b9ee67bf6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:12.362152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027f390-cf47-41d8-9d37-3b9ee67bf6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:12.362434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027f390-cf47-41d8-9d37-3b9ee67bf6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:12.363093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027f390-cf47-41d8-9d37-3b9ee67bf6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:12.364974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027f390-cf47-41d8-9d37-3b9ee67bf6a6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:12.381952 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2469/4858] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:07:12.386060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b5c6b-253a-4cc9-bf8f-7b0fa1caf017 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:12.400356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b5c6b-253a-4cc9-bf8f-7b0fa1caf017 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:12.406794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f773-98f4-425a-8ccc-e509534ea894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:12.409031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f773-98f4-425a-8ccc-e509534ea894 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:12.409991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f773-98f4-425a-8ccc-e509534ea894 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:12.410394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f773-98f4-425a-8ccc-e509534ea894 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:12.411245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6f773-98f4-425a-8ccc-e509534ea894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:12.413646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6f773-98f4-425a-8ccc-e509534ea894 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:12.433113 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2390/4859] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:12.436719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027f390-cf47-41d8-9d37-3b9ee67bf6a6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:12.449309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027f390-cf47-41d8-9d37-3b9ee67bf6a6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:12.455904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce361a7-fb22-494d-8c78-700411a3ff46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:12.456565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce361a7-fb22-494d-8c78-700411a3ff46 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:12.456950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce361a7-fb22-494d-8c78-700411a3ff46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:12.457326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce361a7-fb22-494d-8c78-700411a3ff46 admin admin] PATH_INFO: `/v3/projects/52b16fc2508d49009c3bcf4ce85d7854/users/c0472774a0614615bd9c9f9b60360fe1/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:12.478289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce361a7-fb22-494d-8c78-700411a3ff46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52b16fc2508d49009c3bcf4ce85d7854, user_id=c0472774a0614615bd9c9f9b60360fe1, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:12.488220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce361a7-fb22-494d-8c78-700411a3ff46 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:12.516866 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2470/4860] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:12 2022] PUT /identity/v3/projects/52b16fc2508d49009c3bcf4ce85d7854/users/c0472774a0614615bd9c9f9b60360fe1/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:07:12.518737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db219a30-6a01-466b-aa42-e5928f247124 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:12.519059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db219a30-6a01-466b-aa42-e5928f247124 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:12.519337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db219a30-6a01-466b-aa42-e5928f247124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:12.565815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-db219a30-6a01-466b-aa42-e5928f247124 None None] MFA Rules not processed for user `c0472774a0614615bd9c9f9b60360fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:12.567266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-db219a30-6a01-466b-aa42-e5928f247124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:12.595865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:12.694277 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2391/4861] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:12.696202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce361a7-fb22-494d-8c78-700411a3ff46 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:12.873357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce361a7-fb22-494d-8c78-700411a3ff46 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:12.890523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f46994-168c-40ce-b72c-d81b5a046df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:12.891236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f46994-168c-40ce-b72c-d81b5a046df0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:12.891524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f46994-168c-40ce-b72c-d81b5a046df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:12.891796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f46994-168c-40ce-b72c-d81b5a046df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:12.892755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f46994-168c-40ce-b72c-d81b5a046df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:12.894731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f46994-168c-40ce-b72c-d81b5a046df0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:12.964462 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2471/4862] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:12 2022] POST /identity/v3/projects => generated 377 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:12.968850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db219a30-6a01-466b-aa42-e5928f247124 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:12.995744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db219a30-6a01-466b-aa42-e5928f247124 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:13.140711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15518037-aec9-499a-a2c5-0bbffbffc39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:13.142340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15518037-aec9-499a-a2c5-0bbffbffc39d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:13.143444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15518037-aec9-499a-a2c5-0bbffbffc39d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:13.143864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15518037-aec9-499a-a2c5-0bbffbffc39d admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:13.144643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15518037-aec9-499a-a2c5-0bbffbffc39d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:13.147920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15518037-aec9-499a-a2c5-0bbffbffc39d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:13.199122 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2392/4863] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:12 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:13.203674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f46994-168c-40ce-b72c-d81b5a046df0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:13.244143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f46994-168c-40ce-b72c-d81b5a046df0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:13.256493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf4a16-6c35-472d-aeda-280cd3e5c39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:13.257319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf4a16-6c35-472d-aeda-280cd3e5c39a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:13.257610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf4a16-6c35-472d-aeda-280cd3e5c39a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:13.257889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf4a16-6c35-472d-aeda-280cd3e5c39a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:13.258554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf4a16-6c35-472d-aeda-280cd3e5c39a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:13.260173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf4a16-6c35-472d-aeda-280cd3e5c39a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:13.284883 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2472/4864] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:13.288845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15518037-aec9-499a-a2c5-0bbffbffc39d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:13.310123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15518037-aec9-499a-a2c5-0bbffbffc39d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:13.318657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e539d-a804-4ef5-89f3-e3f98e22616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:13.319446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e539d-a804-4ef5-89f3-e3f98e22616c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:13.319826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e539d-a804-4ef5-89f3-e3f98e22616c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:13.320482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e539d-a804-4ef5-89f3-e3f98e22616c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:13.324970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e539d-a804-4ef5-89f3-e3f98e22616c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:13.327261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e539d-a804-4ef5-89f3-e3f98e22616c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:13.344493 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2393/4865] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:13.349551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf4a16-6c35-472d-aeda-280cd3e5c39a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:13.366798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf4a16-6c35-472d-aeda-280cd3e5c39a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:13.399734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469f9f6-306b-4c46-b509-2e0da28c4fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:13.400538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469f9f6-306b-4c46-b509-2e0da28c4fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:13.403531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469f9f6-306b-4c46-b509-2e0da28c4fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:13.403957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469f9f6-306b-4c46-b509-2e0da28c4fb4 admin admin] PATH_INFO: `/v3/projects/2826e2ef2ade4f0486f74e5515150f20/users/92f7b29887934513a2354fb8b0c37708/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:13.430864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469f9f6-306b-4c46-b509-2e0da28c4fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2826e2ef2ade4f0486f74e5515150f20, user_id=92f7b29887934513a2354fb8b0c37708, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:13.434450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469f9f6-306b-4c46-b509-2e0da28c4fb4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:13.467909 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2473/4866] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:13 2022] PUT /identity/v3/projects/2826e2ef2ade4f0486f74e5515150f20/users/92f7b29887934513a2354fb8b0c37708/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:13.479429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0681bf-553a-4ed1-a448-7f09734bdf58 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:13.479728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0681bf-553a-4ed1-a448-7f09734bdf58 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:13.480012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0681bf-553a-4ed1-a448-7f09734bdf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:13.531687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4c0681bf-553a-4ed1-a448-7f09734bdf58 None None] MFA Rules not processed for user `92f7b29887934513a2354fb8b0c37708`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:13.532961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4c0681bf-553a-4ed1-a448-7f09734bdf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:13.620561 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2394/4867] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:13.629212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469f9f6-306b-4c46-b509-2e0da28c4fb4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:13.631862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8469f9f6-306b-4c46-b509-2e0da28c4fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:13.681218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:13.752086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469f9f6-306b-4c46-b509-2e0da28c4fb4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:13.763082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ef251-6fc0-4785-a2bb-62ae9bf587b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:13.764128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ef251-6fc0-4785-a2bb-62ae9bf587b9 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:13.764620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ef251-6fc0-4785-a2bb-62ae9bf587b9 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:13.765212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ef251-6fc0-4785-a2bb-62ae9bf587b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:13.843637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ef251-6fc0-4785-a2bb-62ae9bf587b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:13.845180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ef251-6fc0-4785-a2bb-62ae9bf587b9 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:13.877544 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2474/4868] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:14.124428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f6a65-ac5f-4fcf-b879-dffa4405e86a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:14.124883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f6a65-ac5f-4fcf-b879-dffa4405e86a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:14.125876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f6a65-ac5f-4fcf-b879-dffa4405e86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:14.155738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fb5f6a65-ac5f-4fcf-b879-dffa4405e86a None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:14.157518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fb5f6a65-ac5f-4fcf-b879-dffa4405e86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:14.171927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:14.236864 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2395/4869] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:14.249265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ef251-6fc0-4785-a2bb-62ae9bf587b9 service glance] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:14.250907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-728ef251-6fc0-4785-a2bb-62ae9bf587b9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:14.335846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ef251-6fc0-4785-a2bb-62ae9bf587b9 service glance] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:14.341111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae7302-e174-43bc-b31a-b8bdeda3a795 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:14.341794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae7302-e174-43bc-b31a-b8bdeda3a795 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:14.342096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae7302-e174-43bc-b31a-b8bdeda3a795 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:14.342367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae7302-e174-43bc-b31a-b8bdeda3a795 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:14.352639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae7302-e174-43bc-b31a-b8bdeda3a795 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:14.354998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae7302-e174-43bc-b31a-b8bdeda3a795 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:14.364692 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2475/4870] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:07:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:07:14.681533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f6a65-ac5f-4fcf-b879-dffa4405e86a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:14.683594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fb5f6a65-ac5f-4fcf-b879-dffa4405e86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:14.702212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f6a65-ac5f-4fcf-b879-dffa4405e86a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:14.706567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f25c9-569f-4b1e-805a-4a54be41b847 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:14.707342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f25c9-569f-4b1e-805a-4a54be41b847 service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:14.709165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f25c9-569f-4b1e-805a-4a54be41b847 service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:14.709568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f25c9-569f-4b1e-805a-4a54be41b847 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:14.730371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f25c9-569f-4b1e-805a-4a54be41b847 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:14.732720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f25c9-569f-4b1e-805a-4a54be41b847 service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:14.746299 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2396/4871] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:14.877840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c8c5b-578f-4e65-a8c5-92da7a06fa4f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:14.878355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c8c5b-578f-4e65-a8c5-92da7a06fa4f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:14.878946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c8c5b-578f-4e65-a8c5-92da7a06fa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:14.901513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8b9c8c5b-578f-4e65-a8c5-92da7a06fa4f None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:14.903024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8b9c8c5b-578f-4e65-a8c5-92da7a06fa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:14.913470 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2476/4872] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:14.926606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f25c9-569f-4b1e-805a-4a54be41b847 service glance] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:14.941871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f25c9-569f-4b1e-805a-4a54be41b847 service glance] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:14.947599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc584bbe-e953-49e2-ad6e-a151d5c40c96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:14.948229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc584bbe-e953-49e2-ad6e-a151d5c40c96 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:14.948515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc584bbe-e953-49e2-ad6e-a151d5c40c96 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:14.948795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc584bbe-e953-49e2-ad6e-a151d5c40c96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:14.959430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc584bbe-e953-49e2-ad6e-a151d5c40c96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:14.961166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc584bbe-e953-49e2-ad6e-a151d5c40c96 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:14.970970 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2397/4873] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:07:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:07:15.089244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747b260-3068-45df-9e07-2c1d957baa57 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:15.091753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747b260-3068-45df-9e07-2c1d957baa57 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:15.092170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747b260-3068-45df-9e07-2c1d957baa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:15.134460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2747b260-3068-45df-9e07-2c1d957baa57 None None] MFA Rules not processed for user `1cf3a90cb1034ae39f67fb0fa74366f7`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:15.135935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2747b260-3068-45df-9e07-2c1d957baa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:15.159836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc584bbe-e953-49e2-ad6e-a151d5c40c96 service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:15.273210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:15.314877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc584bbe-e953-49e2-ad6e-a151d5c40c96 service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:15.322811 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2477/4874] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:15 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:15.323529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d703f-9556-4215-9be8-b3f62946ad35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:15.323909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747b260-3068-45df-9e07-2c1d957baa57 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:15.324743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d703f-9556-4215-9be8-b3f62946ad35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:15.325285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d703f-9556-4215-9be8-b3f62946ad35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:15.325767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d703f-9556-4215-9be8-b3f62946ad35 admin admin] PATH_INFO: `/v3/users/c0472774a0614615bd9c9f9b60360fe1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:15.327170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d703f-9556-4215-9be8-b3f62946ad35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0472774a0614615bd9c9f9b60360fe1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:15.328491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2747b260-3068-45df-9e07-2c1d957baa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:15.329839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d703f-9556-4215-9be8-b3f62946ad35 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:15.372421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747b260-3068-45df-9e07-2c1d957baa57 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:15.385244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09fb5c-87c2-45fb-a844-bf90c33a8711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:15.388122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09fb5c-87c2-45fb-a844-bf90c33a8711 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:15.389542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09fb5c-87c2-45fb-a844-bf90c33a8711 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:15.391183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09fb5c-87c2-45fb-a844-bf90c33a8711 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:15.393447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09fb5c-87c2-45fb-a844-bf90c33a8711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:15.397798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09fb5c-87c2-45fb-a844-bf90c33a8711 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:15.409865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-327d703f-9556-4215-9be8-b3f62946ad35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0472774a0614615bd9c9f9b60360fe1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:15.440483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-327d703f-9556-4215-9be8-b3f62946ad35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0472774a0614615bd9c9f9b60360fe1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:15.441340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-327d703f-9556-4215-9be8-b3f62946ad35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0472774a0614615bd9c9f9b60360fe1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:15.450717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-327d703f-9556-4215-9be8-b3f62946ad35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0472774a0614615bd9c9f9b60360fe1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:15.460503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-327d703f-9556-4215-9be8-b3f62946ad35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0472774a0614615bd9c9f9b60360fe1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:15.467965 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2478/4875] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:15 2022] POST /identity/v3/projects => generated 405 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:15.469192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09fb5c-87c2-45fb-a844-bf90c33a8711 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:15.472092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3d09fb5c-87c2-45fb-a844-bf90c33a8711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:15.483988 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2398/4876] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:15 2022] DELETE /identity/v3/users/c0472774a0614615bd9c9f9b60360fe1 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:07:15.485526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c753b-f449-4a17-bf2c-ee3b24154d18 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:15.485818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c753b-f449-4a17-bf2c-ee3b24154d18 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:15.486141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c753b-f449-4a17-bf2c-ee3b24154d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:15.531770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d43c753b-f449-4a17-bf2c-ee3b24154d18 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:15.535899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d43c753b-f449-4a17-bf2c-ee3b24154d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:15.560944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:15.597829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09fb5c-87c2-45fb-a844-bf90c33a8711 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:15.653919 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2399/4877] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:15.665311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c753b-f449-4a17-bf2c-ee3b24154d18 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:15.669951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d43c753b-f449-4a17-bf2c-ee3b24154d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:15.704432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ecfb7-1bce-4acc-831a-2ade8d60d588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:15.705291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ecfb7-1bce-4acc-831a-2ade8d60d588 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:15.706417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ecfb7-1bce-4acc-831a-2ade8d60d588 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:15.706880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ecfb7-1bce-4acc-831a-2ade8d60d588 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:15.708026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ecfb7-1bce-4acc-831a-2ade8d60d588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:15.710588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ecfb7-1bce-4acc-831a-2ade8d60d588 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:15.739891 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2479/4878] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:15 2022] POST /identity/v3/users => generated 303 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:15.743068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ecfb7-1bce-4acc-831a-2ade8d60d588 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:15.756637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ecfb7-1bce-4acc-831a-2ade8d60d588 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:15.761958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c753b-f449-4a17-bf2c-ee3b24154d18 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:15.762344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25c287-8e9c-42a8-a7f9-9c50fd0e09f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:15.763233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25c287-8e9c-42a8-a7f9-9c50fd0e09f0 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:15.763388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25c287-8e9c-42a8-a7f9-9c50fd0e09f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:15.763636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25c287-8e9c-42a8-a7f9-9c50fd0e09f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:15.764304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25c287-8e9c-42a8-a7f9-9c50fd0e09f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:15.766047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25c287-8e9c-42a8-a7f9-9c50fd0e09f0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:15.770102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd254f-8dda-4bf9-886b-78ececf6a3a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:15.771455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd254f-8dda-4bf9-886b-78ececf6a3a5 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:15.772016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd254f-8dda-4bf9-886b-78ececf6a3a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:15.772444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd254f-8dda-4bf9-886b-78ececf6a3a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:15.782578 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2480/4879] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:07:15.784610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25c287-8e9c-42a8-a7f9-9c50fd0e09f0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:15.787602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd254f-8dda-4bf9-886b-78ececf6a3a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:15.789270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd254f-8dda-4bf9-886b-78ececf6a3a5 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:15.798841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25c287-8e9c-42a8-a7f9-9c50fd0e09f0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:15.805536 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2400/4880] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:07:15.820940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12bf69-764f-476d-9cf2-1ccf90e6be9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:15.821577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12bf69-764f-476d-9cf2-1ccf90e6be9c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:15.821972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12bf69-764f-476d-9cf2-1ccf90e6be9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:15.822255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12bf69-764f-476d-9cf2-1ccf90e6be9c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:15.823831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12bf69-764f-476d-9cf2-1ccf90e6be9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:15.826561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12bf69-764f-476d-9cf2-1ccf90e6be9c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:15.849780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd254f-8dda-4bf9-886b-78ececf6a3a5 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:15.860261 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2481/4881] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:15.862769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd254f-8dda-4bf9-886b-78ececf6a3a5 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:15.868603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee36a3-17f1-4924-9424-279f70e4700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:15.870425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee36a3-17f1-4924-9424-279f70e4700d admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:15.870425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee36a3-17f1-4924-9424-279f70e4700d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:15.870425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee36a3-17f1-4924-9424-279f70e4700d admin admin] PATH_INFO: `/v3/projects/e2834d4f687747f2b38e3ab7ffa8fd56/users/e052a3c319774afb8b373adf58417a46/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:15.893550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee36a3-17f1-4924-9424-279f70e4700d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2834d4f687747f2b38e3ab7ffa8fd56, user_id=e052a3c319774afb8b373adf58417a46, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:15.898057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee36a3-17f1-4924-9424-279f70e4700d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:15.924454 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2401/4882] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:15 2022] PUT /identity/v3/projects/e2834d4f687747f2b38e3ab7ffa8fd56/users/e052a3c319774afb8b373adf58417a46/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:16.548391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12bf69-764f-476d-9cf2-1ccf90e6be9c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:16.550363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1c12bf69-764f-476d-9cf2-1ccf90e6be9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:16.574581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12bf69-764f-476d-9cf2-1ccf90e6be9c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:16.641016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:16.662454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:16.663816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:16.664312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:16.664791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] PATH_INFO: `/v3/projects/52b16fc2508d49009c3bcf4ce85d7854` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:16.666243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52b16fc2508d49009c3bcf4ce85d7854)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:16.668932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:16.769870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52b16fc2508d49009c3bcf4ce85d7854', 'request_id': 'req-0f61921e-de79-4b58-b9bb-fe54a27f9595'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:16.778405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52b16fc2508d49009c3bcf4ce85d7854', 'request_id': 'req-0f61921e-de79-4b58-b9bb-fe54a27f9595'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:16.786817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52b16fc2508d49009c3bcf4ce85d7854', 'request_id': 'req-0f61921e-de79-4b58-b9bb-fe54a27f9595'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:16.801121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] The token cache is being invalidate because project 52b16fc2508d49009c3bcf4ce85d7854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:16.802229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:16.803055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:16.805064 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2482/4883] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:16 2022] DELETE /identity/v3/projects/52b16fc2508d49009c3bcf4ce85d7854 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:16.808942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee36a3-17f1-4924-9424-279f70e4700d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:16.811359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-93ee36a3-17f1-4924-9424-279f70e4700d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:16.827421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee36a3-17f1-4924-9424-279f70e4700d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:16.903453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43075f6-0fc0-404c-ab30-a386073c5f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:16.904261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43075f6-0fc0-404c-ab30-a386073c5f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:16.904656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43075f6-0fc0-404c-ab30-a386073c5f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:16.905102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43075f6-0fc0-404c-ab30-a386073c5f19 admin admin] PATH_INFO: `/v3/users/92f7b29887934513a2354fb8b0c37708` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:16.906365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43075f6-0fc0-404c-ab30-a386073c5f19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92f7b29887934513a2354fb8b0c37708)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:16.908412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43075f6-0fc0-404c-ab30-a386073c5f19 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:16.954192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:16.984310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c43075f6-0fc0-404c-ab30-a386073c5f19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92f7b29887934513a2354fb8b0c37708'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:17.015085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c43075f6-0fc0-404c-ab30-a386073c5f19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92f7b29887934513a2354fb8b0c37708'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:17.015850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c43075f6-0fc0-404c-ab30-a386073c5f19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92f7b29887934513a2354fb8b0c37708'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:17.024860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c43075f6-0fc0-404c-ab30-a386073c5f19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92f7b29887934513a2354fb8b0c37708'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:17.034265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c43075f6-0fc0-404c-ab30-a386073c5f19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92f7b29887934513a2354fb8b0c37708'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:17.048492 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2402/4884] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:16 2022] DELETE /identity/v3/users/92f7b29887934513a2354fb8b0c37708 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:17.428240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:17.440734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61921e-de79-4b58-b9bb-fe54a27f9595 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:17.557162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e7bea-cfea-468c-bc79-b577988a40bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:17.557869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e7bea-cfea-468c-bc79-b577988a40bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:17.558281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e7bea-cfea-468c-bc79-b577988a40bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:17.558641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e7bea-cfea-468c-bc79-b577988a40bb admin admin] PATH_INFO: `/v3/projects/2826e2ef2ade4f0486f74e5515150f20` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:17.560055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e7bea-cfea-468c-bc79-b577988a40bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2826e2ef2ade4f0486f74e5515150f20)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:17.562399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e7bea-cfea-468c-bc79-b577988a40bb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:17.632759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-341e7bea-cfea-468c-bc79-b577988a40bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2826e2ef2ade4f0486f74e5515150f20', 'request_id': 'req-341e7bea-cfea-468c-bc79-b577988a40bb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:17.655089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-341e7bea-cfea-468c-bc79-b577988a40bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2826e2ef2ade4f0486f74e5515150f20', 'request_id': 'req-341e7bea-cfea-468c-bc79-b577988a40bb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:17.668167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-341e7bea-cfea-468c-bc79-b577988a40bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2826e2ef2ade4f0486f74e5515150f20', 'request_id': 'req-341e7bea-cfea-468c-bc79-b577988a40bb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:17.682746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-341e7bea-cfea-468c-bc79-b577988a40bb admin admin] The token cache is being invalidate because project 2826e2ef2ade4f0486f74e5515150f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:17.683034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-341e7bea-cfea-468c-bc79-b577988a40bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:17.683722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-341e7bea-cfea-468c-bc79-b577988a40bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:17.691140 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2483/4885] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:17 2022] DELETE /identity/v3/projects/2826e2ef2ade4f0486f74e5515150f20 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:17.694326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb57352-dd50-4ef8-b2f8-563e423cc036 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:17.696181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb57352-dd50-4ef8-b2f8-563e423cc036 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:17.698853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb57352-dd50-4ef8-b2f8-563e423cc036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:17.807565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2cb57352-dd50-4ef8-b2f8-563e423cc036 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:17.809723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2cb57352-dd50-4ef8-b2f8-563e423cc036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:17.918487 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2403/4886] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:17.927711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1e265-c33b-48ba-b058-41e65eb2a2ac None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:17.928025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1e265-c33b-48ba-b058-41e65eb2a2ac None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:17.928274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1e265-c33b-48ba-b058-41e65eb2a2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:17.943382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:17.972852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-aff1e265-c33b-48ba-b058-41e65eb2a2ac None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:17.975450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-aff1e265-c33b-48ba-b058-41e65eb2a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:18.003766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb57352-dd50-4ef8-b2f8-563e423cc036 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:18.004116 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2484/4887] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:18.023945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb57352-dd50-4ef8-b2f8-563e423cc036 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:18.041177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2620282-df6c-49c5-8812-260c5cf4dc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:18.043580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2620282-df6c-49c5-8812-260c5cf4dc36 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:18.043980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2620282-df6c-49c5-8812-260c5cf4dc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:18.045913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2620282-df6c-49c5-8812-260c5cf4dc36 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:18.046657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2620282-df6c-49c5-8812-260c5cf4dc36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:18.047047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2620282-df6c-49c5-8812-260c5cf4dc36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:18.048681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2620282-df6c-49c5-8812-260c5cf4dc36 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:18.080548 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2404/4888] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:18.082103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1e265-c33b-48ba-b058-41e65eb2a2ac None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:18.101545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1e265-c33b-48ba-b058-41e65eb2a2ac None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:18.122784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e2cf0-5f28-48c1-8335-48af4e1e86e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:18.123274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e2cf0-5f28-48c1-8335-48af4e1e86e8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:18.123567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e2cf0-5f28-48c1-8335-48af4e1e86e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:18.123863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e2cf0-5f28-48c1-8335-48af4e1e86e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:18.124502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e2cf0-5f28-48c1-8335-48af4e1e86e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:18.126543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e2cf0-5f28-48c1-8335-48af4e1e86e8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:18.191930 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2485/4889] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:18 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:18.202929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2620282-df6c-49c5-8812-260c5cf4dc36 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:18.219450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2620282-df6c-49c5-8812-260c5cf4dc36 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:18.309194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00eaae-a2f1-4183-8ff6-610d76a4479e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:18.309877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00eaae-a2f1-4183-8ff6-610d76a4479e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:18.310377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00eaae-a2f1-4183-8ff6-610d76a4479e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:18.310669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00eaae-a2f1-4183-8ff6-610d76a4479e admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:18.311400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f00eaae-a2f1-4183-8ff6-610d76a4479e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:18.313504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f00eaae-a2f1-4183-8ff6-610d76a4479e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:18.350159 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2405/4890] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:18 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:18.353923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e2cf0-5f28-48c1-8335-48af4e1e86e8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:18.368048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e2cf0-5f28-48c1-8335-48af4e1e86e8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:18.373839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47db66-ed45-4f16-ac2a-a61a70b6e9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:18.374508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47db66-ed45-4f16-ac2a-a61a70b6e9ce admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:18.374852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47db66-ed45-4f16-ac2a-a61a70b6e9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:18.375267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47db66-ed45-4f16-ac2a-a61a70b6e9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:18.376108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47db66-ed45-4f16-ac2a-a61a70b6e9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:18.377882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47db66-ed45-4f16-ac2a-a61a70b6e9ce admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:18.402873 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2486/4891] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:18.403436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00eaae-a2f1-4183-8ff6-610d76a4479e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:18.415531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00eaae-a2f1-4183-8ff6-610d76a4479e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:18.420922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01f13f-617e-4f3b-93f1-0c1563d770cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:18.421704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01f13f-617e-4f3b-93f1-0c1563d770cd admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:18.422091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01f13f-617e-4f3b-93f1-0c1563d770cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:18.422465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01f13f-617e-4f3b-93f1-0c1563d770cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:18.423562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01f13f-617e-4f3b-93f1-0c1563d770cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:18.426026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01f13f-617e-4f3b-93f1-0c1563d770cd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:18.429052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:18.466535 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2406/4892] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:18.471076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47db66-ed45-4f16-ac2a-a61a70b6e9ce admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:18.485254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47db66-ed45-4f16-ac2a-a61a70b6e9ce admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:18.491531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa0f8c-0e7b-454a-990a-0aa65ce71ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:18.492461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa0f8c-0e7b-454a-990a-0aa65ce71ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:18.492914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa0f8c-0e7b-454a-990a-0aa65ce71ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:18.493341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa0f8c-0e7b-454a-990a-0aa65ce71ee8 admin admin] PATH_INFO: `/v3/projects/a87f83d4322b4586a08cd05b98282735/users/7fea34c3dfaf489ea736dc738d922b68/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:18.515734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa0f8c-0e7b-454a-990a-0aa65ce71ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87f83d4322b4586a08cd05b98282735, user_id=7fea34c3dfaf489ea736dc738d922b68, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:18.520319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa0f8c-0e7b-454a-990a-0aa65ce71ee8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:18.550432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e024e-edcb-45c0-b1fe-db6d138b682d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:18.550733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e024e-edcb-45c0-b1fe-db6d138b682d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:18.551017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e024e-edcb-45c0-b1fe-db6d138b682d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:18.558575 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2487/4893] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:18 2022] PUT /identity/v3/projects/a87f83d4322b4586a08cd05b98282735/users/7fea34c3dfaf489ea736dc738d922b68/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:18.591464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-dd9e024e-edcb-45c0-b1fe-db6d138b682d None None] MFA Rules not processed for user `7fea34c3dfaf489ea736dc738d922b68`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:18.593618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dd9e024e-edcb-45c0-b1fe-db6d138b682d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:18.724790 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2407/4894] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:18.729914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa0f8c-0e7b-454a-990a-0aa65ce71ee8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:18.835614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa0f8c-0e7b-454a-990a-0aa65ce71ee8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:18.840697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2cf046-a6e6-495d-a6bd-e90cb279c8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:18.842701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2cf046-a6e6-495d-a6bd-e90cb279c8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:18.842701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2cf046-a6e6-495d-a6bd-e90cb279c8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:18.842701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2cf046-a6e6-495d-a6bd-e90cb279c8b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:18.842701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2cf046-a6e6-495d-a6bd-e90cb279c8b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:18.844654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2cf046-a6e6-495d-a6bd-e90cb279c8b8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:18.882752 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2488/4895] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:18 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:18.884951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e024e-edcb-45c0-b1fe-db6d138b682d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:18.901071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e024e-edcb-45c0-b1fe-db6d138b682d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:19.002291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778c3ff-920f-4d99-b6bd-34c4c1f9bf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:19.003566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778c3ff-920f-4d99-b6bd-34c4c1f9bf0f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:19.003919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778c3ff-920f-4d99-b6bd-34c4c1f9bf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:19.004206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778c3ff-920f-4d99-b6bd-34c4c1f9bf0f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:19.005709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778c3ff-920f-4d99-b6bd-34c4c1f9bf0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:19.007702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778c3ff-920f-4d99-b6bd-34c4c1f9bf0f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:19.054263 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2408/4896] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:18 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:19.069065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2cf046-a6e6-495d-a6bd-e90cb279c8b8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:19.081525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2cf046-a6e6-495d-a6bd-e90cb279c8b8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:19.087317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f7ed7-d8b1-45b6-9c65-303511886fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:19.087941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f7ed7-d8b1-45b6-9c65-303511886fce admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:19.088268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f7ed7-d8b1-45b6-9c65-303511886fce admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:19.088628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f7ed7-d8b1-45b6-9c65-303511886fce admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:19.089327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f7ed7-d8b1-45b6-9c65-303511886fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:19.091031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f7ed7-d8b1-45b6-9c65-303511886fce admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:19.108190 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2489/4897] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:19.111932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778c3ff-920f-4d99-b6bd-34c4c1f9bf0f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:19.132995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778c3ff-920f-4d99-b6bd-34c4c1f9bf0f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:19.148419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7272f-30a3-4b5d-bdee-10476cd55fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:19.149017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7272f-30a3-4b5d-bdee-10476cd55fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:19.149308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7272f-30a3-4b5d-bdee-10476cd55fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:19.149592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7272f-30a3-4b5d-bdee-10476cd55fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:19.150249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7272f-30a3-4b5d-bdee-10476cd55fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:19.151899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7272f-30a3-4b5d-bdee-10476cd55fa1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:19.171968 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2409/4898] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:19.174847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f7ed7-d8b1-45b6-9c65-303511886fce admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:19.195890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f7ed7-d8b1-45b6-9c65-303511886fce admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:19.204071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0eddf9-f148-4ef2-b8a8-3646221a283f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:19.204782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0eddf9-f148-4ef2-b8a8-3646221a283f admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:19.205194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0eddf9-f148-4ef2-b8a8-3646221a283f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:19.205497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0eddf9-f148-4ef2-b8a8-3646221a283f admin admin] PATH_INFO: `/v3/projects/84da09e660d94824bc408301edd67fd2/users/8de9ece1e45f4532846a168f09a59268/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:19.235416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0eddf9-f148-4ef2-b8a8-3646221a283f admin admin] RBAC: Authorizing `identity:create_grant(project_id=84da09e660d94824bc408301edd67fd2, user_id=8de9ece1e45f4532846a168f09a59268, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:19.239994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0eddf9-f148-4ef2-b8a8-3646221a283f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:19.262513 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2490/4899] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:19 2022] PUT /identity/v3/projects/84da09e660d94824bc408301edd67fd2/users/8de9ece1e45f4532846a168f09a59268/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:19.359258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b2fbd-271b-4f7f-b1de-4611effff896 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:19.359680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b2fbd-271b-4f7f-b1de-4611effff896 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:19.360072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b2fbd-271b-4f7f-b1de-4611effff896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:19.424221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ba6b2fbd-271b-4f7f-b1de-4611effff896 None None] MFA Rules not processed for user `8de9ece1e45f4532846a168f09a59268`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:19.425863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ba6b2fbd-271b-4f7f-b1de-4611effff896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:19.540689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:19.561319 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2410/4900] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:19.569173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0eddf9-f148-4ef2-b8a8-3646221a283f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:19.571084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3d0eddf9-f148-4ef2-b8a8-3646221a283f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:19.695642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0eddf9-f148-4ef2-b8a8-3646221a283f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:19.701791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89454aa6-d699-43be-ab4a-a659739da188 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:19.702450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89454aa6-d699-43be-ab4a-a659739da188 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:19.702843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89454aa6-d699-43be-ab4a-a659739da188 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:19.703124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89454aa6-d699-43be-ab4a-a659739da188 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:19.707875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:19.795550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89454aa6-d699-43be-ab4a-a659739da188 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:19.797178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89454aa6-d699-43be-ab4a-a659739da188 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:19.808047 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2491/4901] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:19.978582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b2fbd-271b-4f7f-b1de-4611effff896 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:19.993682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b2fbd-271b-4f7f-b1de-4611effff896 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:19.998762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd5914-8cea-431e-8d44-9eb5e3431bc8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:19.999508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd5914-8cea-431e-8d44-9eb5e3431bc8 service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:19.999877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd5914-8cea-431e-8d44-9eb5e3431bc8 service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:20.000229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd5914-8cea-431e-8d44-9eb5e3431bc8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:20.012473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd5914-8cea-431e-8d44-9eb5e3431bc8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:20.014006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd5914-8cea-431e-8d44-9eb5e3431bc8 service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:20.024813 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2411/4902] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:20.467142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89454aa6-d699-43be-ab4a-a659739da188 service glance] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:20.651294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89454aa6-d699-43be-ab4a-a659739da188 service glance] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:20.656082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:20.656738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:20.657062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:20.658149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] PATH_INFO: `/v3/users/7fea34c3dfaf489ea736dc738d922b68` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:20.659246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fea34c3dfaf489ea736dc738d922b68)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:20.661668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:20.742617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fea34c3dfaf489ea736dc738d922b68'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:20.784350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fea34c3dfaf489ea736dc738d922b68'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:20.800239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fea34c3dfaf489ea736dc738d922b68'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:20.802111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fea34c3dfaf489ea736dc738d922b68'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:20.803332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fea34c3dfaf489ea736dc738d922b68'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:20.821516 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2492/4903] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:20 2022] DELETE /identity/v3/users/7fea34c3dfaf489ea736dc738d922b68 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:20.825767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6de698-533a-4a7f-b9a0-4b9631288884 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:20.826207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6de698-533a-4a7f-b9a0-4b9631288884 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:20.826971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6de698-533a-4a7f-b9a0-4b9631288884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:20.868614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2e6de698-533a-4a7f-b9a0-4b9631288884 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:20.874574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2e6de698-533a-4a7f-b9a0-4b9631288884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:21.070180 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2412/4904] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:21.176362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:21.181290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:21.259907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:21.435234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31b184-80b7-4329-addf-15fec40429fa admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:21.450622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c1b0d-65ad-496d-a314-e55bc4de8bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:21.451545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c1b0d-65ad-496d-a314-e55bc4de8bfd service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:21.451993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c1b0d-65ad-496d-a314-e55bc4de8bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:21.453174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c1b0d-65ad-496d-a314-e55bc4de8bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:21.470020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c1b0d-65ad-496d-a314-e55bc4de8bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:21.472540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c1b0d-65ad-496d-a314-e55bc4de8bfd service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:21.493516 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2493/4905] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:07:22.230032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6de698-533a-4a7f-b9a0-4b9631288884 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:22.231575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2e6de698-533a-4a7f-b9a0-4b9631288884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:22.249299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6de698-533a-4a7f-b9a0-4b9631288884 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:22.269658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:22.271693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:22.272232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:22.272815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12 admin admin] PATH_INFO: `/v3/projects/a87f83d4322b4586a08cd05b98282735` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:22.274534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a87f83d4322b4586a08cd05b98282735)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:22.278960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:22.338341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a87f83d4322b4586a08cd05b98282735', 'request_id': 'req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:22.350395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a87f83d4322b4586a08cd05b98282735', 'request_id': 'req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:22.358350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a87f83d4322b4586a08cd05b98282735', 'request_id': 'req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:22.373999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12 admin admin] The token cache is being invalidate because project a87f83d4322b4586a08cd05b98282735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:22.374583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:22.375439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8bed9e6-0178-46db-aebc-4e7cda7d7a12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:22.377661 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2413/4906] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:22 2022] DELETE /identity/v3/projects/a87f83d4322b4586a08cd05b98282735 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:22.381585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c1b0d-65ad-496d-a314-e55bc4de8bfd service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:22.396853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c1b0d-65ad-496d-a314-e55bc4de8bfd service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:22.549236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:22.550256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:22.550754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:22.551637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] PATH_INFO: `/v3/users/8de9ece1e45f4532846a168f09a59268` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:22.553350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de9ece1e45f4532846a168f09a59268)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:22.555652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:22.638221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de9ece1e45f4532846a168f09a59268'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:22.674483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de9ece1e45f4532846a168f09a59268'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:22.692550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de9ece1e45f4532846a168f09a59268'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:22.693239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de9ece1e45f4532846a168f09a59268'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:22.694696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de9ece1e45f4532846a168f09a59268'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:22.717207 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2494/4907] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:22 2022] DELETE /identity/v3/users/8de9ece1e45f4532846a168f09a59268 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:07:22.961801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac958e6-300c-4e0c-b241-5e0807036876 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:22.962262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac958e6-300c-4e0c-b241-5e0807036876 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:22.962570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac958e6-300c-4e0c-b241-5e0807036876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:22.985920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3ac958e6-300c-4e0c-b241-5e0807036876 None None] MFA Rules not processed for user `9e686d314d164930bbcfc9905a8b45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:22.987234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3ac958e6-300c-4e0c-b241-5e0807036876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:23.113072 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2414/4908] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:23.547396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:23.549473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:23.636244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:23.677741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896dd16-8aa2-4c09-b76a-e588ab2f24ee admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:23.684299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43520529-f106-40a9-b677-90b71e1fdc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:23.685127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43520529-f106-40a9-b677-90b71e1fdc73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:23.685427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43520529-f106-40a9-b677-90b71e1fdc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:23.685752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43520529-f106-40a9-b677-90b71e1fdc73 admin admin] PATH_INFO: `/v3/projects/84da09e660d94824bc408301edd67fd2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:23.687053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43520529-f106-40a9-b677-90b71e1fdc73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84da09e660d94824bc408301edd67fd2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:23.690261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43520529-f106-40a9-b677-90b71e1fdc73 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:23.798794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-43520529-f106-40a9-b677-90b71e1fdc73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84da09e660d94824bc408301edd67fd2', 'request_id': 'req-43520529-f106-40a9-b677-90b71e1fdc73'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:23.815567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-43520529-f106-40a9-b677-90b71e1fdc73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84da09e660d94824bc408301edd67fd2', 'request_id': 'req-43520529-f106-40a9-b677-90b71e1fdc73'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:23.824987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-43520529-f106-40a9-b677-90b71e1fdc73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84da09e660d94824bc408301edd67fd2', 'request_id': 'req-43520529-f106-40a9-b677-90b71e1fdc73'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:23.861840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-43520529-f106-40a9-b677-90b71e1fdc73 admin admin] The token cache is being invalidate because project 84da09e660d94824bc408301edd67fd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:23.862149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-43520529-f106-40a9-b677-90b71e1fdc73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:23.866680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-43520529-f106-40a9-b677-90b71e1fdc73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:23.878305 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2495/4909] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:23 2022] DELETE /identity/v3/projects/84da09e660d94824bc408301edd67fd2 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:23.884595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8419b7-09e9-461e-ab7d-c07d49dbbebd None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:23.885152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8419b7-09e9-461e-ab7d-c07d49dbbebd None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:23.886114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8419b7-09e9-461e-ab7d-c07d49dbbebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:23.958628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1f8419b7-09e9-461e-ab7d-c07d49dbbebd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:23.962817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1f8419b7-09e9-461e-ab7d-c07d49dbbebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:23.976502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:24.124360 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2415/4910] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:24.126249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b31e11-de40-4380-a421-cf5d0673fa8a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:24.127083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b31e11-de40-4380-a421-cf5d0673fa8a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:24.127470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b31e11-de40-4380-a421-cf5d0673fa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:24.162386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-42b31e11-de40-4380-a421-cf5d0673fa8a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:24.163863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-42b31e11-de40-4380-a421-cf5d0673fa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:24.184834 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2496/4911] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:24.190025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8419b7-09e9-461e-ab7d-c07d49dbbebd None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:24.202431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8419b7-09e9-461e-ab7d-c07d49dbbebd None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:24.209513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf156553-e6fd-4181-8eac-0bd09d9f5bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:24.220084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf156553-e6fd-4181-8eac-0bd09d9f5bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:24.222938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf156553-e6fd-4181-8eac-0bd09d9f5bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:24.223294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf156553-e6fd-4181-8eac-0bd09d9f5bc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:24.224286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf156553-e6fd-4181-8eac-0bd09d9f5bc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:24.224602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf156553-e6fd-4181-8eac-0bd09d9f5bc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:24.226274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf156553-e6fd-4181-8eac-0bd09d9f5bc7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:24.281034 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2416/4912] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:24.282426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b31e11-de40-4380-a421-cf5d0673fa8a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:24.294759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b31e11-de40-4380-a421-cf5d0673fa8a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:24.300808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6cad8-83d3-4d98-8b64-9ef6eba1a5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:24.301660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6cad8-83d3-4d98-8b64-9ef6eba1a5f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:24.302158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6cad8-83d3-4d98-8b64-9ef6eba1a5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:24.302513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6cad8-83d3-4d98-8b64-9ef6eba1a5f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:24.303257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6cad8-83d3-4d98-8b64-9ef6eba1a5f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:24.308990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6cad8-83d3-4d98-8b64-9ef6eba1a5f8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:24.368146 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2497/4913] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:24 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:24.375197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf156553-e6fd-4181-8eac-0bd09d9f5bc7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:24.392125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf156553-e6fd-4181-8eac-0bd09d9f5bc7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:24.535190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cc2d9-9f24-414f-80a3-1cdd4c59aa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:24.535954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cc2d9-9f24-414f-80a3-1cdd4c59aa22 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:24.536320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cc2d9-9f24-414f-80a3-1cdd4c59aa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:24.536679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cc2d9-9f24-414f-80a3-1cdd4c59aa22 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:24.537442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713cc2d9-9f24-414f-80a3-1cdd4c59aa22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:24.539478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713cc2d9-9f24-414f-80a3-1cdd4c59aa22 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:24.588195 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2417/4914] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:24 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:24.590208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6cad8-83d3-4d98-8b64-9ef6eba1a5f8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:24.606373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125b801-5e80-427d-87ee-30a9fd8c009c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:24.607118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125b801-5e80-427d-87ee-30a9fd8c009c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:24.607118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125b801-5e80-427d-87ee-30a9fd8c009c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:24.617464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6cad8-83d3-4d98-8b64-9ef6eba1a5f8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:24.644321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c63bf-a99d-41ea-bb41-9f75d09fc7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:24.645356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c63bf-a99d-41ea-bb41-9f75d09fc7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:24.645783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c63bf-a99d-41ea-bb41-9f75d09fc7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:24.646223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c63bf-a99d-41ea-bb41-9f75d09fc7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:24.647845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c63bf-a99d-41ea-bb41-9f75d09fc7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:24.654885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c63bf-a99d-41ea-bb41-9f75d09fc7a4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:24.664263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0125b801-5e80-427d-87ee-30a9fd8c009c None None] MFA Rules not processed for user `e052a3c319774afb8b373adf58417a46`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:24.666077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0125b801-5e80-427d-87ee-30a9fd8c009c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:24.692580 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2498/4915] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:24.693600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c63bf-a99d-41ea-bb41-9f75d09fc7a4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:24.719571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c63bf-a99d-41ea-bb41-9f75d09fc7a4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:24.736660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a90e4-a25d-48f5-ba8e-619004310fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:24.737337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a90e4-a25d-48f5-ba8e-619004310fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:24.737623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a90e4-a25d-48f5-ba8e-619004310fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:24.737898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a90e4-a25d-48f5-ba8e-619004310fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:24.738540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a90e4-a25d-48f5-ba8e-619004310fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:24.740694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a90e4-a25d-48f5-ba8e-619004310fe2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:24.764258 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2499/4916] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:24.768544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a90e4-a25d-48f5-ba8e-619004310fe2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:24.795730 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2418/4917] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:24 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:24.795998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a90e4-a25d-48f5-ba8e-619004310fe2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:24.797664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04790a80-f5f2-424e-ae77-f3bb6eba0c14 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:24.801034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04790a80-f5f2-424e-ae77-f3bb6eba0c14 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:24.802265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3594d2-af83-4564-85c6-020852972b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:24.802468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04790a80-f5f2-424e-ae77-f3bb6eba0c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:24.803866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3594d2-af83-4564-85c6-020852972b85 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:24.804530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3594d2-af83-4564-85c6-020852972b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:24.805039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3594d2-af83-4564-85c6-020852972b85 admin admin] PATH_INFO: `/v3/projects/28bbd58184d14b3f98ab374304b86591/users/32384c0852ab4762bb7b1c1117f06289/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:24.824109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3594d2-af83-4564-85c6-020852972b85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28bbd58184d14b3f98ab374304b86591, user_id=32384c0852ab4762bb7b1c1117f06289, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:24.827198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3594d2-af83-4564-85c6-020852972b85 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:24.829807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-04790a80-f5f2-424e-ae77-f3bb6eba0c14 None None] MFA Rules not processed for user `1cf3a90cb1034ae39f67fb0fa74366f7`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:24.831384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-04790a80-f5f2-424e-ae77-f3bb6eba0c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:24.846375 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2500/4918] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:24 2022] PUT /identity/v3/projects/28bbd58184d14b3f98ab374304b86591/users/32384c0852ab4762bb7b1c1117f06289/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:24.855135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:24.861673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f5b29-0c49-419d-b12f-b2db4b7f3649 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:24.862060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f5b29-0c49-419d-b12f-b2db4b7f3649 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:24.862415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f5b29-0c49-419d-b12f-b2db4b7f3649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:24.907603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3f4f5b29-0c49-419d-b12f-b2db4b7f3649 None None] MFA Rules not processed for user `32384c0852ab4762bb7b1c1117f06289`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:24.909767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3f4f5b29-0c49-419d-b12f-b2db4b7f3649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:24.934764 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2419/4919] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:24 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:24.948241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04790a80-f5f2-424e-ae77-f3bb6eba0c14 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:24.950467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-04790a80-f5f2-424e-ae77-f3bb6eba0c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:24.970744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:25.007615 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2501/4920] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:25.032279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f5b29-0c49-419d-b12f-b2db4b7f3649 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:25.034654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3f4f5b29-0c49-419d-b12f-b2db4b7f3649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:25.107783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f5b29-0c49-419d-b12f-b2db4b7f3649 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:25.111690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04790a80-f5f2-424e-ae77-f3bb6eba0c14 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:25.112161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9186c-b26c-467f-aae4-f2c9825bc9c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:25.112796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9186c-b26c-467f-aae4-f2c9825bc9c3 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:25.113131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9186c-b26c-467f-aae4-f2c9825bc9c3 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:25.113436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9186c-b26c-467f-aae4-f2c9825bc9c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:25.125063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9186c-b26c-467f-aae4-f2c9825bc9c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:25.126817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9186c-b26c-467f-aae4-f2c9825bc9c3 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:25.137296 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2502/4921] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:25.138214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9186c-b26c-467f-aae4-f2c9825bc9c3 service glance] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:25.139591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-49d9186c-b26c-467f-aae4-f2c9825bc9c3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:25.187028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f76e4-55bf-43da-9380-06de31b0af7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:25.187756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f76e4-55bf-43da-9380-06de31b0af7e service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:25.188048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f76e4-55bf-43da-9380-06de31b0af7e service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:25.188326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f76e4-55bf-43da-9380-06de31b0af7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:25.190253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b72f76e4-55bf-43da-9380-06de31b0af7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:25.241954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9186c-b26c-467f-aae4-f2c9825bc9c3 service glance] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:25.242147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:25.250578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516c965-88a7-436a-ada0-55d131798693 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:25.253919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516c965-88a7-436a-ada0-55d131798693 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:25.254650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516c965-88a7-436a-ada0-55d131798693 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:25.255034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516c965-88a7-436a-ada0-55d131798693 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:25.257300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0516c965-88a7-436a-ada0-55d131798693 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:25.276153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f76e4-55bf-43da-9380-06de31b0af7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:25.278099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f76e4-55bf-43da-9380-06de31b0af7e service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:25.279442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516c965-88a7-436a-ada0-55d131798693 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:25.282466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516c965-88a7-436a-ada0-55d131798693 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:25.291569 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2420/4922] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:24 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:07:25.300989 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2503/4923] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:25.478362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f76e4-55bf-43da-9380-06de31b0af7e service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:25.556493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f76e4-55bf-43da-9380-06de31b0af7e service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:25.562096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:25.562750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:25.563037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:25.563315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] PATH_INFO: `/v3/users/32384c0852ab4762bb7b1c1117f06289` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:25.564500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32384c0852ab4762bb7b1c1117f06289)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:25.566497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:25.660013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32384c0852ab4762bb7b1c1117f06289'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:25.674794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32384c0852ab4762bb7b1c1117f06289'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:25.676283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32384c0852ab4762bb7b1c1117f06289'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:25.689217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32384c0852ab4762bb7b1c1117f06289'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:25.703115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32384c0852ab4762bb7b1c1117f06289'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:25.713084 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2421/4924] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:25 2022] DELETE /identity/v3/users/32384c0852ab4762bb7b1c1117f06289 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:25.721392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90308385-4a9d-401a-b94d-85370a34822b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:25.721974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90308385-4a9d-401a-b94d-85370a34822b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:25.726334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90308385-4a9d-401a-b94d-85370a34822b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:25.767163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-90308385-4a9d-401a-b94d-85370a34822b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:25.768690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-90308385-4a9d-401a-b94d-85370a34822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:25.868306 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2504/4925] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:25.890246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:25.892820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:25.987023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e9c37-46da-4535-9629-1689839fc9a2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:25.992361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d976a-0856-455f-a8d9-1de51a998590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:25.993225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d976a-0856-455f-a8d9-1de51a998590 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:25.993696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d976a-0856-455f-a8d9-1de51a998590 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:25.994221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d976a-0856-455f-a8d9-1de51a998590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:26.009912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d976a-0856-455f-a8d9-1de51a998590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:26.012413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d976a-0856-455f-a8d9-1de51a998590 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:26.026287 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2422/4926] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:26.664306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90308385-4a9d-401a-b94d-85370a34822b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:26.665922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-90308385-4a9d-401a-b94d-85370a34822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:26.684288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90308385-4a9d-401a-b94d-85370a34822b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:26.712856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b29cc-f3b8-4c0e-aa02-39401d748527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:26.713474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b29cc-f3b8-4c0e-aa02-39401d748527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:26.713757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b29cc-f3b8-4c0e-aa02-39401d748527 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:26.714033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b29cc-f3b8-4c0e-aa02-39401d748527 admin admin] PATH_INFO: `/v3/projects/28bbd58184d14b3f98ab374304b86591` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:26.715578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b29cc-f3b8-4c0e-aa02-39401d748527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28bbd58184d14b3f98ab374304b86591)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:26.717557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b29cc-f3b8-4c0e-aa02-39401d748527 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:26.739981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:26.837885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a8b29cc-f3b8-4c0e-aa02-39401d748527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28bbd58184d14b3f98ab374304b86591', 'request_id': 'req-2a8b29cc-f3b8-4c0e-aa02-39401d748527'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:26.848021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a8b29cc-f3b8-4c0e-aa02-39401d748527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28bbd58184d14b3f98ab374304b86591', 'request_id': 'req-2a8b29cc-f3b8-4c0e-aa02-39401d748527'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:26.862619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a8b29cc-f3b8-4c0e-aa02-39401d748527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28bbd58184d14b3f98ab374304b86591', 'request_id': 'req-2a8b29cc-f3b8-4c0e-aa02-39401d748527'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:26.887432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a8b29cc-f3b8-4c0e-aa02-39401d748527 admin admin] The token cache is being invalidate because project 28bbd58184d14b3f98ab374304b86591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:26.896419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a8b29cc-f3b8-4c0e-aa02-39401d748527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:26.897335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a8b29cc-f3b8-4c0e-aa02-39401d748527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:26.899178 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2505/4927] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:26 2022] DELETE /identity/v3/projects/28bbd58184d14b3f98ab374304b86591 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:26.904719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaba64-580e-4508-bed0-2af3dc235dbc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:26.905054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaba64-580e-4508-bed0-2af3dc235dbc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:26.905380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaba64-580e-4508-bed0-2af3dc235dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:26.919733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:26.943104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7ebaba64-580e-4508-bed0-2af3dc235dbc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:26.947352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7ebaba64-580e-4508-bed0-2af3dc235dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:27.094149 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2423/4928] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:27.095854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f6fac-1bf3-44a0-834b-7f915c669722 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:27.096147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f6fac-1bf3-44a0-834b-7f915c669722 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:27.096425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f6fac-1bf3-44a0-834b-7f915c669722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:27.124710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-438f6fac-1bf3-44a0-834b-7f915c669722 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:27.126742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-438f6fac-1bf3-44a0-834b-7f915c669722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:27.153417 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2506/4929] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:07:27.157867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaba64-580e-4508-bed0-2af3dc235dbc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:27.171143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaba64-580e-4508-bed0-2af3dc235dbc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:27.177919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae0fd3-9952-4d2d-87bb-a524c52fa580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:27.178928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae0fd3-9952-4d2d-87bb-a524c52fa580 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:27.179454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae0fd3-9952-4d2d-87bb-a524c52fa580 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:27.179935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae0fd3-9952-4d2d-87bb-a524c52fa580 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:27.181173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae0fd3-9952-4d2d-87bb-a524c52fa580 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:27.182335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae0fd3-9952-4d2d-87bb-a524c52fa580 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:27.184432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae0fd3-9952-4d2d-87bb-a524c52fa580 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:27.206936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f6fac-1bf3-44a0-834b-7f915c669722 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:27.214705 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2424/4930] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:27.221731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f6fac-1bf3-44a0-834b-7f915c669722 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:27.229829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b667f-b260-48b1-b04c-38b676975896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:27.230626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b667f-b260-48b1-b04c-38b676975896 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:27.231006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b667f-b260-48b1-b04c-38b676975896 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:27.231601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b667f-b260-48b1-b04c-38b676975896 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:27.232575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b667f-b260-48b1-b04c-38b676975896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:27.234865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b667f-b260-48b1-b04c-38b676975896 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:27.285334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae0fd3-9952-4d2d-87bb-a524c52fa580 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:27.293206 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2507/4931] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:27 2022] POST /identity/v3/projects => generated 377 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:27.299774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae0fd3-9952-4d2d-87bb-a524c52fa580 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:27.404083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041cd05-0cf8-4b36-abd4-c588383e0658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:27.406160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041cd05-0cf8-4b36-abd4-c588383e0658 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:27.408185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041cd05-0cf8-4b36-abd4-c588383e0658 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:27.409100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041cd05-0cf8-4b36-abd4-c588383e0658 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:27.412051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041cd05-0cf8-4b36-abd4-c588383e0658 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:27.415202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041cd05-0cf8-4b36-abd4-c588383e0658 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:27.468831 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2425/4932] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:27 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:27.470580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b667f-b260-48b1-b04c-38b676975896 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:27.483378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b667f-b260-48b1-b04c-38b676975896 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:27.497473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d4bbb-5856-4bc7-8f45-f3e8a61a689a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:27.498519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d4bbb-5856-4bc7-8f45-f3e8a61a689a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:27.499021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d4bbb-5856-4bc7-8f45-f3e8a61a689a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:27.499502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d4bbb-5856-4bc7-8f45-f3e8a61a689a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:27.500399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d4bbb-5856-4bc7-8f45-f3e8a61a689a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:27.502334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d4bbb-5856-4bc7-8f45-f3e8a61a689a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:27.520370 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2508/4933] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:27.521622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041cd05-0cf8-4b36-abd4-c588383e0658 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:27.546796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041cd05-0cf8-4b36-abd4-c588383e0658 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:27.585628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23248faa-19b2-473e-9c4a-9ad392160c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:27.586302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23248faa-19b2-473e-9c4a-9ad392160c0a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:27.586541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23248faa-19b2-473e-9c4a-9ad392160c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:27.586828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23248faa-19b2-473e-9c4a-9ad392160c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:27.587489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23248faa-19b2-473e-9c4a-9ad392160c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:27.589130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23248faa-19b2-473e-9c4a-9ad392160c0a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:27.600839 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2426/4934] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:07:27.605624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d4bbb-5856-4bc7-8f45-f3e8a61a689a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:27.617622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d4bbb-5856-4bc7-8f45-f3e8a61a689a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:27.625959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e196818-c7ae-46ae-a93a-47ee008f6b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:27.627307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e196818-c7ae-46ae-a93a-47ee008f6b0a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:27.627909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e196818-c7ae-46ae-a93a-47ee008f6b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:27.628341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e196818-c7ae-46ae-a93a-47ee008f6b0a admin admin] PATH_INFO: `/v3/projects/c8ae32e3cbed4a27972a3441b0612bed/users/138f2e296af041018519068c3a698c9c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:27.646666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e196818-c7ae-46ae-a93a-47ee008f6b0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ae32e3cbed4a27972a3441b0612bed, user_id=138f2e296af041018519068c3a698c9c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:27.652067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e196818-c7ae-46ae-a93a-47ee008f6b0a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:27.694003 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2509/4935] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:27 2022] PUT /identity/v3/projects/c8ae32e3cbed4a27972a3441b0612bed/users/138f2e296af041018519068c3a698c9c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:07:27.704238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177426f-587d-4840-8559-f19742bd7dd3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:27.704535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177426f-587d-4840-8559-f19742bd7dd3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:27.704818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177426f-587d-4840-8559-f19742bd7dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:27.775632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8177426f-587d-4840-8559-f19742bd7dd3 None None] MFA Rules not processed for user `138f2e296af041018519068c3a698c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:27.778190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8177426f-587d-4840-8559-f19742bd7dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:27.902285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:27.923460 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2427/4936] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:27.928589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e196818-c7ae-46ae-a93a-47ee008f6b0a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:28.103241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e196818-c7ae-46ae-a93a-47ee008f6b0a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:28.108446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3b9da-d975-4965-a956-6acc1e80a971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:28.109276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3b9da-d975-4965-a956-6acc1e80a971 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:28.109753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3b9da-d975-4965-a956-6acc1e80a971 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:28.110194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3b9da-d975-4965-a956-6acc1e80a971 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:28.111000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3b9da-d975-4965-a956-6acc1e80a971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:28.113133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3b9da-d975-4965-a956-6acc1e80a971 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:28.180952 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2510/4937] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:27 2022] POST /identity/v3/projects => generated 377 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:28.182553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177426f-587d-4840-8559-f19742bd7dd3 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:28.199161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177426f-587d-4840-8559-f19742bd7dd3 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:28.430707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466aa102-c2ce-41f1-bbee-ac818237d420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:28.431580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466aa102-c2ce-41f1-bbee-ac818237d420 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:28.432216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466aa102-c2ce-41f1-bbee-ac818237d420 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:28.432812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466aa102-c2ce-41f1-bbee-ac818237d420 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:28.433854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466aa102-c2ce-41f1-bbee-ac818237d420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:28.440821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466aa102-c2ce-41f1-bbee-ac818237d420 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:28.491909 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2428/4938] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:28 2022] POST /identity/v3/users => generated 295 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:28.496034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3b9da-d975-4965-a956-6acc1e80a971 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:28.520482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3b9da-d975-4965-a956-6acc1e80a971 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:28.548494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf9787-098c-4fbf-9c67-a88de2f5afd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:28.549603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf9787-098c-4fbf-9c67-a88de2f5afd4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:28.553168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf9787-098c-4fbf-9c67-a88de2f5afd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:28.553576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf9787-098c-4fbf-9c67-a88de2f5afd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:28.554387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf9787-098c-4fbf-9c67-a88de2f5afd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:28.562342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf9787-098c-4fbf-9c67-a88de2f5afd4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:28.597910 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2511/4939] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:28.600869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466aa102-c2ce-41f1-bbee-ac818237d420 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:28.618041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466aa102-c2ce-41f1-bbee-ac818237d420 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:28.633152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9766c-2dbd-420e-809b-a4e0d72f3273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:28.634983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9766c-2dbd-420e-809b-a4e0d72f3273 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:28.634983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9766c-2dbd-420e-809b-a4e0d72f3273 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:28.637325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9766c-2dbd-420e-809b-a4e0d72f3273 admin admin] PATH_INFO: `/v3/projects/99031669eb0147c4a9a6d50b79b32c63/users/e9e7eb357f514efe840a8ceb7b31419f/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:28.676614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9766c-2dbd-420e-809b-a4e0d72f3273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99031669eb0147c4a9a6d50b79b32c63, user_id=e9e7eb357f514efe840a8ceb7b31419f, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:28.685593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9766c-2dbd-420e-809b-a4e0d72f3273 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:28.715075 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2429/4940] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:28 2022] PUT /identity/v3/projects/99031669eb0147c4a9a6d50b79b32c63/users/e9e7eb357f514efe840a8ceb7b31419f/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:28.719724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf9787-098c-4fbf-9c67-a88de2f5afd4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:28.733928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf9787-098c-4fbf-9c67-a88de2f5afd4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:28.743140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:28.890707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d934bf-e71f-4116-b87d-8eb01e2f5707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:28.892518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d934bf-e71f-4116-b87d-8eb01e2f5707 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:28.892811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d934bf-e71f-4116-b87d-8eb01e2f5707 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:28.893139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d934bf-e71f-4116-b87d-8eb01e2f5707 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:28.893899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d934bf-e71f-4116-b87d-8eb01e2f5707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:28.895883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d934bf-e71f-4116-b87d-8eb01e2f5707 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:28.918548 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2512/4941] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:28.919917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9766c-2dbd-420e-809b-a4e0d72f3273 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:28.934435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9766c-2dbd-420e-809b-a4e0d72f3273 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:28.939463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b0475-fcf8-4a6e-aa93-dc560d5fb984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:28.940273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b0475-fcf8-4a6e-aa93-dc560d5fb984 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:28.940997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b0475-fcf8-4a6e-aa93-dc560d5fb984 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:28.941534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b0475-fcf8-4a6e-aa93-dc560d5fb984 admin admin] PATH_INFO: `/v3/projects/99031669eb0147c4a9a6d50b79b32c63/users/e9e7eb357f514efe840a8ceb7b31419f/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:28.945261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b0475-fcf8-4a6e-aa93-dc560d5fb984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99031669eb0147c4a9a6d50b79b32c63, user_id=e9e7eb357f514efe840a8ceb7b31419f, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:28.949461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b0475-fcf8-4a6e-aa93-dc560d5fb984 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:28.972399 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2430/4942] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:28 2022] PUT /identity/v3/projects/99031669eb0147c4a9a6d50b79b32c63/users/e9e7eb357f514efe840a8ceb7b31419f/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:28.974920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c0f7c-2860-4941-bfa4-fad83b07ee5f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:28.975368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c0f7c-2860-4941-bfa4-fad83b07ee5f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:28.975491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c0f7c-2860-4941-bfa4-fad83b07ee5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:29.010858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-067c0f7c-2860-4941-bfa4-fad83b07ee5f None None] MFA Rules not processed for user `e9e7eb357f514efe840a8ceb7b31419f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:29.012420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-067c0f7c-2860-4941-bfa4-fad83b07ee5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:29.118623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b0475-fcf8-4a6e-aa93-dc560d5fb984 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:29.120697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-291b0475-fcf8-4a6e-aa93-dc560d5fb984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:29.131739 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2513/4943] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:29.209041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b0475-fcf8-4a6e-aa93-dc560d5fb984 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:29.214195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad8c2b-d9fc-4e09-a54e-954633c2555d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:29.214930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad8c2b-d9fc-4e09-a54e-954633c2555d service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:29.215462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad8c2b-d9fc-4e09-a54e-954633c2555d service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:29.216085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad8c2b-d9fc-4e09-a54e-954633c2555d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:29.230551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad8c2b-d9fc-4e09-a54e-954633c2555d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:29.235577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad8c2b-d9fc-4e09-a54e-954633c2555d service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:29.269388 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2431/4944] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:07:31.355876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c0f7c-2860-4941-bfa4-fad83b07ee5f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:31.424964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c0f7c-2860-4941-bfa4-fad83b07ee5f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:31.434458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:31.435159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:31.435449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:31.435726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] PATH_INFO: `/v3/users/138f2e296af041018519068c3a698c9c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:31.436924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=138f2e296af041018519068c3a698c9c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:31.438862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:31.476743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:31.492253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138f2e296af041018519068c3a698c9c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:31.505476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138f2e296af041018519068c3a698c9c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:31.513584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138f2e296af041018519068c3a698c9c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:31.514340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138f2e296af041018519068c3a698c9c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:31.514970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138f2e296af041018519068c3a698c9c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:31.528269 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2514/4945] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:31 2022] DELETE /identity/v3/users/138f2e296af041018519068c3a698c9c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:31.532881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f5238-aec5-42be-b14f-355aa75fbe12 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:31.533442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f5238-aec5-42be-b14f-355aa75fbe12 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:31.533545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f5238-aec5-42be-b14f-355aa75fbe12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:31.554745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-245f5238-aec5-42be-b14f-355aa75fbe12 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:31.555996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-245f5238-aec5-42be-b14f-355aa75fbe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:31.642319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:31.658134 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2432/4946] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:31.681788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:31.683300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:31.768792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5e931-6efa-40ce-a531-20ad08858ff7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:31.772729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27a916-b578-46e0-a14f-6a53293565ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:31.773375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27a916-b578-46e0-a14f-6a53293565ee service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:31.773716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27a916-b578-46e0-a14f-6a53293565ee service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:31.774017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27a916-b578-46e0-a14f-6a53293565ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:31.783773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27a916-b578-46e0-a14f-6a53293565ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:31.786934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27a916-b578-46e0-a14f-6a53293565ee service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:31.803803 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2515/4947] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:32.627773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f5238-aec5-42be-b14f-355aa75fbe12 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:32.630161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-245f5238-aec5-42be-b14f-355aa75fbe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:32.643604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f5238-aec5-42be-b14f-355aa75fbe12 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:32.651516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:32.652491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:32.653334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:32.654021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] PATH_INFO: `/v3/projects/c8ae32e3cbed4a27972a3441b0612bed` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:32.655729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ae32e3cbed4a27972a3441b0612bed)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:32.658568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:32.727630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ae32e3cbed4a27972a3441b0612bed', 'request_id': 'req-e408f836-483e-4fd4-aa70-5619f98ab528'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:32.745426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ae32e3cbed4a27972a3441b0612bed', 'request_id': 'req-e408f836-483e-4fd4-aa70-5619f98ab528'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:32.754997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ae32e3cbed4a27972a3441b0612bed', 'request_id': 'req-e408f836-483e-4fd4-aa70-5619f98ab528'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:32.778360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] The token cache is being invalidate because project c8ae32e3cbed4a27972a3441b0612bed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:32.778360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:32.778717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:32.780953 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2433/4948] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:32 2022] DELETE /identity/v3/projects/c8ae32e3cbed4a27972a3441b0612bed => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:32.784960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27a916-b578-46e0-a14f-6a53293565ee service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:32.809537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27a916-b578-46e0-a14f-6a53293565ee service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:32.886186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0db09-e7f6-49fd-885e-b7f185b252da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:32.887007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0db09-e7f6-49fd-885e-b7f185b252da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:32.887398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0db09-e7f6-49fd-885e-b7f185b252da admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:32.889057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0db09-e7f6-49fd-885e-b7f185b252da admin admin] PATH_INFO: `/v3/users/e9e7eb357f514efe840a8ceb7b31419f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:32.889732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0db09-e7f6-49fd-885e-b7f185b252da admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9e7eb357f514efe840a8ceb7b31419f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:32.892463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0db09-e7f6-49fd-885e-b7f185b252da admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:32.971761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26e0db09-e7f6-49fd-885e-b7f185b252da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9e7eb357f514efe840a8ceb7b31419f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:32.977653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:32.992320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26e0db09-e7f6-49fd-885e-b7f185b252da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9e7eb357f514efe840a8ceb7b31419f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:33.002533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26e0db09-e7f6-49fd-885e-b7f185b252da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9e7eb357f514efe840a8ceb7b31419f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:33.003228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26e0db09-e7f6-49fd-885e-b7f185b252da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9e7eb357f514efe840a8ceb7b31419f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:33.003935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26e0db09-e7f6-49fd-885e-b7f185b252da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9e7eb357f514efe840a8ceb7b31419f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:33.028866 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2516/4949] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:32 2022] DELETE /identity/v3/users/e9e7eb357f514efe840a8ceb7b31419f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:33.522910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:33.524945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:33.550596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408f836-483e-4fd4-aa70-5619f98ab528 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:33.620920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:33.621558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:33.621845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:33.622121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b admin admin] PATH_INFO: `/v3/projects/99031669eb0147c4a9a6d50b79b32c63` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:33.623230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=99031669eb0147c4a9a6d50b79b32c63)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:33.625171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:33.671544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99031669eb0147c4a9a6d50b79b32c63', 'request_id': 'req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:33.678460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99031669eb0147c4a9a6d50b79b32c63', 'request_id': 'req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:33.684537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99031669eb0147c4a9a6d50b79b32c63', 'request_id': 'req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:33.695185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b admin admin] The token cache is being invalidate because project 99031669eb0147c4a9a6d50b79b32c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:33.695474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:33.696108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-37cb6a82-eb9f-4578-b422-35b4cfa07a1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:33.697611 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2434/4950] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:33 2022] DELETE /identity/v3/projects/99031669eb0147c4a9a6d50b79b32c63 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:33.702425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc979cd8-e5b1-4ee5-b1bd-8d221bb42ec8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:33.702742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc979cd8-e5b1-4ee5-b1bd-8d221bb42ec8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:33.703020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc979cd8-e5b1-4ee5-b1bd-8d221bb42ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:33.722513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fc979cd8-e5b1-4ee5-b1bd-8d221bb42ec8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:33.723773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fc979cd8-e5b1-4ee5-b1bd-8d221bb42ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:33.785622 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2517/4951] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:33.793740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78addf5a-d3e5-4315-8e41-2ff0ea51c991 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:33.794112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78addf5a-d3e5-4315-8e41-2ff0ea51c991 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:33.794484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78addf5a-d3e5-4315-8e41-2ff0ea51c991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:33.809539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:33.815241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-78addf5a-d3e5-4315-8e41-2ff0ea51c991 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:33.817745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-78addf5a-d3e5-4315-8e41-2ff0ea51c991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:33.826984 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2435/4952] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:07:33.831964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc979cd8-e5b1-4ee5-b1bd-8d221bb42ec8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:33.842558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc979cd8-e5b1-4ee5-b1bd-8d221bb42ec8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:33.847213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c66ba5-ac4a-49a0-ab94-baa54f3280d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:33.847830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c66ba5-ac4a-49a0-ab94-baa54f3280d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:33.848616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c66ba5-ac4a-49a0-ab94-baa54f3280d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:33.848889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c66ba5-ac4a-49a0-ab94-baa54f3280d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:33.849837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c66ba5-ac4a-49a0-ab94-baa54f3280d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:33.850143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c66ba5-ac4a-49a0-ab94-baa54f3280d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:33.851753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c66ba5-ac4a-49a0-ab94-baa54f3280d3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:33.865470 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2518/4953] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:33.870125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78addf5a-d3e5-4315-8e41-2ff0ea51c991 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:33.880273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78addf5a-d3e5-4315-8e41-2ff0ea51c991 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:33.885055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690afed-e99b-4fb0-84f8-54016a76a191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:33.885678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690afed-e99b-4fb0-84f8-54016a76a191 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:33.885962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690afed-e99b-4fb0-84f8-54016a76a191 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:33.886263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690afed-e99b-4fb0-84f8-54016a76a191 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:33.886915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690afed-e99b-4fb0-84f8-54016a76a191 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:33.888847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690afed-e99b-4fb0-84f8-54016a76a191 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:33.922658 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2436/4954] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:33 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:07:33.927974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c66ba5-ac4a-49a0-ab94-baa54f3280d3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:33.942129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c66ba5-ac4a-49a0-ab94-baa54f3280d3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:34.019208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc64618-f8f3-4b09-be7c-226a06d5fec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:34.020042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc64618-f8f3-4b09-be7c-226a06d5fec4 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:34.020336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc64618-f8f3-4b09-be7c-226a06d5fec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:34.020620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc64618-f8f3-4b09-be7c-226a06d5fec4 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:34.021329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc64618-f8f3-4b09-be7c-226a06d5fec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:34.023495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc64618-f8f3-4b09-be7c-226a06d5fec4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:34.059964 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2519/4955] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:33 2022] POST /identity/v3/users => generated 285 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:07:34.061160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690afed-e99b-4fb0-84f8-54016a76a191 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:34.079293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690afed-e99b-4fb0-84f8-54016a76a191 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:34.089390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e15238-91c2-4ad9-b4a8-acd3422bfe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:34.090081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e15238-91c2-4ad9-b4a8-acd3422bfe55 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:34.090524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e15238-91c2-4ad9-b4a8-acd3422bfe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:34.090822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e15238-91c2-4ad9-b4a8-acd3422bfe55 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:34.091477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e15238-91c2-4ad9-b4a8-acd3422bfe55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:34.093329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e15238-91c2-4ad9-b4a8-acd3422bfe55 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:34.112693 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2437/4956] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:34.115121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc64618-f8f3-4b09-be7c-226a06d5fec4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:34.127349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc64618-f8f3-4b09-be7c-226a06d5fec4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:34.135712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b9936-d95c-4328-b0de-e914492bae11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:34.136539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b9936-d95c-4328-b0de-e914492bae11 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:34.137583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b9936-d95c-4328-b0de-e914492bae11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:34.137583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b9936-d95c-4328-b0de-e914492bae11 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:34.138346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b9936-d95c-4328-b0de-e914492bae11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:34.140135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b9936-d95c-4328-b0de-e914492bae11 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:34.154598 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2520/4957] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:07:34.158915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e15238-91c2-4ad9-b4a8-acd3422bfe55 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:34.172961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e15238-91c2-4ad9-b4a8-acd3422bfe55 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:34.179898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3a2ed-27d3-4394-99d0-b899c9a7a4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:34.180165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3a2ed-27d3-4394-99d0-b899c9a7a4d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:34.180426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3a2ed-27d3-4394-99d0-b899c9a7a4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:34.180708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3a2ed-27d3-4394-99d0-b899c9a7a4d6 admin admin] PATH_INFO: `/v3/projects/894b66bfd5794f9fb143fc399ea3fb7c/users/a5557b5764674ab6b1b97916bad075d0/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:34.202015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3a2ed-27d3-4394-99d0-b899c9a7a4d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=894b66bfd5794f9fb143fc399ea3fb7c, user_id=a5557b5764674ab6b1b97916bad075d0, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:34.205183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3a2ed-27d3-4394-99d0-b899c9a7a4d6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:34.220060 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2438/4958] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:34 2022] PUT /identity/v3/projects/894b66bfd5794f9fb143fc399ea3fb7c/users/a5557b5764674ab6b1b97916bad075d0/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:34.231045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f673b44-9d22-4694-9123-1a8d3787ebe1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:34.231342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f673b44-9d22-4694-9123-1a8d3787ebe1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:34.232275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f673b44-9d22-4694-9123-1a8d3787ebe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:34.262197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3f673b44-9d22-4694-9123-1a8d3787ebe1 None None] MFA Rules not processed for user `a5557b5764674ab6b1b97916bad075d0`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:34.263508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3f673b44-9d22-4694-9123-1a8d3787ebe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:34.291111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:34.360055 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2521/4959] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:34.364684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3a2ed-27d3-4394-99d0-b899c9a7a4d6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:34.449210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3a2ed-27d3-4394-99d0-b899c9a7a4d6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:34.458695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68f1f3-f691-4e5b-98f2-8a3584a53bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:34.459906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68f1f3-f691-4e5b-98f2-8a3584a53bcb admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:34.460449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68f1f3-f691-4e5b-98f2-8a3584a53bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:34.460917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68f1f3-f691-4e5b-98f2-8a3584a53bcb admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:34.461696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68f1f3-f691-4e5b-98f2-8a3584a53bcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:34.463867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68f1f3-f691-4e5b-98f2-8a3584a53bcb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:34.495737 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2439/4960] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:34 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:34.499915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f673b44-9d22-4694-9123-1a8d3787ebe1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:34.514095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f673b44-9d22-4694-9123-1a8d3787ebe1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:34.597277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22952455-5b51-49a8-a941-8567de838aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:34.598568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22952455-5b51-49a8-a941-8567de838aaa admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:34.599354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22952455-5b51-49a8-a941-8567de838aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:34.599851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22952455-5b51-49a8-a941-8567de838aaa admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:34.600546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22952455-5b51-49a8-a941-8567de838aaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:34.602920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22952455-5b51-49a8-a941-8567de838aaa admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:34.646190 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2522/4961] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:34 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:34.647711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68f1f3-f691-4e5b-98f2-8a3584a53bcb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:34.661745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68f1f3-f691-4e5b-98f2-8a3584a53bcb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:34.667674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35972a-63ad-4e5c-b039-250abf953f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:34.669320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35972a-63ad-4e5c-b039-250abf953f0d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:34.669768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35972a-63ad-4e5c-b039-250abf953f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:34.670200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35972a-63ad-4e5c-b039-250abf953f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:34.671011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35972a-63ad-4e5c-b039-250abf953f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:34.672819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35972a-63ad-4e5c-b039-250abf953f0d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:34.694673 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2440/4962] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:34.694879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22952455-5b51-49a8-a941-8567de838aaa admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:34.705333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22952455-5b51-49a8-a941-8567de838aaa admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:34.710862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449bdd7-ef26-41a0-9b24-9779aa9263b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:34.711531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449bdd7-ef26-41a0-9b24-9779aa9263b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:34.711866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449bdd7-ef26-41a0-9b24-9779aa9263b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:34.712821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449bdd7-ef26-41a0-9b24-9779aa9263b4 admin admin] PATH_INFO: `/v3/projects/832335d127044b1c8c3d431846dbedaa/users/9f7b5005a78b4feba479e48860c92750/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:34.730681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449bdd7-ef26-41a0-9b24-9779aa9263b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=832335d127044b1c8c3d431846dbedaa, user_id=9f7b5005a78b4feba479e48860c92750, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:34.733461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449bdd7-ef26-41a0-9b24-9779aa9263b4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:34.762105 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2523/4963] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:34 2022] PUT /identity/v3/projects/832335d127044b1c8c3d431846dbedaa/users/9f7b5005a78b4feba479e48860c92750/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:07:34.762309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35972a-63ad-4e5c-b039-250abf953f0d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:34.780281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35972a-63ad-4e5c-b039-250abf953f0d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:34.866071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bce7d0-6560-4174-b49b-612f1d985af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:34.867272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bce7d0-6560-4174-b49b-612f1d985af7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:34.867571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bce7d0-6560-4174-b49b-612f1d985af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:34.867853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bce7d0-6560-4174-b49b-612f1d985af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:34.868505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bce7d0-6560-4174-b49b-612f1d985af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:34.870299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bce7d0-6560-4174-b49b-612f1d985af7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:34.886631 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2441/4964] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:34.891371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449bdd7-ef26-41a0-9b24-9779aa9263b4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:34.902522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449bdd7-ef26-41a0-9b24-9779aa9263b4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:34.923811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63ae37-95bc-4b67-af4c-71842fd35d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:34.924514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63ae37-95bc-4b67-af4c-71842fd35d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:34.924962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63ae37-95bc-4b67-af4c-71842fd35d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:34.925622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63ae37-95bc-4b67-af4c-71842fd35d91 admin admin] PATH_INFO: `/v3/projects/832335d127044b1c8c3d431846dbedaa/users/9f7b5005a78b4feba479e48860c92750/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:34.927723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63ae37-95bc-4b67-af4c-71842fd35d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=832335d127044b1c8c3d431846dbedaa, user_id=9f7b5005a78b4feba479e48860c92750, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:34.930405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63ae37-95bc-4b67-af4c-71842fd35d91 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:34.953737 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2524/4965] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:34 2022] PUT /identity/v3/projects/832335d127044b1c8c3d431846dbedaa/users/9f7b5005a78b4feba479e48860c92750/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:34.960670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a6196-7f59-45fe-b166-869a953aa393 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:34.960989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a6196-7f59-45fe-b166-869a953aa393 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:34.961276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a6196-7f59-45fe-b166-869a953aa393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:34.993604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:35.006725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bd6a6196-7f59-45fe-b166-869a953aa393 None None] MFA Rules not processed for user `9f7b5005a78b4feba479e48860c92750`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:35.008393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bd6a6196-7f59-45fe-b166-869a953aa393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:35.087018 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2442/4966] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:35.098345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63ae37-95bc-4b67-af4c-71842fd35d91 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:35.100361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ea63ae37-95bc-4b67-af4c-71842fd35d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:35.173827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63ae37-95bc-4b67-af4c-71842fd35d91 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:35.178436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e93bf-35d9-4af0-a476-ae99c8708583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:35.178999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e93bf-35d9-4af0-a476-ae99c8708583 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:35.179308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e93bf-35d9-4af0-a476-ae99c8708583 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:35.179585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e93bf-35d9-4af0-a476-ae99c8708583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:35.190143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e93bf-35d9-4af0-a476-ae99c8708583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:35.191677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e93bf-35d9-4af0-a476-ae99c8708583 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:35.202865 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2525/4967] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:40.732190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a6196-7f59-45fe-b166-869a953aa393 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:40.734216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bd6a6196-7f59-45fe-b166-869a953aa393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:40.851184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a6196-7f59-45fe-b166-869a953aa393 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:40.856060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775bcca-3800-402a-910f-a61f6e3d2522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:40.856858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775bcca-3800-402a-910f-a61f6e3d2522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:40.857345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775bcca-3800-402a-910f-a61f6e3d2522 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:40.857644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775bcca-3800-402a-910f-a61f6e3d2522 admin admin] PATH_INFO: `/v3/users/5eff5aacf85a4fda97f6eb375da7cf26` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:40.858994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775bcca-3800-402a-910f-a61f6e3d2522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eff5aacf85a4fda97f6eb375da7cf26)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:40.861257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775bcca-3800-402a-910f-a61f6e3d2522 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:40.923052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0775bcca-3800-402a-910f-a61f6e3d2522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eff5aacf85a4fda97f6eb375da7cf26'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:40.938000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0775bcca-3800-402a-910f-a61f6e3d2522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eff5aacf85a4fda97f6eb375da7cf26'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:40.938987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0775bcca-3800-402a-910f-a61f6e3d2522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eff5aacf85a4fda97f6eb375da7cf26'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:40.947584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0775bcca-3800-402a-910f-a61f6e3d2522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eff5aacf85a4fda97f6eb375da7cf26'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:40.955323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0775bcca-3800-402a-910f-a61f6e3d2522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eff5aacf85a4fda97f6eb375da7cf26'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:40.962071 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2443/4968] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:40 2022] DELETE /identity/v3/users/5eff5aacf85a4fda97f6eb375da7cf26 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:41.195016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e93bf-35d9-4af0-a476-ae99c8708583 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:41.206182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e93bf-35d9-4af0-a476-ae99c8708583 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:41.225051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:41.270305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd5cd6-c7a5-4017-9112-117496c0619a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:41.271086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd5cd6-c7a5-4017-9112-117496c0619a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:41.271395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd5cd6-c7a5-4017-9112-117496c0619a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:41.271725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd5cd6-c7a5-4017-9112-117496c0619a admin admin] PATH_INFO: `/v3/projects/ae91188b2c8b43d4bc6114581be12847` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:41.273253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd5cd6-c7a5-4017-9112-117496c0619a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae91188b2c8b43d4bc6114581be12847)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:41.275220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd5cd6-c7a5-4017-9112-117496c0619a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:41.323345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58fd5cd6-c7a5-4017-9112-117496c0619a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae91188b2c8b43d4bc6114581be12847', 'request_id': 'req-58fd5cd6-c7a5-4017-9112-117496c0619a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:41.331820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58fd5cd6-c7a5-4017-9112-117496c0619a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae91188b2c8b43d4bc6114581be12847', 'request_id': 'req-58fd5cd6-c7a5-4017-9112-117496c0619a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:41.339150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58fd5cd6-c7a5-4017-9112-117496c0619a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae91188b2c8b43d4bc6114581be12847', 'request_id': 'req-58fd5cd6-c7a5-4017-9112-117496c0619a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:41.350897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58fd5cd6-c7a5-4017-9112-117496c0619a admin admin] The token cache is being invalidate because project ae91188b2c8b43d4bc6114581be12847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:41.351189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58fd5cd6-c7a5-4017-9112-117496c0619a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:41.351851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-58fd5cd6-c7a5-4017-9112-117496c0619a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:41.353492 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2526/4969] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:41 2022] DELETE /identity/v3/projects/ae91188b2c8b43d4bc6114581be12847 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:41.358533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99bbf-28fd-49d9-8d14-d496b89a8776 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:41.358900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99bbf-28fd-49d9-8d14-d496b89a8776 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:41.359252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99bbf-28fd-49d9-8d14-d496b89a8776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:41.381302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-38b99bbf-28fd-49d9-8d14-d496b89a8776 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:41.382620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-38b99bbf-28fd-49d9-8d14-d496b89a8776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:41.396268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:41.447212 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2444/4970] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:41.455131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927af1b-d810-42af-bc0b-568568a8c94d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:41.455418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927af1b-d810-42af-bc0b-568568a8c94d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:41.455693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927af1b-d810-42af-bc0b-568568a8c94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:41.476970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7927af1b-d810-42af-bc0b-568568a8c94d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:41.478187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7927af1b-d810-42af-bc0b-568568a8c94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:41.486105 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2527/4971] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:07:41.490912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b99bbf-28fd-49d9-8d14-d496b89a8776 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:41.501431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b99bbf-28fd-49d9-8d14-d496b89a8776 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:41.506679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7009b-69d9-40aa-b8ea-6349b8ab5360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:41.507316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7009b-69d9-40aa-b8ea-6349b8ab5360 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:41.507606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7009b-69d9-40aa-b8ea-6349b8ab5360 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:41.507886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7009b-69d9-40aa-b8ea-6349b8ab5360 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:41.508533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7009b-69d9-40aa-b8ea-6349b8ab5360 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:41.508844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7009b-69d9-40aa-b8ea-6349b8ab5360 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:41.510477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7009b-69d9-40aa-b8ea-6349b8ab5360 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:41.526255 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2445/4972] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:41.527486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7927af1b-d810-42af-bc0b-568568a8c94d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:41.537427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7927af1b-d810-42af-bc0b-568568a8c94d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:41.542410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b91cb-5c13-45e8-a873-ae5995b5f68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:41.543020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b91cb-5c13-45e8-a873-ae5995b5f68c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:41.543307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b91cb-5c13-45e8-a873-ae5995b5f68c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:41.543585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b91cb-5c13-45e8-a873-ae5995b5f68c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:41.544218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b91cb-5c13-45e8-a873-ae5995b5f68c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:41.546165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b91cb-5c13-45e8-a873-ae5995b5f68c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:41.576337 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2528/4973] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:41 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:41.580663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7009b-69d9-40aa-b8ea-6349b8ab5360 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:41.594091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7009b-69d9-40aa-b8ea-6349b8ab5360 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:41.679164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0612ff-16f9-4c53-95f6-5eaf80e26914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:41.679890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0612ff-16f9-4c53-95f6-5eaf80e26914 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:41.680246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0612ff-16f9-4c53-95f6-5eaf80e26914 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:41.680596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0612ff-16f9-4c53-95f6-5eaf80e26914 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:41.681493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0612ff-16f9-4c53-95f6-5eaf80e26914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:41.683557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0612ff-16f9-4c53-95f6-5eaf80e26914 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:41.713365 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2446/4974] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:41 2022] POST /identity/v3/users => generated 282 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:41.717866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b91cb-5c13-45e8-a873-ae5995b5f68c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:41.730076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b91cb-5c13-45e8-a873-ae5995b5f68c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:41.735178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d7a2e-8b12-4cfb-9c0d-66e8225f1e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:41.736425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d7a2e-8b12-4cfb-9c0d-66e8225f1e6f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:41.736718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d7a2e-8b12-4cfb-9c0d-66e8225f1e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:41.737731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d7a2e-8b12-4cfb-9c0d-66e8225f1e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:41.738419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d7a2e-8b12-4cfb-9c0d-66e8225f1e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:41.740685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d7a2e-8b12-4cfb-9c0d-66e8225f1e6f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:41.756632 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2529/4975] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:41.758378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0612ff-16f9-4c53-95f6-5eaf80e26914 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:41.769317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0612ff-16f9-4c53-95f6-5eaf80e26914 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:41.776066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c01e7-cddb-42f2-bdd7-e1a5193f0a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:41.777299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c01e7-cddb-42f2-bdd7-e1a5193f0a8b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:41.777849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c01e7-cddb-42f2-bdd7-e1a5193f0a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:41.778361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c01e7-cddb-42f2-bdd7-e1a5193f0a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:41.779123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c01e7-cddb-42f2-bdd7-e1a5193f0a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:41.780999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c01e7-cddb-42f2-bdd7-e1a5193f0a8b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:41.794715 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2447/4976] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:41.799176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d7a2e-8b12-4cfb-9c0d-66e8225f1e6f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:41.816531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d7a2e-8b12-4cfb-9c0d-66e8225f1e6f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:41.822331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b555a0-3413-4cc6-93da-e0fecd28251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:41.823684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b555a0-3413-4cc6-93da-e0fecd28251a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:41.823684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b555a0-3413-4cc6-93da-e0fecd28251a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:41.823929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b555a0-3413-4cc6-93da-e0fecd28251a admin admin] PATH_INFO: `/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1/users/a23d411e5bdf418faf309ebf380a67e7/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:41.843060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b555a0-3413-4cc6-93da-e0fecd28251a admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c8ada6f940405c9aa2b3fecbfd34e1, user_id=a23d411e5bdf418faf309ebf380a67e7, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:41.846109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b555a0-3413-4cc6-93da-e0fecd28251a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:41.878906 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2530/4977] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:41 2022] PUT /identity/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1/users/a23d411e5bdf418faf309ebf380a67e7/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:41.880541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1bf84-c8f7-4d51-8fcd-c11f437c909b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:41.880882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1bf84-c8f7-4d51-8fcd-c11f437c909b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:41.881205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1bf84-c8f7-4d51-8fcd-c11f437c909b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:41.922951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e4c1bf84-c8f7-4d51-8fcd-c11f437c909b None None] MFA Rules not processed for user `a23d411e5bdf418faf309ebf380a67e7`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:41.924235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e4c1bf84-c8f7-4d51-8fcd-c11f437c909b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:42.090640 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2448/4978] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:07:42.104591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b555a0-3413-4cc6-93da-e0fecd28251a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:42.106332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-86b555a0-3413-4cc6-93da-e0fecd28251a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:42.163189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:42.198690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b555a0-3413-4cc6-93da-e0fecd28251a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:42.206602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065f16a-4326-4a30-8589-29744a9a5e2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:42.207269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065f16a-4326-4a30-8589-29744a9a5e2f service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:42.207574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065f16a-4326-4a30-8589-29744a9a5e2f service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:42.207840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065f16a-4326-4a30-8589-29744a9a5e2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:42.217072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065f16a-4326-4a30-8589-29744a9a5e2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:42.218694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065f16a-4326-4a30-8589-29744a9a5e2f service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:42.229220 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2531/4979] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:42.243911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcabea0-1302-44dc-a4cc-5cbd53ee0f12 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:42.244415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcabea0-1302-44dc-a4cc-5cbd53ee0f12 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:42.244852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcabea0-1302-44dc-a4cc-5cbd53ee0f12 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:42.246664 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2449/4980] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 21:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 21:07:42.252527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065f16a-4326-4a30-8589-29744a9a5e2f service nova] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 21:07:42.266699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065f16a-4326-4a30-8589-29744a9a5e2f service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:42.277307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065f16a-4326-4a30-8589-29744a9a5e2f service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:42.282171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c201e-f634-48a3-8509-19ad944a45b3 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a23d411e5bdf418faf309ebf380a67e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c8ada6f940405c9aa2b3fecbfd34e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:42.283260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c201e-f634-48a3-8509-19ad944a45b3 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:42.283452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c201e-f634-48a3-8509-19ad944a45b3 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:42.284064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c201e-f634-48a3-8509-19ad944a45b3 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] PATH_INFO: `/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:42.285993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c201e-f634-48a3-8509-19ad944a45b3 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: Authorizing `identity:get_project(project_id=76c8ada6f940405c9aa2b3fecbfd34e1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:42.287795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c201e-f634-48a3-8509-19ad944a45b3 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:42.290356 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2532/4981] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 21:07:42 2022] GET /identity/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1 => generated 363 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:42.537491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4758bb-fd8e-4da9-89d2-c4980a4be6e3 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:42.538075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4758bb-fd8e-4da9-89d2-c4980a4be6e3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:42.538519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4758bb-fd8e-4da9-89d2-c4980a4be6e3 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:42.540165 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2450/4982] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 21:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 21:07:42.547069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c201e-f634-48a3-8509-19ad944a45b3 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 21:07:42.564157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c201e-f634-48a3-8509-19ad944a45b3 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:42.573638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c201e-f634-48a3-8509-19ad944a45b3 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:42.578073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e445a-05ed-42d6-959c-88ff1fc7aa29 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a23d411e5bdf418faf309ebf380a67e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c8ada6f940405c9aa2b3fecbfd34e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:42.579455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e445a-05ed-42d6-959c-88ff1fc7aa29 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:42.579903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e445a-05ed-42d6-959c-88ff1fc7aa29 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:42.580200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e445a-05ed-42d6-959c-88ff1fc7aa29 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] PATH_INFO: `/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:42.581528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e445a-05ed-42d6-959c-88ff1fc7aa29 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: Authorizing `identity:get_project(project_id=76c8ada6f940405c9aa2b3fecbfd34e1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:42.583828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e445a-05ed-42d6-959c-88ff1fc7aa29 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:42.585742 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2533/4983] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 21:07:42 2022] GET /identity/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1 => generated 363 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:42.627844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fccc2-1407-4b89-867f-ea19845a8500 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:42.628129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fccc2-1407-4b89-867f-ea19845a8500 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:42.628817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fccc2-1407-4b89-867f-ea19845a8500 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:42.630670 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2451/4984] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 21:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 21:07:42.637031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e445a-05ed-42d6-959c-88ff1fc7aa29 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 21:07:42.655570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e445a-05ed-42d6-959c-88ff1fc7aa29 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:42.666695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e445a-05ed-42d6-959c-88ff1fc7aa29 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:42.671897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cce5a-fa13-42a9-8bb6-26434f391daa tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a23d411e5bdf418faf309ebf380a67e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c8ada6f940405c9aa2b3fecbfd34e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:42.673420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cce5a-fa13-42a9-8bb6-26434f391daa tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:42.674596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cce5a-fa13-42a9-8bb6-26434f391daa tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:42.674596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cce5a-fa13-42a9-8bb6-26434f391daa tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] PATH_INFO: `/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:42.675589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cce5a-fa13-42a9-8bb6-26434f391daa tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: Authorizing `identity:get_project(project_id=76c8ada6f940405c9aa2b3fecbfd34e1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:42.677636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cce5a-fa13-42a9-8bb6-26434f391daa tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:42.682452 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2534/4985] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 21:07:42 2022] GET /identity/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1 => generated 363 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:42.717089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47f1a4-030d-450d-aa4e-cfed0ac02a2e None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:42.717869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47f1a4-030d-450d-aa4e-cfed0ac02a2e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:42.718439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47f1a4-030d-450d-aa4e-cfed0ac02a2e None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:42.720310 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2452/4986] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 21:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 21:07:42.729971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cce5a-fa13-42a9-8bb6-26434f391daa tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 21:07:42.733583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-914cce5a-fa13-42a9-8bb6-26434f391daa tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:42.758053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cce5a-fa13-42a9-8bb6-26434f391daa tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:42.768775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cce5a-fa13-42a9-8bb6-26434f391daa tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:42.774563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4df149-ae41-4279-ad84-771d529c74c2 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a23d411e5bdf418faf309ebf380a67e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c8ada6f940405c9aa2b3fecbfd34e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:42.775213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4df149-ae41-4279-ad84-771d529c74c2 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:42.775572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4df149-ae41-4279-ad84-771d529c74c2 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:42.775788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4df149-ae41-4279-ad84-771d529c74c2 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] PATH_INFO: `/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:42.776986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4df149-ae41-4279-ad84-771d529c74c2 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: Authorizing `identity:get_project(project_id=76c8ada6f940405c9aa2b3fecbfd34e1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:42.779273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4df149-ae41-4279-ad84-771d529c74c2 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:42.781770 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2535/4987] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 21:07:42 2022] GET /identity/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1 => generated 363 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:42.835575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f2485-c4d9-42af-8ecf-3d18ac0e8dc9 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:42.836107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f2485-c4d9-42af-8ecf-3d18ac0e8dc9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:42.836373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f2485-c4d9-42af-8ecf-3d18ac0e8dc9 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:42.837993 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2453/4988] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 21:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 21:07:42.843167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4df149-ae41-4279-ad84-771d529c74c2 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 21:07:42.865181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4df149-ae41-4279-ad84-771d529c74c2 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:42.878588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4df149-ae41-4279-ad84-771d529c74c2 tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:42.884880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ec426-53af-476c-bec1-ad23bbc61edc tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a23d411e5bdf418faf309ebf380a67e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c8ada6f940405c9aa2b3fecbfd34e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:42.885587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec426-53af-476c-bec1-ad23bbc61edc tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:42.885944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec426-53af-476c-bec1-ad23bbc61edc tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:42.886159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec426-53af-476c-bec1-ad23bbc61edc tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] PATH_INFO: `/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:42.888269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ec426-53af-476c-bec1-ad23bbc61edc tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: Authorizing `identity:get_project(project_id=76c8ada6f940405c9aa2b3fecbfd34e1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:42.890747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ec426-53af-476c-bec1-ad23bbc61edc tempest-QuotasTestJSON-102623755 tempest-QuotasTestJSON-102623755-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:42.892958 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2536/4989] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 21:07:42 2022] GET /identity/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1 => generated 363 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:42.968084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f2485-c4d9-42af-8ecf-3d18ac0e8dc9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:43.007553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:43.134441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f2485-c4d9-42af-8ecf-3d18ac0e8dc9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:43.147955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:43.148756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:43.149207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:43.149575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] PATH_INFO: `/v3/users/a23d411e5bdf418faf309ebf380a67e7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:43.151100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a23d411e5bdf418faf309ebf380a67e7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:43.153266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:43.228415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a23d411e5bdf418faf309ebf380a67e7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:43.252697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a23d411e5bdf418faf309ebf380a67e7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:43.253519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a23d411e5bdf418faf309ebf380a67e7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:43.270868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a23d411e5bdf418faf309ebf380a67e7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:43.284299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a23d411e5bdf418faf309ebf380a67e7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:43.295421 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2454/4990] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:42 2022] DELETE /identity/v3/users/a23d411e5bdf418faf309ebf380a67e7 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:43.299759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a27b17-88e1-4730-a218-0cb24baed561 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:43.300232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a27b17-88e1-4730-a218-0cb24baed561 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:43.300312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a27b17-88e1-4730-a218-0cb24baed561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:43.321289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-44a27b17-88e1-4730-a218-0cb24baed561 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:43.322766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-44a27b17-88e1-4730-a218-0cb24baed561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:43.416166 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2537/4991] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:43.459544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:43.461136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:43.604358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c3416-02e2-4b62-8e2e-292d751f9918 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:43.609076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1ae79-773f-498a-a444-2f79c4852d03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:43.609889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1ae79-773f-498a-a444-2f79c4852d03 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:43.610272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1ae79-773f-498a-a444-2f79c4852d03 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:43.610616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1ae79-773f-498a-a444-2f79c4852d03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:43.621325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1ae79-773f-498a-a444-2f79c4852d03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:43.622863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1ae79-773f-498a-a444-2f79c4852d03 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:43.636103 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2455/4992] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:44.191177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a27b17-88e1-4730-a218-0cb24baed561 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:44.193304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-44a27b17-88e1-4730-a218-0cb24baed561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:44.226285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a27b17-88e1-4730-a218-0cb24baed561 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:44.232504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:44.233341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:44.233697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:44.234811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] PATH_INFO: `/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:44.236389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c8ada6f940405c9aa2b3fecbfd34e1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:44.238941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:44.306991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c8ada6f940405c9aa2b3fecbfd34e1', 'request_id': 'req-f8b836cb-c760-4285-92b2-df734fdaaf92'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:44.319076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c8ada6f940405c9aa2b3fecbfd34e1', 'request_id': 'req-f8b836cb-c760-4285-92b2-df734fdaaf92'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:44.330100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c8ada6f940405c9aa2b3fecbfd34e1', 'request_id': 'req-f8b836cb-c760-4285-92b2-df734fdaaf92'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:44.344663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] The token cache is being invalidate because project 76c8ada6f940405c9aa2b3fecbfd34e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:44.345084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:44.347075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:44.349607 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2538/4993] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:44 2022] DELETE /identity/v3/projects/76c8ada6f940405c9aa2b3fecbfd34e1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:44.359730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328b376-f245-4eee-aaac-9b5c900e41e4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:44.360256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328b376-f245-4eee-aaac-9b5c900e41e4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:44.361095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328b376-f245-4eee-aaac-9b5c900e41e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:44.414438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2328b376-f245-4eee-aaac-9b5c900e41e4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:44.416886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2328b376-f245-4eee-aaac-9b5c900e41e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:44.482880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:44.505570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:44.507097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:44.530710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:44.566175 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2456/4994] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:44.574535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaab604-8f40-4ee9-929b-947d5f4ab901 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:44.574925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaab604-8f40-4ee9-929b-947d5f4ab901 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:44.575118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaab604-8f40-4ee9-929b-947d5f4ab901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:44.645875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bcaab604-8f40-4ee9-929b-947d5f4ab901 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:44.647156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bcaab604-8f40-4ee9-929b-947d5f4ab901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:44.680154 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2457/4995] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:07:44.680847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaab604-8f40-4ee9-929b-947d5f4ab901 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:44.683512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b836cb-c760-4285-92b2-df734fdaaf92 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:44.685570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bcaab604-8f40-4ee9-929b-947d5f4ab901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:44.694003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f22cff-b009-4853-baf5-9df3368c74cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:44.694949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f22cff-b009-4853-baf5-9df3368c74cd service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:44.695302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f22cff-b009-4853-baf5-9df3368c74cd service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:44.695677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f22cff-b009-4853-baf5-9df3368c74cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:44.701490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a9f22cff-b009-4853-baf5-9df3368c74cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:44.741651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f22cff-b009-4853-baf5-9df3368c74cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:44.746088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f22cff-b009-4853-baf5-9df3368c74cd service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:44.778681 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2539/4996] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:07:44.779888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f22cff-b009-4853-baf5-9df3368c74cd service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:44.798906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f22cff-b009-4853-baf5-9df3368c74cd service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:44.805129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbaae2-c7d1-4875-ac39-99d7ceff70a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:44.806144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbaae2-c7d1-4875-ac39-99d7ceff70a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:44.807018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbaae2-c7d1-4875-ac39-99d7ceff70a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:44.807503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbaae2-c7d1-4875-ac39-99d7ceff70a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:44.808547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbaae2-c7d1-4875-ac39-99d7ceff70a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:44.809112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbaae2-c7d1-4875-ac39-99d7ceff70a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:44.814992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbaae2-c7d1-4875-ac39-99d7ceff70a6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:44.836245 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2540/4997] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:44.841643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbaae2-c7d1-4875-ac39-99d7ceff70a6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:44.855151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbaae2-c7d1-4875-ac39-99d7ceff70a6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:44.860342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ef398-2faa-4ec7-806f-c6b36fadbd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:44.860593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaab604-8f40-4ee9-929b-947d5f4ab901 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:44.861110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ef398-2faa-4ec7-806f-c6b36fadbd10 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:44.861481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ef398-2faa-4ec7-806f-c6b36fadbd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:44.868132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681b84a-215b-4cea-ade2-2d4c293da2c3 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f7b5005a78b4feba479e48860c92750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '832335d127044b1c8c3d431846dbedaa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:44.868806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681b84a-215b-4cea-ade2-2d4c293da2c3 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:44.869148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681b84a-215b-4cea-ade2-2d4c293da2c3 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:44.869430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681b84a-215b-4cea-ade2-2d4c293da2c3 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:44.870415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681b84a-215b-4cea-ade2-2d4c293da2c3 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:44.870743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681b84a-215b-4cea-ade2-2d4c293da2c3 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:44.871629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ef398-2faa-4ec7-806f-c6b36fadbd10 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:44.873048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681b84a-215b-4cea-ade2-2d4c293da2c3 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:44.873175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ef398-2faa-4ec7-806f-c6b36fadbd10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:44.888017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ef398-2faa-4ec7-806f-c6b36fadbd10 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:44.893439 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2458/4998] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:44.898288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681b84a-215b-4cea-ade2-2d4c293da2c3 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:44.932413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681b84a-215b-4cea-ade2-2d4c293da2c3 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:44.949528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd739ed-4d16-4ef6-95c1-fc30986726a8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f7b5005a78b4feba479e48860c92750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '832335d127044b1c8c3d431846dbedaa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:44.950111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd739ed-4d16-4ef6-95c1-fc30986726a8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:44.952955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd739ed-4d16-4ef6-95c1-fc30986726a8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:44.952955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd739ed-4d16-4ef6-95c1-fc30986726a8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:44.952955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd739ed-4d16-4ef6-95c1-fc30986726a8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:44.957951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd739ed-4d16-4ef6-95c1-fc30986726a8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:44.976639 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2541/4999] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:44 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:44.980341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ef398-2faa-4ec7-806f-c6b36fadbd10 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:45.032863 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2459/5000] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:44 2022] POST /identity/v3/projects => generated 349 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:45.036452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd739ed-4d16-4ef6-95c1-fc30986726a8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:45.156541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ef398-2faa-4ec7-806f-c6b36fadbd10 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:45.247735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd739ed-4d16-4ef6-95c1-fc30986726a8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:45.263034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e40a2-ecdf-4d7d-8aea-27e51cffc771 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f7b5005a78b4feba479e48860c92750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '832335d127044b1c8c3d431846dbedaa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:45.263681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e40a2-ecdf-4d7d-8aea-27e51cffc771 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:45.263975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e40a2-ecdf-4d7d-8aea-27e51cffc771 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:45.264262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e40a2-ecdf-4d7d-8aea-27e51cffc771 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:45.264941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e40a2-ecdf-4d7d-8aea-27e51cffc771 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:45.265263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e40a2-ecdf-4d7d-8aea-27e51cffc771 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:45.285688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e40a2-ecdf-4d7d-8aea-27e51cffc771 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:45.295158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d0bf-c525-4b08-9244-bf70c824ea8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:45.295496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d0bf-c525-4b08-9244-bf70c824ea8d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:45.295781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d0bf-c525-4b08-9244-bf70c824ea8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:45.296057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d0bf-c525-4b08-9244-bf70c824ea8d admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:45.296692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27d0bf-c525-4b08-9244-bf70c824ea8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:45.298692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27d0bf-c525-4b08-9244-bf70c824ea8d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:45.314421 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2460/5001] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:45.333030 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2542/5002] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:44 2022] POST /identity/v3/users => generated 291 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:07:45.338066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e40a2-ecdf-4d7d-8aea-27e51cffc771 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:45.354996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e40a2-ecdf-4d7d-8aea-27e51cffc771 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:45.361668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9e475-987e-4e3c-985a-f99f6a41dedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:45.362519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9e475-987e-4e3c-985a-f99f6a41dedc admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:45.363188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9e475-987e-4e3c-985a-f99f6a41dedc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:45.363787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9e475-987e-4e3c-985a-f99f6a41dedc admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:45.364835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9e475-987e-4e3c-985a-f99f6a41dedc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:45.367087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9e475-987e-4e3c-985a-f99f6a41dedc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:45.380890 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2461/5003] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:45.384629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d0bf-c525-4b08-9244-bf70c824ea8d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:45.391036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:45.400391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d0bf-c525-4b08-9244-bf70c824ea8d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:45.408069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b46a7-0dd8-4a03-86ff-caec42ab374d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:45.408996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b46a7-0dd8-4a03-86ff-caec42ab374d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:45.409541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b46a7-0dd8-4a03-86ff-caec42ab374d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:45.409886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b46a7-0dd8-4a03-86ff-caec42ab374d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:45.410675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b46a7-0dd8-4a03-86ff-caec42ab374d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:45.412826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b46a7-0dd8-4a03-86ff-caec42ab374d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:45.429413 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2543/5004] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:07:45.433174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9e475-987e-4e3c-985a-f99f6a41dedc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:45.454825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9e475-987e-4e3c-985a-f99f6a41dedc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:45.461240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695dc3d-0a8b-4575-986e-f2bb0f66ced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:45.461924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695dc3d-0a8b-4575-986e-f2bb0f66ced5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:45.462212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695dc3d-0a8b-4575-986e-f2bb0f66ced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:45.462786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695dc3d-0a8b-4575-986e-f2bb0f66ced5 admin admin] PATH_INFO: `/v3/projects/15985a3823b5409c8a18e31048f82ff7/users/3e79edf1a33e45ddac92ba8cd8429d9b/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:45.482498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695dc3d-0a8b-4575-986e-f2bb0f66ced5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15985a3823b5409c8a18e31048f82ff7, user_id=3e79edf1a33e45ddac92ba8cd8429d9b, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:45.485559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695dc3d-0a8b-4575-986e-f2bb0f66ced5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:45.498075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:45.508666 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2462/5005] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:45 2022] PUT /identity/v3/projects/15985a3823b5409c8a18e31048f82ff7/users/3e79edf1a33e45ddac92ba8cd8429d9b/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:45.514934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab3a68-8ece-4f5e-b5de-825a8df0dc21 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:45.515231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab3a68-8ece-4f5e-b5de-825a8df0dc21 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:45.515512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab3a68-8ece-4f5e-b5de-825a8df0dc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:45.537595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6dab3a68-8ece-4f5e-b5de-825a8df0dc21 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:45.538842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6dab3a68-8ece-4f5e-b5de-825a8df0dc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:45.608567 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2544/5006] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:45.618703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695dc3d-0a8b-4575-986e-f2bb0f66ced5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:45.620530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0695dc3d-0a8b-4575-986e-f2bb0f66ced5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:45.644144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab3a68-8ece-4f5e-b5de-825a8df0dc21 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:45.645859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6dab3a68-8ece-4f5e-b5de-825a8df0dc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:45.685241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695dc3d-0a8b-4575-986e-f2bb0f66ced5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:45.688956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbb031-5c74-4280-9d3d-8208e9d2dd65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:45.689695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbb031-5c74-4280-9d3d-8208e9d2dd65 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:45.689983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbb031-5c74-4280-9d3d-8208e9d2dd65 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:45.690256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbb031-5c74-4280-9d3d-8208e9d2dd65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:45.700468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbb031-5c74-4280-9d3d-8208e9d2dd65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:45.701932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbb031-5c74-4280-9d3d-8208e9d2dd65 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:45.711735 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2463/5007] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:45.749138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab3a68-8ece-4f5e-b5de-825a8df0dc21 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:45.755853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897695f4-bb50-40c5-8939-19ee5d0eb14a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:45.756636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897695f4-bb50-40c5-8939-19ee5d0eb14a service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:45.757027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897695f4-bb50-40c5-8939-19ee5d0eb14a service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:45.757409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897695f4-bb50-40c5-8939-19ee5d0eb14a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:45.759625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-897695f4-bb50-40c5-8939-19ee5d0eb14a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:45.768931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbb031-5c74-4280-9d3d-8208e9d2dd65 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:45.856699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897695f4-bb50-40c5-8939-19ee5d0eb14a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:45.858380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897695f4-bb50-40c5-8939-19ee5d0eb14a service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:45.862853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbb031-5c74-4280-9d3d-8208e9d2dd65 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:45.868254 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2545/5008] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:45 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:07:45.868823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812689f-642a-48d7-8416-e4c1d7bc1cc8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f7b5005a78b4feba479e48860c92750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '832335d127044b1c8c3d431846dbedaa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:45.869870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812689f-642a-48d7-8416-e4c1d7bc1cc8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:45.870461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812689f-642a-48d7-8416-e4c1d7bc1cc8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:45.870967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812689f-642a-48d7-8416-e4c1d7bc1cc8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] PATH_INFO: `/v3/projects/d980cc32e7954c34819da9136706dafd` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:45.872442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812689f-642a-48d7-8416-e4c1d7bc1cc8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d980cc32e7954c34819da9136706dafd)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:45.875635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812689f-642a-48d7-8416-e4c1d7bc1cc8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:45.946620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1812689f-642a-48d7-8416-e4c1d7bc1cc8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd980cc32e7954c34819da9136706dafd', 'request_id': 'req-1812689f-642a-48d7-8416-e4c1d7bc1cc8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:45.958607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1812689f-642a-48d7-8416-e4c1d7bc1cc8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd980cc32e7954c34819da9136706dafd', 'request_id': 'req-1812689f-642a-48d7-8416-e4c1d7bc1cc8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:45.965584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1812689f-642a-48d7-8416-e4c1d7bc1cc8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd980cc32e7954c34819da9136706dafd', 'request_id': 'req-1812689f-642a-48d7-8416-e4c1d7bc1cc8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:45.978021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1812689f-642a-48d7-8416-e4c1d7bc1cc8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] The token cache is being invalidate because project d980cc32e7954c34819da9136706dafd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:45.978383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1812689f-642a-48d7-8416-e4c1d7bc1cc8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:45.979198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1812689f-642a-48d7-8416-e4c1d7bc1cc8 tempest-RoutersAdminTest-421019420 tempest-RoutersAdminTest-421019420-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:45.980733 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2464/5009] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:45 2022] DELETE /identity/v3/projects/d980cc32e7954c34819da9136706dafd => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:45.992788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897695f4-bb50-40c5-8939-19ee5d0eb14a service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:45.994268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-897695f4-bb50-40c5-8939-19ee5d0eb14a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:46.011149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897695f4-bb50-40c5-8939-19ee5d0eb14a service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:46.047659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:46.071963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1860525-7239-4249-8d39-9e8199dba830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:46.072575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1860525-7239-4249-8d39-9e8199dba830 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:46.072857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1860525-7239-4249-8d39-9e8199dba830 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:46.073167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1860525-7239-4249-8d39-9e8199dba830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:46.074940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f1860525-7239-4249-8d39-9e8199dba830 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:46.152417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1860525-7239-4249-8d39-9e8199dba830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:46.153982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1860525-7239-4249-8d39-9e8199dba830 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:46.165471 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2546/5010] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:49.066045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad374b-301e-4e1c-b836-af8f4b020b6b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:49.067077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad374b-301e-4e1c-b836-af8f4b020b6b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:49.067581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad374b-301e-4e1c-b836-af8f4b020b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:49.091905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-54ad374b-301e-4e1c-b836-af8f4b020b6b None None] MFA Rules not processed for user `9e686d314d164930bbcfc9905a8b45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:49.093259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-54ad374b-301e-4e1c-b836-af8f4b020b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:49.159041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:49.229735 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2465/5011] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:51.080948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1860525-7239-4249-8d39-9e8199dba830 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:51.082461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f1860525-7239-4249-8d39-9e8199dba830 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:51.285572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1860525-7239-4249-8d39-9e8199dba830 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:51.315904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:51.316589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:51.316885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:51.319081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] PATH_INFO: `/v3/users/fb144426dd4a46caa03a9ebe77e95c4c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:51.324412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb144426dd4a46caa03a9ebe77e95c4c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:51.328014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:51.450679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb144426dd4a46caa03a9ebe77e95c4c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:51.481287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb144426dd4a46caa03a9ebe77e95c4c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:51.505662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb144426dd4a46caa03a9ebe77e95c4c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:51.506237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb144426dd4a46caa03a9ebe77e95c4c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:51.507018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb144426dd4a46caa03a9ebe77e95c4c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:51.542570 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2547/5012] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:51 2022] DELETE /identity/v3/users/fb144426dd4a46caa03a9ebe77e95c4c => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:52.124648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485c2fc-0bac-418a-a3cf-af33ea534967 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:52.125481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485c2fc-0bac-418a-a3cf-af33ea534967 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:52.125481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485c2fc-0bac-418a-a3cf-af33ea534967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:52.156809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b485c2fc-0bac-418a-a3cf-af33ea534967 None None] MFA Rules not processed for user `d485394bc49e4380b152d32d09440281`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:52.158497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b485c2fc-0bac-418a-a3cf-af33ea534967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:52.225746 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2466/5013] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:52.916454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:52.918466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:52.928027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:53.059517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e200bd3-2358-464b-8964-aa7a0208cde4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:53.077968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc809a-2bf3-4c26-95d4-bc36b27220de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:53.079662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc809a-2bf3-4c26-95d4-bc36b27220de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:53.080140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc809a-2bf3-4c26-95d4-bc36b27220de admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:53.080577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc809a-2bf3-4c26-95d4-bc36b27220de admin admin] PATH_INFO: `/v3/projects/79959bbc9a594e1f81d7e60e5d175e2e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:53.083298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc809a-2bf3-4c26-95d4-bc36b27220de admin admin] RBAC: Authorizing `identity:delete_project(project_id=79959bbc9a594e1f81d7e60e5d175e2e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:53.088877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc809a-2bf3-4c26-95d4-bc36b27220de admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:53.214391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87bc809a-2bf3-4c26-95d4-bc36b27220de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79959bbc9a594e1f81d7e60e5d175e2e', 'request_id': 'req-87bc809a-2bf3-4c26-95d4-bc36b27220de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:53.226038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87bc809a-2bf3-4c26-95d4-bc36b27220de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79959bbc9a594e1f81d7e60e5d175e2e', 'request_id': 'req-87bc809a-2bf3-4c26-95d4-bc36b27220de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:53.237286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87bc809a-2bf3-4c26-95d4-bc36b27220de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79959bbc9a594e1f81d7e60e5d175e2e', 'request_id': 'req-87bc809a-2bf3-4c26-95d4-bc36b27220de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:53.256533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87bc809a-2bf3-4c26-95d4-bc36b27220de admin admin] The token cache is being invalidate because project 79959bbc9a594e1f81d7e60e5d175e2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:53.257031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87bc809a-2bf3-4c26-95d4-bc36b27220de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:53.258185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87bc809a-2bf3-4c26-95d4-bc36b27220de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:53.260721 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2548/5014] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:52 2022] DELETE /identity/v3/projects/79959bbc9a594e1f81d7e60e5d175e2e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:53.266047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f8485-f94d-4ea1-aa53-7a7a1380dbc7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:53.266612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f8485-f94d-4ea1-aa53-7a7a1380dbc7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:53.266612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f8485-f94d-4ea1-aa53-7a7a1380dbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:53.295642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a98f8485-f94d-4ea1-aa53-7a7a1380dbc7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:53.297064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a98f8485-f94d-4ea1-aa53-7a7a1380dbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:53.397997 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2467/5015] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:53.406883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4e57f-6bd8-4175-afe7-9405f064a056 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:53.407290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4e57f-6bd8-4175-afe7-9405f064a056 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:53.407829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4e57f-6bd8-4175-afe7-9405f064a056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:53.451485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7bb4e57f-6bd8-4175-afe7-9405f064a056 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:53.453099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7bb4e57f-6bd8-4175-afe7-9405f064a056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:53.470926 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2549/5016] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:53.475504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f8485-f94d-4ea1-aa53-7a7a1380dbc7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:53.496407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f8485-f94d-4ea1-aa53-7a7a1380dbc7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:53.509095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17e6df-c4cc-441f-bb48-66de58dbf767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:53.509779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17e6df-c4cc-441f-bb48-66de58dbf767 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:53.510313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17e6df-c4cc-441f-bb48-66de58dbf767 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:53.510773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17e6df-c4cc-441f-bb48-66de58dbf767 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:53.511845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17e6df-c4cc-441f-bb48-66de58dbf767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:53.512436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17e6df-c4cc-441f-bb48-66de58dbf767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:53.514267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17e6df-c4cc-441f-bb48-66de58dbf767 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:53.537445 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2468/5017] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:53.542846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4e57f-6bd8-4175-afe7-9405f064a056 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:53.584103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4e57f-6bd8-4175-afe7-9405f064a056 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:53.593497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f421d-7488-484c-a0ae-2ff96ac454fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:53.594268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f421d-7488-484c-a0ae-2ff96ac454fc admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:53.595409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f421d-7488-484c-a0ae-2ff96ac454fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:53.595409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f421d-7488-484c-a0ae-2ff96ac454fc admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:53.595689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f421d-7488-484c-a0ae-2ff96ac454fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:53.598616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f421d-7488-484c-a0ae-2ff96ac454fc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:53.654261 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2550/5018] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:53 2022] POST /identity/v3/projects => generated 361 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:53.654894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17e6df-c4cc-441f-bb48-66de58dbf767 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:53.674845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17e6df-c4cc-441f-bb48-66de58dbf767 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:53.814654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903a000-019a-42e6-abfa-8fc8c3a351d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:53.815756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903a000-019a-42e6-abfa-8fc8c3a351d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:53.816048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903a000-019a-42e6-abfa-8fc8c3a351d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:53.816869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903a000-019a-42e6-abfa-8fc8c3a351d1 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:53.818029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903a000-019a-42e6-abfa-8fc8c3a351d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:53.822289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903a000-019a-42e6-abfa-8fc8c3a351d1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:53.862208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:53.874733 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2469/5019] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:53 2022] POST /identity/v3/users => generated 281 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:53.876370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f421d-7488-484c-a0ae-2ff96ac454fc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:53.894507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f421d-7488-484c-a0ae-2ff96ac454fc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:53.906159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62681f79-2cef-427b-b1e9-ae8bd6337a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:53.907199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62681f79-2cef-427b-b1e9-ae8bd6337a04 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:53.907919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62681f79-2cef-427b-b1e9-ae8bd6337a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:53.908503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62681f79-2cef-427b-b1e9-ae8bd6337a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:53.909417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62681f79-2cef-427b-b1e9-ae8bd6337a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:53.912151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62681f79-2cef-427b-b1e9-ae8bd6337a04 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:53.929243 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2551/5020] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:07:53.931477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903a000-019a-42e6-abfa-8fc8c3a351d1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:53.958384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903a000-019a-42e6-abfa-8fc8c3a351d1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:53.966490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d03b50-9077-4230-99cf-fbf0a5891033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:53.967188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d03b50-9077-4230-99cf-fbf0a5891033 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:53.967485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d03b50-9077-4230-99cf-fbf0a5891033 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:53.967767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d03b50-9077-4230-99cf-fbf0a5891033 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:53.968428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d03b50-9077-4230-99cf-fbf0a5891033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:53.970917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d03b50-9077-4230-99cf-fbf0a5891033 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:53.989086 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2470/5021] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:53.990295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62681f79-2cef-427b-b1e9-ae8bd6337a04 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:54.007768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62681f79-2cef-427b-b1e9-ae8bd6337a04 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:54.015816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc555e16-b81f-4eb1-bc9f-649e2e2d602f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:54.016765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc555e16-b81f-4eb1-bc9f-649e2e2d602f admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:54.017089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc555e16-b81f-4eb1-bc9f-649e2e2d602f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:54.017443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc555e16-b81f-4eb1-bc9f-649e2e2d602f admin admin] PATH_INFO: `/v3/projects/0174d2dc113d4022852389c99cf89a72/users/851b4f190c8948bd8d31b577d18e074a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:54.038546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc555e16-b81f-4eb1-bc9f-649e2e2d602f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0174d2dc113d4022852389c99cf89a72, user_id=851b4f190c8948bd8d31b577d18e074a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:54.044914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc555e16-b81f-4eb1-bc9f-649e2e2d602f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:54.081665 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2552/5022] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:53 2022] PUT /identity/v3/projects/0174d2dc113d4022852389c99cf89a72/users/851b4f190c8948bd8d31b577d18e074a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:54.084130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c7f34-dc72-49ae-9193-5716d15414ac None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:54.084429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c7f34-dc72-49ae-9193-5716d15414ac None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:54.084736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c7f34-dc72-49ae-9193-5716d15414ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:54.154562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a53c7f34-dc72-49ae-9193-5716d15414ac None None] MFA Rules not processed for user `851b4f190c8948bd8d31b577d18e074a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:54.158340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a53c7f34-dc72-49ae-9193-5716d15414ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:54.303741 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2471/5023] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:54.350542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc555e16-b81f-4eb1-bc9f-649e2e2d602f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:54.405741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:54.458085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc555e16-b81f-4eb1-bc9f-649e2e2d602f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:54.478054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:54.479170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:54.480878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:54.481404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] PATH_INFO: `/v3/users/851b4f190c8948bd8d31b577d18e074a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:54.485421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=851b4f190c8948bd8d31b577d18e074a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:54.490082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:54.585920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '851b4f190c8948bd8d31b577d18e074a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:54.620863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '851b4f190c8948bd8d31b577d18e074a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:54.643945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '851b4f190c8948bd8d31b577d18e074a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:54.646711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '851b4f190c8948bd8d31b577d18e074a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:54.647375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '851b4f190c8948bd8d31b577d18e074a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:54.668360 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2553/5024] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:07:54 2022] DELETE /identity/v3/users/851b4f190c8948bd8d31b577d18e074a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:54.672810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31548a2-cf4e-4c88-b2eb-21e433ad66bf None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:54.673211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31548a2-cf4e-4c88-b2eb-21e433ad66bf None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:54.673499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31548a2-cf4e-4c88-b2eb-21e433ad66bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:54.709255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f31548a2-cf4e-4c88-b2eb-21e433ad66bf None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:54.710573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f31548a2-cf4e-4c88-b2eb-21e433ad66bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:54.825810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2472/5025] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:54.864304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:54.869167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:55.041876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85677754-7155-47fa-b916-b885bf629d06 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:55.051928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fba380-08fd-424f-92f7-62dc7b738549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:55.052884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fba380-08fd-424f-92f7-62dc7b738549 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:55.054018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fba380-08fd-424f-92f7-62dc7b738549 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:55.054621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fba380-08fd-424f-92f7-62dc7b738549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:55.069680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fba380-08fd-424f-92f7-62dc7b738549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:55.071525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fba380-08fd-424f-92f7-62dc7b738549 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:55.105780 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2554/5026] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:07:57.174007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31548a2-cf4e-4c88-b2eb-21e433ad66bf None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:57.177462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f31548a2-cf4e-4c88-b2eb-21e433ad66bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:57.208400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31548a2-cf4e-4c88-b2eb-21e433ad66bf None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:57.217424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74f00f-7c03-4bc5-bf52-302b4820c945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:57.218894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74f00f-7c03-4bc5-bf52-302b4820c945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:57.220345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74f00f-7c03-4bc5-bf52-302b4820c945 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:57.223840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74f00f-7c03-4bc5-bf52-302b4820c945 admin admin] PATH_INFO: `/v3/projects/0174d2dc113d4022852389c99cf89a72` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:57.226665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74f00f-7c03-4bc5-bf52-302b4820c945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0174d2dc113d4022852389c99cf89a72)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:57.236277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74f00f-7c03-4bc5-bf52-302b4820c945 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:57.354554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af74f00f-7c03-4bc5-bf52-302b4820c945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0174d2dc113d4022852389c99cf89a72', 'request_id': 'req-af74f00f-7c03-4bc5-bf52-302b4820c945'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:57.373566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af74f00f-7c03-4bc5-bf52-302b4820c945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0174d2dc113d4022852389c99cf89a72', 'request_id': 'req-af74f00f-7c03-4bc5-bf52-302b4820c945'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:57.382221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af74f00f-7c03-4bc5-bf52-302b4820c945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0174d2dc113d4022852389c99cf89a72', 'request_id': 'req-af74f00f-7c03-4bc5-bf52-302b4820c945'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:57.409033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af74f00f-7c03-4bc5-bf52-302b4820c945 admin admin] The token cache is being invalidate because project 0174d2dc113d4022852389c99cf89a72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:07:57.409444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af74f00f-7c03-4bc5-bf52-302b4820c945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:57.411342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af74f00f-7c03-4bc5-bf52-302b4820c945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:07:57.413606 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2473/5027] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:07:57 2022] DELETE /identity/v3/projects/0174d2dc113d4022852389c99cf89a72 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:57.418668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4496053-7d61-457e-a2b1-f4ae6dfb9efa None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:57.419350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4496053-7d61-457e-a2b1-f4ae6dfb9efa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:57.419350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4496053-7d61-457e-a2b1-f4ae6dfb9efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:57.471886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f4496053-7d61-457e-a2b1-f4ae6dfb9efa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:57.473726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f4496053-7d61-457e-a2b1-f4ae6dfb9efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:57.630604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:57.673385 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2555/5028] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:57.680498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbdbeb-9078-4b51-ace5-7ce17626e6fb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:57.681215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbdbeb-9078-4b51-ace5-7ce17626e6fb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:57.681761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbdbeb-9078-4b51-ace5-7ce17626e6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:57.695453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:07:57.727167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fabbdbeb-9078-4b51-ace5-7ce17626e6fb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:57.727376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fabbdbeb-9078-4b51-ace5-7ce17626e6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:57.747907 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2474/5029] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:57.749970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4496053-7d61-457e-a2b1-f4ae6dfb9efa None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:57.766491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4496053-7d61-457e-a2b1-f4ae6dfb9efa None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:57.790170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e3b23-4bf5-4657-a91f-369071e114a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:57.791905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e3b23-4bf5-4657-a91f-369071e114a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:57.793071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e3b23-4bf5-4657-a91f-369071e114a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:57.793843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e3b23-4bf5-4657-a91f-369071e114a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:57.794731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e3b23-4bf5-4657-a91f-369071e114a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:07:57.795247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e3b23-4bf5-4657-a91f-369071e114a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:57.797482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e3b23-4bf5-4657-a91f-369071e114a7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:57.827457 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2556/5030] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:07:57.829064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbdbeb-9078-4b51-ace5-7ce17626e6fb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:57.845554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbdbeb-9078-4b51-ace5-7ce17626e6fb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:57.859574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adae6b-b805-4ac1-90e7-c6d50706091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:57.862217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adae6b-b805-4ac1-90e7-c6d50706091e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:57.863299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adae6b-b805-4ac1-90e7-c6d50706091e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:57.863299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adae6b-b805-4ac1-90e7-c6d50706091e admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:57.865216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74adae6b-b805-4ac1-90e7-c6d50706091e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:57.872663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74adae6b-b805-4ac1-90e7-c6d50706091e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:57.939610 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2475/5031] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:07:57 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:57.944970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e3b23-4bf5-4657-a91f-369071e114a7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:57.959610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e3b23-4bf5-4657-a91f-369071e114a7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:58.100136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03d8d3-3e3d-4ee6-a9c3-3a697b60caf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:58.100807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03d8d3-3e3d-4ee6-a9c3-3a697b60caf3 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:58.101130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03d8d3-3e3d-4ee6-a9c3-3a697b60caf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:58.101410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03d8d3-3e3d-4ee6-a9c3-3a697b60caf3 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:58.102208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03d8d3-3e3d-4ee6-a9c3-3a697b60caf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:58.104922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03d8d3-3e3d-4ee6-a9c3-3a697b60caf3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:58.144537 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2557/5032] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:07:57 2022] POST /identity/v3/users => generated 284 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:07:58.149267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adae6b-b805-4ac1-90e7-c6d50706091e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:58.163055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adae6b-b805-4ac1-90e7-c6d50706091e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:58.168305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d960f0-316c-4f3a-9c44-f27aa2d6c32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:58.170464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d960f0-316c-4f3a-9c44-f27aa2d6c32f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:58.170821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d960f0-316c-4f3a-9c44-f27aa2d6c32f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:58.171139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d960f0-316c-4f3a-9c44-f27aa2d6c32f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:58.179262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d960f0-316c-4f3a-9c44-f27aa2d6c32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:58.182449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d960f0-316c-4f3a-9c44-f27aa2d6c32f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:58.196684 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2476/5033] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:58.201396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03d8d3-3e3d-4ee6-a9c3-3a697b60caf3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:58.212195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03d8d3-3e3d-4ee6-a9c3-3a697b60caf3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:58.217461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14c5df-3fbe-440f-ab70-5079b501f9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:58.218079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14c5df-3fbe-440f-ab70-5079b501f9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:58.218368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14c5df-3fbe-440f-ab70-5079b501f9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:58.218649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14c5df-3fbe-440f-ab70-5079b501f9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:58.219303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14c5df-3fbe-440f-ab70-5079b501f9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:58.221378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14c5df-3fbe-440f-ab70-5079b501f9f0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:58.237308 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2558/5034] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:07:58.241884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d960f0-316c-4f3a-9c44-f27aa2d6c32f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:58.253523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d960f0-316c-4f3a-9c44-f27aa2d6c32f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:58.259922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d16ca5-8939-46bc-91e0-f4221712379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:58.261080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d16ca5-8939-46bc-91e0-f4221712379c admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:58.261584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d16ca5-8939-46bc-91e0-f4221712379c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:58.262077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d16ca5-8939-46bc-91e0-f4221712379c admin admin] PATH_INFO: `/v3/projects/d7cddc92268c48a08f3b69a7cf5331f8/users/6a3d97dd4c424030b0324ca260ee9250/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:58.282583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d16ca5-8939-46bc-91e0-f4221712379c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7cddc92268c48a08f3b69a7cf5331f8, user_id=6a3d97dd4c424030b0324ca260ee9250, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:58.286619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d16ca5-8939-46bc-91e0-f4221712379c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:58.304067 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2477/5035] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:07:58 2022] PUT /identity/v3/projects/d7cddc92268c48a08f3b69a7cf5331f8/users/6a3d97dd4c424030b0324ca260ee9250/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:07:58.315757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a197e-4078-4215-b57a-c310b2c82861 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:58.316139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a197e-4078-4215-b57a-c310b2c82861 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:58.316418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a197e-4078-4215-b57a-c310b2c82861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:58.361982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-672a197e-4078-4215-b57a-c310b2c82861 None None] MFA Rules not processed for user `6a3d97dd4c424030b0324ca260ee9250`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:07:58.363360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-672a197e-4078-4215-b57a-c310b2c82861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:58.441489 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2559/5036] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:07:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:07:58.453044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d16ca5-8939-46bc-91e0-f4221712379c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:07:58.454886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-58d16ca5-8939-46bc-91e0-f4221712379c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:07:58.555374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d16ca5-8939-46bc-91e0-f4221712379c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:07:58.578216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6295718-33ea-429a-af0f-56cf30801161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:07:58.582526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6295718-33ea-429a-af0f-56cf30801161 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:07:58.583614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6295718-33ea-429a-af0f-56cf30801161 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:07:58.588747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6295718-33ea-429a-af0f-56cf30801161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:07:58.603907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6295718-33ea-429a-af0f-56cf30801161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:07:58.605860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6295718-33ea-429a-af0f-56cf30801161 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:07:58.619442 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2478/5037] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:07:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:00.667651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a197e-4078-4215-b57a-c310b2c82861 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:00.729005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:00.749262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a197e-4078-4215-b57a-c310b2c82861 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:00.754363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b1d26-6df2-4a40-a9ea-f3fdd8b609f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:00.755458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b1d26-6df2-4a40-a9ea-f3fdd8b609f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:00.755968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b1d26-6df2-4a40-a9ea-f3fdd8b609f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:00.756481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b1d26-6df2-4a40-a9ea-f3fdd8b609f4 admin admin] PATH_INFO: `/v3/users/6a3d97dd4c424030b0324ca260ee9250` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:00.758235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b1d26-6df2-4a40-a9ea-f3fdd8b609f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a3d97dd4c424030b0324ca260ee9250)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:00.761392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b1d26-6df2-4a40-a9ea-f3fdd8b609f4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:00.829307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a09b1d26-6df2-4a40-a9ea-f3fdd8b609f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a3d97dd4c424030b0324ca260ee9250'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:00.850221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a09b1d26-6df2-4a40-a9ea-f3fdd8b609f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a3d97dd4c424030b0324ca260ee9250'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:00.861089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a09b1d26-6df2-4a40-a9ea-f3fdd8b609f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a3d97dd4c424030b0324ca260ee9250'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:00.863249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a09b1d26-6df2-4a40-a9ea-f3fdd8b609f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a3d97dd4c424030b0324ca260ee9250'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:00.865400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a09b1d26-6df2-4a40-a9ea-f3fdd8b609f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a3d97dd4c424030b0324ca260ee9250'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:00.904591 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2560/5038] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:00 2022] DELETE /identity/v3/users/6a3d97dd4c424030b0324ca260ee9250 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:00.909890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b63ba-4703-4a9e-b0f0-75610ff8d572 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:00.911192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b63ba-4703-4a9e-b0f0-75610ff8d572 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:00.911696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b63ba-4703-4a9e-b0f0-75610ff8d572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:00.942326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e57b63ba-4703-4a9e-b0f0-75610ff8d572 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:00.943918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e57b63ba-4703-4a9e-b0f0-75610ff8d572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:01.076424 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2479/5039] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:01.078637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deb41f-c67c-4eb4-aa54-386210453c91 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:01.079190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deb41f-c67c-4eb4-aa54-386210453c91 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:01.079686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deb41f-c67c-4eb4-aa54-386210453c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:01.121951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-05deb41f-c67c-4eb4-aa54-386210453c91 None None] MFA Rules not processed for user `3e79edf1a33e45ddac92ba8cd8429d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:01.123983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-05deb41f-c67c-4eb4-aa54-386210453c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:01.133053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b63ba-4703-4a9e-b0f0-75610ff8d572 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:01.143548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e57b63ba-4703-4a9e-b0f0-75610ff8d572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:01.247321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:01.303964 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2561/5040] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:01.311638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b63ba-4703-4a9e-b0f0-75610ff8d572 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:01.316851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdd1ba-ace2-4318-af10-d44715709514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:01.317520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdd1ba-ace2-4318-af10-d44715709514 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:01.317810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdd1ba-ace2-4318-af10-d44715709514 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:01.318093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdd1ba-ace2-4318-af10-d44715709514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:01.323798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deb41f-c67c-4eb4-aa54-386210453c91 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:01.326693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-05deb41f-c67c-4eb4-aa54-386210453c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:01.343716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdd1ba-ace2-4318-af10-d44715709514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:01.345394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdd1ba-ace2-4318-af10-d44715709514 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:01.357905 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2480/5041] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:01.458851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:01.487057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deb41f-c67c-4eb4-aa54-386210453c91 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:01.495003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b039bb5-cf3a-404d-b7c8-a40d5845b839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:01.495697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b039bb5-cf3a-404d-b7c8-a40d5845b839 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:01.496052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b039bb5-cf3a-404d-b7c8-a40d5845b839 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:01.496388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b039bb5-cf3a-404d-b7c8-a40d5845b839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:01.507023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b039bb5-cf3a-404d-b7c8-a40d5845b839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:01.509243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b039bb5-cf3a-404d-b7c8-a40d5845b839 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:01.521268 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2562/5042] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:08:02.365596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdd1ba-ace2-4318-af10-d44715709514 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:02.367458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-30cdd1ba-ace2-4318-af10-d44715709514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:02.391060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdd1ba-ace2-4318-af10-d44715709514 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:02.404774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:02.405823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:02.406220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:02.406761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9 admin admin] PATH_INFO: `/v3/projects/d7cddc92268c48a08f3b69a7cf5331f8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:02.408883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7cddc92268c48a08f3b69a7cf5331f8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:02.413895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:02.496852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7cddc92268c48a08f3b69a7cf5331f8', 'request_id': 'req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:02.512200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7cddc92268c48a08f3b69a7cf5331f8', 'request_id': 'req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:02.522207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7cddc92268c48a08f3b69a7cf5331f8', 'request_id': 'req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:02.555824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9 admin admin] The token cache is being invalidate because project d7cddc92268c48a08f3b69a7cf5331f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:02.556271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:02.558088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3c3a6a63-e3ac-44d7-bab3-34b246731aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:02.562430 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2481/5043] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:02 2022] DELETE /identity/v3/projects/d7cddc92268c48a08f3b69a7cf5331f8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:02.590692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839cb5b-c73f-45ae-8a04-7fc31acaeed6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:02.591481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839cb5b-c73f-45ae-8a04-7fc31acaeed6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:02.591932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839cb5b-c73f-45ae-8a04-7fc31acaeed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:02.626905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2839cb5b-c73f-45ae-8a04-7fc31acaeed6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:02.628639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2839cb5b-c73f-45ae-8a04-7fc31acaeed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:02.768954 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2563/5044] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:02.777877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b93fd-cda6-45dc-9206-abf42d38802f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:02.778174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b93fd-cda6-45dc-9206-abf42d38802f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:02.779081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b93fd-cda6-45dc-9206-abf42d38802f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:02.813518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4c1b93fd-cda6-45dc-9206-abf42d38802f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:02.817512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4c1b93fd-cda6-45dc-9206-abf42d38802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:02.833555 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2482/5045] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:02.842106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839cb5b-c73f-45ae-8a04-7fc31acaeed6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:02.861774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839cb5b-c73f-45ae-8a04-7fc31acaeed6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:02.883269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a3706-cee5-4897-b7be-0a15cd7394aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:02.885971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a3706-cee5-4897-b7be-0a15cd7394aa admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:02.886283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a3706-cee5-4897-b7be-0a15cd7394aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:02.886633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a3706-cee5-4897-b7be-0a15cd7394aa admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:02.888593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a3706-cee5-4897-b7be-0a15cd7394aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:02.891148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a3706-cee5-4897-b7be-0a15cd7394aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:02.895559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a3706-cee5-4897-b7be-0a15cd7394aa admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:02.926818 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2564/5046] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:02.927169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b93fd-cda6-45dc-9206-abf42d38802f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:02.940786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b93fd-cda6-45dc-9206-abf42d38802f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:02.959158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c65913-1a54-4f51-a745-cb388be89828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:02.960690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c65913-1a54-4f51-a745-cb388be89828 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:02.962657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c65913-1a54-4f51-a745-cb388be89828 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:02.963399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c65913-1a54-4f51-a745-cb388be89828 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:02.963894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c65913-1a54-4f51-a745-cb388be89828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:02.967943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c65913-1a54-4f51-a745-cb388be89828 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:03.044415 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2483/5047] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:02 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:03.049188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a3706-cee5-4897-b7be-0a15cd7394aa admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:03.064554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a3706-cee5-4897-b7be-0a15cd7394aa admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:03.165496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48285f-9e84-4ec1-a7f0-fa7858815a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:03.166295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48285f-9e84-4ec1-a7f0-fa7858815a53 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:03.166582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48285f-9e84-4ec1-a7f0-fa7858815a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:03.166865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48285f-9e84-4ec1-a7f0-fa7858815a53 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:03.168009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48285f-9e84-4ec1-a7f0-fa7858815a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:03.170433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48285f-9e84-4ec1-a7f0-fa7858815a53 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:03.209092 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2565/5048] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:03 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:03.213456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c65913-1a54-4f51-a745-cb388be89828 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:03.249387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c65913-1a54-4f51-a745-cb388be89828 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:03.256566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13825d39-8136-4aa5-a5f0-5ef2016ba464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:03.257319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13825d39-8136-4aa5-a5f0-5ef2016ba464 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:03.258361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13825d39-8136-4aa5-a5f0-5ef2016ba464 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:03.258650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13825d39-8136-4aa5-a5f0-5ef2016ba464 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:03.259746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13825d39-8136-4aa5-a5f0-5ef2016ba464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:03.264818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13825d39-8136-4aa5-a5f0-5ef2016ba464 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:03.283551 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2484/5049] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:03.285264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48285f-9e84-4ec1-a7f0-fa7858815a53 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:03.302098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48285f-9e84-4ec1-a7f0-fa7858815a53 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:03.308450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28ab1a-3971-4f31-82c7-9033faaa460a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:03.309951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28ab1a-3971-4f31-82c7-9033faaa460a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:03.310477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28ab1a-3971-4f31-82c7-9033faaa460a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:03.310878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28ab1a-3971-4f31-82c7-9033faaa460a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:03.312340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28ab1a-3971-4f31-82c7-9033faaa460a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:03.315857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28ab1a-3971-4f31-82c7-9033faaa460a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:03.346464 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2566/5050] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:03.350487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13825d39-8136-4aa5-a5f0-5ef2016ba464 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:03.370234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13825d39-8136-4aa5-a5f0-5ef2016ba464 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:03.383716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d0ad1-27c4-4789-a1ec-172407a871ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:03.386161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d0ad1-27c4-4789-a1ec-172407a871ae admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:03.386618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d0ad1-27c4-4789-a1ec-172407a871ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:03.388189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d0ad1-27c4-4789-a1ec-172407a871ae admin admin] PATH_INFO: `/v3/projects/43bddba02efe450ba3a4a85f855bc144/users/adec6122b37440cc94aee4405ff1957d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:03.427770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d0ad1-27c4-4789-a1ec-172407a871ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=43bddba02efe450ba3a4a85f855bc144, user_id=adec6122b37440cc94aee4405ff1957d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:03.432717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d0ad1-27c4-4789-a1ec-172407a871ae admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:03.471037 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2485/5051] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:03 2022] PUT /identity/v3/projects/43bddba02efe450ba3a4a85f855bc144/users/adec6122b37440cc94aee4405ff1957d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:03.473645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78419b-3821-459e-a2e0-688eba8fb743 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:03.476431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78419b-3821-459e-a2e0-688eba8fb743 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:03.476431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78419b-3821-459e-a2e0-688eba8fb743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:03.545759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9d78419b-3821-459e-a2e0-688eba8fb743 None None] MFA Rules not processed for user `adec6122b37440cc94aee4405ff1957d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:03.547811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9d78419b-3821-459e-a2e0-688eba8fb743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:03.679762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:03.795342 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2567/5052] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:03.799939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d0ad1-27c4-4789-a1ec-172407a871ae admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:03.936415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:03.960624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d0ad1-27c4-4789-a1ec-172407a871ae admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:03.981998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63955e0-1ec8-43b5-b8c9-3f29541377c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:03.982661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63955e0-1ec8-43b5-b8c9-3f29541377c1 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:03.982954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63955e0-1ec8-43b5-b8c9-3f29541377c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:03.983585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63955e0-1ec8-43b5-b8c9-3f29541377c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:03.983904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63955e0-1ec8-43b5-b8c9-3f29541377c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:03.985922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63955e0-1ec8-43b5-b8c9-3f29541377c1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:04.040005 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2486/5053] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:03 2022] POST /identity/v3/projects => generated 371 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:04.055150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78419b-3821-459e-a2e0-688eba8fb743 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:04.070235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78419b-3821-459e-a2e0-688eba8fb743 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:04.208122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619d532-b3cf-46c9-9327-ea469ae670bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:04.208998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619d532-b3cf-46c9-9327-ea469ae670bb admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:04.209292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619d532-b3cf-46c9-9327-ea469ae670bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:04.209574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619d532-b3cf-46c9-9327-ea469ae670bb admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:04.210230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619d532-b3cf-46c9-9327-ea469ae670bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:04.212924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619d532-b3cf-46c9-9327-ea469ae670bb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:04.248201 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2568/5054] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:04 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:04.252049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63955e0-1ec8-43b5-b8c9-3f29541377c1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:04.274197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63955e0-1ec8-43b5-b8c9-3f29541377c1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:04.286398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ecfe7-9096-41fa-8245-fa99860c43b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:04.287515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ecfe7-9096-41fa-8245-fa99860c43b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:04.288155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ecfe7-9096-41fa-8245-fa99860c43b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:04.288640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ecfe7-9096-41fa-8245-fa99860c43b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:04.289553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ecfe7-9096-41fa-8245-fa99860c43b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:04.293951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ecfe7-9096-41fa-8245-fa99860c43b3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:04.328055 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2487/5055] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:04.331576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619d532-b3cf-46c9-9327-ea469ae670bb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:04.343517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619d532-b3cf-46c9-9327-ea469ae670bb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:04.348739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35261e33-679b-4ceb-8e62-a8ecd3b24c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:04.350342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35261e33-679b-4ceb-8e62-a8ecd3b24c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:04.350636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35261e33-679b-4ceb-8e62-a8ecd3b24c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:04.350919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35261e33-679b-4ceb-8e62-a8ecd3b24c26 admin admin] PATH_INFO: `/v3/projects/d5852c7b40c240be93634c125c5e13f7/users/d33fd7cf52fd4bf894ec985c168ad0de/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:04.371317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35261e33-679b-4ceb-8e62-a8ecd3b24c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5852c7b40c240be93634c125c5e13f7, user_id=d33fd7cf52fd4bf894ec985c168ad0de, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:04.376310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35261e33-679b-4ceb-8e62-a8ecd3b24c26 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:04.424472 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2569/5056] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:04 2022] PUT /identity/v3/projects/d5852c7b40c240be93634c125c5e13f7/users/d33fd7cf52fd4bf894ec985c168ad0de/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:04.424706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ecfe7-9096-41fa-8245-fa99860c43b3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:04.436799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ecfe7-9096-41fa-8245-fa99860c43b3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:04.628241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa238aff-5883-427f-a27b-1c037e846df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:04.629435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238aff-5883-427f-a27b-1c037e846df8 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:04.629737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238aff-5883-427f-a27b-1c037e846df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:04.630016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238aff-5883-427f-a27b-1c037e846df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:04.630758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa238aff-5883-427f-a27b-1c037e846df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:04.640878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa238aff-5883-427f-a27b-1c037e846df8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:04.663900 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2488/5057] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:04.665820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35261e33-679b-4ceb-8e62-a8ecd3b24c26 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:04.691125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35261e33-679b-4ceb-8e62-a8ecd3b24c26 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:04.704291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f05ad3-a68e-4e61-9e21-240fd788dfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:04.707045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f05ad3-a68e-4e61-9e21-240fd788dfe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:04.708071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f05ad3-a68e-4e61-9e21-240fd788dfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:04.708600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f05ad3-a68e-4e61-9e21-240fd788dfe8 admin admin] PATH_INFO: `/v3/projects/d5852c7b40c240be93634c125c5e13f7/users/d33fd7cf52fd4bf894ec985c168ad0de/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:04.711965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f05ad3-a68e-4e61-9e21-240fd788dfe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5852c7b40c240be93634c125c5e13f7, user_id=d33fd7cf52fd4bf894ec985c168ad0de, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:04.716194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f05ad3-a68e-4e61-9e21-240fd788dfe8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:04.738737 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2570/5058] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:04 2022] PUT /identity/v3/projects/d5852c7b40c240be93634c125c5e13f7/users/d33fd7cf52fd4bf894ec985c168ad0de/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:04.749224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726a4d6-a933-42f3-bb21-18c572b01420 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:04.749523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726a4d6-a933-42f3-bb21-18c572b01420 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:04.749792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726a4d6-a933-42f3-bb21-18c572b01420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:04.832134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f726a4d6-a933-42f3-bb21-18c572b01420 None None] MFA Rules not processed for user `d33fd7cf52fd4bf894ec985c168ad0de`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:04.833475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f726a4d6-a933-42f3-bb21-18c572b01420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:05.036709 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2489/5059] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:05.041867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f05ad3-a68e-4e61-9e21-240fd788dfe8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:05.066887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f05ad3-a68e-4e61-9e21-240fd788dfe8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:05.076793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa308d-8d53-45fd-86b4-c41b6d4bd7e7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:05.077463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa308d-8d53-45fd-86b4-c41b6d4bd7e7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:05.077743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa308d-8d53-45fd-86b4-c41b6d4bd7e7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:05.078246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa308d-8d53-45fd-86b4-c41b6d4bd7e7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:05.078944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa308d-8d53-45fd-86b4-c41b6d4bd7e7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:05.081292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa308d-8d53-45fd-86b4-c41b6d4bd7e7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:05.124786 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2571/5060] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:08:05 2022] POST /identity/v3/domains => generated 277 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:05.126609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726a4d6-a933-42f3-bb21-18c572b01420 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:05.143942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726a4d6-a933-42f3-bb21-18c572b01420 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:05.289849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:05.328911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa308d-8d53-45fd-86b4-c41b6d4bd7e7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:05.330614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-73aa308d-8d53-45fd-86b4-c41b6d4bd7e7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:05.372663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868046b-81f4-418e-91df-e12ebb315786 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:05.373350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868046b-81f4-418e-91df-e12ebb315786 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:05.373649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868046b-81f4-418e-91df-e12ebb315786 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:05.373928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868046b-81f4-418e-91df-e12ebb315786 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:05.374583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868046b-81f4-418e-91df-e12ebb315786 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:05.384316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868046b-81f4-418e-91df-e12ebb315786 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:05.520486 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2490/5061] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:05 2022] POST /identity/v3/projects => generated 403 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:05.525607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868046b-81f4-418e-91df-e12ebb315786 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:05.554162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa308d-8d53-45fd-86b4-c41b6d4bd7e7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:05.653813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:05.776219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868046b-81f4-418e-91df-e12ebb315786 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:05.802774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec61952-e206-4992-a6f6-6256d602fc2a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:05.803422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec61952-e206-4992-a6f6-6256d602fc2a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:05.803706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec61952-e206-4992-a6f6-6256d602fc2a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:05.803985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec61952-e206-4992-a6f6-6256d602fc2a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:05.804635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec61952-e206-4992-a6f6-6256d602fc2a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:05.806640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec61952-e206-4992-a6f6-6256d602fc2a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:05.821915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9b9ca-f610-42b8-8a82-0f7c7196b783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:05.822598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9b9ca-f610-42b8-8a82-0f7c7196b783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:05.822874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9b9ca-f610-42b8-8a82-0f7c7196b783 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:05.823154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9b9ca-f610-42b8-8a82-0f7c7196b783 admin admin] PATH_INFO: `/v3/users/1cf3a90cb1034ae39f67fb0fa74366f7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:05.824819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9b9ca-f610-42b8-8a82-0f7c7196b783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cf3a90cb1034ae39f67fb0fa74366f7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:05.826659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9b9ca-f610-42b8-8a82-0f7c7196b783 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:05.856430 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2491/5062] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Sep 3 21:08:05 2022] POST /identity/v3/groups => generated 302 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:05.861961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec61952-e206-4992-a6f6-6256d602fc2a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:05.888683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec61952-e206-4992-a6f6-6256d602fc2a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:05.904733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a34c4-bf26-4e7e-bf7e-2e57680bea06 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:05.905327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a34c4-bf26-4e7e-bf7e-2e57680bea06 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:05.905616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a34c4-bf26-4e7e-bf7e-2e57680bea06 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:05.905899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a34c4-bf26-4e7e-bf7e-2e57680bea06 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:05.906563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a34c4-bf26-4e7e-bf7e-2e57680bea06 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:05.908604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a34c4-bf26-4e7e-bf7e-2e57680bea06 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:05.932601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ac9b9ca-f610-42b8-8a82-0f7c7196b783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cf3a90cb1034ae39f67fb0fa74366f7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:05.956404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ac9b9ca-f610-42b8-8a82-0f7c7196b783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cf3a90cb1034ae39f67fb0fa74366f7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:05.957377 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2492/5063] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:05 2022] POST /identity/v3/users => generated 445 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:05.962640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a34c4-bf26-4e7e-bf7e-2e57680bea06 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:05.979930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ac9b9ca-f610-42b8-8a82-0f7c7196b783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cf3a90cb1034ae39f67fb0fa74366f7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:05.980534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ac9b9ca-f610-42b8-8a82-0f7c7196b783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cf3a90cb1034ae39f67fb0fa74366f7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:05.981515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ac9b9ca-f610-42b8-8a82-0f7c7196b783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cf3a90cb1034ae39f67fb0fa74366f7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:05.987459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a34c4-bf26-4e7e-bf7e-2e57680bea06 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:06.000164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84636332-36fc-46b4-8be4-1a68c2ef6973 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:06.002273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84636332-36fc-46b4-8be4-1a68c2ef6973 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:06.002625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84636332-36fc-46b4-8be4-1a68c2ef6973 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:06.003030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84636332-36fc-46b4-8be4-1a68c2ef6973 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:06.005667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84636332-36fc-46b4-8be4-1a68c2ef6973 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:06.008304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84636332-36fc-46b4-8be4-1a68c2ef6973 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:06.009834 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2572/5064] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:05 2022] DELETE /identity/v3/users/1cf3a90cb1034ae39f67fb0fa74366f7 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:06.017776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85180b26-4dd6-421f-af99-e9bf81a5948a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:06.018276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85180b26-4dd6-421f-af99-e9bf81a5948a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:06.018969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85180b26-4dd6-421f-af99-e9bf81a5948a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:06.068417 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2493/5065] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:08:05 2022] POST /identity/v3/roles => generated 239 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:06.073311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84636332-36fc-46b4-8be4-1a68c2ef6973 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:06.076264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-84636332-36fc-46b4-8be4-1a68c2ef6973 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:06.085240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-85180b26-4dd6-421f-af99-e9bf81a5948a None None] MFA Rules not processed for user `3e79edf1a33e45ddac92ba8cd8429d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:06.086755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-85180b26-4dd6-421f-af99-e9bf81a5948a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:06.094684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84636332-36fc-46b4-8be4-1a68c2ef6973 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:06.204215 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2573/5066] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:06.204435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1797db3-673c-4cae-9141-549a85e6b1e8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:06.204435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1797db3-673c-4cae-9141-549a85e6b1e8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:06.204435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1797db3-673c-4cae-9141-549a85e6b1e8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:06.217053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1797db3-673c-4cae-9141-549a85e6b1e8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/groups/7951efaead8c49f9b57b03baaa294bed/roles/c69f1bc476b94a17a97dde2b1235984d/inherited_to_projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:06.231189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1797db3-673c-4cae-9141-549a85e6b1e8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7e4a23fa192e47d2aacabb831375ddaa, group_id=7951efaead8c49f9b57b03baaa294bed, role_id=c69f1bc476b94a17a97dde2b1235984d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:06.235728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1797db3-673c-4cae-9141-549a85e6b1e8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:06.281035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85180b26-4dd6-421f-af99-e9bf81a5948a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:06.282798 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2494/5067] 10.222.0.16 () {60 vars in 1585 bytes} [Sat Sep 3 21:08:06 2022] PUT /identity/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/groups/7951efaead8c49f9b57b03baaa294bed/roles/c69f1bc476b94a17a97dde2b1235984d/inherited_to_projects => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:06.296196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85180b26-4dd6-421f-af99-e9bf81a5948a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:06.401778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1797db3-673c-4cae-9141-549a85e6b1e8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:06.403776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b1797db3-673c-4cae-9141-549a85e6b1e8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:06.404473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:06.439142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8ea91-d3be-4386-8893-f71fcb2362a3 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:06.454690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8ea91-d3be-4386-8893-f71fcb2362a3 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:06.456389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8ea91-d3be-4386-8893-f71fcb2362a3 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:06.456509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8ea91-d3be-4386-8893-f71fcb2362a3 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/groups/7951efaead8c49f9b57b03baaa294bed/roles/c69f1bc476b94a17a97dde2b1235984d/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:06.461549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8ea91-d3be-4386-8893-f71fcb2362a3 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:check_grant(project_id=7e4a23fa192e47d2aacabb831375ddaa, group_id=7951efaead8c49f9b57b03baaa294bed, role_id=c69f1bc476b94a17a97dde2b1235984d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:06.466782 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-f3b8ea91-d3be-4386-8893-f71fcb2362a3 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:06.466782 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:06.466782 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:06.468122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:06.473237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8ea91-d3be-4386-8893-f71fcb2362a3 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:06.506336 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2574/5068] 10.222.0.16 () {58 vars in 1567 bytes} [Sat Sep 3 21:08:06 2022] HEAD /identity/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/groups/7951efaead8c49f9b57b03baaa294bed/roles/c69f1bc476b94a17a97dde2b1235984d/inherited_to_projects => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:06.510376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8ea91-d3be-4386-8893-f71fcb2362a3 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:06.529891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8ea91-d3be-4386-8893-f71fcb2362a3 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:06.535533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:06.536331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:06.536729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:06.537164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/groups/7951efaead8c49f9b57b03baaa294bed/roles/c69f1bc476b94a17a97dde2b1235984d/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:06.540590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=7e4a23fa192e47d2aacabb831375ddaa, group_id=7951efaead8c49f9b57b03baaa294bed, role_id=c69f1bc476b94a17a97dde2b1235984d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:06.540800 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:06.540800 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:06.540800 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:06.543231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:06.545187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1797db3-673c-4cae-9141-549a85e6b1e8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:06.550049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invalidating the token cache because role c69f1bc476b94a17a97dde2b1235984d was removed from group 7951efaead8c49f9b57b03baaa294bed on project 7e4a23fa192e47d2aacabb831375ddaa. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:06.552115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:06.552858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:06.553296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:06.553581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] PATH_INFO: `/v3/projects/1646ccd0c1d442bdbab54ad9f44e3aa6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:06.554760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1646ccd0c1d442bdbab54ad9f44e3aa6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:06.555607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.556542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.559196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:06.580120 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2575/5069] 10.222.0.16 () {58 vars in 1569 bytes} [Sat Sep 3 21:08:06 2022] DELETE /identity/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/groups/7951efaead8c49f9b57b03baaa294bed/roles/c69f1bc476b94a17a97dde2b1235984d/inherited_to_projects => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:06.584669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:06.586400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:06.629863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1646ccd0c1d442bdbab54ad9f44e3aa6', 'request_id': 'req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.645970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1646ccd0c1d442bdbab54ad9f44e3aa6', 'request_id': 'req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.655614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1646ccd0c1d442bdbab54ad9f44e3aa6', 'request_id': 'req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.677733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] The token cache is being invalidate because project 1646ccd0c1d442bdbab54ad9f44e3aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:06.678119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.679094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.681095 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2495/5070] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:06 2022] DELETE /identity/v3/projects/1646ccd0c1d442bdbab54ad9f44e3aa6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:06.685623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:06.687604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:06.747448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffb72a-3b35-4fdf-9cac-6f7adc37812f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:06.812434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be14e2f-c644-4c45-a6b2-392c3a9b3f8b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:06.819274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:06.821665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:06.822129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:06.822559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] PATH_INFO: `/v3/users/e052a3c319774afb8b373adf58417a46` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:06.824627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e052a3c319774afb8b373adf58417a46)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:06.827180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:06.854501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65147005-1573-4af1-a34d-509f2d61a051 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:06.855523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65147005-1573-4af1-a34d-509f2d61a051 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:06.855932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65147005-1573-4af1-a34d-509f2d61a051 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:06.856330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65147005-1573-4af1-a34d-509f2d61a051 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/roles/c69f1bc476b94a17a97dde2b1235984d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:06.858413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65147005-1573-4af1-a34d-509f2d61a051 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:delete_role(role_id=c69f1bc476b94a17a97dde2b1235984d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:06.860176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65147005-1573-4af1-a34d-509f2d61a051 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:06.904486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e052a3c319774afb8b373adf58417a46'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.906808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:06.926285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e052a3c319774afb8b373adf58417a46'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.927325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e052a3c319774afb8b373adf58417a46'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.938237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e052a3c319774afb8b373adf58417a46'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.950356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65147005-1573-4af1-a34d-509f2d61a051 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invalidating the token cache because role c69f1bc476b94a17a97dde2b1235984d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:06.950668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65147005-1573-4af1-a34d-509f2d61a051 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.951387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65147005-1573-4af1-a34d-509f2d61a051 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.952305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e052a3c319774afb8b373adf58417a46'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:06.958611 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2576/5071] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:06 2022] DELETE /identity/v3/roles/c69f1bc476b94a17a97dde2b1235984d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:06.959619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65147005-1573-4af1-a34d-509f2d61a051 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:06.961158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-65147005-1573-4af1-a34d-509f2d61a051 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:06.970143 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2496/5072] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:06 2022] DELETE /identity/v3/users/e052a3c319774afb8b373adf58417a46 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:07.051571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:07.099863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65147005-1573-4af1-a34d-509f2d61a051 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:07.195760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cda001-1b7a-467c-826d-09f9d52c57a7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:07.197317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cda001-1b7a-467c-826d-09f9d52c57a7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:07.198122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cda001-1b7a-467c-826d-09f9d52c57a7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:07.198480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cda001-1b7a-467c-826d-09f9d52c57a7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:07.199254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cda001-1b7a-467c-826d-09f9d52c57a7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:07.201152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cda001-1b7a-467c-826d-09f9d52c57a7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:07.233367 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2577/5073] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:08:06 2022] POST /identity/v3/roles => generated 239 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:07.235862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:07.237504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:07.258482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93f611-23ce-45cc-abe3-7c6707825476 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:07.264319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3de8b1-c3aa-4444-98f2-6f70bc872d2e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:07.265067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3de8b1-c3aa-4444-98f2-6f70bc872d2e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:07.265356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3de8b1-c3aa-4444-98f2-6f70bc872d2e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:07.265799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3de8b1-c3aa-4444-98f2-6f70bc872d2e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/users/b8b9d276657d4d65a6f5281e88d61c76/roles/d888d5dcb8d3432faed6cab5092a9106/inherited_to_projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:07.287062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3de8b1-c3aa-4444-98f2-6f70bc872d2e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7e4a23fa192e47d2aacabb831375ddaa, user_id=b8b9d276657d4d65a6f5281e88d61c76, role_id=d888d5dcb8d3432faed6cab5092a9106)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:07.289961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3de8b1-c3aa-4444-98f2-6f70bc872d2e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:07.309416 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2497/5074] 10.222.0.16 () {60 vars in 1582 bytes} [Sat Sep 3 21:08:07 2022] PUT /identity/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/users/b8b9d276657d4d65a6f5281e88d61c76/roles/d888d5dcb8d3432faed6cab5092a9106/inherited_to_projects => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:07.313770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cda001-1b7a-467c-826d-09f9d52c57a7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:07.333187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cda001-1b7a-467c-826d-09f9d52c57a7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:07.403044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3de8b1-c3aa-4444-98f2-6f70bc872d2e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:07.405605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-aa3de8b1-c3aa-4444-98f2-6f70bc872d2e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:07.540918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd64bff-fef8-4d07-85f2-87b6bb87b93e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:07.544153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd64bff-fef8-4d07-85f2-87b6bb87b93e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:07.545019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd64bff-fef8-4d07-85f2-87b6bb87b93e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:07.545870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd64bff-fef8-4d07-85f2-87b6bb87b93e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/users/b8b9d276657d4d65a6f5281e88d61c76/roles/d888d5dcb8d3432faed6cab5092a9106/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:07.555290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd64bff-fef8-4d07-85f2-87b6bb87b93e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:check_grant(project_id=7e4a23fa192e47d2aacabb831375ddaa, user_id=b8b9d276657d4d65a6f5281e88d61c76, role_id=d888d5dcb8d3432faed6cab5092a9106)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:07.563688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd64bff-fef8-4d07-85f2-87b6bb87b93e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:07.583958 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2578/5075] 10.222.0.16 () {58 vars in 1564 bytes} [Sat Sep 3 21:08:07 2022] HEAD /identity/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/users/b8b9d276657d4d65a6f5281e88d61c76/roles/d888d5dcb8d3432faed6cab5092a9106/inherited_to_projects => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:07.585660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd64bff-fef8-4d07-85f2-87b6bb87b93e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:07.615056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd64bff-fef8-4d07-85f2-87b6bb87b93e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:07.624260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3de8b1-c3aa-4444-98f2-6f70bc872d2e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:07.629140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf1510-09b8-45c9-a475-4fef54d67129 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:07.631320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf1510-09b8-45c9-a475-4fef54d67129 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:07.632932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf1510-09b8-45c9-a475-4fef54d67129 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:07.633983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf1510-09b8-45c9-a475-4fef54d67129 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/users/b8b9d276657d4d65a6f5281e88d61c76/roles/d888d5dcb8d3432faed6cab5092a9106/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:07.636991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff9e0c-81d3-4c03-8d50-41951979fe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:07.639025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff9e0c-81d3-4c03-8d50-41951979fe0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:07.639448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf1510-09b8-45c9-a475-4fef54d67129 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=7e4a23fa192e47d2aacabb831375ddaa, user_id=b8b9d276657d4d65a6f5281e88d61c76, role_id=d888d5dcb8d3432faed6cab5092a9106)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:07.639821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff9e0c-81d3-4c03-8d50-41951979fe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:07.640109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff9e0c-81d3-4c03-8d50-41951979fe0f admin admin] PATH_INFO: `/v3/projects/e2834d4f687747f2b38e3ab7ffa8fd56` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:07.646847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff9e0c-81d3-4c03-8d50-41951979fe0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2834d4f687747f2b38e3ab7ffa8fd56)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:07.648810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff9e0c-81d3-4c03-8d50-41951979fe0f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:07.650086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf1510-09b8-45c9-a475-4fef54d67129 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:07.663458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8bf1510-09b8-45c9-a475-4fef54d67129 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invalidating the token cache because role d888d5dcb8d3432faed6cab5092a9106 was removed from user b8b9d276657d4d65a6f5281e88d61c76 on project 7e4a23fa192e47d2aacabb831375ddaa. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:07.663754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8bf1510-09b8-45c9-a475-4fef54d67129 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:07.664514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8bf1510-09b8-45c9-a475-4fef54d67129 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:07.702336 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2579/5076] 10.222.0.16 () {58 vars in 1566 bytes} [Sat Sep 3 21:08:07 2022] DELETE /identity/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/users/b8b9d276657d4d65a6f5281e88d61c76/roles/d888d5dcb8d3432faed6cab5092a9106/inherited_to_projects => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:07.707275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf1510-09b8-45c9-a475-4fef54d67129 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:07.709253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b8bf1510-09b8-45c9-a475-4fef54d67129 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:07.764019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94ff9e0c-81d3-4c03-8d50-41951979fe0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2834d4f687747f2b38e3ab7ffa8fd56', 'request_id': 'req-94ff9e0c-81d3-4c03-8d50-41951979fe0f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:07.776465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94ff9e0c-81d3-4c03-8d50-41951979fe0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2834d4f687747f2b38e3ab7ffa8fd56', 'request_id': 'req-94ff9e0c-81d3-4c03-8d50-41951979fe0f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:07.784107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94ff9e0c-81d3-4c03-8d50-41951979fe0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2834d4f687747f2b38e3ab7ffa8fd56', 'request_id': 'req-94ff9e0c-81d3-4c03-8d50-41951979fe0f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:07.800386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94ff9e0c-81d3-4c03-8d50-41951979fe0f admin admin] The token cache is being invalidate because project e2834d4f687747f2b38e3ab7ffa8fd56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:07.800683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94ff9e0c-81d3-4c03-8d50-41951979fe0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:07.801385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94ff9e0c-81d3-4c03-8d50-41951979fe0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:07.807045 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2498/5077] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:07 2022] DELETE /identity/v3/projects/e2834d4f687747f2b38e3ab7ffa8fd56 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:07.810432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc32b2-378c-4ec4-ae1d-9ab9309fb0b6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:07.810720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc32b2-378c-4ec4-ae1d-9ab9309fb0b6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:07.811007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc32b2-378c-4ec4-ae1d-9ab9309fb0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:07.819156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf1510-09b8-45c9-a475-4fef54d67129 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:07.826372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:07.835336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2acc32b2-378c-4ec4-ae1d-9ab9309fb0b6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:07.837231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2acc32b2-378c-4ec4-ae1d-9ab9309fb0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:07.876029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:07.887874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c6999-1db2-41b4-9fb9-3cb8c6267d01 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:07.889158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c6999-1db2-41b4-9fb9-3cb8c6267d01 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:07.889158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c6999-1db2-41b4-9fb9-3cb8c6267d01 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:07.889348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c6999-1db2-41b4-9fb9-3cb8c6267d01 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/roles/d888d5dcb8d3432faed6cab5092a9106` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:07.890930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c6999-1db2-41b4-9fb9-3cb8c6267d01 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:delete_role(role_id=d888d5dcb8d3432faed6cab5092a9106)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:07.892582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c6999-1db2-41b4-9fb9-3cb8c6267d01 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:07.932938 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2499/5078] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:07.937887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc151d29-9c0f-4173-9bf6-d478c1e16e9f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:07.938108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc151d29-9c0f-4173-9bf6-d478c1e16e9f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:07.938534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc151d29-9c0f-4173-9bf6-d478c1e16e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:07.948147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0a6c6999-1db2-41b4-9fb9-3cb8c6267d01 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invalidating the token cache because role d888d5dcb8d3432faed6cab5092a9106 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:07.948778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0a6c6999-1db2-41b4-9fb9-3cb8c6267d01 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:07.950194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0a6c6999-1db2-41b4-9fb9-3cb8c6267d01 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:07.952757 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2580/5079] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:07 2022] DELETE /identity/v3/roles/d888d5dcb8d3432faed6cab5092a9106 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:07.958820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c6999-1db2-41b4-9fb9-3cb8c6267d01 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:07.961204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0a6c6999-1db2-41b4-9fb9-3cb8c6267d01 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:07.983890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bc151d29-9c0f-4173-9bf6-d478c1e16e9f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:07.985671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bc151d29-9c0f-4173-9bf6-d478c1e16e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:08.087189 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2500/5080] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:08.088146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc151d29-9c0f-4173-9bf6-d478c1e16e9f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:08.099875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc151d29-9c0f-4173-9bf6-d478c1e16e9f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:08.106097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b517020-2034-4135-99b4-de60f93991f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:08.106733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b517020-2034-4135-99b4-de60f93991f0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.107023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b517020-2034-4135-99b4-de60f93991f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.107440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b517020-2034-4135-99b4-de60f93991f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.108111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b517020-2034-4135-99b4-de60f93991f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:08.108426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b517020-2034-4135-99b4-de60f93991f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:08.109004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c6999-1db2-41b4-9fb9-3cb8c6267d01 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:08.111418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b517020-2034-4135-99b4-de60f93991f0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:08.124455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71661d5-0683-45aa-b514-2bb3a1844506 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:08.125199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71661d5-0683-45aa-b514-2bb3a1844506 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.125553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71661d5-0683-45aa-b514-2bb3a1844506 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.125831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71661d5-0683-45aa-b514-2bb3a1844506 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.126006 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2501/5081] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:08.126513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71661d5-0683-45aa-b514-2bb3a1844506 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:08.128155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71661d5-0683-45aa-b514-2bb3a1844506 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:08.133281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b517020-2034-4135-99b4-de60f93991f0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:08.143526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b517020-2034-4135-99b4-de60f93991f0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:08.148382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbadb4-aab7-4c6f-83b9-a5fb7bd8415f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:08.149038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbadb4-aab7-4c6f-83b9-a5fb7bd8415f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.149326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbadb4-aab7-4c6f-83b9-a5fb7bd8415f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.149608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbadb4-aab7-4c6f-83b9-a5fb7bd8415f admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.150290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbadb4-aab7-4c6f-83b9-a5fb7bd8415f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:08.152335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbadb4-aab7-4c6f-83b9-a5fb7bd8415f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:08.154238 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2581/5082] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:08:07 2022] POST /identity/v3/roles => generated 239 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:08.158162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71661d5-0683-45aa-b514-2bb3a1844506 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:08.171615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71661d5-0683-45aa-b514-2bb3a1844506 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:08.178362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293ea7a-cd3a-4cad-acae-1f10192eb047 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:08.179377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293ea7a-cd3a-4cad-acae-1f10192eb047 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.179914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293ea7a-cd3a-4cad-acae-1f10192eb047 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.180289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293ea7a-cd3a-4cad-acae-1f10192eb047 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/groups/7951efaead8c49f9b57b03baaa294bed/roles/f860bfb5a5ef4a8181aa19d07c46813b/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.182834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293ea7a-cd3a-4cad-acae-1f10192eb047 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=624a4f5ac1354a71bf67dccccf8b5337, group_id=7951efaead8c49f9b57b03baaa294bed, role_id=f860bfb5a5ef4a8181aa19d07c46813b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:08.186275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293ea7a-cd3a-4cad-acae-1f10192eb047 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:08.189356 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2502/5083] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:08 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:08.193104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbadb4-aab7-4c6f-83b9-a5fb7bd8415f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:08.211192 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2582/5084] 10.222.0.16 () {60 vars in 1582 bytes} [Sat Sep 3 21:08:08 2022] PUT /identity/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/groups/7951efaead8c49f9b57b03baaa294bed/roles/f860bfb5a5ef4a8181aa19d07c46813b/inherited_to_projects => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:08.212655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293ea7a-cd3a-4cad-acae-1f10192eb047 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:08.298661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:08.301424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbadb4-aab7-4c6f-83b9-a5fb7bd8415f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:08.310883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293ea7a-cd3a-4cad-acae-1f10192eb047 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:08.320177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f9d06-8459-4133-8605-b0006fcb135a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:08.320875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f9d06-8459-4133-8605-b0006fcb135a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.321195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f9d06-8459-4133-8605-b0006fcb135a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.321492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f9d06-8459-4133-8605-b0006fcb135a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/groups/7951efaead8c49f9b57b03baaa294bed/roles/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.323203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f9d06-8459-4133-8605-b0006fcb135a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=624a4f5ac1354a71bf67dccccf8b5337, group_id=7951efaead8c49f9b57b03baaa294bed)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:08.324248 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-980f9d06-8459-4133-8605-b0006fcb135a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:08.324248 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:08.324248 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:08.326300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f9d06-8459-4133-8605-b0006fcb135a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:08.343006 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2583/5085] 10.222.0.16 () {58 vars in 1464 bytes} [Sat Sep 3 21:08:08 2022] GET /identity/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/groups/7951efaead8c49f9b57b03baaa294bed/roles/inherited_to_projects => generated 447 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:08.344646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f9d06-8459-4133-8605-b0006fcb135a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:08.362583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:08.368885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f9d06-8459-4133-8605-b0006fcb135a tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:08.378490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68f641-c92b-4098-8a38-b7ebd180a0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:08.378905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3c875-e37c-4904-b815-dd1327842cdf tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:08.379273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68f641-c92b-4098-8a38-b7ebd180a0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.379569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68f641-c92b-4098-8a38-b7ebd180a0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.379850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68f641-c92b-4098-8a38-b7ebd180a0f5 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.380494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68f641-c92b-4098-8a38-b7ebd180a0f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:08.380612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3c875-e37c-4904-b815-dd1327842cdf tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.381003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3c875-e37c-4904-b815-dd1327842cdf tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.381391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3c875-e37c-4904-b815-dd1327842cdf tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/groups/7951efaead8c49f9b57b03baaa294bed/roles/f860bfb5a5ef4a8181aa19d07c46813b/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.382538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68f641-c92b-4098-8a38-b7ebd180a0f5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:08.384578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3c875-e37c-4904-b815-dd1327842cdf tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=624a4f5ac1354a71bf67dccccf8b5337, group_id=7951efaead8c49f9b57b03baaa294bed, role_id=f860bfb5a5ef4a8181aa19d07c46813b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:08.387848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3c875-e37c-4904-b815-dd1327842cdf tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:08.410447 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2584/5086] 10.222.0.16 () {58 vars in 1564 bytes} [Sat Sep 3 21:08:08 2022] HEAD /identity/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/groups/7951efaead8c49f9b57b03baaa294bed/roles/f860bfb5a5ef4a8181aa19d07c46813b/inherited_to_projects => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:08.413019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3c875-e37c-4904-b815-dd1327842cdf tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:08.428377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3c875-e37c-4904-b815-dd1327842cdf tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:08.444204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549d525-0b6e-4020-8419-1e2894db071b tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:08.445737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549d525-0b6e-4020-8419-1e2894db071b tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.446103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549d525-0b6e-4020-8419-1e2894db071b tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.446698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549d525-0b6e-4020-8419-1e2894db071b tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/groups/7951efaead8c49f9b57b03baaa294bed/roles/f860bfb5a5ef4a8181aa19d07c46813b/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.449687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549d525-0b6e-4020-8419-1e2894db071b tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=624a4f5ac1354a71bf67dccccf8b5337, group_id=7951efaead8c49f9b57b03baaa294bed, role_id=f860bfb5a5ef4a8181aa19d07c46813b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:08.454128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549d525-0b6e-4020-8419-1e2894db071b tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:08.457128 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2503/5087] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:08 2022] POST /identity/v3/users => generated 302 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:08.459901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68f641-c92b-4098-8a38-b7ebd180a0f5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:08.462106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a549d525-0b6e-4020-8419-1e2894db071b tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invalidating the token cache because role f860bfb5a5ef4a8181aa19d07c46813b was removed from group 7951efaead8c49f9b57b03baaa294bed on domain 624a4f5ac1354a71bf67dccccf8b5337. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:08.462825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a549d525-0b6e-4020-8419-1e2894db071b tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:08.463562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a549d525-0b6e-4020-8419-1e2894db071b tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:08.476977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68f641-c92b-4098-8a38-b7ebd180a0f5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:08.483561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000e809-00d9-43f7-85dc-f381ea0b87b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:08.484302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000e809-00d9-43f7-85dc-f381ea0b87b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.485352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000e809-00d9-43f7-85dc-f381ea0b87b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.485841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000e809-00d9-43f7-85dc-f381ea0b87b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.486864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000e809-00d9-43f7-85dc-f381ea0b87b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:08.489172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000e809-00d9-43f7-85dc-f381ea0b87b5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:08.501795 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2585/5088] 10.222.0.16 () {58 vars in 1566 bytes} [Sat Sep 3 21:08:08 2022] DELETE /identity/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/groups/7951efaead8c49f9b57b03baaa294bed/roles/f860bfb5a5ef4a8181aa19d07c46813b/inherited_to_projects => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:08.506703 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2504/5089] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:08 2022] GET /identity/v3/roles => generated 1650 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:08.507461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549d525-0b6e-4020-8419-1e2894db071b tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:08.508270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000e809-00d9-43f7-85dc-f381ea0b87b5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:08.510911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9000e809-00d9-43f7-85dc-f381ea0b87b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:08.511878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a549d525-0b6e-4020-8419-1e2894db071b tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:08.527435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000e809-00d9-43f7-85dc-f381ea0b87b5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:08.694808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549d525-0b6e-4020-8419-1e2894db071b tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:08.696252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e211207-1150-4ea7-93f7-d1b636c5df05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:08.696823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e211207-1150-4ea7-93f7-d1b636c5df05 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.697141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e211207-1150-4ea7-93f7-d1b636c5df05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.697420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e211207-1150-4ea7-93f7-d1b636c5df05 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.698094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e211207-1150-4ea7-93f7-d1b636c5df05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:08.699685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e211207-1150-4ea7-93f7-d1b636c5df05 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:08.708305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eb9c5-28a5-4383-bf22-ef66ffc069b1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:08.709232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb9c5-28a5-4383-bf22-ef66ffc069b1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.709587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb9c5-28a5-4383-bf22-ef66ffc069b1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.709978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb9c5-28a5-4383-bf22-ef66ffc069b1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/roles/f860bfb5a5ef4a8181aa19d07c46813b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.712193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1eb9c5-28a5-4383-bf22-ef66ffc069b1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f860bfb5a5ef4a8181aa19d07c46813b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:08.717014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1eb9c5-28a5-4383-bf22-ef66ffc069b1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:08.769131 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2505/5090] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:08 2022] GET /identity/v3/roles => generated 1650 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:08.770073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e211207-1150-4ea7-93f7-d1b636c5df05 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:08.795212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e211207-1150-4ea7-93f7-d1b636c5df05 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:08.815414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e791c-5496-4664-a188-dbd1a75f705b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:08.817692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e791c-5496-4664-a188-dbd1a75f705b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.817991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e791c-5496-4664-a188-dbd1a75f705b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.818275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e791c-5496-4664-a188-dbd1a75f705b admin admin] PATH_INFO: `/v3/projects/b62a52acdf16425fa5d0d91febe2e1f4/users/a202c5c85a1f4c75a8f8b82c34d7e233/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.836408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f1eb9c5-28a5-4383-bf22-ef66ffc069b1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invalidating the token cache because role f860bfb5a5ef4a8181aa19d07c46813b has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:08.836733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f1eb9c5-28a5-4383-bf22-ef66ffc069b1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:08.838227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f1eb9c5-28a5-4383-bf22-ef66ffc069b1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:08.841644 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2586/5091] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:08 2022] DELETE /identity/v3/roles/f860bfb5a5ef4a8181aa19d07c46813b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:08.847001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eb9c5-28a5-4383-bf22-ef66ffc069b1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:08.850218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2f1eb9c5-28a5-4383-bf22-ef66ffc069b1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:08.858162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e791c-5496-4664-a188-dbd1a75f705b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62a52acdf16425fa5d0d91febe2e1f4, user_id=a202c5c85a1f4c75a8f8b82c34d7e233, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:08.865979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e791c-5496-4664-a188-dbd1a75f705b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:08.898287 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2506/5092] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:08 2022] PUT /identity/v3/projects/b62a52acdf16425fa5d0d91febe2e1f4/users/a202c5c85a1f4c75a8f8b82c34d7e233/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:08.909263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de7744-013c-42bb-b791-0e9678e24aff None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:08.909647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de7744-013c-42bb-b791-0e9678e24aff None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:08.910030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de7744-013c-42bb-b791-0e9678e24aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:08.979702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d0de7744-013c-42bb-b791-0e9678e24aff None None] MFA Rules not processed for user `a202c5c85a1f4c75a8f8b82c34d7e233`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:08.981255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d0de7744-013c-42bb-b791-0e9678e24aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:08.992549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eb9c5-28a5-4383-bf22-ef66ffc069b1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:09.046595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:09.121021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cddf0-a051-42e1-aa3a-8866107e3b6c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:09.121021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cddf0-a051-42e1-aa3a-8866107e3b6c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:09.121021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cddf0-a051-42e1-aa3a-8866107e3b6c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:09.121416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cddf0-a051-42e1-aa3a-8866107e3b6c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:09.124535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cddf0-a051-42e1-aa3a-8866107e3b6c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:09.126245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cddf0-a051-42e1-aa3a-8866107e3b6c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:09.161980 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2587/5093] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:08:08 2022] POST /identity/v3/roles => generated 239 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:09.162926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cddf0-a051-42e1-aa3a-8866107e3b6c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:09.184582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cddf0-a051-42e1-aa3a-8866107e3b6c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:09.187666 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2507/5094] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:08 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:09.203565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccace51-b4a0-47d2-aafd-c53d13f63995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:09.204040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de7744-013c-42bb-b791-0e9678e24aff None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:09.204293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccace51-b4a0-47d2-aafd-c53d13f63995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:09.204718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccace51-b4a0-47d2-aafd-c53d13f63995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:09.205127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccace51-b4a0-47d2-aafd-c53d13f63995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/users/b8b9d276657d4d65a6f5281e88d61c76/roles/3181e952eec94fa886fabb1010916a0d/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:09.205693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d0de7744-013c-42bb-b791-0e9678e24aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:09.209272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccace51-b4a0-47d2-aafd-c53d13f63995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=624a4f5ac1354a71bf67dccccf8b5337, user_id=b8b9d276657d4d65a6f5281e88d61c76, role_id=3181e952eec94fa886fabb1010916a0d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:09.214248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccace51-b4a0-47d2-aafd-c53d13f63995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:09.255268 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2588/5095] 10.222.0.16 () {60 vars in 1579 bytes} [Sat Sep 3 21:08:09 2022] PUT /identity/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/users/b8b9d276657d4d65a6f5281e88d61c76/roles/3181e952eec94fa886fabb1010916a0d/inherited_to_projects => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:09.259036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccace51-b4a0-47d2-aafd-c53d13f63995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:09.267942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:09.369828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de7744-013c-42bb-b791-0e9678e24aff None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:09.448164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccace51-b4a0-47d2-aafd-c53d13f63995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:09.464389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50207957-c4ed-42cd-bbe0-c7004b21f0f9 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:09.465330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50207957-c4ed-42cd-bbe0-c7004b21f0f9 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:09.466382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50207957-c4ed-42cd-bbe0-c7004b21f0f9 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:09.467131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50207957-c4ed-42cd-bbe0-c7004b21f0f9 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/users/b8b9d276657d4d65a6f5281e88d61c76/roles/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:09.471223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50207957-c4ed-42cd-bbe0-c7004b21f0f9 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=624a4f5ac1354a71bf67dccccf8b5337, user_id=b8b9d276657d4d65a6f5281e88d61c76)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:09.474949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50207957-c4ed-42cd-bbe0-c7004b21f0f9 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:09.476383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660e495-fd03-4d24-aa09-75a2f5e24b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:09.477204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660e495-fd03-4d24-aa09-75a2f5e24b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:09.477587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660e495-fd03-4d24-aa09-75a2f5e24b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:09.477968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660e495-fd03-4d24-aa09-75a2f5e24b3b admin admin] PATH_INFO: `/v3/users/a202c5c85a1f4c75a8f8b82c34d7e233` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:09.483199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660e495-fd03-4d24-aa09-75a2f5e24b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a202c5c85a1f4c75a8f8b82c34d7e233)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:09.487104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660e495-fd03-4d24-aa09-75a2f5e24b3b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:09.508330 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2589/5096] 10.222.0.16 () {58 vars in 1461 bytes} [Sat Sep 3 21:08:09 2022] GET /identity/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/users/b8b9d276657d4d65a6f5281e88d61c76/roles/inherited_to_projects => generated 446 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:09.513187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50207957-c4ed-42cd-bbe0-c7004b21f0f9 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:09.555642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50207957-c4ed-42cd-bbe0-c7004b21f0f9 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:09.581586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a0162-d9b7-448f-a04e-c67c7f61e326 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:09.582770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1660e495-fd03-4d24-aa09-75a2f5e24b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a202c5c85a1f4c75a8f8b82c34d7e233'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:09.584452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a0162-d9b7-448f-a04e-c67c7f61e326 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:09.584752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a0162-d9b7-448f-a04e-c67c7f61e326 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:09.585143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a0162-d9b7-448f-a04e-c67c7f61e326 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/users/b8b9d276657d4d65a6f5281e88d61c76/roles/3181e952eec94fa886fabb1010916a0d/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:09.594669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a0162-d9b7-448f-a04e-c67c7f61e326 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=624a4f5ac1354a71bf67dccccf8b5337, user_id=b8b9d276657d4d65a6f5281e88d61c76, role_id=3181e952eec94fa886fabb1010916a0d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:09.598082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a0162-d9b7-448f-a04e-c67c7f61e326 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:09.600197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1660e495-fd03-4d24-aa09-75a2f5e24b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a202c5c85a1f4c75a8f8b82c34d7e233'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:09.601489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1660e495-fd03-4d24-aa09-75a2f5e24b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a202c5c85a1f4c75a8f8b82c34d7e233'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:09.611162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1660e495-fd03-4d24-aa09-75a2f5e24b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a202c5c85a1f4c75a8f8b82c34d7e233'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:09.623195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1660e495-fd03-4d24-aa09-75a2f5e24b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a202c5c85a1f4c75a8f8b82c34d7e233'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:09.624192 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2590/5097] 10.222.0.16 () {58 vars in 1561 bytes} [Sat Sep 3 21:08:09 2022] HEAD /identity/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/users/b8b9d276657d4d65a6f5281e88d61c76/roles/3181e952eec94fa886fabb1010916a0d/inherited_to_projects => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:09.631295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a0162-d9b7-448f-a04e-c67c7f61e326 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:09.633745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3b8a0162-d9b7-448f-a04e-c67c7f61e326 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:09.658359 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2508/5098] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:09 2022] DELETE /identity/v3/users/a202c5c85a1f4c75a8f8b82c34d7e233 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:09.665641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba8130-3dba-44e7-a317-e675a77d19c0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:09.665929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba8130-3dba-44e7-a317-e675a77d19c0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:09.666208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba8130-3dba-44e7-a317-e675a77d19c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:09.687022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a0162-d9b7-448f-a04e-c67c7f61e326 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:09.746169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d1ba8130-3dba-44e7-a317-e675a77d19c0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:09.747412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d1ba8130-3dba-44e7-a317-e675a77d19c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:09.877594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707fb0a8-d720-4b60-8ed9-e6aec445bef7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:09.878355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fb0a8-d720-4b60-8ed9-e6aec445bef7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:09.878705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fb0a8-d720-4b60-8ed9-e6aec445bef7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:09.879050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fb0a8-d720-4b60-8ed9-e6aec445bef7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/users/b8b9d276657d4d65a6f5281e88d61c76/roles/3181e952eec94fa886fabb1010916a0d/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:09.881479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707fb0a8-d720-4b60-8ed9-e6aec445bef7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=624a4f5ac1354a71bf67dccccf8b5337, user_id=b8b9d276657d4d65a6f5281e88d61c76, role_id=3181e952eec94fa886fabb1010916a0d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:09.894915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707fb0a8-d720-4b60-8ed9-e6aec445bef7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:09.907849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-707fb0a8-d720-4b60-8ed9-e6aec445bef7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invalidating the token cache because role 3181e952eec94fa886fabb1010916a0d was removed from user b8b9d276657d4d65a6f5281e88d61c76 on domain 624a4f5ac1354a71bf67dccccf8b5337. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:09.908153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-707fb0a8-d720-4b60-8ed9-e6aec445bef7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:09.909273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-707fb0a8-d720-4b60-8ed9-e6aec445bef7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:09.913498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:09.947649 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2509/5099] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:09.951350 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2591/5100] 10.222.0.16 () {58 vars in 1563 bytes} [Sat Sep 3 21:08:09 2022] DELETE /identity/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/users/b8b9d276657d4d65a6f5281e88d61c76/roles/3181e952eec94fa886fabb1010916a0d/inherited_to_projects => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:09.955655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba8130-3dba-44e7-a317-e675a77d19c0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:09.958531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d1ba8130-3dba-44e7-a317-e675a77d19c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:09.962539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707fb0a8-d720-4b60-8ed9-e6aec445bef7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:09.965817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-707fb0a8-d720-4b60-8ed9-e6aec445bef7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:09.988810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba8130-3dba-44e7-a317-e675a77d19c0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:10.038039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:10.094884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707fb0a8-d720-4b60-8ed9-e6aec445bef7 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:10.100649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5953f0b-152b-417f-94b7-f889219b7753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:10.102135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5953f0b-152b-417f-94b7-f889219b7753 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:10.102613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5953f0b-152b-417f-94b7-f889219b7753 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:10.102991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5953f0b-152b-417f-94b7-f889219b7753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:10.108056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdaca0-9036-44b8-ac21-0854c4274379 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:10.109624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdaca0-9036-44b8-ac21-0854c4274379 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:10.110201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdaca0-9036-44b8-ac21-0854c4274379 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:10.112529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdaca0-9036-44b8-ac21-0854c4274379 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/roles/3181e952eec94fa886fabb1010916a0d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:10.115848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdaca0-9036-44b8-ac21-0854c4274379 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:delete_role(role_id=3181e952eec94fa886fabb1010916a0d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:10.122891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdaca0-9036-44b8-ac21-0854c4274379 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:10.183739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-becdaca0-9036-44b8-ac21-0854c4274379 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invalidating the token cache because role 3181e952eec94fa886fabb1010916a0d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:10.186621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-becdaca0-9036-44b8-ac21-0854c4274379 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:10.188721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-becdaca0-9036-44b8-ac21-0854c4274379 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:10.192178 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2510/5101] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:09 2022] DELETE /identity/v3/roles/3181e952eec94fa886fabb1010916a0d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:10.198340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdaca0-9036-44b8-ac21-0854c4274379 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:10.202376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-becdaca0-9036-44b8-ac21-0854c4274379 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:10.229812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5953f0b-152b-417f-94b7-f889219b7753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:10.234316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5953f0b-152b-417f-94b7-f889219b7753 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:10.264320 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2592/5102] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:10.323572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdaca0-9036-44b8-ac21-0854c4274379 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:10.331285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500d49f-be37-4c71-b6a8-10c66f9c2ebc tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:10.333319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500d49f-be37-4c71-b6a8-10c66f9c2ebc tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:10.333809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500d49f-be37-4c71-b6a8-10c66f9c2ebc tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:10.334673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500d49f-be37-4c71-b6a8-10c66f9c2ebc tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:10.336075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500d49f-be37-4c71-b6a8-10c66f9c2ebc tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:10.338675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500d49f-be37-4c71-b6a8-10c66f9c2ebc tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:10.362432 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2511/5103] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:08:10 2022] POST /identity/v3/roles => generated 240 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:10.366821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5953f0b-152b-417f-94b7-f889219b7753 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:10.378627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5953f0b-152b-417f-94b7-f889219b7753 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:10.384180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f416075-ba19-49e6-84b9-698bba44cfe1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:10.384639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f416075-ba19-49e6-84b9-698bba44cfe1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:10.385097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f416075-ba19-49e6-84b9-698bba44cfe1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:10.385467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f416075-ba19-49e6-84b9-698bba44cfe1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:10.386267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f416075-ba19-49e6-84b9-698bba44cfe1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:10.388489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f416075-ba19-49e6-84b9-698bba44cfe1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:10.468598 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2593/5104] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:10 2022] POST /identity/v3/projects => generated 409 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:10.474303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500d49f-be37-4c71-b6a8-10c66f9c2ebc tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:10.489047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500d49f-be37-4c71-b6a8-10c66f9c2ebc tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:10.602567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:10.625162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd648549-5869-41ec-9772-06b9eeff0040 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:10.625874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd648549-5869-41ec-9772-06b9eeff0040 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:10.626167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd648549-5869-41ec-9772-06b9eeff0040 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:10.626734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd648549-5869-41ec-9772-06b9eeff0040 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/users/b8b9d276657d4d65a6f5281e88d61c76/roles/b7b9f88f64e64f43bcc0da30ba4e9634/inherited_to_projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:10.629863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd648549-5869-41ec-9772-06b9eeff0040 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=624a4f5ac1354a71bf67dccccf8b5337, user_id=b8b9d276657d4d65a6f5281e88d61c76, role_id=b7b9f88f64e64f43bcc0da30ba4e9634)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:10.632636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd648549-5869-41ec-9772-06b9eeff0040 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:10.650601 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2512/5105] 10.222.0.16 () {60 vars in 1579 bytes} [Sat Sep 3 21:08:10 2022] PUT /identity/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/users/b8b9d276657d4d65a6f5281e88d61c76/roles/b7b9f88f64e64f43bcc0da30ba4e9634/inherited_to_projects => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:10.654587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f416075-ba19-49e6-84b9-698bba44cfe1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:10.666654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f416075-ba19-49e6-84b9-698bba44cfe1 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:10.749800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd648549-5869-41ec-9772-06b9eeff0040 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:10.751920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fd648549-5869-41ec-9772-06b9eeff0040 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:10.785898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee41b8-f129-4974-85ac-91637fab2559 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:10.787146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee41b8-f129-4974-85ac-91637fab2559 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:10.787570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee41b8-f129-4974-85ac-91637fab2559 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:10.787956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee41b8-f129-4974-85ac-91637fab2559 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:10.789033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee41b8-f129-4974-85ac-91637fab2559 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Adding query filter params (scope.project.id=7e4a23fa192e47d2aacabb831375ddaa, user.id=b8b9d276657d4d65a6f5281e88d61c76) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:10.789428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee41b8-f129-4974-85ac-91637fab2559 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:10.790958 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-13ee41b8-f129-4974-85ac-91637fab2559 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:10.790958 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:10.790958 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:10.792563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee41b8-f129-4974-85ac-91637fab2559 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:10.865965 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2594/5106] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Sep 3 21:08:10 2022] GET /identity/v3/role_assignments?scope.project.id=7e4a23fa192e47d2aacabb831375ddaa&user.id=b8b9d276657d4d65a6f5281e88d61c76&effective => generated 648 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:10.869966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee41b8-f129-4974-85ac-91637fab2559 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:10.889445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd648549-5869-41ec-9772-06b9eeff0040 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:10.896475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d2afe-a405-4e9c-9920-0b55bcafc529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:10.897151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d2afe-a405-4e9c-9920-0b55bcafc529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:10.897434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d2afe-a405-4e9c-9920-0b55bcafc529 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:10.898139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d2afe-a405-4e9c-9920-0b55bcafc529 admin admin] PATH_INFO: `/v3/projects/b62a52acdf16425fa5d0d91febe2e1f4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:10.899118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d2afe-a405-4e9c-9920-0b55bcafc529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b62a52acdf16425fa5d0d91febe2e1f4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:10.901709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d2afe-a405-4e9c-9920-0b55bcafc529 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:10.902592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee41b8-f129-4974-85ac-91637fab2559 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:10.914151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb931b-6531-4f4f-aede-9ce189a0820c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:10.917290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb931b-6531-4f4f-aede-9ce189a0820c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:10.917706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb931b-6531-4f4f-aede-9ce189a0820c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:10.917999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb931b-6531-4f4f-aede-9ce189a0820c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:10.918840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb931b-6531-4f4f-aede-9ce189a0820c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Adding query filter params (scope.project.id=896970c2a60e481eaab1e47251a638fa, user.id=b8b9d276657d4d65a6f5281e88d61c76) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:10.919163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb931b-6531-4f4f-aede-9ce189a0820c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:10.921582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb931b-6531-4f4f-aede-9ce189a0820c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:10.923817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:11.019418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0c6d2afe-a405-4e9c-9920-0b55bcafc529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b62a52acdf16425fa5d0d91febe2e1f4', 'request_id': 'req-0c6d2afe-a405-4e9c-9920-0b55bcafc529'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:11.035886 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2595/5107] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Sep 3 21:08:10 2022] GET /identity/v3/role_assignments?scope.project.id=896970c2a60e481eaab1e47251a638fa&user.id=b8b9d276657d4d65a6f5281e88d61c76&effective => generated 648 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:11.040160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0c6d2afe-a405-4e9c-9920-0b55bcafc529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b62a52acdf16425fa5d0d91febe2e1f4', 'request_id': 'req-0c6d2afe-a405-4e9c-9920-0b55bcafc529'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:11.040413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb931b-6531-4f4f-aede-9ce189a0820c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:11.049909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0c6d2afe-a405-4e9c-9920-0b55bcafc529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b62a52acdf16425fa5d0d91febe2e1f4', 'request_id': 'req-0c6d2afe-a405-4e9c-9920-0b55bcafc529'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:11.061436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb931b-6531-4f4f-aede-9ce189a0820c tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:11.070781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0c6d2afe-a405-4e9c-9920-0b55bcafc529 admin admin] The token cache is being invalidate because project b62a52acdf16425fa5d0d91febe2e1f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:11.071168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0c6d2afe-a405-4e9c-9920-0b55bcafc529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:11.071885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0c6d2afe-a405-4e9c-9920-0b55bcafc529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:11.083448 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2513/5108] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:10 2022] DELETE /identity/v3/projects/b62a52acdf16425fa5d0d91febe2e1f4 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:11.085194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51961070-ff6d-42c2-b0c0-8c4c48a17e89 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.085516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51961070-ff6d-42c2-b0c0-8c4c48a17e89 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.085776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51961070-ff6d-42c2-b0c0-8c4c48a17e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.110954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-51961070-ff6d-42c2-b0c0-8c4c48a17e89 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:11.112647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-51961070-ff6d-42c2-b0c0-8c4c48a17e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:11.170832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d7d8b-9195-4c8d-ba44-b4911b9f3da6 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:11.171529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d7d8b-9195-4c8d-ba44-b4911b9f3da6 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.171824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d7d8b-9195-4c8d-ba44-b4911b9f3da6 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.172114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d7d8b-9195-4c8d-ba44-b4911b9f3da6 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/users/b8b9d276657d4d65a6f5281e88d61c76/roles/b7b9f88f64e64f43bcc0da30ba4e9634/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.177062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d7d8b-9195-4c8d-ba44-b4911b9f3da6 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=624a4f5ac1354a71bf67dccccf8b5337, user_id=b8b9d276657d4d65a6f5281e88d61c76, role_id=b7b9f88f64e64f43bcc0da30ba4e9634)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:11.179773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d7d8b-9195-4c8d-ba44-b4911b9f3da6 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:11.185787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d34d7d8b-9195-4c8d-ba44-b4911b9f3da6 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invalidating the token cache because role b7b9f88f64e64f43bcc0da30ba4e9634 was removed from user b8b9d276657d4d65a6f5281e88d61c76 on domain 624a4f5ac1354a71bf67dccccf8b5337. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:11.186091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d34d7d8b-9195-4c8d-ba44-b4911b9f3da6 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:11.186816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d34d7d8b-9195-4c8d-ba44-b4911b9f3da6 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:11.200514 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2514/5109] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:11.206808 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2596/5110] 10.222.0.16 () {58 vars in 1563 bytes} [Sat Sep 3 21:08:11 2022] DELETE /identity/v3/OS-INHERIT/domains/624a4f5ac1354a71bf67dccccf8b5337/users/b8b9d276657d4d65a6f5281e88d61c76/roles/b7b9f88f64e64f43bcc0da30ba4e9634/inherited_to_projects => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:11.209504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec1f0b-7fc7-4f21-abc8-4761e59df0eb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.209684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51961070-ff6d-42c2-b0c0-8c4c48a17e89 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:11.210343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec1f0b-7fc7-4f21-abc8-4761e59df0eb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.210928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec1f0b-7fc7-4f21-abc8-4761e59df0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.213063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-51961070-ff6d-42c2-b0c0-8c4c48a17e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:11.231307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51961070-ff6d-42c2-b0c0-8c4c48a17e89 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:11.239373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-48ec1f0b-7fc7-4f21-abc8-4761e59df0eb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:11.240848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-48ec1f0b-7fc7-4f21-abc8-4761e59df0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:11.252742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:11.324885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c046839-11fb-4569-aea9-910da5f77c00 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:11.325751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c046839-11fb-4569-aea9-910da5f77c00 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.326042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c046839-11fb-4569-aea9-910da5f77c00 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.328072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c046839-11fb-4569-aea9-910da5f77c00 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.328886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c046839-11fb-4569-aea9-910da5f77c00 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Adding query filter params (scope.project.id=7e4a23fa192e47d2aacabb831375ddaa, user.id=b8b9d276657d4d65a6f5281e88d61c76) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:11.329387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c046839-11fb-4569-aea9-910da5f77c00 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:11.331846 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-9c046839-11fb-4569-aea9-910da5f77c00 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:11.331846 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:11.331846 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:11.333578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c046839-11fb-4569-aea9-910da5f77c00 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:11.336468 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2597/5111] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:11.351775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec1f0b-7fc7-4f21-abc8-4761e59df0eb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:11.365473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec1f0b-7fc7-4f21-abc8-4761e59df0eb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:11.372010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee8899-1ed4-42f9-b6ce-54298063eb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:11.372889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee8899-1ed4-42f9-b6ce-54298063eb35 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.373209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee8899-1ed4-42f9-b6ce-54298063eb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.373495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee8899-1ed4-42f9-b6ce-54298063eb35 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.374192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee8899-1ed4-42f9-b6ce-54298063eb35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:11.374508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee8899-1ed4-42f9-b6ce-54298063eb35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:11.376135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee8899-1ed4-42f9-b6ce-54298063eb35 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:11.396233 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2515/5112] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Sep 3 21:08:11 2022] GET /identity/v3/role_assignments?scope.project.id=7e4a23fa192e47d2aacabb831375ddaa&user.id=b8b9d276657d4d65a6f5281e88d61c76&effective => generated 228 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:11.396520 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2598/5113] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:11.397377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c046839-11fb-4569-aea9-910da5f77c00 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:11.400462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee8899-1ed4-42f9-b6ce-54298063eb35 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:11.412940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c046839-11fb-4569-aea9-910da5f77c00 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:11.413728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee8899-1ed4-42f9-b6ce-54298063eb35 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:11.418855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033e8e8-9e6f-4dc7-8fbf-b874306637c2 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:11.419563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033e8e8-9e6f-4dc7-8fbf-b874306637c2 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.419923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033e8e8-9e6f-4dc7-8fbf-b874306637c2 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.420284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033e8e8-9e6f-4dc7-8fbf-b874306637c2 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.421148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033e8e8-9e6f-4dc7-8fbf-b874306637c2 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Adding query filter params (scope.project.id=896970c2a60e481eaab1e47251a638fa, user.id=b8b9d276657d4d65a6f5281e88d61c76) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:11.421563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033e8e8-9e6f-4dc7-8fbf-b874306637c2 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:11.422416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3e880-30a0-413b-aba3-ec0147e9d297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:11.423309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3e880-30a0-413b-aba3-ec0147e9d297 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.423659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033e8e8-9e6f-4dc7-8fbf-b874306637c2 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:11.424151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3e880-30a0-413b-aba3-ec0147e9d297 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.424619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3e880-30a0-413b-aba3-ec0147e9d297 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.425840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3e880-30a0-413b-aba3-ec0147e9d297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:11.428318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3e880-30a0-413b-aba3-ec0147e9d297 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:11.457642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:11.474759 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2516/5114] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:11 2022] POST /identity/v3/projects => generated 367 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:11.477628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3e880-30a0-413b-aba3-ec0147e9d297 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:11.519874 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2599/5115] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Sep 3 21:08:11 2022] GET /identity/v3/role_assignments?scope.project.id=896970c2a60e481eaab1e47251a638fa&user.id=b8b9d276657d4d65a6f5281e88d61c76&effective => generated 228 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:11.520802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033e8e8-9e6f-4dc7-8fbf-b874306637c2 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:11.543461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033e8e8-9e6f-4dc7-8fbf-b874306637c2 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:11.577360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3e880-30a0-413b-aba3-ec0147e9d297 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:11.585455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2c9c9-b515-485f-8633-70ac524f0a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:11.585909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2c9c9-b515-485f-8633-70ac524f0a70 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.586199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2c9c9-b515-485f-8633-70ac524f0a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.586537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2c9c9-b515-485f-8633-70ac524f0a70 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.587192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2c9c9-b515-485f-8633-70ac524f0a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:11.589400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2c9c9-b515-485f-8633-70ac524f0a70 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:11.623334 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2517/5116] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:11 2022] POST /identity/v3/users => generated 284 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:11.625978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2c9c9-b515-485f-8633-70ac524f0a70 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:11.640641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2c9c9-b515-485f-8633-70ac524f0a70 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:11.648533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa306667-8ef7-4b01-b2f2-9c91914afc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:11.649465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa306667-8ef7-4b01-b2f2-9c91914afc4f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.649891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa306667-8ef7-4b01-b2f2-9c91914afc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.650244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa306667-8ef7-4b01-b2f2-9c91914afc4f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.650407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:11.651042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.651154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa306667-8ef7-4b01-b2f2-9c91914afc4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:11.651323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.651695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/projects/896970c2a60e481eaab1e47251a638fa` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.653623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:delete_project(project_id=896970c2a60e481eaab1e47251a638fa)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:11.654793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa306667-8ef7-4b01-b2f2-9c91914afc4f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:11.658233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:11.679989 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2518/5117] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:11 2022] GET /identity/v3/roles => generated 1651 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:11.697602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa306667-8ef7-4b01-b2f2-9c91914afc4f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:11.720807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa306667-8ef7-4b01-b2f2-9c91914afc4f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:11.749591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896970c2a60e481eaab1e47251a638fa', 'request_id': 'req-2f57d398-57ed-475b-8eff-8aaa2b783ccb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:11.761059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896970c2a60e481eaab1e47251a638fa', 'request_id': 'req-2f57d398-57ed-475b-8eff-8aaa2b783ccb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:11.775570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896970c2a60e481eaab1e47251a638fa', 'request_id': 'req-2f57d398-57ed-475b-8eff-8aaa2b783ccb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:11.808186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] The token cache is being invalidate because project 896970c2a60e481eaab1e47251a638fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:11.808659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:11.809079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:11.809823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:11.811578 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2600/5118] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:11 2022] DELETE /identity/v3/projects/896970c2a60e481eaab1e47251a638fa => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:11.816871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:11.818373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:11.879745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eab56c-2e91-4197-976c-4cf7a4172763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:11.881554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eab56c-2e91-4197-976c-4cf7a4172763 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.881965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eab56c-2e91-4197-976c-4cf7a4172763 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.882355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eab56c-2e91-4197-976c-4cf7a4172763 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.884049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eab56c-2e91-4197-976c-4cf7a4172763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:11.887520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eab56c-2e91-4197-976c-4cf7a4172763 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:11.901706 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2519/5119] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:11 2022] GET /identity/v3/roles => generated 1651 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:11.906032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eab56c-2e91-4197-976c-4cf7a4172763 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:11.908676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-54eab56c-2e91-4197-976c-4cf7a4172763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:11.927008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eab56c-2e91-4197-976c-4cf7a4172763 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:11.933116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d86685-c093-4a68-85d9-7bc2fcefd394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:11.933965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d86685-c093-4a68-85d9-7bc2fcefd394 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.934359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d86685-c093-4a68-85d9-7bc2fcefd394 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.934743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d86685-c093-4a68-85d9-7bc2fcefd394 admin admin] PATH_INFO: `/v3/projects/23cf0acc0a814fb59b73fe5066fb384c/users/938351e16aaf43f896f06c7ffdb18154/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.964034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57d398-57ed-475b-8eff-8aaa2b783ccb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:11.973101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d86685-c093-4a68-85d9-7bc2fcefd394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23cf0acc0a814fb59b73fe5066fb384c, user_id=938351e16aaf43f896f06c7ffdb18154, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:11.976589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e32df5-662a-418b-b607-1f2413160cde tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:11.979555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e32df5-662a-418b-b607-1f2413160cde tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:11.979914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e32df5-662a-418b-b607-1f2413160cde tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:11.980266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e32df5-662a-418b-b607-1f2413160cde tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/roles/b7b9f88f64e64f43bcc0da30ba4e9634` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:11.983241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d86685-c093-4a68-85d9-7bc2fcefd394 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:11.987221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e32df5-662a-418b-b607-1f2413160cde tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b7b9f88f64e64f43bcc0da30ba4e9634)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:11.993474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e32df5-662a-418b-b607-1f2413160cde tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:12.022453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:12.022778 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2520/5120] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:11 2022] PUT /identity/v3/projects/23cf0acc0a814fb59b73fe5066fb384c/users/938351e16aaf43f896f06c7ffdb18154/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:12.033999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b73f6c-b6be-4b28-901f-f72978bf9e8c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:12.034538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b73f6c-b6be-4b28-901f-f72978bf9e8c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:12.035577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b73f6c-b6be-4b28-901f-f72978bf9e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:12.076867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93e32df5-662a-418b-b607-1f2413160cde tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invalidating the token cache because role b7b9f88f64e64f43bcc0da30ba4e9634 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:12.077960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93e32df5-662a-418b-b607-1f2413160cde tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:12.078896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93e32df5-662a-418b-b607-1f2413160cde tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:12.083568 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2601/5121] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:11 2022] DELETE /identity/v3/roles/b7b9f88f64e64f43bcc0da30ba4e9634 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:12.088809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e32df5-662a-418b-b607-1f2413160cde tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:12.091387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-93e32df5-662a-418b-b607-1f2413160cde tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:12.115337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a1b73f6c-b6be-4b28-901f-f72978bf9e8c None None] MFA Rules not processed for user `938351e16aaf43f896f06c7ffdb18154`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:12.117115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a1b73f6c-b6be-4b28-901f-f72978bf9e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:12.230660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e32df5-662a-418b-b607-1f2413160cde tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:12.247337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c83cab-8803-4db6-bf8a-45a7b3d256da tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:12.247984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c83cab-8803-4db6-bf8a-45a7b3d256da tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:12.248272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c83cab-8803-4db6-bf8a-45a7b3d256da tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:12.248553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c83cab-8803-4db6-bf8a-45a7b3d256da tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:12.249300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c83cab-8803-4db6-bf8a-45a7b3d256da tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:12.250944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c83cab-8803-4db6-bf8a-45a7b3d256da tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:12.256518 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2521/5122] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:12.258250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b73f6c-b6be-4b28-901f-f72978bf9e8c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:12.260241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a1b73f6c-b6be-4b28-901f-f72978bf9e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:12.294868 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2602/5123] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:08:12 2022] POST /identity/v3/roles => generated 239 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:12.306417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c83cab-8803-4db6-bf8a-45a7b3d256da tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:12.318964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c83cab-8803-4db6-bf8a-45a7b3d256da tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:12.325743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82c2cb-e83b-463a-a4f4-93441531de5d tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:12.326375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82c2cb-e83b-463a-a4f4-93441531de5d tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:12.326697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82c2cb-e83b-463a-a4f4-93441531de5d tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:12.326959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82c2cb-e83b-463a-a4f4-93441531de5d tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:12.327587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82c2cb-e83b-463a-a4f4-93441531de5d tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:12.331196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82c2cb-e83b-463a-a4f4-93441531de5d tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:12.384702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b73f6c-b6be-4b28-901f-f72978bf9e8c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:12.395242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cba49-5a3e-4b9f-8a1f-7b14584f27ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:12.396389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cba49-5a3e-4b9f-8a1f-7b14584f27ee admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:12.396916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cba49-5a3e-4b9f-8a1f-7b14584f27ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:12.397422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cba49-5a3e-4b9f-8a1f-7b14584f27ee admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:12.398447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978cba49-5a3e-4b9f-8a1f-7b14584f27ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:12.401804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978cba49-5a3e-4b9f-8a1f-7b14584f27ee admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:12.415903 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2603/5124] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:12 2022] POST /identity/v3/projects => generated 409 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:12.420091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82c2cb-e83b-463a-a4f4-93441531de5d tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:12.490284 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2522/5125] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:12 2022] POST /identity/v3/projects => generated 367 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:12.496425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cba49-5a3e-4b9f-8a1f-7b14584f27ee admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:12.530573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82c2cb-e83b-463a-a4f4-93441531de5d tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:12.530795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:12.587283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cba49-5a3e-4b9f-8a1f-7b14584f27ee admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:12.599695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b96e17-0c95-40bf-bac5-74c9bfa5a2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:12.600508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b96e17-0c95-40bf-bac5-74c9bfa5a2ce admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:12.600945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b96e17-0c95-40bf-bac5-74c9bfa5a2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:12.601221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b96e17-0c95-40bf-bac5-74c9bfa5a2ce admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:12.606687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b96e17-0c95-40bf-bac5-74c9bfa5a2ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:12.607799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b96e17-0c95-40bf-bac5-74c9bfa5a2ce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:12.611618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:12.638148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a7321-fd29-443b-9826-a105ced7b995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:12.639074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a7321-fd29-443b-9826-a105ced7b995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:12.639372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a7321-fd29-443b-9826-a105ced7b995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:12.640118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a7321-fd29-443b-9826-a105ced7b995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/users/b8b9d276657d4d65a6f5281e88d61c76/roles/14998af1b3c44044902569bf61eba98f/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:12.642230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a7321-fd29-443b-9826-a105ced7b995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7e4a23fa192e47d2aacabb831375ddaa, user_id=b8b9d276657d4d65a6f5281e88d61c76, role_id=14998af1b3c44044902569bf61eba98f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:12.645602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a7321-fd29-443b-9826-a105ced7b995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:12.652266 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2523/5126] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:12 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:12.656414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b96e17-0c95-40bf-bac5-74c9bfa5a2ce admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:12.670016 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2604/5127] 10.222.0.16 () {60 vars in 1582 bytes} [Sat Sep 3 21:08:12 2022] PUT /identity/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/users/b8b9d276657d4d65a6f5281e88d61c76/roles/14998af1b3c44044902569bf61eba98f/inherited_to_projects => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:12.674280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b96e17-0c95-40bf-bac5-74c9bfa5a2ce admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:12.675900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a7321-fd29-443b-9826-a105ced7b995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:12.769648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a7321-fd29-443b-9826-a105ced7b995 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:12.775878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2ebc4-0aff-48b0-b301-208f4e27ac5e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:12.776509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2ebc4-0aff-48b0-b301-208f4e27ac5e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:12.776804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2ebc4-0aff-48b0-b301-208f4e27ac5e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:12.777140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2ebc4-0aff-48b0-b301-208f4e27ac5e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:12.777884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2ebc4-0aff-48b0-b301-208f4e27ac5e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Adding query filter params (scope.project.id=0c6a7ed653b8410b8b0df87d79d49a54, user.id=b8b9d276657d4d65a6f5281e88d61c76) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:12.778011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd92687-e7dd-4d68-ab53-7aee4f4ca3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:12.778200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2ebc4-0aff-48b0-b301-208f4e27ac5e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:12.780161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2ebc4-0aff-48b0-b301-208f4e27ac5e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:12.781483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd92687-e7dd-4d68-ab53-7aee4f4ca3da admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:12.781841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd92687-e7dd-4d68-ab53-7aee4f4ca3da admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:12.782253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd92687-e7dd-4d68-ab53-7aee4f4ca3da admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:12.782934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd92687-e7dd-4d68-ab53-7aee4f4ca3da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:12.787786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd92687-e7dd-4d68-ab53-7aee4f4ca3da admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:12.802186 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2524/5128] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:12 2022] GET /identity/v3/roles => generated 1650 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:12.806630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd92687-e7dd-4d68-ab53-7aee4f4ca3da admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:12.819857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd92687-e7dd-4d68-ab53-7aee4f4ca3da admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:12.826646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d223d9-364c-49d5-b6a5-1b233b0e41aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:12.827335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d223d9-364c-49d5-b6a5-1b233b0e41aa admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:12.827623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d223d9-364c-49d5-b6a5-1b233b0e41aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:12.827904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d223d9-364c-49d5-b6a5-1b233b0e41aa admin admin] PATH_INFO: `/v3/projects/c006ac088e114f36b850825686bde529/users/fea51b110f934255bc2e11b76760a730/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:12.848350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d223d9-364c-49d5-b6a5-1b233b0e41aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c006ac088e114f36b850825686bde529, user_id=fea51b110f934255bc2e11b76760a730, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:12.852149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d223d9-364c-49d5-b6a5-1b233b0e41aa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:12.870424 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2525/5129] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:12 2022] PUT /identity/v3/projects/c006ac088e114f36b850825686bde529/users/fea51b110f934255bc2e11b76760a730/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:12.874949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d223d9-364c-49d5-b6a5-1b233b0e41aa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:12.900438 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2605/5130] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Sep 3 21:08:12 2022] GET /identity/v3/role_assignments?scope.project.id=0c6a7ed653b8410b8b0df87d79d49a54&user.id=b8b9d276657d4d65a6f5281e88d61c76&effective => generated 649 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:12.905765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2ebc4-0aff-48b0-b301-208f4e27ac5e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:12.938517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2ebc4-0aff-48b0-b301-208f4e27ac5e tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:12.994180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d223d9-364c-49d5-b6a5-1b233b0e41aa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:13.000947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0e6f8-599c-4fd7-88a3-80c0e95980d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:13.001875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0e6f8-599c-4fd7-88a3-80c0e95980d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:13.002185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0e6f8-599c-4fd7-88a3-80c0e95980d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:13.002467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0e6f8-599c-4fd7-88a3-80c0e95980d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:13.003128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0e6f8-599c-4fd7-88a3-80c0e95980d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:13.007397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0e6f8-599c-4fd7-88a3-80c0e95980d8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:13.034917 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2526/5131] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:12 2022] GET /identity/v3/roles => generated 1650 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:13.036321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0e6f8-599c-4fd7-88a3-80c0e95980d8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:13.046507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a138c-c21f-4bc9-88c1-85e0d94f7963 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:13.047346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a138c-c21f-4bc9-88c1-85e0d94f7963 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:13.048354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a138c-c21f-4bc9-88c1-85e0d94f7963 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:13.048782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a138c-c21f-4bc9-88c1-85e0d94f7963 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/users/b8b9d276657d4d65a6f5281e88d61c76/roles/14998af1b3c44044902569bf61eba98f/inherited_to_projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:13.052640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a138c-c21f-4bc9-88c1-85e0d94f7963 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=7e4a23fa192e47d2aacabb831375ddaa, user_id=b8b9d276657d4d65a6f5281e88d61c76, role_id=14998af1b3c44044902569bf61eba98f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:13.055998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a138c-c21f-4bc9-88c1-85e0d94f7963 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:13.056551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0e6f8-599c-4fd7-88a3-80c0e95980d8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:13.063843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fafa35-4681-4374-ad97-ea267ca2768d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:13.065432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fafa35-4681-4374-ad97-ea267ca2768d admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:13.066915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fafa35-4681-4374-ad97-ea267ca2768d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:13.067225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fafa35-4681-4374-ad97-ea267ca2768d admin admin] PATH_INFO: `/v3/projects/c006ac088e114f36b850825686bde529/users/fea51b110f934255bc2e11b76760a730/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:13.069481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fafa35-4681-4374-ad97-ea267ca2768d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c006ac088e114f36b850825686bde529, user_id=fea51b110f934255bc2e11b76760a730, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:13.077270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-257a138c-c21f-4bc9-88c1-85e0d94f7963 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invalidating the token cache because role 14998af1b3c44044902569bf61eba98f was removed from user b8b9d276657d4d65a6f5281e88d61c76 on project 7e4a23fa192e47d2aacabb831375ddaa. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:13.077686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-257a138c-c21f-4bc9-88c1-85e0d94f7963 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:13.079721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-257a138c-c21f-4bc9-88c1-85e0d94f7963 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:13.089196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fafa35-4681-4374-ad97-ea267ca2768d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:13.117733 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2606/5132] 10.222.0.16 () {58 vars in 1566 bytes} [Sat Sep 3 21:08:12 2022] DELETE /identity/v3/OS-INHERIT/projects/7e4a23fa192e47d2aacabb831375ddaa/users/b8b9d276657d4d65a6f5281e88d61c76/roles/14998af1b3c44044902569bf61eba98f/inherited_to_projects => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:13.127507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a138c-c21f-4bc9-88c1-85e0d94f7963 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:13.132438 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2527/5133] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:13 2022] PUT /identity/v3/projects/c006ac088e114f36b850825686bde529/users/fea51b110f934255bc2e11b76760a730/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:13.134140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-257a138c-c21f-4bc9-88c1-85e0d94f7963 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:13.138213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fc5b7-04ef-492d-9f44-d0e6a47c2d81 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:13.138565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fc5b7-04ef-492d-9f44-d0e6a47c2d81 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:13.138849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fc5b7-04ef-492d-9f44-d0e6a47c2d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:13.171320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:13.212012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c21fc5b7-04ef-492d-9f44-d0e6a47c2d81 None None] MFA Rules not processed for user `fea51b110f934255bc2e11b76760a730`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:13.213631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c21fc5b7-04ef-492d-9f44-d0e6a47c2d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:13.215842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:13.274730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a138c-c21f-4bc9-88c1-85e0d94f7963 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:13.282131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448d03b-2b23-440b-b8b6-3f00b62a789f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:13.282774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448d03b-2b23-440b-b8b6-3f00b62a789f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:13.283073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448d03b-2b23-440b-b8b6-3f00b62a789f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:13.283366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448d03b-2b23-440b-b8b6-3f00b62a789f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:13.284110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448d03b-2b23-440b-b8b6-3f00b62a789f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Adding query filter params (scope.project.id=0c6a7ed653b8410b8b0df87d79d49a54, user.id=b8b9d276657d4d65a6f5281e88d61c76) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:13.284428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448d03b-2b23-440b-b8b6-3f00b62a789f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:13.287129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448d03b-2b23-440b-b8b6-3f00b62a789f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:13.335150 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2528/5134] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:13.339550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fc5b7-04ef-492d-9f44-d0e6a47c2d81 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:13.352985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fc5b7-04ef-492d-9f44-d0e6a47c2d81 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:13.372111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd2eef-98f9-4b50-8060-8e802df91402 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:13.372672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd2eef-98f9-4b50-8060-8e802df91402 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:13.372986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd2eef-98f9-4b50-8060-8e802df91402 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:13.373269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd2eef-98f9-4b50-8060-8e802df91402 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:13.373963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd2eef-98f9-4b50-8060-8e802df91402 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:13.375768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd2eef-98f9-4b50-8060-8e802df91402 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:13.417673 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2529/5135] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:08:13 2022] POST /identity/v3/domains => generated 277 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:13.419691 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2607/5136] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Sep 3 21:08:13 2022] GET /identity/v3/role_assignments?scope.project.id=0c6a7ed653b8410b8b0df87d79d49a54&user.id=b8b9d276657d4d65a6f5281e88d61c76&effective => generated 228 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:13.423870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd2eef-98f9-4b50-8060-8e802df91402 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:13.429575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448d03b-2b23-440b-b8b6-3f00b62a789f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:13.448023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448d03b-2b23-440b-b8b6-3f00b62a789f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:13.562846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd2eef-98f9-4b50-8060-8e802df91402 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:13.614980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80516187-d39a-40d2-b63e-22b2d1e64338 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:13.615938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80516187-d39a-40d2-b63e-22b2d1e64338 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:13.616643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80516187-d39a-40d2-b63e-22b2d1e64338 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:13.617052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80516187-d39a-40d2-b63e-22b2d1e64338 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:13.618017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80516187-d39a-40d2-b63e-22b2d1e64338 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:13.620595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80516187-d39a-40d2-b63e-22b2d1e64338 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:13.632421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:13.633373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:13.633667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:13.633950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/projects/0c6a7ed653b8410b8b0df87d79d49a54` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:13.635234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0c6a7ed653b8410b8b0df87d79d49a54)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:13.637414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:13.700330 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2530/5137] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Sep 3 21:08:13 2022] POST /identity/v3/groups => generated 280 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:13.702514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80516187-d39a-40d2-b63e-22b2d1e64338 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:13.739230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80516187-d39a-40d2-b63e-22b2d1e64338 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:13.746650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6dba4-4331-46fd-a785-61878327991b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:13.747310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6dba4-4331-46fd-a785-61878327991b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:13.747967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6dba4-4331-46fd-a785-61878327991b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:13.748266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6dba4-4331-46fd-a785-61878327991b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/8a5b8cdfe10b4e8e876297e0ca366505` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:13.754774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6dba4-4331-46fd-a785-61878327991b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:update_group(group_id=8a5b8cdfe10b4e8e876297e0ca366505)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:13.755827 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-79c6dba4-4331-46fd-a785-61878327991b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:13.755827 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:13.755827 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:13.757154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6dba4-4331-46fd-a785-61878327991b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:13.783956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c6a7ed653b8410b8b0df87d79d49a54', 'request_id': 'req-92a04e6b-fca1-4411-a1a2-95b06e51cc62'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:13.796727 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2531/5138] 10.222.0.16 () {60 vars in 1247 bytes} [Sat Sep 3 21:08:13 2022] PATCH /identity/v3/groups/8a5b8cdfe10b4e8e876297e0ca366505 => generated 292 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:13.798122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6dba4-4331-46fd-a785-61878327991b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:13.802502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c6a7ed653b8410b8b0df87d79d49a54', 'request_id': 'req-92a04e6b-fca1-4411-a1a2-95b06e51cc62'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:13.814841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c6a7ed653b8410b8b0df87d79d49a54', 'request_id': 'req-92a04e6b-fca1-4411-a1a2-95b06e51cc62'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:13.821553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6dba4-4331-46fd-a785-61878327991b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:13.835755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] The token cache is being invalidate because project 0c6a7ed653b8410b8b0df87d79d49a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:13.836055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:13.839450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:13.847382 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2608/5139] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:13 2022] DELETE /identity/v3/projects/0c6a7ed653b8410b8b0df87d79d49a54 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:13.853492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:13.857254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:13.877012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:13.936431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:13.973180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2bbf7b-ab87-4d61-b109-efdf318b8e76 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:13.973850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bbf7b-ab87-4d61-b109-efdf318b8e76 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:13.974141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bbf7b-ab87-4d61-b109-efdf318b8e76 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:13.975178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bbf7b-ab87-4d61-b109-efdf318b8e76 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/8a5b8cdfe10b4e8e876297e0ca366505` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:13.976422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a04e6b-fca1-4411-a1a2-95b06e51cc62 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:13.980854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2bbf7b-ab87-4d61-b109-efdf318b8e76 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:get_group(group_id=8a5b8cdfe10b4e8e876297e0ca366505)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:13.982018 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-ac2bbf7b-ab87-4d61-b109-efdf318b8e76 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:13.982018 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:13.982018 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:13.982390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449260d-0342-4b7b-8e7f-d475e9aeaaf8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:13.983124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449260d-0342-4b7b-8e7f-d475e9aeaaf8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:13.983718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449260d-0342-4b7b-8e7f-d475e9aeaaf8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:13.983838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2bbf7b-ab87-4d61-b109-efdf318b8e76 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:13.984527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449260d-0342-4b7b-8e7f-d475e9aeaaf8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/roles/14998af1b3c44044902569bf61eba98f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:13.985796 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2532/5140] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:13 2022] GET /identity/v3/groups/8a5b8cdfe10b4e8e876297e0ca366505 => generated 292 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:13.992513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2bbf7b-ab87-4d61-b109-efdf318b8e76 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:13.993037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449260d-0342-4b7b-8e7f-d475e9aeaaf8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:delete_role(role_id=14998af1b3c44044902569bf61eba98f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:13.994257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ac2bbf7b-ab87-4d61-b109-efdf318b8e76 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:13.994937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449260d-0342-4b7b-8e7f-d475e9aeaaf8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:14.015152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2bbf7b-ab87-4d61-b109-efdf318b8e76 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:14.029440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a28f7c-baf4-4153-b967-c3d810100990 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:14.030986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a28f7c-baf4-4153-b967-c3d810100990 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:14.031547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a28f7c-baf4-4153-b967-c3d810100990 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:14.031986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a28f7c-baf4-4153-b967-c3d810100990 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/8a5b8cdfe10b4e8e876297e0ca366505` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:14.033887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a28f7c-baf4-4153-b967-c3d810100990 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:update_group(group_id=8a5b8cdfe10b4e8e876297e0ca366505)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:14.036312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a28f7c-baf4-4153-b967-c3d810100990 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:14.041825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8449260d-0342-4b7b-8e7f-d475e9aeaaf8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invalidating the token cache because role 14998af1b3c44044902569bf61eba98f has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:14.042209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8449260d-0342-4b7b-8e7f-d475e9aeaaf8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:14.043600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8449260d-0342-4b7b-8e7f-d475e9aeaaf8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:14.046132 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2609/5141] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:13 2022] DELETE /identity/v3/roles/14998af1b3c44044902569bf61eba98f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:14.051828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449260d-0342-4b7b-8e7f-d475e9aeaaf8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:14.053533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8449260d-0342-4b7b-8e7f-d475e9aeaaf8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:14.059159 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2533/5142] 10.222.0.16 () {60 vars in 1246 bytes} [Sat Sep 3 21:08:13 2022] PATCH /identity/v3/groups/8a5b8cdfe10b4e8e876297e0ca366505 => generated 309 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:14.063768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a28f7c-baf4-4153-b967-c3d810100990 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:14.065057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-28a28f7c-baf4-4153-b967-c3d810100990 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:14.079349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a28f7c-baf4-4153-b967-c3d810100990 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:14.174008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449260d-0342-4b7b-8e7f-d475e9aeaaf8 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:14.197679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40ff67-0308-458f-b5c0-153b48004bc5 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:14.199007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40ff67-0308-458f-b5c0-153b48004bc5 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:14.199368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40ff67-0308-458f-b5c0-153b48004bc5 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:14.199766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40ff67-0308-458f-b5c0-153b48004bc5 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/8a5b8cdfe10b4e8e876297e0ca366505` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:14.199878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:14.201562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:14.202487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:14.202720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/users/b8b9d276657d4d65a6f5281e88d61c76` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:14.207891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:delete_user(user_id=b8b9d276657d4d65a6f5281e88d61c76)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:14.213262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:14.214840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40ff67-0308-458f-b5c0-153b48004bc5 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:delete_group(group_id=8a5b8cdfe10b4e8e876297e0ca366505)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:14.216708 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-6c40ff67-0308-458f-b5c0-153b48004bc5 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:14.216708 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:14.216708 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:14.219848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40ff67-0308-458f-b5c0-153b48004bc5 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:14.339136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b9d276657d4d65a6f5281e88d61c76'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:14.343960 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2534/5143] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:08:14 2022] DELETE /identity/v3/groups/8a5b8cdfe10b4e8e876297e0ca366505 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:14.350121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40ff67-0308-458f-b5c0-153b48004bc5 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:14.369273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b9d276657d4d65a6f5281e88d61c76'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:14.383783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b9d276657d4d65a6f5281e88d61c76'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:14.387040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b9d276657d4d65a6f5281e88d61c76'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:14.387040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b9d276657d4d65a6f5281e88d61c76'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:14.392785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:14.406725 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2610/5144] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:14 2022] DELETE /identity/v3/users/b8b9d276657d4d65a6f5281e88d61c76 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:14.410594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:14.412626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:14.454835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40ff67-0308-458f-b5c0-153b48004bc5 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:14.488444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:14.495694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4156b2f-bbb6-4f2b-be4d-34bdb2aa87c5 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:14.501318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874acbee-195d-40df-bd21-be8f796aa1fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:14.501960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874acbee-195d-40df-bd21-be8f796aa1fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:14.502250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874acbee-195d-40df-bd21-be8f796aa1fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:14.502531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874acbee-195d-40df-bd21-be8f796aa1fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/groups/7951efaead8c49f9b57b03baaa294bed` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:14.504044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874acbee-195d-40df-bd21-be8f796aa1fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:delete_group(group_id=7951efaead8c49f9b57b03baaa294bed)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:14.504846 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-874acbee-195d-40df-bd21-be8f796aa1fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:14.504846 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:14.504846 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:14.506156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874acbee-195d-40df-bd21-be8f796aa1fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:14.528564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747c1b2-a7ff-4ba0-8ac4-7e7807ae9d53 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:14.529492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747c1b2-a7ff-4ba0-8ac4-7e7807ae9d53 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:14.530094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747c1b2-a7ff-4ba0-8ac4-7e7807ae9d53 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:14.530537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747c1b2-a7ff-4ba0-8ac4-7e7807ae9d53 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:14.532189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0747c1b2-a7ff-4ba0-8ac4-7e7807ae9d53 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:14.535099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0747c1b2-a7ff-4ba0-8ac4-7e7807ae9d53 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:14.554676 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2535/5145] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Sep 3 21:08:14 2022] POST /identity/v3/groups => generated 326 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:14.559606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747c1b2-a7ff-4ba0-8ac4-7e7807ae9d53 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:14.561575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0747c1b2-a7ff-4ba0-8ac4-7e7807ae9d53 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:14.574976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747c1b2-a7ff-4ba0-8ac4-7e7807ae9d53 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:14.578404 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2611/5146] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:08:14 2022] DELETE /identity/v3/groups/7951efaead8c49f9b57b03baaa294bed => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:14.582745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874acbee-195d-40df-bd21-be8f796aa1fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:14.583126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995037a2-ccab-4ed7-9fc2-d06d4d77eae4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:14.583753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995037a2-ccab-4ed7-9fc2-d06d4d77eae4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:14.584180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995037a2-ccab-4ed7-9fc2-d06d4d77eae4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:14.584481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995037a2-ccab-4ed7-9fc2-d06d4d77eae4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:14.585156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995037a2-ccab-4ed7-9fc2-d06d4d77eae4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:14.587025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995037a2-ccab-4ed7-9fc2-d06d4d77eae4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:14.619375 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2536/5147] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:14 2022] POST /identity/v3/users => generated 323 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:14.620640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995037a2-ccab-4ed7-9fc2-d06d4d77eae4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:14.638893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995037a2-ccab-4ed7-9fc2-d06d4d77eae4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:14.665903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874acbee-195d-40df-bd21-be8f796aa1fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:14.670890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:14.671526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:14.671814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:14.672202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/projects/7e4a23fa192e47d2aacabb831375ddaa` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:14.673401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7e4a23fa192e47d2aacabb831375ddaa)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:14.675883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:14.711298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf79dc3-f779-44a6-90d1-f57e66d7084c tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:14.712946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf79dc3-f779-44a6-90d1-f57e66d7084c tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:14.713100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf79dc3-f779-44a6-90d1-f57e66d7084c tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:14.713553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf79dc3-f779-44a6-90d1-f57e66d7084c tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/969b0d7d3d5c4a7386154be08df451e2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:14.731702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e4a23fa192e47d2aacabb831375ddaa', 'request_id': 'req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:14.739842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf79dc3-f779-44a6-90d1-f57e66d7084c tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=29d8f56c03cc42a9959eb1281adfddad, user_id=969b0d7d3d5c4a7386154be08df451e2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:14.739842 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-daf79dc3-f779-44a6-90d1-f57e66d7084c tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:14.739842 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:14.739842 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:14.740311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:14.741158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf79dc3-f779-44a6-90d1-f57e66d7084c tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:14.750645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e4a23fa192e47d2aacabb831375ddaa', 'request_id': 'req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:14.760204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e4a23fa192e47d2aacabb831375ddaa', 'request_id': 'req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:14.775488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] The token cache is being invalidate because project 7e4a23fa192e47d2aacabb831375ddaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:14.775832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:14.776581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:14.778576 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2612/5148] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:14 2022] DELETE /identity/v3/projects/7e4a23fa192e47d2aacabb831375ddaa => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:14.782744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:14.783042 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2537/5149] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Sep 3 21:08:14 2022] PUT /identity/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/969b0d7d3d5c4a7386154be08df451e2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:14.787769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:14.787916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf79dc3-f779-44a6-90d1-f57e66d7084c tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:14.790142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-daf79dc3-f779-44a6-90d1-f57e66d7084c tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:14.890490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf79dc3-f779-44a6-90d1-f57e66d7084c tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:14.898955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11b874-9516-4ab1-b729-37ad8ca2bb96 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:14.899908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11b874-9516-4ab1-b729-37ad8ca2bb96 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:14.900380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11b874-9516-4ab1-b729-37ad8ca2bb96 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:14.900819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11b874-9516-4ab1-b729-37ad8ca2bb96 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:14.901794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e11b874-9516-4ab1-b729-37ad8ca2bb96 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:14.904816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e11b874-9516-4ab1-b729-37ad8ca2bb96 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:14.915279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec6b46-3cfd-4b16-9321-db2c37d7a9fb tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:14.943705 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2538/5150] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:14 2022] POST /identity/v3/users => generated 323 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:14.949804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11b874-9516-4ab1-b729-37ad8ca2bb96 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:14.961381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11b874-9516-4ab1-b729-37ad8ca2bb96 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:14.966451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e5df8-ef61-457c-9842-5e22a5400d40 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:14.967110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e5df8-ef61-457c-9842-5e22a5400d40 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:14.968132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e5df8-ef61-457c-9842-5e22a5400d40 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:14.968421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e5df8-ef61-457c-9842-5e22a5400d40 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/9254c299cddc4b7ca4617acad71dedde` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:14.972036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:14.992081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e5df8-ef61-457c-9842-5e22a5400d40 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=29d8f56c03cc42a9959eb1281adfddad, user_id=9254c299cddc4b7ca4617acad71dedde)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:14.995004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e5df8-ef61-457c-9842-5e22a5400d40 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:14.996142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:14.996845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:14.997104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:14.997386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/domains/624a4f5ac1354a71bf67dccccf8b5337` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:14.998712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=624a4f5ac1354a71bf67dccccf8b5337)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:15.000340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:15.035576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '624a4f5ac1354a71bf67dccccf8b5337'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.036307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '624a4f5ac1354a71bf67dccccf8b5337'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.038252 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2539/5151] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Sep 3 21:08:14 2022] PUT /identity/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/9254c299cddc4b7ca4617acad71dedde => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:15.042360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e5df8-ef61-457c-9842-5e22a5400d40 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:15.044551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0e2e5df8-ef61-457c-9842-5e22a5400d40 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:15.075784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '624a4f5ac1354a71bf67dccccf8b5337'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.076425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '624a4f5ac1354a71bf67dccccf8b5337'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.078950 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2613/5152] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:08:14 2022] PATCH /identity/v3/domains/624a4f5ac1354a71bf67dccccf8b5337 => generated 278 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:15.083418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:15.085873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:15.139848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e5df8-ef61-457c-9842-5e22a5400d40 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:15.163527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:UPDATE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:15.170906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52af1fd-60ef-4750-baf2-427051896c3f tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:15.178653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33fd7cf52fd4bf894ec985c168ad0de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5852c7b40c240be93634c125c5e13f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:15.179326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:15.179582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:15.179860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] PATH_INFO: `/v3/domains/624a4f5ac1354a71bf67dccccf8b5337` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:15.192808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=624a4f5ac1354a71bf67dccccf8b5337)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:15.195062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:15.215081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03530bed-4a44-4165-b0d9-ace4e0f52609 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:15.216094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03530bed-4a44-4165-b0d9-ace4e0f52609 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:15.216382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03530bed-4a44-4165-b0d9-ace4e0f52609 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:15.216671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03530bed-4a44-4165-b0d9-ace4e0f52609 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:15.217666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03530bed-4a44-4165-b0d9-ace4e0f52609 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:15.219654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03530bed-4a44-4165-b0d9-ace4e0f52609 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:15.221594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '624a4f5ac1354a71bf67dccccf8b5337'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.225607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '624a4f5ac1354a71bf67dccccf8b5337'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.261645 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2540/5153] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:15 2022] POST /identity/v3/users => generated 323 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:15.263048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03530bed-4a44-4165-b0d9-ace4e0f52609 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:15.271328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-03530bed-4a44-4165-b0d9-ace4e0f52609 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:15.289336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03530bed-4a44-4165-b0d9-ace4e0f52609 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:15.296627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af25d4d-489b-4c57-8ea5-eea845d18855 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:15.297725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af25d4d-489b-4c57-8ea5-eea845d18855 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:15.298246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af25d4d-489b-4c57-8ea5-eea845d18855 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:15.298588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af25d4d-489b-4c57-8ea5-eea845d18855 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/dee4868ce36b4756a7fad9a27d69cc1d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:15.320872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624a4f5ac1354a71bf67dccccf8b5337', 'request_id': 'req-702ccac8-f352-4bda-b660-32513904f734'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.321629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af25d4d-489b-4c57-8ea5-eea845d18855 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=29d8f56c03cc42a9959eb1281adfddad, user_id=dee4868ce36b4756a7fad9a27d69cc1d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:15.324004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af25d4d-489b-4c57-8ea5-eea845d18855 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:15.332754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624a4f5ac1354a71bf67dccccf8b5337', 'request_id': 'req-702ccac8-f352-4bda-b660-32513904f734'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.341953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624a4f5ac1354a71bf67dccccf8b5337', 'request_id': 'req-702ccac8-f352-4bda-b660-32513904f734'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.356458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] The token cache is being invalidate because project 624a4f5ac1354a71bf67dccccf8b5337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:15.357073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.358341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.371803 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2541/5154] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Sep 3 21:08:15 2022] PUT /identity/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/dee4868ce36b4756a7fad9a27d69cc1d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:15.371958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '624a4f5ac1354a71bf67dccccf8b5337', 'request_id': 'req-702ccac8-f352-4bda-b660-32513904f734'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.373182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af25d4d-489b-4c57-8ea5-eea845d18855 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:15.374241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '624a4f5ac1354a71bf67dccccf8b5337', 'request_id': 'req-702ccac8-f352-4bda-b660-32513904f734'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.375348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6af25d4d-489b-4c57-8ea5-eea845d18855 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:15.375490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '624a4f5ac1354a71bf67dccccf8b5337', 'request_id': 'req-702ccac8-f352-4bda-b660-32513904f734'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.384832 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2614/5155] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:15 2022] DELETE /identity/v3/domains/624a4f5ac1354a71bf67dccccf8b5337 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:15.391108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:15.401180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:15.445278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:15.474028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af25d4d-489b-4c57-8ea5-eea845d18855 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:15.479413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77197333-df36-4b06-ae5b-1f893860fb81 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:15.480068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77197333-df36-4b06-ae5b-1f893860fb81 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:15.480465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77197333-df36-4b06-ae5b-1f893860fb81 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:15.480752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77197333-df36-4b06-ae5b-1f893860fb81 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:15.482145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77197333-df36-4b06-ae5b-1f893860fb81 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=29d8f56c03cc42a9959eb1281adfddad)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:15.483172 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-77197333-df36-4b06-ae5b-1f893860fb81 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:15.483172 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:15.483172 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:15.484920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77197333-df36-4b06-ae5b-1f893860fb81 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:15.486503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ccac8-f352-4bda-b660-32513904f734 tempest-InheritsV3TestJSON-362401242 tempest-InheritsV3TestJSON-362401242-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:15.504048 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2542/5156] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Sep 3 21:08:15 2022] GET /identity/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users => generated 1085 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:15.505529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f7eb7-38bc-496d-9213-6616661b96c4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:15.505888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f7eb7-38bc-496d-9213-6616661b96c4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:15.506235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f7eb7-38bc-496d-9213-6616661b96c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:15.527954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5f1f7eb7-38bc-496d-9213-6616661b96c4 None None] MFA Rules not processed for user `9e686d314d164930bbcfc9905a8b45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:15.529938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5f1f7eb7-38bc-496d-9213-6616661b96c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:15.538745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:15.565329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21592a3e-a7f2-4a7d-9119-cec78fa5722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:15.566515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21592a3e-a7f2-4a7d-9119-cec78fa5722f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:15.566808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21592a3e-a7f2-4a7d-9119-cec78fa5722f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:15.567200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21592a3e-a7f2-4a7d-9119-cec78fa5722f admin admin] PATH_INFO: `/v3/users/adec6122b37440cc94aee4405ff1957d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:15.568378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21592a3e-a7f2-4a7d-9119-cec78fa5722f admin admin] RBAC: Authorizing `identity:delete_user(user_id=adec6122b37440cc94aee4405ff1957d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:15.570496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21592a3e-a7f2-4a7d-9119-cec78fa5722f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:15.598678 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2543/5157] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:15.599851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f7eb7-38bc-496d-9213-6616661b96c4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:15.601286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5f1f7eb7-38bc-496d-9213-6616661b96c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:15.618934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f7eb7-38bc-496d-9213-6616661b96c4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:15.626862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a026e6-86ab-4aba-8bc5-dea74aa59f70 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:15.627285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-21592a3e-a7f2-4a7d-9119-cec78fa5722f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adec6122b37440cc94aee4405ff1957d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.627693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a026e6-86ab-4aba-8bc5-dea74aa59f70 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:15.628061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a026e6-86ab-4aba-8bc5-dea74aa59f70 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:15.628610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a026e6-86ab-4aba-8bc5-dea74aa59f70 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/969b0d7d3d5c4a7386154be08df451e2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:15.631830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a026e6-86ab-4aba-8bc5-dea74aa59f70 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=29d8f56c03cc42a9959eb1281adfddad, user_id=969b0d7d3d5c4a7386154be08df451e2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:15.632958 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-45a026e6-86ab-4aba-8bc5-dea74aa59f70 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_user_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:15.632958 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:15.632958 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:15.634450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a026e6-86ab-4aba-8bc5-dea74aa59f70 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:15.647736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-21592a3e-a7f2-4a7d-9119-cec78fa5722f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adec6122b37440cc94aee4405ff1957d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.661675 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2544/5158] 10.222.0.16 () {58 vars in 1342 bytes} [Sat Sep 3 21:08:15 2022] HEAD /identity/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/969b0d7d3d5c4a7386154be08df451e2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:15.662375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-21592a3e-a7f2-4a7d-9119-cec78fa5722f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adec6122b37440cc94aee4405ff1957d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.662618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a026e6-86ab-4aba-8bc5-dea74aa59f70 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:15.663667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-21592a3e-a7f2-4a7d-9119-cec78fa5722f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adec6122b37440cc94aee4405ff1957d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.664285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-21592a3e-a7f2-4a7d-9119-cec78fa5722f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adec6122b37440cc94aee4405ff1957d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.674727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a026e6-86ab-4aba-8bc5-dea74aa59f70 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:15.680602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ab6f2-a0f8-4b5d-a83e-508e13f6aa1e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:15.681919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ab6f2-a0f8-4b5d-a83e-508e13f6aa1e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:15.681919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ab6f2-a0f8-4b5d-a83e-508e13f6aa1e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:15.682212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ab6f2-a0f8-4b5d-a83e-508e13f6aa1e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/969b0d7d3d5c4a7386154be08df451e2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:15.683325 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2615/5159] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:15 2022] DELETE /identity/v3/users/adec6122b37440cc94aee4405ff1957d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:15.684612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ab6f2-a0f8-4b5d-a83e-508e13f6aa1e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=29d8f56c03cc42a9959eb1281adfddad, user_id=969b0d7d3d5c4a7386154be08df451e2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:15.686073 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-ee4ab6f2-a0f8-4b5d-a83e-508e13f6aa1e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:15.686073 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:15.686073 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:15.687628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7750bde-b069-4e40-b909-1ea417e4ae02 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:15.687752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ab6f2-a0f8-4b5d-a83e-508e13f6aa1e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:15.687991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7750bde-b069-4e40-b909-1ea417e4ae02 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:15.688286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7750bde-b069-4e40-b909-1ea417e4ae02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:15.701749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ee4ab6f2-a0f8-4b5d-a83e-508e13f6aa1e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '969b0d7d3d5c4a7386154be08df451e2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.719116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b7750bde-b069-4e40-b909-1ea417e4ae02 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:15.720478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b7750bde-b069-4e40-b909-1ea417e4ae02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:15.724682 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2545/5160] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Sep 3 21:08:15 2022] DELETE /identity/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/969b0d7d3d5c4a7386154be08df451e2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:15.728954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ab6f2-a0f8-4b5d-a83e-508e13f6aa1e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:15.734141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ee4ab6f2-a0f8-4b5d-a83e-508e13f6aa1e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:15.800927 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2616/5161] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:15.817188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7750bde-b069-4e40-b909-1ea417e4ae02 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:15.820653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b7750bde-b069-4e40-b909-1ea417e4ae02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:15.833066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ab6f2-a0f8-4b5d-a83e-508e13f6aa1e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:15.839425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e316b-3eeb-4398-9e58-02b2282cb119 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:15.840585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e316b-3eeb-4398-9e58-02b2282cb119 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:15.841123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e316b-3eeb-4398-9e58-02b2282cb119 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:15.842334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e316b-3eeb-4398-9e58-02b2282cb119 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/9254c299cddc4b7ca4617acad71dedde` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:15.844589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e316b-3eeb-4398-9e58-02b2282cb119 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=29d8f56c03cc42a9959eb1281adfddad, user_id=9254c299cddc4b7ca4617acad71dedde)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:15.847571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e316b-3eeb-4398-9e58-02b2282cb119 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:15.872038 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2546/5162] 10.222.0.16 () {58 vars in 1342 bytes} [Sat Sep 3 21:08:15 2022] HEAD /identity/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/9254c299cddc4b7ca4617acad71dedde => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:15.873455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e316b-3eeb-4398-9e58-02b2282cb119 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:15.884842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e316b-3eeb-4398-9e58-02b2282cb119 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:15.892606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7750bde-b069-4e40-b909-1ea417e4ae02 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:15.893310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe62f1-b538-41f6-8fb0-06ad06eafac2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:15.894922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe62f1-b538-41f6-8fb0-06ad06eafac2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:15.895218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe62f1-b538-41f6-8fb0-06ad06eafac2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:15.895504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe62f1-b538-41f6-8fb0-06ad06eafac2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/9254c299cddc4b7ca4617acad71dedde` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:15.898017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe62f1-b538-41f6-8fb0-06ad06eafac2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=29d8f56c03cc42a9959eb1281adfddad, user_id=9254c299cddc4b7ca4617acad71dedde)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:15.899734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3a2c0-e269-4582-8d48-557659904b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:15.900973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe62f1-b538-41f6-8fb0-06ad06eafac2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:15.901216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3a2c0-e269-4582-8d48-557659904b56 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:15.901531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3a2c0-e269-4582-8d48-557659904b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:15.901823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3a2c0-e269-4582-8d48-557659904b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:15.911223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b6fe62f1-b538-41f6-8fb0-06ad06eafac2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '9254c299cddc4b7ca4617acad71dedde'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:15.915670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3a2c0-e269-4582-8d48-557659904b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:15.917658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3a2c0-e269-4582-8d48-557659904b56 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:15.928825 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2547/5163] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Sep 3 21:08:15 2022] DELETE /identity/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/9254c299cddc4b7ca4617acad71dedde => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:15.929027 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2617/5164] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:08:15.933125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe62f1-b538-41f6-8fb0-06ad06eafac2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:15.944085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:16.009260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe62f1-b538-41f6-8fb0-06ad06eafac2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:16.014809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b9e12-c41d-48d5-989d-665d13478412 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:16.015502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b9e12-c41d-48d5-989d-665d13478412 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:16.015799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b9e12-c41d-48d5-989d-665d13478412 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:16.016575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b9e12-c41d-48d5-989d-665d13478412 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/dee4868ce36b4756a7fad9a27d69cc1d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:16.017982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b9e12-c41d-48d5-989d-665d13478412 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=29d8f56c03cc42a9959eb1281adfddad, user_id=dee4868ce36b4756a7fad9a27d69cc1d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:16.020471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b9e12-c41d-48d5-989d-665d13478412 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:16.036464 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2548/5165] 10.222.0.16 () {58 vars in 1342 bytes} [Sat Sep 3 21:08:15 2022] HEAD /identity/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/dee4868ce36b4756a7fad9a27d69cc1d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:16.040463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3a2c0-e269-4582-8d48-557659904b56 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:16.046508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:16.054343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3a2c0-e269-4582-8d48-557659904b56 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:16.060471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774c4c5-3791-4001-8c94-bc830a8dd5b3 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:16.061505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774c4c5-3791-4001-8c94-bc830a8dd5b3 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:16.062018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774c4c5-3791-4001-8c94-bc830a8dd5b3 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:16.062493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774c4c5-3791-4001-8c94-bc830a8dd5b3 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/dee4868ce36b4756a7fad9a27d69cc1d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:16.064593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774c4c5-3791-4001-8c94-bc830a8dd5b3 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=29d8f56c03cc42a9959eb1281adfddad, user_id=dee4868ce36b4756a7fad9a27d69cc1d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:16.065886 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-e774c4c5-3791-4001-8c94-bc830a8dd5b3 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:16.065886 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:16.065886 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:16.067635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774c4c5-3791-4001-8c94-bc830a8dd5b3 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:16.081114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e774c4c5-3791-4001-8c94-bc830a8dd5b3 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'dee4868ce36b4756a7fad9a27d69cc1d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.099984 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2618/5166] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Sep 3 21:08:16 2022] DELETE /identity/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users/dee4868ce36b4756a7fad9a27d69cc1d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:16.103961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b9e12-c41d-48d5-989d-665d13478412 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:16.125230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b9e12-c41d-48d5-989d-665d13478412 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:16.201118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c4b52-f059-4da2-a593-6be25dec23dc tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:16.201892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c4b52-f059-4da2-a593-6be25dec23dc tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:16.202379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c4b52-f059-4da2-a593-6be25dec23dc tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:16.202799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c4b52-f059-4da2-a593-6be25dec23dc tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:16.204046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c4b52-f059-4da2-a593-6be25dec23dc tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=29d8f56c03cc42a9959eb1281adfddad)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:16.206054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c4b52-f059-4da2-a593-6be25dec23dc tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:16.220825 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2549/5167] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Sep 3 21:08:16 2022] GET /identity/v3/groups/29d8f56c03cc42a9959eb1281adfddad/users => generated 145 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:16.223202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774c4c5-3791-4001-8c94-bc830a8dd5b3 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:16.238472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774c4c5-3791-4001-8c94-bc830a8dd5b3 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:16.244868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:16.246062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:16.246571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:16.247109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/users/dee4868ce36b4756a7fad9a27d69cc1d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:16.249183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:delete_user(user_id=dee4868ce36b4756a7fad9a27d69cc1d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:16.251627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:16.321880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dee4868ce36b4756a7fad9a27d69cc1d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.340126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dee4868ce36b4756a7fad9a27d69cc1d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.351105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dee4868ce36b4756a7fad9a27d69cc1d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.351822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dee4868ce36b4756a7fad9a27d69cc1d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.352550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dee4868ce36b4756a7fad9a27d69cc1d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.370704 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2619/5168] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:16 2022] DELETE /identity/v3/users/dee4868ce36b4756a7fad9a27d69cc1d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:16.374695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c4b52-f059-4da2-a593-6be25dec23dc tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:16.376485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8c6c4b52-f059-4da2-a593-6be25dec23dc tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:16.391189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c4b52-f059-4da2-a593-6be25dec23dc tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:16.471941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:16.474247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:16.475180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:16.475723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/users/9254c299cddc4b7ca4617acad71dedde` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:16.477954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:delete_user(user_id=9254c299cddc4b7ca4617acad71dedde)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:16.481753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:16.547223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:16.557435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9254c299cddc4b7ca4617acad71dedde'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.575464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9254c299cddc4b7ca4617acad71dedde'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.575848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9254c299cddc4b7ca4617acad71dedde'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.584798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9254c299cddc4b7ca4617acad71dedde'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.605630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9254c299cddc4b7ca4617acad71dedde'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.622340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:16.623832 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2550/5169] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:16 2022] DELETE /identity/v3/users/9254c299cddc4b7ca4617acad71dedde => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:16.637347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1d96b-8809-4656-bec0-2c3d8c01536a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:16.718580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:16.719527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:16.720042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:16.720515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/users/969b0d7d3d5c4a7386154be08df451e2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:16.722706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:delete_user(user_id=969b0d7d3d5c4a7386154be08df451e2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:16.726603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:16.802254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '969b0d7d3d5c4a7386154be08df451e2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.812967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:16.825146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '969b0d7d3d5c4a7386154be08df451e2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.836521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '969b0d7d3d5c4a7386154be08df451e2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.837365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '969b0d7d3d5c4a7386154be08df451e2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.840013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '969b0d7d3d5c4a7386154be08df451e2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:16.899639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:16.901634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:16.904441 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2620/5170] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:16 2022] DELETE /identity/v3/users/969b0d7d3d5c4a7386154be08df451e2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:16.916507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f0537-6ecc-45df-b386-e672d9afc59e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:16.931964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:16.935784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:17.062314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebeb16-d368-45d4-ae3e-b9148c459bd2 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:17.065231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c083e-e328-4bc2-824a-fd28acd9c646 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:17.066432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c083e-e328-4bc2-824a-fd28acd9c646 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:17.066731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c083e-e328-4bc2-824a-fd28acd9c646 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:17.067009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c083e-e328-4bc2-824a-fd28acd9c646 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29d8f56c03cc42a9959eb1281adfddad` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:17.070257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c083e-e328-4bc2-824a-fd28acd9c646 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:delete_group(group_id=29d8f56c03cc42a9959eb1281adfddad)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:17.070585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:17.071254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:17.071543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:17.071824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] PATH_INFO: `/v3/projects/43bddba02efe450ba3a4a85f855bc144` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:17.073313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] RBAC: Authorizing `identity:delete_project(project_id=43bddba02efe450ba3a4a85f855bc144)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:17.073720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c083e-e328-4bc2-824a-fd28acd9c646 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:17.076467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:17.156181 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2551/5171] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:08:16 2022] DELETE /identity/v3/groups/29d8f56c03cc42a9959eb1281adfddad => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:17.163331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c083e-e328-4bc2-824a-fd28acd9c646 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:17.165598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f89c083e-e328-4bc2-824a-fd28acd9c646 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:17.166855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43bddba02efe450ba3a4a85f855bc144', 'request_id': 'req-d2044778-487f-4bfc-85ba-fff4885b775f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:17.178053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43bddba02efe450ba3a4a85f855bc144', 'request_id': 'req-d2044778-487f-4bfc-85ba-fff4885b775f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:17.190004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43bddba02efe450ba3a4a85f855bc144', 'request_id': 'req-d2044778-487f-4bfc-85ba-fff4885b775f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:17.212328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] The token cache is being invalidate because project 43bddba02efe450ba3a4a85f855bc144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:17.212588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:17.213446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:17.220758 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2621/5172] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:16 2022] DELETE /identity/v3/projects/43bddba02efe450ba3a4a85f855bc144 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:17.225781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:17.228933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:17.270713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c083e-e328-4bc2-824a-fd28acd9c646 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:17.276497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2818560-7046-4977-9064-39ee560253b4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:17.277481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2818560-7046-4977-9064-39ee560253b4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:17.277873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2818560-7046-4977-9064-39ee560253b4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:17.278431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2818560-7046-4977-9064-39ee560253b4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:17.279016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2818560-7046-4977-9064-39ee560253b4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:17.281085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2818560-7046-4977-9064-39ee560253b4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:17.303449 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2552/5173] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Sep 3 21:08:17 2022] POST /identity/v3/groups => generated 325 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:17.305452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2818560-7046-4977-9064-39ee560253b4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:17.307753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a2818560-7046-4977-9064-39ee560253b4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:17.326708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2818560-7046-4977-9064-39ee560253b4 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:17.333120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cce77-3835-4861-934e-0799baab4701 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:17.334127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cce77-3835-4861-934e-0799baab4701 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:17.334377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cce77-3835-4861-934e-0799baab4701 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:17.334846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cce77-3835-4861-934e-0799baab4701 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:17.335614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cce77-3835-4861-934e-0799baab4701 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:17.338041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cce77-3835-4861-934e-0799baab4701 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:17.343802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:17.361730 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2553/5174] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Sep 3 21:08:17 2022] POST /identity/v3/groups => generated 324 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:17.362516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2044778-487f-4bfc-85ba-fff4885b775f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:17.364140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cce77-3835-4861-934e-0799baab4701 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:17.368558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:17.377015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cce77-3835-4861-934e-0799baab4701 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:17.383477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc657c13-2ef5-4f78-8c0c-6823c19450fa tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:17.384364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc657c13-2ef5-4f78-8c0c-6823c19450fa tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:17.384571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc657c13-2ef5-4f78-8c0c-6823c19450fa tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:17.384834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc657c13-2ef5-4f78-8c0c-6823c19450fa tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:17.385657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc657c13-2ef5-4f78-8c0c-6823c19450fa tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:17.387717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc657c13-2ef5-4f78-8c0c-6823c19450fa tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:17.409921 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2554/5175] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Sep 3 21:08:17 2022] POST /identity/v3/groups => generated 326 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:17.411285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc657c13-2ef5-4f78-8c0c-6823c19450fa tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:17.428856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc657c13-2ef5-4f78-8c0c-6823c19450fa tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:17.434027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce627e-4372-4827-a14d-a47982ccef87 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:17.434645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce627e-4372-4827-a14d-a47982ccef87 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:17.434946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce627e-4372-4827-a14d-a47982ccef87 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:17.435350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce627e-4372-4827-a14d-a47982ccef87 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:17.436109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce627e-4372-4827-a14d-a47982ccef87 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:list_groups()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:17.437308 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-55ce627e-4372-4827-a14d-a47982ccef87 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_groups": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:17.437308 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:17.437308 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:17.438526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce627e-4372-4827-a14d-a47982ccef87 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:17.440658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:17.441697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:17.442335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:17.442662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] PATH_INFO: `/v3/users/d33fd7cf52fd4bf894ec985c168ad0de` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:17.444099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d33fd7cf52fd4bf894ec985c168ad0de)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:17.446274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:17.448945 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2555/5176] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Sep 3 21:08:17 2022] GET /identity/v3/groups => generated 1485 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:17.451697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce627e-4372-4827-a14d-a47982ccef87 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:17.473811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce627e-4372-4827-a14d-a47982ccef87 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:17.480286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757bebd-432f-42cb-ad19-9efa3afa6572 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:17.481199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757bebd-432f-42cb-ad19-9efa3afa6572 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:17.481436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757bebd-432f-42cb-ad19-9efa3afa6572 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:17.481758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757bebd-432f-42cb-ad19-9efa3afa6572 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/18a570664dca4a2283f81cac274f39ec` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:17.491365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757bebd-432f-42cb-ad19-9efa3afa6572 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:delete_group(group_id=18a570664dca4a2283f81cac274f39ec)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:17.497428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757bebd-432f-42cb-ad19-9efa3afa6572 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:17.508593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd33fd7cf52fd4bf894ec985c168ad0de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:17.531629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd33fd7cf52fd4bf894ec985c168ad0de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:17.550857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd33fd7cf52fd4bf894ec985c168ad0de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:17.551997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd33fd7cf52fd4bf894ec985c168ad0de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:17.553116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd33fd7cf52fd4bf894ec985c168ad0de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:17.564946 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2556/5177] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:08:17 2022] DELETE /identity/v3/groups/18a570664dca4a2283f81cac274f39ec => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:17.567554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757bebd-432f-42cb-ad19-9efa3afa6572 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:17.572192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6757bebd-432f-42cb-ad19-9efa3afa6572 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:17.580385 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2622/5178] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:17 2022] DELETE /identity/v3/users/d33fd7cf52fd4bf894ec985c168ad0de => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:17.694549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757bebd-432f-42cb-ad19-9efa3afa6572 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:17.816822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51478c-a480-4fb5-9a07-777c53fc16dd tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:17.817343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51478c-a480-4fb5-9a07-777c53fc16dd tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:17.817632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51478c-a480-4fb5-9a07-777c53fc16dd tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:17.817928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51478c-a480-4fb5-9a07-777c53fc16dd tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/dd4a05ff695f4cb9b78973e9c3745343` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:17.825598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51478c-a480-4fb5-9a07-777c53fc16dd tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:delete_group(group_id=dd4a05ff695f4cb9b78973e9c3745343)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:17.827109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51478c-a480-4fb5-9a07-777c53fc16dd tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:17.854729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:17.930568 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2557/5179] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:08:17 2022] DELETE /identity/v3/groups/dd4a05ff695f4cb9b78973e9c3745343 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:17.932019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:17.947260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01573a-5fb0-4cdd-98d8-57ae4be99a15 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:18.020662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff79c2a-f9e9-4f91-9a06-a60cbcbc3d7d tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:18.021692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff79c2a-f9e9-4f91-9a06-a60cbcbc3d7d tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.022258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff79c2a-f9e9-4f91-9a06-a60cbcbc3d7d tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.022801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff79c2a-f9e9-4f91-9a06-a60cbcbc3d7d tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/66899cc5ff0248379dc8d1d2ede0e2a7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.028433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff79c2a-f9e9-4f91-9a06-a60cbcbc3d7d tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:delete_group(group_id=66899cc5ff0248379dc8d1d2ede0e2a7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:18.030409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff79c2a-f9e9-4f91-9a06-a60cbcbc3d7d tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:18.051355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:18.081255 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2623/5180] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:08:17 2022] DELETE /identity/v3/groups/66899cc5ff0248379dc8d1d2ede0e2a7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:18.087095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51478c-a480-4fb5-9a07-777c53fc16dd tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:18.098025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51478c-a480-4fb5-9a07-777c53fc16dd tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:18.176630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cffb9-3899-4f0b-a7e8-e4a6d8b4cf2e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:18.177587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cffb9-3899-4f0b-a7e8-e4a6d8b4cf2e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.177880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cffb9-3899-4f0b-a7e8-e4a6d8b4cf2e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.178160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cffb9-3899-4f0b-a7e8-e4a6d8b4cf2e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.178811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cffb9-3899-4f0b-a7e8-e4a6d8b4cf2e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:18.180744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cffb9-3899-4f0b-a7e8-e4a6d8b4cf2e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:18.201037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff79c2a-f9e9-4f91-9a06-a60cbcbc3d7d tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:18.203628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1ff79c2a-f9e9-4f91-9a06-a60cbcbc3d7d tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:18.224945 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2558/5181] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:18 2022] POST /identity/v3/users => generated 321 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:18.226995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cffb9-3899-4f0b-a7e8-e4a6d8b4cf2e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:18.245193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cffb9-3899-4f0b-a7e8-e4a6d8b4cf2e tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:18.250636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3a4f5-77f6-4001-bad5-9973b9670952 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:18.252792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3a4f5-77f6-4001-bad5-9973b9670952 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.253123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3a4f5-77f6-4001-bad5-9973b9670952 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.254715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3a4f5-77f6-4001-bad5-9973b9670952 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.255794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3a4f5-77f6-4001-bad5-9973b9670952 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:18.258467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3a4f5-77f6-4001-bad5-9973b9670952 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:18.281791 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2559/5182] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Sep 3 21:08:18 2022] POST /identity/v3/groups => generated 325 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:18.283219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3a4f5-77f6-4001-bad5-9973b9670952 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:18.313770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff79c2a-f9e9-4f91-9a06-a60cbcbc3d7d tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:18.315554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3a4f5-77f6-4001-bad5-9973b9670952 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:18.321708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:18.322782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.323120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.323456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e admin admin] PATH_INFO: `/v3/projects/d5852c7b40c240be93634c125c5e13f7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.324697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5852c7b40c240be93634c125c5e13f7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:18.325213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dfbbd-6f82-45ed-b42d-a3e47fd89656 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:18.326262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dfbbd-6f82-45ed-b42d-a3e47fd89656 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.326846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dfbbd-6f82-45ed-b42d-a3e47fd89656 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.327282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:18.328096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dfbbd-6f82-45ed-b42d-a3e47fd89656 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29944f61b5c74cb982c07843b15030f7/users/393dfb9e61cd4154bffcf8f3f212eed0` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.359121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71dfbbd-6f82-45ed-b42d-a3e47fd89656 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=29944f61b5c74cb982c07843b15030f7, user_id=393dfb9e61cd4154bffcf8f3f212eed0)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:18.362104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71dfbbd-6f82-45ed-b42d-a3e47fd89656 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:18.406240 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2560/5183] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Sep 3 21:08:18 2022] PUT /identity/v3/groups/29944f61b5c74cb982c07843b15030f7/users/393dfb9e61cd4154bffcf8f3f212eed0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:18.407494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dfbbd-6f82-45ed-b42d-a3e47fd89656 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:18.482593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5852c7b40c240be93634c125c5e13f7', 'request_id': 'req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:18.499308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5852c7b40c240be93634c125c5e13f7', 'request_id': 'req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:18.508023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dfbbd-6f82-45ed-b42d-a3e47fd89656 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:18.518699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5852c7b40c240be93634c125c5e13f7', 'request_id': 'req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:18.545555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:18.548052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e admin admin] The token cache is being invalidate because project d5852c7b40c240be93634c125c5e13f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:18.548443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:18.551176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a5295eb1-33bd-42c4-afb3-b7a4b31c854e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:18.556620 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2624/5184] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:18 2022] DELETE /identity/v3/projects/d5852c7b40c240be93634c125c5e13f7 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:18.561705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf503397-1350-41c8-818f-8db233abb8db None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.562005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf503397-1350-41c8-818f-8db233abb8db None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.563422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf503397-1350-41c8-818f-8db233abb8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.607710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bf503397-1350-41c8-818f-8db233abb8db None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:18.609158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bf503397-1350-41c8-818f-8db233abb8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:18.629596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7edcf-3b17-4908-8f80-df96a7b89ea7 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:18.630799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7edcf-3b17-4908-8f80-df96a7b89ea7 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.631321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7edcf-3b17-4908-8f80-df96a7b89ea7 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.631887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7edcf-3b17-4908-8f80-df96a7b89ea7 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.632809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7edcf-3b17-4908-8f80-df96a7b89ea7 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:18.635328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7edcf-3b17-4908-8f80-df96a7b89ea7 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:18.657697 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2561/5185] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Sep 3 21:08:18 2022] POST /identity/v3/groups => generated 326 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:18.661775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7edcf-3b17-4908-8f80-df96a7b89ea7 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:18.663948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e8e7edcf-3b17-4908-8f80-df96a7b89ea7 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:18.688941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7edcf-3b17-4908-8f80-df96a7b89ea7 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:18.698276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c53e7-820c-4f16-b4ed-021987729333 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:18.698276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c53e7-820c-4f16-b4ed-021987729333 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.698276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c53e7-820c-4f16-b4ed-021987729333 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.698585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c53e7-820c-4f16-b4ed-021987729333 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/756e0b20c076493394a81e2709f2dfd5/users/393dfb9e61cd4154bffcf8f3f212eed0` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.703944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c53e7-820c-4f16-b4ed-021987729333 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=756e0b20c076493394a81e2709f2dfd5, user_id=393dfb9e61cd4154bffcf8f3f212eed0)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:18.707497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c53e7-820c-4f16-b4ed-021987729333 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:18.719666 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2625/5186] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:18.727734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28779c6f-d41b-4ae0-b087-5bb7826e3e38 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.728072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28779c6f-d41b-4ae0-b087-5bb7826e3e38 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.728359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28779c6f-d41b-4ae0-b087-5bb7826e3e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.743162 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2562/5187] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Sep 3 21:08:18 2022] PUT /identity/v3/groups/756e0b20c076493394a81e2709f2dfd5/users/393dfb9e61cd4154bffcf8f3f212eed0 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:18.744081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c53e7-820c-4f16-b4ed-021987729333 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:18.750587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-28779c6f-d41b-4ae0-b087-5bb7826e3e38 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:18.752414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-28779c6f-d41b-4ae0-b087-5bb7826e3e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:18.768486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:18.851580 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2626/5188] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:18.853128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28779c6f-d41b-4ae0-b087-5bb7826e3e38 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:18.867035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c53e7-820c-4f16-b4ed-021987729333 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:18.867325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28779c6f-d41b-4ae0-b087-5bb7826e3e38 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:18.872800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bf125-1106-4f40-9df5-42f3e5ddee62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:18.873556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bf125-1106-4f40-9df5-42f3e5ddee62 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.873718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5090b-fe1a-402b-877f-9cbae290da38 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:18.874013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bf125-1106-4f40-9df5-42f3e5ddee62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.874413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bf125-1106-4f40-9df5-42f3e5ddee62 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.874709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5090b-fe1a-402b-877f-9cbae290da38 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.875210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bf125-1106-4f40-9df5-42f3e5ddee62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:18.875323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5090b-fe1a-402b-877f-9cbae290da38 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.875618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bf125-1106-4f40-9df5-42f3e5ddee62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:18.875738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5090b-fe1a-402b-877f-9cbae290da38 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/users/393dfb9e61cd4154bffcf8f3f212eed0/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.877399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5090b-fe1a-402b-877f-9cbae290da38 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:list_groups_for_user(user_id=393dfb9e61cd4154bffcf8f3f212eed0)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:18.877539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bf125-1106-4f40-9df5-42f3e5ddee62 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:18.880237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5090b-fe1a-402b-877f-9cbae290da38 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:18.896010 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2627/5189] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:18.897419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bf125-1106-4f40-9df5-42f3e5ddee62 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:18.910450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bf125-1106-4f40-9df5-42f3e5ddee62 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:18.915803 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2563/5190] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Sep 3 21:08:18 2022] GET /identity/v3/users/393dfb9e61cd4154bffcf8f3f212eed0/groups => generated 837 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:18.917636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5090b-fe1a-402b-877f-9cbae290da38 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:18.919842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf127c-9538-4122-93d3-609b92b689f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:18.920483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf127c-9538-4122-93d3-609b92b689f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.921147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf127c-9538-4122-93d3-609b92b689f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.921545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf127c-9538-4122-93d3-609b92b689f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.922839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf127c-9538-4122-93d3-609b92b689f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:18.928059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf127c-9538-4122-93d3-609b92b689f7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:18.946089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5090b-fe1a-402b-877f-9cbae290da38 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:18.959479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b36b14-88b8-4489-b1f3-6e3a7407bcac tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:18.960820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b36b14-88b8-4489-b1f3-6e3a7407bcac tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:18.961156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b36b14-88b8-4489-b1f3-6e3a7407bcac tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:18.961467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b36b14-88b8-4489-b1f3-6e3a7407bcac tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/756e0b20c076493394a81e2709f2dfd5` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:18.965056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b36b14-88b8-4489-b1f3-6e3a7407bcac tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:delete_group(group_id=756e0b20c076493394a81e2709f2dfd5)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:18.972427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b36b14-88b8-4489-b1f3-6e3a7407bcac tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:18.974054 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2628/5191] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:18 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:18.978629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf127c-9538-4122-93d3-609b92b689f7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:19.073022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf127c-9538-4122-93d3-609b92b689f7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:19.078356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703cd90-a331-4f22-939c-802abb505079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:19.079107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703cd90-a331-4f22-939c-802abb505079 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:19.079476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703cd90-a331-4f22-939c-802abb505079 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:19.079591 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2564/5192] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:08:18 2022] DELETE /identity/v3/groups/756e0b20c076493394a81e2709f2dfd5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:19.083874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703cd90-a331-4f22-939c-802abb505079 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:19.083874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703cd90-a331-4f22-939c-802abb505079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:19.084371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b36b14-88b8-4489-b1f3-6e3a7407bcac tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:19.084662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703cd90-a331-4f22-939c-802abb505079 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:19.121557 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2629/5193] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:18 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:19.122866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703cd90-a331-4f22-939c-802abb505079 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:19.139868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703cd90-a331-4f22-939c-802abb505079 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:19.191266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:19.229278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e20cb-976a-44fe-b61b-f2a61ef4c668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:19.229904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e20cb-976a-44fe-b61b-f2a61ef4c668 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:19.230190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e20cb-976a-44fe-b61b-f2a61ef4c668 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:19.230764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e20cb-976a-44fe-b61b-f2a61ef4c668 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:19.231425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e20cb-976a-44fe-b61b-f2a61ef4c668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:19.233072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e20cb-976a-44fe-b61b-f2a61ef4c668 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:19.250028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b36b14-88b8-4489-b1f3-6e3a7407bcac tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:19.261000 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2630/5194] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:19.263276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e20cb-976a-44fe-b61b-f2a61ef4c668 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:19.264446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e26bb-9759-4bd2-b591-42e21243d42f tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:19.265262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e26bb-9759-4bd2-b591-42e21243d42f tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:19.265673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e26bb-9759-4bd2-b591-42e21243d42f tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:19.266159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e26bb-9759-4bd2-b591-42e21243d42f tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/groups/29944f61b5c74cb982c07843b15030f7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:19.267485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e26bb-9759-4bd2-b591-42e21243d42f tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:delete_group(group_id=29944f61b5c74cb982c07843b15030f7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:19.269641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e26bb-9759-4bd2-b591-42e21243d42f tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:19.271735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:19.278215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e20cb-976a-44fe-b61b-f2a61ef4c668 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:19.288218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71491325-6435-4986-9113-207aabcb7467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:19.288880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71491325-6435-4986-9113-207aabcb7467 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:19.289194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71491325-6435-4986-9113-207aabcb7467 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:19.290851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71491325-6435-4986-9113-207aabcb7467 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:19.291347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71491325-6435-4986-9113-207aabcb7467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:19.298808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71491325-6435-4986-9113-207aabcb7467 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:19.331666 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2631/5195] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:19.331666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71491325-6435-4986-9113-207aabcb7467 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:19.343663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71491325-6435-4986-9113-207aabcb7467 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:19.350000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ff159-a59f-42e0-ac35-cf2ee5aed5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:19.350799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ff159-a59f-42e0-ac35-cf2ee5aed5cc admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:19.351215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ff159-a59f-42e0-ac35-cf2ee5aed5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:19.351588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ff159-a59f-42e0-ac35-cf2ee5aed5cc admin admin] PATH_INFO: `/v3/projects/ac601057cde6448b859c13a864271d00/users/8e00742eb8c24d9cba3e840f36527fc3/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:19.367530 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2565/5196] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:08:19 2022] DELETE /identity/v3/groups/29944f61b5c74cb982c07843b15030f7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:19.369395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e26bb-9759-4bd2-b591-42e21243d42f tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:19.369858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ff159-a59f-42e0-ac35-cf2ee5aed5cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac601057cde6448b859c13a864271d00, user_id=8e00742eb8c24d9cba3e840f36527fc3, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:19.373038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ff159-a59f-42e0-ac35-cf2ee5aed5cc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:19.398214 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2632/5197] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:19 2022] PUT /identity/v3/projects/ac601057cde6448b859c13a864271d00/users/8e00742eb8c24d9cba3e840f36527fc3/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:19.403039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688aea37-bf22-47b9-bf77-9a2d5b5194b3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:19.404063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688aea37-bf22-47b9-bf77-9a2d5b5194b3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:19.404423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688aea37-bf22-47b9-bf77-9a2d5b5194b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:19.443480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-688aea37-bf22-47b9-bf77-9a2d5b5194b3 None None] MFA Rules not processed for user `8e00742eb8c24d9cba3e840f36527fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:19.445567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-688aea37-bf22-47b9-bf77-9a2d5b5194b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:19.463917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e26bb-9759-4bd2-b591-42e21243d42f tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:19.527045 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2633/5198] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:19.531188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688aea37-bf22-47b9-bf77-9a2d5b5194b3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:19.544419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:19.545111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:19.545519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:19.545817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/users/393dfb9e61cd4154bffcf8f3f212eed0` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:19.547296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:delete_user(user_id=393dfb9e61cd4154bffcf8f3f212eed0)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:19.549769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:19.614024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=10 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:19.618297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '393dfb9e61cd4154bffcf8f3f212eed0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:19.624339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688aea37-bf22-47b9-bf77-9a2d5b5194b3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:19.631240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944d581-a42b-4587-9371-fa9fd89f074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:19.631862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944d581-a42b-4587-9371-fa9fd89f074d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:19.632255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944d581-a42b-4587-9371-fa9fd89f074d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:19.632969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944d581-a42b-4587-9371-fa9fd89f074d admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:19.634010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944d581-a42b-4587-9371-fa9fd89f074d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:19.634127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '393dfb9e61cd4154bffcf8f3f212eed0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:19.634982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '393dfb9e61cd4154bffcf8f3f212eed0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:19.636933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944d581-a42b-4587-9371-fa9fd89f074d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:19.643209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '393dfb9e61cd4154bffcf8f3f212eed0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:19.655550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '393dfb9e61cd4154bffcf8f3f212eed0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:19.664200 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2566/5199] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:19 2022] DELETE /identity/v3/users/393dfb9e61cd4154bffcf8f3f212eed0 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:19.670278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:19.672034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:19.680171 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2634/5200] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:19 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:19.685039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944d581-a42b-4587-9371-fa9fd89f074d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:19.689388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2944d581-a42b-4587-9371-fa9fd89f074d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:19.809230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:19.835172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281a1ef-478a-4a29-bf1a-fec9015cb93b tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:19.866089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944d581-a42b-4587-9371-fa9fd89f074d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:19.879608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509a109-ef95-4cf4-8f10-eb60e23fdde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:19.880934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509a109-ef95-4cf4-8f10-eb60e23fdde5 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:19.880934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509a109-ef95-4cf4-8f10-eb60e23fdde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:19.882405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509a109-ef95-4cf4-8f10-eb60e23fdde5 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:19.882922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509a109-ef95-4cf4-8f10-eb60e23fdde5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:19.887037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509a109-ef95-4cf4-8f10-eb60e23fdde5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:19.910682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:19.911346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:19.911634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:19.911911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/domains/4cd6dbeb05584f2f864fcf980fd09dcd` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:19.913257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=4cd6dbeb05584f2f864fcf980fd09dcd)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:19.915596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:19.949892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '4cd6dbeb05584f2f864fcf980fd09dcd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:19.950961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '4cd6dbeb05584f2f864fcf980fd09dcd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:19.952649 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2635/5201] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:19 2022] POST /identity/v3/users => generated 295 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:19.953748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509a109-ef95-4cf4-8f10-eb60e23fdde5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:19.955716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f509a109-ef95-4cf4-8f10-eb60e23fdde5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:19.997832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '4cd6dbeb05584f2f864fcf980fd09dcd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:19.998728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '4cd6dbeb05584f2f864fcf980fd09dcd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:19.999440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509a109-ef95-4cf4-8f10-eb60e23fdde5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:20.011404 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2567/5202] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:08:19 2022] PATCH /identity/v3/domains/4cd6dbeb05584f2f864fcf980fd09dcd => generated 278 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:20.013613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:20.016246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:20.067169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:20.137787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9308827-208c-4220-84bb-17d37cf17a5a tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:20.143507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea51b110f934255bc2e11b76760a730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c006ac088e114f36b850825686bde529', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:20.144187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:20.144477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:20.144797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] PATH_INFO: `/v3/domains/4cd6dbeb05584f2f864fcf980fd09dcd` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:20.158816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=4cd6dbeb05584f2f864fcf980fd09dcd)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:20.161636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:20.179466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e1711-8e94-4cf4-ab8e-ea26dbc39a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:20.180408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e1711-8e94-4cf4-ab8e-ea26dbc39a00 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:20.181074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e1711-8e94-4cf4-ab8e-ea26dbc39a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:20.181228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e1711-8e94-4cf4-ab8e-ea26dbc39a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:20.181916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e1711-8e94-4cf4-ab8e-ea26dbc39a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:20.183549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e1711-8e94-4cf4-ab8e-ea26dbc39a00 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:20.194658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '4cd6dbeb05584f2f864fcf980fd09dcd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.195920 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2636/5203] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:20.198987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '4cd6dbeb05584f2f864fcf980fd09dcd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.200545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e1711-8e94-4cf4-ab8e-ea26dbc39a00 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:20.202814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-be6e1711-8e94-4cf4-ab8e-ea26dbc39a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:20.222788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e1711-8e94-4cf4-ab8e-ea26dbc39a00 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:20.229904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3762d4-0f7d-4d20-ab84-57c55f1329ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:20.231102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3762d4-0f7d-4d20-ab84-57c55f1329ae admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:20.231586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3762d4-0f7d-4d20-ab84-57c55f1329ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:20.232080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3762d4-0f7d-4d20-ab84-57c55f1329ae admin admin] PATH_INFO: `/v3/projects/4ad55e8b308a45aea31c892a3f1a2623/users/f9cb58b8660a49f9b64349994d358a6a/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:20.259277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3762d4-0f7d-4d20-ab84-57c55f1329ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad55e8b308a45aea31c892a3f1a2623, user_id=f9cb58b8660a49f9b64349994d358a6a, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:20.265734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3762d4-0f7d-4d20-ab84-57c55f1329ae admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:20.270706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd6dbeb05584f2f864fcf980fd09dcd', 'request_id': 'req-4395a371-0626-4da7-a50e-2cad9d105109'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.280439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd6dbeb05584f2f864fcf980fd09dcd', 'request_id': 'req-4395a371-0626-4da7-a50e-2cad9d105109'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.289036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd6dbeb05584f2f864fcf980fd09dcd', 'request_id': 'req-4395a371-0626-4da7-a50e-2cad9d105109'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.290752 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2637/5204] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:20 2022] PUT /identity/v3/projects/4ad55e8b308a45aea31c892a3f1a2623/users/f9cb58b8660a49f9b64349994d358a6a/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:20.296295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3762d4-0f7d-4d20-ab84-57c55f1329ae admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:20.301559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] The token cache is being invalidate because project 4cd6dbeb05584f2f864fcf980fd09dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:20.301874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.302677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.312452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '4cd6dbeb05584f2f864fcf980fd09dcd', 'request_id': 'req-4395a371-0626-4da7-a50e-2cad9d105109'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.317648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '4cd6dbeb05584f2f864fcf980fd09dcd', 'request_id': 'req-4395a371-0626-4da7-a50e-2cad9d105109'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.319116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '4cd6dbeb05584f2f864fcf980fd09dcd', 'request_id': 'req-4395a371-0626-4da7-a50e-2cad9d105109'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.325254 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2568/5205] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:20 2022] DELETE /identity/v3/domains/4cd6dbeb05584f2f864fcf980fd09dcd => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:20.329371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:20.332143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:20.452244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395a371-0626-4da7-a50e-2cad9d105109 tempest-GroupsV3TestJSON-673868280 tempest-GroupsV3TestJSON-673868280-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:20.455937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3762d4-0f7d-4d20-ab84-57c55f1329ae admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:20.458440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f248b6-1d41-4015-b011-a43cd3256999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:20.462822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f248b6-1d41-4015-b011-a43cd3256999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:20.463115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f248b6-1d41-4015-b011-a43cd3256999 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:20.463877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f248b6-1d41-4015-b011-a43cd3256999 admin admin] PATH_INFO: `/v3/users/938351e16aaf43f896f06c7ffdb18154` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:20.465134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f248b6-1d41-4015-b011-a43cd3256999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=938351e16aaf43f896f06c7ffdb18154)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:20.466955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f248b6-1d41-4015-b011-a43cd3256999 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:20.474449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6271a83-c3c4-430c-9538-fef7b586e840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:20.475691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6271a83-c3c4-430c-9538-fef7b586e840 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:20.476175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6271a83-c3c4-430c-9538-fef7b586e840 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:20.476654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6271a83-c3c4-430c-9538-fef7b586e840 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:20.480802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6271a83-c3c4-430c-9538-fef7b586e840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:20.488328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6271a83-c3c4-430c-9538-fef7b586e840 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:20.505492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:20.516665 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2638/5206] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:20.518359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6271a83-c3c4-430c-9538-fef7b586e840 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:20.525798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c6271a83-c3c4-430c-9538-fef7b586e840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:20.566875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6271a83-c3c4-430c-9538-fef7b586e840 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:20.580623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715db644-454f-4693-a153-ff98b05c7b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:20.582143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715db644-454f-4693-a153-ff98b05c7b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:20.582665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715db644-454f-4693-a153-ff98b05c7b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:20.583369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715db644-454f-4693-a153-ff98b05c7b01 admin admin] PATH_INFO: `/v3/projects/4ad55e8b308a45aea31c892a3f1a2623/users/f9cb58b8660a49f9b64349994d358a6a/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:20.586671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715db644-454f-4693-a153-ff98b05c7b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad55e8b308a45aea31c892a3f1a2623, user_id=f9cb58b8660a49f9b64349994d358a6a, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:20.589422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b8f248b6-1d41-4015-b011-a43cd3256999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '938351e16aaf43f896f06c7ffdb18154'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.593190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715db644-454f-4693-a153-ff98b05c7b01 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:20.613414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b8f248b6-1d41-4015-b011-a43cd3256999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '938351e16aaf43f896f06c7ffdb18154'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.616779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b8f248b6-1d41-4015-b011-a43cd3256999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '938351e16aaf43f896f06c7ffdb18154'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.628943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b8f248b6-1d41-4015-b011-a43cd3256999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '938351e16aaf43f896f06c7ffdb18154'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.639824 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2639/5207] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:20 2022] PUT /identity/v3/projects/4ad55e8b308a45aea31c892a3f1a2623/users/f9cb58b8660a49f9b64349994d358a6a/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:20.641810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee6dbb-96f8-4d64-9551-2ab8ce6503b8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:20.642304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee6dbb-96f8-4d64-9551-2ab8ce6503b8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:20.642623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee6dbb-96f8-4d64-9551-2ab8ce6503b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:20.644591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b8f248b6-1d41-4015-b011-a43cd3256999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '938351e16aaf43f896f06c7ffdb18154'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:20.656362 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2569/5208] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:20 2022] DELETE /identity/v3/users/938351e16aaf43f896f06c7ffdb18154 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:20.658095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472d416-544c-46b0-bbf0-6b1f8c720479 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:20.658592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472d416-544c-46b0-bbf0-6b1f8c720479 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:20.659049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472d416-544c-46b0-bbf0-6b1f8c720479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:20.682339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-abee6dbb-96f8-4d64-9551-2ab8ce6503b8 None None] MFA Rules not processed for user `f9cb58b8660a49f9b64349994d358a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:20.683843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-abee6dbb-96f8-4d64-9551-2ab8ce6503b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:20.686450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1472d416-544c-46b0-bbf0-6b1f8c720479 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:20.687884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1472d416-544c-46b0-bbf0-6b1f8c720479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:20.719321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:20.769876 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2570/5209] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:20.788670 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2640/5210] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:20.790104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472d416-544c-46b0-bbf0-6b1f8c720479 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:20.791936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1472d416-544c-46b0-bbf0-6b1f8c720479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:20.793103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee6dbb-96f8-4d64-9551-2ab8ce6503b8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:20.808123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee6dbb-96f8-4d64-9551-2ab8ce6503b8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:20.816775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7520b6-9c3b-40fb-9d02-8f0e8234a4dc tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9cb58b8660a49f9b64349994d358a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad55e8b308a45aea31c892a3f1a2623', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:20.817795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7520b6-9c3b-40fb-9d02-8f0e8234a4dc tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:20.818762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7520b6-9c3b-40fb-9d02-8f0e8234a4dc tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:20.819259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7520b6-9c3b-40fb-9d02-8f0e8234a4dc tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:20.820405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7520b6-9c3b-40fb-9d02-8f0e8234a4dc tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:20.823446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7520b6-9c3b-40fb-9d02-8f0e8234a4dc tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:20.875442 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2641/5211] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:20 2022] POST /identity/v3/projects => generated 334 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:20.890901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7520b6-9c3b-40fb-9d02-8f0e8234a4dc tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:20.931342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472d416-544c-46b0-bbf0-6b1f8c720479 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:20.984230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7520b6-9c3b-40fb-9d02-8f0e8234a4dc tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:20.992232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13296d3-c603-408f-b983-ac4b00483fbf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9cb58b8660a49f9b64349994d358a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad55e8b308a45aea31c892a3f1a2623', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:20.992728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13296d3-c603-408f-b983-ac4b00483fbf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:20.993044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13296d3-c603-408f-b983-ac4b00483fbf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:20.993326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13296d3-c603-408f-b983-ac4b00483fbf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:20.994256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13296d3-c603-408f-b983-ac4b00483fbf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:20.996071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13296d3-c603-408f-b983-ac4b00483fbf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:21.013402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:21.029642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be683378-7db6-4faf-892b-ed199dedff6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:21.030272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be683378-7db6-4faf-892b-ed199dedff6a service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:21.030561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be683378-7db6-4faf-892b-ed199dedff6a service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:21.030844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be683378-7db6-4faf-892b-ed199dedff6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:21.052342 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2642/5212] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:08:20 2022] POST /identity/v3/projects => generated 333 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:21.053725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13296d3-c603-408f-b983-ac4b00483fbf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:21.142824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be683378-7db6-4faf-892b-ed199dedff6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:21.145343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be683378-7db6-4faf-892b-ed199dedff6a service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:21.174204 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2571/5213] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:08:21.178282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13296d3-c603-408f-b983-ac4b00483fbf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:21.183453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb316169-6b83-4a4d-9842-0fc7b4b886cf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9cb58b8660a49f9b64349994d358a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad55e8b308a45aea31c892a3f1a2623', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:21.184078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb316169-6b83-4a4d-9842-0fc7b4b886cf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:21.184468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb316169-6b83-4a4d-9842-0fc7b4b886cf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:21.184757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb316169-6b83-4a4d-9842-0fc7b4b886cf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:21.185425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb316169-6b83-4a4d-9842-0fc7b4b886cf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:21.187600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb316169-6b83-4a4d-9842-0fc7b4b886cf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:21.214355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:21.231291 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2643/5214] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:21 2022] POST /identity/v3/projects => generated 356 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:21.235504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be683378-7db6-4faf-892b-ed199dedff6a service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:21.248029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be683378-7db6-4faf-892b-ed199dedff6a service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:21.314392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bd5ce-6fbd-4ff2-86a1-f7e6499b6972 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9cb58b8660a49f9b64349994d358a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad55e8b308a45aea31c892a3f1a2623', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:21.314801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bd5ce-6fbd-4ff2-86a1-f7e6499b6972 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:21.315095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bd5ce-6fbd-4ff2-86a1-f7e6499b6972 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:21.315380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bd5ce-6fbd-4ff2-86a1-f7e6499b6972 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:21.316644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bd5ce-6fbd-4ff2-86a1-f7e6499b6972 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:21.318576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bd5ce-6fbd-4ff2-86a1-f7e6499b6972 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:21.342460 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2572/5215] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Sep 3 21:08:21 2022] GET /identity/v3/projects => generated 7162 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:21.343435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb316169-6b83-4a4d-9842-0fc7b4b886cf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:21.357006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb316169-6b83-4a4d-9842-0fc7b4b886cf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:21.364249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec4e68-e969-42af-a745-b8a65623a3bf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9cb58b8660a49f9b64349994d358a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad55e8b308a45aea31c892a3f1a2623', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:21.364984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec4e68-e969-42af-a745-b8a65623a3bf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:21.365405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec4e68-e969-42af-a745-b8a65623a3bf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:21.365759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec4e68-e969-42af-a745-b8a65623a3bf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:21.366457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec4e68-e969-42af-a745-b8a65623a3bf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:21.366987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec4e68-e969-42af-a745-b8a65623a3bf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:21.370330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec4e68-e969-42af-a745-b8a65623a3bf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:21.396203 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2644/5216] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Sep 3 21:08:21 2022] GET /identity/v3/projects?enabled=False => generated 445 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:21.397747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bd5ce-6fbd-4ff2-86a1-f7e6499b6972 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:21.408231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bd5ce-6fbd-4ff2-86a1-f7e6499b6972 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:21.413924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f0c97-9bc9-4c1b-8a7a-e7867ede128c tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9cb58b8660a49f9b64349994d358a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad55e8b308a45aea31c892a3f1a2623', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:21.414586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f0c97-9bc9-4c1b-8a7a-e7867ede128c tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:21.415135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f0c97-9bc9-4c1b-8a7a-e7867ede128c tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:21.415514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f0c97-9bc9-4c1b-8a7a-e7867ede128c tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:21.417734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f0c97-9bc9-4c1b-8a7a-e7867ede128c tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Adding query filter params (parent_id=2b934b774b10409f90cdf73d1fc9bf9a) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:21.418060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f0c97-9bc9-4c1b-8a7a-e7867ede128c tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:21.419952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f0c97-9bc9-4c1b-8a7a-e7867ede128c tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:21.472510 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2573/5217] 10.222.0.16 () {58 vars in 1216 bytes} [Sat Sep 3 21:08:21 2022] GET /identity/v3/projects?parent_id=2b934b774b10409f90cdf73d1fc9bf9a => generated 496 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:21.474455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec4e68-e969-42af-a745-b8a65623a3bf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:21.491487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec4e68-e969-42af-a745-b8a65623a3bf tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:21.501836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9cb58b8660a49f9b64349994d358a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad55e8b308a45aea31c892a3f1a2623', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:21.502766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:21.503070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:21.504310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] PATH_INFO: `/v3/projects/c1515191d45c46e883f1eaf0c2aaf205` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:21.506552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c1515191d45c46e883f1eaf0c2aaf205)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:21.509158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:21.593539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1515191d45c46e883f1eaf0c2aaf205', 'request_id': 'req-f76bed21-2f53-449d-a90a-5c9c2b021e01'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:21.609966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1515191d45c46e883f1eaf0c2aaf205', 'request_id': 'req-f76bed21-2f53-449d-a90a-5c9c2b021e01'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:21.617717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1515191d45c46e883f1eaf0c2aaf205', 'request_id': 'req-f76bed21-2f53-449d-a90a-5c9c2b021e01'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:21.646699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] The token cache is being invalidate because project c1515191d45c46e883f1eaf0c2aaf205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:21.646968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:21.648204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:21.651343 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2645/5218] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:21 2022] DELETE /identity/v3/projects/c1515191d45c46e883f1eaf0c2aaf205 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:21.655077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f0c97-9bc9-4c1b-8a7a-e7867ede128c tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:21.656884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-554f0c97-9bc9-4c1b-8a7a-e7867ede128c tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:21.674204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f0c97-9bc9-4c1b-8a7a-e7867ede128c tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:21.758295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9cb58b8660a49f9b64349994d358a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad55e8b308a45aea31c892a3f1a2623', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:21.759383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:21.759778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:21.760138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] PATH_INFO: `/v3/projects/2b934b774b10409f90cdf73d1fc9bf9a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:21.761666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2b934b774b10409f90cdf73d1fc9bf9a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:21.763741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:21.773941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:21.775787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:21.813147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:21.848573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b934b774b10409f90cdf73d1fc9bf9a', 'request_id': 'req-98b7717a-e019-4058-95e0-b8a2c94c6f56'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:21.860602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b934b774b10409f90cdf73d1fc9bf9a', 'request_id': 'req-98b7717a-e019-4058-95e0-b8a2c94c6f56'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:21.882994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b934b774b10409f90cdf73d1fc9bf9a', 'request_id': 'req-98b7717a-e019-4058-95e0-b8a2c94c6f56'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:21.901290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] The token cache is being invalidate because project 2b934b774b10409f90cdf73d1fc9bf9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:21.901476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:21.902174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:21.906145 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2574/5219] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:21 2022] DELETE /identity/v3/projects/2b934b774b10409f90cdf73d1fc9bf9a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:21.908534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:21.911542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:21.941622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bed21-2f53-449d-a90a-5c9c2b021e01 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:22.000193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7717a-e019-4058-95e0-b8a2c94c6f56 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:22.005723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9cb58b8660a49f9b64349994d358a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad55e8b308a45aea31c892a3f1a2623', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:22.006416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:22.006706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:22.007014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] PATH_INFO: `/v3/projects/82f1b6c1c8a54123bd18817fba52e941` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:22.010193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorizing `identity:delete_project(project_id=82f1b6c1c8a54123bd18817fba52e941)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:22.014079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:22.032803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:22.033720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:22.034062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:22.034354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] PATH_INFO: `/v3/projects/23cf0acc0a814fb59b73fe5066fb384c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:22.036691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] RBAC: Authorizing `identity:delete_project(project_id=23cf0acc0a814fb59b73fe5066fb384c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:22.037652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:22.061943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:22.089876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f1b6c1c8a54123bd18817fba52e941', 'request_id': 'req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.098185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f1b6c1c8a54123bd18817fba52e941', 'request_id': 'req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.108849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f1b6c1c8a54123bd18817fba52e941', 'request_id': 'req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.121299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23cf0acc0a814fb59b73fe5066fb384c', 'request_id': 'req-a869d451-16f8-4f99-8b55-ebd7b974427e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.130481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23cf0acc0a814fb59b73fe5066fb384c', 'request_id': 'req-a869d451-16f8-4f99-8b55-ebd7b974427e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.140123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] The token cache is being invalidate because project 82f1b6c1c8a54123bd18817fba52e941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:22.140423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.141183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.143868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23cf0acc0a814fb59b73fe5066fb384c', 'request_id': 'req-a869d451-16f8-4f99-8b55-ebd7b974427e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.144021 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2575/5220] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:21 2022] DELETE /identity/v3/projects/82f1b6c1c8a54123bd18817fba52e941 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:22.154238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:22.156310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:22.159502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] The token cache is being invalidate because project 23cf0acc0a814fb59b73fe5066fb384c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:22.159783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.160453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.162880 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2646/5221] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:21 2022] DELETE /identity/v3/projects/23cf0acc0a814fb59b73fe5066fb384c => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:22.167994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:22.171060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:22.241494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:22.283802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864dec6-1ad5-4ae4-8374-95756d7e2ed1 tempest-ListProjectsTestJSON-1837130067 tempest-ListProjectsTestJSON-1837130067-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:22.290588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:22.291261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:22.291536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869d451-16f8-4f99-8b55-ebd7b974427e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:22.291669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:22.291834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] PATH_INFO: `/v3/users/8e00742eb8c24d9cba3e840f36527fc3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:22.293162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e00742eb8c24d9cba3e840f36527fc3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:22.295515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:22.299036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248f7eb-1d11-4f46-988b-eded760b813d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:22.299949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248f7eb-1d11-4f46-988b-eded760b813d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:22.300733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248f7eb-1d11-4f46-988b-eded760b813d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:22.301213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248f7eb-1d11-4f46-988b-eded760b813d admin admin] PATH_INFO: `/v3/users/fea51b110f934255bc2e11b76760a730` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:22.302915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248f7eb-1d11-4f46-988b-eded760b813d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea51b110f934255bc2e11b76760a730)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:22.305275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248f7eb-1d11-4f46-988b-eded760b813d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:22.377091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e00742eb8c24d9cba3e840f36527fc3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.380059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1248f7eb-1d11-4f46-988b-eded760b813d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea51b110f934255bc2e11b76760a730'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.401886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e00742eb8c24d9cba3e840f36527fc3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.402864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e00742eb8c24d9cba3e840f36527fc3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.403657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1248f7eb-1d11-4f46-988b-eded760b813d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea51b110f934255bc2e11b76760a730'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.413509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e00742eb8c24d9cba3e840f36527fc3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.414481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1248f7eb-1d11-4f46-988b-eded760b813d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea51b110f934255bc2e11b76760a730'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.415772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1248f7eb-1d11-4f46-988b-eded760b813d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea51b110f934255bc2e11b76760a730'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.417495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1248f7eb-1d11-4f46-988b-eded760b813d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea51b110f934255bc2e11b76760a730'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.425479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e00742eb8c24d9cba3e840f36527fc3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:22.438053 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2647/5223] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:22 2022] DELETE /identity/v3/users/fea51b110f934255bc2e11b76760a730 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:22.439223 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2576/5223] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:22 2022] DELETE /identity/v3/users/8e00742eb8c24d9cba3e840f36527fc3 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:22.440445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc4b5a-f287-446e-b6f3-77512d1fc239 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:22.441346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc4b5a-f287-446e-b6f3-77512d1fc239 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:22.441633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc4b5a-f287-446e-b6f3-77512d1fc239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:22.464706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c5bc4b5a-f287-446e-b6f3-77512d1fc239 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:22.466608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c5bc4b5a-f287-446e-b6f3-77512d1fc239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:22.489007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:22.559946 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2648/5224] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:22.580544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:22.584053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:22.664735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063d8a1-446a-4569-8f6f-6c9850f2f552 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:22.739283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b98073-821e-4d0b-adef-d58ed7ba0bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:22.740687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98073-821e-4d0b-adef-d58ed7ba0bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:22.741013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98073-821e-4d0b-adef-d58ed7ba0bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:22.741297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98073-821e-4d0b-adef-d58ed7ba0bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:22.753062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b98073-821e-4d0b-adef-d58ed7ba0bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:22.756716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b98073-821e-4d0b-adef-d58ed7ba0bb3 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:22.767774 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2577/5225] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:23.164003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc4b5a-f287-446e-b6f3-77512d1fc239 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:23.165525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c5bc4b5a-f287-446e-b6f3-77512d1fc239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:23.204835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc4b5a-f287-446e-b6f3-77512d1fc239 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:23.229118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3381df-a874-4a01-86ee-c38d26e22f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:23.229853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3381df-a874-4a01-86ee-c38d26e22f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:23.230096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3381df-a874-4a01-86ee-c38d26e22f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:23.230381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3381df-a874-4a01-86ee-c38d26e22f2f admin admin] PATH_INFO: `/v3/projects/c006ac088e114f36b850825686bde529` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:23.234516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3381df-a874-4a01-86ee-c38d26e22f2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c006ac088e114f36b850825686bde529)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:23.240246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3381df-a874-4a01-86ee-c38d26e22f2f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:23.348609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b3381df-a874-4a01-86ee-c38d26e22f2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c006ac088e114f36b850825686bde529', 'request_id': 'req-1b3381df-a874-4a01-86ee-c38d26e22f2f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:23.368940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b3381df-a874-4a01-86ee-c38d26e22f2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c006ac088e114f36b850825686bde529', 'request_id': 'req-1b3381df-a874-4a01-86ee-c38d26e22f2f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:23.381896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b3381df-a874-4a01-86ee-c38d26e22f2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c006ac088e114f36b850825686bde529', 'request_id': 'req-1b3381df-a874-4a01-86ee-c38d26e22f2f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:23.405071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b3381df-a874-4a01-86ee-c38d26e22f2f admin admin] The token cache is being invalidate because project c006ac088e114f36b850825686bde529 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:23.405378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b3381df-a874-4a01-86ee-c38d26e22f2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:23.410177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b3381df-a874-4a01-86ee-c38d26e22f2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:23.412745 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2649/5226] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:23 2022] DELETE /identity/v3/projects/c006ac088e114f36b850825686bde529 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:23.417699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e94bab-a85b-42e2-b433-810cdf96031d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:23.418467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e94bab-a85b-42e2-b433-810cdf96031d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:23.418754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e94bab-a85b-42e2-b433-810cdf96031d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:23.463632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-98e94bab-a85b-42e2-b433-810cdf96031d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:23.466257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-98e94bab-a85b-42e2-b433-810cdf96031d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:23.482367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:23.557203 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2578/5227] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:23.566124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6708651-4806-4d50-b704-ccf412a050b1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:23.566508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6708651-4806-4d50-b704-ccf412a050b1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:23.567135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6708651-4806-4d50-b704-ccf412a050b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:23.597236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c6708651-4806-4d50-b704-ccf412a050b1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:23.598802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c6708651-4806-4d50-b704-ccf412a050b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:23.619962 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2650/5228] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:23.621227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e94bab-a85b-42e2-b433-810cdf96031d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:23.632238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e94bab-a85b-42e2-b433-810cdf96031d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:23.640570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578eeb9e-fe96-4cb9-a997-f79cddc2fe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:23.641266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578eeb9e-fe96-4cb9-a997-f79cddc2fe1b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:23.641554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578eeb9e-fe96-4cb9-a997-f79cddc2fe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:23.641832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578eeb9e-fe96-4cb9-a997-f79cddc2fe1b admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:23.642561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578eeb9e-fe96-4cb9-a997-f79cddc2fe1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:23.642874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578eeb9e-fe96-4cb9-a997-f79cddc2fe1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:23.644543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578eeb9e-fe96-4cb9-a997-f79cddc2fe1b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:23.660959 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2579/5229] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:23.662833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6708651-4806-4d50-b704-ccf412a050b1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:23.673192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6708651-4806-4d50-b704-ccf412a050b1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:23.681311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372c286-e8f6-4eb9-a76a-fbc1e094832a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:23.682420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372c286-e8f6-4eb9-a76a-fbc1e094832a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:23.682851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372c286-e8f6-4eb9-a76a-fbc1e094832a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:23.683142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372c286-e8f6-4eb9-a76a-fbc1e094832a admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:23.683793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372c286-e8f6-4eb9-a76a-fbc1e094832a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:23.686044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372c286-e8f6-4eb9-a76a-fbc1e094832a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:23.725567 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2651/5230] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:23 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:23.734568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578eeb9e-fe96-4cb9-a997-f79cddc2fe1b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:23.750029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578eeb9e-fe96-4cb9-a997-f79cddc2fe1b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:23.813825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372c286-e8f6-4eb9-a76a-fbc1e094832a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:23.815619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4372c286-e8f6-4eb9-a76a-fbc1e094832a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:23.831183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8046b24-6698-4105-a38b-4f895d1d6e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:23.831183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8046b24-6698-4105-a38b-4f895d1d6e60 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:23.831183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8046b24-6698-4105-a38b-4f895d1d6e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:23.831619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8046b24-6698-4105-a38b-4f895d1d6e60 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:23.832875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8046b24-6698-4105-a38b-4f895d1d6e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:23.834865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8046b24-6698-4105-a38b-4f895d1d6e60 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:23.872210 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2580/5231] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:23 2022] POST /identity/v3/users => generated 285 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:23.873314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8046b24-6698-4105-a38b-4f895d1d6e60 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:23.884708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8046b24-6698-4105-a38b-4f895d1d6e60 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:23.890084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ab927-367e-4e63-b7e3-c9520e2680ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:23.890699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ab927-367e-4e63-b7e3-c9520e2680ef admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:23.890987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ab927-367e-4e63-b7e3-c9520e2680ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:23.891274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ab927-367e-4e63-b7e3-c9520e2680ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:23.891907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ab927-367e-4e63-b7e3-c9520e2680ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:23.893542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ab927-367e-4e63-b7e3-c9520e2680ef admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:23.904782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372c286-e8f6-4eb9-a76a-fbc1e094832a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:23.910393 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2581/5232] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:23.915459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:23.916133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:23.916426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:23.916710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] PATH_INFO: `/v3/projects/ac601057cde6448b859c13a864271d00` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:23.918147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac601057cde6448b859c13a864271d00)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:23.918271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ab927-367e-4e63-b7e3-c9520e2680ef admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:23.920106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:23.937813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ab927-367e-4e63-b7e3-c9520e2680ef admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:23.943098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f53e1-f647-4a3e-b7df-1eec8994328a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:23.943894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f53e1-f647-4a3e-b7df-1eec8994328a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:23.944944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f53e1-f647-4a3e-b7df-1eec8994328a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:23.944944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f53e1-f647-4a3e-b7df-1eec8994328a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:23.945278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f53e1-f647-4a3e-b7df-1eec8994328a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:23.946873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f53e1-f647-4a3e-b7df-1eec8994328a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:23.950033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:23.963495 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2582/5233] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:23.966493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f53e1-f647-4a3e-b7df-1eec8994328a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:23.981489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f53e1-f647-4a3e-b7df-1eec8994328a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:23.990168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac601057cde6448b859c13a864271d00', 'request_id': 'req-02473ce0-962e-468f-8a9f-2c9490737892'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:23.998639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac601057cde6448b859c13a864271d00', 'request_id': 'req-02473ce0-962e-468f-8a9f-2c9490737892'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:24.007524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac601057cde6448b859c13a864271d00', 'request_id': 'req-02473ce0-962e-468f-8a9f-2c9490737892'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:24.022746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] The token cache is being invalidate because project ac601057cde6448b859c13a864271d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:24.023065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:24.023728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:24.028363 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2652/5234] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:23 2022] DELETE /identity/v3/projects/ac601057cde6448b859c13a864271d00 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:24.029512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:24.030987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:24.050189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1b2e2-ce8b-4d0f-b540-d1bd484de30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:24.050951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1b2e2-ce8b-4d0f-b540-d1bd484de30b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:24.051302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1b2e2-ce8b-4d0f-b540-d1bd484de30b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:24.051652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1b2e2-ce8b-4d0f-b540-d1bd484de30b admin admin] PATH_INFO: `/v3/projects/5615a74cb51947208af83a65c64dc0de/users/a388bfebf5f544b2b900e4b950b00a51/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:24.078093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1b2e2-ce8b-4d0f-b540-d1bd484de30b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5615a74cb51947208af83a65c64dc0de, user_id=a388bfebf5f544b2b900e4b950b00a51, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:24.080788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1b2e2-ce8b-4d0f-b540-d1bd484de30b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:24.093811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:24.115278 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2583/5235] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:23 2022] PUT /identity/v3/projects/5615a74cb51947208af83a65c64dc0de/users/a388bfebf5f544b2b900e4b950b00a51/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:24.126088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385a4dd-2ac4-4cba-b647-a2eeddab55e3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:24.126365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385a4dd-2ac4-4cba-b647-a2eeddab55e3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:24.126633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385a4dd-2ac4-4cba-b647-a2eeddab55e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:24.135056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02473ce0-962e-468f-8a9f-2c9490737892 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:24.170796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b385a4dd-2ac4-4cba-b647-a2eeddab55e3 None None] MFA Rules not processed for user `a388bfebf5f544b2b900e4b950b00a51`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:24.172515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b385a4dd-2ac4-4cba-b647-a2eeddab55e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:24.223736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:24.224379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:24.225028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:24.225460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] PATH_INFO: `/v3/users/f9cb58b8660a49f9b64349994d358a6a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:24.226751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9cb58b8660a49f9b64349994d358a6a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:24.233163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:24.267156 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2584/5236] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:24.271372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385a4dd-2ac4-4cba-b647-a2eeddab55e3 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:24.273017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b385a4dd-2ac4-4cba-b647-a2eeddab55e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:24.288588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385a4dd-2ac4-4cba-b647-a2eeddab55e3 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:24.293295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9700f7-01c1-4fe5-bd28-0f9b35b620d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:24.293922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9700f7-01c1-4fe5-bd28-0f9b35b620d5 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:24.294212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9700f7-01c1-4fe5-bd28-0f9b35b620d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:24.294491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9700f7-01c1-4fe5-bd28-0f9b35b620d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:24.295589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9700f7-01c1-4fe5-bd28-0f9b35b620d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:24.297104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9700f7-01c1-4fe5-bd28-0f9b35b620d5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:24.298492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9cb58b8660a49f9b64349994d358a6a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:24.309127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:24.317988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9cb58b8660a49f9b64349994d358a6a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:24.327189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9cb58b8660a49f9b64349994d358a6a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:24.327856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9cb58b8660a49f9b64349994d358a6a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:24.328517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9cb58b8660a49f9b64349994d358a6a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:24.330317 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2585/5237] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:24 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:24.334335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9700f7-01c1-4fe5-bd28-0f9b35b620d5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:24.336818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7f9700f7-01c1-4fe5-bd28-0f9b35b620d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:24.351645 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2653/5238] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:24 2022] DELETE /identity/v3/users/f9cb58b8660a49f9b64349994d358a6a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:24.414790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9700f7-01c1-4fe5-bd28-0f9b35b620d5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:24.482487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce28e11-9716-4b07-b6c1-f1223dd98722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:24.493928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce28e11-9716-4b07-b6c1-f1223dd98722 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:24.494450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce28e11-9716-4b07-b6c1-f1223dd98722 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:24.494797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce28e11-9716-4b07-b6c1-f1223dd98722 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:24.495609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce28e11-9716-4b07-b6c1-f1223dd98722 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:24.497797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce28e11-9716-4b07-b6c1-f1223dd98722 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:24.520003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:24.528667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:24.540515 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2586/5239] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:24 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:24.541581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d89993-8211-47e4-88ab-b6d5e102f3d7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:24.547787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0918e50-e2e1-4031-8007-11e162b84943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:24.548410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0918e50-e2e1-4031-8007-11e162b84943 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:24.548690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0918e50-e2e1-4031-8007-11e162b84943 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:24.549011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0918e50-e2e1-4031-8007-11e162b84943 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:24.549669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0918e50-e2e1-4031-8007-11e162b84943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:24.552378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0918e50-e2e1-4031-8007-11e162b84943 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:24.570081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce28e11-9716-4b07-b6c1-f1223dd98722 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:24.585858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce28e11-9716-4b07-b6c1-f1223dd98722 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:24.586827 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2654/5240] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:24.593144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609dbcd-33ef-48e1-bdc2-8ec1f37ceb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:24.593835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609dbcd-33ef-48e1-bdc2-8ec1f37ceb11 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:24.594127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609dbcd-33ef-48e1-bdc2-8ec1f37ceb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:24.594411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609dbcd-33ef-48e1-bdc2-8ec1f37ceb11 admin admin] PATH_INFO: `/v3/projects/346a136fb78c425aa009ac5c260ee137/users/33e913bc7e6c4f6b9d15e3fdb946b644/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:24.614221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609dbcd-33ef-48e1-bdc2-8ec1f37ceb11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=346a136fb78c425aa009ac5c260ee137, user_id=33e913bc7e6c4f6b9d15e3fdb946b644, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:24.616963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609dbcd-33ef-48e1-bdc2-8ec1f37ceb11 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:24.632960 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2587/5241] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:24 2022] PUT /identity/v3/projects/346a136fb78c425aa009ac5c260ee137/users/33e913bc7e6c4f6b9d15e3fdb946b644/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:24.638919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0918e50-e2e1-4031-8007-11e162b84943 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:24.662098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0918e50-e2e1-4031-8007-11e162b84943 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:24.680387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609dbcd-33ef-48e1-bdc2-8ec1f37ceb11 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:24.682317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b609dbcd-33ef-48e1-bdc2-8ec1f37ceb11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:24.779874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd51690-b71c-444c-9ef5-4157b288785b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:24.780718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd51690-b71c-444c-9ef5-4157b288785b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:24.781043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd51690-b71c-444c-9ef5-4157b288785b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:24.781419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd51690-b71c-444c-9ef5-4157b288785b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:24.782147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd51690-b71c-444c-9ef5-4157b288785b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:24.784314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd51690-b71c-444c-9ef5-4157b288785b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:24.803656 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2655/5242] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:24.804554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd51690-b71c-444c-9ef5-4157b288785b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:24.821372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd51690-b71c-444c-9ef5-4157b288785b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:24.829374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609dbcd-33ef-48e1-bdc2-8ec1f37ceb11 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:24.832947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4d7d6-28d8-479e-b093-1ca96e0cb206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:24.834308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4d7d6-28d8-479e-b093-1ca96e0cb206 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:24.834869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4d7d6-28d8-479e-b093-1ca96e0cb206 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:24.835326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4d7d6-28d8-479e-b093-1ca96e0cb206 admin admin] PATH_INFO: `/v3/projects/346a136fb78c425aa009ac5c260ee137/users/33e913bc7e6c4f6b9d15e3fdb946b644/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:24.837947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:24.838272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4d7d6-28d8-479e-b093-1ca96e0cb206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=346a136fb78c425aa009ac5c260ee137, user_id=33e913bc7e6c4f6b9d15e3fdb946b644, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:24.838924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:24.839321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:24.839708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89 admin admin] PATH_INFO: `/v3/projects/4ad55e8b308a45aea31c892a3f1a2623` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:24.841276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad55e8b308a45aea31c892a3f1a2623)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:24.843615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:24.845613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4d7d6-28d8-479e-b093-1ca96e0cb206 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:24.861262 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2656/5243] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:24 2022] PUT /identity/v3/projects/346a136fb78c425aa009ac5c260ee137/users/33e913bc7e6c4f6b9d15e3fdb946b644/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:24.873316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e138b-181d-4ba1-b6bf-524607b56106 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:24.873616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e138b-181d-4ba1-b6bf-524607b56106 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:24.873895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e138b-181d-4ba1-b6bf-524607b56106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:24.912763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-412e138b-181d-4ba1-b6bf-524607b56106 None None] MFA Rules not processed for user `33e913bc7e6c4f6b9d15e3fdb946b644`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:24.914304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-412e138b-181d-4ba1-b6bf-524607b56106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:24.946567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad55e8b308a45aea31c892a3f1a2623', 'request_id': 'req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:24.960994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad55e8b308a45aea31c892a3f1a2623', 'request_id': 'req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:24.980020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad55e8b308a45aea31c892a3f1a2623', 'request_id': 'req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:25.013077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89 admin admin] The token cache is being invalidate because project 4ad55e8b308a45aea31c892a3f1a2623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:25.013313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:25.016049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c21f5b57-4f98-43e8-a0e9-f9b4b36c2a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:25.021569 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2588/5244] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:24 2022] DELETE /identity/v3/projects/4ad55e8b308a45aea31c892a3f1a2623 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:25.026917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c353e2-8398-4e3b-938d-3790484d0bff None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.027203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c353e2-8398-4e3b-938d-3790484d0bff None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.029850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c353e2-8398-4e3b-938d-3790484d0bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.043207 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2657/5245] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:25.043924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e138b-181d-4ba1-b6bf-524607b56106 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.056661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e138b-181d-4ba1-b6bf-524607b56106 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.062164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da95c-500d-490b-8ff0-65e4030592be tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e913bc7e6c4f6b9d15e3fdb946b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '346a136fb78c425aa009ac5c260ee137', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.063276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da95c-500d-490b-8ff0-65e4030592be tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.063888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da95c-500d-490b-8ff0-65e4030592be tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.064177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da95c-500d-490b-8ff0-65e4030592be tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] PATH_INFO: `/v3/policies` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.065591 ncdv-master-1 devstack@keystone.service[99210]: WARNING oslo_log.versionutils [None req-3a0da95c-500d-490b-8ff0-65e4030592be tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 03 21:08:25.067269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0da95c-500d-490b-8ff0-65e4030592be tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.068614 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-3a0da95c-500d-490b-8ff0-65e4030592be tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:25.068614 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:25.068614 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:25.069611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0da95c-500d-490b-8ff0-65e4030592be tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.111950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-81c353e2-8398-4e3b-938d-3790484d0bff None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:25.113894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-81c353e2-8398-4e3b-938d-3790484d0bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:25.135363 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2658/5246] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:08:25 2022] POST /identity/v3/policies => generated 228 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:25.139934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da95c-500d-490b-8ff0-65e4030592be tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.141843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3a0da95c-500d-490b-8ff0-65e4030592be tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:25.155443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da95c-500d-490b-8ff0-65e4030592be tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.160306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470228b-d4fa-4c54-aab9-aa202a0df76c tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e913bc7e6c4f6b9d15e3fdb946b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '346a136fb78c425aa009ac5c260ee137', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.160984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470228b-d4fa-4c54-aab9-aa202a0df76c tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.161275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470228b-d4fa-4c54-aab9-aa202a0df76c tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.161942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470228b-d4fa-4c54-aab9-aa202a0df76c tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] PATH_INFO: `/v3/policies/43276993ca154ecb8bc507b9b8c1a3e8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.163481 ncdv-master-1 devstack@keystone.service[99210]: WARNING oslo_log.versionutils [None req-a470228b-d4fa-4c54-aab9-aa202a0df76c tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Deprecated: identity:update_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 03 21:08:25.164004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470228b-d4fa-4c54-aab9-aa202a0df76c tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorizing `identity:update_policy(policy_id=43276993ca154ecb8bc507b9b8c1a3e8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.165288 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-a470228b-d4fa-4c54-aab9-aa202a0df76c tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:25.165288 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:25.165288 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:25.166221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470228b-d4fa-4c54-aab9-aa202a0df76c tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.231780 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2659/5247] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Sep 3 21:08:25 2022] PATCH /identity/v3/policies/43276993ca154ecb8bc507b9b8c1a3e8 => generated 236 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:25.243295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470228b-d4fa-4c54-aab9-aa202a0df76c tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.259199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:25.270880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470228b-d4fa-4c54-aab9-aa202a0df76c tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.279838 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2589/5248] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:25.289597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546b5f2-fb5e-4858-9bc9-6699cfdb1946 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e913bc7e6c4f6b9d15e3fdb946b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '346a136fb78c425aa009ac5c260ee137', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.290254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546b5f2-fb5e-4858-9bc9-6699cfdb1946 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.290547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546b5f2-fb5e-4858-9bc9-6699cfdb1946 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.290833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546b5f2-fb5e-4858-9bc9-6699cfdb1946 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] PATH_INFO: `/v3/policies/43276993ca154ecb8bc507b9b8c1a3e8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.291643 ncdv-master-1 devstack@keystone.service[99210]: WARNING oslo_log.versionutils [None req-7546b5f2-fb5e-4858-9bc9-6699cfdb1946 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Deprecated: identity:get_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 03 21:08:25.292554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f0b53-c8cb-42d1-88de-a1921f27ce09 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.292850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f0b53-c8cb-42d1-88de-a1921f27ce09 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.293153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f0b53-c8cb-42d1-88de-a1921f27ce09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.295971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546b5f2-fb5e-4858-9bc9-6699cfdb1946 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorizing `identity:get_policy(policy_id=43276993ca154ecb8bc507b9b8c1a3e8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.297823 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-7546b5f2-fb5e-4858-9bc9-6699cfdb1946 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_policy": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:25.297823 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:25.297823 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:25.298743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546b5f2-fb5e-4858-9bc9-6699cfdb1946 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.309155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:25.321752 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2660/5249] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:25 2022] GET /identity/v3/policies/43276993ca154ecb8bc507b9b8c1a3e8 => generated 236 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:25.326622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546b5f2-fb5e-4858-9bc9-6699cfdb1946 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.331179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bd1f0b53-c8cb-42d1-88de-a1921f27ce09 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:25.332483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bd1f0b53-c8cb-42d1-88de-a1921f27ce09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:25.345010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546b5f2-fb5e-4858-9bc9-6699cfdb1946 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.345722 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2590/5250] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:25.350573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff696ec-a228-4dbb-831a-d7d7532704dd tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e913bc7e6c4f6b9d15e3fdb946b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '346a136fb78c425aa009ac5c260ee137', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.351230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff696ec-a228-4dbb-831a-d7d7532704dd tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.351522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff696ec-a228-4dbb-831a-d7d7532704dd tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.351803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff696ec-a228-4dbb-831a-d7d7532704dd tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] PATH_INFO: `/v3/policies/43276993ca154ecb8bc507b9b8c1a3e8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.352612 ncdv-master-1 devstack@keystone.service[99210]: WARNING oslo_log.versionutils [None req-eff696ec-a228-4dbb-831a-d7d7532704dd tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 03 21:08:25.353103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f0b53-c8cb-42d1-88de-a1921f27ce09 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.353351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff696ec-a228-4dbb-831a-d7d7532704dd tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=43276993ca154ecb8bc507b9b8c1a3e8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.354916 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-eff696ec-a228-4dbb-831a-d7d7532704dd tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:25.354916 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:25.354916 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:25.355941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff696ec-a228-4dbb-831a-d7d7532704dd tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.371087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f0b53-c8cb-42d1-88de-a1921f27ce09 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.378309 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2661/5251] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:25 2022] DELETE /identity/v3/policies/43276993ca154ecb8bc507b9b8c1a3e8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:25.380954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb0f25-dfd1-4644-9cf8-a811e6a612dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.381743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb0f25-dfd1-4644-9cf8-a811e6a612dd admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.382039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb0f25-dfd1-4644-9cf8-a811e6a612dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.382321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb0f25-dfd1-4644-9cf8-a811e6a612dd admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.384952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff696ec-a228-4dbb-831a-d7d7532704dd tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.385120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb0f25-dfd1-4644-9cf8-a811e6a612dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:25.385874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb0f25-dfd1-4644-9cf8-a811e6a612dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.393639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb0f25-dfd1-4644-9cf8-a811e6a612dd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.405441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff696ec-a228-4dbb-831a-d7d7532704dd tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.419462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6d45a-622c-4c8c-99ec-b27672c096eb tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e913bc7e6c4f6b9d15e3fdb946b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '346a136fb78c425aa009ac5c260ee137', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.421310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6d45a-622c-4c8c-99ec-b27672c096eb tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.422332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6d45a-622c-4c8c-99ec-b27672c096eb tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.422974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6d45a-622c-4c8c-99ec-b27672c096eb tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] PATH_INFO: `/v3/policies` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.427845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6d45a-622c-4c8c-99ec-b27672c096eb tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.428979 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2591/5252] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:25.431535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb0f25-dfd1-4644-9cf8-a811e6a612dd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.433352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6d45a-622c-4c8c-99ec-b27672c096eb tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.448093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb0f25-dfd1-4644-9cf8-a811e6a612dd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.456197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb01b04-5651-43b9-a863-f4b13f7fb732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.457398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb01b04-5651-43b9-a863-f4b13f7fb732 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.457872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb01b04-5651-43b9-a863-f4b13f7fb732 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.458351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb01b04-5651-43b9-a863-f4b13f7fb732 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.459317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb01b04-5651-43b9-a863-f4b13f7fb732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.462435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb01b04-5651-43b9-a863-f4b13f7fb732 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.463077 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2662/5253] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:08:25 2022] POST /identity/v3/policies => generated 229 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:25.478327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6d45a-622c-4c8c-99ec-b27672c096eb tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.495265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6d45a-622c-4c8c-99ec-b27672c096eb tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.513551 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2592/5254] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:25 2022] POST /identity/v3/projects => generated 365 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:25.517222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb01b04-5651-43b9-a863-f4b13f7fb732 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.518543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299c799-f709-4c9c-8b91-cd02881de8e8 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e913bc7e6c4f6b9d15e3fdb946b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '346a136fb78c425aa009ac5c260ee137', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.519419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299c799-f709-4c9c-8b91-cd02881de8e8 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.519817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299c799-f709-4c9c-8b91-cd02881de8e8 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.520679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299c799-f709-4c9c-8b91-cd02881de8e8 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] PATH_INFO: `/v3/policies` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.522486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299c799-f709-4c9c-8b91-cd02881de8e8 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.524563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299c799-f709-4c9c-8b91-cd02881de8e8 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.551296 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2663/5255] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:08:25 2022] POST /identity/v3/policies => generated 229 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:25.552706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299c799-f709-4c9c-8b91-cd02881de8e8 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.563821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299c799-f709-4c9c-8b91-cd02881de8e8 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.598350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb01b04-5651-43b9-a863-f4b13f7fb732 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.604187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c904b5-f3ed-494f-8712-190edbefa74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.604847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c904b5-f3ed-494f-8712-190edbefa74c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.605169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c904b5-f3ed-494f-8712-190edbefa74c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.605452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c904b5-f3ed-494f-8712-190edbefa74c admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.606102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c904b5-f3ed-494f-8712-190edbefa74c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.608201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c904b5-f3ed-494f-8712-190edbefa74c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.635998 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2593/5256] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:25 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:25.637080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe306cb4-d36c-452d-ac4f-497ff67a76fc tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e913bc7e6c4f6b9d15e3fdb946b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '346a136fb78c425aa009ac5c260ee137', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.637591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c904b5-f3ed-494f-8712-190edbefa74c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.637913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe306cb4-d36c-452d-ac4f-497ff67a76fc tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.638298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe306cb4-d36c-452d-ac4f-497ff67a76fc tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.638697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe306cb4-d36c-452d-ac4f-497ff67a76fc tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] PATH_INFO: `/v3/policies` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.639973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe306cb4-d36c-452d-ac4f-497ff67a76fc tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.642079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe306cb4-d36c-452d-ac4f-497ff67a76fc tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.651873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c904b5-f3ed-494f-8712-190edbefa74c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.657714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecee30-823a-4da4-b7c0-0d236ed0d51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.658467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecee30-823a-4da4-b7c0-0d236ed0d51e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.658823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecee30-823a-4da4-b7c0-0d236ed0d51e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.659196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecee30-823a-4da4-b7c0-0d236ed0d51e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.660610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecee30-823a-4da4-b7c0-0d236ed0d51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.662319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecee30-823a-4da4-b7c0-0d236ed0d51e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.664577 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2664/5257] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:08:25 2022] POST /identity/v3/policies => generated 230 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:25.668626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe306cb4-d36c-452d-ac4f-497ff67a76fc tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.674442 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2594/5258] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:25.679099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecee30-823a-4da4-b7c0-0d236ed0d51e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.683442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe306cb4-d36c-452d-ac4f-497ff67a76fc tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.688633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ac264-d5e8-41f4-bcf0-ce5f89a931b9 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e913bc7e6c4f6b9d15e3fdb946b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '346a136fb78c425aa009ac5c260ee137', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.689530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ac264-d5e8-41f4-bcf0-ce5f89a931b9 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.689825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ac264-d5e8-41f4-bcf0-ce5f89a931b9 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.690274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ac264-d5e8-41f4-bcf0-ce5f89a931b9 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] PATH_INFO: `/v3/policies` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.691104 ncdv-master-1 devstack@keystone.service[99210]: WARNING oslo_log.versionutils [None req-967ac264-d5e8-41f4-bcf0-ce5f89a931b9 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Deprecated: identity:list_policies of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 03 21:08:25.691462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecee30-823a-4da4-b7c0-0d236ed0d51e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.691673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ac264-d5e8-41f4-bcf0-ce5f89a931b9 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorizing `identity:list_policies()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.695789 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-967ac264-d5e8-41f4-bcf0-ce5f89a931b9 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_policies": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:25.695789 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:25.695789 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:25.695789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ac264-d5e8-41f4-bcf0-ce5f89a931b9 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.702800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5a450-6c25-4ecc-b5f4-d1350ab235c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.703941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5a450-6c25-4ecc-b5f4-d1350ab235c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.704073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5a450-6c25-4ecc-b5f4-d1350ab235c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.704654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5a450-6c25-4ecc-b5f4-d1350ab235c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.705344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5a450-6c25-4ecc-b5f4-d1350ab235c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.705975 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2665/5259] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Sep 3 21:08:25 2022] GET /identity/v3/policies => generated 764 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:25.707648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5a450-6c25-4ecc-b5f4-d1350ab235c7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.710765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ac264-d5e8-41f4-bcf0-ce5f89a931b9 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.721926 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2595/5260] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:25.722629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ac264-d5e8-41f4-bcf0-ce5f89a931b9 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.726972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5a450-6c25-4ecc-b5f4-d1350ab235c7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.728705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97604b-2c26-4174-bada-eea777b25fe3 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e913bc7e6c4f6b9d15e3fdb946b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '346a136fb78c425aa009ac5c260ee137', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.729776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97604b-2c26-4174-bada-eea777b25fe3 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.730178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97604b-2c26-4174-bada-eea777b25fe3 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.730588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97604b-2c26-4174-bada-eea777b25fe3 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] PATH_INFO: `/v3/policies/064ed31a434d43158549c3f61133c11c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.731823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97604b-2c26-4174-bada-eea777b25fe3 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=064ed31a434d43158549c3f61133c11c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.735225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97604b-2c26-4174-bada-eea777b25fe3 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.739586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5a450-6c25-4ecc-b5f4-d1350ab235c7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.747665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ac3f7-44d0-4775-a68d-f1f1c9ba26c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.748593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ac3f7-44d0-4775-a68d-f1f1c9ba26c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.749077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ac3f7-44d0-4775-a68d-f1f1c9ba26c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.749532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ac3f7-44d0-4775-a68d-f1f1c9ba26c2 admin admin] PATH_INFO: `/v3/projects/ff3fe8967ddb43a2b8ba88ccc4f7f84c/users/8c101ebf1e594c71aa8664b40a058764/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.763479 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2666/5261] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:25 2022] DELETE /identity/v3/policies/064ed31a434d43158549c3f61133c11c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:25.765733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97604b-2c26-4174-bada-eea777b25fe3 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.773723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ac3f7-44d0-4775-a68d-f1f1c9ba26c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3fe8967ddb43a2b8ba88ccc4f7f84c, user_id=8c101ebf1e594c71aa8664b40a058764, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.777732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ac3f7-44d0-4775-a68d-f1f1c9ba26c2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.789242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97604b-2c26-4174-bada-eea777b25fe3 tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.796511 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2596/5262] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:25 2022] PUT /identity/v3/projects/ff3fe8967ddb43a2b8ba88ccc4f7f84c/users/8c101ebf1e594c71aa8664b40a058764/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:25.808416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af9bbe-7d39-4f13-b17e-5973ba18c870 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.808416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af9bbe-7d39-4f13-b17e-5973ba18c870 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.808416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af9bbe-7d39-4f13-b17e-5973ba18c870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.848622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-93af9bbe-7d39-4f13-b17e-5973ba18c870 None None] MFA Rules not processed for user `8c101ebf1e594c71aa8664b40a058764`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:25.850736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-93af9bbe-7d39-4f13-b17e-5973ba18c870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:25.878971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb67774-07f8-498e-92e0-2dd40917277a tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e913bc7e6c4f6b9d15e3fdb946b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '346a136fb78c425aa009ac5c260ee137', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.880632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb67774-07f8-498e-92e0-2dd40917277a tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.880632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb67774-07f8-498e-92e0-2dd40917277a tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.881209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb67774-07f8-498e-92e0-2dd40917277a tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] PATH_INFO: `/v3/policies/4aa90ecc6a454b0991ddbdaff00ccdcc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.883739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb67774-07f8-498e-92e0-2dd40917277a tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=4aa90ecc6a454b0991ddbdaff00ccdcc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.887338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb67774-07f8-498e-92e0-2dd40917277a tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.909806 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2667/5263] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:25 2022] DELETE /identity/v3/policies/4aa90ecc6a454b0991ddbdaff00ccdcc => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:25.915842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb67774-07f8-498e-92e0-2dd40917277a tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.933105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb67774-07f8-498e-92e0-2dd40917277a tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:25.936456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:25.943170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4cbfb-1a56-4402-8031-62aa21afc37d tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e913bc7e6c4f6b9d15e3fdb946b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '346a136fb78c425aa009ac5c260ee137', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:25.944165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4cbfb-1a56-4402-8031-62aa21afc37d tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:25.944629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4cbfb-1a56-4402-8031-62aa21afc37d tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:25.945146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4cbfb-1a56-4402-8031-62aa21afc37d tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] PATH_INFO: `/v3/policies/96f258e206634c4d94cb53dce07cd6ce` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:25.946849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4cbfb-1a56-4402-8031-62aa21afc37d tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=96f258e206634c4d94cb53dce07cd6ce)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:25.950756 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2597/5264] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:25.953495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4cbfb-1a56-4402-8031-62aa21afc37d tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:25.956176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af9bbe-7d39-4f13-b17e-5973ba18c870 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.966889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:25.980021 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2668/5265] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:25 2022] DELETE /identity/v3/policies/96f258e206634c4d94cb53dce07cd6ce => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:25.983819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4cbfb-1a56-4402-8031-62aa21afc37d tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:25.991187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1cf4cbfb-1a56-4402-8031-62aa21afc37d tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:26.063966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af9bbe-7d39-4f13-b17e-5973ba18c870 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:26.070167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c908f0-fcb2-4a3d-83b8-43e58654cb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:26.071306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c908f0-fcb2-4a3d-83b8-43e58654cb65 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:26.071664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c908f0-fcb2-4a3d-83b8-43e58654cb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:26.072013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c908f0-fcb2-4a3d-83b8-43e58654cb65 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:26.073223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c908f0-fcb2-4a3d-83b8-43e58654cb65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:26.076404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c908f0-fcb2-4a3d-83b8-43e58654cb65 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:26.102655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4cbfb-1a56-4402-8031-62aa21afc37d tempest-PoliciesTestJSON-863425686 tempest-PoliciesTestJSON-863425686-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:26.108151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac55f0-e404-4469-864b-222e642bc70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:26.109326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac55f0-e404-4469-864b-222e642bc70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:26.109778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac55f0-e404-4469-864b-222e642bc70c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:26.110294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac55f0-e404-4469-864b-222e642bc70c admin admin] PATH_INFO: `/v3/users/a388bfebf5f544b2b900e4b950b00a51` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:26.115003 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2598/5266] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:25 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:26.115201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac55f0-e404-4469-864b-222e642bc70c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a388bfebf5f544b2b900e4b950b00a51)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:26.115808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac55f0-e404-4469-864b-222e642bc70c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:26.116936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c908f0-fcb2-4a3d-83b8-43e58654cb65 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:26.183374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6bac55f0-e404-4469-864b-222e642bc70c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a388bfebf5f544b2b900e4b950b00a51'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:26.195419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c908f0-fcb2-4a3d-83b8-43e58654cb65 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:26.200174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6bac55f0-e404-4469-864b-222e642bc70c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a388bfebf5f544b2b900e4b950b00a51'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:26.201292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc80f3b-f829-4389-b05d-8fad9a9c1bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:26.202136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc80f3b-f829-4389-b05d-8fad9a9c1bbf admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:26.202606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc80f3b-f829-4389-b05d-8fad9a9c1bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:26.202969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc80f3b-f829-4389-b05d-8fad9a9c1bbf admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:26.204026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc80f3b-f829-4389-b05d-8fad9a9c1bbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:26.206260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc80f3b-f829-4389-b05d-8fad9a9c1bbf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:26.211061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6bac55f0-e404-4469-864b-222e642bc70c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a388bfebf5f544b2b900e4b950b00a51'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:26.212597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6bac55f0-e404-4469-864b-222e642bc70c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a388bfebf5f544b2b900e4b950b00a51'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:26.213555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6bac55f0-e404-4469-864b-222e642bc70c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a388bfebf5f544b2b900e4b950b00a51'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:26.234285 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2599/5267] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:26 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:26.238366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc80f3b-f829-4389-b05d-8fad9a9c1bbf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:26.240945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bfc80f3b-f829-4389-b05d-8fad9a9c1bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:26.258260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc80f3b-f829-4389-b05d-8fad9a9c1bbf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:26.265202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d6e31-447c-41a6-af39-98635e300e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:26.266227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d6e31-447c-41a6-af39-98635e300e28 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:26.266706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d6e31-447c-41a6-af39-98635e300e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:26.267092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d6e31-447c-41a6-af39-98635e300e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:26.268001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d6e31-447c-41a6-af39-98635e300e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:26.270849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d6e31-447c-41a6-af39-98635e300e28 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:26.279301 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2669/5268] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:25 2022] DELETE /identity/v3/users/a388bfebf5f544b2b900e4b950b00a51 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:26.284112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48668897-f244-492c-b0b3-2c84624c8c50 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:26.284269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48668897-f244-492c-b0b3-2c84624c8c50 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:26.285132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48668897-f244-492c-b0b3-2c84624c8c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:26.288209 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2600/5269] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:26.289102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d6e31-447c-41a6-af39-98635e300e28 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:26.309347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d6e31-447c-41a6-af39-98635e300e28 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:26.346202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-48668897-f244-492c-b0b3-2c84624c8c50 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:26.348019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-48668897-f244-492c-b0b3-2c84624c8c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:26.389864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23138c8b-1065-4007-ba73-4608412f25d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:26.390854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23138c8b-1065-4007-ba73-4608412f25d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:26.391243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23138c8b-1065-4007-ba73-4608412f25d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:26.392475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23138c8b-1065-4007-ba73-4608412f25d1 admin admin] PATH_INFO: `/v3/projects/53ebcc6052c743119093df58f48d7c46/users/1e4d72f22e5d47d6ae9eac7d6a2bdaaf/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:26.395840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:26.427673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23138c8b-1065-4007-ba73-4608412f25d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ebcc6052c743119093df58f48d7c46, user_id=1e4d72f22e5d47d6ae9eac7d6a2bdaaf, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:26.441362 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2670/5270] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:26.441643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23138c8b-1065-4007-ba73-4608412f25d1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:26.464316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48668897-f244-492c-b0b3-2c84624c8c50 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:26.469233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-48668897-f244-492c-b0b3-2c84624c8c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:26.497794 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2601/5271] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:26 2022] PUT /identity/v3/projects/53ebcc6052c743119093df58f48d7c46/users/1e4d72f22e5d47d6ae9eac7d6a2bdaaf/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:26.498917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23138c8b-1065-4007-ba73-4608412f25d1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:26.514684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:26.616208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48668897-f244-492c-b0b3-2c84624c8c50 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:26.650612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23138c8b-1065-4007-ba73-4608412f25d1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:26.655472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f8a30-31c4-475c-9f58-2082371f635d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:26.656092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f8a30-31c4-475c-9f58-2082371f635d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:26.656381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f8a30-31c4-475c-9f58-2082371f635d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:26.656662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f8a30-31c4-475c-9f58-2082371f635d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:26.657394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f8a30-31c4-475c-9f58-2082371f635d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:26.659029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f8a30-31c4-475c-9f58-2082371f635d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:26.672855 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2602/5272] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:26.675643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f8a30-31c4-475c-9f58-2082371f635d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:26.699169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f8a30-31c4-475c-9f58-2082371f635d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:26.716737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd15bbe-e384-4ce2-b0f3-5c6ccb77ae0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:26.717601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd15bbe-e384-4ce2-b0f3-5c6ccb77ae0d admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:26.717759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053cb50-8e1e-4d12-ba1a-67f8a560e129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:26.718237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd15bbe-e384-4ce2-b0f3-5c6ccb77ae0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:26.718354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053cb50-8e1e-4d12-ba1a-67f8a560e129 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:26.718494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd15bbe-e384-4ce2-b0f3-5c6ccb77ae0d admin admin] PATH_INFO: `/v3/projects/53ebcc6052c743119093df58f48d7c46/users/1e4d72f22e5d47d6ae9eac7d6a2bdaaf/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:26.718609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053cb50-8e1e-4d12-ba1a-67f8a560e129 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:26.718895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053cb50-8e1e-4d12-ba1a-67f8a560e129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:26.723091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd15bbe-e384-4ce2-b0f3-5c6ccb77ae0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ebcc6052c743119093df58f48d7c46, user_id=1e4d72f22e5d47d6ae9eac7d6a2bdaaf, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:26.726239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd15bbe-e384-4ce2-b0f3-5c6ccb77ae0d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:26.733176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053cb50-8e1e-4d12-ba1a-67f8a560e129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:26.734611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053cb50-8e1e-4d12-ba1a-67f8a560e129 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:26.774052 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2603/5273] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:26 2022] PUT /identity/v3/projects/53ebcc6052c743119093df58f48d7c46/users/1e4d72f22e5d47d6ae9eac7d6a2bdaaf/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:26.774266 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2671/5274] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:26.775937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4423485-a89a-4040-b654-ba0b7a808e7f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:26.776333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4423485-a89a-4040-b654-ba0b7a808e7f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:26.776721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4423485-a89a-4040-b654-ba0b7a808e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:26.810329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f4423485-a89a-4040-b654-ba0b7a808e7f None None] MFA Rules not processed for user `1e4d72f22e5d47d6ae9eac7d6a2bdaaf`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:26.811496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f4423485-a89a-4040-b654-ba0b7a808e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:26.900171 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2604/5275] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:26.904928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053cb50-8e1e-4d12-ba1a-67f8a560e129 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:26.979101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053cb50-8e1e-4d12-ba1a-67f8a560e129 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:26.985861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85da356-6c38-4773-822d-5b33bca12718 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ebcc6052c743119093df58f48d7c46', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:26.986770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85da356-6c38-4773-822d-5b33bca12718 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:26.987266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85da356-6c38-4773-822d-5b33bca12718 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:26.988322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85da356-6c38-4773-822d-5b33bca12718 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:26.989867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85da356-6c38-4773-822d-5b33bca12718 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:26.992503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85da356-6c38-4773-822d-5b33bca12718 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:27.017223 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2672/5276] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:08:26 2022] POST /identity/v3/domains => generated 277 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:27.021608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4423485-a89a-4040-b654-ba0b7a808e7f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:27.032835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4423485-a89a-4040-b654-ba0b7a808e7f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:27.096460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:27.115008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40e3bd-17d5-40ce-90b4-88b5d2e0b861 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ebcc6052c743119093df58f48d7c46', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:27.115323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40e3bd-17d5-40ce-90b4-88b5d2e0b861 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:27.115782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40e3bd-17d5-40ce-90b4-88b5d2e0b861 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:27.115971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40e3bd-17d5-40ce-90b4-88b5d2e0b861 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:27.121237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40e3bd-17d5-40ce-90b4-88b5d2e0b861 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:27.126547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40e3bd-17d5-40ce-90b4-88b5d2e0b861 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:27.179028 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2605/5277] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:27 2022] POST /identity/v3/users => generated 347 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:27.180726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85da356-6c38-4773-822d-5b33bca12718 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:27.199641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85da356-6c38-4773-822d-5b33bca12718 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:27.206058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256df65-b3d7-4954-983e-683cdf1d68c9 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ebcc6052c743119093df58f48d7c46', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:27.207967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256df65-b3d7-4954-983e-683cdf1d68c9 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:27.208268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256df65-b3d7-4954-983e-683cdf1d68c9 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:27.208560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256df65-b3d7-4954-983e-683cdf1d68c9 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:27.209269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256df65-b3d7-4954-983e-683cdf1d68c9 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:27.211154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256df65-b3d7-4954-983e-683cdf1d68c9 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:27.273470 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2673/5278] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:27 2022] POST /identity/v3/users => generated 323 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:27.277538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40e3bd-17d5-40ce-90b4-88b5d2e0b861 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:27.290966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40e3bd-17d5-40ce-90b4-88b5d2e0b861 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:27.297075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b16c3d-0031-41af-81ca-dce3e50d2351 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ebcc6052c743119093df58f48d7c46', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:27.297852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b16c3d-0031-41af-81ca-dce3e50d2351 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:27.298291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b16c3d-0031-41af-81ca-dce3e50d2351 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:27.298727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b16c3d-0031-41af-81ca-dce3e50d2351 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] PATH_INFO: `/v3/users/278f57753ccc47c59c0186e76da2dfe3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:27.324037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b16c3d-0031-41af-81ca-dce3e50d2351 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorizing `identity:get_user(user_id=278f57753ccc47c59c0186e76da2dfe3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:27.326689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b16c3d-0031-41af-81ca-dce3e50d2351 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:27.329222 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2606/5279] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Sep 3 21:08:27 2022] GET /identity/v3/users/278f57753ccc47c59c0186e76da2dfe3 => generated 347 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:27.334641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256df65-b3d7-4954-983e-683cdf1d68c9 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:27.363242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256df65-b3d7-4954-983e-683cdf1d68c9 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:27.377508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97534e21-6804-4256-aa4d-162f0fd99099 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ebcc6052c743119093df58f48d7c46', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:27.378214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97534e21-6804-4256-aa4d-162f0fd99099 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:27.378552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97534e21-6804-4256-aa4d-162f0fd99099 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:27.378839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97534e21-6804-4256-aa4d-162f0fd99099 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:27.379608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97534e21-6804-4256-aa4d-162f0fd99099 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Adding query filter params (domain_id=7c43f0bdf6e44ddf9b2baa4695861b50) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:27.380061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97534e21-6804-4256-aa4d-162f0fd99099 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:27.382282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97534e21-6804-4256-aa4d-162f0fd99099 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:27.403149 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2674/5280] 10.222.0.16 () {58 vars in 1207 bytes} [Sat Sep 3 21:08:27 2022] GET /identity/v3/users?domain_id=7c43f0bdf6e44ddf9b2baa4695861b50 => generated 484 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:27.404423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b16c3d-0031-41af-81ca-dce3e50d2351 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:27.433714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b16c3d-0031-41af-81ca-dce3e50d2351 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:27.441959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce18fac-6234-4191-80a0-cf8784fa1769 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ebcc6052c743119093df58f48d7c46', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:27.442880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce18fac-6234-4191-80a0-cf8784fa1769 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:27.443813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce18fac-6234-4191-80a0-cf8784fa1769 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:27.444327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce18fac-6234-4191-80a0-cf8784fa1769 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:27.445179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce18fac-6234-4191-80a0-cf8784fa1769 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:27.447330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce18fac-6234-4191-80a0-cf8784fa1769 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:27.491301 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2607/5281] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:27 2022] GET /identity/v3/users => generated 8287 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:27.494014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97534e21-6804-4256-aa4d-162f0fd99099 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:27.508927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97534e21-6804-4256-aa4d-162f0fd99099 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:27.518077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a888f-944a-40b9-a398-52e5b9a48d98 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ebcc6052c743119093df58f48d7c46', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:27.519438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a888f-944a-40b9-a398-52e5b9a48d98 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:27.519835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a888f-944a-40b9-a398-52e5b9a48d98 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:27.520831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a888f-944a-40b9-a398-52e5b9a48d98 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:27.521699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a888f-944a-40b9-a398-52e5b9a48d98 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Adding query filter params (name=tempest-test_user-451271497) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:27.522118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a888f-944a-40b9-a398-52e5b9a48d98 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:27.524342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a888f-944a-40b9-a398-52e5b9a48d98 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:27.582013 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2675/5282] 10.222.0.16 () {58 vars in 1187 bytes} [Sat Sep 3 21:08:27 2022] GET /identity/v3/users?name=tempest-test_user-451271497 => generated 474 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:27.584276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce18fac-6234-4191-80a0-cf8784fa1769 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:27.608619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce18fac-6234-4191-80a0-cf8784fa1769 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:27.625776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d33c8-e3a7-46c6-a00a-a733f9ec3823 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ebcc6052c743119093df58f48d7c46', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:27.626416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d33c8-e3a7-46c6-a00a-a733f9ec3823 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:27.626705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d33c8-e3a7-46c6-a00a-a733f9ec3823 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:27.626987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d33c8-e3a7-46c6-a00a-a733f9ec3823 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:27.627698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d33c8-e3a7-46c6-a00a-a733f9ec3823 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:27.628559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d33c8-e3a7-46c6-a00a-a733f9ec3823 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:27.630423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d33c8-e3a7-46c6-a00a-a733f9ec3823 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:27.674398 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2608/5283] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Sep 3 21:08:27 2022] GET /identity/v3/users?enabled=False => generated 431 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:27.675055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a888f-944a-40b9-a398-52e5b9a48d98 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:27.688629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a888f-944a-40b9-a398-52e5b9a48d98 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:27.701659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ebcc6052c743119093df58f48d7c46', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:27.702320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:27.702717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:27.703012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] PATH_INFO: `/v3/users/b2332df16a304fc7b3292450d19ff4a0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:27.740399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorizing `identity:delete_user(user_id=b2332df16a304fc7b3292450d19ff4a0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:27.743484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:27.761322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:27.810661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2332df16a304fc7b3292450d19ff4a0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:27.834807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2332df16a304fc7b3292450d19ff4a0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:27.844789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2332df16a304fc7b3292450d19ff4a0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:27.845631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2332df16a304fc7b3292450d19ff4a0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:27.846351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2332df16a304fc7b3292450d19ff4a0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:27.863537 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2676/5284] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:27 2022] DELETE /identity/v3/users/b2332df16a304fc7b3292450d19ff4a0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:27.869376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d33c8-e3a7-46c6-a00a-a733f9ec3823 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:27.873787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9a4d33c8-e3a7-46c6-a00a-a733f9ec3823 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:27.894584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d33c8-e3a7-46c6-a00a-a733f9ec3823 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:27.895971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:27.897750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:28.009770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ebcc6052c743119093df58f48d7c46', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:28.011143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:28.011143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:28.011502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] PATH_INFO: `/v3/users/278f57753ccc47c59c0186e76da2dfe3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:28.013104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorizing `identity:delete_user(user_id=278f57753ccc47c59c0186e76da2dfe3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:28.015686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:28.038032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8a730-f2c8-4b23-ba1e-ee7a8533ae95 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:28.046802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:28.047614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:28.048007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:28.048390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] PATH_INFO: `/v3/projects/5615a74cb51947208af83a65c64dc0de` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:28.049777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5615a74cb51947208af83a65c64dc0de)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:28.052809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:28.091520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '278f57753ccc47c59c0186e76da2dfe3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.142734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '278f57753ccc47c59c0186e76da2dfe3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.143517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '278f57753ccc47c59c0186e76da2dfe3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.154653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '278f57753ccc47c59c0186e76da2dfe3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.158757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5615a74cb51947208af83a65c64dc0de', 'request_id': 'req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.163408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '278f57753ccc47c59c0186e76da2dfe3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.172612 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2609/5285] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:27 2022] DELETE /identity/v3/users/278f57753ccc47c59c0186e76da2dfe3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:28.174030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5615a74cb51947208af83a65c64dc0de', 'request_id': 'req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.181141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:28.182994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:28.197552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5615a74cb51947208af83a65c64dc0de', 'request_id': 'req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.224025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:28.227475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] The token cache is being invalidate because project 5615a74cb51947208af83a65c64dc0de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:28.227798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.229146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.232887 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2677/5286] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:27 2022] DELETE /identity/v3/projects/5615a74cb51947208af83a65c64dc0de => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:28.246962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:28.248662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:28.279853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:28.328868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c00ff-0ced-49ec-83c6-699da656b3ec tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:28.335255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ebcc6052c743119093df58f48d7c46', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:28.336199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:28.336542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:28.337479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] PATH_INFO: `/v3/domains/7c43f0bdf6e44ddf9b2baa4695861b50` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:28.339052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=7c43f0bdf6e44ddf9b2baa4695861b50)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:28.348773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:28.393004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '7c43f0bdf6e44ddf9b2baa4695861b50'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.394756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '7c43f0bdf6e44ddf9b2baa4695861b50'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.398204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ac8a1-cd09-4937-bed9-5ff5dc413909 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:28.427398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '7c43f0bdf6e44ddf9b2baa4695861b50'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.428033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '7c43f0bdf6e44ddf9b2baa4695861b50'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.431075 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2610/5287] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:08:28 2022] PATCH /identity/v3/domains/7c43f0bdf6e44ddf9b2baa4695861b50 => generated 278 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:28.434896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:28.436471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:28.495323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:28.495971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:28.497059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:28.497231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] PATH_INFO: `/v3/users/33e913bc7e6c4f6b9d15e3fdb946b644` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:28.499489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e913bc7e6c4f6b9d15e3fdb946b644)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:28.502683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:28.549191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff518f-f115-4ae2-ae22-793c11a76251 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:28.554914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ebcc6052c743119093df58f48d7c46', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:28.555725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:28.556077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:28.556477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] PATH_INFO: `/v3/domains/7c43f0bdf6e44ddf9b2baa4695861b50` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:28.569797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e913bc7e6c4f6b9d15e3fdb946b644'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.570056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=7c43f0bdf6e44ddf9b2baa4695861b50)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:28.571763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:28.584932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e913bc7e6c4f6b9d15e3fdb946b644'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.610595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e913bc7e6c4f6b9d15e3fdb946b644'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.611432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e913bc7e6c4f6b9d15e3fdb946b644'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.612130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e913bc7e6c4f6b9d15e3fdb946b644'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.614947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '7c43f0bdf6e44ddf9b2baa4695861b50'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.619039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '7c43f0bdf6e44ddf9b2baa4695861b50'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.629044 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2678/5288] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:28 2022] DELETE /identity/v3/users/33e913bc7e6c4f6b9d15e3fdb946b644 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:28.678488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:28.681517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c43f0bdf6e44ddf9b2baa4695861b50', 'request_id': 'req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.691669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c43f0bdf6e44ddf9b2baa4695861b50', 'request_id': 'req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.705977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c43f0bdf6e44ddf9b2baa4695861b50', 'request_id': 'req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.721299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] The token cache is being invalidate because project 7c43f0bdf6e44ddf9b2baa4695861b50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:28.721962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.722647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.731291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '7c43f0bdf6e44ddf9b2baa4695861b50', 'request_id': 'req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.735495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '7c43f0bdf6e44ddf9b2baa4695861b50', 'request_id': 'req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.736276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '7c43f0bdf6e44ddf9b2baa4695861b50', 'request_id': 'req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:28.744821 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2611/5289] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:28 2022] DELETE /identity/v3/domains/7c43f0bdf6e44ddf9b2baa4695861b50 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:28.749097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:28.751288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:28.824636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:28.856098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce8fb7-af6e-4b0a-92b0-62ed480bf0ec admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:28.929024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbf79d-126c-4e4b-a931-b9585067deb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:28.929973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbf79d-126c-4e4b-a931-b9585067deb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:28.930428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbf79d-126c-4e4b-a931-b9585067deb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:28.930858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbf79d-126c-4e4b-a931-b9585067deb9 admin admin] PATH_INFO: `/v3/users/8c101ebf1e594c71aa8664b40a058764` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:28.933144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbf79d-126c-4e4b-a931-b9585067deb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c101ebf1e594c71aa8664b40a058764)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:28.935646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbf79d-126c-4e4b-a931-b9585067deb9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:28.993998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:28.998942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:29.017963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3ebbf79d-126c-4e4b-a931-b9585067deb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c101ebf1e594c71aa8664b40a058764'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.038125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d20c22-fc1e-40fb-ab14-1bb9a53dc161 tempest-UsersV3TestJSON-1480052573 tempest-UsersV3TestJSON-1480052573-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:29.043920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55df76-2ac2-4623-8f9c-c8c175705a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:29.044611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55df76-2ac2-4623-8f9c-c8c175705a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.044925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55df76-2ac2-4623-8f9c-c8c175705a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.045378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55df76-2ac2-4623-8f9c-c8c175705a07 admin admin] PATH_INFO: `/v3/projects/346a136fb78c425aa009ac5c260ee137` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.047600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55df76-2ac2-4623-8f9c-c8c175705a07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=346a136fb78c425aa009ac5c260ee137)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:29.049814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55df76-2ac2-4623-8f9c-c8c175705a07 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:29.069460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3ebbf79d-126c-4e4b-a931-b9585067deb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c101ebf1e594c71aa8664b40a058764'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.079601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3ebbf79d-126c-4e4b-a931-b9585067deb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c101ebf1e594c71aa8664b40a058764'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.080254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3ebbf79d-126c-4e4b-a931-b9585067deb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c101ebf1e594c71aa8664b40a058764'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.080869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3ebbf79d-126c-4e4b-a931-b9585067deb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c101ebf1e594c71aa8664b40a058764'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.097809 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2679/5290] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:28 2022] DELETE /identity/v3/users/8c101ebf1e594c71aa8664b40a058764 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:29.102534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68c3a7-6f38-4126-aaed-f677b6399257 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.102818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68c3a7-6f38-4126-aaed-f677b6399257 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.103159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68c3a7-6f38-4126-aaed-f677b6399257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.106927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae55df76-2ac2-4623-8f9c-c8c175705a07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '346a136fb78c425aa009ac5c260ee137', 'request_id': 'req-ae55df76-2ac2-4623-8f9c-c8c175705a07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.116098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae55df76-2ac2-4623-8f9c-c8c175705a07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '346a136fb78c425aa009ac5c260ee137', 'request_id': 'req-ae55df76-2ac2-4623-8f9c-c8c175705a07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.123807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae55df76-2ac2-4623-8f9c-c8c175705a07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '346a136fb78c425aa009ac5c260ee137', 'request_id': 'req-ae55df76-2ac2-4623-8f9c-c8c175705a07'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.131236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-df68c3a7-6f38-4126-aaed-f677b6399257 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:29.132583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-df68c3a7-6f38-4126-aaed-f677b6399257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:29.147860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae55df76-2ac2-4623-8f9c-c8c175705a07 admin admin] The token cache is being invalidate because project 346a136fb78c425aa009ac5c260ee137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:29.148385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae55df76-2ac2-4623-8f9c-c8c175705a07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.149470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae55df76-2ac2-4623-8f9c-c8c175705a07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.150875 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2612/5291] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:28 2022] DELETE /identity/v3/projects/346a136fb78c425aa009ac5c260ee137 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:29.158757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd80df-226e-419c-bcf9-b1408caf166a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.159319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd80df-226e-419c-bcf9-b1408caf166a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.160472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd80df-226e-419c-bcf9-b1408caf166a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.226523 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2680/5292] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:29.248660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68c3a7-6f38-4126-aaed-f677b6399257 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:29.250566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-df68c3a7-6f38-4126-aaed-f677b6399257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:29.277716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:29.343102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b5fd80df-226e-419c-bcf9-b1408caf166a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:29.344495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b5fd80df-226e-419c-bcf9-b1408caf166a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:29.351204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68c3a7-6f38-4126-aaed-f677b6399257 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:29.357438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d4f96-56b3-4d32-b69c-b642c3b78914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:29.365024 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2613/5293] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:29.366638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdfbef-b37b-4982-b0b9-b00d137cbab0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.366931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdfbef-b37b-4982-b0b9-b00d137cbab0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.367211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdfbef-b37b-4982-b0b9-b00d137cbab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.368488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d4f96-56b3-4d32-b69c-b642c3b78914 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.369685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d4f96-56b3-4d32-b69c-b642c3b78914 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.370157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d4f96-56b3-4d32-b69c-b642c3b78914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.391218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d4f96-56b3-4d32-b69c-b642c3b78914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:29.392685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d4f96-56b3-4d32-b69c-b642c3b78914 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:29.393589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-79fdfbef-b37b-4982-b0b9-b00d137cbab0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:29.395488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-79fdfbef-b37b-4982-b0b9-b00d137cbab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:29.405451 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2614/5294] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:29.413114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdfbef-b37b-4982-b0b9-b00d137cbab0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:29.418123 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2681/5295] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:29.427596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdfbef-b37b-4982-b0b9-b00d137cbab0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:29.433183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b09edd-0cec-409d-bc7a-48576b7a935e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:29.433880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b09edd-0cec-409d-bc7a-48576b7a935e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.434271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b09edd-0cec-409d-bc7a-48576b7a935e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.434615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b09edd-0cec-409d-bc7a-48576b7a935e admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.435424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b09edd-0cec-409d-bc7a-48576b7a935e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:29.435784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b09edd-0cec-409d-bc7a-48576b7a935e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:29.437424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b09edd-0cec-409d-bc7a-48576b7a935e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:29.453074 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2615/5296] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:29.454888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d4f96-56b3-4d32-b69c-b642c3b78914 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:29.467816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d4f96-56b3-4d32-b69c-b642c3b78914 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:29.475152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5e0cb-05f3-4387-9b42-a83f91683cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:29.476036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5e0cb-05f3-4387-9b42-a83f91683cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.476523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5e0cb-05f3-4387-9b42-a83f91683cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.477253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5e0cb-05f3-4387-9b42-a83f91683cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.477967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5e0cb-05f3-4387-9b42-a83f91683cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:29.480815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5e0cb-05f3-4387-9b42-a83f91683cd5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:29.523683 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2682/5297] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:29 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:29.527594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b09edd-0cec-409d-bc7a-48576b7a935e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:29.554009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b09edd-0cec-409d-bc7a-48576b7a935e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:29.654933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4735e-8fee-43e0-bc0b-0be248b0a8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:29.655796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4735e-8fee-43e0-bc0b-0be248b0a8a6 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.655915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4735e-8fee-43e0-bc0b-0be248b0a8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.656152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4735e-8fee-43e0-bc0b-0be248b0a8a6 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.656845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4735e-8fee-43e0-bc0b-0be248b0a8a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:29.659109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4735e-8fee-43e0-bc0b-0be248b0a8a6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:29.687010 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2616/5298] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:29 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:29.689575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5e0cb-05f3-4387-9b42-a83f91683cd5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:29.704648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5e0cb-05f3-4387-9b42-a83f91683cd5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:29.712293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d10c35-2bba-413f-a99e-21b340b0e30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:29.713247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d10c35-2bba-413f-a99e-21b340b0e30f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.713564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d10c35-2bba-413f-a99e-21b340b0e30f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.713848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d10c35-2bba-413f-a99e-21b340b0e30f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.714509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d10c35-2bba-413f-a99e-21b340b0e30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:29.716139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d10c35-2bba-413f-a99e-21b340b0e30f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:29.728941 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2683/5299] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:29.733962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4735e-8fee-43e0-bc0b-0be248b0a8a6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:29.739829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=81,keystone:INSERT=5,keystone:DELETE=14 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:29.750903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4735e-8fee-43e0-bc0b-0be248b0a8a6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:29.757736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88cfbe-5c07-4748-a614-c8f6195aab6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:29.758440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88cfbe-5c07-4748-a614-c8f6195aab6e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.758792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88cfbe-5c07-4748-a614-c8f6195aab6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.759128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88cfbe-5c07-4748-a614-c8f6195aab6e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.759848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88cfbe-5c07-4748-a614-c8f6195aab6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:29.761551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88cfbe-5c07-4748-a614-c8f6195aab6e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:29.777380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d10c35-2bba-413f-a99e-21b340b0e30f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:29.779298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f2d10c35-2bba-413f-a99e-21b340b0e30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:29.781307 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2617/5300] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:29.788871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88cfbe-5c07-4748-a614-c8f6195aab6e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:29.800272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d10c35-2bba-413f-a99e-21b340b0e30f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:29.805305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88cfbe-5c07-4748-a614-c8f6195aab6e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:29.806206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:29.806925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.807283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.807689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] PATH_INFO: `/v3/projects/ff3fe8967ddb43a2b8ba88ccc4f7f84c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.809332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3fe8967ddb43a2b8ba88ccc4f7f84c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:29.812253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:29.814203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58e0bb-cabe-406c-a76a-91b652cbd97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:29.815115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58e0bb-cabe-406c-a76a-91b652cbd97f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.815233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58e0bb-cabe-406c-a76a-91b652cbd97f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.815428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58e0bb-cabe-406c-a76a-91b652cbd97f admin admin] PATH_INFO: `/v3/projects/b0f346da77a147f69bb85dece3cac287/users/9af765193d064e849e92b4ee91821967/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.833581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58e0bb-cabe-406c-a76a-91b652cbd97f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f346da77a147f69bb85dece3cac287, user_id=9af765193d064e849e92b4ee91821967, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:29.836833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58e0bb-cabe-406c-a76a-91b652cbd97f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:29.854148 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2618/5301] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:29 2022] PUT /identity/v3/projects/b0f346da77a147f69bb85dece3cac287/users/9af765193d064e849e92b4ee91821967/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:29.864311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265018ed-eb18-433f-9687-da33c0600c27 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.864603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265018ed-eb18-433f-9687-da33c0600c27 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.864885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265018ed-eb18-433f-9687-da33c0600c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.866272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3fe8967ddb43a2b8ba88ccc4f7f84c', 'request_id': 'req-3f070d24-32ee-412c-a4e9-f742cb52f115'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.875617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3fe8967ddb43a2b8ba88ccc4f7f84c', 'request_id': 'req-3f070d24-32ee-412c-a4e9-f742cb52f115'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.886362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3fe8967ddb43a2b8ba88ccc4f7f84c', 'request_id': 'req-3f070d24-32ee-412c-a4e9-f742cb52f115'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.896972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-265018ed-eb18-433f-9687-da33c0600c27 None None] MFA Rules not processed for user `9af765193d064e849e92b4ee91821967`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:29.898199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-265018ed-eb18-433f-9687-da33c0600c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:29.899231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] The token cache is being invalidate because project ff3fe8967ddb43a2b8ba88ccc4f7f84c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:29.899598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.900429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:29.902332 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2684/5302] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:29 2022] DELETE /identity/v3/projects/ff3fe8967ddb43a2b8ba88ccc4f7f84c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:29.905855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:29.907232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:29.978975 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2619/5303] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:29.986687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f070d24-32ee-412c-a4e9-f742cb52f115 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:29.986871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265018ed-eb18-433f-9687-da33c0600c27 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:29.988589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-265018ed-eb18-433f-9687-da33c0600c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:29.991804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4f1bc-35e7-44bb-bd66-96ebe717aedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:29.992505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f1bc-35e7-44bb-bd66-96ebe717aedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:29.992802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f1bc-35e7-44bb-bd66-96ebe717aedc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:29.993123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f1bc-35e7-44bb-bd66-96ebe717aedc admin admin] PATH_INFO: `/v3/users/1e4d72f22e5d47d6ae9eac7d6a2bdaaf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:29.994290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4f1bc-35e7-44bb-bd66-96ebe717aedc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e4d72f22e5d47d6ae9eac7d6a2bdaaf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:29.996237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4f1bc-35e7-44bb-bd66-96ebe717aedc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:30.002642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:30.057120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2ec4f1bc-35e7-44bb-bd66-96ebe717aedc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:30.058924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265018ed-eb18-433f-9687-da33c0600c27 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:30.062795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654ed79-8203-4944-ad6e-0581ab0b6b57 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:30.063506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654ed79-8203-4944-ad6e-0581ab0b6b57 service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:30.064131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654ed79-8203-4944-ad6e-0581ab0b6b57 service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:30.064131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654ed79-8203-4944-ad6e-0581ab0b6b57 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:30.073554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2ec4f1bc-35e7-44bb-bd66-96ebe717aedc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:30.073742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654ed79-8203-4944-ad6e-0581ab0b6b57 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:30.075272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654ed79-8203-4944-ad6e-0581ab0b6b57 service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:30.082784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2ec4f1bc-35e7-44bb-bd66-96ebe717aedc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:30.083712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2ec4f1bc-35e7-44bb-bd66-96ebe717aedc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:30.084265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2ec4f1bc-35e7-44bb-bd66-96ebe717aedc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e4d72f22e5d47d6ae9eac7d6a2bdaaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:30.086126 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2620/5304] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:30.100938 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2685/5305] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:29 2022] DELETE /identity/v3/users/1e4d72f22e5d47d6ae9eac7d6a2bdaaf => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:30.462792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654ed79-8203-4944-ad6e-0581ab0b6b57 service glance] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:30.476251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654ed79-8203-4944-ad6e-0581ab0b6b57 service glance] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:30.576961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38638bf9-f3d9-41e3-afb2-fa57d87ea735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:30.576961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38638bf9-f3d9-41e3-afb2-fa57d87ea735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:30.576961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38638bf9-f3d9-41e3-afb2-fa57d87ea735 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:30.576961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38638bf9-f3d9-41e3-afb2-fa57d87ea735 admin admin] PATH_INFO: `/v3/projects/53ebcc6052c743119093df58f48d7c46` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:30.579427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38638bf9-f3d9-41e3-afb2-fa57d87ea735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ebcc6052c743119093df58f48d7c46)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:30.583013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38638bf9-f3d9-41e3-afb2-fa57d87ea735 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:30.585171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:30.689666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-38638bf9-f3d9-41e3-afb2-fa57d87ea735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ebcc6052c743119093df58f48d7c46', 'request_id': 'req-38638bf9-f3d9-41e3-afb2-fa57d87ea735'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:30.703730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-38638bf9-f3d9-41e3-afb2-fa57d87ea735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ebcc6052c743119093df58f48d7c46', 'request_id': 'req-38638bf9-f3d9-41e3-afb2-fa57d87ea735'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:30.717764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-38638bf9-f3d9-41e3-afb2-fa57d87ea735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ebcc6052c743119093df58f48d7c46', 'request_id': 'req-38638bf9-f3d9-41e3-afb2-fa57d87ea735'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:30.748377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-38638bf9-f3d9-41e3-afb2-fa57d87ea735 admin admin] The token cache is being invalidate because project 53ebcc6052c743119093df58f48d7c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:30.748807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-38638bf9-f3d9-41e3-afb2-fa57d87ea735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:30.749760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-38638bf9-f3d9-41e3-afb2-fa57d87ea735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:30.751945 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2621/5306] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:30 2022] DELETE /identity/v3/projects/53ebcc6052c743119093df58f48d7c46 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:30.757979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b68510-8905-4c7e-9d74-a830f23924fa None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:30.760094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b68510-8905-4c7e-9d74-a830f23924fa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:30.760390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b68510-8905-4c7e-9d74-a830f23924fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:30.785210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-32b68510-8905-4c7e-9d74-a830f23924fa None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:30.786813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-32b68510-8905-4c7e-9d74-a830f23924fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:30.857850 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2686/5307] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:30.868609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b29d7-0d79-4f15-82c0-c2dde3464b8d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:30.868913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b29d7-0d79-4f15-82c0-c2dde3464b8d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:30.869473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b29d7-0d79-4f15-82c0-c2dde3464b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:30.895496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c05b29d7-0d79-4f15-82c0-c2dde3464b8d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:30.896810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c05b29d7-0d79-4f15-82c0-c2dde3464b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:30.910140 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2622/5308] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:30.911534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b68510-8905-4c7e-9d74-a830f23924fa None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:30.923566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b68510-8905-4c7e-9d74-a830f23924fa None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:30.929128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14df56a-7682-450d-9098-daa2dcc4432b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:30.929765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14df56a-7682-450d-9098-daa2dcc4432b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:30.930176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14df56a-7682-450d-9098-daa2dcc4432b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:30.930461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14df56a-7682-450d-9098-daa2dcc4432b admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:30.931119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14df56a-7682-450d-9098-daa2dcc4432b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:30.931433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14df56a-7682-450d-9098-daa2dcc4432b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:30.933105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14df56a-7682-450d-9098-daa2dcc4432b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:30.948189 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2687/5309] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:30.953246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b29d7-0d79-4f15-82c0-c2dde3464b8d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:30.977513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b29d7-0d79-4f15-82c0-c2dde3464b8d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:30.991406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805672c3-3c8b-470b-9490-cbdbb6c2f72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:30.991406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805672c3-3c8b-470b-9490-cbdbb6c2f72a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:30.991406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805672c3-3c8b-470b-9490-cbdbb6c2f72a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:30.991406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805672c3-3c8b-470b-9490-cbdbb6c2f72a admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:30.991825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805672c3-3c8b-470b-9490-cbdbb6c2f72a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:30.995653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805672c3-3c8b-470b-9490-cbdbb6c2f72a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.030222 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2623/5310] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:30 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:31.034939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14df56a-7682-450d-9098-daa2dcc4432b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.045359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14df56a-7682-450d-9098-daa2dcc4432b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.094505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805672c3-3c8b-470b-9490-cbdbb6c2f72a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.096617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-805672c3-3c8b-470b-9490-cbdbb6c2f72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:31.135295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f15c8-a0e4-4502-8a9f-c4b9dc873102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:31.136037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f15c8-a0e4-4502-8a9f-c4b9dc873102 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.137585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f15c8-a0e4-4502-8a9f-c4b9dc873102 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.137949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f15c8-a0e4-4502-8a9f-c4b9dc873102 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.138739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f15c8-a0e4-4502-8a9f-c4b9dc873102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:31.140961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f15c8-a0e4-4502-8a9f-c4b9dc873102 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.162436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:31.170500 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2688/5311] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:31 2022] POST /identity/v3/users => generated 284 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:31.174089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f15c8-a0e4-4502-8a9f-c4b9dc873102 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.179158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805672c3-3c8b-470b-9490-cbdbb6c2f72a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.184020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9df082-ac71-40ae-a1cb-9a0b56c5f28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:31.184739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df082-ac71-40ae-a1cb-9a0b56c5f28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.185218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df082-ac71-40ae-a1cb-9a0b56c5f28b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.185559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df082-ac71-40ae-a1cb-9a0b56c5f28b admin admin] PATH_INFO: `/v3/users/9af765193d064e849e92b4ee91821967` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.186658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f15c8-a0e4-4502-8a9f-c4b9dc873102 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.187907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9df082-ac71-40ae-a1cb-9a0b56c5f28b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af765193d064e849e92b4ee91821967)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:31.190143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9df082-ac71-40ae-a1cb-9a0b56c5f28b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.192081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364894dc-27a2-4408-ac22-8ca66615b09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:31.192992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364894dc-27a2-4408-ac22-8ca66615b09c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.193126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364894dc-27a2-4408-ac22-8ca66615b09c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.193400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364894dc-27a2-4408-ac22-8ca66615b09c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.194053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364894dc-27a2-4408-ac22-8ca66615b09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:31.195657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364894dc-27a2-4408-ac22-8ca66615b09c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.209036 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2689/5312] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:31.212933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364894dc-27a2-4408-ac22-8ca66615b09c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.227224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364894dc-27a2-4408-ac22-8ca66615b09c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.232719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4b3b6-6427-4318-8d2f-5143cf0c328b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:31.233655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4b3b6-6427-4318-8d2f-5143cf0c328b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.233932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4b3b6-6427-4318-8d2f-5143cf0c328b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.234249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4b3b6-6427-4318-8d2f-5143cf0c328b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.234832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4b3b6-6427-4318-8d2f-5143cf0c328b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:31.236890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4b3b6-6427-4318-8d2f-5143cf0c328b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.246134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8f9df082-ac71-40ae-a1cb-9a0b56c5f28b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af765193d064e849e92b4ee91821967'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:31.247828 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2690/5313] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:31.251358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4b3b6-6427-4318-8d2f-5143cf0c328b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.262426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4b3b6-6427-4318-8d2f-5143cf0c328b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.263658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:31.264697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8f9df082-ac71-40ae-a1cb-9a0b56c5f28b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af765193d064e849e92b4ee91821967'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:31.265948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8f9df082-ac71-40ae-a1cb-9a0b56c5f28b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af765193d064e849e92b4ee91821967'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:31.268189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d2cb6-e5d5-4008-b46a-d58e8cfe431a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:31.269698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d2cb6-e5d5-4008-b46a-d58e8cfe431a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.270202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d2cb6-e5d5-4008-b46a-d58e8cfe431a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.270665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d2cb6-e5d5-4008-b46a-d58e8cfe431a admin admin] PATH_INFO: `/v3/projects/1809ee2f113340dcaed765f3d0d1a460/users/4eaef77d012b4543bca6ea825d9354fd/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.278560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8f9df082-ac71-40ae-a1cb-9a0b56c5f28b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af765193d064e849e92b4ee91821967'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:31.286580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d2cb6-e5d5-4008-b46a-d58e8cfe431a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1809ee2f113340dcaed765f3d0d1a460, user_id=4eaef77d012b4543bca6ea825d9354fd, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:31.287585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8f9df082-ac71-40ae-a1cb-9a0b56c5f28b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af765193d064e849e92b4ee91821967'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:31.289308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d2cb6-e5d5-4008-b46a-d58e8cfe431a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.295741 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2624/5314] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:31 2022] DELETE /identity/v3/users/9af765193d064e849e92b4ee91821967 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:31.300750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a001a60-300a-4b16-9c7e-95b834bb9222 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.301150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a001a60-300a-4b16-9c7e-95b834bb9222 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.301503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a001a60-300a-4b16-9c7e-95b834bb9222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.305122 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2691/5315] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:31 2022] PUT /identity/v3/projects/1809ee2f113340dcaed765f3d0d1a460/users/4eaef77d012b4543bca6ea825d9354fd/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:31.315019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ae182-6832-4d0f-b0c7-8490766246df None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.315377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ae182-6832-4d0f-b0c7-8490766246df None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.316196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ae182-6832-4d0f-b0c7-8490766246df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.322190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5a001a60-300a-4b16-9c7e-95b834bb9222 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:31.323419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5a001a60-300a-4b16-9c7e-95b834bb9222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:31.347927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-969ae182-6832-4d0f-b0c7-8490766246df None None] MFA Rules not processed for user `4eaef77d012b4543bca6ea825d9354fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:31.349220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-969ae182-6832-4d0f-b0c7-8490766246df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:31.397134 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2625/5316] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:31.408015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a001a60-300a-4b16-9c7e-95b834bb9222 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.410073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5a001a60-300a-4b16-9c7e-95b834bb9222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:31.434577 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2692/5317] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:31.438970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ae182-6832-4d0f-b0c7-8490766246df None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.440559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-969ae182-6832-4d0f-b0c7-8490766246df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:31.454633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ae182-6832-4d0f-b0c7-8490766246df None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.459332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd8627-afac-4f0b-a6b1-de9fc25c22cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:31.459937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd8627-afac-4f0b-a6b1-de9fc25c22cc admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.460223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd8627-afac-4f0b-a6b1-de9fc25c22cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.460496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd8627-afac-4f0b-a6b1-de9fc25c22cc admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.461160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd8627-afac-4f0b-a6b1-de9fc25c22cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:31.463196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd8627-afac-4f0b-a6b1-de9fc25c22cc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.482853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a001a60-300a-4b16-9c7e-95b834bb9222 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.486882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7292f1-51f3-4897-bd98-48d6144cb474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:31.488075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7292f1-51f3-4897-bd98-48d6144cb474 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.488360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7292f1-51f3-4897-bd98-48d6144cb474 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.488630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7292f1-51f3-4897-bd98-48d6144cb474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.498541 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2693/5318] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:31 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:31.499578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd8627-afac-4f0b-a6b1-de9fc25c22cc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.501926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7292f1-51f3-4897-bd98-48d6144cb474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:31.503444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7292f1-51f3-4897-bd98-48d6144cb474 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.519111 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2626/5319] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:08:31.568222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd8627-afac-4f0b-a6b1-de9fc25c22cc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.573462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab585d-cf21-4c15-bf9d-938de51a18b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:31.574291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab585d-cf21-4c15-bf9d-938de51a18b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.574642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab585d-cf21-4c15-bf9d-938de51a18b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.575071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab585d-cf21-4c15-bf9d-938de51a18b2 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.575657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab585d-cf21-4c15-bf9d-938de51a18b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:31.578125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab585d-cf21-4c15-bf9d-938de51a18b2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.603509 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2694/5320] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:31 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:31.605117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7292f1-51f3-4897-bd98-48d6144cb474 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.621786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7292f1-51f3-4897-bd98-48d6144cb474 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.626762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c71598-634a-46b4-8554-cc3314210876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:31.627620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c71598-634a-46b4-8554-cc3314210876 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.627887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c71598-634a-46b4-8554-cc3314210876 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.628201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c71598-634a-46b4-8554-cc3314210876 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.628850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c71598-634a-46b4-8554-cc3314210876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:31.630939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c71598-634a-46b4-8554-cc3314210876 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.647440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab585d-cf21-4c15-bf9d-938de51a18b2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.648954 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2627/5321] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:31.657548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab585d-cf21-4c15-bf9d-938de51a18b2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.663506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17b153-8fa0-4cbe-b5cc-72f3f1521d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:31.663887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17b153-8fa0-4cbe-b5cc-72f3f1521d40 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.664042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17b153-8fa0-4cbe-b5cc-72f3f1521d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.664339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17b153-8fa0-4cbe-b5cc-72f3f1521d40 admin admin] PATH_INFO: `/v3/projects/72df0f6759df4d39b842cc44df5474f1/users/366d391b87a141dd8e9c0f77763c5777/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.674546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:31.682489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17b153-8fa0-4cbe-b5cc-72f3f1521d40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72df0f6759df4d39b842cc44df5474f1, user_id=366d391b87a141dd8e9c0f77763c5777, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:31.686989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17b153-8fa0-4cbe-b5cc-72f3f1521d40 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.704555 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2695/5322] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:31 2022] PUT /identity/v3/projects/72df0f6759df4d39b842cc44df5474f1/users/366d391b87a141dd8e9c0f77763c5777/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:31.705972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c71598-634a-46b4-8554-cc3314210876 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.721053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c71598-634a-46b4-8554-cc3314210876 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.800883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17b153-8fa0-4cbe-b5cc-72f3f1521d40 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.803388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8b17b153-8fa0-4cbe-b5cc-72f3f1521d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:31.804823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b648c8-cf6c-4724-a1f7-9de2bfc8c2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:31.806077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b648c8-cf6c-4724-a1f7-9de2bfc8c2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.806420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b648c8-cf6c-4724-a1f7-9de2bfc8c2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.806709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b648c8-cf6c-4724-a1f7-9de2bfc8c2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.807739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b648c8-cf6c-4724-a1f7-9de2bfc8c2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:31.809528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b648c8-cf6c-4724-a1f7-9de2bfc8c2e0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.828118 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2628/5323] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:31.829534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b648c8-cf6c-4724-a1f7-9de2bfc8c2e0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.846488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b648c8-cf6c-4724-a1f7-9de2bfc8c2e0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.852637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1e3d4-18ba-4c38-b07d-9e7cdf2151f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:31.853343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1e3d4-18ba-4c38-b07d-9e7cdf2151f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:31.853630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1e3d4-18ba-4c38-b07d-9e7cdf2151f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:31.853907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1e3d4-18ba-4c38-b07d-9e7cdf2151f6 admin admin] PATH_INFO: `/v3/projects/72df0f6759df4d39b842cc44df5474f1/users/366d391b87a141dd8e9c0f77763c5777/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:31.855864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1e3d4-18ba-4c38-b07d-9e7cdf2151f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72df0f6759df4d39b842cc44df5474f1, user_id=366d391b87a141dd8e9c0f77763c5777, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:31.862513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1e3d4-18ba-4c38-b07d-9e7cdf2151f6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:31.865995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:31.879603 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2629/5324] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:31 2022] PUT /identity/v3/projects/72df0f6759df4d39b842cc44df5474f1/users/366d391b87a141dd8e9c0f77763c5777/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:31.881339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1e3d4-18ba-4c38-b07d-9e7cdf2151f6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:31.896613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-def1e3d4-18ba-4c38-b07d-9e7cdf2151f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:31.938346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17b153-8fa0-4cbe-b5cc-72f3f1521d40 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:31.999400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1e3d4-18ba-4c38-b07d-9e7cdf2151f6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:32.005644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4700d-6009-4f4b-8544-3b5791cd0f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.006404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4700d-6009-4f4b-8544-3b5791cd0f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.007082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4700d-6009-4f4b-8544-3b5791cd0f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.007681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4700d-6009-4f4b-8544-3b5791cd0f36 admin admin] PATH_INFO: `/v3/projects/b0f346da77a147f69bb85dece3cac287` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.009357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4700d-6009-4f4b-8544-3b5791cd0f36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f346da77a147f69bb85dece3cac287)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.012144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4700d-6009-4f4b-8544-3b5791cd0f36 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.034541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09b23c-c0a7-439d-916d-12c491565158 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.035176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09b23c-c0a7-439d-916d-12c491565158 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.035531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09b23c-c0a7-439d-916d-12c491565158 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.035881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09b23c-c0a7-439d-916d-12c491565158 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.038029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ba09b23c-c0a7-439d-916d-12c491565158 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:32.077570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-61e4700d-6009-4f4b-8544-3b5791cd0f36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f346da77a147f69bb85dece3cac287', 'request_id': 'req-61e4700d-6009-4f4b-8544-3b5791cd0f36'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.088569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-61e4700d-6009-4f4b-8544-3b5791cd0f36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f346da77a147f69bb85dece3cac287', 'request_id': 'req-61e4700d-6009-4f4b-8544-3b5791cd0f36'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.096630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-61e4700d-6009-4f4b-8544-3b5791cd0f36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f346da77a147f69bb85dece3cac287', 'request_id': 'req-61e4700d-6009-4f4b-8544-3b5791cd0f36'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.112712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-61e4700d-6009-4f4b-8544-3b5791cd0f36 admin admin] The token cache is being invalidate because project b0f346da77a147f69bb85dece3cac287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:32.113159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-61e4700d-6009-4f4b-8544-3b5791cd0f36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.113451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09b23c-c0a7-439d-916d-12c491565158 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.113728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-61e4700d-6009-4f4b-8544-3b5791cd0f36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.115237 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2630/5325] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:31 2022] DELETE /identity/v3/projects/b0f346da77a147f69bb85dece3cac287 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:32.119363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09b23c-c0a7-439d-916d-12c491565158 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.119490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd3577-025f-4d0c-8737-d48e61fb429f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.119490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd3577-025f-4d0c-8737-d48e61fb429f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.119490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd3577-025f-4d0c-8737-d48e61fb429f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.126825 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2696/5326] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:08:32.129271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab2cb2-6198-4895-85df-5a18bac89e18 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.129639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab2cb2-6198-4895-85df-5a18bac89e18 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.129990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab2cb2-6198-4895-85df-5a18bac89e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.151545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-44ab2cb2-6198-4895-85df-5a18bac89e18 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:32.154231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-44ab2cb2-6198-4895-85df-5a18bac89e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:32.167277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-aafd3577-025f-4d0c-8737-d48e61fb429f None None] MFA Rules not processed for user `366d391b87a141dd8e9c0f77763c5777`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:32.168662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-aafd3577-025f-4d0c-8737-d48e61fb429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:32.183187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:32.265280 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2697/5327] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:32.278131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0b016-579c-47ce-b713-0bef0b48a4d2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.279164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0b016-579c-47ce-b713-0bef0b48a4d2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.279797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0b016-579c-47ce-b713-0bef0b48a4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.288528 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2631/5328] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:32.289411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd3577-025f-4d0c-8737-d48e61fb429f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:32.299874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd3577-025f-4d0c-8737-d48e61fb429f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:32.320237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4515b63-be3e-4e19-aa8a-0b8498f07237 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.320648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-98d0b016-579c-47ce-b713-0bef0b48a4d2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:32.320861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4515b63-be3e-4e19-aa8a-0b8498f07237 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.321323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4515b63-be3e-4e19-aa8a-0b8498f07237 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.321614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4515b63-be3e-4e19-aa8a-0b8498f07237 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.321933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-98d0b016-579c-47ce-b713-0bef0b48a4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:32.322255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4515b63-be3e-4e19-aa8a-0b8498f07237 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.324195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4515b63-be3e-4e19-aa8a-0b8498f07237 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.340935 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2698/5329] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:32.341234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:32.348795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0b016-579c-47ce-b713-0bef0b48a4d2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:32.360410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0b016-579c-47ce-b713-0bef0b48a4d2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:32.365991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c095a7-cb4a-40e5-b19d-573d1926eb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.366607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c095a7-cb4a-40e5-b19d-573d1926eb14 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.366893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c095a7-cb4a-40e5-b19d-573d1926eb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.367172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c095a7-cb4a-40e5-b19d-573d1926eb14 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.367846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c095a7-cb4a-40e5-b19d-573d1926eb14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:32.368131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c095a7-cb4a-40e5-b19d-573d1926eb14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.370026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c095a7-cb4a-40e5-b19d-573d1926eb14 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.375720 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2632/5330] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:32 2022] POST /identity/v3/projects => generated 360 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:32.377012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4515b63-be3e-4e19-aa8a-0b8498f07237 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:32.384939 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2699/5331] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:32.387163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c095a7-cb4a-40e5-b19d-573d1926eb14 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:32.398767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c095a7-cb4a-40e5-b19d-573d1926eb14 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:32.457615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4515b63-be3e-4e19-aa8a-0b8498f07237 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:32.462602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ddcd29-5e9d-40b3-b9b4-9e43f745aefc tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.463208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ddcd29-5e9d-40b3-b9b4-9e43f745aefc tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.463492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ddcd29-5e9d-40b3-b9b4-9e43f745aefc tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.463772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ddcd29-5e9d-40b3-b9b4-9e43f745aefc tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.464417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd085780-ee14-4e16-9bc6-06d60c330773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.464545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ddcd29-5e9d-40b3-b9b4-9e43f745aefc tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.465209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd085780-ee14-4e16-9bc6-06d60c330773 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.465566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd085780-ee14-4e16-9bc6-06d60c330773 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.465913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd085780-ee14-4e16-9bc6-06d60c330773 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.466609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd085780-ee14-4e16-9bc6-06d60c330773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.466778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ddcd29-5e9d-40b3-b9b4-9e43f745aefc tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.468663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd085780-ee14-4e16-9bc6-06d60c330773 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.506243 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2633/5332] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:32 2022] POST /identity/v3/users => generated 410 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:32.507101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ddcd29-5e9d-40b3-b9b4-9e43f745aefc tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:32.516521 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2700/5333] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:32 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:32.517672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd085780-ee14-4e16-9bc6-06d60c330773 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:32.518418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ddcd29-5e9d-40b3-b9b4-9e43f745aefc tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:32.591742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3e50d-609a-426e-a1f1-69418392e935 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.593376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3e50d-609a-426e-a1f1-69418392e935 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.593863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3e50d-609a-426e-a1f1-69418392e935 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.594371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3e50d-609a-426e-a1f1-69418392e935 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/users/f51d8a71832b4285b48c3fefc34d97ee` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.600365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd085780-ee14-4e16-9bc6-06d60c330773 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:32.605726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fe0c1-2fc3-4614-a790-c2c9b31c7d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.606339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fe0c1-2fc3-4614-a790-c2c9b31c7d2d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.606623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fe0c1-2fc3-4614-a790-c2c9b31c7d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.606901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fe0c1-2fc3-4614-a790-c2c9b31c7d2d admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.607535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fe0c1-2fc3-4614-a790-c2c9b31c7d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.609948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fe0c1-2fc3-4614-a790-c2c9b31c7d2d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.612078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3e50d-609a-426e-a1f1-69418392e935 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:get_user(user_id=f51d8a71832b4285b48c3fefc34d97ee)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.615318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3e50d-609a-426e-a1f1-69418392e935 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.617501 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2634/5334] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Sep 3 21:08:32 2022] GET /identity/v3/users/f51d8a71832b4285b48c3fefc34d97ee => generated 410 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:32.621801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3e50d-609a-426e-a1f1-69418392e935 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:32.634102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3e50d-609a-426e-a1f1-69418392e935 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:32.637391 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2701/5335] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:32 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:32.641124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fe0c1-2fc3-4614-a790-c2c9b31c7d2d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:32.642421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.643139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.643500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.644003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/users/f51d8a71832b4285b48c3fefc34d97ee` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.645659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_user(user_id=f51d8a71832b4285b48c3fefc34d97ee)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.648124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.653011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fe0c1-2fc3-4614-a790-c2c9b31c7d2d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:32.658909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f536d3-94ac-4882-9eaf-2bb0182a5ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.659607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f536d3-94ac-4882-9eaf-2bb0182a5ade admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.659882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f536d3-94ac-4882-9eaf-2bb0182a5ade admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.660332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f536d3-94ac-4882-9eaf-2bb0182a5ade admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.662356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f536d3-94ac-4882-9eaf-2bb0182a5ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.662715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f536d3-94ac-4882-9eaf-2bb0182a5ade admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.673093 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2702/5336] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:32.677852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f536d3-94ac-4882-9eaf-2bb0182a5ade admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:32.689581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f536d3-94ac-4882-9eaf-2bb0182a5ade admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:32.696787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f51d8a71832b4285b48c3fefc34d97ee'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.697448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd0aa7-34e0-4360-90ed-5335eddac045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.698291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0aa7-34e0-4360-90ed-5335eddac045 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.698596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0aa7-34e0-4360-90ed-5335eddac045 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.699507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0aa7-34e0-4360-90ed-5335eddac045 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.700984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd0aa7-34e0-4360-90ed-5335eddac045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.702876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd0aa7-34e0-4360-90ed-5335eddac045 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.704438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:32.710233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f51d8a71832b4285b48c3fefc34d97ee'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.711209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f51d8a71832b4285b48c3fefc34d97ee'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.712563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:32.722000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f51d8a71832b4285b48c3fefc34d97ee'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.723353 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2703/5337] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:32.728428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd0aa7-34e0-4360-90ed-5335eddac045 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:32.730317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c2cd0aa7-34e0-4360-90ed-5335eddac045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:32.734632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f51d8a71832b4285b48c3fefc34d97ee'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.743720 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2635/5338] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:32 2022] DELETE /identity/v3/users/f51d8a71832b4285b48c3fefc34d97ee => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:32.746108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd0aa7-34e0-4360-90ed-5335eddac045 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:32.747620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:32.749254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:32.838183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e3d95-3fc7-4e06-9640-4e535f61d270 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:32.843776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.844359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.844649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.844956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/439531fe8ac54e45baab2f075db46b14` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.846260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=439531fe8ac54e45baab2f075db46b14)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.849049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.853736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f48cd-fc49-4ea2-bbaf-bc151d7ba470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:32.855413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f48cd-fc49-4ea2-bbaf-bc151d7ba470 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.855710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f48cd-fc49-4ea2-bbaf-bc151d7ba470 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.856956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f48cd-fc49-4ea2-bbaf-bc151d7ba470 admin admin] PATH_INFO: `/v3/projects/9092516787ff4ea7847227f4a8266ceb/users/5875e3d67b864f3abe87e9c8a179d602/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.877938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f48cd-fc49-4ea2-bbaf-bc151d7ba470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9092516787ff4ea7847227f4a8266ceb, user_id=5875e3d67b864f3abe87e9c8a179d602, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:32.882484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f48cd-fc49-4ea2-bbaf-bc151d7ba470 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:32.899117 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2704/5339] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:32 2022] PUT /identity/v3/projects/9092516787ff4ea7847227f4a8266ceb/users/5875e3d67b864f3abe87e9c8a179d602/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:32.910666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '439531fe8ac54e45baab2f075db46b14', 'request_id': 'req-390014eb-3baf-4bce-af68-64386af3904e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.911153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1945ca-025f-4799-8029-877acabc18a7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:32.911964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1945ca-025f-4799-8029-877acabc18a7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:32.911964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1945ca-025f-4799-8029-877acabc18a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:32.918992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '439531fe8ac54e45baab2f075db46b14', 'request_id': 'req-390014eb-3baf-4bce-af68-64386af3904e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.929843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '439531fe8ac54e45baab2f075db46b14', 'request_id': 'req-390014eb-3baf-4bce-af68-64386af3904e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.946320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project 439531fe8ac54e45baab2f075db46b14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:32.946980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.947836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:32.953671 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2636/5340] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:32 2022] DELETE /identity/v3/projects/439531fe8ac54e45baab2f075db46b14 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:32.955253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:32.955850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8c1945ca-025f-4799-8029-877acabc18a7 None None] MFA Rules not processed for user `5875e3d67b864f3abe87e9c8a179d602`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:32.957119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:32.957596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8c1945ca-025f-4799-8029-877acabc18a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:33.033502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390014eb-3baf-4bce-af68-64386af3904e tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.034872 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2705/5341] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:33.039643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1945ca-025f-4799-8029-877acabc18a7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.041406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8c1945ca-025f-4799-8029-877acabc18a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:33.042256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fb13e-fb69-4de8-bb26-8feaedf8cc32 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.042884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fb13e-fb69-4de8-bb26-8feaedf8cc32 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.043174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fb13e-fb69-4de8-bb26-8feaedf8cc32 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.043456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fb13e-fb69-4de8-bb26-8feaedf8cc32 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.044099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fb13e-fb69-4de8-bb26-8feaedf8cc32 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.045870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fb13e-fb69-4de8-bb26-8feaedf8cc32 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.069112 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2637/5342] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:32 2022] POST /identity/v3/projects => generated 347 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:33.072812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fb13e-fb69-4de8-bb26-8feaedf8cc32 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.083002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:33.134118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1945ca-025f-4799-8029-877acabc18a7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.150086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:33.150327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fb13e-fb69-4de8-bb26-8feaedf8cc32 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.155750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63dc03-e951-4ba3-b879-0b04d7e81ac7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.156374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63dc03-e951-4ba3-b879-0b04d7e81ac7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.156665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63dc03-e951-4ba3-b879-0b04d7e81ac7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.157348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63dc03-e951-4ba3-b879-0b04d7e81ac7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.157652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63dc03-e951-4ba3-b879-0b04d7e81ac7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Adding query filter params (is_domain=True) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:33.157984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63dc03-e951-4ba3-b879-0b04d7e81ac7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.160402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63dc03-e951-4ba3-b879-0b04d7e81ac7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.181782 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2638/5343] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Sep 3 21:08:33 2022] GET /identity/v3/projects?is_domain=True => generated 1019 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:33.182927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63dc03-e951-4ba3-b879-0b04d7e81ac7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.194048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63dc03-e951-4ba3-b879-0b04d7e81ac7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.198781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c3316-9d93-487e-8a3a-723833f1daf0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.199386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c3316-9d93-487e-8a3a-723833f1daf0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.199678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c3316-9d93-487e-8a3a-723833f1daf0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.199956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c3316-9d93-487e-8a3a-723833f1daf0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.200570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c3316-9d93-487e-8a3a-723833f1daf0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.202237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c3316-9d93-487e-8a3a-723833f1daf0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.204933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c578d1-2407-4537-8320-5d0bffcbfa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.205153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c578d1-2407-4537-8320-5d0bffcbfa15 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.205477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c578d1-2407-4537-8320-5d0bffcbfa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.205754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c578d1-2407-4537-8320-5d0bffcbfa15 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.206925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c578d1-2407-4537-8320-5d0bffcbfa15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.208947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c578d1-2407-4537-8320-5d0bffcbfa15 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.216551 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2639/5344] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Sep 3 21:08:33 2022] GET /identity/v3/domains => generated 828 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:33.220643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c3316-9d93-487e-8a3a-723833f1daf0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.233122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c3316-9d93-487e-8a3a-723833f1daf0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.239850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.240663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.241091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.241379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/cbaff2e43ae44f1195b6299d78ad626f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.242760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:update_project(project_id=cbaff2e43ae44f1195b6299d78ad626f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.243220 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2706/5345] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:33 2022] POST /identity/v3/projects => generated 377 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:33.246108 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:33.246108 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:33.246108 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:33.247121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.248377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c578d1-2407-4537-8320-5d0bffcbfa15 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.262346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'cbaff2e43ae44f1195b6299d78ad626f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.263279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'cbaff2e43ae44f1195b6299d78ad626f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.294608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'cbaff2e43ae44f1195b6299d78ad626f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.295251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'cbaff2e43ae44f1195b6299d78ad626f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.297830 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2640/5346] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Sep 3 21:08:33 2022] PATCH /identity/v3/projects/cbaff2e43ae44f1195b6299d78ad626f => generated 361 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:33.302386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.303779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:33.325481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c578d1-2407-4537-8320-5d0bffcbfa15 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.403425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b057f99-192a-4c63-a0d3-3715b1982f13 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.412584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.413468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.413729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.414012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/cbaff2e43ae44f1195b6299d78ad626f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.423106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273f9c1-30f0-4c12-a9c4-06e8cbc16f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.423106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273f9c1-30f0-4c12-a9c4-06e8cbc16f95 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.423106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273f9c1-30f0-4c12-a9c4-06e8cbc16f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.423106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273f9c1-30f0-4c12-a9c4-06e8cbc16f95 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.424094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273f9c1-30f0-4c12-a9c4-06e8cbc16f95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.424094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273f9c1-30f0-4c12-a9c4-06e8cbc16f95 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.426488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cbaff2e43ae44f1195b6299d78ad626f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.428225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.455391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'cbaff2e43ae44f1195b6299d78ad626f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.457101 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2707/5347] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:33 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:33.460817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273f9c1-30f0-4c12-a9c4-06e8cbc16f95 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.462997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3273f9c1-30f0-4c12-a9c4-06e8cbc16f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:33.463124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'cbaff2e43ae44f1195b6299d78ad626f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.477558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273f9c1-30f0-4c12-a9c4-06e8cbc16f95 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.484489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05af14a-cfd7-4f54-8ab5-a40367d84760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.485139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05af14a-cfd7-4f54-8ab5-a40367d84760 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.485931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05af14a-cfd7-4f54-8ab5-a40367d84760 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.486268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05af14a-cfd7-4f54-8ab5-a40367d84760 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.487179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05af14a-cfd7-4f54-8ab5-a40367d84760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.490076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05af14a-cfd7-4f54-8ab5-a40367d84760 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.500786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:33.508930 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2708/5348] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:33.509805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05af14a-cfd7-4f54-8ab5-a40367d84760 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.520563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05af14a-cfd7-4f54-8ab5-a40367d84760 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.533577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbaff2e43ae44f1195b6299d78ad626f', 'request_id': 'req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.556037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbaff2e43ae44f1195b6299d78ad626f', 'request_id': 'req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.559365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:33.569321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbaff2e43ae44f1195b6299d78ad626f', 'request_id': 'req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.581927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project cbaff2e43ae44f1195b6299d78ad626f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:33.582221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.582934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.601112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'cbaff2e43ae44f1195b6299d78ad626f', 'request_id': 'req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.604559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'cbaff2e43ae44f1195b6299d78ad626f', 'request_id': 'req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.605351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'cbaff2e43ae44f1195b6299d78ad626f', 'request_id': 'req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:33.613394 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2641/5349] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:33 2022] DELETE /identity/v3/projects/cbaff2e43ae44f1195b6299d78ad626f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:33.614142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a485d-07a1-418c-88a3-57500dc61008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.614830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a485d-07a1-418c-88a3-57500dc61008 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.616842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a485d-07a1-418c-88a3-57500dc61008 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.616842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a485d-07a1-418c-88a3-57500dc61008 admin admin] PATH_INFO: `/v3/projects/65482ec3d35b4791abdfd3a9d57d8219/users/841e6ce4ab1b4aab94fb0a41cdbc1cd5/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.618333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.619955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:33.637431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a485d-07a1-418c-88a3-57500dc61008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65482ec3d35b4791abdfd3a9d57d8219, user_id=841e6ce4ab1b4aab94fb0a41cdbc1cd5, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.640348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a485d-07a1-418c-88a3-57500dc61008 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.658262 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2709/5350] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:33 2022] PUT /identity/v3/projects/65482ec3d35b4791abdfd3a9d57d8219/users/841e6ce4ab1b4aab94fb0a41cdbc1cd5/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:33.662740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a485d-07a1-418c-88a3-57500dc61008 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.664830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-539a485d-07a1-418c-88a3-57500dc61008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:33.719376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5b48e-8f66-4da0-8bc6-5f48340c2ec2 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.735473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a485d-07a1-418c-88a3-57500dc61008 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.740807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a423a-140e-4cfc-b70a-70b21681ef80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.741458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a423a-140e-4cfc-b70a-70b21681ef80 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.741765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a423a-140e-4cfc-b70a-70b21681ef80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.742160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a423a-140e-4cfc-b70a-70b21681ef80 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.742824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a423a-140e-4cfc-b70a-70b21681ef80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.744447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a423a-140e-4cfc-b70a-70b21681ef80 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.757705 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2710/5351] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:33.758744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a423a-140e-4cfc-b70a-70b21681ef80 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.769284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a423a-140e-4cfc-b70a-70b21681ef80 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.774481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad48bc-2df3-4c7a-a95e-6ec6f1c65322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.775271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad48bc-2df3-4c7a-a95e-6ec6f1c65322 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.775625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad48bc-2df3-4c7a-a95e-6ec6f1c65322 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.775984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad48bc-2df3-4c7a-a95e-6ec6f1c65322 admin admin] PATH_INFO: `/v3/projects/65482ec3d35b4791abdfd3a9d57d8219/users/841e6ce4ab1b4aab94fb0a41cdbc1cd5/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.778106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad48bc-2df3-4c7a-a95e-6ec6f1c65322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65482ec3d35b4791abdfd3a9d57d8219, user_id=841e6ce4ab1b4aab94fb0a41cdbc1cd5, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.780768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad48bc-2df3-4c7a-a95e-6ec6f1c65322 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.791963 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2711/5352] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:33 2022] PUT /identity/v3/projects/65482ec3d35b4791abdfd3a9d57d8219/users/841e6ce4ab1b4aab94fb0a41cdbc1cd5/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:33.793179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c550d-30d7-4783-912f-37257848de4f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.793992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c550d-30d7-4783-912f-37257848de4f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.794385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c550d-30d7-4783-912f-37257848de4f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.794754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c550d-30d7-4783-912f-37257848de4f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.795500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c550d-30d7-4783-912f-37257848de4f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.797538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c550d-30d7-4783-912f-37257848de4f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.803486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e02e65-d5e8-4b4e-a19b-00c4d82699d4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.803783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e02e65-d5e8-4b4e-a19b-00c4d82699d4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.804075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e02e65-d5e8-4b4e-a19b-00c4d82699d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.834952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-09e02e65-d5e8-4b4e-a19b-00c4d82699d4 None None] MFA Rules not processed for user `841e6ce4ab1b4aab94fb0a41cdbc1cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:33.835538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-09e02e65-d5e8-4b4e-a19b-00c4d82699d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:33.836065 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2642/5353] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:33 2022] POST /identity/v3/projects => generated 360 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:33.840072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c550d-30d7-4783-912f-37257848de4f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.883845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:33.920127 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2712/5354] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:33.934896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e02e65-d5e8-4b4e-a19b-00c4d82699d4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.937707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-09e02e65-d5e8-4b4e-a19b-00c4d82699d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:33.938186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c550d-30d7-4783-912f-37257848de4f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.943115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a2035-01f1-49ce-b2cd-79b692634bca tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.943764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a2035-01f1-49ce-b2cd-79b692634bca tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.944081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a2035-01f1-49ce-b2cd-79b692634bca tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.944379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a2035-01f1-49ce-b2cd-79b692634bca tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/3f70ea2f51c94d28bb1a83f325248ee3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.945575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a2035-01f1-49ce-b2cd-79b692634bca tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:get_project(project_id=3f70ea2f51c94d28bb1a83f325248ee3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.947753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a2035-01f1-49ce-b2cd-79b692634bca tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:33.949410 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2643/5355] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:33 2022] GET /identity/v3/projects/3f70ea2f51c94d28bb1a83f325248ee3 => generated 360 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:33.953476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a2035-01f1-49ce-b2cd-79b692634bca tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:33.962539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:33.966402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a2035-01f1-49ce-b2cd-79b692634bca tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:33.972574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:33.973351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:33.973713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:33.974410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/3f70ea2f51c94d28bb1a83f325248ee3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:33.976096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3f70ea2f51c94d28bb1a83f325248ee3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:33.979114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:34.007986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e02e65-d5e8-4b4e-a19b-00c4d82699d4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:34.013261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b363f-848a-40c6-8bec-cf8e6cafeb0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:34.014129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b363f-848a-40c6-8bec-cf8e6cafeb0b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:34.014682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b363f-848a-40c6-8bec-cf8e6cafeb0b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:34.015089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b363f-848a-40c6-8bec-cf8e6cafeb0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:34.026504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b363f-848a-40c6-8bec-cf8e6cafeb0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:34.028476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b363f-848a-40c6-8bec-cf8e6cafeb0b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:34.039070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f70ea2f51c94d28bb1a83f325248ee3', 'request_id': 'req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.040555 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2713/5356] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:08:34.047057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f70ea2f51c94d28bb1a83f325248ee3', 'request_id': 'req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.053695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f70ea2f51c94d28bb1a83f325248ee3', 'request_id': 'req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.064215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project 3f70ea2f51c94d28bb1a83f325248ee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:34.064507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.065432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.067283 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2644/5357] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:33 2022] DELETE /identity/v3/projects/3f70ea2f51c94d28bb1a83f325248ee3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:34.072539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b363f-848a-40c6-8bec-cf8e6cafeb0b service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:34.086440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b363f-848a-40c6-8bec-cf8e6cafeb0b service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:34.156039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e6908-ca31-4f5d-91fe-9704864bee78 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:34.156690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e6908-ca31-4f5d-91fe-9704864bee78 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:34.157043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e6908-ca31-4f5d-91fe-9704864bee78 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:34.157434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e6908-ca31-4f5d-91fe-9704864bee78 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:34.158084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e6908-ca31-4f5d-91fe-9704864bee78 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:34.160485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e6908-ca31-4f5d-91fe-9704864bee78 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:34.193153 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2714/5358] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:34 2022] POST /identity/v3/projects => generated 359 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:34.197275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:34.199026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:34.213958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a22d5-ef21-468a-829e-b66aa1c6b5f8 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:34.277830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710771f-59b5-4e9a-b76f-04be37eaa9bb tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:34.278508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710771f-59b5-4e9a-b76f-04be37eaa9bb tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:34.278789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710771f-59b5-4e9a-b76f-04be37eaa9bb tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:34.279072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710771f-59b5-4e9a-b76f-04be37eaa9bb tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/29d1f5c103514d19bfdc37d94b35cf43` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:34.280245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710771f-59b5-4e9a-b76f-04be37eaa9bb tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:get_project(project_id=29d1f5c103514d19bfdc37d94b35cf43)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:34.282363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710771f-59b5-4e9a-b76f-04be37eaa9bb tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:34.284006 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2645/5359] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:34 2022] GET /identity/v3/projects/29d1f5c103514d19bfdc37d94b35cf43 => generated 359 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:34.288850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e6908-ca31-4f5d-91fe-9704864bee78 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:34.300106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e6908-ca31-4f5d-91fe-9704864bee78 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:34.305159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:34.305627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:34.305917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:34.306195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/29d1f5c103514d19bfdc37d94b35cf43` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:34.307324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=29d1f5c103514d19bfdc37d94b35cf43)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:34.309471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:34.360428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29d1f5c103514d19bfdc37d94b35cf43', 'request_id': 'req-be1d89b3-6522-4566-bb2d-48a290ec84e4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.368474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29d1f5c103514d19bfdc37d94b35cf43', 'request_id': 'req-be1d89b3-6522-4566-bb2d-48a290ec84e4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.375679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29d1f5c103514d19bfdc37d94b35cf43', 'request_id': 'req-be1d89b3-6522-4566-bb2d-48a290ec84e4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.387058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project 29d1f5c103514d19bfdc37d94b35cf43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:34.387384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.388154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.390063 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2715/5360] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:34 2022] DELETE /identity/v3/projects/29d1f5c103514d19bfdc37d94b35cf43 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:34.395191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710771f-59b5-4e9a-b76f-04be37eaa9bb tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:34.397019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f710771f-59b5-4e9a-b76f-04be37eaa9bb tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:34.412273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710771f-59b5-4e9a-b76f-04be37eaa9bb tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:34.464039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:34.477604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f79bec-d417-41f6-bfc0-b3c5b3b9595c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:34.478423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f79bec-d417-41f6-bfc0-b3c5b3b9595c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:34.478974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f79bec-d417-41f6-bfc0-b3c5b3b9595c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:34.479580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f79bec-d417-41f6-bfc0-b3c5b3b9595c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:34.480309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f79bec-d417-41f6-bfc0-b3c5b3b9595c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:34.482322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f79bec-d417-41f6-bfc0-b3c5b3b9595c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:34.517857 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2646/5361] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:08:34 2022] POST /identity/v3/projects => generated 345 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:34.522136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:34.532955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d89b3-6522-4566-bb2d-48a290ec84e4 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:34.561842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:34.600603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4da2d-0a9c-47e6-bf80-560b79f79dd3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:34.601297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4da2d-0a9c-47e6-bf80-560b79f79dd3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:34.601873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4da2d-0a9c-47e6-bf80-560b79f79dd3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:34.602171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4da2d-0a9c-47e6-bf80-560b79f79dd3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/7d1dc9fa145c4301a2b8e9f5aa1ab229` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:34.603364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4da2d-0a9c-47e6-bf80-560b79f79dd3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:get_project(project_id=7d1dc9fa145c4301a2b8e9f5aa1ab229)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:34.605528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4da2d-0a9c-47e6-bf80-560b79f79dd3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:34.611357 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2716/5362] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:34 2022] GET /identity/v3/projects/7d1dc9fa145c4301a2b8e9f5aa1ab229 => generated 345 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:34.613352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f79bec-d417-41f6-bfc0-b3c5b3b9595c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:34.625898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f79bec-d417-41f6-bfc0-b3c5b3b9595c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:34.634741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:34.635297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:34.635615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:34.636108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/7d1dc9fa145c4301a2b8e9f5aa1ab229` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:34.640278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7d1dc9fa145c4301a2b8e9f5aa1ab229)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:34.642671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:34.705167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1dc9fa145c4301a2b8e9f5aa1ab229', 'request_id': 'req-2211de0f-99f8-415c-b863-2b5fd751afe7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.715060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1dc9fa145c4301a2b8e9f5aa1ab229', 'request_id': 'req-2211de0f-99f8-415c-b863-2b5fd751afe7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.725183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1dc9fa145c4301a2b8e9f5aa1ab229', 'request_id': 'req-2211de0f-99f8-415c-b863-2b5fd751afe7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.740786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project 7d1dc9fa145c4301a2b8e9f5aa1ab229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:34.741700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.742613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:34.744701 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2647/5363] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:34 2022] DELETE /identity/v3/projects/7d1dc9fa145c4301a2b8e9f5aa1ab229 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:34.749378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4da2d-0a9c-47e6-bf80-560b79f79dd3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:34.751705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-78e4da2d-0a9c-47e6-bf80-560b79f79dd3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:34.773332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4da2d-0a9c-47e6-bf80-560b79f79dd3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:34.847548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fe03d-2ff2-423d-bcff-1d8ab06ec480 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:34.848174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fe03d-2ff2-423d-bcff-1d8ab06ec480 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:34.848462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fe03d-2ff2-423d-bcff-1d8ab06ec480 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:34.848739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fe03d-2ff2-423d-bcff-1d8ab06ec480 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:34.849462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699fe03d-2ff2-423d-bcff-1d8ab06ec480 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:34.851374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699fe03d-2ff2-423d-bcff-1d8ab06ec480 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:34.868039 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2717/5364] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:08:34 2022] POST /identity/v3/domains => generated 277 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:34.872176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:34.882819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211de0f-99f8-415c-b863-2b5fd751afe7 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:34.952630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246bc2d-3a2e-410e-948a-de658af8c028 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:34.953285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246bc2d-3a2e-410e-948a-de658af8c028 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:34.954114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246bc2d-3a2e-410e-948a-de658af8c028 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:34.954403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246bc2d-3a2e-410e-948a-de658af8c028 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:34.955047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246bc2d-3a2e-410e-948a-de658af8c028 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:34.957519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246bc2d-3a2e-410e-948a-de658af8c028 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:35.010482 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2648/5365] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:34 2022] POST /identity/v3/projects => generated 404 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:35.014861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fe03d-2ff2-423d-bcff-1d8ab06ec480 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:35.025881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fe03d-2ff2-423d-bcff-1d8ab06ec480 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:35.093267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ddbb0-9317-4a54-8e02-769f6ef8132b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:35.093960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ddbb0-9317-4a54-8e02-769f6ef8132b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:35.094289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ddbb0-9317-4a54-8e02-769f6ef8132b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:35.094513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ddbb0-9317-4a54-8e02-769f6ef8132b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/aa58025faec54705985efb138d00d060` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:35.095751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ddbb0-9317-4a54-8e02-769f6ef8132b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:get_project(project_id=aa58025faec54705985efb138d00d060)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:35.098364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ddbb0-9317-4a54-8e02-769f6ef8132b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:35.100640 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2718/5366] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:35 2022] GET /identity/v3/projects/aa58025faec54705985efb138d00d060 => generated 404 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:35.104643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246bc2d-3a2e-410e-948a-de658af8c028 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:35.115832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246bc2d-3a2e-410e-948a-de658af8c028 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:35.128930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:35.129562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:35.129857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:35.130142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/aa58025faec54705985efb138d00d060` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:35.131724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=aa58025faec54705985efb138d00d060)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:35.133838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:35.176041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:35.196403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa58025faec54705985efb138d00d060', 'request_id': 'req-abf3750a-a714-47f9-bb93-f3d9b233f536'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.204635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa58025faec54705985efb138d00d060', 'request_id': 'req-abf3750a-a714-47f9-bb93-f3d9b233f536'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.212963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa58025faec54705985efb138d00d060', 'request_id': 'req-abf3750a-a714-47f9-bb93-f3d9b233f536'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.227967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project aa58025faec54705985efb138d00d060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:35.228260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.229009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.234092 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2649/5367] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:35 2022] DELETE /identity/v3/projects/aa58025faec54705985efb138d00d060 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:35.235327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ddbb0-9317-4a54-8e02-769f6ef8132b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:35.237871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b66ddbb0-9317-4a54-8e02-769f6ef8132b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:35.255501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ddbb0-9317-4a54-8e02-769f6ef8132b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:35.325557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:35.326676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:35.327286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:35.327765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/domains/edce14ee66af44d4bf1b2a87ddb245f8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:35.334242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=edce14ee66af44d4bf1b2a87ddb245f8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:35.335427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:35.351040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:35.363289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'edce14ee66af44d4bf1b2a87ddb245f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.365210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'edce14ee66af44d4bf1b2a87ddb245f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.398454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'edce14ee66af44d4bf1b2a87ddb245f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.399598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'edce14ee66af44d4bf1b2a87ddb245f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.406223 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2719/5368] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:08:35 2022] PATCH /identity/v3/domains/edce14ee66af44d4bf1b2a87ddb245f8 => generated 278 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:35.407467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:35.418218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3750a-a714-47f9-bb93-f3d9b233f536 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:35.485087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:35.485845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:35.486195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:35.486542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/domains/edce14ee66af44d4bf1b2a87ddb245f8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:35.499228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=edce14ee66af44d4bf1b2a87ddb245f8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:35.500770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:35.529178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'edce14ee66af44d4bf1b2a87ddb245f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.535011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'edce14ee66af44d4bf1b2a87ddb245f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.601767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edce14ee66af44d4bf1b2a87ddb245f8', 'request_id': 'req-0b2e0d82-7e5b-4543-b9ee-e0478b028194'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.618935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edce14ee66af44d4bf1b2a87ddb245f8', 'request_id': 'req-0b2e0d82-7e5b-4543-b9ee-e0478b028194'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.630003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edce14ee66af44d4bf1b2a87ddb245f8', 'request_id': 'req-0b2e0d82-7e5b-4543-b9ee-e0478b028194'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.654586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project edce14ee66af44d4bf1b2a87ddb245f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:35.654948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.656065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.666134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'edce14ee66af44d4bf1b2a87ddb245f8', 'request_id': 'req-0b2e0d82-7e5b-4543-b9ee-e0478b028194'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.670224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'edce14ee66af44d4bf1b2a87ddb245f8', 'request_id': 'req-0b2e0d82-7e5b-4543-b9ee-e0478b028194'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.671275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'edce14ee66af44d4bf1b2a87ddb245f8', 'request_id': 'req-0b2e0d82-7e5b-4543-b9ee-e0478b028194'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:35.672799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:35.678879 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2650/5369] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:35 2022] DELETE /identity/v3/domains/edce14ee66af44d4bf1b2a87ddb245f8 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:35.683767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:35.685717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:35.704213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93e9ad-06c0-448b-9b3c-02e710f5817b tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:35.787547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc51e7d-ceec-4637-98fa-7ca2ca293b76 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:35.788349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc51e7d-ceec-4637-98fa-7ca2ca293b76 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:35.788740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc51e7d-ceec-4637-98fa-7ca2ca293b76 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:35.789149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc51e7d-ceec-4637-98fa-7ca2ca293b76 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:35.789948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc51e7d-ceec-4637-98fa-7ca2ca293b76 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:35.791746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc51e7d-ceec-4637-98fa-7ca2ca293b76 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:35.813905 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2720/5370] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:08:35 2022] POST /identity/v3/domains => generated 276 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:35.818097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:35.819872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:35.837611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e0d82-7e5b-4543-b9ee-e0478b028194 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:35.916533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c99e3-c969-47cd-ab96-7ae349891869 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:35.917112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c99e3-c969-47cd-ab96-7ae349891869 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:35.917396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c99e3-c969-47cd-ab96-7ae349891869 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:35.917676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c99e3-c969-47cd-ab96-7ae349891869 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:35.918331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c99e3-c969-47cd-ab96-7ae349891869 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:35.920267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c99e3-c969-47cd-ab96-7ae349891869 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:35.967660 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2651/5371] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:35 2022] POST /identity/v3/projects => generated 408 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:35.971436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc51e7d-ceec-4637-98fa-7ca2ca293b76 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:35.988472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc51e7d-ceec-4637-98fa-7ca2ca293b76 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:36.085188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf04ab-c79e-4db5-ba6f-497c8e88f49c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:36.086042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf04ab-c79e-4db5-ba6f-497c8e88f49c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:36.086436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf04ab-c79e-4db5-ba6f-497c8e88f49c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:36.086833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf04ab-c79e-4db5-ba6f-497c8e88f49c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:36.087710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf04ab-c79e-4db5-ba6f-497c8e88f49c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:36.090190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf04ab-c79e-4db5-ba6f-497c8e88f49c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:36.139741 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2721/5372] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:35 2022] POST /identity/v3/projects => generated 403 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:36.140546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c99e3-c969-47cd-ab96-7ae349891869 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:36.159224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c99e3-c969-47cd-ab96-7ae349891869 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:36.242781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:36.243685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:36.243978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:36.244260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/36e7b52b6de443bca800e263e05587e4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:36.246169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=36e7b52b6de443bca800e263e05587e4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:36.248466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:36.307220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36e7b52b6de443bca800e263e05587e4', 'request_id': 'req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.315270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36e7b52b6de443bca800e263e05587e4', 'request_id': 'req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.323292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36e7b52b6de443bca800e263e05587e4', 'request_id': 'req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.338597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project 36e7b52b6de443bca800e263e05587e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:36.338930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.339761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.347970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf04ab-c79e-4db5-ba6f-497c8e88f49c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:36.348438 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2652/5373] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:36 2022] DELETE /identity/v3/projects/36e7b52b6de443bca800e263e05587e4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:36.350290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d5cf04ab-c79e-4db5-ba6f-497c8e88f49c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:36.369333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf04ab-c79e-4db5-ba6f-497c8e88f49c tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:36.418447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:36.439096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:36.439961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:36.440291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:36.440811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/9e46f9dad8ae4e17a567c2bf20335d64` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:36.442136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9e46f9dad8ae4e17a567c2bf20335d64)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:36.444268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:36.516562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e46f9dad8ae4e17a567c2bf20335d64', 'request_id': 'req-933a8fc0-a59b-4c6c-96f1-516c854aea93'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.526380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e46f9dad8ae4e17a567c2bf20335d64', 'request_id': 'req-933a8fc0-a59b-4c6c-96f1-516c854aea93'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.534920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e46f9dad8ae4e17a567c2bf20335d64', 'request_id': 'req-933a8fc0-a59b-4c6c-96f1-516c854aea93'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.547868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project 9e46f9dad8ae4e17a567c2bf20335d64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:36.548331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.549136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.553885 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2722/5374] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:36 2022] DELETE /identity/v3/projects/9e46f9dad8ae4e17a567c2bf20335d64 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:36.555450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:36.564051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:36.572637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec10ca-f8b5-45aa-a142-e63a7525d7d0 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:36.659172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:36.661193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:36.661193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:36.661193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/domains/b125ac28c4de4d109d413159b4267653` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:36.661657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=b125ac28c4de4d109d413159b4267653)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:36.663740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:36.688834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'b125ac28c4de4d109d413159b4267653'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.689835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'b125ac28c4de4d109d413159b4267653'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.718248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'b125ac28c4de4d109d413159b4267653'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.719119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'b125ac28c4de4d109d413159b4267653'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.725905 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2653/5375] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:08:36 2022] PATCH /identity/v3/domains/b125ac28c4de4d109d413159b4267653 => generated 277 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:36.726183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:36.727891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:36.742177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a8fc0-a59b-4c6c-96f1-516c854aea93 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:36.808773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:36.809722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:36.810233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:36.810624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/domains/b125ac28c4de4d109d413159b4267653` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:36.823663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=b125ac28c4de4d109d413159b4267653)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:36.825560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:36.850054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'b125ac28c4de4d109d413159b4267653'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.853441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'b125ac28c4de4d109d413159b4267653'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.911091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:36.915018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b125ac28c4de4d109d413159b4267653', 'request_id': 'req-4f80a058-3a8f-4891-b69f-20b3c433ac7f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.926529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b125ac28c4de4d109d413159b4267653', 'request_id': 'req-4f80a058-3a8f-4891-b69f-20b3c433ac7f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.934403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b125ac28c4de4d109d413159b4267653', 'request_id': 'req-4f80a058-3a8f-4891-b69f-20b3c433ac7f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.945795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project b125ac28c4de4d109d413159b4267653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:36.946018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.946699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.955058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'b125ac28c4de4d109d413159b4267653', 'request_id': 'req-4f80a058-3a8f-4891-b69f-20b3c433ac7f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.958244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'b125ac28c4de4d109d413159b4267653', 'request_id': 'req-4f80a058-3a8f-4891-b69f-20b3c433ac7f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.958933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'b125ac28c4de4d109d413159b4267653', 'request_id': 'req-4f80a058-3a8f-4891-b69f-20b3c433ac7f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:36.965389 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2723/5376] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:36 2022] DELETE /identity/v3/domains/b125ac28c4de4d109d413159b4267653 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:36.970831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:36.972335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:36.988106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdea7f-e02a-4f3c-af3a-b7fa9f09fd17 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:37.057834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a69da0-8c6e-4969-a044-701b9df4ed87 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:37.058540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a69da0-8c6e-4969-a044-701b9df4ed87 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:37.058905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a69da0-8c6e-4969-a044-701b9df4ed87 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:37.059300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a69da0-8c6e-4969-a044-701b9df4ed87 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:37.060028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a69da0-8c6e-4969-a044-701b9df4ed87 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:37.063874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a69da0-8c6e-4969-a044-701b9df4ed87 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:37.109882 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2654/5377] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:36 2022] POST /identity/v3/projects => generated 376 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:37.123109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:37.124873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:37.145546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80a058-3a8f-4891-b69f-20b3c433ac7f tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:37.222324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486f74e-90af-4246-970a-c2d983be72c3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:37.222889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486f74e-90af-4246-970a-c2d983be72c3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:37.223178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486f74e-90af-4246-970a-c2d983be72c3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:37.223459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486f74e-90af-4246-970a-c2d983be72c3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/4fad328dd8d248faa7b679b174ec9dda` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:37.225194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486f74e-90af-4246-970a-c2d983be72c3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:get_project(project_id=4fad328dd8d248faa7b679b174ec9dda)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:37.227243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486f74e-90af-4246-970a-c2d983be72c3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:37.231745 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2724/5378] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:37 2022] GET /identity/v3/projects/4fad328dd8d248faa7b679b174ec9dda => generated 376 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:37.233447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a69da0-8c6e-4969-a044-701b9df4ed87 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:37.244707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a69da0-8c6e-4969-a044-701b9df4ed87 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:37.250950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb0dc1-323b-405e-9f5b-f4764ad95b82 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:37.251714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb0dc1-323b-405e-9f5b-f4764ad95b82 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:37.252794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb0dc1-323b-405e-9f5b-f4764ad95b82 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:37.253713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb0dc1-323b-405e-9f5b-f4764ad95b82 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:37.254468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb0dc1-323b-405e-9f5b-f4764ad95b82 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:37.256613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb0dc1-323b-405e-9f5b-f4764ad95b82 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:37.281022 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2655/5379] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Sep 3 21:08:37 2022] GET /identity/v3/projects => generated 6537 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:37.284026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486f74e-90af-4246-970a-c2d983be72c3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:37.294919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486f74e-90af-4246-970a-c2d983be72c3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:37.300052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:37.300707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:37.301030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:37.302027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/4fad328dd8d248faa7b679b174ec9dda` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:37.302683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4fad328dd8d248faa7b679b174ec9dda)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:37.304583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:37.372353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fad328dd8d248faa7b679b174ec9dda', 'request_id': 'req-f56eec12-fc22-4194-abe8-df4b5b906811'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:37.382026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fad328dd8d248faa7b679b174ec9dda', 'request_id': 'req-f56eec12-fc22-4194-abe8-df4b5b906811'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:37.390019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fad328dd8d248faa7b679b174ec9dda', 'request_id': 'req-f56eec12-fc22-4194-abe8-df4b5b906811'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:37.402258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project 4fad328dd8d248faa7b679b174ec9dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:37.402756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:37.403489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:37.405326 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2725/5380] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:37 2022] DELETE /identity/v3/projects/4fad328dd8d248faa7b679b174ec9dda => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:37.411565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb0dc1-323b-405e-9f5b-f4764ad95b82 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:37.413724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-90fb0dc1-323b-405e-9f5b-f4764ad95b82 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:37.427795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb0dc1-323b-405e-9f5b-f4764ad95b82 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:37.497174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:37.524183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ffadc-1257-4617-ac1e-43ae03939516 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:37.524663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ffadc-1257-4617-ac1e-43ae03939516 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:37.525057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ffadc-1257-4617-ac1e-43ae03939516 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:37.525430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ffadc-1257-4617-ac1e-43ae03939516 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:37.526282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ffadc-1257-4617-ac1e-43ae03939516 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:37.528330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ffadc-1257-4617-ac1e-43ae03939516 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:37.576605 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2656/5381] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:08:37 2022] POST /identity/v3/projects => generated 346 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:37.578138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:37.590281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56eec12-fc22-4194-abe8-df4b5b906811 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:37.637213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:37.678572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd57c4f-b7f6-4b84-997e-87e2e43d5872 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:37.679282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd57c4f-b7f6-4b84-997e-87e2e43d5872 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:37.679576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd57c4f-b7f6-4b84-997e-87e2e43d5872 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:37.679960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd57c4f-b7f6-4b84-997e-87e2e43d5872 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/7094d17d623941f780d4fdb64b72dd8a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:37.681339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd57c4f-b7f6-4b84-997e-87e2e43d5872 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:update_project(project_id=7094d17d623941f780d4fdb64b72dd8a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:37.682373 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-bfd57c4f-b7f6-4b84-997e-87e2e43d5872 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:37.682373 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:37.682373 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:37.683913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd57c4f-b7f6-4b84-997e-87e2e43d5872 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:37.725113 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2726/5382] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Sep 3 21:08:37 2022] PATCH /identity/v3/projects/7094d17d623941f780d4fdb64b72dd8a => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:37.729341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ffadc-1257-4617-ac1e-43ae03939516 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:37.742757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ffadc-1257-4617-ac1e-43ae03939516 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:37.748391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120360b-208c-4314-a1e4-028a3cec0b5d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:37.749124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120360b-208c-4314-a1e4-028a3cec0b5d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:37.749442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120360b-208c-4314-a1e4-028a3cec0b5d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:37.749729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120360b-208c-4314-a1e4-028a3cec0b5d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/7094d17d623941f780d4fdb64b72dd8a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:37.763921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120360b-208c-4314-a1e4-028a3cec0b5d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:get_project(project_id=7094d17d623941f780d4fdb64b72dd8a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:37.766357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120360b-208c-4314-a1e4-028a3cec0b5d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:37.772994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd57c4f-b7f6-4b84-997e-87e2e43d5872 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:37.776927 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2657/5383] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:37 2022] GET /identity/v3/projects/7094d17d623941f780d4fdb64b72dd8a => generated 348 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:37.801166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd57c4f-b7f6-4b84-997e-87e2e43d5872 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:37.801166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:37.801166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:37.801519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:37.801519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/7094d17d623941f780d4fdb64b72dd8a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:37.802354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7094d17d623941f780d4fdb64b72dd8a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:37.805065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:37.872925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7094d17d623941f780d4fdb64b72dd8a', 'request_id': 'req-c9cdc815-64f7-4136-b287-046abec76e06'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:37.884845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7094d17d623941f780d4fdb64b72dd8a', 'request_id': 'req-c9cdc815-64f7-4136-b287-046abec76e06'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:37.894480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7094d17d623941f780d4fdb64b72dd8a', 'request_id': 'req-c9cdc815-64f7-4136-b287-046abec76e06'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:37.913258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project 7094d17d623941f780d4fdb64b72dd8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:37.913693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:37.915001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:37.921042 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2727/5384] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:37 2022] DELETE /identity/v3/projects/7094d17d623941f780d4fdb64b72dd8a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:37.921863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120360b-208c-4314-a1e4-028a3cec0b5d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:37.923891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f120360b-208c-4314-a1e4-028a3cec0b5d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:37.944152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120360b-208c-4314-a1e4-028a3cec0b5d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:38.068419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b637c-d691-4a8e-bb8c-272df6143541 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:38.069361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b637c-d691-4a8e-bb8c-272df6143541 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:38.069541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b637c-d691-4a8e-bb8c-272df6143541 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:38.069827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b637c-d691-4a8e-bb8c-272df6143541 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:38.070557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b637c-d691-4a8e-bb8c-272df6143541 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:38.073327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b637c-d691-4a8e-bb8c-272df6143541 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:38.117114 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2658/5385] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:37 2022] POST /identity/v3/projects => generated 361 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:38.119463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:38.134312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdc815-64f7-4136-b287-046abec76e06 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:38.219972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b8cf6-c098-4b42-801f-6c7c163b2a57 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:38.220665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b8cf6-c098-4b42-801f-6c7c163b2a57 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:38.221027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b8cf6-c098-4b42-801f-6c7c163b2a57 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:38.221332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b8cf6-c098-4b42-801f-6c7c163b2a57 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/6ac1436bb5a74a71a9f2ee0c27b2c4a7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:38.222964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b8cf6-c098-4b42-801f-6c7c163b2a57 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:update_project(project_id=6ac1436bb5a74a71a9f2ee0c27b2c4a7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:38.225341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b8cf6-c098-4b42-801f-6c7c163b2a57 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:38.295122 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2728/5386] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Sep 3 21:08:38 2022] PATCH /identity/v3/projects/6ac1436bb5a74a71a9f2ee0c27b2c4a7 => generated 373 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:38.296572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b637c-d691-4a8e-bb8c-272df6143541 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:38.308973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b637c-d691-4a8e-bb8c-272df6143541 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:38.316146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6e6bd-ded1-40b2-ab6d-c38ecd7a7709 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:38.316868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6e6bd-ded1-40b2-ab6d-c38ecd7a7709 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:38.317191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6e6bd-ded1-40b2-ab6d-c38ecd7a7709 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:38.317471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6e6bd-ded1-40b2-ab6d-c38ecd7a7709 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/6ac1436bb5a74a71a9f2ee0c27b2c4a7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:38.333869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6e6bd-ded1-40b2-ab6d-c38ecd7a7709 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:get_project(project_id=6ac1436bb5a74a71a9f2ee0c27b2c4a7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:38.336083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6e6bd-ded1-40b2-ab6d-c38ecd7a7709 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:38.340945 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2659/5387] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:38 2022] GET /identity/v3/projects/6ac1436bb5a74a71a9f2ee0c27b2c4a7 => generated 360 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:38.342194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b8cf6-c098-4b42-801f-6c7c163b2a57 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:38.347803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:UPDATE=2,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:38.354381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b8cf6-c098-4b42-801f-6c7c163b2a57 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:38.361267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:38.361929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:38.362456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:38.363297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/6ac1436bb5a74a71a9f2ee0c27b2c4a7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:38.364568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6ac1436bb5a74a71a9f2ee0c27b2c4a7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:38.366956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:38.427005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac1436bb5a74a71a9f2ee0c27b2c4a7', 'request_id': 'req-f9e102da-2d03-4361-83c2-001e23210fc6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:38.436990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac1436bb5a74a71a9f2ee0c27b2c4a7', 'request_id': 'req-f9e102da-2d03-4361-83c2-001e23210fc6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:38.446541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac1436bb5a74a71a9f2ee0c27b2c4a7', 'request_id': 'req-f9e102da-2d03-4361-83c2-001e23210fc6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:38.458098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project 6ac1436bb5a74a71a9f2ee0c27b2c4a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:38.458393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:38.459089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:38.460818 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2729/5388] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:38 2022] DELETE /identity/v3/projects/6ac1436bb5a74a71a9f2ee0c27b2c4a7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:38.465618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6e6bd-ded1-40b2-ab6d-c38ecd7a7709 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:38.467597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3bb6e6bd-ded1-40b2-ab6d-c38ecd7a7709 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:38.482491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6e6bd-ded1-40b2-ab6d-c38ecd7a7709 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:38.555793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a675f-d43f-4bbc-9dfc-0b088a89968d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:38.556748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a675f-d43f-4bbc-9dfc-0b088a89968d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:38.557221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a675f-d43f-4bbc-9dfc-0b088a89968d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:38.557654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a675f-d43f-4bbc-9dfc-0b088a89968d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:38.558491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a675f-d43f-4bbc-9dfc-0b088a89968d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:38.561363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a675f-d43f-4bbc-9dfc-0b088a89968d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:38.604024 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2660/5389] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:38 2022] POST /identity/v3/projects => generated 354 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:38.607749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:38.629375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e102da-2d03-4361-83c2-001e23210fc6 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:38.698488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286107e-9110-4fca-b434-753b7cb71953 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:38.699308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286107e-9110-4fca-b434-753b7cb71953 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:38.699437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286107e-9110-4fca-b434-753b7cb71953 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:38.699720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286107e-9110-4fca-b434-753b7cb71953 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/ea744b1a888c4a9dbc6c5ed79bca2431` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:38.700998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286107e-9110-4fca-b434-753b7cb71953 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:update_project(project_id=ea744b1a888c4a9dbc6c5ed79bca2431)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:38.702930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286107e-9110-4fca-b434-753b7cb71953 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:38.759415 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2730/5390] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Sep 3 21:08:38 2022] PATCH /identity/v3/projects/ea744b1a888c4a9dbc6c5ed79bca2431 => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:38.761149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a675f-d43f-4bbc-9dfc-0b088a89968d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:38.784818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a675f-d43f-4bbc-9dfc-0b088a89968d tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:38.791110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef413970-987c-4bbe-9ab7-65886ed39d09 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:38.791842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef413970-987c-4bbe-9ab7-65886ed39d09 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:38.793023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef413970-987c-4bbe-9ab7-65886ed39d09 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:38.793544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef413970-987c-4bbe-9ab7-65886ed39d09 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/ea744b1a888c4a9dbc6c5ed79bca2431` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:38.802387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:38.818330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef413970-987c-4bbe-9ab7-65886ed39d09 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:get_project(project_id=ea744b1a888c4a9dbc6c5ed79bca2431)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:38.821649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef413970-987c-4bbe-9ab7-65886ed39d09 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:38.825698 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2661/5391] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:38 2022] GET /identity/v3/projects/ea744b1a888c4a9dbc6c5ed79bca2431 => generated 355 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:38.834071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286107e-9110-4fca-b434-753b7cb71953 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:38.844466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286107e-9110-4fca-b434-753b7cb71953 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:38.852378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:38.853413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:38.853708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:38.853991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/projects/ea744b1a888c4a9dbc6c5ed79bca2431` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:38.855296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ea744b1a888c4a9dbc6c5ed79bca2431)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:38.857709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:38.922732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea744b1a888c4a9dbc6c5ed79bca2431', 'request_id': 'req-0c6c059f-109d-4af3-b34a-f3aa157295ea'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:38.933714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea744b1a888c4a9dbc6c5ed79bca2431', 'request_id': 'req-0c6c059f-109d-4af3-b34a-f3aa157295ea'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:38.937167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:38.953601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea744b1a888c4a9dbc6c5ed79bca2431', 'request_id': 'req-0c6c059f-109d-4af3-b34a-f3aa157295ea'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:38.973355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] The token cache is being invalidate because project ea744b1a888c4a9dbc6c5ed79bca2431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:38.973861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:38.974641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:38.977005 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2731/5392] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:38 2022] DELETE /identity/v3/projects/ea744b1a888c4a9dbc6c5ed79bca2431 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:38.985606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef413970-987c-4bbe-9ab7-65886ed39d09 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:38.991627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ef413970-987c-4bbe-9ab7-65886ed39d09 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:39.011749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef413970-987c-4bbe-9ab7-65886ed39d09 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:39.114075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d317-047f-4cda-b2da-8e87ef8d0513 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:39.114735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d317-047f-4cda-b2da-8e87ef8d0513 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:39.115003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d317-047f-4cda-b2da-8e87ef8d0513 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:39.115282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d317-047f-4cda-b2da-8e87ef8d0513 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/domains/b125ac28c4de4d109d413159b4267653` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:39.141825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5d317-047f-4cda-b2da-8e87ef8d0513 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-b0b5d317-047f-4cda-b2da-8e87ef8d0513 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Could not find domain: b125ac28c4de4d109d413159b4267653.: keystone.exception.DomainNotFound: Could not find domain: b125ac28c4de4d109d413159b4267653. Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: b125ac28c4de4d109d413159b4267653. Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:08:39.148578 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:08:39.149875 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: b125ac28c4de4d109d413159b4267653. Sep 03 21:08:39.150834 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:08:39.156660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:39.157145 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2662/5393] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:08:38 2022] PATCH /identity/v3/domains/b125ac28c4de4d109d413159b4267653 => generated 112 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:08:39.168393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c059f-109d-4af3-b34a-f3aa157295ea tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:39.173546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262a681-e3ed-467c-a18a-9547634804b3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366d391b87a141dd8e9c0f77763c5777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72df0f6759df4d39b842cc44df5474f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:39.174349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262a681-e3ed-467c-a18a-9547634804b3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:39.174777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262a681-e3ed-467c-a18a-9547634804b3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:39.175156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262a681-e3ed-467c-a18a-9547634804b3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] PATH_INFO: `/v3/domains/edce14ee66af44d4bf1b2a87ddb245f8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:39.183189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262a681-e3ed-467c-a18a-9547634804b3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-e262a681-e3ed-467c-a18a-9547634804b3 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Could not find domain: edce14ee66af44d4bf1b2a87ddb245f8.: keystone.exception.DomainNotFound: Could not find domain: edce14ee66af44d4bf1b2a87ddb245f8. Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: edce14ee66af44d4bf1b2a87ddb245f8. Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:08:39.188666 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:08:39.189965 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: edce14ee66af44d4bf1b2a87ddb245f8. Sep 03 21:08:39.191167 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:08:39.195779 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2732/5394] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:08:39 2022] PATCH /identity/v3/domains/edce14ee66af44d4bf1b2a87ddb245f8 => generated 112 bytes in 37 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 03 21:08:39.197049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d317-047f-4cda-b2da-8e87ef8d0513 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:39.199520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b0b5d317-047f-4cda-b2da-8e87ef8d0513 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:39.353799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d317-047f-4cda-b2da-8e87ef8d0513 tempest-ProjectsTestJSON-498178888 tempest-ProjectsTestJSON-498178888-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:39.370233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:39.370526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:39.370526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:39.370927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] PATH_INFO: `/v3/users/4eaef77d012b4543bca6ea825d9354fd` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:39.372313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eaef77d012b4543bca6ea825d9354fd)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:39.374096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:39.457169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eaef77d012b4543bca6ea825d9354fd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:39.477915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eaef77d012b4543bca6ea825d9354fd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:39.479310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eaef77d012b4543bca6ea825d9354fd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:39.494030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eaef77d012b4543bca6ea825d9354fd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:39.507855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eaef77d012b4543bca6ea825d9354fd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:39.519157 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2663/5395] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:39 2022] DELETE /identity/v3/users/4eaef77d012b4543bca6ea825d9354fd => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:39.521922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37bbad-711d-4352-90b2-86a30cf0172b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:39.522320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37bbad-711d-4352-90b2-86a30cf0172b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:39.522747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37bbad-711d-4352-90b2-86a30cf0172b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:39.551353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5c37bbad-711d-4352-90b2-86a30cf0172b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:39.553187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5c37bbad-711d-4352-90b2-86a30cf0172b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:39.587274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:39.589647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:39.614034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:39.634257 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2733/5396] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:39.647712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37bbad-711d-4352-90b2-86a30cf0172b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:39.650732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5c37bbad-711d-4352-90b2-86a30cf0172b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:39.736535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45485fb5-8294-43fb-8317-75101a5334f0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:39.748428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:39.749693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:39.750144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:39.750582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] PATH_INFO: `/v3/users/a5557b5764674ab6b1b97916bad075d0` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:39.752563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5557b5764674ab6b1b97916bad075d0)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:39.756275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:39.778335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37bbad-711d-4352-90b2-86a30cf0172b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:39.792752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7b8ed-6e55-41f1-a1d8-d87fde874f9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:39.793421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b8ed-6e55-41f1-a1d8-d87fde874f9f service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:39.793712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b8ed-6e55-41f1-a1d8-d87fde874f9f service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:39.794023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b8ed-6e55-41f1-a1d8-d87fde874f9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:39.808255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7b8ed-6e55-41f1-a1d8-d87fde874f9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:39.817886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7b8ed-6e55-41f1-a1d8-d87fde874f9f service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:39.836716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5557b5764674ab6b1b97916bad075d0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:39.844518 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2734/5397] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:08:39.854111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5557b5764674ab6b1b97916bad075d0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:39.855094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5557b5764674ab6b1b97916bad075d0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:39.865526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5557b5764674ab6b1b97916bad075d0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:39.877533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5557b5764674ab6b1b97916bad075d0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:39.887277 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2664/5398] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:39 2022] DELETE /identity/v3/users/a5557b5764674ab6b1b97916bad075d0 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:39.894225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18496fad-a270-4600-af25-38cd272eff06 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:39.894521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18496fad-a270-4600-af25-38cd272eff06 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:39.894823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18496fad-a270-4600-af25-38cd272eff06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:39.917831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-18496fad-a270-4600-af25-38cd272eff06 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:39.919073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-18496fad-a270-4600-af25-38cd272eff06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:39.986993 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2735/5399] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:40.000054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:40.001752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:40.074093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71330787-3e6c-4b30-825e-cf83bd45fe3e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:40.078089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e83609-eda5-4909-b1bf-bcef73d2ceda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:40.078791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e83609-eda5-4909-b1bf-bcef73d2ceda service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:40.079163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e83609-eda5-4909-b1bf-bcef73d2ceda service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:40.079519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e83609-eda5-4909-b1bf-bcef73d2ceda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:40.090947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e83609-eda5-4909-b1bf-bcef73d2ceda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:40.092606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e83609-eda5-4909-b1bf-bcef73d2ceda service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:40.107847 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2665/5400] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:40.317886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18496fad-a270-4600-af25-38cd272eff06 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:40.320040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-18496fad-a270-4600-af25-38cd272eff06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:40.348544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18496fad-a270-4600-af25-38cd272eff06 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:40.387737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:40.387737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:40.387737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:40.387737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] PATH_INFO: `/v3/projects/1809ee2f113340dcaed765f3d0d1a460` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:40.388122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1809ee2f113340dcaed765f3d0d1a460)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:40.388689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:40.415150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:40.462881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1809ee2f113340dcaed765f3d0d1a460', 'request_id': 'req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:40.476290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1809ee2f113340dcaed765f3d0d1a460', 'request_id': 'req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:40.508565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1809ee2f113340dcaed765f3d0d1a460', 'request_id': 'req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:40.524781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] The token cache is being invalidate because project 1809ee2f113340dcaed765f3d0d1a460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:40.525718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:40.526855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:40.533069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e83609-eda5-4909-b1bf-bcef73d2ceda service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:40.534574 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2736/5401] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:40 2022] DELETE /identity/v3/projects/1809ee2f113340dcaed765f3d0d1a460 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:40.546048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e83609-eda5-4909-b1bf-bcef73d2ceda service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:40.623227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:40.683289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:40.683706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:40.683992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:40.684317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] PATH_INFO: `/v3/users/366d391b87a141dd8e9c0f77763c5777` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:40.685592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=366d391b87a141dd8e9c0f77763c5777)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:40.688577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:40.767013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '366d391b87a141dd8e9c0f77763c5777'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:40.783581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '366d391b87a141dd8e9c0f77763c5777'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:40.785130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '366d391b87a141dd8e9c0f77763c5777'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:40.795612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '366d391b87a141dd8e9c0f77763c5777'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:40.805202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '366d391b87a141dd8e9c0f77763c5777'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:40.824688 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2666/5402] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:40 2022] DELETE /identity/v3/users/366d391b87a141dd8e9c0f77763c5777 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:40.895690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:40.897434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:40.918447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8e5b6-cc9a-4990-899f-bbb8d4ab46fe admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:40.998661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:40.999329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:40.999640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:40.999920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] PATH_INFO: `/v3/projects/894b66bfd5794f9fb143fc399ea3fb7c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:41.001081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] RBAC: Authorizing `identity:delete_project(project_id=894b66bfd5794f9fb143fc399ea3fb7c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:41.003650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:41.055031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '894b66bfd5794f9fb143fc399ea3fb7c', 'request_id': 'req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.063437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '894b66bfd5794f9fb143fc399ea3fb7c', 'request_id': 'req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.072378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '894b66bfd5794f9fb143fc399ea3fb7c', 'request_id': 'req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.085633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] The token cache is being invalidate because project 894b66bfd5794f9fb143fc399ea3fb7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:41.086153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.086939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.088711 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2737/5403] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:40 2022] DELETE /identity/v3/projects/894b66bfd5794f9fb143fc399ea3fb7c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:41.093017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:41.094520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:41.108386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e3ef3-d458-4d49-8ddf-f1b460210d90 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:41.192583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:41.193249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:41.193602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:41.193810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] PATH_INFO: `/v3/users/9f7b5005a78b4feba479e48860c92750` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:41.195530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f7b5005a78b4feba479e48860c92750)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:41.197521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:41.258979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f7b5005a78b4feba479e48860c92750'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.266712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:41.275476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f7b5005a78b4feba479e48860c92750'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.277355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f7b5005a78b4feba479e48860c92750'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.287036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f7b5005a78b4feba479e48860c92750'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.297345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f7b5005a78b4feba479e48860c92750'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.304465 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2667/5404] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:41 2022] DELETE /identity/v3/users/9f7b5005a78b4feba479e48860c92750 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:41.448377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:41.449984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:41.467626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:41.476574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:41.532662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:41.587792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f37bb-1a6e-41e7-a4b0-a7f3db99608f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:41.588218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d78b2f-1815-4104-ae38-d0d3dc196c56 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:41.597845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:41.599308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:41.599631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:41.600978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] PATH_INFO: `/v3/projects/72df0f6759df4d39b842cc44df5474f1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:41.603723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] RBAC: Authorizing `identity:delete_project(project_id=72df0f6759df4d39b842cc44df5474f1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:41.606736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:41.658180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce58ee-9b52-4744-8005-a089bafbf4f0 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e6ce4ab1b4aab94fb0a41cdbc1cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65482ec3d35b4791abdfd3a9d57d8219', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:41.658428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72df0f6759df4d39b842cc44df5474f1', 'request_id': 'req-e3a7a168-a218-40d6-ae50-bd427554980c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.658859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce58ee-9b52-4744-8005-a089bafbf4f0 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:41.659252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce58ee-9b52-4744-8005-a089bafbf4f0 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:41.659599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce58ee-9b52-4744-8005-a089bafbf4f0 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:41.660331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce58ee-9b52-4744-8005-a089bafbf4f0 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:41.660828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce58ee-9b52-4744-8005-a089bafbf4f0 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:41.662774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce58ee-9b52-4744-8005-a089bafbf4f0 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:41.668669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72df0f6759df4d39b842cc44df5474f1', 'request_id': 'req-e3a7a168-a218-40d6-ae50-bd427554980c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.676662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72df0f6759df4d39b842cc44df5474f1', 'request_id': 'req-e3a7a168-a218-40d6-ae50-bd427554980c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.680236 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2738/5405] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:41.680236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce58ee-9b52-4744-8005-a089bafbf4f0 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:41.680236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3bce58ee-9b52-4744-8005-a089bafbf4f0 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:41.693143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] The token cache is being invalidate because project 72df0f6759df4d39b842cc44df5474f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:41.693520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.694344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.695059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce58ee-9b52-4744-8005-a089bafbf4f0 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:41.696270 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2668/5406] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:41 2022] DELETE /identity/v3/projects/72df0f6759df4d39b842cc44df5474f1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:41.696928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:41.698263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:41.773346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:41.774227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:41.774520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:41.774803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] PATH_INFO: `/v3/projects/832335d127044b1c8c3d431846dbedaa` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:41.775978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] RBAC: Authorizing `identity:delete_project(project_id=832335d127044b1c8c3d431846dbedaa)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:41.777967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:41.785257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7a168-a218-40d6-ae50-bd427554980c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:41.790832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a01fe-9390-4353-b5bd-767d1cb65e87 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e6ce4ab1b4aab94fb0a41cdbc1cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65482ec3d35b4791abdfd3a9d57d8219', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:41.791655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a01fe-9390-4353-b5bd-767d1cb65e87 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:41.791980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a01fe-9390-4353-b5bd-767d1cb65e87 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:41.792511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a01fe-9390-4353-b5bd-767d1cb65e87 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:41.793586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a01fe-9390-4353-b5bd-767d1cb65e87 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:41.796566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a01fe-9390-4353-b5bd-767d1cb65e87 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:41.808378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:41.834452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '832335d127044b1c8c3d431846dbedaa', 'request_id': 'req-5c7ac710-7698-441b-ab00-bf7daa37effe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.844289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '832335d127044b1c8c3d431846dbedaa', 'request_id': 'req-5c7ac710-7698-441b-ab00-bf7daa37effe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.854131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '832335d127044b1c8c3d431846dbedaa', 'request_id': 'req-5c7ac710-7698-441b-ab00-bf7daa37effe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.863581 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2669/5407] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:41 2022] POST /identity/v3/projects => generated 347 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:41.868842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af82c4-7906-4599-9ac1-eb8eff531418 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:41.869211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af82c4-7906-4599-9ac1-eb8eff531418 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:41.869466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af82c4-7906-4599-9ac1-eb8eff531418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:41.873468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] The token cache is being invalidate because project 832335d127044b1c8c3d431846dbedaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:41.875753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.876989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:41.881577 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2739/5408] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:41 2022] DELETE /identity/v3/projects/832335d127044b1c8c3d431846dbedaa => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:41.886486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:41.888345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:41.895763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-82af82c4-7906-4599-9ac1-eb8eff531418 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:41.897127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-82af82c4-7906-4599-9ac1-eb8eff531418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:41.900733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:41.981689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ac710-7698-441b-ab00-bf7daa37effe admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.021715 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2670/5409] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:42.029739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ea46-cd72-4f92-8811-61a94808ebc1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.029739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ea46-cd72-4f92-8811-61a94808ebc1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.029739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ea46-cd72-4f92-8811-61a94808ebc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.061212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843d183-7e40-4726-8f85-6fcce4dcb0e5 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e6ce4ab1b4aab94fb0a41cdbc1cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65482ec3d35b4791abdfd3a9d57d8219', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.062002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843d183-7e40-4726-8f85-6fcce4dcb0e5 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.062299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843d183-7e40-4726-8f85-6fcce4dcb0e5 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.062583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843d183-7e40-4726-8f85-6fcce4dcb0e5 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.063272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1843d183-7e40-4726-8f85-6fcce4dcb0e5 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:42.063591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1843d183-7e40-4726-8f85-6fcce4dcb0e5 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.065374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1843d183-7e40-4726-8f85-6fcce4dcb0e5 tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.066562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fbb4ea46-cd72-4f92-8811-61a94808ebc1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:42.067976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fbb4ea46-cd72-4f92-8811-61a94808ebc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:42.078404 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2671/5410] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:42.079642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83c3f8-0195-40eb-903b-56a450b9de5a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.079936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83c3f8-0195-40eb-903b-56a450b9de5a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.083075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83c3f8-0195-40eb-903b-56a450b9de5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.083214 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2740/5411] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:42.087775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534e48e-56b7-4104-a994-4c8d3349a9e9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.100450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534e48e-56b7-4104-a994-4c8d3349a9e9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.100450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534e48e-56b7-4104-a994-4c8d3349a9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.106496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7f83c3f8-0195-40eb-903b-56a450b9de5a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:42.107855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7f83c3f8-0195-40eb-903b-56a450b9de5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:42.116929 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2672/5412] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:42.122318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83c3f8-0195-40eb-903b-56a450b9de5a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.124004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b534e48e-56b7-4104-a994-4c8d3349a9e9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:42.125434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b534e48e-56b7-4104-a994-4c8d3349a9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:42.133512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83c3f8-0195-40eb-903b-56a450b9de5a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.141798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91107dd2-bd73-45f9-95cd-2bfc39cf36db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.142606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91107dd2-bd73-45f9-95cd-2bfc39cf36db admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.142999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91107dd2-bd73-45f9-95cd-2bfc39cf36db admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.143346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91107dd2-bd73-45f9-95cd-2bfc39cf36db admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.144675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91107dd2-bd73-45f9-95cd-2bfc39cf36db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:42.145136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91107dd2-bd73-45f9-95cd-2bfc39cf36db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.147024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91107dd2-bd73-45f9-95cd-2bfc39cf36db admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.151285 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2741/5413] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:42.152432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534e48e-56b7-4104-a994-4c8d3349a9e9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.164086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534e48e-56b7-4104-a994-4c8d3349a9e9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.166266 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2673/5414] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:42.169295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34dace-3dfd-4fff-84ac-e17f86890069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.169915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34dace-3dfd-4fff-84ac-e17f86890069 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.171008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34dace-3dfd-4fff-84ac-e17f86890069 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.171008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34dace-3dfd-4fff-84ac-e17f86890069 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.171259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34dace-3dfd-4fff-84ac-e17f86890069 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:42.171445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34dace-3dfd-4fff-84ac-e17f86890069 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.172290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91107dd2-bd73-45f9-95cd-2bfc39cf36db admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.173157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34dace-3dfd-4fff-84ac-e17f86890069 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.187875 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2742/5415] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:42.192530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91107dd2-bd73-45f9-95cd-2bfc39cf36db admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.193795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34dace-3dfd-4fff-84ac-e17f86890069 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.204101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea655b6-f4cb-479f-8583-29f11497288a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.204746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea655b6-f4cb-479f-8583-29f11497288a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.205071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea655b6-f4cb-479f-8583-29f11497288a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.205366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea655b6-f4cb-479f-8583-29f11497288a admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.205538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34dace-3dfd-4fff-84ac-e17f86890069 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.207567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea655b6-f4cb-479f-8583-29f11497288a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.208430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea655b6-f4cb-479f-8583-29f11497288a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.211661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b361a54-2f9a-4901-b6b0-4e4472869798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.212378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361a54-2f9a-4901-b6b0-4e4472869798 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.212670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361a54-2f9a-4901-b6b0-4e4472869798 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.212973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361a54-2f9a-4901-b6b0-4e4472869798 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.213609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b361a54-2f9a-4901-b6b0-4e4472869798 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.215521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b361a54-2f9a-4901-b6b0-4e4472869798 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.244207 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2674/5416] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:42 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:42.248367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea655b6-f4cb-479f-8583-29f11497288a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.253723 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2743/5417] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:42 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:42.255511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b361a54-2f9a-4901-b6b0-4e4472869798 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.334484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b361a54-2f9a-4901-b6b0-4e4472869798 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.340567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6b525-e1d4-43ad-8a83-4e2d7eae9ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.341326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6b525-e1d4-43ad-8a83-4e2d7eae9ded admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.341622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6b525-e1d4-43ad-8a83-4e2d7eae9ded admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.341908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6b525-e1d4-43ad-8a83-4e2d7eae9ded admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.342553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6b525-e1d4-43ad-8a83-4e2d7eae9ded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.344967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6b525-e1d4-43ad-8a83-4e2d7eae9ded admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.356732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea655b6-f4cb-479f-8583-29f11497288a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.363652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c576c8-563d-4a73-9efc-0ad47e9434ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.364292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c576c8-563d-4a73-9efc-0ad47e9434ae admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.365367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c576c8-563d-4a73-9efc-0ad47e9434ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.365540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c576c8-563d-4a73-9efc-0ad47e9434ae admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.366590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c576c8-563d-4a73-9efc-0ad47e9434ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.367174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:42.369269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c576c8-563d-4a73-9efc-0ad47e9434ae admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.383213 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2744/5418] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:42 2022] POST /identity/v3/users => generated 291 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:42.387533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6b525-e1d4-43ad-8a83-4e2d7eae9ded admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.397059 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2675/5419] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:42 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:42.400133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6b525-e1d4-43ad-8a83-4e2d7eae9ded admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.401545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c576c8-563d-4a73-9efc-0ad47e9434ae admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.405896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b8863-a9d6-4d59-8cea-b91a79e7a672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.406519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b8863-a9d6-4d59-8cea-b91a79e7a672 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.406997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b8863-a9d6-4d59-8cea-b91a79e7a672 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.407289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b8863-a9d6-4d59-8cea-b91a79e7a672 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.407951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b8863-a9d6-4d59-8cea-b91a79e7a672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.410835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b8863-a9d6-4d59-8cea-b91a79e7a672 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.414401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c576c8-563d-4a73-9efc-0ad47e9434ae admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.421777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e509-5dda-4a72-8e62-967272ff3e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.422416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e509-5dda-4a72-8e62-967272ff3e61 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.422772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e509-5dda-4a72-8e62-967272ff3e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.423120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e509-5dda-4a72-8e62-967272ff3e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.423842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e509-5dda-4a72-8e62-967272ff3e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.424091 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2745/5420] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:42.426341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e509-5dda-4a72-8e62-967272ff3e61 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.428411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b8863-a9d6-4d59-8cea-b91a79e7a672 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.438418 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2676/5421] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:42.443012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e509-5dda-4a72-8e62-967272ff3e61 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.445820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b8863-a9d6-4d59-8cea-b91a79e7a672 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.456290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e509-5dda-4a72-8e62-967272ff3e61 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.461627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9c54-a9dc-45b5-8b7b-ad4042fa4259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.462262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9c54-a9dc-45b5-8b7b-ad4042fa4259 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.462552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9c54-a9dc-45b5-8b7b-ad4042fa4259 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.462838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9c54-a9dc-45b5-8b7b-ad4042fa4259 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.463541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff9c54-a9dc-45b5-8b7b-ad4042fa4259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.465337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff9c54-a9dc-45b5-8b7b-ad4042fa4259 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.473909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fbc62-d10c-464f-b9f1-b634b17ba0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.474925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fbc62-d10c-464f-b9f1-b634b17ba0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.475225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fbc62-d10c-464f-b9f1-b634b17ba0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.475502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fbc62-d10c-464f-b9f1-b634b17ba0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.476147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fbc62-d10c-464f-b9f1-b634b17ba0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.477330 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2746/5422] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:42.480815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fbc62-d10c-464f-b9f1-b634b17ba0b2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.496093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9c54-a9dc-45b5-8b7b-ad4042fa4259 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.512989 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2677/5423] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:42.514835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fbc62-d10c-464f-b9f1-b634b17ba0b2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.521420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9c54-a9dc-45b5-8b7b-ad4042fa4259 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.527253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2862adc-f9d6-4012-a721-1bf98e98b414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.527934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2862adc-f9d6-4012-a721-1bf98e98b414 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.528223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2862adc-f9d6-4012-a721-1bf98e98b414 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.528623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2862adc-f9d6-4012-a721-1bf98e98b414 admin admin] PATH_INFO: `/v3/projects/74845811a8b84f909b5035cf6972f936/users/d0c0a1c831a64e659dbda031c99e6eef/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.531832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fbc62-d10c-464f-b9f1-b634b17ba0b2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.539010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf8338-af68-4729-91b8-fc588e5fefb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.539882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf8338-af68-4729-91b8-fc588e5fefb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.540790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf8338-af68-4729-91b8-fc588e5fefb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.541112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf8338-af68-4729-91b8-fc588e5fefb4 admin admin] PATH_INFO: `/v3/projects/9b96e8f3d3864e6688298eeee9f3350b/users/eb9f3434d660490597b754f092f2be99/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.545468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:42.553832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2862adc-f9d6-4012-a721-1bf98e98b414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74845811a8b84f909b5035cf6972f936, user_id=d0c0a1c831a64e659dbda031c99e6eef, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.557630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2862adc-f9d6-4012-a721-1bf98e98b414 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.562431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf8338-af68-4729-91b8-fc588e5fefb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b96e8f3d3864e6688298eeee9f3350b, user_id=eb9f3434d660490597b754f092f2be99, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.574302 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2747/5424] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:42 2022] PUT /identity/v3/projects/74845811a8b84f909b5035cf6972f936/users/d0c0a1c831a64e659dbda031c99e6eef/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:42.585548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf8338-af68-4729-91b8-fc588e5fefb4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.590676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e9c01-8d9c-4dd0-ba2e-1899d88a56bc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.591110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e9c01-8d9c-4dd0-ba2e-1899d88a56bc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.591252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e9c01-8d9c-4dd0-ba2e-1899d88a56bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.615368 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2678/5425] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:42 2022] PUT /identity/v3/projects/9b96e8f3d3864e6688298eeee9f3350b/users/eb9f3434d660490597b754f092f2be99/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:42.615368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf8338-af68-4729-91b8-fc588e5fefb4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.630789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-166e9c01-8d9c-4dd0-ba2e-1899d88a56bc None None] MFA Rules not processed for user `d0c0a1c831a64e659dbda031c99e6eef`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:42.632081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-166e9c01-8d9c-4dd0-ba2e-1899d88a56bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:42.693300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf8338-af68-4729-91b8-fc588e5fefb4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.699340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e6ce4ab1b4aab94fb0a41cdbc1cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65482ec3d35b4791abdfd3a9d57d8219', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.700034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.700327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.700672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] PATH_INFO: `/v3/projects/3ec986fe6e7146ec91ca2fbdd45c978f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.702554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3ec986fe6e7146ec91ca2fbdd45c978f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.703826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.718619 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2748/5426] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:42.718619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9d78c-e54e-4265-b349-c8d1f7043152 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.718619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9d78c-e54e-4265-b349-c8d1f7043152 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.718619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9d78c-e54e-4265-b349-c8d1f7043152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.749384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-10d9d78c-e54e-4265-b349-c8d1f7043152 None None] MFA Rules not processed for user `eb9f3434d660490597b754f092f2be99`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:42.750618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-10d9d78c-e54e-4265-b349-c8d1f7043152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:42.767046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec986fe6e7146ec91ca2fbdd45c978f', 'request_id': 'req-31a53ca3-1d25-447b-a0d9-656de9b4384a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:42.777515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec986fe6e7146ec91ca2fbdd45c978f', 'request_id': 'req-31a53ca3-1d25-447b-a0d9-656de9b4384a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:42.786215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec986fe6e7146ec91ca2fbdd45c978f', 'request_id': 'req-31a53ca3-1d25-447b-a0d9-656de9b4384a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:42.807336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] The token cache is being invalidate because project 3ec986fe6e7146ec91ca2fbdd45c978f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:42.807336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:42.807336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:42.807336 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2679/5427] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:42 2022] DELETE /identity/v3/projects/3ec986fe6e7146ec91ca2fbdd45c978f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:42.807815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.807815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:42.831414 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2749/5428] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:42.834961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9d78c-e54e-4265-b349-c8d1f7043152 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.834961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-10d9d78c-e54e-4265-b349-c8d1f7043152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:42.869314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:42.883674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a53ca3-1d25-447b-a0d9-656de9b4384a tempest-RoutersIpV6AdminTest-1867779522 tempest-RoutersIpV6AdminTest-1867779522-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.887652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7e2c5-357e-48cc-8044-33030d464d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.888439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7e2c5-357e-48cc-8044-33030d464d89 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.888726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7e2c5-357e-48cc-8044-33030d464d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.889044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7e2c5-357e-48cc-8044-33030d464d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.891277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4ed7e2c5-357e-48cc-8044-33030d464d89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:42.903861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9d78c-e54e-4265-b349-c8d1f7043152 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:42.908579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e73d7-8d90-4b04-baeb-d5ae8a74f6f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:42.910402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e73d7-8d90-4b04-baeb-d5ae8a74f6f9 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:42.910402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e73d7-8d90-4b04-baeb-d5ae8a74f6f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:42.910633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e73d7-8d90-4b04-baeb-d5ae8a74f6f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:42.912519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dd6e73d7-8d90-4b04-baeb-d5ae8a74f6f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:42.926559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:42.968507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7e2c5-357e-48cc-8044-33030d464d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.970772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7e2c5-357e-48cc-8044-33030d464d89 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:42.984755 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2680/5429] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:42.985585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7e2c5-357e-48cc-8044-33030d464d89 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:42.987237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4ed7e2c5-357e-48cc-8044-33030d464d89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:42.997013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e73d7-8d90-4b04-baeb-d5ae8a74f6f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:42.997013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e73d7-8d90-4b04-baeb-d5ae8a74f6f9 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.003115 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2750/5430] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:43.066624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7e2c5-357e-48cc-8044-33030d464d89 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.071987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179795eb-0964-446a-827f-41b58efcce14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.072757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179795eb-0964-446a-827f-41b58efcce14 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.073128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179795eb-0964-446a-827f-41b58efcce14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.073587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179795eb-0964-446a-827f-41b58efcce14 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.074409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179795eb-0964-446a-827f-41b58efcce14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.076182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179795eb-0964-446a-827f-41b58efcce14 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.120819 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2681/5431] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:42 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:43.124494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e73d7-8d90-4b04-baeb-d5ae8a74f6f9 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:43.145197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e73d7-8d90-4b04-baeb-d5ae8a74f6f9 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.225594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10953e-8ccc-49d7-abcb-f854ab3400f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.226233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10953e-8ccc-49d7-abcb-f854ab3400f4 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.226524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10953e-8ccc-49d7-abcb-f854ab3400f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.226951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10953e-8ccc-49d7-abcb-f854ab3400f4 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.227494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10953e-8ccc-49d7-abcb-f854ab3400f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.230216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10953e-8ccc-49d7-abcb-f854ab3400f4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.257083 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2751/5432] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:43 2022] POST /identity/v3/users => generated 291 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:43.262390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179795eb-0964-446a-827f-41b58efcce14 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:43.274152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179795eb-0964-446a-827f-41b58efcce14 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.279290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1495577-c476-41a9-a0f5-bf1d4ab07969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.279906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1495577-c476-41a9-a0f5-bf1d4ab07969 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.280319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1495577-c476-41a9-a0f5-bf1d4ab07969 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.280604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1495577-c476-41a9-a0f5-bf1d4ab07969 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.281341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1495577-c476-41a9-a0f5-bf1d4ab07969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.282956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1495577-c476-41a9-a0f5-bf1d4ab07969 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.295121 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2682/5433] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:43.302870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10953e-8ccc-49d7-abcb-f854ab3400f4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:43.313762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10953e-8ccc-49d7-abcb-f854ab3400f4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.319042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd0d4f-c355-41eb-a97c-bfa778761a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.319822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd0d4f-c355-41eb-a97c-bfa778761a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.320206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd0d4f-c355-41eb-a97c-bfa778761a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.320599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd0d4f-c355-41eb-a97c-bfa778761a21 admin admin] PATH_INFO: `/v3/projects/aba8a48e2ae644bbadf6787dc3b87271/users/5393cb8383114a46b853d09fcdea5c94/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.348405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd0d4f-c355-41eb-a97c-bfa778761a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba8a48e2ae644bbadf6787dc3b87271, user_id=5393cb8383114a46b853d09fcdea5c94, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.351412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd0d4f-c355-41eb-a97c-bfa778761a21 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.368668 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2752/5434] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:43 2022] PUT /identity/v3/projects/aba8a48e2ae644bbadf6787dc3b87271/users/5393cb8383114a46b853d09fcdea5c94/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:43.372566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1495577-c476-41a9-a0f5-bf1d4ab07969 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:43.384871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1495577-c476-41a9-a0f5-bf1d4ab07969 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.452060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885900d-ffcd-4f37-a417-613de8289075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.452671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885900d-ffcd-4f37-a417-613de8289075 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.452984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885900d-ffcd-4f37-a417-613de8289075 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.453388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885900d-ffcd-4f37-a417-613de8289075 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.454173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885900d-ffcd-4f37-a417-613de8289075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.455793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885900d-ffcd-4f37-a417-613de8289075 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.470899 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2683/5435] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:43.472123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd0d4f-c355-41eb-a97c-bfa778761a21 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:43.483549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd0d4f-c355-41eb-a97c-bfa778761a21 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.489130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb006d-aa23-4d07-afe7-8df4d61da73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.489835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb006d-aa23-4d07-afe7-8df4d61da73c admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.490078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb006d-aa23-4d07-afe7-8df4d61da73c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.490352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb006d-aa23-4d07-afe7-8df4d61da73c admin admin] PATH_INFO: `/v3/projects/aba8a48e2ae644bbadf6787dc3b87271/users/5393cb8383114a46b853d09fcdea5c94/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.492819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb006d-aa23-4d07-afe7-8df4d61da73c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba8a48e2ae644bbadf6787dc3b87271, user_id=5393cb8383114a46b853d09fcdea5c94, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.501424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb006d-aa23-4d07-afe7-8df4d61da73c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.512934 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2753/5436] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:43 2022] PUT /identity/v3/projects/aba8a48e2ae644bbadf6787dc3b87271/users/5393cb8383114a46b853d09fcdea5c94/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:43.525218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f7f6a-6892-4cf8-aa76-4fd445489ec6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.525543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f7f6a-6892-4cf8-aa76-4fd445489ec6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.527040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f7f6a-6892-4cf8-aa76-4fd445489ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.559101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-124f7f6a-6892-4cf8-aa76-4fd445489ec6 None None] MFA Rules not processed for user `5393cb8383114a46b853d09fcdea5c94`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:43.560412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-124f7f6a-6892-4cf8-aa76-4fd445489ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:43.609350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:43.651685 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2684/5437] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:43.656768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb006d-aa23-4d07-afe7-8df4d61da73c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:43.667626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb006d-aa23-4d07-afe7-8df4d61da73c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.672176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042e4e9-8a44-497d-96e0-e2f56668b537 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.673159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042e4e9-8a44-497d-96e0-e2f56668b537 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.673326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042e4e9-8a44-497d-96e0-e2f56668b537 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.673651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042e4e9-8a44-497d-96e0-e2f56668b537 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.674435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042e4e9-8a44-497d-96e0-e2f56668b537 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.676630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042e4e9-8a44-497d-96e0-e2f56668b537 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.695412 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2754/5438] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:08:43 2022] POST /identity/v3/services => generated 241 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:43.699583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f7f6a-6892-4cf8-aa76-4fd445489ec6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:43.709194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f7f6a-6892-4cf8-aa76-4fd445489ec6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.725287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c16d4-15b5-4d0c-8831-fd369bbc3219 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.725928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c16d4-15b5-4d0c-8831-fd369bbc3219 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.726220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c16d4-15b5-4d0c-8831-fd369bbc3219 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.726502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c16d4-15b5-4d0c-8831-fd369bbc3219 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services/499e0cbbad884e29a4018ea8c7d4ed14` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.727100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c16d4-15b5-4d0c-8831-fd369bbc3219 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:delete_service(service_id=499e0cbbad884e29a4018ea8c7d4ed14)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.728116 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-bb0c16d4-15b5-4d0c-8831-fd369bbc3219 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:43.728116 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:43.728116 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:43.729181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c16d4-15b5-4d0c-8831-fd369bbc3219 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.758268 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2685/5439] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:43 2022] DELETE /identity/v3/services/499e0cbbad884e29a4018ea8c7d4ed14 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:43.763572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042e4e9-8a44-497d-96e0-e2f56668b537 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:43.775113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042e4e9-8a44-497d-96e0-e2f56668b537 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.790373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70fdc9-9fad-4623-95b1-1ad3e6db8e75 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.791137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70fdc9-9fad-4623-95b1-1ad3e6db8e75 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.791427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70fdc9-9fad-4623-95b1-1ad3e6db8e75 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.791708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70fdc9-9fad-4623-95b1-1ad3e6db8e75 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.792507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70fdc9-9fad-4623-95b1-1ad3e6db8e75 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.794232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70fdc9-9fad-4623-95b1-1ad3e6db8e75 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.805351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:43.816058 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2755/5440] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:43 2022] POST /identity/v3/services => generated 289 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:43.817346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c16d4-15b5-4d0c-8831-fd369bbc3219 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:43.828074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c16d4-15b5-4d0c-8831-fd369bbc3219 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.850330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c1cbf-0d1d-4173-b182-f02ac33ca32d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.850977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c1cbf-0d1d-4173-b182-f02ac33ca32d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.851265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c1cbf-0d1d-4173-b182-f02ac33ca32d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.851544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c1cbf-0d1d-4173-b182-f02ac33ca32d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services/a81a3a14d9be4055a62e522e63af2077` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.852211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c1cbf-0d1d-4173-b182-f02ac33ca32d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:update_service(service_id=a81a3a14d9be4055a62e522e63af2077)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.853243 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-039c1cbf-0d1d-4173-b182-f02ac33ca32d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:43.853243 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:43.853243 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:43.854136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c1cbf-0d1d-4173-b182-f02ac33ca32d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.872084 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2686/5441] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Sep 3 21:08:43 2022] PATCH /identity/v3/services/a81a3a14d9be4055a62e522e63af2077 => generated 283 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:43.876226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70fdc9-9fad-4623-95b1-1ad3e6db8e75 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:43.887103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70fdc9-9fad-4623-95b1-1ad3e6db8e75 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.902664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03bca2-efba-4980-a692-56cf9af03698 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.903298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03bca2-efba-4980-a692-56cf9af03698 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.903610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03bca2-efba-4980-a692-56cf9af03698 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.903892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03bca2-efba-4980-a692-56cf9af03698 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services/a81a3a14d9be4055a62e522e63af2077` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.904482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03bca2-efba-4980-a692-56cf9af03698 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:get_service(service_id=a81a3a14d9be4055a62e522e63af2077)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.905530 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-df03bca2-efba-4980-a692-56cf9af03698 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_service": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:43.905530 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:43.905530 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:43.906429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03bca2-efba-4980-a692-56cf9af03698 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.913183 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2756/5442] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:43 2022] GET /identity/v3/services/a81a3a14d9be4055a62e522e63af2077 => generated 283 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:43.917432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c1cbf-0d1d-4173-b182-f02ac33ca32d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:43.928348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c1cbf-0d1d-4173-b182-f02ac33ca32d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.933791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e491f-2bbe-4d7e-a84e-c0af08a395c6 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.934515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e491f-2bbe-4d7e-a84e-c0af08a395c6 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.934750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e491f-2bbe-4d7e-a84e-c0af08a395c6 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.935117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e491f-2bbe-4d7e-a84e-c0af08a395c6 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services/a81a3a14d9be4055a62e522e63af2077` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.935635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e491f-2bbe-4d7e-a84e-c0af08a395c6 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:delete_service(service_id=a81a3a14d9be4055a62e522e63af2077)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.937270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e491f-2bbe-4d7e-a84e-c0af08a395c6 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:43.958507 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2687/5443] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:43 2022] DELETE /identity/v3/services/a81a3a14d9be4055a62e522e63af2077 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:43.962881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03bca2-efba-4980-a692-56cf9af03698 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:43.973685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03bca2-efba-4980-a692-56cf9af03698 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:43.991483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac242e14-8430-4b10-850b-e10e94b3990d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:43.992136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac242e14-8430-4b10-850b-e10e94b3990d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:43.992421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac242e14-8430-4b10-850b-e10e94b3990d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:43.992698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac242e14-8430-4b10-850b-e10e94b3990d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services/a81a3a14d9be4055a62e522e63af2077` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:43.993312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac242e14-8430-4b10-850b-e10e94b3990d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:get_service(service_id=a81a3a14d9be4055a62e522e63af2077)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:43.994969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac242e14-8430-4b10-850b-e10e94b3990d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-ac242e14-8430-4b10-850b-e10e94b3990d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Could not find service: a81a3a14d9be4055a62e522e63af2077.: keystone.exception.ServiceNotFound: Could not find service: a81a3a14d9be4055a62e522e63af2077. Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: a81a3a14d9be4055a62e522e63af2077. Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:08:44.012245 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:08:44.013441 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: a81a3a14d9be4055a62e522e63af2077. Sep 03 21:08:44.014602 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:08:44.015732 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2757/5444] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:08:43 2022] GET /identity/v3/services/a81a3a14d9be4055a62e522e63af2077 => generated 113 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:08:44.022638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e491f-2bbe-4d7e-a84e-c0af08a395c6 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:44.033266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e491f-2bbe-4d7e-a84e-c0af08a395c6 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:44.038005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b70e1f-0336-47f1-915d-770c7803c1db tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:44.038686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b70e1f-0336-47f1-915d-770c7803c1db tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:44.038977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b70e1f-0336-47f1-915d-770c7803c1db tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:44.039259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b70e1f-0336-47f1-915d-770c7803c1db tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:44.039917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b70e1f-0336-47f1-915d-770c7803c1db tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:44.041585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b70e1f-0336-47f1-915d-770c7803c1db tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:44.057284 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2688/5445] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:44 2022] POST /identity/v3/services => generated 273 bytes in 35 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:44.062388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac242e14-8430-4b10-850b-e10e94b3990d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:44.077244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac242e14-8430-4b10-850b-e10e94b3990d tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:44.101478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ba9ce-c409-4201-ba2c-345bd3442045 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:44.102413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ba9ce-c409-4201-ba2c-345bd3442045 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:44.102856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ba9ce-c409-4201-ba2c-345bd3442045 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:44.103144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ba9ce-c409-4201-ba2c-345bd3442045 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:44.104145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ba9ce-c409-4201-ba2c-345bd3442045 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:44.107371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ba9ce-c409-4201-ba2c-345bd3442045 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:44.156868 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2758/5446] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:44 2022] POST /identity/v3/services => generated 275 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:44.158146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b70e1f-0336-47f1-915d-770c7803c1db tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:44.171909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b70e1f-0336-47f1-915d-770c7803c1db tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:44.194178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92a456-1c56-4472-8aa3-2c28ba47adbb tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:44.195195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92a456-1c56-4472-8aa3-2c28ba47adbb tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:44.195195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92a456-1c56-4472-8aa3-2c28ba47adbb tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:44.195654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92a456-1c56-4472-8aa3-2c28ba47adbb tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:44.196486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92a456-1c56-4472-8aa3-2c28ba47adbb tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:44.198293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92a456-1c56-4472-8aa3-2c28ba47adbb tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:44.218270 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2689/5447] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:44 2022] POST /identity/v3/services => generated 275 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:44.222098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ba9ce-c409-4201-ba2c-345bd3442045 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:44.238815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ba9ce-c409-4201-ba2c-345bd3442045 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:44.267083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604a787-67f2-44b6-a770-7a85aed2b551 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:44.268069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604a787-67f2-44b6-a770-7a85aed2b551 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:44.269107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604a787-67f2-44b6-a770-7a85aed2b551 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:44.269914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604a787-67f2-44b6-a770-7a85aed2b551 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:44.270636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604a787-67f2-44b6-a770-7a85aed2b551 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:44.272251 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-e604a787-67f2-44b6-a770-7a85aed2b551 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:44.272251 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:08:44.272251 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:08:44.273414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604a787-67f2-44b6-a770-7a85aed2b551 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:44.284961 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2759/5448] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Sep 3 21:08:44 2022] GET /identity/v3/services => generated 2978 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:44.287916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92a456-1c56-4472-8aa3-2c28ba47adbb tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:44.298824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92a456-1c56-4472-8aa3-2c28ba47adbb tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:44.303768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94abf2-9faa-4e23-ab0a-1ac0d3597d8a tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:44.304389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94abf2-9faa-4e23-ab0a-1ac0d3597d8a tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:44.304678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94abf2-9faa-4e23-ab0a-1ac0d3597d8a tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:44.305001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94abf2-9faa-4e23-ab0a-1ac0d3597d8a tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:44.305640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94abf2-9faa-4e23-ab0a-1ac0d3597d8a tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-807327595) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:44.305955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94abf2-9faa-4e23-ab0a-1ac0d3597d8a tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:44.306967 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-4f94abf2-9faa-4e23-ab0a-1ac0d3597d8a tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:08:44.306967 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:08:44.306967 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:08:44.307859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94abf2-9faa-4e23-ab0a-1ac0d3597d8a tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:44.314000 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2690/5449] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Sep 3 21:08:44 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-807327595 => generated 415 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:44.318633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604a787-67f2-44b6-a770-7a85aed2b551 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:44.330943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604a787-67f2-44b6-a770-7a85aed2b551 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:44.336738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b411d73-5f84-4424-ae26-2b16bf1a7345 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:44.337457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b411d73-5f84-4424-ae26-2b16bf1a7345 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:44.337743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b411d73-5f84-4424-ae26-2b16bf1a7345 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:44.338018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b411d73-5f84-4424-ae26-2b16bf1a7345 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:44.338659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b411d73-5f84-4424-ae26-2b16bf1a7345 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-1594847119) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:44.338968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b411d73-5f84-4424-ae26-2b16bf1a7345 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:44.340556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b411d73-5f84-4424-ae26-2b16bf1a7345 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:44.349234 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2760/5450] 10.222.0.16 () {58 vars in 1222 bytes} [Sat Sep 3 21:08:44 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-1594847119 => generated 418 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:44.350612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94abf2-9faa-4e23-ab0a-1ac0d3597d8a tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:44.360741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94abf2-9faa-4e23-ab0a-1ac0d3597d8a tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:44.366042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26f60d-0f7d-465c-85e1-c3cf31945ee1 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:44.366674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26f60d-0f7d-465c-85e1-c3cf31945ee1 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:44.366958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26f60d-0f7d-465c-85e1-c3cf31945ee1 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:44.367234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26f60d-0f7d-465c-85e1-c3cf31945ee1 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:44.367885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26f60d-0f7d-465c-85e1-c3cf31945ee1 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-2096536011) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:44.368182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26f60d-0f7d-465c-85e1-c3cf31945ee1 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:44.369942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26f60d-0f7d-465c-85e1-c3cf31945ee1 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:44.376045 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2691/5451] 10.222.0.16 () {58 vars in 1222 bytes} [Sat Sep 3 21:08:44 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-2096536011 => generated 418 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:44.380357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b411d73-5f84-4424-ae26-2b16bf1a7345 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:44.391463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b411d73-5f84-4424-ae26-2b16bf1a7345 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:44.397177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0214d-c241-4fc4-b6ee-4ecd58676e4e tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:44.397867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0214d-c241-4fc4-b6ee-4ecd58676e4e tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:44.398157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0214d-c241-4fc4-b6ee-4ecd58676e4e tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:44.398435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0214d-c241-4fc4-b6ee-4ecd58676e4e tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services/b61862c4985b498fac91a012175c4816` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:44.399030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0214d-c241-4fc4-b6ee-4ecd58676e4e tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:delete_service(service_id=b61862c4985b498fac91a012175c4816)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:44.400646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0214d-c241-4fc4-b6ee-4ecd58676e4e tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:44.430973 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2761/5452] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:44 2022] DELETE /identity/v3/services/b61862c4985b498fac91a012175c4816 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:44.435030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26f60d-0f7d-465c-85e1-c3cf31945ee1 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:44.446567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26f60d-0f7d-465c-85e1-c3cf31945ee1 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:44.462996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7592d-2752-4550-bf3a-cdacb74aa3f4 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:44.463672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7592d-2752-4550-bf3a-cdacb74aa3f4 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:44.463956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7592d-2752-4550-bf3a-cdacb74aa3f4 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:44.464230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7592d-2752-4550-bf3a-cdacb74aa3f4 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services/4e768fc89a5e4afd92dac2ea31fc90a9` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:44.464826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7592d-2752-4550-bf3a-cdacb74aa3f4 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:delete_service(service_id=4e768fc89a5e4afd92dac2ea31fc90a9)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:44.466551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7592d-2752-4550-bf3a-cdacb74aa3f4 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:44.494436 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2692/5453] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:44 2022] DELETE /identity/v3/services/4e768fc89a5e4afd92dac2ea31fc90a9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:44.495396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0214d-c241-4fc4-b6ee-4ecd58676e4e tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:44.508313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0214d-c241-4fc4-b6ee-4ecd58676e4e tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:44.525728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1296c4-0f4b-41f7-b1de-ab17a8caa090 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5393cb8383114a46b853d09fcdea5c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aba8a48e2ae644bbadf6787dc3b87271', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:44.525987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1296c4-0f4b-41f7-b1de-ab17a8caa090 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:44.526272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1296c4-0f4b-41f7-b1de-ab17a8caa090 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:44.526549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1296c4-0f4b-41f7-b1de-ab17a8caa090 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] PATH_INFO: `/v3/services/dafd812cdb964a1c9bb40e9b24238c20` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:44.527147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1296c4-0f4b-41f7-b1de-ab17a8caa090 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorizing `identity:delete_service(service_id=dafd812cdb964a1c9bb40e9b24238c20)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:44.528780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1296c4-0f4b-41f7-b1de-ab17a8caa090 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:44.555883 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2762/5454] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:44 2022] DELETE /identity/v3/services/dafd812cdb964a1c9bb40e9b24238c20 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:44.561368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7592d-2752-4550-bf3a-cdacb74aa3f4 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:44.584230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:44.676474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7592d-2752-4550-bf3a-cdacb74aa3f4 tempest-ServicesTestJSON-1058770928 tempest-ServicesTestJSON-1058770928-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:44.709710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:44.710592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:44.711034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:44.711462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] PATH_INFO: `/v3/users/eb9f3434d660490597b754f092f2be99` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:44.712956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb9f3434d660490597b754f092f2be99)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:44.715136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:44.769313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb9f3434d660490597b754f092f2be99'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:44.785677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb9f3434d660490597b754f092f2be99'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:44.786527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb9f3434d660490597b754f092f2be99'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:44.797357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb9f3434d660490597b754f092f2be99'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:44.806065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb9f3434d660490597b754f092f2be99'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:44.816261 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2693/5455] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:44 2022] DELETE /identity/v3/users/eb9f3434d660490597b754f092f2be99 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:44.818087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24fede-19c6-4f4a-92a2-6166fa0f6135 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:44.818414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24fede-19c6-4f4a-92a2-6166fa0f6135 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:44.818703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24fede-19c6-4f4a-92a2-6166fa0f6135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:44.839874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4f24fede-19c6-4f4a-92a2-6166fa0f6135 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:44.841185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4f24fede-19c6-4f4a-92a2-6166fa0f6135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:44.921169 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2763/5456] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:44.944366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:44.945961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:45.041947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1071ea-deb7-4e37-85f2-1e876adb52f6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:45.048387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba1775-b667-4800-b706-30df8654be80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:45.049997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba1775-b667-4800-b706-30df8654be80 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:45.050295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba1775-b667-4800-b706-30df8654be80 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:45.050724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba1775-b667-4800-b706-30df8654be80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:45.063777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba1775-b667-4800-b706-30df8654be80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:45.066980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba1775-b667-4800-b706-30df8654be80 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:45.087238 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2694/5457] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:45.739221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24fede-19c6-4f4a-92a2-6166fa0f6135 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:45.741446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4f24fede-19c6-4f4a-92a2-6166fa0f6135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:45.761867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24fede-19c6-4f4a-92a2-6166fa0f6135 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:45.766942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:45.767767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:45.768240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:45.769116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] PATH_INFO: `/v3/projects/9b96e8f3d3864e6688298eeee9f3350b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:45.773608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b96e8f3d3864e6688298eeee9f3350b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:45.776506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:45.795986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:45.831879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b96e8f3d3864e6688298eeee9f3350b', 'request_id': 'req-f2d40fe5-2007-4321-beda-b1e78588b398'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:45.839941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b96e8f3d3864e6688298eeee9f3350b', 'request_id': 'req-f2d40fe5-2007-4321-beda-b1e78588b398'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:45.846894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b96e8f3d3864e6688298eeee9f3350b', 'request_id': 'req-f2d40fe5-2007-4321-beda-b1e78588b398'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:45.858812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] The token cache is being invalidate because project 9b96e8f3d3864e6688298eeee9f3350b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:45.859281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:45.859963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:45.861547 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2764/5458] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:45 2022] DELETE /identity/v3/projects/9b96e8f3d3864e6688298eeee9f3350b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:45.865691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba1775-b667-4800-b706-30df8654be80 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:45.876235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba1775-b667-4800-b706-30df8654be80 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:45.886564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:45.941616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fa324-7851-43c3-9b88-a4c76550472a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:45.942261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fa324-7851-43c3-9b88-a4c76550472a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:45.942551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fa324-7851-43c3-9b88-a4c76550472a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:45.942832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fa324-7851-43c3-9b88-a4c76550472a admin admin] PATH_INFO: `/v3/users/5393cb8383114a46b853d09fcdea5c94` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:45.943974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fa324-7851-43c3-9b88-a4c76550472a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5393cb8383114a46b853d09fcdea5c94)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:45.945998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fa324-7851-43c3-9b88-a4c76550472a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:46.006403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-700fa324-7851-43c3-9b88-a4c76550472a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5393cb8383114a46b853d09fcdea5c94'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:46.020938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-700fa324-7851-43c3-9b88-a4c76550472a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5393cb8383114a46b853d09fcdea5c94'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:46.021647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-700fa324-7851-43c3-9b88-a4c76550472a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5393cb8383114a46b853d09fcdea5c94'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:46.028959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-700fa324-7851-43c3-9b88-a4c76550472a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5393cb8383114a46b853d09fcdea5c94'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:46.040672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-700fa324-7851-43c3-9b88-a4c76550472a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5393cb8383114a46b853d09fcdea5c94'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:46.048183 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2695/5459] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:45 2022] DELETE /identity/v3/users/5393cb8383114a46b853d09fcdea5c94 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:46.441827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:46.443319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:46.466002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d40fe5-2007-4321-beda-b1e78588b398 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:46.538701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1017148-1bc4-419d-bfd6-adf2fbab210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:46.539544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1017148-1bc4-419d-bfd6-adf2fbab210c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:46.540243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1017148-1bc4-419d-bfd6-adf2fbab210c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:46.540636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1017148-1bc4-419d-bfd6-adf2fbab210c admin admin] PATH_INFO: `/v3/projects/aba8a48e2ae644bbadf6787dc3b87271` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:46.542009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1017148-1bc4-419d-bfd6-adf2fbab210c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba8a48e2ae644bbadf6787dc3b87271)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:46.544454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1017148-1bc4-419d-bfd6-adf2fbab210c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:46.611959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1017148-1bc4-419d-bfd6-adf2fbab210c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba8a48e2ae644bbadf6787dc3b87271', 'request_id': 'req-a1017148-1bc4-419d-bfd6-adf2fbab210c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:46.623268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1017148-1bc4-419d-bfd6-adf2fbab210c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba8a48e2ae644bbadf6787dc3b87271', 'request_id': 'req-a1017148-1bc4-419d-bfd6-adf2fbab210c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:46.631778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1017148-1bc4-419d-bfd6-adf2fbab210c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba8a48e2ae644bbadf6787dc3b87271', 'request_id': 'req-a1017148-1bc4-419d-bfd6-adf2fbab210c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:46.651993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1017148-1bc4-419d-bfd6-adf2fbab210c admin admin] The token cache is being invalidate because project aba8a48e2ae644bbadf6787dc3b87271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:46.652306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1017148-1bc4-419d-bfd6-adf2fbab210c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:46.653023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1017148-1bc4-419d-bfd6-adf2fbab210c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:46.656951 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2765/5460] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:46 2022] DELETE /identity/v3/projects/aba8a48e2ae644bbadf6787dc3b87271 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:46.660122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3fbee-7bda-4298-b3aa-6188dc5eb1c1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:46.660455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3fbee-7bda-4298-b3aa-6188dc5eb1c1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:46.660737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3fbee-7bda-4298-b3aa-6188dc5eb1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:46.697262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fca3fbee-7bda-4298-b3aa-6188dc5eb1c1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:46.699364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fca3fbee-7bda-4298-b3aa-6188dc5eb1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:46.805017 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2696/5461] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:46.810140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904640dc-615b-45ce-9076-0b15d67f941c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:46.810637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904640dc-615b-45ce-9076-0b15d67f941c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:46.810734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904640dc-615b-45ce-9076-0b15d67f941c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:46.831694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-904640dc-615b-45ce-9076-0b15d67f941c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:46.832960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-904640dc-615b-45ce-9076-0b15d67f941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:46.843561 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2766/5462] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:46.848129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3fbee-7bda-4298-b3aa-6188dc5eb1c1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:46.858396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3fbee-7bda-4298-b3aa-6188dc5eb1c1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:46.863685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76acbb0-b4c6-4e24-b448-3e6db7a2170b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:46.864304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76acbb0-b4c6-4e24-b448-3e6db7a2170b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:46.864604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76acbb0-b4c6-4e24-b448-3e6db7a2170b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:46.864870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76acbb0-b4c6-4e24-b448-3e6db7a2170b admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:46.865573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76acbb0-b4c6-4e24-b448-3e6db7a2170b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:46.865883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76acbb0-b4c6-4e24-b448-3e6db7a2170b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:46.867635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76acbb0-b4c6-4e24-b448-3e6db7a2170b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:46.881210 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2697/5463] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:46.885516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904640dc-615b-45ce-9076-0b15d67f941c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:46.896878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904640dc-615b-45ce-9076-0b15d67f941c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:46.901872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179efee-651e-4ca1-b1d7-06a79cb7f9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:46.902507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179efee-651e-4ca1-b1d7-06a79cb7f9ae admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:46.902864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179efee-651e-4ca1-b1d7-06a79cb7f9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:46.903352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179efee-651e-4ca1-b1d7-06a79cb7f9ae admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:46.904030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179efee-651e-4ca1-b1d7-06a79cb7f9ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:46.906324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179efee-651e-4ca1-b1d7-06a79cb7f9ae admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:46.935810 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2767/5464] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:46 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:46.939839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76acbb0-b4c6-4e24-b448-3e6db7a2170b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:46.959982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76acbb0-b4c6-4e24-b448-3e6db7a2170b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:46.979160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:47.027137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b7805-a3d2-47a8-9562-82f1544b3317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:47.027760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b7805-a3d2-47a8-9562-82f1544b3317 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:47.028048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b7805-a3d2-47a8-9562-82f1544b3317 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:47.028328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b7805-a3d2-47a8-9562-82f1544b3317 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:47.029006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b7805-a3d2-47a8-9562-82f1544b3317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:47.030927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b7805-a3d2-47a8-9562-82f1544b3317 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:47.055628 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2698/5465] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:46 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:47.056853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179efee-651e-4ca1-b1d7-06a79cb7f9ae admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:47.063497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:47.076310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179efee-651e-4ca1-b1d7-06a79cb7f9ae admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:47.088058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9097c1-988c-4446-ae65-bb02259e38ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:47.088741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9097c1-988c-4446-ae65-bb02259e38ba admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:47.089134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9097c1-988c-4446-ae65-bb02259e38ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:47.089479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9097c1-988c-4446-ae65-bb02259e38ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:47.090746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9097c1-988c-4446-ae65-bb02259e38ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:47.092327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9097c1-988c-4446-ae65-bb02259e38ba admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:47.107714 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2768/5466] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:08:47.109411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b7805-a3d2-47a8-9562-82f1544b3317 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:47.122108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b7805-a3d2-47a8-9562-82f1544b3317 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:47.128889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2bf21-9e43-45a3-be84-19ac7d771278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:47.129805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2bf21-9e43-45a3-be84-19ac7d771278 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:47.130192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2bf21-9e43-45a3-be84-19ac7d771278 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:47.130691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2bf21-9e43-45a3-be84-19ac7d771278 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:47.131544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2bf21-9e43-45a3-be84-19ac7d771278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:47.134236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2bf21-9e43-45a3-be84-19ac7d771278 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:47.153114 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2699/5467] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:47.154474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9097c1-988c-4446-ae65-bb02259e38ba admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:47.166432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9097c1-988c-4446-ae65-bb02259e38ba admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:47.169403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2bf21-9e43-45a3-be84-19ac7d771278 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:47.171309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-40a2bf21-9e43-45a3-be84-19ac7d771278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:47.172084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b01a3-b5cf-4254-ab00-9b675d3cab37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:47.172851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b01a3-b5cf-4254-ab00-9b675d3cab37 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:47.173293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b01a3-b5cf-4254-ab00-9b675d3cab37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:47.174389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b01a3-b5cf-4254-ab00-9b675d3cab37 admin admin] PATH_INFO: `/v3/projects/7e2e29c22aa94f558ab737abf40b4c9f/users/d088e1e0b1bc42f3bd065713787d68bf/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:47.195059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b01a3-b5cf-4254-ab00-9b675d3cab37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2e29c22aa94f558ab737abf40b4c9f, user_id=d088e1e0b1bc42f3bd065713787d68bf, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:47.198196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b01a3-b5cf-4254-ab00-9b675d3cab37 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:47.216268 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2769/5468] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:47 2022] PUT /identity/v3/projects/7e2e29c22aa94f558ab737abf40b4c9f/users/d088e1e0b1bc42f3bd065713787d68bf/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:47.229318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a52c0-2583-4297-82ef-3d02b7f95db9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:47.229631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a52c0-2583-4297-82ef-3d02b7f95db9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:47.229923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a52c0-2583-4297-82ef-3d02b7f95db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:47.239053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2bf21-9e43-45a3-be84-19ac7d771278 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:47.266192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e14a52c0-2583-4297-82ef-3d02b7f95db9 None None] MFA Rules not processed for user `d088e1e0b1bc42f3bd065713787d68bf`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:47.267447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e14a52c0-2583-4297-82ef-3d02b7f95db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:47.313429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1894c4-23ad-4856-8aaa-eb6d025fdd81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:47.314054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1894c4-23ad-4856-8aaa-eb6d025fdd81 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:47.314373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1894c4-23ad-4856-8aaa-eb6d025fdd81 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:47.314715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1894c4-23ad-4856-8aaa-eb6d025fdd81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:47.318014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bb1894c4-23ad-4856-8aaa-eb6d025fdd81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:47.399276 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2770/5469] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:47.404403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a52c0-2583-4297-82ef-3d02b7f95db9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:47.414478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a52c0-2583-4297-82ef-3d02b7f95db9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:47.419736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814f328-83d9-43e5-92b6-ba7c4614f94a tempest-DefaultDomainTestJSON-2099490494 tempest-DefaultDomainTestJSON-2099490494-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088e1e0b1bc42f3bd065713787d68bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e2e29c22aa94f558ab737abf40b4c9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:47.420466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814f328-83d9-43e5-92b6-ba7c4614f94a tempest-DefaultDomainTestJSON-2099490494 tempest-DefaultDomainTestJSON-2099490494-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:47.420862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814f328-83d9-43e5-92b6-ba7c4614f94a tempest-DefaultDomainTestJSON-2099490494 tempest-DefaultDomainTestJSON-2099490494-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:47.421247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814f328-83d9-43e5-92b6-ba7c4614f94a tempest-DefaultDomainTestJSON-2099490494 tempest-DefaultDomainTestJSON-2099490494-project] PATH_INFO: `/v3/domains/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:47.422585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814f328-83d9-43e5-92b6-ba7c4614f94a tempest-DefaultDomainTestJSON-2099490494 tempest-DefaultDomainTestJSON-2099490494-project] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:47.424562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814f328-83d9-43e5-92b6-ba7c4614f94a tempest-DefaultDomainTestJSON-2099490494 tempest-DefaultDomainTestJSON-2099490494-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:47.430500 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2771/5470] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Sep 3 21:08:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:47.431498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814f328-83d9-43e5-92b6-ba7c4614f94a tempest-DefaultDomainTestJSON-2099490494 tempest-DefaultDomainTestJSON-2099490494-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:47.433281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:47.441889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1894c4-23ad-4856-8aaa-eb6d025fdd81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:47.444509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1894c4-23ad-4856-8aaa-eb6d025fdd81 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:47.478918 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2700/5471] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:47.540202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814f328-83d9-43e5-92b6-ba7c4614f94a tempest-DefaultDomainTestJSON-2099490494 tempest-DefaultDomainTestJSON-2099490494-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:47.548634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:47.549969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:47.550425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:47.550872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] PATH_INFO: `/v3/users/d088e1e0b1bc42f3bd065713787d68bf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:47.553715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d088e1e0b1bc42f3bd065713787d68bf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:47.556586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:47.625371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd088e1e0b1bc42f3bd065713787d68bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:47.635686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:47.642471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd088e1e0b1bc42f3bd065713787d68bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:47.663189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd088e1e0b1bc42f3bd065713787d68bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:47.664709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd088e1e0b1bc42f3bd065713787d68bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:47.666368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd088e1e0b1bc42f3bd065713787d68bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:47.691555 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2772/5472] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:47 2022] DELETE /identity/v3/users/d088e1e0b1bc42f3bd065713787d68bf => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:47.693957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3f90e-b36c-4aa2-b5f3-729695c6781c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:47.694362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3f90e-b36c-4aa2-b5f3-729695c6781c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:47.694648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3f90e-b36c-4aa2-b5f3-729695c6781c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:47.721515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2bc3f90e-b36c-4aa2-b5f3-729695c6781c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:47.723334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2bc3f90e-b36c-4aa2-b5f3-729695c6781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:47.841329 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2701/5473] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:47.861759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:47.863941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:47.953956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786976ab-0756-4823-8492-33bc32f5d5fc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:47.958329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411d3a8-5ba0-40a6-9eb4-170a7385e1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:47.958957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411d3a8-5ba0-40a6-9eb4-170a7385e1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:47.959243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411d3a8-5ba0-40a6-9eb4-170a7385e1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:47.959522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411d3a8-5ba0-40a6-9eb4-170a7385e1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:47.969526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411d3a8-5ba0-40a6-9eb4-170a7385e1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:47.972168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411d3a8-5ba0-40a6-9eb4-170a7385e1b0 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:47.983240 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2773/5474] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:08:48.250379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3f90e-b36c-4aa2-b5f3-729695c6781c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:48.253131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2bc3f90e-b36c-4aa2-b5f3-729695c6781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:48.269055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3f90e-b36c-4aa2-b5f3-729695c6781c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:48.272848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86903ee-aa8e-421c-b3c5-1d1f75f95144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:48.273519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86903ee-aa8e-421c-b3c5-1d1f75f95144 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:48.273804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86903ee-aa8e-421c-b3c5-1d1f75f95144 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:48.274083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86903ee-aa8e-421c-b3c5-1d1f75f95144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:48.276306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b86903ee-aa8e-421c-b3c5-1d1f75f95144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:48.380688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86903ee-aa8e-421c-b3c5-1d1f75f95144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:48.382607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86903ee-aa8e-421c-b3c5-1d1f75f95144 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:48.401376 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2702/5475] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:48.556291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411d3a8-5ba0-40a6-9eb4-170a7385e1b0 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:48.559057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a411d3a8-5ba0-40a6-9eb4-170a7385e1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:48.582494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411d3a8-5ba0-40a6-9eb4-170a7385e1b0 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:48.587723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:48.589707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:48.590184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:48.591425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a admin admin] PATH_INFO: `/v3/projects/7e2e29c22aa94f558ab737abf40b4c9f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:48.592743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e2e29c22aa94f558ab737abf40b4c9f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:48.594860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:48.653315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e2e29c22aa94f558ab737abf40b4c9f', 'request_id': 'req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:48.662343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e2e29c22aa94f558ab737abf40b4c9f', 'request_id': 'req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:48.671112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e2e29c22aa94f558ab737abf40b4c9f', 'request_id': 'req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:48.693353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a admin admin] The token cache is being invalidate because project 7e2e29c22aa94f558ab737abf40b4c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:48.693523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:48.694466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ab4c4b3-3798-4abf-8e2b-bf334797d04a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:48.701642 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2774/5476] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:48 2022] DELETE /identity/v3/projects/7e2e29c22aa94f558ab737abf40b4c9f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:48.707100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490aacdf-c92b-4270-857f-5d34b5f35887 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:48.708399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490aacdf-c92b-4270-857f-5d34b5f35887 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:48.709635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490aacdf-c92b-4270-857f-5d34b5f35887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:48.800198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-490aacdf-c92b-4270-857f-5d34b5f35887 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:48.805303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-490aacdf-c92b-4270-857f-5d34b5f35887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:48.912370 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2703/5477] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:48.919985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb2321-2a83-4b78-acc0-9774dc7b88a8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:48.920499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb2321-2a83-4b78-acc0-9774dc7b88a8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:48.920587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb2321-2a83-4b78-acc0-9774dc7b88a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:48.952365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d7fb2321-2a83-4b78-acc0-9774dc7b88a8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:48.953651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d7fb2321-2a83-4b78-acc0-9774dc7b88a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:48.963412 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2775/5478] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:48.969266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490aacdf-c92b-4270-857f-5d34b5f35887 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:48.980034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490aacdf-c92b-4270-857f-5d34b5f35887 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:48.985213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c5989-e613-4d98-b212-072e28f15217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:48.985835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c5989-e613-4d98-b212-072e28f15217 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:48.986148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c5989-e613-4d98-b212-072e28f15217 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:48.986436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c5989-e613-4d98-b212-072e28f15217 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:48.987648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c5989-e613-4d98-b212-072e28f15217 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:48.987959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c5989-e613-4d98-b212-072e28f15217 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:48.989662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c5989-e613-4d98-b212-072e28f15217 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:48.999703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:49.006905 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2704/5479] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:49.013214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb2321-2a83-4b78-acc0-9774dc7b88a8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:49.033275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb2321-2a83-4b78-acc0-9774dc7b88a8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:49.033275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdfc30-50bc-4a24-9d4c-73f631fe6727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:49.033275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdfc30-50bc-4a24-9d4c-73f631fe6727 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.033275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdfc30-50bc-4a24-9d4c-73f631fe6727 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.033696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdfc30-50bc-4a24-9d4c-73f631fe6727 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.033696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdfc30-50bc-4a24-9d4c-73f631fe6727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:49.033902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdfc30-50bc-4a24-9d4c-73f631fe6727 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:49.066042 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2776/5480] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:49 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:49.070537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c5989-e613-4d98-b212-072e28f15217 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:49.081382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c5989-e613-4d98-b212-072e28f15217 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:49.147200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904eeea-8bfd-457a-b1cd-a34c65976e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:49.147909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904eeea-8bfd-457a-b1cd-a34c65976e5e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.148198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904eeea-8bfd-457a-b1cd-a34c65976e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.148482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904eeea-8bfd-457a-b1cd-a34c65976e5e admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.149173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904eeea-8bfd-457a-b1cd-a34c65976e5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:49.151176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904eeea-8bfd-457a-b1cd-a34c65976e5e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:49.175208 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2705/5481] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:49 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:49.179271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdfc30-50bc-4a24-9d4c-73f631fe6727 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:49.190032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdfc30-50bc-4a24-9d4c-73f631fe6727 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:49.195563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e2469-9f9b-438c-a281-16b8e30a8ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:49.196271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e2469-9f9b-438c-a281-16b8e30a8ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.196474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e2469-9f9b-438c-a281-16b8e30a8ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.196730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e2469-9f9b-438c-a281-16b8e30a8ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.197412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e2469-9f9b-438c-a281-16b8e30a8ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:49.199043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e2469-9f9b-438c-a281-16b8e30a8ae4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:49.217104 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2777/5482] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:49.217228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904eeea-8bfd-457a-b1cd-a34c65976e5e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:49.225550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904eeea-8bfd-457a-b1cd-a34c65976e5e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:49.230193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bfd32-0b7f-4ef6-87c5-ef048ede4396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:49.230788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bfd32-0b7f-4ef6-87c5-ef048ede4396 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.231070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bfd32-0b7f-4ef6-87c5-ef048ede4396 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.231349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bfd32-0b7f-4ef6-87c5-ef048ede4396 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.231995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bfd32-0b7f-4ef6-87c5-ef048ede4396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:49.233826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bfd32-0b7f-4ef6-87c5-ef048ede4396 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:49.246819 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2706/5483] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:49.252134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e2469-9f9b-438c-a281-16b8e30a8ae4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:49.263509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e2469-9f9b-438c-a281-16b8e30a8ae4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:49.269157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d65c0-d753-454f-ba1b-f46a6f532938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:49.269892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d65c0-d753-454f-ba1b-f46a6f532938 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.270246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d65c0-d753-454f-ba1b-f46a6f532938 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.270592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d65c0-d753-454f-ba1b-f46a6f532938 admin admin] PATH_INFO: `/v3/projects/32bc39ba816044eea2cc6333a0804850/users/c084c28bbe3f495b96f5b683e57e5d24/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.285526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d65c0-d753-454f-ba1b-f46a6f532938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32bc39ba816044eea2cc6333a0804850, user_id=c084c28bbe3f495b96f5b683e57e5d24, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:49.288276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d65c0-d753-454f-ba1b-f46a6f532938 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:49.291575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:49.305599 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2778/5484] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:49 2022] PUT /identity/v3/projects/32bc39ba816044eea2cc6333a0804850/users/c084c28bbe3f495b96f5b683e57e5d24/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:49.316166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5fe60-457f-4c72-a8aa-e4155235c4d2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.316698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5fe60-457f-4c72-a8aa-e4155235c4d2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.316885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5fe60-457f-4c72-a8aa-e4155235c4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.349391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-adc5fe60-457f-4c72-a8aa-e4155235c4d2 None None] MFA Rules not processed for user `c084c28bbe3f495b96f5b683e57e5d24`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:49.351017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-adc5fe60-457f-4c72-a8aa-e4155235c4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:49.418109 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2707/5485] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:49.422699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d65c0-d753-454f-ba1b-f46a6f532938 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:49.487031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d65c0-d753-454f-ba1b-f46a6f532938 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:49.492699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a6a10-f0fa-471d-9e3a-5ce98ff0d70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:49.493341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a6a10-f0fa-471d-9e3a-5ce98ff0d70b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.493630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a6a10-f0fa-471d-9e3a-5ce98ff0d70b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.493908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a6a10-f0fa-471d-9e3a-5ce98ff0d70b admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.494543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a6a10-f0fa-471d-9e3a-5ce98ff0d70b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:49.496465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a6a10-f0fa-471d-9e3a-5ce98ff0d70b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:49.524647 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2779/5486] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:49 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:49.528762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5fe60-457f-4c72-a8aa-e4155235c4d2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:49.542431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5fe60-457f-4c72-a8aa-e4155235c4d2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:49.608310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015afb6a-1bef-4465-addd-1efc906aa224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:49.608954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015afb6a-1bef-4465-addd-1efc906aa224 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.609253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015afb6a-1bef-4465-addd-1efc906aa224 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.609533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015afb6a-1bef-4465-addd-1efc906aa224 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.610177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015afb6a-1bef-4465-addd-1efc906aa224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:49.612738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015afb6a-1bef-4465-addd-1efc906aa224 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:49.628499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:49.635579 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2708/5487] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:49 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:49.640994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a6a10-f0fa-471d-9e3a-5ce98ff0d70b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:49.652290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a6a10-f0fa-471d-9e3a-5ce98ff0d70b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:49.657823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3834f82-0684-4dd2-aa1c-8f7b77641ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:49.658502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3834f82-0684-4dd2-aa1c-8f7b77641ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.658792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3834f82-0684-4dd2-aa1c-8f7b77641ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.659610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3834f82-0684-4dd2-aa1c-8f7b77641ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.660247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3834f82-0684-4dd2-aa1c-8f7b77641ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:49.662444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3834f82-0684-4dd2-aa1c-8f7b77641ed6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:49.673835 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2780/5488] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:49.678265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015afb6a-1bef-4465-addd-1efc906aa224 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:49.688620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015afb6a-1bef-4465-addd-1efc906aa224 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:49.693401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603f3d8-6381-42b2-ba2a-97c7acbea554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:49.694006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603f3d8-6381-42b2-ba2a-97c7acbea554 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.694289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603f3d8-6381-42b2-ba2a-97c7acbea554 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.694571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603f3d8-6381-42b2-ba2a-97c7acbea554 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.695211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603f3d8-6381-42b2-ba2a-97c7acbea554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:49.697217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603f3d8-6381-42b2-ba2a-97c7acbea554 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:49.707653 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2709/5489] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:49.711991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3834f82-0684-4dd2-aa1c-8f7b77641ed6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:49.722861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3834f82-0684-4dd2-aa1c-8f7b77641ed6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:49.729247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d8d20-34be-4d8d-9fef-c29882f0f11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:49.729998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d8d20-34be-4d8d-9fef-c29882f0f11d admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.730356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d8d20-34be-4d8d-9fef-c29882f0f11d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.730730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d8d20-34be-4d8d-9fef-c29882f0f11d admin admin] PATH_INFO: `/v3/projects/5403a51b516c480d87767e43797e7de9/users/641ff629e43e434a897ef30ab403ea55/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.748154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d8d20-34be-4d8d-9fef-c29882f0f11d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5403a51b516c480d87767e43797e7de9, user_id=641ff629e43e434a897ef30ab403ea55, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:49.751069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d8d20-34be-4d8d-9fef-c29882f0f11d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:49.765238 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2781/5490] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:49 2022] PUT /identity/v3/projects/5403a51b516c480d87767e43797e7de9/users/641ff629e43e434a897ef30ab403ea55/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:49.776130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232933d9-c9ab-43ac-9508-07498133d1ce None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.776434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232933d9-c9ab-43ac-9508-07498133d1ce None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.776819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232933d9-c9ab-43ac-9508-07498133d1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.808327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-232933d9-c9ab-43ac-9508-07498133d1ce None None] MFA Rules not processed for user `641ff629e43e434a897ef30ab403ea55`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:49.809651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-232933d9-c9ab-43ac-9508-07498133d1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:49.879686 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2710/5491] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:49.884650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d8d20-34be-4d8d-9fef-c29882f0f11d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:49.948363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d8d20-34be-4d8d-9fef-c29882f0f11d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:49.952691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebc4ae-8ca0-4947-9c9e-c51c001c1714 tempest-IdentityV3ProjectsTest-649355362 tempest-IdentityV3ProjectsTest-649355362-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c084c28bbe3f495b96f5b683e57e5d24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bc39ba816044eea2cc6333a0804850', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:49.953369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebc4ae-8ca0-4947-9c9e-c51c001c1714 tempest-IdentityV3ProjectsTest-649355362 tempest-IdentityV3ProjectsTest-649355362-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:49.953670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebc4ae-8ca0-4947-9c9e-c51c001c1714 tempest-IdentityV3ProjectsTest-649355362 tempest-IdentityV3ProjectsTest-649355362-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:49.953934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebc4ae-8ca0-4947-9c9e-c51c001c1714 tempest-IdentityV3ProjectsTest-649355362 tempest-IdentityV3ProjectsTest-649355362-project] PATH_INFO: `/v3/users/c084c28bbe3f495b96f5b683e57e5d24/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:49.955137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebc4ae-8ca0-4947-9c9e-c51c001c1714 tempest-IdentityV3ProjectsTest-649355362 tempest-IdentityV3ProjectsTest-649355362-project] RBAC: Authorizing `identity:list_user_projects(user_id=c084c28bbe3f495b96f5b683e57e5d24)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:49.957511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebc4ae-8ca0-4947-9c9e-c51c001c1714 tempest-IdentityV3ProjectsTest-649355362 tempest-IdentityV3ProjectsTest-649355362-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:50.015416 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2782/5492] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Sep 3 21:08:49 2022] GET /identity/v3/users/c084c28bbe3f495b96f5b683e57e5d24/projects => generated 515 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:08:50.017215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c803e80-b373-4536-a43d-fd1e1047e061 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:50.017511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c803e80-b373-4536-a43d-fd1e1047e061 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:50.017792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c803e80-b373-4536-a43d-fd1e1047e061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:50.054697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2c803e80-b373-4536-a43d-fd1e1047e061 None None] MFA Rules not processed for user `c084c28bbe3f495b96f5b683e57e5d24`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:50.055825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2c803e80-b373-4536-a43d-fd1e1047e061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:50.063493 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2711/5493] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:50.068796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5968e3-87ea-4c06-a9cc-49ee1ff87693 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:50.069138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5968e3-87ea-4c06-a9cc-49ee1ff87693 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:50.069425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5968e3-87ea-4c06-a9cc-49ee1ff87693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:50.091135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0f5968e3-87ea-4c06-a9cc-49ee1ff87693 None None] MFA Rules not processed for user `c084c28bbe3f495b96f5b683e57e5d24`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:50.092371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0f5968e3-87ea-4c06-a9cc-49ee1ff87693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:50.102683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:50.146867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.models.token_model [None req-0f5968e3-87ea-4c06-a9cc-49ee1ff87693 None None] User c084c28bbe3f495b96f5b683e57e5d24 has no access to project 5403a51b516c480d87767e43797e7de9 {{(pid=99210) _validate_project_scope /opt/stack/keystone/keystone/models/token_model.py:539}} Sep 03 21:08:50.150651 ncdv-master-1 devstack@keystone.service[99210]: WARNING keystone.server.flask.application [None req-0f5968e3-87ea-4c06-a9cc-49ee1ff87693 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.16: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 03 21:08:50.152041 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2783/5494] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:50 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 84 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 03 21:08:50.157278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c803e80-b373-4536-a43d-fd1e1047e061 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:50.224724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c803e80-b373-4536-a43d-fd1e1047e061 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:50.231431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:50.232339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:50.232628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:50.232967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] PATH_INFO: `/v3/users/c084c28bbe3f495b96f5b683e57e5d24` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:50.234964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c084c28bbe3f495b96f5b683e57e5d24)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:50.237170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:50.298270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c084c28bbe3f495b96f5b683e57e5d24'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:50.318874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c084c28bbe3f495b96f5b683e57e5d24'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:50.319893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c084c28bbe3f495b96f5b683e57e5d24'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:50.322549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:50.331584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c084c28bbe3f495b96f5b683e57e5d24'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:50.340750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c084c28bbe3f495b96f5b683e57e5d24'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:50.353021 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2712/5495] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:50 2022] DELETE /identity/v3/users/c084c28bbe3f495b96f5b683e57e5d24 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:50.356467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f75526-d4fd-4108-bc6c-cd9b9e49ffd7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:50.356994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f75526-d4fd-4108-bc6c-cd9b9e49ffd7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:50.358340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f75526-d4fd-4108-bc6c-cd9b9e49ffd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:50.380937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c2f75526-d4fd-4108-bc6c-cd9b9e49ffd7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:50.382192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c2f75526-d4fd-4108-bc6c-cd9b9e49ffd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:50.484914 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2784/5496] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:50.498325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:50.501164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:50.576304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93990b0f-d0fe-41ac-beed-24b71f20502d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:50.580609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f3be8-bdb2-418a-947e-90d0f3a9da4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:50.581488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f3be8-bdb2-418a-947e-90d0f3a9da4a service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:50.581785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f3be8-bdb2-418a-947e-90d0f3a9da4a service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:50.582067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f3be8-bdb2-418a-947e-90d0f3a9da4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:50.594822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f3be8-bdb2-418a-947e-90d0f3a9da4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:50.596370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f3be8-bdb2-418a-947e-90d0f3a9da4a service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:50.608626 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2713/5497] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:08:50.994007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f75526-d4fd-4108-bc6c-cd9b9e49ffd7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:50.995962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c2f75526-d4fd-4108-bc6c-cd9b9e49ffd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:51.023623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f75526-d4fd-4108-bc6c-cd9b9e49ffd7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:51.032411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:51.033358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:51.033650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:51.034455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] PATH_INFO: `/v3/projects/32bc39ba816044eea2cc6333a0804850` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:51.035634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32bc39ba816044eea2cc6333a0804850)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:51.037913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:51.137629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32bc39ba816044eea2cc6333a0804850', 'request_id': 'req-9c36d365-b812-47cd-b8f0-0bacda12b044'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:51.149400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32bc39ba816044eea2cc6333a0804850', 'request_id': 'req-9c36d365-b812-47cd-b8f0-0bacda12b044'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:51.161746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32bc39ba816044eea2cc6333a0804850', 'request_id': 'req-9c36d365-b812-47cd-b8f0-0bacda12b044'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:51.177310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] The token cache is being invalidate because project 32bc39ba816044eea2cc6333a0804850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:51.177750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:51.178679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:51.180971 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2785/5498] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:50 2022] DELETE /identity/v3/projects/32bc39ba816044eea2cc6333a0804850 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:51.185008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f3be8-bdb2-418a-947e-90d0f3a9da4a service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:51.201900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f3be8-bdb2-418a-947e-90d0f3a9da4a service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:51.279926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76959a-2b75-4045-8515-6c2f428c06ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:51.280713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76959a-2b75-4045-8515-6c2f428c06ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:51.281182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76959a-2b75-4045-8515-6c2f428c06ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:51.281569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76959a-2b75-4045-8515-6c2f428c06ae admin admin] PATH_INFO: `/v3/users/641ff629e43e434a897ef30ab403ea55` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:51.282892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76959a-2b75-4045-8515-6c2f428c06ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=641ff629e43e434a897ef30ab403ea55)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:51.285696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76959a-2b75-4045-8515-6c2f428c06ae admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:51.354108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e76959a-2b75-4045-8515-6c2f428c06ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '641ff629e43e434a897ef30ab403ea55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:51.368405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e76959a-2b75-4045-8515-6c2f428c06ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '641ff629e43e434a897ef30ab403ea55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:51.369400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e76959a-2b75-4045-8515-6c2f428c06ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '641ff629e43e434a897ef30ab403ea55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:51.377365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e76959a-2b75-4045-8515-6c2f428c06ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '641ff629e43e434a897ef30ab403ea55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:51.387116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e76959a-2b75-4045-8515-6c2f428c06ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '641ff629e43e434a897ef30ab403ea55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:51.408744 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2714/5499] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:51 2022] DELETE /identity/v3/users/641ff629e43e434a897ef30ab403ea55 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:52.046538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:52.048104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:52.066538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36d365-b812-47cd-b8f0-0bacda12b044 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:52.162115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:52.176550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:52.178475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:52.178475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:52.178475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7 admin admin] PATH_INFO: `/v3/projects/5403a51b516c480d87767e43797e7de9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:52.179924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5403a51b516c480d87767e43797e7de9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:52.182397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:52.311467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5403a51b516c480d87767e43797e7de9', 'request_id': 'req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:52.326871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5403a51b516c480d87767e43797e7de9', 'request_id': 'req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:52.337536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5403a51b516c480d87767e43797e7de9', 'request_id': 'req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:52.364690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7 admin admin] The token cache is being invalidate because project 5403a51b516c480d87767e43797e7de9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:52.365090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:52.366408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbfa13e4-6a91-4ef2-9398-7e49be662bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:52.376939 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2786/5500] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:52 2022] DELETE /identity/v3/projects/5403a51b516c480d87767e43797e7de9 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:52.377216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd694-25de-4f1a-a067-bf7174464621 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:52.377896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd694-25de-4f1a-a067-bf7174464621 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:52.377896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd694-25de-4f1a-a067-bf7174464621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:52.400685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:52.426981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7cffd694-25de-4f1a-a067-bf7174464621 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:52.431129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7cffd694-25de-4f1a-a067-bf7174464621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:52.606446 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2715/5501] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:52.716580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cbbb2-3915-461f-be51-e3afb788c000 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:52.717021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cbbb2-3915-461f-be51-e3afb788c000 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:52.717167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cbbb2-3915-461f-be51-e3afb788c000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:52.752015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-861cbbb2-3915-461f-be51-e3afb788c000 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:52.754952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-861cbbb2-3915-461f-be51-e3afb788c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:52.771519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd694-25de-4f1a-a067-bf7174464621 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:52.777321 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2787/5502] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:52.783335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd694-25de-4f1a-a067-bf7174464621 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:52.795129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7cd14-b8f5-4777-85f0-785b8c29e518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:52.796509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7cd14-b8f5-4777-85f0-785b8c29e518 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:52.797008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7cd14-b8f5-4777-85f0-785b8c29e518 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:52.797456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7cd14-b8f5-4777-85f0-785b8c29e518 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:52.798287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7cd14-b8f5-4777-85f0-785b8c29e518 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:52.798773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7cd14-b8f5-4777-85f0-785b8c29e518 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:52.805271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7cd14-b8f5-4777-85f0-785b8c29e518 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:52.837667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cbbb2-3915-461f-be51-e3afb788c000 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:52.838465 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2716/5503] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:52.851735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cbbb2-3915-461f-be51-e3afb788c000 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:52.863727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8c5ed-ea83-4f62-be52-3b471226214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:52.864448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8c5ed-ea83-4f62-be52-3b471226214b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:52.864811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8c5ed-ea83-4f62-be52-3b471226214b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:52.865185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8c5ed-ea83-4f62-be52-3b471226214b admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:52.865912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8c5ed-ea83-4f62-be52-3b471226214b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:52.867949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8c5ed-ea83-4f62-be52-3b471226214b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:52.953524 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2788/5504] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:52 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:52.954779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7cd14-b8f5-4777-85f0-785b8c29e518 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:52.977067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7cd14-b8f5-4777-85f0-785b8c29e518 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:53.087893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15da20-72f8-4fd5-b742-bf2ee5421f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:53.088842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15da20-72f8-4fd5-b742-bf2ee5421f09 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:53.089326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15da20-72f8-4fd5-b742-bf2ee5421f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:53.089771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15da20-72f8-4fd5-b742-bf2ee5421f09 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:53.090576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15da20-72f8-4fd5-b742-bf2ee5421f09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:53.095022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15da20-72f8-4fd5-b742-bf2ee5421f09 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:53.143197 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2717/5505] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:52 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:53.147194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8c5ed-ea83-4f62-be52-3b471226214b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:53.159668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8c5ed-ea83-4f62-be52-3b471226214b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:53.171903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf00883-20fd-4c1d-ae58-5fc56587d303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:53.172927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf00883-20fd-4c1d-ae58-5fc56587d303 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:53.173442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf00883-20fd-4c1d-ae58-5fc56587d303 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:53.174011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf00883-20fd-4c1d-ae58-5fc56587d303 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:53.175944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf00883-20fd-4c1d-ae58-5fc56587d303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:53.181212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf00883-20fd-4c1d-ae58-5fc56587d303 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:53.210534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15da20-72f8-4fd5-b742-bf2ee5421f09 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:53.217326 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2789/5506] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:53.226415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15da20-72f8-4fd5-b742-bf2ee5421f09 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:53.244158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee0638-7ad7-49a4-97c5-587d376d9fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:53.245084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee0638-7ad7-49a4-97c5-587d376d9fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:53.245497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee0638-7ad7-49a4-97c5-587d376d9fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:53.245875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee0638-7ad7-49a4-97c5-587d376d9fe8 admin admin] PATH_INFO: `/v3/projects/afdf10ba5dd64b26b0dd50d68432e407/users/089e6311c8b349ff9106c76288ff9f06/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:53.302438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee0638-7ad7-49a4-97c5-587d376d9fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afdf10ba5dd64b26b0dd50d68432e407, user_id=089e6311c8b349ff9106c76288ff9f06, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:53.305301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee0638-7ad7-49a4-97c5-587d376d9fe8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:53.348580 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2718/5507] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:53 2022] PUT /identity/v3/projects/afdf10ba5dd64b26b0dd50d68432e407/users/089e6311c8b349ff9106c76288ff9f06/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:53.352861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf00883-20fd-4c1d-ae58-5fc56587d303 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:53.380617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf00883-20fd-4c1d-ae58-5fc56587d303 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:53.503081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a59ed-a1f2-4dc1-b2d9-543f327fcb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:53.503711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a59ed-a1f2-4dc1-b2d9-543f327fcb1f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:53.504004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a59ed-a1f2-4dc1-b2d9-543f327fcb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:53.504281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a59ed-a1f2-4dc1-b2d9-543f327fcb1f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:53.504976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a59ed-a1f2-4dc1-b2d9-543f327fcb1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:53.506913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a59ed-a1f2-4dc1-b2d9-543f327fcb1f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:53.527986 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2790/5508] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:53.529290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee0638-7ad7-49a4-97c5-587d376d9fe8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:53.549169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee0638-7ad7-49a4-97c5-587d376d9fe8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:53.560816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a11c6-7759-410a-8c3b-47158c43c203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:53.561534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a11c6-7759-410a-8c3b-47158c43c203 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:53.561821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a11c6-7759-410a-8c3b-47158c43c203 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:53.562110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a11c6-7759-410a-8c3b-47158c43c203 admin admin] PATH_INFO: `/v3/projects/afdf10ba5dd64b26b0dd50d68432e407/users/089e6311c8b349ff9106c76288ff9f06/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:53.567650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a11c6-7759-410a-8c3b-47158c43c203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afdf10ba5dd64b26b0dd50d68432e407, user_id=089e6311c8b349ff9106c76288ff9f06, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:53.570538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a11c6-7759-410a-8c3b-47158c43c203 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:53.597780 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2719/5509] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:53 2022] PUT /identity/v3/projects/afdf10ba5dd64b26b0dd50d68432e407/users/089e6311c8b349ff9106c76288ff9f06/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:53.611415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cbde8-be1b-41c1-a9f7-3ea6053b469e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:53.611728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cbde8-be1b-41c1-a9f7-3ea6053b469e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:53.612032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cbde8-be1b-41c1-a9f7-3ea6053b469e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:53.673607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6a9cbde8-be1b-41c1-a9f7-3ea6053b469e None None] MFA Rules not processed for user `089e6311c8b349ff9106c76288ff9f06`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:53.674937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6a9cbde8-be1b-41c1-a9f7-3ea6053b469e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:53.726904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:53.803623 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2791/5510] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:08:53.808733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a11c6-7759-410a-8c3b-47158c43c203 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:53.886695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:53.929640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a11c6-7759-410a-8c3b-47158c43c203 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:53.936228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25da92f-8a6b-4798-ae8c-8911683b2149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:53.937029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25da92f-8a6b-4798-ae8c-8911683b2149 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:53.937546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25da92f-8a6b-4798-ae8c-8911683b2149 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:53.937824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25da92f-8a6b-4798-ae8c-8911683b2149 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:53.938531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25da92f-8a6b-4798-ae8c-8911683b2149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:53.941734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25da92f-8a6b-4798-ae8c-8911683b2149 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:53.998137 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2720/5511] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:53 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:54.001531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cbde8-be1b-41c1-a9f7-3ea6053b469e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:54.014366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cbde8-be1b-41c1-a9f7-3ea6053b469e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:54.083685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede015ed-7842-416d-86ef-433b51b118be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:54.084327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede015ed-7842-416d-86ef-433b51b118be admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:54.084619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede015ed-7842-416d-86ef-433b51b118be admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:54.084921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede015ed-7842-416d-86ef-433b51b118be admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:54.085574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede015ed-7842-416d-86ef-433b51b118be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:54.087556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede015ed-7842-416d-86ef-433b51b118be admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:54.139239 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2792/5512] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:54 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:08:54.143893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25da92f-8a6b-4798-ae8c-8911683b2149 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:54.170292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25da92f-8a6b-4798-ae8c-8911683b2149 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:54.181834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ec08e-d5da-4fa6-9fdd-58176227b22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:54.182795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ec08e-d5da-4fa6-9fdd-58176227b22a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:54.183252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ec08e-d5da-4fa6-9fdd-58176227b22a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:54.183749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ec08e-d5da-4fa6-9fdd-58176227b22a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:54.184574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ec08e-d5da-4fa6-9fdd-58176227b22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:54.187519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ec08e-d5da-4fa6-9fdd-58176227b22a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:54.214124 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2721/5513] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:54.218351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede015ed-7842-416d-86ef-433b51b118be admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:54.232267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede015ed-7842-416d-86ef-433b51b118be admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:54.245903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1f8a3-c42c-4219-ae54-4e466a02853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:54.246553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1f8a3-c42c-4219-ae54-4e466a02853d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:54.246845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1f8a3-c42c-4219-ae54-4e466a02853d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:54.253357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1f8a3-c42c-4219-ae54-4e466a02853d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:54.255268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1f8a3-c42c-4219-ae54-4e466a02853d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:54.263116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1f8a3-c42c-4219-ae54-4e466a02853d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:54.298420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ec08e-d5da-4fa6-9fdd-58176227b22a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:54.301705 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2793/5514] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:54.320044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ec08e-d5da-4fa6-9fdd-58176227b22a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:54.327869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3a6a9-c28a-4747-8885-d1e6628162bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:54.328832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3a6a9-c28a-4747-8885-d1e6628162bc admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:54.329362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3a6a9-c28a-4747-8885-d1e6628162bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:54.329879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3a6a9-c28a-4747-8885-d1e6628162bc admin admin] PATH_INFO: `/v3/projects/cb0c68df8b504d97b696cea69b6cc0ca/users/8a16dca31f4b4053bbdc94fb506d43fe/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:54.352776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3a6a9-c28a-4747-8885-d1e6628162bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0c68df8b504d97b696cea69b6cc0ca, user_id=8a16dca31f4b4053bbdc94fb506d43fe, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:54.357311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3a6a9-c28a-4747-8885-d1e6628162bc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:54.394984 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2722/5515] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:54 2022] PUT /identity/v3/projects/cb0c68df8b504d97b696cea69b6cc0ca/users/8a16dca31f4b4053bbdc94fb506d43fe/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:54.401997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a24146-4e00-4e77-9a12-e9b09a8d3e07 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:54.402300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a24146-4e00-4e77-9a12-e9b09a8d3e07 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:54.402673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a24146-4e00-4e77-9a12-e9b09a8d3e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:54.468204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-27a24146-4e00-4e77-9a12-e9b09a8d3e07 None None] MFA Rules not processed for user `8a16dca31f4b4053bbdc94fb506d43fe`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:54.469971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-27a24146-4e00-4e77-9a12-e9b09a8d3e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:54.571530 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2794/5516] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:54.585044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3a6a9-c28a-4747-8885-d1e6628162bc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:54.588165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a5f3a6a9-c28a-4747-8885-d1e6628162bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:54.697839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3a6a9-c28a-4747-8885-d1e6628162bc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:54.702916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89161d79-abb1-4cd8-89c4-2a0c189b1912 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:54.703958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89161d79-abb1-4cd8-89c4-2a0c189b1912 service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:54.707549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89161d79-abb1-4cd8-89c4-2a0c189b1912 service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:54.708032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89161d79-abb1-4cd8-89c4-2a0c189b1912 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:54.782528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89161d79-abb1-4cd8-89c4-2a0c189b1912 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:54.784017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89161d79-abb1-4cd8-89c4-2a0c189b1912 service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:54.795511 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2723/5517] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:08:55.768666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a24146-4e00-4e77-9a12-e9b09a8d3e07 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:55.829859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:55.867381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a24146-4e00-4e77-9a12-e9b09a8d3e07 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:55.875813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:55.876544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:55.876834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:55.877162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] PATH_INFO: `/v3/users/089e6311c8b349ff9106c76288ff9f06` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:55.878372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=089e6311c8b349ff9106c76288ff9f06)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:55.880493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:55.966291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '089e6311c8b349ff9106c76288ff9f06'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:55.979972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '089e6311c8b349ff9106c76288ff9f06'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:55.988658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '089e6311c8b349ff9106c76288ff9f06'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:55.989304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '089e6311c8b349ff9106c76288ff9f06'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:55.990082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '089e6311c8b349ff9106c76288ff9f06'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:56.003239 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2795/5518] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:55 2022] DELETE /identity/v3/users/089e6311c8b349ff9106c76288ff9f06 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:56.007840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a69925-3c20-4b7b-99cb-9763e4f3cf13 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:56.009169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a69925-3c20-4b7b-99cb-9763e4f3cf13 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:56.009169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a69925-3c20-4b7b-99cb-9763e4f3cf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:56.030985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a3a69925-3c20-4b7b-99cb-9763e4f3cf13 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:56.032430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a3a69925-3c20-4b7b-99cb-9763e4f3cf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:56.066854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:56.109645 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2724/5519] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:56.121508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:56.123360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:56.210578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfe022-9190-4e35-837c-f91e210da123 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:56.219740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755d307-8eb9-475e-97cd-b80061c95118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:56.220366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755d307-8eb9-475e-97cd-b80061c95118 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:56.220659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755d307-8eb9-475e-97cd-b80061c95118 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:56.220990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755d307-8eb9-475e-97cd-b80061c95118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:56.231594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755d307-8eb9-475e-97cd-b80061c95118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:56.233191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755d307-8eb9-475e-97cd-b80061c95118 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:56.255312 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2796/5520] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:08:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:08:57.352629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a69925-3c20-4b7b-99cb-9763e4f3cf13 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:57.354166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a3a69925-3c20-4b7b-99cb-9763e4f3cf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:57.384657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a69925-3c20-4b7b-99cb-9763e4f3cf13 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:57.417928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:57.420289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:57.420289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:57.420289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] PATH_INFO: `/v3/projects/afdf10ba5dd64b26b0dd50d68432e407` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:57.422030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] RBAC: Authorizing `identity:delete_project(project_id=afdf10ba5dd64b26b0dd50d68432e407)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:57.426109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:57.584228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afdf10ba5dd64b26b0dd50d68432e407', 'request_id': 'req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:57.600305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afdf10ba5dd64b26b0dd50d68432e407', 'request_id': 'req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:57.608300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afdf10ba5dd64b26b0dd50d68432e407', 'request_id': 'req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:57.633717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] The token cache is being invalidate because project afdf10ba5dd64b26b0dd50d68432e407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:57.637716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:57.639509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:57.642599 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2725/5521] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:57 2022] DELETE /identity/v3/projects/afdf10ba5dd64b26b0dd50d68432e407 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:57.648748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755d307-8eb9-475e-97cd-b80061c95118 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:57.662876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755d307-8eb9-475e-97cd-b80061c95118 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:57.764092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:57.775308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcbfe6-c769-43ef-8ad1-ecca9426a018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:57.775951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcbfe6-c769-43ef-8ad1-ecca9426a018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:57.776244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcbfe6-c769-43ef-8ad1-ecca9426a018 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:57.776574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcbfe6-c769-43ef-8ad1-ecca9426a018 admin admin] PATH_INFO: `/v3/users/8a16dca31f4b4053bbdc94fb506d43fe` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:57.778036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcbfe6-c769-43ef-8ad1-ecca9426a018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a16dca31f4b4053bbdc94fb506d43fe)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:57.780196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcbfe6-c769-43ef-8ad1-ecca9426a018 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:57.867046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ffcbfe6-c769-43ef-8ad1-ecca9426a018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a16dca31f4b4053bbdc94fb506d43fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:57.892645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ffcbfe6-c769-43ef-8ad1-ecca9426a018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a16dca31f4b4053bbdc94fb506d43fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:57.910062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ffcbfe6-c769-43ef-8ad1-ecca9426a018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a16dca31f4b4053bbdc94fb506d43fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:57.911032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ffcbfe6-c769-43ef-8ad1-ecca9426a018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a16dca31f4b4053bbdc94fb506d43fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:57.911940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7ffcbfe6-c769-43ef-8ad1-ecca9426a018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a16dca31f4b4053bbdc94fb506d43fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:57.934384 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2797/5522] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:08:57 2022] DELETE /identity/v3/users/8a16dca31f4b4053bbdc94fb506d43fe => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:08:58.927038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:58.928720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:58.955352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eee5d-d654-4bca-bd7a-2f4ad092d72e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:59.071391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56265f6-3c19-447e-b8c4-1c4269ce17e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:59.073368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56265f6-3c19-447e-b8c4-1c4269ce17e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:59.073661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56265f6-3c19-447e-b8c4-1c4269ce17e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:59.073945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56265f6-3c19-447e-b8c4-1c4269ce17e1 admin admin] PATH_INFO: `/v3/projects/cb0c68df8b504d97b696cea69b6cc0ca` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:59.077222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56265f6-3c19-447e-b8c4-1c4269ce17e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb0c68df8b504d97b696cea69b6cc0ca)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:59.079406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56265f6-3c19-447e-b8c4-1c4269ce17e1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:59.174183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c56265f6-3c19-447e-b8c4-1c4269ce17e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb0c68df8b504d97b696cea69b6cc0ca', 'request_id': 'req-c56265f6-3c19-447e-b8c4-1c4269ce17e1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:59.191999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c56265f6-3c19-447e-b8c4-1c4269ce17e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb0c68df8b504d97b696cea69b6cc0ca', 'request_id': 'req-c56265f6-3c19-447e-b8c4-1c4269ce17e1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:59.216331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c56265f6-3c19-447e-b8c4-1c4269ce17e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb0c68df8b504d97b696cea69b6cc0ca', 'request_id': 'req-c56265f6-3c19-447e-b8c4-1c4269ce17e1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:59.219334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:59.246616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c56265f6-3c19-447e-b8c4-1c4269ce17e1 admin admin] The token cache is being invalidate because project cb0c68df8b504d97b696cea69b6cc0ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:08:59.247588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c56265f6-3c19-447e-b8c4-1c4269ce17e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:59.248428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c56265f6-3c19-447e-b8c4-1c4269ce17e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:08:59.256561 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2726/5523] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:08:58 2022] DELETE /identity/v3/projects/cb0c68df8b504d97b696cea69b6cc0ca => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:59.257849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38e986-facb-40d3-98ab-e518917e0018 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:59.258336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38e986-facb-40d3-98ab-e518917e0018 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:59.258436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38e986-facb-40d3-98ab-e518917e0018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:59.281781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bd38e986-facb-40d3-98ab-e518917e0018 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:59.283372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bd38e986-facb-40d3-98ab-e518917e0018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:59.360791 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2798/5524] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:59.370209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112c8ff-30f8-4021-948a-e7c96e8aa7d8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:59.370772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112c8ff-30f8-4021-948a-e7c96e8aa7d8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:59.371205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112c8ff-30f8-4021-948a-e7c96e8aa7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:59.399056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6112c8ff-30f8-4021-948a-e7c96e8aa7d8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:08:59.400228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6112c8ff-30f8-4021-948a-e7c96e8aa7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:08:59.413819 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2727/5525] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:08:59.414924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38e986-facb-40d3-98ab-e518917e0018 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:59.427593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38e986-facb-40d3-98ab-e518917e0018 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:59.433932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee694d9-9e3c-4cd6-8de8-4829409346f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:59.435359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee694d9-9e3c-4cd6-8de8-4829409346f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:59.435359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee694d9-9e3c-4cd6-8de8-4829409346f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:59.435359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee694d9-9e3c-4cd6-8de8-4829409346f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:59.437014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee694d9-9e3c-4cd6-8de8-4829409346f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:08:59.437541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee694d9-9e3c-4cd6-8de8-4829409346f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:59.439779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee694d9-9e3c-4cd6-8de8-4829409346f7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:59.456336 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2799/5526] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:08:59.461920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112c8ff-30f8-4021-948a-e7c96e8aa7d8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:59.471886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112c8ff-30f8-4021-948a-e7c96e8aa7d8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:59.479884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645c999-b43b-4831-b639-3379e808b40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:59.481158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645c999-b43b-4831-b639-3379e808b40e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:59.483206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645c999-b43b-4831-b639-3379e808b40e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:59.483709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645c999-b43b-4831-b639-3379e808b40e admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:59.484740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645c999-b43b-4831-b639-3379e808b40e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:59.488053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645c999-b43b-4831-b639-3379e808b40e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:59.555305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee694d9-9e3c-4cd6-8de8-4829409346f7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:59.562663 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2728/5527] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:08:59 2022] POST /identity/v3/projects => generated 377 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:59.570188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee694d9-9e3c-4cd6-8de8-4829409346f7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:59.649378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b640d-d5e4-4ee3-8490-274f5aaac80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:59.650394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b640d-d5e4-4ee3-8490-274f5aaac80f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:59.650902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b640d-d5e4-4ee3-8490-274f5aaac80f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:59.651413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b640d-d5e4-4ee3-8490-274f5aaac80f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:59.652319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b640d-d5e4-4ee3-8490-274f5aaac80f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:59.655477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b640d-d5e4-4ee3-8490-274f5aaac80f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:59.667413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:08:59.698755 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2800/5528] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:08:59 2022] POST /identity/v3/users => generated 295 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:08:59.702838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645c999-b43b-4831-b639-3379e808b40e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:59.716125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645c999-b43b-4831-b639-3379e808b40e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:59.721926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b9a6c-40fb-433b-bcee-50fc80e2cc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:59.723071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b9a6c-40fb-433b-bcee-50fc80e2cc15 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:59.723635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b9a6c-40fb-433b-bcee-50fc80e2cc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:59.724193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b9a6c-40fb-433b-bcee-50fc80e2cc15 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:59.725533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b9a6c-40fb-433b-bcee-50fc80e2cc15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:59.727732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b9a6c-40fb-433b-bcee-50fc80e2cc15 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:59.748567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b640d-d5e4-4ee3-8490-274f5aaac80f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:59.755517 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2729/5529] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:59.761490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b640d-d5e4-4ee3-8490-274f5aaac80f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:59.767875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e016d1-ffbd-474c-903b-5d08c3ab439a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:59.768928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e016d1-ffbd-474c-903b-5d08c3ab439a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:59.769646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e016d1-ffbd-474c-903b-5d08c3ab439a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:59.770131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e016d1-ffbd-474c-903b-5d08c3ab439a admin admin] PATH_INFO: `/v3/projects/372dee41ddb445bd8fb846ad5b1d45a2/users/cb481f008d1849aca2c0932d8d38bfde/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:59.792618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e016d1-ffbd-474c-903b-5d08c3ab439a admin admin] RBAC: Authorizing `identity:create_grant(project_id=372dee41ddb445bd8fb846ad5b1d45a2, user_id=cb481f008d1849aca2c0932d8d38bfde, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:59.795993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e016d1-ffbd-474c-903b-5d08c3ab439a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:59.822206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b9a6c-40fb-433b-bcee-50fc80e2cc15 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:59.823956 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2801/5530] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:59 2022] PUT /identity/v3/projects/372dee41ddb445bd8fb846ad5b1d45a2/users/cb481f008d1849aca2c0932d8d38bfde/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:08:59.838253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b9a6c-40fb-433b-bcee-50fc80e2cc15 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:59.956658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234c129-88a4-48b0-95e7-fb84f336ace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:59.957349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234c129-88a4-48b0-95e7-fb84f336ace5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:59.957608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234c129-88a4-48b0-95e7-fb84f336ace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:59.957895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234c129-88a4-48b0-95e7-fb84f336ace5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:59.958551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234c129-88a4-48b0-95e7-fb84f336ace5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:08:59.960309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234c129-88a4-48b0-95e7-fb84f336ace5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:08:59.973545 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2730/5531] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:08:59.977543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e016d1-ffbd-474c-903b-5d08c3ab439a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:08:59.988706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e016d1-ffbd-474c-903b-5d08c3ab439a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:08:59.995346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67012c81-7f33-4317-86c2-a7817ca6f31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:08:59.995346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67012c81-7f33-4317-86c2-a7817ca6f31f admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:08:59.995346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67012c81-7f33-4317-86c2-a7817ca6f31f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:08:59.995346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67012c81-7f33-4317-86c2-a7817ca6f31f admin admin] PATH_INFO: `/v3/projects/372dee41ddb445bd8fb846ad5b1d45a2/users/cb481f008d1849aca2c0932d8d38bfde/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:08:59.997371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67012c81-7f33-4317-86c2-a7817ca6f31f admin admin] RBAC: Authorizing `identity:create_grant(project_id=372dee41ddb445bd8fb846ad5b1d45a2, user_id=cb481f008d1849aca2c0932d8d38bfde, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:00.000225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67012c81-7f33-4317-86c2-a7817ca6f31f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:00.015085 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2802/5532] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:08:59 2022] PUT /identity/v3/projects/372dee41ddb445bd8fb846ad5b1d45a2/users/cb481f008d1849aca2c0932d8d38bfde/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:00.025891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b6500-dab6-4e15-8c3e-73ab55b91628 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:00.026283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b6500-dab6-4e15-8c3e-73ab55b91628 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:00.026605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b6500-dab6-4e15-8c3e-73ab55b91628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:00.063156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1f4b6500-dab6-4e15-8c3e-73ab55b91628 None None] MFA Rules not processed for user `cb481f008d1849aca2c0932d8d38bfde`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:00.064848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1f4b6500-dab6-4e15-8c3e-73ab55b91628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:00.173088 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2731/5533] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:00.177112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67012c81-7f33-4317-86c2-a7817ca6f31f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:00.260798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67012c81-7f33-4317-86c2-a7817ca6f31f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:00.266905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a3684-116b-4f1d-a8a9-6d0880623a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:00.269327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a3684-116b-4f1d-a8a9-6d0880623a02 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:00.269722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a3684-116b-4f1d-a8a9-6d0880623a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:00.270277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a3684-116b-4f1d-a8a9-6d0880623a02 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:00.271198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a3684-116b-4f1d-a8a9-6d0880623a02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:00.273604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a3684-116b-4f1d-a8a9-6d0880623a02 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:00.342718 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2803/5534] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:00 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:00.347140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b6500-dab6-4e15-8c3e-73ab55b91628 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:00.365629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b6500-dab6-4e15-8c3e-73ab55b91628 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:00.392176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:00.486380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7823f-1a21-4e7c-b9e4-dfc45d701953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:00.487346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7823f-1a21-4e7c-b9e4-dfc45d701953 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:00.487730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7823f-1a21-4e7c-b9e4-dfc45d701953 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:00.490103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7823f-1a21-4e7c-b9e4-dfc45d701953 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:00.491131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7823f-1a21-4e7c-b9e4-dfc45d701953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:00.494157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7823f-1a21-4e7c-b9e4-dfc45d701953 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:00.562406 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2732/5535] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:00 2022] POST /identity/v3/users => generated 291 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:00.567001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a3684-116b-4f1d-a8a9-6d0880623a02 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:00.587434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a3684-116b-4f1d-a8a9-6d0880623a02 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:00.597523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf238e8-f72c-4aec-a50a-3490f0c74f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:00.598154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf238e8-f72c-4aec-a50a-3490f0c74f71 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:00.598441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf238e8-f72c-4aec-a50a-3490f0c74f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:00.598779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf238e8-f72c-4aec-a50a-3490f0c74f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:00.599367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf238e8-f72c-4aec-a50a-3490f0c74f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:00.601279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf238e8-f72c-4aec-a50a-3490f0c74f71 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:00.625968 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2804/5536] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:00.630977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7823f-1a21-4e7c-b9e4-dfc45d701953 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:00.644112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7823f-1a21-4e7c-b9e4-dfc45d701953 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:00.663018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf681b30-c18c-4d60-935b-3911afef3629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:00.663664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf681b30-c18c-4d60-935b-3911afef3629 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:00.663996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf681b30-c18c-4d60-935b-3911afef3629 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:00.664269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf681b30-c18c-4d60-935b-3911afef3629 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:00.665122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf681b30-c18c-4d60-935b-3911afef3629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:00.668075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf681b30-c18c-4d60-935b-3911afef3629 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:00.691226 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2733/5537] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:00.696535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf238e8-f72c-4aec-a50a-3490f0c74f71 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:00.709053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf238e8-f72c-4aec-a50a-3490f0c74f71 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:00.718344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ec80b-e5f3-4d9e-afbd-25d77b83f448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:00.719578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ec80b-e5f3-4d9e-afbd-25d77b83f448 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:00.720190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ec80b-e5f3-4d9e-afbd-25d77b83f448 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:00.720621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ec80b-e5f3-4d9e-afbd-25d77b83f448 admin admin] PATH_INFO: `/v3/projects/fa5232054dd84b8b8810e5cb5063ba3a/users/9ff27d82a6da499797d77772ed4fdaa0/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:00.741188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ec80b-e5f3-4d9e-afbd-25d77b83f448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa5232054dd84b8b8810e5cb5063ba3a, user_id=9ff27d82a6da499797d77772ed4fdaa0, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:00.744082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ec80b-e5f3-4d9e-afbd-25d77b83f448 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:00.762056 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2805/5538] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:00 2022] PUT /identity/v3/projects/fa5232054dd84b8b8810e5cb5063ba3a/users/9ff27d82a6da499797d77772ed4fdaa0/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:00.774043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d80ae-2c0b-4dd5-b2c2-bfe66a4a3d5f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:00.774467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d80ae-2c0b-4dd5-b2c2-bfe66a4a3d5f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:00.774873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d80ae-2c0b-4dd5-b2c2-bfe66a4a3d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:00.806580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-372d80ae-2c0b-4dd5-b2c2-bfe66a4a3d5f None None] MFA Rules not processed for user `9ff27d82a6da499797d77772ed4fdaa0`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:00.807797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-372d80ae-2c0b-4dd5-b2c2-bfe66a4a3d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:00.904770 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2734/5539] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:00.916260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ec80b-e5f3-4d9e-afbd-25d77b83f448 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:00.918090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-061ec80b-e5f3-4d9e-afbd-25d77b83f448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:00.992591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ec80b-e5f3-4d9e-afbd-25d77b83f448 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:00.997033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b26d2-824c-4c52-b715-d54f4e0565e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:00.997666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b26d2-824c-4c52-b715-d54f4e0565e0 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:00.997953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b26d2-824c-4c52-b715-d54f4e0565e0 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:00.998231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b26d2-824c-4c52-b715-d54f4e0565e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:01.020832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:01.075227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b26d2-824c-4c52-b715-d54f4e0565e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:01.076772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b26d2-824c-4c52-b715-d54f4e0565e0 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:01.086726 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2806/5540] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:01.555699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d80ae-2c0b-4dd5-b2c2-bfe66a4a3d5f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:01.557286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-372d80ae-2c0b-4dd5-b2c2-bfe66a4a3d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:01.652585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d80ae-2c0b-4dd5-b2c2-bfe66a4a3d5f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:01.657572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:01.658447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:01.659570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:01.659866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] PATH_INFO: `/v3/users/d0c0a1c831a64e659dbda031c99e6eef` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:01.661043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0c0a1c831a64e659dbda031c99e6eef)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:01.663137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:01.705136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:01.720746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0c0a1c831a64e659dbda031c99e6eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:01.739502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0c0a1c831a64e659dbda031c99e6eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:01.741293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0c0a1c831a64e659dbda031c99e6eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:01.751301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0c0a1c831a64e659dbda031c99e6eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:01.760621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0c0a1c831a64e659dbda031c99e6eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:01.767735 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2735/5541] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:01 2022] DELETE /identity/v3/users/d0c0a1c831a64e659dbda031c99e6eef => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:01.773405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645863b-c7e6-4be1-9f9a-96d1d7de8301 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:01.773908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645863b-c7e6-4be1-9f9a-96d1d7de8301 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:01.774005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645863b-c7e6-4be1-9f9a-96d1d7de8301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:01.795991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2645863b-c7e6-4be1-9f9a-96d1d7de8301 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:01.797254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2645863b-c7e6-4be1-9f9a-96d1d7de8301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:01.856760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:01.858634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:01.867764 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2807/5542] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:01.874171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8de098-d656-49aa-a3dd-acfb2ceaba92 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:01.880331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645863b-c7e6-4be1-9f9a-96d1d7de8301 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:01.880531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:01.882127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2645863b-c7e6-4be1-9f9a-96d1d7de8301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:01.882703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:01.883005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:01.883505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] PATH_INFO: `/v3/users/cb481f008d1849aca2c0932d8d38bfde` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:01.884999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb481f008d1849aca2c0932d8d38bfde)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:01.887114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:01.959807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645863b-c7e6-4be1-9f9a-96d1d7de8301 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:01.960059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb481f008d1849aca2c0932d8d38bfde'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:01.965214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65018b-fe31-4d7f-b5b8-cdc4cbd6a0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:01.966067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65018b-fe31-4d7f-b5b8-cdc4cbd6a0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:01.966366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65018b-fe31-4d7f-b5b8-cdc4cbd6a0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:01.966742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65018b-fe31-4d7f-b5b8-cdc4cbd6a0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:01.979231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65018b-fe31-4d7f-b5b8-cdc4cbd6a0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:01.980876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65018b-fe31-4d7f-b5b8-cdc4cbd6a0d1 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:01.992843 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2808/5543] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:09:02.001202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb481f008d1849aca2c0932d8d38bfde'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:02.001697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb481f008d1849aca2c0932d8d38bfde'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:02.025345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb481f008d1849aca2c0932d8d38bfde'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:02.053690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb481f008d1849aca2c0932d8d38bfde'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:02.066039 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2736/5544] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:01 2022] DELETE /identity/v3/users/cb481f008d1849aca2c0932d8d38bfde => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:02.068509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee550d3-4690-4ef5-8e3b-278663668ecb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:02.068933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee550d3-4690-4ef5-8e3b-278663668ecb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:02.069329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee550d3-4690-4ef5-8e3b-278663668ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:02.092607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1ee550d3-4690-4ef5-8e3b-278663668ecb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:02.094183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1ee550d3-4690-4ef5-8e3b-278663668ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:02.165213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:02.178169 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2809/5545] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:02.196677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:02.198144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:02.272994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323fb6f-19d1-46c5-90b2-feb6f2387d10 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:02.277556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b522cd5-8443-45f3-9fb2-349c8e4e1c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:02.278855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b522cd5-8443-45f3-9fb2-349c8e4e1c3d service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:02.278855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b522cd5-8443-45f3-9fb2-349c8e4e1c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:02.278855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b522cd5-8443-45f3-9fb2-349c8e4e1c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:02.295611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee550d3-4690-4ef5-8e3b-278663668ecb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:02.297484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1ee550d3-4690-4ef5-8e3b-278663668ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:02.299454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b522cd5-8443-45f3-9fb2-349c8e4e1c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:02.301529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b522cd5-8443-45f3-9fb2-349c8e4e1c3d service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:02.320245 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2737/5546] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:02.323944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee550d3-4690-4ef5-8e3b-278663668ecb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:02.334999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e04454-ec37-4887-bb05-f67b81b6783f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:02.335845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e04454-ec37-4887-bb05-f67b81b6783f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:02.336204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e04454-ec37-4887-bb05-f67b81b6783f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:02.336814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e04454-ec37-4887-bb05-f67b81b6783f admin admin] PATH_INFO: `/v3/projects/74845811a8b84f909b5035cf6972f936` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:02.338418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e04454-ec37-4887-bb05-f67b81b6783f admin admin] RBAC: Authorizing `identity:delete_project(project_id=74845811a8b84f909b5035cf6972f936)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:02.341805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e04454-ec37-4887-bb05-f67b81b6783f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:02.445771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3e04454-ec37-4887-bb05-f67b81b6783f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74845811a8b84f909b5035cf6972f936', 'request_id': 'req-c3e04454-ec37-4887-bb05-f67b81b6783f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:02.458264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3e04454-ec37-4887-bb05-f67b81b6783f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74845811a8b84f909b5035cf6972f936', 'request_id': 'req-c3e04454-ec37-4887-bb05-f67b81b6783f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:02.466972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3e04454-ec37-4887-bb05-f67b81b6783f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74845811a8b84f909b5035cf6972f936', 'request_id': 'req-c3e04454-ec37-4887-bb05-f67b81b6783f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:02.482936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3e04454-ec37-4887-bb05-f67b81b6783f admin admin] The token cache is being invalidate because project 74845811a8b84f909b5035cf6972f936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:02.483271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3e04454-ec37-4887-bb05-f67b81b6783f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:02.484018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3e04454-ec37-4887-bb05-f67b81b6783f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:02.489998 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2810/5547] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:02 2022] DELETE /identity/v3/projects/74845811a8b84f909b5035cf6972f936 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:02.491846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03123d-81c2-4c59-8bd8-feac5f1c757d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:02.492980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03123d-81c2-4c59-8bd8-feac5f1c757d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:02.496246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03123d-81c2-4c59-8bd8-feac5f1c757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:02.534533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6e03123d-81c2-4c59-8bd8-feac5f1c757d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:02.535924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6e03123d-81c2-4c59-8bd8-feac5f1c757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:02.593574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:02.684947 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2738/5548] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:02.690626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f2596-12c6-41c6-97c6-95ff95320d40 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:02.690626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f2596-12c6-41c6-97c6-95ff95320d40 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:02.690626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f2596-12c6-41c6-97c6-95ff95320d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:02.727422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2b6f2596-12c6-41c6-97c6-95ff95320d40 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:02.728671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b6f2596-12c6-41c6-97c6-95ff95320d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:02.737690 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2811/5549] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:02.743595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03123d-81c2-4c59-8bd8-feac5f1c757d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:02.755009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03123d-81c2-4c59-8bd8-feac5f1c757d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:02.760346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef59733-a7bf-43e3-859e-e02085969716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:02.761001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef59733-a7bf-43e3-859e-e02085969716 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:02.761292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef59733-a7bf-43e3-859e-e02085969716 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:02.761572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef59733-a7bf-43e3-859e-e02085969716 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:02.762228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef59733-a7bf-43e3-859e-e02085969716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:02.762540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef59733-a7bf-43e3-859e-e02085969716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:02.765341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef59733-a7bf-43e3-859e-e02085969716 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:02.779208 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2739/5550] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:02.784337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f2596-12c6-41c6-97c6-95ff95320d40 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:02.799019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f2596-12c6-41c6-97c6-95ff95320d40 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:02.805376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e29f46-66dd-412f-b12d-bcd719ff00eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:02.806144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e29f46-66dd-412f-b12d-bcd719ff00eb admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:02.806395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e29f46-66dd-412f-b12d-bcd719ff00eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:02.806839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e29f46-66dd-412f-b12d-bcd719ff00eb admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:02.809588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e29f46-66dd-412f-b12d-bcd719ff00eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:02.821319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e29f46-66dd-412f-b12d-bcd719ff00eb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:02.824004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef59733-a7bf-43e3-859e-e02085969716 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:02.826480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1ef59733-a7bf-43e3-859e-e02085969716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:02.870621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef59733-a7bf-43e3-859e-e02085969716 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:02.884422 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2812/5551] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:02 2022] POST /identity/v3/projects => generated 357 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:02.889547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:02.890717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:02.890845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e29f46-66dd-412f-b12d-bcd719ff00eb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:02.891537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:02.891976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] PATH_INFO: `/v3/projects/372dee41ddb445bd8fb846ad5b1d45a2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:02.894205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=372dee41ddb445bd8fb846ad5b1d45a2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:02.896247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:02.966363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '372dee41ddb445bd8fb846ad5b1d45a2', 'request_id': 'req-9dd6285b-9c84-4b49-bae1-461e8975ae73'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:02.977110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '372dee41ddb445bd8fb846ad5b1d45a2', 'request_id': 'req-9dd6285b-9c84-4b49-bae1-461e8975ae73'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:02.986681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '372dee41ddb445bd8fb846ad5b1d45a2', 'request_id': 'req-9dd6285b-9c84-4b49-bae1-461e8975ae73'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:02.987533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e29f46-66dd-412f-b12d-bcd719ff00eb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:03.009434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] The token cache is being invalidate because project 372dee41ddb445bd8fb846ad5b1d45a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:03.009856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:03.011499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:03.013793 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2740/5552] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:02 2022] DELETE /identity/v3/projects/372dee41ddb445bd8fb846ad5b1d45a2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:03.018164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:03.020001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:03.069128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e16fe-06e1-46ea-a262-c1954321424b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:03.070523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e16fe-06e1-46ea-a262-c1954321424b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:03.070844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e16fe-06e1-46ea-a262-c1954321424b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:03.071126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e16fe-06e1-46ea-a262-c1954321424b admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:03.071778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e16fe-06e1-46ea-a262-c1954321424b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:03.074914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e16fe-06e1-46ea-a262-c1954321424b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:03.097467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:03.130594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6285b-9c84-4b49-bae1-461e8975ae73 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:03.138193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361b3b5-e872-4d9e-8b8c-4cbf25b07ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:03.138880 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2813/5553] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:02 2022] POST /identity/v3/users => generated 279 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:03.139146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361b3b5-e872-4d9e-8b8c-4cbf25b07ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:03.140420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361b3b5-e872-4d9e-8b8c-4cbf25b07ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:03.140804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361b3b5-e872-4d9e-8b8c-4cbf25b07ad3 admin admin] PATH_INFO: `/v3/users/9ff27d82a6da499797d77772ed4fdaa0` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:03.142807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e16fe-06e1-46ea-a262-c1954321424b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:03.142953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361b3b5-e872-4d9e-8b8c-4cbf25b07ad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ff27d82a6da499797d77772ed4fdaa0)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:03.146427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ef3e16fe-06e1-46ea-a262-c1954321424b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:03.146747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361b3b5-e872-4d9e-8b8c-4cbf25b07ad3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:03.188794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e16fe-06e1-46ea-a262-c1954321424b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:03.198665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec14220-dad0-4a21-bae0-8e319d2773ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:03.199313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec14220-dad0-4a21-bae0-8e319d2773ec admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:03.199596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec14220-dad0-4a21-bae0-8e319d2773ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:03.199871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec14220-dad0-4a21-bae0-8e319d2773ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:03.200549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec14220-dad0-4a21-bae0-8e319d2773ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:03.202167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec14220-dad0-4a21-bae0-8e319d2773ec admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:03.205418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:03.218069 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2814/5554] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:03.222609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec14220-dad0-4a21-bae0-8e319d2773ec admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:03.222869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b361b3b5-e872-4d9e-8b8c-4cbf25b07ad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ff27d82a6da499797d77772ed4fdaa0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:03.237112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec14220-dad0-4a21-bae0-8e319d2773ec admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:03.242548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d25e2b-9500-475e-a0f8-7e30aff718cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:03.242834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b361b3b5-e872-4d9e-8b8c-4cbf25b07ad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ff27d82a6da499797d77772ed4fdaa0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:03.243396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d25e2b-9500-475e-a0f8-7e30aff718cf admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:03.243567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b361b3b5-e872-4d9e-8b8c-4cbf25b07ad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ff27d82a6da499797d77772ed4fdaa0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:03.243877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d25e2b-9500-475e-a0f8-7e30aff718cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:03.244259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d25e2b-9500-475e-a0f8-7e30aff718cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:03.245049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d25e2b-9500-475e-a0f8-7e30aff718cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:03.246756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d25e2b-9500-475e-a0f8-7e30aff718cf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:03.251016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b361b3b5-e872-4d9e-8b8c-4cbf25b07ad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ff27d82a6da499797d77772ed4fdaa0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:03.259363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b361b3b5-e872-4d9e-8b8c-4cbf25b07ad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ff27d82a6da499797d77772ed4fdaa0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:03.262863 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2815/5555] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:03.263831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d25e2b-9500-475e-a0f8-7e30aff718cf admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:03.265721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-32d25e2b-9500-475e-a0f8-7e30aff718cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:03.269293 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2741/5556] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:03 2022] DELETE /identity/v3/users/9ff27d82a6da499797d77772ed4fdaa0 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:03.279640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d25e2b-9500-475e-a0f8-7e30aff718cf admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:03.339005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8267e-7380-46ff-82d7-8f86c1914f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:03.339677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8267e-7380-46ff-82d7-8f86c1914f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:03.339968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8267e-7380-46ff-82d7-8f86c1914f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:03.340251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8267e-7380-46ff-82d7-8f86c1914f11 admin admin] PATH_INFO: `/v3/projects/619b709397d94e84bf93df7169217ff2/users/de25c0287dd24be09b62914160863c20/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:03.356646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8267e-7380-46ff-82d7-8f86c1914f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=619b709397d94e84bf93df7169217ff2, user_id=de25c0287dd24be09b62914160863c20, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:03.359445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8267e-7380-46ff-82d7-8f86c1914f11 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:03.373107 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2816/5557] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:03 2022] PUT /identity/v3/projects/619b709397d94e84bf93df7169217ff2/users/de25c0287dd24be09b62914160863c20/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:03.383325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e746d5-c912-42eb-9e32-b7dca4558a48 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:03.383623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e746d5-c912-42eb-9e32-b7dca4558a48 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:03.383910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e746d5-c912-42eb-9e32-b7dca4558a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:03.418822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a7e746d5-c912-42eb-9e32-b7dca4558a48 None None] MFA Rules not processed for user `de25c0287dd24be09b62914160863c20`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:03.420065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a7e746d5-c912-42eb-9e32-b7dca4558a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:03.491726 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2742/5558] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:03.501965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8267e-7380-46ff-82d7-8f86c1914f11 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:03.505212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9ee8267e-7380-46ff-82d7-8f86c1914f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:03.567882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8267e-7380-46ff-82d7-8f86c1914f11 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:03.572188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00702925-0ea0-49d9-8bab-e6e6fb91901f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:03.572795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00702925-0ea0-49d9-8bab-e6e6fb91901f service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:03.573110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00702925-0ea0-49d9-8bab-e6e6fb91901f service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:03.573394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00702925-0ea0-49d9-8bab-e6e6fb91901f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:03.582360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00702925-0ea0-49d9-8bab-e6e6fb91901f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:03.583849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00702925-0ea0-49d9-8bab-e6e6fb91901f service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:03.593189 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2817/5559] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:03.873305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e746d5-c912-42eb-9e32-b7dca4558a48 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:03.875085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a7e746d5-c912-42eb-9e32-b7dca4558a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:03.990461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e746d5-c912-42eb-9e32-b7dca4558a48 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:03.997251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:03.998180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:03.998607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:03.998903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2 admin admin] PATH_INFO: `/v3/projects/fa5232054dd84b8b8810e5cb5063ba3a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:04.000097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa5232054dd84b8b8810e5cb5063ba3a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:04.002206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:04.062999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:04.066426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa5232054dd84b8b8810e5cb5063ba3a', 'request_id': 'req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:04.079133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa5232054dd84b8b8810e5cb5063ba3a', 'request_id': 'req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:04.089618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa5232054dd84b8b8810e5cb5063ba3a', 'request_id': 'req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:04.107172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2 admin admin] The token cache is being invalidate because project fa5232054dd84b8b8810e5cb5063ba3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:04.107565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:04.108303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba3c9cb0-23c5-45a5-8ba3-3d5169998da2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:04.110175 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2743/5560] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:03 2022] DELETE /identity/v3/projects/fa5232054dd84b8b8810e5cb5063ba3a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:04.115959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f7be6-d73d-492d-b6a1-129f4a6afc55 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:04.116459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f7be6-d73d-492d-b6a1-129f4a6afc55 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:04.116584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f7be6-d73d-492d-b6a1-129f4a6afc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:04.144452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7a9f7be6-d73d-492d-b6a1-129f4a6afc55 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:04.145997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7a9f7be6-d73d-492d-b6a1-129f4a6afc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:04.209110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:04.228437 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2818/5561] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:04.236178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed76a53-e127-4542-9e7d-f68944500646 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:04.236643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed76a53-e127-4542-9e7d-f68944500646 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:04.237091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed76a53-e127-4542-9e7d-f68944500646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:04.272355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5ed76a53-e127-4542-9e7d-f68944500646 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:04.273959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5ed76a53-e127-4542-9e7d-f68944500646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:04.295908 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2744/5562] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:04.298011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f7be6-d73d-492d-b6a1-129f4a6afc55 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:04.309813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f7be6-d73d-492d-b6a1-129f4a6afc55 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:04.314777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73f221-6a66-49ae-8f5f-7d6cea55975f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:04.315928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73f221-6a66-49ae-8f5f-7d6cea55975f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:04.316216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73f221-6a66-49ae-8f5f-7d6cea55975f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:04.316653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73f221-6a66-49ae-8f5f-7d6cea55975f admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:04.317346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73f221-6a66-49ae-8f5f-7d6cea55975f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:04.317652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73f221-6a66-49ae-8f5f-7d6cea55975f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:04.319258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73f221-6a66-49ae-8f5f-7d6cea55975f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:04.332943 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2819/5563] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:04.336878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed76a53-e127-4542-9e7d-f68944500646 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:04.349739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed76a53-e127-4542-9e7d-f68944500646 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:04.356640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1123e-14b4-48e3-a788-f3aea8c6fdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:04.357707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1123e-14b4-48e3-a788-f3aea8c6fdac admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:04.359024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1123e-14b4-48e3-a788-f3aea8c6fdac admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:04.359024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1123e-14b4-48e3-a788-f3aea8c6fdac admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:04.359882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1123e-14b4-48e3-a788-f3aea8c6fdac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:04.362390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1123e-14b4-48e3-a788-f3aea8c6fdac admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:04.409139 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2745/5564] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:04 2022] POST /identity/v3/projects => generated 367 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:04.410388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73f221-6a66-49ae-8f5f-7d6cea55975f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:04.422623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73f221-6a66-49ae-8f5f-7d6cea55975f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:04.524852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd70c3-945e-4619-8bda-5e681530ae7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:04.525675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd70c3-945e-4619-8bda-5e681530ae7e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:04.525973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd70c3-945e-4619-8bda-5e681530ae7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:04.526260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd70c3-945e-4619-8bda-5e681530ae7e admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:04.526930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fd70c3-945e-4619-8bda-5e681530ae7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:04.531899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fd70c3-945e-4619-8bda-5e681530ae7e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:04.570301 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2820/5565] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:04 2022] POST /identity/v3/users => generated 284 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:04.570634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1123e-14b4-48e3-a788-f3aea8c6fdac admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:04.592089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1123e-14b4-48e3-a788-f3aea8c6fdac admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:04.600403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa57d0-14ed-4859-89bb-8222a81e2c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:04.601646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa57d0-14ed-4859-89bb-8222a81e2c4d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:04.602095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa57d0-14ed-4859-89bb-8222a81e2c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:04.602530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa57d0-14ed-4859-89bb-8222a81e2c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:04.603356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa57d0-14ed-4859-89bb-8222a81e2c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:04.606150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa57d0-14ed-4859-89bb-8222a81e2c4d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:04.624575 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2746/5566] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:04.626006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd70c3-945e-4619-8bda-5e681530ae7e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:04.640499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd70c3-945e-4619-8bda-5e681530ae7e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:04.654566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74591582-7175-40dc-b891-38e40542b5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:04.654779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74591582-7175-40dc-b891-38e40542b5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:04.654883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74591582-7175-40dc-b891-38e40542b5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:04.655161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74591582-7175-40dc-b891-38e40542b5f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:04.655888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74591582-7175-40dc-b891-38e40542b5f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:04.657910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74591582-7175-40dc-b891-38e40542b5f1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:04.686601 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2821/5567] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:04.693227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa57d0-14ed-4859-89bb-8222a81e2c4d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:04.708192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa57d0-14ed-4859-89bb-8222a81e2c4d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:04.715413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788f21d-1415-42e4-96cb-89bf330589e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:04.716869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788f21d-1415-42e4-96cb-89bf330589e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:04.717372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788f21d-1415-42e4-96cb-89bf330589e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:04.717817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788f21d-1415-42e4-96cb-89bf330589e4 admin admin] PATH_INFO: `/v3/projects/25eb2ecc0e6449a9b3242f46ff3ffcf5/users/f7fd2a68be6f470d93cf56958cdc3b5a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:04.741770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788f21d-1415-42e4-96cb-89bf330589e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25eb2ecc0e6449a9b3242f46ff3ffcf5, user_id=f7fd2a68be6f470d93cf56958cdc3b5a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:04.746264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788f21d-1415-42e4-96cb-89bf330589e4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:04.768156 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2747/5568] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:04 2022] PUT /identity/v3/projects/25eb2ecc0e6449a9b3242f46ff3ffcf5/users/f7fd2a68be6f470d93cf56958cdc3b5a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:04.778947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35674c39-d461-4076-a176-e52bc7b10569 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:04.779615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35674c39-d461-4076-a176-e52bc7b10569 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:04.780130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35674c39-d461-4076-a176-e52bc7b10569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:04.813385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-35674c39-d461-4076-a176-e52bc7b10569 None None] MFA Rules not processed for user `f7fd2a68be6f470d93cf56958cdc3b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:04.814876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-35674c39-d461-4076-a176-e52bc7b10569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:04.891490 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2822/5569] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:04.895505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788f21d-1415-42e4-96cb-89bf330589e4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:04.985579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788f21d-1415-42e4-96cb-89bf330589e4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:04.992388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101178f-a9c3-45d9-bf20-d64933009b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:04.993120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101178f-a9c3-45d9-bf20-d64933009b81 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:04.993360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101178f-a9c3-45d9-bf20-d64933009b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:04.993644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101178f-a9c3-45d9-bf20-d64933009b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:04.994320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101178f-a9c3-45d9-bf20-d64933009b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:05.003348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101178f-a9c3-45d9-bf20-d64933009b81 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:05.066307 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2748/5570] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:04 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:05.067224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35674c39-d461-4076-a176-e52bc7b10569 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:05.081599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35674c39-d461-4076-a176-e52bc7b10569 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:05.188492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:05.215222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6479c0-624e-445c-8e22-3ed2c337ee39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:05.215884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6479c0-624e-445c-8e22-3ed2c337ee39 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:05.216167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6479c0-624e-445c-8e22-3ed2c337ee39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:05.216879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6479c0-624e-445c-8e22-3ed2c337ee39 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:05.217267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6479c0-624e-445c-8e22-3ed2c337ee39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:05.219245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6479c0-624e-445c-8e22-3ed2c337ee39 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:05.256609 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2823/5571] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:05 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:05.260512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101178f-a9c3-45d9-bf20-d64933009b81 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:05.271737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101178f-a9c3-45d9-bf20-d64933009b81 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:05.278634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55567a-8de2-4c66-8951-781cae539c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:05.279366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55567a-8de2-4c66-8951-781cae539c07 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:05.279723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55567a-8de2-4c66-8951-781cae539c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:05.280097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55567a-8de2-4c66-8951-781cae539c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:05.280829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55567a-8de2-4c66-8951-781cae539c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:05.282638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55567a-8de2-4c66-8951-781cae539c07 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:05.304927 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2749/5572] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:05.307926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6479c0-624e-445c-8e22-3ed2c337ee39 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:05.322766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6479c0-624e-445c-8e22-3ed2c337ee39 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:05.331406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c62eb-2056-41d6-8c93-e56668902fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:05.332618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c62eb-2056-41d6-8c93-e56668902fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:05.333244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c62eb-2056-41d6-8c93-e56668902fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:05.333754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c62eb-2056-41d6-8c93-e56668902fdd admin admin] PATH_INFO: `/v3/projects/bcbfd2c4533049aab233bb1fe172522f/users/9159af8f0eef4ccab67871b789d7c808/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:05.369155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c62eb-2056-41d6-8c93-e56668902fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbfd2c4533049aab233bb1fe172522f, user_id=9159af8f0eef4ccab67871b789d7c808, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:05.372334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c62eb-2056-41d6-8c93-e56668902fdd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:05.399190 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2824/5573] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:05 2022] PUT /identity/v3/projects/bcbfd2c4533049aab233bb1fe172522f/users/9159af8f0eef4ccab67871b789d7c808/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:05.400088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55567a-8de2-4c66-8951-781cae539c07 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:05.411517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55567a-8de2-4c66-8951-781cae539c07 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:05.500813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:05.536934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1744f92-dffc-4d81-8851-ced00d53821b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:05.538083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744f92-dffc-4d81-8851-ced00d53821b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:05.538675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744f92-dffc-4d81-8851-ced00d53821b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:05.539442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744f92-dffc-4d81-8851-ced00d53821b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:05.541240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1744f92-dffc-4d81-8851-ced00d53821b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:05.543359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1744f92-dffc-4d81-8851-ced00d53821b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:05.563925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c62eb-2056-41d6-8c93-e56668902fdd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:05.566437 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2750/5574] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:05.579714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c62eb-2056-41d6-8c93-e56668902fdd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:05.585812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c439fa7-5efe-48da-afaf-a676ac55ee48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:05.586732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c439fa7-5efe-48da-afaf-a676ac55ee48 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:05.587174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c439fa7-5efe-48da-afaf-a676ac55ee48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:05.587674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c439fa7-5efe-48da-afaf-a676ac55ee48 admin admin] PATH_INFO: `/v3/projects/bcbfd2c4533049aab233bb1fe172522f/users/9159af8f0eef4ccab67871b789d7c808/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:05.591883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c439fa7-5efe-48da-afaf-a676ac55ee48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbfd2c4533049aab233bb1fe172522f, user_id=9159af8f0eef4ccab67871b789d7c808, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:05.595196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c439fa7-5efe-48da-afaf-a676ac55ee48 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:05.623057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94d16c-afeb-47cc-9e7e-76b164097503 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:05.623703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94d16c-afeb-47cc-9e7e-76b164097503 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:05.623989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94d16c-afeb-47cc-9e7e-76b164097503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:05.628546 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2825/5575] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:05 2022] PUT /identity/v3/projects/bcbfd2c4533049aab233bb1fe172522f/users/9159af8f0eef4ccab67871b789d7c808/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:05.693418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-de94d16c-afeb-47cc-9e7e-76b164097503 None None] MFA Rules not processed for user `9159af8f0eef4ccab67871b789d7c808`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:05.695331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-de94d16c-afeb-47cc-9e7e-76b164097503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:05.798251 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2751/5576] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:05.812407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c439fa7-5efe-48da-afaf-a676ac55ee48 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:05.814574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7c439fa7-5efe-48da-afaf-a676ac55ee48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:05.938342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c439fa7-5efe-48da-afaf-a676ac55ee48 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:05.944221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767dd63-09c3-4584-9606-b0babff905c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:05.946439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767dd63-09c3-4584-9606-b0babff905c3 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:05.947074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767dd63-09c3-4584-9606-b0babff905c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:05.947074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767dd63-09c3-4584-9606-b0babff905c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:05.969835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767dd63-09c3-4584-9606-b0babff905c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:05.971408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767dd63-09c3-4584-9606-b0babff905c3 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:05.989560 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2826/5577] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:08.536032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94d16c-afeb-47cc-9e7e-76b164097503 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:08.686906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94d16c-afeb-47cc-9e7e-76b164097503 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:08.694407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:08.698484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:08.698781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:08.700102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] PATH_INFO: `/v3/users/f7fd2a68be6f470d93cf56958cdc3b5a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:08.704395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7fd2a68be6f470d93cf56958cdc3b5a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:08.707659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:08.786815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7fd2a68be6f470d93cf56958cdc3b5a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:08.806708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7fd2a68be6f470d93cf56958cdc3b5a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:08.809411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7fd2a68be6f470d93cf56958cdc3b5a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:08.811694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:08.821273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7fd2a68be6f470d93cf56958cdc3b5a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:08.833864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7fd2a68be6f470d93cf56958cdc3b5a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:08.854575 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2752/5578] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:08 2022] DELETE /identity/v3/users/f7fd2a68be6f470d93cf56958cdc3b5a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:08.856343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be2827-5593-41bb-86e9-39cfc60a5135 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:08.856786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be2827-5593-41bb-86e9-39cfc60a5135 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:08.856989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be2827-5593-41bb-86e9-39cfc60a5135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:08.892463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e9be2827-5593-41bb-86e9-39cfc60a5135 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:08.894351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e9be2827-5593-41bb-86e9-39cfc60a5135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:08.981763 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2827/5579] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:08.994821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:08.996657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:09.070364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d625c6-d604-4b16-b47b-6c12603053b9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:09.074592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d742-57ca-40c7-81da-62f8aad46300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:09.075327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d742-57ca-40c7-81da-62f8aad46300 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:09.075711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d742-57ca-40c7-81da-62f8aad46300 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:09.076147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d742-57ca-40c7-81da-62f8aad46300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:09.088154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6d742-57ca-40c7-81da-62f8aad46300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:09.089824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6d742-57ca-40c7-81da-62f8aad46300 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:09.101497 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2753/5580] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:09.808318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be2827-5593-41bb-86e9-39cfc60a5135 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:09.809919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e9be2827-5593-41bb-86e9-39cfc60a5135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:09.822712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be2827-5593-41bb-86e9-39cfc60a5135 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:09.827635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:09.828576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:09.828745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:09.829665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] PATH_INFO: `/v3/projects/25eb2ecc0e6449a9b3242f46ff3ffcf5` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:09.830652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25eb2ecc0e6449a9b3242f46ff3ffcf5)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:09.832974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:09.903170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:09.914205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25eb2ecc0e6449a9b3242f46ff3ffcf5', 'request_id': 'req-79fc94b9-6e9b-4773-a025-28b0e9e43784'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:09.925937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25eb2ecc0e6449a9b3242f46ff3ffcf5', 'request_id': 'req-79fc94b9-6e9b-4773-a025-28b0e9e43784'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:09.933394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25eb2ecc0e6449a9b3242f46ff3ffcf5', 'request_id': 'req-79fc94b9-6e9b-4773-a025-28b0e9e43784'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:09.950360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] The token cache is being invalidate because project 25eb2ecc0e6449a9b3242f46ff3ffcf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:09.951106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:09.952029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:09.957408 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2828/5581] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:09 2022] DELETE /identity/v3/projects/25eb2ecc0e6449a9b3242f46ff3ffcf5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:09.958669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d742-57ca-40c7-81da-62f8aad46300 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:09.969216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d742-57ca-40c7-81da-62f8aad46300 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:10.046890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98280d72-43af-42f9-828b-1d34086ece80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:10.047495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98280d72-43af-42f9-828b-1d34086ece80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:10.047855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98280d72-43af-42f9-828b-1d34086ece80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:10.048369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98280d72-43af-42f9-828b-1d34086ece80 admin admin] PATH_INFO: `/v3/users/9159af8f0eef4ccab67871b789d7c808` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:10.049998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98280d72-43af-42f9-828b-1d34086ece80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9159af8f0eef4ccab67871b789d7c808)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:10.051937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98280d72-43af-42f9-828b-1d34086ece80 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:10.106241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98280d72-43af-42f9-828b-1d34086ece80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9159af8f0eef4ccab67871b789d7c808'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:10.129068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98280d72-43af-42f9-828b-1d34086ece80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9159af8f0eef4ccab67871b789d7c808'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:10.129834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98280d72-43af-42f9-828b-1d34086ece80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9159af8f0eef4ccab67871b789d7c808'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:10.140189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98280d72-43af-42f9-828b-1d34086ece80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9159af8f0eef4ccab67871b789d7c808'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:10.151074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98280d72-43af-42f9-828b-1d34086ece80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9159af8f0eef4ccab67871b789d7c808'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:10.159637 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2754/5582] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:09 2022] DELETE /identity/v3/users/9159af8f0eef4ccab67871b789d7c808 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:10.838915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:10.840668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:10.855940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc94b9-6e9b-4773-a025-28b0e9e43784 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:10.922074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016626a-6f64-4839-a1cd-5fbe43863b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:10.923105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016626a-6f64-4839-a1cd-5fbe43863b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:10.923840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016626a-6f64-4839-a1cd-5fbe43863b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:10.925135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016626a-6f64-4839-a1cd-5fbe43863b6f admin admin] PATH_INFO: `/v3/projects/bcbfd2c4533049aab233bb1fe172522f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:10.926905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016626a-6f64-4839-a1cd-5fbe43863b6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcbfd2c4533049aab233bb1fe172522f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:10.929038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016626a-6f64-4839-a1cd-5fbe43863b6f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:11.042526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4016626a-6f64-4839-a1cd-5fbe43863b6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcbfd2c4533049aab233bb1fe172522f', 'request_id': 'req-4016626a-6f64-4839-a1cd-5fbe43863b6f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:11.053655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4016626a-6f64-4839-a1cd-5fbe43863b6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcbfd2c4533049aab233bb1fe172522f', 'request_id': 'req-4016626a-6f64-4839-a1cd-5fbe43863b6f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:11.062274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4016626a-6f64-4839-a1cd-5fbe43863b6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcbfd2c4533049aab233bb1fe172522f', 'request_id': 'req-4016626a-6f64-4839-a1cd-5fbe43863b6f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:11.073683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4016626a-6f64-4839-a1cd-5fbe43863b6f admin admin] The token cache is being invalidate because project bcbfd2c4533049aab233bb1fe172522f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:11.073986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4016626a-6f64-4839-a1cd-5fbe43863b6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:11.074668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4016626a-6f64-4839-a1cd-5fbe43863b6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:11.076289 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2829/5583] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:10 2022] DELETE /identity/v3/projects/bcbfd2c4533049aab233bb1fe172522f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:11.081573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89da02-3131-424d-9a83-ca4d2b4cdfc8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:11.082075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89da02-3131-424d-9a83-ca4d2b4cdfc8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:11.082199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89da02-3131-424d-9a83-ca4d2b4cdfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:11.099324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:11.106732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9f89da02-3131-424d-9a83-ca4d2b4cdfc8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:11.109235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9f89da02-3131-424d-9a83-ca4d2b4cdfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:11.200320 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2755/5584] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:11.207953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b8f5-e6a3-4b0c-a083-071b21bf6118 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:11.208567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b8f5-e6a3-4b0c-a083-071b21bf6118 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:11.208966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b8f5-e6a3-4b0c-a083-071b21bf6118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:11.233154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9999b8f5-e6a3-4b0c-a083-071b21bf6118 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:11.235058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9999b8f5-e6a3-4b0c-a083-071b21bf6118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:11.249575 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2830/5585] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:11.250743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89da02-3131-424d-9a83-ca4d2b4cdfc8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:11.264966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89da02-3131-424d-9a83-ca4d2b4cdfc8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:11.270011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c861763-0c8c-4e1d-aaff-8a49e38afe3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:11.270760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c861763-0c8c-4e1d-aaff-8a49e38afe3c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:11.271053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c861763-0c8c-4e1d-aaff-8a49e38afe3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:11.271332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c861763-0c8c-4e1d-aaff-8a49e38afe3c admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:11.271986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c861763-0c8c-4e1d-aaff-8a49e38afe3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:11.272297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c861763-0c8c-4e1d-aaff-8a49e38afe3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:11.273969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c861763-0c8c-4e1d-aaff-8a49e38afe3c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:11.287463 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2756/5586] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:11.294374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999b8f5-e6a3-4b0c-a083-071b21bf6118 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:11.306550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999b8f5-e6a3-4b0c-a083-071b21bf6118 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:11.312498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b72ce-9409-4e6a-9b77-5558193ecabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:11.313319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b72ce-9409-4e6a-9b77-5558193ecabe admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:11.313616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b72ce-9409-4e6a-9b77-5558193ecabe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:11.314426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b72ce-9409-4e6a-9b77-5558193ecabe admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:11.314720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b72ce-9409-4e6a-9b77-5558193ecabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:11.317047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b72ce-9409-4e6a-9b77-5558193ecabe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:11.365048 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2831/5587] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:11 2022] POST /identity/v3/projects => generated 405 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:11.369319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c861763-0c8c-4e1d-aaff-8a49e38afe3c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:11.382072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c861763-0c8c-4e1d-aaff-8a49e38afe3c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:11.485022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c05b1-c18f-41bc-879b-6dc67ead34d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:11.485756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c05b1-c18f-41bc-879b-6dc67ead34d3 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:11.486120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c05b1-c18f-41bc-879b-6dc67ead34d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:11.486472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c05b1-c18f-41bc-879b-6dc67ead34d3 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:11.487259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c05b1-c18f-41bc-879b-6dc67ead34d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:11.489394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c05b1-c18f-41bc-879b-6dc67ead34d3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:11.522705 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2757/5588] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:11 2022] POST /identity/v3/users => generated 303 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:11.526350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b72ce-9409-4e6a-9b77-5558193ecabe admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:11.542771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b72ce-9409-4e6a-9b77-5558193ecabe admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:11.553271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f06db-253d-45b1-a384-8ae7babe0282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:11.553901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f06db-253d-45b1-a384-8ae7babe0282 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:11.554186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f06db-253d-45b1-a384-8ae7babe0282 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:11.554463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f06db-253d-45b1-a384-8ae7babe0282 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:11.555121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f06db-253d-45b1-a384-8ae7babe0282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:11.556742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f06db-253d-45b1-a384-8ae7babe0282 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:11.574611 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2832/5589] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:11.578787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c05b1-c18f-41bc-879b-6dc67ead34d3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:11.590596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c05b1-c18f-41bc-879b-6dc67ead34d3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:11.595591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a81989-4561-4c78-bb32-bcad2fd6a65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:11.597096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a81989-4561-4c78-bb32-bcad2fd6a65b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:11.597609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a81989-4561-4c78-bb32-bcad2fd6a65b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:11.597609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a81989-4561-4c78-bb32-bcad2fd6a65b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:11.598407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a81989-4561-4c78-bb32-bcad2fd6a65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:11.600989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a81989-4561-4c78-bb32-bcad2fd6a65b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:11.612937 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2758/5590] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:11.616834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f06db-253d-45b1-a384-8ae7babe0282 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:11.627546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f06db-253d-45b1-a384-8ae7babe0282 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:11.632947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b5e3d-cfa6-4e32-955d-7d8bd5697d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:11.633409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b5e3d-cfa6-4e32-955d-7d8bd5697d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:11.633773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b5e3d-cfa6-4e32-955d-7d8bd5697d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:11.634153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b5e3d-cfa6-4e32-955d-7d8bd5697d56 admin admin] PATH_INFO: `/v3/projects/d4277a26f3324841aaa42c8911a1073b/users/07819d45d57c415cb144be268c8dcf53/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:11.651912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b5e3d-cfa6-4e32-955d-7d8bd5697d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4277a26f3324841aaa42c8911a1073b, user_id=07819d45d57c415cb144be268c8dcf53, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:11.654778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b5e3d-cfa6-4e32-955d-7d8bd5697d56 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:11.657335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:11.683773 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2833/5591] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:11 2022] PUT /identity/v3/projects/d4277a26f3324841aaa42c8911a1073b/users/07819d45d57c415cb144be268c8dcf53/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:11.692726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be609d2-1b67-406a-9abd-70c5a9ced127 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:11.693324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be609d2-1b67-406a-9abd-70c5a9ced127 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:11.693681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be609d2-1b67-406a-9abd-70c5a9ced127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:11.730222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1be609d2-1b67-406a-9abd-70c5a9ced127 None None] MFA Rules not processed for user `07819d45d57c415cb144be268c8dcf53`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:11.731589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1be609d2-1b67-406a-9abd-70c5a9ced127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:11.831667 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2759/5592] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:11 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:11.836118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b5e3d-cfa6-4e32-955d-7d8bd5697d56 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:11.936096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b5e3d-cfa6-4e32-955d-7d8bd5697d56 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:11.944752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eca8db-291e-49c3-bc5e-b8eaedd76f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:11.944752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eca8db-291e-49c3-bc5e-b8eaedd76f8e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:11.944752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eca8db-291e-49c3-bc5e-b8eaedd76f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:11.944752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eca8db-291e-49c3-bc5e-b8eaedd76f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:11.945170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eca8db-291e-49c3-bc5e-b8eaedd76f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:11.947183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eca8db-291e-49c3-bc5e-b8eaedd76f8e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:12.008058 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2834/5593] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:11 2022] POST /identity/v3/projects => generated 407 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:12.011968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be609d2-1b67-406a-9abd-70c5a9ced127 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:12.020492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:12.028658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be609d2-1b67-406a-9abd-70c5a9ced127 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:12.108155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787783ce-d805-46f2-9564-c099626f2a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:12.108988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787783ce-d805-46f2-9564-c099626f2a01 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:12.109125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787783ce-d805-46f2-9564-c099626f2a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:12.109411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787783ce-d805-46f2-9564-c099626f2a01 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:12.110110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787783ce-d805-46f2-9564-c099626f2a01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:12.112060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787783ce-d805-46f2-9564-c099626f2a01 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:12.158351 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2760/5594] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:12 2022] POST /identity/v3/users => generated 310 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:12.162562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eca8db-291e-49c3-bc5e-b8eaedd76f8e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:12.179670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eca8db-291e-49c3-bc5e-b8eaedd76f8e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:12.185704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1bab-442c-49d5-aa5b-76d4713c2908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:12.186481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1bab-442c-49d5-aa5b-76d4713c2908 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:12.186768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1bab-442c-49d5-aa5b-76d4713c2908 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:12.187047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1bab-442c-49d5-aa5b-76d4713c2908 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:12.187695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d1bab-442c-49d5-aa5b-76d4713c2908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:12.190336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d1bab-442c-49d5-aa5b-76d4713c2908 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:12.210887 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2835/5595] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:12.212405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787783ce-d805-46f2-9564-c099626f2a01 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:12.223647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787783ce-d805-46f2-9564-c099626f2a01 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:12.228756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5a8fb-0077-473e-92d8-517c2ffceebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:12.229470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5a8fb-0077-473e-92d8-517c2ffceebb admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:12.229751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5a8fb-0077-473e-92d8-517c2ffceebb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:12.230025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5a8fb-0077-473e-92d8-517c2ffceebb admin admin] PATH_INFO: `/v3/projects/830f78404ffc4b7ab6ea476e6a46fc5c/users/eb6f067efc384d17bfbeedb71258d5aa/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:12.247494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5a8fb-0077-473e-92d8-517c2ffceebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=830f78404ffc4b7ab6ea476e6a46fc5c, user_id=eb6f067efc384d17bfbeedb71258d5aa, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:12.250589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5a8fb-0077-473e-92d8-517c2ffceebb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:12.269439 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2761/5596] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:12 2022] PUT /identity/v3/projects/830f78404ffc4b7ab6ea476e6a46fc5c/users/eb6f067efc384d17bfbeedb71258d5aa/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:12.270786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1bab-442c-49d5-aa5b-76d4713c2908 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:12.286420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1bab-442c-49d5-aa5b-76d4713c2908 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:12.359556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5dbca-6456-4b34-a472-5d7f0ad9be2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:12.360189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5dbca-6456-4b34-a472-5d7f0ad9be2f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:12.360480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5dbca-6456-4b34-a472-5d7f0ad9be2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:12.360764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5dbca-6456-4b34-a472-5d7f0ad9be2f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:12.362053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d5dbca-6456-4b34-a472-5d7f0ad9be2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:12.365232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d5dbca-6456-4b34-a472-5d7f0ad9be2f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:12.384783 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2836/5597] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:12.388856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5a8fb-0077-473e-92d8-517c2ffceebb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:12.404593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5a8fb-0077-473e-92d8-517c2ffceebb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:12.413329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be423f53-8373-4604-9e8e-e45289de29d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:12.414374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be423f53-8373-4604-9e8e-e45289de29d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:12.414783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be423f53-8373-4604-9e8e-e45289de29d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:12.415865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be423f53-8373-4604-9e8e-e45289de29d8 admin admin] PATH_INFO: `/v3/projects/830f78404ffc4b7ab6ea476e6a46fc5c/users/eb6f067efc384d17bfbeedb71258d5aa/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:12.418499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be423f53-8373-4604-9e8e-e45289de29d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830f78404ffc4b7ab6ea476e6a46fc5c, user_id=eb6f067efc384d17bfbeedb71258d5aa, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:12.422555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be423f53-8373-4604-9e8e-e45289de29d8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:12.436482 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2762/5598] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:12 2022] PUT /identity/v3/projects/830f78404ffc4b7ab6ea476e6a46fc5c/users/eb6f067efc384d17bfbeedb71258d5aa/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:12.448520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4281f-1b00-44d0-9d64-39f011f93848 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:12.448938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4281f-1b00-44d0-9d64-39f011f93848 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:12.449289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4281f-1b00-44d0-9d64-39f011f93848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:12.493618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-53d4281f-1b00-44d0-9d64-39f011f93848 None None] MFA Rules not processed for user `eb6f067efc384d17bfbeedb71258d5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:12.495350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-53d4281f-1b00-44d0-9d64-39f011f93848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:12.589408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:12.648246 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2837/5599] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:12.665519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be423f53-8373-4604-9e8e-e45289de29d8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:12.667370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-be423f53-8373-4604-9e8e-e45289de29d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:12.786992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be423f53-8373-4604-9e8e-e45289de29d8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:12.791364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2d358-d819-4210-9ac9-3587b71e2c23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:12.792008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2d358-d819-4210-9ac9-3587b71e2c23 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:12.792295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2d358-d819-4210-9ac9-3587b71e2c23 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:12.792572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2d358-d819-4210-9ac9-3587b71e2c23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:12.856155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4281f-1b00-44d0-9d64-39f011f93848 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:12.858128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-53d4281f-1b00-44d0-9d64-39f011f93848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:12.883858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2d358-d819-4210-9ac9-3587b71e2c23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:12.885349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2d358-d819-4210-9ac9-3587b71e2c23 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:12.897844 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2763/5600] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:12 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:12.977998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4281f-1b00-44d0-9d64-39f011f93848 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:12.983713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e431a-7454-4fec-9413-8101a3d64a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:12.985186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e431a-7454-4fec-9413-8101a3d64a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:12.985485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e431a-7454-4fec-9413-8101a3d64a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:12.985777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e431a-7454-4fec-9413-8101a3d64a0b admin admin] PATH_INFO: `/v3/users/9e686d314d164930bbcfc9905a8b45ce` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:12.987004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e431a-7454-4fec-9413-8101a3d64a0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e686d314d164930bbcfc9905a8b45ce)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:12.989031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e431a-7454-4fec-9413-8101a3d64a0b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:13.050420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-870e431a-7454-4fec-9413-8101a3d64a0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e686d314d164930bbcfc9905a8b45ce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:13.066015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-870e431a-7454-4fec-9413-8101a3d64a0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e686d314d164930bbcfc9905a8b45ce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:13.078167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-870e431a-7454-4fec-9413-8101a3d64a0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e686d314d164930bbcfc9905a8b45ce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:13.079588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-870e431a-7454-4fec-9413-8101a3d64a0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e686d314d164930bbcfc9905a8b45ce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:13.079943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-870e431a-7454-4fec-9413-8101a3d64a0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e686d314d164930bbcfc9905a8b45ce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:13.100390 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2838/5601] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:12 2022] DELETE /identity/v3/users/9e686d314d164930bbcfc9905a8b45ce => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:13.465464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2d358-d819-4210-9ac9-3587b71e2c23 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:13.484972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2d358-d819-4210-9ac9-3587b71e2c23 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:13.498537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:13.577525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:13.578420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:13.578667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:13.578952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] PATH_INFO: `/v3/projects/0de6b6fc412742beb94162e078268849` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:13.580732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de6b6fc412742beb94162e078268849)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:13.582209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:13.663285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de6b6fc412742beb94162e078268849', 'request_id': 'req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:13.671136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de6b6fc412742beb94162e078268849', 'request_id': 'req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:13.680752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de6b6fc412742beb94162e078268849', 'request_id': 'req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:13.692031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] The token cache is being invalidate because project 0de6b6fc412742beb94162e078268849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:13.692330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:13.693207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:13.695392 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2764/5602] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:13 2022] DELETE /identity/v3/projects/0de6b6fc412742beb94162e078268849 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:13.706411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652861f-df89-4cf0-a40b-29dd21db0bb8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:13.706940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652861f-df89-4cf0-a40b-29dd21db0bb8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:13.707457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652861f-df89-4cf0-a40b-29dd21db0bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:13.709913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:13.711498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:13.817118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1629b3-2167-45e7-a08a-50b57a1c3f6a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:13.824100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf74f5-72cf-4a66-991c-82031d93a914 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:13.824752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf74f5-72cf-4a66-991c-82031d93a914 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:13.825086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf74f5-72cf-4a66-991c-82031d93a914 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:13.825368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf74f5-72cf-4a66-991c-82031d93a914 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:13.827719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-04bf74f5-72cf-4a66-991c-82031d93a914 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:13.880278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:13.920004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf74f5-72cf-4a66-991c-82031d93a914 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:13.922257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf74f5-72cf-4a66-991c-82031d93a914 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:13.926739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e652861f-df89-4cf0-a40b-29dd21db0bb8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:13.928217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e652861f-df89-4cf0-a40b-29dd21db0bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:13.933597 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2765/5603] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:13 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:09:14.051038 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2839/5604] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:14.060278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa29ed-4358-476c-9d1e-a8b90402120d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:14.060756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa29ed-4358-476c-9d1e-a8b90402120d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:14.061230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa29ed-4358-476c-9d1e-a8b90402120d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:14.105560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fbfa29ed-4358-476c-9d1e-a8b90402120d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:14.106812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fbfa29ed-4358-476c-9d1e-a8b90402120d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:14.132372 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2766/5605] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:14.143743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652861f-df89-4cf0-a40b-29dd21db0bb8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:14.159845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652861f-df89-4cf0-a40b-29dd21db0bb8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:14.165418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4dbf8-6893-4985-aa8b-993b92acfc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:14.166410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4dbf8-6893-4985-aa8b-993b92acfc71 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:14.166900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4dbf8-6893-4985-aa8b-993b92acfc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:14.167373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4dbf8-6893-4985-aa8b-993b92acfc71 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:14.168252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4dbf8-6893-4985-aa8b-993b92acfc71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:14.169585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4dbf8-6893-4985-aa8b-993b92acfc71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:14.172127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4dbf8-6893-4985-aa8b-993b92acfc71 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:14.175371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:14.193955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa29ed-4358-476c-9d1e-a8b90402120d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:14.206365 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2840/5606] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:14.208022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa29ed-4358-476c-9d1e-a8b90402120d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:14.214118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb120a-9a82-46ab-87f3-803143bb55bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:14.214773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb120a-9a82-46ab-87f3-803143bb55bf admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:14.215066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb120a-9a82-46ab-87f3-803143bb55bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:14.215357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb120a-9a82-46ab-87f3-803143bb55bf admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:14.216058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb120a-9a82-46ab-87f3-803143bb55bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:14.222253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb120a-9a82-46ab-87f3-803143bb55bf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:14.270316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4dbf8-6893-4985-aa8b-993b92acfc71 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:14.273181 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2767/5607] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:14 2022] POST /identity/v3/projects => generated 387 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:14.286804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4dbf8-6893-4985-aa8b-993b92acfc71 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:14.372032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ec321-5538-4106-875b-483be9152f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:14.373134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ec321-5538-4106-875b-483be9152f28 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:14.374518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ec321-5538-4106-875b-483be9152f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:14.374518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ec321-5538-4106-875b-483be9152f28 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:14.374705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ec321-5538-4106-875b-483be9152f28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:14.376598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ec321-5538-4106-875b-483be9152f28 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:14.406224 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2841/5608] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:14 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:14.410326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb120a-9a82-46ab-87f3-803143bb55bf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:14.427788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb120a-9a82-46ab-87f3-803143bb55bf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:14.432912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba1460-3d8c-4977-aed0-000a63845f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:14.433809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba1460-3d8c-4977-aed0-000a63845f15 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:14.433932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba1460-3d8c-4977-aed0-000a63845f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:14.434142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba1460-3d8c-4977-aed0-000a63845f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:14.434785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba1460-3d8c-4977-aed0-000a63845f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:14.436694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba1460-3d8c-4977-aed0-000a63845f15 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:14.455360 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2768/5609] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:14.457621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ec321-5538-4106-875b-483be9152f28 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:14.468128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ec321-5538-4106-875b-483be9152f28 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:14.474974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2ea68-b06b-4478-a9bb-7a5cde9718b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:14.475938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2ea68-b06b-4478-a9bb-7a5cde9718b9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:14.476955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2ea68-b06b-4478-a9bb-7a5cde9718b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:14.476955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2ea68-b06b-4478-a9bb-7a5cde9718b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:14.477787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2ea68-b06b-4478-a9bb-7a5cde9718b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:14.479642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2ea68-b06b-4478-a9bb-7a5cde9718b9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:14.495438 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2842/5610] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:14.496950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba1460-3d8c-4977-aed0-000a63845f15 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:14.509935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba1460-3d8c-4977-aed0-000a63845f15 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:14.515289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85339dbc-220b-4135-8401-4c210a99a418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:14.516087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85339dbc-220b-4135-8401-4c210a99a418 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:14.516438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85339dbc-220b-4135-8401-4c210a99a418 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:14.516600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85339dbc-220b-4135-8401-4c210a99a418 admin admin] PATH_INFO: `/v3/projects/b9dc0a9036134cf7bf8f4d6449aa2281/users/ebcd0b229106483aa71d7edbc950af3e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:14.539842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85339dbc-220b-4135-8401-4c210a99a418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9dc0a9036134cf7bf8f4d6449aa2281, user_id=ebcd0b229106483aa71d7edbc950af3e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:14.543665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85339dbc-220b-4135-8401-4c210a99a418 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:14.559475 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2769/5611] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:14 2022] PUT /identity/v3/projects/b9dc0a9036134cf7bf8f4d6449aa2281/users/ebcd0b229106483aa71d7edbc950af3e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:14.565615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f172b-1a68-4a29-a261-a16cb09e03d1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:14.565909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f172b-1a68-4a29-a261-a16cb09e03d1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:14.566469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f172b-1a68-4a29-a261-a16cb09e03d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:14.587519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8d5f172b-1a68-4a29-a261-a16cb09e03d1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:14.588813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8d5f172b-1a68-4a29-a261-a16cb09e03d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:14.669736 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2843/5612] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:14.681737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85339dbc-220b-4135-8401-4c210a99a418 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:14.687732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-85339dbc-220b-4135-8401-4c210a99a418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:14.792312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85339dbc-220b-4135-8401-4c210a99a418 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:14.797112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825490db-5bda-4b6d-bcc7-5ebbaf07b008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:14.801945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825490db-5bda-4b6d-bcc7-5ebbaf07b008 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:14.802299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825490db-5bda-4b6d-bcc7-5ebbaf07b008 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:14.802583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825490db-5bda-4b6d-bcc7-5ebbaf07b008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:14.816114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825490db-5bda-4b6d-bcc7-5ebbaf07b008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:14.818098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825490db-5bda-4b6d-bcc7-5ebbaf07b008 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:14.829938 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2770/5613] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:15.639318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a3ef2-4cf1-40a4-891c-c8658dce8eca None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:15.640305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a3ef2-4cf1-40a4-891c-c8658dce8eca None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:15.640591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a3ef2-4cf1-40a4-891c-c8658dce8eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:15.672736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-382a3ef2-4cf1-40a4-891c-c8658dce8eca None None] MFA Rules not processed for user `ebcd0b229106483aa71d7edbc950af3e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:15.674114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-382a3ef2-4cf1-40a4-891c-c8658dce8eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:15.753735 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2844/5614] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:15.765377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4d097-25be-4166-ad78-e575c166590e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:15.765908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4d097-25be-4166-ad78-e575c166590e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:15.765908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4d097-25be-4166-ad78-e575c166590e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:15.787512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-71e4d097-25be-4166-ad78-e575c166590e None None] MFA Rules not processed for user `ebcd0b229106483aa71d7edbc950af3e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:15.790025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-71e4d097-25be-4166-ad78-e575c166590e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:15.802347 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2771/5615] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:15.815657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a3ef2-4cf1-40a4-891c-c8658dce8eca None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:15.817379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-382a3ef2-4cf1-40a4-891c-c8658dce8eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:15.824917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:15.916874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a3ef2-4cf1-40a4-891c-c8658dce8eca None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:15.923147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560ec97-a422-4a32-a821-2d5766f5a2ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:15.923883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560ec97-a422-4a32-a821-2d5766f5a2ff service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:15.924933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560ec97-a422-4a32-a821-2d5766f5a2ff service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:15.925228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560ec97-a422-4a32-a821-2d5766f5a2ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:15.934137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560ec97-a422-4a32-a821-2d5766f5a2ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:15.936029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560ec97-a422-4a32-a821-2d5766f5a2ff service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:15.947640 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2845/5616] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:16.405686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4d097-25be-4166-ad78-e575c166590e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:16.407808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-71e4d097-25be-4166-ad78-e575c166590e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:16.425488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4d097-25be-4166-ad78-e575c166590e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:16.430275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:16.431101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:16.431502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:16.431850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] PATH_INFO: `/v3/users/de25c0287dd24be09b62914160863c20` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:16.433342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de25c0287dd24be09b62914160863c20)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:16.435746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:16.501564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de25c0287dd24be09b62914160863c20'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:16.511922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:16.517995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de25c0287dd24be09b62914160863c20'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:16.519612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de25c0287dd24be09b62914160863c20'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:16.540287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de25c0287dd24be09b62914160863c20'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:16.559529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de25c0287dd24be09b62914160863c20'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:16.572942 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2772/5617] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:16 2022] DELETE /identity/v3/users/de25c0287dd24be09b62914160863c20 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:16.576574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec55ba5-cf2e-43a6-979e-f9c7639550bc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:16.576868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec55ba5-cf2e-43a6-979e-f9c7639550bc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:16.577175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec55ba5-cf2e-43a6-979e-f9c7639550bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:16.608631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-dec55ba5-cf2e-43a6-979e-f9c7639550bc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:16.610012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dec55ba5-cf2e-43a6-979e-f9c7639550bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:16.699951 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2846/5618] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:16.718849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:16.723576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:16.804956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd2d09-7ab5-459f-aca6-bd3280b59307 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:16.809066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73aef7c-9872-44ff-988f-6f715837cbc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:16.809849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73aef7c-9872-44ff-988f-6f715837cbc9 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:16.810322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73aef7c-9872-44ff-988f-6f715837cbc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:16.810606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73aef7c-9872-44ff-988f-6f715837cbc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:16.821312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73aef7c-9872-44ff-988f-6f715837cbc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:16.822991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73aef7c-9872-44ff-988f-6f715837cbc9 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:16.835762 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2773/5619] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:16.976727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec55ba5-cf2e-43a6-979e-f9c7639550bc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:16.978595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dec55ba5-cf2e-43a6-979e-f9c7639550bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:16.992719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec55ba5-cf2e-43a6-979e-f9c7639550bc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:16.997733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:16.998442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:16.998791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:16.999071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] PATH_INFO: `/v3/users/ebcd0b229106483aa71d7edbc950af3e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:17.000203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebcd0b229106483aa71d7edbc950af3e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:17.002662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:17.074566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebcd0b229106483aa71d7edbc950af3e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.088979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebcd0b229106483aa71d7edbc950af3e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.097585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebcd0b229106483aa71d7edbc950af3e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.098336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebcd0b229106483aa71d7edbc950af3e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.099075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebcd0b229106483aa71d7edbc950af3e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.126893 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2847/5620] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:16 2022] DELETE /identity/v3/users/ebcd0b229106483aa71d7edbc950af3e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:17.532271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73aef7c-9872-44ff-988f-6f715837cbc9 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:17.542729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73aef7c-9872-44ff-988f-6f715837cbc9 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:17.634148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a688998-a523-48f9-b1c3-d87fdac50993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:17.635100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a688998-a523-48f9-b1c3-d87fdac50993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:17.635485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a688998-a523-48f9-b1c3-d87fdac50993 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:17.635837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a688998-a523-48f9-b1c3-d87fdac50993 admin admin] PATH_INFO: `/v3/projects/b9dc0a9036134cf7bf8f4d6449aa2281` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:17.637100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a688998-a523-48f9-b1c3-d87fdac50993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9dc0a9036134cf7bf8f4d6449aa2281)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:17.641276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a688998-a523-48f9-b1c3-d87fdac50993 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:17.642456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:17.644038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:17.652547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:17.673251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f6ac1-6541-4099-a160-52d7dec29221 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:17.678933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff129544-f846-4254-8cf9-f9b1d10d6d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:17.679816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff129544-f846-4254-8cf9-f9b1d10d6d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:17.680262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff129544-f846-4254-8cf9-f9b1d10d6d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:17.680698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff129544-f846-4254-8cf9-f9b1d10d6d50 admin admin] PATH_INFO: `/v3/projects/619b709397d94e84bf93df7169217ff2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:17.682209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff129544-f846-4254-8cf9-f9b1d10d6d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=619b709397d94e84bf93df7169217ff2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:17.684385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff129544-f846-4254-8cf9-f9b1d10d6d50 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:17.736260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a688998-a523-48f9-b1c3-d87fdac50993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9dc0a9036134cf7bf8f4d6449aa2281', 'request_id': 'req-7a688998-a523-48f9-b1c3-d87fdac50993'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.748080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ff129544-f846-4254-8cf9-f9b1d10d6d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '619b709397d94e84bf93df7169217ff2', 'request_id': 'req-ff129544-f846-4254-8cf9-f9b1d10d6d50'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.751825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a688998-a523-48f9-b1c3-d87fdac50993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9dc0a9036134cf7bf8f4d6449aa2281', 'request_id': 'req-7a688998-a523-48f9-b1c3-d87fdac50993'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.762192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ff129544-f846-4254-8cf9-f9b1d10d6d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '619b709397d94e84bf93df7169217ff2', 'request_id': 'req-ff129544-f846-4254-8cf9-f9b1d10d6d50'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.763988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a688998-a523-48f9-b1c3-d87fdac50993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9dc0a9036134cf7bf8f4d6449aa2281', 'request_id': 'req-7a688998-a523-48f9-b1c3-d87fdac50993'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.771453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ff129544-f846-4254-8cf9-f9b1d10d6d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '619b709397d94e84bf93df7169217ff2', 'request_id': 'req-ff129544-f846-4254-8cf9-f9b1d10d6d50'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.783185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ff129544-f846-4254-8cf9-f9b1d10d6d50 admin admin] The token cache is being invalidate because project 619b709397d94e84bf93df7169217ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:17.784284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ff129544-f846-4254-8cf9-f9b1d10d6d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.785436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ff129544-f846-4254-8cf9-f9b1d10d6d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.787803 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2848/5621] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:17 2022] DELETE /identity/v3/projects/619b709397d94e84bf93df7169217ff2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:17.793300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc3d4f-07e2-4fa1-9070-bc91bef9598b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:17.793782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc3d4f-07e2-4fa1-9070-bc91bef9598b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:17.794061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc3d4f-07e2-4fa1-9070-bc91bef9598b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:17.802310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a688998-a523-48f9-b1c3-d87fdac50993 admin admin] The token cache is being invalidate because project b9dc0a9036134cf7bf8f4d6449aa2281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:17.802622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a688998-a523-48f9-b1c3-d87fdac50993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.803318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a688998-a523-48f9-b1c3-d87fdac50993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:17.807103 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2774/5622] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:17 2022] DELETE /identity/v3/projects/b9dc0a9036134cf7bf8f4d6449aa2281 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:17.813494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d94a8c-b616-4282-8167-7f98b09c8a95 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:17.814014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d94a8c-b616-4282-8167-7f98b09c8a95 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:17.814499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d94a8c-b616-4282-8167-7f98b09c8a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:17.816657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-26bc3d4f-07e2-4fa1-9070-bc91bef9598b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:17.818021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-26bc3d4f-07e2-4fa1-9070-bc91bef9598b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:17.840273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c1d94a8c-b616-4282-8167-7f98b09c8a95 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:17.842695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c1d94a8c-b616-4282-8167-7f98b09c8a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:17.849450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:17.894131 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2849/5623] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:17.896052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be53e0-37f4-4f74-bba5-47d3533a461b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:17.896348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be53e0-37f4-4f74-bba5-47d3533a461b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:17.896625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be53e0-37f4-4f74-bba5-47d3533a461b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:17.924042 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2775/5624] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:17.930870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-83be53e0-37f4-4f74-bba5-47d3533a461b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:17.931633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-83be53e0-37f4-4f74-bba5-47d3533a461b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:17.933944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329cee5-f7e6-4063-b0de-123e7edab69d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:17.934238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329cee5-f7e6-4063-b0de-123e7edab69d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:17.934518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329cee5-f7e6-4063-b0de-123e7edab69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:17.943310 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2850/5625] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:17.947869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be53e0-37f4-4f74-bba5-47d3533a461b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:17.957955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be53e0-37f4-4f74-bba5-47d3533a461b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:17.963464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783d928-c364-419d-a19b-2c533b53f8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:17.964138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783d928-c364-419d-a19b-2c533b53f8ae admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:17.964429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783d928-c364-419d-a19b-2c533b53f8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:17.964707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783d928-c364-419d-a19b-2c533b53f8ae admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:17.965393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783d928-c364-419d-a19b-2c533b53f8ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:17.965655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4329cee5-f7e6-4063-b0de-123e7edab69d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:17.965896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783d928-c364-419d-a19b-2c533b53f8ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:17.967291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4329cee5-f7e6-4063-b0de-123e7edab69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:17.967700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783d928-c364-419d-a19b-2c533b53f8ae admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:17.980432 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2776/5626] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:17.984961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329cee5-f7e6-4063-b0de-123e7edab69d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:17.990202 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2851/5627] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:17.991335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783d928-c364-419d-a19b-2c533b53f8ae admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:18.001609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329cee5-f7e6-4063-b0de-123e7edab69d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.004656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783d928-c364-419d-a19b-2c533b53f8ae admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.009149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ece8fd-9c26-413c-a5ad-6364b860fca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.009962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f330fe4-f193-40df-8c9f-0e6a3c236a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.010986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330fe4-f193-40df-8c9f-0e6a3c236a02 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.011214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330fe4-f193-40df-8c9f-0e6a3c236a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.011540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330fe4-f193-40df-8c9f-0e6a3c236a02 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.011759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ece8fd-9c26-413c-a5ad-6364b860fca9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.012183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f330fe4-f193-40df-8c9f-0e6a3c236a02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.012449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ece8fd-9c26-413c-a5ad-6364b860fca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.013134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ece8fd-9c26-413c-a5ad-6364b860fca9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.014012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ece8fd-9c26-413c-a5ad-6364b860fca9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:18.014310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f330fe4-f193-40df-8c9f-0e6a3c236a02 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.014787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ece8fd-9c26-413c-a5ad-6364b860fca9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.016662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ece8fd-9c26-413c-a5ad-6364b860fca9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.031425 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2777/5628] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:18.037511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ece8fd-9c26-413c-a5ad-6364b860fca9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:18.057170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ece8fd-9c26-413c-a5ad-6364b860fca9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.064957 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2852/5629] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:17 2022] POST /identity/v3/projects => generated 405 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:18.065709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c50af-b771-47cd-a121-31ebfa3420aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.066456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c50af-b771-47cd-a121-31ebfa3420aa admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.066688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c50af-b771-47cd-a121-31ebfa3420aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.067020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c50af-b771-47cd-a121-31ebfa3420aa admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.067129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f330fe4-f193-40df-8c9f-0e6a3c236a02 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:18.067688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c50af-b771-47cd-a121-31ebfa3420aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.069995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c50af-b771-47cd-a121-31ebfa3420aa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.105468 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2778/5630] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:18 2022] POST /identity/v3/projects => generated 389 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:18.109637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c50af-b771-47cd-a121-31ebfa3420aa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:18.162247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:18.189873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f330fe4-f193-40df-8c9f-0e6a3c236a02 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.213165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c50af-b771-47cd-a121-31ebfa3420aa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.220743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098de27-d170-448f-8186-28a8d781d84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.222723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098de27-d170-448f-8186-28a8d781d84c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.223171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098de27-d170-448f-8186-28a8d781d84c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.224095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098de27-d170-448f-8186-28a8d781d84c admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.225020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098de27-d170-448f-8186-28a8d781d84c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.229011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098de27-d170-448f-8186-28a8d781d84c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.270152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1044e7-d2dc-4c30-8277-63de320baab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.271090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1044e7-d2dc-4c30-8277-63de320baab7 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.271713 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2779/5631] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:18 2022] POST /identity/v3/users => generated 295 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:18.272286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1044e7-d2dc-4c30-8277-63de320baab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.272986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1044e7-d2dc-4c30-8277-63de320baab7 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.274832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1044e7-d2dc-4c30-8277-63de320baab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.275901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098de27-d170-448f-8186-28a8d781d84c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:18.276937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1044e7-d2dc-4c30-8277-63de320baab7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.292627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098de27-d170-448f-8186-28a8d781d84c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.300170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37feecc7-8b41-4d53-8f7b-ae85746359ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.303665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37feecc7-8b41-4d53-8f7b-ae85746359ed admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.303985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37feecc7-8b41-4d53-8f7b-ae85746359ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.305689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37feecc7-8b41-4d53-8f7b-ae85746359ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.307872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37feecc7-8b41-4d53-8f7b-ae85746359ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.309661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37feecc7-8b41-4d53-8f7b-ae85746359ed admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.313627 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2853/5632] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:18 2022] POST /identity/v3/users => generated 303 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:18.317407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1044e7-d2dc-4c30-8277-63de320baab7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:18.332858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1044e7-d2dc-4c30-8277-63de320baab7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.340113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d02f3a-b304-4b64-a75a-e71fec3499c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.340803 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2780/5633] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:18.344736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37feecc7-8b41-4d53-8f7b-ae85746359ed admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:18.346473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d02f3a-b304-4b64-a75a-e71fec3499c8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.346920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d02f3a-b304-4b64-a75a-e71fec3499c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.347369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d02f3a-b304-4b64-a75a-e71fec3499c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.348550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d02f3a-b304-4b64-a75a-e71fec3499c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.351132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d02f3a-b304-4b64-a75a-e71fec3499c8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.368631 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2854/5634] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:18.371872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d02f3a-b304-4b64-a75a-e71fec3499c8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:18.385478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37feecc7-8b41-4d53-8f7b-ae85746359ed admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.386641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d02f3a-b304-4b64-a75a-e71fec3499c8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.395865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a207b3-0518-4446-a53e-24bbbc996479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.396550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a207b3-0518-4446-a53e-24bbbc996479 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.396849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a207b3-0518-4446-a53e-24bbbc996479 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.397735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a207b3-0518-4446-a53e-24bbbc996479 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.398406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a207b3-0518-4446-a53e-24bbbc996479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.401191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a207b3-0518-4446-a53e-24bbbc996479 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.402786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e7e97-af31-4715-bb3a-dc61fe1d1005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.404208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e7e97-af31-4715-bb3a-dc61fe1d1005 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.406499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e7e97-af31-4715-bb3a-dc61fe1d1005 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.406935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e7e97-af31-4715-bb3a-dc61fe1d1005 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.407709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e7e97-af31-4715-bb3a-dc61fe1d1005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.411775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e7e97-af31-4715-bb3a-dc61fe1d1005 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.418424 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2855/5635] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:18.422658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a207b3-0518-4446-a53e-24bbbc996479 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:18.431009 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2781/5636] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:18.435000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e7e97-af31-4715-bb3a-dc61fe1d1005 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:18.442807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a207b3-0518-4446-a53e-24bbbc996479 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.451610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d9ff6-4b5f-4636-ab0a-9a0600b6f1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.452284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d9ff6-4b5f-4636-ab0a-9a0600b6f1d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.452613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d9ff6-4b5f-4636-ab0a-9a0600b6f1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.452836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d9ff6-4b5f-4636-ab0a-9a0600b6f1d5 admin admin] PATH_INFO: `/v3/projects/79004e1ade2f43df828d0d8eab4306ef/users/12d5c5d48a2d4340a823d19fd3145276/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.455864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e7e97-af31-4715-bb3a-dc61fe1d1005 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.470394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072751d-9843-42ed-9854-a5fe6041e4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.472884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072751d-9843-42ed-9854-a5fe6041e4cd admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.473453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072751d-9843-42ed-9854-a5fe6041e4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.473810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072751d-9843-42ed-9854-a5fe6041e4cd admin admin] PATH_INFO: `/v3/projects/0f4fe674ef004caf8df73b8b06f7dd23/users/38dddfbdc4f0450e8d5fc7b214796758/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.481431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d9ff6-4b5f-4636-ab0a-9a0600b6f1d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79004e1ade2f43df828d0d8eab4306ef, user_id=12d5c5d48a2d4340a823d19fd3145276, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.483843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d9ff6-4b5f-4636-ab0a-9a0600b6f1d5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.503753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072751d-9843-42ed-9854-a5fe6041e4cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4fe674ef004caf8df73b8b06f7dd23, user_id=38dddfbdc4f0450e8d5fc7b214796758, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.504795 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2856/5637] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:18 2022] PUT /identity/v3/projects/79004e1ade2f43df828d0d8eab4306ef/users/12d5c5d48a2d4340a823d19fd3145276/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:18.508341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072751d-9843-42ed-9854-a5fe6041e4cd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.510046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27933ca5-4536-4a87-89ae-8b780a873c62 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.510623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27933ca5-4536-4a87-89ae-8b780a873c62 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.511203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27933ca5-4536-4a87-89ae-8b780a873c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.537215 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2782/5638] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:18 2022] PUT /identity/v3/projects/0f4fe674ef004caf8df73b8b06f7dd23/users/38dddfbdc4f0450e8d5fc7b214796758/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:18.537933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee2f69-8d81-4146-b583-7e1e746b4ae1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.538226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee2f69-8d81-4146-b583-7e1e746b4ae1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.538505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee2f69-8d81-4146-b583-7e1e746b4ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.549130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-27933ca5-4536-4a87-89ae-8b780a873c62 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:18.552759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-27933ca5-4536-4a87-89ae-8b780a873c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:18.577979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a6ee2f69-8d81-4146-b583-7e1e746b4ae1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:18.579133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a6ee2f69-8d81-4146-b583-7e1e746b4ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:18.585549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:18.661553 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2857/5639] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:18.675309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27933ca5-4536-4a87-89ae-8b780a873c62 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:18.676796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-27933ca5-4536-4a87-89ae-8b780a873c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:18.678170 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2783/5640] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:18.687024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee2f69-8d81-4146-b583-7e1e746b4ae1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:18.688445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a6ee2f69-8d81-4146-b583-7e1e746b4ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:18.728767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:18.753876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27933ca5-4536-4a87-89ae-8b780a873c62 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.762767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f101adb-4238-4fd2-a34a-35d3e4c8706d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.763583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f101adb-4238-4fd2-a34a-35d3e4c8706d service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.763874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f101adb-4238-4fd2-a34a-35d3e4c8706d service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.764151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f101adb-4238-4fd2-a34a-35d3e4c8706d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.777708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee2f69-8d81-4146-b583-7e1e746b4ae1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:18.780172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f101adb-4238-4fd2-a34a-35d3e4c8706d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.781812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aede0c6-af8a-4dd7-8343-5bca935015be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:18.782469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f101adb-4238-4fd2-a34a-35d3e4c8706d service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.782616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aede0c6-af8a-4dd7-8343-5bca935015be service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:18.782912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aede0c6-af8a-4dd7-8343-5bca935015be service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:18.783195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aede0c6-af8a-4dd7-8343-5bca935015be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:18.795254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aede0c6-af8a-4dd7-8343-5bca935015be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:18.795941 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2858/5641] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:18.797069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aede0c6-af8a-4dd7-8343-5bca935015be service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:18.809443 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2784/5642] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:09:20.499434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83eb94-15c9-4d78-b1e3-a4ee529e9ba7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:20.500317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83eb94-15c9-4d78-b1e3-a4ee529e9ba7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:20.500762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83eb94-15c9-4d78-b1e3-a4ee529e9ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:20.550504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9b83eb94-15c9-4d78-b1e3-a4ee529e9ba7 None None] MFA Rules not processed for user `12d5c5d48a2d4340a823d19fd3145276`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:20.551854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9b83eb94-15c9-4d78-b1e3-a4ee529e9ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:20.656764 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2859/5643] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:20 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:20.661655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aede0c6-af8a-4dd7-8343-5bca935015be service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:20.688711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aede0c6-af8a-4dd7-8343-5bca935015be service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:20.702104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44bf53-183b-45b0-9295-7227d863db9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:20.703309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44bf53-183b-45b0-9295-7227d863db9b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:20.704515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44bf53-183b-45b0-9295-7227d863db9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:20.705038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44bf53-183b-45b0-9295-7227d863db9b admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:20.706514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44bf53-183b-45b0-9295-7227d863db9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:20.710898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44bf53-183b-45b0-9295-7227d863db9b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:20.777212 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2785/5644] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:20 2022] POST /identity/v3/projects => generated 405 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:20.777839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83eb94-15c9-4d78-b1e3-a4ee529e9ba7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:20.791224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83eb94-15c9-4d78-b1e3-a4ee529e9ba7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:20.852291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7a65c-93e2-4f83-96fd-d59be7cf9e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:20.853243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7a65c-93e2-4f83-96fd-d59be7cf9e69 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:20.853728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7a65c-93e2-4f83-96fd-d59be7cf9e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:20.854175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7a65c-93e2-4f83-96fd-d59be7cf9e69 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:20.854982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7a65c-93e2-4f83-96fd-d59be7cf9e69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:20.857285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7a65c-93e2-4f83-96fd-d59be7cf9e69 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:20.893736 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2860/5645] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:20 2022] POST /identity/v3/users => generated 309 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:20.896704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44bf53-183b-45b0-9295-7227d863db9b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:20.907879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44bf53-183b-45b0-9295-7227d863db9b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:20.913827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6970ed4-2677-4559-a390-6a5d2b714c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:20.914759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6970ed4-2677-4559-a390-6a5d2b714c32 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:20.915292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6970ed4-2677-4559-a390-6a5d2b714c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:20.915802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6970ed4-2677-4559-a390-6a5d2b714c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:20.916666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6970ed4-2677-4559-a390-6a5d2b714c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:20.918533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6970ed4-2677-4559-a390-6a5d2b714c32 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:20.934048 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2786/5646] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:20.935376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7a65c-93e2-4f83-96fd-d59be7cf9e69 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:20.950388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7a65c-93e2-4f83-96fd-d59be7cf9e69 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:20.957419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e606b-0794-45fe-a29a-523f15aee2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:20.958312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e606b-0794-45fe-a29a-523f15aee2f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:20.958473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e606b-0794-45fe-a29a-523f15aee2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:20.959401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e606b-0794-45fe-a29a-523f15aee2f3 admin admin] PATH_INFO: `/v3/projects/0aa5975a7d6c47bcb33dfa40ea68f34f/users/fd61a3d45321405ea295b6cfbc52359d/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:20.980041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e606b-0794-45fe-a29a-523f15aee2f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa5975a7d6c47bcb33dfa40ea68f34f, user_id=fd61a3d45321405ea295b6cfbc52359d, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:20.983513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e606b-0794-45fe-a29a-523f15aee2f3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:21.007569 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2861/5647] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:20 2022] PUT /identity/v3/projects/0aa5975a7d6c47bcb33dfa40ea68f34f/users/fd61a3d45321405ea295b6cfbc52359d/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:21.011819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6970ed4-2677-4559-a390-6a5d2b714c32 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:21.025014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6970ed4-2677-4559-a390-6a5d2b714c32 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:21.094593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce137e-4ffd-47d0-8ffd-8cc34e1db375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:21.096925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce137e-4ffd-47d0-8ffd-8cc34e1db375 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:21.097415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce137e-4ffd-47d0-8ffd-8cc34e1db375 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:21.098851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce137e-4ffd-47d0-8ffd-8cc34e1db375 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:21.099845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce137e-4ffd-47d0-8ffd-8cc34e1db375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:21.101985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce137e-4ffd-47d0-8ffd-8cc34e1db375 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:21.118933 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2787/5648] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:21.119609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e606b-0794-45fe-a29a-523f15aee2f3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:21.135600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e606b-0794-45fe-a29a-523f15aee2f3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:21.147821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29b798-a79c-4cc7-a76f-e0ff6df65649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:21.148807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29b798-a79c-4cc7-a76f-e0ff6df65649 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:21.149467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29b798-a79c-4cc7-a76f-e0ff6df65649 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:21.149865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29b798-a79c-4cc7-a76f-e0ff6df65649 admin admin] PATH_INFO: `/v3/projects/0aa5975a7d6c47bcb33dfa40ea68f34f/users/fd61a3d45321405ea295b6cfbc52359d/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:21.152261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29b798-a79c-4cc7-a76f-e0ff6df65649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa5975a7d6c47bcb33dfa40ea68f34f, user_id=fd61a3d45321405ea295b6cfbc52359d, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:21.155384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29b798-a79c-4cc7-a76f-e0ff6df65649 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:21.172780 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2862/5649] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:21 2022] PUT /identity/v3/projects/0aa5975a7d6c47bcb33dfa40ea68f34f/users/fd61a3d45321405ea295b6cfbc52359d/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:22.731199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce137e-4ffd-47d0-8ffd-8cc34e1db375 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:22.733666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b6ce137e-4ffd-47d0-8ffd-8cc34e1db375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:22.747866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce137e-4ffd-47d0-8ffd-8cc34e1db375 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:22.783087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:22.859199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:22.860226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:22.860768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:22.861283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] PATH_INFO: `/v3/users/5875e3d67b864f3abe87e9c8a179d602` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:22.862842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5875e3d67b864f3abe87e9c8a179d602)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:22.865391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:22.932988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5875e3d67b864f3abe87e9c8a179d602'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:22.948949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5875e3d67b864f3abe87e9c8a179d602'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:22.950235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5875e3d67b864f3abe87e9c8a179d602'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:22.961598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5875e3d67b864f3abe87e9c8a179d602'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:22.974476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5875e3d67b864f3abe87e9c8a179d602'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:22.986168 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2788/5650] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:22 2022] DELETE /identity/v3/users/5875e3d67b864f3abe87e9c8a179d602 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:22.992430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c65483-2005-437c-83db-52d035f321a3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:22.993661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c65483-2005-437c-83db-52d035f321a3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:22.994117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c65483-2005-437c-83db-52d035f321a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:23.039429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-22c65483-2005-437c-83db-52d035f321a3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:23.041760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-22c65483-2005-437c-83db-52d035f321a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:23.062912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:23.173831 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2863/5651] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:23.186054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:23.187686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:23.251201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aa100-6bcb-4c0f-a3f2-42043da1e206 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:23.251433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aa100-6bcb-4c0f-a3f2-42043da1e206 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:23.251676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aa100-6bcb-4c0f-a3f2-42043da1e206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:23.305444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b16aa100-6bcb-4c0f-a3f2-42043da1e206 None None] MFA Rules not processed for user `fd61a3d45321405ea295b6cfbc52359d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:23.307947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b16aa100-6bcb-4c0f-a3f2-42043da1e206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:23.317852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc5248-a781-4e32-aa8b-e3a855239e90 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:23.324690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a3467-561a-44e7-8fa6-4095af7a6787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:23.327601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a3467-561a-44e7-8fa6-4095af7a6787 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:23.327601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a3467-561a-44e7-8fa6-4095af7a6787 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:23.327856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a3467-561a-44e7-8fa6-4095af7a6787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:23.349107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a3467-561a-44e7-8fa6-4095af7a6787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:23.352798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a3467-561a-44e7-8fa6-4095af7a6787 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:23.369004 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2789/5652] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:23.435351 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2864/5653] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:23 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:23.779719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a3467-561a-44e7-8fa6-4095af7a6787 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:23.781506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3c1a3467-561a-44e7-8fa6-4095af7a6787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:23.848636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a3467-561a-44e7-8fa6-4095af7a6787 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:23.859142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1413ff-519a-4a9b-a8ab-55c0fc18be75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:23.862224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1413ff-519a-4a9b-a8ab-55c0fc18be75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:23.862642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1413ff-519a-4a9b-a8ab-55c0fc18be75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:23.865570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1413ff-519a-4a9b-a8ab-55c0fc18be75 admin admin] PATH_INFO: `/v3/users/07819d45d57c415cb144be268c8dcf53` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:23.875841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1413ff-519a-4a9b-a8ab-55c0fc18be75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07819d45d57c415cb144be268c8dcf53)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:23.878348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1413ff-519a-4a9b-a8ab-55c0fc18be75 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:23.938387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:23.949960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db1413ff-519a-4a9b-a8ab-55c0fc18be75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07819d45d57c415cb144be268c8dcf53'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:23.970522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db1413ff-519a-4a9b-a8ab-55c0fc18be75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07819d45d57c415cb144be268c8dcf53'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:23.972126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db1413ff-519a-4a9b-a8ab-55c0fc18be75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07819d45d57c415cb144be268c8dcf53'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:23.983765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db1413ff-519a-4a9b-a8ab-55c0fc18be75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07819d45d57c415cb144be268c8dcf53'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:23.984261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aa100-6bcb-4c0f-a3f2-42043da1e206 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:23.985770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b16aa100-6bcb-4c0f-a3f2-42043da1e206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:23.995298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db1413ff-519a-4a9b-a8ab-55c0fc18be75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07819d45d57c415cb144be268c8dcf53'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:24.002112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aa100-6bcb-4c0f-a3f2-42043da1e206 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:24.003843 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2790/5654] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:23 2022] DELETE /identity/v3/users/07819d45d57c415cb144be268c8dcf53 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:24.009542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d312cf-4db3-42da-8341-90053e6b2b88 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:24.010221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d312cf-4db3-42da-8341-90053e6b2b88 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:24.010516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d312cf-4db3-42da-8341-90053e6b2b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:24.041249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-02d312cf-4db3-42da-8341-90053e6b2b88 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:24.042878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-02d312cf-4db3-42da-8341-90053e6b2b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:24.087244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:24.089162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:24.089652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:24.090099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] PATH_INFO: `/v3/projects/9092516787ff4ea7847227f4a8266ceb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:24.092207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9092516787ff4ea7847227f4a8266ceb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:24.095042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:24.127366 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2791/5655] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:24.142414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d312cf-4db3-42da-8341-90053e6b2b88 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:24.145814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-02d312cf-4db3-42da-8341-90053e6b2b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:24.178497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:24.183172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9092516787ff4ea7847227f4a8266ceb', 'request_id': 'req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:24.202576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9092516787ff4ea7847227f4a8266ceb', 'request_id': 'req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:24.216651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9092516787ff4ea7847227f4a8266ceb', 'request_id': 'req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:24.235702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] The token cache is being invalidate because project 9092516787ff4ea7847227f4a8266ceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:24.236080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:24.236977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:24.240839 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2865/5656] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:23 2022] DELETE /identity/v3/projects/9092516787ff4ea7847227f4a8266ceb => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:24.246714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:24.250110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:24.277120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d312cf-4db3-42da-8341-90053e6b2b88 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:24.378113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d3817-6271-4f75-8fdb-ab0da5dd8f22 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:24.384602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:24.385549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:24.385926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:24.386324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] PATH_INFO: `/v3/users/841e6ce4ab1b4aab94fb0a41cdbc1cd5` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:24.387492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=841e6ce4ab1b4aab94fb0a41cdbc1cd5)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:24.389354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7b765-261d-497b-b117-8f19c358f884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:24.389801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:24.390167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7b765-261d-497b-b117-8f19c358f884 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:24.390595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7b765-261d-497b-b117-8f19c358f884 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:24.390966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7b765-261d-497b-b117-8f19c358f884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:24.395352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-eeb7b765-261d-497b-b117-8f19c358f884 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:24.410676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7b765-261d-497b-b117-8f19c358f884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:24.412719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7b765-261d-497b-b117-8f19c358f884 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:24.426108 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2792/5657] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:24.457968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '841e6ce4ab1b4aab94fb0a41cdbc1cd5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:24.472615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '841e6ce4ab1b4aab94fb0a41cdbc1cd5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:24.482070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '841e6ce4ab1b4aab94fb0a41cdbc1cd5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:24.482709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '841e6ce4ab1b4aab94fb0a41cdbc1cd5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:24.483360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '841e6ce4ab1b4aab94fb0a41cdbc1cd5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:24.502987 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2866/5658] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:24 2022] DELETE /identity/v3/users/841e6ce4ab1b4aab94fb0a41cdbc1cd5 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:24.876759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7b765-261d-497b-b117-8f19c358f884 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:24.890889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:24.893918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:24.909858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7b765-261d-497b-b117-8f19c358f884 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:24.934739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:25.008409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:25.020039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356a051-b7fe-42eb-a7be-69a4d6e1eca1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:25.027143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:25.028645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:25.029169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:25.029748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] PATH_INFO: `/v3/projects/d4277a26f3324841aaa42c8911a1073b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:25.031387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4277a26f3324841aaa42c8911a1073b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:25.033998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:25.061751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab29abe-98e6-4639-8742-dd2e1c39bb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:25.062473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab29abe-98e6-4639-8742-dd2e1c39bb02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:25.063516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab29abe-98e6-4639-8742-dd2e1c39bb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:25.065935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab29abe-98e6-4639-8742-dd2e1c39bb02 admin admin] PATH_INFO: `/v3/projects/65482ec3d35b4791abdfd3a9d57d8219` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:25.067444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab29abe-98e6-4639-8742-dd2e1c39bb02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65482ec3d35b4791abdfd3a9d57d8219)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:25.078116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab29abe-98e6-4639-8742-dd2e1c39bb02 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:25.103476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4277a26f3324841aaa42c8911a1073b', 'request_id': 'req-e9dc8524-c723-4110-b654-708c2b22ac63'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.128438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4277a26f3324841aaa42c8911a1073b', 'request_id': 'req-e9dc8524-c723-4110-b654-708c2b22ac63'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.150017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4277a26f3324841aaa42c8911a1073b', 'request_id': 'req-e9dc8524-c723-4110-b654-708c2b22ac63'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.163103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] The token cache is being invalidate because project d4277a26f3324841aaa42c8911a1073b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:25.163406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.164221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.166928 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2867/5659] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:24 2022] DELETE /identity/v3/projects/d4277a26f3324841aaa42c8911a1073b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:25.172239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:25.180463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:25.180686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0ab29abe-98e6-4639-8742-dd2e1c39bb02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65482ec3d35b4791abdfd3a9d57d8219', 'request_id': 'req-0ab29abe-98e6-4639-8742-dd2e1c39bb02'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.195533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0ab29abe-98e6-4639-8742-dd2e1c39bb02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65482ec3d35b4791abdfd3a9d57d8219', 'request_id': 'req-0ab29abe-98e6-4639-8742-dd2e1c39bb02'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.213680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0ab29abe-98e6-4639-8742-dd2e1c39bb02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65482ec3d35b4791abdfd3a9d57d8219', 'request_id': 'req-0ab29abe-98e6-4639-8742-dd2e1c39bb02'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.246784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0ab29abe-98e6-4639-8742-dd2e1c39bb02 admin admin] The token cache is being invalidate because project 65482ec3d35b4791abdfd3a9d57d8219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:25.249664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0ab29abe-98e6-4639-8742-dd2e1c39bb02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.251386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0ab29abe-98e6-4639-8742-dd2e1c39bb02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.257835 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2793/5660] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:24 2022] DELETE /identity/v3/projects/65482ec3d35b4791abdfd3a9d57d8219 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:25.263805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128cfd5-96e8-4f96-bdba-e507879605a4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:25.264099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128cfd5-96e8-4f96-bdba-e507879605a4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:25.264467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128cfd5-96e8-4f96-bdba-e507879605a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:25.299911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4128cfd5-96e8-4f96-bdba-e507879605a4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:25.301641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4128cfd5-96e8-4f96-bdba-e507879605a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:25.323707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc8524-c723-4110-b654-708c2b22ac63 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:25.377974 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2794/5661] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:25.377974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128cfd5-96e8-4f96-bdba-e507879605a4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:25.388845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4128cfd5-96e8-4f96-bdba-e507879605a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:25.403398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128cfd5-96e8-4f96-bdba-e507879605a4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:25.410157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:25.411214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:25.412511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:25.412936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] PATH_INFO: `/v3/users/12d5c5d48a2d4340a823d19fd3145276` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:25.414380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d5c5d48a2d4340a823d19fd3145276)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:25.417025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:25.423390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657aa9e6-d4d5-4619-bdf5-0b7a8f48e6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:25.425328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aa9e6-d4d5-4619-bdf5-0b7a8f48e6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:25.425914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aa9e6-d4d5-4619-bdf5-0b7a8f48e6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:25.426198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aa9e6-d4d5-4619-bdf5-0b7a8f48e6cd admin admin] PATH_INFO: `/v3/users/eb6f067efc384d17bfbeedb71258d5aa` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:25.428476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657aa9e6-d4d5-4619-bdf5-0b7a8f48e6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb6f067efc384d17bfbeedb71258d5aa)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:25.429540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657aa9e6-d4d5-4619-bdf5-0b7a8f48e6cd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:25.477716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:25.485331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:25.497276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d5c5d48a2d4340a823d19fd3145276'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.505075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-657aa9e6-d4d5-4619-bdf5-0b7a8f48e6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb6f067efc384d17bfbeedb71258d5aa'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.518386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d5c5d48a2d4340a823d19fd3145276'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.519308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d5c5d48a2d4340a823d19fd3145276'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.523659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-657aa9e6-d4d5-4619-bdf5-0b7a8f48e6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb6f067efc384d17bfbeedb71258d5aa'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.536293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-657aa9e6-d4d5-4619-bdf5-0b7a8f48e6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb6f067efc384d17bfbeedb71258d5aa'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.537556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-657aa9e6-d4d5-4619-bdf5-0b7a8f48e6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb6f067efc384d17bfbeedb71258d5aa'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.539113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d5c5d48a2d4340a823d19fd3145276'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.539426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-657aa9e6-d4d5-4619-bdf5-0b7a8f48e6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb6f067efc384d17bfbeedb71258d5aa'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.562705 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2868/5662] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:25 2022] DELETE /identity/v3/users/eb6f067efc384d17bfbeedb71258d5aa => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:25.565302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585483e-45dc-4959-8e43-7cc06b069632 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:25.566032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585483e-45dc-4959-8e43-7cc06b069632 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:25.566527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585483e-45dc-4959-8e43-7cc06b069632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:25.578780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d5c5d48a2d4340a823d19fd3145276'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:25.603956 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2795/5663] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:25 2022] DELETE /identity/v3/users/12d5c5d48a2d4340a823d19fd3145276 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:25.604394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3585483e-45dc-4959-8e43-7cc06b069632 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:25.606028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3585483e-45dc-4959-8e43-7cc06b069632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:25.709518 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2869/5664] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:25.715170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:25.718834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:25.740171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a11ac-79ff-4460-ae37-3fef6c8d3403 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:25.746260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55dcfd-486e-4231-8cfc-d6f935998b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:25.746897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55dcfd-486e-4231-8cfc-d6f935998b7e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:25.747203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55dcfd-486e-4231-8cfc-d6f935998b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:25.747482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55dcfd-486e-4231-8cfc-d6f935998b7e admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:25.748148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55dcfd-486e-4231-8cfc-d6f935998b7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:25.749912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55dcfd-486e-4231-8cfc-d6f935998b7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:25.755896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55dcfd-486e-4231-8cfc-d6f935998b7e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:25.777681 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2796/5665] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:25.780530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585483e-45dc-4959-8e43-7cc06b069632 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:25.794971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585483e-45dc-4959-8e43-7cc06b069632 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:25.803089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd85731-5b34-43de-8982-8fb60c6c4789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:25.804163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd85731-5b34-43de-8982-8fb60c6c4789 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:25.805500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd85731-5b34-43de-8982-8fb60c6c4789 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:25.806045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd85731-5b34-43de-8982-8fb60c6c4789 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:25.807024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd85731-5b34-43de-8982-8fb60c6c4789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:25.809300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd85731-5b34-43de-8982-8fb60c6c4789 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:25.864935 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2870/5666] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:25 2022] POST /identity/v3/projects => generated 365 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:25.866308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55dcfd-486e-4231-8cfc-d6f935998b7e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:25.879092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55dcfd-486e-4231-8cfc-d6f935998b7e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:25.952619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858fef5-3b31-42ec-9894-b44cf7dfd2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:25.953417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858fef5-3b31-42ec-9894-b44cf7dfd2ca admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:25.953713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858fef5-3b31-42ec-9894-b44cf7dfd2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:25.953994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858fef5-3b31-42ec-9894-b44cf7dfd2ca admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:25.954643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858fef5-3b31-42ec-9894-b44cf7dfd2ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:25.957689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858fef5-3b31-42ec-9894-b44cf7dfd2ca admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:25.999989 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2797/5667] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:25 2022] POST /identity/v3/users => generated 283 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:26.002506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd85731-5b34-43de-8982-8fb60c6c4789 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:26.020185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd85731-5b34-43de-8982-8fb60c6c4789 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:26.029223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858fef5-3b31-42ec-9894-b44cf7dfd2ca admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:26.031814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8858fef5-3b31-42ec-9894-b44cf7dfd2ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:26.042455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ac500-8862-4612-9d11-63cf7e46d742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:26.043875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ac500-8862-4612-9d11-63cf7e46d742 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:26.045766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ac500-8862-4612-9d11-63cf7e46d742 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:26.045766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ac500-8862-4612-9d11-63cf7e46d742 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:26.059796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ac500-8862-4612-9d11-63cf7e46d742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:26.059796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ac500-8862-4612-9d11-63cf7e46d742 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:26.080922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858fef5-3b31-42ec-9894-b44cf7dfd2ca admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:26.080922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:26.080922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:26.080922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:26.081533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] PATH_INFO: `/v3/projects/830f78404ffc4b7ab6ea476e6a46fc5c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:26.081533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] RBAC: Authorizing `identity:delete_project(project_id=830f78404ffc4b7ab6ea476e6a46fc5c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:26.083558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:26.090976 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2871/5668] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:26.093719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ac500-8862-4612-9d11-63cf7e46d742 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:26.097161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-788ac500-8862-4612-9d11-63cf7e46d742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:26.158576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ac500-8862-4612-9d11-63cf7e46d742 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:26.159603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830f78404ffc4b7ab6ea476e6a46fc5c', 'request_id': 'req-3750daf2-1cd7-4cc3-a5d8-2641e555388d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.172167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830f78404ffc4b7ab6ea476e6a46fc5c', 'request_id': 'req-3750daf2-1cd7-4cc3-a5d8-2641e555388d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.180507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830f78404ffc4b7ab6ea476e6a46fc5c', 'request_id': 'req-3750daf2-1cd7-4cc3-a5d8-2641e555388d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.198625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] The token cache is being invalidate because project 830f78404ffc4b7ab6ea476e6a46fc5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:26.199173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.199984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.206586 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2798/5669] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:26 2022] DELETE /identity/v3/projects/830f78404ffc4b7ab6ea476e6a46fc5c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:26.207596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:26.209313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:26.224365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:26.242509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:26.243327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:26.243714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:26.244101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] PATH_INFO: `/v3/projects/79004e1ade2f43df828d0d8eab4306ef` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:26.245573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79004e1ade2f43df828d0d8eab4306ef)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:26.248151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:26.277467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:26.304401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750daf2-1cd7-4cc3-a5d8-2641e555388d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:26.314909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f2619-d086-4c48-ae72-252aea05f2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:26.315855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f2619-d086-4c48-ae72-252aea05f2e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:26.316624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f2619-d086-4c48-ae72-252aea05f2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:26.317728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f2619-d086-4c48-ae72-252aea05f2e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:26.318694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f2619-d086-4c48-ae72-252aea05f2e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:26.323150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f2619-d086-4c48-ae72-252aea05f2e1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:26.332069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79004e1ade2f43df828d0d8eab4306ef', 'request_id': 'req-b5a2543d-1076-4595-93e0-bd3ed35e1333'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.341179 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2799/5670] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:26.341179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa3b5d-6485-4a1a-aa17-2859be7b1e26 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:26.341179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa3b5d-6485-4a1a-aa17-2859be7b1e26 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:26.341179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa3b5d-6485-4a1a-aa17-2859be7b1e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:26.348169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79004e1ade2f43df828d0d8eab4306ef', 'request_id': 'req-b5a2543d-1076-4595-93e0-bd3ed35e1333'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.360601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79004e1ade2f43df828d0d8eab4306ef', 'request_id': 'req-b5a2543d-1076-4595-93e0-bd3ed35e1333'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.370694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-38aa3b5d-6485-4a1a-aa17-2859be7b1e26 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:26.371765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-38aa3b5d-6485-4a1a-aa17-2859be7b1e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:26.378111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] The token cache is being invalidate because project 79004e1ade2f43df828d0d8eab4306ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:26.378401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.379196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.381308 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2872/5671] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:26 2022] DELETE /identity/v3/projects/79004e1ade2f43df828d0d8eab4306ef => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:26.382001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:26.383480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:26.482738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2543d-1076-4595-93e0-bd3ed35e1333 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:26.488202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42de1e-3f7f-4163-a567-5eb4aeb20297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:26.488881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42de1e-3f7f-4163-a567-5eb4aeb20297 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:26.489949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42de1e-3f7f-4163-a567-5eb4aeb20297 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:26.490254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42de1e-3f7f-4163-a567-5eb4aeb20297 admin admin] PATH_INFO: `/v3/projects/1b92867441be43dda95548932111025b/users/3fe763f5a47a4844a23ad64316099db4/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:26.512673 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2800/5672] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:26.514170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa3b5d-6485-4a1a-aa17-2859be7b1e26 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:26.515722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-38aa3b5d-6485-4a1a-aa17-2859be7b1e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:26.523307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42de1e-3f7f-4163-a567-5eb4aeb20297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b92867441be43dda95548932111025b, user_id=3fe763f5a47a4844a23ad64316099db4, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:26.526619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42de1e-3f7f-4163-a567-5eb4aeb20297 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:26.533591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa3b5d-6485-4a1a-aa17-2859be7b1e26 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:26.546431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2504e9-9a25-4e5a-8e72-78613b78c893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:26.547094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2504e9-9a25-4e5a-8e72-78613b78c893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:26.547583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2504e9-9a25-4e5a-8e72-78613b78c893 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:26.547875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2504e9-9a25-4e5a-8e72-78613b78c893 admin admin] PATH_INFO: `/v3/users/fd61a3d45321405ea295b6cfbc52359d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:26.552976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2504e9-9a25-4e5a-8e72-78613b78c893 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd61a3d45321405ea295b6cfbc52359d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:26.554957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2504e9-9a25-4e5a-8e72-78613b78c893 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:26.557970 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2873/5673] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:26 2022] PUT /identity/v3/projects/1b92867441be43dda95548932111025b/users/3fe763f5a47a4844a23ad64316099db4/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:26.561193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4db4b5-178c-4903-8f18-5d713ba4ecdc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:26.563386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4db4b5-178c-4903-8f18-5d713ba4ecdc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:26.563852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4db4b5-178c-4903-8f18-5d713ba4ecdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:26.605278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2b4db4b5-178c-4903-8f18-5d713ba4ecdc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:26.606591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b4db4b5-178c-4903-8f18-5d713ba4ecdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:26.666656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f2504e9-9a25-4e5a-8e72-78613b78c893 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd61a3d45321405ea295b6cfbc52359d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.709793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f2504e9-9a25-4e5a-8e72-78613b78c893 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd61a3d45321405ea295b6cfbc52359d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.710875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f2504e9-9a25-4e5a-8e72-78613b78c893 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd61a3d45321405ea295b6cfbc52359d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.727080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f2504e9-9a25-4e5a-8e72-78613b78c893 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd61a3d45321405ea295b6cfbc52359d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.750981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f2504e9-9a25-4e5a-8e72-78613b78c893 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd61a3d45321405ea295b6cfbc52359d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:26.758249 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2874/5674] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:26.758249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b16db-6328-4fbc-9df3-8a59f0b42697 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:26.758249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b16db-6328-4fbc-9df3-8a59f0b42697 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:26.758249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b16db-6328-4fbc-9df3-8a59f0b42697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:26.775344 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2801/5675] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:26 2022] DELETE /identity/v3/users/fd61a3d45321405ea295b6cfbc52359d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:26.781139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee7783-d650-492d-9342-e0c023e20b09 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:26.781625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee7783-d650-492d-9342-e0c023e20b09 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:26.782239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee7783-d650-492d-9342-e0c023e20b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:26.810577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a92b16db-6328-4fbc-9df3-8a59f0b42697 None None] MFA Rules not processed for user `3fe763f5a47a4844a23ad64316099db4`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:26.812260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a92b16db-6328-4fbc-9df3-8a59f0b42697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:26.824665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8fee7783-d650-492d-9342-e0c023e20b09 None None] MFA Rules not processed for user `38dddfbdc4f0450e8d5fc7b214796758`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:26.826133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8fee7783-d650-492d-9342-e0c023e20b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:26.839875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:26.855438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:26.912357 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2875/5676] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:26.913400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b16db-6328-4fbc-9df3-8a59f0b42697 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:26.951045 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2802/5677] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:26.953907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee7783-d650-492d-9342-e0c023e20b09 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:26.960426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8fee7783-d650-492d-9342-e0c023e20b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:27.038897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b16db-6328-4fbc-9df3-8a59f0b42697 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.051618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fec20c-0a8f-4d06-800a-64c75deca25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.052279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fec20c-0a8f-4d06-800a-64c75deca25f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.054842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fec20c-0a8f-4d06-800a-64c75deca25f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.055130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fec20c-0a8f-4d06-800a-64c75deca25f admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.055849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fec20c-0a8f-4d06-800a-64c75deca25f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:27.056350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fec20c-0a8f-4d06-800a-64c75deca25f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.059464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fec20c-0a8f-4d06-800a-64c75deca25f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.075456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee7783-d650-492d-9342-e0c023e20b09 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.080483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539774ff-0186-472e-947f-d5aacd2c530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.081359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539774ff-0186-472e-947f-d5aacd2c530b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.081729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539774ff-0186-472e-947f-d5aacd2c530b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.082110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539774ff-0186-472e-947f-d5aacd2c530b admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.082816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539774ff-0186-472e-947f-d5aacd2c530b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.085156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539774ff-0186-472e-947f-d5aacd2c530b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.087254 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2876/5678] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:27.088145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fec20c-0a8f-4d06-800a-64c75deca25f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.091518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-28fec20c-0a8f-4d06-800a-64c75deca25f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:27.160469 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2803/5679] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:26 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:27.162299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539774ff-0186-472e-947f-d5aacd2c530b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.164629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-539774ff-0186-472e-947f-d5aacd2c530b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:27.233254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fec20c-0a8f-4d06-800a-64c75deca25f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.246455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539774ff-0186-472e-947f-d5aacd2c530b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.251575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.252346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.252696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.253081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] PATH_INFO: `/v3/projects/0aa5975a7d6c47bcb33dfa40ea68f34f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.254395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aa5975a7d6c47bcb33dfa40ea68f34f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.256330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.287977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:27.301946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:27.307077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c31c9a-e92b-4c93-9a75-73526f6e9226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.307727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c31c9a-e92b-4c93-9a75-73526f6e9226 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.308017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c31c9a-e92b-4c93-9a75-73526f6e9226 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.308309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c31c9a-e92b-4c93-9a75-73526f6e9226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.310964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aa5975a7d6c47bcb33dfa40ea68f34f', 'request_id': 'req-906330e2-903f-4123-a675-6604437abed1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:27.321854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aa5975a7d6c47bcb33dfa40ea68f34f', 'request_id': 'req-906330e2-903f-4123-a675-6604437abed1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:27.328502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aa5975a7d6c47bcb33dfa40ea68f34f', 'request_id': 'req-906330e2-903f-4123-a675-6604437abed1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:27.341435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] The token cache is being invalidate because project 0aa5975a7d6c47bcb33dfa40ea68f34f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:27.341753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:27.342568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:27.344113 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2804/5680] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:27 2022] DELETE /identity/v3/projects/0aa5975a7d6c47bcb33dfa40ea68f34f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:27.345323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.347647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:27.377373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c31c9a-e92b-4c93-9a75-73526f6e9226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.379283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c31c9a-e92b-4c93-9a75-73526f6e9226 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.389984 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2877/5681] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:27.390908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c31c9a-e92b-4c93-9a75-73526f6e9226 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.405520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c31c9a-e92b-4c93-9a75-73526f6e9226 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.420337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906330e2-903f-4123-a675-6604437abed1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.426808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6124f1-826a-4940-99cb-e00f401b3e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.427443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6124f1-826a-4940-99cb-e00f401b3e50 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.427734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6124f1-826a-4940-99cb-e00f401b3e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.428018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6124f1-826a-4940-99cb-e00f401b3e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.428671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6124f1-826a-4940-99cb-e00f401b3e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.430660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6124f1-826a-4940-99cb-e00f401b3e50 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.468439 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2805/5682] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:27 2022] POST /identity/v3/projects => generated 389 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:27.471511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d645758-161d-47bb-afee-5042a0a03c8e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.472046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d645758-161d-47bb-afee-5042a0a03c8e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.472529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d645758-161d-47bb-afee-5042a0a03c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.493032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fa520-e725-43c5-a341-2f1a57f7bc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.494289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fa520-e725-43c5-a341-2f1a57f7bc52 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.494750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fa520-e725-43c5-a341-2f1a57f7bc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.495138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2d645758-161d-47bb-afee-5042a0a03c8e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:27.495674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fa520-e725-43c5-a341-2f1a57f7bc52 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.496437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2d645758-161d-47bb-afee-5042a0a03c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:27.497031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fa520-e725-43c5-a341-2f1a57f7bc52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.499680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fa520-e725-43c5-a341-2f1a57f7bc52 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.536989 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2878/5683] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:27 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:27.536989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fa520-e725-43c5-a341-2f1a57f7bc52 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.547442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fa520-e725-43c5-a341-2f1a57f7bc52 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.576959 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2806/5684] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:27.585053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d645758-161d-47bb-afee-5042a0a03c8e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.585053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2d645758-161d-47bb-afee-5042a0a03c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:27.592567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d645758-161d-47bb-afee-5042a0a03c8e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.597471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17942c-9acd-41e6-b9b0-27f661979dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.598089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17942c-9acd-41e6-b9b0-27f661979dbe admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.598379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17942c-9acd-41e6-b9b0-27f661979dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.598667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17942c-9acd-41e6-b9b0-27f661979dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.599320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17942c-9acd-41e6-b9b0-27f661979dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.601843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17942c-9acd-41e6-b9b0-27f661979dbe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.616524 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2807/5685] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:27.616524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2e118-9902-43a0-9088-820c5eae64c3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.616524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2e118-9902-43a0-9088-820c5eae64c3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.616524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2e118-9902-43a0-9088-820c5eae64c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.622733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf3d9b-89cb-4ee7-9c90-faf180601b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.623537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf3d9b-89cb-4ee7-9c90-faf180601b59 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.623899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf3d9b-89cb-4ee7-9c90-faf180601b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.633447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf3d9b-89cb-4ee7-9c90-faf180601b59 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.634284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf3d9b-89cb-4ee7-9c90-faf180601b59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.637324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf3d9b-89cb-4ee7-9c90-faf180601b59 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.641412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1ba2e118-9902-43a0-9088-820c5eae64c3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:27.643272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1ba2e118-9902-43a0-9088-820c5eae64c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:27.653058 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2808/5686] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:27.654096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2e118-9902-43a0-9088-820c5eae64c3 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.666127 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2879/5687] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:27 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:27.666642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2e118-9902-43a0-9088-820c5eae64c3 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.667266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf3d9b-89cb-4ee7-9c90-faf180601b59 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.673844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13096f6-63f3-418a-a634-d83de1895a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.674767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13096f6-63f3-418a-a634-d83de1895a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.675112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13096f6-63f3-418a-a634-d83de1895a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.675426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13096f6-63f3-418a-a634-d83de1895a74 admin admin] PATH_INFO: `/v3/projects/20c08bc074844d13bd9f765880b07668/users/b334362f83b44e40a66080bec3131b98/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.680271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf3d9b-89cb-4ee7-9c90-faf180601b59 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.685533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4aaf3-6e8f-439b-a008-a0c832f1c551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.686544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4aaf3-6e8f-439b-a008-a0c832f1c551 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.687326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4aaf3-6e8f-439b-a008-a0c832f1c551 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.687783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4aaf3-6e8f-439b-a008-a0c832f1c551 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.688616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4aaf3-6e8f-439b-a008-a0c832f1c551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:27.689007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4aaf3-6e8f-439b-a008-a0c832f1c551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.691304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4aaf3-6e8f-439b-a008-a0c832f1c551 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.694446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13096f6-63f3-418a-a634-d83de1895a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c08bc074844d13bd9f765880b07668, user_id=b334362f83b44e40a66080bec3131b98, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.697658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13096f6-63f3-418a-a634-d83de1895a74 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.700515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:27.708920 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2880/5688] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:27.710044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4aaf3-6e8f-439b-a008-a0c832f1c551 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.714973 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2809/5689] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:27 2022] PUT /identity/v3/projects/20c08bc074844d13bd9f765880b07668/users/b334362f83b44e40a66080bec3131b98/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:27.716124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13096f6-63f3-418a-a634-d83de1895a74 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.720480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4aaf3-6e8f-439b-a008-a0c832f1c551 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.730248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:27.784523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d66dd8-f89e-404a-bf65-3d3deb71c86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.785270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d66dd8-f89e-404a-bf65-3d3deb71c86b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.785555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d66dd8-f89e-404a-bf65-3d3deb71c86b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.785831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d66dd8-f89e-404a-bf65-3d3deb71c86b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.786504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d66dd8-f89e-404a-bf65-3d3deb71c86b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.788116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d66dd8-f89e-404a-bf65-3d3deb71c86b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.798061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13096f6-63f3-418a-a634-d83de1895a74 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.798588 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2881/5690] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:27.805852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d66dd8-f89e-404a-bf65-3d3deb71c86b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.811867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb185000-e779-4b1e-8174-4e690cb33f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.812745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb185000-e779-4b1e-8174-4e690cb33f86 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.813449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb185000-e779-4b1e-8174-4e690cb33f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.813931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb185000-e779-4b1e-8174-4e690cb33f86 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.814166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d66dd8-f89e-404a-bf65-3d3deb71c86b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.815959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb185000-e779-4b1e-8174-4e690cb33f86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.818503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb185000-e779-4b1e-8174-4e690cb33f86 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.819966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f99ad-7387-4fb8-a84d-8610c8e648ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.820761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f99ad-7387-4fb8-a84d-8610c8e648ae admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.821165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f99ad-7387-4fb8-a84d-8610c8e648ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.821556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f99ad-7387-4fb8-a84d-8610c8e648ae admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.822311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f99ad-7387-4fb8-a84d-8610c8e648ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.824019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f99ad-7387-4fb8-a84d-8610c8e648ae admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.837362 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2882/5691] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:27.841409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f99ad-7387-4fb8-a84d-8610c8e648ae admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.854359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f99ad-7387-4fb8-a84d-8610c8e648ae admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.861392 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2810/5692] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:27 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:27.861695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583189e-dffa-47ae-8a96-eecfcc077283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.862390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583189e-dffa-47ae-8a96-eecfcc077283 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.862696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583189e-dffa-47ae-8a96-eecfcc077283 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.863109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583189e-dffa-47ae-8a96-eecfcc077283 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.863773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583189e-dffa-47ae-8a96-eecfcc077283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.865266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb185000-e779-4b1e-8174-4e690cb33f86 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.866099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583189e-dffa-47ae-8a96-eecfcc077283 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.882056 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2883/5693] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:27.882056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583189e-dffa-47ae-8a96-eecfcc077283 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.890589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583189e-dffa-47ae-8a96-eecfcc077283 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.947797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb185000-e779-4b1e-8174-4e690cb33f86 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:27.953691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957c8a6-e92c-4964-881c-473d06a10f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.954441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957c8a6-e92c-4964-881c-473d06a10f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.957997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957c8a6-e92c-4964-881c-473d06a10f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.957997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957c8a6-e92c-4964-881c-473d06a10f39 admin admin] PATH_INFO: `/v3/projects/20c08bc074844d13bd9f765880b07668/users/b334362f83b44e40a66080bec3131b98/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.957997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957c8a6-e92c-4964-881c-473d06a10f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c08bc074844d13bd9f765880b07668, user_id=b334362f83b44e40a66080bec3131b98, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.960177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957c8a6-e92c-4964-881c-473d06a10f39 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:27.974245 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2811/5694] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:27 2022] PUT /identity/v3/projects/20c08bc074844d13bd9f765880b07668/users/b334362f83b44e40a66080bec3131b98/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:27.975564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957c8a6-e92c-4964-881c-473d06a10f39 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:27.979175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f6f59-9905-435d-a47b-6f394fae51fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:27.980113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f6f59-9905-435d-a47b-6f394fae51fd admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:27.980451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f6f59-9905-435d-a47b-6f394fae51fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:27.980793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f6f59-9905-435d-a47b-6f394fae51fd admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:27.982166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f6f59-9905-435d-a47b-6f394fae51fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:27.984864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f6f59-9905-435d-a47b-6f394fae51fd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:28.012828 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2884/5695] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:27 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:28.014240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27176854-8930-4599-98d0-42bc100c4be0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:28.014527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27176854-8930-4599-98d0-42bc100c4be0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:28.014811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27176854-8930-4599-98d0-42bc100c4be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:28.044611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-27176854-8930-4599-98d0-42bc100c4be0 None None] MFA Rules not processed for user `b334362f83b44e40a66080bec3131b98`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:28.045930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-27176854-8930-4599-98d0-42bc100c4be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:28.053647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957c8a6-e92c-4964-881c-473d06a10f39 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:28.060566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b6228-34ab-41f8-8152-eadac0b2191c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:28.061288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b6228-34ab-41f8-8152-eadac0b2191c admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:28.061875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b6228-34ab-41f8-8152-eadac0b2191c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:28.062156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b6228-34ab-41f8-8152-eadac0b2191c admin admin] PATH_INFO: `/v3/projects/3623dca2492247ea9c881eae91b76964/users/c27f6d52bbb544a2b56e259de1cd6577/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:28.079710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b6228-34ab-41f8-8152-eadac0b2191c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3623dca2492247ea9c881eae91b76964, user_id=c27f6d52bbb544a2b56e259de1cd6577, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:28.085317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b6228-34ab-41f8-8152-eadac0b2191c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:28.114199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:28.114489 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2812/5696] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:27 2022] PUT /identity/v3/projects/3623dca2492247ea9c881eae91b76964/users/c27f6d52bbb544a2b56e259de1cd6577/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:28.115503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b6228-34ab-41f8-8152-eadac0b2191c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:28.132432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:28.169751 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2885/5697] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:28.171459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1bfd8-baf2-473c-91f6-810a93f8b295 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:28.171807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1bfd8-baf2-473c-91f6-810a93f8b295 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:28.172204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1bfd8-baf2-473c-91f6-810a93f8b295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:28.234467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b6228-34ab-41f8-8152-eadac0b2191c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:28.241417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-61c1bfd8-baf2-473c-91f6-810a93f8b295 None None] MFA Rules not processed for user `c27f6d52bbb544a2b56e259de1cd6577`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:28.242805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-61c1bfd8-baf2-473c-91f6-810a93f8b295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:28.244794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0e25c-d375-4487-90e0-e70ab6c492c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:28.245670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0e25c-d375-4487-90e0-e70ab6c492c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:28.245961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0e25c-d375-4487-90e0-e70ab6c492c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:28.246226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0e25c-d375-4487-90e0-e70ab6c492c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:28.246869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0e25c-d375-4487-90e0-e70ab6c492c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:28.248725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0e25c-d375-4487-90e0-e70ab6c492c7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:28.262027 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2813/5698] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:28.263057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0e25c-d375-4487-90e0-e70ab6c492c7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:28.264868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8ed0e25c-d375-4487-90e0-e70ab6c492c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:28.323776 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2886/5699] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:28.327695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1bfd8-baf2-473c-91f6-810a93f8b295 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:28.339679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1bfd8-baf2-473c-91f6-810a93f8b295 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:28.341180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0e25c-d375-4487-90e0-e70ab6c492c7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:28.344642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3e66e-0574-4f8b-a426-9aacfc165cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:28.345316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3e66e-0574-4f8b-a426-9aacfc165cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:28.346466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3e66e-0574-4f8b-a426-9aacfc165cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:28.346466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3e66e-0574-4f8b-a426-9aacfc165cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:28.346991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3e66e-0574-4f8b-a426-9aacfc165cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:28.347294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6645fc-c33f-4723-9222-d0086e7e63ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:28.348037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6645fc-c33f-4723-9222-d0086e7e63ae service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:28.348426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6645fc-c33f-4723-9222-d0086e7e63ae service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:28.348602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3e66e-0574-4f8b-a426-9aacfc165cf0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:28.349033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6645fc-c33f-4723-9222-d0086e7e63ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:28.359614 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2887/5700] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:28.360267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3e66e-0574-4f8b-a426-9aacfc165cf0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:28.374862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3e66e-0574-4f8b-a426-9aacfc165cf0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:28.379328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd7375-33c6-425b-a571-fec5c149abb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:28.380535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd7375-33c6-425b-a571-fec5c149abb2 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:28.380819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd7375-33c6-425b-a571-fec5c149abb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:28.381133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd7375-33c6-425b-a571-fec5c149abb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:28.394801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd7375-33c6-425b-a571-fec5c149abb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:28.396589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd7375-33c6-425b-a571-fec5c149abb2 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:28.408760 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2888/5701] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:28.409577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd7375-33c6-425b-a571-fec5c149abb2 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:28.439446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd7375-33c6-425b-a571-fec5c149abb2 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:28.445002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6645fc-c33f-4723-9222-d0086e7e63ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:28.447298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6645fc-c33f-4723-9222-d0086e7e63ae service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:28.450393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62328a67-c3ca-4e4d-aeaf-8550528a6b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:28.451131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62328a67-c3ca-4e4d-aeaf-8550528a6b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:28.455064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62328a67-c3ca-4e4d-aeaf-8550528a6b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:28.455364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62328a67-c3ca-4e4d-aeaf-8550528a6b04 admin admin] PATH_INFO: `/v3/projects/118fc22be845490aa23fa6d36fcecaa1/users/4cdce2ff71304bec923102d5c879da5d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:28.461907 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2814/5702] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:28.495911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62328a67-c3ca-4e4d-aeaf-8550528a6b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=118fc22be845490aa23fa6d36fcecaa1, user_id=4cdce2ff71304bec923102d5c879da5d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:28.500390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62328a67-c3ca-4e4d-aeaf-8550528a6b04 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:28.534375 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2889/5703] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:28 2022] PUT /identity/v3/projects/118fc22be845490aa23fa6d36fcecaa1/users/4cdce2ff71304bec923102d5c879da5d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:28.544591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166de9b-952e-4e41-903e-c88b6b883e04 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:28.545197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166de9b-952e-4e41-903e-c88b6b883e04 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:28.545494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166de9b-952e-4e41-903e-c88b6b883e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:28.581178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0166de9b-952e-4e41-903e-c88b6b883e04 None None] MFA Rules not processed for user `4cdce2ff71304bec923102d5c879da5d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:28.582434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0166de9b-952e-4e41-903e-c88b6b883e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:28.695196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:28.731206 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2815/5704] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:28.736419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62328a67-c3ca-4e4d-aeaf-8550528a6b04 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:28.867561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62328a67-c3ca-4e4d-aeaf-8550528a6b04 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:28.878888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95940df-8fbf-490c-89a5-da70eb5ff1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:28.880750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95940df-8fbf-490c-89a5-da70eb5ff1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:28.881726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95940df-8fbf-490c-89a5-da70eb5ff1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:28.882026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95940df-8fbf-490c-89a5-da70eb5ff1c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:28.884930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95940df-8fbf-490c-89a5-da70eb5ff1c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:28.891530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95940df-8fbf-490c-89a5-da70eb5ff1c5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:28.957295 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2890/5705] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:28 2022] POST /identity/v3/projects => generated 387 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:28.961131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166de9b-952e-4e41-903e-c88b6b883e04 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:28.976805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166de9b-952e-4e41-903e-c88b6b883e04 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:29.060209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551f6af-8242-4805-89e1-ebf1025c65d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:29.061177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551f6af-8242-4805-89e1-ebf1025c65d2 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:29.061462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551f6af-8242-4805-89e1-ebf1025c65d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:29.061738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551f6af-8242-4805-89e1-ebf1025c65d2 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:29.062397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551f6af-8242-4805-89e1-ebf1025c65d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:29.066078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551f6af-8242-4805-89e1-ebf1025c65d2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:29.098717 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2816/5706] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:28 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:29.102861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95940df-8fbf-490c-89a5-da70eb5ff1c5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:29.114026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95940df-8fbf-490c-89a5-da70eb5ff1c5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:29.119771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac2f6a-8932-4167-b893-0daf5111b609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:29.120406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac2f6a-8932-4167-b893-0daf5111b609 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:29.120695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac2f6a-8932-4167-b893-0daf5111b609 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:29.121011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac2f6a-8932-4167-b893-0daf5111b609 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:29.121670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac2f6a-8932-4167-b893-0daf5111b609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:29.123881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac2f6a-8932-4167-b893-0daf5111b609 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:29.154150 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2891/5707] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:29.155061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551f6af-8242-4805-89e1-ebf1025c65d2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:29.171623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551f6af-8242-4805-89e1-ebf1025c65d2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:29.177412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f3f46-8eb2-4fc2-b23f-2e8986a29b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:29.177799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f3f46-8eb2-4fc2-b23f-2e8986a29b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:29.178232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f3f46-8eb2-4fc2-b23f-2e8986a29b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:29.178656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f3f46-8eb2-4fc2-b23f-2e8986a29b4e admin admin] PATH_INFO: `/v3/projects/eb554ecbb1cd46f099e2df146dd745de/users/9dd96fce0bdb48b490e890a4f455daa4/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:29.201443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f3f46-8eb2-4fc2-b23f-2e8986a29b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb554ecbb1cd46f099e2df146dd745de, user_id=9dd96fce0bdb48b490e890a4f455daa4, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:29.204731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f3f46-8eb2-4fc2-b23f-2e8986a29b4e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:29.235243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac2f6a-8932-4167-b893-0daf5111b609 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:29.241168 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2817/5708] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:29 2022] PUT /identity/v3/projects/eb554ecbb1cd46f099e2df146dd745de/users/9dd96fce0bdb48b490e890a4f455daa4/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:29.241565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:29.248493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac2f6a-8932-4167-b893-0daf5111b609 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:29.331856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccba084-c7ff-4603-bc24-63d816642707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:29.332490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccba084-c7ff-4603-bc24-63d816642707 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:29.332979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccba084-c7ff-4603-bc24-63d816642707 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:29.333270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccba084-c7ff-4603-bc24-63d816642707 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:29.333927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccba084-c7ff-4603-bc24-63d816642707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:29.335785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccba084-c7ff-4603-bc24-63d816642707 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:29.353168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f3f46-8eb2-4fc2-b23f-2e8986a29b4e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:29.364204 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2892/5709] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:29.367838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f3f46-8eb2-4fc2-b23f-2e8986a29b4e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:29.374490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dcea1-f399-4728-94fd-f3b417c078eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:29.375588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dcea1-f399-4728-94fd-f3b417c078eb admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:29.376155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dcea1-f399-4728-94fd-f3b417c078eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:29.377404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dcea1-f399-4728-94fd-f3b417c078eb admin admin] PATH_INFO: `/v3/projects/eb554ecbb1cd46f099e2df146dd745de/users/9dd96fce0bdb48b490e890a4f455daa4/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:29.379728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dcea1-f399-4728-94fd-f3b417c078eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb554ecbb1cd46f099e2df146dd745de, user_id=9dd96fce0bdb48b490e890a4f455daa4, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:29.382765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dcea1-f399-4728-94fd-f3b417c078eb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:29.405985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92551df4-3cb8-410e-97d2-2238de4891f0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:29.405985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92551df4-3cb8-410e-97d2-2238de4891f0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:29.406387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92551df4-3cb8-410e-97d2-2238de4891f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:29.407359 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2818/5710] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:29 2022] PUT /identity/v3/projects/eb554ecbb1cd46f099e2df146dd745de/users/9dd96fce0bdb48b490e890a4f455daa4/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:29.437201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-92551df4-3cb8-410e-97d2-2238de4891f0 None None] MFA Rules not processed for user `9dd96fce0bdb48b490e890a4f455daa4`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:29.438754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-92551df4-3cb8-410e-97d2-2238de4891f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:29.515412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef499f3-fe98-4b51-9984-0e2123d27119 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:29.515710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef499f3-fe98-4b51-9984-0e2123d27119 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:29.515995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef499f3-fe98-4b51-9984-0e2123d27119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:29.546571 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2893/5711] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:29.551370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3ef499f3-fe98-4b51-9984-0e2123d27119 None None] MFA Rules not processed for user `38dddfbdc4f0450e8d5fc7b214796758`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:29.552965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3ef499f3-fe98-4b51-9984-0e2123d27119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:29.559209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92551df4-3cb8-410e-97d2-2238de4891f0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:29.561861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-92551df4-3cb8-410e-97d2-2238de4891f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:29.660798 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2819/5712] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:29.702960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92551df4-3cb8-410e-97d2-2238de4891f0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:29.714632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cb247-0f71-4cc0-8aec-75c8648a08a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:29.715244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cb247-0f71-4cc0-8aec-75c8648a08a9 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:29.715532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cb247-0f71-4cc0-8aec-75c8648a08a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:29.715822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cb247-0f71-4cc0-8aec-75c8648a08a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:29.738229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7cb247-0f71-4cc0-8aec-75c8648a08a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:29.743492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7cb247-0f71-4cc0-8aec-75c8648a08a9 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:29.761292 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2894/5713] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:29.959887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef499f3-fe98-4b51-9984-0e2123d27119 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:30.035724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:30.060421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef499f3-fe98-4b51-9984-0e2123d27119 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:30.066146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:30.067121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:30.067627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:30.068561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] PATH_INFO: `/v3/users/4cdce2ff71304bec923102d5c879da5d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:30.070755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cdce2ff71304bec923102d5c879da5d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:30.073397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:30.182175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cdce2ff71304bec923102d5c879da5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:30.206844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cdce2ff71304bec923102d5c879da5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:30.207630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cdce2ff71304bec923102d5c879da5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:30.218759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cdce2ff71304bec923102d5c879da5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:30.232474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cdce2ff71304bec923102d5c879da5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:30.244033 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2820/5714] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:29 2022] DELETE /identity/v3/users/4cdce2ff71304bec923102d5c879da5d => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:30.249680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0689376-89f0-4019-b58e-e9c283dc50a1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:30.250265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0689376-89f0-4019-b58e-e9c283dc50a1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:30.250381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0689376-89f0-4019-b58e-e9c283dc50a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:30.273715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a0689376-89f0-4019-b58e-e9c283dc50a1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:30.274870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a0689376-89f0-4019-b58e-e9c283dc50a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:30.292221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:30.416886 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2895/5715] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:30.459445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:30.461051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:30.596638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3557dc-797b-4ac6-9e9d-ce6c6ad284f7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:30.607214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36a74-7b54-46cb-b341-d22f3dc13992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:30.607870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36a74-7b54-46cb-b341-d22f3dc13992 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:30.608160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36a74-7b54-46cb-b341-d22f3dc13992 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:30.608446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36a74-7b54-46cb-b341-d22f3dc13992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:30.634705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36a74-7b54-46cb-b341-d22f3dc13992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:30.636401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36a74-7b54-46cb-b341-d22f3dc13992 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:30.657670 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2821/5716] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:30.897956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0689376-89f0-4019-b58e-e9c283dc50a1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:30.899888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a0689376-89f0-4019-b58e-e9c283dc50a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:31.022628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0689376-89f0-4019-b58e-e9c283dc50a1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:31.029047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b257b-1ba1-49db-8488-2a9bf96cfbf6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:31.030025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b257b-1ba1-49db-8488-2a9bf96cfbf6 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:31.030461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b257b-1ba1-49db-8488-2a9bf96cfbf6 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:31.031041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b257b-1ba1-49db-8488-2a9bf96cfbf6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:31.033179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-417b257b-1ba1-49db-8488-2a9bf96cfbf6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:31.077730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b257b-1ba1-49db-8488-2a9bf96cfbf6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:31.079522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b257b-1ba1-49db-8488-2a9bf96cfbf6 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:31.101556 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2896/5717] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:31.916634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36a74-7b54-46cb-b341-d22f3dc13992 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:31.918183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c4d36a74-7b54-46cb-b341-d22f3dc13992 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:31.940717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36a74-7b54-46cb-b341-d22f3dc13992 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:31.951776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:31.952722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:31.953534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:31.954042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] PATH_INFO: `/v3/projects/118fc22be845490aa23fa6d36fcecaa1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:31.969802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=118fc22be845490aa23fa6d36fcecaa1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:31.977464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:32.000884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b257b-1ba1-49db-8488-2a9bf96cfbf6 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:32.002866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-417b257b-1ba1-49db-8488-2a9bf96cfbf6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:32.024567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b257b-1ba1-49db-8488-2a9bf96cfbf6 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:32.037024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fbcb5-a2d4-40d0-90c3-798816359f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:32.038216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fbcb5-a2d4-40d0-90c3-798816359f09 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:32.038535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fbcb5-a2d4-40d0-90c3-798816359f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:32.038916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fbcb5-a2d4-40d0-90c3-798816359f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:32.041469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8c3fbcb5-a2d4-40d0-90c3-798816359f09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:32.101563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '118fc22be845490aa23fa6d36fcecaa1', 'request_id': 'req-8f68f866-bbe6-4607-8779-84f58907b286'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:32.118999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '118fc22be845490aa23fa6d36fcecaa1', 'request_id': 'req-8f68f866-bbe6-4607-8779-84f58907b286'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:32.135695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '118fc22be845490aa23fa6d36fcecaa1', 'request_id': 'req-8f68f866-bbe6-4607-8779-84f58907b286'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:32.139821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:32.166518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] The token cache is being invalidate because project 118fc22be845490aa23fa6d36fcecaa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:32.177192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:32.178045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:32.179917 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2822/5718] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:31 2022] DELETE /identity/v3/projects/118fc22be845490aa23fa6d36fcecaa1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:32.183696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:32.185264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fbcb5-a2d4-40d0-90c3-798816359f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:32.186701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fbcb5-a2d4-40d0-90c3-798816359f09 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:32.187471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:32.218356 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2897/5719] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:32.348838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68f866-bbe6-4607-8779-84f58907b286 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:32.368098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d597f1-2c67-4fb1-b0d4-6c9ba99bb50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:32.369647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d597f1-2c67-4fb1-b0d4-6c9ba99bb50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:32.369647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d597f1-2c67-4fb1-b0d4-6c9ba99bb50c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:32.369647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d597f1-2c67-4fb1-b0d4-6c9ba99bb50c admin admin] PATH_INFO: `/v3/users/9dd96fce0bdb48b490e890a4f455daa4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:32.373337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d597f1-2c67-4fb1-b0d4-6c9ba99bb50c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dd96fce0bdb48b490e890a4f455daa4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:32.375369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d597f1-2c67-4fb1-b0d4-6c9ba99bb50c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:32.459445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-67d597f1-2c67-4fb1-b0d4-6c9ba99bb50c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dd96fce0bdb48b490e890a4f455daa4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:32.475692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-67d597f1-2c67-4fb1-b0d4-6c9ba99bb50c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dd96fce0bdb48b490e890a4f455daa4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:32.476431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-67d597f1-2c67-4fb1-b0d4-6c9ba99bb50c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dd96fce0bdb48b490e890a4f455daa4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:32.484648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-67d597f1-2c67-4fb1-b0d4-6c9ba99bb50c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dd96fce0bdb48b490e890a4f455daa4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:32.503966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-67d597f1-2c67-4fb1-b0d4-6c9ba99bb50c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dd96fce0bdb48b490e890a4f455daa4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:32.521672 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2823/5720] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:32 2022] DELETE /identity/v3/users/9dd96fce0bdb48b490e890a4f455daa4 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:33.570670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fbcb5-a2d4-40d0-90c3-798816359f09 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:33.595714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fbcb5-a2d4-40d0-90c3-798816359f09 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:33.655736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:33.688479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680344be-95d5-4430-a0c8-eb62b6627844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:33.691246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680344be-95d5-4430-a0c8-eb62b6627844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:33.691543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680344be-95d5-4430-a0c8-eb62b6627844 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:33.691830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680344be-95d5-4430-a0c8-eb62b6627844 admin admin] PATH_INFO: `/v3/projects/eb554ecbb1cd46f099e2df146dd745de` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:33.693167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680344be-95d5-4430-a0c8-eb62b6627844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb554ecbb1cd46f099e2df146dd745de)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:33.695275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680344be-95d5-4430-a0c8-eb62b6627844 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:33.779843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-680344be-95d5-4430-a0c8-eb62b6627844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb554ecbb1cd46f099e2df146dd745de', 'request_id': 'req-680344be-95d5-4430-a0c8-eb62b6627844'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:33.792619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-680344be-95d5-4430-a0c8-eb62b6627844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb554ecbb1cd46f099e2df146dd745de', 'request_id': 'req-680344be-95d5-4430-a0c8-eb62b6627844'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:33.804580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-680344be-95d5-4430-a0c8-eb62b6627844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb554ecbb1cd46f099e2df146dd745de', 'request_id': 'req-680344be-95d5-4430-a0c8-eb62b6627844'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:33.825548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-680344be-95d5-4430-a0c8-eb62b6627844 admin admin] The token cache is being invalidate because project eb554ecbb1cd46f099e2df146dd745de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:33.826014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-680344be-95d5-4430-a0c8-eb62b6627844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:33.827331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-680344be-95d5-4430-a0c8-eb62b6627844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:33.833734 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2898/5721] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:33 2022] DELETE /identity/v3/projects/eb554ecbb1cd46f099e2df146dd745de => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:33.835467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b819853-c9bf-47e8-bb82-3a2c27b55fec None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:33.835978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b819853-c9bf-47e8-bb82-3a2c27b55fec None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:33.836067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b819853-c9bf-47e8-bb82-3a2c27b55fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:33.862255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3b819853-c9bf-47e8-bb82-3a2c27b55fec None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:33.863502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3b819853-c9bf-47e8-bb82-3a2c27b55fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:33.954361 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2824/5722] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:33.956779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77071497-a64a-4952-9c8a-7df85f0f1b53 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:33.957121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77071497-a64a-4952-9c8a-7df85f0f1b53 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:33.957406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77071497-a64a-4952-9c8a-7df85f0f1b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:33.979588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-77071497-a64a-4952-9c8a-7df85f0f1b53 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:33.980870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-77071497-a64a-4952-9c8a-7df85f0f1b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:33.989745 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2899/5723] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:33.994474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b819853-c9bf-47e8-bb82-3a2c27b55fec None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:34.005748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b819853-c9bf-47e8-bb82-3a2c27b55fec None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:34.011207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb39df-a7a0-48a8-8cdf-8260e088ce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:34.011467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb39df-a7a0-48a8-8cdf-8260e088ce6d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.011723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb39df-a7a0-48a8-8cdf-8260e088ce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.011898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb39df-a7a0-48a8-8cdf-8260e088ce6d admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.012569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb39df-a7a0-48a8-8cdf-8260e088ce6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:34.012876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb39df-a7a0-48a8-8cdf-8260e088ce6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:34.014911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb39df-a7a0-48a8-8cdf-8260e088ce6d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:34.023287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:34.035392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77071497-a64a-4952-9c8a-7df85f0f1b53 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:34.042364 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2825/5724] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:34.047302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77071497-a64a-4952-9c8a-7df85f0f1b53 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:34.053532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795ffc1-36f3-49f9-a002-88d960f4899b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:34.054086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795ffc1-36f3-49f9-a002-88d960f4899b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.054368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795ffc1-36f3-49f9-a002-88d960f4899b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.054641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795ffc1-36f3-49f9-a002-88d960f4899b admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.055286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795ffc1-36f3-49f9-a002-88d960f4899b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:34.057643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795ffc1-36f3-49f9-a002-88d960f4899b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:34.102387 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2900/5725] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:34 2022] POST /identity/v3/projects => generated 379 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:34.103642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb39df-a7a0-48a8-8cdf-8260e088ce6d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:34.116021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb39df-a7a0-48a8-8cdf-8260e088ce6d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:34.199370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe28859-14c8-43e5-8388-ca9b45689df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:34.199995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe28859-14c8-43e5-8388-ca9b45689df3 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.200284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe28859-14c8-43e5-8388-ca9b45689df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.200565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe28859-14c8-43e5-8388-ca9b45689df3 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.201252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe28859-14c8-43e5-8388-ca9b45689df3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:34.203208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe28859-14c8-43e5-8388-ca9b45689df3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:34.229375 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2826/5726] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:34 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:34.230694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795ffc1-36f3-49f9-a002-88d960f4899b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:34.241939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795ffc1-36f3-49f9-a002-88d960f4899b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:34.246614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83bbae-929c-4e52-ab74-082b1b1fc109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:34.247219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83bbae-929c-4e52-ab74-082b1b1fc109 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.247508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83bbae-929c-4e52-ab74-082b1b1fc109 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.247790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83bbae-929c-4e52-ab74-082b1b1fc109 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.248439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83bbae-929c-4e52-ab74-082b1b1fc109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:34.250098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83bbae-929c-4e52-ab74-082b1b1fc109 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:34.261946 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2901/5727] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:34.266286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe28859-14c8-43e5-8388-ca9b45689df3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:34.279607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe28859-14c8-43e5-8388-ca9b45689df3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:34.285731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01b76a-f21a-49bf-8b3b-3c7959acf0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:34.286407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01b76a-f21a-49bf-8b3b-3c7959acf0ec admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.286706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01b76a-f21a-49bf-8b3b-3c7959acf0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.286992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01b76a-f21a-49bf-8b3b-3c7959acf0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.287794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01b76a-f21a-49bf-8b3b-3c7959acf0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:34.289954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01b76a-f21a-49bf-8b3b-3c7959acf0ec admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:34.291705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83bbae-929c-4e52-ab74-082b1b1fc109 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:34.294737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1c83bbae-929c-4e52-ab74-082b1b1fc109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:34.310078 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2827/5728] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:34.310253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83bbae-929c-4e52-ab74-082b1b1fc109 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:34.311342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01b76a-f21a-49bf-8b3b-3c7959acf0ec admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:34.314684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3bdb8-a483-4d2b-9bf9-48faf5131721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:34.315464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3bdb8-a483-4d2b-9bf9-48faf5131721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.315745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3bdb8-a483-4d2b-9bf9-48faf5131721 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.316153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3bdb8-a483-4d2b-9bf9-48faf5131721 admin admin] PATH_INFO: `/v3/users/3fe763f5a47a4844a23ad64316099db4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.317435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3bdb8-a483-4d2b-9bf9-48faf5131721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fe763f5a47a4844a23ad64316099db4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:34.319474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3bdb8-a483-4d2b-9bf9-48faf5131721 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:34.323358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01b76a-f21a-49bf-8b3b-3c7959acf0ec admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:34.329480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23772d-82e1-46ec-840e-33c5f1941c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:34.330339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23772d-82e1-46ec-840e-33c5f1941c39 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.330629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23772d-82e1-46ec-840e-33c5f1941c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.330911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23772d-82e1-46ec-840e-33c5f1941c39 admin admin] PATH_INFO: `/v3/projects/726f04e1d9cd4c73b6ee1a7250cb3e83/users/cf4dfb455208485f9d3e7661ad77bbe5/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.352158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23772d-82e1-46ec-840e-33c5f1941c39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=726f04e1d9cd4c73b6ee1a7250cb3e83, user_id=cf4dfb455208485f9d3e7661ad77bbe5, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:34.355764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23772d-82e1-46ec-840e-33c5f1941c39 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:34.375187 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2828/5729] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:34 2022] PUT /identity/v3/projects/726f04e1d9cd4c73b6ee1a7250cb3e83/users/cf4dfb455208485f9d3e7661ad77bbe5/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:34.384691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb74ca-5c92-434c-9f69-ef343913c433 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.385094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb74ca-5c92-434c-9f69-ef343913c433 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.385437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb74ca-5c92-434c-9f69-ef343913c433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.396744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5d3bdb8-a483-4d2b-9bf9-48faf5131721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fe763f5a47a4844a23ad64316099db4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:34.415745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5d3bdb8-a483-4d2b-9bf9-48faf5131721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fe763f5a47a4844a23ad64316099db4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:34.419402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-adbb74ca-5c92-434c-9f69-ef343913c433 None None] MFA Rules not processed for user `cf4dfb455208485f9d3e7661ad77bbe5`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:34.421590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-adbb74ca-5c92-434c-9f69-ef343913c433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:34.424855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5d3bdb8-a483-4d2b-9bf9-48faf5131721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fe763f5a47a4844a23ad64316099db4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:34.425661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5d3bdb8-a483-4d2b-9bf9-48faf5131721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fe763f5a47a4844a23ad64316099db4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:34.426375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5d3bdb8-a483-4d2b-9bf9-48faf5131721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fe763f5a47a4844a23ad64316099db4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:34.437815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:34.444276 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2902/5730] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:34 2022] DELETE /identity/v3/users/3fe763f5a47a4844a23ad64316099db4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:34.449212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249eed9-fe29-458f-ba5c-d39f8db7f928 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.449794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249eed9-fe29-458f-ba5c-d39f8db7f928 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.451078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249eed9-fe29-458f-ba5c-d39f8db7f928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.477530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e249eed9-fe29-458f-ba5c-d39f8db7f928 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:34.479092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e249eed9-fe29-458f-ba5c-d39f8db7f928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:34.518990 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2829/5731] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:34.520439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb74ca-5c92-434c-9f69-ef343913c433 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:34.522841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-adbb74ca-5c92-434c-9f69-ef343913c433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:34.594588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:34.594928 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2903/5732] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:34.607649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249eed9-fe29-458f-ba5c-d39f8db7f928 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:34.610251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e249eed9-fe29-458f-ba5c-d39f8db7f928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:34.627977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb74ca-5c92-434c-9f69-ef343913c433 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:34.634035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d21097-bd3a-4251-88f0-3ac0c2b58e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:34.634801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d21097-bd3a-4251-88f0-3ac0c2b58e1c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.635098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d21097-bd3a-4251-88f0-3ac0c2b58e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.635448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d21097-bd3a-4251-88f0-3ac0c2b58e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.636360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d21097-bd3a-4251-88f0-3ac0c2b58e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:34.638974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d21097-bd3a-4251-88f0-3ac0c2b58e1c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:34.685913 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2830/5733] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:34 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:34.689640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d21097-bd3a-4251-88f0-3ac0c2b58e1c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:34.704546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249eed9-fe29-458f-ba5c-d39f8db7f928 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:34.764727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2fb9c2-1ab7-4778-9658-144fe185b424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:34.767150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fb9c2-1ab7-4778-9658-144fe185b424 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.768109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fb9c2-1ab7-4778-9658-144fe185b424 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.768426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fb9c2-1ab7-4778-9658-144fe185b424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.806289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d21097-bd3a-4251-88f0-3ac0c2b58e1c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:34.811744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2514bf8-32b0-4831-bb22-9734d9f91587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:34.814426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2514bf8-32b0-4831-bb22-9734d9f91587 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.814426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2514bf8-32b0-4831-bb22-9734d9f91587 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.814426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2514bf8-32b0-4831-bb22-9734d9f91587 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.815564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2514bf8-32b0-4831-bb22-9734d9f91587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:34.818714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2514bf8-32b0-4831-bb22-9734d9f91587 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:34.868550 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2831/5734] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:34 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:34.874696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2514bf8-32b0-4831-bb22-9734d9f91587 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:34.888349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2514bf8-32b0-4831-bb22-9734d9f91587 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:34.890998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fb9c2-1ab7-4778-9658-144fe185b424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:34.894032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fb9c2-1ab7-4778-9658-144fe185b424 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:34.894964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdbec2-f37b-49b0-bb6a-fbe7b6ff5f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:34.895695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdbec2-f37b-49b0-bb6a-fbe7b6ff5f61 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.897131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdbec2-f37b-49b0-bb6a-fbe7b6ff5f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.897131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdbec2-f37b-49b0-bb6a-fbe7b6ff5f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.897131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdbec2-f37b-49b0-bb6a-fbe7b6ff5f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:34.898644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdbec2-f37b-49b0-bb6a-fbe7b6ff5f61 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:34.911696 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2904/5735] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:34.911916 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2832/5736] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:34.916274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2fb9c2-1ab7-4778-9658-144fe185b424 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:34.939592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2fb9c2-1ab7-4778-9658-144fe185b424 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:34.945826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6ff17-0518-4064-94b7-3536db9b1646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:34.947151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6ff17-0518-4064-94b7-3536db9b1646 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:34.947438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6ff17-0518-4064-94b7-3536db9b1646 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:34.947717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6ff17-0518-4064-94b7-3536db9b1646 admin admin] PATH_INFO: `/v3/projects/abd7d66723c44cb1983b0953a83e0377/users/f2828c03d1944bcebe32aedf658c5ca5/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:34.950267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:34.972366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6ff17-0518-4064-94b7-3536db9b1646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd7d66723c44cb1983b0953a83e0377, user_id=f2828c03d1944bcebe32aedf658c5ca5, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:34.976215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6ff17-0518-4064-94b7-3536db9b1646 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:34.999068 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2905/5737] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:34 2022] PUT /identity/v3/projects/abd7d66723c44cb1983b0953a83e0377/users/f2828c03d1944bcebe32aedf658c5ca5/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:35.004158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdbec2-f37b-49b0-bb6a-fbe7b6ff5f61 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:35.026670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdbec2-f37b-49b0-bb6a-fbe7b6ff5f61 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:35.166769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39192d-ae30-4393-9e2d-13bd99ea9dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:35.167022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39192d-ae30-4393-9e2d-13bd99ea9dcf admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:35.167114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39192d-ae30-4393-9e2d-13bd99ea9dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:35.167390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39192d-ae30-4393-9e2d-13bd99ea9dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:35.168048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39192d-ae30-4393-9e2d-13bd99ea9dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:35.169882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39192d-ae30-4393-9e2d-13bd99ea9dcf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:35.181942 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2833/5738] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:35.185554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6ff17-0518-4064-94b7-3536db9b1646 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:35.198205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6ff17-0518-4064-94b7-3536db9b1646 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:35.203928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59620f-7a50-47c8-849b-5f94012d5e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:35.205762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59620f-7a50-47c8-849b-5f94012d5e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:35.206045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59620f-7a50-47c8-849b-5f94012d5e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:35.206321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59620f-7a50-47c8-849b-5f94012d5e39 admin admin] PATH_INFO: `/v3/projects/abd7d66723c44cb1983b0953a83e0377/users/f2828c03d1944bcebe32aedf658c5ca5/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:35.208487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59620f-7a50-47c8-849b-5f94012d5e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd7d66723c44cb1983b0953a83e0377, user_id=f2828c03d1944bcebe32aedf658c5ca5, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:35.211189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59620f-7a50-47c8-849b-5f94012d5e39 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:35.234392 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2906/5739] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:35 2022] PUT /identity/v3/projects/abd7d66723c44cb1983b0953a83e0377/users/f2828c03d1944bcebe32aedf658c5ca5/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:35.244561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a6226-5d99-42ff-bb95-cf4c95c75e27 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:35.245082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a6226-5d99-42ff-bb95-cf4c95c75e27 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:35.245529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a6226-5d99-42ff-bb95-cf4c95c75e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:35.273460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59620f-7a50-47c8-849b-5f94012d5e39 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:35.275566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ea59620f-7a50-47c8-849b-5f94012d5e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:35.296196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2e6a6226-5d99-42ff-bb95-cf4c95c75e27 None None] MFA Rules not processed for user `f2828c03d1944bcebe32aedf658c5ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:35.297566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2e6a6226-5d99-42ff-bb95-cf4c95c75e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:35.355954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:35.388018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59620f-7a50-47c8-849b-5f94012d5e39 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:35.394362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d7568-075b-4756-adb8-3d3216bdbb29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:35.395097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d7568-075b-4756-adb8-3d3216bdbb29 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:35.396289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d7568-075b-4756-adb8-3d3216bdbb29 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:35.396876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d7568-075b-4756-adb8-3d3216bdbb29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:35.399492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2c0d7568-075b-4756-adb8-3d3216bdbb29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:35.418232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d7568-075b-4756-adb8-3d3216bdbb29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:35.419643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d7568-075b-4756-adb8-3d3216bdbb29 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:35.437637 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2907/5740] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:35.441466 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2834/5741] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:35.460518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d7568-075b-4756-adb8-3d3216bdbb29 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:35.462569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2c0d7568-075b-4756-adb8-3d3216bdbb29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:35.513610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a6226-5d99-42ff-bb95-cf4c95c75e27 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:35.515425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2e6a6226-5d99-42ff-bb95-cf4c95c75e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:35.609016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d7568-075b-4756-adb8-3d3216bdbb29 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:35.613701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe20a0-b919-4d4a-8f18-1411e1dafe37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:35.614408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe20a0-b919-4d4a-8f18-1411e1dafe37 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:35.614771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe20a0-b919-4d4a-8f18-1411e1dafe37 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:35.615131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe20a0-b919-4d4a-8f18-1411e1dafe37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:35.625368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe20a0-b919-4d4a-8f18-1411e1dafe37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:35.627571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe20a0-b919-4d4a-8f18-1411e1dafe37 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:35.648712 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2908/5742] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:35.655053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a6226-5d99-42ff-bb95-cf4c95c75e27 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:35.661240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:35.662140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:35.662589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:35.663029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] PATH_INFO: `/v3/projects/1b92867441be43dda95548932111025b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:35.669847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b92867441be43dda95548932111025b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:35.672112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:35.754688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b92867441be43dda95548932111025b', 'request_id': 'req-590613c3-a358-4962-942c-0eb9bebdc153'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:35.764963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b92867441be43dda95548932111025b', 'request_id': 'req-590613c3-a358-4962-942c-0eb9bebdc153'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:35.776606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b92867441be43dda95548932111025b', 'request_id': 'req-590613c3-a358-4962-942c-0eb9bebdc153'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:35.827536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] The token cache is being invalidate because project 1b92867441be43dda95548932111025b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:35.828178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:35.829724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:35.834817 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2835/5743] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:35 2022] DELETE /identity/v3/projects/1b92867441be43dda95548932111025b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:35.836161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe20a0-b919-4d4a-8f18-1411e1dafe37 service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:35.852032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe20a0-b919-4d4a-8f18-1411e1dafe37 service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:35.965093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:35.965814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:35.966107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:35.966386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] PATH_INFO: `/v3/users/b334362f83b44e40a66080bec3131b98` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:35.996965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b334362f83b44e40a66080bec3131b98)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:36.003792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:36.026666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:36.085158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b334362f83b44e40a66080bec3131b98'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:36.105539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b334362f83b44e40a66080bec3131b98'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:36.121628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b334362f83b44e40a66080bec3131b98'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:36.122269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b334362f83b44e40a66080bec3131b98'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:36.123066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b334362f83b44e40a66080bec3131b98'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:36.148941 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2909/5744] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:35 2022] DELETE /identity/v3/users/b334362f83b44e40a66080bec3131b98 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:36.223605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:36.226096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:36.320938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:36.341997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590613c3-a358-4962-942c-0eb9bebdc153 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:36.438848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90d20b-99b4-4618-81f3-0fdab88e109c tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2828c03d1944bcebe32aedf658c5ca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abd7d66723c44cb1983b0953a83e0377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:36.447894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90d20b-99b4-4618-81f3-0fdab88e109c tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:36.448199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90d20b-99b4-4618-81f3-0fdab88e109c tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:36.448494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90d20b-99b4-4618-81f3-0fdab88e109c tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:36.449222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90d20b-99b4-4618-81f3-0fdab88e109c tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:36.449990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90d20b-99b4-4618-81f3-0fdab88e109c tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:36.452086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90d20b-99b4-4618-81f3-0fdab88e109c tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:36.468688 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2836/5745] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:36.474008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:36.475892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:36.499275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d34263-6ecc-469e-8abe-b4859153bc08 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:36.506623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed97f91-80f6-4dea-8c8d-227164cc2f35 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2828c03d1944bcebe32aedf658c5ca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abd7d66723c44cb1983b0953a83e0377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:36.507368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed97f91-80f6-4dea-8c8d-227164cc2f35 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:36.507734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed97f91-80f6-4dea-8c8d-227164cc2f35 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:36.508122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed97f91-80f6-4dea-8c8d-227164cc2f35 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:36.509054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed97f91-80f6-4dea-8c8d-227164cc2f35 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:36.511726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed97f91-80f6-4dea-8c8d-227164cc2f35 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:36.545086 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2910/5746] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:36 2022] POST /identity/v3/projects => generated 354 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:36.547391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90d20b-99b4-4618-81f3-0fdab88e109c tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:36.560390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90d20b-99b4-4618-81f3-0fdab88e109c tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:36.645134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da509ef-3354-43ce-a62e-3ac5ddea86fb tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2828c03d1944bcebe32aedf658c5ca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abd7d66723c44cb1983b0953a83e0377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:36.646002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da509ef-3354-43ce-a62e-3ac5ddea86fb tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:36.646373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da509ef-3354-43ce-a62e-3ac5ddea86fb tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:36.646940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da509ef-3354-43ce-a62e-3ac5ddea86fb tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:36.647536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da509ef-3354-43ce-a62e-3ac5ddea86fb tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:36.648005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da509ef-3354-43ce-a62e-3ac5ddea86fb tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:36.649576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da509ef-3354-43ce-a62e-3ac5ddea86fb tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:36.662482 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2837/5747] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:36.704120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed97f91-80f6-4dea-8c8d-227164cc2f35 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:36.716705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed97f91-80f6-4dea-8c8d-227164cc2f35 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:36.724801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2828c03d1944bcebe32aedf658c5ca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abd7d66723c44cb1983b0953a83e0377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:36.725776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:36.726075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:36.727058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] PATH_INFO: `/v3/projects/15e3eaf98a5d40759265d9d480be98a1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:36.728612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: Authorizing `identity:delete_project(project_id=15e3eaf98a5d40759265d9d480be98a1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:36.731666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:36.809259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e3eaf98a5d40759265d9d480be98a1', 'request_id': 'req-b6d07286-9fa6-42da-8249-db4956101e70'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:36.819430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e3eaf98a5d40759265d9d480be98a1', 'request_id': 'req-b6d07286-9fa6-42da-8249-db4956101e70'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:36.827171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e3eaf98a5d40759265d9d480be98a1', 'request_id': 'req-b6d07286-9fa6-42da-8249-db4956101e70'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:36.848994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] The token cache is being invalidate because project 15e3eaf98a5d40759265d9d480be98a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:36.849343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:36.850173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:36.852328 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2911/5748] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:36 2022] DELETE /identity/v3/projects/15e3eaf98a5d40759265d9d480be98a1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:36.872919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da509ef-3354-43ce-a62e-3ac5ddea86fb tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:36.880852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7da509ef-3354-43ce-a62e-3ac5ddea86fb tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:36.976766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da509ef-3354-43ce-a62e-3ac5ddea86fb tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:37.043488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:37.072714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf560aba-2a14-4265-ac94-b43835ce80f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:37.073375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf560aba-2a14-4265-ac94-b43835ce80f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.073665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf560aba-2a14-4265-ac94-b43835ce80f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.073951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf560aba-2a14-4265-ac94-b43835ce80f8 admin admin] PATH_INFO: `/v3/projects/20c08bc074844d13bd9f765880b07668` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.075164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf560aba-2a14-4265-ac94-b43835ce80f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c08bc074844d13bd9f765880b07668)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:37.077491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf560aba-2a14-4265-ac94-b43835ce80f8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:37.140196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:37.141668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:37.157245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bf560aba-2a14-4265-ac94-b43835ce80f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c08bc074844d13bd9f765880b07668', 'request_id': 'req-bf560aba-2a14-4265-ac94-b43835ce80f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:37.163983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d07286-9fa6-42da-8249-db4956101e70 tempest-VolumeQuotaClassesTest-934548207 tempest-VolumeQuotaClassesTest-934548207-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:37.170060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bf560aba-2a14-4265-ac94-b43835ce80f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c08bc074844d13bd9f765880b07668', 'request_id': 'req-bf560aba-2a14-4265-ac94-b43835ce80f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:37.180833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bf560aba-2a14-4265-ac94-b43835ce80f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c08bc074844d13bd9f765880b07668', 'request_id': 'req-bf560aba-2a14-4265-ac94-b43835ce80f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:37.195575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bf560aba-2a14-4265-ac94-b43835ce80f8 admin admin] The token cache is being invalidate because project 20c08bc074844d13bd9f765880b07668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:37.196032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bf560aba-2a14-4265-ac94-b43835ce80f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:37.197099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bf560aba-2a14-4265-ac94-b43835ce80f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:37.199267 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2838/5749] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:36 2022] DELETE /identity/v3/projects/20c08bc074844d13bd9f765880b07668 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:37.204657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48996-8730-4470-a71a-93db777c856b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.205723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48996-8730-4470-a71a-93db777c856b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.206116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48996-8730-4470-a71a-93db777c856b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.242091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:37.246914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ace48996-8730-4470-a71a-93db777c856b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:37.248375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ace48996-8730-4470-a71a-93db777c856b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:37.255257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735e1f7-e3a4-468a-813d-e74e811239ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:37.256431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735e1f7-e3a4-468a-813d-e74e811239ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.256922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735e1f7-e3a4-468a-813d-e74e811239ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.261384 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2839/5750] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:37.272123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735e1f7-e3a4-468a-813d-e74e811239ce admin admin] PATH_INFO: `/v3/users/cf4dfb455208485f9d3e7661ad77bbe5` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.272781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba918d-e9ea-4ff5-8d5b-9fae396efbae None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.273762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba918d-e9ea-4ff5-8d5b-9fae396efbae None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.274547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba918d-e9ea-4ff5-8d5b-9fae396efbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.275444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735e1f7-e3a4-468a-813d-e74e811239ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf4dfb455208485f9d3e7661ad77bbe5)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:37.278200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735e1f7-e3a4-468a-813d-e74e811239ce admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:37.302628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-49ba918d-e9ea-4ff5-8d5b-9fae396efbae None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:37.303798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-49ba918d-e9ea-4ff5-8d5b-9fae396efbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:37.316577 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2840/5751] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:37.321280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba918d-e9ea-4ff5-8d5b-9fae396efbae None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:37.333185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba918d-e9ea-4ff5-8d5b-9fae396efbae None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:37.339372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c08d4-ba64-4a74-994e-f556b9c9a2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:37.339590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c08d4-ba64-4a74-994e-f556b9c9a2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.339878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c08d4-ba64-4a74-994e-f556b9c9a2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.340231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c08d4-ba64-4a74-994e-f556b9c9a2c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.341034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c08d4-ba64-4a74-994e-f556b9c9a2c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:37.341633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c08d4-ba64-4a74-994e-f556b9c9a2c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:37.343975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c08d4-ba64-4a74-994e-f556b9c9a2c6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:37.353515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c735e1f7-e3a4-468a-813d-e74e811239ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf4dfb455208485f9d3e7661ad77bbe5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:37.362128 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2841/5752] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:09:37.363045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c08d4-ba64-4a74-994e-f556b9c9a2c6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:37.373123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c735e1f7-e3a4-468a-813d-e74e811239ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf4dfb455208485f9d3e7661ad77bbe5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:37.386062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c08d4-ba64-4a74-994e-f556b9c9a2c6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:37.386675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c735e1f7-e3a4-468a-813d-e74e811239ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf4dfb455208485f9d3e7661ad77bbe5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:37.387449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c735e1f7-e3a4-468a-813d-e74e811239ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf4dfb455208485f9d3e7661ad77bbe5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:37.388280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c735e1f7-e3a4-468a-813d-e74e811239ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf4dfb455208485f9d3e7661ad77bbe5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:37.392979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e99d9-962b-4a07-afb5-210eb4cf2b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:37.395152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e99d9-962b-4a07-afb5-210eb4cf2b67 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.395519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e99d9-962b-4a07-afb5-210eb4cf2b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.395871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e99d9-962b-4a07-afb5-210eb4cf2b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.396603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e99d9-962b-4a07-afb5-210eb4cf2b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:37.398939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e99d9-962b-4a07-afb5-210eb4cf2b67 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:37.430154 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2912/5753] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:37 2022] DELETE /identity/v3/users/cf4dfb455208485f9d3e7661ad77bbe5 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:37.435010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ed5c2-8e87-4179-9105-91d4f01bc228 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.435398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ed5c2-8e87-4179-9105-91d4f01bc228 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.435659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ed5c2-8e87-4179-9105-91d4f01bc228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.443305 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2842/5754] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:37 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:37.447747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e99d9-962b-4a07-afb5-210eb4cf2b67 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:37.450296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d22e99d9-962b-4a07-afb5-210eb4cf2b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:37.457794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-784ed5c2-8e87-4179-9105-91d4f01bc228 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:37.458981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-784ed5c2-8e87-4179-9105-91d4f01bc228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:37.533447 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2913/5755] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:37.533661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e99d9-962b-4a07-afb5-210eb4cf2b67 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:37.535739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b45e85-e46c-437e-956a-f2ad025ea52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:37.536309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b45e85-e46c-437e-956a-f2ad025ea52b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.536729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b45e85-e46c-437e-956a-f2ad025ea52b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.537321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b45e85-e46c-437e-956a-f2ad025ea52b admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.537685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b45e85-e46c-437e-956a-f2ad025ea52b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:37.539643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b45e85-e46c-437e-956a-f2ad025ea52b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:37.540994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ed5c2-8e87-4179-9105-91d4f01bc228 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:37.543228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-784ed5c2-8e87-4179-9105-91d4f01bc228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:37.564948 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2843/5756] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:37 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:37.569094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b45e85-e46c-437e-956a-f2ad025ea52b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:37.579157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b45e85-e46c-437e-956a-f2ad025ea52b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:37.585140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1718e-37f3-40b9-abba-1cc041d1096c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:37.585788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1718e-37f3-40b9-abba-1cc041d1096c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.586150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1718e-37f3-40b9-abba-1cc041d1096c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.586433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1718e-37f3-40b9-abba-1cc041d1096c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.587156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1718e-37f3-40b9-abba-1cc041d1096c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:37.588874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1718e-37f3-40b9-abba-1cc041d1096c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:37.595499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:37.601750 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2844/5757] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:37.605665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1718e-37f3-40b9-abba-1cc041d1096c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:37.618607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1718e-37f3-40b9-abba-1cc041d1096c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:37.625315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f82ca4-8623-463d-a0e7-f92c284e5ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:37.625735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ed5c2-8e87-4179-9105-91d4f01bc228 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:37.625932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f82ca4-8623-463d-a0e7-f92c284e5ecc admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.626265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f82ca4-8623-463d-a0e7-f92c284e5ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.627240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f82ca4-8623-463d-a0e7-f92c284e5ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.628686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f82ca4-8623-463d-a0e7-f92c284e5ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:37.629877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f82ca4-8623-463d-a0e7-f92c284e5ecc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:37.630521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf973f45-4bc5-4737-9bb2-5592b8964726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:37.631297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf973f45-4bc5-4737-9bb2-5592b8964726 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.631552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf973f45-4bc5-4737-9bb2-5592b8964726 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.631900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf973f45-4bc5-4737-9bb2-5592b8964726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.642015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf973f45-4bc5-4737-9bb2-5592b8964726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:37.644035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf973f45-4bc5-4737-9bb2-5592b8964726 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:37.646040 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2845/5758] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:37.649817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f82ca4-8623-463d-a0e7-f92c284e5ecc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:37.657376 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2914/5759] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:09:37.660966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f82ca4-8623-463d-a0e7-f92c284e5ecc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:37.666714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8eb380-920a-473c-9abe-5fafa1087bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:37.667500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8eb380-920a-473c-9abe-5fafa1087bce admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.667852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8eb380-920a-473c-9abe-5fafa1087bce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.668216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8eb380-920a-473c-9abe-5fafa1087bce admin admin] PATH_INFO: `/v3/projects/dc30295b8af2476dad1901fb791c7ce5/users/469c71743bd440548f6efd5a95136a87/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.685577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8eb380-920a-473c-9abe-5fafa1087bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc30295b8af2476dad1901fb791c7ce5, user_id=469c71743bd440548f6efd5a95136a87, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:37.688558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8eb380-920a-473c-9abe-5fafa1087bce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:37.705596 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2846/5760] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:37 2022] PUT /identity/v3/projects/dc30295b8af2476dad1901fb791c7ce5/users/469c71743bd440548f6efd5a95136a87/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:37.716537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711818a1-8bf3-4307-a45e-7fe34b663bbe None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.717067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711818a1-8bf3-4307-a45e-7fe34b663bbe None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.717269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711818a1-8bf3-4307-a45e-7fe34b663bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.749889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-711818a1-8bf3-4307-a45e-7fe34b663bbe None None] MFA Rules not processed for user `469c71743bd440548f6efd5a95136a87`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:37.752339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-711818a1-8bf3-4307-a45e-7fe34b663bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:37.758327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:37.845203 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2915/5761] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:37.859035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8eb380-920a-473c-9abe-5fafa1087bce admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:37.861538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6e8eb380-920a-473c-9abe-5fafa1087bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:37.938089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8eb380-920a-473c-9abe-5fafa1087bce admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:37.942584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca8a7c-3599-4c44-bd01-97ce59d617d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:37.943204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca8a7c-3599-4c44-bd01-97ce59d617d9 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:37.943494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca8a7c-3599-4c44-bd01-97ce59d617d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:37.943776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca8a7c-3599-4c44-bd01-97ce59d617d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:37.953673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca8a7c-3599-4c44-bd01-97ce59d617d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:37.955212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca8a7c-3599-4c44-bd01-97ce59d617d9 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:37.967826 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2847/5762] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:38.310454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711818a1-8bf3-4307-a45e-7fe34b663bbe None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:38.314452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-711818a1-8bf3-4307-a45e-7fe34b663bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:38.442259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711818a1-8bf3-4307-a45e-7fe34b663bbe None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:38.452078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:38.452757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:38.453082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:38.454276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] PATH_INFO: `/v3/projects/726f04e1d9cd4c73b6ee1a7250cb3e83` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:38.461078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=726f04e1d9cd4c73b6ee1a7250cb3e83)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:38.463527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:38.545870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '726f04e1d9cd4c73b6ee1a7250cb3e83', 'request_id': 'req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.563359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '726f04e1d9cd4c73b6ee1a7250cb3e83', 'request_id': 'req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.573607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '726f04e1d9cd4c73b6ee1a7250cb3e83', 'request_id': 'req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.589626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] The token cache is being invalidate because project 726f04e1d9cd4c73b6ee1a7250cb3e83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:38.589931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.590609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.593098 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2916/5763] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:38 2022] DELETE /identity/v3/projects/726f04e1d9cd4c73b6ee1a7250cb3e83 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:38.596875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca8a7c-3599-4c44-bd01-97ce59d617d9 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:38.610479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca8a7c-3599-4c44-bd01-97ce59d617d9 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:38.689349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35572f27-daf2-4968-9a72-728e8d5a66e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:38.690011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35572f27-daf2-4968-9a72-728e8d5a66e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:38.690305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35572f27-daf2-4968-9a72-728e8d5a66e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:38.690596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35572f27-daf2-4968-9a72-728e8d5a66e4 admin admin] PATH_INFO: `/v3/users/f2828c03d1944bcebe32aedf658c5ca5` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:38.691752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35572f27-daf2-4968-9a72-728e8d5a66e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2828c03d1944bcebe32aedf658c5ca5)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:38.693786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35572f27-daf2-4968-9a72-728e8d5a66e4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:38.765269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-35572f27-daf2-4968-9a72-728e8d5a66e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2828c03d1944bcebe32aedf658c5ca5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.772589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-35572f27-daf2-4968-9a72-728e8d5a66e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2828c03d1944bcebe32aedf658c5ca5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.776354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-35572f27-daf2-4968-9a72-728e8d5a66e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2828c03d1944bcebe32aedf658c5ca5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.784035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:38.788128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:38.790362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:38.790841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-35572f27-daf2-4968-9a72-728e8d5a66e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2828c03d1944bcebe32aedf658c5ca5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.800989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-35572f27-daf2-4968-9a72-728e8d5a66e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2828c03d1944bcebe32aedf658c5ca5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.808223 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2848/5764] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:38 2022] DELETE /identity/v3/users/f2828c03d1944bcebe32aedf658c5ca5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:38.809013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab3cf4-c4ff-4aee-a757-e4473ad009a2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:38.861755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:38.900159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:38.900824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:38.901151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:38.901455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] PATH_INFO: `/v3/users/469c71743bd440548f6efd5a95136a87` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:38.902719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=469c71743bd440548f6efd5a95136a87)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:38.904806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:38.965452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '469c71743bd440548f6efd5a95136a87'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.987896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '469c71743bd440548f6efd5a95136a87'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.998660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '469c71743bd440548f6efd5a95136a87'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.999320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '469c71743bd440548f6efd5a95136a87'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:38.999947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '469c71743bd440548f6efd5a95136a87'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:39.016203 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2917/5765] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:38 2022] DELETE /identity/v3/users/469c71743bd440548f6efd5a95136a87 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:39.021437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978435-99cd-4b82-8ffa-055e9b2e3033 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.021687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978435-99cd-4b82-8ffa-055e9b2e3033 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.022012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978435-99cd-4b82-8ffa-055e9b2e3033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.043485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d1978435-99cd-4b82-8ffa-055e9b2e3033 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:39.044786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d1978435-99cd-4b82-8ffa-055e9b2e3033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:39.107053 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2849/5766] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:39.116742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:39.118194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:39.192661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d13d19-79e5-4c40-95b3-5dd9999dc312 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:39.197250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c7476-513e-4b58-aba3-8a3cc3a5215d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:39.197949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c7476-513e-4b58-aba3-8a3cc3a5215d service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.198241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c7476-513e-4b58-aba3-8a3cc3a5215d service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.198522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c7476-513e-4b58-aba3-8a3cc3a5215d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.209431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c7476-513e-4b58-aba3-8a3cc3a5215d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:39.210955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c7476-513e-4b58-aba3-8a3cc3a5215d service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:39.220977 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2918/5767] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:39.264189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1978435-99cd-4b82-8ffa-055e9b2e3033 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:39.265996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d1978435-99cd-4b82-8ffa-055e9b2e3033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:39.310579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1978435-99cd-4b82-8ffa-055e9b2e3033 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:39.320389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443bcbe-3c9e-433a-b7ba-48c48be880be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:39.321108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443bcbe-3c9e-433a-b7ba-48c48be880be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.321404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443bcbe-3c9e-433a-b7ba-48c48be880be admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.321723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443bcbe-3c9e-433a-b7ba-48c48be880be admin admin] PATH_INFO: `/v3/projects/abd7d66723c44cb1983b0953a83e0377` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.322913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443bcbe-3c9e-433a-b7ba-48c48be880be admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd7d66723c44cb1983b0953a83e0377)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:39.324849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443bcbe-3c9e-433a-b7ba-48c48be880be admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:39.383724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9443bcbe-3c9e-433a-b7ba-48c48be880be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd7d66723c44cb1983b0953a83e0377', 'request_id': 'req-9443bcbe-3c9e-433a-b7ba-48c48be880be'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:39.396587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9443bcbe-3c9e-433a-b7ba-48c48be880be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd7d66723c44cb1983b0953a83e0377', 'request_id': 'req-9443bcbe-3c9e-433a-b7ba-48c48be880be'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:39.404766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9443bcbe-3c9e-433a-b7ba-48c48be880be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd7d66723c44cb1983b0953a83e0377', 'request_id': 'req-9443bcbe-3c9e-433a-b7ba-48c48be880be'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:39.415970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9443bcbe-3c9e-433a-b7ba-48c48be880be admin admin] The token cache is being invalidate because project abd7d66723c44cb1983b0953a83e0377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:39.416276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9443bcbe-3c9e-433a-b7ba-48c48be880be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:39.417011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9443bcbe-3c9e-433a-b7ba-48c48be880be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:39.419049 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2850/5768] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:39 2022] DELETE /identity/v3/projects/abd7d66723c44cb1983b0953a83e0377 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:39.425484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ec72c-fdc2-407d-a38c-372f979967ad None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.425850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ec72c-fdc2-407d-a38c-372f979967ad None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.426202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ec72c-fdc2-407d-a38c-372f979967ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.447063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-938ec72c-fdc2-407d-a38c-372f979967ad None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:39.448300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-938ec72c-fdc2-407d-a38c-372f979967ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:39.517964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:39.534592 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2919/5769] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:39.537411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309f624-999b-4afb-a183-938b635ea386 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.537724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309f624-999b-4afb-a183-938b635ea386 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.538007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309f624-999b-4afb-a183-938b635ea386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.561883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1309f624-999b-4afb-a183-938b635ea386 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:39.563389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1309f624-999b-4afb-a183-938b635ea386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:39.573930 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2851/5770] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:39.577596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ec72c-fdc2-407d-a38c-372f979967ad None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:39.589441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ec72c-fdc2-407d-a38c-372f979967ad None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:39.595121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4884d-88e7-4088-ae5e-8e7cd6b93d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:39.596238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4884d-88e7-4088-ae5e-8e7cd6b93d5f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.596522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4884d-88e7-4088-ae5e-8e7cd6b93d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.596793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4884d-88e7-4088-ae5e-8e7cd6b93d5f admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.597488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4884d-88e7-4088-ae5e-8e7cd6b93d5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:39.598306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4884d-88e7-4088-ae5e-8e7cd6b93d5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:39.600821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4884d-88e7-4088-ae5e-8e7cd6b93d5f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:39.620505 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2920/5771] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:39.621712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309f624-999b-4afb-a183-938b635ea386 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:39.631673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309f624-999b-4afb-a183-938b635ea386 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:39.640996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482fccf-1f65-478e-a6a3-e53db23f9f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:39.641866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482fccf-1f65-478e-a6a3-e53db23f9f6c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.642391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482fccf-1f65-478e-a6a3-e53db23f9f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.642894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482fccf-1f65-478e-a6a3-e53db23f9f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.643894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482fccf-1f65-478e-a6a3-e53db23f9f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:39.646254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482fccf-1f65-478e-a6a3-e53db23f9f6c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:39.653348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4884d-88e7-4088-ae5e-8e7cd6b93d5f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:39.655162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-30a4884d-88e7-4088-ae5e-8e7cd6b93d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:39.679780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4884d-88e7-4088-ae5e-8e7cd6b93d5f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:39.689011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d461639-03f3-48d4-b268-3effcfb23551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:39.689945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d461639-03f3-48d4-b268-3effcfb23551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.690332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d461639-03f3-48d4-b268-3effcfb23551 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.690767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d461639-03f3-48d4-b268-3effcfb23551 admin admin] PATH_INFO: `/v3/projects/dc30295b8af2476dad1901fb791c7ce5` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.692255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d461639-03f3-48d4-b268-3effcfb23551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc30295b8af2476dad1901fb791c7ce5)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:39.695786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d461639-03f3-48d4-b268-3effcfb23551 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:39.734497 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2852/5772] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:39 2022] POST /identity/v3/projects => generated 393 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:39.735800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482fccf-1f65-478e-a6a3-e53db23f9f6c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:39.765548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7d461639-03f3-48d4-b268-3effcfb23551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc30295b8af2476dad1901fb791c7ce5', 'request_id': 'req-7d461639-03f3-48d4-b268-3effcfb23551'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:39.774921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7d461639-03f3-48d4-b268-3effcfb23551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc30295b8af2476dad1901fb791c7ce5', 'request_id': 'req-7d461639-03f3-48d4-b268-3effcfb23551'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:39.787161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7d461639-03f3-48d4-b268-3effcfb23551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc30295b8af2476dad1901fb791c7ce5', 'request_id': 'req-7d461639-03f3-48d4-b268-3effcfb23551'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:39.795126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:39.807686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7d461639-03f3-48d4-b268-3effcfb23551 admin admin] The token cache is being invalidate because project dc30295b8af2476dad1901fb791c7ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:39.808471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7d461639-03f3-48d4-b268-3effcfb23551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:39.809565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7d461639-03f3-48d4-b268-3effcfb23551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:39.811842 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2921/5773] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:39 2022] DELETE /identity/v3/projects/dc30295b8af2476dad1901fb791c7ce5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:39.816401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9175d4d-2322-42ad-b4c1-5d23c2040967 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.818841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9175d4d-2322-42ad-b4c1-5d23c2040967 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.819146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9175d4d-2322-42ad-b4c1-5d23c2040967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.821998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482fccf-1f65-478e-a6a3-e53db23f9f6c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:39.841968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e9175d4d-2322-42ad-b4c1-5d23c2040967 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:39.843928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e9175d4d-2322-42ad-b4c1-5d23c2040967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:39.889721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee065f-2c6b-403d-a6ec-02dde9992978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:39.890386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee065f-2c6b-403d-a6ec-02dde9992978 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.890676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee065f-2c6b-403d-a6ec-02dde9992978 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.891008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee065f-2c6b-403d-a6ec-02dde9992978 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.891664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee065f-2c6b-403d-a6ec-02dde9992978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:39.894270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee065f-2c6b-403d-a6ec-02dde9992978 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:39.922302 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2853/5774] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:39 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:39.922536 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2922/5775] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:39.925961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee065f-2c6b-403d-a6ec-02dde9992978 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:39.928152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8bee065f-2c6b-403d-a6ec-02dde9992978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:39.929129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f957e2-1d51-4dc3-8743-f5c3b7d72aec None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.929430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f957e2-1d51-4dc3-8743-f5c3b7d72aec None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.929717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f957e2-1d51-4dc3-8743-f5c3b7d72aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.944536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee065f-2c6b-403d-a6ec-02dde9992978 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:39.949601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd27e25-63ee-461a-95ad-e192d98ff804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:39.950356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd27e25-63ee-461a-95ad-e192d98ff804 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.950653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd27e25-63ee-461a-95ad-e192d98ff804 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.950936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd27e25-63ee-461a-95ad-e192d98ff804 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.951586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd27e25-63ee-461a-95ad-e192d98ff804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:39.954007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd27e25-63ee-461a-95ad-e192d98ff804 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:39.954539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a9f957e2-1d51-4dc3-8743-f5c3b7d72aec None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:39.957110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a9f957e2-1d51-4dc3-8743-f5c3b7d72aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:39.967623 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2854/5776] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:39.968758 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2923/5777] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:39.968929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd27e25-63ee-461a-95ad-e192d98ff804 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:39.974010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f957e2-1d51-4dc3-8743-f5c3b7d72aec None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:39.983765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd27e25-63ee-461a-95ad-e192d98ff804 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:39.986874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f957e2-1d51-4dc3-8743-f5c3b7d72aec None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:39.989756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df49dd7-a6b7-445f-a8ca-2b6ea4d85c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:39.990559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df49dd7-a6b7-445f-a8ca-2b6ea4d85c91 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.991036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df49dd7-a6b7-445f-a8ca-2b6ea4d85c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.991991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd47ae9-e3b6-4ca9-bf5f-c13e6daa3f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:39.992352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df49dd7-a6b7-445f-a8ca-2b6ea4d85c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.992877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd47ae9-e3b6-4ca9-bf5f-c13e6daa3f29 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:39.993230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd47ae9-e3b6-4ca9-bf5f-c13e6daa3f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:39.993537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd47ae9-e3b6-4ca9-bf5f-c13e6daa3f29 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:39.993732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df49dd7-a6b7-445f-a8ca-2b6ea4d85c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:39.994429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd47ae9-e3b6-4ca9-bf5f-c13e6daa3f29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:39.994704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd47ae9-e3b6-4ca9-bf5f-c13e6daa3f29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:39.996332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd47ae9-e3b6-4ca9-bf5f-c13e6daa3f29 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:39.997340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df49dd7-a6b7-445f-a8ca-2b6ea4d85c91 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.010203 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2924/5778] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:09:40.015256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd47ae9-e3b6-4ca9-bf5f-c13e6daa3f29 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.030690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd47ae9-e3b6-4ca9-bf5f-c13e6daa3f29 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.032502 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2855/5779] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:40.034282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df49dd7-a6b7-445f-a8ca-2b6ea4d85c91 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.036844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc9a10-194c-425e-a1b2-3a058fd8955b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.037495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc9a10-194c-425e-a1b2-3a058fd8955b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.037809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc9a10-194c-425e-a1b2-3a058fd8955b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.038096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc9a10-194c-425e-a1b2-3a058fd8955b admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.038768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc9a10-194c-425e-a1b2-3a058fd8955b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.042370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc9a10-194c-425e-a1b2-3a058fd8955b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.045666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df49dd7-a6b7-445f-a8ca-2b6ea4d85c91 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.050781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a391c7-4494-4dd1-b08a-113d7122b8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.051460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a391c7-4494-4dd1-b08a-113d7122b8b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.051911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a391c7-4494-4dd1-b08a-113d7122b8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.052266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a391c7-4494-4dd1-b08a-113d7122b8b3 admin admin] PATH_INFO: `/v3/projects/315897be82e4487691ea89cd2ab54abb/users/c68e54ec2baf40908333d6e8d5d6a32b/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.081507 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2925/5780] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:40 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:40.085750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc9a10-194c-425e-a1b2-3a058fd8955b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.086292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a391c7-4494-4dd1-b08a-113d7122b8b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315897be82e4487691ea89cd2ab54abb, user_id=c68e54ec2baf40908333d6e8d5d6a32b, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.090395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a391c7-4494-4dd1-b08a-113d7122b8b3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.111641 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2856/5781] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:40 2022] PUT /identity/v3/projects/315897be82e4487691ea89cd2ab54abb/users/c68e54ec2baf40908333d6e8d5d6a32b/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:40.121341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa6f31-5f4d-4f75-b2b2-b42eac993ec9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.121826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa6f31-5f4d-4f75-b2b2-b42eac993ec9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.122183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa6f31-5f4d-4f75-b2b2-b42eac993ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.141835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:40.161627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ddaa6f31-5f4d-4f75-b2b2-b42eac993ec9 None None] MFA Rules not processed for user `c68e54ec2baf40908333d6e8d5d6a32b`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:40.162797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ddaa6f31-5f4d-4f75-b2b2-b42eac993ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:40.169482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc9a10-194c-425e-a1b2-3a058fd8955b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.246159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079f25a-5a47-4b6c-8a07-969fbe66a13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.246783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079f25a-5a47-4b6c-8a07-969fbe66a13f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.247069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079f25a-5a47-4b6c-8a07-969fbe66a13f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.247349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079f25a-5a47-4b6c-8a07-969fbe66a13f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.247979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079f25a-5a47-4b6c-8a07-969fbe66a13f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.249927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079f25a-5a47-4b6c-8a07-969fbe66a13f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.256940 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2857/5782] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:40.260574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa6f31-5f4d-4f75-b2b2-b42eac993ec9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.275659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa6f31-5f4d-4f75-b2b2-b42eac993ec9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.278884 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2926/5783] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:40 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:40.279868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079f25a-5a47-4b6c-8a07-969fbe66a13f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.281415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354e45b-4c41-47c2-97f9-1a2547a1d85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.282270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354e45b-4c41-47c2-97f9-1a2547a1d85c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.282568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354e45b-4c41-47c2-97f9-1a2547a1d85c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.282882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354e45b-4c41-47c2-97f9-1a2547a1d85c admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.283531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354e45b-4c41-47c2-97f9-1a2547a1d85c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.285760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354e45b-4c41-47c2-97f9-1a2547a1d85c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.292157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079f25a-5a47-4b6c-8a07-969fbe66a13f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.297642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff90c1-f623-430d-a600-0972472611d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.298354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff90c1-f623-430d-a600-0972472611d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.298735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff90c1-f623-430d-a600-0972472611d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.299113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff90c1-f623-430d-a600-0972472611d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.299934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff90c1-f623-430d-a600-0972472611d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.304729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff90c1-f623-430d-a600-0972472611d5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.318434 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2927/5784] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:40.322068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:40.323541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff90c1-f623-430d-a600-0972472611d5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.333515 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2858/5785] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:40 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:40.335978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354e45b-4c41-47c2-97f9-1a2547a1d85c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.336429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff90c1-f623-430d-a600-0972472611d5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.428409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544673c-0dac-4add-81f3-342b8adc79d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.429141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544673c-0dac-4add-81f3-342b8adc79d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.429502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544673c-0dac-4add-81f3-342b8adc79d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.429847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544673c-0dac-4add-81f3-342b8adc79d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.430645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544673c-0dac-4add-81f3-342b8adc79d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.432455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544673c-0dac-4add-81f3-342b8adc79d8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.438759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354e45b-4c41-47c2-97f9-1a2547a1d85c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.447163 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2928/5786] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:40.448122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9162384-b553-4897-8821-c7e0439d5e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.448824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9162384-b553-4897-8821-c7e0439d5e72 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.449079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544673c-0dac-4add-81f3-342b8adc79d8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.449259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9162384-b553-4897-8821-c7e0439d5e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.449618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9162384-b553-4897-8821-c7e0439d5e72 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.450299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9162384-b553-4897-8821-c7e0439d5e72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.452329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9162384-b553-4897-8821-c7e0439d5e72 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.462844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544673c-0dac-4add-81f3-342b8adc79d8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.473287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf9f66-117c-410c-a57f-04c5dd600355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.474309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf9f66-117c-410c-a57f-04c5dd600355 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.475675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf9f66-117c-410c-a57f-04c5dd600355 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.475675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf9f66-117c-410c-a57f-04c5dd600355 admin admin] PATH_INFO: `/v3/projects/73e86a5495fd4f618208750f7c6d6f14/users/7120c389feb14e319b2d3a8ace28dbe2/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.478707 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2859/5787] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:40 2022] POST /identity/v3/users => generated 300 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:40.482635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9162384-b553-4897-8821-c7e0439d5e72 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.499878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf9f66-117c-410c-a57f-04c5dd600355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e86a5495fd4f618208750f7c6d6f14, user_id=7120c389feb14e319b2d3a8ace28dbe2, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.500058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9162384-b553-4897-8821-c7e0439d5e72 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.503863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf9f66-117c-410c-a57f-04c5dd600355 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.509789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b062dc-2dd1-4c51-b400-2da798d703fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.510899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b062dc-2dd1-4c51-b400-2da798d703fe admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.511249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b062dc-2dd1-4c51-b400-2da798d703fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.511649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b062dc-2dd1-4c51-b400-2da798d703fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.512576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b062dc-2dd1-4c51-b400-2da798d703fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.514522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b062dc-2dd1-4c51-b400-2da798d703fe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.523293 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2929/5788] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:40 2022] PUT /identity/v3/projects/73e86a5495fd4f618208750f7c6d6f14/users/7120c389feb14e319b2d3a8ace28dbe2/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:40.528219 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2860/5789] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:40.533617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf9f66-117c-410c-a57f-04c5dd600355 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.534237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade3f06-e29c-4533-bc0e-68c1d726f55f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.534636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade3f06-e29c-4533-bc0e-68c1d726f55f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.535150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade3f06-e29c-4533-bc0e-68c1d726f55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.581659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0ade3f06-e29c-4533-bc0e-68c1d726f55f None None] MFA Rules not processed for user `7120c389feb14e319b2d3a8ace28dbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:40.583133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0ade3f06-e29c-4533-bc0e-68c1d726f55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:40.591229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:40.630785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf9f66-117c-410c-a57f-04c5dd600355 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.636304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebf90e-e461-414b-8b03-9904f14f6b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.637015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebf90e-e461-414b-8b03-9904f14f6b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.637429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebf90e-e461-414b-8b03-9904f14f6b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.637711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebf90e-e461-414b-8b03-9904f14f6b93 admin admin] PATH_INFO: `/v3/projects/de7adc4281a140989ed9b89019aaaa2a/users/b9663d439fda41c186b907e0d48ccb8c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.655549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebf90e-e461-414b-8b03-9904f14f6b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7adc4281a140989ed9b89019aaaa2a, user_id=b9663d439fda41c186b907e0d48ccb8c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.658513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebf90e-e461-414b-8b03-9904f14f6b93 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.674908 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2861/5790] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:40.679327 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2930/5791] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:40 2022] PUT /identity/v3/projects/de7adc4281a140989ed9b89019aaaa2a/users/b9663d439fda41c186b907e0d48ccb8c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:40.680257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade3f06-e29c-4533-bc0e-68c1d726f55f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.685908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebf90e-e461-414b-8b03-9904f14f6b93 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.688571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-91ebf90e-e461-414b-8b03-9904f14f6b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:40.692672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade3f06-e29c-4533-bc0e-68c1d726f55f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.775256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0867fb9-5f01-4e09-939b-d6006fd6748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.775879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0867fb9-5f01-4e09-939b-d6006fd6748c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.776166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0867fb9-5f01-4e09-939b-d6006fd6748c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.776448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0867fb9-5f01-4e09-939b-d6006fd6748c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.777175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0867fb9-5f01-4e09-939b-d6006fd6748c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.777545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebf90e-e461-414b-8b03-9904f14f6b93 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.779036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0867fb9-5f01-4e09-939b-d6006fd6748c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.781561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de27765-7f69-44b1-9044-56433fa5547f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.782187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de27765-7f69-44b1-9044-56433fa5547f service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.782456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de27765-7f69-44b1-9044-56433fa5547f service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.782734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de27765-7f69-44b1-9044-56433fa5547f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.789551 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2862/5792] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:40.793387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0867fb9-5f01-4e09-939b-d6006fd6748c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:40.801656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:40.806099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0867fb9-5f01-4e09-939b-d6006fd6748c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:40.812150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65197687-cbc2-42e6-b366-b1b159040d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:40.812826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65197687-cbc2-42e6-b366-b1b159040d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.813165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65197687-cbc2-42e6-b366-b1b159040d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.813551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65197687-cbc2-42e6-b366-b1b159040d3b admin admin] PATH_INFO: `/v3/projects/de7adc4281a140989ed9b89019aaaa2a/users/b9663d439fda41c186b907e0d48ccb8c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.816327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65197687-cbc2-42e6-b366-b1b159040d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7adc4281a140989ed9b89019aaaa2a, user_id=b9663d439fda41c186b907e0d48ccb8c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.819304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65197687-cbc2-42e6-b366-b1b159040d3b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.830450 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2863/5793] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:40 2022] PUT /identity/v3/projects/de7adc4281a140989ed9b89019aaaa2a/users/b9663d439fda41c186b907e0d48ccb8c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:40.859561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de27765-7f69-44b1-9044-56433fa5547f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:40.862462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de27765-7f69-44b1-9044-56433fa5547f service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:40.873833 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2931/5794] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:09:40.937027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefd82e-467e-4dae-8229-a25a67dcbaac None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:40.937558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefd82e-467e-4dae-8229-a25a67dcbaac None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:40.937851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefd82e-467e-4dae-8229-a25a67dcbaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:40.975224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-deefd82e-467e-4dae-8229-a25a67dcbaac None None] MFA Rules not processed for user `b9663d439fda41c186b907e0d48ccb8c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:40.977437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-deefd82e-467e-4dae-8229-a25a67dcbaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:41.120462 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2864/5795] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:41.131912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de27765-7f69-44b1-9044-56433fa5547f service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:41.133889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5de27765-7f69-44b1-9044-56433fa5547f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:41.227440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de27765-7f69-44b1-9044-56433fa5547f service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:41.255356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:41.339984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18c9c2-e2c1-4ffc-95c9-635a4a8c59d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:41.341250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18c9c2-e2c1-4ffc-95c9-635a4a8c59d5 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:41.341759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18c9c2-e2c1-4ffc-95c9-635a4a8c59d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:41.342269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18c9c2-e2c1-4ffc-95c9-635a4a8c59d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:41.354927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18c9c2-e2c1-4ffc-95c9-635a4a8c59d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:41.357169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18c9c2-e2c1-4ffc-95c9-635a4a8c59d5 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:41.377100 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2932/5796] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:41.444671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefd82e-467e-4dae-8229-a25a67dcbaac None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:41.446190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-deefd82e-467e-4dae-8229-a25a67dcbaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:41.461571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefd82e-467e-4dae-8229-a25a67dcbaac None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:41.466128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd1651-a243-4a0f-9bc9-563b2133bd43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:41.466855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd1651-a243-4a0f-9bc9-563b2133bd43 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:41.467240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd1651-a243-4a0f-9bc9-563b2133bd43 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:41.467617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd1651-a243-4a0f-9bc9-563b2133bd43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:41.545652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dd1651-a243-4a0f-9bc9-563b2133bd43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:41.547432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dd1651-a243-4a0f-9bc9-563b2133bd43 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:41.559015 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2865/5797] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:09:43.489253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18c9c2-e2c1-4ffc-95c9-635a4a8c59d5 service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:43.491127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4a18c9c2-e2c1-4ffc-95c9-635a4a8c59d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:43.531280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd1651-a243-4a0f-9bc9-563b2133bd43 service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:43.649184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18c9c2-e2c1-4ffc-95c9-635a4a8c59d5 service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:43.657983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e7311-39b8-4198-b370-6dcda7ae9e29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:43.658912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e7311-39b8-4198-b370-6dcda7ae9e29 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:43.659423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e7311-39b8-4198-b370-6dcda7ae9e29 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:43.659873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e7311-39b8-4198-b370-6dcda7ae9e29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:43.661884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5a3e7311-39b8-4198-b370-6dcda7ae9e29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:43.677524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd1651-a243-4a0f-9bc9-563b2133bd43 service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:43.679696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e7311-39b8-4198-b370-6dcda7ae9e29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:43.681566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e7311-39b8-4198-b370-6dcda7ae9e29 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:43.694666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:43.695583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:43.696018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:43.696421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] PATH_INFO: `/v3/users/c68e54ec2baf40908333d6e8d5d6a32b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:43.699040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c68e54ec2baf40908333d6e8d5d6a32b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:43.701980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:43.713190 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2933/5798] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:43.747515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:43.808139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c68e54ec2baf40908333d6e8d5d6a32b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:43.832870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c68e54ec2baf40908333d6e8d5d6a32b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:43.833319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c68e54ec2baf40908333d6e8d5d6a32b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:43.858764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c68e54ec2baf40908333d6e8d5d6a32b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:43.882853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c68e54ec2baf40908333d6e8d5d6a32b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:43.908349 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2866/5799] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:43 2022] DELETE /identity/v3/users/c68e54ec2baf40908333d6e8d5d6a32b => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:43.913090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37085c1e-503c-4dd3-af37-0c7b62bbd940 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:43.913387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37085c1e-503c-4dd3-af37-0c7b62bbd940 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:43.913674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37085c1e-503c-4dd3-af37-0c7b62bbd940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:43.947205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-37085c1e-503c-4dd3-af37-0c7b62bbd940 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:43.948597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-37085c1e-503c-4dd3-af37-0c7b62bbd940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:44.096947 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2934/5800] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:44.125403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:44.127605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:44.215429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bf3d3-54cc-4b8b-961e-466698530ba1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:44.221496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a2c37-837f-4709-8379-66bc7ee680c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:44.222397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a2c37-837f-4709-8379-66bc7ee680c6 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:44.224200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a2c37-837f-4709-8379-66bc7ee680c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:44.224714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a2c37-837f-4709-8379-66bc7ee680c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:44.239199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a2c37-837f-4709-8379-66bc7ee680c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:44.241875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a2c37-837f-4709-8379-66bc7ee680c6 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:44.256250 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2867/5801] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:44.909802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37085c1e-503c-4dd3-af37-0c7b62bbd940 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:44.911305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-37085c1e-503c-4dd3-af37-0c7b62bbd940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:44.934922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37085c1e-503c-4dd3-af37-0c7b62bbd940 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:44.942632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:44.943345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:44.943644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:44.943924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] PATH_INFO: `/v3/projects/315897be82e4487691ea89cd2ab54abb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:44.947501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=315897be82e4487691ea89cd2ab54abb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:44.949541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:44.998905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:45.005878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315897be82e4487691ea89cd2ab54abb', 'request_id': 'req-76ad2ab9-b77e-4396-bcc3-9805388e2356'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:45.015259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315897be82e4487691ea89cd2ab54abb', 'request_id': 'req-76ad2ab9-b77e-4396-bcc3-9805388e2356'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:45.024778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315897be82e4487691ea89cd2ab54abb', 'request_id': 'req-76ad2ab9-b77e-4396-bcc3-9805388e2356'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:45.045979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] The token cache is being invalidate because project 315897be82e4487691ea89cd2ab54abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:45.046240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:45.047008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:45.049226 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2935/5802] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:44 2022] DELETE /identity/v3/projects/315897be82e4487691ea89cd2ab54abb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:45.054860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a2c37-837f-4709-8379-66bc7ee680c6 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:45.066164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a2c37-837f-4709-8379-66bc7ee680c6 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:45.192524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:45.193316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:45.193603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:45.193881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] PATH_INFO: `/v3/users/b9663d439fda41c186b907e0d48ccb8c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:45.196051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9663d439fda41c186b907e0d48ccb8c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:45.198790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:45.276002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:45.287019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9663d439fda41c186b907e0d48ccb8c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:45.302722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9663d439fda41c186b907e0d48ccb8c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:45.303397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9663d439fda41c186b907e0d48ccb8c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:45.312717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9663d439fda41c186b907e0d48ccb8c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:45.323120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9663d439fda41c186b907e0d48ccb8c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:45.340092 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2868/5803] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:45 2022] DELETE /identity/v3/users/b9663d439fda41c186b907e0d48ccb8c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:45.392470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:45.394473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:45.468376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad2ab9-b77e-4396-bcc3-9805388e2356 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:45.551588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221ee8e-432e-474e-8307-4a5eddae9055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:45.553934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221ee8e-432e-474e-8307-4a5eddae9055 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:45.553934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221ee8e-432e-474e-8307-4a5eddae9055 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:45.554451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221ee8e-432e-474e-8307-4a5eddae9055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:45.558478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1221ee8e-432e-474e-8307-4a5eddae9055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:45.577072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221ee8e-432e-474e-8307-4a5eddae9055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:45.578027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221ee8e-432e-474e-8307-4a5eddae9055 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:45.620979 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2936/5804] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:46.732505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:46.733577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:46.841156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094de227-477a-498d-98fc-080f89f01be6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:46.848078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059006ff-e1aa-4e4b-9b80-194cd33b8086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:46.848741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059006ff-e1aa-4e4b-9b80-194cd33b8086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:46.849058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059006ff-e1aa-4e4b-9b80-194cd33b8086 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:46.849382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059006ff-e1aa-4e4b-9b80-194cd33b8086 admin admin] PATH_INFO: `/v3/projects/de7adc4281a140989ed9b89019aaaa2a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:46.851203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059006ff-e1aa-4e4b-9b80-194cd33b8086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de7adc4281a140989ed9b89019aaaa2a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:46.853688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059006ff-e1aa-4e4b-9b80-194cd33b8086 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:46.912805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-059006ff-e1aa-4e4b-9b80-194cd33b8086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de7adc4281a140989ed9b89019aaaa2a', 'request_id': 'req-059006ff-e1aa-4e4b-9b80-194cd33b8086'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:46.923822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-059006ff-e1aa-4e4b-9b80-194cd33b8086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de7adc4281a140989ed9b89019aaaa2a', 'request_id': 'req-059006ff-e1aa-4e4b-9b80-194cd33b8086'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:46.937696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-059006ff-e1aa-4e4b-9b80-194cd33b8086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de7adc4281a140989ed9b89019aaaa2a', 'request_id': 'req-059006ff-e1aa-4e4b-9b80-194cd33b8086'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:46.953774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-059006ff-e1aa-4e4b-9b80-194cd33b8086 admin admin] The token cache is being invalidate because project de7adc4281a140989ed9b89019aaaa2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:46.954442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-059006ff-e1aa-4e4b-9b80-194cd33b8086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:46.955834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-059006ff-e1aa-4e4b-9b80-194cd33b8086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:46.958083 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2869/5805] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:46 2022] DELETE /identity/v3/projects/de7adc4281a140989ed9b89019aaaa2a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:46.969048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2d3c9-00bc-4650-ad40-51f78bf27fff None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:46.969564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2d3c9-00bc-4650-ad40-51f78bf27fff None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:46.970849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2d3c9-00bc-4650-ad40-51f78bf27fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:47.004207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-81c2d3c9-00bc-4650-ad40-51f78bf27fff None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:47.011475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-81c2d3c9-00bc-4650-ad40-51f78bf27fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:47.098583 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2937/5806] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:47.107149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8471b45-1b1c-4d48-902e-f001d862fcf0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:47.107481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8471b45-1b1c-4d48-902e-f001d862fcf0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:47.107713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8471b45-1b1c-4d48-902e-f001d862fcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:47.133658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a8471b45-1b1c-4d48-902e-f001d862fcf0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:47.135215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a8471b45-1b1c-4d48-902e-f001d862fcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:47.144188 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2870/5807] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:47.148406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2d3c9-00bc-4650-ad40-51f78bf27fff None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:47.159737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2d3c9-00bc-4650-ad40-51f78bf27fff None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:47.166109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff6f2e-3d97-4a24-b262-492d54a170cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:47.166815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff6f2e-3d97-4a24-b262-492d54a170cc admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:47.167174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff6f2e-3d97-4a24-b262-492d54a170cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:47.167641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff6f2e-3d97-4a24-b262-492d54a170cc admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:47.168567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff6f2e-3d97-4a24-b262-492d54a170cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:47.169088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff6f2e-3d97-4a24-b262-492d54a170cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:47.170905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff6f2e-3d97-4a24-b262-492d54a170cc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:47.184405 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2938/5808] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:47.188797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8471b45-1b1c-4d48-902e-f001d862fcf0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:47.199146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8471b45-1b1c-4d48-902e-f001d862fcf0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:47.204471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c301b1-5e4d-46e3-8d07-f207bf1ec41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:47.205491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c301b1-5e4d-46e3-8d07-f207bf1ec41b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:47.205847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c301b1-5e4d-46e3-8d07-f207bf1ec41b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:47.206192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c301b1-5e4d-46e3-8d07-f207bf1ec41b admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:47.207075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c301b1-5e4d-46e3-8d07-f207bf1ec41b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:47.209445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c301b1-5e4d-46e3-8d07-f207bf1ec41b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:47.244370 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2871/5809] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:47 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:47.248471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff6f2e-3d97-4a24-b262-492d54a170cc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:47.260880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff6f2e-3d97-4a24-b262-492d54a170cc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:47.280120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:47.310582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c301b1-5e4d-46e3-8d07-f207bf1ec41b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:47.315273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c3c301b1-5e4d-46e3-8d07-f207bf1ec41b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:47.392945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430faa-6abc-4e3c-9c6b-d89d06b374c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:47.393587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430faa-6abc-4e3c-9c6b-d89d06b374c0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:47.393928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430faa-6abc-4e3c-9c6b-d89d06b374c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:47.394702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430faa-6abc-4e3c-9c6b-d89d06b374c0 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:47.395634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:47.395955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5430faa-6abc-4e3c-9c6b-d89d06b374c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:47.400233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5430faa-6abc-4e3c-9c6b-d89d06b374c0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:47.436993 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2939/5810] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:47 2022] POST /identity/v3/users => generated 291 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:47.439605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430faa-6abc-4e3c-9c6b-d89d06b374c0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:47.463939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430faa-6abc-4e3c-9c6b-d89d06b374c0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:47.477917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631657f0-1825-4849-806d-45d82096d4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:47.479190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631657f0-1825-4849-806d-45d82096d4e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:47.479704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631657f0-1825-4849-806d-45d82096d4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:47.480213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631657f0-1825-4849-806d-45d82096d4e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:47.481129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631657f0-1825-4849-806d-45d82096d4e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:47.483005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631657f0-1825-4849-806d-45d82096d4e6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:47.495817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c301b1-5e4d-46e3-8d07-f207bf1ec41b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:47.506904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a19df-afac-4874-8090-b0a04f18e344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:47.507591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a19df-afac-4874-8090-b0a04f18e344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:47.507861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a19df-afac-4874-8090-b0a04f18e344 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:47.508136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a19df-afac-4874-8090-b0a04f18e344 admin admin] PATH_INFO: `/v3/users/c27f6d52bbb544a2b56e259de1cd6577` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:47.511510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a19df-afac-4874-8090-b0a04f18e344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c27f6d52bbb544a2b56e259de1cd6577)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:47.515552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a19df-afac-4874-8090-b0a04f18e344 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:47.519183 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2940/5811] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:47.523378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631657f0-1825-4849-806d-45d82096d4e6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:47.547186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631657f0-1825-4849-806d-45d82096d4e6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:47.564631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c7a72-56bc-4ec9-8b02-993339412c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:47.565537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c7a72-56bc-4ec9-8b02-993339412c9c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:47.565892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c7a72-56bc-4ec9-8b02-993339412c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:47.566222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c7a72-56bc-4ec9-8b02-993339412c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:47.566899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c7a72-56bc-4ec9-8b02-993339412c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:47.568520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c7a72-56bc-4ec9-8b02-993339412c9c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:47.588878 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2941/5812] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:47.590060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c7a72-56bc-4ec9-8b02-993339412c9c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:47.601647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c7a72-56bc-4ec9-8b02-993339412c9c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:47.606398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc91a9-f7f4-46c6-a5f3-44cd66011803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:47.606941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc91a9-f7f4-46c6-a5f3-44cd66011803 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:47.607306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc91a9-f7f4-46c6-a5f3-44cd66011803 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:47.607657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc91a9-f7f4-46c6-a5f3-44cd66011803 admin admin] PATH_INFO: `/v3/projects/b4bf7143069a44a1b2d8e7129aabd5f3/users/d7bd5e318e2e4e21ab0b39dc59619f18/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:47.623017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a78a19df-afac-4874-8090-b0a04f18e344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c27f6d52bbb544a2b56e259de1cd6577'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:47.633107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc91a9-f7f4-46c6-a5f3-44cd66011803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4bf7143069a44a1b2d8e7129aabd5f3, user_id=d7bd5e318e2e4e21ab0b39dc59619f18, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:47.635987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc91a9-f7f4-46c6-a5f3-44cd66011803 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:47.646452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a78a19df-afac-4874-8090-b0a04f18e344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c27f6d52bbb544a2b56e259de1cd6577'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:47.647433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a78a19df-afac-4874-8090-b0a04f18e344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c27f6d52bbb544a2b56e259de1cd6577'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:47.656260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a78a19df-afac-4874-8090-b0a04f18e344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c27f6d52bbb544a2b56e259de1cd6577'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:47.666570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a78a19df-afac-4874-8090-b0a04f18e344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c27f6d52bbb544a2b56e259de1cd6577'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:47.676445 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2872/5813] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:47 2022] DELETE /identity/v3/users/c27f6d52bbb544a2b56e259de1cd6577 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:47.679095 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2942/5814] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:47 2022] PUT /identity/v3/projects/b4bf7143069a44a1b2d8e7129aabd5f3/users/d7bd5e318e2e4e21ab0b39dc59619f18/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:47.681410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df8ebc-2047-4087-9a90-3155888e0093 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:47.681733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df8ebc-2047-4087-9a90-3155888e0093 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:47.682014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df8ebc-2047-4087-9a90-3155888e0093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:47.688707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b2f71-9bdd-47f5-ac3f-164a7d3fe20e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:47.689085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b2f71-9bdd-47f5-ac3f-164a7d3fe20e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:47.689365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b2f71-9bdd-47f5-ac3f-164a7d3fe20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:47.723453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-43df8ebc-2047-4087-9a90-3155888e0093 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:47.724859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-43df8ebc-2047-4087-9a90-3155888e0093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:47.754812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ce0b2f71-9bdd-47f5-ac3f-164a7d3fe20e None None] MFA Rules not processed for user `d7bd5e318e2e4e21ab0b39dc59619f18`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:47.756291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ce0b2f71-9bdd-47f5-ac3f-164a7d3fe20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:47.842888 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2873/5815] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:47.857600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df8ebc-2047-4087-9a90-3155888e0093 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:47.860415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-43df8ebc-2047-4087-9a90-3155888e0093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:47.923380 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2943/5816] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:47.928543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b2f71-9bdd-47f5-ac3f-164a7d3fe20e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:47.933979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ce0b2f71-9bdd-47f5-ac3f-164a7d3fe20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:47.973972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b2f71-9bdd-47f5-ac3f-164a7d3fe20e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:47.979606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:47.992688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e8d50-1898-40c6-9cb7-03724004b13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:47.993581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e8d50-1898-40c6-9cb7-03724004b13a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:47.994077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e8d50-1898-40c6-9cb7-03724004b13a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:47.994515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e8d50-1898-40c6-9cb7-03724004b13a admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:47.995326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e8d50-1898-40c6-9cb7-03724004b13a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:48.000081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e8d50-1898-40c6-9cb7-03724004b13a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:48.020951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df8ebc-2047-4087-9a90-3155888e0093 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:48.029362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becf296-f1ff-42d8-a545-9d32e491aa7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:48.030066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becf296-f1ff-42d8-a545-9d32e491aa7a service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:48.030350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becf296-f1ff-42d8-a545-9d32e491aa7a service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:48.030626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becf296-f1ff-42d8-a545-9d32e491aa7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:48.055715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becf296-f1ff-42d8-a545-9d32e491aa7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:48.058651 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2944/5817] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:47 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:48.059004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becf296-f1ff-42d8-a545-9d32e491aa7a service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:48.063228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e8d50-1898-40c6-9cb7-03724004b13a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:48.083615 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2874/5818] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:48.116291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:48.165272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e8d50-1898-40c6-9cb7-03724004b13a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:48.175166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60529b7b-421e-4fb4-a1c5-57f6e7d7225e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:48.176588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60529b7b-421e-4fb4-a1c5-57f6e7d7225e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:48.177347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60529b7b-421e-4fb4-a1c5-57f6e7d7225e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:48.178517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60529b7b-421e-4fb4-a1c5-57f6e7d7225e admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:48.180475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60529b7b-421e-4fb4-a1c5-57f6e7d7225e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:48.183332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60529b7b-421e-4fb4-a1c5-57f6e7d7225e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:48.227491 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2945/5819] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:48 2022] POST /identity/v3/users => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:48.231683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becf296-f1ff-42d8-a545-9d32e491aa7a service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:48.242589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becf296-f1ff-42d8-a545-9d32e491aa7a service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:48.247723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af5866-365e-451d-97a7-f5ccc6a8a625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:48.248335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af5866-365e-451d-97a7-f5ccc6a8a625 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:48.248621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af5866-365e-451d-97a7-f5ccc6a8a625 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:48.249078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af5866-365e-451d-97a7-f5ccc6a8a625 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:48.249726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af5866-365e-451d-97a7-f5ccc6a8a625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:48.251339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af5866-365e-451d-97a7-f5ccc6a8a625 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:48.269789 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2875/5820] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:48.271484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60529b7b-421e-4fb4-a1c5-57f6e7d7225e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:48.284307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60529b7b-421e-4fb4-a1c5-57f6e7d7225e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:48.292493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691f75e-a376-4b75-833d-5dd667ed485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:48.293647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691f75e-a376-4b75-833d-5dd667ed485b admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:48.294289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691f75e-a376-4b75-833d-5dd667ed485b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:48.294790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691f75e-a376-4b75-833d-5dd667ed485b admin admin] PATH_INFO: `/v3/projects/8ef3bc87a63e4716a53bfd1b7814f2f9/users/bf455629bd744ec3bb2710f886c359c6/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:48.316703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691f75e-a376-4b75-833d-5dd667ed485b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef3bc87a63e4716a53bfd1b7814f2f9, user_id=bf455629bd744ec3bb2710f886c359c6, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:48.319828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691f75e-a376-4b75-833d-5dd667ed485b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:48.343045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af5866-365e-451d-97a7-f5ccc6a8a625 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:48.343927 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2946/5821] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:48 2022] PUT /identity/v3/projects/8ef3bc87a63e4716a53bfd1b7814f2f9/users/bf455629bd744ec3bb2710f886c359c6/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:48.385086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af5866-365e-451d-97a7-f5ccc6a8a625 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:48.458245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30189d36-ec92-40a1-abb9-cba50088242c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:48.459015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30189d36-ec92-40a1-abb9-cba50088242c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:48.459309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30189d36-ec92-40a1-abb9-cba50088242c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:48.459660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30189d36-ec92-40a1-abb9-cba50088242c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:48.460490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30189d36-ec92-40a1-abb9-cba50088242c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:48.462164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30189d36-ec92-40a1-abb9-cba50088242c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:48.479405 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2876/5822] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:48.483057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691f75e-a376-4b75-833d-5dd667ed485b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:48.497787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691f75e-a376-4b75-833d-5dd667ed485b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:48.505333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49b3f5-42d9-48fa-878f-81c84ae21174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:48.506231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49b3f5-42d9-48fa-878f-81c84ae21174 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:48.506711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49b3f5-42d9-48fa-878f-81c84ae21174 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:48.507151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49b3f5-42d9-48fa-878f-81c84ae21174 admin admin] PATH_INFO: `/v3/projects/8ef3bc87a63e4716a53bfd1b7814f2f9/users/bf455629bd744ec3bb2710f886c359c6/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:48.509827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49b3f5-42d9-48fa-878f-81c84ae21174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef3bc87a63e4716a53bfd1b7814f2f9, user_id=bf455629bd744ec3bb2710f886c359c6, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:48.512518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49b3f5-42d9-48fa-878f-81c84ae21174 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:48.526100 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2947/5823] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:48 2022] PUT /identity/v3/projects/8ef3bc87a63e4716a53bfd1b7814f2f9/users/bf455629bd744ec3bb2710f886c359c6/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:48.539337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fe6d1-eb98-4a1b-b8d8-8752de414180 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:48.539684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fe6d1-eb98-4a1b-b8d8-8752de414180 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:48.540259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fe6d1-eb98-4a1b-b8d8-8752de414180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:48.595797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a97fe6d1-eb98-4a1b-b8d8-8752de414180 None None] MFA Rules not processed for user `bf455629bd744ec3bb2710f886c359c6`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:48.597493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a97fe6d1-eb98-4a1b-b8d8-8752de414180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:48.715863 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2877/5824] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:48.728372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49b3f5-42d9-48fa-878f-81c84ae21174 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:48.730597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cb49b3f5-42d9-48fa-878f-81c84ae21174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:48.764208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fe6d1-eb98-4a1b-b8d8-8752de414180 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:48.766584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a97fe6d1-eb98-4a1b-b8d8-8752de414180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:48.857492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:48.897064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fe6d1-eb98-4a1b-b8d8-8752de414180 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:48.909392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:48.910060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:48.911031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:48.911322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] PATH_INFO: `/v3/projects/3623dca2492247ea9c881eae91b76964` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:48.912527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3623dca2492247ea9c881eae91b76964)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:48.912915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49b3f5-42d9-48fa-878f-81c84ae21174 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:48.916008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:48.926387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133be05-13f3-4651-938a-563019241026 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:48.929073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133be05-13f3-4651-938a-563019241026 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:48.929073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133be05-13f3-4651-938a-563019241026 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:48.932486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133be05-13f3-4651-938a-563019241026 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:48.975141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3623dca2492247ea9c881eae91b76964', 'request_id': 'req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:48.984567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133be05-13f3-4651-938a-563019241026 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:48.986722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133be05-13f3-4651-938a-563019241026 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:48.992182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3623dca2492247ea9c881eae91b76964', 'request_id': 'req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:49.005887 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2948/5825] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:49.013335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3623dca2492247ea9c881eae91b76964', 'request_id': 'req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:49.032161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] The token cache is being invalidate because project 3623dca2492247ea9c881eae91b76964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:49.033000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:49.033914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:49.036397 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2878/5826] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:48 2022] DELETE /identity/v3/projects/3623dca2492247ea9c881eae91b76964 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:49.041115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bba58-e646-4c46-9ea7-7960f9b4db2d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:49.041411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bba58-e646-4c46-9ea7-7960f9b4db2d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:49.041754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bba58-e646-4c46-9ea7-7960f9b4db2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:49.077945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-518bba58-e646-4c46-9ea7-7960f9b4db2d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:49.080483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-518bba58-e646-4c46-9ea7-7960f9b4db2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:49.150709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:49.152583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:49.244663 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2949/5827] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:49.253957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb403ce2-6703-4d1b-8039-f7b3dc5607fb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:49.255046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb403ce2-6703-4d1b-8039-f7b3dc5607fb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:49.255981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb403ce2-6703-4d1b-8039-f7b3dc5607fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:49.280105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d48aeb-ba58-4a8c-a124-ffe58c37482a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:49.285411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34ce14-af4b-4071-a04e-177a7b14119a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:49.286305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34ce14-af4b-4071-a04e-177a7b14119a service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:49.286959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34ce14-af4b-4071-a04e-177a7b14119a service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:49.287321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34ce14-af4b-4071-a04e-177a7b14119a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:49.289345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-af34ce14-af4b-4071-a04e-177a7b14119a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:49.310209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-eb403ce2-6703-4d1b-8039-f7b3dc5607fb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:49.311492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-eb403ce2-6703-4d1b-8039-f7b3dc5607fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:49.321409 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2950/5828] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:49.325363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb403ce2-6703-4d1b-8039-f7b3dc5607fb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:49.336601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb403ce2-6703-4d1b-8039-f7b3dc5607fb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:49.343467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114ab5a-1c12-430c-a6d9-81df9931e0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:49.344108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114ab5a-1c12-430c-a6d9-81df9931e0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:49.344460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114ab5a-1c12-430c-a6d9-81df9931e0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:49.344746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114ab5a-1c12-430c-a6d9-81df9931e0d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:49.345495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114ab5a-1c12-430c-a6d9-81df9931e0d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:49.345833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114ab5a-1c12-430c-a6d9-81df9931e0d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:49.347600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114ab5a-1c12-430c-a6d9-81df9931e0d7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:49.362015 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2951/5829] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:49.366441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114ab5a-1c12-430c-a6d9-81df9931e0d7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:49.378082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:49.393634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114ab5a-1c12-430c-a6d9-81df9931e0d7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:49.410310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4a384-84c2-46b7-90d4-8c836268801f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:49.410954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4a384-84c2-46b7-90d4-8c836268801f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:49.411245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4a384-84c2-46b7-90d4-8c836268801f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:49.411530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4a384-84c2-46b7-90d4-8c836268801f admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:49.412257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4a384-84c2-46b7-90d4-8c836268801f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:49.415547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4a384-84c2-46b7-90d4-8c836268801f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:49.431928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34ce14-af4b-4071-a04e-177a7b14119a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:49.433498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34ce14-af4b-4071-a04e-177a7b14119a service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:49.478045 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2952/5830] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:49 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:49.482053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4a384-84c2-46b7-90d4-8c836268801f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:49.482555 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2879/5831] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:09:49.708197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4a384-84c2-46b7-90d4-8c836268801f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:49.752032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94d91b-f7f6-4ad4-b48b-a39feb468896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:49.753213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94d91b-f7f6-4ad4-b48b-a39feb468896 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:49.755774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94d91b-f7f6-4ad4-b48b-a39feb468896 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:49.758091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94d91b-f7f6-4ad4-b48b-a39feb468896 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:49.758977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94d91b-f7f6-4ad4-b48b-a39feb468896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:49.761471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94d91b-f7f6-4ad4-b48b-a39feb468896 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:49.810101 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2953/5832] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:49 2022] POST /identity/v3/users => generated 287 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:49.811394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34ce14-af4b-4071-a04e-177a7b14119a service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:49.825497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34ce14-af4b-4071-a04e-177a7b14119a service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:49.835626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1f796-efbf-4caa-8b30-432bd309cab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:49.836439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1f796-efbf-4caa-8b30-432bd309cab1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:49.836885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1f796-efbf-4caa-8b30-432bd309cab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:49.837409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1f796-efbf-4caa-8b30-432bd309cab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:49.849869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1f796-efbf-4caa-8b30-432bd309cab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:49.853949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1f796-efbf-4caa-8b30-432bd309cab1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:49.885167 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2880/5833] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:49.889841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94d91b-f7f6-4ad4-b48b-a39feb468896 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:49.907081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94d91b-f7f6-4ad4-b48b-a39feb468896 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:49.921417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0730c-a519-42f5-9aee-8865f8795d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:49.922404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0730c-a519-42f5-9aee-8865f8795d77 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:49.922766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0730c-a519-42f5-9aee-8865f8795d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:49.923220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0730c-a519-42f5-9aee-8865f8795d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:49.924490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0730c-a519-42f5-9aee-8865f8795d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:49.926185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0730c-a519-42f5-9aee-8865f8795d77 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:49.943313 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2954/5834] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:49.944263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1f796-efbf-4caa-8b30-432bd309cab1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:49.960865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1f796-efbf-4caa-8b30-432bd309cab1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:49.971898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c2aeb-ca9a-4ac8-8449-73fb1b0e689b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:49.972934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c2aeb-ca9a-4ac8-8449-73fb1b0e689b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:49.972934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c2aeb-ca9a-4ac8-8449-73fb1b0e689b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:49.973190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c2aeb-ca9a-4ac8-8449-73fb1b0e689b admin admin] PATH_INFO: `/v3/projects/77a2358e02c446288638170b6ec6d144/users/ced09232d29a41d3a0d075fcc53a7e4e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:49.999676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c2aeb-ca9a-4ac8-8449-73fb1b0e689b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a2358e02c446288638170b6ec6d144, user_id=ced09232d29a41d3a0d075fcc53a7e4e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:50.003836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c2aeb-ca9a-4ac8-8449-73fb1b0e689b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:50.022189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:50.031161 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2881/5835] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:49 2022] PUT /identity/v3/projects/77a2358e02c446288638170b6ec6d144/users/ced09232d29a41d3a0d075fcc53a7e4e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:50.043235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263cf96-601a-44bc-b1b6-5b48154523d1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:50.044146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263cf96-601a-44bc-b1b6-5b48154523d1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:50.044502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263cf96-601a-44bc-b1b6-5b48154523d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:50.069668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c2aeb-ca9a-4ac8-8449-73fb1b0e689b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:50.071565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f28c2aeb-ca9a-4ac8-8449-73fb1b0e689b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:50.082254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0263cf96-601a-44bc-b1b6-5b48154523d1 None None] MFA Rules not processed for user `ced09232d29a41d3a0d075fcc53a7e4e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:50.083491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0263cf96-601a-44bc-b1b6-5b48154523d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:50.195134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c2aeb-ca9a-4ac8-8449-73fb1b0e689b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:50.201065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60a293-c5cd-4323-805b-658a93bb1210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:50.201984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60a293-c5cd-4323-805b-658a93bb1210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:50.206175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60a293-c5cd-4323-805b-658a93bb1210 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:50.206175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60a293-c5cd-4323-805b-658a93bb1210 admin admin] PATH_INFO: `/v3/users/d7bd5e318e2e4e21ab0b39dc59619f18` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:50.207816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60a293-c5cd-4323-805b-658a93bb1210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7bd5e318e2e4e21ab0b39dc59619f18)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:50.210245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60a293-c5cd-4323-805b-658a93bb1210 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:50.230563 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2955/5836] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:50.255482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263cf96-601a-44bc-b1b6-5b48154523d1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:50.257107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0263cf96-601a-44bc-b1b6-5b48154523d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:50.311215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f60a293-c5cd-4323-805b-658a93bb1210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7bd5e318e2e4e21ab0b39dc59619f18'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:50.337325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f60a293-c5cd-4323-805b-658a93bb1210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7bd5e318e2e4e21ab0b39dc59619f18'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:50.338351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f60a293-c5cd-4323-805b-658a93bb1210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7bd5e318e2e4e21ab0b39dc59619f18'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:50.348029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f60a293-c5cd-4323-805b-658a93bb1210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7bd5e318e2e4e21ab0b39dc59619f18'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:50.359077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f60a293-c5cd-4323-805b-658a93bb1210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7bd5e318e2e4e21ab0b39dc59619f18'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:50.368607 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2882/5837] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:50 2022] DELETE /identity/v3/users/d7bd5e318e2e4e21ab0b39dc59619f18 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:50.374042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ef29d-4a41-47e4-8650-ebde294004ef None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:50.374439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ef29d-4a41-47e4-8650-ebde294004ef None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:50.374821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ef29d-4a41-47e4-8650-ebde294004ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:50.400946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-437ef29d-4a41-47e4-8650-ebde294004ef None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:50.402655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-437ef29d-4a41-47e4-8650-ebde294004ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:50.404490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263cf96-601a-44bc-b1b6-5b48154523d1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:50.411937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:50.498181 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2883/5838] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:50.506203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527aebf-849a-4199-a64c-818d6beaa965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:50.507061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527aebf-849a-4199-a64c-818d6beaa965 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:50.507425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527aebf-849a-4199-a64c-818d6beaa965 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:50.507770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527aebf-849a-4199-a64c-818d6beaa965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:50.509724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4527aebf-849a-4199-a64c-818d6beaa965 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:50.516001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ef29d-4a41-47e4-8650-ebde294004ef None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:50.517892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-437ef29d-4a41-47e4-8650-ebde294004ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:50.541119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ef29d-4a41-47e4-8650-ebde294004ef None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:50.554421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971d554-2181-47cd-9a27-9b6fb44a6cde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:50.555064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971d554-2181-47cd-9a27-9b6fb44a6cde service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:50.555349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971d554-2181-47cd-9a27-9b6fb44a6cde service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:50.555631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971d554-2181-47cd-9a27-9b6fb44a6cde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:50.587063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971d554-2181-47cd-9a27-9b6fb44a6cde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:50.595381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971d554-2181-47cd-9a27-9b6fb44a6cde service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:50.619017 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2884/5839] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:50.662994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527aebf-849a-4199-a64c-818d6beaa965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:50.664862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527aebf-849a-4199-a64c-818d6beaa965 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:50.678381 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2956/5840] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:51.286937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971d554-2181-47cd-9a27-9b6fb44a6cde service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:51.288429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e971d554-2181-47cd-9a27-9b6fb44a6cde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:51.317451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971d554-2181-47cd-9a27-9b6fb44a6cde service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:51.324306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:51.324996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:51.325279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:51.325813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] PATH_INFO: `/v3/projects/b4bf7143069a44a1b2d8e7129aabd5f3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:51.328015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4bf7143069a44a1b2d8e7129aabd5f3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:51.330468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:51.333344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:51.404490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4bf7143069a44a1b2d8e7129aabd5f3', 'request_id': 'req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:51.413530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4bf7143069a44a1b2d8e7129aabd5f3', 'request_id': 'req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:51.424210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4bf7143069a44a1b2d8e7129aabd5f3', 'request_id': 'req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:51.439965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] The token cache is being invalidate because project b4bf7143069a44a1b2d8e7129aabd5f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:51.440443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:51.442254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:51.444846 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2885/5841] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:51 2022] DELETE /identity/v3/projects/b4bf7143069a44a1b2d8e7129aabd5f3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:51.448672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527aebf-849a-4199-a64c-818d6beaa965 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:51.464734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527aebf-849a-4199-a64c-818d6beaa965 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:51.554217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb731b92-cf56-4be1-8a26-58a33e4ee3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:51.554936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb731b92-cf56-4be1-8a26-58a33e4ee3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:51.555229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb731b92-cf56-4be1-8a26-58a33e4ee3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:51.555508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb731b92-cf56-4be1-8a26-58a33e4ee3b8 admin admin] PATH_INFO: `/v3/users/bf455629bd744ec3bb2710f886c359c6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:51.556702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb731b92-cf56-4be1-8a26-58a33e4ee3b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf455629bd744ec3bb2710f886c359c6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:51.558616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb731b92-cf56-4be1-8a26-58a33e4ee3b8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:51.614467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fb731b92-cf56-4be1-8a26-58a33e4ee3b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf455629bd744ec3bb2710f886c359c6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:51.627062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:51.628953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fb731b92-cf56-4be1-8a26-58a33e4ee3b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf455629bd744ec3bb2710f886c359c6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:51.638024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fb731b92-cf56-4be1-8a26-58a33e4ee3b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf455629bd744ec3bb2710f886c359c6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:51.638772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fb731b92-cf56-4be1-8a26-58a33e4ee3b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf455629bd744ec3bb2710f886c359c6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:51.639391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fb731b92-cf56-4be1-8a26-58a33e4ee3b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf455629bd744ec3bb2710f886c359c6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:51.656006 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2957/5842] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:51 2022] DELETE /identity/v3/users/bf455629bd744ec3bb2710f886c359c6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:52.164325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:52.166080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:52.181175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775e87f-4bf3-405e-a8bb-1af4b9f5b7d9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:52.293051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f988ddf-a3d0-4b20-9557-104ab042487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:52.294651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f988ddf-a3d0-4b20-9557-104ab042487a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:52.295278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f988ddf-a3d0-4b20-9557-104ab042487a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:52.295779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f988ddf-a3d0-4b20-9557-104ab042487a admin admin] PATH_INFO: `/v3/projects/8ef3bc87a63e4716a53bfd1b7814f2f9` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:52.297539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f988ddf-a3d0-4b20-9557-104ab042487a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef3bc87a63e4716a53bfd1b7814f2f9)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:52.300250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f988ddf-a3d0-4b20-9557-104ab042487a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:52.368603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f988ddf-a3d0-4b20-9557-104ab042487a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef3bc87a63e4716a53bfd1b7814f2f9', 'request_id': 'req-9f988ddf-a3d0-4b20-9557-104ab042487a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:52.378012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f988ddf-a3d0-4b20-9557-104ab042487a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef3bc87a63e4716a53bfd1b7814f2f9', 'request_id': 'req-9f988ddf-a3d0-4b20-9557-104ab042487a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:52.385157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f988ddf-a3d0-4b20-9557-104ab042487a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef3bc87a63e4716a53bfd1b7814f2f9', 'request_id': 'req-9f988ddf-a3d0-4b20-9557-104ab042487a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:52.398400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f988ddf-a3d0-4b20-9557-104ab042487a admin admin] The token cache is being invalidate because project 8ef3bc87a63e4716a53bfd1b7814f2f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:52.398758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f988ddf-a3d0-4b20-9557-104ab042487a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:52.399429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f988ddf-a3d0-4b20-9557-104ab042487a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:52.402104 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2886/5843] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:52 2022] DELETE /identity/v3/projects/8ef3bc87a63e4716a53bfd1b7814f2f9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:52.406923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768055f8-b2b6-4cba-a4db-58fa2e3c9aaf None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:52.407403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768055f8-b2b6-4cba-a4db-58fa2e3c9aaf None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:52.408080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768055f8-b2b6-4cba-a4db-58fa2e3c9aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:52.435158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-768055f8-b2b6-4cba-a4db-58fa2e3c9aaf None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:52.436708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-768055f8-b2b6-4cba-a4db-58fa2e3c9aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:52.533002 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2958/5844] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:52.542955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd42e2-7caa-43fe-be0f-8fe9817a5666 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:52.543248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd42e2-7caa-43fe-be0f-8fe9817a5666 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:52.543551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd42e2-7caa-43fe-be0f-8fe9817a5666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:52.583215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-24fd42e2-7caa-43fe-be0f-8fe9817a5666 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:52.588647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-24fd42e2-7caa-43fe-be0f-8fe9817a5666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:52.640006 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2887/5845] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:52.645261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768055f8-b2b6-4cba-a4db-58fa2e3c9aaf None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:52.661872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768055f8-b2b6-4cba-a4db-58fa2e3c9aaf None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:52.672500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbda2cb-b617-4584-91ae-74afe6171d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:52.674216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbda2cb-b617-4584-91ae-74afe6171d4e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:52.674813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbda2cb-b617-4584-91ae-74afe6171d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:52.675347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbda2cb-b617-4584-91ae-74afe6171d4e admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:52.676582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbda2cb-b617-4584-91ae-74afe6171d4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:52.677482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbda2cb-b617-4584-91ae-74afe6171d4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:52.680228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbda2cb-b617-4584-91ae-74afe6171d4e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:52.709866 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2959/5846] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:52.714097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd42e2-7caa-43fe-be0f-8fe9817a5666 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:52.739169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd42e2-7caa-43fe-be0f-8fe9817a5666 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:52.750330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be04321-084d-4e7e-a7f5-c78a3b0077e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:52.752089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be04321-084d-4e7e-a7f5-c78a3b0077e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:52.752609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be04321-084d-4e7e-a7f5-c78a3b0077e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:52.753159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be04321-084d-4e7e-a7f5-c78a3b0077e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:52.754843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be04321-084d-4e7e-a7f5-c78a3b0077e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:52.758244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be04321-084d-4e7e-a7f5-c78a3b0077e9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:52.773985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:52.802043 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2888/5847] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:52 2022] POST /identity/v3/projects => generated 369 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:52.806765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbda2cb-b617-4584-91ae-74afe6171d4e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:52.832527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbda2cb-b617-4584-91ae-74afe6171d4e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:52.933224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0850f6-e4e0-4306-abd4-ef7cc88f0315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:52.934206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0850f6-e4e0-4306-abd4-ef7cc88f0315 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:52.934935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0850f6-e4e0-4306-abd4-ef7cc88f0315 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:52.935441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0850f6-e4e0-4306-abd4-ef7cc88f0315 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:52.936876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0850f6-e4e0-4306-abd4-ef7cc88f0315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:52.939715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0850f6-e4e0-4306-abd4-ef7cc88f0315 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:52.965798 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2960/5848] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:52 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:52.969863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be04321-084d-4e7e-a7f5-c78a3b0077e9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:52.981953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be04321-084d-4e7e-a7f5-c78a3b0077e9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:52.987316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209cf43-a9bc-4b21-8eef-5e3ed2a2f037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:52.987929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209cf43-a9bc-4b21-8eef-5e3ed2a2f037 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:52.988218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209cf43-a9bc-4b21-8eef-5e3ed2a2f037 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:52.988494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209cf43-a9bc-4b21-8eef-5e3ed2a2f037 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:52.989165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209cf43-a9bc-4b21-8eef-5e3ed2a2f037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:52.990910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209cf43-a9bc-4b21-8eef-5e3ed2a2f037 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:53.005514 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2889/5849] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:53.006952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0850f6-e4e0-4306-abd4-ef7cc88f0315 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:53.017897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0850f6-e4e0-4306-abd4-ef7cc88f0315 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:53.024735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c948359-9d03-44dc-8b0d-531f07a32f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:53.025575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c948359-9d03-44dc-8b0d-531f07a32f66 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:53.025993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c948359-9d03-44dc-8b0d-531f07a32f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:53.026384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c948359-9d03-44dc-8b0d-531f07a32f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:53.027406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c948359-9d03-44dc-8b0d-531f07a32f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:53.029129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c948359-9d03-44dc-8b0d-531f07a32f66 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:53.041854 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2961/5850] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:53.046075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209cf43-a9bc-4b21-8eef-5e3ed2a2f037 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:53.056461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209cf43-a9bc-4b21-8eef-5e3ed2a2f037 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:53.061267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f32bb-0831-4e9c-b378-035477c35aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:53.062092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f32bb-0831-4e9c-b378-035477c35aca admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:53.062341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f32bb-0831-4e9c-b378-035477c35aca admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:53.062640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f32bb-0831-4e9c-b378-035477c35aca admin admin] PATH_INFO: `/v3/projects/05721125bedf48bfa63e34906eaa5f8b/users/d458390532f741c6afb028530be2de23/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:53.080034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f32bb-0831-4e9c-b378-035477c35aca admin admin] RBAC: Authorizing `identity:create_grant(project_id=05721125bedf48bfa63e34906eaa5f8b, user_id=d458390532f741c6afb028530be2de23, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:53.083007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f32bb-0831-4e9c-b378-035477c35aca admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:53.101644 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2890/5851] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:53 2022] PUT /identity/v3/projects/05721125bedf48bfa63e34906eaa5f8b/users/d458390532f741c6afb028530be2de23/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:53.123642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08518393-613e-4d3a-b200-086b380f3421 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:53.124424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08518393-613e-4d3a-b200-086b380f3421 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:53.125631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08518393-613e-4d3a-b200-086b380f3421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:53.166496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-08518393-613e-4d3a-b200-086b380f3421 None None] MFA Rules not processed for user `d458390532f741c6afb028530be2de23`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:53.184458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-08518393-613e-4d3a-b200-086b380f3421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:53.248280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:53.300149 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2962/5852] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:53.314407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f32bb-0831-4e9c-b378-035477c35aca admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:53.316883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-085f32bb-0831-4e9c-b378-035477c35aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:53.394682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f32bb-0831-4e9c-b378-035477c35aca admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:53.398415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee4eb9-d2f6-4a96-8d91-d1c19c636045 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:53.399036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee4eb9-d2f6-4a96-8d91-d1c19c636045 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:53.399323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee4eb9-d2f6-4a96-8d91-d1c19c636045 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:53.399603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee4eb9-d2f6-4a96-8d91-d1c19c636045 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:53.409305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee4eb9-d2f6-4a96-8d91-d1c19c636045 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:53.410780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee4eb9-d2f6-4a96-8d91-d1c19c636045 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:53.420450 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2891/5853] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:53.443941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08518393-613e-4d3a-b200-086b380f3421 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:53.511524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08518393-613e-4d3a-b200-086b380f3421 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:53.516285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:53.517488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:53.518030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:53.518520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] PATH_INFO: `/v3/users/d458390532f741c6afb028530be2de23` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:53.519888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=d458390532f741c6afb028530be2de23)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:53.522119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:53.595871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd458390532f741c6afb028530be2de23'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:53.612193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd458390532f741c6afb028530be2de23'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:53.625079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd458390532f741c6afb028530be2de23'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:53.630118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd458390532f741c6afb028530be2de23'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:53.630790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd458390532f741c6afb028530be2de23'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:53.669812 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2963/5854] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:53 2022] DELETE /identity/v3/users/d458390532f741c6afb028530be2de23 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:53.671646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d706ee-3ad9-400c-b9df-0142dec43a4e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:53.671934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d706ee-3ad9-400c-b9df-0142dec43a4e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:53.672212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d706ee-3ad9-400c-b9df-0142dec43a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:53.696670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:53.698183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:53.707123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a2d706ee-3ad9-400c-b9df-0142dec43a4e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:53.708366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a2d706ee-3ad9-400c-b9df-0142dec43a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:53.773404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:53.783366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c66b8f-dfcd-4833-ad94-3ecc798762ee admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:53.788171 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2892/5855] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:53.791908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990e84f-a636-448e-acd3-4f53d998741e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:53.792528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990e84f-a636-448e-acd3-4f53d998741e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:53.792881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990e84f-a636-448e-acd3-4f53d998741e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:53.793257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990e84f-a636-448e-acd3-4f53d998741e admin admin] PATH_INFO: `/v3/users/7120c389feb14e319b2d3a8ace28dbe2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:53.794488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990e84f-a636-448e-acd3-4f53d998741e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7120c389feb14e319b2d3a8ace28dbe2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:53.796495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990e84f-a636-448e-acd3-4f53d998741e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:53.800483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d706ee-3ad9-400c-b9df-0142dec43a4e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:53.802549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a2d706ee-3ad9-400c-b9df-0142dec43a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:53.852032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a990e84f-a636-448e-acd3-4f53d998741e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7120c389feb14e319b2d3a8ace28dbe2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:53.867427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a990e84f-a636-448e-acd3-4f53d998741e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7120c389feb14e319b2d3a8ace28dbe2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:53.878057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a990e84f-a636-448e-acd3-4f53d998741e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7120c389feb14e319b2d3a8ace28dbe2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:53.878299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d706ee-3ad9-400c-b9df-0142dec43a4e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:53.878928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a990e84f-a636-448e-acd3-4f53d998741e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7120c389feb14e319b2d3a8ace28dbe2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:53.879671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a990e84f-a636-448e-acd3-4f53d998741e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7120c389feb14e319b2d3a8ace28dbe2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:53.882439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ea1c3-bbfe-4f43-8745-266abdc830a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:53.883060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ea1c3-bbfe-4f43-8745-266abdc830a3 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:53.883348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ea1c3-bbfe-4f43-8745-266abdc830a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:53.883643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ea1c3-bbfe-4f43-8745-266abdc830a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:53.885420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-144ea1c3-bbfe-4f43-8745-266abdc830a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:53.895048 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2964/5856] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:53 2022] DELETE /identity/v3/users/7120c389feb14e319b2d3a8ace28dbe2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:53.899371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1e020-497a-428a-9e29-e1ef8316dfba None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:53.899670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1e020-497a-428a-9e29-e1ef8316dfba None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:53.900001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ea1c3-bbfe-4f43-8745-266abdc830a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:53.900122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1e020-497a-428a-9e29-e1ef8316dfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:53.901636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ea1c3-bbfe-4f43-8745-266abdc830a3 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:53.923815 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2893/5857] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:09:53.940366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-43e1e020-497a-428a-9e29-e1ef8316dfba None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:53.942375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-43e1e020-497a-428a-9e29-e1ef8316dfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:54.023948 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2965/5858] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:54.034076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ea1c3-bbfe-4f43-8745-266abdc830a3 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:54.040297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-144ea1c3-bbfe-4f43-8745-266abdc830a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:54.044206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:54.054718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ea1c3-bbfe-4f43-8745-266abdc830a3 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:54.112360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37999ef-cbc6-462b-9f5d-9677f4f81f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:54.113494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37999ef-cbc6-462b-9f5d-9677f4f81f2a service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:54.113794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37999ef-cbc6-462b-9f5d-9677f4f81f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:54.114079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37999ef-cbc6-462b-9f5d-9677f4f81f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:54.127776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37999ef-cbc6-462b-9f5d-9677f4f81f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:54.129286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37999ef-cbc6-462b-9f5d-9677f4f81f2a service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:54.143979 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2894/5859] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:54.433243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1e020-497a-428a-9e29-e1ef8316dfba None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:54.435901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-43e1e020-497a-428a-9e29-e1ef8316dfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:54.470655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1e020-497a-428a-9e29-e1ef8316dfba None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:54.484692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:54.486293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:54.496302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:54.496586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96 admin admin] PATH_INFO: `/v3/projects/05721125bedf48bfa63e34906eaa5f8b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:54.500679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05721125bedf48bfa63e34906eaa5f8b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:54.504219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:54.540007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37999ef-cbc6-462b-9f5d-9677f4f81f2a service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:54.542056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c37999ef-cbc6-462b-9f5d-9677f4f81f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:54.589392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37999ef-cbc6-462b-9f5d-9677f4f81f2a service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:54.606832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:54.607745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:54.608035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:54.608334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54 admin admin] PATH_INFO: `/v3/projects/73e86a5495fd4f618208750f7c6d6f14` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:54.610004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e86a5495fd4f618208750f7c6d6f14)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:54.613777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:54.632559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05721125bedf48bfa63e34906eaa5f8b', 'request_id': 'req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:54.645447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05721125bedf48bfa63e34906eaa5f8b', 'request_id': 'req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:54.655603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05721125bedf48bfa63e34906eaa5f8b', 'request_id': 'req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:54.663960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:54.676118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96 admin admin] The token cache is being invalidate because project 05721125bedf48bfa63e34906eaa5f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:54.676341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:54.682028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4efc33ea-4c11-46e5-8d3d-51bb7b7a4e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:54.684400 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2966/5860] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:54 2022] DELETE /identity/v3/projects/05721125bedf48bfa63e34906eaa5f8b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:54.688129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e86a5495fd4f618208750f7c6d6f14', 'request_id': 'req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:54.689448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1a3db-621b-4350-b488-4acbb2d55db5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:54.689906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1a3db-621b-4350-b488-4acbb2d55db5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:54.693026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1a3db-621b-4350-b488-4acbb2d55db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:54.711810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e86a5495fd4f618208750f7c6d6f14', 'request_id': 'req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:54.717140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-89d1a3db-621b-4350-b488-4acbb2d55db5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:54.718523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-89d1a3db-621b-4350-b488-4acbb2d55db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:54.721517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e86a5495fd4f618208750f7c6d6f14', 'request_id': 'req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:54.751131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54 admin admin] The token cache is being invalidate because project 73e86a5495fd4f618208750f7c6d6f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:54.752471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:54.754377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1a9ab825-7fb0-40d6-85fc-b5ee049e8e54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:54.758286 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2895/5861] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:54 2022] DELETE /identity/v3/projects/73e86a5495fd4f618208750f7c6d6f14 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:54.764097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc047f7-7598-4d27-8635-0741eaf09912 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:54.764557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc047f7-7598-4d27-8635-0741eaf09912 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:54.765068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc047f7-7598-4d27-8635-0741eaf09912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:54.808255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3cc047f7-7598-4d27-8635-0741eaf09912 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:54.810418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3cc047f7-7598-4d27-8635-0741eaf09912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:54.830961 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2967/5862] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:54.833700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5141b-4d33-4d8a-befd-01d26b8ffa4d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:54.834073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5141b-4d33-4d8a-befd-01d26b8ffa4d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:54.834563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5141b-4d33-4d8a-befd-01d26b8ffa4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:54.865343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3ed5141b-4d33-4d8a-befd-01d26b8ffa4d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:54.866593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3ed5141b-4d33-4d8a-befd-01d26b8ffa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:54.879118 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2968/5863] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:54.880637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5141b-4d33-4d8a-befd-01d26b8ffa4d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:54.895672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5141b-4d33-4d8a-befd-01d26b8ffa4d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:54.903791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaafe3-e0f1-4461-a161-59178cdd6096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:54.904843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaafe3-e0f1-4461-a161-59178cdd6096 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:54.905376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaafe3-e0f1-4461-a161-59178cdd6096 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:54.905895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaafe3-e0f1-4461-a161-59178cdd6096 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:54.906810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbaafe3-e0f1-4461-a161-59178cdd6096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:54.907484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbaafe3-e0f1-4461-a161-59178cdd6096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:54.912037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbaafe3-e0f1-4461-a161-59178cdd6096 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:54.913051 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2896/5864] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:54.914600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38103522-7942-4bdf-88af-f8d225977b6e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:54.914958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38103522-7942-4bdf-88af-f8d225977b6e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:54.915302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38103522-7942-4bdf-88af-f8d225977b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:54.934453 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2969/5865] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:09:54.936861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaafe3-e0f1-4461-a161-59178cdd6096 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:54.941954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-38103522-7942-4bdf-88af-f8d225977b6e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:54.947887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-38103522-7942-4bdf-88af-f8d225977b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:54.951742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaafe3-e0f1-4461-a161-59178cdd6096 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:54.957400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd27299-4cd9-4fa2-a755-f1dcb13ec828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:54.958294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd27299-4cd9-4fa2-a755-f1dcb13ec828 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:54.958586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd27299-4cd9-4fa2-a755-f1dcb13ec828 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:54.959368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd27299-4cd9-4fa2-a755-f1dcb13ec828 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:54.959516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd27299-4cd9-4fa2-a755-f1dcb13ec828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:54.961491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd27299-4cd9-4fa2-a755-f1dcb13ec828 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:54.962466 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2897/5866] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:54.967699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38103522-7942-4bdf-88af-f8d225977b6e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:54.980096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38103522-7942-4bdf-88af-f8d225977b6e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:54.986214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1147aea-0c07-4dad-8437-ec1e521088c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:54.987250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1147aea-0c07-4dad-8437-ec1e521088c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:54.987542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1147aea-0c07-4dad-8437-ec1e521088c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:54.987829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1147aea-0c07-4dad-8437-ec1e521088c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:54.988502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1147aea-0c07-4dad-8437-ec1e521088c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:54.988814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1147aea-0c07-4dad-8437-ec1e521088c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:54.990548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1147aea-0c07-4dad-8437-ec1e521088c7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:55.000886 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2970/5867] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:54 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:55.005310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd27299-4cd9-4fa2-a755-f1dcb13ec828 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:55.013736 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2898/5868] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:55.015370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1147aea-0c07-4dad-8437-ec1e521088c7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:55.031813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1147aea-0c07-4dad-8437-ec1e521088c7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:55.040092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:55.093371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd27299-4cd9-4fa2-a755-f1dcb13ec828 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:55.101143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e24b96-ecbc-486c-8f76-09e0e437f3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:55.101777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e24b96-ecbc-486c-8f76-09e0e437f3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.102069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e24b96-ecbc-486c-8f76-09e0e437f3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.102350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e24b96-ecbc-486c-8f76-09e0e437f3b6 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.103193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e24b96-ecbc-486c-8f76-09e0e437f3b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:55.105604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e24b96-ecbc-486c-8f76-09e0e437f3b6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:55.119976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbb87-6f81-4343-94eb-a204ca47844e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:55.120847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbb87-6f81-4343-94eb-a204ca47844e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.121168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbb87-6f81-4343-94eb-a204ca47844e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.121493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbb87-6f81-4343-94eb-a204ca47844e admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.122149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbb87-6f81-4343-94eb-a204ca47844e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:55.124977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbb87-6f81-4343-94eb-a204ca47844e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:55.136327 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2971/5869] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:55 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:55.140790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e24b96-ecbc-486c-8f76-09e0e437f3b6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:55.153844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e24b96-ecbc-486c-8f76-09e0e437f3b6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:55.160075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8853e64-559c-4dd5-b751-f080bcfe2eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:55.160935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8853e64-559c-4dd5-b751-f080bcfe2eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.161370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8853e64-559c-4dd5-b751-f080bcfe2eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.161868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8853e64-559c-4dd5-b751-f080bcfe2eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.162354 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2899/5870] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:55 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:55.166033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8853e64-559c-4dd5-b751-f080bcfe2eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:55.166394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbb87-6f81-4343-94eb-a204ca47844e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:55.168021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8853e64-559c-4dd5-b751-f080bcfe2eb2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:55.180230 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2972/5871] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:55.183923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8853e64-559c-4dd5-b751-f080bcfe2eb2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:55.195134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8853e64-559c-4dd5-b751-f080bcfe2eb2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:55.233615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:55.236871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbb87-6f81-4343-94eb-a204ca47844e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:55.242468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87166924-fbee-4c69-b72e-b30a3757f829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:55.242856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87166924-fbee-4c69-b72e-b30a3757f829 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.243130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87166924-fbee-4c69-b72e-b30a3757f829 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.244006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87166924-fbee-4c69-b72e-b30a3757f829 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.244146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87166924-fbee-4c69-b72e-b30a3757f829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:55.246134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87166924-fbee-4c69-b72e-b30a3757f829 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:55.267808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891a6b1-7f15-4ae5-a378-76500bd14c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:55.268527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891a6b1-7f15-4ae5-a378-76500bd14c1f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.268913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891a6b1-7f15-4ae5-a378-76500bd14c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.272441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891a6b1-7f15-4ae5-a378-76500bd14c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.272584 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2900/5872] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:55 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:09:55.272689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891a6b1-7f15-4ae5-a378-76500bd14c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:55.272689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891a6b1-7f15-4ae5-a378-76500bd14c1f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:55.273554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87166924-fbee-4c69-b72e-b30a3757f829 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:55.282430 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2973/5873] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:55.286364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891a6b1-7f15-4ae5-a378-76500bd14c1f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:55.289017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87166924-fbee-4c69-b72e-b30a3757f829 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:55.295735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e7c74-1a34-4205-98d6-83f174d64a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:55.296445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e7c74-1a34-4205-98d6-83f174d64a52 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.296803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e7c74-1a34-4205-98d6-83f174d64a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.297116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e7c74-1a34-4205-98d6-83f174d64a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.297764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e7c74-1a34-4205-98d6-83f174d64a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:55.298294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891a6b1-7f15-4ae5-a378-76500bd14c1f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:55.299499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e7c74-1a34-4205-98d6-83f174d64a52 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:55.303248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0d06d-6c3b-4d69-ba53-b1e48ead1ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:55.303916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0d06d-6c3b-4d69-ba53-b1e48ead1ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.304233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0d06d-6c3b-4d69-ba53-b1e48ead1ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.304516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0d06d-6c3b-4d69-ba53-b1e48ead1ce2 admin admin] PATH_INFO: `/v3/projects/06c1b8cc275d405eb94f2664b881c84f/users/b5a24024925c4982820f9ac3a23bf46c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.313369 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2901/5874] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:55.315962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e7c74-1a34-4205-98d6-83f174d64a52 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:55.325366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0d06d-6c3b-4d69-ba53-b1e48ead1ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c1b8cc275d405eb94f2664b881c84f, user_id=b5a24024925c4982820f9ac3a23bf46c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:55.325880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e7c74-1a34-4205-98d6-83f174d64a52 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:55.330460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0d06d-6c3b-4d69-ba53-b1e48ead1ce2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:55.331578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ea7d2-5eb0-42c9-a2b3-547dc7805ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:55.332312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ea7d2-5eb0-42c9-a2b3-547dc7805ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.332705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ea7d2-5eb0-42c9-a2b3-547dc7805ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.333125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ea7d2-5eb0-42c9-a2b3-547dc7805ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.334016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ea7d2-5eb0-42c9-a2b3-547dc7805ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:55.335759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ea7d2-5eb0-42c9-a2b3-547dc7805ae7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:55.358896 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2902/5876] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:55.359355 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2974/5876] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:55 2022] PUT /identity/v3/projects/06c1b8cc275d405eb94f2664b881c84f/users/b5a24024925c4982820f9ac3a23bf46c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:55.359468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ea7d2-5eb0-42c9-a2b3-547dc7805ae7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:55.372345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae5440-0813-497b-b234-a5d75368aabb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.372641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae5440-0813-497b-b234-a5d75368aabb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.372942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae5440-0813-497b-b234-a5d75368aabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.373435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ea7d2-5eb0-42c9-a2b3-547dc7805ae7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:55.406063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-13ae5440-0813-497b-b234-a5d75368aabb None None] MFA Rules not processed for user `b5a24024925c4982820f9ac3a23bf46c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:55.407443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-13ae5440-0813-497b-b234-a5d75368aabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:55.455604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e753f9-a767-438a-b8e6-93dc54fed2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:55.457045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e753f9-a767-438a-b8e6-93dc54fed2ec admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.457537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e753f9-a767-438a-b8e6-93dc54fed2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.458915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e753f9-a767-438a-b8e6-93dc54fed2ec admin admin] PATH_INFO: `/v3/projects/9e8e78802f0942e58f8fc3e49cb47073/users/d96fd998f3b24c348163e8b07a93af1c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.464257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:55.479190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e753f9-a767-438a-b8e6-93dc54fed2ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e8e78802f0942e58f8fc3e49cb47073, user_id=d96fd998f3b24c348163e8b07a93af1c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:55.482222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e753f9-a767-438a-b8e6-93dc54fed2ec admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:55.488183 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2975/5877] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:55.497711 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2903/5878] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:55 2022] PUT /identity/v3/projects/9e8e78802f0942e58f8fc3e49cb47073/users/d96fd998f3b24c348163e8b07a93af1c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:55.504998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae5440-0813-497b-b234-a5d75368aabb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:55.506474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-13ae5440-0813-497b-b234-a5d75368aabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:55.592287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2c478-e532-433a-9344-a45d571a9529 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.593064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2c478-e532-433a-9344-a45d571a9529 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.593064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2c478-e532-433a-9344-a45d571a9529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.594261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae5440-0813-497b-b234-a5d75368aabb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:55.623964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f7c2c478-e532-433a-9344-a45d571a9529 None None] MFA Rules not processed for user `d96fd998f3b24c348163e8b07a93af1c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:55.630305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f7c2c478-e532-433a-9344-a45d571a9529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:55.652555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7eb588-0b0c-4d1c-aa3c-c98a9058cfdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:55.653295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7eb588-0b0c-4d1c-aa3c-c98a9058cfdb service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.653597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7eb588-0b0c-4d1c-aa3c-c98a9058cfdb service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.653857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7eb588-0b0c-4d1c-aa3c-c98a9058cfdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.662114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:55.703574 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2904/5879] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:55.717273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2c478-e532-433a-9344-a45d571a9529 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:55.719574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f7c2c478-e532-433a-9344-a45d571a9529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:55.731673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7eb588-0b0c-4d1c-aa3c-c98a9058cfdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:55.733427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7eb588-0b0c-4d1c-aa3c-c98a9058cfdb service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:55.745977 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2976/5880] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:55.790352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2c478-e532-433a-9344-a45d571a9529 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:55.795296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f84606-b019-417d-b3e1-094a3e513a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:55.795296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f84606-b019-417d-b3e1-094a3e513a49 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:55.795717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f84606-b019-417d-b3e1-094a3e513a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:55.795717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f84606-b019-417d-b3e1-094a3e513a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:55.808294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f84606-b019-417d-b3e1-094a3e513a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:55.811311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f84606-b019-417d-b3e1-094a3e513a49 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:55.822907 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2905/5881] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:57.358375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7eb588-0b0c-4d1c-aa3c-c98a9058cfdb service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:57.475039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7eb588-0b0c-4d1c-aa3c-c98a9058cfdb service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:57.480604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:57.481331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:57.481576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:57.481846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] PATH_INFO: `/v3/users/b5a24024925c4982820f9ac3a23bf46c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:57.483189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5a24024925c4982820f9ac3a23bf46c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:57.485366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:57.550909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5a24024925c4982820f9ac3a23bf46c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:57.578024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5a24024925c4982820f9ac3a23bf46c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:57.589760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5a24024925c4982820f9ac3a23bf46c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:57.590991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5a24024925c4982820f9ac3a23bf46c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:57.591779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5a24024925c4982820f9ac3a23bf46c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:57.620686 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2977/5882] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:09:57 2022] DELETE /identity/v3/users/b5a24024925c4982820f9ac3a23bf46c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:57.625069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76df164-8cef-40b3-a5c8-33dc37356ac6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:57.626016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76df164-8cef-40b3-a5c8-33dc37356ac6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:57.627179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76df164-8cef-40b3-a5c8-33dc37356ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:57.660855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d76df164-8cef-40b3-a5c8-33dc37356ac6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:57.662328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d76df164-8cef-40b3-a5c8-33dc37356ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:57.763838 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2906/5883] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:57.780701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:57.782237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:57.833119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:57.873388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef0ad7-1bf5-4595-9db9-d99a253965bc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:57.878568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18052fe8-f19b-4946-b1e8-6838ef24b037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:57.879300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18052fe8-f19b-4946-b1e8-6838ef24b037 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:57.879697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18052fe8-f19b-4946-b1e8-6838ef24b037 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:57.880080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18052fe8-f19b-4946-b1e8-6838ef24b037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:57.890780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18052fe8-f19b-4946-b1e8-6838ef24b037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:57.893140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18052fe8-f19b-4946-b1e8-6838ef24b037 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:57.904245 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2978/5884] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:09:58.507924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76df164-8cef-40b3-a5c8-33dc37356ac6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:58.509400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d76df164-8cef-40b3-a5c8-33dc37356ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:58.522018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76df164-8cef-40b3-a5c8-33dc37356ac6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:58.530068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:58.531538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:58.532062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:58.532428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181 admin admin] PATH_INFO: `/v3/projects/06c1b8cc275d405eb94f2664b881c84f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:58.534668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06c1b8cc275d405eb94f2664b881c84f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:58.538167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:58.582961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:58.652526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06c1b8cc275d405eb94f2664b881c84f', 'request_id': 'req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:58.670215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06c1b8cc275d405eb94f2664b881c84f', 'request_id': 'req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:58.680006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06c1b8cc275d405eb94f2664b881c84f', 'request_id': 'req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:58.693533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181 admin admin] The token cache is being invalidate because project 06c1b8cc275d405eb94f2664b881c84f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:09:58.693533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:58.694262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bdbe4346-e37a-41f0-bd2c-0413bbc0d181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:09:58.695756 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2907/5885] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:09:58 2022] DELETE /identity/v3/projects/06c1b8cc275d405eb94f2664b881c84f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:09:58.700967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa41e2-d148-4a96-8a67-7d3e26051a6a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:58.701490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa41e2-d148-4a96-8a67-7d3e26051a6a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:58.701734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa41e2-d148-4a96-8a67-7d3e26051a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:58.725074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-20aa41e2-d148-4a96-8a67-7d3e26051a6a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:58.727294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-20aa41e2-d148-4a96-8a67-7d3e26051a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:58.816182 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2979/5886] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:58.823780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e1cc0-49da-42bc-b889-a49dbe70c37d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:58.824451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e1cc0-49da-42bc-b889-a49dbe70c37d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:58.824451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e1cc0-49da-42bc-b889-a49dbe70c37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:58.846072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-634e1cc0-49da-42bc-b889-a49dbe70c37d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:58.847438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-634e1cc0-49da-42bc-b889-a49dbe70c37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:58.857694 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2908/5887] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:58.862373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa41e2-d148-4a96-8a67-7d3e26051a6a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:58.875259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa41e2-d148-4a96-8a67-7d3e26051a6a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:58.882018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b324a-5104-47a1-b65f-6c665b2fc846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:58.882772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b324a-5104-47a1-b65f-6c665b2fc846 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:58.883133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b324a-5104-47a1-b65f-6c665b2fc846 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:58.883410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b324a-5104-47a1-b65f-6c665b2fc846 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:58.884142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b324a-5104-47a1-b65f-6c665b2fc846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:09:58.884369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b324a-5104-47a1-b65f-6c665b2fc846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:58.886319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b324a-5104-47a1-b65f-6c665b2fc846 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:58.900109 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2980/5888] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:09:58.904575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e1cc0-49da-42bc-b889-a49dbe70c37d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:58.914815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e1cc0-49da-42bc-b889-a49dbe70c37d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:58.919996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ff165-330c-4079-9970-5bf01d9bccb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:58.920615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ff165-330c-4079-9970-5bf01d9bccb5 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:58.920932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ff165-330c-4079-9970-5bf01d9bccb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:58.921210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ff165-330c-4079-9970-5bf01d9bccb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:58.921852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ff165-330c-4079-9970-5bf01d9bccb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:58.924349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ff165-330c-4079-9970-5bf01d9bccb5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:58.958059 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2909/5889] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:58 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:58.961991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b324a-5104-47a1-b65f-6c665b2fc846 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:58.976989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b324a-5104-47a1-b65f-6c665b2fc846 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:59.060737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca6055-b93e-4dd6-abb4-e4e4b8f9fa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:59.061515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca6055-b93e-4dd6-abb4-e4e4b8f9fa2d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:59.061899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca6055-b93e-4dd6-abb4-e4e4b8f9fa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:59.062250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca6055-b93e-4dd6-abb4-e4e4b8f9fa2d admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:59.062959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca6055-b93e-4dd6-abb4-e4e4b8f9fa2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:59.065083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca6055-b93e-4dd6-abb4-e4e4b8f9fa2d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:59.092631 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2981/5890] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:58 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:59.097394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ff165-330c-4079-9970-5bf01d9bccb5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:59.109111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ff165-330c-4079-9970-5bf01d9bccb5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:59.124649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acef942-a721-4ac9-bced-07033bbadd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:59.125462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acef942-a721-4ac9-bced-07033bbadd6c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:59.125946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acef942-a721-4ac9-bced-07033bbadd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:59.126418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acef942-a721-4ac9-bced-07033bbadd6c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:59.127256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acef942-a721-4ac9-bced-07033bbadd6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:59.134051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acef942-a721-4ac9-bced-07033bbadd6c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:59.153366 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2910/5891] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:09:59.158344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca6055-b93e-4dd6-abb4-e4e4b8f9fa2d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:59.182608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca6055-b93e-4dd6-abb4-e4e4b8f9fa2d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:59.191081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f0674-c233-4fbc-a381-1a173ddbfa81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:59.192155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f0674-c233-4fbc-a381-1a173ddbfa81 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:59.192671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f0674-c233-4fbc-a381-1a173ddbfa81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:59.194292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f0674-c233-4fbc-a381-1a173ddbfa81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:59.195180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f0674-c233-4fbc-a381-1a173ddbfa81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:59.196866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f0674-c233-4fbc-a381-1a173ddbfa81 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:59.198916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b27fbe-e03b-4484-8b07-526d1d5b2414 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:59.199295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b27fbe-e03b-4484-8b07-526d1d5b2414 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:59.199653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b27fbe-e03b-4484-8b07-526d1d5b2414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:59.213781 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2982/5892] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:59.215176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f0674-c233-4fbc-a381-1a173ddbfa81 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:59.223399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f2b27fbe-e03b-4484-8b07-526d1d5b2414 None None] MFA Rules not processed for user `3e79edf1a33e45ddac92ba8cd8429d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:59.224682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f2b27fbe-e03b-4484-8b07-526d1d5b2414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:59.226751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f0674-c233-4fbc-a381-1a173ddbfa81 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:59.232947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df743350-6844-45fc-b5e9-2b1735da3d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:59.233247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df743350-6844-45fc-b5e9-2b1735da3d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:59.233700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df743350-6844-45fc-b5e9-2b1735da3d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:59.233987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df743350-6844-45fc-b5e9-2b1735da3d42 admin admin] PATH_INFO: `/v3/projects/c0a700e9c8974a7fb762f9224359decf/users/fcbc8851d2124305a4f4426c3b8f9dc2/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:59.261804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df743350-6844-45fc-b5e9-2b1735da3d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a700e9c8974a7fb762f9224359decf, user_id=fcbc8851d2124305a4f4426c3b8f9dc2, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:59.264756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df743350-6844-45fc-b5e9-2b1735da3d42 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:59.280244 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2983/5893] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:59 2022] PUT /identity/v3/projects/c0a700e9c8974a7fb762f9224359decf/users/fcbc8851d2124305a4f4426c3b8f9dc2/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:59.291868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7365f-5dc5-44a6-ac44-3aa6420a7250 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:59.292226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7365f-5dc5-44a6-ac44-3aa6420a7250 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:59.292707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7365f-5dc5-44a6-ac44-3aa6420a7250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:59.315520 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2911/5894] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:09:59.317981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:59.330782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e6a7365f-5dc5-44a6-ac44-3aa6420a7250 None None] MFA Rules not processed for user `fcbc8851d2124305a4f4426c3b8f9dc2`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:59.332221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e6a7365f-5dc5-44a6-ac44-3aa6420a7250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:59.435025 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2984/5895] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:09:59.439806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b27fbe-e03b-4484-8b07-526d1d5b2414 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:59.455311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b27fbe-e03b-4484-8b07-526d1d5b2414 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:59.528732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:09:59.548085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dce4c-b570-44bc-b462-0a9579988b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:59.548716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dce4c-b570-44bc-b462-0a9579988b10 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:59.549064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dce4c-b570-44bc-b462-0a9579988b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:59.549341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dce4c-b570-44bc-b462-0a9579988b10 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:59.550015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7dce4c-b570-44bc-b462-0a9579988b10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:59.552321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7dce4c-b570-44bc-b462-0a9579988b10 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:59.583098 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2912/5896] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:09:59 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:59.588246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7365f-5dc5-44a6-ac44-3aa6420a7250 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:59.604027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7365f-5dc5-44a6-ac44-3aa6420a7250 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:59.691933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae1bf2-2571-4aef-ba42-6975058a2aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:59.693550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae1bf2-2571-4aef-ba42-6975058a2aae admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:59.693945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae1bf2-2571-4aef-ba42-6975058a2aae admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:59.694334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae1bf2-2571-4aef-ba42-6975058a2aae admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:59.695094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae1bf2-2571-4aef-ba42-6975058a2aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:59.697186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae1bf2-2571-4aef-ba42-6975058a2aae admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:59.738317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dce4c-b570-44bc-b462-0a9579988b10 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:59.740968 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2985/5897] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:09:59 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:09:59.752690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dce4c-b570-44bc-b462-0a9579988b10 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:59.760445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3149ef-55e9-4e2c-b563-dc5e53fdbfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:59.761286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3149ef-55e9-4e2c-b563-dc5e53fdbfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:59.761696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3149ef-55e9-4e2c-b563-dc5e53fdbfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:59.762087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3149ef-55e9-4e2c-b563-dc5e53fdbfa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:59.762853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3149ef-55e9-4e2c-b563-dc5e53fdbfa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:59.764680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3149ef-55e9-4e2c-b563-dc5e53fdbfa6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:59.777500 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2913/5898] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:09:59.781794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae1bf2-2571-4aef-ba42-6975058a2aae admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:09:59.793597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae1bf2-2571-4aef-ba42-6975058a2aae admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:09:59.798865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f457-2cf4-4c0f-87d7-2e9ec4ac6b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:09:59.799142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f457-2cf4-4c0f-87d7-2e9ec4ac6b30 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:59.799429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f457-2cf4-4c0f-87d7-2e9ec4ac6b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:59.799883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f457-2cf4-4c0f-87d7-2e9ec4ac6b30 admin admin] PATH_INFO: `/v3/projects/bbe173bb54b54a73a6ad36e3504f448b/users/ff6e0a8b12b244d287bcb7bc9b9b48fe/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:59.818741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f457-2cf4-4c0f-87d7-2e9ec4ac6b30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe173bb54b54a73a6ad36e3504f448b, user_id=ff6e0a8b12b244d287bcb7bc9b9b48fe, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:09:59.821338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f457-2cf4-4c0f-87d7-2e9ec4ac6b30 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:09:59.838717 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2986/5899] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:09:59 2022] PUT /identity/v3/projects/bbe173bb54b54a73a6ad36e3504f448b/users/ff6e0a8b12b244d287bcb7bc9b9b48fe/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:09:59.849716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c6239-0d05-493a-a36c-5489976fbad7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:09:59.850301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c6239-0d05-493a-a36c-5489976fbad7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:09:59.851404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c6239-0d05-493a-a36c-5489976fbad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:09:59.895428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c44c6239-0d05-493a-a36c-5489976fbad7 None None] MFA Rules not processed for user `ff6e0a8b12b244d287bcb7bc9b9b48fe`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:09:59.897025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c44c6239-0d05-493a-a36c-5489976fbad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:09:59.991776 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2914/5900] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3269 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:00.034825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f457-2cf4-4c0f-87d7-2e9ec4ac6b30 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:00.036604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ddd1f457-2cf4-4c0f-87d7-2e9ec4ac6b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:00.132178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:00.145127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f457-2cf4-4c0f-87d7-2e9ec4ac6b30 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:00.151287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80b0b1-bf90-46fc-99d9-654bf659d03c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:00.152080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80b0b1-bf90-46fc-99d9-654bf659d03c service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:00.152537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80b0b1-bf90-46fc-99d9-654bf659d03c service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:00.153083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80b0b1-bf90-46fc-99d9-654bf659d03c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:00.163623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80b0b1-bf90-46fc-99d9-654bf659d03c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:00.165886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80b0b1-bf90-46fc-99d9-654bf659d03c service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:00.176650 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2987/5901] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:10:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 678 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:10:01.009046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c6239-0d05-493a-a36c-5489976fbad7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:01.095638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c6239-0d05-493a-a36c-5489976fbad7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:01.104572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:01.106120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:01.107119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:01.107965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] PATH_INFO: `/v3/users/fcbc8851d2124305a4f4426c3b8f9dc2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:01.110931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcbc8851d2124305a4f4426c3b8f9dc2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:01.114669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:01.171782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcbc8851d2124305a4f4426c3b8f9dc2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:01.186447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:01.187751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcbc8851d2124305a4f4426c3b8f9dc2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:01.189008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcbc8851d2124305a4f4426c3b8f9dc2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:01.199778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcbc8851d2124305a4f4426c3b8f9dc2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:01.209651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcbc8851d2124305a4f4426c3b8f9dc2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:01.216395 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2915/5902] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:01 2022] DELETE /identity/v3/users/fcbc8851d2124305a4f4426c3b8f9dc2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:01.221558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4269d7-a6ca-4a4d-a8d9-d3c16149d25f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:01.222094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4269d7-a6ca-4a4d-a8d9-d3c16149d25f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:01.222276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4269d7-a6ca-4a4d-a8d9-d3c16149d25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:01.244865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ac4269d7-a6ca-4a4d-a8d9-d3c16149d25f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:01.246248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ac4269d7-a6ca-4a4d-a8d9-d3c16149d25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:01.312006 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2988/5903] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:01.322802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:01.324283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:01.402423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ee881-cbde-47d1-b31c-ebdafbdf7387 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:01.408996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159ad7d-3f64-4bd3-b3f5-09873ff22075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:01.409776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159ad7d-3f64-4bd3-b3f5-09873ff22075 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:01.410257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159ad7d-3f64-4bd3-b3f5-09873ff22075 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:01.410647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159ad7d-3f64-4bd3-b3f5-09873ff22075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:01.440819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159ad7d-3f64-4bd3-b3f5-09873ff22075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:01.444542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159ad7d-3f64-4bd3-b3f5-09873ff22075 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:01.457739 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2916/5904] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:01.887327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4269d7-a6ca-4a4d-a8d9-d3c16149d25f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:01.889010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ac4269d7-a6ca-4a4d-a8d9-d3c16149d25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:01.903764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4269d7-a6ca-4a4d-a8d9-d3c16149d25f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:01.910028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:01.910883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:01.911176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:01.911457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] PATH_INFO: `/v3/projects/c0a700e9c8974a7fb762f9224359decf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:01.912704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0a700e9c8974a7fb762f9224359decf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:01.914371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159ad7d-3f64-4bd3-b3f5-09873ff22075 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:01.915068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:01.915776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9159ad7d-3f64-4bd3-b3f5-09873ff22075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:01.932280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159ad7d-3f64-4bd3-b3f5-09873ff22075 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:01.937207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076f8a7-dc5b-4e51-83fb-3c6e5fd67dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:01.937870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076f8a7-dc5b-4e51-83fb-3c6e5fd67dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:01.938154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076f8a7-dc5b-4e51-83fb-3c6e5fd67dde admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:01.938431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076f8a7-dc5b-4e51-83fb-3c6e5fd67dde admin admin] PATH_INFO: `/v3/users/d96fd998f3b24c348163e8b07a93af1c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:01.939651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076f8a7-dc5b-4e51-83fb-3c6e5fd67dde admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96fd998f3b24c348163e8b07a93af1c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:01.941651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076f8a7-dc5b-4e51-83fb-3c6e5fd67dde admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:01.987979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0a700e9c8974a7fb762f9224359decf', 'request_id': 'req-a6c2a7e8-a766-495c-9d43-969ac21d2d44'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:01.997353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0a700e9c8974a7fb762f9224359decf', 'request_id': 'req-a6c2a7e8-a766-495c-9d43-969ac21d2d44'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.005285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0a700e9c8974a7fb762f9224359decf', 'request_id': 'req-a6c2a7e8-a766-495c-9d43-969ac21d2d44'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.006964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b076f8a7-dc5b-4e51-83fb-3c6e5fd67dde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96fd998f3b24c348163e8b07a93af1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.020817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b076f8a7-dc5b-4e51-83fb-3c6e5fd67dde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96fd998f3b24c348163e8b07a93af1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.022053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b076f8a7-dc5b-4e51-83fb-3c6e5fd67dde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96fd998f3b24c348163e8b07a93af1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.022597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] The token cache is being invalidate because project c0a700e9c8974a7fb762f9224359decf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:02.023092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.023964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.030027 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2989/5905] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:01 2022] DELETE /identity/v3/projects/c0a700e9c8974a7fb762f9224359decf => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:02.030304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b076f8a7-dc5b-4e51-83fb-3c6e5fd67dde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96fd998f3b24c348163e8b07a93af1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.031209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:02.033216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:02.039659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b076f8a7-dc5b-4e51-83fb-3c6e5fd67dde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96fd998f3b24c348163e8b07a93af1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.050683 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2917/5906] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:01 2022] DELETE /identity/v3/users/d96fd998f3b24c348163e8b07a93af1c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:02.055448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce8397-b77c-467c-83c3-d80a14bc2782 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:02.055742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce8397-b77c-467c-83c3-d80a14bc2782 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:02.056018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce8397-b77c-467c-83c3-d80a14bc2782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:02.080287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-70ce8397-b77c-467c-83c3-d80a14bc2782 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:02.081527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-70ce8397-b77c-467c-83c3-d80a14bc2782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:02.126737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2a7e8-a766-495c-9d43-969ac21d2d44 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:02.150797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:02.153177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:02.171215 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2918/5907] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:02.182877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce8397-b77c-467c-83c3-d80a14bc2782 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:02.186140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-70ce8397-b77c-467c-83c3-d80a14bc2782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:02.240033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:02.241401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:02.241693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:02.241974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] PATH_INFO: `/v3/users/ff6e0a8b12b244d287bcb7bc9b9b48fe` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:02.243305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6e0a8b12b244d287bcb7bc9b9b48fe)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:02.245289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:02.267663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce8397-b77c-467c-83c3-d80a14bc2782 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:02.273681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dad26a-f982-41b8-9ed7-c0e385aaadeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:02.274834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dad26a-f982-41b8-9ed7-c0e385aaadeb service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:02.275997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dad26a-f982-41b8-9ed7-c0e385aaadeb service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:02.276530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dad26a-f982-41b8-9ed7-c0e385aaadeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:02.293029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dad26a-f982-41b8-9ed7-c0e385aaadeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:02.295289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dad26a-f982-41b8-9ed7-c0e385aaadeb service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:02.305454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6e0a8b12b244d287bcb7bc9b9b48fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.314090 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2919/5908] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:02.324027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6e0a8b12b244d287bcb7bc9b9b48fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.338515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6e0a8b12b244d287bcb7bc9b9b48fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.342414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6e0a8b12b244d287bcb7bc9b9b48fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.343413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6e0a8b12b244d287bcb7bc9b9b48fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:02.360473 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2990/5909] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:02 2022] DELETE /identity/v3/users/ff6e0a8b12b244d287bcb7bc9b9b48fe => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:02.842753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dad26a-f982-41b8-9ed7-c0e385aaadeb service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:02.844260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-65dad26a-f982-41b8-9ed7-c0e385aaadeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:02.858435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dad26a-f982-41b8-9ed7-c0e385aaadeb service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:03.002287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684675c3-b6ef-41a5-9b26-64615d0d1dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:03.003533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684675c3-b6ef-41a5-9b26-64615d0d1dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:03.003948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684675c3-b6ef-41a5-9b26-64615d0d1dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:03.004266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684675c3-b6ef-41a5-9b26-64615d0d1dc2 admin admin] PATH_INFO: `/v3/projects/9e8e78802f0942e58f8fc3e49cb47073` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:03.005607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684675c3-b6ef-41a5-9b26-64615d0d1dc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e8e78802f0942e58f8fc3e49cb47073)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:03.007912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684675c3-b6ef-41a5-9b26-64615d0d1dc2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:03.071879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-684675c3-b6ef-41a5-9b26-64615d0d1dc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e8e78802f0942e58f8fc3e49cb47073', 'request_id': 'req-684675c3-b6ef-41a5-9b26-64615d0d1dc2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:03.085041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-684675c3-b6ef-41a5-9b26-64615d0d1dc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e8e78802f0942e58f8fc3e49cb47073', 'request_id': 'req-684675c3-b6ef-41a5-9b26-64615d0d1dc2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:03.095643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-684675c3-b6ef-41a5-9b26-64615d0d1dc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e8e78802f0942e58f8fc3e49cb47073', 'request_id': 'req-684675c3-b6ef-41a5-9b26-64615d0d1dc2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:03.104355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:03.105973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:03.119524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-684675c3-b6ef-41a5-9b26-64615d0d1dc2 admin admin] The token cache is being invalidate because project 9e8e78802f0942e58f8fc3e49cb47073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:03.119524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-684675c3-b6ef-41a5-9b26-64615d0d1dc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:03.120362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-684675c3-b6ef-41a5-9b26-64615d0d1dc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:03.125645 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2920/5910] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:02 2022] DELETE /identity/v3/projects/9e8e78802f0942e58f8fc3e49cb47073 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:03.132413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240bde2-28cd-4104-9986-ada61d06ce0c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:03.140642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2f358-2eca-475d-8bc3-c7d332630c7e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:03.140962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2f358-2eca-475d-8bc3-c7d332630c7e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:03.141266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2f358-2eca-475d-8bc3-c7d332630c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:03.179534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f7e2f358-2eca-475d-8bc3-c7d332630c7e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:03.181174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f7e2f358-2eca-475d-8bc3-c7d332630c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:03.221379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:03.241419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:03.242099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:03.242348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:03.242625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] PATH_INFO: `/v3/projects/bbe173bb54b54a73a6ad36e3504f448b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:03.243955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbe173bb54b54a73a6ad36e3504f448b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:03.246584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:03.275241 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2921/5911] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:03.284038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75373646-f1d5-4393-8cca-8ddbc35d1710 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:03.284704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75373646-f1d5-4393-8cca-8ddbc35d1710 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:03.285530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75373646-f1d5-4393-8cca-8ddbc35d1710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:03.313154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:03.313517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-75373646-f1d5-4393-8cca-8ddbc35d1710 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:03.315103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-75373646-f1d5-4393-8cca-8ddbc35d1710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:03.324731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbe173bb54b54a73a6ad36e3504f448b', 'request_id': 'req-93dd044e-098b-4bd7-9912-2747f5ea1df2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:03.341717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbe173bb54b54a73a6ad36e3504f448b', 'request_id': 'req-93dd044e-098b-4bd7-9912-2747f5ea1df2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:03.353101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbe173bb54b54a73a6ad36e3504f448b', 'request_id': 'req-93dd044e-098b-4bd7-9912-2747f5ea1df2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:03.367350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] The token cache is being invalidate because project bbe173bb54b54a73a6ad36e3504f448b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:03.368002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:03.368838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:03.370922 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2991/5912] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:03 2022] DELETE /identity/v3/projects/bbe173bb54b54a73a6ad36e3504f448b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:10:03.408004 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2922/5913] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:03.409031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:03.420839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd044e-098b-4bd7-9912-2747f5ea1df2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:03.426815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5510f9e-0ea5-4578-8dac-97c88d6e10e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:03.427697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5510f9e-0ea5-4578-8dac-97c88d6e10e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:03.428188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5510f9e-0ea5-4578-8dac-97c88d6e10e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:03.428621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5510f9e-0ea5-4578-8dac-97c88d6e10e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:03.429618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5510f9e-0ea5-4578-8dac-97c88d6e10e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:10:03.430096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5510f9e-0ea5-4578-8dac-97c88d6e10e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:03.432412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5510f9e-0ea5-4578-8dac-97c88d6e10e2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:03.473913 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2992/5914] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:10:03.478792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75373646-f1d5-4393-8cca-8ddbc35d1710 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:03.504181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75373646-f1d5-4393-8cca-8ddbc35d1710 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:03.523679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac7fdc-9aa6-4edf-947f-bcac7850e412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:03.524335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac7fdc-9aa6-4edf-947f-bcac7850e412 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:03.524622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac7fdc-9aa6-4edf-947f-bcac7850e412 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:03.524942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac7fdc-9aa6-4edf-947f-bcac7850e412 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:03.525602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac7fdc-9aa6-4edf-947f-bcac7850e412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:03.536131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac7fdc-9aa6-4edf-947f-bcac7850e412 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:03.604590 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2923/5915] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:03 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:03.606006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5510f9e-0ea5-4578-8dac-97c88d6e10e2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:03.625908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5510f9e-0ea5-4578-8dac-97c88d6e10e2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:03.755637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7cc9b-9aaf-483f-9a50-c7034d3c9cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:03.756739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7cc9b-9aaf-483f-9a50-c7034d3c9cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:03.757254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7cc9b-9aaf-483f-9a50-c7034d3c9cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:03.757734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7cc9b-9aaf-483f-9a50-c7034d3c9cc2 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:03.758578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7cc9b-9aaf-483f-9a50-c7034d3c9cc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:03.761995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7cc9b-9aaf-483f-9a50-c7034d3c9cc2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:03.800880 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2993/5916] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:03 2022] POST /identity/v3/users => generated 288 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:03.802208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac7fdc-9aa6-4edf-947f-bcac7850e412 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:03.813509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac7fdc-9aa6-4edf-947f-bcac7850e412 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:03.819229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbbea5-1913-44f4-b02b-4e49d552a331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:03.819899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbbea5-1913-44f4-b02b-4e49d552a331 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:03.820172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbbea5-1913-44f4-b02b-4e49d552a331 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:03.820455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbbea5-1913-44f4-b02b-4e49d552a331 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:03.821185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbbea5-1913-44f4-b02b-4e49d552a331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:03.833920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbbea5-1913-44f4-b02b-4e49d552a331 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:03.863230 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2924/5917] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:03.864617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7cc9b-9aaf-483f-9a50-c7034d3c9cc2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:03.876796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7cc9b-9aaf-483f-9a50-c7034d3c9cc2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:03.882132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7aafa-c022-47b7-87d6-3dde7ff14280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:03.883256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7aafa-c022-47b7-87d6-3dde7ff14280 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:03.883256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7aafa-c022-47b7-87d6-3dde7ff14280 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:03.883539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7aafa-c022-47b7-87d6-3dde7ff14280 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:03.884161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7aafa-c022-47b7-87d6-3dde7ff14280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:03.886207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7aafa-c022-47b7-87d6-3dde7ff14280 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:03.901001 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2994/5918] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:03.904104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbbea5-1913-44f4-b02b-4e49d552a331 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:03.915722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbbea5-1913-44f4-b02b-4e49d552a331 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:03.920905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dfbc8-0326-446b-aeae-18f654bc2538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:03.921777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dfbc8-0326-446b-aeae-18f654bc2538 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:03.922168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dfbc8-0326-446b-aeae-18f654bc2538 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:03.922586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dfbc8-0326-446b-aeae-18f654bc2538 admin admin] PATH_INFO: `/v3/projects/6be2f8502e5941c8bee08b7397dbdeff/users/766a5a8af8894ca890b30da2cb547f66/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:03.940302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dfbc8-0326-446b-aeae-18f654bc2538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be2f8502e5941c8bee08b7397dbdeff, user_id=766a5a8af8894ca890b30da2cb547f66, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:03.943052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dfbc8-0326-446b-aeae-18f654bc2538 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:03.958840 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2925/5919] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:03 2022] PUT /identity/v3/projects/6be2f8502e5941c8bee08b7397dbdeff/users/766a5a8af8894ca890b30da2cb547f66/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:03.971311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1b49d-3145-40cd-b281-3a16424f42d0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:03.972758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1b49d-3145-40cd-b281-3a16424f42d0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:03.973089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1b49d-3145-40cd-b281-3a16424f42d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:04.005370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4fc1b49d-3145-40cd-b281-3a16424f42d0 None None] MFA Rules not processed for user `766a5a8af8894ca890b30da2cb547f66`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:04.006872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4fc1b49d-3145-40cd-b281-3a16424f42d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:04.079101 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2995/5920] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:04.091351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dfbc8-0326-446b-aeae-18f654bc2538 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:04.093328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b10dfbc8-0326-446b-aeae-18f654bc2538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:04.162512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dfbc8-0326-446b-aeae-18f654bc2538 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:04.166330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868d335-68f6-419e-acd1-736823adfef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:04.166979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868d335-68f6-419e-acd1-736823adfef5 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:04.167266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868d335-68f6-419e-acd1-736823adfef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:04.167553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868d335-68f6-419e-acd1-736823adfef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:04.176207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868d335-68f6-419e-acd1-736823adfef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:04.177682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868d335-68f6-419e-acd1-736823adfef5 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:04.186834 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2926/5921] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:04.595475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1b49d-3145-40cd-b281-3a16424f42d0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:04.627044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:04.674336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1b49d-3145-40cd-b281-3a16424f42d0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:04.678865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:04.679640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:04.679930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:04.680207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] PATH_INFO: `/v3/users/766a5a8af8894ca890b30da2cb547f66` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:04.681376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=766a5a8af8894ca890b30da2cb547f66)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:04.683316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:04.738569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '766a5a8af8894ca890b30da2cb547f66'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:04.750612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '766a5a8af8894ca890b30da2cb547f66'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:04.759092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '766a5a8af8894ca890b30da2cb547f66'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:04.759741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '766a5a8af8894ca890b30da2cb547f66'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:04.760423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '766a5a8af8894ca890b30da2cb547f66'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:04.772953 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2996/5922] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:04 2022] DELETE /identity/v3/users/766a5a8af8894ca890b30da2cb547f66 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:04.777567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6459dbd-137e-4776-96c5-b159418a615a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:04.778059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6459dbd-137e-4776-96c5-b159418a615a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:04.778448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6459dbd-137e-4776-96c5-b159418a615a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:04.800412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b6459dbd-137e-4776-96c5-b159418a615a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:04.801659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b6459dbd-137e-4776-96c5-b159418a615a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:04.811680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:04.879360 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2927/5923] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:04.895967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:04.897833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:04.973187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19a081-340e-4fbc-beba-cc07b1d10406 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:04.977658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f15afc-c66c-482e-851d-95107ff9367b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:04.978288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f15afc-c66c-482e-851d-95107ff9367b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:04.978580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f15afc-c66c-482e-851d-95107ff9367b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:04.978865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f15afc-c66c-482e-851d-95107ff9367b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:04.992144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f15afc-c66c-482e-851d-95107ff9367b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:04.993844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f15afc-c66c-482e-851d-95107ff9367b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:05.007076 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2997/5924] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:05.030713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6459dbd-137e-4776-96c5-b159418a615a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:05.032821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b6459dbd-137e-4776-96c5-b159418a615a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:05.177983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6459dbd-137e-4776-96c5-b159418a615a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:05.184833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996edc45-30f5-4fe9-919b-58d3d5bc0100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:05.185726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996edc45-30f5-4fe9-919b-58d3d5bc0100 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:05.185859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996edc45-30f5-4fe9-919b-58d3d5bc0100 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:05.186337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996edc45-30f5-4fe9-919b-58d3d5bc0100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:05.188253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-996edc45-30f5-4fe9-919b-58d3d5bc0100 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:05.206491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996edc45-30f5-4fe9-919b-58d3d5bc0100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:05.208117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996edc45-30f5-4fe9-919b-58d3d5bc0100 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:05.224273 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2928/5925] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:05.656091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f15afc-c66c-482e-851d-95107ff9367b service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:05.660081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-92f15afc-c66c-482e-851d-95107ff9367b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:05.695667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f15afc-c66c-482e-851d-95107ff9367b service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:05.701978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:05.702910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:05.703335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:05.703693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92 admin admin] PATH_INFO: `/v3/projects/6be2f8502e5941c8bee08b7397dbdeff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:05.705644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be2f8502e5941c8bee08b7397dbdeff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:05.707787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:05.753478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:05.764577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be2f8502e5941c8bee08b7397dbdeff', 'request_id': 'req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:05.773488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be2f8502e5941c8bee08b7397dbdeff', 'request_id': 'req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:05.780553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be2f8502e5941c8bee08b7397dbdeff', 'request_id': 'req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:05.792176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92 admin admin] The token cache is being invalidate because project 6be2f8502e5941c8bee08b7397dbdeff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:05.792555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:05.793239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f4176d9d-cdf3-4e9c-9a09-63ee7a95bf92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:05.794745 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2998/5926] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:05 2022] DELETE /identity/v3/projects/6be2f8502e5941c8bee08b7397dbdeff => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:05.799686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5e3c2-e845-4a29-8360-63903e4c2e86 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:05.799988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5e3c2-e845-4a29-8360-63903e4c2e86 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:05.800271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5e3c2-e845-4a29-8360-63903e4c2e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:05.821035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-18d5e3c2-e845-4a29-8360-63903e4c2e86 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:05.822929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-18d5e3c2-e845-4a29-8360-63903e4c2e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:05.893172 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2929/5927] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:05.896226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e525c66-32bb-495c-a4f2-8cd1f5fa259b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:05.896525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e525c66-32bb-495c-a4f2-8cd1f5fa259b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:05.896806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e525c66-32bb-495c-a4f2-8cd1f5fa259b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:05.916845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5e525c66-32bb-495c-a4f2-8cd1f5fa259b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:05.918100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5e525c66-32bb-495c-a4f2-8cd1f5fa259b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:05.926559 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 2999/5928] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:05.930976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5e3c2-e845-4a29-8360-63903e4c2e86 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:05.942603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5e3c2-e845-4a29-8360-63903e4c2e86 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:05.948238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d39e63-c2af-4e0f-91bf-044d274a5459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:05.948888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d39e63-c2af-4e0f-91bf-044d274a5459 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:05.949204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d39e63-c2af-4e0f-91bf-044d274a5459 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:05.949485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d39e63-c2af-4e0f-91bf-044d274a5459 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:05.950139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d39e63-c2af-4e0f-91bf-044d274a5459 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:10:05.950486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d39e63-c2af-4e0f-91bf-044d274a5459 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:05.952112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d39e63-c2af-4e0f-91bf-044d274a5459 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:05.968793 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2930/5929] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:10:05.970036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e525c66-32bb-495c-a4f2-8cd1f5fa259b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:05.979852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e525c66-32bb-495c-a4f2-8cd1f5fa259b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:05.984553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90665e86-e3e2-4fa5-8641-eaa17e95ff38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:05.985216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90665e86-e3e2-4fa5-8641-eaa17e95ff38 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:05.985552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90665e86-e3e2-4fa5-8641-eaa17e95ff38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:05.985827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90665e86-e3e2-4fa5-8641-eaa17e95ff38 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:05.986479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90665e86-e3e2-4fa5-8641-eaa17e95ff38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:05.988403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90665e86-e3e2-4fa5-8641-eaa17e95ff38 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:06.018285 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3000/5930] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:05 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:06.023004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d39e63-c2af-4e0f-91bf-044d274a5459 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:06.033800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d39e63-c2af-4e0f-91bf-044d274a5459 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:06.060988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:06.094425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb0eb9-0187-4e35-b163-aa8ab225d407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:06.095073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb0eb9-0187-4e35-b163-aa8ab225d407 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:06.095361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb0eb9-0187-4e35-b163-aa8ab225d407 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:06.095640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb0eb9-0187-4e35-b163-aa8ab225d407 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:06.096272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb0eb9-0187-4e35-b163-aa8ab225d407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:06.098239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb0eb9-0187-4e35-b163-aa8ab225d407 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:06.129627 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2931/5931] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:06 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:06.130787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90665e86-e3e2-4fa5-8641-eaa17e95ff38 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:06.148350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90665e86-e3e2-4fa5-8641-eaa17e95ff38 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:06.153468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19434d-0741-4f9a-848d-3bd1f9a6b546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:06.154062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19434d-0741-4f9a-848d-3bd1f9a6b546 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:06.154350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19434d-0741-4f9a-848d-3bd1f9a6b546 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:06.154628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19434d-0741-4f9a-848d-3bd1f9a6b546 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:06.155406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19434d-0741-4f9a-848d-3bd1f9a6b546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:06.157083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19434d-0741-4f9a-848d-3bd1f9a6b546 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:06.171624 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3001/5932] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:06.173740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb0eb9-0187-4e35-b163-aa8ab225d407 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:06.186686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb0eb9-0187-4e35-b163-aa8ab225d407 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:06.195325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eb5d1-cb84-4aa8-a49a-100769dd51ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:06.195945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eb5d1-cb84-4aa8-a49a-100769dd51ff admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:06.196228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eb5d1-cb84-4aa8-a49a-100769dd51ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:06.196503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eb5d1-cb84-4aa8-a49a-100769dd51ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:06.197175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eb5d1-cb84-4aa8-a49a-100769dd51ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:06.198783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eb5d1-cb84-4aa8-a49a-100769dd51ff admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:06.209535 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2932/5933] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:06.213535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19434d-0741-4f9a-848d-3bd1f9a6b546 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:06.223649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19434d-0741-4f9a-848d-3bd1f9a6b546 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:06.228734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668d728-6f15-4c4b-b463-a7885f983125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:06.229426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668d728-6f15-4c4b-b463-a7885f983125 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:06.229714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668d728-6f15-4c4b-b463-a7885f983125 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:06.229995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668d728-6f15-4c4b-b463-a7885f983125 admin admin] PATH_INFO: `/v3/projects/489665c75740426db3d4473135f51e93/users/d12be039076242aa8ffff6914cddc3e9/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:06.245625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668d728-6f15-4c4b-b463-a7885f983125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=489665c75740426db3d4473135f51e93, user_id=d12be039076242aa8ffff6914cddc3e9, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:06.248334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668d728-6f15-4c4b-b463-a7885f983125 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:06.261803 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3002/5934] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:06 2022] PUT /identity/v3/projects/489665c75740426db3d4473135f51e93/users/d12be039076242aa8ffff6914cddc3e9/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:10:06.272269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1631ab-8c98-4ba9-9633-f450964090a3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:06.272573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1631ab-8c98-4ba9-9633-f450964090a3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:06.272879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1631ab-8c98-4ba9-9633-f450964090a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:06.302871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7f1631ab-8c98-4ba9-9633-f450964090a3 None None] MFA Rules not processed for user `d12be039076242aa8ffff6914cddc3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:06.304004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7f1631ab-8c98-4ba9-9633-f450964090a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:06.373158 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2933/5935] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:06.374577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668d728-6f15-4c4b-b463-a7885f983125 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:06.443073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668d728-6f15-4c4b-b463-a7885f983125 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:06.447994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c31103-5d6a-4e54-bd9c-5fd367c9fac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:06.448603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c31103-5d6a-4e54-bd9c-5fd367c9fac6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:06.448888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c31103-5d6a-4e54-bd9c-5fd367c9fac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:06.449199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c31103-5d6a-4e54-bd9c-5fd367c9fac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:06.449861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c31103-5d6a-4e54-bd9c-5fd367c9fac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:06.451830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c31103-5d6a-4e54-bd9c-5fd367c9fac6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:06.481962 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3003/5936] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:06 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:06.483144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1631ab-8c98-4ba9-9633-f450964090a3 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:06.494579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1631ab-8c98-4ba9-9633-f450964090a3 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:06.554438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1f0fd-40c5-446a-b0a1-7c25a8796a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:06.555360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1f0fd-40c5-446a-b0a1-7c25a8796a09 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:06.555360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1f0fd-40c5-446a-b0a1-7c25a8796a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:06.556039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1f0fd-40c5-446a-b0a1-7c25a8796a09 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:06.556502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1f0fd-40c5-446a-b0a1-7c25a8796a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:06.559351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1f0fd-40c5-446a-b0a1-7c25a8796a09 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:06.586086 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2934/5937] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:06 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:06.587381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c31103-5d6a-4e54-bd9c-5fd367c9fac6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:06.599565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c31103-5d6a-4e54-bd9c-5fd367c9fac6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:06.605439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc15c38-5221-41e7-9d63-75e2dd4cd4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:06.606128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc15c38-5221-41e7-9d63-75e2dd4cd4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:06.606490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc15c38-5221-41e7-9d63-75e2dd4cd4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:06.606844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc15c38-5221-41e7-9d63-75e2dd4cd4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:06.607641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc15c38-5221-41e7-9d63-75e2dd4cd4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:06.609705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc15c38-5221-41e7-9d63-75e2dd4cd4e4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:06.625310 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3004/5938] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:06.626551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1f0fd-40c5-446a-b0a1-7c25a8796a09 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:06.639926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1f0fd-40c5-446a-b0a1-7c25a8796a09 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:06.646931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12306f9-c0f3-417a-ba9e-dbb387ba4a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:06.647711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12306f9-c0f3-417a-ba9e-dbb387ba4a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:06.648009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12306f9-c0f3-417a-ba9e-dbb387ba4a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:06.648612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12306f9-c0f3-417a-ba9e-dbb387ba4a65 admin admin] PATH_INFO: `/v3/projects/da6aa505e139438ab57e8ce228606e41/users/dde2fa10475b419a83e79e83677859a6/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:06.676930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12306f9-c0f3-417a-ba9e-dbb387ba4a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6aa505e139438ab57e8ce228606e41, user_id=dde2fa10475b419a83e79e83677859a6, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:06.680011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12306f9-c0f3-417a-ba9e-dbb387ba4a65 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:06.697719 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2935/5939] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:06 2022] PUT /identity/v3/projects/da6aa505e139438ab57e8ce228606e41/users/dde2fa10475b419a83e79e83677859a6/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:06.701716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc15c38-5221-41e7-9d63-75e2dd4cd4e4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:06.713342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc15c38-5221-41e7-9d63-75e2dd4cd4e4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:06.753121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:06.787850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476224a7-85b5-4d48-a947-799c52eca8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:06.788511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476224a7-85b5-4d48-a947-799c52eca8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:06.789022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476224a7-85b5-4d48-a947-799c52eca8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:06.789310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476224a7-85b5-4d48-a947-799c52eca8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:06.789962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476224a7-85b5-4d48-a947-799c52eca8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:06.791707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476224a7-85b5-4d48-a947-799c52eca8b7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:06.802002 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3005/5940] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:06.806437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12306f9-c0f3-417a-ba9e-dbb387ba4a65 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:06.817119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:06.825300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12306f9-c0f3-417a-ba9e-dbb387ba4a65 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:06.830364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1dcb9-4afb-4d79-b13f-bbefc9cfb297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:06.831105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1dcb9-4afb-4d79-b13f-bbefc9cfb297 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:06.831399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1dcb9-4afb-4d79-b13f-bbefc9cfb297 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:06.831675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1dcb9-4afb-4d79-b13f-bbefc9cfb297 admin admin] PATH_INFO: `/v3/projects/da6aa505e139438ab57e8ce228606e41/users/dde2fa10475b419a83e79e83677859a6/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:06.833691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1dcb9-4afb-4d79-b13f-bbefc9cfb297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6aa505e139438ab57e8ce228606e41, user_id=dde2fa10475b419a83e79e83677859a6, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:06.836499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1dcb9-4afb-4d79-b13f-bbefc9cfb297 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:06.848352 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2936/5941] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:06 2022] PUT /identity/v3/projects/da6aa505e139438ab57e8ce228606e41/users/dde2fa10475b419a83e79e83677859a6/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:06.860022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575adb34-885a-401a-8065-eabd5f6f438d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:06.860319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575adb34-885a-401a-8065-eabd5f6f438d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:06.861250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575adb34-885a-401a-8065-eabd5f6f438d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:06.896066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-575adb34-885a-401a-8065-eabd5f6f438d None None] MFA Rules not processed for user `dde2fa10475b419a83e79e83677859a6`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:06.897862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-575adb34-885a-401a-8065-eabd5f6f438d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:06.979641 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3006/5942] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:06.992559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1dcb9-4afb-4d79-b13f-bbefc9cfb297 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:06.994327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-63d1dcb9-4afb-4d79-b13f-bbefc9cfb297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:07.065554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1dcb9-4afb-4d79-b13f-bbefc9cfb297 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:07.069719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b1b7c-f1e6-4b0f-8e67-dd8a1d8e0453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:07.070383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b1b7c-f1e6-4b0f-8e67-dd8a1d8e0453 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:07.070672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b1b7c-f1e6-4b0f-8e67-dd8a1d8e0453 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:07.070950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b1b7c-f1e6-4b0f-8e67-dd8a1d8e0453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:07.082026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b1b7c-f1e6-4b0f-8e67-dd8a1d8e0453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:07.083788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b1b7c-f1e6-4b0f-8e67-dd8a1d8e0453 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:07.095884 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2937/5943] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:07.147033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575adb34-885a-401a-8065-eabd5f6f438d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:07.148716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-575adb34-885a-401a-8065-eabd5f6f438d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:07.166545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575adb34-885a-401a-8065-eabd5f6f438d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:07.170955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7c091-d559-45b2-875a-146e5d1876b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:07.171972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7c091-d559-45b2-875a-146e5d1876b0 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:07.172127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7c091-d559-45b2-875a-146e5d1876b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:07.172513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7c091-d559-45b2-875a-146e5d1876b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:07.258063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7c091-d559-45b2-875a-146e5d1876b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:07.259696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7c091-d559-45b2-875a-146e5d1876b0 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:07.272243 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3007/5944] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:10.704967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b1b7c-f1e6-4b0f-8e67-dd8a1d8e0453 service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:10.770433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b1b7c-f1e6-4b0f-8e67-dd8a1d8e0453 service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:10.776343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:10.777061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:10.777375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:10.777772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] PATH_INFO: `/v3/users/d12be039076242aa8ffff6914cddc3e9` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:10.780509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d12be039076242aa8ffff6914cddc3e9)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:10.783021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:10.837287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd12be039076242aa8ffff6914cddc3e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:10.850015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd12be039076242aa8ffff6914cddc3e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:10.851044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd12be039076242aa8ffff6914cddc3e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:10.858594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd12be039076242aa8ffff6914cddc3e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:10.867518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd12be039076242aa8ffff6914cddc3e9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:10.873808 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2938/5945] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:10 2022] DELETE /identity/v3/users/d12be039076242aa8ffff6914cddc3e9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:10.878463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9415d6f-0ddd-49e2-8653-e31409102020 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:10.878955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9415d6f-0ddd-49e2-8653-e31409102020 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:10.879040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9415d6f-0ddd-49e2-8653-e31409102020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:10.898617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d9415d6f-0ddd-49e2-8653-e31409102020 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:10.899831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d9415d6f-0ddd-49e2-8653-e31409102020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:10.955942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:10.966620 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3008/5946] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:10.976819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:10.978355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:10.999344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:11.060578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df0c31-d3fa-4842-993b-10c229c38702 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:11.065268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192c049-1f76-433b-8adf-a59f004c119d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:11.065909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192c049-1f76-433b-8adf-a59f004c119d service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:11.066223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192c049-1f76-433b-8adf-a59f004c119d service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:11.066465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192c049-1f76-433b-8adf-a59f004c119d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:11.076355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192c049-1f76-433b-8adf-a59f004c119d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:11.078185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192c049-1f76-433b-8adf-a59f004c119d service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:11.088465 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2939/5947] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:11.806170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9415d6f-0ddd-49e2-8653-e31409102020 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:11.807683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d9415d6f-0ddd-49e2-8653-e31409102020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:11.820063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9415d6f-0ddd-49e2-8653-e31409102020 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:11.824686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:11.825359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:11.825646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:11.825923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] PATH_INFO: `/v3/projects/489665c75740426db3d4473135f51e93` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:11.827328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=489665c75740426db3d4473135f51e93)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:11.829419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:11.892850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '489665c75740426db3d4473135f51e93', 'request_id': 'req-bc7ab746-0930-43e0-ae7c-8aa821736e42'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:11.900576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '489665c75740426db3d4473135f51e93', 'request_id': 'req-bc7ab746-0930-43e0-ae7c-8aa821736e42'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:11.907035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '489665c75740426db3d4473135f51e93', 'request_id': 'req-bc7ab746-0930-43e0-ae7c-8aa821736e42'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:11.918567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] The token cache is being invalidate because project 489665c75740426db3d4473135f51e93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:11.918953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:11.919624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:11.921216 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3009/5948] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:11 2022] DELETE /identity/v3/projects/489665c75740426db3d4473135f51e93 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:11.925780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192c049-1f76-433b-8adf-a59f004c119d service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:11.936248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192c049-1f76-433b-8adf-a59f004c119d service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:12.022724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ccc14-d258-4c46-848a-1f076d559015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:12.023338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ccc14-d258-4c46-848a-1f076d559015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:12.023625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ccc14-d258-4c46-848a-1f076d559015 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:12.023901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ccc14-d258-4c46-848a-1f076d559015 admin admin] PATH_INFO: `/v3/users/dde2fa10475b419a83e79e83677859a6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:12.025533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ccc14-d258-4c46-848a-1f076d559015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dde2fa10475b419a83e79e83677859a6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:12.028012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ccc14-d258-4c46-848a-1f076d559015 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:12.094165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f9ccc14-d258-4c46-848a-1f076d559015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dde2fa10475b419a83e79e83677859a6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:12.109938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f9ccc14-d258-4c46-848a-1f076d559015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dde2fa10475b419a83e79e83677859a6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:12.111235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f9ccc14-d258-4c46-848a-1f076d559015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dde2fa10475b419a83e79e83677859a6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:12.120769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f9ccc14-d258-4c46-848a-1f076d559015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dde2fa10475b419a83e79e83677859a6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:12.129795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9f9ccc14-d258-4c46-848a-1f076d559015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dde2fa10475b419a83e79e83677859a6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:12.137687 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2940/5949] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:11 2022] DELETE /identity/v3/users/dde2fa10475b419a83e79e83677859a6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:12.779179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:12.780787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:12.800625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ab746-0930-43e0-ae7c-8aa821736e42 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:12.889451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba858dee-f0ea-4be6-86d1-e1574b4d7623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:12.890119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba858dee-f0ea-4be6-86d1-e1574b4d7623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:12.890427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba858dee-f0ea-4be6-86d1-e1574b4d7623 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:12.890741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba858dee-f0ea-4be6-86d1-e1574b4d7623 admin admin] PATH_INFO: `/v3/projects/da6aa505e139438ab57e8ce228606e41` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:12.892562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba858dee-f0ea-4be6-86d1-e1574b4d7623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da6aa505e139438ab57e8ce228606e41)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:12.894632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba858dee-f0ea-4be6-86d1-e1574b4d7623 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:12.960847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ba858dee-f0ea-4be6-86d1-e1574b4d7623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da6aa505e139438ab57e8ce228606e41', 'request_id': 'req-ba858dee-f0ea-4be6-86d1-e1574b4d7623'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:12.970656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ba858dee-f0ea-4be6-86d1-e1574b4d7623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da6aa505e139438ab57e8ce228606e41', 'request_id': 'req-ba858dee-f0ea-4be6-86d1-e1574b4d7623'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:12.978043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ba858dee-f0ea-4be6-86d1-e1574b4d7623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da6aa505e139438ab57e8ce228606e41', 'request_id': 'req-ba858dee-f0ea-4be6-86d1-e1574b4d7623'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:12.988844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ba858dee-f0ea-4be6-86d1-e1574b4d7623 admin admin] The token cache is being invalidate because project da6aa505e139438ab57e8ce228606e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:12.989182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ba858dee-f0ea-4be6-86d1-e1574b4d7623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:12.989846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ba858dee-f0ea-4be6-86d1-e1574b4d7623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:12.991589 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3010/5950] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:12 2022] DELETE /identity/v3/projects/da6aa505e139438ab57e8ce228606e41 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:12.996220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad06e5-f8c8-461f-a593-60fd0b592704 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:12.996732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad06e5-f8c8-461f-a593-60fd0b592704 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:12.996873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad06e5-f8c8-461f-a593-60fd0b592704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.017517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f8ad06e5-f8c8-461f-a593-60fd0b592704 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:13.018759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f8ad06e5-f8c8-461f-a593-60fd0b592704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:13.052173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:13.088850 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2941/5951] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:13.096794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10220cf-368a-4444-909b-d67458e48f29 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:13.097121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10220cf-368a-4444-909b-d67458e48f29 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:13.097427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10220cf-368a-4444-909b-d67458e48f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.117994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b10220cf-368a-4444-909b-d67458e48f29 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:13.119398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b10220cf-368a-4444-909b-d67458e48f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:13.129435 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3011/5952] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:13.133247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad06e5-f8c8-461f-a593-60fd0b592704 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:13.144548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad06e5-f8c8-461f-a593-60fd0b592704 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:13.149313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3897e26-0271-447b-8f14-5ad653e0dee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:13.149995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3897e26-0271-447b-8f14-5ad653e0dee2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:13.150351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3897e26-0271-447b-8f14-5ad653e0dee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:13.150697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3897e26-0271-447b-8f14-5ad653e0dee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.151436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3897e26-0271-447b-8f14-5ad653e0dee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:10:13.151810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3897e26-0271-447b-8f14-5ad653e0dee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:13.153550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3897e26-0271-447b-8f14-5ad653e0dee2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:13.168931 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2942/5953] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:10:13.171196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10220cf-368a-4444-909b-d67458e48f29 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:13.177502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:13.181745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10220cf-368a-4444-909b-d67458e48f29 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:13.187772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30d046-3a52-4b88-bb07-01dfa9d0687d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:13.188570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30d046-3a52-4b88-bb07-01dfa9d0687d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:13.189088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30d046-3a52-4b88-bb07-01dfa9d0687d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:13.189554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30d046-3a52-4b88-bb07-01dfa9d0687d admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.190491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30d046-3a52-4b88-bb07-01dfa9d0687d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:13.192535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30d046-3a52-4b88-bb07-01dfa9d0687d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:13.225549 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3012/5954] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:13 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:13.227229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3897e26-0271-447b-8f14-5ad653e0dee2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:13.238038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3897e26-0271-447b-8f14-5ad653e0dee2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:13.295036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826acff-a587-4461-8411-56cbb2c09726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:13.295648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826acff-a587-4461-8411-56cbb2c09726 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:13.295931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826acff-a587-4461-8411-56cbb2c09726 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:13.296202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826acff-a587-4461-8411-56cbb2c09726 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.296822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826acff-a587-4461-8411-56cbb2c09726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:13.298810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826acff-a587-4461-8411-56cbb2c09726 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:13.321480 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2943/5955] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:13 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:13.325643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30d046-3a52-4b88-bb07-01dfa9d0687d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:13.336739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30d046-3a52-4b88-bb07-01dfa9d0687d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:13.341719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9af4eb-157f-4ac6-b67a-44b3e2bc9360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:13.342258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9af4eb-157f-4ac6-b67a-44b3e2bc9360 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:13.342543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9af4eb-157f-4ac6-b67a-44b3e2bc9360 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:13.342817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9af4eb-157f-4ac6-b67a-44b3e2bc9360 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.343451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9af4eb-157f-4ac6-b67a-44b3e2bc9360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:13.345112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9af4eb-157f-4ac6-b67a-44b3e2bc9360 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:13.355365 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3013/5956] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:13.359480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826acff-a587-4461-8411-56cbb2c09726 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:13.371680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826acff-a587-4461-8411-56cbb2c09726 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:13.377759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8fa33-783a-438a-88a3-39a570044ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:13.378463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8fa33-783a-438a-88a3-39a570044ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:13.378870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8fa33-783a-438a-88a3-39a570044ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:13.379249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8fa33-783a-438a-88a3-39a570044ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.380348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8fa33-783a-438a-88a3-39a570044ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:13.383802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8fa33-783a-438a-88a3-39a570044ee7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:13.398495 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2944/5957] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:13.400096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9af4eb-157f-4ac6-b67a-44b3e2bc9360 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:13.411522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9af4eb-157f-4ac6-b67a-44b3e2bc9360 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:13.417373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1307bc-079f-4df0-93a7-7ccaed1d98c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:13.418195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1307bc-079f-4df0-93a7-7ccaed1d98c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:13.418602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1307bc-079f-4df0-93a7-7ccaed1d98c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:13.419006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1307bc-079f-4df0-93a7-7ccaed1d98c8 admin admin] PATH_INFO: `/v3/projects/00749088762b4b06bdf5a794b64eaa41/users/eb4f739c44a84c3ba42b7715a44e1d5d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.436511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1307bc-079f-4df0-93a7-7ccaed1d98c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00749088762b4b06bdf5a794b64eaa41, user_id=eb4f739c44a84c3ba42b7715a44e1d5d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:13.440067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1307bc-079f-4df0-93a7-7ccaed1d98c8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:13.464477 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3014/5958] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:13 2022] PUT /identity/v3/projects/00749088762b4b06bdf5a794b64eaa41/users/eb4f739c44a84c3ba42b7715a44e1d5d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:13.468800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66d7f6-62e8-4fda-9389-867e749ae7b5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:13.469184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66d7f6-62e8-4fda-9389-867e749ae7b5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:13.469523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66d7f6-62e8-4fda-9389-867e749ae7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.502144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5e66d7f6-62e8-4fda-9389-867e749ae7b5 None None] MFA Rules not processed for user `eb4f739c44a84c3ba42b7715a44e1d5d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:13.503392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5e66d7f6-62e8-4fda-9389-867e749ae7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:13.579810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2945/5959] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:13.580157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1307bc-079f-4df0-93a7-7ccaed1d98c8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:13.655171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1307bc-079f-4df0-93a7-7ccaed1d98c8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:13.659859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897aaab2-9a35-4741-93c9-e33eaf1b62e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:13.660463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897aaab2-9a35-4741-93c9-e33eaf1b62e8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:13.660748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897aaab2-9a35-4741-93c9-e33eaf1b62e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:13.661058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897aaab2-9a35-4741-93c9-e33eaf1b62e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.661691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897aaab2-9a35-4741-93c9-e33eaf1b62e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:13.663654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897aaab2-9a35-4741-93c9-e33eaf1b62e8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:13.698396 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3015/5960] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:13 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:10:13.702097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66d7f6-62e8-4fda-9389-867e749ae7b5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:13.721561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66d7f6-62e8-4fda-9389-867e749ae7b5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:13.751268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:13.796568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e5829-3dd3-4f74-93d2-a375e7f17c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:13.797313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e5829-3dd3-4f74-93d2-a375e7f17c3e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:13.797670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e5829-3dd3-4f74-93d2-a375e7f17c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:13.798568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e5829-3dd3-4f74-93d2-a375e7f17c3e admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.799305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e5829-3dd3-4f74-93d2-a375e7f17c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:13.801363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e5829-3dd3-4f74-93d2-a375e7f17c3e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:13.831159 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2946/5961] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:13 2022] POST /identity/v3/users => generated 298 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:13.832439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897aaab2-9a35-4741-93c9-e33eaf1b62e8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:13.844967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897aaab2-9a35-4741-93c9-e33eaf1b62e8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:13.850256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea73ec-54ef-45ff-aba1-86d61b834910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:13.851242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea73ec-54ef-45ff-aba1-86d61b834910 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:13.851533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea73ec-54ef-45ff-aba1-86d61b834910 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:13.851813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea73ec-54ef-45ff-aba1-86d61b834910 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.852461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea73ec-54ef-45ff-aba1-86d61b834910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:13.854395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea73ec-54ef-45ff-aba1-86d61b834910 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:13.870822 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3016/5962] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:13.872031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e5829-3dd3-4f74-93d2-a375e7f17c3e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:13.883119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e5829-3dd3-4f74-93d2-a375e7f17c3e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:13.888931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1745b76-f1fe-47eb-bc50-9ed20f11f94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:13.889643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1745b76-f1fe-47eb-bc50-9ed20f11f94f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:13.889950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1745b76-f1fe-47eb-bc50-9ed20f11f94f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:13.890716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1745b76-f1fe-47eb-bc50-9ed20f11f94f admin admin] PATH_INFO: `/v3/projects/5f7fd41ac5ba4d788daa97c5c8185ddb/users/bf7930adfbd04996a6e1af32b56cfd3b/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:13.908141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1745b76-f1fe-47eb-bc50-9ed20f11f94f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7fd41ac5ba4d788daa97c5c8185ddb, user_id=bf7930adfbd04996a6e1af32b56cfd3b, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:13.910962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1745b76-f1fe-47eb-bc50-9ed20f11f94f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:13.932918 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2947/5963] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:13 2022] PUT /identity/v3/projects/5f7fd41ac5ba4d788daa97c5c8185ddb/users/bf7930adfbd04996a6e1af32b56cfd3b/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:13.934101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea73ec-54ef-45ff-aba1-86d61b834910 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:13.945235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea73ec-54ef-45ff-aba1-86d61b834910 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:14.014770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031827e-63af-44d0-b720-a838c700b53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:14.015402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031827e-63af-44d0-b720-a838c700b53d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:14.015693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031827e-63af-44d0-b720-a838c700b53d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:14.015978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031827e-63af-44d0-b720-a838c700b53d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:14.016670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031827e-63af-44d0-b720-a838c700b53d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:14.019570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031827e-63af-44d0-b720-a838c700b53d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:14.034231 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3017/5964] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:14.036025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1745b76-f1fe-47eb-bc50-9ed20f11f94f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:14.047168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1745b76-f1fe-47eb-bc50-9ed20f11f94f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:14.054639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025870c-a0ea-4ed3-803e-9cc7ad76e9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:14.054985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025870c-a0ea-4ed3-803e-9cc7ad76e9fb admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:14.055349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025870c-a0ea-4ed3-803e-9cc7ad76e9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:14.055704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025870c-a0ea-4ed3-803e-9cc7ad76e9fb admin admin] PATH_INFO: `/v3/projects/5f7fd41ac5ba4d788daa97c5c8185ddb/users/bf7930adfbd04996a6e1af32b56cfd3b/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:14.058078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025870c-a0ea-4ed3-803e-9cc7ad76e9fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7fd41ac5ba4d788daa97c5c8185ddb, user_id=bf7930adfbd04996a6e1af32b56cfd3b, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:14.060803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025870c-a0ea-4ed3-803e-9cc7ad76e9fb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:14.079512 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2948/5965] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:14 2022] PUT /identity/v3/projects/5f7fd41ac5ba4d788daa97c5c8185ddb/users/bf7930adfbd04996a6e1af32b56cfd3b/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:14.089575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb422e-fe88-4568-8107-b20f294a95ea None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:14.089876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb422e-fe88-4568-8107-b20f294a95ea None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:14.090636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb422e-fe88-4568-8107-b20f294a95ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:14.124444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-69fb422e-fe88-4568-8107-b20f294a95ea None None] MFA Rules not processed for user `bf7930adfbd04996a6e1af32b56cfd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:14.125555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-69fb422e-fe88-4568-8107-b20f294a95ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:14.131987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:14.210854 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3018/5966] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:14.221823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025870c-a0ea-4ed3-803e-9cc7ad76e9fb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:14.223686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0025870c-a0ea-4ed3-803e-9cc7ad76e9fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:14.302639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025870c-a0ea-4ed3-803e-9cc7ad76e9fb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:14.306513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08abca-93a1-4328-9518-e5a90d79bfbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:14.307231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08abca-93a1-4328-9518-e5a90d79bfbd service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:14.307752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08abca-93a1-4328-9518-e5a90d79bfbd service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:14.308128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08abca-93a1-4328-9518-e5a90d79bfbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:14.323340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08abca-93a1-4328-9518-e5a90d79bfbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:14.325016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08abca-93a1-4328-9518-e5a90d79bfbd service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:14.337843 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2949/5967] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:14.457153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb422e-fe88-4568-8107-b20f294a95ea None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:14.471896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb422e-fe88-4568-8107-b20f294a95ea None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:14.475564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0132ed-e5ee-45f6-b06a-0c3a373df4c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:14.476943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0132ed-e5ee-45f6-b06a-0c3a373df4c1 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:14.476943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0132ed-e5ee-45f6-b06a-0c3a373df4c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:14.477188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0132ed-e5ee-45f6-b06a-0c3a373df4c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:14.541215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0132ed-e5ee-45f6-b06a-0c3a373df4c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:14.542850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0132ed-e5ee-45f6-b06a-0c3a373df4c1 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:14.553586 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3019/5968] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:16.245887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08abca-93a1-4328-9518-e5a90d79bfbd service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:16.319882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08abca-93a1-4328-9518-e5a90d79bfbd service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:16.325425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:16.326184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:16.326539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:16.326894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] PATH_INFO: `/v3/users/eb4f739c44a84c3ba42b7715a44e1d5d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:16.328286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb4f739c44a84c3ba42b7715a44e1d5d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:16.330509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:16.374670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:16.401040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb4f739c44a84c3ba42b7715a44e1d5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:16.413839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb4f739c44a84c3ba42b7715a44e1d5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:16.414641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb4f739c44a84c3ba42b7715a44e1d5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:16.421961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb4f739c44a84c3ba42b7715a44e1d5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:16.433320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb4f739c44a84c3ba42b7715a44e1d5d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:16.439473 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2950/5969] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:16 2022] DELETE /identity/v3/users/eb4f739c44a84c3ba42b7715a44e1d5d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:16.444223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41105ca3-7f0a-42da-854c-306a6e1f4a33 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:16.444706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41105ca3-7f0a-42da-854c-306a6e1f4a33 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:16.444968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41105ca3-7f0a-42da-854c-306a6e1f4a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:16.465419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-41105ca3-7f0a-42da-854c-306a6e1f4a33 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:16.466649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-41105ca3-7f0a-42da-854c-306a6e1f4a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:16.535359 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3020/5970] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:16.545232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:16.546671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:16.610962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6af7a-e2e9-477c-9d71-e59a24089693 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:16.614692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96a23a-2e91-41b8-91a2-a0d670fa5b33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:16.615307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96a23a-2e91-41b8-91a2-a0d670fa5b33 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:16.615736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96a23a-2e91-41b8-91a2-a0d670fa5b33 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:16.616026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96a23a-2e91-41b8-91a2-a0d670fa5b33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:16.625420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96a23a-2e91-41b8-91a2-a0d670fa5b33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:16.627527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96a23a-2e91-41b8-91a2-a0d670fa5b33 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:16.639099 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2951/5971] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:10:17.062231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41105ca3-7f0a-42da-854c-306a6e1f4a33 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:17.063732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-41105ca3-7f0a-42da-854c-306a6e1f4a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:17.075843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41105ca3-7f0a-42da-854c-306a6e1f4a33 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:17.080515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:17.081183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:17.081476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:17.081773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] PATH_INFO: `/v3/projects/00749088762b4b06bdf5a794b64eaa41` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:17.082904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=00749088762b4b06bdf5a794b64eaa41)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:17.085377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:17.140865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:17.151761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00749088762b4b06bdf5a794b64eaa41', 'request_id': 'req-da081510-34bf-4eaa-adc8-d319a17392cc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.160337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00749088762b4b06bdf5a794b64eaa41', 'request_id': 'req-da081510-34bf-4eaa-adc8-d319a17392cc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.167027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00749088762b4b06bdf5a794b64eaa41', 'request_id': 'req-da081510-34bf-4eaa-adc8-d319a17392cc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.180394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] The token cache is being invalidate because project 00749088762b4b06bdf5a794b64eaa41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:17.180778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.181546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.183163 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3021/5972] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:17 2022] DELETE /identity/v3/projects/00749088762b4b06bdf5a794b64eaa41 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:17.187249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96a23a-2e91-41b8-91a2-a0d670fa5b33 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:17.197324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96a23a-2e91-41b8-91a2-a0d670fa5b33 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:17.258536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee90d7e-b913-4e36-bd8f-2b9ab46e11a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:17.259248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee90d7e-b913-4e36-bd8f-2b9ab46e11a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:17.259455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee90d7e-b913-4e36-bd8f-2b9ab46e11a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:17.259738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee90d7e-b913-4e36-bd8f-2b9ab46e11a7 admin admin] PATH_INFO: `/v3/users/bf7930adfbd04996a6e1af32b56cfd3b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:17.260888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee90d7e-b913-4e36-bd8f-2b9ab46e11a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf7930adfbd04996a6e1af32b56cfd3b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:17.262859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee90d7e-b913-4e36-bd8f-2b9ab46e11a7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:17.312554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bee90d7e-b913-4e36-bd8f-2b9ab46e11a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf7930adfbd04996a6e1af32b56cfd3b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.314416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:17.329814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bee90d7e-b913-4e36-bd8f-2b9ab46e11a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf7930adfbd04996a6e1af32b56cfd3b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.330556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bee90d7e-b913-4e36-bd8f-2b9ab46e11a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf7930adfbd04996a6e1af32b56cfd3b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.337730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bee90d7e-b913-4e36-bd8f-2b9ab46e11a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf7930adfbd04996a6e1af32b56cfd3b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.345119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bee90d7e-b913-4e36-bd8f-2b9ab46e11a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf7930adfbd04996a6e1af32b56cfd3b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.351530 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2952/5973] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:17 2022] DELETE /identity/v3/users/bf7930adfbd04996a6e1af32b56cfd3b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:17.634113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:17.635635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:17.651740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da081510-34bf-4eaa-adc8-d319a17392cc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:17.713770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:17.714518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:17.714879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:17.715171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4 admin admin] PATH_INFO: `/v3/projects/5f7fd41ac5ba4d788daa97c5c8185ddb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:17.716321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f7fd41ac5ba4d788daa97c5c8185ddb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:17.718303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:17.768944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f7fd41ac5ba4d788daa97c5c8185ddb', 'request_id': 'req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.776637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f7fd41ac5ba4d788daa97c5c8185ddb', 'request_id': 'req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.783736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f7fd41ac5ba4d788daa97c5c8185ddb', 'request_id': 'req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.794713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4 admin admin] The token cache is being invalidate because project 5f7fd41ac5ba4d788daa97c5c8185ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:17.795008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.795959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b2bb2b9-9bec-42e2-a495-ea4cb288b4a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:17.797520 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3022/5974] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:17 2022] DELETE /identity/v3/projects/5f7fd41ac5ba4d788daa97c5c8185ddb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:17.802843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5f72-67c1-418b-adc0-93387d64c535 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:17.803142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5f72-67c1-418b-adc0-93387d64c535 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:17.803429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5f72-67c1-418b-adc0-93387d64c535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:17.823668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b3bb5f72-67c1-418b-adc0-93387d64c535 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:17.824800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b3bb5f72-67c1-418b-adc0-93387d64c535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:17.893710 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2953/5975] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:17.896287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f2b69-721d-4f18-b73b-80fc75b23086 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:17.896816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f2b69-721d-4f18-b73b-80fc75b23086 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:17.897500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f2b69-721d-4f18-b73b-80fc75b23086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:17.931138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f50f2b69-721d-4f18-b73b-80fc75b23086 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:17.932467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f50f2b69-721d-4f18-b73b-80fc75b23086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:17.940804 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3023/5976] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:17.945105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5f72-67c1-418b-adc0-93387d64c535 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:17.955066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5f72-67c1-418b-adc0-93387d64c535 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:17.959886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d6864-a245-4e60-9b64-6cb2ca202d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:17.960521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d6864-a245-4e60-9b64-6cb2ca202d2a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:17.960832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d6864-a245-4e60-9b64-6cb2ca202d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:17.961145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d6864-a245-4e60-9b64-6cb2ca202d2a admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:17.961790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d6864-a245-4e60-9b64-6cb2ca202d2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:10:17.962096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d6864-a245-4e60-9b64-6cb2ca202d2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:17.963694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d6864-a245-4e60-9b64-6cb2ca202d2a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:17.976434 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2954/5977] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:10:17.980716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f2b69-721d-4f18-b73b-80fc75b23086 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:17.990934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f2b69-721d-4f18-b73b-80fc75b23086 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:17.995676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc899a2-5682-4bfd-b038-dacac7e13dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:17.996279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc899a2-5682-4bfd-b038-dacac7e13dfc admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:17.996570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc899a2-5682-4bfd-b038-dacac7e13dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:17.996846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc899a2-5682-4bfd-b038-dacac7e13dfc admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:17.997577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc899a2-5682-4bfd-b038-dacac7e13dfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:17.999489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc899a2-5682-4bfd-b038-dacac7e13dfc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:18.031803 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3024/5978] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:17 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:10:18.033824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d6864-a245-4e60-9b64-6cb2ca202d2a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:18.044208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d6864-a245-4e60-9b64-6cb2ca202d2a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:18.108018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054b51a-5136-480a-8294-2bceb4ba358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:18.108641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054b51a-5136-480a-8294-2bceb4ba358f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:18.109103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054b51a-5136-480a-8294-2bceb4ba358f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:18.109386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054b51a-5136-480a-8294-2bceb4ba358f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:18.110028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054b51a-5136-480a-8294-2bceb4ba358f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:18.112643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054b51a-5136-480a-8294-2bceb4ba358f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:18.151073 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2955/5979] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:18 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:18.152074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc899a2-5682-4bfd-b038-dacac7e13dfc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:18.163930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc899a2-5682-4bfd-b038-dacac7e13dfc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:18.169424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f6aaf-38d0-4401-a0ce-d5758ec1f798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:18.170142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f6aaf-38d0-4401-a0ce-d5758ec1f798 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:18.170505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f6aaf-38d0-4401-a0ce-d5758ec1f798 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:18.170878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f6aaf-38d0-4401-a0ce-d5758ec1f798 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:18.171650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f6aaf-38d0-4401-a0ce-d5758ec1f798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:18.173441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f6aaf-38d0-4401-a0ce-d5758ec1f798 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:18.192022 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3025/5980] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:10:18.193336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054b51a-5136-480a-8294-2bceb4ba358f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:18.205078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054b51a-5136-480a-8294-2bceb4ba358f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:18.209809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6a6a5-050b-41a3-b867-bcc491e87ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:18.210421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6a6a5-050b-41a3-b867-bcc491e87ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:18.210705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6a6a5-050b-41a3-b867-bcc491e87ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:18.210984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6a6a5-050b-41a3-b867-bcc491e87ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:18.211627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6a6a5-050b-41a3-b867-bcc491e87ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:18.213281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6a6a5-050b-41a3-b867-bcc491e87ab0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:18.225141 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2956/5981] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:18.229575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f6aaf-38d0-4401-a0ce-d5758ec1f798 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:18.239770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f6aaf-38d0-4401-a0ce-d5758ec1f798 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:18.246011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f5fdb-cd3b-4396-b8de-c6916a622922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:18.246011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f5fdb-cd3b-4396-b8de-c6916a622922 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:18.246283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f5fdb-cd3b-4396-b8de-c6916a622922 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:18.246423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f5fdb-cd3b-4396-b8de-c6916a622922 admin admin] PATH_INFO: `/v3/projects/f1fb30b6cd4d4012889c2d1b66c15a2b/users/490b6431459e4ea395d072892dc1c3f0/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:18.258711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:18.262160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f5fdb-cd3b-4396-b8de-c6916a622922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1fb30b6cd4d4012889c2d1b66c15a2b, user_id=490b6431459e4ea395d072892dc1c3f0, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:18.265313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f5fdb-cd3b-4396-b8de-c6916a622922 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:18.280486 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3026/5982] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:18 2022] PUT /identity/v3/projects/f1fb30b6cd4d4012889c2d1b66c15a2b/users/490b6431459e4ea395d072892dc1c3f0/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:18.292773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24d9b-9066-49b1-996e-df2d7b2807ad None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:18.293327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24d9b-9066-49b1-996e-df2d7b2807ad None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:18.293452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24d9b-9066-49b1-996e-df2d7b2807ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:18.321335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d8c24d9b-9066-49b1-996e-df2d7b2807ad None None] MFA Rules not processed for user `490b6431459e4ea395d072892dc1c3f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:18.322563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d8c24d9b-9066-49b1-996e-df2d7b2807ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:18.342946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:18.394653 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2957/5983] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:18.399227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f5fdb-cd3b-4396-b8de-c6916a622922 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:18.464330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f5fdb-cd3b-4396-b8de-c6916a622922 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:18.469033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e0ac7-cf3d-44a8-9bb1-7d2eec700d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:18.469645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e0ac7-cf3d-44a8-9bb1-7d2eec700d2e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:18.469969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e0ac7-cf3d-44a8-9bb1-7d2eec700d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:18.470201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e0ac7-cf3d-44a8-9bb1-7d2eec700d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:18.470841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e0ac7-cf3d-44a8-9bb1-7d2eec700d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:18.472773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e0ac7-cf3d-44a8-9bb1-7d2eec700d2e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:18.502890 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3027/5984] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:18 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:18.507066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c24d9b-9066-49b1-996e-df2d7b2807ad None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:18.518734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c24d9b-9066-49b1-996e-df2d7b2807ad None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:18.578341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a201f0-a410-4b8b-a597-25d70ed038db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:18.578814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a201f0-a410-4b8b-a597-25d70ed038db admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:18.579125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a201f0-a410-4b8b-a597-25d70ed038db admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:18.579402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a201f0-a410-4b8b-a597-25d70ed038db admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:18.580084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a201f0-a410-4b8b-a597-25d70ed038db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:18.582099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a201f0-a410-4b8b-a597-25d70ed038db admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:18.605003 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2958/5985] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:18 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:18.608279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e0ac7-cf3d-44a8-9bb1-7d2eec700d2e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:18.619722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e0ac7-cf3d-44a8-9bb1-7d2eec700d2e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:18.624613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54710e1-9a16-415f-a7ca-c937b65a2ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:18.625255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54710e1-9a16-415f-a7ca-c937b65a2ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:18.625544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54710e1-9a16-415f-a7ca-c937b65a2ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:18.625828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54710e1-9a16-415f-a7ca-c937b65a2ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:18.626475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54710e1-9a16-415f-a7ca-c937b65a2ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:18.628104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54710e1-9a16-415f-a7ca-c937b65a2ad5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:18.639363 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3028/5986] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:10:18.643574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a201f0-a410-4b8b-a597-25d70ed038db admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:18.654303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a201f0-a410-4b8b-a597-25d70ed038db admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:18.660182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5a711-9d5e-41dd-a31d-4d28025def22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:18.660863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5a711-9d5e-41dd-a31d-4d28025def22 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:18.661197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5a711-9d5e-41dd-a31d-4d28025def22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:18.661488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5a711-9d5e-41dd-a31d-4d28025def22 admin admin] PATH_INFO: `/v3/projects/60e2dc3e82534d4fa522a07d2ec07077/users/c93f19794119413f8e9873335473c208/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:18.683748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5a711-9d5e-41dd-a31d-4d28025def22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60e2dc3e82534d4fa522a07d2ec07077, user_id=c93f19794119413f8e9873335473c208, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:18.686625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5a711-9d5e-41dd-a31d-4d28025def22 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:18.702008 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2959/5987] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:18 2022] PUT /identity/v3/projects/60e2dc3e82534d4fa522a07d2ec07077/users/c93f19794119413f8e9873335473c208/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:10:18.706856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54710e1-9a16-415f-a7ca-c937b65a2ad5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:18.717359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54710e1-9a16-415f-a7ca-c937b65a2ad5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:18.777549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037668b-a502-4dec-83ce-efedc0b302df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:18.778170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037668b-a502-4dec-83ce-efedc0b302df admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:18.778455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037668b-a502-4dec-83ce-efedc0b302df admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:18.778733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037668b-a502-4dec-83ce-efedc0b302df admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:18.779376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037668b-a502-4dec-83ce-efedc0b302df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:18.781009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037668b-a502-4dec-83ce-efedc0b302df admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:18.791956 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3029/5988] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:18.796246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5a711-9d5e-41dd-a31d-4d28025def22 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:18.807180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5a711-9d5e-41dd-a31d-4d28025def22 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:18.812191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce3036-7d0a-4d16-a945-7db43a0fcd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:18.812856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce3036-7d0a-4d16-a945-7db43a0fcd22 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:18.813171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce3036-7d0a-4d16-a945-7db43a0fcd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:18.813976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce3036-7d0a-4d16-a945-7db43a0fcd22 admin admin] PATH_INFO: `/v3/projects/60e2dc3e82534d4fa522a07d2ec07077/users/c93f19794119413f8e9873335473c208/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:18.815936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce3036-7d0a-4d16-a945-7db43a0fcd22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60e2dc3e82534d4fa522a07d2ec07077, user_id=c93f19794119413f8e9873335473c208, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:18.818596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce3036-7d0a-4d16-a945-7db43a0fcd22 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:18.829659 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2960/5989] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:18 2022] PUT /identity/v3/projects/60e2dc3e82534d4fa522a07d2ec07077/users/c93f19794119413f8e9873335473c208/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:18.839970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84784353-5a97-4e49-85c5-ab3b370f9d0b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:18.840267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84784353-5a97-4e49-85c5-ab3b370f9d0b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:18.840550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84784353-5a97-4e49-85c5-ab3b370f9d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:18.868769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-84784353-5a97-4e49-85c5-ab3b370f9d0b None None] MFA Rules not processed for user `c93f19794119413f8e9873335473c208`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:18.869940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-84784353-5a97-4e49-85c5-ab3b370f9d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:18.924088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:18.956016 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3030/5990] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:18.964375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce3036-7d0a-4d16-a945-7db43a0fcd22 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:18.966148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-10ce3036-7d0a-4d16-a945-7db43a0fcd22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:19.058131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce3036-7d0a-4d16-a945-7db43a0fcd22 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:19.062015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256793e-a5cb-4700-a6d5-2e3667338eec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:19.062662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256793e-a5cb-4700-a6d5-2e3667338eec service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:19.062950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256793e-a5cb-4700-a6d5-2e3667338eec service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:19.063256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256793e-a5cb-4700-a6d5-2e3667338eec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:19.092001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:19.125382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256793e-a5cb-4700-a6d5-2e3667338eec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:19.127364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256793e-a5cb-4700-a6d5-2e3667338eec service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:19.137977 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2961/5991] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:20.721573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84784353-5a97-4e49-85c5-ab3b370f9d0b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:20.789161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84784353-5a97-4e49-85c5-ab3b370f9d0b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:20.794151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:20.794846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:20.795086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:20.795395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] PATH_INFO: `/v3/users/490b6431459e4ea395d072892dc1c3f0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:20.796562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=490b6431459e4ea395d072892dc1c3f0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:20.799408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:20.847665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '490b6431459e4ea395d072892dc1c3f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:20.861710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '490b6431459e4ea395d072892dc1c3f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:20.869805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '490b6431459e4ea395d072892dc1c3f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:20.870442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '490b6431459e4ea395d072892dc1c3f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:20.871080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '490b6431459e4ea395d072892dc1c3f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:20.885375 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3031/5992] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:20 2022] DELETE /identity/v3/users/490b6431459e4ea395d072892dc1c3f0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:20.889983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dec41-cbac-413b-b8af-50b61f48e06f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:20.890488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dec41-cbac-413b-b8af-50b61f48e06f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:20.890671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dec41-cbac-413b-b8af-50b61f48e06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:20.911206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-305dec41-cbac-413b-b8af-50b61f48e06f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:20.912448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-305dec41-cbac-413b-b8af-50b61f48e06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:20.973771 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2962/5993] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:20.984434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:20.986433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:21.049546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f56f77-35a1-48c7-904a-708641fb5b71 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:21.053187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f3b26-853e-459f-b4dd-7b9538b8d683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:21.053800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f3b26-853e-459f-b4dd-7b9538b8d683 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:21.054084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f3b26-853e-459f-b4dd-7b9538b8d683 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:21.054363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f3b26-853e-459f-b4dd-7b9538b8d683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:21.063728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f3b26-853e-459f-b4dd-7b9538b8d683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:21.065210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f3b26-853e-459f-b4dd-7b9538b8d683 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:21.076357 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3032/5994] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:21.414059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dec41-cbac-413b-b8af-50b61f48e06f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:21.415518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-305dec41-cbac-413b-b8af-50b61f48e06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:21.428080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dec41-cbac-413b-b8af-50b61f48e06f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:21.433101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:21.433957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:21.434249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:21.434529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] PATH_INFO: `/v3/projects/f1fb30b6cd4d4012889c2d1b66c15a2b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:21.435700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1fb30b6cd4d4012889c2d1b66c15a2b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:21.437685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:21.493435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1fb30b6cd4d4012889c2d1b66c15a2b', 'request_id': 'req-1143aed6-d54d-4129-8298-359dce0cd8eb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:21.502571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1fb30b6cd4d4012889c2d1b66c15a2b', 'request_id': 'req-1143aed6-d54d-4129-8298-359dce0cd8eb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:21.509376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1fb30b6cd4d4012889c2d1b66c15a2b', 'request_id': 'req-1143aed6-d54d-4129-8298-359dce0cd8eb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:21.521196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] The token cache is being invalidate because project f1fb30b6cd4d4012889c2d1b66c15a2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:21.521490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:21.522165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:21.523698 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2963/5995] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:21 2022] DELETE /identity/v3/projects/f1fb30b6cd4d4012889c2d1b66c15a2b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:10:21.527753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f3b26-853e-459f-b4dd-7b9538b8d683 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:21.538785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f3b26-853e-459f-b4dd-7b9538b8d683 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:21.544182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:21.610596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e8fdf-91ec-4ef0-8f91-a7fa0dbfc7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:21.611267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e8fdf-91ec-4ef0-8f91-a7fa0dbfc7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:21.611549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e8fdf-91ec-4ef0-8f91-a7fa0dbfc7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:21.611824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e8fdf-91ec-4ef0-8f91-a7fa0dbfc7dd admin admin] PATH_INFO: `/v3/users/c93f19794119413f8e9873335473c208` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:21.612983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e8fdf-91ec-4ef0-8f91-a7fa0dbfc7dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93f19794119413f8e9873335473c208)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:21.615036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e8fdf-91ec-4ef0-8f91-a7fa0dbfc7dd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:21.674179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a5e8fdf-91ec-4ef0-8f91-a7fa0dbfc7dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93f19794119413f8e9873335473c208'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:21.693455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a5e8fdf-91ec-4ef0-8f91-a7fa0dbfc7dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93f19794119413f8e9873335473c208'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:21.707664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a5e8fdf-91ec-4ef0-8f91-a7fa0dbfc7dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93f19794119413f8e9873335473c208'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:21.708383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a5e8fdf-91ec-4ef0-8f91-a7fa0dbfc7dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93f19794119413f8e9873335473c208'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:21.709075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5a5e8fdf-91ec-4ef0-8f91-a7fa0dbfc7dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93f19794119413f8e9873335473c208'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:21.725201 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3033/5996] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:21 2022] DELETE /identity/v3/users/c93f19794119413f8e9873335473c208 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:10:22.013013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:22.015043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:22.034459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143aed6-d54d-4129-8298-359dce0cd8eb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:22.093403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:22.114093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:22.114850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:22.115189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:22.115537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3 admin admin] PATH_INFO: `/v3/projects/60e2dc3e82534d4fa522a07d2ec07077` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:22.116773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60e2dc3e82534d4fa522a07d2ec07077)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:22.123692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:22.190877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60e2dc3e82534d4fa522a07d2ec07077', 'request_id': 'req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:22.199860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60e2dc3e82534d4fa522a07d2ec07077', 'request_id': 'req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:22.207421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60e2dc3e82534d4fa522a07d2ec07077', 'request_id': 'req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:22.221927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3 admin admin] The token cache is being invalidate because project 60e2dc3e82534d4fa522a07d2ec07077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:22.222570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:22.223474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1669ed93-4b93-4305-ae54-ca5f8a1c51a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:22.225617 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2964/5997] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:22 2022] DELETE /identity/v3/projects/60e2dc3e82534d4fa522a07d2ec07077 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:22.237354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dba42a-e774-45c5-baed-4590a0b77d0b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:22.237740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dba42a-e774-45c5-baed-4590a0b77d0b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:22.238121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dba42a-e774-45c5-baed-4590a0b77d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:22.259692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-85dba42a-e774-45c5-baed-4590a0b77d0b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:22.261247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-85dba42a-e774-45c5-baed-4590a0b77d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:22.337330 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3034/5998] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:22.345400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab306a03-2c0d-419c-9743-e465a968ca16 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:22.345756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab306a03-2c0d-419c-9743-e465a968ca16 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:22.346146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab306a03-2c0d-419c-9743-e465a968ca16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:22.378536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ab306a03-2c0d-419c-9743-e465a968ca16 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:22.379788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ab306a03-2c0d-419c-9743-e465a968ca16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:22.389852 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2965/5999] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:22.394760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dba42a-e774-45c5-baed-4590a0b77d0b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:22.406936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dba42a-e774-45c5-baed-4590a0b77d0b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:22.413136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594f5a3-34d8-4255-9135-b40d0e545f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:22.414002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594f5a3-34d8-4255-9135-b40d0e545f91 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:22.414410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594f5a3-34d8-4255-9135-b40d0e545f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:22.414793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594f5a3-34d8-4255-9135-b40d0e545f91 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:22.415688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594f5a3-34d8-4255-9135-b40d0e545f91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:10:22.416139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594f5a3-34d8-4255-9135-b40d0e545f91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:22.418071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594f5a3-34d8-4255-9135-b40d0e545f91 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:22.446929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab306a03-2c0d-419c-9743-e465a968ca16 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:22.458760 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3035/6000] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:10:22.464277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab306a03-2c0d-419c-9743-e465a968ca16 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:22.481923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e6bdb-02b5-47fa-b7e7-a38d514efaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:22.482593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e6bdb-02b5-47fa-b7e7-a38d514efaae admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:22.482881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e6bdb-02b5-47fa-b7e7-a38d514efaae admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:22.483161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e6bdb-02b5-47fa-b7e7-a38d514efaae admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:22.483808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e6bdb-02b5-47fa-b7e7-a38d514efaae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:22.486025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e6bdb-02b5-47fa-b7e7-a38d514efaae admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:22.521118 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2966/6001] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:22 2022] POST /identity/v3/projects => generated 361 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:22.524774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594f5a3-34d8-4255-9135-b40d0e545f91 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:22.538401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594f5a3-34d8-4255-9135-b40d0e545f91 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:22.558213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:22.623551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354a195-40ab-4293-ab93-351c9c44845f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:22.624380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a195-40ab-4293-ab93-351c9c44845f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:22.624854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a195-40ab-4293-ab93-351c9c44845f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:22.625326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a195-40ab-4293-ab93-351c9c44845f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:22.626707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354a195-40ab-4293-ab93-351c9c44845f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:22.629040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354a195-40ab-4293-ab93-351c9c44845f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:22.659647 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3036/6002] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:22 2022] POST /identity/v3/users => generated 281 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:22.662790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e6bdb-02b5-47fa-b7e7-a38d514efaae admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:22.679013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e6bdb-02b5-47fa-b7e7-a38d514efaae admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:22.699923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87ef12-dd69-4a71-b722-21da28cf9556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:22.700710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87ef12-dd69-4a71-b722-21da28cf9556 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:22.701152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87ef12-dd69-4a71-b722-21da28cf9556 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:22.701535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87ef12-dd69-4a71-b722-21da28cf9556 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:22.702315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87ef12-dd69-4a71-b722-21da28cf9556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:22.704101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87ef12-dd69-4a71-b722-21da28cf9556 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:22.718863 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2967/6003] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:22.722614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354a195-40ab-4293-ab93-351c9c44845f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:22.736061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354a195-40ab-4293-ab93-351c9c44845f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:22.747034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a67975-8e49-4d36-8358-a9aec67caddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:22.747710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a67975-8e49-4d36-8358-a9aec67caddb admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:22.747963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a67975-8e49-4d36-8358-a9aec67caddb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:22.748247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a67975-8e49-4d36-8358-a9aec67caddb admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:22.748969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a67975-8e49-4d36-8358-a9aec67caddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:22.751493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a67975-8e49-4d36-8358-a9aec67caddb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:22.771597 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3037/6004] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:10:22.777508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87ef12-dd69-4a71-b722-21da28cf9556 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:22.788234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87ef12-dd69-4a71-b722-21da28cf9556 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:22.793210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb57d7b-f43f-4f1c-9496-1e46338adc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:22.793949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb57d7b-f43f-4f1c-9496-1e46338adc49 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:22.794301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb57d7b-f43f-4f1c-9496-1e46338adc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:22.794650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb57d7b-f43f-4f1c-9496-1e46338adc49 admin admin] PATH_INFO: `/v3/projects/8bb09fe634f44f119f0e394f65e5c116/users/415f14f3de354f428fcf10f9ab0cdeb2/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:22.817207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb57d7b-f43f-4f1c-9496-1e46338adc49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bb09fe634f44f119f0e394f65e5c116, user_id=415f14f3de354f428fcf10f9ab0cdeb2, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:22.819739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb57d7b-f43f-4f1c-9496-1e46338adc49 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:22.834697 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2968/6005] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:22 2022] PUT /identity/v3/projects/8bb09fe634f44f119f0e394f65e5c116/users/415f14f3de354f428fcf10f9ab0cdeb2/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:22.850441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664102c-d884-44d5-9422-890ae6eb1953 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:22.851022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664102c-d884-44d5-9422-890ae6eb1953 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:22.851301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664102c-d884-44d5-9422-890ae6eb1953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:22.885987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0664102c-d884-44d5-9422-890ae6eb1953 None None] MFA Rules not processed for user `415f14f3de354f428fcf10f9ab0cdeb2`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:22.887226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0664102c-d884-44d5-9422-890ae6eb1953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:22.976071 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3038/6006] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:23.012485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb57d7b-f43f-4f1c-9496-1e46338adc49 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:23.014390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1fb57d7b-f43f-4f1c-9496-1e46338adc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:23.097143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb57d7b-f43f-4f1c-9496-1e46338adc49 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:23.111758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2830f-1aa6-455a-aa8e-9198b1f19a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:23.112449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2830f-1aa6-455a-aa8e-9198b1f19a82 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:23.112769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2830f-1aa6-455a-aa8e-9198b1f19a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:23.113083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2830f-1aa6-455a-aa8e-9198b1f19a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:23.123031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2830f-1aa6-455a-aa8e-9198b1f19a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:23.124518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2830f-1aa6-455a-aa8e-9198b1f19a82 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:23.143668 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2969/6007] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:23.161206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664102c-d884-44d5-9422-890ae6eb1953 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:23.259310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664102c-d884-44d5-9422-890ae6eb1953 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:23.265794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:23.266640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:23.267022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:23.267434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] PATH_INFO: `/v3/users/415f14f3de354f428fcf10f9ab0cdeb2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:23.268714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] RBAC: Authorizing `identity:delete_user(user_id=415f14f3de354f428fcf10f9ab0cdeb2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:23.270804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:23.303062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:23.324665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '415f14f3de354f428fcf10f9ab0cdeb2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:23.338773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '415f14f3de354f428fcf10f9ab0cdeb2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:23.347514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '415f14f3de354f428fcf10f9ab0cdeb2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:23.348147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '415f14f3de354f428fcf10f9ab0cdeb2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:23.348762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '415f14f3de354f428fcf10f9ab0cdeb2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:23.362629 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3039/6008] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:23 2022] DELETE /identity/v3/users/415f14f3de354f428fcf10f9ab0cdeb2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:23.367173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ba55f-7cd1-4f12-b52c-91de27b168a7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:23.367555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ba55f-7cd1-4f12-b52c-91de27b168a7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:23.367904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ba55f-7cd1-4f12-b52c-91de27b168a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:23.388579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9b8ba55f-7cd1-4f12-b52c-91de27b168a7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:23.389804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9b8ba55f-7cd1-4f12-b52c-91de27b168a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:23.408761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:23.477619 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2970/6009] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:23.487115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:23.488660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:23.557171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c68cd-b8ba-49a1-85fa-a8917f9ca56d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:23.565196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f98bf5-6b42-42b7-ac37-1aec97c9b44a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:23.566381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f98bf5-6b42-42b7-ac37-1aec97c9b44a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:23.566678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f98bf5-6b42-42b7-ac37-1aec97c9b44a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:23.566964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f98bf5-6b42-42b7-ac37-1aec97c9b44a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:23.578154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f98bf5-6b42-42b7-ac37-1aec97c9b44a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:23.580182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f98bf5-6b42-42b7-ac37-1aec97c9b44a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:23.591914 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3040/6010] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:24.090737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ba55f-7cd1-4f12-b52c-91de27b168a7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:24.092278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9b8ba55f-7cd1-4f12-b52c-91de27b168a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:24.121925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ba55f-7cd1-4f12-b52c-91de27b168a7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:24.139236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:24.139810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:24.140091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:24.140366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f admin admin] PATH_INFO: `/v3/projects/8bb09fe634f44f119f0e394f65e5c116` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:24.142756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bb09fe634f44f119f0e394f65e5c116)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:24.144694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:24.201337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bb09fe634f44f119f0e394f65e5c116', 'request_id': 'req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:24.212722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bb09fe634f44f119f0e394f65e5c116', 'request_id': 'req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:24.220830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bb09fe634f44f119f0e394f65e5c116', 'request_id': 'req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:24.233097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f admin admin] The token cache is being invalidate because project 8bb09fe634f44f119f0e394f65e5c116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:24.233411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:24.234078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0dbefe6e-6cd8-44d0-81fc-20ea3f56153f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:24.236426 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2971/6011] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:24 2022] DELETE /identity/v3/projects/8bb09fe634f44f119f0e394f65e5c116 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:24.242632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9250bf-7057-41e5-94d3-f30b73a18366 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:24.244237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9250bf-7057-41e5-94d3-f30b73a18366 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:24.244536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9250bf-7057-41e5-94d3-f30b73a18366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:24.270686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1a9250bf-7057-41e5-94d3-f30b73a18366 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:24.272493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1a9250bf-7057-41e5-94d3-f30b73a18366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:24.398811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c8880-28c3-4c16-8473-d0b73943a239 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:24.399202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c8880-28c3-4c16-8473-d0b73943a239 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:24.399491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c8880-28c3-4c16-8473-d0b73943a239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:24.403756 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3041/6012] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:24.422087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-992c8880-28c3-4c16-8473-d0b73943a239 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:24.424436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-992c8880-28c3-4c16-8473-d0b73943a239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:24.437982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9250bf-7057-41e5-94d3-f30b73a18366 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:24.443393 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2972/6013] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:24.458232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9250bf-7057-41e5-94d3-f30b73a18366 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:24.464181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20580eaa-6acb-4358-9e6d-3e17dad3ab02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:24.465115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20580eaa-6acb-4358-9e6d-3e17dad3ab02 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:24.465720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20580eaa-6acb-4358-9e6d-3e17dad3ab02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:24.466185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20580eaa-6acb-4358-9e6d-3e17dad3ab02 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:24.467032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20580eaa-6acb-4358-9e6d-3e17dad3ab02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:10:24.467526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20580eaa-6acb-4358-9e6d-3e17dad3ab02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:24.469550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20580eaa-6acb-4358-9e6d-3e17dad3ab02 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:24.487936 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3042/6014] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:10:24.489964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c8880-28c3-4c16-8473-d0b73943a239 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:24.505687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c8880-28c3-4c16-8473-d0b73943a239 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:24.513349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327e3ae-7377-463b-b4e7-cbf9df271d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:24.514021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327e3ae-7377-463b-b4e7-cbf9df271d39 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:24.514329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327e3ae-7377-463b-b4e7-cbf9df271d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:24.515359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327e3ae-7377-463b-b4e7-cbf9df271d39 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:24.516362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327e3ae-7377-463b-b4e7-cbf9df271d39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:24.518162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327e3ae-7377-463b-b4e7-cbf9df271d39 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:24.552776 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2973/6015] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:24 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:24.554352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20580eaa-6acb-4358-9e6d-3e17dad3ab02 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:24.565232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20580eaa-6acb-4358-9e6d-3e17dad3ab02 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:24.614149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:24.657431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d9dc4-3dd3-40e3-84eb-5e5f9267e1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:24.658414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d9dc4-3dd3-40e3-84eb-5e5f9267e1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:24.658414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d9dc4-3dd3-40e3-84eb-5e5f9267e1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:24.658695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d9dc4-3dd3-40e3-84eb-5e5f9267e1a9 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:24.659357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d9dc4-3dd3-40e3-84eb-5e5f9267e1a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:24.661439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d9dc4-3dd3-40e3-84eb-5e5f9267e1a9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:24.693217 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3043/6016] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:24 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:24.697215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327e3ae-7377-463b-b4e7-cbf9df271d39 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:24.708992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327e3ae-7377-463b-b4e7-cbf9df271d39 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:24.718958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17599cc5-98e4-4be1-8b01-c350fbb79350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:24.720546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17599cc5-98e4-4be1-8b01-c350fbb79350 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:24.721034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17599cc5-98e4-4be1-8b01-c350fbb79350 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:24.721535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17599cc5-98e4-4be1-8b01-c350fbb79350 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:24.722348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17599cc5-98e4-4be1-8b01-c350fbb79350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:24.724577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17599cc5-98e4-4be1-8b01-c350fbb79350 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:24.737514 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2974/6017] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:24.741711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d9dc4-3dd3-40e3-84eb-5e5f9267e1a9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:24.752596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d9dc4-3dd3-40e3-84eb-5e5f9267e1a9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:24.759306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d802af-db1f-49e6-8e97-38523d5d53d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:24.764358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d802af-db1f-49e6-8e97-38523d5d53d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:24.764662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d802af-db1f-49e6-8e97-38523d5d53d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:24.764969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d802af-db1f-49e6-8e97-38523d5d53d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:24.766809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d802af-db1f-49e6-8e97-38523d5d53d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:24.768745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d802af-db1f-49e6-8e97-38523d5d53d2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:24.784460 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3044/6018] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:24.785729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17599cc5-98e4-4be1-8b01-c350fbb79350 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:24.799966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17599cc5-98e4-4be1-8b01-c350fbb79350 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:24.804912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28a364-2baa-4f85-8381-75da4fc57c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:24.805610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28a364-2baa-4f85-8381-75da4fc57c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:24.805901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28a364-2baa-4f85-8381-75da4fc57c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:24.806274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28a364-2baa-4f85-8381-75da4fc57c68 admin admin] PATH_INFO: `/v3/projects/4f9fe9a91a2542989f69fa801f7c47a2/users/2a72da83a068496da1c4c01400807ca9/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:24.824146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28a364-2baa-4f85-8381-75da4fc57c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f9fe9a91a2542989f69fa801f7c47a2, user_id=2a72da83a068496da1c4c01400807ca9, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:24.826890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28a364-2baa-4f85-8381-75da4fc57c68 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:24.853364 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2975/6019] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:24 2022] PUT /identity/v3/projects/4f9fe9a91a2542989f69fa801f7c47a2/users/2a72da83a068496da1c4c01400807ca9/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:24.855028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54acee2-9a66-4e7a-8390-c631cd138d65 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:24.855319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54acee2-9a66-4e7a-8390-c631cd138d65 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:24.856963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54acee2-9a66-4e7a-8390-c631cd138d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:24.894390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f54acee2-9a66-4e7a-8390-c631cd138d65 None None] MFA Rules not processed for user `2a72da83a068496da1c4c01400807ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:24.895641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f54acee2-9a66-4e7a-8390-c631cd138d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:24.978161 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3045/6020] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:24.979708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28a364-2baa-4f85-8381-75da4fc57c68 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:25.037218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:25.075653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28a364-2baa-4f85-8381-75da4fc57c68 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:25.085732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaedc24-c2f6-420e-8e0b-6897a30f654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:25.086670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaedc24-c2f6-420e-8e0b-6897a30f654a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:25.087052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaedc24-c2f6-420e-8e0b-6897a30f654a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:25.087457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaedc24-c2f6-420e-8e0b-6897a30f654a admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:25.088202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaedc24-c2f6-420e-8e0b-6897a30f654a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:25.090330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaedc24-c2f6-420e-8e0b-6897a30f654a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:25.125410 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2976/6021] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:24 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:10:25.127560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54acee2-9a66-4e7a-8390-c631cd138d65 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:25.142579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54acee2-9a66-4e7a-8390-c631cd138d65 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:25.223675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97146192-36d8-4114-8955-fba5286e65cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:25.224381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97146192-36d8-4114-8955-fba5286e65cc admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:25.224673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97146192-36d8-4114-8955-fba5286e65cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:25.225063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97146192-36d8-4114-8955-fba5286e65cc admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:25.226121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97146192-36d8-4114-8955-fba5286e65cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:25.228082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97146192-36d8-4114-8955-fba5286e65cc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:25.261844 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3046/6022] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:25 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:10:25.266106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaedc24-c2f6-420e-8e0b-6897a30f654a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:25.279834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaedc24-c2f6-420e-8e0b-6897a30f654a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:25.285496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2989a2-8014-4541-a1ce-11edc4b1921a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:25.286184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2989a2-8014-4541-a1ce-11edc4b1921a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:25.286473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2989a2-8014-4541-a1ce-11edc4b1921a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:25.286749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2989a2-8014-4541-a1ce-11edc4b1921a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:25.287401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2989a2-8014-4541-a1ce-11edc4b1921a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:25.289257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2989a2-8014-4541-a1ce-11edc4b1921a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:25.301454 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2977/6023] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:25.305792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97146192-36d8-4114-8955-fba5286e65cc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:25.318810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97146192-36d8-4114-8955-fba5286e65cc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:25.331613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feae82-b3ef-4e20-995b-566a2a13bf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:25.333167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feae82-b3ef-4e20-995b-566a2a13bf97 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:25.333351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feae82-b3ef-4e20-995b-566a2a13bf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:25.333644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feae82-b3ef-4e20-995b-566a2a13bf97 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:25.334318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6feae82-b3ef-4e20-995b-566a2a13bf97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:25.336482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6feae82-b3ef-4e20-995b-566a2a13bf97 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:25.338636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2989a2-8014-4541-a1ce-11edc4b1921a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:25.340391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5b2989a2-8014-4541-a1ce-11edc4b1921a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:25.355335 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3047/6024] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:25.359279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feae82-b3ef-4e20-995b-566a2a13bf97 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:25.372447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feae82-b3ef-4e20-995b-566a2a13bf97 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:25.378520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d46ce-30e5-4100-83c5-09df5d9e1894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:25.379337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d46ce-30e5-4100-83c5-09df5d9e1894 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:25.379694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d46ce-30e5-4100-83c5-09df5d9e1894 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:25.380049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d46ce-30e5-4100-83c5-09df5d9e1894 admin admin] PATH_INFO: `/v3/projects/fc19e848a9834dfe9a5935ba19038b7f/users/64183d6f717349f9bd8de315ba22c02c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:25.399615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d46ce-30e5-4100-83c5-09df5d9e1894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc19e848a9834dfe9a5935ba19038b7f, user_id=64183d6f717349f9bd8de315ba22c02c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:25.402957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d46ce-30e5-4100-83c5-09df5d9e1894 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:25.410010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:25.420971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2989a2-8014-4541-a1ce-11edc4b1921a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:25.427474 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3048/6025] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:25 2022] PUT /identity/v3/projects/fc19e848a9834dfe9a5935ba19038b7f/users/64183d6f717349f9bd8de315ba22c02c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:10:25.440225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595b065-2dc5-4dd2-b4a9-16b044b47850 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:25.440520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595b065-2dc5-4dd2-b4a9-16b044b47850 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:25.442156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595b065-2dc5-4dd2-b4a9-16b044b47850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:25.480761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7595b065-2dc5-4dd2-b4a9-16b044b47850 None None] MFA Rules not processed for user `64183d6f717349f9bd8de315ba22c02c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:25.482139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7595b065-2dc5-4dd2-b4a9-16b044b47850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:25.512556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131f6f1-21da-46ed-9e9a-aefaf42179d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:25.514127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131f6f1-21da-46ed-9e9a-aefaf42179d0 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:25.514612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131f6f1-21da-46ed-9e9a-aefaf42179d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:25.515054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131f6f1-21da-46ed-9e9a-aefaf42179d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:25.518799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a131f6f1-21da-46ed-9e9a-aefaf42179d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:25.559918 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3049/6026] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:25.564930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595b065-2dc5-4dd2-b4a9-16b044b47850 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:25.615535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131f6f1-21da-46ed-9e9a-aefaf42179d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:25.617400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131f6f1-21da-46ed-9e9a-aefaf42179d0 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:25.641111 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2978/6027] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:25.656638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595b065-2dc5-4dd2-b4a9-16b044b47850 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:25.661698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cb5b7-2e8c-4a79-a043-d5ee44f007a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:25.662409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cb5b7-2e8c-4a79-a043-d5ee44f007a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:25.662773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cb5b7-2e8c-4a79-a043-d5ee44f007a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:25.663126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cb5b7-2e8c-4a79-a043-d5ee44f007a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:25.663965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cb5b7-2e8c-4a79-a043-d5ee44f007a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:25.666103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cb5b7-2e8c-4a79-a043-d5ee44f007a9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:25.708939 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3050/6028] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:25 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:25.711814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131f6f1-21da-46ed-9e9a-aefaf42179d0 service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:25.723591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131f6f1-21da-46ed-9e9a-aefaf42179d0 service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:25.736106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:25.786702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a3943-2c16-4b33-9403-2f58f289e477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:25.787327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a3943-2c16-4b33-9403-2f58f289e477 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:25.787616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a3943-2c16-4b33-9403-2f58f289e477 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:25.787899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a3943-2c16-4b33-9403-2f58f289e477 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:25.788573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a3943-2c16-4b33-9403-2f58f289e477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:25.790541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a3943-2c16-4b33-9403-2f58f289e477 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:25.812655 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2979/6029] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:25 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:10:25.817333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cb5b7-2e8c-4a79-a043-d5ee44f007a9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:25.827817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cb5b7-2e8c-4a79-a043-d5ee44f007a9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:25.832610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba2cb1-3662-4966-b65a-88da322e8dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:25.833254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba2cb1-3662-4966-b65a-88da322e8dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:25.833543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba2cb1-3662-4966-b65a-88da322e8dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:25.833823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba2cb1-3662-4966-b65a-88da322e8dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:25.834471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba2cb1-3662-4966-b65a-88da322e8dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:25.836092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba2cb1-3662-4966-b65a-88da322e8dd9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:25.847408 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3051/6030] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:25.851575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a3943-2c16-4b33-9403-2f58f289e477 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:25.862559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a3943-2c16-4b33-9403-2f58f289e477 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:25.867332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd425d-2555-4123-afa1-ce2fd04bb50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:25.867993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd425d-2555-4123-afa1-ce2fd04bb50b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:25.868278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd425d-2555-4123-afa1-ce2fd04bb50b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:25.868554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd425d-2555-4123-afa1-ce2fd04bb50b admin admin] PATH_INFO: `/v3/projects/89b4db4f7e3545689b25d862880f8c14/users/b30d291b783e4fb3b0b501b20172b0ca/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:25.884607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd425d-2555-4123-afa1-ce2fd04bb50b admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b4db4f7e3545689b25d862880f8c14, user_id=b30d291b783e4fb3b0b501b20172b0ca, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:25.887375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd425d-2555-4123-afa1-ce2fd04bb50b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:25.901524 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2980/6031] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:25 2022] PUT /identity/v3/projects/89b4db4f7e3545689b25d862880f8c14/users/b30d291b783e4fb3b0b501b20172b0ca/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:25.905418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba2cb1-3662-4966-b65a-88da322e8dd9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:25.915858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba2cb1-3662-4966-b65a-88da322e8dd9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:25.976507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac822439-e3b1-464f-9b18-ccc9b423bc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:25.977197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac822439-e3b1-464f-9b18-ccc9b423bc84 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:25.977551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac822439-e3b1-464f-9b18-ccc9b423bc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:25.977902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac822439-e3b1-464f-9b18-ccc9b423bc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:25.978626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac822439-e3b1-464f-9b18-ccc9b423bc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:25.980319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac822439-e3b1-464f-9b18-ccc9b423bc84 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:25.981789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:25.992608 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3052/6032] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:10:25.996637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd425d-2555-4123-afa1-ce2fd04bb50b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:26.007874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd425d-2555-4123-afa1-ce2fd04bb50b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:26.012618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7672c74-981e-4575-9025-e8f58b5dba17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:26.013318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7672c74-981e-4575-9025-e8f58b5dba17 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:26.013609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7672c74-981e-4575-9025-e8f58b5dba17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:26.013893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7672c74-981e-4575-9025-e8f58b5dba17 admin admin] PATH_INFO: `/v3/projects/89b4db4f7e3545689b25d862880f8c14/users/b30d291b783e4fb3b0b501b20172b0ca/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:26.015850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7672c74-981e-4575-9025-e8f58b5dba17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b4db4f7e3545689b25d862880f8c14, user_id=b30d291b783e4fb3b0b501b20172b0ca, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:26.018640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7672c74-981e-4575-9025-e8f58b5dba17 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:26.040891 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2981/6033] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:25 2022] PUT /identity/v3/projects/89b4db4f7e3545689b25d862880f8c14/users/b30d291b783e4fb3b0b501b20172b0ca/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:26.043840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87193d57-0f19-4fcb-b236-4711cafab109 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:26.044139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87193d57-0f19-4fcb-b236-4711cafab109 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:26.044437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87193d57-0f19-4fcb-b236-4711cafab109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:26.079663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-87193d57-0f19-4fcb-b236-4711cafab109 None None] MFA Rules not processed for user `b30d291b783e4fb3b0b501b20172b0ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:26.080938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-87193d57-0f19-4fcb-b236-4711cafab109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:26.178039 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3053/6034] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:26.189422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7672c74-981e-4575-9025-e8f58b5dba17 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:26.191874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e7672c74-981e-4575-9025-e8f58b5dba17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:26.264624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7672c74-981e-4575-9025-e8f58b5dba17 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:26.268505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacc8eb-b66c-4acb-ad9d-88c316650671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:26.269232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacc8eb-b66c-4acb-ad9d-88c316650671 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:26.269804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacc8eb-b66c-4acb-ad9d-88c316650671 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:26.269933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacc8eb-b66c-4acb-ad9d-88c316650671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:26.383282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffacc8eb-b66c-4acb-ad9d-88c316650671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:26.384786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffacc8eb-b66c-4acb-ad9d-88c316650671 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:26.398094 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2982/6035] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:29.761103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87193d57-0f19-4fcb-b236-4711cafab109 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:29.841602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87193d57-0f19-4fcb-b236-4711cafab109 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:29.849058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:29.850080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:29.850407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:29.851335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] PATH_INFO: `/v3/users/2a72da83a068496da1c4c01400807ca9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:29.852697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a72da83a068496da1c4c01400807ca9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:29.855025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:29.918342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a72da83a068496da1c4c01400807ca9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:29.937961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a72da83a068496da1c4c01400807ca9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:29.946511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a72da83a068496da1c4c01400807ca9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:29.947234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a72da83a068496da1c4c01400807ca9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:29.948561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a72da83a068496da1c4c01400807ca9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:29.958996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:29.964801 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3054/6036] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:29 2022] DELETE /identity/v3/users/2a72da83a068496da1c4c01400807ca9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:29.971467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f101e43-857f-4337-a1d5-47fca12f0aa7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:29.974177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f101e43-857f-4337-a1d5-47fca12f0aa7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:29.974640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f101e43-857f-4337-a1d5-47fca12f0aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:29.998894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0f101e43-857f-4337-a1d5-47fca12f0aa7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:30.000168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0f101e43-857f-4337-a1d5-47fca12f0aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:30.020583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:30.095253 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2983/6037] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:30.105467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:30.106873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:30.179806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b300c76-6c05-47be-a706-a264de545c2e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:30.184183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9841e-bf51-4e99-9bfb-df845aa5b340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:30.185092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9841e-bf51-4e99-9bfb-df845aa5b340 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:30.185561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9841e-bf51-4e99-9bfb-df845aa5b340 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:30.185841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9841e-bf51-4e99-9bfb-df845aa5b340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:30.195334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9841e-bf51-4e99-9bfb-df845aa5b340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:30.196780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9841e-bf51-4e99-9bfb-df845aa5b340 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:30.207038 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3055/6038] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:30.499905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f101e43-857f-4337-a1d5-47fca12f0aa7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:30.502636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0f101e43-857f-4337-a1d5-47fca12f0aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:30.518966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f101e43-857f-4337-a1d5-47fca12f0aa7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:30.524640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:30.525485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:30.525960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:30.526822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] PATH_INFO: `/v3/projects/4f9fe9a91a2542989f69fa801f7c47a2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:30.527645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f9fe9a91a2542989f69fa801f7c47a2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:30.529827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:30.592239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f9fe9a91a2542989f69fa801f7c47a2', 'request_id': 'req-8ea34599-e2cc-40c1-9b08-0fe17228a493'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:30.600362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f9fe9a91a2542989f69fa801f7c47a2', 'request_id': 'req-8ea34599-e2cc-40c1-9b08-0fe17228a493'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:30.607440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f9fe9a91a2542989f69fa801f7c47a2', 'request_id': 'req-8ea34599-e2cc-40c1-9b08-0fe17228a493'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:30.619562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] The token cache is being invalidate because project 4f9fe9a91a2542989f69fa801f7c47a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:30.619853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:30.620528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:30.622194 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2984/6039] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:30 2022] DELETE /identity/v3/projects/4f9fe9a91a2542989f69fa801f7c47a2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:30.626732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9841e-bf51-4e99-9bfb-df845aa5b340 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:30.636597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9841e-bf51-4e99-9bfb-df845aa5b340 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:30.696254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:30.697095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:30.697388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:30.697677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] PATH_INFO: `/v3/users/64183d6f717349f9bd8de315ba22c02c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:30.698887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64183d6f717349f9bd8de315ba22c02c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:30.700814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:30.748513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64183d6f717349f9bd8de315ba22c02c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:30.762647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64183d6f717349f9bd8de315ba22c02c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:30.770493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64183d6f717349f9bd8de315ba22c02c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:30.771076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64183d6f717349f9bd8de315ba22c02c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:30.771696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64183d6f717349f9bd8de315ba22c02c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:30.784956 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3056/6040] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:30 2022] DELETE /identity/v3/users/64183d6f717349f9bd8de315ba22c02c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:31.397251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:31.398728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:31.414473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea34599-e2cc-40c1-9b08-0fe17228a493 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:31.490560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:31.491329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:31.491620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:31.491900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] PATH_INFO: `/v3/projects/fc19e848a9834dfe9a5935ba19038b7f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:31.493072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc19e848a9834dfe9a5935ba19038b7f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:31.495312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:31.542932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:31.544340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc19e848a9834dfe9a5935ba19038b7f', 'request_id': 'req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:31.552696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc19e848a9834dfe9a5935ba19038b7f', 'request_id': 'req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:31.559314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc19e848a9834dfe9a5935ba19038b7f', 'request_id': 'req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:31.569852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] The token cache is being invalidate because project fc19e848a9834dfe9a5935ba19038b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:31.570146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:31.570901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:31.572391 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2985/6041] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:31 2022] DELETE /identity/v3/projects/fc19e848a9834dfe9a5935ba19038b7f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:31.576440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:31.577888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:31.591230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa7cae-fd89-40c9-9319-e373d8de9cb5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:31.611406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:31.653692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90cce10-633c-4f51-8f9e-9ed823628bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:31.654437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cce10-633c-4f51-8f9e-9ed823628bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:31.654884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cce10-633c-4f51-8f9e-9ed823628bda admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:31.655166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cce10-633c-4f51-8f9e-9ed823628bda admin admin] PATH_INFO: `/v3/users/b30d291b783e4fb3b0b501b20172b0ca` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:31.656332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90cce10-633c-4f51-8f9e-9ed823628bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30d291b783e4fb3b0b501b20172b0ca)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:31.659002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90cce10-633c-4f51-8f9e-9ed823628bda admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:31.706213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b90cce10-633c-4f51-8f9e-9ed823628bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30d291b783e4fb3b0b501b20172b0ca'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:31.718611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b90cce10-633c-4f51-8f9e-9ed823628bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30d291b783e4fb3b0b501b20172b0ca'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:31.726412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b90cce10-633c-4f51-8f9e-9ed823628bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30d291b783e4fb3b0b501b20172b0ca'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:31.727191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b90cce10-633c-4f51-8f9e-9ed823628bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30d291b783e4fb3b0b501b20172b0ca'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:31.727892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b90cce10-633c-4f51-8f9e-9ed823628bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30d291b783e4fb3b0b501b20172b0ca'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:31.741019 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3057/6042] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:31 2022] DELETE /identity/v3/users/b30d291b783e4fb3b0b501b20172b0ca => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:32.026083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:32.028229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:32.043173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bac851-4ad1-4124-8f8c-07268fa7f3ce admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:32.077468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526daf9f-2073-479e-a612-4f994d665225 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:32.077962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526daf9f-2073-479e-a612-4f994d665225 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:32.078053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526daf9f-2073-479e-a612-4f994d665225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:32.099113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-526daf9f-2073-479e-a612-4f994d665225 None None] MFA Rules not processed for user `38dddfbdc4f0450e8d5fc7b214796758`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:32.100254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-526daf9f-2073-479e-a612-4f994d665225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:32.111073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:32.112342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:32.112629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:32.113048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc admin admin] PATH_INFO: `/v3/projects/89b4db4f7e3545689b25d862880f8c14` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:32.114244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b4db4f7e3545689b25d862880f8c14)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:32.117718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:32.184963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b4db4f7e3545689b25d862880f8c14', 'request_id': 'req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:32.187378 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3058/6043] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:32.194723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b4db4f7e3545689b25d862880f8c14', 'request_id': 'req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:32.201924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b4db4f7e3545689b25d862880f8c14', 'request_id': 'req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:32.213957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc admin admin] The token cache is being invalidate because project 89b4db4f7e3545689b25d862880f8c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:32.214347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:32.215229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-05c425b5-dfcf-48de-9b8d-b5f9ab696afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:32.220951 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2986/6044] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:32 2022] DELETE /identity/v3/projects/89b4db4f7e3545689b25d862880f8c14 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:32.223928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f85eb5-89d9-4bb2-a3a0-62ceabf6c818 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:32.224441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f85eb5-89d9-4bb2-a3a0-62ceabf6c818 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:32.224786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f85eb5-89d9-4bb2-a3a0-62ceabf6c818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:32.253802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-49f85eb5-89d9-4bb2-a3a0-62ceabf6c818 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:32.254818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-49f85eb5-89d9-4bb2-a3a0-62ceabf6c818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:32.318338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:32.345695 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3059/6045] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:32.351662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0901-cd8b-4325-8efe-9322e7213a6d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:32.351875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0901-cd8b-4325-8efe-9322e7213a6d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:32.352155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0901-cd8b-4325-8efe-9322e7213a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:32.375380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3a9e0901-cd8b-4325-8efe-9322e7213a6d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:32.377543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3a9e0901-cd8b-4325-8efe-9322e7213a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:32.385844 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2987/6046] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:32.390159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f85eb5-89d9-4bb2-a3a0-62ceabf6c818 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:32.404092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f85eb5-89d9-4bb2-a3a0-62ceabf6c818 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:32.416074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5834b-92cb-4c06-b904-73d308d305d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:32.417679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5834b-92cb-4c06-b904-73d308d305d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:32.418023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5834b-92cb-4c06-b904-73d308d305d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:32.418929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5834b-92cb-4c06-b904-73d308d305d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:32.419645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5834b-92cb-4c06-b904-73d308d305d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:10:32.419966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5834b-92cb-4c06-b904-73d308d305d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:32.423007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5834b-92cb-4c06-b904-73d308d305d7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:32.441043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e0901-cd8b-4325-8efe-9322e7213a6d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:32.452083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e0901-cd8b-4325-8efe-9322e7213a6d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:32.453057 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3060/6047] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:10:32.457784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da3445-23c3-45af-8c14-8047378c86fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:32.465080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da3445-23c3-45af-8c14-8047378c86fa admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:32.465634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da3445-23c3-45af-8c14-8047378c86fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:32.466104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da3445-23c3-45af-8c14-8047378c86fa admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:32.466969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da3445-23c3-45af-8c14-8047378c86fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:32.470303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da3445-23c3-45af-8c14-8047378c86fa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:32.529583 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2988/6048] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:32 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:10:32.533581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5834b-92cb-4c06-b904-73d308d305d7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:32.552805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5834b-92cb-4c06-b904-73d308d305d7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:32.652526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c2a66-e958-45ce-9b71-b3fa39c4b5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:32.652747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c2a66-e958-45ce-9b71-b3fa39c4b5bc admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:32.652950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c2a66-e958-45ce-9b71-b3fa39c4b5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:32.653383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c2a66-e958-45ce-9b71-b3fa39c4b5bc admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:32.654119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c2a66-e958-45ce-9b71-b3fa39c4b5bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:32.656447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c2a66-e958-45ce-9b71-b3fa39c4b5bc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:32.693732 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3061/6049] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:32 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:32.697982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da3445-23c3-45af-8c14-8047378c86fa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:32.709790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da3445-23c3-45af-8c14-8047378c86fa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:32.714692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955f139-22cc-4cbb-836c-c90876a7cd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:32.715468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955f139-22cc-4cbb-836c-c90876a7cd06 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:32.715911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955f139-22cc-4cbb-836c-c90876a7cd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:32.716216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955f139-22cc-4cbb-836c-c90876a7cd06 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:32.717025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955f139-22cc-4cbb-836c-c90876a7cd06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:32.718778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955f139-22cc-4cbb-836c-c90876a7cd06 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:32.732718 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2989/6050] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:32.735275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c2a66-e958-45ce-9b71-b3fa39c4b5bc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:32.746295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c2a66-e958-45ce-9b71-b3fa39c4b5bc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:32.752363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eef09-608a-42b6-92f8-5aafce27882a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:32.753567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eef09-608a-42b6-92f8-5aafce27882a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:32.753858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eef09-608a-42b6-92f8-5aafce27882a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:32.754138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eef09-608a-42b6-92f8-5aafce27882a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:32.754792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673eef09-608a-42b6-92f8-5aafce27882a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:32.756408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673eef09-608a-42b6-92f8-5aafce27882a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:32.772198 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3062/6051] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:32.773468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955f139-22cc-4cbb-836c-c90876a7cd06 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:32.784929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955f139-22cc-4cbb-836c-c90876a7cd06 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:32.791472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a2a8e-017e-46fc-b187-46d967349a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:32.792829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a2a8e-017e-46fc-b187-46d967349a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:32.793179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a2a8e-017e-46fc-b187-46d967349a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:32.793449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a2a8e-017e-46fc-b187-46d967349a0c admin admin] PATH_INFO: `/v3/projects/03a487178f754f4fb76046eae2de5598/users/675162cb4820496cb7471bdabd8cc4eb/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:32.825154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a2a8e-017e-46fc-b187-46d967349a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a487178f754f4fb76046eae2de5598, user_id=675162cb4820496cb7471bdabd8cc4eb, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:32.828138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a2a8e-017e-46fc-b187-46d967349a0c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:32.843673 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2990/6052] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:32 2022] PUT /identity/v3/projects/03a487178f754f4fb76046eae2de5598/users/675162cb4820496cb7471bdabd8cc4eb/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:32.855573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01175402-e475-4ecb-b907-5584bbec45cc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:32.855805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01175402-e475-4ecb-b907-5584bbec45cc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:32.856087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01175402-e475-4ecb-b907-5584bbec45cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:32.888199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-01175402-e475-4ecb-b907-5584bbec45cc None None] MFA Rules not processed for user `675162cb4820496cb7471bdabd8cc4eb`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:32.890074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-01175402-e475-4ecb-b907-5584bbec45cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:33.002870 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3063/6053] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:33.013783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a2a8e-017e-46fc-b187-46d967349a0c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:33.017822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-438a2a8e-017e-46fc-b187-46d967349a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:33.027064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:33.108816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a2a8e-017e-46fc-b187-46d967349a0c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:33.113056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a03400-bcfb-4c63-b033-3f20064a8129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:33.113821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a03400-bcfb-4c63-b033-3f20064a8129 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:33.114116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a03400-bcfb-4c63-b033-3f20064a8129 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:33.114390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a03400-bcfb-4c63-b033-3f20064a8129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:33.127847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a03400-bcfb-4c63-b033-3f20064a8129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:33.129357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a03400-bcfb-4c63-b033-3f20064a8129 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:33.141124 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2991/6054] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:10:34.786904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01175402-e475-4ecb-b907-5584bbec45cc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:34.850252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:34.876000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01175402-e475-4ecb-b907-5584bbec45cc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:34.881536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:34.882723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:34.883043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:34.883373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] PATH_INFO: `/v3/users/675162cb4820496cb7471bdabd8cc4eb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:34.885894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=675162cb4820496cb7471bdabd8cc4eb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:34.887978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:34.947165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '675162cb4820496cb7471bdabd8cc4eb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:34.979823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '675162cb4820496cb7471bdabd8cc4eb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:34.992571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '675162cb4820496cb7471bdabd8cc4eb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:34.993730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '675162cb4820496cb7471bdabd8cc4eb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:34.995889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '675162cb4820496cb7471bdabd8cc4eb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:35.020119 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3064/6055] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:34 2022] DELETE /identity/v3/users/675162cb4820496cb7471bdabd8cc4eb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:35.024611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6146c5a-4c37-48d7-ada9-9a1a55afb484 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:35.026186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6146c5a-4c37-48d7-ada9-9a1a55afb484 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:35.026480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6146c5a-4c37-48d7-ada9-9a1a55afb484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:35.060865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b6146c5a-4c37-48d7-ada9-9a1a55afb484 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:35.062141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b6146c5a-4c37-48d7-ada9-9a1a55afb484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:35.160457 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2992/6056] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:35.168660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:35.170235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:35.262395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5384a-2346-40ae-bc8d-a64cadf2b2b1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:35.268211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3d8f6-7a7f-4ee0-9074-eb356d13d688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:35.268831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3d8f6-7a7f-4ee0-9074-eb356d13d688 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:35.269192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3d8f6-7a7f-4ee0-9074-eb356d13d688 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:35.269473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3d8f6-7a7f-4ee0-9074-eb356d13d688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:35.289573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3d8f6-7a7f-4ee0-9074-eb356d13d688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:35.291427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3d8f6-7a7f-4ee0-9074-eb356d13d688 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:35.307676 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3065/6057] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:36.056464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6146c5a-4c37-48d7-ada9-9a1a55afb484 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:36.058070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b6146c5a-4c37-48d7-ada9-9a1a55afb484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:36.089318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6146c5a-4c37-48d7-ada9-9a1a55afb484 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:36.104372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c8268-d5b6-4032-85a2-70166291ae9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:36.105275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c8268-d5b6-4032-85a2-70166291ae9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:36.105735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c8268-d5b6-4032-85a2-70166291ae9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:36.106255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c8268-d5b6-4032-85a2-70166291ae9a admin admin] PATH_INFO: `/v3/projects/03a487178f754f4fb76046eae2de5598` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:36.108082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c8268-d5b6-4032-85a2-70166291ae9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=03a487178f754f4fb76046eae2de5598)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:36.110867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c8268-d5b6-4032-85a2-70166291ae9a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:36.187186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-015c8268-d5b6-4032-85a2-70166291ae9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03a487178f754f4fb76046eae2de5598', 'request_id': 'req-015c8268-d5b6-4032-85a2-70166291ae9a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:36.196317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-015c8268-d5b6-4032-85a2-70166291ae9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03a487178f754f4fb76046eae2de5598', 'request_id': 'req-015c8268-d5b6-4032-85a2-70166291ae9a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:36.204294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-015c8268-d5b6-4032-85a2-70166291ae9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03a487178f754f4fb76046eae2de5598', 'request_id': 'req-015c8268-d5b6-4032-85a2-70166291ae9a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:36.221540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-015c8268-d5b6-4032-85a2-70166291ae9a admin admin] The token cache is being invalidate because project 03a487178f754f4fb76046eae2de5598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:36.222130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-015c8268-d5b6-4032-85a2-70166291ae9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:36.223120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-015c8268-d5b6-4032-85a2-70166291ae9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:36.226312 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2993/6058] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:36 2022] DELETE /identity/v3/projects/03a487178f754f4fb76046eae2de5598 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:36.233785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc4242-82d7-4524-82ff-f18b8c1f91ba None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:36.234321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc4242-82d7-4524-82ff-f18b8c1f91ba None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:36.234321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc4242-82d7-4524-82ff-f18b8c1f91ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:36.271668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ecfc4242-82d7-4524-82ff-f18b8c1f91ba None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:36.274550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ecfc4242-82d7-4524-82ff-f18b8c1f91ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:36.357272 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3066/6059] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:36.365320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b542f0-81c6-4287-90d2-64dd0f6cfeab None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:36.365621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b542f0-81c6-4287-90d2-64dd0f6cfeab None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:36.365907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b542f0-81c6-4287-90d2-64dd0f6cfeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:36.386970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-95b542f0-81c6-4287-90d2-64dd0f6cfeab None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:36.388355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-95b542f0-81c6-4287-90d2-64dd0f6cfeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:36.405427 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2994/6060] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:36.408328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc4242-82d7-4524-82ff-f18b8c1f91ba None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:36.414541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:36.420500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc4242-82d7-4524-82ff-f18b8c1f91ba None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:36.426378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c637ad8-b105-4f4e-bbd0-9a56e7f8d62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:36.427059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c637ad8-b105-4f4e-bbd0-9a56e7f8d62f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:36.427350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c637ad8-b105-4f4e-bbd0-9a56e7f8d62f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:36.427679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c637ad8-b105-4f4e-bbd0-9a56e7f8d62f admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:36.428341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c637ad8-b105-4f4e-bbd0-9a56e7f8d62f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:10:36.428652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c637ad8-b105-4f4e-bbd0-9a56e7f8d62f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:36.430566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c637ad8-b105-4f4e-bbd0-9a56e7f8d62f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:36.450496 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3067/6061] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:10:36.451569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b542f0-81c6-4287-90d2-64dd0f6cfeab None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:36.466627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b542f0-81c6-4287-90d2-64dd0f6cfeab None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:36.472584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1df0d-4ccc-4c14-a73d-afdffc047300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:36.473249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1df0d-4ccc-4c14-a73d-afdffc047300 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:36.473533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1df0d-4ccc-4c14-a73d-afdffc047300 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:36.473809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1df0d-4ccc-4c14-a73d-afdffc047300 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:36.474459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1df0d-4ccc-4c14-a73d-afdffc047300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:36.476692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1df0d-4ccc-4c14-a73d-afdffc047300 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:36.511318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:36.520504 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2995/6062] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:36 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:10:36.524201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c637ad8-b105-4f4e-bbd0-9a56e7f8d62f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:36.534561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c637ad8-b105-4f4e-bbd0-9a56e7f8d62f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:36.603996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c69844-aedf-4949-8cdd-9d734224d153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:36.606112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c69844-aedf-4949-8cdd-9d734224d153 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:36.606439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c69844-aedf-4949-8cdd-9d734224d153 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:36.606882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c69844-aedf-4949-8cdd-9d734224d153 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:36.607741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c69844-aedf-4949-8cdd-9d734224d153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:36.609838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c69844-aedf-4949-8cdd-9d734224d153 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:36.639038 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3068/6063] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:36 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:36.643382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1df0d-4ccc-4c14-a73d-afdffc047300 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:36.656889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1df0d-4ccc-4c14-a73d-afdffc047300 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:36.666079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f09b0-bbaa-475a-9f3d-533d933ae030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:36.666932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f09b0-bbaa-475a-9f3d-533d933ae030 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:36.667470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f09b0-bbaa-475a-9f3d-533d933ae030 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:36.667953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f09b0-bbaa-475a-9f3d-533d933ae030 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:36.668890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f09b0-bbaa-475a-9f3d-533d933ae030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:36.671036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f09b0-bbaa-475a-9f3d-533d933ae030 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:36.691048 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2996/6064] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:10:36.692363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c69844-aedf-4949-8cdd-9d734224d153 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:36.708733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c69844-aedf-4949-8cdd-9d734224d153 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:36.714788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c6465-4784-468d-9667-90ba2c23ffab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:36.715540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c6465-4784-468d-9667-90ba2c23ffab admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:36.715871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c6465-4784-468d-9667-90ba2c23ffab admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:36.716395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c6465-4784-468d-9667-90ba2c23ffab admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:36.717211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c6465-4784-468d-9667-90ba2c23ffab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:36.718890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c6465-4784-468d-9667-90ba2c23ffab admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:36.734446 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3069/6065] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:36.738077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f09b0-bbaa-475a-9f3d-533d933ae030 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:36.752842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f09b0-bbaa-475a-9f3d-533d933ae030 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:36.767878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05031ba-0429-4454-ac65-0703963acb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:36.769075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05031ba-0429-4454-ac65-0703963acb94 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:36.769335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05031ba-0429-4454-ac65-0703963acb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:36.769630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05031ba-0429-4454-ac65-0703963acb94 admin admin] PATH_INFO: `/v3/projects/e203df4336854849801de4d6ae8605ce/users/fa4c59b32e6b4b99855ba2617954400f/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:36.792716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05031ba-0429-4454-ac65-0703963acb94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e203df4336854849801de4d6ae8605ce, user_id=fa4c59b32e6b4b99855ba2617954400f, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:36.795786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05031ba-0429-4454-ac65-0703963acb94 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:36.819862 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2997/6066] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:36 2022] PUT /identity/v3/projects/e203df4336854849801de4d6ae8605ce/users/fa4c59b32e6b4b99855ba2617954400f/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:36.832081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b7a6e-fb2b-4e1f-9aea-052c853facec None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:36.832396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b7a6e-fb2b-4e1f-9aea-052c853facec None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:36.832702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b7a6e-fb2b-4e1f-9aea-052c853facec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:36.864479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b20b7a6e-fb2b-4e1f-9aea-052c853facec None None] MFA Rules not processed for user `fa4c59b32e6b4b99855ba2617954400f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:36.865764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b20b7a6e-fb2b-4e1f-9aea-052c853facec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:36.936427 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3070/6067] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:36.948557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05031ba-0429-4454-ac65-0703963acb94 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:36.950446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e05031ba-0429-4454-ac65-0703963acb94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:37.016855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05031ba-0429-4454-ac65-0703963acb94 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:37.023351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e223ea2-bd8e-4c62-94c2-9fa50c31ac0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:37.023896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e223ea2-bd8e-4c62-94c2-9fa50c31ac0a service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:37.024139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e223ea2-bd8e-4c62-94c2-9fa50c31ac0a service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:37.024441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e223ea2-bd8e-4c62-94c2-9fa50c31ac0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:37.034540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e223ea2-bd8e-4c62-94c2-9fa50c31ac0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:37.036077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e223ea2-bd8e-4c62-94c2-9fa50c31ac0a service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:37.046021 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2998/6068] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:38.551103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b7a6e-fb2b-4e1f-9aea-052c853facec None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:38.625075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b7a6e-fb2b-4e1f-9aea-052c853facec None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:38.630252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:38.630895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:38.631181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:38.631464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] PATH_INFO: `/v3/users/fa4c59b32e6b4b99855ba2617954400f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:38.632582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa4c59b32e6b4b99855ba2617954400f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:38.634549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:38.636100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:38.685103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa4c59b32e6b4b99855ba2617954400f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:38.704265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa4c59b32e6b4b99855ba2617954400f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:38.713355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa4c59b32e6b4b99855ba2617954400f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:38.713981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa4c59b32e6b4b99855ba2617954400f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:38.714597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa4c59b32e6b4b99855ba2617954400f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:38.729728 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3071/6069] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:38 2022] DELETE /identity/v3/users/fa4c59b32e6b4b99855ba2617954400f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:38.734223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d172a2-1831-49b4-8971-86e782488742 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:38.734685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d172a2-1831-49b4-8971-86e782488742 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:38.735316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d172a2-1831-49b4-8971-86e782488742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:38.755685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e8d172a2-1831-49b4-8971-86e782488742 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:38.756917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e8d172a2-1831-49b4-8971-86e782488742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:38.818357 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 2999/6070] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:38.828048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:38.829575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:38.899255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21291d60-9ac5-4035-91d2-ce42e8161dc5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:38.902543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f6c98-8788-428f-88c7-e64f1dfc1311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:38.903176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f6c98-8788-428f-88c7-e64f1dfc1311 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:38.903463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f6c98-8788-428f-88c7-e64f1dfc1311 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:38.903741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f6c98-8788-428f-88c7-e64f1dfc1311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:38.914438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f6c98-8788-428f-88c7-e64f1dfc1311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:38.915918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f6c98-8788-428f-88c7-e64f1dfc1311 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:38.927393 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3072/6071] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:39.595625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d172a2-1831-49b4-8971-86e782488742 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:39.597183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e8d172a2-1831-49b4-8971-86e782488742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:39.609324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d172a2-1831-49b4-8971-86e782488742 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:39.614451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:39.615075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:39.615362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:39.615644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a admin admin] PATH_INFO: `/v3/projects/e203df4336854849801de4d6ae8605ce` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:39.616755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e203df4336854849801de4d6ae8605ce)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:39.618702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:39.702062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e203df4336854849801de4d6ae8605ce', 'request_id': 'req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:39.718045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e203df4336854849801de4d6ae8605ce', 'request_id': 'req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:39.727019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e203df4336854849801de4d6ae8605ce', 'request_id': 'req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:39.729217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:39.740799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a admin admin] The token cache is being invalidate because project e203df4336854849801de4d6ae8605ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:39.741494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:39.742380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b7d4f4df-2dd7-439a-9538-2b5cf6c5265a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:39.744417 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3000/6072] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:39 2022] DELETE /identity/v3/projects/e203df4336854849801de4d6ae8605ce => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:39.751889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8af21-d727-46f5-84ed-0e0bf6846dac None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:39.752425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8af21-d727-46f5-84ed-0e0bf6846dac None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:39.752425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8af21-d727-46f5-84ed-0e0bf6846dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:39.772224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-76d8af21-d727-46f5-84ed-0e0bf6846dac None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:39.773489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-76d8af21-d727-46f5-84ed-0e0bf6846dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:39.835805 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3073/6073] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:39.844612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1787a3-fda8-4860-a2bd-6daa3eceba66 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:39.845119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1787a3-fda8-4860-a2bd-6daa3eceba66 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:39.845495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1787a3-fda8-4860-a2bd-6daa3eceba66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:39.866938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ae1787a3-fda8-4860-a2bd-6daa3eceba66 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:39.868303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ae1787a3-fda8-4860-a2bd-6daa3eceba66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:39.877053 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3001/6074] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:39.881584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8af21-d727-46f5-84ed-0e0bf6846dac None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:39.892045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8af21-d727-46f5-84ed-0e0bf6846dac None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:39.896749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578f394-c51b-4237-8c56-a5e76f1bf76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:39.897399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578f394-c51b-4237-8c56-a5e76f1bf76e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:39.897690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578f394-c51b-4237-8c56-a5e76f1bf76e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:39.897970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578f394-c51b-4237-8c56-a5e76f1bf76e admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:39.898626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578f394-c51b-4237-8c56-a5e76f1bf76e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:10:39.898938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578f394-c51b-4237-8c56-a5e76f1bf76e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:39.900556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578f394-c51b-4237-8c56-a5e76f1bf76e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:39.913423 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3074/6075] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:10:39.917964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1787a3-fda8-4860-a2bd-6daa3eceba66 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:39.927574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1787a3-fda8-4860-a2bd-6daa3eceba66 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:39.932294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d01ce-bdd9-44a0-b071-fd306ede1f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:39.932949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d01ce-bdd9-44a0-b071-fd306ede1f9f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:39.933266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d01ce-bdd9-44a0-b071-fd306ede1f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:39.933562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d01ce-bdd9-44a0-b071-fd306ede1f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:39.934239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d01ce-bdd9-44a0-b071-fd306ede1f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:39.936290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d01ce-bdd9-44a0-b071-fd306ede1f9f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:39.967876 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3002/6076] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:39 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:39.973202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578f394-c51b-4237-8c56-a5e76f1bf76e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:39.984487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578f394-c51b-4237-8c56-a5e76f1bf76e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:39.989421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:40.043803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac28c1-3d93-4fe8-9f9e-41a83ae43322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:40.044583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac28c1-3d93-4fe8-9f9e-41a83ae43322 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:40.044867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac28c1-3d93-4fe8-9f9e-41a83ae43322 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:40.045174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac28c1-3d93-4fe8-9f9e-41a83ae43322 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:40.046025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac28c1-3d93-4fe8-9f9e-41a83ae43322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:40.047983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac28c1-3d93-4fe8-9f9e-41a83ae43322 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:40.071581 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3075/6077] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:39 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:40.072766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d01ce-bdd9-44a0-b071-fd306ede1f9f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:40.084041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d01ce-bdd9-44a0-b071-fd306ede1f9f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:40.089423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8476a0-a539-415d-ae39-c59450cb70c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:40.090047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8476a0-a539-415d-ae39-c59450cb70c0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:40.090323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8476a0-a539-415d-ae39-c59450cb70c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:40.090629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8476a0-a539-415d-ae39-c59450cb70c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:40.091389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8476a0-a539-415d-ae39-c59450cb70c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:40.093104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8476a0-a539-415d-ae39-c59450cb70c0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:40.104494 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3003/6078] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:40.108606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac28c1-3d93-4fe8-9f9e-41a83ae43322 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:40.123935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac28c1-3d93-4fe8-9f9e-41a83ae43322 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:40.129505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0eb3b-20c6-465e-8c53-c991cdcb9177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:40.130361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0eb3b-20c6-465e-8c53-c991cdcb9177 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:40.130662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0eb3b-20c6-465e-8c53-c991cdcb9177 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:40.130949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0eb3b-20c6-465e-8c53-c991cdcb9177 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:40.131604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0eb3b-20c6-465e-8c53-c991cdcb9177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:40.133462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0eb3b-20c6-465e-8c53-c991cdcb9177 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:40.149817 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3076/6079] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:10:40.151190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8476a0-a539-415d-ae39-c59450cb70c0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:40.162797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8476a0-a539-415d-ae39-c59450cb70c0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:40.167630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cefeda-bfa7-4420-b1f9-ab9e766d3907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:40.168289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cefeda-bfa7-4420-b1f9-ab9e766d3907 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:40.169012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cefeda-bfa7-4420-b1f9-ab9e766d3907 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:40.169228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cefeda-bfa7-4420-b1f9-ab9e766d3907 admin admin] PATH_INFO: `/v3/projects/5057ebef2e6d4a5880e6bc8e0bdc382c/users/8213fcba4a3643e3a091dd28e2f79138/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:40.186215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cefeda-bfa7-4420-b1f9-ab9e766d3907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5057ebef2e6d4a5880e6bc8e0bdc382c, user_id=8213fcba4a3643e3a091dd28e2f79138, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:40.188929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cefeda-bfa7-4420-b1f9-ab9e766d3907 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:40.203329 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3004/6080] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:40 2022] PUT /identity/v3/projects/5057ebef2e6d4a5880e6bc8e0bdc382c/users/8213fcba4a3643e3a091dd28e2f79138/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:40.214274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225c24-3ede-4277-aed4-0b0ac53b8053 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:40.214635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225c24-3ede-4277-aed4-0b0ac53b8053 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:40.214982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225c24-3ede-4277-aed4-0b0ac53b8053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:40.247545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8e225c24-3ede-4277-aed4-0b0ac53b8053 None None] MFA Rules not processed for user `8213fcba4a3643e3a091dd28e2f79138`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:40.248786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8e225c24-3ede-4277-aed4-0b0ac53b8053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:40.313886 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3077/6081] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:40 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:40.325330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cefeda-bfa7-4420-b1f9-ab9e766d3907 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:40.327526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-56cefeda-bfa7-4420-b1f9-ab9e766d3907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:40.394887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cefeda-bfa7-4420-b1f9-ab9e766d3907 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:40.398952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f52e8-a256-4011-a44f-83d06e700a47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:40.399604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f52e8-a256-4011-a44f-83d06e700a47 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:40.399893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f52e8-a256-4011-a44f-83d06e700a47 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:40.400171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f52e8-a256-4011-a44f-83d06e700a47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:40.415573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f52e8-a256-4011-a44f-83d06e700a47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:40.417355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f52e8-a256-4011-a44f-83d06e700a47 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:40.429179 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3005/6082] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:40 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:41.421078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e225c24-3ede-4277-aed4-0b0ac53b8053 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:41.423426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8e225c24-3ede-4277-aed4-0b0ac53b8053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:41.493899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e225c24-3ede-4277-aed4-0b0ac53b8053 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:41.499597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:41.500292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:41.501156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:41.501440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] PATH_INFO: `/v3/users/ced09232d29a41d3a0d075fcc53a7e4e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:41.502583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced09232d29a41d3a0d075fcc53a7e4e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:41.504615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:41.526891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:41.553428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced09232d29a41d3a0d075fcc53a7e4e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:41.565796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced09232d29a41d3a0d075fcc53a7e4e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:41.572912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced09232d29a41d3a0d075fcc53a7e4e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:41.573571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced09232d29a41d3a0d075fcc53a7e4e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:41.574216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced09232d29a41d3a0d075fcc53a7e4e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:41.587030 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3078/6083] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:41 2022] DELETE /identity/v3/users/ced09232d29a41d3a0d075fcc53a7e4e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:41.591567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc9813-9d8a-4353-a123-ae2508910151 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:41.592028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc9813-9d8a-4353-a123-ae2508910151 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:41.592150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc9813-9d8a-4353-a123-ae2508910151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:41.611608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-50bc9813-9d8a-4353-a123-ae2508910151 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:41.612727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-50bc9813-9d8a-4353-a123-ae2508910151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:41.669562 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3006/6084] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:41.679666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:41.681083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:41.750346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48d5f2-d977-44b7-b9a9-664881fe359f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:41.754749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaac01f-2373-4401-8e31-bf6ff2639396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:41.755366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaac01f-2373-4401-8e31-bf6ff2639396 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:41.755656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaac01f-2373-4401-8e31-bf6ff2639396 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:41.755943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaac01f-2373-4401-8e31-bf6ff2639396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:41.765327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaac01f-2373-4401-8e31-bf6ff2639396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:41.766770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaac01f-2373-4401-8e31-bf6ff2639396 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:41.779449 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3079/6085] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:41.954045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc9813-9d8a-4353-a123-ae2508910151 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:41.955473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-50bc9813-9d8a-4353-a123-ae2508910151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:41.970306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc9813-9d8a-4353-a123-ae2508910151 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:41.976747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7d3d1-6a15-4a2c-a9df-343c4d703645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:41.977619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7d3d1-6a15-4a2c-a9df-343c4d703645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:41.978016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7d3d1-6a15-4a2c-a9df-343c4d703645 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:41.978483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7d3d1-6a15-4a2c-a9df-343c4d703645 admin admin] PATH_INFO: `/v3/projects/77a2358e02c446288638170b6ec6d144` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:41.979691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7d3d1-6a15-4a2c-a9df-343c4d703645 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77a2358e02c446288638170b6ec6d144)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:41.982683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7d3d1-6a15-4a2c-a9df-343c4d703645 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:42.017189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:42.039077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-03d7d3d1-6a15-4a2c-a9df-343c4d703645 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77a2358e02c446288638170b6ec6d144', 'request_id': 'req-03d7d3d1-6a15-4a2c-a9df-343c4d703645'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:42.053751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-03d7d3d1-6a15-4a2c-a9df-343c4d703645 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77a2358e02c446288638170b6ec6d144', 'request_id': 'req-03d7d3d1-6a15-4a2c-a9df-343c4d703645'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:42.060551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-03d7d3d1-6a15-4a2c-a9df-343c4d703645 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77a2358e02c446288638170b6ec6d144', 'request_id': 'req-03d7d3d1-6a15-4a2c-a9df-343c4d703645'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:42.071679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-03d7d3d1-6a15-4a2c-a9df-343c4d703645 admin admin] The token cache is being invalidate because project 77a2358e02c446288638170b6ec6d144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:42.071975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-03d7d3d1-6a15-4a2c-a9df-343c4d703645 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:42.072634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-03d7d3d1-6a15-4a2c-a9df-343c4d703645 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:42.074141 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3007/6086] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:41 2022] DELETE /identity/v3/projects/77a2358e02c446288638170b6ec6d144 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:42.079053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf9849-d118-4da4-bca0-ffe319565d98 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.079341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf9849-d118-4da4-bca0-ffe319565d98 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.079617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf9849-d118-4da4-bca0-ffe319565d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.099630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3fbf9849-d118-4da4-bca0-ffe319565d98 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:42.101190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3fbf9849-d118-4da4-bca0-ffe319565d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:42.167856 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3080/6087] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:42.176583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9499fc-d4a3-46ac-a3f2-b5bbd3f37040 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.176984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9499fc-d4a3-46ac-a3f2-b5bbd3f37040 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.177340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9499fc-d4a3-46ac-a3f2-b5bbd3f37040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.197778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2a9499fc-d4a3-46ac-a3f2-b5bbd3f37040 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:42.199010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2a9499fc-d4a3-46ac-a3f2-b5bbd3f37040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:42.201307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf9849-d118-4da4-bca0-ffe319565d98 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:42.203047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3fbf9849-d118-4da4-bca0-ffe319565d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:42.207734 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3008/6088] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:42.211778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9499fc-d4a3-46ac-a3f2-b5bbd3f37040 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:42.217172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf9849-d118-4da4-bca0-ffe319565d98 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:42.221572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9499fc-d4a3-46ac-a3f2-b5bbd3f37040 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:42.222767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c5749-4cff-4503-9bde-a06c278d1219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:42.223350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c5749-4cff-4503-9bde-a06c278d1219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.223637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c5749-4cff-4503-9bde-a06c278d1219 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.223915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c5749-4cff-4503-9bde-a06c278d1219 admin admin] PATH_INFO: `/v3/users/8213fcba4a3643e3a091dd28e2f79138` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.225340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c5749-4cff-4503-9bde-a06c278d1219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8213fcba4a3643e3a091dd28e2f79138)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:42.227148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff70e4-3c14-4b90-8730-3380f53bc879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:42.227746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff70e4-3c14-4b90-8730-3380f53bc879 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.227853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c5749-4cff-4503-9bde-a06c278d1219 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:42.228036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff70e4-3c14-4b90-8730-3380f53bc879 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.228321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff70e4-3c14-4b90-8730-3380f53bc879 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.229003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff70e4-3c14-4b90-8730-3380f53bc879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:10:42.229320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff70e4-3c14-4b90-8730-3380f53bc879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:42.230919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff70e4-3c14-4b90-8730-3380f53bc879 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:42.244009 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3009/6089] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:10:42.248437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff70e4-3c14-4b90-8730-3380f53bc879 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:42.259854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff70e4-3c14-4b90-8730-3380f53bc879 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:42.266541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a31cb-a173-468c-bf56-6e60728588bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:42.266671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a31cb-a173-468c-bf56-6e60728588bc admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.267062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a31cb-a173-468c-bf56-6e60728588bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.267176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a31cb-a173-468c-bf56-6e60728588bc admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.267798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a31cb-a173-468c-bf56-6e60728588bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:42.269756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a31cb-a173-468c-bf56-6e60728588bc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:42.269900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:42.285138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6c5c5749-4cff-4503-9bde-a06c278d1219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8213fcba4a3643e3a091dd28e2f79138'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:42.299148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6c5c5749-4cff-4503-9bde-a06c278d1219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8213fcba4a3643e3a091dd28e2f79138'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:42.304026 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3010/6090] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:42 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:10:42.308054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6c5c5749-4cff-4503-9bde-a06c278d1219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8213fcba4a3643e3a091dd28e2f79138'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:42.308476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a31cb-a173-468c-bf56-6e60728588bc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:42.308845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6c5c5749-4cff-4503-9bde-a06c278d1219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8213fcba4a3643e3a091dd28e2f79138'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:42.309682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6c5c5749-4cff-4503-9bde-a06c278d1219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8213fcba4a3643e3a091dd28e2f79138'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:42.310159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2d0a31cb-a173-468c-bf56-6e60728588bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:42.330651 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3081/6091] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:42 2022] DELETE /identity/v3/users/8213fcba4a3643e3a091dd28e2f79138 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:42.332406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030ba41-32ba-4986-98d3-e36fb88a7409 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.332705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030ba41-32ba-4986-98d3-e36fb88a7409 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.333020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030ba41-32ba-4986-98d3-e36fb88a7409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.353928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2030ba41-32ba-4986-98d3-e36fb88a7409 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:42.355160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2030ba41-32ba-4986-98d3-e36fb88a7409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:42.387707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a31cb-a173-468c-bf56-6e60728588bc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:42.431463 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3082/6092] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:42.442197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030ba41-32ba-4986-98d3-e36fb88a7409 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:42.443573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2030ba41-32ba-4986-98d3-e36fb88a7409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:42.463900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b001e44-620a-4bf1-a65e-3575e375cd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:42.464714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b001e44-620a-4bf1-a65e-3575e375cd68 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.464812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b001e44-620a-4bf1-a65e-3575e375cd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.465109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b001e44-620a-4bf1-a65e-3575e375cd68 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.465761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b001e44-620a-4bf1-a65e-3575e375cd68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:42.467700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b001e44-620a-4bf1-a65e-3575e375cd68 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:42.497646 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3011/6093] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:42 2022] POST /identity/v3/users => generated 295 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:10:42.501219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b001e44-620a-4bf1-a65e-3575e375cd68 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:42.512585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b001e44-620a-4bf1-a65e-3575e375cd68 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:42.517536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0249ae4-0c15-464e-8774-fba8cf5f1f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:42.518114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030ba41-32ba-4986-98d3-e36fb88a7409 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:42.518259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0249ae4-0c15-464e-8774-fba8cf5f1f6a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.518431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0249ae4-0c15-464e-8774-fba8cf5f1f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.518715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0249ae4-0c15-464e-8774-fba8cf5f1f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.519367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0249ae4-0c15-464e-8774-fba8cf5f1f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:42.521036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0249ae4-0c15-464e-8774-fba8cf5f1f6a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:42.522259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f17520-13f5-4515-9894-8ebb3dca13a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:42.523189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f17520-13f5-4515-9894-8ebb3dca13a5 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.523822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:42.524013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f17520-13f5-4515-9894-8ebb3dca13a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.524323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f17520-13f5-4515-9894-8ebb3dca13a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.533560 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3012/6094] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:42.537722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0249ae4-0c15-464e-8774-fba8cf5f1f6a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:42.541797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f17520-13f5-4515-9894-8ebb3dca13a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:42.543255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f17520-13f5-4515-9894-8ebb3dca13a5 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:42.549598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0249ae4-0c15-464e-8774-fba8cf5f1f6a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:42.556127 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3083/6095] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:10:42.557609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474baaf-2977-4166-b2d4-722405b23665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:42.558256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474baaf-2977-4166-b2d4-722405b23665 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.558608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474baaf-2977-4166-b2d4-722405b23665 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.558895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474baaf-2977-4166-b2d4-722405b23665 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.559556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474baaf-2977-4166-b2d4-722405b23665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:42.561257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474baaf-2977-4166-b2d4-722405b23665 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:42.576956 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3013/6096] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:42.578591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f17520-13f5-4515-9894-8ebb3dca13a5 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:42.589658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f17520-13f5-4515-9894-8ebb3dca13a5 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:42.595707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52a77b-211d-4bd7-93d7-10fe3dec4fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:42.596420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52a77b-211d-4bd7-93d7-10fe3dec4fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.596712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52a77b-211d-4bd7-93d7-10fe3dec4fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.597105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52a77b-211d-4bd7-93d7-10fe3dec4fd6 admin admin] PATH_INFO: `/v3/projects/0d7d6354069a450e9dd7ce54bfc2d420/users/0bfbb00dc0574dc9a3cafd8d671ca1b1/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.616357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52a77b-211d-4bd7-93d7-10fe3dec4fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d7d6354069a450e9dd7ce54bfc2d420, user_id=0bfbb00dc0574dc9a3cafd8d671ca1b1, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:42.619115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52a77b-211d-4bd7-93d7-10fe3dec4fd6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:42.634561 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3084/6097] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:42 2022] PUT /identity/v3/projects/0d7d6354069a450e9dd7ce54bfc2d420/users/0bfbb00dc0574dc9a3cafd8d671ca1b1/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:42.645061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e5d9a-7928-4c2a-8178-eb34f9c9a896 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.645588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e5d9a-7928-4c2a-8178-eb34f9c9a896 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.645588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e5d9a-7928-4c2a-8178-eb34f9c9a896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.677450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bc8e5d9a-7928-4c2a-8178-eb34f9c9a896 None None] MFA Rules not processed for user `0bfbb00dc0574dc9a3cafd8d671ca1b1`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:42.678880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bc8e5d9a-7928-4c2a-8178-eb34f9c9a896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:42.783948 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3014/6098] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:42.803675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52a77b-211d-4bd7-93d7-10fe3dec4fd6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:42.811882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1d52a77b-211d-4bd7-93d7-10fe3dec4fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:42.861216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:42.914487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52a77b-211d-4bd7-93d7-10fe3dec4fd6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:42.920994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b44343-bcb4-42e6-aace-83e5632e27cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:42.921717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b44343-bcb4-42e6-aace-83e5632e27cc service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:42.921896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b44343-bcb4-42e6-aace-83e5632e27cc service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:42.922204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b44343-bcb4-42e6-aace-83e5632e27cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:42.943412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b44343-bcb4-42e6-aace-83e5632e27cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:42.944829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b44343-bcb4-42e6-aace-83e5632e27cc service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:42.980345 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3085/6099] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:10:43.080106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e5d9a-7928-4c2a-8178-eb34f9c9a896 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:43.081695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bc8e5d9a-7928-4c2a-8178-eb34f9c9a896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:43.211746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e5d9a-7928-4c2a-8178-eb34f9c9a896 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:43.217123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b76065-3b23-439d-a8e6-52cba6309709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:43.217861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b76065-3b23-439d-a8e6-52cba6309709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:43.218227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b76065-3b23-439d-a8e6-52cba6309709 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:43.218583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b76065-3b23-439d-a8e6-52cba6309709 admin admin] PATH_INFO: `/v3/projects/5057ebef2e6d4a5880e6bc8e0bdc382c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:43.220464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b76065-3b23-439d-a8e6-52cba6309709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5057ebef2e6d4a5880e6bc8e0bdc382c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:43.222668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b76065-3b23-439d-a8e6-52cba6309709 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:43.283525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04b76065-3b23-439d-a8e6-52cba6309709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5057ebef2e6d4a5880e6bc8e0bdc382c', 'request_id': 'req-04b76065-3b23-439d-a8e6-52cba6309709'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:43.292655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04b76065-3b23-439d-a8e6-52cba6309709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5057ebef2e6d4a5880e6bc8e0bdc382c', 'request_id': 'req-04b76065-3b23-439d-a8e6-52cba6309709'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:43.302383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04b76065-3b23-439d-a8e6-52cba6309709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5057ebef2e6d4a5880e6bc8e0bdc382c', 'request_id': 'req-04b76065-3b23-439d-a8e6-52cba6309709'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:43.319747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb08aa8-32fd-4d89-9661-590ef39f76b5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:43.320127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb08aa8-32fd-4d89-9661-590ef39f76b5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:43.320317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb08aa8-32fd-4d89-9661-590ef39f76b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:43.320423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04b76065-3b23-439d-a8e6-52cba6309709 admin admin] The token cache is being invalidate because project 5057ebef2e6d4a5880e6bc8e0bdc382c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:43.320680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04b76065-3b23-439d-a8e6-52cba6309709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:43.321420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04b76065-3b23-439d-a8e6-52cba6309709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:43.325153 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3015/6100] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:43 2022] DELETE /identity/v3/projects/5057ebef2e6d4a5880e6bc8e0bdc382c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:43.331557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da110983-6cbe-43ae-8540-152c93e4ceb0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:43.331865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da110983-6cbe-43ae-8540-152c93e4ceb0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:43.332154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da110983-6cbe-43ae-8540-152c93e4ceb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:43.341650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bfb08aa8-32fd-4d89-9661-590ef39f76b5 None None] MFA Rules not processed for user `d485394bc49e4380b152d32d09440281`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:43.342889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bfb08aa8-32fd-4d89-9661-590ef39f76b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:43.352930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:43.360577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-da110983-6cbe-43ae-8540-152c93e4ceb0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:43.362530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-da110983-6cbe-43ae-8540-152c93e4ceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:43.420669 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3086/6101] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:43.456057 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3016/6102] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:43.464940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2c267-6714-441a-a3b1-a63796ad7073 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:43.465441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2c267-6714-441a-a3b1-a63796ad7073 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:43.465525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2c267-6714-441a-a3b1-a63796ad7073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:43.492113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f8a2c267-6714-441a-a3b1-a63796ad7073 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:43.493361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f8a2c267-6714-441a-a3b1-a63796ad7073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:43.502407 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3087/6103] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:43.506502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da110983-6cbe-43ae-8540-152c93e4ceb0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:43.517234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da110983-6cbe-43ae-8540-152c93e4ceb0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:43.521950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1a8c7-4671-4e0c-8e71-894564c0b25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:43.522563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1a8c7-4671-4e0c-8e71-894564c0b25c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:43.522848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1a8c7-4671-4e0c-8e71-894564c0b25c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:43.523125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1a8c7-4671-4e0c-8e71-894564c0b25c admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:43.523769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1a8c7-4671-4e0c-8e71-894564c0b25c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:10:43.524075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1a8c7-4671-4e0c-8e71-894564c0b25c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:43.525726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1a8c7-4671-4e0c-8e71-894564c0b25c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:43.538946 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3017/6104] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:10:43.544002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2c267-6714-441a-a3b1-a63796ad7073 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:43.555451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2c267-6714-441a-a3b1-a63796ad7073 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:43.560805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362919ca-e65c-4125-b4e8-70cb446744c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:43.561259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362919ca-e65c-4125-b4e8-70cb446744c1 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:43.561552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362919ca-e65c-4125-b4e8-70cb446744c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:43.561833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362919ca-e65c-4125-b4e8-70cb446744c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:43.562556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362919ca-e65c-4125-b4e8-70cb446744c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:43.565097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362919ca-e65c-4125-b4e8-70cb446744c1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:43.601732 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3088/6105] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:43 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:43.602990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1a8c7-4671-4e0c-8e71-894564c0b25c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:43.614689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1a8c7-4671-4e0c-8e71-894564c0b25c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:43.678611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12e9c1-ba4d-458c-9e8b-78bf1f4fd04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:43.679250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12e9c1-ba4d-458c-9e8b-78bf1f4fd04c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:43.679511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12e9c1-ba4d-458c-9e8b-78bf1f4fd04c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:43.679793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12e9c1-ba4d-458c-9e8b-78bf1f4fd04c admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:43.680436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12e9c1-ba4d-458c-9e8b-78bf1f4fd04c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:43.682416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12e9c1-ba4d-458c-9e8b-78bf1f4fd04c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:43.707242 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3018/6106] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:43 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:43.711014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362919ca-e65c-4125-b4e8-70cb446744c1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:43.722063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362919ca-e65c-4125-b4e8-70cb446744c1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:43.727254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89e757-695c-494a-a69e-31e4f68f59e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:43.728169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89e757-695c-494a-a69e-31e4f68f59e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:43.728643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89e757-695c-494a-a69e-31e4f68f59e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:43.728960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89e757-695c-494a-a69e-31e4f68f59e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:43.729618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89e757-695c-494a-a69e-31e4f68f59e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:43.731282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89e757-695c-494a-a69e-31e4f68f59e5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:43.743188 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3089/6107] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:10:43.747206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12e9c1-ba4d-458c-9e8b-78bf1f4fd04c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:43.757715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12e9c1-ba4d-458c-9e8b-78bf1f4fd04c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:43.762883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11862ad9-7fea-4f34-8557-73ccdcd6b056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:43.763497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11862ad9-7fea-4f34-8557-73ccdcd6b056 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:43.763787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11862ad9-7fea-4f34-8557-73ccdcd6b056 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:43.764070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11862ad9-7fea-4f34-8557-73ccdcd6b056 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:43.764762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11862ad9-7fea-4f34-8557-73ccdcd6b056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:43.766984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11862ad9-7fea-4f34-8557-73ccdcd6b056 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:43.778828 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3019/6108] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:43.783560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89e757-695c-494a-a69e-31e4f68f59e5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:43.794619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89e757-695c-494a-a69e-31e4f68f59e5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:43.801980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866bf66-fabc-4484-8d5d-39dfbc4c3bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:43.802647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866bf66-fabc-4484-8d5d-39dfbc4c3bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:43.802934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866bf66-fabc-4484-8d5d-39dfbc4c3bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:43.803213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866bf66-fabc-4484-8d5d-39dfbc4c3bf0 admin admin] PATH_INFO: `/v3/projects/d389730e46ff4f35bf0fec366e2be537/users/9f792275f53f445ea72b6de441314a49/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:43.820044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866bf66-fabc-4484-8d5d-39dfbc4c3bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d389730e46ff4f35bf0fec366e2be537, user_id=9f792275f53f445ea72b6de441314a49, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:43.822764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866bf66-fabc-4484-8d5d-39dfbc4c3bf0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:43.836094 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3090/6109] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:43 2022] PUT /identity/v3/projects/d389730e46ff4f35bf0fec366e2be537/users/9f792275f53f445ea72b6de441314a49/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:43.847017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07b7e9-b3fa-4226-b249-9965d2b1c7e4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:43.847318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07b7e9-b3fa-4226-b249-9965d2b1c7e4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:43.847580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07b7e9-b3fa-4226-b249-9965d2b1c7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:43.876919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-da07b7e9-b3fa-4226-b249-9965d2b1c7e4 None None] MFA Rules not processed for user `9f792275f53f445ea72b6de441314a49`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:43.878353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-da07b7e9-b3fa-4226-b249-9965d2b1c7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:43.943816 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3020/6110] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:43.948728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866bf66-fabc-4484-8d5d-39dfbc4c3bf0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:44.037114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866bf66-fabc-4484-8d5d-39dfbc4c3bf0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:44.037114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bdf2f-3439-4670-9554-4d56df06c3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:44.037114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bdf2f-3439-4670-9554-4d56df06c3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:44.037114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bdf2f-3439-4670-9554-4d56df06c3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:44.037490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bdf2f-3439-4670-9554-4d56df06c3d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:44.037490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bdf2f-3439-4670-9554-4d56df06c3d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:44.037490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bdf2f-3439-4670-9554-4d56df06c3d0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:44.049387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:44.071994 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3091/6111] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:43 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:44.073210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07b7e9-b3fa-4226-b249-9965d2b1c7e4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:44.079530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:44.087158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07b7e9-b3fa-4226-b249-9965d2b1c7e4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:44.150075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e21bc-c3e4-4b3d-b387-98db77525d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:44.150699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e21bc-c3e4-4b3d-b387-98db77525d5c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:44.150979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e21bc-c3e4-4b3d-b387-98db77525d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:44.151251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e21bc-c3e4-4b3d-b387-98db77525d5c admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:44.151887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e21bc-c3e4-4b3d-b387-98db77525d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:44.154047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e21bc-c3e4-4b3d-b387-98db77525d5c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:44.178826 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3021/6112] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:44 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:44.182595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bdf2f-3439-4670-9554-4d56df06c3d0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:44.194127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bdf2f-3439-4670-9554-4d56df06c3d0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:44.199911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67979eae-34cc-4469-bc57-6c3b04bd8d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:44.200595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67979eae-34cc-4469-bc57-6c3b04bd8d00 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:44.200879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67979eae-34cc-4469-bc57-6c3b04bd8d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:44.201348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67979eae-34cc-4469-bc57-6c3b04bd8d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:44.202338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67979eae-34cc-4469-bc57-6c3b04bd8d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:44.204955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67979eae-34cc-4469-bc57-6c3b04bd8d00 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:44.217649 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3092/6113] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:44.221706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e21bc-c3e4-4b3d-b387-98db77525d5c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:44.234492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e21bc-c3e4-4b3d-b387-98db77525d5c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:44.240029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc190250-ee60-47a1-ac2e-00f5e04a8e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:44.240978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc190250-ee60-47a1-ac2e-00f5e04a8e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:44.241360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc190250-ee60-47a1-ac2e-00f5e04a8e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:44.241770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc190250-ee60-47a1-ac2e-00f5e04a8e1f admin admin] PATH_INFO: `/v3/projects/8b76aac08280467fafda70600597ae29/users/9a11dffdf5834c0bb605a50574e926c1/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:44.260529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc190250-ee60-47a1-ac2e-00f5e04a8e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b76aac08280467fafda70600597ae29, user_id=9a11dffdf5834c0bb605a50574e926c1, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:44.263851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc190250-ee60-47a1-ac2e-00f5e04a8e1f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:44.287825 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3022/6114] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:44 2022] PUT /identity/v3/projects/8b76aac08280467fafda70600597ae29/users/9a11dffdf5834c0bb605a50574e926c1/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:10:44.292051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67979eae-34cc-4469-bc57-6c3b04bd8d00 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:44.307565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67979eae-34cc-4469-bc57-6c3b04bd8d00 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:44.446073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca54e0-5fab-47c1-81a2-3b5c9f6a7096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:44.446697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca54e0-5fab-47c1-81a2-3b5c9f6a7096 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:44.446998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca54e0-5fab-47c1-81a2-3b5c9f6a7096 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:44.447290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca54e0-5fab-47c1-81a2-3b5c9f6a7096 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:44.447994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca54e0-5fab-47c1-81a2-3b5c9f6a7096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:44.450300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca54e0-5fab-47c1-81a2-3b5c9f6a7096 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:44.475829 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3093/6115] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:10:44.481173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc190250-ee60-47a1-ac2e-00f5e04a8e1f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:44.499523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc190250-ee60-47a1-ac2e-00f5e04a8e1f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:44.529649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28499b1-4803-4281-837e-45c3bdff2142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:44.530346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28499b1-4803-4281-837e-45c3bdff2142 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:44.530639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28499b1-4803-4281-837e-45c3bdff2142 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:44.530974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28499b1-4803-4281-837e-45c3bdff2142 admin admin] PATH_INFO: `/v3/projects/8b76aac08280467fafda70600597ae29/users/9a11dffdf5834c0bb605a50574e926c1/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:44.536468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28499b1-4803-4281-837e-45c3bdff2142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b76aac08280467fafda70600597ae29, user_id=9a11dffdf5834c0bb605a50574e926c1, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:44.540759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28499b1-4803-4281-837e-45c3bdff2142 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:44.573061 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3023/6116] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:10:44 2022] PUT /identity/v3/projects/8b76aac08280467fafda70600597ae29/users/9a11dffdf5834c0bb605a50574e926c1/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:44.586413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d04622-d1e3-472f-b3b8-6bdac8e8ff86 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:44.586662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d04622-d1e3-472f-b3b8-6bdac8e8ff86 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:44.586931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d04622-d1e3-472f-b3b8-6bdac8e8ff86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:44.629299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-67d04622-d1e3-472f-b3b8-6bdac8e8ff86 None None] MFA Rules not processed for user `9a11dffdf5834c0bb605a50574e926c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:44.630802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-67d04622-d1e3-472f-b3b8-6bdac8e8ff86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:44.746456 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3094/6117] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:44.753614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28499b1-4803-4281-837e-45c3bdff2142 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:44.756939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d28499b1-4803-4281-837e-45c3bdff2142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:44.892262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28499b1-4803-4281-837e-45c3bdff2142 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:44.896942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf44a79-a419-4282-82cb-902a1787274a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:44.897718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf44a79-a419-4282-82cb-902a1787274a service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:44.898116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf44a79-a419-4282-82cb-902a1787274a service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:44.898616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf44a79-a419-4282-82cb-902a1787274a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:45.021799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf44a79-a419-4282-82cb-902a1787274a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:45.023761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf44a79-a419-4282-82cb-902a1787274a service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:45.041152 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3024/6118] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:10:48.490250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d04622-d1e3-472f-b3b8-6bdac8e8ff86 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:48.491934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-67d04622-d1e3-472f-b3b8-6bdac8e8ff86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:48.562399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:48.614164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d04622-d1e3-472f-b3b8-6bdac8e8ff86 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:48.623516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae498f8-775d-44ad-a3bb-6d91705c3d78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:48.624313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae498f8-775d-44ad-a3bb-6d91705c3d78 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:48.624696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae498f8-775d-44ad-a3bb-6d91705c3d78 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:48.626442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae498f8-775d-44ad-a3bb-6d91705c3d78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:48.629243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4ae498f8-775d-44ad-a3bb-6d91705c3d78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:48.678672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae498f8-775d-44ad-a3bb-6d91705c3d78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:48.681331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae498f8-775d-44ad-a3bb-6d91705c3d78 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:48.724455 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3095/6119] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:49.086438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf44a79-a419-4282-82cb-902a1787274a service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:49.088055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2cf44a79-a419-4282-82cb-902a1787274a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:49.108422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf44a79-a419-4282-82cb-902a1787274a service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:49.124195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52252745-0122-41e2-8080-ac7222c964d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:49.126054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52252745-0122-41e2-8080-ac7222c964d9 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:49.126348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52252745-0122-41e2-8080-ac7222c964d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:49.126628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52252745-0122-41e2-8080-ac7222c964d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:49.146594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:49.150229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52252745-0122-41e2-8080-ac7222c964d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:49.152068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52252745-0122-41e2-8080-ac7222c964d9 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:49.170745 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3025/6120] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:58.377414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf00193-69f6-4191-b992-6a2c21f01029 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:58.378482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf00193-69f6-4191-b992-6a2c21f01029 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:58.379026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf00193-69f6-4191-b992-6a2c21f01029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:58.452790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fdf00193-69f6-4191-b992-6a2c21f01029 None None] MFA Rules not processed for user `9f792275f53f445ea72b6de441314a49`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:58.457282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fdf00193-69f6-4191-b992-6a2c21f01029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:58.469893 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3096/6121] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:58.592338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52252745-0122-41e2-8080-ac7222c964d9 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:58.594272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-52252745-0122-41e2-8080-ac7222c964d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:58.733112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52252745-0122-41e2-8080-ac7222c964d9 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:58.738877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:58.739650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:58.740038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:58.740551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] PATH_INFO: `/v3/users/0bfbb00dc0574dc9a3cafd8d671ca1b1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:58.742202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bfbb00dc0574dc9a3cafd8d671ca1b1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:58.744435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:58.827106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bfbb00dc0574dc9a3cafd8d671ca1b1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:58.842950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bfbb00dc0574dc9a3cafd8d671ca1b1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:58.843771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bfbb00dc0574dc9a3cafd8d671ca1b1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:58.851465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bfbb00dc0574dc9a3cafd8d671ca1b1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:58.862133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bfbb00dc0574dc9a3cafd8d671ca1b1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:58.869106 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3026/6122] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:10:58 2022] DELETE /identity/v3/users/0bfbb00dc0574dc9a3cafd8d671ca1b1 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:58.873405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509b77d-4202-4eba-ab3e-2d7d54aa920e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:58.873858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509b77d-4202-4eba-ab3e-2d7d54aa920e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:58.874142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509b77d-4202-4eba-ab3e-2d7d54aa920e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:58.894527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8509b77d-4202-4eba-ab3e-2d7d54aa920e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:58.895746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8509b77d-4202-4eba-ab3e-2d7d54aa920e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:58.947280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:58.948866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:58.985566 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3097/6123] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:10:58.994165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509b77d-4202-4eba-ab3e-2d7d54aa920e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:58.996371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8509b77d-4202-4eba-ab3e-2d7d54aa920e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:59.035631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f04abd-af71-431f-b099-735728300187 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:59.044598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2d735-d38e-4fd1-9573-b2648e4bf4e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:59.045484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2d735-d38e-4fd1-9573-b2648e4bf4e7 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:59.045823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2d735-d38e-4fd1-9573-b2648e4bf4e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:59.046996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2d735-d38e-4fd1-9573-b2648e4bf4e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:59.055427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-08c2d735-d38e-4fd1-9573-b2648e4bf4e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:59.078758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2d735-d38e-4fd1-9573-b2648e4bf4e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:59.081013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2d735-d38e-4fd1-9573-b2648e4bf4e7 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:59.096717 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3027/6124] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:10:59.133576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509b77d-4202-4eba-ab3e-2d7d54aa920e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:59.139196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3103484-2c3a-4d56-a726-bc6998f38190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:59.139892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3103484-2c3a-4d56-a726-bc6998f38190 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:59.140348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3103484-2c3a-4d56-a726-bc6998f38190 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:59.140750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3103484-2c3a-4d56-a726-bc6998f38190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:59.151531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3103484-2c3a-4d56-a726-bc6998f38190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:59.153102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3103484-2c3a-4d56-a726-bc6998f38190 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:59.169018 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3098/6125] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:10:59.434201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2d735-d38e-4fd1-9573-b2648e4bf4e7 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:59.435721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-08c2d735-d38e-4fd1-9573-b2648e4bf4e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:59.461484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2d735-d38e-4fd1-9573-b2648e4bf4e7 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:59.466327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72112b1-7db2-4190-8368-b88ad01ff3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:59.466970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72112b1-7db2-4190-8368-b88ad01ff3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:59.467253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72112b1-7db2-4190-8368-b88ad01ff3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:59.467692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72112b1-7db2-4190-8368-b88ad01ff3dd admin admin] PATH_INFO: `/v3/projects/0d7d6354069a450e9dd7ce54bfc2d420` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:59.469537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72112b1-7db2-4190-8368-b88ad01ff3dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d7d6354069a450e9dd7ce54bfc2d420)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:59.483704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72112b1-7db2-4190-8368-b88ad01ff3dd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:59.510296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:59.567265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c72112b1-7db2-4190-8368-b88ad01ff3dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d7d6354069a450e9dd7ce54bfc2d420', 'request_id': 'req-c72112b1-7db2-4190-8368-b88ad01ff3dd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:59.599926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c72112b1-7db2-4190-8368-b88ad01ff3dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d7d6354069a450e9dd7ce54bfc2d420', 'request_id': 'req-c72112b1-7db2-4190-8368-b88ad01ff3dd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:59.613822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c72112b1-7db2-4190-8368-b88ad01ff3dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d7d6354069a450e9dd7ce54bfc2d420', 'request_id': 'req-c72112b1-7db2-4190-8368-b88ad01ff3dd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:59.639837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c72112b1-7db2-4190-8368-b88ad01ff3dd admin admin] The token cache is being invalidate because project 0d7d6354069a450e9dd7ce54bfc2d420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:10:59.640424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c72112b1-7db2-4190-8368-b88ad01ff3dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:59.643007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c72112b1-7db2-4190-8368-b88ad01ff3dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:10:59.645191 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3028/6126] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:10:59 2022] DELETE /identity/v3/projects/0d7d6354069a450e9dd7ce54bfc2d420 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:10:59.654100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577adfe3-603a-491f-8e3d-85db58a6f3fe None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:59.654644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577adfe3-603a-491f-8e3d-85db58a6f3fe None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:59.654754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577adfe3-603a-491f-8e3d-85db58a6f3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:59.677700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-577adfe3-603a-491f-8e3d-85db58a6f3fe None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:59.679036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-577adfe3-603a-491f-8e3d-85db58a6f3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:59.752320 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3099/6127] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:59.759539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabda114-8653-4c7f-b4ed-3a2aa0d324e0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:59.759833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabda114-8653-4c7f-b4ed-3a2aa0d324e0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:59.760181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabda114-8653-4c7f-b4ed-3a2aa0d324e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:59.781257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fabda114-8653-4c7f-b4ed-3a2aa0d324e0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:10:59.782621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fabda114-8653-4c7f-b4ed-3a2aa0d324e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:10:59.791088 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3029/6128] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:10:59.795674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577adfe3-603a-491f-8e3d-85db58a6f3fe None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:59.806501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577adfe3-603a-491f-8e3d-85db58a6f3fe None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:59.811640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fd9dc-d9bf-4ba5-a3e7-f7d601786b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:59.812340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fd9dc-d9bf-4ba5-a3e7-f7d601786b65 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:59.813286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fd9dc-d9bf-4ba5-a3e7-f7d601786b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:59.813527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fd9dc-d9bf-4ba5-a3e7-f7d601786b65 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:59.814202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fd9dc-d9bf-4ba5-a3e7-f7d601786b65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:10:59.814543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fd9dc-d9bf-4ba5-a3e7-f7d601786b65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:59.816271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fd9dc-d9bf-4ba5-a3e7-f7d601786b65 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:59.833674 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3100/6129] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:10:59.834950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabda114-8653-4c7f-b4ed-3a2aa0d324e0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:59.855430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabda114-8653-4c7f-b4ed-3a2aa0d324e0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:59.861138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617867e6-6c13-4c44-9724-aa1b4a6ab12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:59.861768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617867e6-6c13-4c44-9724-aa1b4a6ab12d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:59.862088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617867e6-6c13-4c44-9724-aa1b4a6ab12d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:59.862371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617867e6-6c13-4c44-9724-aa1b4a6ab12d admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:59.863013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617867e6-6c13-4c44-9724-aa1b4a6ab12d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:10:59.864978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617867e6-6c13-4c44-9724-aa1b4a6ab12d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:10:59.902159 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3030/6130] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:10:59 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:10:59.907126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fd9dc-d9bf-4ba5-a3e7-f7d601786b65 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:10:59.919432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fd9dc-d9bf-4ba5-a3e7-f7d601786b65 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:10:59.929752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:10:59.996440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a4cb3-f0bb-43bb-b6c7-a597dd47e468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:10:59.997712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a4cb3-f0bb-43bb-b6c7-a597dd47e468 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:10:59.998027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a4cb3-f0bb-43bb-b6c7-a597dd47e468 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:10:59.998561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a4cb3-f0bb-43bb-b6c7-a597dd47e468 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:10:59.999399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a4cb3-f0bb-43bb-b6c7-a597dd47e468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:00.005108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a4cb3-f0bb-43bb-b6c7-a597dd47e468 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:00.036659 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3101/6131] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:10:59 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:11:00.037763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617867e6-6c13-4c44-9724-aa1b4a6ab12d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:00.051734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617867e6-6c13-4c44-9724-aa1b4a6ab12d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:00.056452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74211b53-88fb-4d93-9f8c-1081fe391804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:00.057145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74211b53-88fb-4d93-9f8c-1081fe391804 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:00.057424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74211b53-88fb-4d93-9f8c-1081fe391804 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:00.057724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74211b53-88fb-4d93-9f8c-1081fe391804 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:00.058543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74211b53-88fb-4d93-9f8c-1081fe391804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:00.060310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74211b53-88fb-4d93-9f8c-1081fe391804 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:00.074472 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3031/6132] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:00.078337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a4cb3-f0bb-43bb-b6c7-a597dd47e468 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:00.094586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a4cb3-f0bb-43bb-b6c7-a597dd47e468 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:00.100983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34564f6-b856-42ea-bf04-5e2c625323f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:00.102103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34564f6-b856-42ea-bf04-5e2c625323f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:00.102398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34564f6-b856-42ea-bf04-5e2c625323f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:00.102685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34564f6-b856-42ea-bf04-5e2c625323f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:00.103406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34564f6-b856-42ea-bf04-5e2c625323f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:00.105091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34564f6-b856-42ea-bf04-5e2c625323f7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:00.124961 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3102/6133] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:00.128250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74211b53-88fb-4d93-9f8c-1081fe391804 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:00.141348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74211b53-88fb-4d93-9f8c-1081fe391804 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:00.146761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0691eea-2f91-4966-b51d-6837501fcadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:00.147722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0691eea-2f91-4966-b51d-6837501fcadb admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:00.148061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0691eea-2f91-4966-b51d-6837501fcadb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:00.148343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0691eea-2f91-4966-b51d-6837501fcadb admin admin] PATH_INFO: `/v3/projects/cf49e825b134472091ca4c254dcf8972/users/19eb56414375436ca59d8798fc251606/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:00.169497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0691eea-2f91-4966-b51d-6837501fcadb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf49e825b134472091ca4c254dcf8972, user_id=19eb56414375436ca59d8798fc251606, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:00.174471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0691eea-2f91-4966-b51d-6837501fcadb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:00.188442 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3032/6134] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:00 2022] PUT /identity/v3/projects/cf49e825b134472091ca4c254dcf8972/users/19eb56414375436ca59d8798fc251606/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:00.199933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036a79b-b88a-4d60-bace-e35afbb964d5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:00.200456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036a79b-b88a-4d60-bace-e35afbb964d5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:00.201252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036a79b-b88a-4d60-bace-e35afbb964d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:00.239979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1036a79b-b88a-4d60-bace-e35afbb964d5 None None] MFA Rules not processed for user `19eb56414375436ca59d8798fc251606`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:00.241196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1036a79b-b88a-4d60-bace-e35afbb964d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:00.338112 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3103/6135] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:00.349574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0691eea-2f91-4966-b51d-6837501fcadb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:00.353112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f0691eea-2f91-4966-b51d-6837501fcadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:00.432724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0691eea-2f91-4966-b51d-6837501fcadb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:00.436852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb1096-5938-417d-9f10-7ac211e9730a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:00.437677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb1096-5938-417d-9f10-7ac211e9730a service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:00.437980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb1096-5938-417d-9f10-7ac211e9730a service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:00.438256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb1096-5938-417d-9f10-7ac211e9730a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:00.453837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb1096-5938-417d-9f10-7ac211e9730a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:00.455805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb1096-5938-417d-9f10-7ac211e9730a service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:00.469888 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3033/6136] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:02.791197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfaaff-1057-401a-80a4-73328bebb3b2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:02.791697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfaaff-1057-401a-80a4-73328bebb3b2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:02.791797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfaaff-1057-401a-80a4-73328bebb3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:02.818971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-63bfaaff-1057-401a-80a4-73328bebb3b2 None None] MFA Rules not processed for user `3e79edf1a33e45ddac92ba8cd8429d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:02.820695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-63bfaaff-1057-401a-80a4-73328bebb3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:02.912461 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3104/6137] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:04.213766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff623273-c3b7-4e55-82d6-dd2b9d2d8798 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:04.215340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff623273-c3b7-4e55-82d6-dd2b9d2d8798 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:04.217875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff623273-c3b7-4e55-82d6-dd2b9d2d8798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:04.243131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ff623273-c3b7-4e55-82d6-dd2b9d2d8798 None None] MFA Rules not processed for user `d485394bc49e4380b152d32d09440281`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:04.244794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ff623273-c3b7-4e55-82d6-dd2b9d2d8798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:04.308227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:04.338137 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3034/6138] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:11.402573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfaaff-1057-401a-80a4-73328bebb3b2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:11.404019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-63bfaaff-1057-401a-80a4-73328bebb3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:11.433013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:11.489438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfaaff-1057-401a-80a4-73328bebb3b2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:11.494402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51928375-816a-4f0b-adaa-7316e99a00e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:11.495013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51928375-816a-4f0b-adaa-7316e99a00e0 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:11.495309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51928375-816a-4f0b-adaa-7316e99a00e0 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:11.495579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51928375-816a-4f0b-adaa-7316e99a00e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:11.497325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-51928375-816a-4f0b-adaa-7316e99a00e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:11.512540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51928375-816a-4f0b-adaa-7316e99a00e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:11.514195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51928375-816a-4f0b-adaa-7316e99a00e0 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:11.526696 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3105/6139] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:11.897284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff623273-c3b7-4e55-82d6-dd2b9d2d8798 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:11.969746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff623273-c3b7-4e55-82d6-dd2b9d2d8798 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:11.975476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:11.976723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:11.977831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:11.978280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] PATH_INFO: `/v3/users/19eb56414375436ca59d8798fc251606` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:11.979654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] RBAC: Authorizing `identity:delete_user(user_id=19eb56414375436ca59d8798fc251606)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:11.982094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:12.033549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19eb56414375436ca59d8798fc251606'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:12.045563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19eb56414375436ca59d8798fc251606'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:12.046237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19eb56414375436ca59d8798fc251606'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:12.053863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19eb56414375436ca59d8798fc251606'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:12.061553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19eb56414375436ca59d8798fc251606'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:12.067915 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3035/6140] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:11:11 2022] DELETE /identity/v3/users/19eb56414375436ca59d8798fc251606 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:12.072563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd62e1-f640-4482-9780-608aae0d2770 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:12.072983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd62e1-f640-4482-9780-608aae0d2770 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:12.073285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd62e1-f640-4482-9780-608aae0d2770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:12.093401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ecdd62e1-f640-4482-9780-608aae0d2770 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:12.095159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ecdd62e1-f640-4482-9780-608aae0d2770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:12.154660 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3106/6141] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:12.165243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:12.166725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:12.228397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411071c-1f32-49f3-8079-ae7a152f061f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:12.232135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf292da2-8fc3-447b-a2e5-c6230b492361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:12.232745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf292da2-8fc3-447b-a2e5-c6230b492361 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:12.233077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf292da2-8fc3-447b-a2e5-c6230b492361 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:12.233358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf292da2-8fc3-447b-a2e5-c6230b492361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:12.243269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf292da2-8fc3-447b-a2e5-c6230b492361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:12.244715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf292da2-8fc3-447b-a2e5-c6230b492361 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:12.254698 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3036/6142] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:12.440459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd62e1-f640-4482-9780-608aae0d2770 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:12.442058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ecdd62e1-f640-4482-9780-608aae0d2770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:12.455474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd62e1-f640-4482-9780-608aae0d2770 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:12.460214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:12.461142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:12.461434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:12.461715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd admin admin] PATH_INFO: `/v3/projects/cf49e825b134472091ca4c254dcf8972` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:12.462886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf49e825b134472091ca4c254dcf8972)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:12.465491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:12.521979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf49e825b134472091ca4c254dcf8972', 'request_id': 'req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:12.530420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf49e825b134472091ca4c254dcf8972', 'request_id': 'req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:12.537283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf49e825b134472091ca4c254dcf8972', 'request_id': 'req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:12.547855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd admin admin] The token cache is being invalidate because project cf49e825b134472091ca4c254dcf8972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:11:12.548147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:12.548812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f00f8811-9ca8-4529-9ad2-bb0ade3e25cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:12.550350 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3107/6143] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:11:12 2022] DELETE /identity/v3/projects/cf49e825b134472091ca4c254dcf8972 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:12.555488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dc854-c12d-4671-9c2f-53cd674053ac None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:12.555488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dc854-c12d-4671-9c2f-53cd674053ac None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:12.555686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dc854-c12d-4671-9c2f-53cd674053ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:12.571117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:12.578528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2b6dc854-c12d-4671-9c2f-53cd674053ac None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:12.581186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2b6dc854-c12d-4671-9c2f-53cd674053ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:12.646985 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3037/6144] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:12.654427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51fb35-7f84-4473-9a61-4686343df95d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:12.654714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51fb35-7f84-4473-9a61-4686343df95d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:12.654991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51fb35-7f84-4473-9a61-4686343df95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:12.674502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4b51fb35-7f84-4473-9a61-4686343df95d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:12.675721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4b51fb35-7f84-4473-9a61-4686343df95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:12.683731 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3108/6145] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:12.688387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6dc854-c12d-4671-9c2f-53cd674053ac None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:12.698709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6dc854-c12d-4671-9c2f-53cd674053ac None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:12.703948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ded9e-507a-4d51-8ab9-5ef179052fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:12.704564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ded9e-507a-4d51-8ab9-5ef179052fff admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:12.704848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ded9e-507a-4d51-8ab9-5ef179052fff admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:12.705156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ded9e-507a-4d51-8ab9-5ef179052fff admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:12.705798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ded9e-507a-4d51-8ab9-5ef179052fff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:11:12.706100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ded9e-507a-4d51-8ab9-5ef179052fff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:12.707689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ded9e-507a-4d51-8ab9-5ef179052fff admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:12.719330 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3038/6146] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:11:12.723372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51fb35-7f84-4473-9a61-4686343df95d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:12.732883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51fb35-7f84-4473-9a61-4686343df95d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:12.737549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ed26d-0511-4c23-98f6-a2cf277c5739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:12.738138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ed26d-0511-4c23-98f6-a2cf277c5739 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:12.738422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ed26d-0511-4c23-98f6-a2cf277c5739 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:12.738694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ed26d-0511-4c23-98f6-a2cf277c5739 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:12.739324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ed26d-0511-4c23-98f6-a2cf277c5739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:12.741275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ed26d-0511-4c23-98f6-a2cf277c5739 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:12.772270 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3109/6147] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:11:12 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:12.773375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ded9e-507a-4d51-8ab9-5ef179052fff admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:12.783539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ded9e-507a-4d51-8ab9-5ef179052fff admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:12.847021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c624cba-b8b8-4c0d-90d1-2fda005c6c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:12.847657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c624cba-b8b8-4c0d-90d1-2fda005c6c80 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:12.847946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c624cba-b8b8-4c0d-90d1-2fda005c6c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:12.848222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c624cba-b8b8-4c0d-90d1-2fda005c6c80 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:12.848855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c624cba-b8b8-4c0d-90d1-2fda005c6c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:12.850945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c624cba-b8b8-4c0d-90d1-2fda005c6c80 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:12.882617 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3039/6148] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:11:12 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:11:12.886821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ed26d-0511-4c23-98f6-a2cf277c5739 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:12.901038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ed26d-0511-4c23-98f6-a2cf277c5739 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:12.918530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf22a8a-e37a-4034-89b6-dd9d656e7f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:12.919165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf22a8a-e37a-4034-89b6-dd9d656e7f7a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:12.919456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf22a8a-e37a-4034-89b6-dd9d656e7f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:12.919978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf22a8a-e37a-4034-89b6-dd9d656e7f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:12.920716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf22a8a-e37a-4034-89b6-dd9d656e7f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:12.926622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf22a8a-e37a-4034-89b6-dd9d656e7f7a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:12.944119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:12.964297 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3110/6149] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:12.968658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c624cba-b8b8-4c0d-90d1-2fda005c6c80 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:12.985705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c624cba-b8b8-4c0d-90d1-2fda005c6c80 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:12.991991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aceb150-fd1c-48f9-b779-2757924c4e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:12.992701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aceb150-fd1c-48f9-b779-2757924c4e7f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:12.993088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aceb150-fd1c-48f9-b779-2757924c4e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:12.993438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aceb150-fd1c-48f9-b779-2757924c4e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:12.994194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aceb150-fd1c-48f9-b779-2757924c4e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:12.996226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aceb150-fd1c-48f9-b779-2757924c4e7f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:13.011311 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3040/6150] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:13.016581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf22a8a-e37a-4034-89b6-dd9d656e7f7a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:13.037906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf22a8a-e37a-4034-89b6-dd9d656e7f7a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:13.051142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a9434-6c6a-4891-becb-31683f102e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:13.051844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a9434-6c6a-4891-becb-31683f102e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:13.052136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a9434-6c6a-4891-becb-31683f102e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:13.052426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a9434-6c6a-4891-becb-31683f102e13 admin admin] PATH_INFO: `/v3/projects/9bc6c6f6fc1b41c2bc69d0949bb2e037/users/5650d7db57274de9a3fca3c9c10774cd/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:13.072232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a9434-6c6a-4891-becb-31683f102e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc6c6f6fc1b41c2bc69d0949bb2e037, user_id=5650d7db57274de9a3fca3c9c10774cd, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:13.076625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a9434-6c6a-4891-becb-31683f102e13 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:13.102617 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3111/6151] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:13 2022] PUT /identity/v3/projects/9bc6c6f6fc1b41c2bc69d0949bb2e037/users/5650d7db57274de9a3fca3c9c10774cd/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:11:13.114659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d5f17-4538-48dd-bb01-6f5a1cdb73aa None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:13.114867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d5f17-4538-48dd-bb01-6f5a1cdb73aa None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:13.117388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d5f17-4538-48dd-bb01-6f5a1cdb73aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:13.153546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d18d5f17-4538-48dd-bb01-6f5a1cdb73aa None None] MFA Rules not processed for user `5650d7db57274de9a3fca3c9c10774cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:13.154898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d18d5f17-4538-48dd-bb01-6f5a1cdb73aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:13.241718 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3041/6152] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:13.267482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a9434-6c6a-4891-becb-31683f102e13 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:13.269623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bb6a9434-6c6a-4891-becb-31683f102e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:13.345077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a9434-6c6a-4891-becb-31683f102e13 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:13.349216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2000f1-2fde-4ab8-9b63-6800a34387ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:13.350000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2000f1-2fde-4ab8-9b63-6800a34387ea service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:13.350296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2000f1-2fde-4ab8-9b63-6800a34387ea service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:13.350855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2000f1-2fde-4ab8-9b63-6800a34387ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:13.363709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2000f1-2fde-4ab8-9b63-6800a34387ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:13.366231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2000f1-2fde-4ab8-9b63-6800a34387ea service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:13.378102 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3112/6153] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:11:23.237285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:23.373630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:27.058135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e262922-ae29-4efa-90d3-0b06b18f6a27 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:27.058678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e262922-ae29-4efa-90d3-0b06b18f6a27 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:27.058678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e262922-ae29-4efa-90d3-0b06b18f6a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:27.089243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6e262922-ae29-4efa-90d3-0b06b18f6a27 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:27.090466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6e262922-ae29-4efa-90d3-0b06b18f6a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:27.173069 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3042/6154] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:11:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:27.232465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2000f1-2fde-4ab8-9b63-6800a34387ea service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:27.233726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4b2000f1-2fde-4ab8-9b63-6800a34387ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:27.325587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2000f1-2fde-4ab8-9b63-6800a34387ea service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:27.332137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795ec4c-6208-46ca-b790-6839bd3fb459 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:27.332866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795ec4c-6208-46ca-b790-6839bd3fb459 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:27.333359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795ec4c-6208-46ca-b790-6839bd3fb459 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:27.333717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795ec4c-6208-46ca-b790-6839bd3fb459 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:27.343475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795ec4c-6208-46ca-b790-6839bd3fb459 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:27.345828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795ec4c-6208-46ca-b790-6839bd3fb459 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:27.361550 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3113/6155] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:11:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:11:32.762673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e262922-ae29-4efa-90d3-0b06b18f6a27 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:32.863637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e262922-ae29-4efa-90d3-0b06b18f6a27 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:32.868303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:32.868992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:32.869284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:32.869566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] PATH_INFO: `/v3/users/5650d7db57274de9a3fca3c9c10774cd` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:32.872805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5650d7db57274de9a3fca3c9c10774cd)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:32.873042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:32.934994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5650d7db57274de9a3fca3c9c10774cd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:32.965271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5650d7db57274de9a3fca3c9c10774cd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:32.966465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5650d7db57274de9a3fca3c9c10774cd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:32.985259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5650d7db57274de9a3fca3c9c10774cd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:32.995649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5650d7db57274de9a3fca3c9c10774cd'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:33.003827 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3043/6156] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:11:32 2022] DELETE /identity/v3/users/5650d7db57274de9a3fca3c9c10774cd => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:33.009880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d8e6c-c4d1-4a3c-b1fe-c449f65823a6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:33.010372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d8e6c-c4d1-4a3c-b1fe-c449f65823a6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:33.010952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d8e6c-c4d1-4a3c-b1fe-c449f65823a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:33.045909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d87d8e6c-c4d1-4a3c-b1fe-c449f65823a6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:33.053159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d87d8e6c-c4d1-4a3c-b1fe-c449f65823a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:33.127967 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3114/6157] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:33.138419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:33.141348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:33.153461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:33.228055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53272f-37eb-4e32-a4ed-94cd7176ea50 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:33.232161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb59df-4ce3-4a8e-910a-983d6af196dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:33.232785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb59df-4ce3-4a8e-910a-983d6af196dc service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:33.233174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb59df-4ce3-4a8e-910a-983d6af196dc service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:33.233455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb59df-4ce3-4a8e-910a-983d6af196dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:33.245379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb59df-4ce3-4a8e-910a-983d6af196dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:33.246893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb59df-4ce3-4a8e-910a-983d6af196dc service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:33.259173 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3044/6158] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:33.470551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d8e6c-c4d1-4a3c-b1fe-c449f65823a6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:33.472086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d87d8e6c-c4d1-4a3c-b1fe-c449f65823a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:33.490631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d8e6c-c4d1-4a3c-b1fe-c449f65823a6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:33.495757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:33.498679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:33.498679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:33.498679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f admin admin] PATH_INFO: `/v3/projects/9bc6c6f6fc1b41c2bc69d0949bb2e037` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:33.501198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bc6c6f6fc1b41c2bc69d0949bb2e037)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:33.503280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:33.563979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bc6c6f6fc1b41c2bc69d0949bb2e037', 'request_id': 'req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:33.572110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bc6c6f6fc1b41c2bc69d0949bb2e037', 'request_id': 'req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:33.579874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bc6c6f6fc1b41c2bc69d0949bb2e037', 'request_id': 'req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:33.599574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f admin admin] The token cache is being invalidate because project 9bc6c6f6fc1b41c2bc69d0949bb2e037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:11:33.600367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:33.601218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-64aa27a6-c58f-43dc-bb7f-b2008ed5a64f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:33.603423 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3115/6159] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:11:33 2022] DELETE /identity/v3/projects/9bc6c6f6fc1b41c2bc69d0949bb2e037 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:33.608687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a4e9e-b6c3-4ad8-bccf-b1a28df731f1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:33.609605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a4e9e-b6c3-4ad8-bccf-b1a28df731f1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:33.609887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a4e9e-b6c3-4ad8-bccf-b1a28df731f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:33.635168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-db8a4e9e-b6c3-4ad8-bccf-b1a28df731f1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:33.636434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-db8a4e9e-b6c3-4ad8-bccf-b1a28df731f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:33.707466 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3045/6160] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:33.715530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7f2d9-af92-4f4f-a438-23d2627d9cab None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:33.715996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7f2d9-af92-4f4f-a438-23d2627d9cab None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:33.716112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7f2d9-af92-4f4f-a438-23d2627d9cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:33.765536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-18b7f2d9-af92-4f4f-a438-23d2627d9cab None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:33.766898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-18b7f2d9-af92-4f4f-a438-23d2627d9cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:33.776473 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3116/6161] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:33.781319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a4e9e-b6c3-4ad8-bccf-b1a28df731f1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:33.793534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a4e9e-b6c3-4ad8-bccf-b1a28df731f1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:33.799316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4179ff3-e4dd-4b25-be84-95c6c6b1f201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:33.799930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4179ff3-e4dd-4b25-be84-95c6c6b1f201 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:33.800214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4179ff3-e4dd-4b25-be84-95c6c6b1f201 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:33.800490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4179ff3-e4dd-4b25-be84-95c6c6b1f201 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:33.801167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4179ff3-e4dd-4b25-be84-95c6c6b1f201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:11:33.802052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4179ff3-e4dd-4b25-be84-95c6c6b1f201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:33.803126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4179ff3-e4dd-4b25-be84-95c6c6b1f201 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:33.816966 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3046/6162] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:11:33.819822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7f2d9-af92-4f4f-a438-23d2627d9cab None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:33.829680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7f2d9-af92-4f4f-a438-23d2627d9cab None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:33.834407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c51a89-fb76-4da5-b316-2c35030d0699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:33.835023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c51a89-fb76-4da5-b316-2c35030d0699 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:33.835310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c51a89-fb76-4da5-b316-2c35030d0699 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:33.835616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c51a89-fb76-4da5-b316-2c35030d0699 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:33.836282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c51a89-fb76-4da5-b316-2c35030d0699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:33.838269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c51a89-fb76-4da5-b316-2c35030d0699 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:33.843907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:33.884017 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3117/6163] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:11:33 2022] POST /identity/v3/projects => generated 361 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:33.886223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4179ff3-e4dd-4b25-be84-95c6c6b1f201 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:33.898596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4179ff3-e4dd-4b25-be84-95c6c6b1f201 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:33.987896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2dc16-003e-41d7-87d3-ddd410386203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:33.988779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2dc16-003e-41d7-87d3-ddd410386203 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:33.989109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2dc16-003e-41d7-87d3-ddd410386203 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:33.989392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2dc16-003e-41d7-87d3-ddd410386203 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:33.990047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2dc16-003e-41d7-87d3-ddd410386203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:33.992214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2dc16-003e-41d7-87d3-ddd410386203 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:34.017024 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3047/6164] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:11:33 2022] POST /identity/v3/users => generated 281 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:34.021914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c51a89-fb76-4da5-b316-2c35030d0699 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:34.032863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c51a89-fb76-4da5-b316-2c35030d0699 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:34.038681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110b56c-26dd-455c-ba68-391f8a90685a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:34.039364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110b56c-26dd-455c-ba68-391f8a90685a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:34.039678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110b56c-26dd-455c-ba68-391f8a90685a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:34.040001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110b56c-26dd-455c-ba68-391f8a90685a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:34.040731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110b56c-26dd-455c-ba68-391f8a90685a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:34.042493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110b56c-26dd-455c-ba68-391f8a90685a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:34.057781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2dc16-003e-41d7-87d3-ddd410386203 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:34.058202 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3118/6165] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:11:34.069252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2dc16-003e-41d7-87d3-ddd410386203 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:34.076808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57ed5f-6b0c-40d2-a215-da8a28c01572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:34.078333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ed5f-6b0c-40d2-a215-da8a28c01572 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:34.078891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ed5f-6b0c-40d2-a215-da8a28c01572 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:34.079682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ed5f-6b0c-40d2-a215-da8a28c01572 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:34.080501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57ed5f-6b0c-40d2-a215-da8a28c01572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:34.082336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57ed5f-6b0c-40d2-a215-da8a28c01572 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:34.096024 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3048/6166] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:34.100459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110b56c-26dd-455c-ba68-391f8a90685a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:34.112469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110b56c-26dd-455c-ba68-391f8a90685a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:34.122168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21105caa-3e61-4464-b171-6146b5562d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:34.125179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21105caa-3e61-4464-b171-6146b5562d3c admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:34.125522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21105caa-3e61-4464-b171-6146b5562d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:34.125809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21105caa-3e61-4464-b171-6146b5562d3c admin admin] PATH_INFO: `/v3/projects/20bd0bc8f18b4e2a88444d7891ab5359/users/dbdad3d6d5c842c1a0c226b08635fdc8/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:34.159038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21105caa-3e61-4464-b171-6146b5562d3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=20bd0bc8f18b4e2a88444d7891ab5359, user_id=dbdad3d6d5c842c1a0c226b08635fdc8, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:34.161847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21105caa-3e61-4464-b171-6146b5562d3c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:34.178448 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3119/6167] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:34 2022] PUT /identity/v3/projects/20bd0bc8f18b4e2a88444d7891ab5359/users/dbdad3d6d5c842c1a0c226b08635fdc8/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:34.190671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95913c8-a736-4824-84cd-3f6284ee9237 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:34.190963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95913c8-a736-4824-84cd-3f6284ee9237 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:34.191243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95913c8-a736-4824-84cd-3f6284ee9237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:34.223167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e95913c8-a736-4824-84cd-3f6284ee9237 None None] MFA Rules not processed for user `dbdad3d6d5c842c1a0c226b08635fdc8`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:34.224684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e95913c8-a736-4824-84cd-3f6284ee9237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:34.231067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:34.318576 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3049/6168] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:34.325417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21105caa-3e61-4464-b171-6146b5562d3c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:34.327300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-21105caa-3e61-4464-b171-6146b5562d3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:34.406926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21105caa-3e61-4464-b171-6146b5562d3c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:34.410780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0058840-a475-4f2f-be58-7d3c4d4c1680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:34.411406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058840-a475-4f2f-be58-7d3c4d4c1680 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:34.411683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058840-a475-4f2f-be58-7d3c4d4c1680 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:34.411961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058840-a475-4f2f-be58-7d3c4d4c1680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:34.422133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0058840-a475-4f2f-be58-7d3c4d4c1680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:34.423597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0058840-a475-4f2f-be58-7d3c4d4c1680 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:34.433031 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3120/6169] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:41.961257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95913c8-a736-4824-84cd-3f6284ee9237 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:42.023847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95913c8-a736-4824-84cd-3f6284ee9237 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:42.028387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:42.029043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:42.029432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:42.029717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] PATH_INFO: `/v3/users/dbdad3d6d5c842c1a0c226b08635fdc8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:42.030876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbdad3d6d5c842c1a0c226b08635fdc8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:42.032796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:42.078252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbdad3d6d5c842c1a0c226b08635fdc8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:42.090371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbdad3d6d5c842c1a0c226b08635fdc8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:42.091045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbdad3d6d5c842c1a0c226b08635fdc8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:42.098073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbdad3d6d5c842c1a0c226b08635fdc8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:42.105804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbdad3d6d5c842c1a0c226b08635fdc8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:42.111785 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3050/6170] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:11:41 2022] DELETE /identity/v3/users/dbdad3d6d5c842c1a0c226b08635fdc8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:42.116292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f211be-def5-4d76-8989-8daafaecf204 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:42.116803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f211be-def5-4d76-8989-8daafaecf204 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:42.117027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f211be-def5-4d76-8989-8daafaecf204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:42.137099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d6f211be-def5-4d76-8989-8daafaecf204 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:42.138306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d6f211be-def5-4d76-8989-8daafaecf204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:42.196262 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3121/6171] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:42.205907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:42.207307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:42.235764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:42.269214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425fd12-6ed2-4fad-bee3-c73e2dbf9622 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:42.272858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911dfb9-4451-4455-9285-51a8073b3a21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:42.273495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911dfb9-4451-4455-9285-51a8073b3a21 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:42.273781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911dfb9-4451-4455-9285-51a8073b3a21 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:42.274060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911dfb9-4451-4455-9285-51a8073b3a21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:42.283010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911dfb9-4451-4455-9285-51a8073b3a21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:42.284439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911dfb9-4451-4455-9285-51a8073b3a21 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:42.293918 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3051/6172] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:42.460963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f211be-def5-4d76-8989-8daafaecf204 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:42.462410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d6f211be-def5-4d76-8989-8daafaecf204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:42.476106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f211be-def5-4d76-8989-8daafaecf204 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:42.481186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b607d-85f3-496d-ab98-2f4cc9686843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:42.482309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b607d-85f3-496d-ab98-2f4cc9686843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:42.482605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b607d-85f3-496d-ab98-2f4cc9686843 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:42.482884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b607d-85f3-496d-ab98-2f4cc9686843 admin admin] PATH_INFO: `/v3/projects/20bd0bc8f18b4e2a88444d7891ab5359` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:42.484681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b607d-85f3-496d-ab98-2f4cc9686843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20bd0bc8f18b4e2a88444d7891ab5359)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:42.486650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b607d-85f3-496d-ab98-2f4cc9686843 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:42.532503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:42.534101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d40b607d-85f3-496d-ab98-2f4cc9686843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20bd0bc8f18b4e2a88444d7891ab5359', 'request_id': 'req-d40b607d-85f3-496d-ab98-2f4cc9686843'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:42.542319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d40b607d-85f3-496d-ab98-2f4cc9686843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20bd0bc8f18b4e2a88444d7891ab5359', 'request_id': 'req-d40b607d-85f3-496d-ab98-2f4cc9686843'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:42.549655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d40b607d-85f3-496d-ab98-2f4cc9686843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20bd0bc8f18b4e2a88444d7891ab5359', 'request_id': 'req-d40b607d-85f3-496d-ab98-2f4cc9686843'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:42.559871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d40b607d-85f3-496d-ab98-2f4cc9686843 admin admin] The token cache is being invalidate because project 20bd0bc8f18b4e2a88444d7891ab5359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:11:42.560156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d40b607d-85f3-496d-ab98-2f4cc9686843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:42.560784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d40b607d-85f3-496d-ab98-2f4cc9686843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:42.562545 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3122/6173] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:11:42 2022] DELETE /identity/v3/projects/20bd0bc8f18b4e2a88444d7891ab5359 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:42.567337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00da41-1954-4a37-befe-2bc303445bad None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:42.567671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00da41-1954-4a37-befe-2bc303445bad None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:42.568394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00da41-1954-4a37-befe-2bc303445bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:42.588420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1c00da41-1954-4a37-befe-2bc303445bad None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:42.589678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1c00da41-1954-4a37-befe-2bc303445bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:42.669354 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3052/6174] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:42.677422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcc298-f459-4d54-850a-98dc590e988f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:42.677710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcc298-f459-4d54-850a-98dc590e988f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:42.677984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcc298-f459-4d54-850a-98dc590e988f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:42.698773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c7bcc298-f459-4d54-850a-98dc590e988f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:42.700411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c7bcc298-f459-4d54-850a-98dc590e988f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:42.708407 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3123/6175] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:42.712510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00da41-1954-4a37-befe-2bc303445bad None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:42.722855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00da41-1954-4a37-befe-2bc303445bad None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:42.727979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74548c38-27dc-40ca-8e4b-e360e719c4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:42.728592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74548c38-27dc-40ca-8e4b-e360e719c4ee admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:42.728878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74548c38-27dc-40ca-8e4b-e360e719c4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:42.729188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74548c38-27dc-40ca-8e4b-e360e719c4ee admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:42.729852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74548c38-27dc-40ca-8e4b-e360e719c4ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:11:42.730144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74548c38-27dc-40ca-8e4b-e360e719c4ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:42.731737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74548c38-27dc-40ca-8e4b-e360e719c4ee admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:42.745246 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3053/6176] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:11:42.749642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcc298-f459-4d54-850a-98dc590e988f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:42.758984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcc298-f459-4d54-850a-98dc590e988f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:42.763706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64372a-f843-4b59-9661-2713c01bc789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:42.764300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64372a-f843-4b59-9661-2713c01bc789 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:42.764588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64372a-f843-4b59-9661-2713c01bc789 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:42.764889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64372a-f843-4b59-9661-2713c01bc789 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:42.765562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64372a-f843-4b59-9661-2713c01bc789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:42.767474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64372a-f843-4b59-9661-2713c01bc789 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:42.794125 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3124/6177] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:11:42 2022] POST /identity/v3/projects => generated 351 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:42.798846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74548c38-27dc-40ca-8e4b-e360e719c4ee admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:42.808984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74548c38-27dc-40ca-8e4b-e360e719c4ee admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:42.864819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76fc53-44ce-490a-bb35-4c11b7a666cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:42.865383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76fc53-44ce-490a-bb35-4c11b7a666cd admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:42.865669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76fc53-44ce-490a-bb35-4c11b7a666cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:42.865948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76fc53-44ce-490a-bb35-4c11b7a666cd admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:42.866581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76fc53-44ce-490a-bb35-4c11b7a666cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:42.868531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76fc53-44ce-490a-bb35-4c11b7a666cd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:42.892133 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3054/6178] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:11:42 2022] POST /identity/v3/users => generated 276 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:42.893457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64372a-f843-4b59-9661-2713c01bc789 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:42.904249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64372a-f843-4b59-9661-2713c01bc789 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:42.909520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01efa4e-c7d1-45d0-8046-34cd8d80bb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:42.910150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01efa4e-c7d1-45d0-8046-34cd8d80bb17 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:42.910441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01efa4e-c7d1-45d0-8046-34cd8d80bb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:42.910865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01efa4e-c7d1-45d0-8046-34cd8d80bb17 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:42.911493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01efa4e-c7d1-45d0-8046-34cd8d80bb17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:42.913126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01efa4e-c7d1-45d0-8046-34cd8d80bb17 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:42.923084 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3125/6179] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:42.927042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76fc53-44ce-490a-bb35-4c11b7a666cd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:42.937486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76fc53-44ce-490a-bb35-4c11b7a666cd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:42.942557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b22af4-c9be-4688-baa4-2e0dba35ffce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:42.943167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b22af4-c9be-4688-baa4-2e0dba35ffce admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:42.943454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b22af4-c9be-4688-baa4-2e0dba35ffce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:42.943736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b22af4-c9be-4688-baa4-2e0dba35ffce admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:42.944456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b22af4-c9be-4688-baa4-2e0dba35ffce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:42.946086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b22af4-c9be-4688-baa4-2e0dba35ffce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:42.957774 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3055/6180] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:42.962235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01efa4e-c7d1-45d0-8046-34cd8d80bb17 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:42.973354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01efa4e-c7d1-45d0-8046-34cd8d80bb17 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:42.978326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97f5d4-2296-4c6f-b1d6-8dfce6e87955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:42.978978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97f5d4-2296-4c6f-b1d6-8dfce6e87955 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:42.979268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97f5d4-2296-4c6f-b1d6-8dfce6e87955 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:42.979548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97f5d4-2296-4c6f-b1d6-8dfce6e87955 admin admin] PATH_INFO: `/v3/projects/001d8c3437b041998caf7320096e40bf/users/59ec999015f345e7b542c141a66037fa/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:42.995060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97f5d4-2296-4c6f-b1d6-8dfce6e87955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=001d8c3437b041998caf7320096e40bf, user_id=59ec999015f345e7b542c141a66037fa, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:42.997833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97f5d4-2296-4c6f-b1d6-8dfce6e87955 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:43.010397 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3126/6181] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:42 2022] PUT /identity/v3/projects/001d8c3437b041998caf7320096e40bf/users/59ec999015f345e7b542c141a66037fa/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:11:43.021271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3a37e-341e-4eb4-ad2e-757c9b76f1be None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:43.021800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3a37e-341e-4eb4-ad2e-757c9b76f1be None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:43.021942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3a37e-341e-4eb4-ad2e-757c9b76f1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:43.049880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-95e3a37e-341e-4eb4-ad2e-757c9b76f1be None None] MFA Rules not processed for user `59ec999015f345e7b542c141a66037fa`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:43.051097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-95e3a37e-341e-4eb4-ad2e-757c9b76f1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:43.071397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:43.117617 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3056/6182] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:43.127660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97f5d4-2296-4c6f-b1d6-8dfce6e87955 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:43.129597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5e97f5d4-2296-4c6f-b1d6-8dfce6e87955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:43.188212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97f5d4-2296-4c6f-b1d6-8dfce6e87955 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:43.191917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29883cf9-dede-4101-aff3-5d399b42788e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:43.192521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29883cf9-dede-4101-aff3-5d399b42788e service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:43.192811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29883cf9-dede-4101-aff3-5d399b42788e service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:43.193120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29883cf9-dede-4101-aff3-5d399b42788e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:43.201407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29883cf9-dede-4101-aff3-5d399b42788e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:43.203346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29883cf9-dede-4101-aff3-5d399b42788e service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:43.212194 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3127/6183] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:44.739462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3a37e-341e-4eb4-ad2e-757c9b76f1be None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:44.815640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3a37e-341e-4eb4-ad2e-757c9b76f1be None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:44.820120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:44.820800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:44.821113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:44.821503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] PATH_INFO: `/v3/users/59ec999015f345e7b542c141a66037fa` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:44.822629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59ec999015f345e7b542c141a66037fa)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:44.824551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:44.870344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59ec999015f345e7b542c141a66037fa'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:44.882202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59ec999015f345e7b542c141a66037fa'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:44.882882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59ec999015f345e7b542c141a66037fa'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:44.889949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59ec999015f345e7b542c141a66037fa'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:44.896990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59ec999015f345e7b542c141a66037fa'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:44.902698 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3057/6184] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:11:44 2022] DELETE /identity/v3/users/59ec999015f345e7b542c141a66037fa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:44.907672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147045c-0b89-4c00-9dcf-db3a946898d1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:44.908155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147045c-0b89-4c00-9dcf-db3a946898d1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:44.908308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147045c-0b89-4c00-9dcf-db3a946898d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:44.927884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d147045c-0b89-4c00-9dcf-db3a946898d1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:44.929127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d147045c-0b89-4c00-9dcf-db3a946898d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:44.995461 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3128/6185] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:45.005175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:45.006596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:45.054063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:45.071092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de88f3-9462-4683-bd8c-5ec9b1edd913 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:45.074847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a776a5-cde5-4913-b0a3-1d16b2567843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:45.075466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a776a5-cde5-4913-b0a3-1d16b2567843 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:45.075754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a776a5-cde5-4913-b0a3-1d16b2567843 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:45.076038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a776a5-cde5-4913-b0a3-1d16b2567843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:45.085614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a776a5-cde5-4913-b0a3-1d16b2567843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:45.087050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a776a5-cde5-4913-b0a3-1d16b2567843 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:45.097393 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3058/6186] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:45.263032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147045c-0b89-4c00-9dcf-db3a946898d1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:45.264453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d147045c-0b89-4c00-9dcf-db3a946898d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:45.278364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147045c-0b89-4c00-9dcf-db3a946898d1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:45.283044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:45.284093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:45.284386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:45.284829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f admin admin] PATH_INFO: `/v3/projects/001d8c3437b041998caf7320096e40bf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:45.286016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=001d8c3437b041998caf7320096e40bf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:45.287921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:45.299659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:45.342127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '001d8c3437b041998caf7320096e40bf', 'request_id': 'req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:45.348928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '001d8c3437b041998caf7320096e40bf', 'request_id': 'req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:45.355238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '001d8c3437b041998caf7320096e40bf', 'request_id': 'req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:45.366049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f admin admin] The token cache is being invalidate because project 001d8c3437b041998caf7320096e40bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:11:45.366341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:45.366981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6d59bfa8-57ed-49b3-aa1a-b81313d85c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:45.368524 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3129/6187] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:11:45 2022] DELETE /identity/v3/projects/001d8c3437b041998caf7320096e40bf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:45.373526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90685e1e-c275-4911-9d70-300895437411 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:45.373814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90685e1e-c275-4911-9d70-300895437411 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:45.374174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90685e1e-c275-4911-9d70-300895437411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:45.394295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-90685e1e-c275-4911-9d70-300895437411 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:45.395506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-90685e1e-c275-4911-9d70-300895437411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:45.461454 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3059/6188] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:45.470483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0895f-c84c-4865-9c54-1464efc5cbac None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:45.470785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0895f-c84c-4865-9c54-1464efc5cbac None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:45.471070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0895f-c84c-4865-9c54-1464efc5cbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:45.491920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bfa0895f-c84c-4865-9c54-1464efc5cbac None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:45.493173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bfa0895f-c84c-4865-9c54-1464efc5cbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:45.502127 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3130/6189] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:45.506428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90685e1e-c275-4911-9d70-300895437411 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:45.516183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90685e1e-c275-4911-9d70-300895437411 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:45.521005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca70b12-277b-4075-94d5-86139ca5d8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:45.521614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca70b12-277b-4075-94d5-86139ca5d8dc admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:45.521904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca70b12-277b-4075-94d5-86139ca5d8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:45.522211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca70b12-277b-4075-94d5-86139ca5d8dc admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:45.522864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca70b12-277b-4075-94d5-86139ca5d8dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:11:45.523179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca70b12-277b-4075-94d5-86139ca5d8dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:45.524843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca70b12-277b-4075-94d5-86139ca5d8dc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:45.537137 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3060/6190] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:11:45.541358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0895f-c84c-4865-9c54-1464efc5cbac None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:45.551354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0895f-c84c-4865-9c54-1464efc5cbac None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:45.556089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e0825-6cac-41d4-a5d4-933c5cd891eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:45.556734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e0825-6cac-41d4-a5d4-933c5cd891eb admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:45.557096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e0825-6cac-41d4-a5d4-933c5cd891eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:45.557377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e0825-6cac-41d4-a5d4-933c5cd891eb admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:45.558009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e0825-6cac-41d4-a5d4-933c5cd891eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:45.559937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e0825-6cac-41d4-a5d4-933c5cd891eb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:45.587353 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3131/6191] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:11:45 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:45.591473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca70b12-277b-4075-94d5-86139ca5d8dc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:45.602359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca70b12-277b-4075-94d5-86139ca5d8dc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:45.657977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cd4c6-711c-40d5-a21d-634b29d2a049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:45.658575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cd4c6-711c-40d5-a21d-634b29d2a049 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:45.658861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cd4c6-711c-40d5-a21d-634b29d2a049 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:45.659139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cd4c6-711c-40d5-a21d-634b29d2a049 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:45.659767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1cd4c6-711c-40d5-a21d-634b29d2a049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:45.661723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1cd4c6-711c-40d5-a21d-634b29d2a049 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:45.681761 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3061/6192] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:11:45 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:45.685904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e0825-6cac-41d4-a5d4-933c5cd891eb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:45.696665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e0825-6cac-41d4-a5d4-933c5cd891eb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:45.701310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84c6ca-b5a6-4c7f-872a-e8a13dc04071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:45.701900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84c6ca-b5a6-4c7f-872a-e8a13dc04071 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:45.702280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84c6ca-b5a6-4c7f-872a-e8a13dc04071 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:45.703051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84c6ca-b5a6-4c7f-872a-e8a13dc04071 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:45.703689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84c6ca-b5a6-4c7f-872a-e8a13dc04071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:45.705337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84c6ca-b5a6-4c7f-872a-e8a13dc04071 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:45.715343 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3132/6193] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:45.719449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cd4c6-711c-40d5-a21d-634b29d2a049 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:45.729636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cd4c6-711c-40d5-a21d-634b29d2a049 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:45.734478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f350af-bf1a-48a7-8fb4-2256debae55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:45.735074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f350af-bf1a-48a7-8fb4-2256debae55d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:45.735355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f350af-bf1a-48a7-8fb4-2256debae55d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:45.735637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f350af-bf1a-48a7-8fb4-2256debae55d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:45.736276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f350af-bf1a-48a7-8fb4-2256debae55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:45.737892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f350af-bf1a-48a7-8fb4-2256debae55d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:45.749006 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3062/6194] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:45.753644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84c6ca-b5a6-4c7f-872a-e8a13dc04071 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:45.763884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84c6ca-b5a6-4c7f-872a-e8a13dc04071 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:45.769303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b8261-4cb8-43cd-8915-bcf2ccf40364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:45.770134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b8261-4cb8-43cd-8915-bcf2ccf40364 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:45.770418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b8261-4cb8-43cd-8915-bcf2ccf40364 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:45.770694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b8261-4cb8-43cd-8915-bcf2ccf40364 admin admin] PATH_INFO: `/v3/projects/5d3a007f1ae84fb78596c03d4ff871f8/users/0eb9f6e450144b1fbc7650fe9f088113/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:45.785898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b8261-4cb8-43cd-8915-bcf2ccf40364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d3a007f1ae84fb78596c03d4ff871f8, user_id=0eb9f6e450144b1fbc7650fe9f088113, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:45.788632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b8261-4cb8-43cd-8915-bcf2ccf40364 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:45.802505 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3133/6195] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:45 2022] PUT /identity/v3/projects/5d3a007f1ae84fb78596c03d4ff871f8/users/0eb9f6e450144b1fbc7650fe9f088113/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:45.813622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3477e59-28d7-4ad3-b999-92cf02e69fd8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:45.813917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3477e59-28d7-4ad3-b999-92cf02e69fd8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:45.814200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3477e59-28d7-4ad3-b999-92cf02e69fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:45.844696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d3477e59-28d7-4ad3-b999-92cf02e69fd8 None None] MFA Rules not processed for user `0eb9f6e450144b1fbc7650fe9f088113`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:45.845980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d3477e59-28d7-4ad3-b999-92cf02e69fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:45.885936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:45.911285 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3063/6196] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:45.915762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b8261-4cb8-43cd-8915-bcf2ccf40364 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:45.977488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b8261-4cb8-43cd-8915-bcf2ccf40364 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:45.982133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70425cb-a2d5-4fc8-b9a6-bfa773b69bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:45.982735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70425cb-a2d5-4fc8-b9a6-bfa773b69bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:45.983025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70425cb-a2d5-4fc8-b9a6-bfa773b69bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:45.983303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70425cb-a2d5-4fc8-b9a6-bfa773b69bf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:45.984428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70425cb-a2d5-4fc8-b9a6-bfa773b69bf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:45.986377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70425cb-a2d5-4fc8-b9a6-bfa773b69bf6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:46.012492 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3134/6197] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:11:45 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:46.016615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3477e59-28d7-4ad3-b999-92cf02e69fd8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:46.027533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3477e59-28d7-4ad3-b999-92cf02e69fd8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:46.084610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1c654-9836-4c6f-b7c5-2a7fd3ebe8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:46.085188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1c654-9836-4c6f-b7c5-2a7fd3ebe8ed admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:46.085477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1c654-9836-4c6f-b7c5-2a7fd3ebe8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:46.085759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1c654-9836-4c6f-b7c5-2a7fd3ebe8ed admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:46.086388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1c654-9836-4c6f-b7c5-2a7fd3ebe8ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:46.088305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1c654-9836-4c6f-b7c5-2a7fd3ebe8ed admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:46.111550 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3064/6198] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:11:46 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:11:46.115534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70425cb-a2d5-4fc8-b9a6-bfa773b69bf6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:46.134114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70425cb-a2d5-4fc8-b9a6-bfa773b69bf6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:46.139311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba242d-f0da-45c2-adc2-8235283483d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:46.139933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba242d-f0da-45c2-adc2-8235283483d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:46.140224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba242d-f0da-45c2-adc2-8235283483d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:46.140543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba242d-f0da-45c2-adc2-8235283483d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:46.141225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba242d-f0da-45c2-adc2-8235283483d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:46.142850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba242d-f0da-45c2-adc2-8235283483d9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:46.154266 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3135/6199] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:46.158522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1c654-9836-4c6f-b7c5-2a7fd3ebe8ed admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:46.168783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1c654-9836-4c6f-b7c5-2a7fd3ebe8ed admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:46.173995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ee7ab-a16d-4531-8c9c-b9fa1ba4c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:46.174659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ee7ab-a16d-4531-8c9c-b9fa1ba4c3ad admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:46.174946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ee7ab-a16d-4531-8c9c-b9fa1ba4c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:46.175231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ee7ab-a16d-4531-8c9c-b9fa1ba4c3ad admin admin] PATH_INFO: `/v3/projects/f611096ed30744c89bd9752180f6bf14/users/a9580ca7f8df4851b6cd06435d349b43/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:46.191366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ee7ab-a16d-4531-8c9c-b9fa1ba4c3ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f611096ed30744c89bd9752180f6bf14, user_id=a9580ca7f8df4851b6cd06435d349b43, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:46.194156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ee7ab-a16d-4531-8c9c-b9fa1ba4c3ad admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:46.207210 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3065/6200] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:46 2022] PUT /identity/v3/projects/f611096ed30744c89bd9752180f6bf14/users/a9580ca7f8df4851b6cd06435d349b43/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:46.211186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba242d-f0da-45c2-adc2-8235283483d9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:46.221811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba242d-f0da-45c2-adc2-8235283483d9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:46.227336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:46.284666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee73167-9f3e-4e89-8086-c265de53bf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:46.285374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee73167-9f3e-4e89-8086-c265de53bf93 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:46.285665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee73167-9f3e-4e89-8086-c265de53bf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:46.285949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee73167-9f3e-4e89-8086-c265de53bf93 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:46.286596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee73167-9f3e-4e89-8086-c265de53bf93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:46.288218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee73167-9f3e-4e89-8086-c265de53bf93 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:46.298320 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3136/6201] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:46.302395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ee7ab-a16d-4531-8c9c-b9fa1ba4c3ad admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:46.313610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ee7ab-a16d-4531-8c9c-b9fa1ba4c3ad admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:46.318414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e7809-53e1-4d77-a7ad-1ddc15d11437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:46.319067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e7809-53e1-4d77-a7ad-1ddc15d11437 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:46.319375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e7809-53e1-4d77-a7ad-1ddc15d11437 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:46.319659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e7809-53e1-4d77-a7ad-1ddc15d11437 admin admin] PATH_INFO: `/v3/projects/f611096ed30744c89bd9752180f6bf14/users/a9580ca7f8df4851b6cd06435d349b43/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:46.321630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e7809-53e1-4d77-a7ad-1ddc15d11437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f611096ed30744c89bd9752180f6bf14, user_id=a9580ca7f8df4851b6cd06435d349b43, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:46.324350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e7809-53e1-4d77-a7ad-1ddc15d11437 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:46.334972 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3066/6202] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:46 2022] PUT /identity/v3/projects/f611096ed30744c89bd9752180f6bf14/users/a9580ca7f8df4851b6cd06435d349b43/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:46.345193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ca830-8e88-4858-9ab2-125f6b9deeea None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:46.345594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ca830-8e88-4858-9ab2-125f6b9deeea None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:46.345915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ca830-8e88-4858-9ab2-125f6b9deeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:46.374737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8e7ca830-8e88-4858-9ab2-125f6b9deeea None None] MFA Rules not processed for user `a9580ca7f8df4851b6cd06435d349b43`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:46.376224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8e7ca830-8e88-4858-9ab2-125f6b9deeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:46.443851 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3137/6203] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:46.455798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e7809-53e1-4d77-a7ad-1ddc15d11437 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:46.457585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-046e7809-53e1-4d77-a7ad-1ddc15d11437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:46.522400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e7809-53e1-4d77-a7ad-1ddc15d11437 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:46.526120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832446e-2907-4e58-9094-66b97028c52f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:46.526730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832446e-2907-4e58-9094-66b97028c52f service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:46.527018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832446e-2907-4e58-9094-66b97028c52f service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:46.527297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832446e-2907-4e58-9094-66b97028c52f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:46.536485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832446e-2907-4e58-9094-66b97028c52f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:46.537969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832446e-2907-4e58-9094-66b97028c52f service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:46.547680 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3067/6204] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:46.621094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ca830-8e88-4858-9ab2-125f6b9deeea None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:46.633343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ca830-8e88-4858-9ab2-125f6b9deeea None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:46.637280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d085f7-42e7-4d18-a4cc-3332f25b28c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:46.637892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d085f7-42e7-4d18-a4cc-3332f25b28c2 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:46.638182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d085f7-42e7-4d18-a4cc-3332f25b28c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:46.638506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d085f7-42e7-4d18-a4cc-3332f25b28c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:46.698216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d085f7-42e7-4d18-a4cc-3332f25b28c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:46.699674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d085f7-42e7-4d18-a4cc-3332f25b28c2 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:46.709129 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3138/6205] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:11:48.188531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832446e-2907-4e58-9094-66b97028c52f service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:48.225838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:48.251940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832446e-2907-4e58-9094-66b97028c52f service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:48.256480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:48.257140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:48.257424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:48.257702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] PATH_INFO: `/v3/users/0eb9f6e450144b1fbc7650fe9f088113` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:48.258812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eb9f6e450144b1fbc7650fe9f088113)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:48.260711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:48.306290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eb9f6e450144b1fbc7650fe9f088113'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:48.318771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eb9f6e450144b1fbc7650fe9f088113'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:48.319435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eb9f6e450144b1fbc7650fe9f088113'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:48.326273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eb9f6e450144b1fbc7650fe9f088113'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:48.333455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eb9f6e450144b1fbc7650fe9f088113'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:48.339463 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3068/6206] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:11:48 2022] DELETE /identity/v3/users/0eb9f6e450144b1fbc7650fe9f088113 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:48.344083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047be1f-8597-47d2-af57-df5d433b5c6c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:48.344563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047be1f-8597-47d2-af57-df5d433b5c6c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:48.344674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047be1f-8597-47d2-af57-df5d433b5c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:48.364446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9047be1f-8597-47d2-af57-df5d433b5c6c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:48.365705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9047be1f-8597-47d2-af57-df5d433b5c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:48.390201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:48.425617 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3139/6207] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:11:48.435441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:48.437523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:48.501209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee6b61-6bc3-44bb-892b-7cf933901127 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:48.504795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400f91d-1cd9-4a82-9926-f4e0a4f1ac46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:48.505462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400f91d-1cd9-4a82-9926-f4e0a4f1ac46 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:48.505748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400f91d-1cd9-4a82-9926-f4e0a4f1ac46 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:48.506023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400f91d-1cd9-4a82-9926-f4e0a4f1ac46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:48.515114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400f91d-1cd9-4a82-9926-f4e0a4f1ac46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:48.516540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400f91d-1cd9-4a82-9926-f4e0a4f1ac46 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:48.526346 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3069/6208] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:48.873439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047be1f-8597-47d2-af57-df5d433b5c6c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:48.875019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9047be1f-8597-47d2-af57-df5d433b5c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:48.888138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047be1f-8597-47d2-af57-df5d433b5c6c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:48.894153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:48.895066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:48.895312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:48.895767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] PATH_INFO: `/v3/projects/5d3a007f1ae84fb78596c03d4ff871f8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:48.897269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d3a007f1ae84fb78596c03d4ff871f8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:48.899064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:48.945624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d3a007f1ae84fb78596c03d4ff871f8', 'request_id': 'req-292701cb-2842-4531-ab5d-55cc2756f823'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:48.953329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d3a007f1ae84fb78596c03d4ff871f8', 'request_id': 'req-292701cb-2842-4531-ab5d-55cc2756f823'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:48.959851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d3a007f1ae84fb78596c03d4ff871f8', 'request_id': 'req-292701cb-2842-4531-ab5d-55cc2756f823'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:48.972429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] The token cache is being invalidate because project 5d3a007f1ae84fb78596c03d4ff871f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:11:48.972728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:48.973448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:48.975671 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3140/6209] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:11:48 2022] DELETE /identity/v3/projects/5d3a007f1ae84fb78596c03d4ff871f8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:48.979862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400f91d-1cd9-4a82-9926-f4e0a4f1ac46 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:48.989825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400f91d-1cd9-4a82-9926-f4e0a4f1ac46 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:49.041496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:49.048325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1486b506-958d-443d-a916-d3e2483c921f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:49.049198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486b506-958d-443d-a916-d3e2483c921f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:49.049650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486b506-958d-443d-a916-d3e2483c921f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:49.050283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486b506-958d-443d-a916-d3e2483c921f admin admin] PATH_INFO: `/v3/users/a9580ca7f8df4851b6cd06435d349b43` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:49.051586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1486b506-958d-443d-a916-d3e2483c921f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9580ca7f8df4851b6cd06435d349b43)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:49.053993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1486b506-958d-443d-a916-d3e2483c921f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:49.105132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1486b506-958d-443d-a916-d3e2483c921f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9580ca7f8df4851b6cd06435d349b43'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:49.117434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1486b506-958d-443d-a916-d3e2483c921f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9580ca7f8df4851b6cd06435d349b43'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:49.118171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1486b506-958d-443d-a916-d3e2483c921f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9580ca7f8df4851b6cd06435d349b43'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:49.125417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1486b506-958d-443d-a916-d3e2483c921f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9580ca7f8df4851b6cd06435d349b43'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:49.133185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1486b506-958d-443d-a916-d3e2483c921f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9580ca7f8df4851b6cd06435d349b43'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:49.140328 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3070/6210] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:11:48 2022] DELETE /identity/v3/users/a9580ca7f8df4851b6cd06435d349b43 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:11:49.418907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:49.420931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:49.436005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292701cb-2842-4531-ab5d-55cc2756f823 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:49.498673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec4b3b-d7d3-4bf2-adc9-5d148479879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:49.499898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec4b3b-d7d3-4bf2-adc9-5d148479879b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:49.500137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec4b3b-d7d3-4bf2-adc9-5d148479879b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:49.500432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec4b3b-d7d3-4bf2-adc9-5d148479879b admin admin] PATH_INFO: `/v3/projects/f611096ed30744c89bd9752180f6bf14` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:49.501736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec4b3b-d7d3-4bf2-adc9-5d148479879b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f611096ed30744c89bd9752180f6bf14)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:49.503664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec4b3b-d7d3-4bf2-adc9-5d148479879b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:49.553602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-acec4b3b-d7d3-4bf2-adc9-5d148479879b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f611096ed30744c89bd9752180f6bf14', 'request_id': 'req-acec4b3b-d7d3-4bf2-adc9-5d148479879b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:49.557200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:49.562156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-acec4b3b-d7d3-4bf2-adc9-5d148479879b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f611096ed30744c89bd9752180f6bf14', 'request_id': 'req-acec4b3b-d7d3-4bf2-adc9-5d148479879b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:49.570328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-acec4b3b-d7d3-4bf2-adc9-5d148479879b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f611096ed30744c89bd9752180f6bf14', 'request_id': 'req-acec4b3b-d7d3-4bf2-adc9-5d148479879b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:49.582037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-acec4b3b-d7d3-4bf2-adc9-5d148479879b admin admin] The token cache is being invalidate because project f611096ed30744c89bd9752180f6bf14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:11:49.582616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-acec4b3b-d7d3-4bf2-adc9-5d148479879b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:49.583372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-acec4b3b-d7d3-4bf2-adc9-5d148479879b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:49.585077 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3141/6211] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:11:49 2022] DELETE /identity/v3/projects/f611096ed30744c89bd9752180f6bf14 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:49.590020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f1a78-170b-450f-b7de-0c048ee6994a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:49.590487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f1a78-170b-450f-b7de-0c048ee6994a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:49.590735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f1a78-170b-450f-b7de-0c048ee6994a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:49.612131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4e9f1a78-170b-450f-b7de-0c048ee6994a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:49.613579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4e9f1a78-170b-450f-b7de-0c048ee6994a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:49.674404 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3071/6212] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:49.778771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da54fcf-fcd4-4ebf-8f32-09bfe30935fc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:49.779047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da54fcf-fcd4-4ebf-8f32-09bfe30935fc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:49.779329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da54fcf-fcd4-4ebf-8f32-09bfe30935fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:49.799488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7da54fcf-fcd4-4ebf-8f32-09bfe30935fc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:49.800696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7da54fcf-fcd4-4ebf-8f32-09bfe30935fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:49.808618 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3142/6213] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:49.813008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f1a78-170b-450f-b7de-0c048ee6994a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:49.823847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f1a78-170b-450f-b7de-0c048ee6994a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:49.830241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aafc26-a94d-44af-9bb9-4a7df012e954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:49.830866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aafc26-a94d-44af-9bb9-4a7df012e954 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:49.831154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aafc26-a94d-44af-9bb9-4a7df012e954 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:49.831462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aafc26-a94d-44af-9bb9-4a7df012e954 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:49.832119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aafc26-a94d-44af-9bb9-4a7df012e954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:11:49.832527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aafc26-a94d-44af-9bb9-4a7df012e954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:49.834102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aafc26-a94d-44af-9bb9-4a7df012e954 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:49.848985 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3072/6214] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:11:49.851497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da54fcf-fcd4-4ebf-8f32-09bfe30935fc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:49.861528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da54fcf-fcd4-4ebf-8f32-09bfe30935fc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:49.866479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb70bc0-9c7a-48e2-9770-b15cb74b7299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:49.867094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb70bc0-9c7a-48e2-9770-b15cb74b7299 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:49.867378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb70bc0-9c7a-48e2-9770-b15cb74b7299 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:49.867657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb70bc0-9c7a-48e2-9770-b15cb74b7299 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:49.868359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb70bc0-9c7a-48e2-9770-b15cb74b7299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:49.870406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb70bc0-9c7a-48e2-9770-b15cb74b7299 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:49.897615 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3143/6215] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:11:49 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:49.901750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aafc26-a94d-44af-9bb9-4a7df012e954 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:49.912632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aafc26-a94d-44af-9bb9-4a7df012e954 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:49.969291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f1352-8490-4089-8198-46961ed79c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:49.969894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f1352-8490-4089-8198-46961ed79c9b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:49.970180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f1352-8490-4089-8198-46961ed79c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:49.970452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f1352-8490-4089-8198-46961ed79c9b admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:49.971069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f1352-8490-4089-8198-46961ed79c9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:49.973020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f1352-8490-4089-8198-46961ed79c9b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:49.983280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:49.996699 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3073/6216] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:11:49 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:50.001452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb70bc0-9c7a-48e2-9770-b15cb74b7299 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:50.012110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb70bc0-9c7a-48e2-9770-b15cb74b7299 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:50.017043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87e55d-0283-4d6d-b8f3-4e0779e2f9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:50.017632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87e55d-0283-4d6d-b8f3-4e0779e2f9fc admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.017915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87e55d-0283-4d6d-b8f3-4e0779e2f9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.018188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87e55d-0283-4d6d-b8f3-4e0779e2f9fc admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.018951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87e55d-0283-4d6d-b8f3-4e0779e2f9fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:50.020545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87e55d-0283-4d6d-b8f3-4e0779e2f9fc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:50.034478 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3144/6217] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:50.035632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f1352-8490-4089-8198-46961ed79c9b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:50.046848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f1352-8490-4089-8198-46961ed79c9b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:50.051657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb391c-bc9b-4591-aa6e-8948b8a1d76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:50.052263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb391c-bc9b-4591-aa6e-8948b8a1d76a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.052552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb391c-bc9b-4591-aa6e-8948b8a1d76a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.052837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb391c-bc9b-4591-aa6e-8948b8a1d76a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.053535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb391c-bc9b-4591-aa6e-8948b8a1d76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:50.055114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb391c-bc9b-4591-aa6e-8948b8a1d76a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:50.070136 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3074/6218] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:50.074324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87e55d-0283-4d6d-b8f3-4e0779e2f9fc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:50.085250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87e55d-0283-4d6d-b8f3-4e0779e2f9fc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:50.089978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc677c-40a3-4870-8e09-5a9b8d66974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:50.090631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc677c-40a3-4870-8e09-5a9b8d66974f admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.090924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc677c-40a3-4870-8e09-5a9b8d66974f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.091206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc677c-40a3-4870-8e09-5a9b8d66974f admin admin] PATH_INFO: `/v3/projects/34fda59b3d81492aa10149446ea412ee/users/4be20e648e664caea55ee3f793899b74/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.105950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc677c-40a3-4870-8e09-5a9b8d66974f admin admin] RBAC: Authorizing `identity:create_grant(project_id=34fda59b3d81492aa10149446ea412ee, user_id=4be20e648e664caea55ee3f793899b74, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:50.108638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc677c-40a3-4870-8e09-5a9b8d66974f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:50.121202 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3145/6219] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:50 2022] PUT /identity/v3/projects/34fda59b3d81492aa10149446ea412ee/users/4be20e648e664caea55ee3f793899b74/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:50.131873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a860b28-4898-444c-9573-23df5b0ba7f0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.132175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a860b28-4898-444c-9573-23df5b0ba7f0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.132460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a860b28-4898-444c-9573-23df5b0ba7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.160830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6a860b28-4898-444c-9573-23df5b0ba7f0 None None] MFA Rules not processed for user `4be20e648e664caea55ee3f793899b74`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:50.162083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6a860b28-4898-444c-9573-23df5b0ba7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:50.224955 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3075/6220] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:50.229598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc677c-40a3-4870-8e09-5a9b8d66974f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:50.290760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc677c-40a3-4870-8e09-5a9b8d66974f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:50.295464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc82f4-ebef-4a0e-a84b-ca5c4aa3e500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:50.296096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc82f4-ebef-4a0e-a84b-ca5c4aa3e500 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.296384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc82f4-ebef-4a0e-a84b-ca5c4aa3e500 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.296664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc82f4-ebef-4a0e-a84b-ca5c4aa3e500 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.297337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc82f4-ebef-4a0e-a84b-ca5c4aa3e500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:50.299253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc82f4-ebef-4a0e-a84b-ca5c4aa3e500 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:50.325402 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3146/6221] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:11:50 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:11:50.330068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a860b28-4898-444c-9573-23df5b0ba7f0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:50.341098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a860b28-4898-444c-9573-23df5b0ba7f0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:50.397888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae0efe-6bc8-4eab-a3c8-6f9edbf19b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:50.398518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae0efe-6bc8-4eab-a3c8-6f9edbf19b7f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.398803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae0efe-6bc8-4eab-a3c8-6f9edbf19b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.399077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae0efe-6bc8-4eab-a3c8-6f9edbf19b7f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.399708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae0efe-6bc8-4eab-a3c8-6f9edbf19b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:50.401663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae0efe-6bc8-4eab-a3c8-6f9edbf19b7f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:50.422833 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3076/6222] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:11:50 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:11:50.426863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc82f4-ebef-4a0e-a84b-ca5c4aa3e500 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:50.437722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc82f4-ebef-4a0e-a84b-ca5c4aa3e500 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:50.442736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf4571-76c9-4667-a698-10553e977811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:50.443371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf4571-76c9-4667-a698-10553e977811 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.443646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf4571-76c9-4667-a698-10553e977811 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.443929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf4571-76c9-4667-a698-10553e977811 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.444580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf4571-76c9-4667-a698-10553e977811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:50.446282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf4571-76c9-4667-a698-10553e977811 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:50.457186 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3147/6223] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:50.461174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae0efe-6bc8-4eab-a3c8-6f9edbf19b7f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:50.471208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae0efe-6bc8-4eab-a3c8-6f9edbf19b7f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:50.476107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895c24a-b4e7-4f53-851b-ad7e6d12e926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:50.477044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895c24a-b4e7-4f53-851b-ad7e6d12e926 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.477332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895c24a-b4e7-4f53-851b-ad7e6d12e926 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.477618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895c24a-b4e7-4f53-851b-ad7e6d12e926 admin admin] PATH_INFO: `/v3/projects/a468d30e47334a4793de98eeef778286/users/b6c2ffd5805f427983d108ac9f10f633/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.493022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895c24a-b4e7-4f53-851b-ad7e6d12e926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a468d30e47334a4793de98eeef778286, user_id=b6c2ffd5805f427983d108ac9f10f633, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:50.495706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895c24a-b4e7-4f53-851b-ad7e6d12e926 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:50.508835 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3077/6224] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:50 2022] PUT /identity/v3/projects/a468d30e47334a4793de98eeef778286/users/b6c2ffd5805f427983d108ac9f10f633/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:11:50.512713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf4571-76c9-4667-a698-10553e977811 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:50.522946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf4571-76c9-4667-a698-10553e977811 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:50.570066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:50.581041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90573c-1ef4-4055-aab0-d3f214eac256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:50.581673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90573c-1ef4-4055-aab0-d3f214eac256 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.581959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90573c-1ef4-4055-aab0-d3f214eac256 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.582244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90573c-1ef4-4055-aab0-d3f214eac256 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.582892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90573c-1ef4-4055-aab0-d3f214eac256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:50.584613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90573c-1ef4-4055-aab0-d3f214eac256 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:50.594645 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3148/6225] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:50.598598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895c24a-b4e7-4f53-851b-ad7e6d12e926 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:50.609699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895c24a-b4e7-4f53-851b-ad7e6d12e926 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:50.614322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b159bc-1d9e-40bb-b89a-c8bece309b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:50.614971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b159bc-1d9e-40bb-b89a-c8bece309b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.615255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b159bc-1d9e-40bb-b89a-c8bece309b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.615561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b159bc-1d9e-40bb-b89a-c8bece309b56 admin admin] PATH_INFO: `/v3/projects/a468d30e47334a4793de98eeef778286/users/b6c2ffd5805f427983d108ac9f10f633/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.617537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b159bc-1d9e-40bb-b89a-c8bece309b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a468d30e47334a4793de98eeef778286, user_id=b6c2ffd5805f427983d108ac9f10f633, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:50.620178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b159bc-1d9e-40bb-b89a-c8bece309b56 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:50.630786 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3078/6226] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:50 2022] PUT /identity/v3/projects/a468d30e47334a4793de98eeef778286/users/b6c2ffd5805f427983d108ac9f10f633/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:50.640743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d357c-51f3-4cac-9e23-3c49a49c1aa5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.641078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d357c-51f3-4cac-9e23-3c49a49c1aa5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.641363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d357c-51f3-4cac-9e23-3c49a49c1aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.669700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e46d357c-51f3-4cac-9e23-3c49a49c1aa5 None None] MFA Rules not processed for user `b6c2ffd5805f427983d108ac9f10f633`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:50.671463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e46d357c-51f3-4cac-9e23-3c49a49c1aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:50.740050 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3149/6227] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:50.753022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b159bc-1d9e-40bb-b89a-c8bece309b56 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:50.754728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-57b159bc-1d9e-40bb-b89a-c8bece309b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:50.783980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:50.817377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b159bc-1d9e-40bb-b89a-c8bece309b56 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:50.821116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e8138-be94-4735-bd9b-b85c93e74e8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:50.821725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e8138-be94-4735-bd9b-b85c93e74e8c service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.822011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e8138-be94-4735-bd9b-b85c93e74e8c service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.822292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e8138-be94-4735-bd9b-b85c93e74e8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.832120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e8138-be94-4735-bd9b-b85c93e74e8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:50.833664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e8138-be94-4735-bd9b-b85c93e74e8c service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:50.843271 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3079/6228] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:50.918953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d357c-51f3-4cac-9e23-3c49a49c1aa5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:50.931308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d357c-51f3-4cac-9e23-3c49a49c1aa5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:50.935172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb70ae-9344-4eef-b7b6-850ee6adc9a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:50.935804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb70ae-9344-4eef-b7b6-850ee6adc9a5 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:50.936090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb70ae-9344-4eef-b7b6-850ee6adc9a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:50.936366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb70ae-9344-4eef-b7b6-850ee6adc9a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:50.995521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb70ae-9344-4eef-b7b6-850ee6adc9a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:50.996920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb70ae-9344-4eef-b7b6-850ee6adc9a5 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:51.006541 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3150/6229] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:11:52.476387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e8138-be94-4735-bd9b-b85c93e74e8c service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:52.552239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e8138-be94-4735-bd9b-b85c93e74e8c service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:52.557131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:52.557792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:52.558081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:52.558360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] PATH_INFO: `/v3/users/4be20e648e664caea55ee3f793899b74` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:52.559500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4be20e648e664caea55ee3f793899b74)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:52.561528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:52.617491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4be20e648e664caea55ee3f793899b74'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:52.629603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4be20e648e664caea55ee3f793899b74'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:52.630325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4be20e648e664caea55ee3f793899b74'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:52.637422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4be20e648e664caea55ee3f793899b74'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:52.645848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4be20e648e664caea55ee3f793899b74'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:52.652188 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3080/6230] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:11:52 2022] DELETE /identity/v3/users/4be20e648e664caea55ee3f793899b74 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:52.657243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a6745-d8a7-4690-bc89-cd36deae2af9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:52.657650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a6745-d8a7-4690-bc89-cd36deae2af9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:52.657827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a6745-d8a7-4690-bc89-cd36deae2af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:52.677254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b72a6745-d8a7-4690-bc89-cd36deae2af9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:52.678478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b72a6745-d8a7-4690-bc89-cd36deae2af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:52.743233 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3151/6231] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:52.752773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:52.754277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:52.808395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:52.818541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6ca36-46a2-409b-84fe-a0abf0f7ac42 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:52.822309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcab995-51b8-458c-bfe4-0354df6c17e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:52.822904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcab995-51b8-458c-bfe4-0354df6c17e3 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:52.823192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcab995-51b8-458c-bfe4-0354df6c17e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:52.823466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcab995-51b8-458c-bfe4-0354df6c17e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:52.833771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcab995-51b8-458c-bfe4-0354df6c17e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:52.835255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcab995-51b8-458c-bfe4-0354df6c17e3 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:52.847344 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3081/6232] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:53.187330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a6745-d8a7-4690-bc89-cd36deae2af9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:53.189137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b72a6745-d8a7-4690-bc89-cd36deae2af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:53.207492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a6745-d8a7-4690-bc89-cd36deae2af9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:53.219842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:53.220501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:53.220791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:53.221111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] PATH_INFO: `/v3/projects/34fda59b3d81492aa10149446ea412ee` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:53.222292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34fda59b3d81492aa10149446ea412ee)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:53.224224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:53.236714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:53.282997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34fda59b3d81492aa10149446ea412ee', 'request_id': 'req-d29282a8-5dac-43a7-98d1-cee126c66d38'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.291856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34fda59b3d81492aa10149446ea412ee', 'request_id': 'req-d29282a8-5dac-43a7-98d1-cee126c66d38'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.299487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34fda59b3d81492aa10149446ea412ee', 'request_id': 'req-d29282a8-5dac-43a7-98d1-cee126c66d38'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.311064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] The token cache is being invalidate because project 34fda59b3d81492aa10149446ea412ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:11:53.311357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.312559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.314330 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3152/6233] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:11:53 2022] DELETE /identity/v3/projects/34fda59b3d81492aa10149446ea412ee => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:53.318267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcab995-51b8-458c-bfe4-0354df6c17e3 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:53.329122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcab995-51b8-458c-bfe4-0354df6c17e3 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:53.394365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a8054-d4cd-4b3b-83c4-f75af48820aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:53.395020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a8054-d4cd-4b3b-83c4-f75af48820aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:53.395429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a8054-d4cd-4b3b-83c4-f75af48820aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:53.395707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a8054-d4cd-4b3b-83c4-f75af48820aa admin admin] PATH_INFO: `/v3/users/b6c2ffd5805f427983d108ac9f10f633` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:53.396881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a8054-d4cd-4b3b-83c4-f75af48820aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6c2ffd5805f427983d108ac9f10f633)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:53.398950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a8054-d4cd-4b3b-83c4-f75af48820aa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:53.446044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af7a8054-d4cd-4b3b-83c4-f75af48820aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6c2ffd5805f427983d108ac9f10f633'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.458256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af7a8054-d4cd-4b3b-83c4-f75af48820aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6c2ffd5805f427983d108ac9f10f633'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.458939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af7a8054-d4cd-4b3b-83c4-f75af48820aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6c2ffd5805f427983d108ac9f10f633'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.465779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af7a8054-d4cd-4b3b-83c4-f75af48820aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6c2ffd5805f427983d108ac9f10f633'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.473079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af7a8054-d4cd-4b3b-83c4-f75af48820aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6c2ffd5805f427983d108ac9f10f633'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.479286 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3082/6234] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:11:53 2022] DELETE /identity/v3/users/b6c2ffd5805f427983d108ac9f10f633 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:53.762185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:53.765633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:53.792018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29282a8-5dac-43a7-98d1-cee126c66d38 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:53.857993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5d30f-e562-4c77-822c-646f193349de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:53.858776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5d30f-e562-4c77-822c-646f193349de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:53.859194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5d30f-e562-4c77-822c-646f193349de admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:53.860107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5d30f-e562-4c77-822c-646f193349de admin admin] PATH_INFO: `/v3/projects/a468d30e47334a4793de98eeef778286` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:53.860782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5d30f-e562-4c77-822c-646f193349de admin admin] RBAC: Authorizing `identity:delete_project(project_id=a468d30e47334a4793de98eeef778286)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:53.863083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5d30f-e562-4c77-822c-646f193349de admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:53.912792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9da5d30f-e562-4c77-822c-646f193349de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a468d30e47334a4793de98eeef778286', 'request_id': 'req-9da5d30f-e562-4c77-822c-646f193349de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.921677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9da5d30f-e562-4c77-822c-646f193349de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a468d30e47334a4793de98eeef778286', 'request_id': 'req-9da5d30f-e562-4c77-822c-646f193349de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.928461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9da5d30f-e562-4c77-822c-646f193349de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a468d30e47334a4793de98eeef778286', 'request_id': 'req-9da5d30f-e562-4c77-822c-646f193349de'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.942251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9da5d30f-e562-4c77-822c-646f193349de admin admin] The token cache is being invalidate because project a468d30e47334a4793de98eeef778286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:11:53.942614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9da5d30f-e562-4c77-822c-646f193349de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.943498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9da5d30f-e562-4c77-822c-646f193349de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:53.945140 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3153/6235] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:11:53 2022] DELETE /identity/v3/projects/a468d30e47334a4793de98eeef778286 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:53.950222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cbb9d-4969-48ad-8660-002a1e22d946 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:53.950518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cbb9d-4969-48ad-8660-002a1e22d946 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:53.950796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cbb9d-4969-48ad-8660-002a1e22d946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:53.971759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-676cbb9d-4969-48ad-8660-002a1e22d946 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:53.973001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-676cbb9d-4969-48ad-8660-002a1e22d946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:54.045446 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3083/6236] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:11:54.048010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320bb46-011f-4b48-a9f8-a93f5b75158e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.048768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320bb46-011f-4b48-a9f8-a93f5b75158e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.049269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320bb46-011f-4b48-a9f8-a93f5b75158e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.077905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1320bb46-011f-4b48-a9f8-a93f5b75158e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:54.079165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1320bb46-011f-4b48-a9f8-a93f5b75158e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:54.088029 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3154/6237] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:54.092214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cbb9d-4969-48ad-8660-002a1e22d946 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:54.103609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cbb9d-4969-48ad-8660-002a1e22d946 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:54.108782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d567dba-d1ba-4d40-9051-fe5878013d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:54.109561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d567dba-d1ba-4d40-9051-fe5878013d5a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.109924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d567dba-d1ba-4d40-9051-fe5878013d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.110196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d567dba-d1ba-4d40-9051-fe5878013d5a admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.110966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d567dba-d1ba-4d40-9051-fe5878013d5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:11:54.111317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d567dba-d1ba-4d40-9051-fe5878013d5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:54.112972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d567dba-d1ba-4d40-9051-fe5878013d5a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:54.115477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:54.128940 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3084/6238] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:11:54.132229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320bb46-011f-4b48-a9f8-a93f5b75158e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:54.142515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320bb46-011f-4b48-a9f8-a93f5b75158e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:54.147592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc27b6-e58e-4570-a0df-1950a87e51a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:54.148219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc27b6-e58e-4570-a0df-1950a87e51a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.148534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc27b6-e58e-4570-a0df-1950a87e51a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.148815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc27b6-e58e-4570-a0df-1950a87e51a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.149483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc27b6-e58e-4570-a0df-1950a87e51a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:54.151438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc27b6-e58e-4570-a0df-1950a87e51a0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:54.172533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:54.181613 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3155/6239] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:11:54 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:54.185361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d567dba-d1ba-4d40-9051-fe5878013d5a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:54.196677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d567dba-d1ba-4d40-9051-fe5878013d5a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:54.253228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7838807-ff71-47a8-8284-4cee0cf4e157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:54.253847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7838807-ff71-47a8-8284-4cee0cf4e157 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.254133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7838807-ff71-47a8-8284-4cee0cf4e157 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.254419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7838807-ff71-47a8-8284-4cee0cf4e157 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.255053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7838807-ff71-47a8-8284-4cee0cf4e157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:54.257047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7838807-ff71-47a8-8284-4cee0cf4e157 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:54.278333 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3085/6240] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:11:54 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:54.282442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc27b6-e58e-4570-a0df-1950a87e51a0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:54.292946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc27b6-e58e-4570-a0df-1950a87e51a0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:54.298183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b305f8-e302-4fd7-b697-bdec9fdb08d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:54.298779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b305f8-e302-4fd7-b697-bdec9fdb08d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.299064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b305f8-e302-4fd7-b697-bdec9fdb08d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.299347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b305f8-e302-4fd7-b697-bdec9fdb08d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.299983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b305f8-e302-4fd7-b697-bdec9fdb08d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:54.301794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b305f8-e302-4fd7-b697-bdec9fdb08d8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:54.312314 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3156/6241] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:54.316198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7838807-ff71-47a8-8284-4cee0cf4e157 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:54.326371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7838807-ff71-47a8-8284-4cee0cf4e157 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:54.330991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff2891-510c-4c96-bd55-7de7c3248a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:54.331591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff2891-510c-4c96-bd55-7de7c3248a71 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.331882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff2891-510c-4c96-bd55-7de7c3248a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.332164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff2891-510c-4c96-bd55-7de7c3248a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.332818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff2891-510c-4c96-bd55-7de7c3248a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:54.334448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff2891-510c-4c96-bd55-7de7c3248a71 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:54.345211 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3086/6242] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:11:54.349511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b305f8-e302-4fd7-b697-bdec9fdb08d8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:54.360574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b305f8-e302-4fd7-b697-bdec9fdb08d8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:54.365213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0ddb8-e935-4476-b807-1bbcf0fb1588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:54.365874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0ddb8-e935-4476-b807-1bbcf0fb1588 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.366165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0ddb8-e935-4476-b807-1bbcf0fb1588 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.366446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0ddb8-e935-4476-b807-1bbcf0fb1588 admin admin] PATH_INFO: `/v3/projects/701273b8464a446d81db404a6fec5ec4/users/5cbd8b379df2450c93f23ff1894f55b4/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.381657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0ddb8-e935-4476-b807-1bbcf0fb1588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=701273b8464a446d81db404a6fec5ec4, user_id=5cbd8b379df2450c93f23ff1894f55b4, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:54.384376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0ddb8-e935-4476-b807-1bbcf0fb1588 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:54.398184 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3157/6243] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:54 2022] PUT /identity/v3/projects/701273b8464a446d81db404a6fec5ec4/users/5cbd8b379df2450c93f23ff1894f55b4/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:54.408382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb9a0a-a764-41cd-bc15-2816f00b1cc1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.408686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb9a0a-a764-41cd-bc15-2816f00b1cc1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.409009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb9a0a-a764-41cd-bc15-2816f00b1cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.436296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-17eb9a0a-a764-41cd-bc15-2816f00b1cc1 None None] MFA Rules not processed for user `5cbd8b379df2450c93f23ff1894f55b4`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:54.438072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-17eb9a0a-a764-41cd-bc15-2816f00b1cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:54.500153 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3087/6244] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:54.504734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0ddb8-e935-4476-b807-1bbcf0fb1588 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:54.565507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0ddb8-e935-4476-b807-1bbcf0fb1588 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:54.570505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674638bf-d3ee-4460-b5e2-c73ed2c1f56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:54.571124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674638bf-d3ee-4460-b5e2-c73ed2c1f56c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.571413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674638bf-d3ee-4460-b5e2-c73ed2c1f56c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.571693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674638bf-d3ee-4460-b5e2-c73ed2c1f56c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.572322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674638bf-d3ee-4460-b5e2-c73ed2c1f56c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:54.574286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674638bf-d3ee-4460-b5e2-c73ed2c1f56c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:54.602292 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3158/6245] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:11:54 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:54.606475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb9a0a-a764-41cd-bc15-2816f00b1cc1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:54.617846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb9a0a-a764-41cd-bc15-2816f00b1cc1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:54.675202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8422115-b9f7-464d-b346-d4d9eea9e09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:54.675837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8422115-b9f7-464d-b346-d4d9eea9e09d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.676111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8422115-b9f7-464d-b346-d4d9eea9e09d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.676389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8422115-b9f7-464d-b346-d4d9eea9e09d admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.677068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8422115-b9f7-464d-b346-d4d9eea9e09d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:54.678997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8422115-b9f7-464d-b346-d4d9eea9e09d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:54.689208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:54.701963 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3088/6246] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:11:54 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:54.705804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674638bf-d3ee-4460-b5e2-c73ed2c1f56c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:54.716235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674638bf-d3ee-4460-b5e2-c73ed2c1f56c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:54.721457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecc018-8028-4cc8-9079-2709a63cea98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:54.722056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecc018-8028-4cc8-9079-2709a63cea98 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.722345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecc018-8028-4cc8-9079-2709a63cea98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.722627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecc018-8028-4cc8-9079-2709a63cea98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.723271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecc018-8028-4cc8-9079-2709a63cea98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:54.724885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecc018-8028-4cc8-9079-2709a63cea98 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:54.736559 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3159/6247] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:54.740510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8422115-b9f7-464d-b346-d4d9eea9e09d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:54.751683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8422115-b9f7-464d-b346-d4d9eea9e09d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:54.756582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a4253-081d-4f95-bbed-32e89e0d37fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:54.757292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a4253-081d-4f95-bbed-32e89e0d37fa admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.757586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a4253-081d-4f95-bbed-32e89e0d37fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.757871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a4253-081d-4f95-bbed-32e89e0d37fa admin admin] PATH_INFO: `/v3/projects/0051398908c8429686c005c9f1ab2dd5/users/47feb3ad0c374adfa481bc9f808fcc91/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.772991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a4253-081d-4f95-bbed-32e89e0d37fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0051398908c8429686c005c9f1ab2dd5, user_id=47feb3ad0c374adfa481bc9f808fcc91, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:54.775668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a4253-081d-4f95-bbed-32e89e0d37fa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:54.789504 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3089/6248] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:54 2022] PUT /identity/v3/projects/0051398908c8429686c005c9f1ab2dd5/users/47feb3ad0c374adfa481bc9f808fcc91/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:11:54.793470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecc018-8028-4cc8-9079-2709a63cea98 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:54.804510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecc018-8028-4cc8-9079-2709a63cea98 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:54.864752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a9b50-1b18-4673-a926-39d328dc687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:54.865404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a9b50-1b18-4673-a926-39d328dc687c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.865696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a9b50-1b18-4673-a926-39d328dc687c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.865978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a9b50-1b18-4673-a926-39d328dc687c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.866627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a9b50-1b18-4673-a926-39d328dc687c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:54.868234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a9b50-1b18-4673-a926-39d328dc687c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:54.881641 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3160/6249] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:54.883172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a4253-081d-4f95-bbed-32e89e0d37fa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:54.894724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a4253-081d-4f95-bbed-32e89e0d37fa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:54.899454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eeb533-244d-4611-9a4b-120b0ab6c1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:54.900105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eeb533-244d-4611-9a4b-120b0ab6c1cc admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.900389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eeb533-244d-4611-9a4b-120b0ab6c1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.900666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eeb533-244d-4611-9a4b-120b0ab6c1cc admin admin] PATH_INFO: `/v3/projects/0051398908c8429686c005c9f1ab2dd5/users/47feb3ad0c374adfa481bc9f808fcc91/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.902616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eeb533-244d-4611-9a4b-120b0ab6c1cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0051398908c8429686c005c9f1ab2dd5, user_id=47feb3ad0c374adfa481bc9f808fcc91, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:54.905902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eeb533-244d-4611-9a4b-120b0ab6c1cc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:54.916130 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3090/6250] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:54 2022] PUT /identity/v3/projects/0051398908c8429686c005c9f1ab2dd5/users/47feb3ad0c374adfa481bc9f808fcc91/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:11:54.927260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2e83a-4c66-4a72-9aca-7556dc9020fc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:54.927561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2e83a-4c66-4a72-9aca-7556dc9020fc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:54.927852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2e83a-4c66-4a72-9aca-7556dc9020fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:54.957714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-23a2e83a-4c66-4a72-9aca-7556dc9020fc None None] MFA Rules not processed for user `47feb3ad0c374adfa481bc9f808fcc91`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:54.958967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-23a2e83a-4c66-4a72-9aca-7556dc9020fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:54.980401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:55.032368 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3161/6251] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:55.044784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eeb533-244d-4611-9a4b-120b0ab6c1cc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:55.046619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d3eeb533-244d-4611-9a4b-120b0ab6c1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:55.110891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eeb533-244d-4611-9a4b-120b0ab6c1cc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:55.115174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17527b7d-4a84-4f94-b37f-c468aec8f851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:55.115876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17527b7d-4a84-4f94-b37f-c468aec8f851 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:55.116189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17527b7d-4a84-4f94-b37f-c468aec8f851 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:55.116527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17527b7d-4a84-4f94-b37f-c468aec8f851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:55.132417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17527b7d-4a84-4f94-b37f-c468aec8f851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:55.134045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17527b7d-4a84-4f94-b37f-c468aec8f851 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:55.145784 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3091/6252] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:55.240581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2e83a-4c66-4a72-9aca-7556dc9020fc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:55.323668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2e83a-4c66-4a72-9aca-7556dc9020fc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:55.329314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:55.329907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:55.330194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:55.330470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] PATH_INFO: `/v3/users/5cbd8b379df2450c93f23ff1894f55b4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:55.331626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cbd8b379df2450c93f23ff1894f55b4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:55.333596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:55.412048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cbd8b379df2450c93f23ff1894f55b4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:55.426314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cbd8b379df2450c93f23ff1894f55b4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:55.439324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cbd8b379df2450c93f23ff1894f55b4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:55.440242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cbd8b379df2450c93f23ff1894f55b4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:55.441014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cbd8b379df2450c93f23ff1894f55b4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:55.459010 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3162/6253] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:11:55 2022] DELETE /identity/v3/users/5cbd8b379df2450c93f23ff1894f55b4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:11:55.463583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d5670-7322-459a-adbf-15b0452e8636 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:55.463879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d5670-7322-459a-adbf-15b0452e8636 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:55.464161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d5670-7322-459a-adbf-15b0452e8636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:55.488930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-525d5670-7322-459a-adbf-15b0452e8636 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:55.490211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-525d5670-7322-459a-adbf-15b0452e8636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:55.553165 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3092/6254] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:55.563306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:55.564731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:55.611502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:55.632540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2b1d2-c79b-4f58-98e7-9aec9653e89c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:55.636449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5686c8-6169-4492-b154-cc6e8998a81c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:55.637179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5686c8-6169-4492-b154-cc6e8998a81c service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:55.637531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5686c8-6169-4492-b154-cc6e8998a81c service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:55.637773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5686c8-6169-4492-b154-cc6e8998a81c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:55.650922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5686c8-6169-4492-b154-cc6e8998a81c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:55.652406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5686c8-6169-4492-b154-cc6e8998a81c service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:55.663363 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3163/6255] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:55.983465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d5670-7322-459a-adbf-15b0452e8636 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:55.988995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-525d5670-7322-459a-adbf-15b0452e8636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:56.010667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d5670-7322-459a-adbf-15b0452e8636 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:56.015949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:56.016838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:56.016838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:56.017396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] PATH_INFO: `/v3/projects/701273b8464a446d81db404a6fec5ec4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:56.018559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=701273b8464a446d81db404a6fec5ec4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:56.023143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:56.089036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '701273b8464a446d81db404a6fec5ec4', 'request_id': 'req-ca82374e-8358-4a07-8d37-38f7accd7fa5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.093625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:56.098388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '701273b8464a446d81db404a6fec5ec4', 'request_id': 'req-ca82374e-8358-4a07-8d37-38f7accd7fa5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.108400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '701273b8464a446d81db404a6fec5ec4', 'request_id': 'req-ca82374e-8358-4a07-8d37-38f7accd7fa5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.124885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] The token cache is being invalidate because project 701273b8464a446d81db404a6fec5ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:11:56.125337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.126117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.128067 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3093/6256] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:11:55 2022] DELETE /identity/v3/projects/701273b8464a446d81db404a6fec5ec4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:56.132559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5686c8-6169-4492-b154-cc6e8998a81c service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:56.156273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5686c8-6169-4492-b154-cc6e8998a81c service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:56.216780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f23ec-639d-4c3a-bef4-f76912645020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:56.217565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f23ec-639d-4c3a-bef4-f76912645020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:56.217929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f23ec-639d-4c3a-bef4-f76912645020 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:56.218282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f23ec-639d-4c3a-bef4-f76912645020 admin admin] PATH_INFO: `/v3/users/47feb3ad0c374adfa481bc9f808fcc91` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:56.219493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f23ec-639d-4c3a-bef4-f76912645020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47feb3ad0c374adfa481bc9f808fcc91)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:56.221586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f23ec-639d-4c3a-bef4-f76912645020 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:56.272982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c66f23ec-639d-4c3a-bef4-f76912645020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47feb3ad0c374adfa481bc9f808fcc91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.290463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c66f23ec-639d-4c3a-bef4-f76912645020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47feb3ad0c374adfa481bc9f808fcc91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.298500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c66f23ec-639d-4c3a-bef4-f76912645020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47feb3ad0c374adfa481bc9f808fcc91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.299233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c66f23ec-639d-4c3a-bef4-f76912645020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47feb3ad0c374adfa481bc9f808fcc91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.299930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c66f23ec-639d-4c3a-bef4-f76912645020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47feb3ad0c374adfa481bc9f808fcc91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.315908 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3164/6257] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:11:56 2022] DELETE /identity/v3/users/47feb3ad0c374adfa481bc9f808fcc91 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:56.754164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:56.757290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:56.771069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82374e-8358-4a07-8d37-38f7accd7fa5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:56.856273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff756006-e1ec-48df-89cf-4a2c742a5b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:56.856955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff756006-e1ec-48df-89cf-4a2c742a5b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:56.857248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff756006-e1ec-48df-89cf-4a2c742a5b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:56.857529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff756006-e1ec-48df-89cf-4a2c742a5b04 admin admin] PATH_INFO: `/v3/projects/0051398908c8429686c005c9f1ab2dd5` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:56.859229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff756006-e1ec-48df-89cf-4a2c742a5b04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0051398908c8429686c005c9f1ab2dd5)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:56.862354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff756006-e1ec-48df-89cf-4a2c742a5b04 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:56.911807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ff756006-e1ec-48df-89cf-4a2c742a5b04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0051398908c8429686c005c9f1ab2dd5', 'request_id': 'req-ff756006-e1ec-48df-89cf-4a2c742a5b04'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.923047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ff756006-e1ec-48df-89cf-4a2c742a5b04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0051398908c8429686c005c9f1ab2dd5', 'request_id': 'req-ff756006-e1ec-48df-89cf-4a2c742a5b04'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.930097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ff756006-e1ec-48df-89cf-4a2c742a5b04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0051398908c8429686c005c9f1ab2dd5', 'request_id': 'req-ff756006-e1ec-48df-89cf-4a2c742a5b04'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.940727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ff756006-e1ec-48df-89cf-4a2c742a5b04 admin admin] The token cache is being invalidate because project 0051398908c8429686c005c9f1ab2dd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:11:56.941063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ff756006-e1ec-48df-89cf-4a2c742a5b04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.941725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ff756006-e1ec-48df-89cf-4a2c742a5b04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:11:56.943206 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3094/6258] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:11:56 2022] DELETE /identity/v3/projects/0051398908c8429686c005c9f1ab2dd5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:56.948986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f284b-cc8a-4ec1-999e-1c5a0b553951 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:56.949281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f284b-cc8a-4ec1-999e-1c5a0b553951 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:56.949557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f284b-cc8a-4ec1-999e-1c5a0b553951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:56.970279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-764f284b-cc8a-4ec1-999e-1c5a0b553951 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:56.971509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-764f284b-cc8a-4ec1-999e-1c5a0b553951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:57.031855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:57.043894 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3165/6259] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:57.049275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e8f8a-fc31-493d-be6e-5c6769019f2a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:57.050067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e8f8a-fc31-493d-be6e-5c6769019f2a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:57.050710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e8f8a-fc31-493d-be6e-5c6769019f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:57.073765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0b3e8f8a-fc31-493d-be6e-5c6769019f2a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:57.074303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0b3e8f8a-fc31-493d-be6e-5c6769019f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:57.095217 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3095/6260] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:11:57.100296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f284b-cc8a-4ec1-999e-1c5a0b553951 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:57.113135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f284b-cc8a-4ec1-999e-1c5a0b553951 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:57.120077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e341cf5-5acd-475b-a978-5c09464b30d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:57.120732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e341cf5-5acd-475b-a978-5c09464b30d4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:57.121067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e341cf5-5acd-475b-a978-5c09464b30d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:57.121439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e341cf5-5acd-475b-a978-5c09464b30d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:57.122115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e341cf5-5acd-475b-a978-5c09464b30d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:11:57.122482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e341cf5-5acd-475b-a978-5c09464b30d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:57.125723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e341cf5-5acd-475b-a978-5c09464b30d4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:57.141667 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3166/6261] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:11:57.146394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e8f8a-fc31-493d-be6e-5c6769019f2a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:57.159023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e8f8a-fc31-493d-be6e-5c6769019f2a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:57.173867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00035870-47ab-485b-b4b2-ebbf020a67ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:57.174195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00035870-47ab-485b-b4b2-ebbf020a67ac admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:57.174856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00035870-47ab-485b-b4b2-ebbf020a67ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:57.175257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00035870-47ab-485b-b4b2-ebbf020a67ac admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:57.175917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00035870-47ab-485b-b4b2-ebbf020a67ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:57.178103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00035870-47ab-485b-b4b2-ebbf020a67ac admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:57.220588 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3096/6262] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:11:57 2022] POST /identity/v3/projects => generated 365 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:11:57.224938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e341cf5-5acd-475b-a978-5c09464b30d4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:57.236975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e341cf5-5acd-475b-a978-5c09464b30d4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:57.369327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1baf06-d3f3-4b86-b68a-162385d18973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:57.371751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1baf06-d3f3-4b86-b68a-162385d18973 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:57.372953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1baf06-d3f3-4b86-b68a-162385d18973 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:57.373120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1baf06-d3f3-4b86-b68a-162385d18973 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:57.377075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1baf06-d3f3-4b86-b68a-162385d18973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:57.381631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1baf06-d3f3-4b86-b68a-162385d18973 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:57.424863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00035870-47ab-485b-b4b2-ebbf020a67ac admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:57.437915 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3167/6263] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:11:57 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:11:57.441534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00035870-47ab-485b-b4b2-ebbf020a67ac admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:57.447839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173bfde-6498-4301-81a9-5c9fd841b283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:57.449143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173bfde-6498-4301-81a9-5c9fd841b283 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:57.449451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173bfde-6498-4301-81a9-5c9fd841b283 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:57.449728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173bfde-6498-4301-81a9-5c9fd841b283 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:57.450531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173bfde-6498-4301-81a9-5c9fd841b283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:57.453240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173bfde-6498-4301-81a9-5c9fd841b283 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:57.473867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1baf06-d3f3-4b86-b68a-162385d18973 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:57.474810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3097/6264] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:57.487394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1baf06-d3f3-4b86-b68a-162385d18973 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:57.494242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c576a5b-e554-4b0f-8ced-22c989d3533f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:57.495015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c576a5b-e554-4b0f-8ced-22c989d3533f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:57.495311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c576a5b-e554-4b0f-8ced-22c989d3533f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:57.495586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c576a5b-e554-4b0f-8ced-22c989d3533f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:57.496240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c576a5b-e554-4b0f-8ced-22c989d3533f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:57.500377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c576a5b-e554-4b0f-8ced-22c989d3533f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:57.522301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173bfde-6498-4301-81a9-5c9fd841b283 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:57.522558 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3168/6265] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:57.541225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173bfde-6498-4301-81a9-5c9fd841b283 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:57.544393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b54636-fd44-45ad-8006-2e442709c4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:57.545762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b54636-fd44-45ad-8006-2e442709c4f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:57.547026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b54636-fd44-45ad-8006-2e442709c4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:57.547546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b54636-fd44-45ad-8006-2e442709c4f9 admin admin] PATH_INFO: `/v3/projects/09bb57542ced4ce7979b9b4eb385be1a/users/adc8d4feb0f8454d89acc52dbffcfb3b/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:57.571154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b54636-fd44-45ad-8006-2e442709c4f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09bb57542ced4ce7979b9b4eb385be1a, user_id=adc8d4feb0f8454d89acc52dbffcfb3b, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:57.575344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b54636-fd44-45ad-8006-2e442709c4f9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:57.603283 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3098/6266] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:57 2022] PUT /identity/v3/projects/09bb57542ced4ce7979b9b4eb385be1a/users/adc8d4feb0f8454d89acc52dbffcfb3b/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:11:57.616308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e98bd0-484b-4b74-a08d-a23c1bc4ff9b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:57.616648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e98bd0-484b-4b74-a08d-a23c1bc4ff9b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:57.617126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e98bd0-484b-4b74-a08d-a23c1bc4ff9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:57.662086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-33e98bd0-484b-4b74-a08d-a23c1bc4ff9b None None] MFA Rules not processed for user `adc8d4feb0f8454d89acc52dbffcfb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:57.663432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-33e98bd0-484b-4b74-a08d-a23c1bc4ff9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:57.743280 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3169/6267] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:57.750372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b54636-fd44-45ad-8006-2e442709c4f9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:57.765259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:57.893847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b54636-fd44-45ad-8006-2e442709c4f9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:57.900639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ff6d7-1bfd-442b-8ce3-d87a8a98d326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:57.902276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ff6d7-1bfd-442b-8ce3-d87a8a98d326 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:57.902767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ff6d7-1bfd-442b-8ce3-d87a8a98d326 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:57.904113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ff6d7-1bfd-442b-8ce3-d87a8a98d326 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:57.905175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ff6d7-1bfd-442b-8ce3-d87a8a98d326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:57.909599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ff6d7-1bfd-442b-8ce3-d87a8a98d326 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:57.963700 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3099/6268] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:11:57 2022] POST /identity/v3/projects => generated 367 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:11:57.965124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e98bd0-484b-4b74-a08d-a23c1bc4ff9b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:57.978066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e98bd0-484b-4b74-a08d-a23c1bc4ff9b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:58.093344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:58.111857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8d529-9b19-47c5-ac82-915a2e0d0c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:58.112552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8d529-9b19-47c5-ac82-915a2e0d0c69 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:58.112950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8d529-9b19-47c5-ac82-915a2e0d0c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:58.113598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8d529-9b19-47c5-ac82-915a2e0d0c69 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:58.115097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8d529-9b19-47c5-ac82-915a2e0d0c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:58.124938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8d529-9b19-47c5-ac82-915a2e0d0c69 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:58.162264 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3170/6269] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:11:57 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:11:58.163671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ff6d7-1bfd-442b-8ce3-d87a8a98d326 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:58.176393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ff6d7-1bfd-442b-8ce3-d87a8a98d326 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:58.181556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dbb49-70cc-43ed-81bd-459b15d702ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:58.182296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dbb49-70cc-43ed-81bd-459b15d702ad admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:58.182613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dbb49-70cc-43ed-81bd-459b15d702ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:58.182890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dbb49-70cc-43ed-81bd-459b15d702ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:58.183698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752dbb49-70cc-43ed-81bd-459b15d702ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:58.185514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752dbb49-70cc-43ed-81bd-459b15d702ad admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:58.210168 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3100/6270] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:58.211399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8d529-9b19-47c5-ac82-915a2e0d0c69 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:58.223610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8d529-9b19-47c5-ac82-915a2e0d0c69 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:58.243791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2ca9a-a8d5-4952-853d-81996e312828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:58.244531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2ca9a-a8d5-4952-853d-81996e312828 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:58.244852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2ca9a-a8d5-4952-853d-81996e312828 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:58.245219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2ca9a-a8d5-4952-853d-81996e312828 admin admin] PATH_INFO: `/v3/projects/5d535e5f94a14ff29c118ccd4f5f0d04/users/03ee7d2ea7854c84925c0df91a5e66ff/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:58.277974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2ca9a-a8d5-4952-853d-81996e312828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d535e5f94a14ff29c118ccd4f5f0d04, user_id=03ee7d2ea7854c84925c0df91a5e66ff, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:58.283765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2ca9a-a8d5-4952-853d-81996e312828 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:58.310383 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3171/6271] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:58 2022] PUT /identity/v3/projects/5d535e5f94a14ff29c118ccd4f5f0d04/users/03ee7d2ea7854c84925c0df91a5e66ff/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:58.314248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dbb49-70cc-43ed-81bd-459b15d702ad admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:58.373945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dbb49-70cc-43ed-81bd-459b15d702ad admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:58.473271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ea20d-c6b8-432a-839c-ae62ee3e038a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:58.473919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ea20d-c6b8-432a-839c-ae62ee3e038a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:58.474564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ea20d-c6b8-432a-839c-ae62ee3e038a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:58.474564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ea20d-c6b8-432a-839c-ae62ee3e038a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:58.475290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ea20d-c6b8-432a-839c-ae62ee3e038a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:58.477319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ea20d-c6b8-432a-839c-ae62ee3e038a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:58.490209 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3101/6272] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:11:58.493775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2ca9a-a8d5-4952-853d-81996e312828 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:58.508247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2ca9a-a8d5-4952-853d-81996e312828 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:58.515617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce32ae4-fa50-41fa-947c-c6d4f798d140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:58.516340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce32ae4-fa50-41fa-947c-c6d4f798d140 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:58.516648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce32ae4-fa50-41fa-947c-c6d4f798d140 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:58.516962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce32ae4-fa50-41fa-947c-c6d4f798d140 admin admin] PATH_INFO: `/v3/projects/5d535e5f94a14ff29c118ccd4f5f0d04/users/03ee7d2ea7854c84925c0df91a5e66ff/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:58.519735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce32ae4-fa50-41fa-947c-c6d4f798d140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d535e5f94a14ff29c118ccd4f5f0d04, user_id=03ee7d2ea7854c84925c0df91a5e66ff, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:58.523383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce32ae4-fa50-41fa-947c-c6d4f798d140 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:58.551202 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3172/6273] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:11:58 2022] PUT /identity/v3/projects/5d535e5f94a14ff29c118ccd4f5f0d04/users/03ee7d2ea7854c84925c0df91a5e66ff/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:11:58.562879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3768bf-86cb-489c-b06c-2b72365a6eee None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:58.563627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3768bf-86cb-489c-b06c-2b72365a6eee None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:58.564125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3768bf-86cb-489c-b06c-2b72365a6eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:58.602950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4a3768bf-86cb-489c-b06c-2b72365a6eee None None] MFA Rules not processed for user `03ee7d2ea7854c84925c0df91a5e66ff`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:58.606822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4a3768bf-86cb-489c-b06c-2b72365a6eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:58.757973 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3102/6274] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:11:58.772754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce32ae4-fa50-41fa-947c-c6d4f798d140 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:58.784091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0ce32ae4-fa50-41fa-947c-c6d4f798d140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:58.953980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce32ae4-fa50-41fa-947c-c6d4f798d140 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:58.960163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bbd43-3a71-4626-8e64-c140f65dc7f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:58.960872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bbd43-3a71-4626-8e64-c140f65dc7f2 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:58.961200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bbd43-3a71-4626-8e64-c140f65dc7f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:58.962158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bbd43-3a71-4626-8e64-c140f65dc7f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:58.975429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bbd43-3a71-4626-8e64-c140f65dc7f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:58.978082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bbd43-3a71-4626-8e64-c140f65dc7f2 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:59.030753 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3173/6275] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:11:59.229383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3768bf-86cb-489c-b06c-2b72365a6eee None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:11:59.236518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:11:59.250655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3768bf-86cb-489c-b06c-2b72365a6eee None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:11:59.261178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0be4c-6e86-49f1-9877-42f943fc357f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:11:59.262245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0be4c-6e86-49f1-9877-42f943fc357f service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:59.262772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0be4c-6e86-49f1-9877-42f943fc357f service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:59.263170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0be4c-6e86-49f1-9877-42f943fc357f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:59.303677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbd01e-e68c-45ed-ad48-2b5f5607eb76 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:11:59.304268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbd01e-e68c-45ed-ad48-2b5f5607eb76 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:11:59.304268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbd01e-e68c-45ed-ad48-2b5f5607eb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:11:59.336952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6edbd01e-e68c-45ed-ad48-2b5f5607eb76 None None] MFA Rules not processed for user `9f792275f53f445ea72b6de441314a49`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:11:59.338946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6edbd01e-e68c-45ed-ad48-2b5f5607eb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:11:59.448886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0be4c-6e86-49f1-9877-42f943fc357f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:11:59.450791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0be4c-6e86-49f1-9877-42f943fc357f service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:11:59.479629 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3103/6276] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:11:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:11:59.491009 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3174/6277] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:02.493718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0be4c-6e86-49f1-9877-42f943fc357f service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:02.618203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0be4c-6e86-49f1-9877-42f943fc357f service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:02.629885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c654ede-8ce1-46ed-9e96-3c62cb0004b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:02.630556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c654ede-8ce1-46ed-9e96-3c62cb0004b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:02.630849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c654ede-8ce1-46ed-9e96-3c62cb0004b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:02.631134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c654ede-8ce1-46ed-9e96-3c62cb0004b5 admin admin] PATH_INFO: `/v3/users/adc8d4feb0f8454d89acc52dbffcfb3b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:02.632552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c654ede-8ce1-46ed-9e96-3c62cb0004b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adc8d4feb0f8454d89acc52dbffcfb3b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:02.634612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c654ede-8ce1-46ed-9e96-3c62cb0004b5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:02.641504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031eaede-cc9c-48e4-af3a-ff33aa9c7ed3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:02.642006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031eaede-cc9c-48e4-af3a-ff33aa9c7ed3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:02.642286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031eaede-cc9c-48e4-af3a-ff33aa9c7ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:02.709840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-031eaede-cc9c-48e4-af3a-ff33aa9c7ed3 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:02.711645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-031eaede-cc9c-48e4-af3a-ff33aa9c7ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:02.713317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c654ede-8ce1-46ed-9e96-3c62cb0004b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adc8d4feb0f8454d89acc52dbffcfb3b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:02.735749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c654ede-8ce1-46ed-9e96-3c62cb0004b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adc8d4feb0f8454d89acc52dbffcfb3b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:02.737199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c654ede-8ce1-46ed-9e96-3c62cb0004b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adc8d4feb0f8454d89acc52dbffcfb3b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:02.747716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c654ede-8ce1-46ed-9e96-3c62cb0004b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adc8d4feb0f8454d89acc52dbffcfb3b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:02.750405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:02.759599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c654ede-8ce1-46ed-9e96-3c62cb0004b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adc8d4feb0f8454d89acc52dbffcfb3b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:02.767627 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3104/6278] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:02 2022] DELETE /identity/v3/users/adc8d4feb0f8454d89acc52dbffcfb3b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:02.772362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61812c20-1183-4c91-8c02-63124b903d31 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:02.772660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61812c20-1183-4c91-8c02-63124b903d31 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:02.772973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61812c20-1183-4c91-8c02-63124b903d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:02.795142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-61812c20-1183-4c91-8c02-63124b903d31 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:02.797090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-61812c20-1183-4c91-8c02-63124b903d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:02.805379 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3175/6279] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:02.818675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031eaede-cc9c-48e4-af3a-ff33aa9c7ed3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:02.820444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-031eaede-cc9c-48e4-af3a-ff33aa9c7ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:02.833847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:02.888495 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3105/6280] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:02.910680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61812c20-1183-4c91-8c02-63124b903d31 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:02.915386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-61812c20-1183-4c91-8c02-63124b903d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:03.008391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031eaede-cc9c-48e4-af3a-ff33aa9c7ed3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:03.095882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61812c20-1183-4c91-8c02-63124b903d31 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:03.100769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01aaa07-1e9b-4470-b5f6-d9916be69d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:03.102421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01aaa07-1e9b-4470-b5f6-d9916be69d31 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:03.102707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01aaa07-1e9b-4470-b5f6-d9916be69d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:03.105088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01aaa07-1e9b-4470-b5f6-d9916be69d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:03.130402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01aaa07-1e9b-4470-b5f6-d9916be69d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:03.132259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01aaa07-1e9b-4470-b5f6-d9916be69d31 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:03.149117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199b0ff-1d4e-4c70-aee2-f19b87a5ac80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:03.149816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199b0ff-1d4e-4c70-aee2-f19b87a5ac80 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:03.150195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199b0ff-1d4e-4c70-aee2-f19b87a5ac80 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:03.150388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199b0ff-1d4e-4c70-aee2-f19b87a5ac80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:03.159627 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3106/6281] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:12:03.235715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199b0ff-1d4e-4c70-aee2-f19b87a5ac80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:03.237744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199b0ff-1d4e-4c70-aee2-f19b87a5ac80 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:03.255626 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3176/6282] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:12:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 21:12:03.884287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01aaa07-1e9b-4470-b5f6-d9916be69d31 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:03.884287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d01aaa07-1e9b-4470-b5f6-d9916be69d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:03.900729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01aaa07-1e9b-4470-b5f6-d9916be69d31 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:03.907533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:03.909195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:03.909729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:03.910326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] PATH_INFO: `/v3/projects/09bb57542ced4ce7979b9b4eb385be1a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:03.911939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=09bb57542ced4ce7979b9b4eb385be1a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:03.914890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:04.058609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09bb57542ced4ce7979b9b4eb385be1a', 'request_id': 'req-e110b7ea-86dd-4609-b41c-fc49509afd2c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:04.071870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09bb57542ced4ce7979b9b4eb385be1a', 'request_id': 'req-e110b7ea-86dd-4609-b41c-fc49509afd2c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:04.084178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09bb57542ced4ce7979b9b4eb385be1a', 'request_id': 'req-e110b7ea-86dd-4609-b41c-fc49509afd2c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:04.101581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] The token cache is being invalidate because project 09bb57542ced4ce7979b9b4eb385be1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:04.102418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:04.103342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:04.105762 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3107/6283] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:03 2022] DELETE /identity/v3/projects/09bb57542ced4ce7979b9b4eb385be1a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:04.111483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199b0ff-1d4e-4c70-aee2-f19b87a5ac80 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:04.131933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199b0ff-1d4e-4c70-aee2-f19b87a5ac80 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:04.146409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:04.225060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:04.225989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:04.226489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:04.227207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] PATH_INFO: `/v3/users/03ee7d2ea7854c84925c0df91a5e66ff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:04.228569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=03ee7d2ea7854c84925c0df91a5e66ff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:04.230654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:04.301389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ee7d2ea7854c84925c0df91a5e66ff'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:04.324399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ee7d2ea7854c84925c0df91a5e66ff'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:04.333620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ee7d2ea7854c84925c0df91a5e66ff'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:04.334796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ee7d2ea7854c84925c0df91a5e66ff'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:04.335512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ee7d2ea7854c84925c0df91a5e66ff'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:04.357058 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3177/6284] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:04 2022] DELETE /identity/v3/users/03ee7d2ea7854c84925c0df91a5e66ff => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:04.970043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:04.971539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:05.076016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:05.145648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110b7ea-86dd-4609-b41c-fc49509afd2c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:05.264293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8436bc9-64d6-44d9-af78-17f63f1b5223 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:05.265475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8436bc9-64d6-44d9-af78-17f63f1b5223 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:05.265758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8436bc9-64d6-44d9-af78-17f63f1b5223 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:05.266030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8436bc9-64d6-44d9-af78-17f63f1b5223 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:05.270754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c8436bc9-64d6-44d9-af78-17f63f1b5223 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:05.295003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8436bc9-64d6-44d9-af78-17f63f1b5223 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:05.298211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8436bc9-64d6-44d9-af78-17f63f1b5223 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:05.317059 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3108/6285] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:05.415272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:05.417802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:05.533018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98aa5b-f66f-43da-9099-9c1ad85f29dd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:05.541112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71063495-5875-4858-bd74-e48d8f5837c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:05.541707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71063495-5875-4858-bd74-e48d8f5837c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:05.542006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71063495-5875-4858-bd74-e48d8f5837c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:05.542286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71063495-5875-4858-bd74-e48d8f5837c0 admin admin] PATH_INFO: `/v3/projects/5d535e5f94a14ff29c118ccd4f5f0d04` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:05.543787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71063495-5875-4858-bd74-e48d8f5837c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d535e5f94a14ff29c118ccd4f5f0d04)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:05.549627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71063495-5875-4858-bd74-e48d8f5837c0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:05.597712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:05.616988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-71063495-5875-4858-bd74-e48d8f5837c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d535e5f94a14ff29c118ccd4f5f0d04', 'request_id': 'req-71063495-5875-4858-bd74-e48d8f5837c0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:05.625703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-71063495-5875-4858-bd74-e48d8f5837c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d535e5f94a14ff29c118ccd4f5f0d04', 'request_id': 'req-71063495-5875-4858-bd74-e48d8f5837c0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:05.633146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-71063495-5875-4858-bd74-e48d8f5837c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d535e5f94a14ff29c118ccd4f5f0d04', 'request_id': 'req-71063495-5875-4858-bd74-e48d8f5837c0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:05.646669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-71063495-5875-4858-bd74-e48d8f5837c0 admin admin] The token cache is being invalidate because project 5d535e5f94a14ff29c118ccd4f5f0d04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:05.646945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-71063495-5875-4858-bd74-e48d8f5837c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:05.647604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-71063495-5875-4858-bd74-e48d8f5837c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:05.649299 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3178/6286] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:05 2022] DELETE /identity/v3/projects/5d535e5f94a14ff29c118ccd4f5f0d04 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:05.654223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ad8cd-c0a0-4ba4-96db-e6e24c2f0101 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:05.654551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ad8cd-c0a0-4ba4-96db-e6e24c2f0101 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:05.654804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ad8cd-c0a0-4ba4-96db-e6e24c2f0101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:05.681223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c16ad8cd-c0a0-4ba4-96db-e6e24c2f0101 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:05.682609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c16ad8cd-c0a0-4ba4-96db-e6e24c2f0101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:05.755118 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3109/6287] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:05.763602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367be6ee-64d6-4b7c-9d77-20dc877abe3f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:05.763973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367be6ee-64d6-4b7c-9d77-20dc877abe3f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:05.764285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367be6ee-64d6-4b7c-9d77-20dc877abe3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:05.784796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-367be6ee-64d6-4b7c-9d77-20dc877abe3f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:05.786061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-367be6ee-64d6-4b7c-9d77-20dc877abe3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:05.794153 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3179/6288] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:05.798581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ad8cd-c0a0-4ba4-96db-e6e24c2f0101 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:05.808502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ad8cd-c0a0-4ba4-96db-e6e24c2f0101 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:05.813801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b2a39-c4ce-44a4-b17b-23692bf47cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:05.814656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b2a39-c4ce-44a4-b17b-23692bf47cbb admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:05.814781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b2a39-c4ce-44a4-b17b-23692bf47cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:05.814967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b2a39-c4ce-44a4-b17b-23692bf47cbb admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:05.815615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b2a39-c4ce-44a4-b17b-23692bf47cbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:12:05.815920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b2a39-c4ce-44a4-b17b-23692bf47cbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:05.817585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b2a39-c4ce-44a4-b17b-23692bf47cbb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:05.833854 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3110/6289] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:05.834960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367be6ee-64d6-4b7c-9d77-20dc877abe3f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:05.845343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367be6ee-64d6-4b7c-9d77-20dc877abe3f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:05.850727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc0db6-ea34-410a-b596-f0e666c7b816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:05.851352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc0db6-ea34-410a-b596-f0e666c7b816 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:05.859880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc0db6-ea34-410a-b596-f0e666c7b816 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:05.860652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc0db6-ea34-410a-b596-f0e666c7b816 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:05.862301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc0db6-ea34-410a-b596-f0e666c7b816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:05.866343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc0db6-ea34-410a-b596-f0e666c7b816 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:05.911039 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3180/6290] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:05 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:05.915198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b2a39-c4ce-44a4-b17b-23692bf47cbb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:05.931551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b2a39-c4ce-44a4-b17b-23692bf47cbb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:05.997623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e50b6a-3ead-459f-a293-43305a7c09a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:05.998281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e50b6a-3ead-459f-a293-43305a7c09a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:05.998532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e50b6a-3ead-459f-a293-43305a7c09a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:05.999340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e50b6a-3ead-459f-a293-43305a7c09a7 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:06.000268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e50b6a-3ead-459f-a293-43305a7c09a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:06.002295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e50b6a-3ead-459f-a293-43305a7c09a7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:06.023352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:06.030955 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3111/6291] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:05 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:06.035462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc0db6-ea34-410a-b596-f0e666c7b816 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:06.047679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc0db6-ea34-410a-b596-f0e666c7b816 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:06.053128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883bea8-f1b4-4258-b225-391ff95b2f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:06.053969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883bea8-f1b4-4258-b225-391ff95b2f43 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:06.054257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883bea8-f1b4-4258-b225-391ff95b2f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:06.054535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883bea8-f1b4-4258-b225-391ff95b2f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:06.055179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883bea8-f1b4-4258-b225-391ff95b2f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:06.056859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883bea8-f1b4-4258-b225-391ff95b2f43 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:06.073681 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3181/6292] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:06.074976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e50b6a-3ead-459f-a293-43305a7c09a7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:06.085600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e50b6a-3ead-459f-a293-43305a7c09a7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:06.090610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730c6ad-0ca4-4496-8909-e09fbc2aed10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:06.091199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730c6ad-0ca4-4496-8909-e09fbc2aed10 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:06.091557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730c6ad-0ca4-4496-8909-e09fbc2aed10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:06.091978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730c6ad-0ca4-4496-8909-e09fbc2aed10 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:06.093282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730c6ad-0ca4-4496-8909-e09fbc2aed10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:06.094991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730c6ad-0ca4-4496-8909-e09fbc2aed10 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:06.107913 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3112/6293] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:06.112349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883bea8-f1b4-4258-b225-391ff95b2f43 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:06.136390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883bea8-f1b4-4258-b225-391ff95b2f43 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:06.141767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612008e5-be87-44f9-b0c7-8808e184cff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:06.142610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612008e5-be87-44f9-b0c7-8808e184cff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:06.142902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612008e5-be87-44f9-b0c7-8808e184cff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:06.143207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612008e5-be87-44f9-b0c7-8808e184cff8 admin admin] PATH_INFO: `/v3/projects/ed7802c1f2d04316aa5a26fa45e61b3a/users/04ef3241ccad461a97b857c58af91e5b/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:06.164645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612008e5-be87-44f9-b0c7-8808e184cff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7802c1f2d04316aa5a26fa45e61b3a, user_id=04ef3241ccad461a97b857c58af91e5b, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:06.168517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612008e5-be87-44f9-b0c7-8808e184cff8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:06.184966 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3182/6294] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:06 2022] PUT /identity/v3/projects/ed7802c1f2d04316aa5a26fa45e61b3a/users/04ef3241ccad461a97b857c58af91e5b/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:06.197495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509d32b-7a9e-40e1-a6f6-736af017758e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:06.197788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509d32b-7a9e-40e1-a6f6-736af017758e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:06.198121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509d32b-7a9e-40e1-a6f6-736af017758e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:06.237933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8509d32b-7a9e-40e1-a6f6-736af017758e None None] MFA Rules not processed for user `04ef3241ccad461a97b857c58af91e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:06.239170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8509d32b-7a9e-40e1-a6f6-736af017758e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:06.331215 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3113/6295] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:06.333898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612008e5-be87-44f9-b0c7-8808e184cff8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:06.441684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612008e5-be87-44f9-b0c7-8808e184cff8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:06.447832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71171de-7f1d-4ae3-94d1-6cd7635089ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:06.448676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71171de-7f1d-4ae3-94d1-6cd7635089ed admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:06.449056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71171de-7f1d-4ae3-94d1-6cd7635089ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:06.449402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71171de-7f1d-4ae3-94d1-6cd7635089ed admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:06.450367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71171de-7f1d-4ae3-94d1-6cd7635089ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:06.453642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71171de-7f1d-4ae3-94d1-6cd7635089ed admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:06.502474 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3183/6296] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:06 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:06.503896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509d32b-7a9e-40e1-a6f6-736af017758e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:06.516581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509d32b-7a9e-40e1-a6f6-736af017758e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:06.589633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bfac4-a979-4078-811c-eb0fc6c43c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:06.590402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bfac4-a979-4078-811c-eb0fc6c43c9b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:06.590733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bfac4-a979-4078-811c-eb0fc6c43c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:06.591162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bfac4-a979-4078-811c-eb0fc6c43c9b admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:06.591830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bfac4-a979-4078-811c-eb0fc6c43c9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:06.593823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bfac4-a979-4078-811c-eb0fc6c43c9b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:06.616231 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3114/6297] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:06 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:06.620229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71171de-7f1d-4ae3-94d1-6cd7635089ed admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:06.631709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71171de-7f1d-4ae3-94d1-6cd7635089ed admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:06.636475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e0142-69c7-4874-b03c-eb2963e9cc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:06.637126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e0142-69c7-4874-b03c-eb2963e9cc89 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:06.637418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e0142-69c7-4874-b03c-eb2963e9cc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:06.637697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e0142-69c7-4874-b03c-eb2963e9cc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:06.638395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e0142-69c7-4874-b03c-eb2963e9cc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:06.640659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e0142-69c7-4874-b03c-eb2963e9cc89 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:06.664700 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3184/6298] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:06.668748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bfac4-a979-4078-811c-eb0fc6c43c9b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:06.680399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bfac4-a979-4078-811c-eb0fc6c43c9b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:06.689651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f67a1e-4fdf-40f9-970e-5ac02538e719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:06.690434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f67a1e-4fdf-40f9-970e-5ac02538e719 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:06.691342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f67a1e-4fdf-40f9-970e-5ac02538e719 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:06.691342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f67a1e-4fdf-40f9-970e-5ac02538e719 admin admin] PATH_INFO: `/v3/projects/e3fdb7ff52834e2383900d7f133f00de/users/05cf2b72fd6b45ccb254c7b7af607789/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:06.722189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f67a1e-4fdf-40f9-970e-5ac02538e719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3fdb7ff52834e2383900d7f133f00de, user_id=05cf2b72fd6b45ccb254c7b7af607789, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:06.727308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f67a1e-4fdf-40f9-970e-5ac02538e719 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:06.744698 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3115/6299] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:06 2022] PUT /identity/v3/projects/e3fdb7ff52834e2383900d7f133f00de/users/05cf2b72fd6b45ccb254c7b7af607789/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:06.749384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e0142-69c7-4874-b03c-eb2963e9cc89 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:06.771101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e0142-69c7-4874-b03c-eb2963e9cc89 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:06.814860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:06.863030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74e6ce-9bb5-4c9a-837b-379fb00d5056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:06.864018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74e6ce-9bb5-4c9a-837b-379fb00d5056 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:06.864311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74e6ce-9bb5-4c9a-837b-379fb00d5056 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:06.864926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74e6ce-9bb5-4c9a-837b-379fb00d5056 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:06.865348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74e6ce-9bb5-4c9a-837b-379fb00d5056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:06.867725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74e6ce-9bb5-4c9a-837b-379fb00d5056 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:06.905324 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3185/6300] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:06.907379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f67a1e-4fdf-40f9-970e-5ac02538e719 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:06.923104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f67a1e-4fdf-40f9-970e-5ac02538e719 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:06.943047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22832b-baa7-4682-8a09-2a7529d3ff74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:06.943842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22832b-baa7-4682-8a09-2a7529d3ff74 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:06.944139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22832b-baa7-4682-8a09-2a7529d3ff74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:06.944602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22832b-baa7-4682-8a09-2a7529d3ff74 admin admin] PATH_INFO: `/v3/projects/e3fdb7ff52834e2383900d7f133f00de/users/05cf2b72fd6b45ccb254c7b7af607789/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:06.950827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22832b-baa7-4682-8a09-2a7529d3ff74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3fdb7ff52834e2383900d7f133f00de, user_id=05cf2b72fd6b45ccb254c7b7af607789, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:06.954183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22832b-baa7-4682-8a09-2a7529d3ff74 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:06.968941 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3116/6301] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:06 2022] PUT /identity/v3/projects/e3fdb7ff52834e2383900d7f133f00de/users/05cf2b72fd6b45ccb254c7b7af607789/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:06.980662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff04ec-ac6f-4efb-bb30-c77d4e924aa7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:06.981018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff04ec-ac6f-4efb-bb30-c77d4e924aa7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:06.981274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff04ec-ac6f-4efb-bb30-c77d4e924aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:07.020355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-71ff04ec-ac6f-4efb-bb30-c77d4e924aa7 None None] MFA Rules not processed for user `05cf2b72fd6b45ccb254c7b7af607789`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:07.021700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-71ff04ec-ac6f-4efb-bb30-c77d4e924aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:07.168572 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3186/6302] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:07.180383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22832b-baa7-4682-8a09-2a7529d3ff74 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:07.183524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ca22832b-baa7-4682-8a09-2a7529d3ff74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:07.198498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff04ec-ac6f-4efb-bb30-c77d4e924aa7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:07.200163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-71ff04ec-ac6f-4efb-bb30-c77d4e924aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:07.225533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:07.297094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22832b-baa7-4682-8a09-2a7529d3ff74 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:07.304293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff04ec-ac6f-4efb-bb30-c77d4e924aa7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:07.308301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd91e01-c4ba-4788-ae16-624cbd9855b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:07.309002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd91e01-c4ba-4788-ae16-624cbd9855b4 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:07.309168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51765b0e-08bb-4d3c-8385-189a14c2de03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:07.309294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd91e01-c4ba-4788-ae16-624cbd9855b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:07.309603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd91e01-c4ba-4788-ae16-624cbd9855b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:07.309812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51765b0e-08bb-4d3c-8385-189a14c2de03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:07.310099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51765b0e-08bb-4d3c-8385-189a14c2de03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:07.310379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51765b0e-08bb-4d3c-8385-189a14c2de03 admin admin] PATH_INFO: `/v3/users/38dddfbdc4f0450e8d5fc7b214796758` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:07.311689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51765b0e-08bb-4d3c-8385-189a14c2de03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38dddfbdc4f0450e8d5fc7b214796758)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:07.313673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51765b0e-08bb-4d3c-8385-189a14c2de03 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:07.320375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd91e01-c4ba-4788-ae16-624cbd9855b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:07.321677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd91e01-c4ba-4788-ae16-624cbd9855b4 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:07.334569 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3117/6303] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:07.370989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-51765b0e-08bb-4d3c-8385-189a14c2de03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38dddfbdc4f0450e8d5fc7b214796758'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:07.378938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:07.392794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-51765b0e-08bb-4d3c-8385-189a14c2de03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38dddfbdc4f0450e8d5fc7b214796758'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:07.402699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-51765b0e-08bb-4d3c-8385-189a14c2de03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38dddfbdc4f0450e8d5fc7b214796758'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:07.405738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-51765b0e-08bb-4d3c-8385-189a14c2de03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38dddfbdc4f0450e8d5fc7b214796758'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:07.406267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-51765b0e-08bb-4d3c-8385-189a14c2de03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38dddfbdc4f0450e8d5fc7b214796758'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:07.423317 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3187/6304] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:07 2022] DELETE /identity/v3/users/38dddfbdc4f0450e8d5fc7b214796758 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:07.745674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd91e01-c4ba-4788-ae16-624cbd9855b4 service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:07.765481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd91e01-c4ba-4788-ae16-624cbd9855b4 service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:07.838970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:07.839932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:07.840290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:07.840645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17 admin admin] PATH_INFO: `/v3/projects/0f4fe674ef004caf8df73b8b06f7dd23` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:07.842981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f4fe674ef004caf8df73b8b06f7dd23)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:07.845080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:07.910239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f4fe674ef004caf8df73b8b06f7dd23', 'request_id': 'req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:07.922149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f4fe674ef004caf8df73b8b06f7dd23', 'request_id': 'req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:07.929734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f4fe674ef004caf8df73b8b06f7dd23', 'request_id': 'req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:07.942648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17 admin admin] The token cache is being invalidate because project 0f4fe674ef004caf8df73b8b06f7dd23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:07.942936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:07.943626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-36cfc97d-10e8-4d42-b1de-cfb3fab88c17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:07.949991 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3118/6305] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:07 2022] DELETE /identity/v3/projects/0f4fe674ef004caf8df73b8b06f7dd23 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:07.952258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97da7d-ce38-4330-b3cc-43ac9bb9b302 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:07.953169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97da7d-ce38-4330-b3cc-43ac9bb9b302 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:07.953453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97da7d-ce38-4330-b3cc-43ac9bb9b302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:07.978487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bf97da7d-ce38-4330-b3cc-43ac9bb9b302 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:07.979742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bf97da7d-ce38-4330-b3cc-43ac9bb9b302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:08.084122 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3188/6306] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:08.092269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a745f-e1e6-4559-b54a-42d6e3e45ee1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:08.092588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a745f-e1e6-4559-b54a-42d6e3e45ee1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:08.092814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a745f-e1e6-4559-b54a-42d6e3e45ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:08.118847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-420a745f-e1e6-4559-b54a-42d6e3e45ee1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:08.120363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-420a745f-e1e6-4559-b54a-42d6e3e45ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:08.147366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97da7d-ce38-4330-b3cc-43ac9bb9b302 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:08.148074 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3119/6307] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:08.149087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bf97da7d-ce38-4330-b3cc-43ac9bb9b302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:08.152205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a745f-e1e6-4559-b54a-42d6e3e45ee1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:08.163554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a745f-e1e6-4559-b54a-42d6e3e45ee1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:08.166172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97da7d-ce38-4330-b3cc-43ac9bb9b302 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:08.171743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc6ada-315d-4e9e-b8f7-2501969a1c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:08.173022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc6ada-315d-4e9e-b8f7-2501969a1c2d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:08.173843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc6ada-315d-4e9e-b8f7-2501969a1c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:08.174431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc6ada-315d-4e9e-b8f7-2501969a1c2d admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:08.175207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc6ada-315d-4e9e-b8f7-2501969a1c2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:12:08.175525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc6ada-315d-4e9e-b8f7-2501969a1c2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:08.175649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab88c89-5dd3-4495-90b7-9c896e0c7a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:08.176351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab88c89-5dd3-4495-90b7-9c896e0c7a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:08.176646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab88c89-5dd3-4495-90b7-9c896e0c7a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:08.176989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab88c89-5dd3-4495-90b7-9c896e0c7a32 admin admin] PATH_INFO: `/v3/users/04ef3241ccad461a97b857c58af91e5b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:08.177450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc6ada-315d-4e9e-b8f7-2501969a1c2d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:08.178217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab88c89-5dd3-4495-90b7-9c896e0c7a32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04ef3241ccad461a97b857c58af91e5b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:08.181389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab88c89-5dd3-4495-90b7-9c896e0c7a32 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:08.200455 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3120/6308] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:12:08.202566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc6ada-315d-4e9e-b8f7-2501969a1c2d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:08.215624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc6ada-315d-4e9e-b8f7-2501969a1c2d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:08.224527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ef492-6a96-4875-a883-8214770929ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:08.226204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ef492-6a96-4875-a883-8214770929ba admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:08.227098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ef492-6a96-4875-a883-8214770929ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:08.227986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ef492-6a96-4875-a883-8214770929ba admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:08.228841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ef492-6a96-4875-a883-8214770929ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:08.232694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ef492-6a96-4875-a883-8214770929ba admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:08.266116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bab88c89-5dd3-4495-90b7-9c896e0c7a32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04ef3241ccad461a97b857c58af91e5b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:08.274263 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3121/6309] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:08 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:08.276669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ef492-6a96-4875-a883-8214770929ba admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:08.290386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bab88c89-5dd3-4495-90b7-9c896e0c7a32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04ef3241ccad461a97b857c58af91e5b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:08.302303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bab88c89-5dd3-4495-90b7-9c896e0c7a32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04ef3241ccad461a97b857c58af91e5b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:08.303271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bab88c89-5dd3-4495-90b7-9c896e0c7a32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04ef3241ccad461a97b857c58af91e5b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:08.304568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bab88c89-5dd3-4495-90b7-9c896e0c7a32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04ef3241ccad461a97b857c58af91e5b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:08.321065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:08.354322 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3189/6310] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:08 2022] DELETE /identity/v3/users/04ef3241ccad461a97b857c58af91e5b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:08.358635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb21ce5-4767-4121-a147-a92ff0ca681b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:08.359032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb21ce5-4767-4121-a147-a92ff0ca681b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:08.359219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb21ce5-4767-4121-a147-a92ff0ca681b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:08.397158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ef492-6a96-4875-a883-8214770929ba admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:08.397785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-eeb21ce5-4767-4121-a147-a92ff0ca681b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:08.399044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-eeb21ce5-4767-4121-a147-a92ff0ca681b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:08.462567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:08.476400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525098-e9e2-4626-8af3-8e1650c97086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:08.477597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525098-e9e2-4626-8af3-8e1650c97086 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:08.478010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525098-e9e2-4626-8af3-8e1650c97086 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:08.478339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525098-e9e2-4626-8af3-8e1650c97086 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:08.479027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37525098-e9e2-4626-8af3-8e1650c97086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:08.481302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37525098-e9e2-4626-8af3-8e1650c97086 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:08.502740 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3190/6311] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:08.512942 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3122/6312] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:08 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:08.513263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb21ce5-4767-4121-a147-a92ff0ca681b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:08.514760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-eeb21ce5-4767-4121-a147-a92ff0ca681b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:08.515816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525098-e9e2-4626-8af3-8e1650c97086 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:08.518592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-37525098-e9e2-4626-8af3-8e1650c97086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:08.532336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525098-e9e2-4626-8af3-8e1650c97086 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:08.538556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a113e0-6f11-41d7-b17a-8ef958cbb1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:08.539554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a113e0-6f11-41d7-b17a-8ef958cbb1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:08.540201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a113e0-6f11-41d7-b17a-8ef958cbb1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:08.540751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a113e0-6f11-41d7-b17a-8ef958cbb1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:08.541778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a113e0-6f11-41d7-b17a-8ef958cbb1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:08.544616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a113e0-6f11-41d7-b17a-8ef958cbb1d5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:08.562021 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3123/6313] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:08.564155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a113e0-6f11-41d7-b17a-8ef958cbb1d5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:08.575560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a113e0-6f11-41d7-b17a-8ef958cbb1d5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:08.580422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c59d0-5778-46f9-92fb-c1bf6a0b04ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:08.581048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c59d0-5778-46f9-92fb-c1bf6a0b04ba admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:08.581376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c59d0-5778-46f9-92fb-c1bf6a0b04ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:08.581655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c59d0-5778-46f9-92fb-c1bf6a0b04ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:08.582326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c59d0-5778-46f9-92fb-c1bf6a0b04ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:08.583907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb21ce5-4767-4121-a147-a92ff0ca681b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:08.584044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c59d0-5778-46f9-92fb-c1bf6a0b04ba admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:08.589721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d8788-aa06-484f-9388-a81c16f49ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:08.590332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d8788-aa06-484f-9388-a81c16f49ffa service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:08.590614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d8788-aa06-484f-9388-a81c16f49ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:08.590919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d8788-aa06-484f-9388-a81c16f49ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:08.598053 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3124/6314] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:08.602132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c59d0-5778-46f9-92fb-c1bf6a0b04ba admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:08.603601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d8788-aa06-484f-9388-a81c16f49ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:08.605133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d8788-aa06-484f-9388-a81c16f49ffa service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:08.614749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c59d0-5778-46f9-92fb-c1bf6a0b04ba admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:08.616375 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3191/6315] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:08.620678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be393472-1e37-4a83-a6cf-c45884fca9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:08.621679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be393472-1e37-4a83-a6cf-c45884fca9cf admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:08.622217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be393472-1e37-4a83-a6cf-c45884fca9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:08.622743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be393472-1e37-4a83-a6cf-c45884fca9cf admin admin] PATH_INFO: `/v3/projects/ae7df719c1d54fddbf9956b4429735ec/users/cb24530016974c70884bb3bc9b91e69c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:08.641804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be393472-1e37-4a83-a6cf-c45884fca9cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7df719c1d54fddbf9956b4429735ec, user_id=cb24530016974c70884bb3bc9b91e69c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:08.645516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be393472-1e37-4a83-a6cf-c45884fca9cf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:08.660132 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3125/6316] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:08 2022] PUT /identity/v3/projects/ae7df719c1d54fddbf9956b4429735ec/users/cb24530016974c70884bb3bc9b91e69c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:08.671085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3f57b-0884-4a50-a53c-ccb986896e4b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:08.671374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3f57b-0884-4a50-a53c-ccb986896e4b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:08.671741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3f57b-0884-4a50-a53c-ccb986896e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:08.726138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-05a3f57b-0884-4a50-a53c-ccb986896e4b None None] MFA Rules not processed for user `cb24530016974c70884bb3bc9b91e69c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:08.727750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-05a3f57b-0884-4a50-a53c-ccb986896e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:08.813981 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3192/6317] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:08.819475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be393472-1e37-4a83-a6cf-c45884fca9cf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:08.910742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be393472-1e37-4a83-a6cf-c45884fca9cf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:08.917476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44beb9-8b0b-47df-9c46-940334f242dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:08.918139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44beb9-8b0b-47df-9c46-940334f242dc admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:08.918433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44beb9-8b0b-47df-9c46-940334f242dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:08.919167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44beb9-8b0b-47df-9c46-940334f242dc admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:08.919961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44beb9-8b0b-47df-9c46-940334f242dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:08.922935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44beb9-8b0b-47df-9c46-940334f242dc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:08.959012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:08.985911 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3126/6318] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:08 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:08.989789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3f57b-0884-4a50-a53c-ccb986896e4b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:09.003540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3f57b-0884-4a50-a53c-ccb986896e4b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:09.073037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44beb9-8b0b-47df-9c46-940334f242dc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:09.075173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fc44beb9-8b0b-47df-9c46-940334f242dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:09.076711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b4217-70c0-4c7d-bbb6-8dc6a8bb48e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:09.077379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b4217-70c0-4c7d-bbb6-8dc6a8bb48e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:09.077665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b4217-70c0-4c7d-bbb6-8dc6a8bb48e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:09.078747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b4217-70c0-4c7d-bbb6-8dc6a8bb48e0 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:09.079396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b4217-70c0-4c7d-bbb6-8dc6a8bb48e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:09.081923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b4217-70c0-4c7d-bbb6-8dc6a8bb48e0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:09.097929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44beb9-8b0b-47df-9c46-940334f242dc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:09.105784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:09.106642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:09.106949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:09.107363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] PATH_INFO: `/v3/projects/ed7802c1f2d04316aa5a26fa45e61b3a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:09.109441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed7802c1f2d04316aa5a26fa45e61b3a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:09.109866 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3193/6319] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:08 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:09.111589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:09.113830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b4217-70c0-4c7d-bbb6-8dc6a8bb48e0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:09.122428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:09.128403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b4217-70c0-4c7d-bbb6-8dc6a8bb48e0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:09.142615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817a082-cfe8-4270-a032-1cabfaa4132a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:09.143957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817a082-cfe8-4270-a032-1cabfaa4132a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:09.144307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817a082-cfe8-4270-a032-1cabfaa4132a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:09.144657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817a082-cfe8-4270-a032-1cabfaa4132a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:09.145715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817a082-cfe8-4270-a032-1cabfaa4132a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:09.149021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817a082-cfe8-4270-a032-1cabfaa4132a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:09.167811 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3194/6320] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:12:09.169437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817a082-cfe8-4270-a032-1cabfaa4132a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:09.182440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed7802c1f2d04316aa5a26fa45e61b3a', 'request_id': 'req-86c06d64-9bc5-417c-bb43-193b4bcd6da3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:09.186920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817a082-cfe8-4270-a032-1cabfaa4132a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:09.191908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed7802c1f2d04316aa5a26fa45e61b3a', 'request_id': 'req-86c06d64-9bc5-417c-bb43-193b4bcd6da3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:09.204255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed7802c1f2d04316aa5a26fa45e61b3a', 'request_id': 'req-86c06d64-9bc5-417c-bb43-193b4bcd6da3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:09.226351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] The token cache is being invalidate because project ed7802c1f2d04316aa5a26fa45e61b3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:09.226616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:09.227575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:09.230143 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3127/6321] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:09 2022] DELETE /identity/v3/projects/ed7802c1f2d04316aa5a26fa45e61b3a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:09.234020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:09.235965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:09.256745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e3109-2e0f-4cef-a8e2-e53aab4c765c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:09.259103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e3109-2e0f-4cef-a8e2-e53aab4c765c admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:09.259402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e3109-2e0f-4cef-a8e2-e53aab4c765c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:09.259718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e3109-2e0f-4cef-a8e2-e53aab4c765c admin admin] PATH_INFO: `/v3/projects/3531cf1808a1403280f66316941d2501/users/adaae89daed141838dba7860e2085b7b/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:09.279722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e3109-2e0f-4cef-a8e2-e53aab4c765c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3531cf1808a1403280f66316941d2501, user_id=adaae89daed141838dba7860e2085b7b, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:09.283886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e3109-2e0f-4cef-a8e2-e53aab4c765c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:09.304882 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3195/6322] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:09 2022] PUT /identity/v3/projects/3531cf1808a1403280f66316941d2501/users/adaae89daed141838dba7860e2085b7b/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:09.305825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e3109-2e0f-4cef-a8e2-e53aab4c765c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:09.307485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e59e3109-2e0f-4cef-a8e2-e53aab4c765c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:09.332292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c06d64-9bc5-417c-bb43-193b4bcd6da3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:09.410779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e3109-2e0f-4cef-a8e2-e53aab4c765c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:09.417123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618be959-afd2-42a8-9908-311d599fe23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:09.418046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618be959-afd2-42a8-9908-311d599fe23b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:09.418340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618be959-afd2-42a8-9908-311d599fe23b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:09.419145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618be959-afd2-42a8-9908-311d599fe23b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:09.420312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618be959-afd2-42a8-9908-311d599fe23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:09.422275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618be959-afd2-42a8-9908-311d599fe23b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:09.434077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:09.434838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:09.435195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:09.435551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] PATH_INFO: `/v3/users/05cf2b72fd6b45ccb254c7b7af607789` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:09.437028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=05cf2b72fd6b45ccb254c7b7af607789)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:09.441039 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3196/6323] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:12:09.441201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:09.441296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618be959-afd2-42a8-9908-311d599fe23b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:09.444478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-618be959-afd2-42a8-9908-311d599fe23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:09.493977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:09.529027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05cf2b72fd6b45ccb254c7b7af607789'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:09.544931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05cf2b72fd6b45ccb254c7b7af607789'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:09.545832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05cf2b72fd6b45ccb254c7b7af607789'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:09.554181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05cf2b72fd6b45ccb254c7b7af607789'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:09.568733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05cf2b72fd6b45ccb254c7b7af607789'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:09.577706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618be959-afd2-42a8-9908-311d599fe23b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:09.580002 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3128/6324] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:09 2022] DELETE /identity/v3/users/05cf2b72fd6b45ccb254c7b7af607789 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:09.580839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:09.583436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:09.595675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:09.665516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330ab43-193f-43e4-9081-f57a94b02edf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:09.670664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69698c0d-2d40-43fa-a10f-ef1f94c87f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:09.671357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69698c0d-2d40-43fa-a10f-ef1f94c87f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:09.671629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69698c0d-2d40-43fa-a10f-ef1f94c87f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:09.671913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69698c0d-2d40-43fa-a10f-ef1f94c87f3b admin admin] PATH_INFO: `/v3/projects/3531cf1808a1403280f66316941d2501/users/adaae89daed141838dba7860e2085b7b/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:09.673950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69698c0d-2d40-43fa-a10f-ef1f94c87f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3531cf1808a1403280f66316941d2501, user_id=adaae89daed141838dba7860e2085b7b, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:09.676627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69698c0d-2d40-43fa-a10f-ef1f94c87f3b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:09.679454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850aafd-0f53-4586-be2a-fc07889c0063 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:09.680451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850aafd-0f53-4586-be2a-fc07889c0063 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:09.681074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850aafd-0f53-4586-be2a-fc07889c0063 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:09.681267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850aafd-0f53-4586-be2a-fc07889c0063 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:09.683700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2850aafd-0f53-4586-be2a-fc07889c0063 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:09.688228 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3129/6325] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:09 2022] PUT /identity/v3/projects/3531cf1808a1403280f66316941d2501/users/adaae89daed141838dba7860e2085b7b/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:09.699587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850aafd-0f53-4586-be2a-fc07889c0063 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:09.699848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e283e-2171-40a5-a4f0-8bb2f539db0c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:09.699979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e283e-2171-40a5-a4f0-8bb2f539db0c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:09.700259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e283e-2171-40a5-a4f0-8bb2f539db0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:09.701049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850aafd-0f53-4586-be2a-fc07889c0063 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:09.715593 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3197/6326] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:09.738912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c55e283e-2171-40a5-a4f0-8bb2f539db0c None None] MFA Rules not processed for user `adaae89daed141838dba7860e2085b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:09.740181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c55e283e-2171-40a5-a4f0-8bb2f539db0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:09.827461 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3130/6327] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:09.827981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850aafd-0f53-4586-be2a-fc07889c0063 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:09.902244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850aafd-0f53-4586-be2a-fc07889c0063 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:09.910813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a36a10-8c00-4801-9962-2a1da450c3c8 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adaae89daed141838dba7860e2085b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3531cf1808a1403280f66316941d2501', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:09.911448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a36a10-8c00-4801-9962-2a1da450c3c8 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:09.911796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a36a10-8c00-4801-9962-2a1da450c3c8 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:09.912084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a36a10-8c00-4801-9962-2a1da450c3c8 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:09.912750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a36a10-8c00-4801-9962-2a1da450c3c8 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:09.914415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a36a10-8c00-4801-9962-2a1da450c3c8 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-d3a36a10-8c00-4801-9962-2a1da450c3c8 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invalid input for field 'name': '' is too short Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: Failed validating 'minLength' in schema['properties']['name']: Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: On instance['name']: Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: Failed validating 'minLength' in schema['properties']['name']: Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: On instance['name']: Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: '' Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise error Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application On instance['name']: Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application '' Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:09.922001 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 03 21:12:09.923745 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 03 21:12:09.925007 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 03 21:12:09.925007 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 03 21:12:09.925007 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:09.925007 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 03 21:12:09.925007 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:12:09.925007 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:09.925007 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application On instance['name']: Sep 03 21:12:09.925007 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application '' Sep 03 21:12:09.925007 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:09.925007 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3198/6328] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:12:09 2022] POST /identity/v3/domains => generated 283 bytes in 96 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 03 21:12:09.928372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e283e-2171-40a5-a4f0-8bb2f539db0c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:09.938856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e283e-2171-40a5-a4f0-8bb2f539db0c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:09.944316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f3464-ba92-4885-8fa2-d64fbdaea822 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adaae89daed141838dba7860e2085b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3531cf1808a1403280f66316941d2501', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:09.945051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f3464-ba92-4885-8fa2-d64fbdaea822 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:09.945521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f3464-ba92-4885-8fa2-d64fbdaea822 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:09.945705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f3464-ba92-4885-8fa2-d64fbdaea822 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:09.946724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f3464-ba92-4885-8fa2-d64fbdaea822 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:09.948172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f3464-ba92-4885-8fa2-d64fbdaea822 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-c02f3464-ba92-4885-8fa2-d64fbdaea822 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: Failed validating 'maxLength' in schema['properties']['name']: Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: On instance['name']: Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: Failed validating 'maxLength' in schema['properties']['name']: Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: On instance['name']: Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise error Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:12:09.955338 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application On instance['name']: Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Sep 03 21:12:09.956869 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application On instance['name']: Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:09.958140 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3131/6329] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:12:09 2022] POST /identity/v3/domains => generated 412 bytes in 29 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 03 21:12:09.961772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a36a10-8c00-4801-9962-2a1da450c3c8 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:09.973783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a36a10-8c00-4801-9962-2a1da450c3c8 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:09.980850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986bd7f-44d5-4e0e-9a98-95f975b50d14 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adaae89daed141838dba7860e2085b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3531cf1808a1403280f66316941d2501', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:09.981615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986bd7f-44d5-4e0e-9a98-95f975b50d14 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:09.981959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986bd7f-44d5-4e0e-9a98-95f975b50d14 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:09.982309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986bd7f-44d5-4e0e-9a98-95f975b50d14 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:09.983707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986bd7f-44d5-4e0e-9a98-95f975b50d14 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:09.985517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986bd7f-44d5-4e0e-9a98-95f975b50d14 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.006511 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3199/6330] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:12:09 2022] POST /identity/v3/domains => generated 277 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:10.011154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f3464-ba92-4885-8fa2-d64fbdaea822 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.029104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f3464-ba92-4885-8fa2-d64fbdaea822 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.033443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986bd7f-44d5-4e0e-9a98-95f975b50d14 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.035513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1986bd7f-44d5-4e0e-9a98-95f975b50d14 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:10.063961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:10.142240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a83e1-6a5e-4042-8457-83fcbc0de05b tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adaae89daed141838dba7860e2085b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3531cf1808a1403280f66316941d2501', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.142997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a83e1-6a5e-4042-8457-83fcbc0de05b tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.143355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a83e1-6a5e-4042-8457-83fcbc0de05b tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.143710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a83e1-6a5e-4042-8457-83fcbc0de05b tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] PATH_INFO: `/v3/domains/0153438b465c414283682dff164bd3ea` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.150826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986bd7f-44d5-4e0e-9a98-95f975b50d14 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.156825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.157267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a83e1-6a5e-4042-8457-83fcbc0de05b tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=0153438b465c414283682dff164bd3ea)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:10.157629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.157927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.158208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] PATH_INFO: `/v3/projects/e3fdb7ff52834e2383900d7f133f00de` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.159315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a83e1-6a5e-4042-8457-83fcbc0de05b tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.159435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3fdb7ff52834e2383900d7f133f00de)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:10.161552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-d16a83e1-6a5e-4042-8457-83fcbc0de05b tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Cannot delete a domain that is enabled, please disable it first.: keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 151, in delete Sep 03 21:12:10.176765 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_domain( Sep 03 21:12:10.177853 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:12:10.177853 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:12:10.177853 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 802, in delete_domain Sep 03 21:12:10.177853 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._delete_domain(domain, initiator) Sep 03 21:12:10.177853 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 815, in _delete_domain Sep 03 21:12:10.177853 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Sep 03 21:12:10.177853 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Sep 03 21:12:10.177853 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:10.178426 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3132/6331] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:12:10 2022] DELETE /identity/v3/domains/0153438b465c414283682dff164bd3ea => generated 120 bytes in 168 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:12:10.182056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a83e1-6a5e-4042-8457-83fcbc0de05b tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.193679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a83e1-6a5e-4042-8457-83fcbc0de05b tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.199334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316e439-552d-4e36-ad93-9de7b79ea538 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adaae89daed141838dba7860e2085b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3531cf1808a1403280f66316941d2501', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.200290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316e439-552d-4e36-ad93-9de7b79ea538 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.200672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316e439-552d-4e36-ad93-9de7b79ea538 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.201000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316e439-552d-4e36-ad93-9de7b79ea538 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] PATH_INFO: `/v3/domains/0153438b465c414283682dff164bd3ea` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.202879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316e439-552d-4e36-ad93-9de7b79ea538 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=0153438b465c414283682dff164bd3ea)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:10.205191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316e439-552d-4e36-ad93-9de7b79ea538 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.224757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:10.234389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5316e439-552d-4e36-ad93-9de7b79ea538 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '0153438b465c414283682dff164bd3ea'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.235169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5316e439-552d-4e36-ad93-9de7b79ea538 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '0153438b465c414283682dff164bd3ea'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.240415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3fdb7ff52834e2383900d7f133f00de', 'request_id': 'req-76f17200-b031-480b-a4dd-666ce5c30b0e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.251447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3fdb7ff52834e2383900d7f133f00de', 'request_id': 'req-76f17200-b031-480b-a4dd-666ce5c30b0e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.261052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3fdb7ff52834e2383900d7f133f00de', 'request_id': 'req-76f17200-b031-480b-a4dd-666ce5c30b0e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.271498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5316e439-552d-4e36-ad93-9de7b79ea538 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '0153438b465c414283682dff164bd3ea'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.272737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5316e439-552d-4e36-ad93-9de7b79ea538 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '0153438b465c414283682dff164bd3ea'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.276169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] The token cache is being invalidate because project e3fdb7ff52834e2383900d7f133f00de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:10.276404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.277178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.278769 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3200/6332] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:10 2022] DELETE /identity/v3/projects/e3fdb7ff52834e2383900d7f133f00de => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:10.279079 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3133/6333] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:12:10 2022] PATCH /identity/v3/domains/0153438b465c414283682dff164bd3ea => generated 278 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:10.283686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.285102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a89174-4774-403e-be4f-acea9a737c2e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.285299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:10.286501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a89174-4774-403e-be4f-acea9a737c2e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.287144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a89174-4774-403e-be4f-acea9a737c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.318331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-19a89174-4774-403e-be4f-acea9a737c2e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:10.319627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-19a89174-4774-403e-be4f-acea9a737c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:10.369611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f17200-b031-480b-a4dd-666ce5c30b0e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.399390 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3134/6334] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:10.408024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f135234-86fb-4c19-b5e8-e86612040347 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.408313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f135234-86fb-4c19-b5e8-e86612040347 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.408595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f135234-86fb-4c19-b5e8-e86612040347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.429042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2f135234-86fb-4c19-b5e8-e86612040347 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:10.430261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2f135234-86fb-4c19-b5e8-e86612040347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:10.440186 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3135/6335] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:10.444683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f135234-86fb-4c19-b5e8-e86612040347 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.450686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adaae89daed141838dba7860e2085b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3531cf1808a1403280f66316941d2501', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.451551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.451911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.454541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f135234-86fb-4c19-b5e8-e86612040347 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.459693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da00504-44ae-4bb6-b138-fb6fd95b13b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.460312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da00504-44ae-4bb6-b138-fb6fd95b13b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.460626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da00504-44ae-4bb6-b138-fb6fd95b13b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.460874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da00504-44ae-4bb6-b138-fb6fd95b13b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.461576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da00504-44ae-4bb6-b138-fb6fd95b13b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:12:10.462050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da00504-44ae-4bb6-b138-fb6fd95b13b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:10.463746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da00504-44ae-4bb6-b138-fb6fd95b13b8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.464353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] PATH_INFO: `/v3/domains/0153438b465c414283682dff164bd3ea` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.487250 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3136/6336] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:10.488152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da00504-44ae-4bb6-b138-fb6fd95b13b8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.498098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=0153438b465c414283682dff164bd3ea)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:10.499379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da00504-44ae-4bb6-b138-fb6fd95b13b8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.501636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.511280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a869af-50d2-416a-a5d2-6b1ea5ca3814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.512005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a869af-50d2-416a-a5d2-6b1ea5ca3814 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.512298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a869af-50d2-416a-a5d2-6b1ea5ca3814 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.512633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a869af-50d2-416a-a5d2-6b1ea5ca3814 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.513501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a869af-50d2-416a-a5d2-6b1ea5ca3814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:10.516167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a869af-50d2-416a-a5d2-6b1ea5ca3814 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.550747 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3137/6337] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:10 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:10.555154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a869af-50d2-416a-a5d2-6b1ea5ca3814 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.556684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '0153438b465c414283682dff164bd3ea'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.569464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '0153438b465c414283682dff164bd3ea'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.601463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:10.620562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:10.631698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0153438b465c414283682dff164bd3ea', 'request_id': 'req-2b58a3a7-2f84-4798-a3b8-920967c6164e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.641021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0153438b465c414283682dff164bd3ea', 'request_id': 'req-2b58a3a7-2f84-4798-a3b8-920967c6164e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.648421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0153438b465c414283682dff164bd3ea', 'request_id': 'req-2b58a3a7-2f84-4798-a3b8-920967c6164e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.660671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a869af-50d2-416a-a5d2-6b1ea5ca3814 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.661111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] The token cache is being invalidate because project 0153438b465c414283682dff164bd3ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:10.661410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.662098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.673159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '0153438b465c414283682dff164bd3ea', 'request_id': 'req-2b58a3a7-2f84-4798-a3b8-920967c6164e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.677335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '0153438b465c414283682dff164bd3ea', 'request_id': 'req-2b58a3a7-2f84-4798-a3b8-920967c6164e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.678101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '0153438b465c414283682dff164bd3ea', 'request_id': 'req-2b58a3a7-2f84-4798-a3b8-920967c6164e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:10.684341 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3201/6338] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:12:10 2022] DELETE /identity/v3/domains/0153438b465c414283682dff164bd3ea => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:10.690216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.691638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:10.732696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d53f9-b2e9-443c-971e-2d9ae35767dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.733957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d53f9-b2e9-443c-971e-2d9ae35767dc admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.734308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d53f9-b2e9-443c-971e-2d9ae35767dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.734588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d53f9-b2e9-443c-971e-2d9ae35767dc admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.735407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d53f9-b2e9-443c-971e-2d9ae35767dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:10.737428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d53f9-b2e9-443c-971e-2d9ae35767dc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.765859 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3138/6339] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:10 2022] POST /identity/v3/users => generated 287 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:10.767021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d53f9-b2e9-443c-971e-2d9ae35767dc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.768659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58a3a7-2f84-4798-a3b8-920967c6164e tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.768834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7b8d53f9-b2e9-443c-971e-2d9ae35767dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:10.773786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a1d10-b916-46f2-9725-308903d32086 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adaae89daed141838dba7860e2085b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3531cf1808a1403280f66316941d2501', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.774571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a1d10-b916-46f2-9725-308903d32086 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.774865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a1d10-b916-46f2-9725-308903d32086 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.775156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a1d10-b916-46f2-9725-308903d32086 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] PATH_INFO: `/v3/domains/013a91793d7a49158078b86b77981f32` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.781732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d53f9-b2e9-443c-971e-2d9ae35767dc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.782875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a1d10-b916-46f2-9725-308903d32086 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:12:10.786730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6541178-5f48-43f4-9877-24b080e484e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.787434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6541178-5f48-43f4-9877-24b080e484e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.787789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6541178-5f48-43f4-9877-24b080e484e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-1e2a1d10-b916-46f2-9725-308903d32086 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Could not find domain: 013a91793d7a49158078b86b77981f32.: keystone.exception.DomainNotFound: Could not find domain: 013a91793d7a49158078b86b77981f32. Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 013a91793d7a49158078b86b77981f32. Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:12:10.787993 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 150, in delete Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_domain') Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:12:10.789196 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:12:10.790318 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 013a91793d7a49158078b86b77981f32. Sep 03 21:12:10.791466 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:10.791551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6541178-5f48-43f4-9877-24b080e484e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.791551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6541178-5f48-43f4-9877-24b080e484e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:10.791704 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3202/6340] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:12:10 2022] DELETE /identity/v3/domains/013a91793d7a49158078b86b77981f32 => generated 112 bytes in 99 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 03 21:12:10.791876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6541178-5f48-43f4-9877-24b080e484e1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.793931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a1d10-b916-46f2-9725-308903d32086 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.803239 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3139/6341] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:10.807245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6541178-5f48-43f4-9877-24b080e484e1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.807398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a1d10-b916-46f2-9725-308903d32086 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.812823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4109fb8-2d9a-4b46-9eb6-8122491833fc tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adaae89daed141838dba7860e2085b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3531cf1808a1403280f66316941d2501', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.813491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4109fb8-2d9a-4b46-9eb6-8122491833fc tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.813787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4109fb8-2d9a-4b46-9eb6-8122491833fc tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.814070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4109fb8-2d9a-4b46-9eb6-8122491833fc tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.814728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4109fb8-2d9a-4b46-9eb6-8122491833fc tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:10.816348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4109fb8-2d9a-4b46-9eb6-8122491833fc tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.817644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6541178-5f48-43f4-9877-24b080e484e1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.822330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096dccd-e480-4c38-9534-91f1897899f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.822944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096dccd-e480-4c38-9534-91f1897899f9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.823323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096dccd-e480-4c38-9534-91f1897899f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.823601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096dccd-e480-4c38-9534-91f1897899f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.824316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096dccd-e480-4c38-9534-91f1897899f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:10.826054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096dccd-e480-4c38-9534-91f1897899f9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.832752 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3203/6342] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:12:10 2022] POST /identity/v3/domains => generated 252 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:10.836364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4109fb8-2d9a-4b46-9eb6-8122491833fc tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.840935 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3140/6343] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:12:10.845109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096dccd-e480-4c38-9534-91f1897899f9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.856480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096dccd-e480-4c38-9534-91f1897899f9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.914767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4109fb8-2d9a-4b46-9eb6-8122491833fc tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:10.920237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc1189-8233-4a1f-b0b7-c1b4c3e35431 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adaae89daed141838dba7860e2085b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3531cf1808a1403280f66316941d2501', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.921445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc1189-8233-4a1f-b0b7-c1b4c3e35431 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.921854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc1189-8233-4a1f-b0b7-c1b4c3e35431 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.922139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc1189-8233-4a1f-b0b7-c1b4c3e35431 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.922960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc1189-8233-4a1f-b0b7-c1b4c3e35431 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:10.924713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc1189-8233-4a1f-b0b7-c1b4c3e35431 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.932148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4776a-35ba-47ac-9aa0-2008acb46f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:10.933229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4776a-35ba-47ac-9aa0-2008acb46f78 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.933229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4776a-35ba-47ac-9aa0-2008acb46f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.933504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.sql.core [None req-bbbc1189-8233-4a1f-b0b7-c1b4c3e35431 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-636357733' for key 'project.ixu_project_name_domain_id'") Sep 03 21:12:10.933504 ncdv-master-1 devstack@keystone.service[99210]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 03 21:12:10.933504 ncdv-master-1 devstack@keystone.service[99210]: [parameters: {'id': 'c4715ed7d6bb432d8de80e98095c297f', 'name': 'tempest-domain-dup-636357733', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Sep 03 21:12:10.933504 ncdv-master-1 devstack@keystone.service[99210]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99210) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 03 21:12:10.933882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4776a-35ba-47ac-9aa0-2008acb46f78 admin admin] PATH_INFO: `/v3/projects/ea7a4854a6a0475d99f2a5664f741533/users/01c449f3fd1946508ab455af5b231c14/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.949688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4776a-35ba-47ac-9aa0-2008acb46f78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7a4854a6a0475d99f2a5664f741533, user_id=01c449f3fd1946508ab455af5b231c14, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:10.952406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4776a-35ba-47ac-9aa0-2008acb46f78 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:10.965607 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3141/6344] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:10 2022] PUT /identity/v3/projects/ea7a4854a6a0475d99f2a5664f741533/users/01c449f3fd1946508ab455af5b231c14/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-bbbc1189-8233-4a1f-b0b7-c1b4c3e35431 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-636357733.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-636357733. Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application conn.query(q) Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 21:12:10.967186 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application result.read() Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-tempest-domain-dup-636357733' for key 'project.ixu_project_name_domain_id'") Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 03 21:12:10.968212 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application next(self.gen) Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application yield resource Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application next(self.gen) Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.session.rollback() Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.force_reraise() Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise self.value Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application session.commit() Sep 03 21:12:10.969578 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._prepare_impl() Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.session.flush() Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._flush(objects) Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application compat.raise_( Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application flush_context.execute() Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 03 21:12:10.970788 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rec.execute(self) Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 03 21:12:10.972009 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application conn.query(q) Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application result.read() Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 21:12:10.973109 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-636357733' for key 'project.ixu_project_name_domain_id'") Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [parameters: {'id': 'c4715ed7d6bb432d8de80e98095c297f', 'name': 'tempest-domain-dup-636357733', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:10.974237 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-domain-dup-636357733. Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:12:10.975184 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-636357733. Sep 03 21:12:10.976352 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:10.980730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965b6b7-1e8f-460f-ac0b-301800664a53 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:10.980730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965b6b7-1e8f-460f-ac0b-301800664a53 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:10.980730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965b6b7-1e8f-460f-ac0b-301800664a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:10.981047 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3204/6345] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:12:10 2022] POST /identity/v3/domains => generated 211 bytes in 144 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 03 21:12:10.984772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc1189-8233-4a1f-b0b7-c1b4c3e35431 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:10.992656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8965b6b7-1e8f-460f-ac0b-301800664a53 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:10.994078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8965b6b7-1e8f-460f-ac0b-301800664a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:10.996758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc1189-8233-4a1f-b0b7-c1b4c3e35431 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:11.008726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:11.036079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:11.071935 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3142/6346] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:11.073200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adaae89daed141838dba7860e2085b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3531cf1808a1403280f66316941d2501', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:11.074087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:11.074390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:11.074673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] PATH_INFO: `/v3/domains/0343d480740d474e8fb80b363063c97d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:11.083452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965b6b7-1e8f-460f-ac0b-301800664a53 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:11.084858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8965b6b7-1e8f-460f-ac0b-301800664a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:11.087262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=0343d480740d474e8fb80b363063c97d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:11.089181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:11.116424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '0343d480740d474e8fb80b363063c97d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.125999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '0343d480740d474e8fb80b363063c97d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.163939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965b6b7-1e8f-460f-ac0b-301800664a53 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:11.167897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '0343d480740d474e8fb80b363063c97d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.168526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '0343d480740d474e8fb80b363063c97d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.174967 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3205/6347] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:12:10 2022] PATCH /identity/v3/domains/0343d480740d474e8fb80b363063c97d => generated 253 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:11.176229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:11.177876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:11.224280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e96d2b-52ba-4090-b891-eb8eda6f5063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:11.225071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e96d2b-52ba-4090-b891-eb8eda6f5063 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:11.225464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e96d2b-52ba-4090-b891-eb8eda6f5063 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:11.225864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e96d2b-52ba-4090-b891-eb8eda6f5063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:11.228149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e8e96d2b-52ba-4090-b891-eb8eda6f5063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:11.258778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d7ba2-8bf0-4da0-be69-6da753c39436 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:11.264515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adaae89daed141838dba7860e2085b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3531cf1808a1403280f66316941d2501', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:11.265160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:11.265598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:11.265890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] PATH_INFO: `/v3/domains/0343d480740d474e8fb80b363063c97d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:11.277535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=0343d480740d474e8fb80b363063c97d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:11.279180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:11.299298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e96d2b-52ba-4090-b891-eb8eda6f5063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:11.300929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e96d2b-52ba-4090-b891-eb8eda6f5063 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:11.305777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '0343d480740d474e8fb80b363063c97d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.309844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '0343d480740d474e8fb80b363063c97d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.311944 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3143/6348] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:11.374107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0343d480740d474e8fb80b363063c97d', 'request_id': 'req-31ed1679-40e8-4338-83d9-a08e4546a0d3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.386129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0343d480740d474e8fb80b363063c97d', 'request_id': 'req-31ed1679-40e8-4338-83d9-a08e4546a0d3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.391049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:UPDATE=1,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:11.406798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0343d480740d474e8fb80b363063c97d', 'request_id': 'req-31ed1679-40e8-4338-83d9-a08e4546a0d3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.426603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] The token cache is being invalidate because project 0343d480740d474e8fb80b363063c97d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:11.426911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.428451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.439718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '0343d480740d474e8fb80b363063c97d', 'request_id': 'req-31ed1679-40e8-4338-83d9-a08e4546a0d3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.444539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '0343d480740d474e8fb80b363063c97d', 'request_id': 'req-31ed1679-40e8-4338-83d9-a08e4546a0d3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.445601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '0343d480740d474e8fb80b363063c97d', 'request_id': 'req-31ed1679-40e8-4338-83d9-a08e4546a0d3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.457679 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3206/6349] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:12:11 2022] DELETE /identity/v3/domains/0343d480740d474e8fb80b363063c97d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:11.462626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e96d2b-52ba-4090-b891-eb8eda6f5063 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:11.474834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e96d2b-52ba-4090-b891-eb8eda6f5063 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:11.517468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:11.568770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b20a92-66c0-4744-a700-b1c7fe71e601 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adaae89daed141838dba7860e2085b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3531cf1808a1403280f66316941d2501', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:11.569653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b20a92-66c0-4744-a700-b1c7fe71e601 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:11.570105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b20a92-66c0-4744-a700-b1c7fe71e601 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:11.570570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b20a92-66c0-4744-a700-b1c7fe71e601 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] PATH_INFO: `/v3/domains/0153438b465c414283682dff164bd3ea` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:11.581218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b20a92-66c0-4744-a700-b1c7fe71e601 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-b4b20a92-66c0-4744-a700-b1c7fe71e601 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Could not find domain: 0153438b465c414283682dff164bd3ea.: keystone.exception.DomainNotFound: Could not find domain: 0153438b465c414283682dff164bd3ea. Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 0153438b465c414283682dff164bd3ea. Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:12:11.587319 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:12:11.588702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:12:11.590072 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 0153438b465c414283682dff164bd3ea. Sep 03 21:12:11.591451 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:11.594433 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3144/6350] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:12:11 2022] PATCH /identity/v3/domains/0153438b465c414283682dff164bd3ea => generated 112 bytes in 132 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:12:11.599957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:11.604377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:11.746482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed1679-40e8-4338-83d9-a08e4546a0d3 tempest-DomainsNegativeTestJSON-149142539 tempest-DomainsNegativeTestJSON-149142539-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:11.751767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:11.752489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:11.752844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:11.753229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] PATH_INFO: `/v3/users/cb24530016974c70884bb3bc9b91e69c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:11.754469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb24530016974c70884bb3bc9b91e69c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:11.756575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:11.832804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb24530016974c70884bb3bc9b91e69c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.854004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb24530016974c70884bb3bc9b91e69c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.866089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb24530016974c70884bb3bc9b91e69c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.869260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb24530016974c70884bb3bc9b91e69c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.870002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb24530016974c70884bb3bc9b91e69c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:11.894887 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3207/6351] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:11 2022] DELETE /identity/v3/users/cb24530016974c70884bb3bc9b91e69c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:11.896631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaee1c9-1981-4fc9-ac75-fe20ab649ec2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:11.897438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaee1c9-1981-4fc9-ac75-fe20ab649ec2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:11.898273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaee1c9-1981-4fc9-ac75-fe20ab649ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:11.945427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cdaee1c9-1981-4fc9-ac75-fe20ab649ec2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:11.946774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cdaee1c9-1981-4fc9-ac75-fe20ab649ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:12.077705 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3145/6352] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:12.090198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:12.093797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:12.176555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:12.186043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d78a9c-6e10-4b4f-91fb-becff042d6bf admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:12.191402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922769d-8331-4edd-a092-12c0ba0243e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:12.192407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922769d-8331-4edd-a092-12c0ba0243e9 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:12.192755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922769d-8331-4edd-a092-12c0ba0243e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:12.193190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922769d-8331-4edd-a092-12c0ba0243e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:12.202636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922769d-8331-4edd-a092-12c0ba0243e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:12.204216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922769d-8331-4edd-a092-12c0ba0243e9 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:12.213951 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3208/6353] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:12.527859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaee1c9-1981-4fc9-ac75-fe20ab649ec2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:12.529458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cdaee1c9-1981-4fc9-ac75-fe20ab649ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:12.542420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaee1c9-1981-4fc9-ac75-fe20ab649ec2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:12.547353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:12.548570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:12.548570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:12.548570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] PATH_INFO: `/v3/projects/ae7df719c1d54fddbf9956b4429735ec` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:12.549734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae7df719c1d54fddbf9956b4429735ec)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:12.551856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:12.611297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae7df719c1d54fddbf9956b4429735ec', 'request_id': 'req-e72b9023-42b0-4d88-9685-1c3ee852b052'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:12.636410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae7df719c1d54fddbf9956b4429735ec', 'request_id': 'req-e72b9023-42b0-4d88-9685-1c3ee852b052'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:12.645869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae7df719c1d54fddbf9956b4429735ec', 'request_id': 'req-e72b9023-42b0-4d88-9685-1c3ee852b052'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:12.659423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] The token cache is being invalidate because project ae7df719c1d54fddbf9956b4429735ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:12.659807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:12.660488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:12.665617 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3146/6354] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:12 2022] DELETE /identity/v3/projects/ae7df719c1d54fddbf9956b4429735ec => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:12.667133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922769d-8331-4edd-a092-12c0ba0243e9 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:12.691576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922769d-8331-4edd-a092-12c0ba0243e9 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:12.786258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6babac-8679-4071-a0e4-898a6bb20a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:12.786920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6babac-8679-4071-a0e4-898a6bb20a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:12.787214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6babac-8679-4071-a0e4-898a6bb20a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:12.787515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6babac-8679-4071-a0e4-898a6bb20a2b admin admin] PATH_INFO: `/v3/users/adaae89daed141838dba7860e2085b7b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:12.789654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6babac-8679-4071-a0e4-898a6bb20a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=adaae89daed141838dba7860e2085b7b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:12.791650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6babac-8679-4071-a0e4-898a6bb20a2b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:12.860867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bf6babac-8679-4071-a0e4-898a6bb20a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adaae89daed141838dba7860e2085b7b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:12.877400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bf6babac-8679-4071-a0e4-898a6bb20a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adaae89daed141838dba7860e2085b7b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:12.886998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bf6babac-8679-4071-a0e4-898a6bb20a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adaae89daed141838dba7860e2085b7b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:12.887714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bf6babac-8679-4071-a0e4-898a6bb20a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adaae89daed141838dba7860e2085b7b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:12.889991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bf6babac-8679-4071-a0e4-898a6bb20a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adaae89daed141838dba7860e2085b7b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:12.910864 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3209/6355] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:12 2022] DELETE /identity/v3/users/adaae89daed141838dba7860e2085b7b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:13.367068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:13.370809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:13.420059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b9023-42b0-4d88-9685-1c3ee852b052 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:13.496630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:13.528152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d781f0d-474d-4928-9b11-8f7cbf714520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:13.528946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d781f0d-474d-4928-9b11-8f7cbf714520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:13.529148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d781f0d-474d-4928-9b11-8f7cbf714520 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:13.529619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d781f0d-474d-4928-9b11-8f7cbf714520 admin admin] PATH_INFO: `/v3/projects/3531cf1808a1403280f66316941d2501` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:13.533109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d781f0d-474d-4928-9b11-8f7cbf714520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3531cf1808a1403280f66316941d2501)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:13.535260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d781f0d-474d-4928-9b11-8f7cbf714520 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:13.605798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6d781f0d-474d-4928-9b11-8f7cbf714520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3531cf1808a1403280f66316941d2501', 'request_id': 'req-6d781f0d-474d-4928-9b11-8f7cbf714520'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:13.615140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6d781f0d-474d-4928-9b11-8f7cbf714520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3531cf1808a1403280f66316941d2501', 'request_id': 'req-6d781f0d-474d-4928-9b11-8f7cbf714520'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:13.624452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6d781f0d-474d-4928-9b11-8f7cbf714520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3531cf1808a1403280f66316941d2501', 'request_id': 'req-6d781f0d-474d-4928-9b11-8f7cbf714520'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:13.657684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6d781f0d-474d-4928-9b11-8f7cbf714520 admin admin] The token cache is being invalidate because project 3531cf1808a1403280f66316941d2501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:13.658140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6d781f0d-474d-4928-9b11-8f7cbf714520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:13.659702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6d781f0d-474d-4928-9b11-8f7cbf714520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:13.661492 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3147/6356] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:13 2022] DELETE /identity/v3/projects/3531cf1808a1403280f66316941d2501 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:13.666447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2799cc-1a32-4d1b-9a44-ecc48fce0a1e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:13.666942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2799cc-1a32-4d1b-9a44-ecc48fce0a1e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:13.667061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2799cc-1a32-4d1b-9a44-ecc48fce0a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:13.690615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6a2799cc-1a32-4d1b-9a44-ecc48fce0a1e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:13.691965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6a2799cc-1a32-4d1b-9a44-ecc48fce0a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:13.760238 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3210/6357] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:13.769626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105bd63-af6f-4670-957c-8664e7ea73c8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:13.769921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105bd63-af6f-4670-957c-8664e7ea73c8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:13.770206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105bd63-af6f-4670-957c-8664e7ea73c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:13.800287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0105bd63-af6f-4670-957c-8664e7ea73c8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:13.801993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0105bd63-af6f-4670-957c-8664e7ea73c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:13.812958 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3148/6358] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:13.814921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2799cc-1a32-4d1b-9a44-ecc48fce0a1e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:13.829896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2799cc-1a32-4d1b-9a44-ecc48fce0a1e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:13.835694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e947fb-18ea-44a7-9ab3-98ca7a89ce97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:13.836361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e947fb-18ea-44a7-9ab3-98ca7a89ce97 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:13.836652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e947fb-18ea-44a7-9ab3-98ca7a89ce97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:13.836960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e947fb-18ea-44a7-9ab3-98ca7a89ce97 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:13.837617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e947fb-18ea-44a7-9ab3-98ca7a89ce97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:12:13.837976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e947fb-18ea-44a7-9ab3-98ca7a89ce97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:13.839580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e947fb-18ea-44a7-9ab3-98ca7a89ce97 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:13.842390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:13.856845 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3211/6359] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:12:13.858967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105bd63-af6f-4670-957c-8664e7ea73c8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:13.868598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105bd63-af6f-4670-957c-8664e7ea73c8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:13.875080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a412d-fded-4691-8b73-f2ba07930143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:13.875702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a412d-fded-4691-8b73-f2ba07930143 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:13.875992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a412d-fded-4691-8b73-f2ba07930143 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:13.876267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a412d-fded-4691-8b73-f2ba07930143 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:13.877080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a412d-fded-4691-8b73-f2ba07930143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:13.878979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a412d-fded-4691-8b73-f2ba07930143 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:13.909307 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3149/6360] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:13 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:13.913572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e947fb-18ea-44a7-9ab3-98ca7a89ce97 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:13.925353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e947fb-18ea-44a7-9ab3-98ca7a89ce97 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:14.002683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f49f5-a491-489c-a292-a6e0ad1c5d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:14.003361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f49f5-a491-489c-a292-a6e0ad1c5d2c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:14.003651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f49f5-a491-489c-a292-a6e0ad1c5d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:14.004039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f49f5-a491-489c-a292-a6e0ad1c5d2c admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:14.005638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4f49f5-a491-489c-a292-a6e0ad1c5d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:14.008810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4f49f5-a491-489c-a292-a6e0ad1c5d2c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:14.036533 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3212/6361] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:13 2022] POST /identity/v3/users => generated 284 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:14.040984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a412d-fded-4691-8b73-f2ba07930143 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:14.057793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a412d-fded-4691-8b73-f2ba07930143 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:14.064808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd741d-d5de-49a7-923b-3de170e32939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:14.065595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd741d-d5de-49a7-923b-3de170e32939 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:14.065881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd741d-d5de-49a7-923b-3de170e32939 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:14.066159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd741d-d5de-49a7-923b-3de170e32939 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:14.067603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd741d-d5de-49a7-923b-3de170e32939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:14.076852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd741d-d5de-49a7-923b-3de170e32939 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:14.091700 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3150/6362] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:14.093286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f49f5-a491-489c-a292-a6e0ad1c5d2c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:14.105246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f49f5-a491-489c-a292-a6e0ad1c5d2c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:14.110390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca0459-777a-437a-8a72-41ca62144200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:14.111174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca0459-777a-437a-8a72-41ca62144200 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:14.111530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca0459-777a-437a-8a72-41ca62144200 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:14.111865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca0459-777a-437a-8a72-41ca62144200 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:14.112525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca0459-777a-437a-8a72-41ca62144200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:14.114191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca0459-777a-437a-8a72-41ca62144200 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:14.134038 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3213/6363] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:12:14.140965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd741d-d5de-49a7-923b-3de170e32939 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:14.152588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd741d-d5de-49a7-923b-3de170e32939 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:14.158216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d6c29-b3f5-47e4-96cd-5f35b2a9cf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:14.163835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d6c29-b3f5-47e4-96cd-5f35b2a9cf4e admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:14.164265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d6c29-b3f5-47e4-96cd-5f35b2a9cf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:14.164651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d6c29-b3f5-47e4-96cd-5f35b2a9cf4e admin admin] PATH_INFO: `/v3/projects/2474af174b1f4087a11d696643570ef9/users/7a2a473ffab94adbaf5b74a65bec60fc/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:14.186992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d6c29-b3f5-47e4-96cd-5f35b2a9cf4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2474af174b1f4087a11d696643570ef9, user_id=7a2a473ffab94adbaf5b74a65bec60fc, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:14.190089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d6c29-b3f5-47e4-96cd-5f35b2a9cf4e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:14.208234 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3151/6364] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:14 2022] PUT /identity/v3/projects/2474af174b1f4087a11d696643570ef9/users/7a2a473ffab94adbaf5b74a65bec60fc/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:14.218725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3f926-35c4-41b7-9d08-cd1341d63b84 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:14.219022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3f926-35c4-41b7-9d08-cd1341d63b84 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:14.219304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3f926-35c4-41b7-9d08-cd1341d63b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:14.250279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3ee3f926-35c4-41b7-9d08-cd1341d63b84 None None] MFA Rules not processed for user `7a2a473ffab94adbaf5b74a65bec60fc`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:14.251523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3ee3f926-35c4-41b7-9d08-cd1341d63b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:14.370347 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3214/6365] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:14.375462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d6c29-b3f5-47e4-96cd-5f35b2a9cf4e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:14.462853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d6c29-b3f5-47e4-96cd-5f35b2a9cf4e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:14.469824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bfa31-5359-4031-9368-95d6ce4f9b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:14.470587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bfa31-5359-4031-9368-95d6ce4f9b6e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:14.471566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bfa31-5359-4031-9368-95d6ce4f9b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:14.471944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bfa31-5359-4031-9368-95d6ce4f9b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:14.472686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bfa31-5359-4031-9368-95d6ce4f9b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:14.475981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bfa31-5359-4031-9368-95d6ce4f9b6e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:14.521073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:14.543284 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3152/6366] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:14 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:14.548069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3f926-35c4-41b7-9d08-cd1341d63b84 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:14.564933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3f926-35c4-41b7-9d08-cd1341d63b84 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:14.661261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa8063-f94f-479d-ad62-5c1fef749b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:14.664954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa8063-f94f-479d-ad62-5c1fef749b29 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:14.665161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa8063-f94f-479d-ad62-5c1fef749b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:14.665268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa8063-f94f-479d-ad62-5c1fef749b29 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:14.665950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa8063-f94f-479d-ad62-5c1fef749b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:14.668003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa8063-f94f-479d-ad62-5c1fef749b29 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:14.707828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:14.728111 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3215/6367] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:14 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:14.732559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bfa31-5359-4031-9368-95d6ce4f9b6e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:14.761741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bfa31-5359-4031-9368-95d6ce4f9b6e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:14.778486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b699fc-7b3f-4616-9264-aec9521c5ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:14.778486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b699fc-7b3f-4616-9264-aec9521c5ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:14.778486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b699fc-7b3f-4616-9264-aec9521c5ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:14.778486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b699fc-7b3f-4616-9264-aec9521c5ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:14.781940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b699fc-7b3f-4616-9264-aec9521c5ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:14.788524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b699fc-7b3f-4616-9264-aec9521c5ee7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:14.822314 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3153/6368] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:14.830828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa8063-f94f-479d-ad62-5c1fef749b29 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:14.859540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa8063-f94f-479d-ad62-5c1fef749b29 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:14.875960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc82d3-1f5a-43d4-8fca-670679fdeec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:14.876805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc82d3-1f5a-43d4-8fca-670679fdeec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:14.877119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc82d3-1f5a-43d4-8fca-670679fdeec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:14.880936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc82d3-1f5a-43d4-8fca-670679fdeec3 admin admin] PATH_INFO: `/v3/projects/715e5d7d48d84c4594e859a12b70290d/users/8b6096f19b1a4028aa81c2b52a30e67f/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:14.909158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc82d3-1f5a-43d4-8fca-670679fdeec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=715e5d7d48d84c4594e859a12b70290d, user_id=8b6096f19b1a4028aa81c2b52a30e67f, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:14.911862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc82d3-1f5a-43d4-8fca-670679fdeec3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:14.935167 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3216/6369] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:14 2022] PUT /identity/v3/projects/715e5d7d48d84c4594e859a12b70290d/users/8b6096f19b1a4028aa81c2b52a30e67f/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:14.938965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b699fc-7b3f-4616-9264-aec9521c5ee7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:14.958839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b699fc-7b3f-4616-9264-aec9521c5ee7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:15.124503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662afb2-01cc-4abe-bb3f-1cb78b0b087d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:15.125199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662afb2-01cc-4abe-bb3f-1cb78b0b087d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:15.125654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662afb2-01cc-4abe-bb3f-1cb78b0b087d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:15.125955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662afb2-01cc-4abe-bb3f-1cb78b0b087d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:15.126616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662afb2-01cc-4abe-bb3f-1cb78b0b087d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:15.128823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662afb2-01cc-4abe-bb3f-1cb78b0b087d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:15.149370 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3154/6370] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:15.153554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc82d3-1f5a-43d4-8fca-670679fdeec3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:15.174272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc82d3-1f5a-43d4-8fca-670679fdeec3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:15.180277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a6dae-48bc-43f9-b722-e5d8c8f0f4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:15.181100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a6dae-48bc-43f9-b722-e5d8c8f0f4d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:15.181488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a6dae-48bc-43f9-b722-e5d8c8f0f4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:15.181842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a6dae-48bc-43f9-b722-e5d8c8f0f4d8 admin admin] PATH_INFO: `/v3/projects/715e5d7d48d84c4594e859a12b70290d/users/8b6096f19b1a4028aa81c2b52a30e67f/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:15.184640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a6dae-48bc-43f9-b722-e5d8c8f0f4d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=715e5d7d48d84c4594e859a12b70290d, user_id=8b6096f19b1a4028aa81c2b52a30e67f, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:15.188629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a6dae-48bc-43f9-b722-e5d8c8f0f4d8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:15.201261 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3217/6371] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:15 2022] PUT /identity/v3/projects/715e5d7d48d84c4594e859a12b70290d/users/8b6096f19b1a4028aa81c2b52a30e67f/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:15.212998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca2597-3af6-442f-bec9-b162b3d195d4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:15.213396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca2597-3af6-442f-bec9-b162b3d195d4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:15.213750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca2597-3af6-442f-bec9-b162b3d195d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:15.278123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c1ca2597-3af6-442f-bec9-b162b3d195d4 None None] MFA Rules not processed for user `8b6096f19b1a4028aa81c2b52a30e67f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:15.280976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c1ca2597-3af6-442f-bec9-b162b3d195d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:15.400779 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3155/6372] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:15.401985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a6dae-48bc-43f9-b722-e5d8c8f0f4d8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:15.492073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a6dae-48bc-43f9-b722-e5d8c8f0f4d8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:15.497376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34212424-52ec-4ba9-980e-83aedabb06dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:15.498079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34212424-52ec-4ba9-980e-83aedabb06dc admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:15.499334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34212424-52ec-4ba9-980e-83aedabb06dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:15.499334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34212424-52ec-4ba9-980e-83aedabb06dc admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:15.500119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34212424-52ec-4ba9-980e-83aedabb06dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:15.502091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34212424-52ec-4ba9-980e-83aedabb06dc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:15.535815 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3218/6373] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:15 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:15.537579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca2597-3af6-442f-bec9-b162b3d195d4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:15.558141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca2597-3af6-442f-bec9-b162b3d195d4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:15.628276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aeb7f8-66d3-4641-b81a-389d6f3ccb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:15.629095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeb7f8-66d3-4641-b81a-389d6f3ccb47 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:15.629836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeb7f8-66d3-4641-b81a-389d6f3ccb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:15.629836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeb7f8-66d3-4641-b81a-389d6f3ccb47 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:15.630716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aeb7f8-66d3-4641-b81a-389d6f3ccb47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:15.632692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aeb7f8-66d3-4641-b81a-389d6f3ccb47 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:15.656085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:15.671616 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3156/6374] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:15 2022] POST /identity/v3/users => generated 285 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:15.673735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34212424-52ec-4ba9-980e-83aedabb06dc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:15.687416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34212424-52ec-4ba9-980e-83aedabb06dc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:15.693254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feede3ec-e91e-45c5-8200-c4c394a37443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:15.694027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feede3ec-e91e-45c5-8200-c4c394a37443 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:15.694387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feede3ec-e91e-45c5-8200-c4c394a37443 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:15.694828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feede3ec-e91e-45c5-8200-c4c394a37443 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:15.697743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feede3ec-e91e-45c5-8200-c4c394a37443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:15.700031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feede3ec-e91e-45c5-8200-c4c394a37443 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:15.718380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aeb7f8-66d3-4641-b81a-389d6f3ccb47 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:15.718865 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3219/6375] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:12:15.736458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aeb7f8-66d3-4641-b81a-389d6f3ccb47 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:15.743334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a10c90-4623-4e0f-b5b4-87edfabb3099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:15.744013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a10c90-4623-4e0f-b5b4-87edfabb3099 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:15.744142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a10c90-4623-4e0f-b5b4-87edfabb3099 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:15.744413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a10c90-4623-4e0f-b5b4-87edfabb3099 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:15.745613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a10c90-4623-4e0f-b5b4-87edfabb3099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:15.747364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a10c90-4623-4e0f-b5b4-87edfabb3099 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:15.766700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feede3ec-e91e-45c5-8200-c4c394a37443 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:15.766928 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3157/6376] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:15.780685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feede3ec-e91e-45c5-8200-c4c394a37443 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:15.786705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134165c3-3921-413f-a06c-ae0193f9746b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:15.787687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134165c3-3921-413f-a06c-ae0193f9746b admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:15.788617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134165c3-3921-413f-a06c-ae0193f9746b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:15.788987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134165c3-3921-413f-a06c-ae0193f9746b admin admin] PATH_INFO: `/v3/projects/8ae5331900fd4b30969af9c156df5392/users/7fb1781551ca40769e67f106be4523d7/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:15.809341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134165c3-3921-413f-a06c-ae0193f9746b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae5331900fd4b30969af9c156df5392, user_id=7fb1781551ca40769e67f106be4523d7, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:15.812669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134165c3-3921-413f-a06c-ae0193f9746b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:15.830458 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3220/6377] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:15 2022] PUT /identity/v3/projects/8ae5331900fd4b30969af9c156df5392/users/7fb1781551ca40769e67f106be4523d7/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:15.840600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a32016-af63-41f3-baa1-fbbccf54f8e3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:15.840953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a32016-af63-41f3-baa1-fbbccf54f8e3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:15.843053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a32016-af63-41f3-baa1-fbbccf54f8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:15.887203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-89a32016-af63-41f3-baa1-fbbccf54f8e3 None None] MFA Rules not processed for user `7fb1781551ca40769e67f106be4523d7`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:15.890153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-89a32016-af63-41f3-baa1-fbbccf54f8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:15.994328 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3158/6378] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:15.999174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134165c3-3921-413f-a06c-ae0193f9746b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:16.091886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134165c3-3921-413f-a06c-ae0193f9746b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:16.114421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb988546-65f8-46df-895c-1103125e064f tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:16.115274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb988546-65f8-46df-895c-1103125e064f tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:16.115569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb988546-65f8-46df-895c-1103125e064f tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:16.115881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb988546-65f8-46df-895c-1103125e064f tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:16.116543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb988546-65f8-46df-895c-1103125e064f tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:16.118252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb988546-65f8-46df-895c-1103125e064f tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:16.146516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:16.159197 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3221/6379] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:12:15 2022] POST /identity/v3/domains => generated 276 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:16.163762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a32016-af63-41f3-baa1-fbbccf54f8e3 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:16.183245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a32016-af63-41f3-baa1-fbbccf54f8e3 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:16.277871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd9c0-2a93-427e-87b0-4c4973a1d3ac tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:16.279317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd9c0-2a93-427e-87b0-4c4973a1d3ac tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:16.282218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd9c0-2a93-427e-87b0-4c4973a1d3ac tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:16.282218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd9c0-2a93-427e-87b0-4c4973a1d3ac tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:16.283693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4acd9c0-2a93-427e-87b0-4c4973a1d3ac tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:16.285745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4acd9c0-2a93-427e-87b0-4c4973a1d3ac tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:16.306322 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3159/6380] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:12:16 2022] POST /identity/v3/roles => generated 239 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:16.310332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb988546-65f8-46df-895c-1103125e064f tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:16.325090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb988546-65f8-46df-895c-1103125e064f tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:16.331142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f3c2c-394c-4622-9a4c-5f78083987f6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:16.332273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f3c2c-394c-4622-9a4c-5f78083987f6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:16.332668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f3c2c-394c-4622-9a4c-5f78083987f6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:16.333091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f3c2c-394c-4622-9a4c-5f78083987f6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/6c54abfb4d3e4de1bd05e0106ab6a180/users/7fb1781551ca40769e67f106be4523d7/roles/75fa868bca424a319fb678e215ace386` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:16.354483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f3c2c-394c-4622-9a4c-5f78083987f6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=6c54abfb4d3e4de1bd05e0106ab6a180, user_id=7fb1781551ca40769e67f106be4523d7, role_id=75fa868bca424a319fb678e215ace386)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:16.358756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f3c2c-394c-4622-9a4c-5f78083987f6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:16.375596 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3222/6381] 10.222.0.16 () {60 vars in 1480 bytes} [Sat Sep 3 21:12:16 2022] PUT /identity/v3/domains/6c54abfb4d3e4de1bd05e0106ab6a180/users/7fb1781551ca40769e67f106be4523d7/roles/75fa868bca424a319fb678e215ace386 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:16.381172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd9c0-2a93-427e-87b0-4c4973a1d3ac tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:16.395414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd9c0-2a93-427e-87b0-4c4973a1d3ac tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:16.468342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27829293-1c02-40f1-8f40-67f5b51a8a33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:16.469275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27829293-1c02-40f1-8f40-67f5b51a8a33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:16.469675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27829293-1c02-40f1-8f40-67f5b51a8a33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:16.470060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27829293-1c02-40f1-8f40-67f5b51a8a33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:16.471151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27829293-1c02-40f1-8f40-67f5b51a8a33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:16.472650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27829293-1c02-40f1-8f40-67f5b51a8a33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:16.494631 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3160/6382] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:12:16 2022] POST /identity/v3/domains => generated 276 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:16.498679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f3c2c-394c-4622-9a4c-5f78083987f6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:16.511423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f3c2c-394c-4622-9a4c-5f78083987f6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:16.579201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce7b7a-4a81-45b3-ba9e-9f2b2e45e3e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:16.579997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce7b7a-4a81-45b3-ba9e-9f2b2e45e3e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:16.580461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce7b7a-4a81-45b3-ba9e-9f2b2e45e3e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:16.581336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce7b7a-4a81-45b3-ba9e-9f2b2e45e3e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:16.581931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce7b7a-4a81-45b3-ba9e-9f2b2e45e3e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:16.583714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce7b7a-4a81-45b3-ba9e-9f2b2e45e3e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:16.602689 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3223/6383] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:12:16 2022] POST /identity/v3/roles => generated 240 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:16.607181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27829293-1c02-40f1-8f40-67f5b51a8a33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:16.620369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27829293-1c02-40f1-8f40-67f5b51a8a33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:16.627286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1c7c6-ad14-4a65-83af-8d9f56d98a09 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:16.628612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1c7c6-ad14-4a65-83af-8d9f56d98a09 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:16.629055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1c7c6-ad14-4a65-83af-8d9f56d98a09 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:16.629445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1c7c6-ad14-4a65-83af-8d9f56d98a09 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:16.630496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1c7c6-ad14-4a65-83af-8d9f56d98a09 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:16.633271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1c7c6-ad14-4a65-83af-8d9f56d98a09 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:16.638503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:16.673829 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3161/6384] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Sep 3 21:12:16 2022] POST /identity/v3/groups => generated 327 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:16.675650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce7b7a-4a81-45b3-ba9e-9f2b2e45e3e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:16.691431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce7b7a-4a81-45b3-ba9e-9f2b2e45e3e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:16.697906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6f4f9-818b-442b-8a94-bb0dd2f065f2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:16.698643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6f4f9-818b-442b-8a94-bb0dd2f065f2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:16.699018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6f4f9-818b-442b-8a94-bb0dd2f065f2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:16.699368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6f4f9-818b-442b-8a94-bb0dd2f065f2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/groups/bba8e8bc274a40eeb3a2f7cca4a1c43c/users/7fb1781551ca40769e67f106be4523d7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:16.708250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6f4f9-818b-442b-8a94-bb0dd2f065f2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=bba8e8bc274a40eeb3a2f7cca4a1c43c, user_id=7fb1781551ca40769e67f106be4523d7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:16.710180 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-efd6f4f9-818b-442b-8a94-bb0dd2f065f2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:12:16.710180 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:12:16.710180 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:12:16.712080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6f4f9-818b-442b-8a94-bb0dd2f065f2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:16.773278 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3224/6385] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Sep 3 21:12:16 2022] PUT /identity/v3/groups/bba8e8bc274a40eeb3a2f7cca4a1c43c/users/7fb1781551ca40769e67f106be4523d7 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:16.775759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1c7c6-ad14-4a65-83af-8d9f56d98a09 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:16.803107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1c7c6-ad14-4a65-83af-8d9f56d98a09 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:16.911325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3a036-fb39-47c2-80a4-a53dbaa4433c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:16.912030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3a036-fb39-47c2-80a4-a53dbaa4433c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:16.912324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3a036-fb39-47c2-80a4-a53dbaa4433c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:16.912610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3a036-fb39-47c2-80a4-a53dbaa4433c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/4045427e38754c50a5a05015816db9d3/groups/bba8e8bc274a40eeb3a2f7cca4a1c43c/roles/c4e0622210b447b6951911eacd99612a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:16.923942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3a036-fb39-47c2-80a4-a53dbaa4433c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=4045427e38754c50a5a05015816db9d3, group_id=bba8e8bc274a40eeb3a2f7cca4a1c43c, role_id=c4e0622210b447b6951911eacd99612a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:16.928050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3a036-fb39-47c2-80a4-a53dbaa4433c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:16.955837 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3162/6386] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:16 2022] PUT /identity/v3/domains/4045427e38754c50a5a05015816db9d3/groups/bba8e8bc274a40eeb3a2f7cca4a1c43c/roles/c4e0622210b447b6951911eacd99612a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:16.960101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6f4f9-818b-442b-8a94-bb0dd2f065f2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:16.975003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6f4f9-818b-442b-8a94-bb0dd2f065f2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:17.087942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cb75f-6106-4b3c-a06d-532c2b97f9ee tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:17.088506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cb75f-6106-4b3c-a06d-532c2b97f9ee tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:17.089035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cb75f-6106-4b3c-a06d-532c2b97f9ee tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:17.089329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cb75f-6106-4b3c-a06d-532c2b97f9ee tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:17.090027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cb75f-6106-4b3c-a06d-532c2b97f9ee tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:17.091687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cb75f-6106-4b3c-a06d-532c2b97f9ee tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:17.114922 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3225/6387] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:12:16 2022] POST /identity/v3/domains => generated 277 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:17.119423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3a036-fb39-47c2-80a4-a53dbaa4433c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:17.138414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3a036-fb39-47c2-80a4-a53dbaa4433c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:17.264691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4bf77-88f3-4466-bccb-5fec3bc17486 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:17.265800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4bf77-88f3-4466-bccb-5fec3bc17486 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:17.266185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4bf77-88f3-4466-bccb-5fec3bc17486 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:17.266574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4bf77-88f3-4466-bccb-5fec3bc17486 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:17.267370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4bf77-88f3-4466-bccb-5fec3bc17486 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:17.269138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4bf77-88f3-4466-bccb-5fec3bc17486 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:17.291774 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3163/6388] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:12:17 2022] POST /identity/v3/roles => generated 239 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:17.299793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cb75f-6106-4b3c-a06d-532c2b97f9ee tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:17.312359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cb75f-6106-4b3c-a06d-532c2b97f9ee tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:17.317697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4981a-5241-4ef6-8d26-251a5d04045c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:17.318306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4981a-5241-4ef6-8d26-251a5d04045c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:17.318755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4981a-5241-4ef6-8d26-251a5d04045c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:17.319050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4981a-5241-4ef6-8d26-251a5d04045c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/cdfb0c12dd3c4046a2707153e0fab4f2/users/7fb1781551ca40769e67f106be4523d7/roles/2cdb15ea6edc4b129e9fb5d29409c2e6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:17.334268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c4981a-5241-4ef6-8d26-251a5d04045c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=cdfb0c12dd3c4046a2707153e0fab4f2, user_id=7fb1781551ca40769e67f106be4523d7, role_id=2cdb15ea6edc4b129e9fb5d29409c2e6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:17.337299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c4981a-5241-4ef6-8d26-251a5d04045c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:17.355092 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3226/6389] 10.222.0.16 () {60 vars in 1480 bytes} [Sat Sep 3 21:12:17 2022] PUT /identity/v3/domains/cdfb0c12dd3c4046a2707153e0fab4f2/users/7fb1781551ca40769e67f106be4523d7/roles/2cdb15ea6edc4b129e9fb5d29409c2e6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:17.363346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4bf77-88f3-4466-bccb-5fec3bc17486 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:17.378552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4bf77-88f3-4466-bccb-5fec3bc17486 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:17.444189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff432313-e546-455b-9828-bbf2835387cb tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:17.444950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff432313-e546-455b-9828-bbf2835387cb tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:17.445330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff432313-e546-455b-9828-bbf2835387cb tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:17.445739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff432313-e546-455b-9828-bbf2835387cb tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:17.446534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff432313-e546-455b-9828-bbf2835387cb tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:17.448315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff432313-e546-455b-9828-bbf2835387cb tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:17.468950 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3164/6390] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:12:17 2022] POST /identity/v3/domains => generated 277 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:17.473048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4981a-5241-4ef6-8d26-251a5d04045c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:17.488492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4981a-5241-4ef6-8d26-251a5d04045c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:17.505408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:17.565555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66038ef3-3030-445a-af2b-9d5f4571ff3c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:17.567055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66038ef3-3030-445a-af2b-9d5f4571ff3c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:17.567939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66038ef3-3030-445a-af2b-9d5f4571ff3c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:17.568603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66038ef3-3030-445a-af2b-9d5f4571ff3c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:17.571160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66038ef3-3030-445a-af2b-9d5f4571ff3c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:17.573286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66038ef3-3030-445a-af2b-9d5f4571ff3c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:17.596187 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3227/6391] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:12:17 2022] POST /identity/v3/roles => generated 239 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:17.601584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff432313-e546-455b-9828-bbf2835387cb tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:17.617927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff432313-e546-455b-9828-bbf2835387cb tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:17.624249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc1038-e2b3-44d8-a470-7a4d033d04dc tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:17.625208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc1038-e2b3-44d8-a470-7a4d033d04dc tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:17.625576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc1038-e2b3-44d8-a470-7a4d033d04dc tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:17.625926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc1038-e2b3-44d8-a470-7a4d033d04dc tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/groups` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:17.626641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc1038-e2b3-44d8-a470-7a4d033d04dc tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:17.628800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc1038-e2b3-44d8-a470-7a4d033d04dc tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:17.635849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:17.667071 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3165/6392] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Sep 3 21:12:17 2022] POST /identity/v3/groups => generated 326 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:17.671205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66038ef3-3030-445a-af2b-9d5f4571ff3c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:17.683233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66038ef3-3030-445a-af2b-9d5f4571ff3c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:17.689166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27c548-f323-48c1-8d0e-4b7056f22987 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:17.690246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27c548-f323-48c1-8d0e-4b7056f22987 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:17.690872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27c548-f323-48c1-8d0e-4b7056f22987 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:17.691376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27c548-f323-48c1-8d0e-4b7056f22987 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/groups/dc24167fbcb44faa83000fd69f6d036d/users/7fb1781551ca40769e67f106be4523d7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:17.717916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27c548-f323-48c1-8d0e-4b7056f22987 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=dc24167fbcb44faa83000fd69f6d036d, user_id=7fb1781551ca40769e67f106be4523d7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:17.725562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27c548-f323-48c1-8d0e-4b7056f22987 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:17.765180 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3228/6393] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Sep 3 21:12:17 2022] PUT /identity/v3/groups/dc24167fbcb44faa83000fd69f6d036d/users/7fb1781551ca40769e67f106be4523d7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:17.769123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc1038-e2b3-44d8-a470-7a4d033d04dc tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:17.782834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc1038-e2b3-44d8-a470-7a4d033d04dc tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:17.852207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f90eb-be74-4f91-82ab-49cf507156e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:17.865218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f90eb-be74-4f91-82ab-49cf507156e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:17.868387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f90eb-be74-4f91-82ab-49cf507156e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:17.868387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f90eb-be74-4f91-82ab-49cf507156e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/a8b40208fdc04ae18d7a4a5898dbf1fb/groups/dc24167fbcb44faa83000fd69f6d036d/roles/efcd1a5826ae46729d9cc3190710abac` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:17.883883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f90eb-be74-4f91-82ab-49cf507156e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=a8b40208fdc04ae18d7a4a5898dbf1fb, group_id=dc24167fbcb44faa83000fd69f6d036d, role_id=efcd1a5826ae46729d9cc3190710abac)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:17.886986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f90eb-be74-4f91-82ab-49cf507156e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:17.920057 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3166/6394] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:17 2022] PUT /identity/v3/domains/a8b40208fdc04ae18d7a4a5898dbf1fb/groups/dc24167fbcb44faa83000fd69f6d036d/roles/efcd1a5826ae46729d9cc3190710abac => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:17.921861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27c548-f323-48c1-8d0e-4b7056f22987 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:18.059969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27c548-f323-48c1-8d0e-4b7056f22987 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:18.165266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:18.194821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a7b1e-52d5-46a2-b927-ec11a947db88 tempest-TokensV3TestJSON-1976583448 tempest-TokensV3TestJSON-1976583448-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb1781551ca40769e67f106be4523d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ae5331900fd4b30969af9c156df5392', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:18.195552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a7b1e-52d5-46a2-b927-ec11a947db88 tempest-TokensV3TestJSON-1976583448 tempest-TokensV3TestJSON-1976583448-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:18.195945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a7b1e-52d5-46a2-b927-ec11a947db88 tempest-TokensV3TestJSON-1976583448 tempest-TokensV3TestJSON-1976583448-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:18.196332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a7b1e-52d5-46a2-b927-ec11a947db88 tempest-TokensV3TestJSON-1976583448 tempest-TokensV3TestJSON-1976583448-project] PATH_INFO: `/v3/auth/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:18.197584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a7b1e-52d5-46a2-b927-ec11a947db88 tempest-TokensV3TestJSON-1976583448 tempest-TokensV3TestJSON-1976583448-project] RBAC: Authorizing `identity:get_auth_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:18.198817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a7b1e-52d5-46a2-b927-ec11a947db88 tempest-TokensV3TestJSON-1976583448 tempest-TokensV3TestJSON-1976583448-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:18.278087 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3229/6395] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Sep 3 21:12:17 2022] GET /identity/v3/auth/domains => generated 1174 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:18.284132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f90eb-be74-4f91-82ab-49cf507156e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:18.374053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f90eb-be74-4f91-82ab-49cf507156e9 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:18.378677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1dc7d-cffe-424d-923f-46473612373b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:18.379345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1dc7d-cffe-424d-923f-46473612373b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:18.379772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1dc7d-cffe-424d-923f-46473612373b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:18.380057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1dc7d-cffe-424d-923f-46473612373b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/a8b40208fdc04ae18d7a4a5898dbf1fb/groups/dc24167fbcb44faa83000fd69f6d036d/roles/efcd1a5826ae46729d9cc3190710abac` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:18.382807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1dc7d-cffe-424d-923f-46473612373b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=a8b40208fdc04ae18d7a4a5898dbf1fb, group_id=dc24167fbcb44faa83000fd69f6d036d, role_id=efcd1a5826ae46729d9cc3190710abac)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:18.383452 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-ebc1dc7d-cffe-424d-923f-46473612373b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:12:18.383452 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:12:18.383452 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:12:18.385462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1dc7d-cffe-424d-923f-46473612373b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:18.393432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ebc1dc7d-cffe-424d-923f-46473612373b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invalidating the token cache because role efcd1a5826ae46729d9cc3190710abac was removed from group dc24167fbcb44faa83000fd69f6d036d on domain a8b40208fdc04ae18d7a4a5898dbf1fb. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:18.393432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ebc1dc7d-cffe-424d-923f-46473612373b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:18.394247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ebc1dc7d-cffe-424d-923f-46473612373b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:18.413912 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3167/6396] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Sep 3 21:12:18 2022] DELETE /identity/v3/domains/a8b40208fdc04ae18d7a4a5898dbf1fb/groups/dc24167fbcb44faa83000fd69f6d036d/roles/efcd1a5826ae46729d9cc3190710abac => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:18.418114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a7b1e-52d5-46a2-b927-ec11a947db88 tempest-TokensV3TestJSON-1976583448 tempest-TokensV3TestJSON-1976583448-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:18.419940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-220a7b1e-52d5-46a2-b927-ec11a947db88 tempest-TokensV3TestJSON-1976583448 tempest-TokensV3TestJSON-1976583448-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:18.443821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a7b1e-52d5-46a2-b927-ec11a947db88 tempest-TokensV3TestJSON-1976583448 tempest-TokensV3TestJSON-1976583448-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:18.533091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daddca6-a1f9-4431-be3c-656c618df8ca tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:18.534281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daddca6-a1f9-4431-be3c-656c618df8ca tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:18.534747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daddca6-a1f9-4431-be3c-656c618df8ca tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:18.535198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daddca6-a1f9-4431-be3c-656c618df8ca tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/groups/dc24167fbcb44faa83000fd69f6d036d/users/7fb1781551ca40769e67f106be4523d7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:18.542063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daddca6-a1f9-4431-be3c-656c618df8ca tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=dc24167fbcb44faa83000fd69f6d036d, user_id=7fb1781551ca40769e67f106be4523d7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:18.543756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daddca6-a1f9-4431-be3c-656c618df8ca tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:18.563518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1daddca6-a1f9-4431-be3c-656c618df8ca tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '7fb1781551ca40769e67f106be4523d7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:18.581992 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3230/6397] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Sep 3 21:12:18 2022] DELETE /identity/v3/groups/dc24167fbcb44faa83000fd69f6d036d/users/7fb1781551ca40769e67f106be4523d7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:18.586800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1dc7d-cffe-424d-923f-46473612373b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:18.599621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1dc7d-cffe-424d-923f-46473612373b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:18.698952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908c950-91ea-4d2a-a391-7fd564736538 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:18.699626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908c950-91ea-4d2a-a391-7fd564736538 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:18.699915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908c950-91ea-4d2a-a391-7fd564736538 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:18.700241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908c950-91ea-4d2a-a391-7fd564736538 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/groups/dc24167fbcb44faa83000fd69f6d036d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:18.702595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908c950-91ea-4d2a-a391-7fd564736538 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_group(group_id=dc24167fbcb44faa83000fd69f6d036d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:18.705513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908c950-91ea-4d2a-a391-7fd564736538 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:18.722799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:18.809078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daddca6-a1f9-4431-be3c-656c618df8ca tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:18.825019 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3168/6398] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:12:18 2022] DELETE /identity/v3/groups/dc24167fbcb44faa83000fd69f6d036d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:18.825692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daddca6-a1f9-4431-be3c-656c618df8ca tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:18.904389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0b995-2ca1-4ac7-9f43-a95978243c0e tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:18.905671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0b995-2ca1-4ac7-9f43-a95978243c0e tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:18.906193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0b995-2ca1-4ac7-9f43-a95978243c0e tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:18.906654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0b995-2ca1-4ac7-9f43-a95978243c0e tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/roles/efcd1a5826ae46729d9cc3190710abac` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:18.909765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0b995-2ca1-4ac7-9f43-a95978243c0e tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_role(role_id=efcd1a5826ae46729d9cc3190710abac)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:18.911655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0b995-2ca1-4ac7-9f43-a95978243c0e tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:18.923412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:18.963513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-53d0b995-2ca1-4ac7-9f43-a95978243c0e tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invalidating the token cache because role efcd1a5826ae46729d9cc3190710abac has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:18.963831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-53d0b995-2ca1-4ac7-9f43-a95978243c0e tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:18.964588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-53d0b995-2ca1-4ac7-9f43-a95978243c0e tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:18.968124 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3231/6399] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:18 2022] DELETE /identity/v3/roles/efcd1a5826ae46729d9cc3190710abac => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:18.972559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908c950-91ea-4d2a-a391-7fd564736538 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:18.974668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a908c950-91ea-4d2a-a391-7fd564736538 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:18.991401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908c950-91ea-4d2a-a391-7fd564736538 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:19.092649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90e5b5-bdfc-4b63-9742-1fd0d92c9067 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:19.093268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90e5b5-bdfc-4b63-9742-1fd0d92c9067 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:19.093569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90e5b5-bdfc-4b63-9742-1fd0d92c9067 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:19.093862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90e5b5-bdfc-4b63-9742-1fd0d92c9067 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/a8b40208fdc04ae18d7a4a5898dbf1fb` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:19.096301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90e5b5-bdfc-4b63-9742-1fd0d92c9067 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=a8b40208fdc04ae18d7a4a5898dbf1fb)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:19.098168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90e5b5-bdfc-4b63-9742-1fd0d92c9067 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:19.137482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af90e5b5-bdfc-4b63-9742-1fd0d92c9067 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a8b40208fdc04ae18d7a4a5898dbf1fb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.138623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af90e5b5-bdfc-4b63-9742-1fd0d92c9067 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a8b40208fdc04ae18d7a4a5898dbf1fb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.183557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af90e5b5-bdfc-4b63-9742-1fd0d92c9067 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a8b40208fdc04ae18d7a4a5898dbf1fb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.186882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-af90e5b5-bdfc-4b63-9742-1fd0d92c9067 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a8b40208fdc04ae18d7a4a5898dbf1fb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.191682 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3169/6400] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:12:18 2022] PATCH /identity/v3/domains/a8b40208fdc04ae18d7a4a5898dbf1fb => generated 278 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:19.195675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0b995-2ca1-4ac7-9f43-a95978243c0e tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:19.210807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0b995-2ca1-4ac7-9f43-a95978243c0e tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:19.239090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e785f3c-6c38-4d2a-ab28-7875cf140c71 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:19.241591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e785f3c-6c38-4d2a-ab28-7875cf140c71 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:19.242612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e785f3c-6c38-4d2a-ab28-7875cf140c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:19.307337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1e785f3c-6c38-4d2a-ab28-7875cf140c71 None None] MFA Rules not processed for user `01c449f3fd1946508ab455af5b231c14`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:19.309355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1e785f3c-6c38-4d2a-ab28-7875cf140c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:19.415763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:19.417493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:19.417969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:19.418411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/a8b40208fdc04ae18d7a4a5898dbf1fb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:19.437419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=a8b40208fdc04ae18d7a4a5898dbf1fb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:19.440322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:19.476337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a8b40208fdc04ae18d7a4a5898dbf1fb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.482495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a8b40208fdc04ae18d7a4a5898dbf1fb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.484304 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3170/6401] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:19.489090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e785f3c-6c38-4d2a-ab28-7875cf140c71 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:19.491773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1e785f3c-6c38-4d2a-ab28-7875cf140c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:19.561961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8b40208fdc04ae18d7a4a5898dbf1fb', 'request_id': 'req-b8863f1a-5a12-44f3-af99-53b631d5fe2d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.572709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:19.573805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8b40208fdc04ae18d7a4a5898dbf1fb', 'request_id': 'req-b8863f1a-5a12-44f3-af99-53b631d5fe2d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.591671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8b40208fdc04ae18d7a4a5898dbf1fb', 'request_id': 'req-b8863f1a-5a12-44f3-af99-53b631d5fe2d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.608567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] The token cache is being invalidate because project a8b40208fdc04ae18d7a4a5898dbf1fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:19.608882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.609614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.623025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a8b40208fdc04ae18d7a4a5898dbf1fb', 'request_id': 'req-b8863f1a-5a12-44f3-af99-53b631d5fe2d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.628759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a8b40208fdc04ae18d7a4a5898dbf1fb', 'request_id': 'req-b8863f1a-5a12-44f3-af99-53b631d5fe2d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.629576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a8b40208fdc04ae18d7a4a5898dbf1fb', 'request_id': 'req-b8863f1a-5a12-44f3-af99-53b631d5fe2d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.639643 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3232/6402] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:12:19 2022] DELETE /identity/v3/domains/a8b40208fdc04ae18d7a4a5898dbf1fb => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:19.645300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:19.649762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:19.682163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:19.685329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e785f3c-6c38-4d2a-ab28-7875cf140c71 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:19.821868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8863f1a-5a12-44f3-af99-53b631d5fe2d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:19.824552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e51844-09c4-49f8-9af6-a059135127a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:19.825204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e51844-09c4-49f8-9af6-a059135127a6 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:19.825488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e51844-09c4-49f8-9af6-a059135127a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:19.825763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e51844-09c4-49f8-9af6-a059135127a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:19.826405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e51844-09c4-49f8-9af6-a059135127a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:19.828858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e51844-09c4-49f8-9af6-a059135127a6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:19.838858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27effc-080c-42c0-9169-16439e3d886b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:19.839540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27effc-080c-42c0-9169-16439e3d886b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:19.839828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27effc-080c-42c0-9169-16439e3d886b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:19.840109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27effc-080c-42c0-9169-16439e3d886b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/cdfb0c12dd3c4046a2707153e0fab4f2/users/7fb1781551ca40769e67f106be4523d7/roles/2cdb15ea6edc4b129e9fb5d29409c2e6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:19.843572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27effc-080c-42c0-9169-16439e3d886b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=cdfb0c12dd3c4046a2707153e0fab4f2, user_id=7fb1781551ca40769e67f106be4523d7, role_id=2cdb15ea6edc4b129e9fb5d29409c2e6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:19.847081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27effc-080c-42c0-9169-16439e3d886b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:19.855393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ec27effc-080c-42c0-9169-16439e3d886b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invalidating the token cache because role 2cdb15ea6edc4b129e9fb5d29409c2e6 was removed from user 7fb1781551ca40769e67f106be4523d7 on domain cdfb0c12dd3c4046a2707153e0fab4f2. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:19.855695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ec27effc-080c-42c0-9169-16439e3d886b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.856418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ec27effc-080c-42c0-9169-16439e3d886b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:19.881082 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3233/6403] 10.222.0.16 () {58 vars in 1464 bytes} [Sat Sep 3 21:12:19 2022] DELETE /identity/v3/domains/cdfb0c12dd3c4046a2707153e0fab4f2/users/7fb1781551ca40769e67f106be4523d7/roles/2cdb15ea6edc4b129e9fb5d29409c2e6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:19.888105 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3171/6404] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:19 2022] POST /identity/v3/projects => generated 373 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:19.899966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e51844-09c4-49f8-9af6-a059135127a6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:19.901084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27effc-080c-42c0-9169-16439e3d886b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:19.905943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ec27effc-080c-42c0-9169-16439e3d886b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:19.906205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-09e51844-09c4-49f8-9af6-a059135127a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:19.994238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27effc-080c-42c0-9169-16439e3d886b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:19.994500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e51844-09c4-49f8-9af6-a059135127a6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:20.000250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78228f3e-48b6-4b36-8c9b-97d2942db093 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:20.001081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78228f3e-48b6-4b36-8c9b-97d2942db093 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:20.001864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78228f3e-48b6-4b36-8c9b-97d2942db093 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:20.002014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78228f3e-48b6-4b36-8c9b-97d2942db093 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/roles/2cdb15ea6edc4b129e9fb5d29409c2e6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:20.003353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78228f3e-48b6-4b36-8c9b-97d2942db093 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_role(role_id=2cdb15ea6edc4b129e9fb5d29409c2e6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:20.005153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78228f3e-48b6-4b36-8c9b-97d2942db093 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:20.052703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78228f3e-48b6-4b36-8c9b-97d2942db093 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invalidating the token cache because role 2cdb15ea6edc4b129e9fb5d29409c2e6 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:20.053110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78228f3e-48b6-4b36-8c9b-97d2942db093 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.053763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-78228f3e-48b6-4b36-8c9b-97d2942db093 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.059257 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3172/6405] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:19 2022] DELETE /identity/v3/roles/2cdb15ea6edc4b129e9fb5d29409c2e6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:20.060416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78228f3e-48b6-4b36-8c9b-97d2942db093 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:20.064432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-78228f3e-48b6-4b36-8c9b-97d2942db093 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:20.078195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5c412-f7d7-453b-b5d8-3bb93b472697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:20.080577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5c412-f7d7-453b-b5d8-3bb93b472697 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:20.081110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5c412-f7d7-453b-b5d8-3bb93b472697 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:20.081650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5c412-f7d7-453b-b5d8-3bb93b472697 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:20.082395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5c412-f7d7-453b-b5d8-3bb93b472697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:20.085664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5c412-f7d7-453b-b5d8-3bb93b472697 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:20.095701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:20.125062 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3234/6406] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:19 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:20.129978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5c412-f7d7-453b-b5d8-3bb93b472697 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:20.137403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-25f5c412-f7d7-453b-b5d8-3bb93b472697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:20.156001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5c412-f7d7-453b-b5d8-3bb93b472697 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:20.175888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78228f3e-48b6-4b36-8c9b-97d2942db093 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:20.183344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:20.184016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:20.184307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:20.184588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/cdfb0c12dd3c4046a2707153e0fab4f2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:20.186491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=cdfb0c12dd3c4046a2707153e0fab4f2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:20.192255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:20.229865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'cdfb0c12dd3c4046a2707153e0fab4f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.230940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'cdfb0c12dd3c4046a2707153e0fab4f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.259464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:20.292208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'cdfb0c12dd3c4046a2707153e0fab4f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.293136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'cdfb0c12dd3c4046a2707153e0fab4f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.294855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cd1a3-3927-4d1d-a8a0-9fc1bb1b2b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:20.295687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cd1a3-3927-4d1d-a8a0-9fc1bb1b2b6f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:20.296077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cd1a3-3927-4d1d-a8a0-9fc1bb1b2b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:20.296461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cd1a3-3927-4d1d-a8a0-9fc1bb1b2b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:20.300320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cd1a3-3927-4d1d-a8a0-9fc1bb1b2b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:20.300528 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3173/6407] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:12:20 2022] PATCH /identity/v3/domains/cdfb0c12dd3c4046a2707153e0fab4f2 => generated 278 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:20.300624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cd1a3-3927-4d1d-a8a0-9fc1bb1b2b6f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:20.301700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:20.306396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:20.320431 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3235/6408] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:20 2022] GET /identity/v3/roles => generated 1881 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:20.322527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cd1a3-3927-4d1d-a8a0-9fc1bb1b2b6f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:20.325674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-249cd1a3-3927-4d1d-a8a0-9fc1bb1b2b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:20.343703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cd1a3-3927-4d1d-a8a0-9fc1bb1b2b6f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:20.520568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd84c2-bc3b-41f8-887e-cadcc6ed77ea tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:20.521670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6fc39-8ad3-441b-8064-bc7dff609894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:20.522722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6fc39-8ad3-441b-8064-bc7dff609894 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:20.523255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6fc39-8ad3-441b-8064-bc7dff609894 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:20.524973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6fc39-8ad3-441b-8064-bc7dff609894 admin admin] PATH_INFO: `/v3/projects/d226984bcb6d4be48a4f884c26010afa/users/c276bdddd15f4682a19ba89403a1aefc/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:20.526213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:20.526943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:20.527307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:20.527662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/cdfb0c12dd3c4046a2707153e0fab4f2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:20.541356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=cdfb0c12dd3c4046a2707153e0fab4f2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:20.543367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:20.543882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6fc39-8ad3-441b-8064-bc7dff609894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d226984bcb6d4be48a4f884c26010afa, user_id=c276bdddd15f4682a19ba89403a1aefc, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:20.547207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6fc39-8ad3-441b-8064-bc7dff609894 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:20.562678 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3236/6409] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:20 2022] PUT /identity/v3/projects/d226984bcb6d4be48a4f884c26010afa/users/c276bdddd15f4682a19ba89403a1aefc/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:20.566833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6fc39-8ad3-441b-8064-bc7dff609894 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:20.571210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'cdfb0c12dd3c4046a2707153e0fab4f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.575446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'cdfb0c12dd3c4046a2707153e0fab4f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.659295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdfb0c12dd3c4046a2707153e0fab4f2', 'request_id': 'req-714fbc9d-1121-4506-8892-98db099f23b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.666474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6fc39-8ad3-441b-8064-bc7dff609894 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:20.666707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:UPDATE=1,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:20.672884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdfb0c12dd3c4046a2707153e0fab4f2', 'request_id': 'req-714fbc9d-1121-4506-8892-98db099f23b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.684220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdfb0c12dd3c4046a2707153e0fab4f2', 'request_id': 'req-714fbc9d-1121-4506-8892-98db099f23b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.702461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] The token cache is being invalidate because project cdfb0c12dd3c4046a2707153e0fab4f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:20.702758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.703457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.716094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'cdfb0c12dd3c4046a2707153e0fab4f2', 'request_id': 'req-714fbc9d-1121-4506-8892-98db099f23b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.724408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'cdfb0c12dd3c4046a2707153e0fab4f2', 'request_id': 'req-714fbc9d-1121-4506-8892-98db099f23b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.725303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'cdfb0c12dd3c4046a2707153e0fab4f2', 'request_id': 'req-714fbc9d-1121-4506-8892-98db099f23b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.735461 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3174/6410] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:12:20 2022] DELETE /identity/v3/domains/cdfb0c12dd3c4046a2707153e0fab4f2 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:20.742229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:20.743648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:20.774074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd68777-77b1-4c6b-95bb-ea927b335eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:20.775059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd68777-77b1-4c6b-95bb-ea927b335eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:20.775578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd68777-77b1-4c6b-95bb-ea927b335eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:20.776456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd68777-77b1-4c6b-95bb-ea927b335eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:20.777692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd68777-77b1-4c6b-95bb-ea927b335eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:20.782698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd68777-77b1-4c6b-95bb-ea927b335eb2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:20.793390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:20.802897 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3237/6411] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:20 2022] GET /identity/v3/roles => generated 1881 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:20.806695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd68777-77b1-4c6b-95bb-ea927b335eb2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:20.808410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bcd68777-77b1-4c6b-95bb-ea927b335eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:20.829653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd68777-77b1-4c6b-95bb-ea927b335eb2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:20.834897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebce42-ff78-4e7f-b600-075a0df00dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:20.836370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebce42-ff78-4e7f-b600-075a0df00dfc admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:20.836370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebce42-ff78-4e7f-b600-075a0df00dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:20.837071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebce42-ff78-4e7f-b600-075a0df00dfc admin admin] PATH_INFO: `/v3/projects/d226984bcb6d4be48a4f884c26010afa/users/c276bdddd15f4682a19ba89403a1aefc/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:20.839138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebce42-ff78-4e7f-b600-075a0df00dfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d226984bcb6d4be48a4f884c26010afa, user_id=c276bdddd15f4682a19ba89403a1aefc, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:20.842853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebce42-ff78-4e7f-b600-075a0df00dfc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:20.857430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fbc9d-1121-4506-8892-98db099f23b2 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:20.865909 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3238/6412] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:20 2022] PUT /identity/v3/projects/d226984bcb6d4be48a4f884c26010afa/users/c276bdddd15f4682a19ba89403a1aefc/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:20.957507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dae67-998e-4722-85b9-30473541c7d8 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:20.960441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dae67-998e-4722-85b9-30473541c7d8 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:20.960674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dae67-998e-4722-85b9-30473541c7d8 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:20.960994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dae67-998e-4722-85b9-30473541c7d8 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/4045427e38754c50a5a05015816db9d3/groups/bba8e8bc274a40eeb3a2f7cca4a1c43c/roles/c4e0622210b447b6951911eacd99612a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:20.971138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98dae67-998e-4722-85b9-30473541c7d8 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=4045427e38754c50a5a05015816db9d3, group_id=bba8e8bc274a40eeb3a2f7cca4a1c43c, role_id=c4e0622210b447b6951911eacd99612a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:20.975147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98dae67-998e-4722-85b9-30473541c7d8 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:20.984938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f98dae67-998e-4722-85b9-30473541c7d8 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invalidating the token cache because role c4e0622210b447b6951911eacd99612a was removed from group bba8e8bc274a40eeb3a2f7cca4a1c43c on domain 4045427e38754c50a5a05015816db9d3. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:20.985410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f98dae67-998e-4722-85b9-30473541c7d8 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:20.991545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f98dae67-998e-4722-85b9-30473541c7d8 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:21.016965 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3175/6413] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Sep 3 21:12:20 2022] DELETE /identity/v3/domains/4045427e38754c50a5a05015816db9d3/groups/bba8e8bc274a40eeb3a2f7cca4a1c43c/roles/c4e0622210b447b6951911eacd99612a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:21.018490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebce42-ff78-4e7f-b600-075a0df00dfc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:21.020632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2aebce42-ff78-4e7f-b600-075a0df00dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:21.043623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebce42-ff78-4e7f-b600-075a0df00dfc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:21.152707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470eb69-4dee-4b7c-8a7e-96c994199b47 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:21.153703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470eb69-4dee-4b7c-8a7e-96c994199b47 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:21.154221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470eb69-4dee-4b7c-8a7e-96c994199b47 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:21.155300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470eb69-4dee-4b7c-8a7e-96c994199b47 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/groups/bba8e8bc274a40eeb3a2f7cca4a1c43c/users/7fb1781551ca40769e67f106be4523d7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:21.157636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470eb69-4dee-4b7c-8a7e-96c994199b47 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=bba8e8bc274a40eeb3a2f7cca4a1c43c, user_id=7fb1781551ca40769e67f106be4523d7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:21.159816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470eb69-4dee-4b7c-8a7e-96c994199b47 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:21.169668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7470eb69-4dee-4b7c-8a7e-96c994199b47 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '7fb1781551ca40769e67f106be4523d7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:21.190948 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3239/6414] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Sep 3 21:12:21 2022] DELETE /identity/v3/groups/bba8e8bc274a40eeb3a2f7cca4a1c43c/users/7fb1781551ca40769e67f106be4523d7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:21.195431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dae67-998e-4722-85b9-30473541c7d8 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:21.208424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dae67-998e-4722-85b9-30473541c7d8 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:21.309657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac9bd2-ec80-4bee-8867-6b07c1358f78 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:21.310317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac9bd2-ec80-4bee-8867-6b07c1358f78 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:21.314002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac9bd2-ec80-4bee-8867-6b07c1358f78 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:21.314185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac9bd2-ec80-4bee-8867-6b07c1358f78 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/groups/bba8e8bc274a40eeb3a2f7cca4a1c43c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:21.316586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac9bd2-ec80-4bee-8867-6b07c1358f78 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_group(group_id=bba8e8bc274a40eeb3a2f7cca4a1c43c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:21.318231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac9bd2-ec80-4bee-8867-6b07c1358f78 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:21.320432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:21.404006 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3176/6415] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:12:21 2022] DELETE /identity/v3/groups/bba8e8bc274a40eeb3a2f7cca4a1c43c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:21.409447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470eb69-4dee-4b7c-8a7e-96c994199b47 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:21.426893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470eb69-4dee-4b7c-8a7e-96c994199b47 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:21.516051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbd9db-a3df-4387-8b88-c965b880cc33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:21.516420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbd9db-a3df-4387-8b88-c965b880cc33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:21.516675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbd9db-a3df-4387-8b88-c965b880cc33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:21.516994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbd9db-a3df-4387-8b88-c965b880cc33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/roles/c4e0622210b447b6951911eacd99612a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:21.518973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbd9db-a3df-4387-8b88-c965b880cc33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_role(role_id=c4e0622210b447b6951911eacd99612a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:21.520611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbd9db-a3df-4387-8b88-c965b880cc33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:21.560480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cdfbd9db-a3df-4387-8b88-c965b880cc33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invalidating the token cache because role c4e0622210b447b6951911eacd99612a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:21.561573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cdfbd9db-a3df-4387-8b88-c965b880cc33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:21.562697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cdfbd9db-a3df-4387-8b88-c965b880cc33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:21.564600 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3240/6416] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:21 2022] DELETE /identity/v3/roles/c4e0622210b447b6951911eacd99612a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:21.568695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac9bd2-ec80-4bee-8867-6b07c1358f78 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:21.570607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c4ac9bd2-ec80-4bee-8867-6b07c1358f78 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:21.584418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac9bd2-ec80-4bee-8867-6b07c1358f78 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:21.730124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:21.730807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:21.731084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:21.731372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/4045427e38754c50a5a05015816db9d3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:21.737210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=4045427e38754c50a5a05015816db9d3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:21.739378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:21.793224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '4045427e38754c50a5a05015816db9d3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:21.794057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '4045427e38754c50a5a05015816db9d3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:21.852234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '4045427e38754c50a5a05015816db9d3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:21.854605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '4045427e38754c50a5a05015816db9d3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:21.862451 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3177/6417] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:12:21 2022] PATCH /identity/v3/domains/4045427e38754c50a5a05015816db9d3 => generated 277 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:21.865883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbd9db-a3df-4387-8b88-c965b880cc33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:21.882469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbd9db-a3df-4387-8b88-c965b880cc33 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:21.987553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:22.047695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:22.048335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:22.050164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:22.050461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/4045427e38754c50a5a05015816db9d3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:22.091638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=4045427e38754c50a5a05015816db9d3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:22.098964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:22.161216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '4045427e38754c50a5a05015816db9d3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.172260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '4045427e38754c50a5a05015816db9d3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.311055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4045427e38754c50a5a05015816db9d3', 'request_id': 'req-2442e4f4-7da5-45a5-a6eb-46f52d582014'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.325910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4045427e38754c50a5a05015816db9d3', 'request_id': 'req-2442e4f4-7da5-45a5-a6eb-46f52d582014'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.351388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4045427e38754c50a5a05015816db9d3', 'request_id': 'req-2442e4f4-7da5-45a5-a6eb-46f52d582014'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.374687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] The token cache is being invalidate because project 4045427e38754c50a5a05015816db9d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:22.375277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.376249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.401776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '4045427e38754c50a5a05015816db9d3', 'request_id': 'req-2442e4f4-7da5-45a5-a6eb-46f52d582014'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.411914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '4045427e38754c50a5a05015816db9d3', 'request_id': 'req-2442e4f4-7da5-45a5-a6eb-46f52d582014'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.413422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '4045427e38754c50a5a05015816db9d3', 'request_id': 'req-2442e4f4-7da5-45a5-a6eb-46f52d582014'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.423165 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3241/6418] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:12:21 2022] DELETE /identity/v3/domains/4045427e38754c50a5a05015816db9d3 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:22.427315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:22.428963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:22.452609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588a39a-b43e-4973-bd92-8c85a636e5f4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:22.581045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748c479-9842-4db5-a337-97c67d7e78a6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:22.582396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748c479-9842-4db5-a337-97c67d7e78a6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:22.582396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748c479-9842-4db5-a337-97c67d7e78a6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:22.584086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748c479-9842-4db5-a337-97c67d7e78a6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/6c54abfb4d3e4de1bd05e0106ab6a180/users/7fb1781551ca40769e67f106be4523d7/roles/75fa868bca424a319fb678e215ace386` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:22.594645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748c479-9842-4db5-a337-97c67d7e78a6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=6c54abfb4d3e4de1bd05e0106ab6a180, user_id=7fb1781551ca40769e67f106be4523d7, role_id=75fa868bca424a319fb678e215ace386)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:22.598372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748c479-9842-4db5-a337-97c67d7e78a6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:22.608863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a748c479-9842-4db5-a337-97c67d7e78a6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invalidating the token cache because role 75fa868bca424a319fb678e215ace386 was removed from user 7fb1781551ca40769e67f106be4523d7 on domain 6c54abfb4d3e4de1bd05e0106ab6a180. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:22.609384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a748c479-9842-4db5-a337-97c67d7e78a6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.610322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a748c479-9842-4db5-a337-97c67d7e78a6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.637903 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3178/6419] 10.222.0.16 () {58 vars in 1464 bytes} [Sat Sep 3 21:12:22 2022] DELETE /identity/v3/domains/6c54abfb4d3e4de1bd05e0106ab6a180/users/7fb1781551ca40769e67f106be4523d7/roles/75fa868bca424a319fb678e215ace386 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:22.642299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:22.646430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:22.664554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442e4f4-7da5-45a5-a6eb-46f52d582014 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:22.827915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15696e28-cf59-4450-a41a-83e695402857 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:22.828217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696e28-cf59-4450-a41a-83e695402857 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:22.828616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696e28-cf59-4450-a41a-83e695402857 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:22.829039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696e28-cf59-4450-a41a-83e695402857 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/roles/75fa868bca424a319fb678e215ace386` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:22.830646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15696e28-cf59-4450-a41a-83e695402857 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_role(role_id=75fa868bca424a319fb678e215ace386)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:22.832616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15696e28-cf59-4450-a41a-83e695402857 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:22.849510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:22.927897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-15696e28-cf59-4450-a41a-83e695402857 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invalidating the token cache because role 75fa868bca424a319fb678e215ace386 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:22.934055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-15696e28-cf59-4450-a41a-83e695402857 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.935094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-15696e28-cf59-4450-a41a-83e695402857 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:22.957801 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3242/6420] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:22 2022] DELETE /identity/v3/roles/75fa868bca424a319fb678e215ace386 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:22.962856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748c479-9842-4db5-a337-97c67d7e78a6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:22.964985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a748c479-9842-4db5-a337-97c67d7e78a6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:22.999113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748c479-9842-4db5-a337-97c67d7e78a6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:23.011704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15696e28-cf59-4450-a41a-83e695402857 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:23.015004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-15696e28-cf59-4450-a41a-83e695402857 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:23.056820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:UPDATE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:23.125399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15696e28-cf59-4450-a41a-83e695402857 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:23.135821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a5823-6e30-4f92-9cb1-3a124dc7ffcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:23.136966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a5823-6e30-4f92-9cb1-3a124dc7ffcf service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:23.137484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a5823-6e30-4f92-9cb1-3a124dc7ffcf service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:23.138571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a5823-6e30-4f92-9cb1-3a124dc7ffcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:23.142487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1f9a5823-6e30-4f92-9cb1-3a124dc7ffcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:23.231174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:23.233047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:23.233343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:23.233943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/6c54abfb4d3e4de1bd05e0106ab6a180` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:23.236357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=6c54abfb4d3e4de1bd05e0106ab6a180)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:23.241582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:23.275756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a5823-6e30-4f92-9cb1-3a124dc7ffcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:23.277511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a5823-6e30-4f92-9cb1-3a124dc7ffcf service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:23.303156 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3243/6421] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:12:23.322523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '6c54abfb4d3e4de1bd05e0106ab6a180'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.331988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '6c54abfb4d3e4de1bd05e0106ab6a180'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.399662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '6c54abfb4d3e4de1bd05e0106ab6a180'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.401705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '6c54abfb4d3e4de1bd05e0106ab6a180'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.408451 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3179/6422] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:12:22 2022] PATCH /identity/v3/domains/6c54abfb4d3e4de1bd05e0106ab6a180 => generated 277 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:23.410630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a5823-6e30-4f92-9cb1-3a124dc7ffcf service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:23.412349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1f9a5823-6e30-4f92-9cb1-3a124dc7ffcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:23.442013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a5823-6e30-4f92-9cb1-3a124dc7ffcf service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:23.615266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:23.616077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:23.616372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:23.616760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/6c54abfb4d3e4de1bd05e0106ab6a180` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:23.652778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=6c54abfb4d3e4de1bd05e0106ab6a180)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:23.654538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:23.754577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '6c54abfb4d3e4de1bd05e0106ab6a180'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.759972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:23.766594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '6c54abfb4d3e4de1bd05e0106ab6a180'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.862327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c54abfb4d3e4de1bd05e0106ab6a180', 'request_id': 'req-6eeea657-c217-42c9-beed-bf123c57d27a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.871455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c54abfb4d3e4de1bd05e0106ab6a180', 'request_id': 'req-6eeea657-c217-42c9-beed-bf123c57d27a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.893337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c54abfb4d3e4de1bd05e0106ab6a180', 'request_id': 'req-6eeea657-c217-42c9-beed-bf123c57d27a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.912845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] The token cache is being invalidate because project 6c54abfb4d3e4de1bd05e0106ab6a180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:23.914202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.915124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.943306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '6c54abfb4d3e4de1bd05e0106ab6a180', 'request_id': 'req-6eeea657-c217-42c9-beed-bf123c57d27a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.948376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '6c54abfb4d3e4de1bd05e0106ab6a180', 'request_id': 'req-6eeea657-c217-42c9-beed-bf123c57d27a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.949497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '6c54abfb4d3e4de1bd05e0106ab6a180', 'request_id': 'req-6eeea657-c217-42c9-beed-bf123c57d27a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:23.960814 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3244/6423] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:12:23 2022] DELETE /identity/v3/domains/6c54abfb4d3e4de1bd05e0106ab6a180 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:23.970969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:23.972850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:24.005542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5f92f-ef22-4da5-bfea-d4c3459476e6 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:24.181669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff582f-074a-47c4-a561-96030d0c92c5 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:24.182499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff582f-074a-47c4-a561-96030d0c92c5 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:24.182618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff582f-074a-47c4-a561-96030d0c92c5 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:24.182874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff582f-074a-47c4-a561-96030d0c92c5 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:24.183538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff582f-074a-47c4-a561-96030d0c92c5 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:24.185200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff582f-074a-47c4-a561-96030d0c92c5 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:24.205860 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3180/6424] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:24.210045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:24.213872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:24.238148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeea657-c217-42c9-beed-bf123c57d27a tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:24.249086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3893d-85be-4078-a9e7-f5f6343f592b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:24.250031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3893d-85be-4078-a9e7-f5f6343f592b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:24.250531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3893d-85be-4078-a9e7-f5f6343f592b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:24.250817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3893d-85be-4078-a9e7-f5f6343f592b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/projects/2474af174b1f4087a11d696643570ef9/users/8b6096f19b1a4028aa81c2b52a30e67f/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:24.255594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3893d-85be-4078-a9e7-f5f6343f592b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_grant(project_id=2474af174b1f4087a11d696643570ef9, user_id=8b6096f19b1a4028aa81c2b52a30e67f, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:24.273824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3893d-85be-4078-a9e7-f5f6343f592b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:24.300184 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3245/6425] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:24 2022] PUT /identity/v3/projects/2474af174b1f4087a11d696643570ef9/users/8b6096f19b1a4028aa81c2b52a30e67f/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:24.301571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff582f-074a-47c4-a561-96030d0c92c5 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:24.319708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff582f-074a-47c4-a561-96030d0c92c5 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:24.427156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd5633-97a6-4412-83fc-d04d056dd30c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:24.428085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd5633-97a6-4412-83fc-d04d056dd30c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:24.428335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd5633-97a6-4412-83fc-d04d056dd30c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:24.428716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd5633-97a6-4412-83fc-d04d056dd30c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/auth/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:24.429440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd5633-97a6-4412-83fc-d04d056dd30c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:get_auth_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:24.430641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd5633-97a6-4412-83fc-d04d056dd30c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:24.460057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:24.503611 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3181/6426] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Sep 3 21:12:24 2022] GET /identity/v3/auth/projects => generated 826 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:24.507613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3893d-85be-4078-a9e7-f5f6343f592b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:24.519575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3893d-85be-4078-a9e7-f5f6343f592b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:24.524238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2e238-af8b-4053-a9a6-0087f612b130 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:24.524949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2e238-af8b-4053-a9a6-0087f612b130 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:24.525249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2e238-af8b-4053-a9a6-0087f612b130 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:24.525534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2e238-af8b-4053-a9a6-0087f612b130 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/projects/2474af174b1f4087a11d696643570ef9/users/8b6096f19b1a4028aa81c2b52a30e67f/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:24.527552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2e238-af8b-4053-a9a6-0087f612b130 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=2474af174b1f4087a11d696643570ef9, user_id=8b6096f19b1a4028aa81c2b52a30e67f, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:24.530271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2e238-af8b-4053-a9a6-0087f612b130 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:24.536265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ac2e238-af8b-4053-a9a6-0087f612b130 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invalidating the token cache because role bc768522dc984239be4a1b39a45ab437 was removed from user 8b6096f19b1a4028aa81c2b52a30e67f on project 2474af174b1f4087a11d696643570ef9. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:24.536564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ac2e238-af8b-4053-a9a6-0087f612b130 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:24.537292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ac2e238-af8b-4053-a9a6-0087f612b130 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:24.555420 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3246/6427] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Sep 3 21:12:24 2022] DELETE /identity/v3/projects/2474af174b1f4087a11d696643570ef9/users/8b6096f19b1a4028aa81c2b52a30e67f/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:24.557550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd5633-97a6-4412-83fc-d04d056dd30c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:24.561778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b7dd5633-97a6-4412-83fc-d04d056dd30c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:24.582455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd5633-97a6-4412-83fc-d04d056dd30c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:24.673384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c9b8-923d-4e1c-ae01-f440ff29f75b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:24.674772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c9b8-923d-4e1c-ae01-f440ff29f75b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:24.675222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c9b8-923d-4e1c-ae01-f440ff29f75b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:24.675693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c9b8-923d-4e1c-ae01-f440ff29f75b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:24.677759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641c9b8-923d-4e1c-ae01-f440ff29f75b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:24.682815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641c9b8-923d-4e1c-ae01-f440ff29f75b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:24.735873 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3182/6428] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:24 2022] POST /identity/v3/users => generated 323 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:24.739865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2e238-af8b-4053-a9a6-0087f612b130 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:24.755991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2e238-af8b-4053-a9a6-0087f612b130 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:24.770859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad066b-25b5-4670-bde1-a1c597174574 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:24.771508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad066b-25b5-4670-bde1-a1c597174574 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:24.773285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad066b-25b5-4670-bde1-a1c597174574 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:24.773910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad066b-25b5-4670-bde1-a1c597174574 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:24.776867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad066b-25b5-4670-bde1-a1c597174574 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:24.778949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad066b-25b5-4670-bde1-a1c597174574 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:24.824399 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3247/6429] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:24 2022] POST /identity/v3/projects => generated 363 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:24.828696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c9b8-923d-4e1c-ae01-f440ff29f75b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:24.869372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c9b8-923d-4e1c-ae01-f440ff29f75b tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:25.034448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84003d36-37a3-4199-a981-6de4de1d6582 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:25.035128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84003d36-37a3-4199-a981-6de4de1d6582 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:25.035363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84003d36-37a3-4199-a981-6de4de1d6582 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:25.035639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84003d36-37a3-4199-a981-6de4de1d6582 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:25.036286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84003d36-37a3-4199-a981-6de4de1d6582 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:25.038283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84003d36-37a3-4199-a981-6de4de1d6582 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:25.089966 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3183/6430] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:24 2022] POST /identity/v3/projects => generated 362 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:25.093510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad066b-25b5-4670-bde1-a1c597174574 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:25.104882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad066b-25b5-4670-bde1-a1c597174574 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:25.235151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030937a3-6c98-4c1e-ac3f-d2a958e8d967 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:25.235786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030937a3-6c98-4c1e-ac3f-d2a958e8d967 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:25.236075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030937a3-6c98-4c1e-ac3f-d2a958e8d967 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:25.236356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030937a3-6c98-4c1e-ac3f-d2a958e8d967 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:25.237480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030937a3-6c98-4c1e-ac3f-d2a958e8d967 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:25.239100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030937a3-6c98-4c1e-ac3f-d2a958e8d967 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:25.261747 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3248/6431] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:12:25 2022] POST /identity/v3/roles => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:25.264273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84003d36-37a3-4199-a981-6de4de1d6582 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:25.275780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84003d36-37a3-4199-a981-6de4de1d6582 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:25.293766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e668a-2eb6-4b92-a35a-fbf8ad32f262 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:25.294477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e668a-2eb6-4b92-a35a-fbf8ad32f262 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:25.294747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e668a-2eb6-4b92-a35a-fbf8ad32f262 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:25.295240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e668a-2eb6-4b92-a35a-fbf8ad32f262 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/projects/bf6b830f79364bf0890dc05c2bb3a3e9/users/eabc29cf0de044b6971ea394fb5b9a99/roles/4865976d2c6445b295f1e41498db8354` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:25.323199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e668a-2eb6-4b92-a35a-fbf8ad32f262 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_grant(project_id=bf6b830f79364bf0890dc05c2bb3a3e9, user_id=eabc29cf0de044b6971ea394fb5b9a99, role_id=4865976d2c6445b295f1e41498db8354)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:25.328994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e668a-2eb6-4b92-a35a-fbf8ad32f262 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:25.355449 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3184/6432] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:25 2022] PUT /identity/v3/projects/bf6b830f79364bf0890dc05c2bb3a3e9/users/eabc29cf0de044b6971ea394fb5b9a99/roles/4865976d2c6445b295f1e41498db8354 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:25.360385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030937a3-6c98-4c1e-ac3f-d2a958e8d967 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:25.377493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030937a3-6c98-4c1e-ac3f-d2a958e8d967 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:25.388748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:25.510884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056285b-b785-41c3-a592-ec7cfcad336d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:25.512985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056285b-b785-41c3-a592-ec7cfcad336d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:25.514835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056285b-b785-41c3-a592-ec7cfcad336d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:25.515368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056285b-b785-41c3-a592-ec7cfcad336d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/projects/35f686834c564f4c83fd9c773f18b1f8/users/eabc29cf0de044b6971ea394fb5b9a99/roles/4865976d2c6445b295f1e41498db8354` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:25.519386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056285b-b785-41c3-a592-ec7cfcad336d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:create_grant(project_id=35f686834c564f4c83fd9c773f18b1f8, user_id=eabc29cf0de044b6971ea394fb5b9a99, role_id=4865976d2c6445b295f1e41498db8354)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:25.530758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056285b-b785-41c3-a592-ec7cfcad336d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:25.562112 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3249/6433] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:25 2022] PUT /identity/v3/projects/35f686834c564f4c83fd9c773f18b1f8/users/eabc29cf0de044b6971ea394fb5b9a99/roles/4865976d2c6445b295f1e41498db8354 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:25.566552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb836da0-01fa-47c3-86d7-5d4214b0019e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:25.567195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb836da0-01fa-47c3-86d7-5d4214b0019e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:25.567735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb836da0-01fa-47c3-86d7-5d4214b0019e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:25.636228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bb836da0-01fa-47c3-86d7-5d4214b0019e None None] MFA Rules not processed for user `eabc29cf0de044b6971ea394fb5b9a99`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:25.638154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bb836da0-01fa-47c3-86d7-5d4214b0019e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:25.649189 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3185/6434] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:25 2022] POST /identity/v3/auth/tokens => generated 335 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:12:25.653391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b594c-538c-4194-8843-aff11878fbb7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:25.653710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b594c-538c-4194-8843-aff11878fbb7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:25.654070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b594c-538c-4194-8843-aff11878fbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:25.666220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7e3b594c-538c-4194-8843-aff11878fbb7 None None] MFA Rules not processed for user `eabc29cf0de044b6971ea394fb5b9a99`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:25.667580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7e3b594c-538c-4194-8843-aff11878fbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:25.794973 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3250/6435] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:25 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 03 21:12:25.800281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb836da0-01fa-47c3-86d7-5d4214b0019e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:25.834951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:25.955866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb836da0-01fa-47c3-86d7-5d4214b0019e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:25.961490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d0c3e-c27c-42ca-afa5-694afba67678 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:25.962261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d0c3e-c27c-42ca-afa5-694afba67678 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:25.962666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d0c3e-c27c-42ca-afa5-694afba67678 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:25.963092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d0c3e-c27c-42ca-afa5-694afba67678 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:25.981821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d0c3e-c27c-42ca-afa5-694afba67678 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:revoke_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:25.995163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d0c3e-c27c-42ca-afa5-694afba67678 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:26.041772 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3186/6436] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Sep 3 21:12:25 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:26.045843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf1943-b618-4242-81e9-4e1a811fee7f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:26.046179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf1943-b618-4242-81e9-4e1a811fee7f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:26.046460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf1943-b618-4242-81e9-4e1a811fee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:26.081710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f1cf1943-b618-4242-81e9-4e1a811fee7f None None] MFA Rules not processed for user `eabc29cf0de044b6971ea394fb5b9a99`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:26.081710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f1cf1943-b618-4242-81e9-4e1a811fee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:26.190129 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3251/6437] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:26 2022] POST /identity/v3/auth/tokens => generated 3225 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 03 21:12:26.194318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d0c3e-c27c-42ca-afa5-694afba67678 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:26.207710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d0c3e-c27c-42ca-afa5-694afba67678 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:26.224389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1c9e2-a1db-40c5-bc81-26e42e912c94 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:26.225084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1c9e2-a1db-40c5-bc81-26e42e912c94 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:26.225376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1c9e2-a1db-40c5-bc81-26e42e912c94 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:26.225660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1c9e2-a1db-40c5-bc81-26e42e912c94 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/roles/4865976d2c6445b295f1e41498db8354` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:26.227346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e1c9e2-a1db-40c5-bc81-26e42e912c94 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_role(role_id=4865976d2c6445b295f1e41498db8354)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:26.229000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e1c9e2-a1db-40c5-bc81-26e42e912c94 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:26.321659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c5e1c9e2-a1db-40c5-bc81-26e42e912c94 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invalidating the token cache because role 4865976d2c6445b295f1e41498db8354 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:26.322000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c5e1c9e2-a1db-40c5-bc81-26e42e912c94 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:26.324696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c5e1c9e2-a1db-40c5-bc81-26e42e912c94 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:26.330138 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3187/6438] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:26 2022] DELETE /identity/v3/roles/4865976d2c6445b295f1e41498db8354 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:26.334301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf1943-b618-4242-81e9-4e1a811fee7f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:26.352655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf1943-b618-4242-81e9-4e1a811fee7f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:26.471495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:26.512169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:26.512813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:26.515042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:26.516113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/projects/35f686834c564f4c83fd9c773f18b1f8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:26.518510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_project(project_id=35f686834c564f4c83fd9c773f18b1f8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:26.525221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:26.617991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f686834c564f4c83fd9c773f18b1f8', 'request_id': 'req-b3d44485-08a3-49cc-a8b7-ede63497e63d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:26.629526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f686834c564f4c83fd9c773f18b1f8', 'request_id': 'req-b3d44485-08a3-49cc-a8b7-ede63497e63d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:26.637572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f686834c564f4c83fd9c773f18b1f8', 'request_id': 'req-b3d44485-08a3-49cc-a8b7-ede63497e63d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:26.654741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] The token cache is being invalidate because project 35f686834c564f4c83fd9c773f18b1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:26.655149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:26.656799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:26.658977 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3252/6439] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:26 2022] DELETE /identity/v3/projects/35f686834c564f4c83fd9c773f18b1f8 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:26.662837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1c9e2-a1db-40c5-bc81-26e42e912c94 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:26.664261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c5e1c9e2-a1db-40c5-bc81-26e42e912c94 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:26.688004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1c9e2-a1db-40c5-bc81-26e42e912c94 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:26.852455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de776f7-4937-4d0a-b4cd-375b35bd3168 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:26.853834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de776f7-4937-4d0a-b4cd-375b35bd3168 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:26.854131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de776f7-4937-4d0a-b4cd-375b35bd3168 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:26.854417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de776f7-4937-4d0a-b4cd-375b35bd3168 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/projects/35f686834c564f4c83fd9c773f18b1f8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:26.873311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de776f7-4937-4d0a-b4cd-375b35bd3168 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_project(project_id=35f686834c564f4c83fd9c773f18b1f8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:26.875271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de776f7-4937-4d0a-b4cd-375b35bd3168 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-1de776f7-4937-4d0a-b4cd-375b35bd3168 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Could not find project: 35f686834c564f4c83fd9c773f18b1f8.: keystone.exception.ProjectNotFound: Could not find project: 35f686834c564f4c83fd9c773f18b1f8. Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Sep 03 21:12:26.899204 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Sep 03 21:12:26.900519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:12:26.900519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:12:26.900519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Sep 03 21:12:26.900519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Sep 03 21:12:26.900519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:12:26.900519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:12:26.900519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:12:26.900519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:12:26.900519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 35f686834c564f4c83fd9c773f18b1f8. Sep 03 21:12:26.900519 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:26.903574 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3188/6440] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:26 2022] DELETE /identity/v3/projects/35f686834c564f4c83fd9c773f18b1f8 => generated 113 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:12:26.905709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:26.907689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:26.932836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d44485-08a3-49cc-a8b7-ede63497e63d tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:26.949045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:26.949721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:26.950045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:26.950290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/projects/bf6b830f79364bf0890dc05c2bb3a3e9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:26.959076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bf6b830f79364bf0890dc05c2bb3a3e9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:26.968996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:27.111058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf6b830f79364bf0890dc05c2bb3a3e9', 'request_id': 'req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:27.137119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf6b830f79364bf0890dc05c2bb3a3e9', 'request_id': 'req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:27.153359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf6b830f79364bf0890dc05c2bb3a3e9', 'request_id': 'req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:27.187195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] The token cache is being invalidate because project bf6b830f79364bf0890dc05c2bb3a3e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:27.187510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:27.201071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:27.206659 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3253/6441] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:26 2022] DELETE /identity/v3/projects/bf6b830f79364bf0890dc05c2bb3a3e9 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:27.213212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de776f7-4937-4d0a-b4cd-375b35bd3168 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:27.215123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1de776f7-4937-4d0a-b4cd-375b35bd3168 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:27.239112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de776f7-4937-4d0a-b4cd-375b35bd3168 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:27.351633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:27.372875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:27.374103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:27.374463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:27.374815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/users/eabc29cf0de044b6971ea394fb5b9a99` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:27.376208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorizing `identity:delete_user(user_id=eabc29cf0de044b6971ea394fb5b9a99)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:27.378980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:27.441039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eabc29cf0de044b6971ea394fb5b9a99'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:27.455765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eabc29cf0de044b6971ea394fb5b9a99'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:27.456764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eabc29cf0de044b6971ea394fb5b9a99'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:27.464932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eabc29cf0de044b6971ea394fb5b9a99'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:27.484150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eabc29cf0de044b6971ea394fb5b9a99'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:27.491625 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3189/6442] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:27 2022] DELETE /identity/v3/users/eabc29cf0de044b6971ea394fb5b9a99 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:27.496562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:27.526961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ef3eb-8d9e-4cec-9f18-fc33c02c9564 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:27.617690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04254ae-15b8-46be-a358-f8fd4d7e2448 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:27.619006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04254ae-15b8-46be-a358-f8fd4d7e2448 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:27.619547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04254ae-15b8-46be-a358-f8fd4d7e2448 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:27.620078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04254ae-15b8-46be-a358-f8fd4d7e2448 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/a8b40208fdc04ae18d7a4a5898dbf1fb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:27.624479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:27.637912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04254ae-15b8-46be-a358-f8fd4d7e2448 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-c04254ae-15b8-46be-a358-f8fd4d7e2448 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Could not find domain: a8b40208fdc04ae18d7a4a5898dbf1fb.: keystone.exception.DomainNotFound: Could not find domain: a8b40208fdc04ae18d7a4a5898dbf1fb. Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: a8b40208fdc04ae18d7a4a5898dbf1fb. Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:12:27.647192 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:12:27.648568 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: a8b40208fdc04ae18d7a4a5898dbf1fb. Sep 03 21:12:27.650403 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:27.656132 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3254/6443] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:12:27 2022] PATCH /identity/v3/domains/a8b40208fdc04ae18d7a4a5898dbf1fb => generated 112 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:12:27.659834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:27.662044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:27.685024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23adf6e-787f-4169-a38e-b7c2ad40de7c tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:27.702231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37165027-98a9-4040-a042-afcb22d5f5f7 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:27.703167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37165027-98a9-4040-a042-afcb22d5f5f7 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:27.703167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37165027-98a9-4040-a042-afcb22d5f5f7 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:27.703533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37165027-98a9-4040-a042-afcb22d5f5f7 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/cdfb0c12dd3c4046a2707153e0fab4f2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:27.718541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37165027-98a9-4040-a042-afcb22d5f5f7 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-37165027-98a9-4040-a042-afcb22d5f5f7 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Could not find domain: cdfb0c12dd3c4046a2707153e0fab4f2.: keystone.exception.DomainNotFound: Could not find domain: cdfb0c12dd3c4046a2707153e0fab4f2. Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: cdfb0c12dd3c4046a2707153e0fab4f2. Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:12:27.739679 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:12:27.741148 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: cdfb0c12dd3c4046a2707153e0fab4f2. Sep 03 21:12:27.742571 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:27.744010 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3190/6444] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:12:27 2022] PATCH /identity/v3/domains/cdfb0c12dd3c4046a2707153e0fab4f2 => generated 112 bytes in 82 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:12:27.744927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04254ae-15b8-46be-a358-f8fd4d7e2448 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:27.756687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04254ae-15b8-46be-a358-f8fd4d7e2448 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:27.774436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64046fd0-b33c-48e7-8238-da4f5d7eede3 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:27.775155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64046fd0-b33c-48e7-8238-da4f5d7eede3 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:27.775452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64046fd0-b33c-48e7-8238-da4f5d7eede3 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:27.775735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64046fd0-b33c-48e7-8238-da4f5d7eede3 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/4045427e38754c50a5a05015816db9d3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:27.790887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64046fd0-b33c-48e7-8238-da4f5d7eede3 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-64046fd0-b33c-48e7-8238-da4f5d7eede3 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Could not find domain: 4045427e38754c50a5a05015816db9d3.: keystone.exception.DomainNotFound: Could not find domain: 4045427e38754c50a5a05015816db9d3. Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 4045427e38754c50a5a05015816db9d3. Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:12:27.798929 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:12:27.800332 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 4045427e38754c50a5a05015816db9d3. Sep 03 21:12:27.801609 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:27.806725 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3255/6445] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:12:27 2022] PATCH /identity/v3/domains/4045427e38754c50a5a05015816db9d3 => generated 112 bytes in 62 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 03 21:12:27.810748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37165027-98a9-4040-a042-afcb22d5f5f7 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:27.831896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37165027-98a9-4040-a042-afcb22d5f5f7 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:27.848021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb58e8-c2d0-4b7f-aa1e-1aa581680ae4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6096f19b1a4028aa81c2b52a30e67f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '715e5d7d48d84c4594e859a12b70290d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:27.848934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb58e8-c2d0-4b7f-aa1e-1aa581680ae4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:27.849403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb58e8-c2d0-4b7f-aa1e-1aa581680ae4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:27.849860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb58e8-c2d0-4b7f-aa1e-1aa581680ae4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] PATH_INFO: `/v3/domains/6c54abfb4d3e4de1bd05e0106ab6a180` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:27.863429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb58e8-c2d0-4b7f-aa1e-1aa581680ae4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-97bb58e8-c2d0-4b7f-aa1e-1aa581680ae4 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Could not find domain: 6c54abfb4d3e4de1bd05e0106ab6a180.: keystone.exception.DomainNotFound: Could not find domain: 6c54abfb4d3e4de1bd05e0106ab6a180. Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 6c54abfb4d3e4de1bd05e0106ab6a180. Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:12:27.869699 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:12:27.871141 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 6c54abfb4d3e4de1bd05e0106ab6a180. Sep 03 21:12:27.872490 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:12:27.874204 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3191/6446] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:12:27 2022] PATCH /identity/v3/domains/6c54abfb4d3e4de1bd05e0106ab6a180 => generated 112 bytes in 61 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 03 21:12:27.876000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64046fd0-b33c-48e7-8238-da4f5d7eede3 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:27.878284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-64046fd0-b33c-48e7-8238-da4f5d7eede3 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:28.028275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64046fd0-b33c-48e7-8238-da4f5d7eede3 tempest-TokensV3TestJSON-1236422260 tempest-TokensV3TestJSON-1236422260-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:28.045329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1788d8-179a-4f28-81f0-1522c6526a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:28.046007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1788d8-179a-4f28-81f0-1522c6526a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:28.046303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1788d8-179a-4f28-81f0-1522c6526a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:28.046574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1788d8-179a-4f28-81f0-1522c6526a8b admin admin] PATH_INFO: `/v3/users/7a2a473ffab94adbaf5b74a65bec60fc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:28.047960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1788d8-179a-4f28-81f0-1522c6526a8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a2a473ffab94adbaf5b74a65bec60fc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:28.050080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1788d8-179a-4f28-81f0-1522c6526a8b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:28.144825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c1788d8-179a-4f28-81f0-1522c6526a8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a2a473ffab94adbaf5b74a65bec60fc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:28.145828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b17fa-a64d-443e-8765-cf4718a23d06 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:28.146840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b17fa-a64d-443e-8765-cf4718a23d06 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:28.148681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b17fa-a64d-443e-8765-cf4718a23d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:28.162057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c1788d8-179a-4f28-81f0-1522c6526a8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a2a473ffab94adbaf5b74a65bec60fc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:28.172466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c1788d8-179a-4f28-81f0-1522c6526a8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a2a473ffab94adbaf5b74a65bec60fc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:28.173526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c1788d8-179a-4f28-81f0-1522c6526a8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a2a473ffab94adbaf5b74a65bec60fc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:28.174658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c1788d8-179a-4f28-81f0-1522c6526a8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a2a473ffab94adbaf5b74a65bec60fc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:28.195827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-336b17fa-a64d-443e-8765-cf4718a23d06 None None] MFA Rules not processed for user `c276bdddd15f4682a19ba89403a1aefc`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:28.198180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-336b17fa-a64d-443e-8765-cf4718a23d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:28.201258 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3256/6447] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:27 2022] DELETE /identity/v3/users/7a2a473ffab94adbaf5b74a65bec60fc => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:28.224170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e99d3-cc2d-487e-a9cb-2632554aee8d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:28.224515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e99d3-cc2d-487e-a9cb-2632554aee8d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:28.224757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e99d3-cc2d-487e-a9cb-2632554aee8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:28.273680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-432e99d3-cc2d-487e-a9cb-2632554aee8d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:28.274939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-432e99d3-cc2d-487e-a9cb-2632554aee8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:28.363828 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3192/6448] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:28.368120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b17fa-a64d-443e-8765-cf4718a23d06 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:28.369946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-336b17fa-a64d-443e-8765-cf4718a23d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:28.424091 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3257/6449] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:28.435566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e99d3-cc2d-487e-a9cb-2632554aee8d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:28.438479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-432e99d3-cc2d-487e-a9cb-2632554aee8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:28.460013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:28.498829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:28.568743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e99d3-cc2d-487e-a9cb-2632554aee8d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:28.573103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b17fa-a64d-443e-8765-cf4718a23d06 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:28.577886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c2c05-ba87-4664-9f61-947876671bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:28.580632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b31f4-4db3-4a3f-877c-255b568630c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:28.581087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c2c05-ba87-4664-9f61-947876671bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:28.581286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b31f4-4db3-4a3f-877c-255b568630c1 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:28.581663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b31f4-4db3-4a3f-877c-255b568630c1 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:28.581890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b31f4-4db3-4a3f-877c-255b568630c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:28.582379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c2c05-ba87-4664-9f61-947876671bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:28.582923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c2c05-ba87-4664-9f61-947876671bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:28.593707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b31f4-4db3-4a3f-877c-255b568630c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:28.598939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b31f4-4db3-4a3f-877c-255b568630c1 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:28.610427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c2c05-ba87-4664-9f61-947876671bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:28.612136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c2c05-ba87-4664-9f61-947876671bb4 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:28.627632 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3193/6450] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:28.645111 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3258/6451] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:12:28.697129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9d0b1-080e-41e8-93bb-54af675311fc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:28.697436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9d0b1-080e-41e8-93bb-54af675311fc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:28.697791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9d0b1-080e-41e8-93bb-54af675311fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:28.748950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a7c9d0b1-080e-41e8-93bb-54af675311fc None None] MFA Rules not processed for user `01c449f3fd1946508ab455af5b231c14`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:28.753166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a7c9d0b1-080e-41e8-93bb-54af675311fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:28.847995 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3194/6452] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:28.971071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c2c05-ba87-4664-9f61-947876671bb4 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:28.973536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-067c2c05-ba87-4664-9f61-947876671bb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:29.002064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c2c05-ba87-4664-9f61-947876671bb4 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:29.009999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:29.011409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:29.012545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:29.013609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] PATH_INFO: `/v3/users/d485394bc49e4380b152d32d09440281` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:29.016565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d485394bc49e4380b152d32d09440281)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:29.019382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:29.111740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd485394bc49e4380b152d32d09440281'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.136344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd485394bc49e4380b152d32d09440281'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.150040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd485394bc49e4380b152d32d09440281'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.150801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd485394bc49e4380b152d32d09440281'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.152955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd485394bc49e4380b152d32d09440281'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.177865 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3259/6453] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:28 2022] DELETE /identity/v3/users/d485394bc49e4380b152d32d09440281 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:29.426164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9d0b1-080e-41e8-93bb-54af675311fc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:29.428222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a7c9d0b1-080e-41e8-93bb-54af675311fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:29.463328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:29.465238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9d0b1-080e-41e8-93bb-54af675311fc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:29.465762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:29.551236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9624fb0-c030-4d25-8939-30cb8086ec2b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:29.555282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bac18-daed-40b3-a0da-549956863d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:29.555980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bac18-daed-40b3-a0da-549956863d23 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:29.556205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bac18-daed-40b3-a0da-549956863d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:29.556494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bac18-daed-40b3-a0da-549956863d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:29.558388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-498bac18-daed-40b3-a0da-549956863d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:29.571342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498bac18-daed-40b3-a0da-549956863d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:29.572926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498bac18-daed-40b3-a0da-549956863d23 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:29.585035 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3260/6454] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:12:29.585880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bac18-daed-40b3-a0da-549956863d23 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:29.587208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-498bac18-daed-40b3-a0da-549956863d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:29.593465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:29.594340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:29.594810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:29.595324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] PATH_INFO: `/v3/projects/2474af174b1f4087a11d696643570ef9` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:29.596563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2474af174b1f4087a11d696643570ef9)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:29.600301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:29.604425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bac18-daed-40b3-a0da-549956863d23 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:29.609916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:29.610661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:29.610960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:29.611246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b admin admin] PATH_INFO: `/v3/projects/8915b5da4f224da89e9699a7599350a1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:29.612427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8915b5da4f224da89e9699a7599350a1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:29.614674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:29.644656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:29.671696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2474af174b1f4087a11d696643570ef9', 'request_id': 'req-805e1140-363d-4889-b1a9-36ee2d78441f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.674117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8915b5da4f224da89e9699a7599350a1', 'request_id': 'req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.683271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2474af174b1f4087a11d696643570ef9', 'request_id': 'req-805e1140-363d-4889-b1a9-36ee2d78441f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.687492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8915b5da4f224da89e9699a7599350a1', 'request_id': 'req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.691487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2474af174b1f4087a11d696643570ef9', 'request_id': 'req-805e1140-363d-4889-b1a9-36ee2d78441f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.695366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8915b5da4f224da89e9699a7599350a1', 'request_id': 'req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.702870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:29.709744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] The token cache is being invalidate because project 2474af174b1f4087a11d696643570ef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:29.710253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.711552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.714013 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3195/6455] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:29 2022] DELETE /identity/v3/projects/2474af174b1f4087a11d696643570ef9 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:29.719981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:29.721850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:29.722484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b admin admin] The token cache is being invalidate because project 8915b5da4f224da89e9699a7599350a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:29.722866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.723708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b7f36380-c28d-4cb8-a6ae-5bd0dc226f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.725721 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3261/6456] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:29 2022] DELETE /identity/v3/projects/8915b5da4f224da89e9699a7599350a1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:29.730397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b7684-4e4c-4d9d-87d6-a3b9a9463509 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:29.730890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b7684-4e4c-4d9d-87d6-a3b9a9463509 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:29.731215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b7684-4e4c-4d9d-87d6-a3b9a9463509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:29.755357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-590b7684-4e4c-4d9d-87d6-a3b9a9463509 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:29.756674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-590b7684-4e4c-4d9d-87d6-a3b9a9463509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:29.800966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e1140-363d-4889-b1a9-36ee2d78441f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:29.833086 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3262/6457] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:29.840221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c547c93-9974-4a02-80f4-5f3192762cce None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:29.840655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c547c93-9974-4a02-80f4-5f3192762cce None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:29.840975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c547c93-9974-4a02-80f4-5f3192762cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:29.862992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9c547c93-9974-4a02-80f4-5f3192762cce None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:29.864231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9c547c93-9974-4a02-80f4-5f3192762cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:29.874135 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3263/6458] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:29.879779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c547c93-9974-4a02-80f4-5f3192762cce None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:29.885746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:29.886701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:29.887207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:29.887718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] PATH_INFO: `/v3/users/8b6096f19b1a4028aa81c2b52a30e67f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:29.889998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b6096f19b1a4028aa81c2b52a30e67f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:29.890502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c547c93-9974-4a02-80f4-5f3192762cce None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:29.892625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:29.895663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20b8b8-feba-4214-a326-e41c91aa438e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:29.896447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20b8b8-feba-4214-a326-e41c91aa438e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:29.896739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20b8b8-feba-4214-a326-e41c91aa438e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:29.897070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20b8b8-feba-4214-a326-e41c91aa438e admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:29.897734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20b8b8-feba-4214-a326-e41c91aa438e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:12:29.898046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20b8b8-feba-4214-a326-e41c91aa438e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:29.899768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20b8b8-feba-4214-a326-e41c91aa438e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:29.916441 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3264/6459] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:29.917483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20b8b8-feba-4214-a326-e41c91aa438e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:29.931130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20b8b8-feba-4214-a326-e41c91aa438e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:29.941042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1f785-ceb3-4e3f-b9b0-b3afd98754b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:29.941442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1f785-ceb3-4e3f-b9b0-b3afd98754b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:29.941733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1f785-ceb3-4e3f-b9b0-b3afd98754b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:29.942207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1f785-ceb3-4e3f-b9b0-b3afd98754b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:29.943110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1f785-ceb3-4e3f-b9b0-b3afd98754b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:29.945225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1f785-ceb3-4e3f-b9b0-b3afd98754b8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:29.950540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b6096f19b1a4028aa81c2b52a30e67f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.964713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b6096f19b1a4028aa81c2b52a30e67f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.965528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b6096f19b1a4028aa81c2b52a30e67f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.974289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b6096f19b1a4028aa81c2b52a30e67f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.980332 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3265/6460] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:29 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:29.983727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b6096f19b1a4028aa81c2b52a30e67f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:29.984478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1f785-ceb3-4e3f-b9b0-b3afd98754b8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:29.989545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d3a1f785-ceb3-4e3f-b9b0-b3afd98754b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:29.993101 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3196/6461] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:29 2022] DELETE /identity/v3/users/8b6096f19b1a4028aa81c2b52a30e67f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:30.078559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1f785-ceb3-4e3f-b9b0-b3afd98754b8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:30.098521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:30.165948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4ad12-eadd-4bee-824c-173159871d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:30.166582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4ad12-eadd-4bee-824c-173159871d36 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:30.166873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4ad12-eadd-4bee-824c-173159871d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:30.167172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4ad12-eadd-4bee-824c-173159871d36 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:30.167853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4ad12-eadd-4bee-824c-173159871d36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:30.170587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4ad12-eadd-4bee-824c-173159871d36 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:30.197798 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3266/6462] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:29 2022] POST /identity/v3/users => generated 292 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:30.203271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:30.220000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3f7e4-6d51-4d11-864c-87015f1da14e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:30.226027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6dfe5-3393-4551-a54c-2c208c61352d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:30.226857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6dfe5-3393-4551-a54c-2c208c61352d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:30.227681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6dfe5-3393-4551-a54c-2c208c61352d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:30.228215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6dfe5-3393-4551-a54c-2c208c61352d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:30.229075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6dfe5-3393-4551-a54c-2c208c61352d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:30.233210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6dfe5-3393-4551-a54c-2c208c61352d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:30.265295 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3197/6463] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:30.266548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4ad12-eadd-4bee-824c-173159871d36 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:30.278051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4ad12-eadd-4bee-824c-173159871d36 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:30.283198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9e633-0a6d-4734-a68d-43d65a1e6488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:30.283817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9e633-0a6d-4734-a68d-43d65a1e6488 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:30.284108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9e633-0a6d-4734-a68d-43d65a1e6488 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:30.284389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9e633-0a6d-4734-a68d-43d65a1e6488 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:30.285087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9e633-0a6d-4734-a68d-43d65a1e6488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:30.286734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9e633-0a6d-4734-a68d-43d65a1e6488 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:30.298831 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3267/6464] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:30.303127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6dfe5-3393-4551-a54c-2c208c61352d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:30.315968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6dfe5-3393-4551-a54c-2c208c61352d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:30.321258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135572d8-0705-4346-904e-4d38ce908607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:30.321987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135572d8-0705-4346-904e-4d38ce908607 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:30.322240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135572d8-0705-4346-904e-4d38ce908607 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:30.322521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135572d8-0705-4346-904e-4d38ce908607 admin admin] PATH_INFO: `/v3/projects/77941b15bab7435fa20336fa6dd26d58/users/be1f616143114ef2bc2036efb0d11eef/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:30.344379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:30.351053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135572d8-0705-4346-904e-4d38ce908607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77941b15bab7435fa20336fa6dd26d58, user_id=be1f616143114ef2bc2036efb0d11eef, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:30.354738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135572d8-0705-4346-904e-4d38ce908607 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:30.380548 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3198/6465] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:30 2022] PUT /identity/v3/projects/77941b15bab7435fa20336fa6dd26d58/users/be1f616143114ef2bc2036efb0d11eef/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:30.387043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba7bda-cdf6-42e6-a9fc-e8a833edbe7a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:30.387329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba7bda-cdf6-42e6-a9fc-e8a833edbe7a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:30.387611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba7bda-cdf6-42e6-a9fc-e8a833edbe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:30.412760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d5ba7bda-cdf6-42e6-a9fc-e8a833edbe7a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:30.415452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d5ba7bda-cdf6-42e6-a9fc-e8a833edbe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:30.431795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135572d8-0705-4346-904e-4d38ce908607 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:30.433966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-135572d8-0705-4346-904e-4d38ce908607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:30.535213 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3268/6466] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:30.546408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135572d8-0705-4346-904e-4d38ce908607 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:30.548238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba7bda-cdf6-42e6-a9fc-e8a833edbe7a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:30.549975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d5ba7bda-cdf6-42e6-a9fc-e8a833edbe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:30.551896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:30.552779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:30.553195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:30.553671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] PATH_INFO: `/v3/projects/715e5d7d48d84c4594e859a12b70290d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:30.554957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=715e5d7d48d84c4594e859a12b70290d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:30.556955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:30.624688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '715e5d7d48d84c4594e859a12b70290d', 'request_id': 'req-62f06b39-3fcf-439e-9485-fce74c6b3e15'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:30.635133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '715e5d7d48d84c4594e859a12b70290d', 'request_id': 'req-62f06b39-3fcf-439e-9485-fce74c6b3e15'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:30.636232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba7bda-cdf6-42e6-a9fc-e8a833edbe7a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:30.643754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '715e5d7d48d84c4594e859a12b70290d', 'request_id': 'req-62f06b39-3fcf-439e-9485-fce74c6b3e15'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:30.660827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] The token cache is being invalidate because project 715e5d7d48d84c4594e859a12b70290d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:30.661076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:30.661738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:30.663531 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3199/6467] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:30 2022] DELETE /identity/v3/projects/715e5d7d48d84c4594e859a12b70290d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:30.667191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:30.668582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:30.698440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3a3de-66aa-4dc3-a8fd-9bb4dd8991b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:30.700157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3a3de-66aa-4dc3-a8fd-9bb4dd8991b5 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:30.700758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3a3de-66aa-4dc3-a8fd-9bb4dd8991b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:30.701216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3a3de-66aa-4dc3-a8fd-9bb4dd8991b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:30.706483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5ca3a3de-66aa-4dc3-a8fd-9bb4dd8991b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:30.714147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:30.802629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f06b39-3fcf-439e-9485-fce74c6b3e15 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:30.810471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120190bb-e745-4afd-a617-52b1dd01f25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:30.811221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120190bb-e745-4afd-a617-52b1dd01f25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:30.811551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120190bb-e745-4afd-a617-52b1dd01f25b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:30.811842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120190bb-e745-4afd-a617-52b1dd01f25b admin admin] PATH_INFO: `/v3/users/7fb1781551ca40769e67f106be4523d7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:30.813201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120190bb-e745-4afd-a617-52b1dd01f25b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb1781551ca40769e67f106be4523d7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:30.815413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120190bb-e745-4afd-a617-52b1dd01f25b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:30.860866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:30.863227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3a3de-66aa-4dc3-a8fd-9bb4dd8991b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:30.864801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3a3de-66aa-4dc3-a8fd-9bb4dd8991b5 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:30.875700 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3269/6468] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:30.895496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-120190bb-e745-4afd-a617-52b1dd01f25b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb1781551ca40769e67f106be4523d7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:30.918392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-120190bb-e745-4afd-a617-52b1dd01f25b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb1781551ca40769e67f106be4523d7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:30.919457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-120190bb-e745-4afd-a617-52b1dd01f25b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb1781551ca40769e67f106be4523d7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:30.929652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-120190bb-e745-4afd-a617-52b1dd01f25b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb1781551ca40769e67f106be4523d7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:30.938898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-120190bb-e745-4afd-a617-52b1dd01f25b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb1781551ca40769e67f106be4523d7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:30.948245 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3200/6469] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:30 2022] DELETE /identity/v3/users/7fb1781551ca40769e67f106be4523d7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:31.436358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3a3de-66aa-4dc3-a8fd-9bb4dd8991b5 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:31.448636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3a3de-66aa-4dc3-a8fd-9bb4dd8991b5 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:31.564138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13523cc9-187b-4af8-9975-f7b99d3a8355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:31.564728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13523cc9-187b-4af8-9975-f7b99d3a8355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:31.565082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13523cc9-187b-4af8-9975-f7b99d3a8355 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:31.565370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13523cc9-187b-4af8-9975-f7b99d3a8355 admin admin] PATH_INFO: `/v3/projects/8ae5331900fd4b30969af9c156df5392` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:31.579492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13523cc9-187b-4af8-9975-f7b99d3a8355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae5331900fd4b30969af9c156df5392)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:31.579492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13523cc9-187b-4af8-9975-f7b99d3a8355 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:31.633130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-13523cc9-187b-4af8-9975-f7b99d3a8355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae5331900fd4b30969af9c156df5392', 'request_id': 'req-13523cc9-187b-4af8-9975-f7b99d3a8355'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:31.650839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-13523cc9-187b-4af8-9975-f7b99d3a8355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae5331900fd4b30969af9c156df5392', 'request_id': 'req-13523cc9-187b-4af8-9975-f7b99d3a8355'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:31.662531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-13523cc9-187b-4af8-9975-f7b99d3a8355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae5331900fd4b30969af9c156df5392', 'request_id': 'req-13523cc9-187b-4af8-9975-f7b99d3a8355'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:31.678316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-13523cc9-187b-4af8-9975-f7b99d3a8355 admin admin] The token cache is being invalidate because project 8ae5331900fd4b30969af9c156df5392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:31.678606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-13523cc9-187b-4af8-9975-f7b99d3a8355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:31.679769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-13523cc9-187b-4af8-9975-f7b99d3a8355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:31.681743 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3270/6470] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:31 2022] DELETE /identity/v3/projects/8ae5331900fd4b30969af9c156df5392 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:31.687005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a127844-7bae-4e05-86bf-a2bbea4f8ddb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:31.687551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a127844-7bae-4e05-86bf-a2bbea4f8ddb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:31.687551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a127844-7bae-4e05-86bf-a2bbea4f8ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:31.726007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6a127844-7bae-4e05-86bf-a2bbea4f8ddb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:31.727915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6a127844-7bae-4e05-86bf-a2bbea4f8ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:31.870326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1b176-f038-4c76-95e7-22c1f7649885 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:31.871122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1b176-f038-4c76-95e7-22c1f7649885 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:31.871734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1b176-f038-4c76-95e7-22c1f7649885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:31.882502 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3201/6471] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:31.902525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-95c1b176-f038-4c76-95e7-22c1f7649885 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:31.905126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-95c1b176-f038-4c76-95e7-22c1f7649885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:31.919309 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3271/6472] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:31.923954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a127844-7bae-4e05-86bf-a2bbea4f8ddb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:31.941073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a127844-7bae-4e05-86bf-a2bbea4f8ddb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:31.955559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81a9f-a47c-4b87-aa39-10836116dee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:31.956468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81a9f-a47c-4b87-aa39-10836116dee0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:31.956840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81a9f-a47c-4b87-aa39-10836116dee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:31.957402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81a9f-a47c-4b87-aa39-10836116dee0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:31.958507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa81a9f-a47c-4b87-aa39-10836116dee0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:12:31.958647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa81a9f-a47c-4b87-aa39-10836116dee0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:31.960550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa81a9f-a47c-4b87-aa39-10836116dee0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:31.976335 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3202/6473] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:31.980994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1b176-f038-4c76-95e7-22c1f7649885 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:31.995060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1b176-f038-4c76-95e7-22c1f7649885 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:32.005937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065e21d-d96c-47ba-ba91-e2c17e173a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:32.006586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065e21d-d96c-47ba-ba91-e2c17e173a97 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:32.006876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065e21d-d96c-47ba-ba91-e2c17e173a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:32.007156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065e21d-d96c-47ba-ba91-e2c17e173a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:32.007919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065e21d-d96c-47ba-ba91-e2c17e173a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:32.009941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065e21d-d96c-47ba-ba91-e2c17e173a97 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:32.061651 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3272/6474] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:31 2022] POST /identity/v3/projects => generated 369 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:32.066053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81a9f-a47c-4b87-aa39-10836116dee0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:32.086360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81a9f-a47c-4b87-aa39-10836116dee0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:32.192368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234d5d5-ae2c-4c27-96a0-e12b96cd7231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:32.193110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234d5d5-ae2c-4c27-96a0-e12b96cd7231 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:32.193405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234d5d5-ae2c-4c27-96a0-e12b96cd7231 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:32.194001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234d5d5-ae2c-4c27-96a0-e12b96cd7231 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:32.194329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234d5d5-ae2c-4c27-96a0-e12b96cd7231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:32.196461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234d5d5-ae2c-4c27-96a0-e12b96cd7231 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:32.235110 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3203/6475] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:32 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:32.236673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065e21d-d96c-47ba-ba91-e2c17e173a97 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:32.247354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:32.266956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065e21d-d96c-47ba-ba91-e2c17e173a97 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:32.293755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7846190-b035-44f0-956a-8fe9c5592d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:32.294800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7846190-b035-44f0-956a-8fe9c5592d2e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:32.295290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7846190-b035-44f0-956a-8fe9c5592d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:32.295749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7846190-b035-44f0-956a-8fe9c5592d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:32.296576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7846190-b035-44f0-956a-8fe9c5592d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:32.298422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7846190-b035-44f0-956a-8fe9c5592d2e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:32.328748 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3273/6476] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:32.332800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234d5d5-ae2c-4c27-96a0-e12b96cd7231 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:32.346855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234d5d5-ae2c-4c27-96a0-e12b96cd7231 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:32.362101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8656b-4e82-4f63-a323-9cbf684f0dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:32.362839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8656b-4e82-4f63-a323-9cbf684f0dcc admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:32.363141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8656b-4e82-4f63-a323-9cbf684f0dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:32.363474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8656b-4e82-4f63-a323-9cbf684f0dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:32.364140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8656b-4e82-4f63-a323-9cbf684f0dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:32.369531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8656b-4e82-4f63-a323-9cbf684f0dcc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:32.393288 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3204/6477] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:32.397562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7846190-b035-44f0-956a-8fe9c5592d2e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:32.412430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7846190-b035-44f0-956a-8fe9c5592d2e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:32.425054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacdd94-540f-49f9-9365-34f64496ab7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:32.425754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacdd94-540f-49f9-9365-34f64496ab7a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:32.426046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacdd94-540f-49f9-9365-34f64496ab7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:32.426334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacdd94-540f-49f9-9365-34f64496ab7a admin admin] PATH_INFO: `/v3/projects/600b850d77d0450e90e1e4d4f555ddc8/users/c3f88cc4a3a24e9cb2d78e01df17e141/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:32.458763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacdd94-540f-49f9-9365-34f64496ab7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=600b850d77d0450e90e1e4d4f555ddc8, user_id=c3f88cc4a3a24e9cb2d78e01df17e141, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:32.461522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacdd94-540f-49f9-9365-34f64496ab7a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:32.491148 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3274/6478] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:32 2022] PUT /identity/v3/projects/600b850d77d0450e90e1e4d4f555ddc8/users/c3f88cc4a3a24e9cb2d78e01df17e141/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:32.503040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35db6f-440f-4821-a6d8-938dd6ef5142 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:32.503407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35db6f-440f-4821-a6d8-938dd6ef5142 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:32.503603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35db6f-440f-4821-a6d8-938dd6ef5142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:32.526691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:32.563881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-dd35db6f-440f-4821-a6d8-938dd6ef5142 None None] MFA Rules not processed for user `c3f88cc4a3a24e9cb2d78e01df17e141`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:32.567169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dd35db6f-440f-4821-a6d8-938dd6ef5142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:32.783654 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3205/6479] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:32.787858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacdd94-540f-49f9-9365-34f64496ab7a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:32.919125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacdd94-540f-49f9-9365-34f64496ab7a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:32.930904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614f6e3-49c6-4d6c-8346-8d40bd789650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:32.932044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f6e3-49c6-4d6c-8346-8d40bd789650 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:32.932804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f6e3-49c6-4d6c-8346-8d40bd789650 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:32.933138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f6e3-49c6-4d6c-8346-8d40bd789650 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:32.935418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614f6e3-49c6-4d6c-8346-8d40bd789650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:32.941791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614f6e3-49c6-4d6c-8346-8d40bd789650 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:33.026157 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3275/6480] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:32 2022] POST /identity/v3/projects => generated 371 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:33.027357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35db6f-440f-4821-a6d8-938dd6ef5142 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:33.043327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35db6f-440f-4821-a6d8-938dd6ef5142 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:33.245196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f4da6-60be-45c4-ae05-1cf37016d992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:33.245995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f4da6-60be-45c4-ae05-1cf37016d992 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:33.246393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f4da6-60be-45c4-ae05-1cf37016d992 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:33.246788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f4da6-60be-45c4-ae05-1cf37016d992 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:33.247546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f4da6-60be-45c4-ae05-1cf37016d992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:33.254136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f4da6-60be-45c4-ae05-1cf37016d992 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:33.315594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614f6e3-49c6-4d6c-8346-8d40bd789650 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:33.325034 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3206/6481] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:33 2022] POST /identity/v3/users => generated 292 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:33.336330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614f6e3-49c6-4d6c-8346-8d40bd789650 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:33.358401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310eb45-9e50-42f7-ad40-40cdff4303da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:33.359049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310eb45-9e50-42f7-ad40-40cdff4303da admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:33.359360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310eb45-9e50-42f7-ad40-40cdff4303da admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:33.359644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310eb45-9e50-42f7-ad40-40cdff4303da admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:33.360310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310eb45-9e50-42f7-ad40-40cdff4303da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:33.374367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310eb45-9e50-42f7-ad40-40cdff4303da admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:33.394911 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3276/6482] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:33.396322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f4da6-60be-45c4-ae05-1cf37016d992 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:33.413477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f4da6-60be-45c4-ae05-1cf37016d992 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:33.425782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4d0dd-d49f-4e56-89af-df4c31a985f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:33.426491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4d0dd-d49f-4e56-89af-df4c31a985f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:33.426788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4d0dd-d49f-4e56-89af-df4c31a985f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:33.427079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4d0dd-d49f-4e56-89af-df4c31a985f8 admin admin] PATH_INFO: `/v3/projects/1a02d37ae1784079a0b8c9c0d3683767/users/e37ab884e357416e9d11cd36c5307ffe/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:33.464206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4d0dd-d49f-4e56-89af-df4c31a985f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a02d37ae1784079a0b8c9c0d3683767, user_id=e37ab884e357416e9d11cd36c5307ffe, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:33.467441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4d0dd-d49f-4e56-89af-df4c31a985f8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:33.503642 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3207/6483] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:33 2022] PUT /identity/v3/projects/1a02d37ae1784079a0b8c9c0d3683767/users/e37ab884e357416e9d11cd36c5307ffe/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:33.504709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310eb45-9e50-42f7-ad40-40cdff4303da admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:33.516863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310eb45-9e50-42f7-ad40-40cdff4303da admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:33.671439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86429c13-65c5-4b90-a2dd-c644d44ec801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:33.672160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86429c13-65c5-4b90-a2dd-c644d44ec801 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:33.672456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86429c13-65c5-4b90-a2dd-c644d44ec801 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:33.672732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86429c13-65c5-4b90-a2dd-c644d44ec801 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:33.674245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86429c13-65c5-4b90-a2dd-c644d44ec801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:33.677205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86429c13-65c5-4b90-a2dd-c644d44ec801 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:33.700698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4d0dd-d49f-4e56-89af-df4c31a985f8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:33.708870 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3277/6484] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:33.718962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4d0dd-d49f-4e56-89af-df4c31a985f8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:33.726790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad00957-bb39-463a-985e-a098cc237207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:33.728078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad00957-bb39-463a-985e-a098cc237207 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:33.728546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad00957-bb39-463a-985e-a098cc237207 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:33.729033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad00957-bb39-463a-985e-a098cc237207 admin admin] PATH_INFO: `/v3/projects/1a02d37ae1784079a0b8c9c0d3683767/users/e37ab884e357416e9d11cd36c5307ffe/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:33.731421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad00957-bb39-463a-985e-a098cc237207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a02d37ae1784079a0b8c9c0d3683767, user_id=e37ab884e357416e9d11cd36c5307ffe, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:33.735214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad00957-bb39-463a-985e-a098cc237207 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:33.769556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad081ce-82e0-45ae-bf44-e80b0d2186dd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:33.769856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad081ce-82e0-45ae-bf44-e80b0d2186dd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:33.770148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad081ce-82e0-45ae-bf44-e80b0d2186dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:33.774612 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3208/6485] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:33 2022] PUT /identity/v3/projects/1a02d37ae1784079a0b8c9c0d3683767/users/e37ab884e357416e9d11cd36c5307ffe/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:33.817633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cad081ce-82e0-45ae-bf44-e80b0d2186dd None None] MFA Rules not processed for user `e37ab884e357416e9d11cd36c5307ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:33.819507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cad081ce-82e0-45ae-bf44-e80b0d2186dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:33.858860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:33.919033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad00957-bb39-463a-985e-a098cc237207 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:33.919639 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3278/6486] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:33.932306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad00957-bb39-463a-985e-a098cc237207 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:33.941589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c92e8-d01b-46ec-8191-a9c0d3e6e8d0 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e37ab884e357416e9d11cd36c5307ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a02d37ae1784079a0b8c9c0d3683767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:33.942479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c92e8-d01b-46ec-8191-a9c0d3e6e8d0 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:33.942990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c92e8-d01b-46ec-8191-a9c0d3e6e8d0 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:33.943445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c92e8-d01b-46ec-8191-a9c0d3e6e8d0 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:33.944279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c92e8-d01b-46ec-8191-a9c0d3e6e8d0 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:33.946339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c92e8-d01b-46ec-8191-a9c0d3e6e8d0 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:33.985426 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3209/6487] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:33 2022] POST /identity/v3/projects => generated 358 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:33.987818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad081ce-82e0-45ae-bf44-e80b0d2186dd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:34.023514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad081ce-82e0-45ae-bf44-e80b0d2186dd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:34.127971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dacbb0-dfe3-4a78-8508-bad375f631a8 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e37ab884e357416e9d11cd36c5307ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a02d37ae1784079a0b8c9c0d3683767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:34.139427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dacbb0-dfe3-4a78-8508-bad375f631a8 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:34.139727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dacbb0-dfe3-4a78-8508-bad375f631a8 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:34.140024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dacbb0-dfe3-4a78-8508-bad375f631a8 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:34.140821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dacbb0-dfe3-4a78-8508-bad375f631a8 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:34.143644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dacbb0-dfe3-4a78-8508-bad375f631a8 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:34.170988 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3279/6488] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:33 2022] POST /identity/v3/users => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:34.174969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c92e8-d01b-46ec-8191-a9c0d3e6e8d0 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:34.208866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c92e8-d01b-46ec-8191-a9c0d3e6e8d0 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:34.220229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7cef8-8621-4f3b-a707-846b4bfd2c2c tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e37ab884e357416e9d11cd36c5307ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a02d37ae1784079a0b8c9c0d3683767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:34.240671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7cef8-8621-4f3b-a707-846b4bfd2c2c tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:34.241049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7cef8-8621-4f3b-a707-846b4bfd2c2c tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:34.242262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7cef8-8621-4f3b-a707-846b4bfd2c2c tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:34.243245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7cef8-8621-4f3b-a707-846b4bfd2c2c tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:12:34.243591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7cef8-8621-4f3b-a707-846b4bfd2c2c tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:34.245917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7cef8-8621-4f3b-a707-846b4bfd2c2c tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:34.286573 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3210/6489] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Sep 3 21:12:34 2022] GET /identity/v3/users?domain_id=default => generated 7731 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:34.288018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dacbb0-dfe3-4a78-8508-bad375f631a8 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:34.299347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dacbb0-dfe3-4a78-8508-bad375f631a8 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:34.307472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e37ab884e357416e9d11cd36c5307ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a02d37ae1784079a0b8c9c0d3683767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:34.308330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:34.309173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:34.309173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] PATH_INFO: `/v3/users/a9fbbd4bf1df457588aaf508af2eecbe` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:34.333671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorizing `identity:update_user(user_id=a9fbbd4bf1df457588aaf508af2eecbe)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:34.334892 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:12:34.334892 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:12:34.334892 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:12:34.336138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:34.395680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'a9fbbd4bf1df457588aaf508af2eecbe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:34.417664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invalidating the token cache because user a9fbbd4bf1df457588aaf508af2eecbe was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:34.418376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:34.419219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:34.424567 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3280/6490] 10.222.0.16 () {60 vars in 1244 bytes} [Sat Sep 3 21:12:34 2022] PATCH /identity/v3/users/a9fbbd4bf1df457588aaf508af2eecbe => generated 485 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:34.430212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee6985-12ee-4cb7-a664-a3c993ff3658 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:34.430849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee6985-12ee-4cb7-a664-a3c993ff3658 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:34.431708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee6985-12ee-4cb7-a664-a3c993ff3658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:34.454293 ncdv-master-1 devstack@keystone.service[99211]: WARNING keystone.auth.plugins.core [None req-2fee6985-12ee-4cb7-a664-a3c993ff3658 None None] User is disabled: a9fbbd4bf1df457588aaf508af2eecbe: AssertionError: User is disabled: a9fbbd4bf1df457588aaf508af2eecbe Sep 03 21:12:34.468282 ncdv-master-1 devstack@keystone.service[99211]: WARNING keystone.server.flask.application [None req-2fee6985-12ee-4cb7-a664-a3c993ff3658 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.16: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 03 21:12:34.470530 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3211/6491] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:34 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 42 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 03 21:12:34.474286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:34.476116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:34.496620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be84e3-a063-47f4-b065-853f2330ad1e tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:34.503446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e37ab884e357416e9d11cd36c5307ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a02d37ae1784079a0b8c9c0d3683767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:34.504120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:34.504426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:34.504696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] PATH_INFO: `/v3/users/a9fbbd4bf1df457588aaf508af2eecbe` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:34.521337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorizing `identity:delete_user(user_id=a9fbbd4bf1df457588aaf508af2eecbe)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:34.523683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:34.578388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9fbbd4bf1df457588aaf508af2eecbe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:34.587653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:UPDATE=1,keystone:DELETE=9 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:34.599946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9fbbd4bf1df457588aaf508af2eecbe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:34.618038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9fbbd4bf1df457588aaf508af2eecbe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:34.618761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9fbbd4bf1df457588aaf508af2eecbe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:34.619453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9fbbd4bf1df457588aaf508af2eecbe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:34.639908 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3281/6492] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:34 2022] DELETE /identity/v3/users/a9fbbd4bf1df457588aaf508af2eecbe => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:34.643749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee6985-12ee-4cb7-a664-a3c993ff3658 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:34.646082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2fee6985-12ee-4cb7-a664-a3c993ff3658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:34.659660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:34.815014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee6985-12ee-4cb7-a664-a3c993ff3658 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:34.823056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e37ab884e357416e9d11cd36c5307ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a02d37ae1784079a0b8c9c0d3683767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:34.823983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:34.824455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:34.824939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] PATH_INFO: `/v3/projects/7842f57764c14246a4d48b2423a7702c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:34.827052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7842f57764c14246a4d48b2423a7702c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:34.830714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:35.018791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7842f57764c14246a4d48b2423a7702c', 'request_id': 'req-64d9efc0-aa43-4b04-8fbe-d9595475a52f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:35.036865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7842f57764c14246a4d48b2423a7702c', 'request_id': 'req-64d9efc0-aa43-4b04-8fbe-d9595475a52f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:35.054442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7842f57764c14246a4d48b2423a7702c', 'request_id': 'req-64d9efc0-aa43-4b04-8fbe-d9595475a52f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:35.083395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] The token cache is being invalidate because project 7842f57764c14246a4d48b2423a7702c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:35.084644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:35.087309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:35.093281 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3212/6493] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:34 2022] DELETE /identity/v3/projects/7842f57764c14246a4d48b2423a7702c => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:35.097562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:35.110378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad5828-a887-470b-bb99-19c0308ab306 tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:35.240221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b216ca7-50c3-4cd0-aa90-bdae349f3e1a tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e37ab884e357416e9d11cd36c5307ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a02d37ae1784079a0b8c9c0d3683767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:35.242061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b216ca7-50c3-4cd0-aa90-bdae349f3e1a tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:35.242954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b216ca7-50c3-4cd0-aa90-bdae349f3e1a tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:35.243605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b216ca7-50c3-4cd0-aa90-bdae349f3e1a tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:35.244544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b216ca7-50c3-4cd0-aa90-bdae349f3e1a tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:35.247619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b216ca7-50c3-4cd0-aa90-bdae349f3e1a tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-2b216ca7-50c3-4cd0-aa90-bdae349f3e1a tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Could not find domain: d3f1ae5c99af40fc814e9b73fee062c4.: keystone.exception.DomainNotFound: Could not find domain: d3f1ae5c99af40fc814e9b73fee062c4. Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d3f1ae5c99af40fc814e9b73fee062c4. Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:12:35.300244 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 21:12:35.301472 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 987, in create_user Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_domain(domain_id) Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:12:35.302694 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:12:35.303902 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:12:35.303902 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:12:35.303902 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: d3f1ae5c99af40fc814e9b73fee062c4. Sep 03 21:12:35.303902 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:12:35.306328 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3282/6494] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:35 2022] POST /identity/v3/users => generated 112 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:12:35.311149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:35.312616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:35.483897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9efc0-aa43-4b04-8fbe-d9595475a52f tempest-UsersNegativeTest-1775539970 tempest-UsersNegativeTest-1775539970-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:35.490859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:35.492268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:35.492967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:35.493447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] PATH_INFO: `/v3/users/c3f88cc4a3a24e9cb2d78e01df17e141` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:35.495122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f88cc4a3a24e9cb2d78e01df17e141)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:35.497940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:35.586268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:35.593688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f88cc4a3a24e9cb2d78e01df17e141'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:35.610424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f88cc4a3a24e9cb2d78e01df17e141'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:35.611323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f88cc4a3a24e9cb2d78e01df17e141'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:35.627589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f88cc4a3a24e9cb2d78e01df17e141'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:35.638169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f88cc4a3a24e9cb2d78e01df17e141'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:35.645855 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3213/6495] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:35 2022] DELETE /identity/v3/users/c3f88cc4a3a24e9cb2d78e01df17e141 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:35.650319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13374d-298f-4ea4-bdd7-9a2b4f591329 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:35.650613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13374d-298f-4ea4-bdd7-9a2b4f591329 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:35.650943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13374d-298f-4ea4-bdd7-9a2b4f591329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:35.671764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cd13374d-298f-4ea4-bdd7-9a2b4f591329 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:35.673065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cd13374d-298f-4ea4-bdd7-9a2b4f591329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:35.754207 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3283/6496] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:35.767224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:35.768732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:35.847401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c62e6-8bdc-4fcd-a5dc-fe6ef4a6522b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:35.851775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2144a-1bc7-4a0e-977c-76126edc7ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:35.853091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2144a-1bc7-4a0e-977c-76126edc7ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:35.853639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2144a-1bc7-4a0e-977c-76126edc7ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:35.854144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2144a-1bc7-4a0e-977c-76126edc7ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:35.865740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2144a-1bc7-4a0e-977c-76126edc7ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:35.867575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2144a-1bc7-4a0e-977c-76126edc7ab0 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:35.880341 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3214/6497] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:36.388347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13374d-298f-4ea4-bdd7-9a2b4f591329 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:36.390657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cd13374d-298f-4ea4-bdd7-9a2b4f591329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:36.408007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13374d-298f-4ea4-bdd7-9a2b4f591329 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:36.414745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:36.415674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:36.416166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:36.416761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] PATH_INFO: `/v3/projects/600b850d77d0450e90e1e4d4f555ddc8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:36.418195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] RBAC: Authorizing `identity:delete_project(project_id=600b850d77d0450e90e1e4d4f555ddc8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:36.421086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:36.519836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '600b850d77d0450e90e1e4d4f555ddc8', 'request_id': 'req-30d5a0f2-4c4b-43ca-882f-afcef377625e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:36.547066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '600b850d77d0450e90e1e4d4f555ddc8', 'request_id': 'req-30d5a0f2-4c4b-43ca-882f-afcef377625e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:36.558113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '600b850d77d0450e90e1e4d4f555ddc8', 'request_id': 'req-30d5a0f2-4c4b-43ca-882f-afcef377625e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:36.567306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:36.578613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] The token cache is being invalidate because project 600b850d77d0450e90e1e4d4f555ddc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:36.579070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:36.580889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:36.583544 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3284/6498] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:36 2022] DELETE /identity/v3/projects/600b850d77d0450e90e1e4d4f555ddc8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:36.588635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2144a-1bc7-4a0e-977c-76126edc7ab0 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:36.603621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2144a-1bc7-4a0e-977c-76126edc7ab0 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:36.699560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff1ecd-289e-4b7d-8b58-650b9aa523f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:36.700610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff1ecd-289e-4b7d-8b58-650b9aa523f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:36.701236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff1ecd-289e-4b7d-8b58-650b9aa523f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:36.702132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff1ecd-289e-4b7d-8b58-650b9aa523f1 admin admin] PATH_INFO: `/v3/users/e37ab884e357416e9d11cd36c5307ffe` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:36.704606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff1ecd-289e-4b7d-8b58-650b9aa523f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e37ab884e357416e9d11cd36c5307ffe)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:36.707389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff1ecd-289e-4b7d-8b58-650b9aa523f1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:36.815973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fcff1ecd-289e-4b7d-8b58-650b9aa523f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e37ab884e357416e9d11cd36c5307ffe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:36.835627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:36.838116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fcff1ecd-289e-4b7d-8b58-650b9aa523f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e37ab884e357416e9d11cd36c5307ffe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:36.839281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fcff1ecd-289e-4b7d-8b58-650b9aa523f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e37ab884e357416e9d11cd36c5307ffe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:36.855589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fcff1ecd-289e-4b7d-8b58-650b9aa523f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e37ab884e357416e9d11cd36c5307ffe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:36.865201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fcff1ecd-289e-4b7d-8b58-650b9aa523f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e37ab884e357416e9d11cd36c5307ffe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:36.874856 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3215/6499] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:36 2022] DELETE /identity/v3/users/e37ab884e357416e9d11cd36c5307ffe => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:37.447698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:37.449415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:37.480793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5a0f2-4c4b-43ca-882f-afcef377625e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:37.491246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37d597-763e-4b6b-8983-77744114830e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:37.491653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37d597-763e-4b6b-8983-77744114830e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:37.491819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37d597-763e-4b6b-8983-77744114830e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:37.530349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1e37d597-763e-4b6b-8983-77744114830e None None] MFA Rules not processed for user `be1f616143114ef2bc2036efb0d11eef`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:37.537217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1e37d597-763e-4b6b-8983-77744114830e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:37.574323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803d807-9159-423c-a4b5-e22d166cd86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:37.574907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803d807-9159-423c-a4b5-e22d166cd86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:37.575200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803d807-9159-423c-a4b5-e22d166cd86b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:37.575480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803d807-9159-423c-a4b5-e22d166cd86b admin admin] PATH_INFO: `/v3/projects/1a02d37ae1784079a0b8c9c0d3683767` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:37.576675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803d807-9159-423c-a4b5-e22d166cd86b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a02d37ae1784079a0b8c9c0d3683767)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:37.578664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803d807-9159-423c-a4b5-e22d166cd86b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:37.678428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d803d807-9159-423c-a4b5-e22d166cd86b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a02d37ae1784079a0b8c9c0d3683767', 'request_id': 'req-d803d807-9159-423c-a4b5-e22d166cd86b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:37.681583 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3216/6500] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:37.692131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411bad1-f324-4f3a-bdfd-3489daaf9b2d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:37.692463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411bad1-f324-4f3a-bdfd-3489daaf9b2d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:37.692783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411bad1-f324-4f3a-bdfd-3489daaf9b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:37.694889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d803d807-9159-423c-a4b5-e22d166cd86b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a02d37ae1784079a0b8c9c0d3683767', 'request_id': 'req-d803d807-9159-423c-a4b5-e22d166cd86b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:37.713422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d803d807-9159-423c-a4b5-e22d166cd86b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a02d37ae1784079a0b8c9c0d3683767', 'request_id': 'req-d803d807-9159-423c-a4b5-e22d166cd86b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:37.723239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5411bad1-f324-4f3a-bdfd-3489daaf9b2d None None] MFA Rules not processed for user `be1f616143114ef2bc2036efb0d11eef`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:37.724486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5411bad1-f324-4f3a-bdfd-3489daaf9b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:37.729280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d803d807-9159-423c-a4b5-e22d166cd86b admin admin] The token cache is being invalidate because project 1a02d37ae1784079a0b8c9c0d3683767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:37.729620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d803d807-9159-423c-a4b5-e22d166cd86b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:37.730351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d803d807-9159-423c-a4b5-e22d166cd86b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:37.732207 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3285/6501] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:37 2022] DELETE /identity/v3/projects/1a02d37ae1784079a0b8c9c0d3683767 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:37.740386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b82ba-0f93-4daa-9a81-d56fac0d6155 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:37.740656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b82ba-0f93-4daa-9a81-d56fac0d6155 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:37.741137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b82ba-0f93-4daa-9a81-d56fac0d6155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:37.767220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-876b82ba-0f93-4daa-9a81-d56fac0d6155 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:37.768601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-876b82ba-0f93-4daa-9a81-d56fac0d6155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:37.798156 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3217/6502] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:37.805176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411bad1-f324-4f3a-bdfd-3489daaf9b2d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:37.807800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5411bad1-f324-4f3a-bdfd-3489daaf9b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:37.845043 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3286/6503] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:37.853191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f322d2a-55db-4037-9fa2-c47577d61c5a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:37.853491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f322d2a-55db-4037-9fa2-c47577d61c5a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:37.854201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f322d2a-55db-4037-9fa2-c47577d61c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:37.875910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2f322d2a-55db-4037-9fa2-c47577d61c5a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:37.877137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2f322d2a-55db-4037-9fa2-c47577d61c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:37.887057 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3287/6504] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:37.891158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f322d2a-55db-4037-9fa2-c47577d61c5a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:37.901834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f322d2a-55db-4037-9fa2-c47577d61c5a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:37.909976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961bea0-7eba-40af-a698-dd8891573919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:37.910573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961bea0-7eba-40af-a698-dd8891573919 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:37.911021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961bea0-7eba-40af-a698-dd8891573919 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:37.911341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961bea0-7eba-40af-a698-dd8891573919 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:37.912172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961bea0-7eba-40af-a698-dd8891573919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:12:37.912514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961bea0-7eba-40af-a698-dd8891573919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:37.913177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411bad1-f324-4f3a-bdfd-3489daaf9b2d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:37.914491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961bea0-7eba-40af-a698-dd8891573919 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:37.919488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5584093-571d-4037-ab50-48ccd76fa61a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:37.920797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5584093-571d-4037-ab50-48ccd76fa61a service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:37.921144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5584093-571d-4037-ab50-48ccd76fa61a service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:37.921430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5584093-571d-4037-ab50-48ccd76fa61a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:37.923437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e5584093-571d-4037-ab50-48ccd76fa61a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:37.933477 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3288/6505] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:12:37.936485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5584093-571d-4037-ab50-48ccd76fa61a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:37.938174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5584093-571d-4037-ab50-48ccd76fa61a service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:37.938928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961bea0-7eba-40af-a698-dd8891573919 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:37.944263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:37.950988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961bea0-7eba-40af-a698-dd8891573919 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:37.954427 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3218/6506] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:12:37.956736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4646783-c527-4abd-84e9-f13fb7b7cb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:37.957396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4646783-c527-4abd-84e9-f13fb7b7cb6b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:37.957679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4646783-c527-4abd-84e9-f13fb7b7cb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:37.958182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4646783-c527-4abd-84e9-f13fb7b7cb6b admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:37.958822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4646783-c527-4abd-84e9-f13fb7b7cb6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:37.960798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4646783-c527-4abd-84e9-f13fb7b7cb6b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:37.967325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:38.004749 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3289/6507] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:37 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:38.005234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5584093-571d-4037-ab50-48ccd76fa61a service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:38.021487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5584093-571d-4037-ab50-48ccd76fa61a service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:38.163810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e866f-2805-428c-a05b-ed0070d073cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:38.165061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e866f-2805-428c-a05b-ed0070d073cd admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:38.165378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e866f-2805-428c-a05b-ed0070d073cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:38.167887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e866f-2805-428c-a05b-ed0070d073cd admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:38.168573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e866f-2805-428c-a05b-ed0070d073cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:38.173842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e866f-2805-428c-a05b-ed0070d073cd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:38.216606 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3219/6508] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:38 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:38.222176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4646783-c527-4abd-84e9-f13fb7b7cb6b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:38.254149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4646783-c527-4abd-84e9-f13fb7b7cb6b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:38.260380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373ba97-2faf-4349-a05b-dae0fa87beda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:38.261650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373ba97-2faf-4349-a05b-dae0fa87beda admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:38.262004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373ba97-2faf-4349-a05b-dae0fa87beda admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:38.262287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373ba97-2faf-4349-a05b-dae0fa87beda admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:38.262993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373ba97-2faf-4349-a05b-dae0fa87beda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:38.265491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373ba97-2faf-4349-a05b-dae0fa87beda admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:38.285123 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3290/6509] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:38.286130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e866f-2805-428c-a05b-ed0070d073cd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:38.301508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e866f-2805-428c-a05b-ed0070d073cd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:38.307357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58b0ad-04f5-4ed5-a5b0-cfb3d4f1bc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:38.308135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58b0ad-04f5-4ed5-a5b0-cfb3d4f1bc9e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:38.308640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58b0ad-04f5-4ed5-a5b0-cfb3d4f1bc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:38.309077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58b0ad-04f5-4ed5-a5b0-cfb3d4f1bc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:38.309842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58b0ad-04f5-4ed5-a5b0-cfb3d4f1bc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:38.311612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58b0ad-04f5-4ed5-a5b0-cfb3d4f1bc9e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:38.328129 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3220/6510] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:38.330280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373ba97-2faf-4349-a05b-dae0fa87beda admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:38.344424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373ba97-2faf-4349-a05b-dae0fa87beda admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:38.356418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef0c72-af96-412c-aa0c-1bfece70c88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:38.358130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef0c72-af96-412c-aa0c-1bfece70c88f admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:38.358378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef0c72-af96-412c-aa0c-1bfece70c88f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:38.358729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef0c72-af96-412c-aa0c-1bfece70c88f admin admin] PATH_INFO: `/v3/projects/58b6410bf7614ef8ad8dad70ddc3a049/users/cfec7ee1abdb4892b5cd1995599d7357/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:38.381091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef0c72-af96-412c-aa0c-1bfece70c88f admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b6410bf7614ef8ad8dad70ddc3a049, user_id=cfec7ee1abdb4892b5cd1995599d7357, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:38.384482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef0c72-af96-412c-aa0c-1bfece70c88f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:38.400946 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3291/6511] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:38 2022] PUT /identity/v3/projects/58b6410bf7614ef8ad8dad70ddc3a049/users/cfec7ee1abdb4892b5cd1995599d7357/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:38.424888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c4e70-dd28-4841-b07a-1caf1aa0c62d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:38.425581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c4e70-dd28-4841-b07a-1caf1aa0c62d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:38.428284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c4e70-dd28-4841-b07a-1caf1aa0c62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:38.466746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e07c4e70-dd28-4841-b07a-1caf1aa0c62d None None] MFA Rules not processed for user `cfec7ee1abdb4892b5cd1995599d7357`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:38.468379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e07c4e70-dd28-4841-b07a-1caf1aa0c62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:38.563751 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3221/6512] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:38.568407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef0c72-af96-412c-aa0c-1bfece70c88f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:38.676845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef0c72-af96-412c-aa0c-1bfece70c88f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:38.686766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4371ab5-7b00-442b-8dc6-68970cdfc446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:38.687934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4371ab5-7b00-442b-8dc6-68970cdfc446 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:38.688239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4371ab5-7b00-442b-8dc6-68970cdfc446 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:38.688527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4371ab5-7b00-442b-8dc6-68970cdfc446 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:38.689755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4371ab5-7b00-442b-8dc6-68970cdfc446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:38.692072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4371ab5-7b00-442b-8dc6-68970cdfc446 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:38.750257 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3292/6513] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:38 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:38.751728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c4e70-dd28-4841-b07a-1caf1aa0c62d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:38.775614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c4e70-dd28-4841-b07a-1caf1aa0c62d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:38.924854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f43366-649f-474c-b532-bdaf0295146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:38.925451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f43366-649f-474c-b532-bdaf0295146c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:38.925739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f43366-649f-474c-b532-bdaf0295146c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:38.926023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f43366-649f-474c-b532-bdaf0295146c admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:38.926667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f43366-649f-474c-b532-bdaf0295146c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:38.928796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f43366-649f-474c-b532-bdaf0295146c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:38.955118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:38.976354 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3222/6514] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:38 2022] POST /identity/v3/users => generated 298 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:38.977552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4371ab5-7b00-442b-8dc6-68970cdfc446 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:38.995887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4371ab5-7b00-442b-8dc6-68970cdfc446 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:39.012132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884f757-4840-4ada-a6e8-a39f853713a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:39.012778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884f757-4840-4ada-a6e8-a39f853713a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:39.013345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884f757-4840-4ada-a6e8-a39f853713a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:39.013750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884f757-4840-4ada-a6e8-a39f853713a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:39.014528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884f757-4840-4ada-a6e8-a39f853713a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:39.016282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884f757-4840-4ada-a6e8-a39f853713a4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:39.046820 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3293/6515] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:39.055979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f43366-649f-474c-b532-bdaf0295146c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:39.076093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f43366-649f-474c-b532-bdaf0295146c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:39.090775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bf48e-192d-4f11-a0b9-810999e85135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:39.091495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bf48e-192d-4f11-a0b9-810999e85135 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:39.092320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bf48e-192d-4f11-a0b9-810999e85135 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:39.092605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bf48e-192d-4f11-a0b9-810999e85135 admin admin] PATH_INFO: `/v3/projects/82291c52950d4229b2adb2b92985396e/users/07d5fba4627346a48e2acb49ed0e3049/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:39.133957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bf48e-192d-4f11-a0b9-810999e85135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82291c52950d4229b2adb2b92985396e, user_id=07d5fba4627346a48e2acb49ed0e3049, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:39.141436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bf48e-192d-4f11-a0b9-810999e85135 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:39.175758 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3223/6516] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:39 2022] PUT /identity/v3/projects/82291c52950d4229b2adb2b92985396e/users/07d5fba4627346a48e2acb49ed0e3049/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:39.179773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884f757-4840-4ada-a6e8-a39f853713a4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:39.195581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884f757-4840-4ada-a6e8-a39f853713a4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:39.324721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295e3e2-054e-4134-89df-2483d71e9f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:39.325394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e3e2-054e-4134-89df-2483d71e9f35 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:39.325688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e3e2-054e-4134-89df-2483d71e9f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:39.325971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e3e2-054e-4134-89df-2483d71e9f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:39.327411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295e3e2-054e-4134-89df-2483d71e9f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:39.329795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295e3e2-054e-4134-89df-2483d71e9f35 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:39.350043 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3294/6517] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:39.356672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bf48e-192d-4f11-a0b9-810999e85135 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:39.383617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bf48e-192d-4f11-a0b9-810999e85135 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:39.398901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3aa0a-91fc-4e18-bf9b-b4a79ee1b32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:39.401907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3aa0a-91fc-4e18-bf9b-b4a79ee1b32e admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:39.403050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3aa0a-91fc-4e18-bf9b-b4a79ee1b32e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:39.404988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3aa0a-91fc-4e18-bf9b-b4a79ee1b32e admin admin] PATH_INFO: `/v3/projects/82291c52950d4229b2adb2b92985396e/users/07d5fba4627346a48e2acb49ed0e3049/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:39.410451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3aa0a-91fc-4e18-bf9b-b4a79ee1b32e admin admin] RBAC: Authorizing `identity:create_grant(project_id=82291c52950d4229b2adb2b92985396e, user_id=07d5fba4627346a48e2acb49ed0e3049, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:39.419335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3aa0a-91fc-4e18-bf9b-b4a79ee1b32e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:39.459294 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3224/6518] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:39 2022] PUT /identity/v3/projects/82291c52950d4229b2adb2b92985396e/users/07d5fba4627346a48e2acb49ed0e3049/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:39.473234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73378e-399d-439d-b802-7588ccc67050 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:39.473533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73378e-399d-439d-b802-7588ccc67050 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:39.473884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73378e-399d-439d-b802-7588ccc67050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:39.525837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8b73378e-399d-439d-b802-7588ccc67050 None None] MFA Rules not processed for user `07d5fba4627346a48e2acb49ed0e3049`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:39.527510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8b73378e-399d-439d-b802-7588ccc67050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:39.547922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:39.649356 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3295/6519] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:39.670782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3aa0a-91fc-4e18-bf9b-b4a79ee1b32e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:39.672729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-79c3aa0a-91fc-4e18-bf9b-b4a79ee1b32e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:39.796345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3aa0a-91fc-4e18-bf9b-b4a79ee1b32e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:39.806686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ed394-f27f-423a-a3c1-009e95e0f5bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:39.807401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed394-f27f-423a-a3c1-009e95e0f5bc service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:39.807679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed394-f27f-423a-a3c1-009e95e0f5bc service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:39.807949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed394-f27f-423a-a3c1-009e95e0f5bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:39.835218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ed394-f27f-423a-a3c1-009e95e0f5bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:39.836690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ed394-f27f-423a-a3c1-009e95e0f5bc service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:39.850027 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3225/6520] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:43.875047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6eeb4-907c-45b2-bb91-d80bd44920ab None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:43.877268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6eeb4-907c-45b2-bb91-d80bd44920ab None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:43.877580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6eeb4-907c-45b2-bb91-d80bd44920ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:43.918128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-eed6eeb4-907c-45b2-bb91-d80bd44920ab None None] MFA Rules not processed for user `01c449f3fd1946508ab455af5b231c14`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:43.922235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-eed6eeb4-907c-45b2-bb91-d80bd44920ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:44.062624 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3296/6521] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:44.337257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ed394-f27f-423a-a3c1-009e95e0f5bc service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:44.445076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ed394-f27f-423a-a3c1-009e95e0f5bc service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:44.452675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:44.453666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:44.454207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:44.454761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] PATH_INFO: `/v3/users/cfec7ee1abdb4892b5cd1995599d7357` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:44.456170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfec7ee1abdb4892b5cd1995599d7357)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:44.458392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:44.530396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfec7ee1abdb4892b5cd1995599d7357'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:44.546759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfec7ee1abdb4892b5cd1995599d7357'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:44.547925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfec7ee1abdb4892b5cd1995599d7357'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:44.549522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:44.569615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfec7ee1abdb4892b5cd1995599d7357'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:44.580578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfec7ee1abdb4892b5cd1995599d7357'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:44.591152 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3226/6522] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:44 2022] DELETE /identity/v3/users/cfec7ee1abdb4892b5cd1995599d7357 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:44.596087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e1ee9-c78d-46b5-97c2-2a8111328e72 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:44.596385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e1ee9-c78d-46b5-97c2-2a8111328e72 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:44.596700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e1ee9-c78d-46b5-97c2-2a8111328e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:44.616807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d96e1ee9-c78d-46b5-97c2-2a8111328e72 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:44.618180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d96e1ee9-c78d-46b5-97c2-2a8111328e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:44.700094 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3297/6523] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:44.770383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:44.772717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:44.887729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade95bb-b5a0-4371-b093-c3e01024c42e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:44.896031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a26f3-37ba-41d5-9503-08c1b3a1b60e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:44.897057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a26f3-37ba-41d5-9503-08c1b3a1b60e service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:44.897574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a26f3-37ba-41d5-9503-08c1b3a1b60e service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:44.898076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a26f3-37ba-41d5-9503-08c1b3a1b60e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:44.922655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a26f3-37ba-41d5-9503-08c1b3a1b60e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:44.925231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a26f3-37ba-41d5-9503-08c1b3a1b60e service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:44.936777 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3227/6524] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:45.798700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e1ee9-c78d-46b5-97c2-2a8111328e72 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:45.800267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d96e1ee9-c78d-46b5-97c2-2a8111328e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:45.816882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e1ee9-c78d-46b5-97c2-2a8111328e72 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:45.840046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:45.842547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:45.843749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:45.844502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] PATH_INFO: `/v3/projects/58b6410bf7614ef8ad8dad70ddc3a049` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:45.850004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58b6410bf7614ef8ad8dad70ddc3a049)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:45.853051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:45.956433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:45.958172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58b6410bf7614ef8ad8dad70ddc3a049', 'request_id': 'req-49f67af1-fd8e-4e95-8700-a469d4f75295'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:45.979553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58b6410bf7614ef8ad8dad70ddc3a049', 'request_id': 'req-49f67af1-fd8e-4e95-8700-a469d4f75295'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:45.982417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a26f3-37ba-41d5-9503-08c1b3a1b60e service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:45.984927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fc5a26f3-37ba-41d5-9503-08c1b3a1b60e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:45.993578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58b6410bf7614ef8ad8dad70ddc3a049', 'request_id': 'req-49f67af1-fd8e-4e95-8700-a469d4f75295'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:46.011825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] The token cache is being invalidate because project 58b6410bf7614ef8ad8dad70ddc3a049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:46.012650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:46.014515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:46.017007 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3298/6525] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:45 2022] DELETE /identity/v3/projects/58b6410bf7614ef8ad8dad70ddc3a049 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:46.021487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:46.024458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:46.145365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a26f3-37ba-41d5-9503-08c1b3a1b60e service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:46.169119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f67af1-fd8e-4e95-8700-a469d4f75295 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:46.183187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:46.184159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:46.184451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:46.184736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] PATH_INFO: `/v3/users/07d5fba4627346a48e2acb49ed0e3049` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:46.186272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] RBAC: Authorizing `identity:delete_user(user_id=07d5fba4627346a48e2acb49ed0e3049)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:46.188889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:46.254139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07d5fba4627346a48e2acb49ed0e3049'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:46.270337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc0452-c1db-4485-96bd-8dc818529dc8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:46.270337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc0452-c1db-4485-96bd-8dc818529dc8 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:46.270337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc0452-c1db-4485-96bd-8dc818529dc8 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:46.270337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc0452-c1db-4485-96bd-8dc818529dc8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:46.277796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-01bc0452-c1db-4485-96bd-8dc818529dc8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:46.285231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07d5fba4627346a48e2acb49ed0e3049'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:46.301427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07d5fba4627346a48e2acb49ed0e3049'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:46.302558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07d5fba4627346a48e2acb49ed0e3049'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:46.303344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07d5fba4627346a48e2acb49ed0e3049'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:46.306447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc0452-c1db-4485-96bd-8dc818529dc8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:46.311709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc0452-c1db-4485-96bd-8dc818529dc8 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:46.340138 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3228/6526] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:46.344885 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3299/6527] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:46 2022] DELETE /identity/v3/users/07d5fba4627346a48e2acb49ed0e3049 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:46.667246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc0452-c1db-4485-96bd-8dc818529dc8 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:46.668655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-01bc0452-c1db-4485-96bd-8dc818529dc8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:46.738367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:46.789895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc0452-c1db-4485-96bd-8dc818529dc8 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:46.863617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf39617-f686-4bc5-ad04-a6f01f9d2691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:46.864391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf39617-f686-4bc5-ad04-a6f01f9d2691 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:46.864555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:46.865038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf39617-f686-4bc5-ad04-a6f01f9d2691 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:46.865167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf39617-f686-4bc5-ad04-a6f01f9d2691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:46.866044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:46.867018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fdf39617-f686-4bc5-ad04-a6f01f9d2691 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:46.949663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55998f7c-0ec9-47da-82f5-cfb7b5e9d69c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:46.950661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:46.952804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf39617-f686-4bc5-ad04-a6f01f9d2691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:46.954528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf39617-f686-4bc5-ad04-a6f01f9d2691 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:46.959005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:46.960127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:46.960653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:46.964280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2 admin admin] PATH_INFO: `/v3/projects/82291c52950d4229b2adb2b92985396e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:46.967760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82291c52950d4229b2adb2b92985396e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:46.969809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:46.985299 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3229/6528] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:47.036445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82291c52950d4229b2adb2b92985396e', 'request_id': 'req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:47.045841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82291c52950d4229b2adb2b92985396e', 'request_id': 'req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:47.053474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82291c52950d4229b2adb2b92985396e', 'request_id': 'req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:47.064477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2 admin admin] The token cache is being invalidate because project 82291c52950d4229b2adb2b92985396e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:47.064772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:47.065479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3cbfc8f5-5722-48fb-8b9e-8b63424065f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:47.066984 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3300/6529] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:46 2022] DELETE /identity/v3/projects/82291c52950d4229b2adb2b92985396e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:47.072009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d99ac-bdb2-4c8b-b9dd-66edcb93d908 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:47.072327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d99ac-bdb2-4c8b-b9dd-66edcb93d908 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:47.072632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d99ac-bdb2-4c8b-b9dd-66edcb93d908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:47.093535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7f0d99ac-bdb2-4c8b-b9dd-66edcb93d908 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:47.094778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7f0d99ac-bdb2-4c8b-b9dd-66edcb93d908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:47.172718 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3230/6530] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:47.181567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef7c55-6902-4d02-a093-bbdea5ee9f1d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:47.181849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef7c55-6902-4d02-a093-bbdea5ee9f1d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:47.182162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef7c55-6902-4d02-a093-bbdea5ee9f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:47.225934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-14ef7c55-6902-4d02-a093-bbdea5ee9f1d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:47.227540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-14ef7c55-6902-4d02-a093-bbdea5ee9f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:47.249090 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3301/6531] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:12:47.250453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d99ac-bdb2-4c8b-b9dd-66edcb93d908 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:47.268118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d99ac-bdb2-4c8b-b9dd-66edcb93d908 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:47.275117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fc1c2-07d0-4fc5-8552-b4c378b557e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:47.277090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fc1c2-07d0-4fc5-8552-b4c378b557e0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:47.277470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fc1c2-07d0-4fc5-8552-b4c378b557e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:47.277673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fc1c2-07d0-4fc5-8552-b4c378b557e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:47.278327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fc1c2-07d0-4fc5-8552-b4c378b557e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:12:47.278647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fc1c2-07d0-4fc5-8552-b4c378b557e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:47.280794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fc1c2-07d0-4fc5-8552-b4c378b557e0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:47.303848 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3231/6532] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:47.308282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef7c55-6902-4d02-a093-bbdea5ee9f1d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:47.332140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef7c55-6902-4d02-a093-bbdea5ee9f1d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:47.338965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92169f0d-7719-4af6-897a-3d9e60ba946c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:47.339596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92169f0d-7719-4af6-897a-3d9e60ba946c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:47.339887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92169f0d-7719-4af6-897a-3d9e60ba946c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:47.340167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92169f0d-7719-4af6-897a-3d9e60ba946c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:47.340831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92169f0d-7719-4af6-897a-3d9e60ba946c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:47.343415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92169f0d-7719-4af6-897a-3d9e60ba946c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:47.389816 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3302/6533] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:47 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:47.395244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fc1c2-07d0-4fc5-8552-b4c378b557e0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:47.408549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fc1c2-07d0-4fc5-8552-b4c378b557e0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:47.420687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:47.477509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2ae6d-44bc-4f27-a0d2-521917c615e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:47.478127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2ae6d-44bc-4f27-a0d2-521917c615e4 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:47.478415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2ae6d-44bc-4f27-a0d2-521917c615e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:47.478697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2ae6d-44bc-4f27-a0d2-521917c615e4 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:47.479356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2ae6d-44bc-4f27-a0d2-521917c615e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:47.481324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2ae6d-44bc-4f27-a0d2-521917c615e4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:47.506420 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3232/6534] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:47 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:12:47.507855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92169f0d-7719-4af6-897a-3d9e60ba946c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:47.519611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92169f0d-7719-4af6-897a-3d9e60ba946c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:47.525456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d9d89-a3e1-464e-b370-fc634bdda2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:47.526076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d9d89-a3e1-464e-b370-fc634bdda2ea admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:47.526498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d9d89-a3e1-464e-b370-fc634bdda2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:47.526782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d9d89-a3e1-464e-b370-fc634bdda2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:47.527441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d9d89-a3e1-464e-b370-fc634bdda2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:47.529153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d9d89-a3e1-464e-b370-fc634bdda2ea admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:47.550852 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3303/6535] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:47.552148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2ae6d-44bc-4f27-a0d2-521917c615e4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:47.560187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2ae6d-44bc-4f27-a0d2-521917c615e4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:47.565044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0b9d4-4515-497d-9e46-4ca2432985fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:47.565705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0b9d4-4515-497d-9e46-4ca2432985fa admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:47.566017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0b9d4-4515-497d-9e46-4ca2432985fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:47.566302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0b9d4-4515-497d-9e46-4ca2432985fa admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:47.566944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0b9d4-4515-497d-9e46-4ca2432985fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:47.568571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0b9d4-4515-497d-9e46-4ca2432985fa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:47.580247 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3233/6536] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:47.584417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d9d89-a3e1-464e-b370-fc634bdda2ea admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:47.595077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d9d89-a3e1-464e-b370-fc634bdda2ea admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:47.599766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb672866-5f4d-48f3-847c-0292e366745f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:47.600428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb672866-5f4d-48f3-847c-0292e366745f admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:47.600718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb672866-5f4d-48f3-847c-0292e366745f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:47.601052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb672866-5f4d-48f3-847c-0292e366745f admin admin] PATH_INFO: `/v3/projects/374831c019614361a782e1edb7ea5324/users/a02def928c42450b9bca99d965a5c1c7/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:47.617288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb672866-5f4d-48f3-847c-0292e366745f admin admin] RBAC: Authorizing `identity:create_grant(project_id=374831c019614361a782e1edb7ea5324, user_id=a02def928c42450b9bca99d965a5c1c7, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:47.620077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb672866-5f4d-48f3-847c-0292e366745f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:47.634947 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3304/6537] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:47 2022] PUT /identity/v3/projects/374831c019614361a782e1edb7ea5324/users/a02def928c42450b9bca99d965a5c1c7/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:47.730594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545a46c-6ac4-4ce8-9c8c-2476f754cd0e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:47.730899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545a46c-6ac4-4ce8-9c8c-2476f754cd0e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:47.731171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545a46c-6ac4-4ce8-9c8c-2476f754cd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:47.762555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3545a46c-6ac4-4ce8-9c8c-2476f754cd0e None None] MFA Rules not processed for user `a02def928c42450b9bca99d965a5c1c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:47.764436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3545a46c-6ac4-4ce8-9c8c-2476f754cd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:47.840921 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3234/6538] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:47.851385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb672866-5f4d-48f3-847c-0292e366745f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:47.853564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bb672866-5f4d-48f3-847c-0292e366745f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:47.916858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb672866-5f4d-48f3-847c-0292e366745f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:47.920766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34fdb8-ebd3-41bd-825a-c01a6a934b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:47.922204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34fdb8-ebd3-41bd-825a-c01a6a934b29 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:47.922204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34fdb8-ebd3-41bd-825a-c01a6a934b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:47.922204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34fdb8-ebd3-41bd-825a-c01a6a934b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:47.931914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34fdb8-ebd3-41bd-825a-c01a6a934b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:47.933412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34fdb8-ebd3-41bd-825a-c01a6a934b29 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:47.944868 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3305/6539] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:50.577628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545a46c-6ac4-4ce8-9c8c-2476f754cd0e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:50.579457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3545a46c-6ac4-4ce8-9c8c-2476f754cd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:50.769129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545a46c-6ac4-4ce8-9c8c-2476f754cd0e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:50.789901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19ea43-7d17-4c7e-879a-9450c57a4ea0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:50.790772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19ea43-7d17-4c7e-879a-9450c57a4ea0 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:50.791188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19ea43-7d17-4c7e-879a-9450c57a4ea0 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:50.791619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19ea43-7d17-4c7e-879a-9450c57a4ea0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:50.793907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0c19ea43-7d17-4c7e-879a-9450c57a4ea0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:50.819219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19ea43-7d17-4c7e-879a-9450c57a4ea0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:50.820843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19ea43-7d17-4c7e-879a-9450c57a4ea0 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:50.847192 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3235/6540] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:54.312459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34fdb8-ebd3-41bd-825a-c01a6a934b29 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:54.314765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5a34fdb8-ebd3-41bd-825a-c01a6a934b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:54.370186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:54.435364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34fdb8-ebd3-41bd-825a-c01a6a934b29 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:54.440065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a7728-110b-4ebb-afeb-5a1a22fae462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:54.440831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a7728-110b-4ebb-afeb-5a1a22fae462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:54.441284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a7728-110b-4ebb-afeb-5a1a22fae462 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:54.441676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a7728-110b-4ebb-afeb-5a1a22fae462 admin admin] PATH_INFO: `/v3/users/3e79edf1a33e45ddac92ba8cd8429d9b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:54.443048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a7728-110b-4ebb-afeb-5a1a22fae462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e79edf1a33e45ddac92ba8cd8429d9b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:54.445176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a7728-110b-4ebb-afeb-5a1a22fae462 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:54.519237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d12a7728-110b-4ebb-afeb-5a1a22fae462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e79edf1a33e45ddac92ba8cd8429d9b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:54.539262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d12a7728-110b-4ebb-afeb-5a1a22fae462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e79edf1a33e45ddac92ba8cd8429d9b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:54.548784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d12a7728-110b-4ebb-afeb-5a1a22fae462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e79edf1a33e45ddac92ba8cd8429d9b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:54.551898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d12a7728-110b-4ebb-afeb-5a1a22fae462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e79edf1a33e45ddac92ba8cd8429d9b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:54.553086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d12a7728-110b-4ebb-afeb-5a1a22fae462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e79edf1a33e45ddac92ba8cd8429d9b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:54.586485 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3306/6541] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:54 2022] DELETE /identity/v3/users/3e79edf1a33e45ddac92ba8cd8429d9b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:55.334455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19ea43-7d17-4c7e-879a-9450c57a4ea0 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:55.351067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19ea43-7d17-4c7e-879a-9450c57a4ea0 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:55.362673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:55.497255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe891d1-65fb-4fda-980a-0916fdba5e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:55.497980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe891d1-65fb-4fda-980a-0916fdba5e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:55.498269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe891d1-65fb-4fda-980a-0916fdba5e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:55.498549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe891d1-65fb-4fda-980a-0916fdba5e66 admin admin] PATH_INFO: `/v3/projects/15985a3823b5409c8a18e31048f82ff7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:55.505755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe891d1-65fb-4fda-980a-0916fdba5e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15985a3823b5409c8a18e31048f82ff7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:55.507771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe891d1-65fb-4fda-980a-0916fdba5e66 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:55.613787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ffe891d1-65fb-4fda-980a-0916fdba5e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15985a3823b5409c8a18e31048f82ff7', 'request_id': 'req-ffe891d1-65fb-4fda-980a-0916fdba5e66'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:55.630157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ffe891d1-65fb-4fda-980a-0916fdba5e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15985a3823b5409c8a18e31048f82ff7', 'request_id': 'req-ffe891d1-65fb-4fda-980a-0916fdba5e66'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:55.641748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ffe891d1-65fb-4fda-980a-0916fdba5e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15985a3823b5409c8a18e31048f82ff7', 'request_id': 'req-ffe891d1-65fb-4fda-980a-0916fdba5e66'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:55.662401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ffe891d1-65fb-4fda-980a-0916fdba5e66 admin admin] The token cache is being invalidate because project 15985a3823b5409c8a18e31048f82ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:12:55.662714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ffe891d1-65fb-4fda-980a-0916fdba5e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:55.666003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ffe891d1-65fb-4fda-980a-0916fdba5e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:55.674378 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3236/6542] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:55 2022] DELETE /identity/v3/projects/15985a3823b5409c8a18e31048f82ff7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:55.680460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d570a4c-7a3f-400f-a788-22656b2f0c6e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:55.681546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d570a4c-7a3f-400f-a788-22656b2f0c6e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:55.682059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d570a4c-7a3f-400f-a788-22656b2f0c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:55.728569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6d570a4c-7a3f-400f-a788-22656b2f0c6e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:55.730635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6d570a4c-7a3f-400f-a788-22656b2f0c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:55.942457 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3307/6543] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:55.951744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92ae6a-e085-46d0-b0ba-4fc2d3688ec1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:55.952044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92ae6a-e085-46d0-b0ba-4fc2d3688ec1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:55.952334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92ae6a-e085-46d0-b0ba-4fc2d3688ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:55.994787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ca92ae6a-e085-46d0-b0ba-4fc2d3688ec1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:55.996057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ca92ae6a-e085-46d0-b0ba-4fc2d3688ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:56.028348 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3237/6544] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:56.032348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d570a4c-7a3f-400f-a788-22656b2f0c6e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:56.042900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d570a4c-7a3f-400f-a788-22656b2f0c6e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:56.050009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb172f9-aeac-4c6c-9981-5258ce6836e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:56.050857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb172f9-aeac-4c6c-9981-5258ce6836e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:56.051336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb172f9-aeac-4c6c-9981-5258ce6836e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:56.051819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb172f9-aeac-4c6c-9981-5258ce6836e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:56.052921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb172f9-aeac-4c6c-9981-5258ce6836e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:12:56.053622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb172f9-aeac-4c6c-9981-5258ce6836e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:56.058454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb172f9-aeac-4c6c-9981-5258ce6836e6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:56.075821 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3308/6545] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:12:56.081654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92ae6a-e085-46d0-b0ba-4fc2d3688ec1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:56.100731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92ae6a-e085-46d0-b0ba-4fc2d3688ec1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:56.108375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865971-adf0-4d9e-9d2a-e4956d283250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:56.109185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865971-adf0-4d9e-9d2a-e4956d283250 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:56.109599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865971-adf0-4d9e-9d2a-e4956d283250 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:56.110046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865971-adf0-4d9e-9d2a-e4956d283250 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:56.122454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865971-adf0-4d9e-9d2a-e4956d283250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:56.127504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865971-adf0-4d9e-9d2a-e4956d283250 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:56.215386 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3238/6546] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:12:56 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:56.219430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb172f9-aeac-4c6c-9981-5258ce6836e6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:56.231654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb172f9-aeac-4c6c-9981-5258ce6836e6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:56.280256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:56.338883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1a384-8a04-4e71-b18d-fb73055f689c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:56.339669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1a384-8a04-4e71-b18d-fb73055f689c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:56.339986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1a384-8a04-4e71-b18d-fb73055f689c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:56.340322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1a384-8a04-4e71-b18d-fb73055f689c admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:56.341146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1a384-8a04-4e71-b18d-fb73055f689c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:56.350306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1a384-8a04-4e71-b18d-fb73055f689c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:56.409813 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3309/6547] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:12:56 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:12:56.415016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865971-adf0-4d9e-9d2a-e4956d283250 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:56.432542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865971-adf0-4d9e-9d2a-e4956d283250 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:56.439076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837331ac-b75f-4eb2-bf87-2900707290c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:56.439886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837331ac-b75f-4eb2-bf87-2900707290c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:56.440278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837331ac-b75f-4eb2-bf87-2900707290c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:56.440803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837331ac-b75f-4eb2-bf87-2900707290c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:56.442465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837331ac-b75f-4eb2-bf87-2900707290c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:56.444960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837331ac-b75f-4eb2-bf87-2900707290c4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:56.459647 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3239/6548] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:56.463378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1a384-8a04-4e71-b18d-fb73055f689c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:56.480028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1a384-8a04-4e71-b18d-fb73055f689c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:56.486499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1e925-2880-4880-b2a9-08b5973ae648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:56.487178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1e925-2880-4880-b2a9-08b5973ae648 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:56.487465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1e925-2880-4880-b2a9-08b5973ae648 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:56.487794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1e925-2880-4880-b2a9-08b5973ae648 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:56.488447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea1e925-2880-4880-b2a9-08b5973ae648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:56.490163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea1e925-2880-4880-b2a9-08b5973ae648 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:56.532837 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3310/6549] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:12:56.534247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837331ac-b75f-4eb2-bf87-2900707290c4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:56.549930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837331ac-b75f-4eb2-bf87-2900707290c4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:56.572820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c26c4-1630-4ccd-b938-83b7f8e90148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:56.573605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c26c4-1630-4ccd-b938-83b7f8e90148 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:56.573897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c26c4-1630-4ccd-b938-83b7f8e90148 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:56.574180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c26c4-1630-4ccd-b938-83b7f8e90148 admin admin] PATH_INFO: `/v3/projects/9cc6c8498d994cbdbdd965a848504ee9/users/85e7703452424a6bacf85a0aee78444c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:56.604632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c26c4-1630-4ccd-b938-83b7f8e90148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cc6c8498d994cbdbdd965a848504ee9, user_id=85e7703452424a6bacf85a0aee78444c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:56.607080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c26c4-1630-4ccd-b938-83b7f8e90148 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:56.628977 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3240/6550] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:12:56 2022] PUT /identity/v3/projects/9cc6c8498d994cbdbdd965a848504ee9/users/85e7703452424a6bacf85a0aee78444c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:12:56.641357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e49993-c151-4a46-ac4b-62c42b097297 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:56.641642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e49993-c151-4a46-ac4b-62c42b097297 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:56.641947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e49993-c151-4a46-ac4b-62c42b097297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:56.680674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-79e49993-c151-4a46-ac4b-62c42b097297 None None] MFA Rules not processed for user `85e7703452424a6bacf85a0aee78444c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:56.682028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-79e49993-c151-4a46-ac4b-62c42b097297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:56.904218 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3311/6551] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:56.923751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c26c4-1630-4ccd-b938-83b7f8e90148 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:56.927323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e52c26c4-1630-4ccd-b938-83b7f8e90148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:57.034717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:57.104993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c26c4-1630-4ccd-b938-83b7f8e90148 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:57.112986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd432c9d-4e57-4956-9a24-306ac628b4c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:57.114421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd432c9d-4e57-4956-9a24-306ac628b4c2 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:57.114925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd432c9d-4e57-4956-9a24-306ac628b4c2 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:57.115370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd432c9d-4e57-4956-9a24-306ac628b4c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:57.136126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd432c9d-4e57-4956-9a24-306ac628b4c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:57.137901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd432c9d-4e57-4956-9a24-306ac628b4c2 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:57.149346 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3241/6552] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:12:59.164241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e49993-c151-4a46-ac4b-62c42b097297 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:59.247808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e49993-c151-4a46-ac4b-62c42b097297 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:59.255324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:59.256396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:59.256695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:59.257034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] PATH_INFO: `/v3/users/85e7703452424a6bacf85a0aee78444c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:59.258399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] RBAC: Authorizing `identity:delete_user(user_id=85e7703452424a6bacf85a0aee78444c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:59.260841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:59.332311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:12:59.347896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85e7703452424a6bacf85a0aee78444c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:59.368225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85e7703452424a6bacf85a0aee78444c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:59.379973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85e7703452424a6bacf85a0aee78444c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:59.380921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85e7703452424a6bacf85a0aee78444c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:59.381640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85e7703452424a6bacf85a0aee78444c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:12:59.406727 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3312/6553] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:12:59 2022] DELETE /identity/v3/users/85e7703452424a6bacf85a0aee78444c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:12:59.411091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb6587-8002-467c-91ec-2a807b947ed5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:59.411617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb6587-8002-467c-91ec-2a807b947ed5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:59.411776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb6587-8002-467c-91ec-2a807b947ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:59.439613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-88fb6587-8002-467c-91ec-2a807b947ed5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:12:59.441811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-88fb6587-8002-467c-91ec-2a807b947ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:59.517104 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3242/6554] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:12:59.523957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:12:59.525448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:12:59.596490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720dabb-7761-48f0-8fc4-e4f44b2c036d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:12:59.602594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416648a-dfac-4ea7-8b28-09b25688b8c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:12:59.603233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416648a-dfac-4ea7-8b28-09b25688b8c6 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:12:59.603677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416648a-dfac-4ea7-8b28-09b25688b8c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:12:59.604092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416648a-dfac-4ea7-8b28-09b25688b8c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:12:59.614596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416648a-dfac-4ea7-8b28-09b25688b8c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:12:59.616192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416648a-dfac-4ea7-8b28-09b25688b8c6 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:12:59.626603 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3313/6555] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:12:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:13:00.000246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb6587-8002-467c-91ec-2a807b947ed5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:00.002549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-88fb6587-8002-467c-91ec-2a807b947ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:00.032118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb6587-8002-467c-91ec-2a807b947ed5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:00.054082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e883b9-75be-4f11-b558-3af7fee2d799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:00.054951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e883b9-75be-4f11-b558-3af7fee2d799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:00.055401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e883b9-75be-4f11-b558-3af7fee2d799 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:00.055878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e883b9-75be-4f11-b558-3af7fee2d799 admin admin] PATH_INFO: `/v3/projects/9cc6c8498d994cbdbdd965a848504ee9` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:00.057669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e883b9-75be-4f11-b558-3af7fee2d799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cc6c8498d994cbdbdd965a848504ee9)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:00.060318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e883b9-75be-4f11-b558-3af7fee2d799 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:00.128651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53e883b9-75be-4f11-b558-3af7fee2d799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cc6c8498d994cbdbdd965a848504ee9', 'request_id': 'req-53e883b9-75be-4f11-b558-3af7fee2d799'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:00.141506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53e883b9-75be-4f11-b558-3af7fee2d799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cc6c8498d994cbdbdd965a848504ee9', 'request_id': 'req-53e883b9-75be-4f11-b558-3af7fee2d799'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:00.151403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53e883b9-75be-4f11-b558-3af7fee2d799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cc6c8498d994cbdbdd965a848504ee9', 'request_id': 'req-53e883b9-75be-4f11-b558-3af7fee2d799'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:00.182221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53e883b9-75be-4f11-b558-3af7fee2d799 admin admin] The token cache is being invalidate because project 9cc6c8498d994cbdbdd965a848504ee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:00.182502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53e883b9-75be-4f11-b558-3af7fee2d799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:00.184159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-53e883b9-75be-4f11-b558-3af7fee2d799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:00.186872 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3243/6556] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:12:59 2022] DELETE /identity/v3/projects/9cc6c8498d994cbdbdd965a848504ee9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:00.193403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64dc6e-05bb-459c-94ab-95ca3e98c5ae None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:00.193943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64dc6e-05bb-459c-94ab-95ca3e98c5ae None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:00.194051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64dc6e-05bb-459c-94ab-95ca3e98c5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:00.226647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2d64dc6e-05bb-459c-94ab-95ca3e98c5ae None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:00.228337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2d64dc6e-05bb-459c-94ab-95ca3e98c5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:00.303405 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3314/6557] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:00.311485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce35133-f62f-4e27-8751-fadf9f3cb24f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:00.311778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce35133-f62f-4e27-8751-fadf9f3cb24f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:00.312061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce35133-f62f-4e27-8751-fadf9f3cb24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:00.343772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bce35133-f62f-4e27-8751-fadf9f3cb24f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:00.345335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bce35133-f62f-4e27-8751-fadf9f3cb24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:00.358084 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3244/6558] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:00.362217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64dc6e-05bb-459c-94ab-95ca3e98c5ae None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:00.373202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64dc6e-05bb-459c-94ab-95ca3e98c5ae None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:00.379053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b6e9a-8f8e-4e14-b00a-f6f440000fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:00.380367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b6e9a-8f8e-4e14-b00a-f6f440000fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:00.380490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b6e9a-8f8e-4e14-b00a-f6f440000fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:00.380729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b6e9a-8f8e-4e14-b00a-f6f440000fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:00.381607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b6e9a-8f8e-4e14-b00a-f6f440000fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:00.382315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b6e9a-8f8e-4e14-b00a-f6f440000fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:00.383931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b6e9a-8f8e-4e14-b00a-f6f440000fd7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:00.416227 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3315/6559] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:00.417175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce35133-f62f-4e27-8751-fadf9f3cb24f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:00.428671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce35133-f62f-4e27-8751-fadf9f3cb24f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:00.434587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73af4f5-116a-4bfe-92e8-728e42459746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:00.434948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73af4f5-116a-4bfe-92e8-728e42459746 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:00.435348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73af4f5-116a-4bfe-92e8-728e42459746 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:00.435698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73af4f5-116a-4bfe-92e8-728e42459746 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:00.436435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73af4f5-116a-4bfe-92e8-728e42459746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:00.438622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73af4f5-116a-4bfe-92e8-728e42459746 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:00.472954 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3245/6560] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:00 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:00.474559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b6e9a-8f8e-4e14-b00a-f6f440000fd7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:00.485414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b6e9a-8f8e-4e14-b00a-f6f440000fd7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:00.535199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:00.553949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e979518-002b-4f6a-8015-168b1649ddb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:00.554581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e979518-002b-4f6a-8015-168b1649ddb0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:00.554933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e979518-002b-4f6a-8015-168b1649ddb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:00.555358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e979518-002b-4f6a-8015-168b1649ddb0 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:00.556078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e979518-002b-4f6a-8015-168b1649ddb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:00.558088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e979518-002b-4f6a-8015-168b1649ddb0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:00.587645 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3316/6561] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:00 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:00.591822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73af4f5-116a-4bfe-92e8-728e42459746 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:00.604580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73af4f5-116a-4bfe-92e8-728e42459746 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:00.609564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b35563-ec20-4054-b90a-5f421e47265b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:00.610269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b35563-ec20-4054-b90a-5f421e47265b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:00.610645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b35563-ec20-4054-b90a-5f421e47265b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:00.610986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b35563-ec20-4054-b90a-5f421e47265b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:00.611701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b35563-ec20-4054-b90a-5f421e47265b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:00.613449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b35563-ec20-4054-b90a-5f421e47265b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:00.627150 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3246/6562] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:00.632869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e979518-002b-4f6a-8015-168b1649ddb0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:00.647294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e979518-002b-4f6a-8015-168b1649ddb0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:00.655810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1de3f-1b10-407e-b96c-11ff81c8bb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:00.656450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1de3f-1b10-407e-b96c-11ff81c8bb41 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:00.656769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1de3f-1b10-407e-b96c-11ff81c8bb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:00.657110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1de3f-1b10-407e-b96c-11ff81c8bb41 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:00.657771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1de3f-1b10-407e-b96c-11ff81c8bb41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:00.659478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1de3f-1b10-407e-b96c-11ff81c8bb41 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:00.673505 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3317/6563] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:00.677781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b35563-ec20-4054-b90a-5f421e47265b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:00.685519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:00.691578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b35563-ec20-4054-b90a-5f421e47265b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:00.698895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ccb78-3f7a-4e25-94ad-e6447e7a6d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:00.700035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ccb78-3f7a-4e25-94ad-e6447e7a6d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:00.700718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ccb78-3f7a-4e25-94ad-e6447e7a6d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:00.701132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ccb78-3f7a-4e25-94ad-e6447e7a6d6e admin admin] PATH_INFO: `/v3/projects/f4e2fd25078746cc803f0545e10fc176/users/b241663a9dce4ffcb78660b96596cfd0/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:00.739101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ccb78-3f7a-4e25-94ad-e6447e7a6d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e2fd25078746cc803f0545e10fc176, user_id=b241663a9dce4ffcb78660b96596cfd0, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:00.743556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ccb78-3f7a-4e25-94ad-e6447e7a6d6e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:00.779385 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3247/6564] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:00 2022] PUT /identity/v3/projects/f4e2fd25078746cc803f0545e10fc176/users/b241663a9dce4ffcb78660b96596cfd0/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:00.790827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0274a0c-83bc-48c6-a0f0-98eeb50070fd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:00.791305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0274a0c-83bc-48c6-a0f0-98eeb50070fd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:00.791695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0274a0c-83bc-48c6-a0f0-98eeb50070fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:00.851788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e0274a0c-83bc-48c6-a0f0-98eeb50070fd None None] MFA Rules not processed for user `b241663a9dce4ffcb78660b96596cfd0`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:00.853437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e0274a0c-83bc-48c6-a0f0-98eeb50070fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:01.022043 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3318/6565] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:01.025985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ccb78-3f7a-4e25-94ad-e6447e7a6d6e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:01.166941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ccb78-3f7a-4e25-94ad-e6447e7a6d6e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:01.172366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6c7c0-682b-4beb-9667-36af3952fb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:01.173624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6c7c0-682b-4beb-9667-36af3952fb64 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:01.173915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6c7c0-682b-4beb-9667-36af3952fb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:01.174193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6c7c0-682b-4beb-9667-36af3952fb64 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:01.174999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6c7c0-682b-4beb-9667-36af3952fb64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:01.176955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6c7c0-682b-4beb-9667-36af3952fb64 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:01.229894 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3248/6566] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:01 2022] POST /identity/v3/projects => generated 375 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:01.230876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0274a0c-83bc-48c6-a0f0-98eeb50070fd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:01.271279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0274a0c-83bc-48c6-a0f0-98eeb50070fd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:01.506264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c09753-dbe4-41bc-acfc-bc3f6cd9ba4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:01.506908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c09753-dbe4-41bc-acfc-bc3f6cd9ba4a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:01.528159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c09753-dbe4-41bc-acfc-bc3f6cd9ba4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:01.528492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c09753-dbe4-41bc-acfc-bc3f6cd9ba4a admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:01.529940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c09753-dbe4-41bc-acfc-bc3f6cd9ba4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:01.539572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c09753-dbe4-41bc-acfc-bc3f6cd9ba4a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:01.644161 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3319/6567] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:01 2022] POST /identity/v3/users => generated 294 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:01.645541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6c7c0-682b-4beb-9667-36af3952fb64 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:01.661934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6c7c0-682b-4beb-9667-36af3952fb64 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:01.682477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc243106-6be9-41b1-bebe-7bee96c9eb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:01.683121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc243106-6be9-41b1-bebe-7bee96c9eb51 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:01.683410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc243106-6be9-41b1-bebe-7bee96c9eb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:01.683693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc243106-6be9-41b1-bebe-7bee96c9eb51 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:01.684350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc243106-6be9-41b1-bebe-7bee96c9eb51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:01.686119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc243106-6be9-41b1-bebe-7bee96c9eb51 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:01.707707 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3249/6568] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:01.711686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c09753-dbe4-41bc-acfc-bc3f6cd9ba4a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:01.728759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c09753-dbe4-41bc-acfc-bc3f6cd9ba4a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:01.752431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4fb04-61c1-4e93-a7af-cf58c27a1ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:01.755887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4fb04-61c1-4e93-a7af-cf58c27a1ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:01.756243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4fb04-61c1-4e93-a7af-cf58c27a1ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:01.756583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4fb04-61c1-4e93-a7af-cf58c27a1ca1 admin admin] PATH_INFO: `/v3/projects/b14433a868db406ca9e5fd3af1c82eb1/users/1ef44d9b07e74c4aad46aa9e7a5fc75f/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:01.784825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4fb04-61c1-4e93-a7af-cf58c27a1ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14433a868db406ca9e5fd3af1c82eb1, user_id=1ef44d9b07e74c4aad46aa9e7a5fc75f, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:01.788864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4fb04-61c1-4e93-a7af-cf58c27a1ca1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:01.807831 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3320/6569] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:01 2022] PUT /identity/v3/projects/b14433a868db406ca9e5fd3af1c82eb1/users/1ef44d9b07e74c4aad46aa9e7a5fc75f/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:01.811607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc243106-6be9-41b1-bebe-7bee96c9eb51 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:01.825141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc243106-6be9-41b1-bebe-7bee96c9eb51 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:01.960034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11c2f3-2d8b-4f92-8316-21211af5434a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:01.960592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11c2f3-2d8b-4f92-8316-21211af5434a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:01.961068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11c2f3-2d8b-4f92-8316-21211af5434a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:01.961482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11c2f3-2d8b-4f92-8316-21211af5434a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:01.962224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11c2f3-2d8b-4f92-8316-21211af5434a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:01.967108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11c2f3-2d8b-4f92-8316-21211af5434a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:01.991166 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3250/6570] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:01.995800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4fb04-61c1-4e93-a7af-cf58c27a1ca1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:02.010244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4fb04-61c1-4e93-a7af-cf58c27a1ca1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:02.017325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268775d4-9852-4438-9938-aacbe81e382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:02.018090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268775d4-9852-4438-9938-aacbe81e382e admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:02.018445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268775d4-9852-4438-9938-aacbe81e382e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:02.018813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268775d4-9852-4438-9938-aacbe81e382e admin admin] PATH_INFO: `/v3/projects/b14433a868db406ca9e5fd3af1c82eb1/users/1ef44d9b07e74c4aad46aa9e7a5fc75f/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:02.021083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268775d4-9852-4438-9938-aacbe81e382e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14433a868db406ca9e5fd3af1c82eb1, user_id=1ef44d9b07e74c4aad46aa9e7a5fc75f, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:02.023995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268775d4-9852-4438-9938-aacbe81e382e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:02.038519 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3321/6571] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:01 2022] PUT /identity/v3/projects/b14433a868db406ca9e5fd3af1c82eb1/users/1ef44d9b07e74c4aad46aa9e7a5fc75f/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:02.049404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be4498-6fce-4631-824d-086091a30a92 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:02.049874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be4498-6fce-4631-824d-086091a30a92 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:02.050362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be4498-6fce-4631-824d-086091a30a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:02.089743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-02be4498-6fce-4631-824d-086091a30a92 None None] MFA Rules not processed for user `1ef44d9b07e74c4aad46aa9e7a5fc75f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:02.091551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-02be4498-6fce-4631-824d-086091a30a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:02.154851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:02.205991 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3251/6572] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:02.210649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268775d4-9852-4438-9938-aacbe81e382e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:02.222536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268775d4-9852-4438-9938-aacbe81e382e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:02.231038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58124942-28f7-46d5-a0ce-d62905dad6c8 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ef44d9b07e74c4aad46aa9e7a5fc75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14433a868db406ca9e5fd3af1c82eb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:02.231735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58124942-28f7-46d5-a0ce-d62905dad6c8 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:02.232126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58124942-28f7-46d5-a0ce-d62905dad6c8 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:02.232472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58124942-28f7-46d5-a0ce-d62905dad6c8 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:02.233320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58124942-28f7-46d5-a0ce-d62905dad6c8 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:02.235052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58124942-28f7-46d5-a0ce-d62905dad6c8 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:02.241129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:02.264064 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3322/6573] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:13:02 2022] POST /identity/v3/domains => generated 245 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:02.264647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be4498-6fce-4631-824d-086091a30a92 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:02.279302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be4498-6fce-4631-824d-086091a30a92 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:02.377373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35161ae-32c4-4e6e-ab57-6eff610ab8af tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ef44d9b07e74c4aad46aa9e7a5fc75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14433a868db406ca9e5fd3af1c82eb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:02.378067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35161ae-32c4-4e6e-ab57-6eff610ab8af tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:02.378360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35161ae-32c4-4e6e-ab57-6eff610ab8af tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:02.378639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35161ae-32c4-4e6e-ab57-6eff610ab8af tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:02.382291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35161ae-32c4-4e6e-ab57-6eff610ab8af tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:02.385854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35161ae-32c4-4e6e-ab57-6eff610ab8af tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:02.460417 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3252/6574] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:02 2022] POST /identity/v3/projects => generated 409 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:02.464744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58124942-28f7-46d5-a0ce-d62905dad6c8 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:02.482956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58124942-28f7-46d5-a0ce-d62905dad6c8 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:02.627309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db622ddc-fcee-438b-ad53-867c5b793822 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ef44d9b07e74c4aad46aa9e7a5fc75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14433a868db406ca9e5fd3af1c82eb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:02.628658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db622ddc-fcee-438b-ad53-867c5b793822 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:02.629001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db622ddc-fcee-438b-ad53-867c5b793822 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:02.629289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db622ddc-fcee-438b-ad53-867c5b793822 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:02.632197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db622ddc-fcee-438b-ad53-867c5b793822 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:02.635756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db622ddc-fcee-438b-ad53-867c5b793822 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:02.695967 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3323/6575] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:02 2022] POST /identity/v3/users => generated 348 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:02.705735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35161ae-32c4-4e6e-ab57-6eff610ab8af tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:02.751788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35161ae-32c4-4e6e-ab57-6eff610ab8af tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:02.778646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252271f-3fd7-4c26-a5da-3612175522d2 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ef44d9b07e74c4aad46aa9e7a5fc75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14433a868db406ca9e5fd3af1c82eb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:02.779934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252271f-3fd7-4c26-a5da-3612175522d2 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:02.780309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252271f-3fd7-4c26-a5da-3612175522d2 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:02.780598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252271f-3fd7-4c26-a5da-3612175522d2 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:02.781303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252271f-3fd7-4c26-a5da-3612175522d2 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:02.783001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252271f-3fd7-4c26-a5da-3612175522d2 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:02.810269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db622ddc-fcee-438b-ad53-867c5b793822 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:02.822162 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3253/6576] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:02.824592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db622ddc-fcee-438b-ad53-867c5b793822 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:02.835776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f32cd-9140-437e-bf5f-4fd9faa2e270 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ef44d9b07e74c4aad46aa9e7a5fc75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14433a868db406ca9e5fd3af1c82eb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:02.836474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f32cd-9140-437e-bf5f-4fd9faa2e270 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:02.836768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f32cd-9140-437e-bf5f-4fd9faa2e270 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:02.837096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f32cd-9140-437e-bf5f-4fd9faa2e270 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] PATH_INFO: `/v3/projects/4ee98f23ccb44b7295a28a3c1c75659a/users/0e63680e5f3844999db0a7d92d563bec/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:02.871852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f32cd-9140-437e-bf5f-4fd9faa2e270 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorizing `identity:create_grant(project_id=4ee98f23ccb44b7295a28a3c1c75659a, user_id=0e63680e5f3844999db0a7d92d563bec, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:02.877357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f32cd-9140-437e-bf5f-4fd9faa2e270 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:02.907616 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3324/6577] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:02 2022] PUT /identity/v3/projects/4ee98f23ccb44b7295a28a3c1c75659a/users/0e63680e5f3844999db0a7d92d563bec/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:02.912378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b26ea-2c7a-4742-a353-6dcc9e64ef29 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:02.913009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b26ea-2c7a-4742-a353-6dcc9e64ef29 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:02.913759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b26ea-2c7a-4742-a353-6dcc9e64ef29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:03.123547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-376b26ea-2c7a-4742-a353-6dcc9e64ef29 None None] MFA Rules not processed for user `0e63680e5f3844999db0a7d92d563bec`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:03.128152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-376b26ea-2c7a-4742-a353-6dcc9e64ef29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:03.171650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad64818-97da-40cf-a2fd-0b7daaca44b2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:03.171937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad64818-97da-40cf-a2fd-0b7daaca44b2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:03.172222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad64818-97da-40cf-a2fd-0b7daaca44b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:03.177165 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3254/6578] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:02 2022] POST /identity/v3/auth/tokens => generated 3360 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:03.198125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0ad64818-97da-40cf-a2fd-0b7daaca44b2 None None] MFA Rules not processed for user `0e63680e5f3844999db0a7d92d563bec`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:03.199016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0ad64818-97da-40cf-a2fd-0b7daaca44b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:03.207428 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3325/6579] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:03 2022] POST /identity/v3/auth/tokens => generated 3360 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:03.212142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b26ea-2c7a-4742-a353-6dcc9e64ef29 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:03.225224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b26ea-2c7a-4742-a353-6dcc9e64ef29 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:03.230782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4d521-c0ea-45c4-97f1-aaff60e2214b tempest-test_project-242949628 tempest-user-1875491406] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e63680e5f3844999db0a7d92d563bec', 'user_domain_id': '6ac14c31011a48d5a366935f0c6f3b5a', 'system_scope': None, 'project_id': '4ee98f23ccb44b7295a28a3c1c75659a', 'project_domain_id': '6ac14c31011a48d5a366935f0c6f3b5a', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:03.231415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4d521-c0ea-45c4-97f1-aaff60e2214b tempest-test_project-242949628 tempest-user-1875491406] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:03.232054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4d521-c0ea-45c4-97f1-aaff60e2214b tempest-test_project-242949628 tempest-user-1875491406] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:03.232487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4d521-c0ea-45c4-97f1-aaff60e2214b tempest-test_project-242949628 tempest-user-1875491406] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:03.242851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4d521-c0ea-45c4-97f1-aaff60e2214b tempest-test_project-242949628 tempest-user-1875491406] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:03.245037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4d521-c0ea-45c4-97f1-aaff60e2214b tempest-test_project-242949628 tempest-user-1875491406] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:03.249572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:03.261125 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3255/6580] 10.222.0.16 () {56 vars in 1284 bytes} [Sat Sep 3 21:13:03 2022] GET /identity/v3/auth/tokens => generated 3360 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:13:03.261992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad64818-97da-40cf-a2fd-0b7daaca44b2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:03.393536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad64818-97da-40cf-a2fd-0b7daaca44b2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:03.403534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ef44d9b07e74c4aad46aa9e7a5fc75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14433a868db406ca9e5fd3af1c82eb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:03.407389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:03.407837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:03.408127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] PATH_INFO: `/v3/users/0e63680e5f3844999db0a7d92d563bec` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:03.411441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorizing `identity:delete_user(user_id=0e63680e5f3844999db0a7d92d563bec)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:03.413707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:03.503196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e63680e5f3844999db0a7d92d563bec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:03.519488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:03.530888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e63680e5f3844999db0a7d92d563bec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:03.551970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e63680e5f3844999db0a7d92d563bec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:03.553354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e63680e5f3844999db0a7d92d563bec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:03.554811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e63680e5f3844999db0a7d92d563bec'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:03.604945 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3326/6581] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:03 2022] DELETE /identity/v3/users/0e63680e5f3844999db0a7d92d563bec => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:03.607343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4d521-c0ea-45c4-97f1-aaff60e2214b tempest-test_project-242949628 tempest-user-1875491406] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:03.640755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4d521-c0ea-45c4-97f1-aaff60e2214b tempest-test_project-242949628 tempest-user-1875491406] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:03.741869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ef44d9b07e74c4aad46aa9e7a5fc75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14433a868db406ca9e5fd3af1c82eb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:03.743461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:03.743461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:03.744052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] PATH_INFO: `/v3/projects/4ee98f23ccb44b7295a28a3c1c75659a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:03.745437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4ee98f23ccb44b7295a28a3c1c75659a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:03.748262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:03.824535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee98f23ccb44b7295a28a3c1c75659a', 'request_id': 'req-9d765587-04d7-4234-8edc-61fc2f5f2685'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:03.847888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee98f23ccb44b7295a28a3c1c75659a', 'request_id': 'req-9d765587-04d7-4234-8edc-61fc2f5f2685'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:03.861727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee98f23ccb44b7295a28a3c1c75659a', 'request_id': 'req-9d765587-04d7-4234-8edc-61fc2f5f2685'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:03.906452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] The token cache is being invalidate because project 4ee98f23ccb44b7295a28a3c1c75659a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:03.907032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:03.907969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:03.910790 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3256/6582] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:03 2022] DELETE /identity/v3/projects/4ee98f23ccb44b7295a28a3c1c75659a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:03.914883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:03.916377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:03.939005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0838c4b-6791-41d4-9b73-5f8f5d71adac tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:04.051164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ef44d9b07e74c4aad46aa9e7a5fc75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14433a868db406ca9e5fd3af1c82eb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:04.052802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:04.057131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:04.057723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] PATH_INFO: `/v3/domains/6ac14c31011a48d5a366935f0c6f3b5a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:04.063728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=6ac14c31011a48d5a366935f0c6f3b5a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:04.077757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:04.157838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '6ac14c31011a48d5a366935f0c6f3b5a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.159090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '6ac14c31011a48d5a366935f0c6f3b5a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.229469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '6ac14c31011a48d5a366935f0c6f3b5a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.230933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '6ac14c31011a48d5a366935f0c6f3b5a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.243501 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3327/6583] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:13:03 2022] PATCH /identity/v3/domains/6ac14c31011a48d5a366935f0c6f3b5a => generated 246 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:04.244726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:04.246489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:04.268580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d765587-04d7-4234-8edc-61fc2f5f2685 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:04.391758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ef44d9b07e74c4aad46aa9e7a5fc75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14433a868db406ca9e5fd3af1c82eb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:04.392531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:04.392838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:04.393156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] PATH_INFO: `/v3/domains/6ac14c31011a48d5a366935f0c6f3b5a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:04.410788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=6ac14c31011a48d5a366935f0c6f3b5a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:04.412984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:04.459354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '6ac14c31011a48d5a366935f0c6f3b5a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.460615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:04.472800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '6ac14c31011a48d5a366935f0c6f3b5a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.564459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac14c31011a48d5a366935f0c6f3b5a', 'request_id': 'req-d78b96ab-5894-466d-ae05-3a16b0b35681'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.574580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac14c31011a48d5a366935f0c6f3b5a', 'request_id': 'req-d78b96ab-5894-466d-ae05-3a16b0b35681'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.586281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac14c31011a48d5a366935f0c6f3b5a', 'request_id': 'req-d78b96ab-5894-466d-ae05-3a16b0b35681'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.604021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] The token cache is being invalidate because project 6ac14c31011a48d5a366935f0c6f3b5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:04.604591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.605530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.617531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '6ac14c31011a48d5a366935f0c6f3b5a', 'request_id': 'req-d78b96ab-5894-466d-ae05-3a16b0b35681'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.623049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '6ac14c31011a48d5a366935f0c6f3b5a', 'request_id': 'req-d78b96ab-5894-466d-ae05-3a16b0b35681'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.624983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d78b96ab-5894-466d-ae05-3a16b0b35681 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '6ac14c31011a48d5a366935f0c6f3b5a', 'request_id': 'req-d78b96ab-5894-466d-ae05-3a16b0b35681'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:04.655258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:04.657343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:04.666099 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3257/6584] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:13:04 2022] DELETE /identity/v3/domains/6ac14c31011a48d5a366935f0c6f3b5a => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:04.799254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddce019-be69-43f4-8bcd-ce02ed960e66 tempest-TestDefaultProjectId-286551088 tempest-TestDefaultProjectId-286551088-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:04.910095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:04.926313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:04.927478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:04.928122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:04.929471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] PATH_INFO: `/v3/users/b241663a9dce4ffcb78660b96596cfd0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:04.931361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b241663a9dce4ffcb78660b96596cfd0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:04.933683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:05.022901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b241663a9dce4ffcb78660b96596cfd0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:05.050501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b241663a9dce4ffcb78660b96596cfd0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:05.063930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b241663a9dce4ffcb78660b96596cfd0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:05.064865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b241663a9dce4ffcb78660b96596cfd0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:05.066017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b241663a9dce4ffcb78660b96596cfd0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:05.099368 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3328/6585] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:04 2022] DELETE /identity/v3/users/b241663a9dce4ffcb78660b96596cfd0 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:05.107097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671a58a-e34a-48d2-9f01-e7e0a68f1ea1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:05.107663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671a58a-e34a-48d2-9f01-e7e0a68f1ea1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:05.107968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671a58a-e34a-48d2-9f01-e7e0a68f1ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:05.164787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c671a58a-e34a-48d2-9f01-e7e0a68f1ea1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:05.168762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c671a58a-e34a-48d2-9f01-e7e0a68f1ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:05.282464 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3258/6586] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:05.332155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:05.334279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:05.444795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8fdbf-42fd-44ab-8406-5754c273d314 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:05.448614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d657fb-fac2-4e8a-bdfa-f638f7dc078b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:05.449305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d657fb-fac2-4e8a-bdfa-f638f7dc078b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:05.449604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d657fb-fac2-4e8a-bdfa-f638f7dc078b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:05.449888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d657fb-fac2-4e8a-bdfa-f638f7dc078b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:05.462056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d657fb-fac2-4e8a-bdfa-f638f7dc078b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:05.463245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d657fb-fac2-4e8a-bdfa-f638f7dc078b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:05.481842 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3329/6587] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:13:06.526726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671a58a-e34a-48d2-9f01-e7e0a68f1ea1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:06.528059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c671a58a-e34a-48d2-9f01-e7e0a68f1ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:06.547685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671a58a-e34a-48d2-9f01-e7e0a68f1ea1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:06.564688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:06.565398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:06.565688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:06.566684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] PATH_INFO: `/v3/projects/f4e2fd25078746cc803f0545e10fc176` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:06.567261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e2fd25078746cc803f0545e10fc176)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:06.583677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:06.722956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e2fd25078746cc803f0545e10fc176', 'request_id': 'req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:06.748292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e2fd25078746cc803f0545e10fc176', 'request_id': 'req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:06.765550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e2fd25078746cc803f0545e10fc176', 'request_id': 'req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:06.771140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:06.789158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] The token cache is being invalidate because project f4e2fd25078746cc803f0545e10fc176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:06.798456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:06.801155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:06.809736 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3259/6588] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:06 2022] DELETE /identity/v3/projects/f4e2fd25078746cc803f0545e10fc176 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:06.811877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d657fb-fac2-4e8a-bdfa-f638f7dc078b service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:06.835598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d657fb-fac2-4e8a-bdfa-f638f7dc078b service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:07.033203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387b84b-2e59-4425-8b43-df67beacf5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:07.033856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387b84b-2e59-4425-8b43-df67beacf5fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:07.034142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387b84b-2e59-4425-8b43-df67beacf5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:07.034419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387b84b-2e59-4425-8b43-df67beacf5fe admin admin] PATH_INFO: `/v3/users/1ef44d9b07e74c4aad46aa9e7a5fc75f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:07.035680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387b84b-2e59-4425-8b43-df67beacf5fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ef44d9b07e74c4aad46aa9e7a5fc75f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:07.038272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387b84b-2e59-4425-8b43-df67beacf5fe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:07.070154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:07.229953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e387b84b-2e59-4425-8b43-df67beacf5fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ef44d9b07e74c4aad46aa9e7a5fc75f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:07.262612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e387b84b-2e59-4425-8b43-df67beacf5fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ef44d9b07e74c4aad46aa9e7a5fc75f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:07.281213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e387b84b-2e59-4425-8b43-df67beacf5fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ef44d9b07e74c4aad46aa9e7a5fc75f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:07.282419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e387b84b-2e59-4425-8b43-df67beacf5fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ef44d9b07e74c4aad46aa9e7a5fc75f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:07.283331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e387b84b-2e59-4425-8b43-df67beacf5fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ef44d9b07e74c4aad46aa9e7a5fc75f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:07.314870 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3330/6589] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:06 2022] DELETE /identity/v3/users/1ef44d9b07e74c4aad46aa9e7a5fc75f => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:07.898259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:07.901359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:07.945537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c7ad-99d7-43b2-a19d-5cb7352e9bd4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:08.043280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5017242-997d-482c-bc03-853e69b4f47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:08.044673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5017242-997d-482c-bc03-853e69b4f47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:08.045156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5017242-997d-482c-bc03-853e69b4f47a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:08.045478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5017242-997d-482c-bc03-853e69b4f47a admin admin] PATH_INFO: `/v3/projects/b14433a868db406ca9e5fd3af1c82eb1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:08.047026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5017242-997d-482c-bc03-853e69b4f47a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b14433a868db406ca9e5fd3af1c82eb1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:08.050355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5017242-997d-482c-bc03-853e69b4f47a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:08.148629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a5017242-997d-482c-bc03-853e69b4f47a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b14433a868db406ca9e5fd3af1c82eb1', 'request_id': 'req-a5017242-997d-482c-bc03-853e69b4f47a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:08.160351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a5017242-997d-482c-bc03-853e69b4f47a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b14433a868db406ca9e5fd3af1c82eb1', 'request_id': 'req-a5017242-997d-482c-bc03-853e69b4f47a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:08.171181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a5017242-997d-482c-bc03-853e69b4f47a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b14433a868db406ca9e5fd3af1c82eb1', 'request_id': 'req-a5017242-997d-482c-bc03-853e69b4f47a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:08.188243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a5017242-997d-482c-bc03-853e69b4f47a admin admin] The token cache is being invalidate because project b14433a868db406ca9e5fd3af1c82eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:08.189430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a5017242-997d-482c-bc03-853e69b4f47a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:08.190304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a5017242-997d-482c-bc03-853e69b4f47a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:08.192504 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3260/6590] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:07 2022] DELETE /identity/v3/projects/b14433a868db406ca9e5fd3af1c82eb1 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:08.198633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d65c25-344f-4360-a1f8-4bdde4d78cfc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:08.199094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d65c25-344f-4360-a1f8-4bdde4d78cfc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:08.199453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d65c25-344f-4360-a1f8-4bdde4d78cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:08.235798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-54d65c25-344f-4360-a1f8-4bdde4d78cfc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:08.237475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-54d65c25-344f-4360-a1f8-4bdde4d78cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:08.330696 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3331/6591] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:13:08.341473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f17695-1b72-4357-8656-99e94e4b283e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:08.341473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f17695-1b72-4357-8656-99e94e4b283e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:08.341473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f17695-1b72-4357-8656-99e94e4b283e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:08.366121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-11f17695-1b72-4357-8656-99e94e4b283e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:08.367410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-11f17695-1b72-4357-8656-99e94e4b283e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:08.380027 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3261/6592] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:08.382186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d65c25-344f-4360-a1f8-4bdde4d78cfc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:08.394144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d65c25-344f-4360-a1f8-4bdde4d78cfc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:08.399176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04822c6-7b82-4778-8f81-1f48faecfb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:08.399931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04822c6-7b82-4778-8f81-1f48faecfb9c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:08.400292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04822c6-7b82-4778-8f81-1f48faecfb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:08.400633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04822c6-7b82-4778-8f81-1f48faecfb9c admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:08.401618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04822c6-7b82-4778-8f81-1f48faecfb9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:08.402155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04822c6-7b82-4778-8f81-1f48faecfb9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:08.404138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04822c6-7b82-4778-8f81-1f48faecfb9c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:08.423730 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3332/6593] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:08.424846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f17695-1b72-4357-8656-99e94e4b283e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:08.437246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f17695-1b72-4357-8656-99e94e4b283e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:08.442405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583f265-6e59-4d1e-b4c7-bc0ae3b90874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:08.443070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583f265-6e59-4d1e-b4c7-bc0ae3b90874 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:08.443360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583f265-6e59-4d1e-b4c7-bc0ae3b90874 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:08.443637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583f265-6e59-4d1e-b4c7-bc0ae3b90874 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:08.444401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583f265-6e59-4d1e-b4c7-bc0ae3b90874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:08.446271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583f265-6e59-4d1e-b4c7-bc0ae3b90874 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:08.475819 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3262/6594] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:08 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:08.480287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04822c6-7b82-4778-8f81-1f48faecfb9c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:08.495027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04822c6-7b82-4778-8f81-1f48faecfb9c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:08.576146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302fa52-70d4-4210-8e00-2a34ea0299ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:08.576780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302fa52-70d4-4210-8e00-2a34ea0299ff admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:08.577116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302fa52-70d4-4210-8e00-2a34ea0299ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:08.577501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302fa52-70d4-4210-8e00-2a34ea0299ff admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:08.578188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302fa52-70d4-4210-8e00-2a34ea0299ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:08.582277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302fa52-70d4-4210-8e00-2a34ea0299ff admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:08.605648 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3333/6595] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:08 2022] POST /identity/v3/users => generated 285 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:08.607738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583f265-6e59-4d1e-b4c7-bc0ae3b90874 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:08.618429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583f265-6e59-4d1e-b4c7-bc0ae3b90874 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:08.623942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67568ae7-2c27-44d8-aea3-9dad6362e0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:08.624651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67568ae7-2c27-44d8-aea3-9dad6362e0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:08.625039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67568ae7-2c27-44d8-aea3-9dad6362e0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:08.625384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67568ae7-2c27-44d8-aea3-9dad6362e0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:08.626099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67568ae7-2c27-44d8-aea3-9dad6362e0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:08.627883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67568ae7-2c27-44d8-aea3-9dad6362e0a5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:08.638950 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3263/6596] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:08.642942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302fa52-70d4-4210-8e00-2a34ea0299ff admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:08.649880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:08.653904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302fa52-70d4-4210-8e00-2a34ea0299ff admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:08.660866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6356671-d4a7-464e-8c5b-5e0c7be07192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:08.661943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6356671-d4a7-464e-8c5b-5e0c7be07192 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:08.662582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6356671-d4a7-464e-8c5b-5e0c7be07192 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:08.663268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6356671-d4a7-464e-8c5b-5e0c7be07192 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:08.664362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6356671-d4a7-464e-8c5b-5e0c7be07192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:08.666300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6356671-d4a7-464e-8c5b-5e0c7be07192 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:08.678286 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3334/6597] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:08.682807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67568ae7-2c27-44d8-aea3-9dad6362e0a5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:08.693745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67568ae7-2c27-44d8-aea3-9dad6362e0a5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:08.698385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d13c6-062d-44a2-81ab-d80ff380f337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:08.699044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d13c6-062d-44a2-81ab-d80ff380f337 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:08.699333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d13c6-062d-44a2-81ab-d80ff380f337 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:08.699613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d13c6-062d-44a2-81ab-d80ff380f337 admin admin] PATH_INFO: `/v3/projects/ad07973708e1421d8c7f83d9d425b5e7/users/5182f08ca3ea4714bcb6af09c02190fb/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:08.714580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d13c6-062d-44a2-81ab-d80ff380f337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad07973708e1421d8c7f83d9d425b5e7, user_id=5182f08ca3ea4714bcb6af09c02190fb, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:08.717401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d13c6-062d-44a2-81ab-d80ff380f337 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:08.740340 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3264/6598] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:08 2022] PUT /identity/v3/projects/ad07973708e1421d8c7f83d9d425b5e7/users/5182f08ca3ea4714bcb6af09c02190fb/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:08.752285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ac536-1bf9-4ae3-881c-3b499d461184 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:08.752712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ac536-1bf9-4ae3-881c-3b499d461184 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:08.753245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ac536-1bf9-4ae3-881c-3b499d461184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:08.801613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0b0ac536-1bf9-4ae3-881c-3b499d461184 None None] MFA Rules not processed for user `5182f08ca3ea4714bcb6af09c02190fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:08.805777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0b0ac536-1bf9-4ae3-881c-3b499d461184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:09.016566 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3335/6599] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:13:09.018372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d13c6-062d-44a2-81ab-d80ff380f337 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:09.042619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:09.117019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d13c6-062d-44a2-81ab-d80ff380f337 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:09.130061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083739d-7594-43d7-b67b-521c854fa49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:09.131146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083739d-7594-43d7-b67b-521c854fa49a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:09.131654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083739d-7594-43d7-b67b-521c854fa49a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:09.132177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083739d-7594-43d7-b67b-521c854fa49a admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:09.133152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083739d-7594-43d7-b67b-521c854fa49a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:09.138672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083739d-7594-43d7-b67b-521c854fa49a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:09.213210 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3265/6600] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:09 2022] POST /identity/v3/projects => generated 371 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:09.213646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ac536-1bf9-4ae3-881c-3b499d461184 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:09.229814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ac536-1bf9-4ae3-881c-3b499d461184 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:09.313578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d6794-da57-4098-b348-843a01dab18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:09.314541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d6794-da57-4098-b348-843a01dab18a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:09.314838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d6794-da57-4098-b348-843a01dab18a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:09.315242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d6794-da57-4098-b348-843a01dab18a admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:09.316055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d6794-da57-4098-b348-843a01dab18a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:09.318257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d6794-da57-4098-b348-843a01dab18a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:09.351721 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3336/6601] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:09 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:09.354148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083739d-7594-43d7-b67b-521c854fa49a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:09.368977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083739d-7594-43d7-b67b-521c854fa49a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:09.374701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012ecef-24a3-4b1c-b2ec-99ecea7cdc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:09.375578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012ecef-24a3-4b1c-b2ec-99ecea7cdc6f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:09.376005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012ecef-24a3-4b1c-b2ec-99ecea7cdc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:09.376374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012ecef-24a3-4b1c-b2ec-99ecea7cdc6f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:09.377159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012ecef-24a3-4b1c-b2ec-99ecea7cdc6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:09.378907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012ecef-24a3-4b1c-b2ec-99ecea7cdc6f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:09.391024 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3266/6602] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:13:09.396246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d6794-da57-4098-b348-843a01dab18a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:09.408840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d6794-da57-4098-b348-843a01dab18a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:09.414310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c6670-db86-4e27-88b5-97f203cda419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:09.415186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c6670-db86-4e27-88b5-97f203cda419 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:09.415663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c6670-db86-4e27-88b5-97f203cda419 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:09.416111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c6670-db86-4e27-88b5-97f203cda419 admin admin] PATH_INFO: `/v3/projects/4a453c8219ed404fbbba7c51e5a3c3e8/users/52a16490ce76474ebc912dc7ed27a706/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:09.436047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c6670-db86-4e27-88b5-97f203cda419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a453c8219ed404fbbba7c51e5a3c3e8, user_id=52a16490ce76474ebc912dc7ed27a706, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:09.438778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c6670-db86-4e27-88b5-97f203cda419 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:09.457287 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3337/6603] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:09 2022] PUT /identity/v3/projects/4a453c8219ed404fbbba7c51e5a3c3e8/users/52a16490ce76474ebc912dc7ed27a706/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:09.461442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012ecef-24a3-4b1c-b2ec-99ecea7cdc6f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:09.473325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012ecef-24a3-4b1c-b2ec-99ecea7cdc6f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:09.550872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc069f8-959a-4cea-b7d4-c6d4cff8b43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:09.552044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc069f8-959a-4cea-b7d4-c6d4cff8b43f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:09.552405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc069f8-959a-4cea-b7d4-c6d4cff8b43f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:09.552938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc069f8-959a-4cea-b7d4-c6d4cff8b43f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:09.553830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc069f8-959a-4cea-b7d4-c6d4cff8b43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:09.555368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc069f8-959a-4cea-b7d4-c6d4cff8b43f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:09.570775 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3267/6604] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:09.572030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c6670-db86-4e27-88b5-97f203cda419 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:09.602897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c6670-db86-4e27-88b5-97f203cda419 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:09.602897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb506566-223a-4c28-8090-8b59e246f098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:09.602897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb506566-223a-4c28-8090-8b59e246f098 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:09.602897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb506566-223a-4c28-8090-8b59e246f098 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:09.603633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb506566-223a-4c28-8090-8b59e246f098 admin admin] PATH_INFO: `/v3/projects/4a453c8219ed404fbbba7c51e5a3c3e8/users/52a16490ce76474ebc912dc7ed27a706/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:09.610379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb506566-223a-4c28-8090-8b59e246f098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a453c8219ed404fbbba7c51e5a3c3e8, user_id=52a16490ce76474ebc912dc7ed27a706, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:09.610379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb506566-223a-4c28-8090-8b59e246f098 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:09.656986 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3338/6605] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:09 2022] PUT /identity/v3/projects/4a453c8219ed404fbbba7c51e5a3c3e8/users/52a16490ce76474ebc912dc7ed27a706/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:09.658033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908b827-82fc-4272-adac-88aa8c7d2983 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:09.658336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908b827-82fc-4272-adac-88aa8c7d2983 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:09.658627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908b827-82fc-4272-adac-88aa8c7d2983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:09.699013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9908b827-82fc-4272-adac-88aa8c7d2983 None None] MFA Rules not processed for user `52a16490ce76474ebc912dc7ed27a706`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:09.700336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9908b827-82fc-4272-adac-88aa8c7d2983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:09.794369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb506566-223a-4c28-8090-8b59e246f098 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:09.804512 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3268/6606] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:13:09.811483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb506566-223a-4c28-8090-8b59e246f098 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:09.817989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67e896-1170-4a60-bfdf-f8fa515e6314 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:09.818598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67e896-1170-4a60-bfdf-f8fa515e6314 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:09.819054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67e896-1170-4a60-bfdf-f8fa515e6314 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:09.819347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67e896-1170-4a60-bfdf-f8fa515e6314 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:09.820053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67e896-1170-4a60-bfdf-f8fa515e6314 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:09.821794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67e896-1170-4a60-bfdf-f8fa515e6314 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:09.853833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908b827-82fc-4272-adac-88aa8c7d2983 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:09.860971 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3339/6607] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:09 2022] POST /identity/v3/services => generated 289 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:09.867796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908b827-82fc-4272-adac-88aa8c7d2983 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:09.902984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:09.910682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987142d2-8adb-4234-bdd9-ae5a5e95297d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:09.911286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987142d2-8adb-4234-bdd9-ae5a5e95297d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:09.911575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987142d2-8adb-4234-bdd9-ae5a5e95297d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:09.911870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987142d2-8adb-4234-bdd9-ae5a5e95297d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:09.912774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987142d2-8adb-4234-bdd9-ae5a5e95297d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:09.924560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987142d2-8adb-4234-bdd9-ae5a5e95297d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:09.966657 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-987142d2-8adb-4234-bdd9-ae5a5e95297d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-342946949. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:13:09.966657 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:13:09.966657 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:10.020352 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3269/6608] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Sep 3 21:13:09 2022] POST /identity/v3/endpoints => generated 359 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:10.024091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67e896-1170-4a60-bfdf-f8fa515e6314 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.049639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67e896-1170-4a60-bfdf-f8fa515e6314 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:10.089205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38163b-7700-4fd1-b12b-7a7d5b0c6153 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:10.090048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38163b-7700-4fd1-b12b-7a7d5b0c6153 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:10.090458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38163b-7700-4fd1-b12b-7a7d5b0c6153 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:10.090816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38163b-7700-4fd1-b12b-7a7d5b0c6153 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/regions/tempest-region-342946949` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:10.091696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38163b-7700-4fd1-b12b-7a7d5b0c6153 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-342946949)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:10.095012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38163b-7700-4fd1-b12b-7a7d5b0c6153 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:10.099368 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3340/6609] 10.222.0.16 () {58 vars in 1203 bytes} [Sat Sep 3 21:13:10 2022] GET /identity/v3/regions/tempest-region-342946949 => generated 178 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:10.104108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987142d2-8adb-4234-bdd9-ae5a5e95297d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.121698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987142d2-8adb-4234-bdd9-ae5a5e95297d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:10.136852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62cfb4-e0c3-49ca-94ea-7e0df1daee21 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:10.137458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62cfb4-e0c3-49ca-94ea-7e0df1daee21 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:10.137755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62cfb4-e0c3-49ca-94ea-7e0df1daee21 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:10.138083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62cfb4-e0c3-49ca-94ea-7e0df1daee21 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:10.138775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e62cfb4-e0c3-49ca-94ea-7e0df1daee21 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:10.140539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e62cfb4-e0c3-49ca-94ea-7e0df1daee21 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:10.186156 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3270/6610] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:10 2022] POST /identity/v3/services => generated 288 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:10.188044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38163b-7700-4fd1-b12b-7a7d5b0c6153 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.203436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38163b-7700-4fd1-b12b-7a7d5b0c6153 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:10.214201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:10.229373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8c204-3ef2-44d1-a466-c1035b8cf161 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:10.230380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8c204-3ef2-44d1-a466-c1035b8cf161 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:10.231417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8c204-3ef2-44d1-a466-c1035b8cf161 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:10.232496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8c204-3ef2-44d1-a466-c1035b8cf161 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:10.233368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8c204-3ef2-44d1-a466-c1035b8cf161 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:10.236355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8c204-3ef2-44d1-a466-c1035b8cf161 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:10.287850 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-18a8c204-3ef2-44d1-a466-c1035b8cf161 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-75693915. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:13:10.287850 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:13:10.287850 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:10.349152 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3341/6611] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Sep 3 21:13:10 2022] POST /identity/v3/endpoints => generated 359 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:10.354185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62cfb4-e0c3-49ca-94ea-7e0df1daee21 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.385423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62cfb4-e0c3-49ca-94ea-7e0df1daee21 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:10.424734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9549c-83fa-4d53-ada8-ea947fe4a25d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:10.425707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9549c-83fa-4d53-ada8-ea947fe4a25d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:10.426397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9549c-83fa-4d53-ada8-ea947fe4a25d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:10.426914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9549c-83fa-4d53-ada8-ea947fe4a25d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/regions/tempest-region-75693915` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:10.427770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9549c-83fa-4d53-ada8-ea947fe4a25d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-75693915)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:10.429114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9549c-83fa-4d53-ada8-ea947fe4a25d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:10.440137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8c204-3ef2-44d1-a466-c1035b8cf161 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.448076 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3271/6612] 10.222.0.16 () {58 vars in 1200 bytes} [Sat Sep 3 21:13:10 2022] GET /identity/v3/regions/tempest-region-75693915 => generated 176 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:10.457680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8c204-3ef2-44d1-a466-c1035b8cf161 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:10.463294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04339c-4624-4c12-94c8-6138329fcdb6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:10.464932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04339c-4624-4c12-94c8-6138329fcdb6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:10.464932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04339c-4624-4c12-94c8-6138329fcdb6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:10.464932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04339c-4624-4c12-94c8-6138329fcdb6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:10.465385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04339c-4624-4c12-94c8-6138329fcdb6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:10.468141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04339c-4624-4c12-94c8-6138329fcdb6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:10.492261 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-7f04339c-4624-4c12-94c8-6138329fcdb6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1318797494. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:13:10.492261 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:13:10.492261 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:10.540479 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3342/6613] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Sep 3 21:13:10 2022] POST /identity/v3/endpoints => generated 361 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:10.541961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9549c-83fa-4d53-ada8-ea947fe4a25d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.570521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9549c-83fa-4d53-ada8-ea947fe4a25d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:10.601299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ebbf7-1152-40d3-8c21-70fa179a82ec tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:10.602833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ebbf7-1152-40d3-8c21-70fa179a82ec tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:10.603315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ebbf7-1152-40d3-8c21-70fa179a82ec tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:10.603810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ebbf7-1152-40d3-8c21-70fa179a82ec tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/regions/tempest-region-1318797494` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:10.604640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ebbf7-1152-40d3-8c21-70fa179a82ec tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1318797494)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:10.606291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ebbf7-1152-40d3-8c21-70fa179a82ec tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:10.613851 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3272/6614] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Sep 3 21:13:10 2022] GET /identity/v3/regions/tempest-region-1318797494 => generated 180 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:13:10.615171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04339c-4624-4c12-94c8-6138329fcdb6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.626678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04339c-4624-4c12-94c8-6138329fcdb6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:10.632120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabb1be-266c-4fe5-bc2c-f39db8774067 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:10.632743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabb1be-266c-4fe5-bc2c-f39db8774067 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:10.633112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabb1be-266c-4fe5-bc2c-f39db8774067 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:10.633401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabb1be-266c-4fe5-bc2c-f39db8774067 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:10.634035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabb1be-266c-4fe5-bc2c-f39db8774067 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:10.635231 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-aaabb1be-266c-4fe5-bc2c-f39db8774067 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:10.635231 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:10.635231 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:10.636138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabb1be-266c-4fe5-bc2c-f39db8774067 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:10.651058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ebbf7-1152-40d3-8c21-70fa179a82ec tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.651177 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3343/6615] 10.222.0.16 () {58 vars in 1134 bytes} [Sat Sep 3 21:13:10 2022] GET /identity/v3/endpoints => generated 4393 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:10.663268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ebbf7-1152-40d3-8c21-70fa179a82ec tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:10.678695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4dfe6-cf20-4563-a4cf-3a3312a23a0b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:10.679212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4dfe6-cf20-4563-a4cf-3a3312a23a0b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:10.679567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4dfe6-cf20-4563-a4cf-3a3312a23a0b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:10.679849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4dfe6-cf20-4563-a4cf-3a3312a23a0b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints/ebe658c04fc14848bc03e96fd571a3f2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:10.691117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4dfe6-cf20-4563-a4cf-3a3312a23a0b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:get_endpoint(endpoint_id=ebe658c04fc14848bc03e96fd571a3f2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:10.691117 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-04f4dfe6-cf20-4563-a4cf-3a3312a23a0b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:10.691117 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:10.691117 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:10.694017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4dfe6-cf20-4563-a4cf-3a3312a23a0b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:10.696291 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3273/6616] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:10 2022] GET /identity/v3/endpoints/ebe658c04fc14848bc03e96fd571a3f2 => generated 361 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:10.701705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabb1be-266c-4fe5-bc2c-f39db8774067 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.713422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabb1be-266c-4fe5-bc2c-f39db8774067 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:10.719307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773711f7-d5cc-42d9-a453-dc04ca799e1b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:10.720042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773711f7-d5cc-42d9-a453-dc04ca799e1b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:10.720394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773711f7-d5cc-42d9-a453-dc04ca799e1b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:10.720830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773711f7-d5cc-42d9-a453-dc04ca799e1b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints/ebe658c04fc14848bc03e96fd571a3f2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:10.722250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773711f7-d5cc-42d9-a453-dc04ca799e1b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=ebe658c04fc14848bc03e96fd571a3f2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:10.724062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773711f7-d5cc-42d9-a453-dc04ca799e1b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:10.734566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-773711f7-d5cc-42d9-a453-dc04ca799e1b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'ebe658c04fc14848bc03e96fd571a3f2', 'request_id': 'req-773711f7-d5cc-42d9-a453-dc04ca799e1b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:10.762914 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3344/6617] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Sep 3 21:13:10 2022] DELETE /identity/v3/endpoints/ebe658c04fc14848bc03e96fd571a3f2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:10.767254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4dfe6-cf20-4563-a4cf-3a3312a23a0b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.802435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4dfe6-cf20-4563-a4cf-3a3312a23a0b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:10.819484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f020143-bf08-432d-8c6b-0f711d6ce897 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:10.820101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f020143-bf08-432d-8c6b-0f711d6ce897 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:10.820391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f020143-bf08-432d-8c6b-0f711d6ce897 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:10.820783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f020143-bf08-432d-8c6b-0f711d6ce897 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:10.821471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f020143-bf08-432d-8c6b-0f711d6ce897 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:10.822495 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-6f020143-bf08-432d-8c6b-0f711d6ce897 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:10.822495 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:10.822495 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:10.823393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f020143-bf08-432d-8c6b-0f711d6ce897 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:10.833457 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3274/6618] 10.222.0.16 () {58 vars in 1134 bytes} [Sat Sep 3 21:13:10 2022] GET /identity/v3/endpoints => generated 4045 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:10.838273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773711f7-d5cc-42d9-a453-dc04ca799e1b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.852327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773711f7-d5cc-42d9-a453-dc04ca799e1b tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:10.859386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cc2bb-03af-48cf-9a89-4a6baa014af2 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:10.859939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cc2bb-03af-48cf-9a89-4a6baa014af2 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:10.860297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cc2bb-03af-48cf-9a89-4a6baa014af2 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:10.860645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cc2bb-03af-48cf-9a89-4a6baa014af2 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints/ebe658c04fc14848bc03e96fd571a3f2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:10.867534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cc2bb-03af-48cf-9a89-4a6baa014af2 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-1f6cc2bb-03af-48cf-9a89-4a6baa014af2 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Could not find endpoint: ebe658c04fc14848bc03e96fd571a3f2.: keystone.exception.EndpointNotFound: Could not find endpoint: ebe658c04fc14848bc03e96fd571a3f2. Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 236, in _get_endpoint Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return session.query(Endpoint).filter_by(id=endpoint_id).one() Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._iter().one() Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._only_one_row( Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:10.876489 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 228, in get_endpoint Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.driver.get_endpoint(endpoint_id) Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 242, in get_endpoint Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_endpoint(session, endpoint_id).to_dict() Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 238, in _get_endpoint Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: ebe658c04fc14848bc03e96fd571a3f2. Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:13:10.877959 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 113, in delete Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_endpoint') Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:13:10.879328 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 230, in get_endpoint Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: ebe658c04fc14848bc03e96fd571a3f2. Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:10.880505 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3345/6619] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Sep 3 21:13:10 2022] DELETE /identity/v3/endpoints/ebe658c04fc14848bc03e96fd571a3f2 => generated 114 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 03 21:13:10.886024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f020143-bf08-432d-8c6b-0f711d6ce897 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.905895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f020143-bf08-432d-8c6b-0f711d6ce897 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:10.917946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077eb89-8ab6-493a-919d-88291e6ba5fa tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:10.919423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077eb89-8ab6-493a-919d-88291e6ba5fa tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:10.920225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077eb89-8ab6-493a-919d-88291e6ba5fa tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:10.920690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077eb89-8ab6-493a-919d-88291e6ba5fa tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/regions/tempest-region-1318797494` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:10.921305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077eb89-8ab6-493a-919d-88291e6ba5fa tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1318797494)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:10.923372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077eb89-8ab6-493a-919d-88291e6ba5fa tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:10.953844 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-9077eb89-8ab6-493a-919d-88291e6ba5fa tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1318797494. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:13:10.953844 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:13:10.953844 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:10.966000 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3275/6620] 10.222.0.16 () {58 vars in 1209 bytes} [Sat Sep 3 21:13:10 2022] DELETE /identity/v3/regions/tempest-region-1318797494 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:10.971496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cc2bb-03af-48cf-9a89-4a6baa014af2 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:10.991257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cc2bb-03af-48cf-9a89-4a6baa014af2 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.024624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dac67-6b92-4c88-9db3-87d6930d145a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.025798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dac67-6b92-4c88-9db3-87d6930d145a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.026219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dac67-6b92-4c88-9db3-87d6930d145a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.026496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dac67-6b92-4c88-9db3-87d6930d145a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.027137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3dac67-6b92-4c88-9db3-87d6930d145a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.028983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3dac67-6b92-4c88-9db3-87d6930d145a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.038090 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3346/6621] 10.222.0.16 () {58 vars in 1134 bytes} [Sat Sep 3 21:13:10 2022] GET /identity/v3/endpoints => generated 4045 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:11.042262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077eb89-8ab6-493a-919d-88291e6ba5fa tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.055638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077eb89-8ab6-493a-919d-88291e6ba5fa tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.062053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561973e1-de38-493c-a0f3-da67a84dcf31 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.062744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561973e1-de38-493c-a0f3-da67a84dcf31 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.063095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561973e1-de38-493c-a0f3-da67a84dcf31 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.063460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561973e1-de38-493c-a0f3-da67a84dcf31 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.064211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561973e1-de38-493c-a0f3-da67a84dcf31 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Adding query filter params (service_id=92a4c0da6c914a5c8c8a658c7a83b058) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:11.064611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561973e1-de38-493c-a0f3-da67a84dcf31 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.066409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561973e1-de38-493c-a0f3-da67a84dcf31 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.073567 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3276/6622] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Sep 3 21:13:11 2022] GET /identity/v3/endpoints?service_id=92a4c0da6c914a5c8c8a658c7a83b058 => generated 501 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:11.078098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dac67-6b92-4c88-9db3-87d6930d145a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.091240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dac67-6b92-4c88-9db3-87d6930d145a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.097576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cbcfc-dd3b-4af8-82fe-759444c34281 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.098799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cbcfc-dd3b-4af8-82fe-759444c34281 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.099344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cbcfc-dd3b-4af8-82fe-759444c34281 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.099824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cbcfc-dd3b-4af8-82fe-759444c34281 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.101640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cbcfc-dd3b-4af8-82fe-759444c34281 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Adding query filter params (service_id=e0a4a0e2b8b9457d8e315a0a79129250) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:11.103680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cbcfc-dd3b-4af8-82fe-759444c34281 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.107361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cbcfc-dd3b-4af8-82fe-759444c34281 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.120982 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3347/6623] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Sep 3 21:13:11 2022] GET /identity/v3/endpoints?service_id=e0a4a0e2b8b9457d8e315a0a79129250 => generated 501 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:13:11.123083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561973e1-de38-493c-a0f3-da67a84dcf31 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.136486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561973e1-de38-493c-a0f3-da67a84dcf31 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.144064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd046b-38c3-4344-bf9e-344492aff0e6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.145473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd046b-38c3-4344-bf9e-344492aff0e6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.146712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd046b-38c3-4344-bf9e-344492aff0e6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.147323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd046b-38c3-4344-bf9e-344492aff0e6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.147989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd046b-38c3-4344-bf9e-344492aff0e6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Adding query filter params (interface=public) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:11.148315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd046b-38c3-4344-bf9e-344492aff0e6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.150244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd046b-38c3-4344-bf9e-344492aff0e6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.166710 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3277/6624] 10.222.0.16 () {58 vars in 1167 bytes} [Sat Sep 3 21:13:11 2022] GET /identity/v3/endpoints?interface=public => generated 3404 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:11.167291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cbcfc-dd3b-4af8-82fe-759444c34281 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.180477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cbcfc-dd3b-4af8-82fe-759444c34281 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.187050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef02573-a125-4080-bcf6-28caf10e80b1 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.187671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef02573-a125-4080-bcf6-28caf10e80b1 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.187956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef02573-a125-4080-bcf6-28caf10e80b1 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.188362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef02573-a125-4080-bcf6-28caf10e80b1 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.189067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef02573-a125-4080-bcf6-28caf10e80b1 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Adding query filter params (interface=internal) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:11.189695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef02573-a125-4080-bcf6-28caf10e80b1 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.191030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef02573-a125-4080-bcf6-28caf10e80b1 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.201160 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3348/6625] 10.222.0.16 () {58 vars in 1171 bytes} [Sat Sep 3 21:13:11 2022] GET /identity/v3/endpoints?interface=internal => generated 476 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:13:11.205559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd046b-38c3-4344-bf9e-344492aff0e6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.220726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd046b-38c3-4344-bf9e-344492aff0e6 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.228672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f1bef-27e8-45df-a124-fe939152dccb tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.230559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f1bef-27e8-45df-a124-fe939152dccb tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.230946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f1bef-27e8-45df-a124-fe939152dccb tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.231289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f1bef-27e8-45df-a124-fe939152dccb tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/services` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.232312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f1bef-27e8-45df-a124-fe939152dccb tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.235519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f1bef-27e8-45df-a124-fe939152dccb tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.259013 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3278/6626] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:11 2022] POST /identity/v3/services => generated 287 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:11.263794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef02573-a125-4080-bcf6-28caf10e80b1 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.276718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef02573-a125-4080-bcf6-28caf10e80b1 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.300159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257880d-ce69-4ace-b6f1-24b352416416 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.301050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257880d-ce69-4ace-b6f1-24b352416416 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.301401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257880d-ce69-4ace-b6f1-24b352416416 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.301795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257880d-ce69-4ace-b6f1-24b352416416 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.304661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f1bef-27e8-45df-a124-fe939152dccb tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.307547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6a3f1bef-27e8-45df-a124-fe939152dccb tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:11.315104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257880d-ce69-4ace-b6f1-24b352416416 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.319557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257880d-ce69-4ace-b6f1-24b352416416 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.349897 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-6257880d-ce69-4ace-b6f1-24b352416416 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-125537679. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:13:11.349897 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:13:11.349897 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:11.389514 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3349/6627] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Sep 3 21:13:11 2022] POST /identity/v3/endpoints => generated 359 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:11.394057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257880d-ce69-4ace-b6f1-24b352416416 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.439478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257880d-ce69-4ace-b6f1-24b352416416 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.446787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a26381-9d41-4e84-8fee-ee67379b9076 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.448412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a26381-9d41-4e84-8fee-ee67379b9076 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.449133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a26381-9d41-4e84-8fee-ee67379b9076 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.450338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a26381-9d41-4e84-8fee-ee67379b9076 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/regions/tempest-region-125537679` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.451212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a26381-9d41-4e84-8fee-ee67379b9076 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-125537679)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.453107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a26381-9d41-4e84-8fee-ee67379b9076 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.456287 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3350/6628] 10.222.0.16 () {58 vars in 1203 bytes} [Sat Sep 3 21:13:11 2022] GET /identity/v3/regions/tempest-region-125537679 => generated 178 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:11.461412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a26381-9d41-4e84-8fee-ee67379b9076 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.465407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:11.472496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f1bef-27e8-45df-a124-fe939152dccb tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.488961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a26381-9d41-4e84-8fee-ee67379b9076 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.495678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd54f98-d726-4e87-afc4-f0849478e42e tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.496300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd54f98-d726-4e87-afc4-f0849478e42e tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.496640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd54f98-d726-4e87-afc4-f0849478e42e tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.496959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd54f98-d726-4e87-afc4-f0849478e42e tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints/d9da09f4827446c288b2c320ac4046a4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.502778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.503936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.504494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.505039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] PATH_INFO: `/v3/users/be1f616143114ef2bc2036efb0d11eef` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.506478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1f616143114ef2bc2036efb0d11eef)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.509832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.515613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd54f98-d726-4e87-afc4-f0849478e42e tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=d9da09f4827446c288b2c320ac4046a4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.516767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd54f98-d726-4e87-afc4-f0849478e42e tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.537157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:11.556069 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-dfd54f98-d726-4e87-afc4-f0849478e42e tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-2104347922. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:13:11.556069 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:13:11.556069 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:11.603331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1f616143114ef2bc2036efb0d11eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:11.618951 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3351/6629] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Sep 3 21:13:11 2022] PATCH /identity/v3/endpoints/d9da09f4827446c288b2c320ac4046a4 => generated 364 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:11.624810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd54f98-d726-4e87-afc4-f0849478e42e tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.636631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1f616143114ef2bc2036efb0d11eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:11.639161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1f616143114ef2bc2036efb0d11eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:11.656006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1f616143114ef2bc2036efb0d11eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:11.659476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd54f98-d726-4e87-afc4-f0849478e42e tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.666457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe22bd5-3402-4393-974a-4fde8c357475 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.666457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe22bd5-3402-4393-974a-4fde8c357475 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.666457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe22bd5-3402-4393-974a-4fde8c357475 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.666961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe22bd5-3402-4393-974a-4fde8c357475 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/regions/tempest-region-2104347922` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.667555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe22bd5-3402-4393-974a-4fde8c357475 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-2104347922)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.668865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe22bd5-3402-4393-974a-4fde8c357475 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.672962 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3352/6630] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Sep 3 21:13:11 2022] GET /identity/v3/regions/tempest-region-2104347922 => generated 180 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:11.677378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe22bd5-3402-4393-974a-4fde8c357475 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.678897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1f616143114ef2bc2036efb0d11eef'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:11.680441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bbe22bd5-3402-4393-974a-4fde8c357475 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:11.691013 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3279/6631] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:11 2022] DELETE /identity/v3/users/be1f616143114ef2bc2036efb0d11eef => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:11.701599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe22bd5-3402-4393-974a-4fde8c357475 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.801774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c92cb-18c2-4413-bf74-d4880da00c84 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.802978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c92cb-18c2-4413-bf74-d4880da00c84 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.803636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c92cb-18c2-4413-bf74-d4880da00c84 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.804850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c92cb-18c2-4413-bf74-d4880da00c84 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints/d9da09f4827446c288b2c320ac4046a4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.812328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c92cb-18c2-4413-bf74-d4880da00c84 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=d9da09f4827446c288b2c320ac4046a4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.815373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c92cb-18c2-4413-bf74-d4880da00c84 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.833852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9b9c92cb-18c2-4413-bf74-d4880da00c84 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'd9da09f4827446c288b2c320ac4046a4', 'request_id': 'req-9b9c92cb-18c2-4413-bf74-d4880da00c84'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:11.863122 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3353/6632] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Sep 3 21:13:11 2022] DELETE /identity/v3/endpoints/d9da09f4827446c288b2c320ac4046a4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:11.867588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.883304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a490eb-f710-4e5e-be0e-ac5259da48e7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:11.910925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fb95a-d8e3-45c7-8769-d4c7d4d26d64 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:11.911576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb95a-d8e3-45c7-8769-d4c7d4d26d64 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:11.911863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb95a-d8e3-45c7-8769-d4c7d4d26d64 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:11.912707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb95a-d8e3-45c7-8769-d4c7d4d26d64 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/regions/tempest-region-2104347922` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:11.914973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fb95a-d8e3-45c7-8769-d4c7d4d26d64 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-2104347922)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:11.916622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fb95a-d8e3-45c7-8769-d4c7d4d26d64 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:11.957803 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-312fb95a-d8e3-45c7-8769-d4c7d4d26d64 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-2104347922. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:13:11.957803 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:13:11.957803 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:11.959717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c92cb-18c2-4413-bf74-d4880da00c84 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:11.961772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9b9c92cb-18c2-4413-bf74-d4880da00c84 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:11.997260 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3280/6633] 10.222.0.16 () {58 vars in 1209 bytes} [Sat Sep 3 21:13:11 2022] DELETE /identity/v3/regions/tempest-region-2104347922 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:11.998628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fb95a-d8e3-45c7-8769-d4c7d4d26d64 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:12.051869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fb95a-d8e3-45c7-8769-d4c7d4d26d64 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:12.063584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e63549-94a6-4830-95e7-31fc0482e4df tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:12.064749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e63549-94a6-4830-95e7-31fc0482e4df tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:12.065088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e63549-94a6-4830-95e7-31fc0482e4df tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:12.065422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e63549-94a6-4830-95e7-31fc0482e4df tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/regions/tempest-region-125537679` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:12.100652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e63549-94a6-4830-95e7-31fc0482e4df tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-125537679)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:12.103480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e63549-94a6-4830-95e7-31fc0482e4df tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:12.156183 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-61e63549-94a6-4830-95e7-31fc0482e4df tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-125537679. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:13:12.156183 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:13:12.156183 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:12.179527 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3281/6634] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Sep 3 21:13:11 2022] DELETE /identity/v3/regions/tempest-region-125537679 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:12.183117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e63549-94a6-4830-95e7-31fc0482e4df tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:12.210427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c92cb-18c2-4413-bf74-d4880da00c84 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:12.226131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e63549-94a6-4830-95e7-31fc0482e4df tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:12.233897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7c0e7-c8a5-4516-bb0e-11621c702ac7 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:12.234694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:12.236210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:12.237165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7c0e7-c8a5-4516-bb0e-11621c702ac7 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:12.237687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:12.238208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7c0e7-c8a5-4516-bb0e-11621c702ac7 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:12.238705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf admin admin] PATH_INFO: `/v3/projects/77941b15bab7435fa20336fa6dd26d58` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:12.239209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7c0e7-c8a5-4516-bb0e-11621c702ac7 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/services/8bf2f4349160446d9afbcdfc5994e112` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:12.240659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7c0e7-c8a5-4516-bb0e-11621c702ac7 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:delete_service(service_id=8bf2f4349160446d9afbcdfc5994e112)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:12.241621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=77941b15bab7435fa20336fa6dd26d58)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:12.245297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7c0e7-c8a5-4516-bb0e-11621c702ac7 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:12.248458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:12.286894 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3282/6635] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:12 2022] DELETE /identity/v3/services/8bf2f4349160446d9afbcdfc5994e112 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:12.293448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7c0e7-c8a5-4516-bb0e-11621c702ac7 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:12.315139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:UPDATE=1,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:12.354572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7c0e7-c8a5-4516-bb0e-11621c702ac7 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:12.371158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77941b15bab7435fa20336fa6dd26d58', 'request_id': 'req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:12.390126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77941b15bab7435fa20336fa6dd26d58', 'request_id': 'req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:12.402057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77941b15bab7435fa20336fa6dd26d58', 'request_id': 'req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:12.412175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:12.424114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf admin admin] The token cache is being invalidate because project 77941b15bab7435fa20336fa6dd26d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:12.424510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:12.425235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d6ca0e23-fb98-4896-bfa8-0db52669e4bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:12.426839 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3354/6636] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:11 2022] DELETE /identity/v3/projects/77941b15bab7435fa20336fa6dd26d58 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:12.431803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ead08-7f4a-4764-96cf-144c5456f661 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:12.432168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ead08-7f4a-4764-96cf-144c5456f661 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:12.432461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ead08-7f4a-4764-96cf-144c5456f661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:12.476868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-465ead08-7f4a-4764-96cf-144c5456f661 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:12.478424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-465ead08-7f4a-4764-96cf-144c5456f661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:12.493166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265c38b-622f-41e3-b842-6d68cb6a91e4 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:12.493967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265c38b-622f-41e3-b842-6d68cb6a91e4 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:12.494317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265c38b-622f-41e3-b842-6d68cb6a91e4 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:12.494741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265c38b-622f-41e3-b842-6d68cb6a91e4 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints/03d2fd77f75e4290b1b916200148e48d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:12.509130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265c38b-622f-41e3-b842-6d68cb6a91e4 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=03d2fd77f75e4290b1b916200148e48d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:12.512644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265c38b-622f-41e3-b842-6d68cb6a91e4 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:12.530073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f265c38b-622f-41e3-b842-6d68cb6a91e4 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '03d2fd77f75e4290b1b916200148e48d', 'request_id': 'req-f265c38b-622f-41e3-b842-6d68cb6a91e4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:12.550342 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3283/6637] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Sep 3 21:13:12 2022] DELETE /identity/v3/endpoints/03d2fd77f75e4290b1b916200148e48d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:12.554134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265c38b-622f-41e3-b842-6d68cb6a91e4 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:12.555995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f265c38b-622f-41e3-b842-6d68cb6a91e4 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:12.602235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265c38b-622f-41e3-b842-6d68cb6a91e4 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:12.616252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584e211-6664-476b-bc8f-d66c7d9cfa00 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:12.618720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584e211-6664-476b-bc8f-d66c7d9cfa00 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:12.619039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584e211-6664-476b-bc8f-d66c7d9cfa00 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:12.619372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584e211-6664-476b-bc8f-d66c7d9cfa00 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/regions/tempest-region-75693915` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:12.624066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584e211-6664-476b-bc8f-d66c7d9cfa00 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-75693915)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:12.627805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584e211-6664-476b-bc8f-d66c7d9cfa00 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:12.650723 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3355/6638] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:12 2022] POST /identity/v3/auth/tokens => generated 3685 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:12.654759 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-6584e211-6664-476b-bc8f-d66c7d9cfa00 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-75693915. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:13:12.654759 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:13:12.654759 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:12.662090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd8698-a467-442d-adc1-79d5d427f8b4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:12.662758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd8698-a467-442d-adc1-79d5d427f8b4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:12.663401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd8698-a467-442d-adc1-79d5d427f8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:12.686082 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3284/6639] 10.222.0.16 () {58 vars in 1203 bytes} [Sat Sep 3 21:13:12 2022] DELETE /identity/v3/regions/tempest-region-75693915 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:12.690797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584e211-6664-476b-bc8f-d66c7d9cfa00 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:12.739352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584e211-6664-476b-bc8f-d66c7d9cfa00 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:12.741279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-79bd8698-a467-442d-adc1-79d5d427f8b4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:12.743174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-79bd8698-a467-442d-adc1-79d5d427f8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:12.758671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8ecfe-1bd7-4b21-a074-ee5d8dd11140 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:12.759547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ecfe-1bd7-4b21-a074-ee5d8dd11140 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:12.760272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ecfe-1bd7-4b21-a074-ee5d8dd11140 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:12.761454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ecfe-1bd7-4b21-a074-ee5d8dd11140 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/services/e0a4a0e2b8b9457d8e315a0a79129250` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:12.762147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8ecfe-1bd7-4b21-a074-ee5d8dd11140 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:delete_service(service_id=e0a4a0e2b8b9457d8e315a0a79129250)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:12.764011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8ecfe-1bd7-4b21-a074-ee5d8dd11140 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:12.803295 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3285/6640] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:12 2022] DELETE /identity/v3/services/e0a4a0e2b8b9457d8e315a0a79129250 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:12.809434 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3356/6641] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:12 2022] POST /identity/v3/auth/tokens => generated 3685 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:13:12.810737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8ecfe-1bd7-4b21-a074-ee5d8dd11140 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:12.813810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd8698-a467-442d-adc1-79d5d427f8b4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:12.844830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd8698-a467-442d-adc1-79d5d427f8b4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:12.875640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8ecfe-1bd7-4b21-a074-ee5d8dd11140 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:12.883561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac642be5-77d3-433a-9088-0d5b0aeea93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:12.884505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac642be5-77d3-433a-9088-0d5b0aeea93d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:12.885268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac642be5-77d3-433a-9088-0d5b0aeea93d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:12.886130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac642be5-77d3-433a-9088-0d5b0aeea93d admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:12.887085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac642be5-77d3-433a-9088-0d5b0aeea93d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:12.887732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac642be5-77d3-433a-9088-0d5b0aeea93d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:12.890960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac642be5-77d3-433a-9088-0d5b0aeea93d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:12.893144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f649f-ea05-4e97-ace3-97f40c36641a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:12.893144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f649f-ea05-4e97-ace3-97f40c36641a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:12.893144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f649f-ea05-4e97-ace3-97f40c36641a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:12.893490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f649f-ea05-4e97-ace3-97f40c36641a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/endpoints/e29eebdc508b46d08b2855c2dca9e082` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:12.913511 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3357/6642] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:12.917352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac642be5-77d3-433a-9088-0d5b0aeea93d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:12.919303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f649f-ea05-4e97-ace3-97f40c36641a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=e29eebdc508b46d08b2855c2dca9e082)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:12.922739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f649f-ea05-4e97-ace3-97f40c36641a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:12.935720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a09f649f-ea05-4e97-ace3-97f40c36641a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'e29eebdc508b46d08b2855c2dca9e082', 'request_id': 'req-a09f649f-ea05-4e97-ace3-97f40c36641a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:12.945502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac642be5-77d3-433a-9088-0d5b0aeea93d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:12.958750 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3286/6643] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Sep 3 21:13:12 2022] DELETE /identity/v3/endpoints/e29eebdc508b46d08b2855c2dca9e082 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:12.959866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f649f-ea05-4e97-ace3-97f40c36641a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:13.004761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3dc4-ae30-4147-8c49-6fecef1487e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:13.009848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3dc4-ae30-4147-8c49-6fecef1487e5 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:13.011265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3dc4-ae30-4147-8c49-6fecef1487e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:13.011265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3dc4-ae30-4147-8c49-6fecef1487e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:13.011971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec3dc4-ae30-4147-8c49-6fecef1487e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:13.020183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec3dc4-ae30-4147-8c49-6fecef1487e5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:13.045990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f649f-ea05-4e97-ace3-97f40c36641a tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:13.061648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce142f-e61e-48a6-88c4-4c3ccd812de5 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:13.062358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce142f-e61e-48a6-88c4-4c3ccd812de5 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:13.062578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce142f-e61e-48a6-88c4-4c3ccd812de5 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:13.062910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce142f-e61e-48a6-88c4-4c3ccd812de5 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/regions/tempest-region-342946949` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:13.063445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce142f-e61e-48a6-88c4-4c3ccd812de5 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-342946949)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:13.066090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce142f-e61e-48a6-88c4-4c3ccd812de5 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:13.093008 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3358/6644] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:12 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:13.094187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3dc4-ae30-4147-8c49-6fecef1487e5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:13.099285 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3287/6645] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Sep 3 21:13:12 2022] DELETE /identity/v3/regions/tempest-region-342946949 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:13.104653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce142f-e61e-48a6-88c4-4c3ccd812de5 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:13.175164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce142f-e61e-48a6-88c4-4c3ccd812de5 tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:13.210039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:13.239266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:13.265919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3dc4-ae30-4147-8c49-6fecef1487e5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:13.296386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5222938-415d-4302-80c5-1063c197465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:13.297239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5222938-415d-4302-80c5-1063c197465d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:13.297794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5222938-415d-4302-80c5-1063c197465d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:13.298848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5222938-415d-4302-80c5-1063c197465d admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:13.301233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5222938-415d-4302-80c5-1063c197465d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:13.306711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5222938-415d-4302-80c5-1063c197465d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:13.310475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2936d3-d42e-4845-bed3-26872e69b05d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a16490ce76474ebc912dc7ed27a706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a453c8219ed404fbbba7c51e5a3c3e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:13.311149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2936d3-d42e-4845-bed3-26872e69b05d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:13.311747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2936d3-d42e-4845-bed3-26872e69b05d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:13.311747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2936d3-d42e-4845-bed3-26872e69b05d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] PATH_INFO: `/v3/services/92a4c0da6c914a5c8c8a658c7a83b058` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:13.312293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2936d3-d42e-4845-bed3-26872e69b05d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorizing `identity:delete_service(service_id=92a4c0da6c914a5c8c8a658c7a83b058)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:13.313965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2936d3-d42e-4845-bed3-26872e69b05d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:13.358753 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3359/6646] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:13 2022] POST /identity/v3/users => generated 291 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:13.360746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5222938-415d-4302-80c5-1063c197465d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:13.361170 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3288/6647] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:13 2022] DELETE /identity/v3/services/92a4c0da6c914a5c8c8a658c7a83b058 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:13.366854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2936d3-d42e-4845-bed3-26872e69b05d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:13.371597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dd2936d3-d42e-4845-bed3-26872e69b05d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:13.381990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5222938-415d-4302-80c5-1063c197465d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:13.409253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2936d3-d42e-4845-bed3-26872e69b05d tempest-EndPointsTestJSON-1333929568 tempest-EndPointsTestJSON-1333929568-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:13.415811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff7e0c-fc0d-470c-90f4-96049d469231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:13.416549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff7e0c-fc0d-470c-90f4-96049d469231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:13.417187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff7e0c-fc0d-470c-90f4-96049d469231 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:13.417992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff7e0c-fc0d-470c-90f4-96049d469231 admin admin] PATH_INFO: `/v3/users/5182f08ca3ea4714bcb6af09c02190fb` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:13.419385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff7e0c-fc0d-470c-90f4-96049d469231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5182f08ca3ea4714bcb6af09c02190fb)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:13.421631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff7e0c-fc0d-470c-90f4-96049d469231 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:13.426964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb552d1e-713f-45b9-9f16-10c66672df6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:13.428037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb552d1e-713f-45b9-9f16-10c66672df6b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:13.428174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb552d1e-713f-45b9-9f16-10c66672df6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:13.428394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb552d1e-713f-45b9-9f16-10c66672df6b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:13.429087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb552d1e-713f-45b9-9f16-10c66672df6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:13.433399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb552d1e-713f-45b9-9f16-10c66672df6b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:13.456540 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3360/6648] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:13.458900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb552d1e-713f-45b9-9f16-10c66672df6b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:13.472085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb552d1e-713f-45b9-9f16-10c66672df6b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:13.482579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3af40b-d119-4a03-9d30-68f83306ecbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:13.487195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3af40b-d119-4a03-9d30-68f83306ecbe admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:13.488134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3af40b-d119-4a03-9d30-68f83306ecbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:13.488309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3af40b-d119-4a03-9d30-68f83306ecbe admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:13.490897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3af40b-d119-4a03-9d30-68f83306ecbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:13.497128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3af40b-d119-4a03-9d30-68f83306ecbe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:13.531095 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3361/6649] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:13.532606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3af40b-d119-4a03-9d30-68f83306ecbe admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:13.534244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-afff7e0c-fc0d-470c-90f4-96049d469231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5182f08ca3ea4714bcb6af09c02190fb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:13.554390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-afff7e0c-fc0d-470c-90f4-96049d469231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5182f08ca3ea4714bcb6af09c02190fb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:13.555168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-afff7e0c-fc0d-470c-90f4-96049d469231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5182f08ca3ea4714bcb6af09c02190fb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:13.566517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-afff7e0c-fc0d-470c-90f4-96049d469231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5182f08ca3ea4714bcb6af09c02190fb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:13.569458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3af40b-d119-4a03-9d30-68f83306ecbe admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:13.577969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-afff7e0c-fc0d-470c-90f4-96049d469231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5182f08ca3ea4714bcb6af09c02190fb'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:13.584132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb56b7e-5ce6-428b-a79a-85fbe73f2dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:13.584826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb56b7e-5ce6-428b-a79a-85fbe73f2dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:13.585143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb56b7e-5ce6-428b-a79a-85fbe73f2dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:13.585444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb56b7e-5ce6-428b-a79a-85fbe73f2dc9 admin admin] PATH_INFO: `/v3/projects/d09f639e121545c3bc4a91bb3e761f71/users/079a84f13e1443519dd360c46b296815/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:13.587332 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3289/6650] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:13 2022] DELETE /identity/v3/users/5182f08ca3ea4714bcb6af09c02190fb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:13.594212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1746bfa-974f-4e2e-b300-cae10699e209 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:13.597796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1746bfa-974f-4e2e-b300-cae10699e209 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:13.598161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1746bfa-974f-4e2e-b300-cae10699e209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:13.618287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb56b7e-5ce6-428b-a79a-85fbe73f2dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09f639e121545c3bc4a91bb3e761f71, user_id=079a84f13e1443519dd360c46b296815, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:13.630645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb56b7e-5ce6-428b-a79a-85fbe73f2dc9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:13.661434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b1746bfa-974f-4e2e-b300-cae10699e209 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:13.670214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b1746bfa-974f-4e2e-b300-cae10699e209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:13.672682 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3362/6651] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:13 2022] PUT /identity/v3/projects/d09f639e121545c3bc4a91bb3e761f71/users/079a84f13e1443519dd360c46b296815/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:13.674250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245537f8-bc7f-4ed7-b7ce-4f249a7b6bb0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:13.674595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245537f8-bc7f-4ed7-b7ce-4f249a7b6bb0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:13.674876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245537f8-bc7f-4ed7-b7ce-4f249a7b6bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:13.717758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-245537f8-bc7f-4ed7-b7ce-4f249a7b6bb0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:13.719324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-245537f8-bc7f-4ed7-b7ce-4f249a7b6bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:13.803223 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3290/6652] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:13.817267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1746bfa-974f-4e2e-b300-cae10699e209 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:13.819276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b1746bfa-974f-4e2e-b300-cae10699e209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:13.855256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:13.895504 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3363/6653] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:13.947759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1746bfa-974f-4e2e-b300-cae10699e209 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:13.951618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368e8f0-7847-481e-b3d1-5812d08eba6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:13.956558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368e8f0-7847-481e-b3d1-5812d08eba6c service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:13.967218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368e8f0-7847-481e-b3d1-5812d08eba6c service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:13.967524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368e8f0-7847-481e-b3d1-5812d08eba6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:13.981214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368e8f0-7847-481e-b3d1-5812d08eba6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:13.983244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368e8f0-7847-481e-b3d1-5812d08eba6c service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:13.996394 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3291/6654] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:13:14.021946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245537f8-bc7f-4ed7-b7ce-4f249a7b6bb0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:14.036367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245537f8-bc7f-4ed7-b7ce-4f249a7b6bb0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:14.043951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831537e-1feb-4c23-9f4a-477ac32dfd00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:14.045550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831537e-1feb-4c23-9f4a-477ac32dfd00 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:14.046085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831537e-1feb-4c23-9f4a-477ac32dfd00 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:14.046432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831537e-1feb-4c23-9f4a-477ac32dfd00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:14.065577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831537e-1feb-4c23-9f4a-477ac32dfd00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:14.067228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831537e-1feb-4c23-9f4a-477ac32dfd00 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:14.088028 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3364/6655] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:13:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:13:15.492418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368e8f0-7847-481e-b3d1-5812d08eba6c service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:15.494811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b368e8f0-7847-481e-b3d1-5812d08eba6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:15.517647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368e8f0-7847-481e-b3d1-5812d08eba6c service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:15.535493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:15.536673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:15.537215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:15.537726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] PATH_INFO: `/v3/projects/ad07973708e1421d8c7f83d9d425b5e7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:15.539771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad07973708e1421d8c7f83d9d425b5e7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:15.542168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:15.623304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad07973708e1421d8c7f83d9d425b5e7', 'request_id': 'req-f93a50ba-05a9-4672-baf5-369ad9b5585f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:15.649745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad07973708e1421d8c7f83d9d425b5e7', 'request_id': 'req-f93a50ba-05a9-4672-baf5-369ad9b5585f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:15.666905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad07973708e1421d8c7f83d9d425b5e7', 'request_id': 'req-f93a50ba-05a9-4672-baf5-369ad9b5585f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:15.725963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] The token cache is being invalidate because project ad07973708e1421d8c7f83d9d425b5e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:15.726581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:15.730542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:15.734041 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3292/6656] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:15 2022] DELETE /identity/v3/projects/ad07973708e1421d8c7f83d9d425b5e7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:15.738069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831537e-1feb-4c23-9f4a-477ac32dfd00 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:15.768655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831537e-1feb-4c23-9f4a-477ac32dfd00 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:15.872600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:15.897026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec77ffb-517b-49e7-a496-ccddb538fe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:15.898350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec77ffb-517b-49e7-a496-ccddb538fe78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:15.899230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec77ffb-517b-49e7-a496-ccddb538fe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:15.900041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec77ffb-517b-49e7-a496-ccddb538fe78 admin admin] PATH_INFO: `/v3/users/52a16490ce76474ebc912dc7ed27a706` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:15.918618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec77ffb-517b-49e7-a496-ccddb538fe78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52a16490ce76474ebc912dc7ed27a706)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:15.922146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec77ffb-517b-49e7-a496-ccddb538fe78 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:16.066547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4ec77ffb-517b-49e7-a496-ccddb538fe78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52a16490ce76474ebc912dc7ed27a706'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:16.099704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4ec77ffb-517b-49e7-a496-ccddb538fe78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52a16490ce76474ebc912dc7ed27a706'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:16.111321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4ec77ffb-517b-49e7-a496-ccddb538fe78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52a16490ce76474ebc912dc7ed27a706'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:16.112253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4ec77ffb-517b-49e7-a496-ccddb538fe78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52a16490ce76474ebc912dc7ed27a706'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:16.113109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4ec77ffb-517b-49e7-a496-ccddb538fe78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52a16490ce76474ebc912dc7ed27a706'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:16.139944 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3365/6657] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:15 2022] DELETE /identity/v3/users/52a16490ce76474ebc912dc7ed27a706 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:17.521147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:17.523274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:17.553141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a50ba-05a9-4672-baf5-369ad9b5585f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:17.684435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c832b17-2785-46ab-b5e8-cfe5d3d59976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:17.687549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c832b17-2785-46ab-b5e8-cfe5d3d59976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:17.688100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c832b17-2785-46ab-b5e8-cfe5d3d59976 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:17.689899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c832b17-2785-46ab-b5e8-cfe5d3d59976 admin admin] PATH_INFO: `/v3/projects/4a453c8219ed404fbbba7c51e5a3c3e8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:17.693546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c832b17-2785-46ab-b5e8-cfe5d3d59976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a453c8219ed404fbbba7c51e5a3c3e8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:17.698796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c832b17-2785-46ab-b5e8-cfe5d3d59976 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:17.794666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:17.820425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8c832b17-2785-46ab-b5e8-cfe5d3d59976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a453c8219ed404fbbba7c51e5a3c3e8', 'request_id': 'req-8c832b17-2785-46ab-b5e8-cfe5d3d59976'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:17.838261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8c832b17-2785-46ab-b5e8-cfe5d3d59976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a453c8219ed404fbbba7c51e5a3c3e8', 'request_id': 'req-8c832b17-2785-46ab-b5e8-cfe5d3d59976'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:17.854827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8c832b17-2785-46ab-b5e8-cfe5d3d59976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a453c8219ed404fbbba7c51e5a3c3e8', 'request_id': 'req-8c832b17-2785-46ab-b5e8-cfe5d3d59976'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:17.897078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8c832b17-2785-46ab-b5e8-cfe5d3d59976 admin admin] The token cache is being invalidate because project 4a453c8219ed404fbbba7c51e5a3c3e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:17.897401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8c832b17-2785-46ab-b5e8-cfe5d3d59976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:17.898077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8c832b17-2785-46ab-b5e8-cfe5d3d59976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:17.899631 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3293/6658] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:17 2022] DELETE /identity/v3/projects/4a453c8219ed404fbbba7c51e5a3c3e8 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:17.905066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ec9c3-f729-49e9-ad76-f7482f23224e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:17.906452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ec9c3-f729-49e9-ad76-f7482f23224e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:17.906907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ec9c3-f729-49e9-ad76-f7482f23224e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:17.946257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9b9ec9c3-f729-49e9-ad76-f7482f23224e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:17.948229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9b9ec9c3-f729-49e9-ad76-f7482f23224e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:18.076612 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3366/6659] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:18.083738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2ef92-026e-4043-bc09-6caef2f5d951 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:18.084038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2ef92-026e-4043-bc09-6caef2f5d951 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:18.084327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2ef92-026e-4043-bc09-6caef2f5d951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:18.111579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1cf2ef92-026e-4043-bc09-6caef2f5d951 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:18.114951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1cf2ef92-026e-4043-bc09-6caef2f5d951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:18.154769 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3294/6660] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:18.158574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ec9c3-f729-49e9-ad76-f7482f23224e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:18.182748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ec9c3-f729-49e9-ad76-f7482f23224e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:18.200432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f4264-02e5-48e3-b892-533d5dabe1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:18.202513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f4264-02e5-48e3-b892-533d5dabe1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:18.202805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f4264-02e5-48e3-b892-533d5dabe1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:18.204684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f4264-02e5-48e3-b892-533d5dabe1b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:18.205497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f4264-02e5-48e3-b892-533d5dabe1b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:18.205820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f4264-02e5-48e3-b892-533d5dabe1b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:18.214760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f4264-02e5-48e3-b892-533d5dabe1b6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:18.238613 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3367/6661] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:18.239296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2ef92-026e-4043-bc09-6caef2f5d951 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:18.251267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2ef92-026e-4043-bc09-6caef2f5d951 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:18.259483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f361cd0-4f4f-46ea-878c-f871e7a461ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:18.260176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f361cd0-4f4f-46ea-878c-f871e7a461ff admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:18.260459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f361cd0-4f4f-46ea-878c-f871e7a461ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:18.260742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f361cd0-4f4f-46ea-878c-f871e7a461ff admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:18.261510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f361cd0-4f4f-46ea-878c-f871e7a461ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:18.264973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f361cd0-4f4f-46ea-878c-f871e7a461ff admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:18.338143 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3295/6662] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:18 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:18.339691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f4264-02e5-48e3-b892-533d5dabe1b6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:18.360530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f4264-02e5-48e3-b892-533d5dabe1b6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:18.451678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:18.472248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b303506-bdba-48a1-b249-da830b451e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:18.473201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b303506-bdba-48a1-b249-da830b451e4f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:18.474327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b303506-bdba-48a1-b249-da830b451e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:18.474841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b303506-bdba-48a1-b249-da830b451e4f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:18.476607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b303506-bdba-48a1-b249-da830b451e4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:18.480105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b303506-bdba-48a1-b249-da830b451e4f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:18.519878 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3368/6663] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:18 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:18.524063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f361cd0-4f4f-46ea-878c-f871e7a461ff admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:18.538599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f361cd0-4f4f-46ea-878c-f871e7a461ff admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:18.553054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a4b65-876f-4bf8-9037-4305be516070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:18.554090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a4b65-876f-4bf8-9037-4305be516070 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:18.555044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a4b65-876f-4bf8-9037-4305be516070 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:18.555538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a4b65-876f-4bf8-9037-4305be516070 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:18.556497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a4b65-876f-4bf8-9037-4305be516070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:18.558406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a4b65-876f-4bf8-9037-4305be516070 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:18.591973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b303506-bdba-48a1-b249-da830b451e4f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:18.598914 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3296/6664] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:18.605220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b303506-bdba-48a1-b249-da830b451e4f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:18.615588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820c399-c120-4292-a185-1d83cdf452b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:18.616584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820c399-c120-4292-a185-1d83cdf452b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:18.617196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820c399-c120-4292-a185-1d83cdf452b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:18.617738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820c399-c120-4292-a185-1d83cdf452b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:18.618679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2820c399-c120-4292-a185-1d83cdf452b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:18.621055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2820c399-c120-4292-a185-1d83cdf452b3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:18.649038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a4b65-876f-4bf8-9037-4305be516070 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:18.653056 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3369/6665] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:18.666367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a4b65-876f-4bf8-9037-4305be516070 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:18.679480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfb682-c8bd-454e-bd78-e71f199879d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:18.680293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfb682-c8bd-454e-bd78-e71f199879d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:18.680679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfb682-c8bd-454e-bd78-e71f199879d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:18.681102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfb682-c8bd-454e-bd78-e71f199879d2 admin admin] PATH_INFO: `/v3/projects/1bbc5b8a835b4680af57e7346eee5385/users/51db0b56734944079335a38c7083c5c4/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:18.713643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfb682-c8bd-454e-bd78-e71f199879d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bbc5b8a835b4680af57e7346eee5385, user_id=51db0b56734944079335a38c7083c5c4, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:18.721046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfb682-c8bd-454e-bd78-e71f199879d2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:18.756721 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3297/6666] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:18 2022] PUT /identity/v3/projects/1bbc5b8a835b4680af57e7346eee5385/users/51db0b56734944079335a38c7083c5c4/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:18.758149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187af3f-b2b5-410a-adba-9428028d3a4c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:18.758464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187af3f-b2b5-410a-adba-9428028d3a4c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:18.758791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187af3f-b2b5-410a-adba-9428028d3a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:18.858735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c187af3f-b2b5-410a-adba-9428028d3a4c None None] MFA Rules not processed for user `51db0b56734944079335a38c7083c5c4`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:18.861535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c187af3f-b2b5-410a-adba-9428028d3a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:19.098639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfb682-c8bd-454e-bd78-e71f199879d2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:19.100541 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3370/6667] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:13:19.237612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfb682-c8bd-454e-bd78-e71f199879d2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:19.254681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3c943-7142-4544-8806-139ebd944763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:19.255837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3c943-7142-4544-8806-139ebd944763 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:19.255837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3c943-7142-4544-8806-139ebd944763 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:19.256221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3c943-7142-4544-8806-139ebd944763 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:19.257012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3c943-7142-4544-8806-139ebd944763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:19.259365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3c943-7142-4544-8806-139ebd944763 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:19.317913 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3298/6668] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:19 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:19.321788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187af3f-b2b5-410a-adba-9428028d3a4c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:19.340207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187af3f-b2b5-410a-adba-9428028d3a4c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:19.491574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce12f1-3b5b-482c-9163-b1e5a0c2697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:19.492519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce12f1-3b5b-482c-9163-b1e5a0c2697e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:19.493057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce12f1-3b5b-482c-9163-b1e5a0c2697e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:19.493853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce12f1-3b5b-482c-9163-b1e5a0c2697e admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:19.494695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce12f1-3b5b-482c-9163-b1e5a0c2697e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:19.509289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce12f1-3b5b-482c-9163-b1e5a0c2697e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:19.562542 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3371/6669] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:19 2022] POST /identity/v3/users => generated 299 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:19.567328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3c943-7142-4544-8806-139ebd944763 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:19.588336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3c943-7142-4544-8806-139ebd944763 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:19.599424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281b7c3-89da-483d-94cf-9e761a437376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:19.600437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281b7c3-89da-483d-94cf-9e761a437376 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:19.601035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281b7c3-89da-483d-94cf-9e761a437376 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:19.601715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281b7c3-89da-483d-94cf-9e761a437376 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:19.602887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281b7c3-89da-483d-94cf-9e761a437376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:19.604775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281b7c3-89da-483d-94cf-9e761a437376 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:19.618900 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3299/6670] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:13:19.624501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce12f1-3b5b-482c-9163-b1e5a0c2697e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:19.638952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce12f1-3b5b-482c-9163-b1e5a0c2697e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:19.646696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de80227-b18b-4ad8-9b8c-a1c2ec80698b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:19.647539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de80227-b18b-4ad8-9b8c-a1c2ec80698b admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:19.647987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de80227-b18b-4ad8-9b8c-a1c2ec80698b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:19.648424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de80227-b18b-4ad8-9b8c-a1c2ec80698b admin admin] PATH_INFO: `/v3/projects/23a645754e294c669134881fc88b1a8c/users/ea6305dc0fdc42bd9d382115a10e6f6c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:19.680693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de80227-b18b-4ad8-9b8c-a1c2ec80698b admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a645754e294c669134881fc88b1a8c, user_id=ea6305dc0fdc42bd9d382115a10e6f6c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:19.683795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de80227-b18b-4ad8-9b8c-a1c2ec80698b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:19.720662 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3372/6671] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:19 2022] PUT /identity/v3/projects/23a645754e294c669134881fc88b1a8c/users/ea6305dc0fdc42bd9d382115a10e6f6c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:19.724532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281b7c3-89da-483d-94cf-9e761a437376 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:19.739882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281b7c3-89da-483d-94cf-9e761a437376 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:19.799216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:19.854533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d81cc7-e33f-4673-8052-4120a207071a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:19.855619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d81cc7-e33f-4673-8052-4120a207071a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:19.855940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d81cc7-e33f-4673-8052-4120a207071a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:19.857546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d81cc7-e33f-4673-8052-4120a207071a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:19.859741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d81cc7-e33f-4673-8052-4120a207071a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:19.877050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d81cc7-e33f-4673-8052-4120a207071a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:19.905710 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3300/6672] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:19.910957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de80227-b18b-4ad8-9b8c-a1c2ec80698b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:19.929280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de80227-b18b-4ad8-9b8c-a1c2ec80698b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:19.979689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c0b70-c4e9-4b4c-a75f-a7d3fb05c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:19.980665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c0b70-c4e9-4b4c-a75f-a7d3fb05c62d admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:19.987183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c0b70-c4e9-4b4c-a75f-a7d3fb05c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:19.991354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c0b70-c4e9-4b4c-a75f-a7d3fb05c62d admin admin] PATH_INFO: `/v3/projects/23a645754e294c669134881fc88b1a8c/users/ea6305dc0fdc42bd9d382115a10e6f6c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:20.009271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c0b70-c4e9-4b4c-a75f-a7d3fb05c62d admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a645754e294c669134881fc88b1a8c, user_id=ea6305dc0fdc42bd9d382115a10e6f6c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:20.012148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c0b70-c4e9-4b4c-a75f-a7d3fb05c62d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:20.025756 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3373/6673] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:19 2022] PUT /identity/v3/projects/23a645754e294c669134881fc88b1a8c/users/ea6305dc0fdc42bd9d382115a10e6f6c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:20.039928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b417529-8ff5-4cca-bc15-f777ffc678fc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:20.041324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b417529-8ff5-4cca-bc15-f777ffc678fc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:20.042039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b417529-8ff5-4cca-bc15-f777ffc678fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:20.098428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8b417529-8ff5-4cca-bc15-f777ffc678fc None None] MFA Rules not processed for user `ea6305dc0fdc42bd9d382115a10e6f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:20.100062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8b417529-8ff5-4cca-bc15-f777ffc678fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:20.245053 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3301/6674] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:20.245763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c0b70-c4e9-4b4c-a75f-a7d3fb05c62d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:20.263272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c0b70-c4e9-4b4c-a75f-a7d3fb05c62d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:20.272347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b480fac-8b68-4ced-a258-236bbe2ac013 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea6305dc0fdc42bd9d382115a10e6f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23a645754e294c669134881fc88b1a8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:20.273974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b480fac-8b68-4ced-a258-236bbe2ac013 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:20.274484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b480fac-8b68-4ced-a258-236bbe2ac013 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:20.274978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b480fac-8b68-4ced-a258-236bbe2ac013 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:20.276150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b480fac-8b68-4ced-a258-236bbe2ac013 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:20.279925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b480fac-8b68-4ced-a258-236bbe2ac013 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:20.343168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:20.372015 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3374/6675] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:20 2022] POST /identity/v3/projects => generated 359 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:20.376208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b417529-8ff5-4cca-bc15-f777ffc678fc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:20.391600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b417529-8ff5-4cca-bc15-f777ffc678fc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:20.506967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd937b-0b51-454e-832e-1fe7862be319 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea6305dc0fdc42bd9d382115a10e6f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23a645754e294c669134881fc88b1a8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:20.507717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd937b-0b51-454e-832e-1fe7862be319 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:20.508121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd937b-0b51-454e-832e-1fe7862be319 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:20.508503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd937b-0b51-454e-832e-1fe7862be319 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] PATH_INFO: `/v3/projects/d998f2323df0404bb286247fa828dcff/tags/tempest-tag-1315735043` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:20.509892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd937b-0b51-454e-832e-1fe7862be319 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorizing `identity:create_project_tag(project_id=d998f2323df0404bb286247fa828dcff, value=tempest-tag-1315735043)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:20.512214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd937b-0b51-454e-832e-1fe7862be319 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:20.575186 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-12dd937b-0b51-454e-832e-1fe7862be319 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-1315735043. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:13:20.575186 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:13:20.575186 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:20.582904 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3302/6676] 10.222.0.16 () {60 vars in 1333 bytes} [Sat Sep 3 21:13:20 2022] PUT /identity/v3/projects/d998f2323df0404bb286247fa828dcff/tags/tempest-tag-1315735043 => generated 0 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 291 bytes (1 switches on core 0) Sep 03 21:13:20.587293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b480fac-8b68-4ced-a258-236bbe2ac013 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:20.600752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b480fac-8b68-4ced-a258-236bbe2ac013 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:20.608691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519dde2-b907-4fa4-b288-0dbf2b0b4071 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea6305dc0fdc42bd9d382115a10e6f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23a645754e294c669134881fc88b1a8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:20.609756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519dde2-b907-4fa4-b288-0dbf2b0b4071 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:20.610296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519dde2-b907-4fa4-b288-0dbf2b0b4071 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:20.610816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519dde2-b907-4fa4-b288-0dbf2b0b4071 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] PATH_INFO: `/v3/projects/d998f2323df0404bb286247fa828dcff/tags/tempest-tag-1315735043` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:20.653340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519dde2-b907-4fa4-b288-0dbf2b0b4071 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=d998f2323df0404bb286247fa828dcff, value=tempest-tag-1315735043)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:20.657072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519dde2-b907-4fa4-b288-0dbf2b0b4071 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:20.696918 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3375/6677] 10.222.0.16 () {58 vars in 1314 bytes} [Sat Sep 3 21:13:20 2022] GET /identity/v3/projects/d998f2323df0404bb286247fa828dcff/tags/tempest-tag-1315735043 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:20.702886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd937b-0b51-454e-832e-1fe7862be319 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:20.719272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd937b-0b51-454e-832e-1fe7862be319 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:20.734566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565ae53-ea56-4167-979b-05b6d555c6a4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea6305dc0fdc42bd9d382115a10e6f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23a645754e294c669134881fc88b1a8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:20.735382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565ae53-ea56-4167-979b-05b6d555c6a4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:20.736021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565ae53-ea56-4167-979b-05b6d555c6a4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:20.736404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565ae53-ea56-4167-979b-05b6d555c6a4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] PATH_INFO: `/v3/projects/d998f2323df0404bb286247fa828dcff/tags` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:20.737809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565ae53-ea56-4167-979b-05b6d555c6a4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorizing `identity:update_project_tags(project_id=d998f2323df0404bb286247fa828dcff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:20.740223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565ae53-ea56-4167-979b-05b6d555c6a4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:20.767211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:20.835989 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3303/6678] 10.222.0.16 () {60 vars in 1265 bytes} [Sat Sep 3 21:13:20 2022] PUT /identity/v3/projects/d998f2323df0404bb286247fa828dcff/tags => generated 138 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:20.838596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519dde2-b907-4fa4-b288-0dbf2b0b4071 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:20.860639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519dde2-b907-4fa4-b288-0dbf2b0b4071 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:20.867145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc156ee-89e6-4d13-9a3e-ccfb1675464b tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea6305dc0fdc42bd9d382115a10e6f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23a645754e294c669134881fc88b1a8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:20.868370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc156ee-89e6-4d13-9a3e-ccfb1675464b tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:20.868724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc156ee-89e6-4d13-9a3e-ccfb1675464b tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:20.869117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc156ee-89e6-4d13-9a3e-ccfb1675464b tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] PATH_INFO: `/v3/projects/d998f2323df0404bb286247fa828dcff/tags` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:20.890593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc156ee-89e6-4d13-9a3e-ccfb1675464b tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=d998f2323df0404bb286247fa828dcff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:20.895897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc156ee-89e6-4d13-9a3e-ccfb1675464b tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:20.914862 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3376/6679] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Sep 3 21:13:20 2022] GET /identity/v3/projects/d998f2323df0404bb286247fa828dcff/tags => generated 138 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:20.919394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565ae53-ea56-4167-979b-05b6d555c6a4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:20.932859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565ae53-ea56-4167-979b-05b6d555c6a4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:20.941738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805f204-3c3c-4a21-b2e4-bcf231b29c8e tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea6305dc0fdc42bd9d382115a10e6f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23a645754e294c669134881fc88b1a8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:20.942483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805f204-3c3c-4a21-b2e4-bcf231b29c8e tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:20.942842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805f204-3c3c-4a21-b2e4-bcf231b29c8e tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:20.943190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805f204-3c3c-4a21-b2e4-bcf231b29c8e tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] PATH_INFO: `/v3/projects/d998f2323df0404bb286247fa828dcff/tags/tempest-tag-45948877` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:20.945803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805f204-3c3c-4a21-b2e4-bcf231b29c8e tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorizing `identity:delete_project_tag(project_id=d998f2323df0404bb286247fa828dcff, value=tempest-tag-45948877)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:20.948038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805f204-3c3c-4a21-b2e4-bcf231b29c8e tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:21.018624 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-6805f204-3c3c-4a21-b2e4-bcf231b29c8e tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-45948877. To ensure interoperability, identifiers should be a valid uuid. Sep 03 21:13:21.018624 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 03 21:13:21.018624 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:21.039772 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3304/6680] 10.222.0.16 () {58 vars in 1311 bytes} [Sat Sep 3 21:13:20 2022] DELETE /identity/v3/projects/d998f2323df0404bb286247fa828dcff/tags/tempest-tag-45948877 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:21.041175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc156ee-89e6-4d13-9a3e-ccfb1675464b tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:21.067582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc156ee-89e6-4d13-9a3e-ccfb1675464b tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:21.087141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda0a80-5b39-40af-bf21-862719d66365 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea6305dc0fdc42bd9d382115a10e6f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23a645754e294c669134881fc88b1a8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:21.088165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda0a80-5b39-40af-bf21-862719d66365 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:21.088956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda0a80-5b39-40af-bf21-862719d66365 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:21.089468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda0a80-5b39-40af-bf21-862719d66365 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] PATH_INFO: `/v3/projects/d998f2323df0404bb286247fa828dcff/tags/tempest-tag-45948877` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:21.119933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda0a80-5b39-40af-bf21-862719d66365 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=d998f2323df0404bb286247fa828dcff, value=tempest-tag-45948877)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:21.123092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda0a80-5b39-40af-bf21-862719d66365 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-fcda0a80-5b39-40af-bf21-862719d66365 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Could not find project tag: tempest-tag-45948877.: keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-45948877. Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 284, in get Sep 03 21:13:21.160386 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_project_tag(project_id, value) Sep 03 21:13:21.161821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:13:21.161821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:13:21.161821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 960, in get_project_tag Sep 03 21:13:21.161821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectTagNotFound(project_tag=tag_name) Sep 03 21:13:21.161821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-45948877. Sep 03 21:13:21.161821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:21.166261 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3377/6681] 10.222.0.16 () {58 vars in 1308 bytes} [Sat Sep 3 21:13:21 2022] GET /identity/v3/projects/d998f2323df0404bb286247fa828dcff/tags/tempest-tag-45948877 => generated 105 bytes in 125 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:13:21.167655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805f204-3c3c-4a21-b2e4-bcf231b29c8e tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:21.179112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805f204-3c3c-4a21-b2e4-bcf231b29c8e tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:21.209263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c8621-d9e7-4421-8cc4-a2160289e838 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea6305dc0fdc42bd9d382115a10e6f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23a645754e294c669134881fc88b1a8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:21.209969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c8621-d9e7-4421-8cc4-a2160289e838 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:21.210264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c8621-d9e7-4421-8cc4-a2160289e838 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:21.210660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c8621-d9e7-4421-8cc4-a2160289e838 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] PATH_INFO: `/v3/projects/d998f2323df0404bb286247fa828dcff/tags` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:21.217385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c8621-d9e7-4421-8cc4-a2160289e838 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorizing `identity:delete_project_tags(project_id=d998f2323df0404bb286247fa828dcff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:21.222672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c8621-d9e7-4421-8cc4-a2160289e838 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:21.335645 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3305/6682] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Sep 3 21:13:21 2022] DELETE /identity/v3/projects/d998f2323df0404bb286247fa828dcff/tags => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:21.339607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda0a80-5b39-40af-bf21-862719d66365 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:21.353266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda0a80-5b39-40af-bf21-862719d66365 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:21.369664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec742cb-7f3d-4b1e-accd-040bc51b7df4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea6305dc0fdc42bd9d382115a10e6f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23a645754e294c669134881fc88b1a8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:21.371430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec742cb-7f3d-4b1e-accd-040bc51b7df4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:21.375043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec742cb-7f3d-4b1e-accd-040bc51b7df4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:21.376529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec742cb-7f3d-4b1e-accd-040bc51b7df4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] PATH_INFO: `/v3/projects/d998f2323df0404bb286247fa828dcff/tags` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:21.412328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec742cb-7f3d-4b1e-accd-040bc51b7df4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=d998f2323df0404bb286247fa828dcff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:21.415087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec742cb-7f3d-4b1e-accd-040bc51b7df4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:21.429399 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3378/6683] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Sep 3 21:13:21 2022] GET /identity/v3/projects/d998f2323df0404bb286247fa828dcff/tags => generated 66 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:13:21.433826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c8621-d9e7-4421-8cc4-a2160289e838 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:21.447957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c8621-d9e7-4421-8cc4-a2160289e838 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:21.454729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518dad23-5d68-496c-94fb-b3a871c04f02 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea6305dc0fdc42bd9d382115a10e6f6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23a645754e294c669134881fc88b1a8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:21.455852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518dad23-5d68-496c-94fb-b3a871c04f02 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:21.456315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518dad23-5d68-496c-94fb-b3a871c04f02 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:21.456764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518dad23-5d68-496c-94fb-b3a871c04f02 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] PATH_INFO: `/v3/projects/d998f2323df0404bb286247fa828dcff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:21.458692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518dad23-5d68-496c-94fb-b3a871c04f02 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d998f2323df0404bb286247fa828dcff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:21.466204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518dad23-5d68-496c-94fb-b3a871c04f02 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:21.552587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-518dad23-5d68-496c-94fb-b3a871c04f02 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd998f2323df0404bb286247fa828dcff', 'request_id': 'req-518dad23-5d68-496c-94fb-b3a871c04f02'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:21.573133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-518dad23-5d68-496c-94fb-b3a871c04f02 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd998f2323df0404bb286247fa828dcff', 'request_id': 'req-518dad23-5d68-496c-94fb-b3a871c04f02'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:21.584263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-518dad23-5d68-496c-94fb-b3a871c04f02 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd998f2323df0404bb286247fa828dcff', 'request_id': 'req-518dad23-5d68-496c-94fb-b3a871c04f02'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:21.626635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-518dad23-5d68-496c-94fb-b3a871c04f02 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] The token cache is being invalidate because project d998f2323df0404bb286247fa828dcff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:21.626945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-518dad23-5d68-496c-94fb-b3a871c04f02 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:21.627715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-518dad23-5d68-496c-94fb-b3a871c04f02 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:21.637027 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3306/6684] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:21 2022] DELETE /identity/v3/projects/d998f2323df0404bb286247fa828dcff => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:21.640249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec742cb-7f3d-4b1e-accd-040bc51b7df4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:21.650301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cec742cb-7f3d-4b1e-accd-040bc51b7df4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:21.756581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec742cb-7f3d-4b1e-accd-040bc51b7df4 tempest-IdentityV3ProjectTagsTest-412829340 tempest-IdentityV3ProjectTagsTest-412829340-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:21.874037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:21.886887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:21.893892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:21.894269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] PATH_INFO: `/v3/users/51db0b56734944079335a38c7083c5c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:21.899445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51db0b56734944079335a38c7083c5c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:21.912679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:21.961263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=1,keystone:SELECT=98,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:22.022572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51db0b56734944079335a38c7083c5c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:22.040243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51db0b56734944079335a38c7083c5c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:22.053762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51db0b56734944079335a38c7083c5c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:22.055054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51db0b56734944079335a38c7083c5c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:22.056195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51db0b56734944079335a38c7083c5c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:22.074573 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3379/6685] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:21 2022] DELETE /identity/v3/users/51db0b56734944079335a38c7083c5c4 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:22.079956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88513c7-4e90-41e3-9c8a-3f7aacf20af0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:22.081210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88513c7-4e90-41e3-9c8a-3f7aacf20af0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:22.081542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88513c7-4e90-41e3-9c8a-3f7aacf20af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:22.128662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e88513c7-4e90-41e3-9c8a-3f7aacf20af0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:22.130266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e88513c7-4e90-41e3-9c8a-3f7aacf20af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:22.195703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:22.265066 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3307/6686] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:13:22.278700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:22.280657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:22.378623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955f9bd-7371-461b-980c-326af30cb526 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:22.389160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e09d5-cbca-4c29-a50d-6f57ed738564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:22.389994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e09d5-cbca-4c29-a50d-6f57ed738564 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:22.390334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e09d5-cbca-4c29-a50d-6f57ed738564 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:22.393752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e09d5-cbca-4c29-a50d-6f57ed738564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:22.412081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e09d5-cbca-4c29-a50d-6f57ed738564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:22.417125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e09d5-cbca-4c29-a50d-6f57ed738564 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:22.468037 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3380/6687] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:13:23.988400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88513c7-4e90-41e3-9c8a-3f7aacf20af0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:23.990575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e88513c7-4e90-41e3-9c8a-3f7aacf20af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:24.021746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88513c7-4e90-41e3-9c8a-3f7aacf20af0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:24.036065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f21bde-985b-4548-b800-c4eeb740f5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:24.042914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f21bde-985b-4548-b800-c4eeb740f5b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:24.043549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f21bde-985b-4548-b800-c4eeb740f5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:24.044079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f21bde-985b-4548-b800-c4eeb740f5b9 admin admin] PATH_INFO: `/v3/projects/1bbc5b8a835b4680af57e7346eee5385` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:24.053350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f21bde-985b-4548-b800-c4eeb740f5b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bbc5b8a835b4680af57e7346eee5385)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:24.056282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f21bde-985b-4548-b800-c4eeb740f5b9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:24.242438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-67f21bde-985b-4548-b800-c4eeb740f5b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bbc5b8a835b4680af57e7346eee5385', 'request_id': 'req-67f21bde-985b-4548-b800-c4eeb740f5b9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:24.291598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-67f21bde-985b-4548-b800-c4eeb740f5b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bbc5b8a835b4680af57e7346eee5385', 'request_id': 'req-67f21bde-985b-4548-b800-c4eeb740f5b9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:24.317434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-67f21bde-985b-4548-b800-c4eeb740f5b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bbc5b8a835b4680af57e7346eee5385', 'request_id': 'req-67f21bde-985b-4548-b800-c4eeb740f5b9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:24.355252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-67f21bde-985b-4548-b800-c4eeb740f5b9 admin admin] The token cache is being invalidate because project 1bbc5b8a835b4680af57e7346eee5385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:24.355594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-67f21bde-985b-4548-b800-c4eeb740f5b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:24.356251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-67f21bde-985b-4548-b800-c4eeb740f5b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:24.361134 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3308/6688] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:23 2022] DELETE /identity/v3/projects/1bbc5b8a835b4680af57e7346eee5385 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:24.362592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e09d5-cbca-4c29-a50d-6f57ed738564 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:24.383764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e09d5-cbca-4c29-a50d-6f57ed738564 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:24.521779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:24.522736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:24.523035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:24.523389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] PATH_INFO: `/v3/users/ea6305dc0fdc42bd9d382115a10e6f6c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:24.525166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea6305dc0fdc42bd9d382115a10e6f6c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:24.527602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:24.614473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:24.634269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea6305dc0fdc42bd9d382115a10e6f6c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:24.660281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea6305dc0fdc42bd9d382115a10e6f6c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:24.680164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea6305dc0fdc42bd9d382115a10e6f6c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:24.680842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea6305dc0fdc42bd9d382115a10e6f6c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:24.682979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea6305dc0fdc42bd9d382115a10e6f6c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:24.708852 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3381/6689] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:24 2022] DELETE /identity/v3/users/ea6305dc0fdc42bd9d382115a10e6f6c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:24.872545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0bfb2-6ca6-432f-a606-ab084364f22c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:24.872845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0bfb2-6ca6-432f-a606-ab084364f22c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:24.873159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0bfb2-6ca6-432f-a606-ab084364f22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:24.909651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-dfd0bfb2-6ca6-432f-a606-ab084364f22c None None] MFA Rules not processed for user `9f792275f53f445ea72b6de441314a49`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:24.911039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dfd0bfb2-6ca6-432f-a606-ab084364f22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:25.082306 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3309/6690] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:25.919013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:25.920665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:26.097399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c512a-84a0-4a9f-a522-bacae78aaa5a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:26.111934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8232f-6b48-42e3-8f6b-45a42d3839da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:26.112993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8232f-6b48-42e3-8f6b-45a42d3839da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:26.113475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8232f-6b48-42e3-8f6b-45a42d3839da admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:26.113928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8232f-6b48-42e3-8f6b-45a42d3839da admin admin] PATH_INFO: `/v3/projects/23a645754e294c669134881fc88b1a8c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:26.116958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8232f-6b48-42e3-8f6b-45a42d3839da admin admin] RBAC: Authorizing `identity:delete_project(project_id=23a645754e294c669134881fc88b1a8c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:26.119224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8232f-6b48-42e3-8f6b-45a42d3839da admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:26.257431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1ce8232f-6b48-42e3-8f6b-45a42d3839da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23a645754e294c669134881fc88b1a8c', 'request_id': 'req-1ce8232f-6b48-42e3-8f6b-45a42d3839da'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:26.276502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1ce8232f-6b48-42e3-8f6b-45a42d3839da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23a645754e294c669134881fc88b1a8c', 'request_id': 'req-1ce8232f-6b48-42e3-8f6b-45a42d3839da'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:26.288597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1ce8232f-6b48-42e3-8f6b-45a42d3839da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23a645754e294c669134881fc88b1a8c', 'request_id': 'req-1ce8232f-6b48-42e3-8f6b-45a42d3839da'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:26.308979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1ce8232f-6b48-42e3-8f6b-45a42d3839da admin admin] The token cache is being invalidate because project 23a645754e294c669134881fc88b1a8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:26.309469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1ce8232f-6b48-42e3-8f6b-45a42d3839da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:26.316620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1ce8232f-6b48-42e3-8f6b-45a42d3839da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:26.319147 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3382/6691] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:25 2022] DELETE /identity/v3/projects/23a645754e294c669134881fc88b1a8c => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:26.327158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19b95e-36e9-4a0e-8f9e-8b33d8e67fd2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:26.327730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19b95e-36e9-4a0e-8f9e-8b33d8e67fd2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:26.327730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19b95e-36e9-4a0e-8f9e-8b33d8e67fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:26.370527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2e19b95e-36e9-4a0e-8f9e-8b33d8e67fd2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:26.372758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2e19b95e-36e9-4a0e-8f9e-8b33d8e67fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:26.467805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:26.498419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929a162-3f7d-49f8-9a74-d6162fa30081 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:26.501371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929a162-3f7d-49f8-9a74-d6162fa30081 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:26.501911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929a162-3f7d-49f8-9a74-d6162fa30081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:26.511721 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3310/6692] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:26.543182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9929a162-3f7d-49f8-9a74-d6162fa30081 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:26.544471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9929a162-3f7d-49f8-9a74-d6162fa30081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:26.569431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19b95e-36e9-4a0e-8f9e-8b33d8e67fd2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:26.582240 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3383/6693] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:13:26.583671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19b95e-36e9-4a0e-8f9e-8b33d8e67fd2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:26.594148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f88c-2b13-4d2e-a30a-63da64459cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:26.595837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f88c-2b13-4d2e-a30a-63da64459cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:26.596304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f88c-2b13-4d2e-a30a-63da64459cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:26.596779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f88c-2b13-4d2e-a30a-63da64459cd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:26.598157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966f88c-2b13-4d2e-a30a-63da64459cd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:26.604116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966f88c-2b13-4d2e-a30a-63da64459cd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:26.606047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966f88c-2b13-4d2e-a30a-63da64459cd0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:26.638190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929a162-3f7d-49f8-9a74-d6162fa30081 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:26.648163 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3311/6694] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:13:26.654645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929a162-3f7d-49f8-9a74-d6162fa30081 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:26.661209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cf802-387c-42ce-9e3a-267c12d1fc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:26.662215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cf802-387c-42ce-9e3a-267c12d1fc58 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:26.662913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cf802-387c-42ce-9e3a-267c12d1fc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:26.663386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cf802-387c-42ce-9e3a-267c12d1fc58 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:26.664780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cf802-387c-42ce-9e3a-267c12d1fc58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:26.668099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cf802-387c-42ce-9e3a-267c12d1fc58 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:26.724643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f88c-2b13-4d2e-a30a-63da64459cd0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:26.737225 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3384/6695] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:26 2022] POST /identity/v3/projects => generated 365 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:26.741073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f88c-2b13-4d2e-a30a-63da64459cd0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:26.864281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4c7e8-934c-4e94-a882-44938fd068da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:26.865085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4c7e8-934c-4e94-a882-44938fd068da admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:26.865487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4c7e8-934c-4e94-a882-44938fd068da admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:26.865879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4c7e8-934c-4e94-a882-44938fd068da admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:26.866644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4c7e8-934c-4e94-a882-44938fd068da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:26.873596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4c7e8-934c-4e94-a882-44938fd068da admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:26.921320 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3312/6696] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:26 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:26.922745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cf802-387c-42ce-9e3a-267c12d1fc58 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:26.966686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cf802-387c-42ce-9e3a-267c12d1fc58 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:26.974408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb835e6-bf9b-4347-a1a3-580be494b95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:26.975288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb835e6-bf9b-4347-a1a3-580be494b95e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:26.975748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb835e6-bf9b-4347-a1a3-580be494b95e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:26.976217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb835e6-bf9b-4347-a1a3-580be494b95e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:26.977069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb835e6-bf9b-4347-a1a3-580be494b95e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:26.979463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb835e6-bf9b-4347-a1a3-580be494b95e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:26.993410 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3385/6697] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:26.997413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4c7e8-934c-4e94-a882-44938fd068da admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:27.039516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4c7e8-934c-4e94-a882-44938fd068da admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:27.064702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efe05a-c37f-447a-af67-ad0e09fcc664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:27.066627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efe05a-c37f-447a-af67-ad0e09fcc664 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:27.068979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efe05a-c37f-447a-af67-ad0e09fcc664 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:27.069290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efe05a-c37f-447a-af67-ad0e09fcc664 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:27.070000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efe05a-c37f-447a-af67-ad0e09fcc664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:27.080070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efe05a-c37f-447a-af67-ad0e09fcc664 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:27.127786 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3313/6698] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:13:27.150550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb835e6-bf9b-4347-a1a3-580be494b95e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:27.163107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:27.169045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb835e6-bf9b-4347-a1a3-580be494b95e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:27.178207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5a8c8-fb43-4ebe-87be-c43195463d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:27.181660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5a8c8-fb43-4ebe-87be-c43195463d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:27.182479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5a8c8-fb43-4ebe-87be-c43195463d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:27.183122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5a8c8-fb43-4ebe-87be-c43195463d84 admin admin] PATH_INFO: `/v3/projects/0946f0650f4a478ea203f724f77785b5/users/bc34ef7e503547cf9222a41dc41f686e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:27.209796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5a8c8-fb43-4ebe-87be-c43195463d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0946f0650f4a478ea203f724f77785b5, user_id=bc34ef7e503547cf9222a41dc41f686e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:27.213639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5a8c8-fb43-4ebe-87be-c43195463d84 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:27.238344 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3386/6699] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:27 2022] PUT /identity/v3/projects/0946f0650f4a478ea203f724f77785b5/users/bc34ef7e503547cf9222a41dc41f686e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:27.252477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea49d30-9a38-444f-9229-e2b8ffb553d5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:27.252754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea49d30-9a38-444f-9229-e2b8ffb553d5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:27.253725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea49d30-9a38-444f-9229-e2b8ffb553d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:27.301770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7ea49d30-9a38-444f-9229-e2b8ffb553d5 None None] MFA Rules not processed for user `bc34ef7e503547cf9222a41dc41f686e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:27.303346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7ea49d30-9a38-444f-9229-e2b8ffb553d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:27.390119 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3314/6700] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:27.396765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5a8c8-fb43-4ebe-87be-c43195463d84 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:27.471379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5a8c8-fb43-4ebe-87be-c43195463d84 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:27.483959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffce8c-4663-4c8e-9c70-b98acdfee388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:27.484661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffce8c-4663-4c8e-9c70-b98acdfee388 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:27.484919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffce8c-4663-4c8e-9c70-b98acdfee388 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:27.485223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffce8c-4663-4c8e-9c70-b98acdfee388 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:27.485881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffce8c-4663-4c8e-9c70-b98acdfee388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:27.487847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffce8c-4663-4c8e-9c70-b98acdfee388 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:27.531321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea49d30-9a38-444f-9229-e2b8ffb553d5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:27.532646 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3387/6701] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:27 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:27.547841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea49d30-9a38-444f-9229-e2b8ffb553d5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:27.626648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:27.645494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d74187-fc8b-4f23-a209-c48f4850c06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:27.646511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d74187-fc8b-4f23-a209-c48f4850c06a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:27.646814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d74187-fc8b-4f23-a209-c48f4850c06a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:27.647099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d74187-fc8b-4f23-a209-c48f4850c06a admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:27.647774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d74187-fc8b-4f23-a209-c48f4850c06a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:27.652045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d74187-fc8b-4f23-a209-c48f4850c06a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:27.688201 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3315/6702] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:27 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:27.690878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffce8c-4663-4c8e-9c70-b98acdfee388 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:27.703571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffce8c-4663-4c8e-9c70-b98acdfee388 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:27.716960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247a205-2eab-486c-b47c-e3dbe9203e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:27.717602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247a205-2eab-486c-b47c-e3dbe9203e8d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:27.717893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247a205-2eab-486c-b47c-e3dbe9203e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:27.718374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247a205-2eab-486c-b47c-e3dbe9203e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:27.719041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247a205-2eab-486c-b47c-e3dbe9203e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:27.726130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247a205-2eab-486c-b47c-e3dbe9203e8d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:27.743467 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3388/6703] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:27.744921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d74187-fc8b-4f23-a209-c48f4850c06a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:27.757119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d74187-fc8b-4f23-a209-c48f4850c06a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:27.768374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d50c6f-e327-48ea-9210-81c0d958b83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:27.769284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d50c6f-e327-48ea-9210-81c0d958b83b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:27.769678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d50c6f-e327-48ea-9210-81c0d958b83b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:27.770067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d50c6f-e327-48ea-9210-81c0d958b83b admin admin] PATH_INFO: `/v3/projects/0278b5398bf8479eb776c31065ab6e0b/users/3bdf4c013b9a495f9909d9ca4c7a36b4/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:27.790479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d50c6f-e327-48ea-9210-81c0d958b83b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0278b5398bf8479eb776c31065ab6e0b, user_id=3bdf4c013b9a495f9909d9ca4c7a36b4, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:27.793737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d50c6f-e327-48ea-9210-81c0d958b83b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:27.845330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247a205-2eab-486c-b47c-e3dbe9203e8d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:27.857401 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3316/6704] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:27 2022] PUT /identity/v3/projects/0278b5398bf8479eb776c31065ab6e0b/users/3bdf4c013b9a495f9909d9ca4c7a36b4/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:27.864800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247a205-2eab-486c-b47c-e3dbe9203e8d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:28.005804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaab2d-a826-4333-9c58-e0aea07e6356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:28.006478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaab2d-a826-4333-9c58-e0aea07e6356 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:28.006790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaab2d-a826-4333-9c58-e0aea07e6356 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:28.006986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaab2d-a826-4333-9c58-e0aea07e6356 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:28.007823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedaab2d-a826-4333-9c58-e0aea07e6356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:28.009446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedaab2d-a826-4333-9c58-e0aea07e6356 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:28.051598 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3389/6705] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:28.052429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d50c6f-e327-48ea-9210-81c0d958b83b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:28.071286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d50c6f-e327-48ea-9210-81c0d958b83b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:28.085246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d53e48-0222-41ad-a4ef-a1cbc65eb5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:28.091040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d53e48-0222-41ad-a4ef-a1cbc65eb5af admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:28.091959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d53e48-0222-41ad-a4ef-a1cbc65eb5af admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:28.092371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d53e48-0222-41ad-a4ef-a1cbc65eb5af admin admin] PATH_INFO: `/v3/projects/0278b5398bf8479eb776c31065ab6e0b/users/3bdf4c013b9a495f9909d9ca4c7a36b4/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:28.098870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d53e48-0222-41ad-a4ef-a1cbc65eb5af admin admin] RBAC: Authorizing `identity:create_grant(project_id=0278b5398bf8479eb776c31065ab6e0b, user_id=3bdf4c013b9a495f9909d9ca4c7a36b4, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:28.100408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d53e48-0222-41ad-a4ef-a1cbc65eb5af admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:28.179377 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3317/6706] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:28 2022] PUT /identity/v3/projects/0278b5398bf8479eb776c31065ab6e0b/users/3bdf4c013b9a495f9909d9ca4c7a36b4/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:28.190821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0af0f-7562-4041-9221-431467713ce3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:28.192100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0af0f-7562-4041-9221-431467713ce3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:28.193582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0af0f-7562-4041-9221-431467713ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:28.277154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ead0af0f-7562-4041-9221-431467713ce3 None None] MFA Rules not processed for user `3bdf4c013b9a495f9909d9ca4c7a36b4`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:28.278959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ead0af0f-7562-4041-9221-431467713ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:28.407035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:28.453681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fbb3e-5fed-46ca-9d6c-96af2abf537f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:28.454175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fbb3e-5fed-46ca-9d6c-96af2abf537f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:28.454305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fbb3e-5fed-46ca-9d6c-96af2abf537f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:28.521798 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3390/6707] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:28.522770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0af0f-7562-4041-9221-431467713ce3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:28.546452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7d4fbb3e-5fed-46ca-9d6c-96af2abf537f None None] MFA Rules not processed for user `079a84f13e1443519dd360c46b296815`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:28.548560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0af0f-7562-4041-9221-431467713ce3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:28.553176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7d4fbb3e-5fed-46ca-9d6c-96af2abf537f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:28.565399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af60c16-76f6-46f1-83db-5fb9901043df tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:28.566075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af60c16-76f6-46f1-83db-5fb9901043df tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:28.566368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af60c16-76f6-46f1-83db-5fb9901043df tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:28.568106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af60c16-76f6-46f1-83db-5fb9901043df tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:28.569036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af60c16-76f6-46f1-83db-5fb9901043df tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:28.571227 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-0af60c16-76f6-46f1-83db-5fb9901043df tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:28.571227 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:28.571227 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:28.582113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af60c16-76f6-46f1-83db-5fb9901043df tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:28.618654 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3391/6708] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:13:28 2022] POST /identity/v3/regions => generated 224 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:28.622815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af60c16-76f6-46f1-83db-5fb9901043df tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:28.675498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af60c16-76f6-46f1-83db-5fb9901043df tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:28.683684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8be296-bac3-47d1-a4aa-3b43e26276b5 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:28.684318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8be296-bac3-47d1-a4aa-3b43e26276b5 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:28.684715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8be296-bac3-47d1-a4aa-3b43e26276b5 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:28.685391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8be296-bac3-47d1-a4aa-3b43e26276b5 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:28.686240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8be296-bac3-47d1-a4aa-3b43e26276b5 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:28.689291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8be296-bac3-47d1-a4aa-3b43e26276b5 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:28.734691 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3392/6709] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:13:28 2022] POST /identity/v3/regions => generated 222 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:28.738072 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3318/6710] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:28.738296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8be296-bac3-47d1-a4aa-3b43e26276b5 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:28.748481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fbb3e-5fed-46ca-9d6c-96af2abf537f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:28.751010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7d4fbb3e-5fed-46ca-9d6c-96af2abf537f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:28.785671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8be296-bac3-47d1-a4aa-3b43e26276b5 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:28.792129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60048b04-a4cc-43b5-8220-1cccf9b5d078 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:28.792880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60048b04-a4cc-43b5-8220-1cccf9b5d078 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:28.793356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60048b04-a4cc-43b5-8220-1cccf9b5d078 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:28.794139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60048b04-a4cc-43b5-8220-1cccf9b5d078 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions/0a2058ef-84f8-4934-90a9-26d92e9b3473` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:28.794875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60048b04-a4cc-43b5-8220-1cccf9b5d078 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:create_region(region_id=0a2058ef-84f8-4934-90a9-26d92e9b3473)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:28.797200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60048b04-a4cc-43b5-8220-1cccf9b5d078 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:28.829303 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3393/6711] 10.222.0.16 () {60 vars in 1259 bytes} [Sat Sep 3 21:13:28 2022] PUT /identity/v3/regions/0a2058ef-84f8-4934-90a9-26d92e9b3473 => generated 232 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:28.836015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60048b04-a4cc-43b5-8220-1cccf9b5d078 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:28.874230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60048b04-a4cc-43b5-8220-1cccf9b5d078 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:28.879543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0629f-6e56-4da7-a0a1-91c8d38a8c52 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:28.881690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0629f-6e56-4da7-a0a1-91c8d38a8c52 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:28.882013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0629f-6e56-4da7-a0a1-91c8d38a8c52 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:28.882302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0629f-6e56-4da7-a0a1-91c8d38a8c52 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions/0a2058ef-84f8-4934-90a9-26d92e9b3473` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:28.882917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0629f-6e56-4da7-a0a1-91c8d38a8c52 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:delete_region(region_id=0a2058ef-84f8-4934-90a9-26d92e9b3473)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:28.884558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0629f-6e56-4da7-a0a1-91c8d38a8c52 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:28.941644 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3394/6712] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Sep 3 21:13:28 2022] DELETE /identity/v3/regions/0a2058ef-84f8-4934-90a9-26d92e9b3473 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:28.946638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0629f-6e56-4da7-a0a1-91c8d38a8c52 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:28.951045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fbb3e-5fed-46ca-9d6c-96af2abf537f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:28.975678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade2408-8cf6-425d-a79d-d665bb84bad9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:28.976311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade2408-8cf6-425d-a79d-d665bb84bad9 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:28.976599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade2408-8cf6-425d-a79d-d665bb84bad9 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:28.976876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade2408-8cf6-425d-a79d-d665bb84bad9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:28.987846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade2408-8cf6-425d-a79d-d665bb84bad9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:28.990456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade2408-8cf6-425d-a79d-d665bb84bad9 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:28.998245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:29.015498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0629f-6e56-4da7-a0a1-91c8d38a8c52 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:29.027553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbcf88-27e1-4ddc-89a1-7124496e08e8 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:29.028116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbcf88-27e1-4ddc-89a1-7124496e08e8 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:29.028457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbcf88-27e1-4ddc-89a1-7124496e08e8 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:29.028739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbcf88-27e1-4ddc-89a1-7124496e08e8 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:29.034846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbcf88-27e1-4ddc-89a1-7124496e08e8 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:29.036498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbcf88-27e1-4ddc-89a1-7124496e08e8 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:29.040388 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3319/6713] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:13:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:13:29.119361 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3395/6714] 10.222.0.16 () {60 vars in 1150 bytes} [Sat Sep 3 21:13:28 2022] POST /identity/v3/regions => generated 254 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:29.123958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade2408-8cf6-425d-a79d-d665bb84bad9 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:29.151484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade2408-8cf6-425d-a79d-d665bb84bad9 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:29.194813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41913a6-60cf-4602-8e92-35d9e3aef8ca tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:29.195480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41913a6-60cf-4602-8e92-35d9e3aef8ca tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:29.195769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41913a6-60cf-4602-8e92-35d9e3aef8ca tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:29.196050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41913a6-60cf-4602-8e92-35d9e3aef8ca tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions/571b9d6b5cdb4f7a851022bcb0aab35c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:29.196721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41913a6-60cf-4602-8e92-35d9e3aef8ca tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:update_region(region_id=571b9d6b5cdb4f7a851022bcb0aab35c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:29.202656 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-b41913a6-60cf-4602-8e92-35d9e3aef8ca tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:29.202656 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:29.202656 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:29.206449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41913a6-60cf-4602-8e92-35d9e3aef8ca tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:29.261846 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3320/6715] 10.222.0.16 () {60 vars in 1250 bytes} [Sat Sep 3 21:13:29 2022] PATCH /identity/v3/regions/571b9d6b5cdb4f7a851022bcb0aab35c => generated 254 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:29.266809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbcf88-27e1-4ddc-89a1-7124496e08e8 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:29.296218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbcf88-27e1-4ddc-89a1-7124496e08e8 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:29.386934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a24d6-51ee-4d7a-9c4f-761eea564cbc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:29.391400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a24d6-51ee-4d7a-9c4f-761eea564cbc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:29.391752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a24d6-51ee-4d7a-9c4f-761eea564cbc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:29.391983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a24d6-51ee-4d7a-9c4f-761eea564cbc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions/571b9d6b5cdb4f7a851022bcb0aab35c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:29.392708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a24d6-51ee-4d7a-9c4f-761eea564cbc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:get_region(region_id=571b9d6b5cdb4f7a851022bcb0aab35c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:29.393784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a24d6-51ee-4d7a-9c4f-761eea564cbc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:29.413636 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3396/6716] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:13:29 2022] GET /identity/v3/regions/571b9d6b5cdb4f7a851022bcb0aab35c => generated 254 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:29.418557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41913a6-60cf-4602-8e92-35d9e3aef8ca tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:29.455594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41913a6-60cf-4602-8e92-35d9e3aef8ca tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:29.478768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f8b1b-80bf-4bd9-9b86-536c4ff25474 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:29.482604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f8b1b-80bf-4bd9-9b86-536c4ff25474 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:29.484169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f8b1b-80bf-4bd9-9b86-536c4ff25474 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:29.485002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f8b1b-80bf-4bd9-9b86-536c4ff25474 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions/571b9d6b5cdb4f7a851022bcb0aab35c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:29.486536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f8b1b-80bf-4bd9-9b86-536c4ff25474 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:delete_region(region_id=571b9d6b5cdb4f7a851022bcb0aab35c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:29.491161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f8b1b-80bf-4bd9-9b86-536c4ff25474 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:29.540418 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3321/6717] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:13:29 2022] DELETE /identity/v3/regions/571b9d6b5cdb4f7a851022bcb0aab35c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:29.542091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a24d6-51ee-4d7a-9c4f-761eea564cbc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:29.567241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a24d6-51ee-4d7a-9c4f-761eea564cbc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:29.603627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294ff1f-0d50-40d0-81a4-9de00328ba2c tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:29.604582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294ff1f-0d50-40d0-81a4-9de00328ba2c tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:29.605079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294ff1f-0d50-40d0-81a4-9de00328ba2c tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:29.605534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294ff1f-0d50-40d0-81a4-9de00328ba2c tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:29.606880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294ff1f-0d50-40d0-81a4-9de00328ba2c tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:29.608784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294ff1f-0d50-40d0-81a4-9de00328ba2c tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:29.623554 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3397/6718] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Sep 3 21:13:29 2022] GET /identity/v3/regions => generated 668 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:13:29.627678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f8b1b-80bf-4bd9-9b86-536c4ff25474 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:29.647431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f8b1b-80bf-4bd9-9b86-536c4ff25474 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:29.655854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a1292-eda2-4c69-956b-a268e57166bc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:29.657322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a1292-eda2-4c69-956b-a268e57166bc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:29.657904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a1292-eda2-4c69-956b-a268e57166bc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:29.658493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a1292-eda2-4c69-956b-a268e57166bc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions/571b9d6b5cdb4f7a851022bcb0aab35c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:29.662491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a1292-eda2-4c69-956b-a268e57166bc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:delete_region(region_id=571b9d6b5cdb4f7a851022bcb0aab35c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:29.665135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a1292-eda2-4c69-956b-a268e57166bc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-117a1292-eda2-4c69-956b-a268e57166bc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Could not find region: 571b9d6b5cdb4f7a851022bcb0aab35c.: keystone.exception.RegionNotFound: Could not find region: 571b9d6b5cdb4f7a851022bcb0aab35c. Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 129, in delete_region Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ret = self.driver.delete_region(region_id) Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 152, in delete_region Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = self._get_region(session, region_id) Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 103, in _get_region Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 571b9d6b5cdb4f7a851022bcb0aab35c. Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:13:29.692373 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/regions.py", line 92, in delete Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return PROVIDERS.catalog_api.delete_region( Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 135, in delete_region Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 571b9d6b5cdb4f7a851022bcb0aab35c. Sep 03 21:13:29.693894 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:13:29.701927 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3322/6719] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:13:29 2022] DELETE /identity/v3/regions/571b9d6b5cdb4f7a851022bcb0aab35c => generated 112 bytes in 75 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:13:29.703433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294ff1f-0d50-40d0-81a4-9de00328ba2c tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:29.725800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294ff1f-0d50-40d0-81a4-9de00328ba2c tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:29.732991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb49069-f922-429d-9b41-f3b19b53a7be tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:29.733802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49069-f922-429d-9b41-f3b19b53a7be tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:29.734233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49069-f922-429d-9b41-f3b19b53a7be tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:29.734618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49069-f922-429d-9b41-f3b19b53a7be tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:29.735493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb49069-f922-429d-9b41-f3b19b53a7be tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:29.737088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb49069-f922-429d-9b41-f3b19b53a7be tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:29.746952 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3398/6720] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Sep 3 21:13:29 2022] GET /identity/v3/regions => generated 668 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:29.752482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a1292-eda2-4c69-956b-a268e57166bc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:29.767261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a1292-eda2-4c69-956b-a268e57166bc tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:29.784948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc2067-bbff-4754-b3b1-b6fd9f1d5032 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:29.785942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc2067-bbff-4754-b3b1-b6fd9f1d5032 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:29.787280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc2067-bbff-4754-b3b1-b6fd9f1d5032 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:29.787280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc2067-bbff-4754-b3b1-b6fd9f1d5032 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:29.787775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc2067-bbff-4754-b3b1-b6fd9f1d5032 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:29.788926 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-f7fc2067-bbff-4754-b3b1-b6fd9f1d5032 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:29.788926 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:29.788926 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:29.792414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc2067-bbff-4754-b3b1-b6fd9f1d5032 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:29.881017 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3323/6721] 10.222.0.16 () {60 vars in 1150 bytes} [Sat Sep 3 21:13:29 2022] POST /identity/v3/regions => generated 253 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:29.882353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb49069-f922-429d-9b41-f3b19b53a7be tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:29.901588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb49069-f922-429d-9b41-f3b19b53a7be tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:29.952169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1f000-fd7d-4816-ba59-ed3a50f4d1aa tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:29.952812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1f000-fd7d-4816-ba59-ed3a50f4d1aa tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:29.953150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1f000-fd7d-4816-ba59-ed3a50f4d1aa tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:29.953434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1f000-fd7d-4816-ba59-ed3a50f4d1aa tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:29.954062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1f000-fd7d-4816-ba59-ed3a50f4d1aa tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Adding query filter params (parent_region_id=2112b7a728bf439ab207ca2e85e971c9) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:29.954374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1f000-fd7d-4816-ba59-ed3a50f4d1aa tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:29.955468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1f000-fd7d-4816-ba59-ed3a50f4d1aa tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:29.985144 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3399/6722] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:13:29 2022] GET /identity/v3/regions?parent_region_id=2112b7a728bf439ab207ca2e85e971c9 => generated 399 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:29.990033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc2067-bbff-4754-b3b1-b6fd9f1d5032 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:30.015185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc2067-bbff-4754-b3b1-b6fd9f1d5032 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:30.034698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198ef72-43ec-4fac-a58c-d85324a5e3f6 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:30.035575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198ef72-43ec-4fac-a58c-d85324a5e3f6 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:30.036054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198ef72-43ec-4fac-a58c-d85324a5e3f6 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:30.038758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198ef72-43ec-4fac-a58c-d85324a5e3f6 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions/288ad46755394eba9f2a21e7cdbeaa16` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:30.041112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198ef72-43ec-4fac-a58c-d85324a5e3f6 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:delete_region(region_id=288ad46755394eba9f2a21e7cdbeaa16)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:30.052929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198ef72-43ec-4fac-a58c-d85324a5e3f6 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:30.093985 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3324/6723] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:13:29 2022] DELETE /identity/v3/regions/288ad46755394eba9f2a21e7cdbeaa16 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:30.098454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1f000-fd7d-4816-ba59-ed3a50f4d1aa tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:30.110809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:30.126283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1f000-fd7d-4816-ba59-ed3a50f4d1aa tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:30.175620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfb19e-46a5-4abd-88fd-b9e0f22c9684 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:30.179132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfb19e-46a5-4abd-88fd-b9e0f22c9684 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:30.179707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfb19e-46a5-4abd-88fd-b9e0f22c9684 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:30.180239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfb19e-46a5-4abd-88fd-b9e0f22c9684 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions/603e7a033d714f57ba8ce8773ace0354` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:30.188645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfb19e-46a5-4abd-88fd-b9e0f22c9684 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:delete_region(region_id=603e7a033d714f57ba8ce8773ace0354)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:30.192834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfb19e-46a5-4abd-88fd-b9e0f22c9684 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:30.226258 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3400/6724] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:13:30 2022] DELETE /identity/v3/regions/603e7a033d714f57ba8ce8773ace0354 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:30.230339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198ef72-43ec-4fac-a58c-d85324a5e3f6 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:30.257504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198ef72-43ec-4fac-a58c-d85324a5e3f6 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:30.296920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa4d8a-33eb-46a8-9d4f-7cdab71902d3 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdf4c013b9a495f9909d9ca4c7a36b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0278b5398bf8479eb776c31065ab6e0b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:30.298573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa4d8a-33eb-46a8-9d4f-7cdab71902d3 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:30.299636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa4d8a-33eb-46a8-9d4f-7cdab71902d3 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:30.302553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa4d8a-33eb-46a8-9d4f-7cdab71902d3 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] PATH_INFO: `/v3/regions/2112b7a728bf439ab207ca2e85e971c9` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:30.303951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa4d8a-33eb-46a8-9d4f-7cdab71902d3 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorizing `identity:delete_region(region_id=2112b7a728bf439ab207ca2e85e971c9)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:30.308297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa4d8a-33eb-46a8-9d4f-7cdab71902d3 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:30.338080 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3325/6725] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:13:30 2022] DELETE /identity/v3/regions/2112b7a728bf439ab207ca2e85e971c9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:30.342333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfb19e-46a5-4abd-88fd-b9e0f22c9684 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:30.502053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfb19e-46a5-4abd-88fd-b9e0f22c9684 tempest-RegionsTestJSON-1093251202 tempest-RegionsTestJSON-1093251202-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:30.570254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:30.571961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:30.572362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:30.572749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] PATH_INFO: `/v3/users/bc34ef7e503547cf9222a41dc41f686e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:30.574203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc34ef7e503547cf9222a41dc41f686e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:30.581818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:30.740215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc34ef7e503547cf9222a41dc41f686e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:30.788644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc34ef7e503547cf9222a41dc41f686e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:30.802290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc34ef7e503547cf9222a41dc41f686e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:30.802889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc34ef7e503547cf9222a41dc41f686e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:30.804327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc34ef7e503547cf9222a41dc41f686e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:30.864691 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3401/6726] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:30 2022] DELETE /identity/v3/users/bc34ef7e503547cf9222a41dc41f686e => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:30.872677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a976a2c-1410-4c7d-8043-7de2e84aa098 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:30.873207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a976a2c-1410-4c7d-8043-7de2e84aa098 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:30.873313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a976a2c-1410-4c7d-8043-7de2e84aa098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:30.943862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4a976a2c-1410-4c7d-8043-7de2e84aa098 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:30.950547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4a976a2c-1410-4c7d-8043-7de2e84aa098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:31.032875 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3326/6727] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:31.116693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:31.118251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:31.180024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:31.220620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e96cf0-822f-4c9c-a6c1-d36baa30719c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:31.226043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08324fe-9fad-427d-9dcb-251e45c29bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:31.227372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08324fe-9fad-427d-9dcb-251e45c29bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:31.228133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08324fe-9fad-427d-9dcb-251e45c29bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:31.228626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08324fe-9fad-427d-9dcb-251e45c29bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:31.240010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08324fe-9fad-427d-9dcb-251e45c29bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:31.243563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08324fe-9fad-427d-9dcb-251e45c29bf9 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:31.257593 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3402/6728] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:13:32.627056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a976a2c-1410-4c7d-8043-7de2e84aa098 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:32.630297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4a976a2c-1410-4c7d-8043-7de2e84aa098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:32.658614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a976a2c-1410-4c7d-8043-7de2e84aa098 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:32.670779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:32.671648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:32.672114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:32.672574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] PATH_INFO: `/v3/users/01c449f3fd1946508ab455af5b231c14` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:32.673962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c449f3fd1946508ab455af5b231c14)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:32.676087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:32.702389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:32.780328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c449f3fd1946508ab455af5b231c14'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:32.815908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c449f3fd1946508ab455af5b231c14'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:32.816767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c449f3fd1946508ab455af5b231c14'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:32.833081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c449f3fd1946508ab455af5b231c14'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:32.867834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c449f3fd1946508ab455af5b231c14'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:32.882226 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3327/6729] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:32 2022] DELETE /identity/v3/users/01c449f3fd1946508ab455af5b231c14 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:33.437069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08324fe-9fad-427d-9dcb-251e45c29bf9 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:33.456250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08324fe-9fad-427d-9dcb-251e45c29bf9 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:33.580876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:33.582324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:33.585376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:33.586846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:33.587152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:33.587430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] PATH_INFO: `/v3/projects/ea7a4854a6a0475d99f2a5664f741533` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:33.588618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea7a4854a6a0475d99f2a5664f741533)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:33.590694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:33.604790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503c18d-c07c-483f-acbf-bcf3aa186c42 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:33.623333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:33.624320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:33.624786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:33.625247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] PATH_INFO: `/v3/projects/0946f0650f4a478ea203f724f77785b5` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:33.627065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0946f0650f4a478ea203f724f77785b5)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:33.629359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:33.752556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea7a4854a6a0475d99f2a5664f741533', 'request_id': 'req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:33.774869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea7a4854a6a0475d99f2a5664f741533', 'request_id': 'req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:33.795458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea7a4854a6a0475d99f2a5664f741533', 'request_id': 'req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:33.833365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0946f0650f4a478ea203f724f77785b5', 'request_id': 'req-0d17fd33-5769-41b6-a64f-9e922f671baf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:33.844426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] The token cache is being invalidate because project ea7a4854a6a0475d99f2a5664f741533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:33.844738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:33.847645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:33.849466 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3403/6730] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:33 2022] DELETE /identity/v3/projects/ea7a4854a6a0475d99f2a5664f741533 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:33.853724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:33.855184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:33.858126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0946f0650f4a478ea203f724f77785b5', 'request_id': 'req-0d17fd33-5769-41b6-a64f-9e922f671baf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:33.880977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0946f0650f4a478ea203f724f77785b5', 'request_id': 'req-0d17fd33-5769-41b6-a64f-9e922f671baf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:33.912146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] The token cache is being invalidate because project 0946f0650f4a478ea203f724f77785b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:33.912565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:33.926029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:33.933812 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3328/6731] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:33 2022] DELETE /identity/v3/projects/0946f0650f4a478ea203f724f77785b5 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:33.935938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:33.941947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:33.993569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb78bbf-e0b7-4f0c-9069-5f554e33b1f7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:33.999859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:34.117197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17fd33-5769-41b6-a64f-9e922f671baf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:34.137655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:34.138312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:34.139407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:34.140473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] PATH_INFO: `/v3/users/3bdf4c013b9a495f9909d9ca4c7a36b4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:34.144198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bdf4c013b9a495f9909d9ca4c7a36b4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:34.148428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:34.148428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:34.148428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:34.148428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] PATH_INFO: `/v3/users/c276bdddd15f4682a19ba89403a1aefc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:34.148988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c276bdddd15f4682a19ba89403a1aefc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:34.149745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:34.150406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:34.186153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:34.249817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c276bdddd15f4682a19ba89403a1aefc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:34.265204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bdf4c013b9a495f9909d9ca4c7a36b4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:34.272550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c276bdddd15f4682a19ba89403a1aefc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:34.283207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bdf4c013b9a495f9909d9ca4c7a36b4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:34.284855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bdf4c013b9a495f9909d9ca4c7a36b4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:34.285854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c276bdddd15f4682a19ba89403a1aefc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:34.286670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c276bdddd15f4682a19ba89403a1aefc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:34.287803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c276bdddd15f4682a19ba89403a1aefc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:34.299862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bdf4c013b9a495f9909d9ca4c7a36b4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:34.308667 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3404/6732] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:33 2022] DELETE /identity/v3/users/c276bdddd15f4682a19ba89403a1aefc => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:34.311101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bdf4c013b9a495f9909d9ca4c7a36b4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:34.321895 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3329/6733] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:33 2022] DELETE /identity/v3/users/3bdf4c013b9a495f9909d9ca4c7a36b4 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:34.942200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:34.943672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:35.128858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01500fd-0934-47f6-9de6-b4a2da8e8cf6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:35.216612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:35.270313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:35.271162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:35.271556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:35.272025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] PATH_INFO: `/v3/projects/d226984bcb6d4be48a4f884c26010afa` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:35.277727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d226984bcb6d4be48a4f884c26010afa)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:35.280476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:35.345574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd226984bcb6d4be48a4f884c26010afa', 'request_id': 'req-bf08a306-1c81-4e42-b202-301ec1f81d83'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:35.358330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd226984bcb6d4be48a4f884c26010afa', 'request_id': 'req-bf08a306-1c81-4e42-b202-301ec1f81d83'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:35.369860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd226984bcb6d4be48a4f884c26010afa', 'request_id': 'req-bf08a306-1c81-4e42-b202-301ec1f81d83'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:35.386262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] The token cache is being invalidate because project d226984bcb6d4be48a4f884c26010afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:35.386674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:35.387550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:35.390138 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3405/6734] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:34 2022] DELETE /identity/v3/projects/d226984bcb6d4be48a4f884c26010afa => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:35.997275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:35.999131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:36.018911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57008fcf-dbb4-4b2d-a34e-f07914bd01bb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:36.100208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6324762-149e-4363-997e-1303edae6153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:36.100882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6324762-149e-4363-997e-1303edae6153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:36.101216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6324762-149e-4363-997e-1303edae6153 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:36.101517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6324762-149e-4363-997e-1303edae6153 admin admin] PATH_INFO: `/v3/users/079a84f13e1443519dd360c46b296815` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:36.104416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6324762-149e-4363-997e-1303edae6153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=079a84f13e1443519dd360c46b296815)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:36.108579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6324762-149e-4363-997e-1303edae6153 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:36.224068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b6324762-149e-4363-997e-1303edae6153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '079a84f13e1443519dd360c46b296815'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:36.248142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b6324762-149e-4363-997e-1303edae6153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '079a84f13e1443519dd360c46b296815'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:36.249179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b6324762-149e-4363-997e-1303edae6153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '079a84f13e1443519dd360c46b296815'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:36.257934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b6324762-149e-4363-997e-1303edae6153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '079a84f13e1443519dd360c46b296815'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:36.269094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b6324762-149e-4363-997e-1303edae6153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '079a84f13e1443519dd360c46b296815'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:36.280148 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3330/6735] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:35 2022] DELETE /identity/v3/users/079a84f13e1443519dd360c46b296815 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:36.329652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:36.332854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:36.363178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08a306-1c81-4e42-b202-301ec1f81d83 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:36.462425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:36.463892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:36.466490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:36.466713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102 admin admin] PATH_INFO: `/v3/projects/0278b5398bf8479eb776c31065ab6e0b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:36.469147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0278b5398bf8479eb776c31065ab6e0b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:36.473421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:36.539514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0278b5398bf8479eb776c31065ab6e0b', 'request_id': 'req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:36.554814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0278b5398bf8479eb776c31065ab6e0b', 'request_id': 'req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:36.566226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0278b5398bf8479eb776c31065ab6e0b', 'request_id': 'req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:36.571589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:36.581338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102 admin admin] The token cache is being invalidate because project 0278b5398bf8479eb776c31065ab6e0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:36.581679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:36.582603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-87121085-4c2d-4ec2-b7c7-6d1e20b5e102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:36.584760 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3406/6736] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:36 2022] DELETE /identity/v3/projects/0278b5398bf8479eb776c31065ab6e0b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:36.590488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ec09e-474d-4a7f-8f92-e01012c5a8b1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:36.591209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ec09e-474d-4a7f-8f92-e01012c5a8b1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:36.591684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ec09e-474d-4a7f-8f92-e01012c5a8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:36.620047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8e5ec09e-474d-4a7f-8f92-e01012c5a8b1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:36.621698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8e5ec09e-474d-4a7f-8f92-e01012c5a8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:36.633652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:36.753558 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3331/6737] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:36.763809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3a609-decc-4830-b44c-f643126fb4fb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:36.764122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3a609-decc-4830-b44c-f643126fb4fb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:36.764418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3a609-decc-4830-b44c-f643126fb4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:36.818955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d9e3a609-decc-4830-b44c-f643126fb4fb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:36.820378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d9e3a609-decc-4830-b44c-f643126fb4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:36.831997 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3407/6738] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:36.836536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3a609-decc-4830-b44c-f643126fb4fb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:36.838035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ec09e-474d-4a7f-8f92-e01012c5a8b1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:36.838645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d9e3a609-decc-4830-b44c-f643126fb4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:36.853837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ec09e-474d-4a7f-8f92-e01012c5a8b1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:36.862871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7433b-c39c-435d-8a30-88f3ecadc479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:36.864932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7433b-c39c-435d-8a30-88f3ecadc479 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:36.865165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7433b-c39c-435d-8a30-88f3ecadc479 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:36.866060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7433b-c39c-435d-8a30-88f3ecadc479 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:36.866644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7433b-c39c-435d-8a30-88f3ecadc479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:36.866922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7433b-c39c-435d-8a30-88f3ecadc479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:36.873017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3a609-decc-4830-b44c-f643126fb4fb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:36.876233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7433b-c39c-435d-8a30-88f3ecadc479 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:36.889277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aea02b-376b-4906-9248-ef3dafa1a322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:36.891149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aea02b-376b-4906-9248-ef3dafa1a322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:36.891464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aea02b-376b-4906-9248-ef3dafa1a322 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:36.891788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aea02b-376b-4906-9248-ef3dafa1a322 admin admin] PATH_INFO: `/v3/projects/d09f639e121545c3bc4a91bb3e761f71` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:36.893385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aea02b-376b-4906-9248-ef3dafa1a322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d09f639e121545c3bc4a91bb3e761f71)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:36.896104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aea02b-376b-4906-9248-ef3dafa1a322 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:36.902600 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3332/6739] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:36.903808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7433b-c39c-435d-8a30-88f3ecadc479 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:36.934004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7433b-c39c-435d-8a30-88f3ecadc479 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:36.943680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a974129-ee4e-4b30-9543-29735632b57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:36.945344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974129-ee4e-4b30-9543-29735632b57d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:36.945610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974129-ee4e-4b30-9543-29735632b57d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:36.945924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974129-ee4e-4b30-9543-29735632b57d admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:36.946536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a974129-ee4e-4b30-9543-29735632b57d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:36.949750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a974129-ee4e-4b30-9543-29735632b57d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:36.962237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96aea02b-376b-4906-9248-ef3dafa1a322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd09f639e121545c3bc4a91bb3e761f71', 'request_id': 'req-96aea02b-376b-4906-9248-ef3dafa1a322'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:36.975515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96aea02b-376b-4906-9248-ef3dafa1a322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd09f639e121545c3bc4a91bb3e761f71', 'request_id': 'req-96aea02b-376b-4906-9248-ef3dafa1a322'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:36.990015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96aea02b-376b-4906-9248-ef3dafa1a322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd09f639e121545c3bc4a91bb3e761f71', 'request_id': 'req-96aea02b-376b-4906-9248-ef3dafa1a322'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:37.000165 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3333/6740] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:36 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:37.006424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a974129-ee4e-4b30-9543-29735632b57d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.008788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96aea02b-376b-4906-9248-ef3dafa1a322 admin admin] The token cache is being invalidate because project d09f639e121545c3bc4a91bb3e761f71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:37.009512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96aea02b-376b-4906-9248-ef3dafa1a322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:37.010744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-96aea02b-376b-4906-9248-ef3dafa1a322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:37.012719 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3408/6741] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:36 2022] DELETE /identity/v3/projects/d09f639e121545c3bc4a91bb3e761f71 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:37.018731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a7f7d-f5b7-47f1-8850-fd388c0c7193 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.019171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a7f7d-f5b7-47f1-8850-fd388c0c7193 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.020175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a7f7d-f5b7-47f1-8850-fd388c0c7193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.048083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bd9a7f7d-f5b7-47f1-8850-fd388c0c7193 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:37.051581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bd9a7f7d-f5b7-47f1-8850-fd388c0c7193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:37.086678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a974129-ee4e-4b30-9543-29735632b57d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.092799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc8bc4-0de7-4f18-a814-5347517015e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.093578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc8bc4-0de7-4f18-a814-5347517015e4 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.093942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc8bc4-0de7-4f18-a814-5347517015e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.094299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc8bc4-0de7-4f18-a814-5347517015e4 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.095053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc8bc4-0de7-4f18-a814-5347517015e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.097412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc8bc4-0de7-4f18-a814-5347517015e4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.147475 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3409/6742] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:37.155762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccba74-a377-4fc4-8dbf-1f49bdd79f03 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.156181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccba74-a377-4fc4-8dbf-1f49bdd79f03 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.156687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccba74-a377-4fc4-8dbf-1f49bdd79f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.157905 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3334/6743] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:37 2022] POST /identity/v3/users => generated 284 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:37.162377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc8bc4-0de7-4f18-a814-5347517015e4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.166864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-63bc8bc4-0de7-4f18-a814-5347517015e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:37.190964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e5ccba74-a377-4fc4-8dbf-1f49bdd79f03 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:37.192300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e5ccba74-a377-4fc4-8dbf-1f49bdd79f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:37.208553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc8bc4-0de7-4f18-a814-5347517015e4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.217248 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3410/6744] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:37.219067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccba74-a377-4fc4-8dbf-1f49bdd79f03 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.220518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dd74a-0f9d-4c08-9e00-18d17eae3b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.221199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dd74a-0f9d-4c08-9e00-18d17eae3b20 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.221626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dd74a-0f9d-4c08-9e00-18d17eae3b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.221806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dd74a-0f9d-4c08-9e00-18d17eae3b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.225348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854dd74a-0f9d-4c08-9e00-18d17eae3b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.227132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854dd74a-0f9d-4c08-9e00-18d17eae3b20 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.252620 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3335/6745] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:37.256232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dd74a-0f9d-4c08-9e00-18d17eae3b20 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.258555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccba74-a377-4fc4-8dbf-1f49bdd79f03 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.267711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de1002-e447-4c3e-9d51-f4bd7fcfb199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.268603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de1002-e447-4c3e-9d51-f4bd7fcfb199 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.269192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de1002-e447-4c3e-9d51-f4bd7fcfb199 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.269818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de1002-e447-4c3e-9d51-f4bd7fcfb199 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.271127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de1002-e447-4c3e-9d51-f4bd7fcfb199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:37.271704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de1002-e447-4c3e-9d51-f4bd7fcfb199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.274731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dd74a-0f9d-4c08-9e00-18d17eae3b20 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.277686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de1002-e447-4c3e-9d51-f4bd7fcfb199 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.285621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964940c-99e6-46fa-86d0-15580fee0ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.286866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964940c-99e6-46fa-86d0-15580fee0ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.287390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964940c-99e6-46fa-86d0-15580fee0ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.287707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964940c-99e6-46fa-86d0-15580fee0ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.288367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964940c-99e6-46fa-86d0-15580fee0ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.294579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964940c-99e6-46fa-86d0-15580fee0ee9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.315170 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3336/6747] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:37.316311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964940c-99e6-46fa-86d0-15580fee0ee9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.320292 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3411/6747] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:37.321310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de1002-e447-4c3e-9d51-f4bd7fcfb199 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.332918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964940c-99e6-46fa-86d0-15580fee0ee9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.341585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ba86f-fc26-4e94-9239-6cabb9f87fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.342689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ba86f-fc26-4e94-9239-6cabb9f87fcb admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.343183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ba86f-fc26-4e94-9239-6cabb9f87fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.343689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ba86f-fc26-4e94-9239-6cabb9f87fcb admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.344768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ba86f-fc26-4e94-9239-6cabb9f87fcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.347532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ba86f-fc26-4e94-9239-6cabb9f87fcb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.349978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de1002-e447-4c3e-9d51-f4bd7fcfb199 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.360797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470e221-248e-459c-9921-4ce68f7284d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.362868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470e221-248e-459c-9921-4ce68f7284d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.363167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470e221-248e-459c-9921-4ce68f7284d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.364755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470e221-248e-459c-9921-4ce68f7284d7 admin admin] PATH_INFO: `/v3/projects/80b48fe72d5144beaba0ac195a3914d3/users/afee6e474cb6418aad4b56ed919166c4/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.392163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470e221-248e-459c-9921-4ce68f7284d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b48fe72d5144beaba0ac195a3914d3, user_id=afee6e474cb6418aad4b56ed919166c4, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.396797 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3337/6748] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:37 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:37.398771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ba86f-fc26-4e94-9239-6cabb9f87fcb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.399969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470e221-248e-459c-9921-4ce68f7284d7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.433197 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3412/6749] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:37 2022] PUT /identity/v3/projects/80b48fe72d5144beaba0ac195a3914d3/users/afee6e474cb6418aad4b56ed919166c4/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:37.436200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824195eb-7888-4348-a06f-643b09f72839 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.436806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824195eb-7888-4348-a06f-643b09f72839 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.437316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824195eb-7888-4348-a06f-643b09f72839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.467067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:37.489179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-824195eb-7888-4348-a06f-643b09f72839 None None] MFA Rules not processed for user `afee6e474cb6418aad4b56ed919166c4`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:37.491049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-824195eb-7888-4348-a06f-643b09f72839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:37.509070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:37.554328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ba86f-fc26-4e94-9239-6cabb9f87fcb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.578182 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3413/6750] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:37.582492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824195eb-7888-4348-a06f-643b09f72839 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.648391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09c117-84b1-4bbd-8689-da1138331529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.650109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09c117-84b1-4bbd-8689-da1138331529 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.650401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09c117-84b1-4bbd-8689-da1138331529 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.650682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09c117-84b1-4bbd-8689-da1138331529 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.651323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09c117-84b1-4bbd-8689-da1138331529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.654764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09c117-84b1-4bbd-8689-da1138331529 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.686477 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3338/6751] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:37 2022] POST /identity/v3/users => generated 291 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:37.690328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09c117-84b1-4bbd-8689-da1138331529 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.703898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824195eb-7888-4348-a06f-643b09f72839 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.708655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09c117-84b1-4bbd-8689-da1138331529 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.714604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418c102-b314-47c4-8897-fb0fe6a8e375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.715285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418c102-b314-47c4-8897-fb0fe6a8e375 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.716126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418c102-b314-47c4-8897-fb0fe6a8e375 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.716532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418c102-b314-47c4-8897-fb0fe6a8e375 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.717871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418c102-b314-47c4-8897-fb0fe6a8e375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.719222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418c102-b314-47c4-8897-fb0fe6a8e375 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.725107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838715f-d4b7-4361-87a8-f2f7b54a36fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.726231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838715f-d4b7-4361-87a8-f2f7b54a36fc admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.726231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838715f-d4b7-4361-87a8-f2f7b54a36fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.726453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838715f-d4b7-4361-87a8-f2f7b54a36fc admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.727067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838715f-d4b7-4361-87a8-f2f7b54a36fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.730884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838715f-d4b7-4361-87a8-f2f7b54a36fc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.751667 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3339/6752] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:37.752370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838715f-d4b7-4361-87a8-f2f7b54a36fc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.767153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838715f-d4b7-4361-87a8-f2f7b54a36fc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.777513 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3414/6753] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:37 2022] POST /identity/v3/projects => generated 367 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:37.777750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a71f7-77b6-4f2c-a7a6-5c2c689f549e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.777750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a71f7-77b6-4f2c-a7a6-5c2c689f549e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.777750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a71f7-77b6-4f2c-a7a6-5c2c689f549e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.777750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a71f7-77b6-4f2c-a7a6-5c2c689f549e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.778681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a71f7-77b6-4f2c-a7a6-5c2c689f549e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.779735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a71f7-77b6-4f2c-a7a6-5c2c689f549e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.781894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418c102-b314-47c4-8897-fb0fe6a8e375 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.792579 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3340/6754] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:37.797591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a71f7-77b6-4f2c-a7a6-5c2c689f549e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.826087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a71f7-77b6-4f2c-a7a6-5c2c689f549e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.874240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418c102-b314-47c4-8897-fb0fe6a8e375 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.881450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95240d5c-98b9-4af0-a4ec-bb3eec74ce7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.882662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95240d5c-98b9-4af0-a4ec-bb3eec74ce7f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.883162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95240d5c-98b9-4af0-a4ec-bb3eec74ce7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.883659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95240d5c-98b9-4af0-a4ec-bb3eec74ce7f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.884377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95240d5c-98b9-4af0-a4ec-bb3eec74ce7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.887984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95240d5c-98b9-4af0-a4ec-bb3eec74ce7f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.912719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7944f0c-b5e5-4844-8c4b-b5001001f1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.913574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7944f0c-b5e5-4844-8c4b-b5001001f1b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.914082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7944f0c-b5e5-4844-8c4b-b5001001f1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.914433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7944f0c-b5e5-4844-8c4b-b5001001f1b7 admin admin] PATH_INFO: `/v3/projects/42698a69a75f4d6583e8dd7df3926aa6/users/ec4d4dfdda4f4f86a6cffadc35d9e68e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.918073 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3415/6755] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:37 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:37.921885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95240d5c-98b9-4af0-a4ec-bb3eec74ce7f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.934623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95240d5c-98b9-4af0-a4ec-bb3eec74ce7f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.939144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7944f0c-b5e5-4844-8c4b-b5001001f1b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42698a69a75f4d6583e8dd7df3926aa6, user_id=ec4d4dfdda4f4f86a6cffadc35d9e68e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.939858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b5ca2-5c37-411d-bc27-944b1cb93f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:37.940559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b5ca2-5c37-411d-bc27-944b1cb93f6c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.941161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b5ca2-5c37-411d-bc27-944b1cb93f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.941412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b5ca2-5c37-411d-bc27-944b1cb93f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:37.942447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b5ca2-5c37-411d-bc27-944b1cb93f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:37.942699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7944f0c-b5e5-4844-8c4b-b5001001f1b7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.944344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b5ca2-5c37-411d-bc27-944b1cb93f6c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:37.956938 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3416/6756] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:13:37.961452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b5ca2-5c37-411d-bc27-944b1cb93f6c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:37.981041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b5ca2-5c37-411d-bc27-944b1cb93f6c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:37.981287 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3341/6757] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:37 2022] PUT /identity/v3/projects/42698a69a75f4d6583e8dd7df3926aa6/users/ec4d4dfdda4f4f86a6cffadc35d9e68e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:37.983358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e016ce-949f-433e-a513-675bcef294c4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:37.983832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e016ce-949f-433e-a513-675bcef294c4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:37.984085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e016ce-949f-433e-a513-675bcef294c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:38.015429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:38.019237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-67e016ce-949f-433e-a513-675bcef294c4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:38.022214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-67e016ce-949f-433e-a513-675bcef294c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:38.054521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:38.085393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e69d5-2fde-4cf3-a954-2126dc07ab0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:38.086282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e69d5-2fde-4cf3-a954-2126dc07ab0d admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:38.086577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e69d5-2fde-4cf3-a954-2126dc07ab0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:38.087022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e69d5-2fde-4cf3-a954-2126dc07ab0d admin admin] PATH_INFO: `/v3/projects/49c8bbdbd1054ea2be53eb1c38eeeb78/users/7bf3461f48be47f48e9b73e0d7edaba4/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:38.106216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e69d5-2fde-4cf3-a954-2126dc07ab0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c8bbdbd1054ea2be53eb1c38eeeb78, user_id=7bf3461f48be47f48e9b73e0d7edaba4, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:38.110088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e69d5-2fde-4cf3-a954-2126dc07ab0d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:38.126338 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3417/6758] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:37 2022] PUT /identity/v3/projects/49c8bbdbd1054ea2be53eb1c38eeeb78/users/7bf3461f48be47f48e9b73e0d7edaba4/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:38.132983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e69d5-2fde-4cf3-a954-2126dc07ab0d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:38.147603 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3342/6759] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:38.158595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e016ce-949f-433e-a513-675bcef294c4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:38.163987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-67e016ce-949f-433e-a513-675bcef294c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:38.250372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e69d5-2fde-4cf3-a954-2126dc07ab0d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:38.256524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace4e2f-2ae0-4dfa-b8d0-93aa8492b49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:38.258154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace4e2f-2ae0-4dfa-b8d0-93aa8492b49c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:38.258690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace4e2f-2ae0-4dfa-b8d0-93aa8492b49c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:38.259262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace4e2f-2ae0-4dfa-b8d0-93aa8492b49c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:38.260150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace4e2f-2ae0-4dfa-b8d0-93aa8492b49c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:38.264456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e016ce-949f-433e-a513-675bcef294c4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:38.265761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace4e2f-2ae0-4dfa-b8d0-93aa8492b49c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:38.278896 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3418/6760] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:38.282974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace4e2f-2ae0-4dfa-b8d0-93aa8492b49c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:38.306911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace4e2f-2ae0-4dfa-b8d0-93aa8492b49c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:38.314887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42b23e-bc22-47c6-84c0-f47546f0ad17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:38.315586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42b23e-bc22-47c6-84c0-f47546f0ad17 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:38.315880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42b23e-bc22-47c6-84c0-f47546f0ad17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:38.316168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42b23e-bc22-47c6-84c0-f47546f0ad17 admin admin] PATH_INFO: `/v3/projects/49c8bbdbd1054ea2be53eb1c38eeeb78/users/7bf3461f48be47f48e9b73e0d7edaba4/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:38.318584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42b23e-bc22-47c6-84c0-f47546f0ad17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c8bbdbd1054ea2be53eb1c38eeeb78, user_id=7bf3461f48be47f48e9b73e0d7edaba4, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:38.321803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42b23e-bc22-47c6-84c0-f47546f0ad17 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:38.339807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889075f-e0c1-4627-a4b2-6ced725aff23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:38.340754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889075f-e0c1-4627-a4b2-6ced725aff23 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:38.341253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889075f-e0c1-4627-a4b2-6ced725aff23 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:38.341709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889075f-e0c1-4627-a4b2-6ced725aff23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:38.359233 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3419/6761] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:38 2022] PUT /identity/v3/projects/49c8bbdbd1054ea2be53eb1c38eeeb78/users/7bf3461f48be47f48e9b73e0d7edaba4/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:38.362274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0aaf7-4b81-483d-a6db-2e260b4100d2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:38.362561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0aaf7-4b81-483d-a6db-2e260b4100d2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:38.362847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0aaf7-4b81-483d-a6db-2e260b4100d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:38.366567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889075f-e0c1-4627-a4b2-6ced725aff23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:38.368241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889075f-e0c1-4627-a4b2-6ced725aff23 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:38.398492 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3343/6762] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:13:38.401241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3df0aaf7-4b81-483d-a6db-2e260b4100d2 None None] MFA Rules not processed for user `7bf3461f48be47f48e9b73e0d7edaba4`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:38.402801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3df0aaf7-4b81-483d-a6db-2e260b4100d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:38.511127 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3420/6763] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:13:38.515382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889075f-e0c1-4627-a4b2-6ced725aff23 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:38.604930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889075f-e0c1-4627-a4b2-6ced725aff23 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:38.622865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa48348-d659-4eb2-a765-585bd6144dbf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:38.624092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa48348-d659-4eb2-a765-585bd6144dbf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:38.625262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa48348-d659-4eb2-a765-585bd6144dbf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:38.625495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa48348-d659-4eb2-a765-585bd6144dbf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:38.627822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa48348-d659-4eb2-a765-585bd6144dbf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:38.629934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa48348-d659-4eb2-a765-585bd6144dbf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:38.655490 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3344/6764] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:13:38 2022] POST /identity/v3/roles => generated 234 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:38.655760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0aaf7-4b81-483d-a6db-2e260b4100d2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:38.672599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0aaf7-4b81-483d-a6db-2e260b4100d2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:38.680625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569075d-5f7a-4240-903a-17df502441b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:38.681305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569075d-5f7a-4240-903a-17df502441b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:38.681603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569075d-5f7a-4240-903a-17df502441b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:38.682059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569075d-5f7a-4240-903a-17df502441b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:38.685585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569075d-5f7a-4240-903a-17df502441b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:38.689378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569075d-5f7a-4240-903a-17df502441b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:38.705391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:38.717581 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3421/6765] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:13:38 2022] POST /identity/v3/roles => generated 235 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:38.718924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa48348-d659-4eb2-a765-585bd6144dbf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:38.736544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa48348-d659-4eb2-a765-585bd6144dbf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:38.742323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319d52f-b521-42f5-854d-92402b40bf83 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:38.743070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319d52f-b521-42f5-854d-92402b40bf83 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:38.743460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319d52f-b521-42f5-854d-92402b40bf83 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:38.743838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319d52f-b521-42f5-854d-92402b40bf83 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:38.744603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319d52f-b521-42f5-854d-92402b40bf83 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:38.746354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319d52f-b521-42f5-854d-92402b40bf83 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:38.781476 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3345/6766] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:13:38 2022] POST /identity/v3/roles => generated 233 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:38.785368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569075d-5f7a-4240-903a-17df502441b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:38.798751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569075d-5f7a-4240-903a-17df502441b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:38.814040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed07824-1d19-478e-b4c1-898e7a370c4c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:38.814775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed07824-1d19-478e-b4c1-898e7a370c4c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:38.815188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed07824-1d19-478e-b4c1-898e7a370c4c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:38.815539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed07824-1d19-478e-b4c1-898e7a370c4c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:38.823789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed07824-1d19-478e-b4c1-898e7a370c4c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:38.831298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed07824-1d19-478e-b4c1-898e7a370c4c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:38.900123 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3422/6767] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:13:38 2022] POST /identity/v3/domains => generated 277 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:38.901535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319d52f-b521-42f5-854d-92402b40bf83 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:38.910981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:38.920001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319d52f-b521-42f5-854d-92402b40bf83 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:39.044738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bc8bb-dc0c-4d33-b4ac-f82f6af16158 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:39.045474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bc8bb-dc0c-4d33-b4ac-f82f6af16158 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:39.045701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bc8bb-dc0c-4d33-b4ac-f82f6af16158 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:39.045982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bc8bb-dc0c-4d33-b4ac-f82f6af16158 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:39.046647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bc8bb-dc0c-4d33-b4ac-f82f6af16158 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:39.048629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bc8bb-dc0c-4d33-b4ac-f82f6af16158 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:39.126737 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3346/6768] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:38 2022] POST /identity/v3/projects => generated 401 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:39.134129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed07824-1d19-478e-b4c1-898e7a370c4c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:39.151720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed07824-1d19-478e-b4c1-898e7a370c4c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:39.247367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a36ab1-021d-49fb-962e-0c1e8fb8158c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:39.248579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a36ab1-021d-49fb-962e-0c1e8fb8158c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:39.248953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a36ab1-021d-49fb-962e-0c1e8fb8158c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:39.249253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a36ab1-021d-49fb-962e-0c1e8fb8158c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/groups` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:39.249903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a36ab1-021d-49fb-962e-0c1e8fb8158c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:39.250945 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-22a36ab1-021d-49fb-962e-0c1e8fb8158c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:39.250945 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:39.250945 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:39.252364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a36ab1-021d-49fb-962e-0c1e8fb8158c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:39.275813 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3423/6769] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Sep 3 21:13:39 2022] POST /identity/v3/groups => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:39.277298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bc8bb-dc0c-4d33-b4ac-f82f6af16158 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:39.302834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bc8bb-dc0c-4d33-b4ac-f82f6af16158 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:39.320973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d92ae-8b15-484d-acf5-9736adfdf05d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:39.321751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d92ae-8b15-484d-acf5-9736adfdf05d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:39.322092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d92ae-8b15-484d-acf5-9736adfdf05d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:39.322308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d92ae-8b15-484d-acf5-9736adfdf05d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:39.322995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d92ae-8b15-484d-acf5-9736adfdf05d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:39.325845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d92ae-8b15-484d-acf5-9736adfdf05d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:39.352513 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3347/6770] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:13:39 2022] POST /identity/v3/roles => generated 235 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:39.356988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a36ab1-021d-49fb-962e-0c1e8fb8158c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:39.390569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a36ab1-021d-49fb-962e-0c1e8fb8158c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:39.408282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6727c-da34-4c51-88ca-9e45f13a07c1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:39.408935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6727c-da34-4c51-88ca-9e45f13a07c1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:39.409225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6727c-da34-4c51-88ca-9e45f13a07c1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:39.409701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6727c-da34-4c51-88ca-9e45f13a07c1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:39.410558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6727c-da34-4c51-88ca-9e45f13a07c1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:39.413312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6727c-da34-4c51-88ca-9e45f13a07c1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:39.490644 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3424/6771] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:39 2022] POST /identity/v3/users => generated 442 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:39.491845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d92ae-8b15-484d-acf5-9736adfdf05d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:39.536780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d92ae-8b15-484d-acf5-9736adfdf05d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:39.547107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afc185-a87e-422f-bd87-1ed68a7aed37 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:39.547755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afc185-a87e-422f-bd87-1ed68a7aed37 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:39.548074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afc185-a87e-422f-bd87-1ed68a7aed37 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:39.548331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afc185-a87e-422f-bd87-1ed68a7aed37 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:39.549063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afc185-a87e-422f-bd87-1ed68a7aed37 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:39.550057 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-07afc185-a87e-422f-bd87-1ed68a7aed37 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:39.550057 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:39.550057 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:39.550958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afc185-a87e-422f-bd87-1ed68a7aed37 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:39.588042 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3348/6772] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:13:39 2022] POST /identity/v3/roles => generated 270 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:39.592460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6727c-da34-4c51-88ca-9e45f13a07c1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:39.614258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6727c-da34-4c51-88ca-9e45f13a07c1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:39.623840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d715b8a-1d4d-4824-bb60-6d55e9b0b35b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:39.625056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d715b8a-1d4d-4824-bb60-6d55e9b0b35b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:39.625693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d715b8a-1d4d-4824-bb60-6d55e9b0b35b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:39.626146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d715b8a-1d4d-4824-bb60-6d55e9b0b35b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/2c84f5876ec84a95a2a5bc97ef7f6cde` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:39.648584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d715b8a-1d4d-4824-bb60-6d55e9b0b35b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_grant(project_id=926f43c2e60741b99282332544bb4d4d, user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=2c84f5876ec84a95a2a5bc97ef7f6cde)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:39.652523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d715b8a-1d4d-4824-bb60-6d55e9b0b35b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:39.677548 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3425/6773] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:39 2022] PUT /identity/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/2c84f5876ec84a95a2a5bc97ef7f6cde => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:39.681894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afc185-a87e-422f-bd87-1ed68a7aed37 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:39.705423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afc185-a87e-422f-bd87-1ed68a7aed37 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:39.827075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d00ee-36b1-4787-845f-d83ed1363bf1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:39.827676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d00ee-36b1-4787-845f-d83ed1363bf1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:39.827971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d00ee-36b1-4787-845f-d83ed1363bf1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:39.828254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d00ee-36b1-4787-845f-d83ed1363bf1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:39.829358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d00ee-36b1-4787-845f-d83ed1363bf1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Adding query filter params (scope.project.id=926f43c2e60741b99282332544bb4d4d, user.id=c42ff7a66af7464b97e9c2450dc38558) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:39.829691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d00ee-36b1-4787-845f-d83ed1363bf1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:39.831575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d00ee-36b1-4787-845f-d83ed1363bf1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:39.957496 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3349/6774] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Sep 3 21:13:39 2022] GET /identity/v3/role_assignments?scope.project.id=926f43c2e60741b99282332544bb4d4d&user.id=c42ff7a66af7464b97e9c2450dc38558&effective => generated 228 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:13:39.961700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d715b8a-1d4d-4824-bb60-6d55e9b0b35b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:39.977223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d715b8a-1d4d-4824-bb60-6d55e9b0b35b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:39.993456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f57cd7-00e9-42ac-bcad-c00a0df504e4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:39.994367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f57cd7-00e9-42ac-bcad-c00a0df504e4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:39.994766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f57cd7-00e9-42ac-bcad-c00a0df504e4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:39.995151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f57cd7-00e9-42ac-bcad-c00a0df504e4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/2c84f5876ec84a95a2a5bc97ef7f6cde` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:40.002320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f57cd7-00e9-42ac-bcad-c00a0df504e4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=926f43c2e60741b99282332544bb4d4d, user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=2c84f5876ec84a95a2a5bc97ef7f6cde)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:40.007973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f57cd7-00e9-42ac-bcad-c00a0df504e4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:40.022387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-72f57cd7-00e9-42ac-bcad-c00a0df504e4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 2c84f5876ec84a95a2a5bc97ef7f6cde was removed from user c42ff7a66af7464b97e9c2450dc38558 on project 926f43c2e60741b99282332544bb4d4d. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:40.022822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-72f57cd7-00e9-42ac-bcad-c00a0df504e4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:40.026214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-72f57cd7-00e9-42ac-bcad-c00a0df504e4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:40.072258 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3426/6775] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Sep 3 21:13:39 2022] DELETE /identity/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/2c84f5876ec84a95a2a5bc97ef7f6cde => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:40.076669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d00ee-36b1-4787-845f-d83ed1363bf1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:40.079175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f81d00ee-36b1-4787-845f-d83ed1363bf1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:40.106738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d00ee-36b1-4787-845f-d83ed1363bf1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:40.139476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:40.241037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:40.241411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:40.241770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:40.242219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/2c84f5876ec84a95a2a5bc97ef7f6cde` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:40.244284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=2c84f5876ec84a95a2a5bc97ef7f6cde)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:40.245561 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:40.245561 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:40.245561 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:40.250851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:40.323274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 2c84f5876ec84a95a2a5bc97ef7f6cde has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:40.323980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:40.324890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:40.330565 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3350/6776] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:40 2022] DELETE /identity/v3/roles/2c84f5876ec84a95a2a5bc97ef7f6cde => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:40.335757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f57cd7-00e9-42ac-bcad-c00a0df504e4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:40.354909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f57cd7-00e9-42ac-bcad-c00a0df504e4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:40.440720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425f3f2-47ae-4c53-a8ee-973282c22ef0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:40.442853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425f3f2-47ae-4c53-a8ee-973282c22ef0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:40.443323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425f3f2-47ae-4c53-a8ee-973282c22ef0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:40.443775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425f3f2-47ae-4c53-a8ee-973282c22ef0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/94f4ea5bd8d54d608c134889670b1995` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:40.454493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425f3f2-47ae-4c53-a8ee-973282c22ef0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_grant(project_id=926f43c2e60741b99282332544bb4d4d, user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=94f4ea5bd8d54d608c134889670b1995)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:40.459000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425f3f2-47ae-4c53-a8ee-973282c22ef0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:40.491602 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3427/6777] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:40 2022] PUT /identity/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/94f4ea5bd8d54d608c134889670b1995 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:40.496020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:40.497566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:40.514250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcbfa3-70e9-4fe9-bf7e-c95b0de1a3a7 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:40.606255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edbfd8-9f2a-4068-8f20-60cee92b9dcd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:40.607323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edbfd8-9f2a-4068-8f20-60cee92b9dcd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:40.607614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edbfd8-9f2a-4068-8f20-60cee92b9dcd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:40.607902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edbfd8-9f2a-4068-8f20-60cee92b9dcd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:40.615446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edbfd8-9f2a-4068-8f20-60cee92b9dcd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:40.616545 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-01edbfd8-9f2a-4068-8f20-60cee92b9dcd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:40.616545 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:40.616545 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:40.617621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edbfd8-9f2a-4068-8f20-60cee92b9dcd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:40.670715 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3351/6778] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Sep 3 21:13:40 2022] PUT /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 520 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:40.674888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425f3f2-47ae-4c53-a8ee-973282c22ef0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:40.688017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425f3f2-47ae-4c53-a8ee-973282c22ef0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:40.701083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:INSERT=7,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:40.764105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826aab61-65cd-435d-9b20-5a8ae36f49be tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:40.764752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826aab61-65cd-435d-9b20-5a8ae36f49be tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:40.765215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826aab61-65cd-435d-9b20-5a8ae36f49be tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:40.765618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826aab61-65cd-435d-9b20-5a8ae36f49be tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:40.766334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826aab61-65cd-435d-9b20-5a8ae36f49be tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Adding query filter params (scope.project.id=926f43c2e60741b99282332544bb4d4d, user.id=c42ff7a66af7464b97e9c2450dc38558) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:40.766563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826aab61-65cd-435d-9b20-5a8ae36f49be tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:40.769398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826aab61-65cd-435d-9b20-5a8ae36f49be tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:40.846621 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3428/6779] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Sep 3 21:13:40 2022] GET /identity/v3/role_assignments?scope.project.id=926f43c2e60741b99282332544bb4d4d&user.id=c42ff7a66af7464b97e9c2450dc38558&effective => generated 1061 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:13:40.850270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edbfd8-9f2a-4068-8f20-60cee92b9dcd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:40.889827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edbfd8-9f2a-4068-8f20-60cee92b9dcd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:40.898526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565c553-333c-4978-b1e8-69c70ca34266 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:40.900236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565c553-333c-4978-b1e8-69c70ca34266 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:40.900707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565c553-333c-4978-b1e8-69c70ca34266 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:40.901192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565c553-333c-4978-b1e8-69c70ca34266 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:40.904323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565c553-333c-4978-b1e8-69c70ca34266 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:40.906729 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-2565c553-333c-4978-b1e8-69c70ca34266 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:40.906729 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:40.906729 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:40.908201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565c553-333c-4978-b1e8-69c70ca34266 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:40.938248 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3352/6780] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:40 2022] DELETE /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:40.939323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826aab61-65cd-435d-9b20-5a8ae36f49be tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:40.953721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826aab61-65cd-435d-9b20-5a8ae36f49be tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:41.035692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f698f-9d89-4ab4-bb4a-a4cdb101985f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:41.036881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f698f-9d89-4ab4-bb4a-a4cdb101985f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:41.037439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f698f-9d89-4ab4-bb4a-a4cdb101985f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:41.038109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f698f-9d89-4ab4-bb4a-a4cdb101985f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:41.039080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f698f-9d89-4ab4-bb4a-a4cdb101985f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Adding query filter params (scope.project.id=926f43c2e60741b99282332544bb4d4d, user.id=c42ff7a66af7464b97e9c2450dc38558) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:41.039643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f698f-9d89-4ab4-bb4a-a4cdb101985f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:41.042413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f698f-9d89-4ab4-bb4a-a4cdb101985f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:41.143085 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3429/6781] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Sep 3 21:13:40 2022] GET /identity/v3/role_assignments?scope.project.id=926f43c2e60741b99282332544bb4d4d&user.id=c42ff7a66af7464b97e9c2450dc38558&effective => generated 577 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:41.147791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565c553-333c-4978-b1e8-69c70ca34266 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:41.159388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565c553-333c-4978-b1e8-69c70ca34266 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:41.164740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93cb88-781a-4ec6-b561-be03aba3caa0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:41.165507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93cb88-781a-4ec6-b561-be03aba3caa0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:41.165804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93cb88-781a-4ec6-b561-be03aba3caa0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:41.166165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93cb88-781a-4ec6-b561-be03aba3caa0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:41.167894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93cb88-781a-4ec6-b561-be03aba3caa0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:41.169696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93cb88-781a-4ec6-b561-be03aba3caa0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-5e93cb88-781a-4ec6-b561-be03aba3caa0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] 94f4ea5bd8d54d608c134889670b1995 does not imply 7159359626c74e5793a5237927e9933c.: keystone.exception.ImpliedRoleNotFound: 94f4ea5bd8d54d608c134889670b1995 does not imply 7159359626c74e5793a5237927e9933c. Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = query.one() Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._iter().one() Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._only_one_row( Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:41.193003 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Sep 03 21:13:41.194520 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Sep 03 21:13:41.195854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 03 21:13:41.195854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 03 21:13:41.195854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 03 21:13:41.195854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 94f4ea5bd8d54d608c134889670b1995 does not imply 7159359626c74e5793a5237927e9933c. Sep 03 21:13:41.195854 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:13:41.195854 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3353/6782] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:41 2022] DELETE /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 137 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:13:41.201467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f698f-9d89-4ab4-bb4a-a4cdb101985f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:41.217253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f698f-9d89-4ab4-bb4a-a4cdb101985f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:41.222869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ac640-d507-4ec6-9d3a-f88840c38d78 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:41.223755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ac640-d507-4ec6-9d3a-f88840c38d78 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:41.224024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ac640-d507-4ec6-9d3a-f88840c38d78 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:41.224495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ac640-d507-4ec6-9d3a-f88840c38d78 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/94f4ea5bd8d54d608c134889670b1995` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:41.227156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ac640-d507-4ec6-9d3a-f88840c38d78 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=926f43c2e60741b99282332544bb4d4d, user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=94f4ea5bd8d54d608c134889670b1995)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:41.230652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ac640-d507-4ec6-9d3a-f88840c38d78 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:41.237912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d95ac640-d507-4ec6-9d3a-f88840c38d78 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 94f4ea5bd8d54d608c134889670b1995 was removed from user c42ff7a66af7464b97e9c2450dc38558 on project 926f43c2e60741b99282332544bb4d4d. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:41.237912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d95ac640-d507-4ec6-9d3a-f88840c38d78 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:41.238666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d95ac640-d507-4ec6-9d3a-f88840c38d78 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:41.259168 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3430/6783] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Sep 3 21:13:41 2022] DELETE /identity/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/94f4ea5bd8d54d608c134889670b1995 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:41.264966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93cb88-781a-4ec6-b561-be03aba3caa0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:41.267670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5e93cb88-781a-4ec6-b561-be03aba3caa0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:41.282870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93cb88-781a-4ec6-b561-be03aba3caa0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:41.342450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:41.376180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d19fb5-0a4b-4304-bd7f-8e5c41375023 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:41.378099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d19fb5-0a4b-4304-bd7f-8e5c41375023 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:41.378500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d19fb5-0a4b-4304-bd7f-8e5c41375023 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:41.378888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d19fb5-0a4b-4304-bd7f-8e5c41375023 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:41.379688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d19fb5-0a4b-4304-bd7f-8e5c41375023 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:41.381495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d19fb5-0a4b-4304-bd7f-8e5c41375023 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:41.405261 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3354/6784] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:41 2022] POST /identity/v3/roles => generated 271 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:41.410035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ac640-d507-4ec6-9d3a-f88840c38d78 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:41.421127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ac640-d507-4ec6-9d3a-f88840c38d78 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:41.426262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467cfa4-c2dc-4a07-843f-1d5902788929 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:41.427110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467cfa4-c2dc-4a07-843f-1d5902788929 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:41.427400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467cfa4-c2dc-4a07-843f-1d5902788929 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:41.427678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467cfa4-c2dc-4a07-843f-1d5902788929 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:41.428335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467cfa4-c2dc-4a07-843f-1d5902788929 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Adding query filter params (domain_id=69e8bdfecc4e401c92c144fed43e3a65) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:41.428876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467cfa4-c2dc-4a07-843f-1d5902788929 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:41.429993 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-0467cfa4-c2dc-4a07-843f-1d5902788929 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domain_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:41.429993 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:41.429993 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:41.430908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467cfa4-c2dc-4a07-843f-1d5902788929 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:41.433683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:41.445384 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3431/6785] 10.222.0.16 () {58 vars in 1207 bytes} [Sat Sep 3 21:13:41 2022] GET /identity/v3/roles?domain_id=69e8bdfecc4e401c92c144fed43e3a65 => generated 408 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:41.447672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d19fb5-0a4b-4304-bd7f-8e5c41375023 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:41.463359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d19fb5-0a4b-4304-bd7f-8e5c41375023 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:41.470810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e33ff8-48e0-4aa9-bc22-0ec2a1735a18 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:41.471723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e33ff8-48e0-4aa9-bc22-0ec2a1735a18 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:41.472049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e33ff8-48e0-4aa9-bc22-0ec2a1735a18 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:41.472450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e33ff8-48e0-4aa9-bc22-0ec2a1735a18 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/40b77b004c0642d59aa8dd7906b9ecd7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:41.473924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e33ff8-48e0-4aa9-bc22-0ec2a1735a18 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=40b77b004c0642d59aa8dd7906b9ecd7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:41.475888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e33ff8-48e0-4aa9-bc22-0ec2a1735a18 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:41.544610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-40e33ff8-48e0-4aa9-bc22-0ec2a1735a18 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 40b77b004c0642d59aa8dd7906b9ecd7 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:41.544969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-40e33ff8-48e0-4aa9-bc22-0ec2a1735a18 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:41.545687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-40e33ff8-48e0-4aa9-bc22-0ec2a1735a18 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:41.550231 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3355/6786] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:41 2022] DELETE /identity/v3/roles/40b77b004c0642d59aa8dd7906b9ecd7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:41.555605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467cfa4-c2dc-4a07-843f-1d5902788929 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:41.557404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0467cfa4-c2dc-4a07-843f-1d5902788929 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:41.579004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467cfa4-c2dc-4a07-843f-1d5902788929 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:41.698548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1c655-6519-478b-9ddb-21195df43eeb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:41.699353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1c655-6519-478b-9ddb-21195df43eeb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:41.699651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1c655-6519-478b-9ddb-21195df43eeb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:41.707990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1c655-6519-478b-9ddb-21195df43eeb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:41.708481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1c655-6519-478b-9ddb-21195df43eeb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Adding query filter params (domain_id=69e8bdfecc4e401c92c144fed43e3a65) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:41.708810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1c655-6519-478b-9ddb-21195df43eeb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:41.710663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1c655-6519-478b-9ddb-21195df43eeb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:41.735039 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3432/6787] 10.222.0.16 () {58 vars in 1207 bytes} [Sat Sep 3 21:13:41 2022] GET /identity/v3/roles?domain_id=69e8bdfecc4e401c92c144fed43e3a65 => generated 148 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:41.735288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e33ff8-48e0-4aa9-bc22-0ec2a1735a18 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:41.753251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e33ff8-48e0-4aa9-bc22-0ec2a1735a18 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:41.776509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c83491-e0e2-47bd-87aa-98f1690f3ed4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:41.777317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c83491-e0e2-47bd-87aa-98f1690f3ed4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:41.777606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c83491-e0e2-47bd-87aa-98f1690f3ed4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:41.777884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c83491-e0e2-47bd-87aa-98f1690f3ed4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/40b77b004c0642d59aa8dd7906b9ecd7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:41.800036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c83491-e0e2-47bd-87aa-98f1690f3ed4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-b3c83491-e0e2-47bd-87aa-98f1690f3ed4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Could not find role: 40b77b004c0642d59aa8dd7906b9ecd7.: keystone.exception.RoleNotFound: Could not find role: 40b77b004c0642d59aa8dd7906b9ecd7. Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 165, in delete Sep 03 21:13:41.808191 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_role') Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:13:41.809995 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 40b77b004c0642d59aa8dd7906b9ecd7. Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:13:41.811178 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3356/6788] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:41 2022] DELETE /identity/v3/roles/40b77b004c0642d59aa8dd7906b9ecd7 => generated 110 bytes in 76 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:13:41.815719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1c655-6519-478b-9ddb-21195df43eeb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:41.834327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1c655-6519-478b-9ddb-21195df43eeb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:41.850065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701dda-81dc-43e3-a1dc-de5019c56ea4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:41.850897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701dda-81dc-43e3-a1dc-de5019c56ea4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:41.851291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701dda-81dc-43e3-a1dc-de5019c56ea4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:41.851681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701dda-81dc-43e3-a1dc-de5019c56ea4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:41.861559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701dda-81dc-43e3-a1dc-de5019c56ea4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=69e8bdfecc4e401c92c144fed43e3a65, group_id=194d7fcccb8d4f5abf13afc233ee43b7, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:41.866704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701dda-81dc-43e3-a1dc-de5019c56ea4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:41.897855 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3433/6789] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:41 2022] PUT /identity/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:41.901621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c83491-e0e2-47bd-87aa-98f1690f3ed4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:41.920453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c83491-e0e2-47bd-87aa-98f1690f3ed4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:42.002085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243a148-81bd-4362-bc8a-7512cead739a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:42.003060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a148-81bd-4362-bc8a-7512cead739a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:42.003599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a148-81bd-4362-bc8a-7512cead739a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:42.004160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a148-81bd-4362-bc8a-7512cead739a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:42.008360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243a148-81bd-4362-bc8a-7512cead739a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=69e8bdfecc4e401c92c144fed43e3a65, group_id=194d7fcccb8d4f5abf13afc233ee43b7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:42.012942 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-b243a148-81bd-4362-bc8a-7512cead739a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:42.012942 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:42.012942 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:42.017065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243a148-81bd-4362-bc8a-7512cead739a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:42.050258 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3357/6790] 10.222.0.16 () {58 vars in 1365 bytes} [Sat Sep 3 21:13:41 2022] GET /identity/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles => generated 410 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:13:42.054484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701dda-81dc-43e3-a1dc-de5019c56ea4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:42.094151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701dda-81dc-43e3-a1dc-de5019c56ea4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:42.103096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bbda0-01e4-4408-9f5e-702f88147649 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:42.103792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bbda0-01e4-4408-9f5e-702f88147649 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:42.104086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bbda0-01e4-4408-9f5e-702f88147649 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:42.105027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bbda0-01e4-4408-9f5e-702f88147649 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:42.106865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bbda0-01e4-4408-9f5e-702f88147649 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=69e8bdfecc4e401c92c144fed43e3a65, group_id=194d7fcccb8d4f5abf13afc233ee43b7, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:42.109657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bbda0-01e4-4408-9f5e-702f88147649 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:42.123740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243a148-81bd-4362-bc8a-7512cead739a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:42.127535 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3434/6791] 10.222.0.16 () {58 vars in 1465 bytes} [Sat Sep 3 21:13:42 2022] HEAD /identity/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:42.138861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243a148-81bd-4362-bc8a-7512cead739a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:42.144443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdd992-cc36-415b-a655-8e74b89d6e90 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:42.145552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdd992-cc36-415b-a655-8e74b89d6e90 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:42.146071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdd992-cc36-415b-a655-8e74b89d6e90 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:42.146464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdd992-cc36-415b-a655-8e74b89d6e90 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:42.149194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdd992-cc36-415b-a655-8e74b89d6e90 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=69e8bdfecc4e401c92c144fed43e3a65, group_id=194d7fcccb8d4f5abf13afc233ee43b7, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:42.152022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdd992-cc36-415b-a655-8e74b89d6e90 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:42.158723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98bdd992-cc36-415b-a655-8e74b89d6e90 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 90a1e59bc57f417581c7b6fadb9dfcb3 was removed from group 194d7fcccb8d4f5abf13afc233ee43b7 on domain 69e8bdfecc4e401c92c144fed43e3a65. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:42.159166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98bdd992-cc36-415b-a655-8e74b89d6e90 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:42.160121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-98bdd992-cc36-415b-a655-8e74b89d6e90 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:42.186036 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3358/6792] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Sep 3 21:13:42 2022] DELETE /identity/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:42.187894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bbda0-01e4-4408-9f5e-702f88147649 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:42.190766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-354bbda0-01e4-4408-9f5e-702f88147649 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:42.208744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bbda0-01e4-4408-9f5e-702f88147649 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:42.280336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2c766-2801-4bdc-b454-a8068e54e87d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:42.281557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2c766-2801-4bdc-b454-a8068e54e87d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:42.282233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2c766-2801-4bdc-b454-a8068e54e87d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:42.282705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2c766-2801-4bdc-b454-a8068e54e87d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:42.285804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2c766-2801-4bdc-b454-a8068e54e87d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_grant(project_id=926f43c2e60741b99282332544bb4d4d, group_id=194d7fcccb8d4f5abf13afc233ee43b7, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:42.289643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2c766-2801-4bdc-b454-a8068e54e87d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:42.314631 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3435/6793] 10.222.0.16 () {60 vars in 1486 bytes} [Sat Sep 3 21:13:42 2022] PUT /identity/v3/projects/926f43c2e60741b99282332544bb4d4d/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:42.319262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdd992-cc36-415b-a655-8e74b89d6e90 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:42.330927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdd992-cc36-415b-a655-8e74b89d6e90 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:42.405314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:42.415142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b420e4f-938f-49a3-be54-2c09210b4903 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:42.417187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b420e4f-938f-49a3-be54-2c09210b4903 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:42.417780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b420e4f-938f-49a3-be54-2c09210b4903 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:42.419598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b420e4f-938f-49a3-be54-2c09210b4903 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:42.422210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b420e4f-938f-49a3-be54-2c09210b4903 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_grants(project_id=926f43c2e60741b99282332544bb4d4d, group_id=194d7fcccb8d4f5abf13afc233ee43b7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:42.425140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b420e4f-938f-49a3-be54-2c09210b4903 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:42.452806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2c766-2801-4bdc-b454-a8068e54e87d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:42.463136 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3359/6794] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Sep 3 21:13:42 2022] GET /identity/v3/projects/926f43c2e60741b99282332544bb4d4d/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles => generated 411 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:42.465984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2c766-2801-4bdc-b454-a8068e54e87d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:42.472458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570883ed-8de4-4c59-8a9d-e037235eed0d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:42.473187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570883ed-8de4-4c59-8a9d-e037235eed0d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:42.473567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570883ed-8de4-4c59-8a9d-e037235eed0d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:42.473921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570883ed-8de4-4c59-8a9d-e037235eed0d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/groups/194d7fcccb8d4f5abf13afc233ee43b7/users/c42ff7a66af7464b97e9c2450dc38558` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:42.476691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570883ed-8de4-4c59-8a9d-e037235eed0d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=194d7fcccb8d4f5abf13afc233ee43b7, user_id=c42ff7a66af7464b97e9c2450dc38558)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:42.480517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570883ed-8de4-4c59-8a9d-e037235eed0d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:42.514236 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3436/6795] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Sep 3 21:13:42 2022] PUT /identity/v3/groups/194d7fcccb8d4f5abf13afc233ee43b7/users/c42ff7a66af7464b97e9c2450dc38558 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:42.518650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81b848-67fb-4836-a950-06bcca24cacb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:42.519037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81b848-67fb-4836-a950-06bcca24cacb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:42.519391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81b848-67fb-4836-a950-06bcca24cacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:42.609795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cc81b848-67fb-4836-a950-06bcca24cacb None None] MFA Rules not processed for user `c42ff7a66af7464b97e9c2450dc38558`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:42.612458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cc81b848-67fb-4836-a950-06bcca24cacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:42.812952 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3360/6796] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:42.815545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570883ed-8de4-4c59-8a9d-e037235eed0d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:42.915879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:42.945287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570883ed-8de4-4c59-8a9d-e037235eed0d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:42.961730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534c512-a14c-41b7-9c72-4cd4b6ed66df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:42.962451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534c512-a14c-41b7-9c72-4cd4b6ed66df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:42.964885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534c512-a14c-41b7-9c72-4cd4b6ed66df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:42.965275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534c512-a14c-41b7-9c72-4cd4b6ed66df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:42.968513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534c512-a14c-41b7-9c72-4cd4b6ed66df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:check_grant(project_id=926f43c2e60741b99282332544bb4d4d, group_id=194d7fcccb8d4f5abf13afc233ee43b7, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:42.971955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534c512-a14c-41b7-9c72-4cd4b6ed66df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:42.983424 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3437/6797] 10.222.0.16 () {58 vars in 1468 bytes} [Sat Sep 3 21:13:42 2022] HEAD /identity/v3/projects/926f43c2e60741b99282332544bb4d4d/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:42.984676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81b848-67fb-4836-a950-06bcca24cacb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:43.001315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81b848-67fb-4836-a950-06bcca24cacb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:43.010137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d5655-347b-426e-a218-0c043e72818d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:43.011214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d5655-347b-426e-a218-0c043e72818d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:43.019197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d5655-347b-426e-a218-0c043e72818d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:43.019368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d5655-347b-426e-a218-0c043e72818d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:43.022315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d5655-347b-426e-a218-0c043e72818d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=926f43c2e60741b99282332544bb4d4d, group_id=194d7fcccb8d4f5abf13afc233ee43b7, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:43.025539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d5655-347b-426e-a218-0c043e72818d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:43.043192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f15d5655-347b-426e-a218-0c043e72818d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 90a1e59bc57f417581c7b6fadb9dfcb3 was removed from group 194d7fcccb8d4f5abf13afc233ee43b7 on project 926f43c2e60741b99282332544bb4d4d. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:43.044801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f15d5655-347b-426e-a218-0c043e72818d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:43.057562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f15d5655-347b-426e-a218-0c043e72818d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:43.088974 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3361/6798] 10.222.0.16 () {58 vars in 1470 bytes} [Sat Sep 3 21:13:42 2022] DELETE /identity/v3/projects/926f43c2e60741b99282332544bb4d4d/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:43.093711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534c512-a14c-41b7-9c72-4cd4b6ed66df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:43.096793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f534c512-a14c-41b7-9c72-4cd4b6ed66df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:43.113055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534c512-a14c-41b7-9c72-4cd4b6ed66df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:43.193051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3408646-35ee-4bd8-852d-b548e1408345 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:43.193722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3408646-35ee-4bd8-852d-b548e1408345 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:43.194034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3408646-35ee-4bd8-852d-b548e1408345 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:43.194297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3408646-35ee-4bd8-852d-b548e1408345 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/groups/194d7fcccb8d4f5abf13afc233ee43b7/users/c42ff7a66af7464b97e9c2450dc38558` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:43.196312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3408646-35ee-4bd8-852d-b548e1408345 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=194d7fcccb8d4f5abf13afc233ee43b7, user_id=c42ff7a66af7464b97e9c2450dc38558)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:43.198422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3408646-35ee-4bd8-852d-b548e1408345 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:43.207650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3408646-35ee-4bd8-852d-b548e1408345 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'c42ff7a66af7464b97e9c2450dc38558'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:43.224520 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3438/6799] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Sep 3 21:13:43 2022] DELETE /identity/v3/groups/194d7fcccb8d4f5abf13afc233ee43b7/users/c42ff7a66af7464b97e9c2450dc38558 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:43.229812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d5655-347b-426e-a218-0c043e72818d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:43.241929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d5655-347b-426e-a218-0c043e72818d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:43.327970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36f816-694a-4776-b92f-281fa0b2210a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:43.328589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36f816-694a-4776-b92f-281fa0b2210a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:43.328827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36f816-694a-4776-b92f-281fa0b2210a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:43.329078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36f816-694a-4776-b92f-281fa0b2210a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/system/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:43.330810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36f816-694a-4776-b92f-281fa0b2210a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_system_grant_for_group(group_id=194d7fcccb8d4f5abf13afc233ee43b7, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:43.331845 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-1e36f816-694a-4776-b92f-281fa0b2210a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:43.331845 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:43.331845 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:43.332756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36f816-694a-4776-b92f-281fa0b2210a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:43.343574 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3362/6800] 10.222.0.16 () {60 vars in 1381 bytes} [Sat Sep 3 21:13:43 2022] PUT /identity/v3/system/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:43.348329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3408646-35ee-4bd8-852d-b548e1408345 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:43.361158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3408646-35ee-4bd8-852d-b548e1408345 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:43.366957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f088f99-814e-45fb-b7e8-127ae1130447 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:43.367728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f088f99-814e-45fb-b7e8-127ae1130447 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:43.368021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f088f99-814e-45fb-b7e8-127ae1130447 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:43.368304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f088f99-814e-45fb-b7e8-127ae1130447 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/system/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:43.369527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f088f99-814e-45fb-b7e8-127ae1130447 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_system_grants_for_group(group_id=194d7fcccb8d4f5abf13afc233ee43b7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:43.370696 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-9f088f99-814e-45fb-b7e8-127ae1130447 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:43.370696 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:43.370696 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:43.372609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f088f99-814e-45fb-b7e8-127ae1130447 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:43.390989 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3439/6801] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Sep 3 21:13:43 2022] GET /identity/v3/system/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles => generated 376 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:43.395282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36f816-694a-4776-b92f-281fa0b2210a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:43.409657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36f816-694a-4776-b92f-281fa0b2210a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:43.414698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71907d36-8aaf-4729-a58b-18b1e800c93f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:43.415515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71907d36-8aaf-4729-a58b-18b1e800c93f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:43.415884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71907d36-8aaf-4729-a58b-18b1e800c93f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:43.416275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71907d36-8aaf-4729-a58b-18b1e800c93f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/system/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:43.418562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71907d36-8aaf-4729-a58b-18b1e800c93f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:check_system_grant_for_group(group_id=194d7fcccb8d4f5abf13afc233ee43b7, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:43.419673 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-71907d36-8aaf-4729-a58b-18b1e800c93f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:43.419673 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:43.419673 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:43.420829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71907d36-8aaf-4729-a58b-18b1e800c93f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:43.452509 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3363/6802] 10.222.0.16 () {58 vars in 1363 bytes} [Sat Sep 3 21:13:43 2022] HEAD /identity/v3/system/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:43.456858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f088f99-814e-45fb-b7e8-127ae1130447 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:43.467792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f088f99-814e-45fb-b7e8-127ae1130447 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:43.472741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973df73-ae4b-4616-98ff-aa98342277aa tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:43.473578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973df73-ae4b-4616-98ff-aa98342277aa tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:43.473934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973df73-ae4b-4616-98ff-aa98342277aa tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:43.474287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973df73-ae4b-4616-98ff-aa98342277aa tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/system/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:43.476141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973df73-ae4b-4616-98ff-aa98342277aa tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_group(group_id=194d7fcccb8d4f5abf13afc233ee43b7, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:43.477345 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-4973df73-ae4b-4616-98ff-aa98342277aa tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:43.477345 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:43.477345 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:43.478678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973df73-ae4b-4616-98ff-aa98342277aa tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:43.489051 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3440/6803] 10.222.0.16 () {58 vars in 1365 bytes} [Sat Sep 3 21:13:43 2022] DELETE /identity/v3/system/groups/194d7fcccb8d4f5abf13afc233ee43b7/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:43.494423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71907d36-8aaf-4729-a58b-18b1e800c93f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:43.510289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71907d36-8aaf-4729-a58b-18b1e800c93f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:43.521552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c822853-50b1-4139-b93c-95384763c2da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:43.522256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c822853-50b1-4139-b93c-95384763c2da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:43.522551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c822853-50b1-4139-b93c-95384763c2da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:43.522839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c822853-50b1-4139-b93c-95384763c2da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:43.525958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c822853-50b1-4139-b93c-95384763c2da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=69e8bdfecc4e401c92c144fed43e3a65, user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:43.529260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c822853-50b1-4139-b93c-95384763c2da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:43.539794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:43.560958 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3364/6804] 10.222.0.16 () {60 vars in 1480 bytes} [Sat Sep 3 21:13:43 2022] PUT /identity/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:43.565059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973df73-ae4b-4616-98ff-aa98342277aa tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:43.576938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973df73-ae4b-4616-98ff-aa98342277aa tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:43.638915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fea25-fe02-42cd-88f7-c88fac3cd40a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:43.639861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fea25-fe02-42cd-88f7-c88fac3cd40a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:43.640157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fea25-fe02-42cd-88f7-c88fac3cd40a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:43.640496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fea25-fe02-42cd-88f7-c88fac3cd40a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/users/c42ff7a66af7464b97e9c2450dc38558/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:43.642429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fea25-fe02-42cd-88f7-c88fac3cd40a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=69e8bdfecc4e401c92c144fed43e3a65, user_id=c42ff7a66af7464b97e9c2450dc38558)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:43.645249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fea25-fe02-42cd-88f7-c88fac3cd40a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:43.658152 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3441/6805] 10.222.0.16 () {58 vars in 1362 bytes} [Sat Sep 3 21:13:43 2022] GET /identity/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/users/c42ff7a66af7464b97e9c2450dc38558/roles => generated 409 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:43.662373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c822853-50b1-4139-b93c-95384763c2da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:43.673951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c822853-50b1-4139-b93c-95384763c2da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:43.678765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0f758-fe02-44c7-b37e-473f6f6a5ce6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:43.679435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0f758-fe02-44c7-b37e-473f6f6a5ce6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:43.679727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0f758-fe02-44c7-b37e-473f6f6a5ce6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:43.680011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0f758-fe02-44c7-b37e-473f6f6a5ce6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:43.682347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0f758-fe02-44c7-b37e-473f6f6a5ce6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=69e8bdfecc4e401c92c144fed43e3a65, user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:43.683370 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-5ca0f758-fe02-44c7-b37e-473f6f6a5ce6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:43.683370 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:43.683370 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:43.686020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0f758-fe02-44c7-b37e-473f6f6a5ce6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:43.694604 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3365/6806] 10.222.0.16 () {58 vars in 1462 bytes} [Sat Sep 3 21:13:43 2022] HEAD /identity/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:43.699817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fea25-fe02-42cd-88f7-c88fac3cd40a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:43.711391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fea25-fe02-42cd-88f7-c88fac3cd40a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:43.720604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15236a3a-af0e-4f41-a206-1c8f3e3a3368 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:43.721335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15236a3a-af0e-4f41-a206-1c8f3e3a3368 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:43.721644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15236a3a-af0e-4f41-a206-1c8f3e3a3368 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:43.721900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15236a3a-af0e-4f41-a206-1c8f3e3a3368 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:43.724701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15236a3a-af0e-4f41-a206-1c8f3e3a3368 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=69e8bdfecc4e401c92c144fed43e3a65, user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:43.728063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15236a3a-af0e-4f41-a206-1c8f3e3a3368 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:43.734503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-15236a3a-af0e-4f41-a206-1c8f3e3a3368 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 90a1e59bc57f417581c7b6fadb9dfcb3 was removed from user c42ff7a66af7464b97e9c2450dc38558 on domain 69e8bdfecc4e401c92c144fed43e3a65. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:43.734800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-15236a3a-af0e-4f41-a206-1c8f3e3a3368 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:43.735819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-15236a3a-af0e-4f41-a206-1c8f3e3a3368 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:43.760084 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3442/6807] 10.222.0.16 () {58 vars in 1464 bytes} [Sat Sep 3 21:13:43 2022] DELETE /identity/v3/domains/69e8bdfecc4e401c92c144fed43e3a65/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:43.764768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0f758-fe02-44c7-b37e-473f6f6a5ce6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:43.766995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5ca0f758-fe02-44c7-b37e-473f6f6a5ce6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:43.780617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0f758-fe02-44c7-b37e-473f6f6a5ce6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:43.853792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426452b7-ee81-4e2d-bd67-070adaedbfe9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:43.854643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426452b7-ee81-4e2d-bd67-070adaedbfe9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:43.855033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426452b7-ee81-4e2d-bd67-070adaedbfe9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:43.855412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426452b7-ee81-4e2d-bd67-070adaedbfe9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:43.858321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426452b7-ee81-4e2d-bd67-070adaedbfe9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_grant(project_id=926f43c2e60741b99282332544bb4d4d, user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:43.861111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426452b7-ee81-4e2d-bd67-070adaedbfe9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:43.878118 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3366/6808] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:43 2022] PUT /identity/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:43.882355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15236a3a-af0e-4f41-a206-1c8f3e3a3368 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:43.901008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15236a3a-af0e-4f41-a206-1c8f3e3a3368 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:43.927038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:44.014725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af5b6b-1d20-40c8-9564-58f6331d7983 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:44.015685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af5b6b-1d20-40c8-9564-58f6331d7983 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:44.016074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af5b6b-1d20-40c8-9564-58f6331d7983 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:44.016460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af5b6b-1d20-40c8-9564-58f6331d7983 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:44.018569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af5b6b-1d20-40c8-9564-58f6331d7983 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_grants(project_id=926f43c2e60741b99282332544bb4d4d, user_id=c42ff7a66af7464b97e9c2450dc38558)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:44.023138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af5b6b-1d20-40c8-9564-58f6331d7983 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:44.054512 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3443/6809] 10.222.0.16 () {58 vars in 1365 bytes} [Sat Sep 3 21:13:43 2022] GET /identity/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles => generated 410 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:44.058911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426452b7-ee81-4e2d-bd67-070adaedbfe9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:44.075117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426452b7-ee81-4e2d-bd67-070adaedbfe9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:44.084713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9d6eb-270e-45c5-a6f0-87d291a3f17d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:44.085952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9d6eb-270e-45c5-a6f0-87d291a3f17d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:44.086470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9d6eb-270e-45c5-a6f0-87d291a3f17d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:44.086856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9d6eb-270e-45c5-a6f0-87d291a3f17d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:44.090183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9d6eb-270e-45c5-a6f0-87d291a3f17d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:check_grant(project_id=926f43c2e60741b99282332544bb4d4d, user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:44.097707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9d6eb-270e-45c5-a6f0-87d291a3f17d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:44.110786 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3367/6810] 10.222.0.16 () {58 vars in 1465 bytes} [Sat Sep 3 21:13:44 2022] HEAD /identity/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:44.111421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af5b6b-1d20-40c8-9564-58f6331d7983 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:44.122101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af5b6b-1d20-40c8-9564-58f6331d7983 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:44.135149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fbecf-493e-486b-9b51-6ab873c7e52b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:44.135851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fbecf-493e-486b-9b51-6ab873c7e52b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:44.136143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fbecf-493e-486b-9b51-6ab873c7e52b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:44.136429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fbecf-493e-486b-9b51-6ab873c7e52b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:44.143676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fbecf-493e-486b-9b51-6ab873c7e52b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=926f43c2e60741b99282332544bb4d4d, user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:44.148356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fbecf-493e-486b-9b51-6ab873c7e52b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:44.158204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d73fbecf-493e-486b-9b51-6ab873c7e52b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 90a1e59bc57f417581c7b6fadb9dfcb3 was removed from user c42ff7a66af7464b97e9c2450dc38558 on project 926f43c2e60741b99282332544bb4d4d. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:44.158642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d73fbecf-493e-486b-9b51-6ab873c7e52b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:44.159713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d73fbecf-493e-486b-9b51-6ab873c7e52b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:44.184524 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3444/6811] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Sep 3 21:13:44 2022] DELETE /identity/v3/projects/926f43c2e60741b99282332544bb4d4d/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:44.188197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9d6eb-270e-45c5-a6f0-87d291a3f17d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:44.191931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-aed9d6eb-270e-45c5-a6f0-87d291a3f17d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:44.214173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9d6eb-270e-45c5-a6f0-87d291a3f17d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:44.304803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07775a86-74d3-435a-bfa5-acf203f233a0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:44.305930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07775a86-74d3-435a-bfa5-acf203f233a0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:44.306366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07775a86-74d3-435a-bfa5-acf203f233a0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:44.307521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07775a86-74d3-435a-bfa5-acf203f233a0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/system/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:44.311156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07775a86-74d3-435a-bfa5-acf203f233a0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:44.313763 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-07775a86-74d3-435a-bfa5-acf203f233a0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:44.313763 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:44.313763 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:44.317004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07775a86-74d3-435a-bfa5-acf203f233a0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:44.338789 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3368/6812] 10.222.0.16 () {60 vars in 1378 bytes} [Sat Sep 3 21:13:44 2022] PUT /identity/v3/system/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:44.339470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fbecf-493e-486b-9b51-6ab873c7e52b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:44.352978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fbecf-493e-486b-9b51-6ab873c7e52b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:44.360378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59f01d-2c3d-4f5e-865e-8f96942c7ddb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:44.361180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59f01d-2c3d-4f5e-865e-8f96942c7ddb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:44.361684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59f01d-2c3d-4f5e-865e-8f96942c7ddb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:44.361981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59f01d-2c3d-4f5e-865e-8f96942c7ddb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/system/users/c42ff7a66af7464b97e9c2450dc38558/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:44.363584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59f01d-2c3d-4f5e-865e-8f96942c7ddb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_system_grants_for_user(user_id=c42ff7a66af7464b97e9c2450dc38558)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:44.364874 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-9d59f01d-2c3d-4f5e-865e-8f96942c7ddb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:44.364874 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:44.364874 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:44.365993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59f01d-2c3d-4f5e-865e-8f96942c7ddb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:44.389040 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3445/6813] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Sep 3 21:13:44 2022] GET /identity/v3/system/users/c42ff7a66af7464b97e9c2450dc38558/roles => generated 375 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:44.391333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07775a86-74d3-435a-bfa5-acf203f233a0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:44.413027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07775a86-74d3-435a-bfa5-acf203f233a0 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:44.419553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07a5a1-967b-4481-af07-26449d220dc3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:44.420601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07a5a1-967b-4481-af07-26449d220dc3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:44.421109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07a5a1-967b-4481-af07-26449d220dc3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:44.421622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07a5a1-967b-4481-af07-26449d220dc3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/system/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:44.423726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07a5a1-967b-4481-af07-26449d220dc3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:check_system_grant_for_user(user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:44.425797 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-bd07a5a1-967b-4481-af07-26449d220dc3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:44.425797 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:44.425797 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:44.428058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07a5a1-967b-4481-af07-26449d220dc3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:44.439043 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3369/6814] 10.222.0.16 () {58 vars in 1360 bytes} [Sat Sep 3 21:13:44 2022] HEAD /identity/v3/system/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:44.443421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59f01d-2c3d-4f5e-865e-8f96942c7ddb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:44.460474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59f01d-2c3d-4f5e-865e-8f96942c7ddb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:44.468578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b643d0d-f648-4324-8802-a9febb56f943 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:44.469942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b643d0d-f648-4324-8802-a9febb56f943 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:44.470572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b643d0d-f648-4324-8802-a9febb56f943 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:44.471065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b643d0d-f648-4324-8802-a9febb56f943 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/system/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:44.473356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b643d0d-f648-4324-8802-a9febb56f943 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_user(user_id=c42ff7a66af7464b97e9c2450dc38558, role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:44.474590 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-1b643d0d-f648-4324-8802-a9febb56f943 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:44.474590 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:44.474590 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:44.477512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b643d0d-f648-4324-8802-a9febb56f943 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:44.492467 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3446/6815] 10.222.0.16 () {58 vars in 1362 bytes} [Sat Sep 3 21:13:44 2022] DELETE /identity/v3/system/users/c42ff7a66af7464b97e9c2450dc38558/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:44.492683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07a5a1-967b-4481-af07-26449d220dc3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:44.517215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07a5a1-967b-4481-af07-26449d220dc3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:44.524671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ce1d6-fec7-4efb-868d-263430d95da5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:44.524671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ce1d6-fec7-4efb-868d-263430d95da5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:44.524671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ce1d6-fec7-4efb-868d-263430d95da5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:44.525034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ce1d6-fec7-4efb-868d-263430d95da5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:44.525034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ce1d6-fec7-4efb-868d-263430d95da5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:44.525444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ce1d6-fec7-4efb-868d-263430d95da5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:44.550960 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3370/6816] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:13:44 2022] POST /identity/v3/roles => generated 270 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:44.552312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b643d0d-f648-4324-8802-a9febb56f943 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:44.566461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b643d0d-f648-4324-8802-a9febb56f943 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:44.580543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9928b76-b5c4-45d8-bc32-6797803907da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:44.582648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9928b76-b5c4-45d8-bc32-6797803907da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:44.583177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9928b76-b5c4-45d8-bc32-6797803907da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:44.583671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9928b76-b5c4-45d8-bc32-6797803907da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:44.584607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9928b76-b5c4-45d8-bc32-6797803907da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:44.585910 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-f9928b76-b5c4-45d8-bc32-6797803907da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:44.585910 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:44.585910 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:44.589426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9928b76-b5c4-45d8-bc32-6797803907da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:44.620539 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3447/6817] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:13:44 2022] POST /identity/v3/roles => generated 269 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:44.625355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ce1d6-fec7-4efb-868d-263430d95da5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:44.642098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ce1d6-fec7-4efb-868d-263430d95da5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:44.667155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aabeef-2de2-4e02-8a0d-121a6d3bb3fd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:44.668340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aabeef-2de2-4e02-8a0d-121a6d3bb3fd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:44.668991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aabeef-2de2-4e02-8a0d-121a6d3bb3fd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:44.669505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aabeef-2de2-4e02-8a0d-121a6d3bb3fd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/b641be0d110147699efff68ac7243970/implies/a1d6bc8a87d6496fbcf3d15a216f9488` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:44.671483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aabeef-2de2-4e02-8a0d-121a6d3bb3fd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b641be0d110147699efff68ac7243970, implied_role_id=a1d6bc8a87d6496fbcf3d15a216f9488)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:44.673212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aabeef-2de2-4e02-8a0d-121a6d3bb3fd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:44.724229 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3371/6818] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Sep 3 21:13:44 2022] PUT /identity/v3/roles/b641be0d110147699efff68ac7243970/implies/a1d6bc8a87d6496fbcf3d15a216f9488 => generated 530 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:44.725683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9928b76-b5c4-45d8-bc32-6797803907da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:44.741151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9928b76-b5c4-45d8-bc32-6797803907da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:44.881360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae9a75-8f10-434b-b058-0ea1ad070adf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:44.881995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae9a75-8f10-434b-b058-0ea1ad070adf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:44.882285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae9a75-8f10-434b-b058-0ea1ad070adf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:44.882718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae9a75-8f10-434b-b058-0ea1ad070adf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:44.884074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae9a75-8f10-434b-b058-0ea1ad070adf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:44.885823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae9a75-8f10-434b-b058-0ea1ad070adf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:44.919004 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3448/6819] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:13:44 2022] POST /identity/v3/domains => generated 275 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:44.923110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aabeef-2de2-4e02-8a0d-121a6d3bb3fd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:44.943185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aabeef-2de2-4e02-8a0d-121a6d3bb3fd tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:45.010825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:45.100623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdade460-7354-4d7e-be8a-609f2e3e0811 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:45.102294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdade460-7354-4d7e-be8a-609f2e3e0811 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:45.103255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdade460-7354-4d7e-be8a-609f2e3e0811 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:45.105292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdade460-7354-4d7e-be8a-609f2e3e0811 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:45.106242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdade460-7354-4d7e-be8a-609f2e3e0811 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:45.116214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdade460-7354-4d7e-be8a-609f2e3e0811 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:45.153686 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3372/6820] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:13:44 2022] POST /identity/v3/roles => generated 270 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:45.154778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae9a75-8f10-434b-b058-0ea1ad070adf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:45.173169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae9a75-8f10-434b-b058-0ea1ad070adf tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:45.179877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16168a-7886-4e76-9b78-a9fe1a532e46 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:45.180641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16168a-7886-4e76-9b78-a9fe1a532e46 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:45.181557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16168a-7886-4e76-9b78-a9fe1a532e46 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:45.181910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16168a-7886-4e76-9b78-a9fe1a532e46 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/b641be0d110147699efff68ac7243970/implies/1e124bd9d34d4084897be221d28812ed` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:45.191472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16168a-7886-4e76-9b78-a9fe1a532e46 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b641be0d110147699efff68ac7243970, implied_role_id=1e124bd9d34d4084897be221d28812ed)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:45.192597 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-3f16168a-7886-4e76-9b78-a9fe1a532e46 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:45.192597 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:45.192597 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:45.193733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16168a-7886-4e76-9b78-a9fe1a532e46 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:45.259110 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3449/6821] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Sep 3 21:13:45 2022] PUT /identity/v3/roles/b641be0d110147699efff68ac7243970/implies/1e124bd9d34d4084897be221d28812ed => generated 531 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:45.263371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdade460-7354-4d7e-be8a-609f2e3e0811 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:45.280088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdade460-7354-4d7e-be8a-609f2e3e0811 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:45.473652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65a9bc-b863-4b93-ac97-3d5279b89e0f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:45.479150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65a9bc-b863-4b93-ac97-3d5279b89e0f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:45.479732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65a9bc-b863-4b93-ac97-3d5279b89e0f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:45.480139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65a9bc-b863-4b93-ac97-3d5279b89e0f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/b641be0d110147699efff68ac7243970/implies/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:45.482860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65a9bc-b863-4b93-ac97-3d5279b89e0f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b641be0d110147699efff68ac7243970, implied_role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:45.485329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65a9bc-b863-4b93-ac97-3d5279b89e0f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:45.521198 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3373/6822] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Sep 3 21:13:45 2022] PUT /identity/v3/roles/b641be0d110147699efff68ac7243970/implies/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 526 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:45.525159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16168a-7886-4e76-9b78-a9fe1a532e46 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:45.545137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16168a-7886-4e76-9b78-a9fe1a532e46 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:45.581702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:45.647624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a7f60-d4d3-4257-bb42-439f9a5c1144 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:45.650247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a7f60-d4d3-4257-bb42-439f9a5c1144 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:45.650591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a7f60-d4d3-4257-bb42-439f9a5c1144 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:45.650907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a7f60-d4d3-4257-bb42-439f9a5c1144 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/90a1e59bc57f417581c7b6fadb9dfcb3/implies/b641be0d110147699efff68ac7243970` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:45.653926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a7f60-d4d3-4257-bb42-439f9a5c1144 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=90a1e59bc57f417581c7b6fadb9dfcb3, implied_role_id=b641be0d110147699efff68ac7243970)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:45.657365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a7f60-d4d3-4257-bb42-439f9a5c1144 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-b94a7f60-d4d3-4257-bb42-439f9a5c1144 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] b641be0d110147699efff68ac7243970 cannot be an implied roles.: keystone.exception.InvalidImpliedRole: b641be0d110147699efff68ac7243970 cannot be an implied roles. Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 259, in put Sep 03 21:13:45.685341 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application PROVIDERS.role_api.create_implied_role(prior_role_id, implied_role_id) Sep 03 21:13:45.686501 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:13:45.686501 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:13:45.686501 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1361, in create_implied_role Sep 03 21:13:45.686501 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.InvalidImpliedRole(msg, Sep 03 21:13:45.686501 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.InvalidImpliedRole: b641be0d110147699efff68ac7243970 cannot be an implied roles. Sep 03 21:13:45.686501 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:45.687396 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3450/6823] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Sep 3 21:13:45 2022] PUT /identity/v3/roles/90a1e59bc57f417581c7b6fadb9dfcb3/implies/b641be0d110147699efff68ac7243970 => generated 116 bytes in 163 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Sep 03 21:13:45.691956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65a9bc-b863-4b93-ac97-3d5279b89e0f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:45.713920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65a9bc-b863-4b93-ac97-3d5279b89e0f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:45.734063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e12156-d737-44e3-be4e-8ec6ac347bbe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:45.735554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e12156-d737-44e3-be4e-8ec6ac347bbe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:45.736755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e12156-d737-44e3-be4e-8ec6ac347bbe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:45.737160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e12156-d737-44e3-be4e-8ec6ac347bbe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/b641be0d110147699efff68ac7243970/implies/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:45.742525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e12156-d737-44e3-be4e-8ec6ac347bbe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b641be0d110147699efff68ac7243970, implied_role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:45.746765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e12156-d737-44e3-be4e-8ec6ac347bbe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:45.767188 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3374/6824] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:45 2022] DELETE /identity/v3/roles/b641be0d110147699efff68ac7243970/implies/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:45.771727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a7f60-d4d3-4257-bb42-439f9a5c1144 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:45.803632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a7f60-d4d3-4257-bb42-439f9a5c1144 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:46.006096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0d700-9e76-489f-8385-e688cf33eb20 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:46.006601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0d700-9e76-489f-8385-e688cf33eb20 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:46.006893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0d700-9e76-489f-8385-e688cf33eb20 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:46.007174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0d700-9e76-489f-8385-e688cf33eb20 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/b641be0d110147699efff68ac7243970/implies/1e124bd9d34d4084897be221d28812ed` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:46.011709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0d700-9e76-489f-8385-e688cf33eb20 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b641be0d110147699efff68ac7243970, implied_role_id=1e124bd9d34d4084897be221d28812ed)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:46.016286 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-6ca0d700-9e76-489f-8385-e688cf33eb20 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:46.016286 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:46.016286 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:46.016286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0d700-9e76-489f-8385-e688cf33eb20 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:46.045988 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3451/6825] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:45 2022] DELETE /identity/v3/roles/b641be0d110147699efff68ac7243970/implies/1e124bd9d34d4084897be221d28812ed => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:46.050228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e12156-d737-44e3-be4e-8ec6ac347bbe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:46.078279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e12156-d737-44e3-be4e-8ec6ac347bbe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:46.243905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33c69e-0338-4d7f-957f-902b28ed2c6a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:46.244575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33c69e-0338-4d7f-957f-902b28ed2c6a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:46.244864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33c69e-0338-4d7f-957f-902b28ed2c6a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:46.251126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33c69e-0338-4d7f-957f-902b28ed2c6a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/1e124bd9d34d4084897be221d28812ed` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:46.254238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33c69e-0338-4d7f-957f-902b28ed2c6a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=1e124bd9d34d4084897be221d28812ed)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:46.255929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33c69e-0338-4d7f-957f-902b28ed2c6a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:46.336713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:46.353559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab33c69e-0338-4d7f-957f-902b28ed2c6a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 1e124bd9d34d4084897be221d28812ed has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:46.354733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab33c69e-0338-4d7f-957f-902b28ed2c6a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:46.358193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ab33c69e-0338-4d7f-957f-902b28ed2c6a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:46.360525 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3375/6826] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:46 2022] DELETE /identity/v3/roles/1e124bd9d34d4084897be221d28812ed => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:46.365152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0d700-9e76-489f-8385-e688cf33eb20 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:46.367163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6ca0d700-9e76-489f-8385-e688cf33eb20 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:46.392590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0d700-9e76-489f-8385-e688cf33eb20 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:46.506617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:46.507343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:46.507643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:46.507929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/37353ca33f324b02a35b8388ba8ef3f8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:46.527217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=37353ca33f324b02a35b8388ba8ef3f8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:46.529372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:46.577531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '37353ca33f324b02a35b8388ba8ef3f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:46.584429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '37353ca33f324b02a35b8388ba8ef3f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:46.648794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '37353ca33f324b02a35b8388ba8ef3f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:46.651235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '37353ca33f324b02a35b8388ba8ef3f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:46.654236 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3452/6827] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:13:46 2022] PATCH /identity/v3/domains/37353ca33f324b02a35b8388ba8ef3f8 => generated 276 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:46.658639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33c69e-0338-4d7f-957f-902b28ed2c6a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:46.672646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33c69e-0338-4d7f-957f-902b28ed2c6a tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:46.822331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:46.824402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:46.824750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:46.826154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/37353ca33f324b02a35b8388ba8ef3f8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:46.851405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=37353ca33f324b02a35b8388ba8ef3f8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:46.853407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:46.887468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '37353ca33f324b02a35b8388ba8ef3f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:46.893228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '37353ca33f324b02a35b8388ba8ef3f8'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:46.974759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37353ca33f324b02a35b8388ba8ef3f8', 'request_id': 'req-7f268298-1842-4f49-8ba5-9def551d8ba1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:46.993302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37353ca33f324b02a35b8388ba8ef3f8', 'request_id': 'req-7f268298-1842-4f49-8ba5-9def551d8ba1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:47.003228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37353ca33f324b02a35b8388ba8ef3f8', 'request_id': 'req-7f268298-1842-4f49-8ba5-9def551d8ba1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:47.019257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] The token cache is being invalidate because project 37353ca33f324b02a35b8388ba8ef3f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:47.019843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:47.022191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:47.055308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '37353ca33f324b02a35b8388ba8ef3f8', 'request_id': 'req-7f268298-1842-4f49-8ba5-9def551d8ba1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:47.062102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '37353ca33f324b02a35b8388ba8ef3f8', 'request_id': 'req-7f268298-1842-4f49-8ba5-9def551d8ba1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:47.064509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7f268298-1842-4f49-8ba5-9def551d8ba1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '37353ca33f324b02a35b8388ba8ef3f8', 'request_id': 'req-7f268298-1842-4f49-8ba5-9def551d8ba1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:47.085233 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3376/6828] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:13:46 2022] DELETE /identity/v3/domains/37353ca33f324b02a35b8388ba8ef3f8 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:47.089848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:47.091932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:47.107591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:UPDATE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:47.125341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d86758-94b6-45de-aa35-15bfa026197f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:47.256830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646ce10-98a6-4576-9ec4-dabf159e0ccd None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:47.257164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646ce10-98a6-4576-9ec4-dabf159e0ccd None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:47.257446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646ce10-98a6-4576-9ec4-dabf159e0ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:47.269634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c7b30-45bc-4b2e-895d-2fe46ecef49d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:47.270312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c7b30-45bc-4b2e-895d-2fe46ecef49d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:47.271534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c7b30-45bc-4b2e-895d-2fe46ecef49d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:47.271830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c7b30-45bc-4b2e-895d-2fe46ecef49d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/b641be0d110147699efff68ac7243970/implies/a1d6bc8a87d6496fbcf3d15a216f9488` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:47.273895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c7b30-45bc-4b2e-895d-2fe46ecef49d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b641be0d110147699efff68ac7243970, implied_role_id=a1d6bc8a87d6496fbcf3d15a216f9488)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:47.275720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c7b30-45bc-4b2e-895d-2fe46ecef49d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:47.297814 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3453/6829] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:47 2022] DELETE /identity/v3/roles/b641be0d110147699efff68ac7243970/implies/a1d6bc8a87d6496fbcf3d15a216f9488 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:47.299300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c7b30-45bc-4b2e-895d-2fe46ecef49d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:47.302972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-371c7b30-45bc-4b2e-895d-2fe46ecef49d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:47.320722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a646ce10-98a6-4576-9ec4-dabf159e0ccd None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:47.322993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a646ce10-98a6-4576-9ec4-dabf159e0ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:47.398186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:47.449392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c7b30-45bc-4b2e-895d-2fe46ecef49d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:47.459534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:47.460198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:47.460489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:47.460769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/a1d6bc8a87d6496fbcf3d15a216f9488` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:47.467458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=a1d6bc8a87d6496fbcf3d15a216f9488)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:47.468666 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:47.468666 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:47.468666 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:47.469624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:47.513165 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3377/6830] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:13:47.514800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb6367-9ea6-49eb-9ca6-8489ac994541 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:47.515091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb6367-9ea6-49eb-9ca6-8489ac994541 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:47.515478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb6367-9ea6-49eb-9ca6-8489ac994541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:47.554822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-99cb6367-9ea6-49eb-9ca6-8489ac994541 None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:47.556360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role a1d6bc8a87d6496fbcf3d15a216f9488 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:47.556848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:47.557375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-99cb6367-9ea6-49eb-9ca6-8489ac994541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:47.560649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:47.566549 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3454/6831] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:47 2022] DELETE /identity/v3/roles/a1d6bc8a87d6496fbcf3d15a216f9488 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:47.569096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:47.571403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:47.573280 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3378/6832] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:47.597936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb6367-9ea6-49eb-9ca6-8489ac994541 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:47.605726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-99cb6367-9ea6-49eb-9ca6-8489ac994541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:47.757664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156de97-83cf-47b5-a89f-c234771c315c tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:47.764798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f397a-5a6d-4a6a-aca7-e8d2d08d2e39 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:47.765747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f397a-5a6d-4a6a-aca7-e8d2d08d2e39 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:47.766146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f397a-5a6d-4a6a-aca7-e8d2d08d2e39 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:47.766537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f397a-5a6d-4a6a-aca7-e8d2d08d2e39 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/b641be0d110147699efff68ac7243970` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:47.767939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f397a-5a6d-4a6a-aca7-e8d2d08d2e39 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=b641be0d110147699efff68ac7243970)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:47.769924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f397a-5a6d-4a6a-aca7-e8d2d08d2e39 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:47.776771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:47.841218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c90f397a-5a6d-4a6a-aca7-e8d2d08d2e39 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role b641be0d110147699efff68ac7243970 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:47.841700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c90f397a-5a6d-4a6a-aca7-e8d2d08d2e39 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:47.842664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c90f397a-5a6d-4a6a-aca7-e8d2d08d2e39 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:47.850815 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3455/6833] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:47 2022] DELETE /identity/v3/roles/b641be0d110147699efff68ac7243970 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:47.859799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb6367-9ea6-49eb-9ca6-8489ac994541 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:47.863442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f397a-5a6d-4a6a-aca7-e8d2d08d2e39 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:47.866410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c90f397a-5a6d-4a6a-aca7-e8d2d08d2e39 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:47.987617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f397a-5a6d-4a6a-aca7-e8d2d08d2e39 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:47.995204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3c2e2-4c65-43eb-9ce9-6872baf2ba88 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:47.996351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3c2e2-4c65-43eb-9ce9-6872baf2ba88 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:47.996528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3c2e2-4c65-43eb-9ce9-6872baf2ba88 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:47.998776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3c2e2-4c65-43eb-9ce9-6872baf2ba88 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:48.000074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3c2e2-4c65-43eb-9ce9-6872baf2ba88 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:48.002391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3c2e2-4c65-43eb-9ce9-6872baf2ba88 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:48.014724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5fce7-90dc-42dc-8231-c3bc5575a72d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:48.015965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5fce7-90dc-42dc-8231-c3bc5575a72d service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:48.016542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5fce7-90dc-42dc-8231-c3bc5575a72d service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:48.019263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5fce7-90dc-42dc-8231-c3bc5575a72d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:48.021637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-54f5fce7-90dc-42dc-8231-c3bc5575a72d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:48.042667 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3456/6834] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Sep 3 21:13:47 2022] PUT /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 520 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:48.046558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3c2e2-4c65-43eb-9ce9-6872baf2ba88 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:48.078214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:48.120380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5fce7-90dc-42dc-8231-c3bc5575a72d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:48.122681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5fce7-90dc-42dc-8231-c3bc5575a72d service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:48.143493 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3379/6835] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:13:48.206041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5fce7-90dc-42dc-8231-c3bc5575a72d service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:48.207534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-54f5fce7-90dc-42dc-8231-c3bc5575a72d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:48.232715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3c2e2-4c65-43eb-9ce9-6872baf2ba88 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:48.239545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb5bc9-6563-4c75-886d-f3bedc9c9642 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:48.243606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb5bc9-6563-4c75-886d-f3bedc9c9642 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:48.244010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb5bc9-6563-4c75-886d-f3bedc9c9642 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:48.244884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb5bc9-6563-4c75-886d-f3bedc9c9642 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:48.247536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb5bc9-6563-4c75-886d-f3bedc9c9642 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:check_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:48.248690 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-7dbb5bc9-6563-4c75-886d-f3bedc9c9642 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:48.248690 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:48.248690 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:48.250004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb5bc9-6563-4c75-886d-f3bedc9c9642 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:48.251048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb5bc9-6563-4c75-886d-f3bedc9c9642 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:48.252059 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-7dbb5bc9-6563-4c75-886d-f3bedc9c9642 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:48.252059 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:48.252059 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:48.253409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb5bc9-6563-4c75-886d-f3bedc9c9642 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:48.262846 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3457/6836] 10.222.0.16 () {58 vars in 1345 bytes} [Sat Sep 3 21:13:48 2022] HEAD /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:48.264307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb5bc9-6563-4c75-886d-f3bedc9c9642 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:48.283758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb5bc9-6563-4c75-886d-f3bedc9c9642 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:48.289566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb65a9-74f9-41ac-b0bc-b8a7798418da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:48.290236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb65a9-74f9-41ac-b0bc-b8a7798418da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:48.290526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb65a9-74f9-41ac-b0bc-b8a7798418da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:48.290809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb65a9-74f9-41ac-b0bc-b8a7798418da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:48.292512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb65a9-74f9-41ac-b0bc-b8a7798418da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:48.294892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb65a9-74f9-41ac-b0bc-b8a7798418da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:48.302224 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3458/6837] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Sep 3 21:13:48 2022] GET /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 520 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:48.306646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb65a9-74f9-41ac-b0bc-b8a7798418da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:48.322055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb65a9-74f9-41ac-b0bc-b8a7798418da tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:48.327545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a6a18-3f1c-4632-b771-972f6c5ccff6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:48.328229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a6a18-3f1c-4632-b771-972f6c5ccff6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:48.328514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a6a18-3f1c-4632-b771-972f6c5ccff6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:48.328796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a6a18-3f1c-4632-b771-972f6c5ccff6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:48.331002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a6a18-3f1c-4632-b771-972f6c5ccff6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:48.332771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a6a18-3f1c-4632-b771-972f6c5ccff6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:48.349186 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3459/6838] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:48 2022] DELETE /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:48.350387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a6a18-3f1c-4632-b771-972f6c5ccff6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:48.388987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5fce7-90dc-42dc-8231-c3bc5575a72d service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:48.397855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:48.440163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a6a18-3f1c-4632-b771-972f6c5ccff6 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:48.449397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b872a98-7b0d-401b-a862-ccf46af59405 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:48.450102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b872a98-7b0d-401b-a862-ccf46af59405 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:48.450391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b872a98-7b0d-401b-a862-ccf46af59405 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:48.450672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b872a98-7b0d-401b-a862-ccf46af59405 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:48.452349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b872a98-7b0d-401b-a862-ccf46af59405 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:48.454816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b872a98-7b0d-401b-a862-ccf46af59405 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-6b872a98-7b0d-401b-a862-ccf46af59405 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] 94f4ea5bd8d54d608c134889670b1995 does not imply 7159359626c74e5793a5237927e9933c.: keystone.exception.ImpliedRoleNotFound: 94f4ea5bd8d54d608c134889670b1995 does not imply 7159359626c74e5793a5237927e9933c. Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = query.one() Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._iter().one() Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._only_one_row( Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:48.467507 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 235, in get Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_implied_role(prior_role_id, implied_role_id) Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 239, in _get_implied_role Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application PROVIDERS.role_api.get_implied_role( Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 149, in get_implied_role Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 03 21:13:48.468847 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 03 21:13:48.470129 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 03 21:13:48.470129 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 94f4ea5bd8d54d608c134889670b1995 does not imply 7159359626c74e5793a5237927e9933c. Sep 03 21:13:48.470129 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:48.470129 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3460/6839] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Sep 3 21:13:48 2022] GET /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 137 bytes in 119 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:13:48.473657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b872a98-7b0d-401b-a862-ccf46af59405 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:48.487157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6f54e-3181-4928-a2ad-6892ba9b3c9b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:48.487794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6f54e-3181-4928-a2ad-6892ba9b3c9b service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:48.488076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6f54e-3181-4928-a2ad-6892ba9b3c9b service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:48.488359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6f54e-3181-4928-a2ad-6892ba9b3c9b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:48.488495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b872a98-7b0d-401b-a862-ccf46af59405 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:48.490344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8ff6f54e-3181-4928-a2ad-6892ba9b3c9b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:48.494720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ad02c-0be6-4787-be13-7a9a9957a96d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:48.495401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ad02c-0be6-4787-be13-7a9a9957a96d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:48.495969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ad02c-0be6-4787-be13-7a9a9957a96d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:48.496252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ad02c-0be6-4787-be13-7a9a9957a96d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:48.500870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ad02c-0be6-4787-be13-7a9a9957a96d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:48.503543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ad02c-0be6-4787-be13-7a9a9957a96d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-ac3ad02c-0be6-4787-be13-7a9a9957a96d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] 94f4ea5bd8d54d608c134889670b1995 does not imply 7159359626c74e5793a5237927e9933c.: keystone.exception.ImpliedRoleNotFound: 94f4ea5bd8d54d608c134889670b1995 does not imply 7159359626c74e5793a5237927e9933c. Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = query.one() Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._iter().one() Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._only_one_row( Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:48.516979 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Sep 03 21:13:48.518508 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Sep 03 21:13:48.519821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 03 21:13:48.519821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 03 21:13:48.519821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 03 21:13:48.519821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 94f4ea5bd8d54d608c134889670b1995 does not imply 7159359626c74e5793a5237927e9933c. Sep 03 21:13:48.519821 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:48.522824 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3461/6840] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:48 2022] DELETE /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 137 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:13:48.523781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ad02c-0be6-4787-be13-7a9a9957a96d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:48.538263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ad02c-0be6-4787-be13-7a9a9957a96d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:48.544521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbe0bd-b73a-45e9-af58-f3ae10094f07 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:48.545248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbe0bd-b73a-45e9-af58-f3ae10094f07 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:48.545538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbe0bd-b73a-45e9-af58-f3ae10094f07 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:48.545889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbe0bd-b73a-45e9-af58-f3ae10094f07 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:48.547853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbe0bd-b73a-45e9-af58-f3ae10094f07 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:48.550208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbe0bd-b73a-45e9-af58-f3ae10094f07 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:48.604797 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3462/6841] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Sep 3 21:13:48 2022] PUT /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 520 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:48.606892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbe0bd-b73a-45e9-af58-f3ae10094f07 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:48.712999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbe0bd-b73a-45e9-af58-f3ae10094f07 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:48.719931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf79d5-45ce-49b5-b825-4d6e2c642911 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:48.721279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf79d5-45ce-49b5-b825-4d6e2c642911 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:48.721570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf79d5-45ce-49b5-b825-4d6e2c642911 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:48.721851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf79d5-45ce-49b5-b825-4d6e2c642911 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/23541c3d49744a43a459b173d65a78c1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:48.724737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf79d5-45ce-49b5-b825-4d6e2c642911 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=23541c3d49744a43a459b173d65a78c1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:48.728305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf79d5-45ce-49b5-b825-4d6e2c642911 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:48.775584 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3463/6842] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Sep 3 21:13:48 2022] PUT /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/23541c3d49744a43a459b173d65a78c1 => generated 518 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:48.778317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf79d5-45ce-49b5-b825-4d6e2c642911 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:48.864679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6f54e-3181-4928-a2ad-6892ba9b3c9b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:48.869055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6f54e-3181-4928-a2ad-6892ba9b3c9b service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:48.901276 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3380/6843] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:13:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:13:48.920781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf79d5-45ce-49b5-b825-4d6e2c642911 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:48.931352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605dba97-a183-4220-b205-85ec5b9ac07e tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:48.931633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605dba97-a183-4220-b205-85ec5b9ac07e tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:48.931968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605dba97-a183-4220-b205-85ec5b9ac07e tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:48.932667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605dba97-a183-4220-b205-85ec5b9ac07e tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/23541c3d49744a43a459b173d65a78c1/implies/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:48.934543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605dba97-a183-4220-b205-85ec5b9ac07e tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=23541c3d49744a43a459b173d65a78c1, implied_role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:48.936448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605dba97-a183-4220-b205-85ec5b9ac07e tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:49.000036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:49.026096 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3464/6844] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Sep 3 21:13:48 2022] PUT /identity/v3/roles/23541c3d49744a43a459b173d65a78c1/implies/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 519 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:49.026720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6f54e-3181-4928-a2ad-6892ba9b3c9b service glance] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:49.045255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6f54e-3181-4928-a2ad-6892ba9b3c9b service glance] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:49.064522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:49.166866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd94d5f-2bd2-4ae8-b7aa-dfd485431ea8 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:49.167728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd94d5f-2bd2-4ae8-b7aa-dfd485431ea8 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:49.168115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd94d5f-2bd2-4ae8-b7aa-dfd485431ea8 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:49.168527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd94d5f-2bd2-4ae8-b7aa-dfd485431ea8 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/role_inferences` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:49.169431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd94d5f-2bd2-4ae8-b7aa-dfd485431ea8 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_role_inference_rules()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:49.170650 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-3fd94d5f-2bd2-4ae8-b7aa-dfd485431ea8 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_inference_rules": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:49.170650 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:49.170650 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:49.171807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd94d5f-2bd2-4ae8-b7aa-dfd485431ea8 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:49.215175 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3381/6845] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Sep 3 21:13:49 2022] GET /identity/v3/role_inferences => generated 1603 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:49.217513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605dba97-a183-4220-b205-85ec5b9ac07e tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:49.241287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605dba97-a183-4220-b205-85ec5b9ac07e tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:49.268001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05633e7-367b-499e-8a4c-9d2c875ffd09 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:49.268742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05633e7-367b-499e-8a4c-9d2c875ffd09 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:49.269074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05633e7-367b-499e-8a4c-9d2c875ffd09 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:49.269373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05633e7-367b-499e-8a4c-9d2c875ffd09 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/23541c3d49744a43a459b173d65a78c1/implies/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:49.272316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05633e7-367b-499e-8a4c-9d2c875ffd09 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=23541c3d49744a43a459b173d65a78c1, implied_role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:49.274019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05633e7-367b-499e-8a4c-9d2c875ffd09 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:49.299534 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3465/6846] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:49 2022] DELETE /identity/v3/roles/23541c3d49744a43a459b173d65a78c1/implies/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:49.303440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd94d5f-2bd2-4ae8-b7aa-dfd485431ea8 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:49.317620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd94d5f-2bd2-4ae8-b7aa-dfd485431ea8 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:49.461333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b2cd3-46e4-408a-82c1-00ef067085ca tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:49.462511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b2cd3-46e4-408a-82c1-00ef067085ca tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:49.463031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b2cd3-46e4-408a-82c1-00ef067085ca tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:49.463562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b2cd3-46e4-408a-82c1-00ef067085ca tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/23541c3d49744a43a459b173d65a78c1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:49.468907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b2cd3-46e4-408a-82c1-00ef067085ca tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=23541c3d49744a43a459b173d65a78c1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:49.474434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b2cd3-46e4-408a-82c1-00ef067085ca tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:49.498755 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3382/6847] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:49 2022] DELETE /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/23541c3d49744a43a459b173d65a78c1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:49.502560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05633e7-367b-499e-8a4c-9d2c875ffd09 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:49.516293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05633e7-367b-499e-8a4c-9d2c875ffd09 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:49.620264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ba1f7-5732-488c-8795-dcd894417306 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:49.620957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ba1f7-5732-488c-8795-dcd894417306 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:49.621255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ba1f7-5732-488c-8795-dcd894417306 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:49.622081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ba1f7-5732-488c-8795-dcd894417306 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:49.623944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ba1f7-5732-488c-8795-dcd894417306 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:49.625651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ba1f7-5732-488c-8795-dcd894417306 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:49.649358 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3466/6848] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:49 2022] DELETE /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:49.654734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b2cd3-46e4-408a-82c1-00ef067085ca tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:49.669418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b2cd3-46e4-408a-82c1-00ef067085ca tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:49.834608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60177175-213b-491f-b8c1-c3e0b15fc2f1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:49.835115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60177175-213b-491f-b8c1-c3e0b15fc2f1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:49.835561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60177175-213b-491f-b8c1-c3e0b15fc2f1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:49.835943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60177175-213b-491f-b8c1-c3e0b15fc2f1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:49.836696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60177175-213b-491f-b8c1-c3e0b15fc2f1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:49.839062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60177175-213b-491f-b8c1-c3e0b15fc2f1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:49.868190 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3383/6849] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:49 2022] GET /identity/v3/roles => generated 2321 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:49.875129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ba1f7-5732-488c-8795-dcd894417306 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:49.897642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ba1f7-5732-488c-8795-dcd894417306 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:49.905307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030bcaa-db67-41ff-8f0d-caba831d6021 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:49.905943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030bcaa-db67-41ff-8f0d-caba831d6021 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:49.906235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030bcaa-db67-41ff-8f0d-caba831d6021 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:49.906520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030bcaa-db67-41ff-8f0d-caba831d6021 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:49.907218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030bcaa-db67-41ff-8f0d-caba831d6021 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:49.908848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030bcaa-db67-41ff-8f0d-caba831d6021 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:49.944716 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3467/6850] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:13:49 2022] POST /identity/v3/roles => generated 235 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:49.948883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60177175-213b-491f-b8c1-c3e0b15fc2f1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:49.979525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60177175-213b-491f-b8c1-c3e0b15fc2f1 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:49.988583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288e763-26d6-4d07-9d0e-8cce86b1616b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:49.989669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288e763-26d6-4d07-9d0e-8cce86b1616b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:49.990059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288e763-26d6-4d07-9d0e-8cce86b1616b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:49.990529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288e763-26d6-4d07-9d0e-8cce86b1616b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/f57caf756ef449c5bb04e20ade0c34af` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:49.997647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288e763-26d6-4d07-9d0e-8cce86b1616b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:update_role(role_id=f57caf756ef449c5bb04e20ade0c34af)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:49.999039 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-2288e763-26d6-4d07-9d0e-8cce86b1616b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:49.999039 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:49.999039 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:50.000213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288e763-26d6-4d07-9d0e-8cce86b1616b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:50.050543 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3384/6851] 10.222.0.16 () {60 vars in 1243 bytes} [Sat Sep 3 21:13:49 2022] PATCH /identity/v3/roles/f57caf756ef449c5bb04e20ade0c34af => generated 238 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:50.050929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030bcaa-db67-41ff-8f0d-caba831d6021 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:50.073306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030bcaa-db67-41ff-8f0d-caba831d6021 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:50.086563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891fa6-b1e5-4bdd-8a73-938fa00db462 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:50.087232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891fa6-b1e5-4bdd-8a73-938fa00db462 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:50.088115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891fa6-b1e5-4bdd-8a73-938fa00db462 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:50.088415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891fa6-b1e5-4bdd-8a73-938fa00db462 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/f57caf756ef449c5bb04e20ade0c34af` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:50.110012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d891fa6-b1e5-4bdd-8a73-938fa00db462 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:get_role(role_id=f57caf756ef449c5bb04e20ade0c34af)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:50.111069 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-9d891fa6-b1e5-4bdd-8a73-938fa00db462 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:50.111069 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:50.111069 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:50.114135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d891fa6-b1e5-4bdd-8a73-938fa00db462 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:50.121080 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3468/6852] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Sep 3 21:13:50 2022] GET /identity/v3/roles/f57caf756ef449c5bb04e20ade0c34af => generated 238 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:50.121304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288e763-26d6-4d07-9d0e-8cce86b1616b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:50.139885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288e763-26d6-4d07-9d0e-8cce86b1616b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:50.147128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce87f0-d24e-449a-9c5f-624f104f74fe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:50.147811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce87f0-d24e-449a-9c5f-624f104f74fe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:50.148102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce87f0-d24e-449a-9c5f-624f104f74fe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:50.148524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce87f0-d24e-449a-9c5f-624f104f74fe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:50.149221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce87f0-d24e-449a-9c5f-624f104f74fe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:50.152203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce87f0-d24e-449a-9c5f-624f104f74fe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:50.180954 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3385/6853] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:50 2022] GET /identity/v3/roles => generated 2550 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:50.184387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891fa6-b1e5-4bdd-8a73-938fa00db462 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:50.199217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891fa6-b1e5-4bdd-8a73-938fa00db462 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:50.214932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd584b-1ca4-441c-a2ab-50b4e8a088bb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:50.216147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd584b-1ca4-441c-a2ab-50b4e8a088bb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:50.216460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd584b-1ca4-441c-a2ab-50b4e8a088bb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:50.216709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd584b-1ca4-441c-a2ab-50b4e8a088bb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/f57caf756ef449c5bb04e20ade0c34af` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:50.218203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd584b-1ca4-441c-a2ab-50b4e8a088bb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f57caf756ef449c5bb04e20ade0c34af)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:50.219979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd584b-1ca4-441c-a2ab-50b4e8a088bb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:50.296611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0cbd584b-1ca4-441c-a2ab-50b4e8a088bb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role f57caf756ef449c5bb04e20ade0c34af has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:50.296940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0cbd584b-1ca4-441c-a2ab-50b4e8a088bb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:50.297643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0cbd584b-1ca4-441c-a2ab-50b4e8a088bb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:50.301523 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3469/6854] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:50 2022] DELETE /identity/v3/roles/f57caf756ef449c5bb04e20ade0c34af => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:50.306797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce87f0-d24e-449a-9c5f-624f104f74fe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:50.308630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f8ce87f0-d24e-449a-9c5f-624f104f74fe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:50.334771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce87f0-d24e-449a-9c5f-624f104f74fe tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:50.339176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:UPDATE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:50.459818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8c14c-b158-462c-bc48-b80a4874da1d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:50.460855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8c14c-b158-462c-bc48-b80a4874da1d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:50.461298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8c14c-b158-462c-bc48-b80a4874da1d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:50.461815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8c14c-b158-462c-bc48-b80a4874da1d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:50.463906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8c14c-b158-462c-bc48-b80a4874da1d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:50.466260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8c14c-b158-462c-bc48-b80a4874da1d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:50.530200 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3386/6855] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Sep 3 21:13:50 2022] PUT /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 520 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:50.530459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd584b-1ca4-441c-a2ab-50b4e8a088bb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:50.559901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd584b-1ca4-441c-a2ab-50b4e8a088bb tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:50.596603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8c14c-b158-462c-bc48-b80a4874da1d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:50.598890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e9f8c14c-b158-462c-bc48-b80a4874da1d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:50.714973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6255a83-21cd-4e81-81c8-6706cf752690 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:50.716213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6255a83-21cd-4e81-81c8-6706cf752690 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:50.716506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6255a83-21cd-4e81-81c8-6706cf752690 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:50.716786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6255a83-21cd-4e81-81c8-6706cf752690 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/23541c3d49744a43a459b173d65a78c1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:50.722721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6255a83-21cd-4e81-81c8-6706cf752690 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=23541c3d49744a43a459b173d65a78c1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:50.724374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6255a83-21cd-4e81-81c8-6706cf752690 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:50.728468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8c14c-b158-462c-bc48-b80a4874da1d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:50.734508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab4c63-0190-4716-ac3d-733fb8738799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:50.736272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab4c63-0190-4716-ac3d-733fb8738799 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:50.736272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab4c63-0190-4716-ac3d-733fb8738799 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:50.736272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab4c63-0190-4716-ac3d-733fb8738799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:50.739471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d5ab4c63-0190-4716-ac3d-733fb8738799 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:50.778935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:50.816929 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3470/6856] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Sep 3 21:13:50 2022] PUT /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/23541c3d49744a43a459b173d65a78c1 => generated 518 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:50.818208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6255a83-21cd-4e81-81c8-6706cf752690 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:50.943539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab4c63-0190-4716-ac3d-733fb8738799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:50.946051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab4c63-0190-4716-ac3d-733fb8738799 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:50.972387 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3387/6857] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:13:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:13:51.010372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6255a83-21cd-4e81-81c8-6706cf752690 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:51.022739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434121d-dda7-43d7-9aef-c31b066ad430 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:51.022739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434121d-dda7-43d7-9aef-c31b066ad430 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:51.022739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434121d-dda7-43d7-9aef-c31b066ad430 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:51.023362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434121d-dda7-43d7-9aef-c31b066ad430 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/23541c3d49744a43a459b173d65a78c1/implies/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:51.024381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434121d-dda7-43d7-9aef-c31b066ad430 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=23541c3d49744a43a459b173d65a78c1, implied_role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:51.027444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434121d-dda7-43d7-9aef-c31b066ad430 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:51.095285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab4c63-0190-4716-ac3d-733fb8738799 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:51.103399 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3471/6858] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Sep 3 21:13:50 2022] PUT /identity/v3/roles/23541c3d49744a43a459b173d65a78c1/implies/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 519 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:51.133233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab4c63-0190-4716-ac3d-733fb8738799 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:51.183912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:51.225104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4b446-656a-44d2-9d44-12adaf6445a2 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:51.225950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4b446-656a-44d2-9d44-12adaf6445a2 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:51.226245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4b446-656a-44d2-9d44-12adaf6445a2 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:51.226522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4b446-656a-44d2-9d44-12adaf6445a2 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/23541c3d49744a43a459b173d65a78c1/implies` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:51.227722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4b446-656a-44d2-9d44-12adaf6445a2 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=23541c3d49744a43a459b173d65a78c1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:51.229003 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-d6d4b446-656a-44d2-9d44-12adaf6445a2 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:51.229003 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:13:51.229003 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:13:51.229966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4b446-656a-44d2-9d44-12adaf6445a2 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:51.241384 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3388/6859] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Sep 3 21:13:51 2022] GET /identity/v3/roles/23541c3d49744a43a459b173d65a78c1/implies => generated 488 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:51.244271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434121d-dda7-43d7-9aef-c31b066ad430 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:51.259856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434121d-dda7-43d7-9aef-c31b066ad430 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:51.273673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc79e1-410d-41d2-920d-7f0eae039557 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:51.274422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc79e1-410d-41d2-920d-7f0eae039557 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:51.274718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc79e1-410d-41d2-920d-7f0eae039557 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:51.275167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc79e1-410d-41d2-920d-7f0eae039557 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:51.276798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc79e1-410d-41d2-920d-7f0eae039557 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=94f4ea5bd8d54d608c134889670b1995)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:51.278716 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-c2fc79e1-410d-41d2-920d-7f0eae039557 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:13:51.278716 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:13:51.278716 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:13:51.281457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc79e1-410d-41d2-920d-7f0eae039557 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:51.308334 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3472/6860] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Sep 3 21:13:51 2022] GET /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies => generated 658 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:51.313738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4b446-656a-44d2-9d44-12adaf6445a2 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:51.337015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4b446-656a-44d2-9d44-12adaf6445a2 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:51.355204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6668d9a-6c7d-452a-a2cf-ffb973f3f34f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:51.357201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6668d9a-6c7d-452a-a2cf-ffb973f3f34f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:51.357456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6668d9a-6c7d-452a-a2cf-ffb973f3f34f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:51.358199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6668d9a-6c7d-452a-a2cf-ffb973f3f34f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/23541c3d49744a43a459b173d65a78c1/implies/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:51.362926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6668d9a-6c7d-452a-a2cf-ffb973f3f34f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=23541c3d49744a43a459b173d65a78c1, implied_role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:51.364673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6668d9a-6c7d-452a-a2cf-ffb973f3f34f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:51.392044 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3389/6861] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:51 2022] DELETE /identity/v3/roles/23541c3d49744a43a459b173d65a78c1/implies/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:51.396247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc79e1-410d-41d2-920d-7f0eae039557 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:51.413348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc79e1-410d-41d2-920d-7f0eae039557 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:51.678716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d01bb-880c-4862-89ff-340a8c750dc4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:51.679939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d01bb-880c-4862-89ff-340a8c750dc4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:51.680519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d01bb-880c-4862-89ff-340a8c750dc4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:51.681231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d01bb-880c-4862-89ff-340a8c750dc4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/23541c3d49744a43a459b173d65a78c1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:51.683895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d01bb-880c-4862-89ff-340a8c750dc4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=23541c3d49744a43a459b173d65a78c1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:51.686683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d01bb-880c-4862-89ff-340a8c750dc4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:51.707451 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3473/6862] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:51 2022] DELETE /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/23541c3d49744a43a459b173d65a78c1 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:51.709596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6668d9a-6c7d-452a-a2cf-ffb973f3f34f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:51.727134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6668d9a-6c7d-452a-a2cf-ffb973f3f34f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:51.827926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4433bd-b7fd-48f6-9038-22c84c734f6b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:51.828748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4433bd-b7fd-48f6-9038-22c84c734f6b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:51.830029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4433bd-b7fd-48f6-9038-22c84c734f6b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:51.830575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4433bd-b7fd-48f6-9038-22c84c734f6b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:51.832931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4433bd-b7fd-48f6-9038-22c84c734f6b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=94f4ea5bd8d54d608c134889670b1995, implied_role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:51.836821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4433bd-b7fd-48f6-9038-22c84c734f6b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:51.857842 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3390/6863] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Sep 3 21:13:51 2022] DELETE /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995/implies/7159359626c74e5793a5237927e9933c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:51.863387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d01bb-880c-4862-89ff-340a8c750dc4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:51.876013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d01bb-880c-4862-89ff-340a8c750dc4 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:51.971725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94765ed4-24be-47dd-94c9-2279a97c83b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:51.972702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94765ed4-24be-47dd-94c9-2279a97c83b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:51.973222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94765ed4-24be-47dd-94c9-2279a97c83b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:51.973803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94765ed4-24be-47dd-94c9-2279a97c83b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/37353ca33f324b02a35b8388ba8ef3f8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:51.979661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=2,keystone:SELECT=97,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:51.990966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94765ed4-24be-47dd-94c9-2279a97c83b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-94765ed4-24be-47dd-94c9-2279a97c83b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Could not find domain: 37353ca33f324b02a35b8388ba8ef3f8.: keystone.exception.DomainNotFound: Could not find domain: 37353ca33f324b02a35b8388ba8ef3f8. Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 37353ca33f324b02a35b8388ba8ef3f8. Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:13:52.015732 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:13:52.017114 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 37353ca33f324b02a35b8388ba8ef3f8. Sep 03 21:13:52.018766 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:13:52.019976 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3474/6864] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:13:51 2022] PATCH /identity/v3/domains/37353ca33f324b02a35b8388ba8ef3f8 => generated 112 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:13:52.022673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4433bd-b7fd-48f6-9038-22c84c734f6b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:52.042171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4433bd-b7fd-48f6-9038-22c84c734f6b tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:52.060560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:52.061263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:52.061557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:52.062062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/users/c42ff7a66af7464b97e9c2450dc38558` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:52.074078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_user(user_id=c42ff7a66af7464b97e9c2450dc38558)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:52.076644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:52.189020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c42ff7a66af7464b97e9c2450dc38558'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:52.217944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c42ff7a66af7464b97e9c2450dc38558'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:52.219153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c42ff7a66af7464b97e9c2450dc38558'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:52.229632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c42ff7a66af7464b97e9c2450dc38558'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:52.243691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c42ff7a66af7464b97e9c2450dc38558'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:52.252951 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3391/6865] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:52 2022] DELETE /identity/v3/users/c42ff7a66af7464b97e9c2450dc38558 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:52.257496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94765ed4-24be-47dd-94c9-2279a97c83b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:52.265503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-94765ed4-24be-47dd-94c9-2279a97c83b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:52.283077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94765ed4-24be-47dd-94c9-2279a97c83b3 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:52.370731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68b21d-6aaf-4b40-bf5b-a38840d55f32 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:52.372187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68b21d-6aaf-4b40-bf5b-a38840d55f32 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:52.372424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68b21d-6aaf-4b40-bf5b-a38840d55f32 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:52.372825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68b21d-6aaf-4b40-bf5b-a38840d55f32 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/90a1e59bc57f417581c7b6fadb9dfcb3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:52.374436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68b21d-6aaf-4b40-bf5b-a38840d55f32 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_role(role_id=90a1e59bc57f417581c7b6fadb9dfcb3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:52.376385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68b21d-6aaf-4b40-bf5b-a38840d55f32 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:52.437068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ac68b21d-6aaf-4b40-bf5b-a38840d55f32 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 90a1e59bc57f417581c7b6fadb9dfcb3 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:52.437517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ac68b21d-6aaf-4b40-bf5b-a38840d55f32 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:52.438394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ac68b21d-6aaf-4b40-bf5b-a38840d55f32 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:52.440370 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3475/6866] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:52 2022] DELETE /identity/v3/roles/90a1e59bc57f417581c7b6fadb9dfcb3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:52.444550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:52.458064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636a915-08e7-41fa-a994-7fcd0705c4b5 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:52.486755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:52.555210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1344d-fbb2-4021-96b2-1c31f2b831f9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:52.556412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1344d-fbb2-4021-96b2-1c31f2b831f9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:52.557016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1344d-fbb2-4021-96b2-1c31f2b831f9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:52.557527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1344d-fbb2-4021-96b2-1c31f2b831f9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/groups/194d7fcccb8d4f5abf13afc233ee43b7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:52.560175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc1344d-fbb2-4021-96b2-1c31f2b831f9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_group(group_id=194d7fcccb8d4f5abf13afc233ee43b7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:52.563223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc1344d-fbb2-4021-96b2-1c31f2b831f9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:52.680852 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3392/6867] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:13:52 2022] DELETE /identity/v3/groups/194d7fcccb8d4f5abf13afc233ee43b7 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:52.701540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68b21d-6aaf-4b40-bf5b-a38840d55f32 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:52.705770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ac68b21d-6aaf-4b40-bf5b-a38840d55f32 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:52.723043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68b21d-6aaf-4b40-bf5b-a38840d55f32 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:52.886153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:52.945306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:52.945306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:52.945306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/projects/926f43c2e60741b99282332544bb4d4d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:52.945306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_project(project_id=926f43c2e60741b99282332544bb4d4d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:52.945306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:53.036658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '926f43c2e60741b99282332544bb4d4d', 'request_id': 'req-df09488b-85cc-45a8-8b7e-a92b8ce098df'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.060657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '926f43c2e60741b99282332544bb4d4d', 'request_id': 'req-df09488b-85cc-45a8-8b7e-a92b8ce098df'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.079186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '926f43c2e60741b99282332544bb4d4d', 'request_id': 'req-df09488b-85cc-45a8-8b7e-a92b8ce098df'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.085875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:53.096083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] The token cache is being invalidate because project 926f43c2e60741b99282332544bb4d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:53.096652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.098010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.101641 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3476/6868] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:52 2022] DELETE /identity/v3/projects/926f43c2e60741b99282332544bb4d4d => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:53.106916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1344d-fbb2-4021-96b2-1c31f2b831f9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:53.109527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7cc1344d-fbb2-4021-96b2-1c31f2b831f9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:53.139055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1344d-fbb2-4021-96b2-1c31f2b831f9 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:53.232145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:53.233336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:53.234284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:53.234742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/69e8bdfecc4e401c92c144fed43e3a65` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:53.236133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=69e8bdfecc4e401c92c144fed43e3a65)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:53.238535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:53.273999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '69e8bdfecc4e401c92c144fed43e3a65'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.275115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '69e8bdfecc4e401c92c144fed43e3a65'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.332146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '69e8bdfecc4e401c92c144fed43e3a65'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.333095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '69e8bdfecc4e401c92c144fed43e3a65'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.350959 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3393/6869] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:13:53 2022] PATCH /identity/v3/domains/69e8bdfecc4e401c92c144fed43e3a65 => generated 278 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:53.354361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:53.369775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09488b-85cc-45a8-8b7e-a92b8ce098df tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:53.460029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:53.462293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:53.462591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:53.463489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/domains/69e8bdfecc4e401c92c144fed43e3a65` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:53.482424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=69e8bdfecc4e401c92c144fed43e3a65)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:53.484735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:53.518105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '69e8bdfecc4e401c92c144fed43e3a65'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.524262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '69e8bdfecc4e401c92c144fed43e3a65'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.635931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69e8bdfecc4e401c92c144fed43e3a65', 'request_id': 'req-6e164dcd-8856-494a-889c-a83f913c2d7d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.651166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69e8bdfecc4e401c92c144fed43e3a65', 'request_id': 'req-6e164dcd-8856-494a-889c-a83f913c2d7d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.663377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69e8bdfecc4e401c92c144fed43e3a65', 'request_id': 'req-6e164dcd-8856-494a-889c-a83f913c2d7d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.690103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] The token cache is being invalidate because project 69e8bdfecc4e401c92c144fed43e3a65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:53.690418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.691169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.721589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '69e8bdfecc4e401c92c144fed43e3a65', 'request_id': 'req-6e164dcd-8856-494a-889c-a83f913c2d7d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.732025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '69e8bdfecc4e401c92c144fed43e3a65', 'request_id': 'req-6e164dcd-8856-494a-889c-a83f913c2d7d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.733269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '69e8bdfecc4e401c92c144fed43e3a65', 'request_id': 'req-6e164dcd-8856-494a-889c-a83f913c2d7d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.750545 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3477/6870] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:13:53 2022] DELETE /identity/v3/domains/69e8bdfecc4e401c92c144fed43e3a65 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:53.754441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:53.756635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:53.781375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05225c-9e8b-4d81-9723-94800ce38b9f tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:53.837864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:53.856280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d9977-7a91-4f3b-995d-defbfc18d948 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:53.858042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d9977-7a91-4f3b-995d-defbfc18d948 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:53.858520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d9977-7a91-4f3b-995d-defbfc18d948 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:53.858876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d9977-7a91-4f3b-995d-defbfc18d948 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/23541c3d49744a43a459b173d65a78c1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:53.860436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d9977-7a91-4f3b-995d-defbfc18d948 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_role(role_id=23541c3d49744a43a459b173d65a78c1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:53.863227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d9977-7a91-4f3b-995d-defbfc18d948 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:53.914044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-189d9977-7a91-4f3b-995d-defbfc18d948 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 23541c3d49744a43a459b173d65a78c1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:53.914348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-189d9977-7a91-4f3b-995d-defbfc18d948 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.915109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-189d9977-7a91-4f3b-995d-defbfc18d948 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:53.917230 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3394/6871] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:53 2022] DELETE /identity/v3/roles/23541c3d49744a43a459b173d65a78c1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:53.921398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:53.929010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:53.971245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e164dcd-8856-494a-889c-a83f913c2d7d tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:54.068136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:54.111556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27bc570-607f-4143-92b2-214b267ee9cc tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:54.112222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bc570-607f-4143-92b2-214b267ee9cc tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:54.112509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bc570-607f-4143-92b2-214b267ee9cc tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:54.112790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bc570-607f-4143-92b2-214b267ee9cc tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/7159359626c74e5793a5237927e9933c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:54.120496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27bc570-607f-4143-92b2-214b267ee9cc tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_role(role_id=7159359626c74e5793a5237927e9933c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:54.123415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27bc570-607f-4143-92b2-214b267ee9cc tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:54.189967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c27bc570-607f-4143-92b2-214b267ee9cc tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 7159359626c74e5793a5237927e9933c has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:54.190309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c27bc570-607f-4143-92b2-214b267ee9cc tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:54.190982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c27bc570-607f-4143-92b2-214b267ee9cc tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:54.211046 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3478/6872] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:53 2022] DELETE /identity/v3/roles/7159359626c74e5793a5237927e9933c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:54.212383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d9977-7a91-4f3b-995d-defbfc18d948 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:54.213954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-189d9977-7a91-4f3b-995d-defbfc18d948 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:54.228297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d9977-7a91-4f3b-995d-defbfc18d948 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:54.388946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b75e3-4c27-4df8-9190-bf3b3a3db513 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bf3461f48be47f48e9b73e0d7edaba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:54.392875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b75e3-4c27-4df8-9190-bf3b3a3db513 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:54.394698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b75e3-4c27-4df8-9190-bf3b3a3db513 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:54.394698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b75e3-4c27-4df8-9190-bf3b3a3db513 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] PATH_INFO: `/v3/roles/94f4ea5bd8d54d608c134889670b1995` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:54.396310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b75e3-4c27-4df8-9190-bf3b3a3db513 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorizing `identity:delete_role(role_id=94f4ea5bd8d54d608c134889670b1995)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:54.398603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b75e3-4c27-4df8-9190-bf3b3a3db513 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:54.485800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-593b75e3-4c27-4df8-9190-bf3b3a3db513 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invalidating the token cache because role 94f4ea5bd8d54d608c134889670b1995 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:54.488865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-593b75e3-4c27-4df8-9190-bf3b3a3db513 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:54.491262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-593b75e3-4c27-4df8-9190-bf3b3a3db513 tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:54.494264 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3395/6873] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:54 2022] DELETE /identity/v3/roles/94f4ea5bd8d54d608c134889670b1995 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:54.499468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27bc570-607f-4143-92b2-214b267ee9cc tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:54.504375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c27bc570-607f-4143-92b2-214b267ee9cc tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:54.664240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27bc570-607f-4143-92b2-214b267ee9cc tempest-RolesV3TestJSON-1466368128 tempest-RolesV3TestJSON-1466368128-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:54.806828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:54.807515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:54.807810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:54.808093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] PATH_INFO: `/v3/users/afee6e474cb6418aad4b56ed919166c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:54.811400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=afee6e474cb6418aad4b56ed919166c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:54.813491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:54.944029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afee6e474cb6418aad4b56ed919166c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:54.945240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:54.962802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afee6e474cb6418aad4b56ed919166c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:54.978407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afee6e474cb6418aad4b56ed919166c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:54.979259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afee6e474cb6418aad4b56ed919166c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:54.979904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afee6e474cb6418aad4b56ed919166c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:55.003806 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3479/6874] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:54 2022] DELETE /identity/v3/users/afee6e474cb6418aad4b56ed919166c4 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:55.008055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed82b6-1809-48bd-86bd-fbddf74dbbbe None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:55.008378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed82b6-1809-48bd-86bd-fbddf74dbbbe None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:55.008658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed82b6-1809-48bd-86bd-fbddf74dbbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:55.041922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-67ed82b6-1809-48bd-86bd-fbddf74dbbbe None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:55.047311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-67ed82b6-1809-48bd-86bd-fbddf74dbbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:55.186904 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3396/6875] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:55.218847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:55.220977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:55.311522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61e425-61dd-4a37-a88f-5168eb8a51ea admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:55.317001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbd889-fa3c-47c2-80f6-550fa2cbbc68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:55.318040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbd889-fa3c-47c2-80f6-550fa2cbbc68 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:55.318405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbd889-fa3c-47c2-80f6-550fa2cbbc68 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:55.318756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbd889-fa3c-47c2-80f6-550fa2cbbc68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:55.336690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbd889-fa3c-47c2-80f6-550fa2cbbc68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:55.340663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbd889-fa3c-47c2-80f6-550fa2cbbc68 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:55.357148 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3480/6876] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:13:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:13:55.810844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed82b6-1809-48bd-86bd-fbddf74dbbbe None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:55.812654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-67ed82b6-1809-48bd-86bd-fbddf74dbbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:55.829079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed82b6-1809-48bd-86bd-fbddf74dbbbe None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:55.834385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:55.835254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:55.835540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:55.835879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] PATH_INFO: `/v3/projects/80b48fe72d5144beaba0ac195a3914d3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:55.837252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b48fe72d5144beaba0ac195a3914d3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:55.839358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:55.857352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:55.892295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b48fe72d5144beaba0ac195a3914d3', 'request_id': 'req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:55.900991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b48fe72d5144beaba0ac195a3914d3', 'request_id': 'req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:55.908530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b48fe72d5144beaba0ac195a3914d3', 'request_id': 'req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:55.926377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] The token cache is being invalidate because project 80b48fe72d5144beaba0ac195a3914d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:55.926677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:55.927360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:55.928868 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3397/6877] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:55 2022] DELETE /identity/v3/projects/80b48fe72d5144beaba0ac195a3914d3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:55.933132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbd889-fa3c-47c2-80f6-550fa2cbbc68 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:55.946364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbd889-fa3c-47c2-80f6-550fa2cbbc68 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:56.027087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0a452-c573-4520-8fa5-08321cfab6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:56.028279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0a452-c573-4520-8fa5-08321cfab6ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:56.028570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0a452-c573-4520-8fa5-08321cfab6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:56.028844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0a452-c573-4520-8fa5-08321cfab6ef admin admin] PATH_INFO: `/v3/users/7bf3461f48be47f48e9b73e0d7edaba4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:56.030039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0a452-c573-4520-8fa5-08321cfab6ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf3461f48be47f48e9b73e0d7edaba4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:56.032005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0a452-c573-4520-8fa5-08321cfab6ef admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:56.102324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3a0a452-c573-4520-8fa5-08321cfab6ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf3461f48be47f48e9b73e0d7edaba4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:56.116986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3a0a452-c573-4520-8fa5-08321cfab6ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf3461f48be47f48e9b73e0d7edaba4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:56.121541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:56.127238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3a0a452-c573-4520-8fa5-08321cfab6ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf3461f48be47f48e9b73e0d7edaba4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:56.129287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3a0a452-c573-4520-8fa5-08321cfab6ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf3461f48be47f48e9b73e0d7edaba4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:56.131305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3a0a452-c573-4520-8fa5-08321cfab6ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf3461f48be47f48e9b73e0d7edaba4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:56.145701 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3481/6878] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:55 2022] DELETE /identity/v3/users/7bf3461f48be47f48e9b73e0d7edaba4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:56.431158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:56.432682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:56.452795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1d70b-2f93-4203-b3ce-5ab9f57be6c3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:56.518877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb3ade-02f4-41a9-9fea-5548c9e9493a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:56.519483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb3ade-02f4-41a9-9fea-5548c9e9493a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:56.519798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb3ade-02f4-41a9-9fea-5548c9e9493a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:56.520075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb3ade-02f4-41a9-9fea-5548c9e9493a admin admin] PATH_INFO: `/v3/projects/49c8bbdbd1054ea2be53eb1c38eeeb78` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:56.521285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb3ade-02f4-41a9-9fea-5548c9e9493a admin admin] RBAC: Authorizing `identity:delete_project(project_id=49c8bbdbd1054ea2be53eb1c38eeeb78)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:56.523322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb3ade-02f4-41a9-9fea-5548c9e9493a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:56.574241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cacb3ade-02f4-41a9-9fea-5548c9e9493a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'request_id': 'req-cacb3ade-02f4-41a9-9fea-5548c9e9493a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:56.582877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cacb3ade-02f4-41a9-9fea-5548c9e9493a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'request_id': 'req-cacb3ade-02f4-41a9-9fea-5548c9e9493a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:56.590237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cacb3ade-02f4-41a9-9fea-5548c9e9493a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49c8bbdbd1054ea2be53eb1c38eeeb78', 'request_id': 'req-cacb3ade-02f4-41a9-9fea-5548c9e9493a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:56.602536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cacb3ade-02f4-41a9-9fea-5548c9e9493a admin admin] The token cache is being invalidate because project 49c8bbdbd1054ea2be53eb1c38eeeb78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:13:56.603076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cacb3ade-02f4-41a9-9fea-5548c9e9493a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:56.603863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cacb3ade-02f4-41a9-9fea-5548c9e9493a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:56.605606 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3398/6879] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:13:56 2022] DELETE /identity/v3/projects/49c8bbdbd1054ea2be53eb1c38eeeb78 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:56.610550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba337dd2-f2a0-486c-ac41-537b96dbfa2e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:56.610842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba337dd2-f2a0-486c-ac41-537b96dbfa2e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:56.611122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba337dd2-f2a0-486c-ac41-537b96dbfa2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:56.630946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ba337dd2-f2a0-486c-ac41-537b96dbfa2e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:56.632191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ba337dd2-f2a0-486c-ac41-537b96dbfa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:56.695690 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3482/6880] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:13:56.704051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2296c-0f9c-4f14-a0c9-c5f9418e7958 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:56.704347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2296c-0f9c-4f14-a0c9-c5f9418e7958 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:56.704657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2296c-0f9c-4f14-a0c9-c5f9418e7958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:56.727809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-33b2296c-0f9c-4f14-a0c9-c5f9418e7958 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:56.729376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-33b2296c-0f9c-4f14-a0c9-c5f9418e7958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:56.743329 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3399/6881] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:13:56.743697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba337dd2-f2a0-486c-ac41-537b96dbfa2e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:56.754769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba337dd2-f2a0-486c-ac41-537b96dbfa2e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:56.761917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785685c0-ffc9-4b7b-b117-872add2712fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:56.763757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785685c0-ffc9-4b7b-b117-872add2712fd admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:56.764153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785685c0-ffc9-4b7b-b117-872add2712fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:56.764522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785685c0-ffc9-4b7b-b117-872add2712fd admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:56.765510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785685c0-ffc9-4b7b-b117-872add2712fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:56.765920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785685c0-ffc9-4b7b-b117-872add2712fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:56.767564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785685c0-ffc9-4b7b-b117-872add2712fd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:56.787652 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3483/6882] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:13:56.788778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2296c-0f9c-4f14-a0c9-c5f9418e7958 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:56.810073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2296c-0f9c-4f14-a0c9-c5f9418e7958 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:56.820673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d32771-244a-44f2-a446-e9e97419eae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:56.822677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d32771-244a-44f2-a446-e9e97419eae7 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:56.823202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d32771-244a-44f2-a446-e9e97419eae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:56.823646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d32771-244a-44f2-a446-e9e97419eae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:56.825526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d32771-244a-44f2-a446-e9e97419eae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:56.827540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d32771-244a-44f2-a446-e9e97419eae7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:56.862725 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3400/6883] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:13:56 2022] POST /identity/v3/projects => generated 393 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:56.866630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785685c0-ffc9-4b7b-b117-872add2712fd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:56.888815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785685c0-ffc9-4b7b-b117-872add2712fd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:56.975761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307b7b6-f9a1-44eb-bcb7-34b3eb7396b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:56.976398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307b7b6-f9a1-44eb-bcb7-34b3eb7396b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:56.976694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307b7b6-f9a1-44eb-bcb7-34b3eb7396b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:56.977004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307b7b6-f9a1-44eb-bcb7-34b3eb7396b0 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:56.977638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307b7b6-f9a1-44eb-bcb7-34b3eb7396b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:56.980285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307b7b6-f9a1-44eb-bcb7-34b3eb7396b0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:57.017684 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3484/6884] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:13:56 2022] POST /identity/v3/users => generated 297 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:57.021709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d32771-244a-44f2-a446-e9e97419eae7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:57.040662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:57.047853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d32771-244a-44f2-a446-e9e97419eae7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:57.058717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5caf9-6ccb-41eb-bdaf-4f109e0d82ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:57.058717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5caf9-6ccb-41eb-bdaf-4f109e0d82ea admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:57.059009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5caf9-6ccb-41eb-bdaf-4f109e0d82ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:57.059247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5caf9-6ccb-41eb-bdaf-4f109e0d82ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:57.059847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5caf9-6ccb-41eb-bdaf-4f109e0d82ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:57.061587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5caf9-6ccb-41eb-bdaf-4f109e0d82ea admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:57.074350 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3401/6885] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:13:57.078152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307b7b6-f9a1-44eb-bcb7-34b3eb7396b0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:57.092612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307b7b6-f9a1-44eb-bcb7-34b3eb7396b0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:57.098368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0a91c-c3a0-41e2-b393-dfcb5b1384e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:57.099344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0a91c-c3a0-41e2-b393-dfcb5b1384e3 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:57.099732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0a91c-c3a0-41e2-b393-dfcb5b1384e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:57.100071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0a91c-c3a0-41e2-b393-dfcb5b1384e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:57.100766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0a91c-c3a0-41e2-b393-dfcb5b1384e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:57.102611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0a91c-c3a0-41e2-b393-dfcb5b1384e3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:57.115346 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3485/6886] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:13:57.119622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5caf9-6ccb-41eb-bdaf-4f109e0d82ea admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:57.133326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5caf9-6ccb-41eb-bdaf-4f109e0d82ea admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:57.139871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a544743-0ebc-46bf-afac-45ff90bb48f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:57.140998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a544743-0ebc-46bf-afac-45ff90bb48f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:57.140998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a544743-0ebc-46bf-afac-45ff90bb48f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:57.141170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a544743-0ebc-46bf-afac-45ff90bb48f8 admin admin] PATH_INFO: `/v3/projects/a4653fbe71fc4649bb1d25794e6d281a/users/2cc634a3519744e2b0a032c2d7c8ff2a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:57.157915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a544743-0ebc-46bf-afac-45ff90bb48f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4653fbe71fc4649bb1d25794e6d281a, user_id=2cc634a3519744e2b0a032c2d7c8ff2a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:57.160610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a544743-0ebc-46bf-afac-45ff90bb48f8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:57.192994 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3402/6887] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:13:57 2022] PUT /identity/v3/projects/a4653fbe71fc4649bb1d25794e6d281a/users/2cc634a3519744e2b0a032c2d7c8ff2a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:57.197529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3125eb-95b1-4827-82d7-5bdba2b2ba83 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:57.198042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3125eb-95b1-4827-82d7-5bdba2b2ba83 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:57.198400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3125eb-95b1-4827-82d7-5bdba2b2ba83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:57.231720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cb3125eb-95b1-4827-82d7-5bdba2b2ba83 None None] MFA Rules not processed for user `2cc634a3519744e2b0a032c2d7c8ff2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:57.233153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cb3125eb-95b1-4827-82d7-5bdba2b2ba83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:57.284269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:57.319062 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3486/6888] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:13:57.323795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a544743-0ebc-46bf-afac-45ff90bb48f8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:57.339672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a544743-0ebc-46bf-afac-45ff90bb48f8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:57.346003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb011416-8025-4977-a8af-0e0650d56b93 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:57.346929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb011416-8025-4977-a8af-0e0650d56b93 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:57.347480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb011416-8025-4977-a8af-0e0650d56b93 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:57.347833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb011416-8025-4977-a8af-0e0650d56b93 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:57.348627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb011416-8025-4977-a8af-0e0650d56b93 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:create_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:57.350169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb011416-8025-4977-a8af-0e0650d56b93 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:57.447382 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3403/6889] 10.222.0.16 () {60 vars in 1314 bytes} [Sat Sep 3 21:13:57 2022] POST /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials => generated 788 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:13:57.448795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3125eb-95b1-4827-82d7-5bdba2b2ba83 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:57.460787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3125eb-95b1-4827-82d7-5bdba2b2ba83 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:57.467381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e639d3e-85bf-4b2a-90f8-7740b0a691df tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:57.469683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e639d3e-85bf-4b2a-90f8-7740b0a691df tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:57.470166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e639d3e-85bf-4b2a-90f8-7740b0a691df tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:57.470534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e639d3e-85bf-4b2a-90f8-7740b0a691df tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/7d040d824da74e50855eed0c6dbd511c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:57.492379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e639d3e-85bf-4b2a-90f8-7740b0a691df tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:get_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a, application_credential_id=7d040d824da74e50855eed0c6dbd511c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:57.494207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e639d3e-85bf-4b2a-90f8-7740b0a691df tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:57.499270 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3487/6890] 10.222.0.16 () {58 vars in 1392 bytes} [Sat Sep 3 21:13:57 2022] GET /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/7d040d824da74e50855eed0c6dbd511c => generated 688 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:13:57.501135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088248d2-2a07-4814-9e12-10bf1f93242b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:57.501533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088248d2-2a07-4814-9e12-10bf1f93242b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:57.501911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088248d2-2a07-4814-9e12-10bf1f93242b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:57.525947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-088248d2-2a07-4814-9e12-10bf1f93242b None None] MFA Rules not processed for user `2cc634a3519744e2b0a032c2d7c8ff2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:57.528560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-088248d2-2a07-4814-9e12-10bf1f93242b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:57.643535 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3404/6891] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3426 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 03 21:13:57.647795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e639d3e-85bf-4b2a-90f8-7740b0a691df tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:57.660227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e639d3e-85bf-4b2a-90f8-7740b0a691df tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:57.681697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405753e-fa8e-42e8-b752-b0a7f601fa67 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:57.683584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405753e-fa8e-42e8-b752-b0a7f601fa67 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:57.684588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405753e-fa8e-42e8-b752-b0a7f601fa67 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:57.684879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405753e-fa8e-42e8-b752-b0a7f601fa67 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/7d040d824da74e50855eed0c6dbd511c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:57.687340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405753e-fa8e-42e8-b752-b0a7f601fa67 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a, application_credential_id=7d040d824da74e50855eed0c6dbd511c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:57.689142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405753e-fa8e-42e8-b752-b0a7f601fa67 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:57.721604 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3488/6892] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Sep 3 21:13:57 2022] DELETE /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/7d040d824da74e50855eed0c6dbd511c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:57.729753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088248d2-2a07-4814-9e12-10bf1f93242b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:57.751378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088248d2-2a07-4814-9e12-10bf1f93242b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:57.756439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545bfeb-8acd-41c8-985d-3cfaa96e5efc tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:57.757452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545bfeb-8acd-41c8-985d-3cfaa96e5efc tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:57.757883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545bfeb-8acd-41c8-985d-3cfaa96e5efc tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:57.758305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545bfeb-8acd-41c8-985d-3cfaa96e5efc tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:57.759167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545bfeb-8acd-41c8-985d-3cfaa96e5efc tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:create_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:57.760844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545bfeb-8acd-41c8-985d-3cfaa96e5efc tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:57.783272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:57.871874 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3405/6893] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Sep 3 21:13:57 2022] POST /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials => generated 920 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:57.876474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405753e-fa8e-42e8-b752-b0a7f601fa67 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:57.901223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405753e-fa8e-42e8-b752-b0a7f601fa67 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:57.916487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940a1fa-fcd4-428b-aada-d5ea10d91d31 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:57.917440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940a1fa-fcd4-428b-aada-d5ea10d91d31 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:57.917952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940a1fa-fcd4-428b-aada-d5ea10d91d31 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:57.918475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940a1fa-fcd4-428b-aada-d5ea10d91d31 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/1daa2d3d4e52487b8c6eb215561eb0ea` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:57.947907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940a1fa-fcd4-428b-aada-d5ea10d91d31 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a, application_credential_id=1daa2d3d4e52487b8c6eb215561eb0ea)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:57.950186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940a1fa-fcd4-428b-aada-d5ea10d91d31 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:57.992160 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3489/6894] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Sep 3 21:13:57 2022] DELETE /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/1daa2d3d4e52487b8c6eb215561eb0ea => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:57.998484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545bfeb-8acd-41c8-985d-3cfaa96e5efc tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:58.015413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545bfeb-8acd-41c8-985d-3cfaa96e5efc tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:58.024929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b81dcd-98e4-47bb-b700-0f0b30e50c9f tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:58.026035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b81dcd-98e4-47bb-b700-0f0b30e50c9f tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:58.026427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b81dcd-98e4-47bb-b700-0f0b30e50c9f tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:58.026818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b81dcd-98e4-47bb-b700-0f0b30e50c9f tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:58.028580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b81dcd-98e4-47bb-b700-0f0b30e50c9f tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:create_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:58.029799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b81dcd-98e4-47bb-b700-0f0b30e50c9f tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:58.168250 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3406/6895] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Sep 3 21:13:57 2022] POST /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials => generated 813 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:58.172256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940a1fa-fcd4-428b-aada-d5ea10d91d31 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:58.183641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940a1fa-fcd4-428b-aada-d5ea10d91d31 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:58.188705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfea751-9022-45de-9d28-e0f452887055 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:58.189533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfea751-9022-45de-9d28-e0f452887055 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:58.189933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfea751-9022-45de-9d28-e0f452887055 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:58.190721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfea751-9022-45de-9d28-e0f452887055 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/c0209287eceb44d8820bc6aa3793eaf2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:58.206013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfea751-9022-45de-9d28-e0f452887055 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a, application_credential_id=c0209287eceb44d8820bc6aa3793eaf2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:58.208779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfea751-9022-45de-9d28-e0f452887055 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:58.254690 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3490/6896] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Sep 3 21:13:58 2022] DELETE /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/c0209287eceb44d8820bc6aa3793eaf2 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:58.260494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b81dcd-98e4-47bb-b700-0f0b30e50c9f tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:58.271941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b81dcd-98e4-47bb-b700-0f0b30e50c9f tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:58.276708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a259517-7be4-4185-87b7-96fde3767217 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:58.278058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a259517-7be4-4185-87b7-96fde3767217 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:58.278493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a259517-7be4-4185-87b7-96fde3767217 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:58.278786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a259517-7be4-4185-87b7-96fde3767217 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:58.280138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a259517-7be4-4185-87b7-96fde3767217 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:create_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:58.281376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a259517-7be4-4185-87b7-96fde3767217 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:58.361932 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3407/6897] 10.222.0.16 () {60 vars in 1314 bytes} [Sat Sep 3 21:13:58 2022] POST /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials => generated 789 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:58.365255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfea751-9022-45de-9d28-e0f452887055 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:58.384034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfea751-9022-45de-9d28-e0f452887055 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:58.390652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beefb45-3660-4d87-b652-fbdcb5d69fee tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:58.391705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beefb45-3660-4d87-b652-fbdcb5d69fee tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:58.392036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beefb45-3660-4d87-b652-fbdcb5d69fee tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:58.392294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beefb45-3660-4d87-b652-fbdcb5d69fee tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:58.394541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beefb45-3660-4d87-b652-fbdcb5d69fee tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:create_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:58.396503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beefb45-3660-4d87-b652-fbdcb5d69fee tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:58.501260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a259517-7be4-4185-87b7-96fde3767217 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:58.501391 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3491/6898] 10.222.0.16 () {60 vars in 1314 bytes} [Sat Sep 3 21:13:58 2022] POST /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials => generated 787 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:58.515733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a259517-7be4-4185-87b7-96fde3767217 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:58.521298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6e37d-8d91-4570-81f9-aa0feb4448a4 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:58.521540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6e37d-8d91-4570-81f9-aa0feb4448a4 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:58.521818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6e37d-8d91-4570-81f9-aa0feb4448a4 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:58.522153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6e37d-8d91-4570-81f9-aa0feb4448a4 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:58.522795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6e37d-8d91-4570-81f9-aa0feb4448a4 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:list_application_credentials(user_id=2cc634a3519744e2b0a032c2d7c8ff2a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:58.524786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6e37d-8d91-4570-81f9-aa0feb4448a4 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:58.538624 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3408/6899] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Sep 3 21:13:58 2022] GET /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials => generated 1500 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:13:58.542996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beefb45-3660-4d87-b652-fbdcb5d69fee tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:58.556549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beefb45-3660-4d87-b652-fbdcb5d69fee tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:58.563008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787c05-762e-4ada-82d3-e9f20bfaad49 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:58.564220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787c05-762e-4ada-82d3-e9f20bfaad49 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:58.564513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787c05-762e-4ada-82d3-e9f20bfaad49 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:58.564800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787c05-762e-4ada-82d3-e9f20bfaad49 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/080e3c0912224031a05f733449f0ebda` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:58.585782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51787c05-762e-4ada-82d3-e9f20bfaad49 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a, application_credential_id=080e3c0912224031a05f733449f0ebda)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:58.587763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51787c05-762e-4ada-82d3-e9f20bfaad49 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:58.624618 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3492/6900] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Sep 3 21:13:58 2022] DELETE /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/080e3c0912224031a05f733449f0ebda => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:13:58.626215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6e37d-8d91-4570-81f9-aa0feb4448a4 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:58.640587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6e37d-8d91-4570-81f9-aa0feb4448a4 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:58.646561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53af2f7-3911-4f5f-ab9a-c29b4a8c4c53 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:58.647864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53af2f7-3911-4f5f-ab9a-c29b4a8c4c53 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:58.648292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53af2f7-3911-4f5f-ab9a-c29b4a8c4c53 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:58.648648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53af2f7-3911-4f5f-ab9a-c29b4a8c4c53 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/4c0d8d2ba8db49bfbb376151cc7fd6c7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:58.662536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53af2f7-3911-4f5f-ab9a-c29b4a8c4c53 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a, application_credential_id=4c0d8d2ba8db49bfbb376151cc7fd6c7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:58.664266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53af2f7-3911-4f5f-ab9a-c29b4a8c4c53 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:58.673959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:58.702034 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3409/6901] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Sep 3 21:13:58 2022] DELETE /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/4c0d8d2ba8db49bfbb376151cc7fd6c7 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:58.703256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787c05-762e-4ada-82d3-e9f20bfaad49 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:58.722754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787c05-762e-4ada-82d3-e9f20bfaad49 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:58.736536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c5c36-aa81-492f-bbc5-323845c4c0cb tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:58.737636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c5c36-aa81-492f-bbc5-323845c4c0cb tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:58.738295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c5c36-aa81-492f-bbc5-323845c4c0cb tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:58.738799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c5c36-aa81-492f-bbc5-323845c4c0cb tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:58.739684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c5c36-aa81-492f-bbc5-323845c4c0cb tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:create_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:58.741783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c5c36-aa81-492f-bbc5-323845c4c0cb tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:58.751355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:58.906078 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3493/6902] 10.222.0.16 () {60 vars in 1314 bytes} [Sat Sep 3 21:13:58 2022] POST /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials => generated 789 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:58.910619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53af2f7-3911-4f5f-ab9a-c29b4a8c4c53 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:58.921399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53af2f7-3911-4f5f-ab9a-c29b4a8c4c53 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:58.926046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50c90d-d60f-4954-9424-8f3b268c85f8 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:58.926696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50c90d-d60f-4954-9424-8f3b268c85f8 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:58.927006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50c90d-d60f-4954-9424-8f3b268c85f8 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:58.927338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50c90d-d60f-4954-9424-8f3b268c85f8 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:58.928066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50c90d-d60f-4954-9424-8f3b268c85f8 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:create_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:58.929278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50c90d-d60f-4954-9424-8f3b268c85f8 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:59.012459 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3410/6903] 10.222.0.16 () {60 vars in 1314 bytes} [Sat Sep 3 21:13:58 2022] POST /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials => generated 788 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:13:59.017561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c5c36-aa81-492f-bbc5-323845c4c0cb tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:59.028382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c5c36-aa81-492f-bbc5-323845c4c0cb tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:59.034037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab79b74-ae2f-4b49-9c5b-7fce5131b1d7 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:59.034780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab79b74-ae2f-4b49-9c5b-7fce5131b1d7 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:59.035139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab79b74-ae2f-4b49-9c5b-7fce5131b1d7 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:59.035492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab79b74-ae2f-4b49-9c5b-7fce5131b1d7 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:59.036238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab79b74-ae2f-4b49-9c5b-7fce5131b1d7 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Adding query filter params (name=tempest-application_credential-354122591) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:13:59.036630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab79b74-ae2f-4b49-9c5b-7fce5131b1d7 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:list_application_credentials(user_id=2cc634a3519744e2b0a032c2d7c8ff2a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:59.039108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab79b74-ae2f-4b49-9c5b-7fce5131b1d7 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:59.051583 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3494/6904] 10.222.0.16 () {58 vars in 1384 bytes} [Sat Sep 3 21:13:59 2022] GET /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials?name=tempest-application_credential-354122591 => generated 885 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:13:59.055809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50c90d-d60f-4954-9424-8f3b268c85f8 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:59.067600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50c90d-d60f-4954-9424-8f3b268c85f8 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:59.072122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868fcf1-0f33-46a6-a7ef-80e6fe0a2c1b tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:59.072858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868fcf1-0f33-46a6-a7ef-80e6fe0a2c1b tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:59.073263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868fcf1-0f33-46a6-a7ef-80e6fe0a2c1b tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:59.073623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868fcf1-0f33-46a6-a7ef-80e6fe0a2c1b tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/8e914f47806841dc98a49ea3c233923d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:59.084176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868fcf1-0f33-46a6-a7ef-80e6fe0a2c1b tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a, application_credential_id=8e914f47806841dc98a49ea3c233923d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:59.086179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868fcf1-0f33-46a6-a7ef-80e6fe0a2c1b tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:59.115066 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3411/6905] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Sep 3 21:13:59 2022] DELETE /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/8e914f47806841dc98a49ea3c233923d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:59.119086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab79b74-ae2f-4b49-9c5b-7fce5131b1d7 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:59.133192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab79b74-ae2f-4b49-9c5b-7fce5131b1d7 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:59.134400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a3a14-c78f-44a6-b3b3-fd96cf173e94 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc634a3519744e2b0a032c2d7c8ff2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4653fbe71fc4649bb1d25794e6d281a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:59.135330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a3a14-c78f-44a6-b3b3-fd96cf173e94 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:59.135330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a3a14-c78f-44a6-b3b3-fd96cf173e94 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:59.135494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a3a14-c78f-44a6-b3b3-fd96cf173e94 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/901a44c9e0314abb815b4c9f325326e4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:59.146155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a3a14-c78f-44a6-b3b3-fd96cf173e94 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2cc634a3519744e2b0a032c2d7c8ff2a, application_credential_id=901a44c9e0314abb815b4c9f325326e4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:59.147692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a3a14-c78f-44a6-b3b3-fd96cf173e94 tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:59.185150 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3495/6906] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Sep 3 21:13:59 2022] DELETE /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a/application_credentials/901a44c9e0314abb815b4c9f325326e4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:59.185344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868fcf1-0f33-46a6-a7ef-80e6fe0a2c1b tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:59.267979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868fcf1-0f33-46a6-a7ef-80e6fe0a2c1b tempest-ApplicationCredentialsV3Test-1960887933 tempest-ApplicationCredentialsV3Test-1960887933-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:59.275699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:59.276098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:59.276481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:59.276919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] PATH_INFO: `/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:59.278895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cc634a3519744e2b0a032c2d7c8ff2a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:59.281667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:59.366045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cc634a3519744e2b0a032c2d7c8ff2a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:59.377715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:13:59.406507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cc634a3519744e2b0a032c2d7c8ff2a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:59.408192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cc634a3519744e2b0a032c2d7c8ff2a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:59.418852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cc634a3519744e2b0a032c2d7c8ff2a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:59.433047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cc634a3519744e2b0a032c2d7c8ff2a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:13:59.442058 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3412/6907] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:13:59 2022] DELETE /identity/v3/users/2cc634a3519744e2b0a032c2d7c8ff2a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:13:59.446592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcbaa7-585d-4c95-9bd8-817a14cb82a0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:59.446884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcbaa7-585d-4c95-9bd8-817a14cb82a0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:59.447386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcbaa7-585d-4c95-9bd8-817a14cb82a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:59.484666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6fbcbaa7-585d-4c95-9bd8-817a14cb82a0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:13:59.489579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6fbcbaa7-585d-4c95-9bd8-817a14cb82a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:59.568944 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3496/6908] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:13:59.587762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:13:59.589476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:13:59.664368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125ec32-84aa-4161-8a27-9bbc461c4afa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:13:59.667826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c61ab-0a3f-4642-aa2d-f42457b158db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:13:59.668506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c61ab-0a3f-4642-aa2d-f42457b158db service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:13:59.669337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c61ab-0a3f-4642-aa2d-f42457b158db service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:13:59.669640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c61ab-0a3f-4642-aa2d-f42457b158db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:13:59.679885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c61ab-0a3f-4642-aa2d-f42457b158db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:13:59.681379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c61ab-0a3f-4642-aa2d-f42457b158db service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:13:59.693159 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3413/6909] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:14:00.119272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcbaa7-585d-4c95-9bd8-817a14cb82a0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:00.122644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6fbcbaa7-585d-4c95-9bd8-817a14cb82a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:00.142156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcbaa7-585d-4c95-9bd8-817a14cb82a0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:00.152784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:00.154730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:00.155255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:00.155694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8 admin admin] PATH_INFO: `/v3/projects/a4653fbe71fc4649bb1d25794e6d281a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:00.157335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4653fbe71fc4649bb1d25794e6d281a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:00.160835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:00.218441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4653fbe71fc4649bb1d25794e6d281a', 'request_id': 'req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:00.222487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:00.230390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4653fbe71fc4649bb1d25794e6d281a', 'request_id': 'req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:00.239766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4653fbe71fc4649bb1d25794e6d281a', 'request_id': 'req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:00.255319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8 admin admin] The token cache is being invalidate because project a4653fbe71fc4649bb1d25794e6d281a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:14:00.255623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:00.256300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a3fed34f-eeaa-4a80-842a-6bd5b8d5e4e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:00.263541 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3497/6910] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:14:00 2022] DELETE /identity/v3/projects/a4653fbe71fc4649bb1d25794e6d281a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:00.265368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32e69e-cee2-418a-8824-e0a0f0c6ee34 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:00.265814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32e69e-cee2-418a-8824-e0a0f0c6ee34 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:00.265952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32e69e-cee2-418a-8824-e0a0f0c6ee34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:00.288496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7b32e69e-cee2-418a-8824-e0a0f0c6ee34 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:00.290671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7b32e69e-cee2-418a-8824-e0a0f0c6ee34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:00.361999 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3414/6911] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:00.369937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ac6cb-fa43-451e-a9de-e843309aacb7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:00.370196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ac6cb-fa43-451e-a9de-e843309aacb7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:00.370441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ac6cb-fa43-451e-a9de-e843309aacb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:00.392623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cf3ac6cb-fa43-451e-a9de-e843309aacb7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:00.394373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cf3ac6cb-fa43-451e-a9de-e843309aacb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:00.402934 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3498/6912] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:00.407506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32e69e-cee2-418a-8824-e0a0f0c6ee34 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:00.420073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32e69e-cee2-418a-8824-e0a0f0c6ee34 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:00.424890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551808b-cf9c-4dca-90ce-028e54f859d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:00.425552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551808b-cf9c-4dca-90ce-028e54f859d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:00.425844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551808b-cf9c-4dca-90ce-028e54f859d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:00.426123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551808b-cf9c-4dca-90ce-028e54f859d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:00.426811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551808b-cf9c-4dca-90ce-028e54f859d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:14:00.427098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551808b-cf9c-4dca-90ce-028e54f859d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:00.428758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551808b-cf9c-4dca-90ce-028e54f859d0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:00.446334 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3415/6913] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:14:00.447790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ac6cb-fa43-451e-a9de-e843309aacb7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:00.458722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ac6cb-fa43-451e-a9de-e843309aacb7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:00.464094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6cb2a-3094-4c60-ba94-de97f7575f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:00.464714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6cb2a-3094-4c60-ba94-de97f7575f8c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:00.465038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6cb2a-3094-4c60-ba94-de97f7575f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:00.465364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6cb2a-3094-4c60-ba94-de97f7575f8c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:00.466011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6cb2a-3094-4c60-ba94-de97f7575f8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:00.467955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6cb2a-3094-4c60-ba94-de97f7575f8c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:00.517539 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3499/6914] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:14:00 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:14:00.519168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551808b-cf9c-4dca-90ce-028e54f859d0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:00.532520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551808b-cf9c-4dca-90ce-028e54f859d0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:00.613719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa7507-05bb-473b-a9dd-efffda872a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:00.614352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa7507-05bb-473b-a9dd-efffda872a29 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:00.614646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa7507-05bb-473b-a9dd-efffda872a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:00.615462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa7507-05bb-473b-a9dd-efffda872a29 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:00.615568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa7507-05bb-473b-a9dd-efffda872a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:00.617915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa7507-05bb-473b-a9dd-efffda872a29 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:00.627774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:00.655725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6cb2a-3094-4c60-ba94-de97f7575f8c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:00.665895 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3416/6915] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:14:00 2022] POST /identity/v3/users => generated 286 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:14:00.666941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6cb2a-3094-4c60-ba94-de97f7575f8c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:00.674448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa2281-607c-4339-83b7-f89e55d55783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:00.675170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa2281-607c-4339-83b7-f89e55d55783 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:00.675910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa2281-607c-4339-83b7-f89e55d55783 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:00.675910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa2281-607c-4339-83b7-f89e55d55783 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:00.676530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa2281-607c-4339-83b7-f89e55d55783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:00.678291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa2281-607c-4339-83b7-f89e55d55783 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:00.692400 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3500/6916] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:14:00.694400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa7507-05bb-473b-a9dd-efffda872a29 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:00.705725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa7507-05bb-473b-a9dd-efffda872a29 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:00.711300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c978df-a4af-4e73-9a2f-c1a9199e2a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:00.711924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c978df-a4af-4e73-9a2f-c1a9199e2a02 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:00.712521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c978df-a4af-4e73-9a2f-c1a9199e2a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:00.712521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c978df-a4af-4e73-9a2f-c1a9199e2a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:00.713650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c978df-a4af-4e73-9a2f-c1a9199e2a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:00.716197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c978df-a4af-4e73-9a2f-c1a9199e2a02 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:00.733067 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3417/6917] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:14:00.736170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa2281-607c-4339-83b7-f89e55d55783 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:00.748447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa2281-607c-4339-83b7-f89e55d55783 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:00.755050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c66d76-a2f4-4b78-a5d8-0169d3235531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:00.755820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c66d76-a2f4-4b78-a5d8-0169d3235531 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:00.756276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c66d76-a2f4-4b78-a5d8-0169d3235531 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:00.756525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c66d76-a2f4-4b78-a5d8-0169d3235531 admin admin] PATH_INFO: `/v3/projects/055069b8643c4ebb9ad1b0d08cdcdfe3/users/ed6fcb37d5db4e06889dfbbb5d458596/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:00.785327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c66d76-a2f4-4b78-a5d8-0169d3235531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=055069b8643c4ebb9ad1b0d08cdcdfe3, user_id=ed6fcb37d5db4e06889dfbbb5d458596, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:00.789721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c66d76-a2f4-4b78-a5d8-0169d3235531 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:00.817213 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3501/6918] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:14:00 2022] PUT /identity/v3/projects/055069b8643c4ebb9ad1b0d08cdcdfe3/users/ed6fcb37d5db4e06889dfbbb5d458596/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:00.820026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b264d-c7cf-4d22-bf0a-b90a14c86e85 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:00.820989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b264d-c7cf-4d22-bf0a-b90a14c86e85 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:00.821710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b264d-c7cf-4d22-bf0a-b90a14c86e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:00.866305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3a1b264d-c7cf-4d22-bf0a-b90a14c86e85 None None] MFA Rules not processed for user `ed6fcb37d5db4e06889dfbbb5d458596`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:00.867699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3a1b264d-c7cf-4d22-bf0a-b90a14c86e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:00.950258 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3418/6919] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:14:00.968018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c66d76-a2f4-4b78-a5d8-0169d3235531 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:00.970531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-26c66d76-a2f4-4b78-a5d8-0169d3235531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:01.044978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c66d76-a2f4-4b78-a5d8-0169d3235531 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:01.050915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c9d43-4c8f-478b-886e-7351908ae3e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:01.051535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c9d43-4c8f-478b-886e-7351908ae3e5 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:01.051977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c9d43-4c8f-478b-886e-7351908ae3e5 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:01.052256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c9d43-4c8f-478b-886e-7351908ae3e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:01.062340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c9d43-4c8f-478b-886e-7351908ae3e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:01.064209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c9d43-4c8f-478b-886e-7351908ae3e5 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:01.080248 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3502/6920] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:14:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:14:01.264278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074b0ca-01c5-43ea-a786-f98e609f9ed5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:01.264780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074b0ca-01c5-43ea-a786-f98e609f9ed5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:01.265280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074b0ca-01c5-43ea-a786-f98e609f9ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:01.290523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b074b0ca-01c5-43ea-a786-f98e609f9ed5 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:01.292741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b074b0ca-01c5-43ea-a786-f98e609f9ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:01.410388 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3419/6921] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:01.461509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c9d43-4c8f-478b-886e-7351908ae3e5 service glance] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:01.463010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-755c9d43-4c8f-478b-886e-7351908ae3e5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:01.595654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c9d43-4c8f-478b-886e-7351908ae3e5 service glance] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:01.601276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f6783-58a8-45d0-8e46-1ca7ce4249aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:01.602050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f6783-58a8-45d0-8e46-1ca7ce4249aa service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:01.602229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f6783-58a8-45d0-8e46-1ca7ce4249aa service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:01.602486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f6783-58a8-45d0-8e46-1ca7ce4249aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:01.614501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f6783-58a8-45d0-8e46-1ca7ce4249aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:01.616135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f6783-58a8-45d0-8e46-1ca7ce4249aa service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:01.626747 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3503/6922] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:14:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:14:01.916428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cb800-e75b-43b9-b2dc-f323556cb198 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:01.916959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cb800-e75b-43b9-b2dc-f323556cb198 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:01.917448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cb800-e75b-43b9-b2dc-f323556cb198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:01.951299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4a9cb800-e75b-43b9-b2dc-f323556cb198 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:01.953531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4a9cb800-e75b-43b9-b2dc-f323556cb198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:01.969801 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3420/6923] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:01.990928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f6783-58a8-45d0-8e46-1ca7ce4249aa service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:02.001477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:02.015637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f6783-58a8-45d0-8e46-1ca7ce4249aa service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:02.023178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ef309-0090-4fca-9c36-bf7eb41c2ebb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:02.023461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ef309-0090-4fca-9c36-bf7eb41c2ebb service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:02.023829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ef309-0090-4fca-9c36-bf7eb41c2ebb service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:02.023995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ef309-0090-4fca-9c36-bf7eb41c2ebb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:02.033947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ef309-0090-4fca-9c36-bf7eb41c2ebb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:02.037697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ef309-0090-4fca-9c36-bf7eb41c2ebb service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:02.046937 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3504/6924] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:14:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 21:14:02.400000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a118d-18e4-42d5-a31d-ee893211d618 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:02.401540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a118d-18e4-42d5-a31d-ee893211d618 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:02.402625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a118d-18e4-42d5-a31d-ee893211d618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:02.430315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e87a118d-18e4-42d5-a31d-ee893211d618 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:02.431546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e87a118d-18e4-42d5-a31d-ee893211d618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:02.451012 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3421/6925] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:02.466264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ef309-0090-4fca-9c36-bf7eb41c2ebb service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:02.479932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ef309-0090-4fca-9c36-bf7eb41c2ebb service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:02.487233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c9669-0c3a-4c0c-b54c-088e694b7332 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:02.488020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c9669-0c3a-4c0c-b54c-088e694b7332 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:02.488668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c9669-0c3a-4c0c-b54c-088e694b7332 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:02.489190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c9669-0c3a-4c0c-b54c-088e694b7332 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:02.498954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c9669-0c3a-4c0c-b54c-088e694b7332 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:02.500738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c9669-0c3a-4c0c-b54c-088e694b7332 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:02.526656 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3505/6926] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:14:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:14:02.884052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8be90-c294-43f5-9743-e2dd017f4aec None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:02.884598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8be90-c294-43f5-9743-e2dd017f4aec None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:02.885009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8be90-c294-43f5-9743-e2dd017f4aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:02.908543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b4f8be90-c294-43f5-9743-e2dd017f4aec None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:02.909925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b4f8be90-c294-43f5-9743-e2dd017f4aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:02.917773 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3422/6927] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:02.931113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c9669-0c3a-4c0c-b54c-088e694b7332 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:02.947726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c9669-0c3a-4c0c-b54c-088e694b7332 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:02.954832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d02882-4c38-48bc-9960-4860cb59ad2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:02.955648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d02882-4c38-48bc-9960-4860cb59ad2f service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:02.956674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d02882-4c38-48bc-9960-4860cb59ad2f service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:02.957174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d02882-4c38-48bc-9960-4860cb59ad2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:02.967573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d02882-4c38-48bc-9960-4860cb59ad2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:02.969451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d02882-4c38-48bc-9960-4860cb59ad2f service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:02.989096 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3506/6928] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:14:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:14:03.386584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d080c7-ee93-4d3a-8b16-19014bc22f7f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:03.387490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d080c7-ee93-4d3a-8b16-19014bc22f7f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:03.387847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d080c7-ee93-4d3a-8b16-19014bc22f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:03.413189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-13d080c7-ee93-4d3a-8b16-19014bc22f7f None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:03.415262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-13d080c7-ee93-4d3a-8b16-19014bc22f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:03.441409 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3423/6929] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:14:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:03.453787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d02882-4c38-48bc-9960-4860cb59ad2f service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:03.470635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d02882-4c38-48bc-9960-4860cb59ad2f service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:03.488400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0482e-96b5-4c15-a08d-bc69e6e021bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:03.489055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0482e-96b5-4c15-a08d-bc69e6e021bd service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:03.489349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0482e-96b5-4c15-a08d-bc69e6e021bd service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:03.489630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0482e-96b5-4c15-a08d-bc69e6e021bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:03.522000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0482e-96b5-4c15-a08d-bc69e6e021bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:03.523896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0482e-96b5-4c15-a08d-bc69e6e021bd service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:03.547184 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3507/6930] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:14:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:14:03.848445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae47f68-a550-4f16-95fc-9a3e34377e45 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:03.849050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae47f68-a550-4f16-95fc-9a3e34377e45 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:03.849514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae47f68-a550-4f16-95fc-9a3e34377e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:03.872797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2ae47f68-a550-4f16-95fc-9a3e34377e45 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:03.874247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2ae47f68-a550-4f16-95fc-9a3e34377e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:03.888531 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3424/6931] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:14:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:03.897528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0482e-96b5-4c15-a08d-bc69e6e021bd service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:03.912025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0482e-96b5-4c15-a08d-bc69e6e021bd service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:03.919854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504fbf7-b9bf-41f9-8206-02a9b2d82e90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:03.920589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504fbf7-b9bf-41f9-8206-02a9b2d82e90 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:03.921091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504fbf7-b9bf-41f9-8206-02a9b2d82e90 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:03.921655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504fbf7-b9bf-41f9-8206-02a9b2d82e90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:03.932255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504fbf7-b9bf-41f9-8206-02a9b2d82e90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:03.934584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504fbf7-b9bf-41f9-8206-02a9b2d82e90 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:03.958063 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3508/6932] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:14:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 21:14:04.957196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb066d6-0428-4665-aa5c-72b7a28e755f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:04.958025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb066d6-0428-4665-aa5c-72b7a28e755f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:04.958456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb066d6-0428-4665-aa5c-72b7a28e755f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:04.983114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fcb066d6-0428-4665-aa5c-72b7a28e755f None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:04.984501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fcb066d6-0428-4665-aa5c-72b7a28e755f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:04.992678 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3425/6933] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:14:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:05.012968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504fbf7-b9bf-41f9-8206-02a9b2d82e90 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:05.027718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504fbf7-b9bf-41f9-8206-02a9b2d82e90 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:05.033149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b6b09-5ed4-4e5f-9a8a-fb36f72fff11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:05.033776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b6b09-5ed4-4e5f-9a8a-fb36f72fff11 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:05.034069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b6b09-5ed4-4e5f-9a8a-fb36f72fff11 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:05.034353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b6b09-5ed4-4e5f-9a8a-fb36f72fff11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:05.045206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b6b09-5ed4-4e5f-9a8a-fb36f72fff11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:05.046840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b6b09-5ed4-4e5f-9a8a-fb36f72fff11 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:05.057641 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3509/6934] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:14:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:14:05.271960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a04686-bf8d-49cc-b450-a9c35e48b1d4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:05.272342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a04686-bf8d-49cc-b450-a9c35e48b1d4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:05.272865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a04686-bf8d-49cc-b450-a9c35e48b1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:05.277339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e165d44-2832-4673-bf88-5542c7805820 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:05.278432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e165d44-2832-4673-bf88-5542c7805820 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:05.278945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e165d44-2832-4673-bf88-5542c7805820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:05.302360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-73a04686-bf8d-49cc-b450-a9c35e48b1d4 None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:05.303712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-73a04686-bf8d-49cc-b450-a9c35e48b1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:05.320280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1e165d44-2832-4673-bf88-5542c7805820 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:05.321582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1e165d44-2832-4673-bf88-5542c7805820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:05.331259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:05.332232 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3510/6935] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:14:05.347505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e165d44-2832-4673-bf88-5542c7805820 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:05.372060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e165d44-2832-4673-bf88-5542c7805820 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:05.380064 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3426/6936] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:14:05.380219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d8815-34ca-4e4a-8a18-1c2919f31c86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:05.380813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d8815-34ca-4e4a-8a18-1c2919f31c86 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:05.381130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d8815-34ca-4e4a-8a18-1c2919f31c86 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:05.381409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d8815-34ca-4e4a-8a18-1c2919f31c86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:05.391790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d8815-34ca-4e4a-8a18-1c2919f31c86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:05.393610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d8815-34ca-4e4a-8a18-1c2919f31c86 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:05.406184 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3511/6937] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:14:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:14:05.672000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1adcc4-41f8-495d-80bc-0a1cb7dab939 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:05.672544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1adcc4-41f8-495d-80bc-0a1cb7dab939 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:05.673063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1adcc4-41f8-495d-80bc-0a1cb7dab939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:05.695835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ea1adcc4-41f8-495d-80bc-0a1cb7dab939 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:05.697717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ea1adcc4-41f8-495d-80bc-0a1cb7dab939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:05.716678 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3427/6938] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:05.737845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d8815-34ca-4e4a-8a18-1c2919f31c86 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:05.751318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d8815-34ca-4e4a-8a18-1c2919f31c86 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:05.757510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdceb78a-8d57-4d11-bc0c-d4812cccf8cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:05.758400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdceb78a-8d57-4d11-bc0c-d4812cccf8cb service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:05.758830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdceb78a-8d57-4d11-bc0c-d4812cccf8cb service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:05.759250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdceb78a-8d57-4d11-bc0c-d4812cccf8cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:05.768299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdceb78a-8d57-4d11-bc0c-d4812cccf8cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:05.770438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdceb78a-8d57-4d11-bc0c-d4812cccf8cb service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:05.780055 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3512/6939] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:14:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:14:05.976387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d60d16-3d05-42a0-a0ac-09e4e6fa1ee7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:05.978692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d60d16-3d05-42a0-a0ac-09e4e6fa1ee7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:05.979204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d60d16-3d05-42a0-a0ac-09e4e6fa1ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:06.007179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-94d60d16-3d05-42a0-a0ac-09e4e6fa1ee7 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:06.008933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-94d60d16-3d05-42a0-a0ac-09e4e6fa1ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:06.021062 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3428/6940] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:06.042003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdceb78a-8d57-4d11-bc0c-d4812cccf8cb service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:06.061520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdceb78a-8d57-4d11-bc0c-d4812cccf8cb service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:06.070020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3117f-e123-4f17-85ab-65fab9e38bd2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:06.071041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3117f-e123-4f17-85ab-65fab9e38bd2 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:06.071493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3117f-e123-4f17-85ab-65fab9e38bd2 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:06.072021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3117f-e123-4f17-85ab-65fab9e38bd2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:06.082695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3117f-e123-4f17-85ab-65fab9e38bd2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:06.084336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3117f-e123-4f17-85ab-65fab9e38bd2 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:06.093392 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3513/6941] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:14:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:14:06.386211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bcb63-e9b9-4e78-9a40-e05d739d530b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:06.386672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bcb63-e9b9-4e78-9a40-e05d739d530b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:06.387615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bcb63-e9b9-4e78-9a40-e05d739d530b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:06.411023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-026bcb63-e9b9-4e78-9a40-e05d739d530b None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:06.412457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-026bcb63-e9b9-4e78-9a40-e05d739d530b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:06.429663 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3429/6942] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:14:06.445203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3117f-e123-4f17-85ab-65fab9e38bd2 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:06.466504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3117f-e123-4f17-85ab-65fab9e38bd2 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:06.486969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bf60b-6722-4e41-8475-b288ccbaa463 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:06.488156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bf60b-6722-4e41-8475-b288ccbaa463 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:06.489302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bf60b-6722-4e41-8475-b288ccbaa463 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:06.489782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bf60b-6722-4e41-8475-b288ccbaa463 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:06.503284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bf60b-6722-4e41-8475-b288ccbaa463 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:06.505843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bf60b-6722-4e41-8475-b288ccbaa463 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:06.517845 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3514/6943] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:14:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:14:06.771960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373b402-e26e-44f8-94d7-c817ff98e163 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:06.772635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373b402-e26e-44f8-94d7-c817ff98e163 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:06.772635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373b402-e26e-44f8-94d7-c817ff98e163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:06.800016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1373b402-e26e-44f8-94d7-c817ff98e163 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:06.801856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1373b402-e26e-44f8-94d7-c817ff98e163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:06.811788 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3430/6944] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:06.834279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bf60b-6722-4e41-8475-b288ccbaa463 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:06.858882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bf60b-6722-4e41-8475-b288ccbaa463 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:06.864336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d678e-455c-4b98-ac5d-93b5ca528237 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:06.864990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d678e-455c-4b98-ac5d-93b5ca528237 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:06.865280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d678e-455c-4b98-ac5d-93b5ca528237 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:06.865559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d678e-455c-4b98-ac5d-93b5ca528237 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:06.901527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d678e-455c-4b98-ac5d-93b5ca528237 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:06.903153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d678e-455c-4b98-ac5d-93b5ca528237 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:06.914205 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3515/6945] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:14:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:14:07.090837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373b402-e26e-44f8-94d7-c817ff98e163 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:07.192436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373b402-e26e-44f8-94d7-c817ff98e163 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:07.197003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:07.197655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:07.197940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:07.198379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] PATH_INFO: `/v3/users/ed6fcb37d5db4e06889dfbbb5d458596` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:07.199667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed6fcb37d5db4e06889dfbbb5d458596)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:07.201778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:07.271868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed6fcb37d5db4e06889dfbbb5d458596'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:07.287948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed6fcb37d5db4e06889dfbbb5d458596'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:07.288826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed6fcb37d5db4e06889dfbbb5d458596'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:07.301515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed6fcb37d5db4e06889dfbbb5d458596'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:07.314324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed6fcb37d5db4e06889dfbbb5d458596'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:07.322872 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3431/6946] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:14:07 2022] DELETE /identity/v3/users/ed6fcb37d5db4e06889dfbbb5d458596 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:07.327295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f45d1e-7e37-4e8a-b871-8147c02198ec None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:07.327756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f45d1e-7e37-4e8a-b871-8147c02198ec None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:07.327863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f45d1e-7e37-4e8a-b871-8147c02198ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:07.349189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a7f45d1e-7e37-4e8a-b871-8147c02198ec None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:07.349814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a7f45d1e-7e37-4e8a-b871-8147c02198ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:07.425718 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3516/6947] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:07.436842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:07.438942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:07.474147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:07.520265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500d917-ee24-410f-818e-713e0854edfe admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:07.524323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c6cc0-509a-41c0-904a-4fece84919d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:07.524980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c6cc0-509a-41c0-904a-4fece84919d8 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:07.525340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c6cc0-509a-41c0-904a-4fece84919d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:07.525625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c6cc0-509a-41c0-904a-4fece84919d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:07.536525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c6cc0-509a-41c0-904a-4fece84919d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:07.538336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c6cc0-509a-41c0-904a-4fece84919d8 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:07.562383 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3432/6948] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:14:08.411222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f45d1e-7e37-4e8a-b871-8147c02198ec None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:08.412884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a7f45d1e-7e37-4e8a-b871-8147c02198ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:08.428169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f45d1e-7e37-4e8a-b871-8147c02198ec None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:08.435046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439df42-0002-4005-8f30-0d60fab5abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:08.436682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439df42-0002-4005-8f30-0d60fab5abc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:08.437507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439df42-0002-4005-8f30-0d60fab5abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:08.438021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439df42-0002-4005-8f30-0d60fab5abc3 admin admin] PATH_INFO: `/v3/projects/055069b8643c4ebb9ad1b0d08cdcdfe3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:08.439619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439df42-0002-4005-8f30-0d60fab5abc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=055069b8643c4ebb9ad1b0d08cdcdfe3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:08.442727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439df42-0002-4005-8f30-0d60fab5abc3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:08.453717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:08.509477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c439df42-0002-4005-8f30-0d60fab5abc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '055069b8643c4ebb9ad1b0d08cdcdfe3', 'request_id': 'req-c439df42-0002-4005-8f30-0d60fab5abc3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:08.523088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c439df42-0002-4005-8f30-0d60fab5abc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '055069b8643c4ebb9ad1b0d08cdcdfe3', 'request_id': 'req-c439df42-0002-4005-8f30-0d60fab5abc3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:08.534995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c439df42-0002-4005-8f30-0d60fab5abc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '055069b8643c4ebb9ad1b0d08cdcdfe3', 'request_id': 'req-c439df42-0002-4005-8f30-0d60fab5abc3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:08.550856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c439df42-0002-4005-8f30-0d60fab5abc3 admin admin] The token cache is being invalidate because project 055069b8643c4ebb9ad1b0d08cdcdfe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:14:08.551412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c439df42-0002-4005-8f30-0d60fab5abc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:08.552309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c439df42-0002-4005-8f30-0d60fab5abc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:08.558393 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3517/6949] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:14:08 2022] DELETE /identity/v3/projects/055069b8643c4ebb9ad1b0d08cdcdfe3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:08.559646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce0e1b-e659-4128-94b4-158082d35c04 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:08.560084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce0e1b-e659-4128-94b4-158082d35c04 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:08.560314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce0e1b-e659-4128-94b4-158082d35c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:08.591545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-17ce0e1b-e659-4128-94b4-158082d35c04 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:08.593025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-17ce0e1b-e659-4128-94b4-158082d35c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:08.693493 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3433/6950] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:08.705607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b3070-bdb9-4d34-b01c-bb5a79cc0d58 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:08.706150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b3070-bdb9-4d34-b01c-bb5a79cc0d58 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:08.706841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b3070-bdb9-4d34-b01c-bb5a79cc0d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:08.751693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b02b3070-bdb9-4d34-b01c-bb5a79cc0d58 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:08.753048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b02b3070-bdb9-4d34-b01c-bb5a79cc0d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:08.763428 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3518/6951] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:08.767666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce0e1b-e659-4128-94b4-158082d35c04 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:08.779887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce0e1b-e659-4128-94b4-158082d35c04 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:08.786664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a56e4-5e67-47eb-9aaf-cc9bb04ae321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:08.787301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a56e4-5e67-47eb-9aaf-cc9bb04ae321 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:08.787714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a56e4-5e67-47eb-9aaf-cc9bb04ae321 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:08.787845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a56e4-5e67-47eb-9aaf-cc9bb04ae321 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:08.788500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a56e4-5e67-47eb-9aaf-cc9bb04ae321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:14:08.788807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a56e4-5e67-47eb-9aaf-cc9bb04ae321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:08.790459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a56e4-5e67-47eb-9aaf-cc9bb04ae321 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:08.810514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b3070-bdb9-4d34-b01c-bb5a79cc0d58 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:08.826617 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3434/6952] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:14:08.828468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b3070-bdb9-4d34-b01c-bb5a79cc0d58 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:08.839213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700195e-8ad9-41e9-9a45-932fb8022f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:08.839838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700195e-8ad9-41e9-9a45-932fb8022f61 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:08.841046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700195e-8ad9-41e9-9a45-932fb8022f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:08.841705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700195e-8ad9-41e9-9a45-932fb8022f61 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:08.845787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700195e-8ad9-41e9-9a45-932fb8022f61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:08.853690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700195e-8ad9-41e9-9a45-932fb8022f61 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:08.921221 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3519/6953] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:14:08 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:14:08.921500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a56e4-5e67-47eb-9aaf-cc9bb04ae321 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:08.938833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a56e4-5e67-47eb-9aaf-cc9bb04ae321 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:09.037550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5d5f8-552b-4f84-a9ce-89153b8cdb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:09.038569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5d5f8-552b-4f84-a9ce-89153b8cdb83 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:09.039009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5d5f8-552b-4f84-a9ce-89153b8cdb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:09.039439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5d5f8-552b-4f84-a9ce-89153b8cdb83 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:09.040573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5d5f8-552b-4f84-a9ce-89153b8cdb83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:09.044219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5d5f8-552b-4f84-a9ce-89153b8cdb83 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:09.092386 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3435/6954] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:14:08 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:14:09.092983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700195e-8ad9-41e9-9a45-932fb8022f61 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:09.115808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700195e-8ad9-41e9-9a45-932fb8022f61 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:09.122149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0c890-2ab5-45b1-ae8a-3d31bfeda39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:09.122531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0c890-2ab5-45b1-ae8a-3d31bfeda39d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:09.122885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0c890-2ab5-45b1-ae8a-3d31bfeda39d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:09.123962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0c890-2ab5-45b1-ae8a-3d31bfeda39d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:09.124545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0c890-2ab5-45b1-ae8a-3d31bfeda39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:09.126519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0c890-2ab5-45b1-ae8a-3d31bfeda39d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:09.147836 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3520/6955] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:14:09.152763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5d5f8-552b-4f84-a9ce-89153b8cdb83 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:09.178439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5d5f8-552b-4f84-a9ce-89153b8cdb83 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:09.189860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739c379-4f5c-4d51-9152-3f201ff6f872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:09.191297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739c379-4f5c-4d51-9152-3f201ff6f872 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:09.191872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739c379-4f5c-4d51-9152-3f201ff6f872 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:09.192298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739c379-4f5c-4d51-9152-3f201ff6f872 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:09.193083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739c379-4f5c-4d51-9152-3f201ff6f872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:09.195609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739c379-4f5c-4d51-9152-3f201ff6f872 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:09.217976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0c890-2ab5-45b1-ae8a-3d31bfeda39d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:09.228969 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3436/6956] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:14:09.234592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0c890-2ab5-45b1-ae8a-3d31bfeda39d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:09.248677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3c291-7606-47cc-8e03-011eb4e3570d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:09.249411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3c291-7606-47cc-8e03-011eb4e3570d admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:09.249946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3c291-7606-47cc-8e03-011eb4e3570d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:09.250233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3c291-7606-47cc-8e03-011eb4e3570d admin admin] PATH_INFO: `/v3/projects/c5e13bd064ef489fb7fc7ee85de4cf37/users/1257b21a0315401b86e34e68db0424a5/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:09.284337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3c291-7606-47cc-8e03-011eb4e3570d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e13bd064ef489fb7fc7ee85de4cf37, user_id=1257b21a0315401b86e34e68db0424a5, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:09.293934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3c291-7606-47cc-8e03-011eb4e3570d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:09.335739 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3521/6957] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:14:09 2022] PUT /identity/v3/projects/c5e13bd064ef489fb7fc7ee85de4cf37/users/1257b21a0315401b86e34e68db0424a5/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:09.337662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2fbcc-7f85-48c8-9d53-9be3ac840aec None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:09.338151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2fbcc-7f85-48c8-9d53-9be3ac840aec None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:09.338644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2fbcc-7f85-48c8-9d53-9be3ac840aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:09.405159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ebe2fbcc-7f85-48c8-9d53-9be3ac840aec None None] MFA Rules not processed for user `1257b21a0315401b86e34e68db0424a5`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:09.406968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ebe2fbcc-7f85-48c8-9d53-9be3ac840aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:09.451514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:09.540720 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3437/6958] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:09.553836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3c291-7606-47cc-8e03-011eb4e3570d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:09.555751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-07a3c291-7606-47cc-8e03-011eb4e3570d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:09.639585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3c291-7606-47cc-8e03-011eb4e3570d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:09.646228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e40b78-3782-4fc5-a5dd-a6d0a2e52793 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:09.647104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e40b78-3782-4fc5-a5dd-a6d0a2e52793 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:09.647570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e40b78-3782-4fc5-a5dd-a6d0a2e52793 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:09.648275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e40b78-3782-4fc5-a5dd-a6d0a2e52793 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:09.660604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e40b78-3782-4fc5-a5dd-a6d0a2e52793 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:09.663390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e40b78-3782-4fc5-a5dd-a6d0a2e52793 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:09.677836 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3522/6959] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:14:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:14:09.835547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2fbcc-7f85-48c8-9d53-9be3ac840aec None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:09.939733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2fbcc-7f85-48c8-9d53-9be3ac840aec None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:09.950008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:09.950587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:09.950863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:09.955792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] PATH_INFO: `/v3/users/1257b21a0315401b86e34e68db0424a5` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:09.958353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1257b21a0315401b86e34e68db0424a5)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:09.977806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:10.080455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1257b21a0315401b86e34e68db0424a5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:10.097752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1257b21a0315401b86e34e68db0424a5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:10.099127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1257b21a0315401b86e34e68db0424a5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:10.119971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1257b21a0315401b86e34e68db0424a5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:10.143674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1257b21a0315401b86e34e68db0424a5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:10.151023 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3438/6960] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:14:09 2022] DELETE /identity/v3/users/1257b21a0315401b86e34e68db0424a5 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:10.155695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28699bcf-907e-4afa-9212-ed997756ff7f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:10.155991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28699bcf-907e-4afa-9212-ed997756ff7f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:10.156319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28699bcf-907e-4afa-9212-ed997756ff7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:10.181945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-28699bcf-907e-4afa-9212-ed997756ff7f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:10.185075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-28699bcf-907e-4afa-9212-ed997756ff7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:10.249618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:10.331235 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3523/6961] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:10.349669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:10.351134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:10.414243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:10.445209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d72211-cd0e-40fe-aae3-e576f6daa0d7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:10.449317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbecc6e-6af4-45d5-b781-c060ae731f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:10.450232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbecc6e-6af4-45d5-b781-c060ae731f31 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:10.450722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbecc6e-6af4-45d5-b781-c060ae731f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:10.451204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbecc6e-6af4-45d5-b781-c060ae731f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:10.462465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbecc6e-6af4-45d5-b781-c060ae731f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:10.464386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbecc6e-6af4-45d5-b781-c060ae731f31 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:10.479112 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3439/6962] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:14:11.101550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28699bcf-907e-4afa-9212-ed997756ff7f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:11.104030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-28699bcf-907e-4afa-9212-ed997756ff7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:11.120373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28699bcf-907e-4afa-9212-ed997756ff7f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:11.146028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b84ab7-906c-497a-9806-b4e6ff924858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:11.149717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b84ab7-906c-497a-9806-b4e6ff924858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:11.150015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b84ab7-906c-497a-9806-b4e6ff924858 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:11.150302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b84ab7-906c-497a-9806-b4e6ff924858 admin admin] PATH_INFO: `/v3/projects/c5e13bd064ef489fb7fc7ee85de4cf37` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:11.153683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b84ab7-906c-497a-9806-b4e6ff924858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e13bd064ef489fb7fc7ee85de4cf37)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:11.157387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b84ab7-906c-497a-9806-b4e6ff924858 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:11.258432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f1b84ab7-906c-497a-9806-b4e6ff924858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e13bd064ef489fb7fc7ee85de4cf37', 'request_id': 'req-f1b84ab7-906c-497a-9806-b4e6ff924858'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:11.267784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f1b84ab7-906c-497a-9806-b4e6ff924858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e13bd064ef489fb7fc7ee85de4cf37', 'request_id': 'req-f1b84ab7-906c-497a-9806-b4e6ff924858'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:11.275985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f1b84ab7-906c-497a-9806-b4e6ff924858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e13bd064ef489fb7fc7ee85de4cf37', 'request_id': 'req-f1b84ab7-906c-497a-9806-b4e6ff924858'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:11.288768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f1b84ab7-906c-497a-9806-b4e6ff924858 admin admin] The token cache is being invalidate because project c5e13bd064ef489fb7fc7ee85de4cf37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:14:11.289110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f1b84ab7-906c-497a-9806-b4e6ff924858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:11.289792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f1b84ab7-906c-497a-9806-b4e6ff924858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:11.291581 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3524/6963] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:14:11 2022] DELETE /identity/v3/projects/c5e13bd064ef489fb7fc7ee85de4cf37 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:11.296645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122d2ca-a184-4016-a340-b92f7b230537 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:11.297645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122d2ca-a184-4016-a340-b92f7b230537 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:11.298126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122d2ca-a184-4016-a340-b92f7b230537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:11.322514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c122d2ca-a184-4016-a340-b92f7b230537 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:11.323865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c122d2ca-a184-4016-a340-b92f7b230537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:11.397053 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3440/6964] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:11.405302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d335ac-2c0a-413c-a9d7-1fe7b30fcb37 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:11.405590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d335ac-2c0a-413c-a9d7-1fe7b30fcb37 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:11.405868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d335ac-2c0a-413c-a9d7-1fe7b30fcb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:11.427288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e3d335ac-2c0a-413c-a9d7-1fe7b30fcb37 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:11.428759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e3d335ac-2c0a-413c-a9d7-1fe7b30fcb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:11.437463 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3525/6965] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:11.441756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122d2ca-a184-4016-a340-b92f7b230537 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:11.451863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122d2ca-a184-4016-a340-b92f7b230537 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:11.456504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977f303-03b1-444b-95b2-6190477030f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:11.457153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977f303-03b1-444b-95b2-6190477030f6 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:11.458067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977f303-03b1-444b-95b2-6190477030f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:11.458067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977f303-03b1-444b-95b2-6190477030f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:11.458359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977f303-03b1-444b-95b2-6190477030f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:14:11.458668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977f303-03b1-444b-95b2-6190477030f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:11.460293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977f303-03b1-444b-95b2-6190477030f6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:11.472211 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3441/6966] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:14:11.476524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d335ac-2c0a-413c-a9d7-1fe7b30fcb37 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:11.487390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d335ac-2c0a-413c-a9d7-1fe7b30fcb37 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:11.492768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25cb66-0800-40fa-aac8-8cfbfb51d996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:11.493387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25cb66-0800-40fa-aac8-8cfbfb51d996 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:11.493673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25cb66-0800-40fa-aac8-8cfbfb51d996 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:11.493953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25cb66-0800-40fa-aac8-8cfbfb51d996 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:11.494609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25cb66-0800-40fa-aac8-8cfbfb51d996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:11.496722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25cb66-0800-40fa-aac8-8cfbfb51d996 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:11.534139 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3526/6967] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:14:11 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:14:11.538350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977f303-03b1-444b-95b2-6190477030f6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:11.550542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977f303-03b1-444b-95b2-6190477030f6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:11.616633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e41f6-7d8c-4f72-a7db-7a7d9cdc5943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:11.617313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e41f6-7d8c-4f72-a7db-7a7d9cdc5943 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:11.617602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e41f6-7d8c-4f72-a7db-7a7d9cdc5943 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:11.617882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e41f6-7d8c-4f72-a7db-7a7d9cdc5943 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:11.618532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e41f6-7d8c-4f72-a7db-7a7d9cdc5943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:11.620560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e41f6-7d8c-4f72-a7db-7a7d9cdc5943 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:11.652466 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3442/6968] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:14:11 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:14:11.653591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25cb66-0800-40fa-aac8-8cfbfb51d996 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:11.664717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25cb66-0800-40fa-aac8-8cfbfb51d996 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:11.672048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4524687-d65c-4c0c-bcd6-0ee7abca036b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:11.672954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524687-d65c-4c0c-bcd6-0ee7abca036b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:11.673245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524687-d65c-4c0c-bcd6-0ee7abca036b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:11.673521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524687-d65c-4c0c-bcd6-0ee7abca036b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:11.674170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4524687-d65c-4c0c-bcd6-0ee7abca036b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:11.675816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4524687-d65c-4c0c-bcd6-0ee7abca036b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:11.691317 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3527/6969] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:14:11.695157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e41f6-7d8c-4f72-a7db-7a7d9cdc5943 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:11.707224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e41f6-7d8c-4f72-a7db-7a7d9cdc5943 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:11.713239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978ddbe-1c4a-4e17-8eb8-18700dc6fb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:11.714124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978ddbe-1c4a-4e17-8eb8-18700dc6fb05 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:11.714414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978ddbe-1c4a-4e17-8eb8-18700dc6fb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:11.714692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978ddbe-1c4a-4e17-8eb8-18700dc6fb05 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:11.715408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978ddbe-1c4a-4e17-8eb8-18700dc6fb05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:11.717408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978ddbe-1c4a-4e17-8eb8-18700dc6fb05 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:11.730178 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3443/6970] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:14:11.734866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4524687-d65c-4c0c-bcd6-0ee7abca036b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:11.747355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4524687-d65c-4c0c-bcd6-0ee7abca036b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:11.752425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d29cc8-6ada-417e-b334-2fb9f5d037aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:11.753129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d29cc8-6ada-417e-b334-2fb9f5d037aa admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:11.753416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d29cc8-6ada-417e-b334-2fb9f5d037aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:11.753695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d29cc8-6ada-417e-b334-2fb9f5d037aa admin admin] PATH_INFO: `/v3/projects/a2bfb57a877145299451eb6a6a7b66ef/users/fbbf75ba2dc64d93b2e3f75aaac0d5b5/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:11.775780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d29cc8-6ada-417e-b334-2fb9f5d037aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2bfb57a877145299451eb6a6a7b66ef, user_id=fbbf75ba2dc64d93b2e3f75aaac0d5b5, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:11.779173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d29cc8-6ada-417e-b334-2fb9f5d037aa admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:11.795537 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3528/6971] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:14:11 2022] PUT /identity/v3/projects/a2bfb57a877145299451eb6a6a7b66ef/users/fbbf75ba2dc64d93b2e3f75aaac0d5b5/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:11.887135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f3f6-71b7-4268-a68c-54e3d17dc41b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:11.887380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f3f6-71b7-4268-a68c-54e3d17dc41b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:11.887661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f3f6-71b7-4268-a68c-54e3d17dc41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:11.926042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-be61f3f6-71b7-4268-a68c-54e3d17dc41b None None] MFA Rules not processed for user `fbbf75ba2dc64d93b2e3f75aaac0d5b5`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:11.927293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-be61f3f6-71b7-4268-a68c-54e3d17dc41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:11.971777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:12.007802 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3444/6972] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:12.020789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d29cc8-6ada-417e-b334-2fb9f5d037aa admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:12.023102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f7d29cc8-6ada-417e-b334-2fb9f5d037aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:12.102779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d29cc8-6ada-417e-b334-2fb9f5d037aa admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:12.107861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9433a54-e80b-45c9-b9a9-b390d9dd2b71 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:12.108876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9433a54-e80b-45c9-b9a9-b390d9dd2b71 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:12.109360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9433a54-e80b-45c9-b9a9-b390d9dd2b71 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:12.109929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9433a54-e80b-45c9-b9a9-b390d9dd2b71 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:12.121736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:12.128143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9433a54-e80b-45c9-b9a9-b390d9dd2b71 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:12.139657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9433a54-e80b-45c9-b9a9-b390d9dd2b71 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:12.161594 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3529/6973] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:14:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:14:12.645537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61f3f6-71b7-4268-a68c-54e3d17dc41b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:12.716041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61f3f6-71b7-4268-a68c-54e3d17dc41b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:12.722710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:12.723367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:12.723651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:12.723928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] PATH_INFO: `/v3/users/fbbf75ba2dc64d93b2e3f75aaac0d5b5` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:12.725105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbbf75ba2dc64d93b2e3f75aaac0d5b5)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:12.727129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:12.786686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbbf75ba2dc64d93b2e3f75aaac0d5b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:12.811280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbbf75ba2dc64d93b2e3f75aaac0d5b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:12.812278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbbf75ba2dc64d93b2e3f75aaac0d5b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:12.824853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbbf75ba2dc64d93b2e3f75aaac0d5b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:12.838862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbbf75ba2dc64d93b2e3f75aaac0d5b5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:12.852192 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3445/6974] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:14:12 2022] DELETE /identity/v3/users/fbbf75ba2dc64d93b2e3f75aaac0d5b5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:12.854158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2174abe-c472-474c-a520-b732d97aca14 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:12.854541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2174abe-c472-474c-a520-b732d97aca14 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:12.854891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2174abe-c472-474c-a520-b732d97aca14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:12.878335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f2174abe-c472-474c-a520-b732d97aca14 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:12.879939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f2174abe-c472-474c-a520-b732d97aca14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:12.969066 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3530/6975] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:14:12.996093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:12.999369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:13.086334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7e0c1-fe45-4ab4-bf41-ccb16ed4911e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:13.090976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2aba5-aecc-43ca-959f-312c6f63f07c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:13.091661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2aba5-aecc-43ca-959f-312c6f63f07c service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:13.091937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2aba5-aecc-43ca-959f-312c6f63f07c service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:13.092246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2aba5-aecc-43ca-959f-312c6f63f07c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:13.103253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2aba5-aecc-43ca-959f-312c6f63f07c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:13.104822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2aba5-aecc-43ca-959f-312c6f63f07c service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:13.111624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:13.118777 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3446/6976] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:14:13.540711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2174abe-c472-474c-a520-b732d97aca14 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:13.543838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f2174abe-c472-474c-a520-b732d97aca14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:13.564452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2174abe-c472-474c-a520-b732d97aca14 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:13.570072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f603082-343b-45fa-979e-285fdcacb75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:13.571027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f603082-343b-45fa-979e-285fdcacb75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:13.571497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f603082-343b-45fa-979e-285fdcacb75e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:13.571956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f603082-343b-45fa-979e-285fdcacb75e admin admin] PATH_INFO: `/v3/projects/a2bfb57a877145299451eb6a6a7b66ef` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:13.573456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f603082-343b-45fa-979e-285fdcacb75e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2bfb57a877145299451eb6a6a7b66ef)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:13.575618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f603082-343b-45fa-979e-285fdcacb75e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:13.627623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1f603082-343b-45fa-979e-285fdcacb75e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2bfb57a877145299451eb6a6a7b66ef', 'request_id': 'req-1f603082-343b-45fa-979e-285fdcacb75e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:13.637766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1f603082-343b-45fa-979e-285fdcacb75e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2bfb57a877145299451eb6a6a7b66ef', 'request_id': 'req-1f603082-343b-45fa-979e-285fdcacb75e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:13.644637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1f603082-343b-45fa-979e-285fdcacb75e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2bfb57a877145299451eb6a6a7b66ef', 'request_id': 'req-1f603082-343b-45fa-979e-285fdcacb75e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:13.658720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1f603082-343b-45fa-979e-285fdcacb75e admin admin] The token cache is being invalidate because project a2bfb57a877145299451eb6a6a7b66ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:14:13.659026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1f603082-343b-45fa-979e-285fdcacb75e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:13.659736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1f603082-343b-45fa-979e-285fdcacb75e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:13.663073 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3531/6977] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:14:13 2022] DELETE /identity/v3/projects/a2bfb57a877145299451eb6a6a7b66ef => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:13.668306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32915ac6-33dc-479b-842f-bc3f9fb76464 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:13.668807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32915ac6-33dc-479b-842f-bc3f9fb76464 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:13.668878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32915ac6-33dc-479b-842f-bc3f9fb76464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:13.689991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-32915ac6-33dc-479b-842f-bc3f9fb76464 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:13.691514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-32915ac6-33dc-479b-842f-bc3f9fb76464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:13.773632 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3447/6978] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:14:13.782052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b442d63-fca2-47d0-99cb-fb63103b64e3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:13.782399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b442d63-fca2-47d0-99cb-fb63103b64e3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:13.782708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b442d63-fca2-47d0-99cb-fb63103b64e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:13.805520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4b442d63-fca2-47d0-99cb-fb63103b64e3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:13.806679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4b442d63-fca2-47d0-99cb-fb63103b64e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:13.815867 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3532/6979] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:13.821561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32915ac6-33dc-479b-842f-bc3f9fb76464 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:13.834042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32915ac6-33dc-479b-842f-bc3f9fb76464 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:13.839081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b109d-7d2d-4378-bb6d-1a6995b5dd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:13.840782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b109d-7d2d-4378-bb6d-1a6995b5dd3c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:13.840782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b109d-7d2d-4378-bb6d-1a6995b5dd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:13.841086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b109d-7d2d-4378-bb6d-1a6995b5dd3c admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:13.841976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b109d-7d2d-4378-bb6d-1a6995b5dd3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:14:13.841976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b109d-7d2d-4378-bb6d-1a6995b5dd3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:13.843470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b109d-7d2d-4378-bb6d-1a6995b5dd3c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:13.856531 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3448/6980] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:14:13.861453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b442d63-fca2-47d0-99cb-fb63103b64e3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:13.872826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b442d63-fca2-47d0-99cb-fb63103b64e3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:13.878120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6eff2-b72d-4cae-b637-c9d0d3cf5a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:13.878727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6eff2-b72d-4cae-b637-c9d0d3cf5a15 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:13.879009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6eff2-b72d-4cae-b637-c9d0d3cf5a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:13.879442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6eff2-b72d-4cae-b637-c9d0d3cf5a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:13.880245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6eff2-b72d-4cae-b637-c9d0d3cf5a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:13.882362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6eff2-b72d-4cae-b637-c9d0d3cf5a15 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:13.916725 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3533/6981] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:14:13 2022] POST /identity/v3/projects => generated 355 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:14:13.918008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b109d-7d2d-4378-bb6d-1a6995b5dd3c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:13.928844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b109d-7d2d-4378-bb6d-1a6995b5dd3c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:13.989744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db7f92-6049-4106-a4e5-fd75f3837e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:13.990356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db7f92-6049-4106-a4e5-fd75f3837e05 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:13.990642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db7f92-6049-4106-a4e5-fd75f3837e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:13.990921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db7f92-6049-4106-a4e5-fd75f3837e05 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:13.991552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db7f92-6049-4106-a4e5-fd75f3837e05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:13.993512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db7f92-6049-4106-a4e5-fd75f3837e05 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:14.019071 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3449/6982] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:14:13 2022] POST /identity/v3/users => generated 278 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:14:14.020629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6eff2-b72d-4cae-b637-c9d0d3cf5a15 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:14.032141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6eff2-b72d-4cae-b637-c9d0d3cf5a15 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:14.037777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7fd4c-bc90-4e92-96b2-ac85ecf52289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:14.038584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7fd4c-bc90-4e92-96b2-ac85ecf52289 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.038969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7fd4c-bc90-4e92-96b2-ac85ecf52289 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.039346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7fd4c-bc90-4e92-96b2-ac85ecf52289 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.040391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7fd4c-bc90-4e92-96b2-ac85ecf52289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:14.042683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7fd4c-bc90-4e92-96b2-ac85ecf52289 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:14.059840 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3534/6983] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:14:14.061389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db7f92-6049-4106-a4e5-fd75f3837e05 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:14.072448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db7f92-6049-4106-a4e5-fd75f3837e05 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:14.078390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10535d69-f91d-4058-aad4-86f423ca3728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:14.079080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10535d69-f91d-4058-aad4-86f423ca3728 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.079433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10535d69-f91d-4058-aad4-86f423ca3728 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.079797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10535d69-f91d-4058-aad4-86f423ca3728 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.080543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10535d69-f91d-4058-aad4-86f423ca3728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:14.082252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10535d69-f91d-4058-aad4-86f423ca3728 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:14.093321 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3450/6984] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:14:14.097576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7fd4c-bc90-4e92-96b2-ac85ecf52289 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:14.108081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7fd4c-bc90-4e92-96b2-ac85ecf52289 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:14.112688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bd0c6-289f-4aa6-b201-20709ce79e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:14.113381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bd0c6-289f-4aa6-b201-20709ce79e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.113668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bd0c6-289f-4aa6-b201-20709ce79e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.113946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bd0c6-289f-4aa6-b201-20709ce79e1a admin admin] PATH_INFO: `/v3/projects/3398a7c60ec644fcaa72f63b18975809/users/349581f434e4411cb97503e4c33b9fea/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.129392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bd0c6-289f-4aa6-b201-20709ce79e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3398a7c60ec644fcaa72f63b18975809, user_id=349581f434e4411cb97503e4c33b9fea, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:14.132173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bd0c6-289f-4aa6-b201-20709ce79e1a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:14.146932 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3535/6985] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:14:14 2022] PUT /identity/v3/projects/3398a7c60ec644fcaa72f63b18975809/users/349581f434e4411cb97503e4c33b9fea/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:14:14.159500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb908f-917a-4258-9af0-62ff9f107729 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.159861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb908f-917a-4258-9af0-62ff9f107729 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.160339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb908f-917a-4258-9af0-62ff9f107729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.201744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-93fb908f-917a-4258-9af0-62ff9f107729 None None] MFA Rules not processed for user `349581f434e4411cb97503e4c33b9fea`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:14.203193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-93fb908f-917a-4258-9af0-62ff9f107729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:14.286795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:14.299898 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3451/6986] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:14.304303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bd0c6-289f-4aa6-b201-20709ce79e1a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:14.322999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:14.379183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bd0c6-289f-4aa6-b201-20709ce79e1a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:14.384823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8670b2-9efe-411a-aaf2-3a02bacfac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:14.385613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8670b2-9efe-411a-aaf2-3a02bacfac7d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.385972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8670b2-9efe-411a-aaf2-3a02bacfac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.386325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8670b2-9efe-411a-aaf2-3a02bacfac7d admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.387598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8670b2-9efe-411a-aaf2-3a02bacfac7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:14.389642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8670b2-9efe-411a-aaf2-3a02bacfac7d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:14.423102 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3536/6987] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:14:14 2022] POST /identity/v3/projects => generated 357 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:14:14.424459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb908f-917a-4258-9af0-62ff9f107729 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:14.437299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb908f-917a-4258-9af0-62ff9f107729 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:14.499597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332cd1f-c81d-4ef8-9a18-af313651414a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:14.500223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332cd1f-c81d-4ef8-9a18-af313651414a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.500515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332cd1f-c81d-4ef8-9a18-af313651414a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.500797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332cd1f-c81d-4ef8-9a18-af313651414a admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.501504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332cd1f-c81d-4ef8-9a18-af313651414a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:14.503451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332cd1f-c81d-4ef8-9a18-af313651414a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:14.528245 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3452/6988] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:14:14 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:14:14.529602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8670b2-9efe-411a-aaf2-3a02bacfac7d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:14.540852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8670b2-9efe-411a-aaf2-3a02bacfac7d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:14.546931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f3ab3-ef7e-42e5-a6f1-96d88e0149e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:14.547584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f3ab3-ef7e-42e5-a6f1-96d88e0149e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.547869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f3ab3-ef7e-42e5-a6f1-96d88e0149e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.548197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f3ab3-ef7e-42e5-a6f1-96d88e0149e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.548940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f3ab3-ef7e-42e5-a6f1-96d88e0149e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:14.550644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f3ab3-ef7e-42e5-a6f1-96d88e0149e7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:14.563224 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3537/6989] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:14:14.567863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332cd1f-c81d-4ef8-9a18-af313651414a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:14.578500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332cd1f-c81d-4ef8-9a18-af313651414a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:14.583306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2cfa3-f085-4255-adf3-88a861ff5288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:14.584409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2cfa3-f085-4255-adf3-88a861ff5288 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.584409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2cfa3-f085-4255-adf3-88a861ff5288 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.584613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2cfa3-f085-4255-adf3-88a861ff5288 admin admin] PATH_INFO: `/v3/projects/84bc40f5c68d489caac597304ed15bc0/users/bdd212be2fbe496f81af0687506fd363/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.602797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2cfa3-f085-4255-adf3-88a861ff5288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84bc40f5c68d489caac597304ed15bc0, user_id=bdd212be2fbe496f81af0687506fd363, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:14.606341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2cfa3-f085-4255-adf3-88a861ff5288 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:14.620068 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3453/6990] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:14:14 2022] PUT /identity/v3/projects/84bc40f5c68d489caac597304ed15bc0/users/bdd212be2fbe496f81af0687506fd363/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:14.624697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f3ab3-ef7e-42e5-a6f1-96d88e0149e7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:14.635523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f3ab3-ef7e-42e5-a6f1-96d88e0149e7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:14.695506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e649c-5e38-449e-ac3f-e717a0832bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:14.696145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e649c-5e38-449e-ac3f-e717a0832bda admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.696433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e649c-5e38-449e-ac3f-e717a0832bda admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.696708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e649c-5e38-449e-ac3f-e717a0832bda admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.697383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e649c-5e38-449e-ac3f-e717a0832bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:14.699048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e649c-5e38-449e-ac3f-e717a0832bda admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:14.709224 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3538/6991] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:14:14.713399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2cfa3-f085-4255-adf3-88a861ff5288 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:14.724816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2cfa3-f085-4255-adf3-88a861ff5288 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:14.730209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e77759-9db8-4367-aa9f-2f8715a5f608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:14.731500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e77759-9db8-4367-aa9f-2f8715a5f608 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.731780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e77759-9db8-4367-aa9f-2f8715a5f608 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.732106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e77759-9db8-4367-aa9f-2f8715a5f608 admin admin] PATH_INFO: `/v3/projects/84bc40f5c68d489caac597304ed15bc0/users/bdd212be2fbe496f81af0687506fd363/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.734205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e77759-9db8-4367-aa9f-2f8715a5f608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84bc40f5c68d489caac597304ed15bc0, user_id=bdd212be2fbe496f81af0687506fd363, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:14.737285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e77759-9db8-4367-aa9f-2f8715a5f608 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:14.748944 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3454/6992] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:14:14 2022] PUT /identity/v3/projects/84bc40f5c68d489caac597304ed15bc0/users/bdd212be2fbe496f81af0687506fd363/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:14.761093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23f8ea-39e1-412c-a0c3-71ce940be722 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.761509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23f8ea-39e1-412c-a0c3-71ce940be722 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.761857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23f8ea-39e1-412c-a0c3-71ce940be722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.794159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ff23f8ea-39e1-412c-a0c3-71ce940be722 None None] MFA Rules not processed for user `bdd212be2fbe496f81af0687506fd363`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:14.795406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ff23f8ea-39e1-412c-a0c3-71ce940be722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:14.883328 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3539/6993] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:14:14.884957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e77759-9db8-4367-aa9f-2f8715a5f608 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:14.901926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e77759-9db8-4367-aa9f-2f8715a5f608 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:14.907342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1824c0-c111-484c-a72e-9804b4ffd54d tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdd212be2fbe496f81af0687506fd363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bc40f5c68d489caac597304ed15bc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:14.908059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1824c0-c111-484c-a72e-9804b4ffd54d tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.908411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1824c0-c111-484c-a72e-9804b4ffd54d tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.908757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1824c0-c111-484c-a72e-9804b4ffd54d tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.909514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1824c0-c111-484c-a72e-9804b4ffd54d tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:14:14.909883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1824c0-c111-484c-a72e-9804b4ffd54d tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:14.911567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1824c0-c111-484c-a72e-9804b4ffd54d tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:14.926389 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3455/6994] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:14:14.930686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23f8ea-39e1-412c-a0c3-71ce940be722 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:14.940627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23f8ea-39e1-412c-a0c3-71ce940be722 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:14.945472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e01386-092b-4c40-9d00-2e8695d8f763 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdd212be2fbe496f81af0687506fd363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bc40f5c68d489caac597304ed15bc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:14.946186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e01386-092b-4c40-9d00-2e8695d8f763 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:14.946535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e01386-092b-4c40-9d00-2e8695d8f763 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:14.946882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e01386-092b-4c40-9d00-2e8695d8f763 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:14.947633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e01386-092b-4c40-9d00-2e8695d8f763 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:14.949691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e01386-092b-4c40-9d00-2e8695d8f763 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:14.960142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:14.981032 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3540/6995] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:14:14 2022] POST /identity/v3/projects => generated 349 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:14:14.985348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1824c0-c111-484c-a72e-9804b4ffd54d tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:14.995952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1824c0-c111-484c-a72e-9804b4ffd54d tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:15.057756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015c20d-3af0-4428-b57f-c1b1b80289c7 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdd212be2fbe496f81af0687506fd363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bc40f5c68d489caac597304ed15bc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:15.058545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015c20d-3af0-4428-b57f-c1b1b80289c7 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:15.058838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015c20d-3af0-4428-b57f-c1b1b80289c7 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:15.059490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015c20d-3af0-4428-b57f-c1b1b80289c7 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:15.060013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015c20d-3af0-4428-b57f-c1b1b80289c7 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:14:15.069237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015c20d-3af0-4428-b57f-c1b1b80289c7 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:15.072384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015c20d-3af0-4428-b57f-c1b1b80289c7 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:15.093653 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3456/6996] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:14:15.101745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e01386-092b-4c40-9d00-2e8695d8f763 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:15.104487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c1e01386-092b-4c40-9d00-2e8695d8f763 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:15.177497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e01386-092b-4c40-9d00-2e8695d8f763 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:15.182921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238e2b2-df48-4d0e-acc9-e614fb324450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:15.183865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238e2b2-df48-4d0e-acc9-e614fb324450 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:15.184299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238e2b2-df48-4d0e-acc9-e614fb324450 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:15.184722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238e2b2-df48-4d0e-acc9-e614fb324450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:15.196711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238e2b2-df48-4d0e-acc9-e614fb324450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:15.198411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238e2b2-df48-4d0e-acc9-e614fb324450 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:15.211031 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3541/6997] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:14:15 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:14:15.390637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015c20d-3af0-4428-b57f-c1b1b80289c7 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:15.401091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015c20d-3af0-4428-b57f-c1b1b80289c7 tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:15.407272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf7556-94e9-4719-b8c7-4a23399e75df tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdd212be2fbe496f81af0687506fd363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bc40f5c68d489caac597304ed15bc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:15.408125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf7556-94e9-4719-b8c7-4a23399e75df tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:15.408397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf7556-94e9-4719-b8c7-4a23399e75df tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:15.408753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf7556-94e9-4719-b8c7-4a23399e75df tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] PATH_INFO: `/v3/projects/a1faae10c5eb444990ea224a518d8283` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:15.410173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf7556-94e9-4719-b8c7-4a23399e75df tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a1faae10c5eb444990ea224a518d8283)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:15.412190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf7556-94e9-4719-b8c7-4a23399e75df tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:15.447394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:15.470704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6fcf7556-94e9-4719-b8c7-4a23399e75df tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1faae10c5eb444990ea224a518d8283', 'request_id': 'req-6fcf7556-94e9-4719-b8c7-4a23399e75df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:15.479651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6fcf7556-94e9-4719-b8c7-4a23399e75df tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1faae10c5eb444990ea224a518d8283', 'request_id': 'req-6fcf7556-94e9-4719-b8c7-4a23399e75df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:15.488062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6fcf7556-94e9-4719-b8c7-4a23399e75df tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1faae10c5eb444990ea224a518d8283', 'request_id': 'req-6fcf7556-94e9-4719-b8c7-4a23399e75df'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:15.502056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6fcf7556-94e9-4719-b8c7-4a23399e75df tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] The token cache is being invalidate because project a1faae10c5eb444990ea224a518d8283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:14:15.502376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6fcf7556-94e9-4719-b8c7-4a23399e75df tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:15.503044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6fcf7556-94e9-4719-b8c7-4a23399e75df tempest-QuotasTest-1888904020 tempest-QuotasTest-1888904020-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:15.504618 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3457/6998] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:14:15 2022] DELETE /identity/v3/projects/a1faae10c5eb444990ea224a518d8283 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:15.595076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238e2b2-df48-4d0e-acc9-e614fb324450 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:15.677129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238e2b2-df48-4d0e-acc9-e614fb324450 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:15.745653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:15.746610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:15.747043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:15.747620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] PATH_INFO: `/v3/users/349581f434e4411cb97503e4c33b9fea` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:15.748970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=349581f434e4411cb97503e4c33b9fea)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:15.751069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:15.812724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:15.825517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349581f434e4411cb97503e4c33b9fea'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:15.848089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349581f434e4411cb97503e4c33b9fea'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:15.860606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349581f434e4411cb97503e4c33b9fea'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:15.861358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349581f434e4411cb97503e4c33b9fea'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:15.862126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349581f434e4411cb97503e4c33b9fea'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:15.879738 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3542/6999] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:14:15 2022] DELETE /identity/v3/users/349581f434e4411cb97503e4c33b9fea => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:15.884180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44bf17-d3dd-45b6-b73a-8a844cbca640 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:15.884680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44bf17-d3dd-45b6-b73a-8a844cbca640 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:15.884765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44bf17-d3dd-45b6-b73a-8a844cbca640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:15.912844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2d44bf17-d3dd-45b6-b73a-8a844cbca640 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:15.914767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2d44bf17-d3dd-45b6-b73a-8a844cbca640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:16.027085 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3458/7000] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:16.039519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:16.043727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:16.163293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0351e-0a0b-4d7d-abe2-705d55fdfe87 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:16.167072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67ded9-f472-4bd7-a2d2-0cfc30eebba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:16.167703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67ded9-f472-4bd7-a2d2-0cfc30eebba1 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:16.168036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67ded9-f472-4bd7-a2d2-0cfc30eebba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:16.168315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67ded9-f472-4bd7-a2d2-0cfc30eebba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:16.179351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67ded9-f472-4bd7-a2d2-0cfc30eebba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:16.181054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67ded9-f472-4bd7-a2d2-0cfc30eebba1 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:16.203516 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3543/7001] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:14:16.788921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44bf17-d3dd-45b6-b73a-8a844cbca640 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:16.790953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2d44bf17-d3dd-45b6-b73a-8a844cbca640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:16.827398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44bf17-d3dd-45b6-b73a-8a844cbca640 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:16.834259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:16.834926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:16.835216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:16.835501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] PATH_INFO: `/v3/projects/3398a7c60ec644fcaa72f63b18975809` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:16.837410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3398a7c60ec644fcaa72f63b18975809)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:16.839584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:16.930023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3398a7c60ec644fcaa72f63b18975809', 'request_id': 'req-73e7eab0-c59d-4c8b-9323-59475f4a0057'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:16.941923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3398a7c60ec644fcaa72f63b18975809', 'request_id': 'req-73e7eab0-c59d-4c8b-9323-59475f4a0057'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:16.951175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3398a7c60ec644fcaa72f63b18975809', 'request_id': 'req-73e7eab0-c59d-4c8b-9323-59475f4a0057'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:16.964050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] The token cache is being invalidate because project 3398a7c60ec644fcaa72f63b18975809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:14:16.964453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:16.965864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:16.967521 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3459/7002] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:14:16 2022] DELETE /identity/v3/projects/3398a7c60ec644fcaa72f63b18975809 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:16.972374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67ded9-f472-4bd7-a2d2-0cfc30eebba1 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:16.983688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67ded9-f472-4bd7-a2d2-0cfc30eebba1 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:17.065603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756b1a3-3cd0-4a3d-a150-8559849197ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:17.066420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756b1a3-3cd0-4a3d-a150-8559849197ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:17.066798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756b1a3-3cd0-4a3d-a150-8559849197ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:17.067063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756b1a3-3cd0-4a3d-a150-8559849197ea admin admin] PATH_INFO: `/v3/users/bdd212be2fbe496f81af0687506fd363` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:17.068472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756b1a3-3cd0-4a3d-a150-8559849197ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd212be2fbe496f81af0687506fd363)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:17.070864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756b1a3-3cd0-4a3d-a150-8559849197ea admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:17.162719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9756b1a3-3cd0-4a3d-a150-8559849197ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd212be2fbe496f81af0687506fd363'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:17.177887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:17.180537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9756b1a3-3cd0-4a3d-a150-8559849197ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd212be2fbe496f81af0687506fd363'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:17.198852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9756b1a3-3cd0-4a3d-a150-8559849197ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd212be2fbe496f81af0687506fd363'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:17.200120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9756b1a3-3cd0-4a3d-a150-8559849197ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd212be2fbe496f81af0687506fd363'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:17.201154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9756b1a3-3cd0-4a3d-a150-8559849197ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd212be2fbe496f81af0687506fd363'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:17.232252 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3544/7003] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:14:16 2022] DELETE /identity/v3/users/bdd212be2fbe496f81af0687506fd363 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:17.576437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:17.578042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:17.595750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7eab0-c59d-4c8b-9323-59475f4a0057 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:17.639813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:17.677573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:17.678230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:17.678521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:17.678801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889 admin admin] PATH_INFO: `/v3/projects/84bc40f5c68d489caac597304ed15bc0` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:17.679988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84bc40f5c68d489caac597304ed15bc0)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:17.682012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:17.783129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84bc40f5c68d489caac597304ed15bc0', 'request_id': 'req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:17.793569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84bc40f5c68d489caac597304ed15bc0', 'request_id': 'req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:17.801192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84bc40f5c68d489caac597304ed15bc0', 'request_id': 'req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:17.813664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889 admin admin] The token cache is being invalidate because project 84bc40f5c68d489caac597304ed15bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:14:17.814356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:17.815254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cb9a5d25-22af-4ddd-9051-7e4ebcb21889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:17.821697 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3460/7004] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:14:17 2022] DELETE /identity/v3/projects/84bc40f5c68d489caac597304ed15bc0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:17.823490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95201f-42e3-4f2f-b36a-8047f590dae5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:17.823818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95201f-42e3-4f2f-b36a-8047f590dae5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:17.824201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95201f-42e3-4f2f-b36a-8047f590dae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:17.860999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8b95201f-42e3-4f2f-b36a-8047f590dae5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:17.862660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8b95201f-42e3-4f2f-b36a-8047f590dae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:17.946347 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3545/7005] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:17.948004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48cfc1-ba75-45f6-81f5-367f57db0816 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:17.948439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48cfc1-ba75-45f6-81f5-367f57db0816 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:17.948595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48cfc1-ba75-45f6-81f5-367f57db0816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:17.970747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5b48cfc1-ba75-45f6-81f5-367f57db0816 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:17.972375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5b48cfc1-ba75-45f6-81f5-367f57db0816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:17.981298 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3461/7006] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:17.985700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95201f-42e3-4f2f-b36a-8047f590dae5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:17.996958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95201f-42e3-4f2f-b36a-8047f590dae5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:18.006636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0677e-98b8-4f0d-88ea-7d5e5e9e43cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:18.007590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0677e-98b8-4f0d-88ea-7d5e5e9e43cf admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:18.008142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0677e-98b8-4f0d-88ea-7d5e5e9e43cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:18.008424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0677e-98b8-4f0d-88ea-7d5e5e9e43cf admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:18.009158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0677e-98b8-4f0d-88ea-7d5e5e9e43cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:14:18.009472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0677e-98b8-4f0d-88ea-7d5e5e9e43cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:18.011254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0677e-98b8-4f0d-88ea-7d5e5e9e43cf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:18.025945 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3546/7007] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:14:18.031675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48cfc1-ba75-45f6-81f5-367f57db0816 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:18.043731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48cfc1-ba75-45f6-81f5-367f57db0816 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:18.049215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6e6e1-98cf-4b3a-881c-1e52217d1075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:18.049919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6e6e1-98cf-4b3a-881c-1e52217d1075 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:18.050282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6e6e1-98cf-4b3a-881c-1e52217d1075 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:18.050634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6e6e1-98cf-4b3a-881c-1e52217d1075 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:18.051494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6e6e1-98cf-4b3a-881c-1e52217d1075 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:18.053808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6e6e1-98cf-4b3a-881c-1e52217d1075 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:18.098417 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3462/7008] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:14:18 2022] POST /identity/v3/projects => generated 397 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:14:18.103292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0677e-98b8-4f0d-88ea-7d5e5e9e43cf admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:18.115348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0677e-98b8-4f0d-88ea-7d5e5e9e43cf admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:18.196540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f92af-9611-49ca-a644-1d92d82f1bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:18.197287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f92af-9611-49ca-a644-1d92d82f1bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:18.197681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f92af-9611-49ca-a644-1d92d82f1bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:18.198079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f92af-9611-49ca-a644-1d92d82f1bd5 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:18.199265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f92af-9611-49ca-a644-1d92d82f1bd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:18.201155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f92af-9611-49ca-a644-1d92d82f1bd5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:18.229605 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3547/7009] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:14:18 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:14:18.232417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6e6e1-98cf-4b3a-881c-1e52217d1075 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:18.248972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6e6e1-98cf-4b3a-881c-1e52217d1075 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:18.255898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0b13c-add6-493b-93d4-f470845025a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:18.257043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0b13c-add6-493b-93d4-f470845025a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:18.257330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0b13c-add6-493b-93d4-f470845025a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:18.257612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0b13c-add6-493b-93d4-f470845025a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:18.258294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0b13c-add6-493b-93d4-f470845025a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:18.259902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0b13c-add6-493b-93d4-f470845025a9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:18.278060 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3463/7010] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:14:18.278950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f92af-9611-49ca-a644-1d92d82f1bd5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:18.291233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f92af-9611-49ca-a644-1d92d82f1bd5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:18.296025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0e69b-b579-4693-982a-6af378277d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:18.296718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0e69b-b579-4693-982a-6af378277d1c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:18.297153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0e69b-b579-4693-982a-6af378277d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:18.297782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0e69b-b579-4693-982a-6af378277d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:18.299050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0e69b-b579-4693-982a-6af378277d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:18.300767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0e69b-b579-4693-982a-6af378277d1c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:18.321724 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3548/7011] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:14:18.325081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0b13c-add6-493b-93d4-f470845025a9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:18.341781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0b13c-add6-493b-93d4-f470845025a9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:18.348618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276563dc-9d8b-48aa-aacf-8cd82dcb0034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:18.349484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276563dc-9d8b-48aa-aacf-8cd82dcb0034 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:18.349861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276563dc-9d8b-48aa-aacf-8cd82dcb0034 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:18.350247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276563dc-9d8b-48aa-aacf-8cd82dcb0034 admin admin] PATH_INFO: `/v3/projects/07b5d9c71416465cbf710d574e82ea6c/users/40d746e32b8540ad9b5cb474411fc60f/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:18.372401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276563dc-9d8b-48aa-aacf-8cd82dcb0034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b5d9c71416465cbf710d574e82ea6c, user_id=40d746e32b8540ad9b5cb474411fc60f, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:18.376446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276563dc-9d8b-48aa-aacf-8cd82dcb0034 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:18.394577 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3464/7012] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:14:18 2022] PUT /identity/v3/projects/07b5d9c71416465cbf710d574e82ea6c/users/40d746e32b8540ad9b5cb474411fc60f/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:18.406680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecc7bf-7a03-4879-b30b-e0eead8d5e17 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:18.406680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecc7bf-7a03-4879-b30b-e0eead8d5e17 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:18.407004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecc7bf-7a03-4879-b30b-e0eead8d5e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:18.437004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4cecc7bf-7a03-4879-b30b-e0eead8d5e17 None None] MFA Rules not processed for user `40d746e32b8540ad9b5cb474411fc60f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:18.438290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4cecc7bf-7a03-4879-b30b-e0eead8d5e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:18.497903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:18.555501 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3549/7013] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:18 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:18.566165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276563dc-9d8b-48aa-aacf-8cd82dcb0034 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:18.568147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-276563dc-9d8b-48aa-aacf-8cd82dcb0034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:18.660876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276563dc-9d8b-48aa-aacf-8cd82dcb0034 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:18.666677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b52ae-f204-480a-b4f4-e02d2667d998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:18.667485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b52ae-f204-480a-b4f4-e02d2667d998 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:18.668789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b52ae-f204-480a-b4f4-e02d2667d998 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:18.669129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b52ae-f204-480a-b4f4-e02d2667d998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:18.678341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b52ae-f204-480a-b4f4-e02d2667d998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:18.679869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b52ae-f204-480a-b4f4-e02d2667d998 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:18.698324 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3465/7014] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:14:18 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:14:26.958586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0751e-c4e6-4386-a714-30777272e8bc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:26.959071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0751e-c4e6-4386-a714-30777272e8bc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:26.959177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0751e-c4e6-4386-a714-30777272e8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:26.986624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2ac0751e-c4e6-4386-a714-30777272e8bc None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:26.988182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2ac0751e-c4e6-4386-a714-30777272e8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:27.074056 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3550/7015] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:28.689902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:31.991806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc4f7b-aa32-4d74-ba03-f5e3d89b13f7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:31.992386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc4f7b-aa32-4d74-ba03-f5e3d89b13f7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:31.992386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc4f7b-aa32-4d74-ba03-f5e3d89b13f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:32.012743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-86bc4f7b-aa32-4d74-ba03-f5e3d89b13f7 None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:32.014023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-86bc4f7b-aa32-4d74-ba03-f5e3d89b13f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:32.022116 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3466/7016] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:14:32.933373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128aaa72-5c89-4ecf-ad95-d852acdbc2e9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:32.933868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128aaa72-5c89-4ecf-ad95-d852acdbc2e9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:32.933955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128aaa72-5c89-4ecf-ad95-d852acdbc2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:32.955199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-128aaa72-5c89-4ecf-ad95-d852acdbc2e9 None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:32.956430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-128aaa72-5c89-4ecf-ad95-d852acdbc2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:32.966426 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3551/7017] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:33.507467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fe8b8-e1c9-4f71-8c0c-1efbfca16c75 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:33.508339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fe8b8-e1c9-4f71-8c0c-1efbfca16c75 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:33.508768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fe8b8-e1c9-4f71-8c0c-1efbfca16c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:33.531487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-398fe8b8-e1c9-4f71-8c0c-1efbfca16c75 None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:33.533146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-398fe8b8-e1c9-4f71-8c0c-1efbfca16c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:33.545495 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3467/7018] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:34.167661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251bc62-9a6f-426a-b1e8-5ff97e7b33f8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:34.168686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251bc62-9a6f-426a-b1e8-5ff97e7b33f8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:34.169309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251bc62-9a6f-426a-b1e8-5ff97e7b33f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:34.195817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8251bc62-9a6f-426a-b1e8-5ff97e7b33f8 None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:34.198375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8251bc62-9a6f-426a-b1e8-5ff97e7b33f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:34.209315 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3552/7019] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:14:34.852707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded03f5-a65d-43cc-ab23-5f53ea04c8e5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:34.853183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded03f5-a65d-43cc-ab23-5f53ea04c8e5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:34.853535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded03f5-a65d-43cc-ab23-5f53ea04c8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:34.878370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fded03f5-a65d-43cc-ab23-5f53ea04c8e5 None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:34.879764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fded03f5-a65d-43cc-ab23-5f53ea04c8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:34.888129 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3468/7020] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:35.148112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251bc62-9a6f-426a-b1e8-5ff97e7b33f8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:35.266381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251bc62-9a6f-426a-b1e8-5ff97e7b33f8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:35.272144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82006f94-8a88-4665-aa2e-1433095b2f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:35.273048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82006f94-8a88-4665-aa2e-1433095b2f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:35.273510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82006f94-8a88-4665-aa2e-1433095b2f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:35.273962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82006f94-8a88-4665-aa2e-1433095b2f2b admin admin] PATH_INFO: `/v3/users/40d746e32b8540ad9b5cb474411fc60f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:35.275364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82006f94-8a88-4665-aa2e-1433095b2f2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=40d746e32b8540ad9b5cb474411fc60f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:35.277430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82006f94-8a88-4665-aa2e-1433095b2f2b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:35.342077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-82006f94-8a88-4665-aa2e-1433095b2f2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40d746e32b8540ad9b5cb474411fc60f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:35.356771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-82006f94-8a88-4665-aa2e-1433095b2f2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40d746e32b8540ad9b5cb474411fc60f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:35.361644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:35.365534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-82006f94-8a88-4665-aa2e-1433095b2f2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40d746e32b8540ad9b5cb474411fc60f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:35.366477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-82006f94-8a88-4665-aa2e-1433095b2f2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40d746e32b8540ad9b5cb474411fc60f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:35.367371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-82006f94-8a88-4665-aa2e-1433095b2f2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40d746e32b8540ad9b5cb474411fc60f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:35.388994 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3553/7021] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:14:35 2022] DELETE /identity/v3/users/40d746e32b8540ad9b5cb474411fc60f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:35.392290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119683e-317f-443d-82bf-1a7e0c3e3dce None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:35.392579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119683e-317f-443d-82bf-1a7e0c3e3dce None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:35.393820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119683e-317f-443d-82bf-1a7e0c3e3dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:35.415550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2119683e-317f-443d-82bf-1a7e0c3e3dce None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:35.416991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2119683e-317f-443d-82bf-1a7e0c3e3dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:35.466721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a8eea-94df-462c-b819-1d5dd291f502 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:35.467055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a8eea-94df-462c-b819-1d5dd291f502 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:35.467308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a8eea-94df-462c-b819-1d5dd291f502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:35.495051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2a5a8eea-94df-462c-b819-1d5dd291f502 None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:35.497108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2a5a8eea-94df-462c-b819-1d5dd291f502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:35.516448 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3469/7022] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:35.527170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119683e-317f-443d-82bf-1a7e0c3e3dce None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:35.529369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2119683e-317f-443d-82bf-1a7e0c3e3dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:35.564934 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3554/7023] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:14:35.613587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119683e-317f-443d-82bf-1a7e0c3e3dce None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:35.617882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c2ee6-3c9b-4ac7-95cb-799da32a540e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:35.618679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c2ee6-3c9b-4ac7-95cb-799da32a540e service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:35.619273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c2ee6-3c9b-4ac7-95cb-799da32a540e service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:35.625352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c2ee6-3c9b-4ac7-95cb-799da32a540e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:35.637007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c2ee6-3c9b-4ac7-95cb-799da32a540e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:35.638763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c2ee6-3c9b-4ac7-95cb-799da32a540e service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:35.660424 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3470/7024] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:14:36.033718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a8eea-94df-462c-b819-1d5dd291f502 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:36.035245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2a5a8eea-94df-462c-b819-1d5dd291f502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:36.052149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a8eea-94df-462c-b819-1d5dd291f502 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:36.057400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677d7ce-f12b-4595-8d17-201810030df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:36.058471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677d7ce-f12b-4595-8d17-201810030df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:36.059084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677d7ce-f12b-4595-8d17-201810030df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:36.059666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677d7ce-f12b-4595-8d17-201810030df0 admin admin] PATH_INFO: `/v3/projects/07b5d9c71416465cbf710d574e82ea6c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:36.062208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677d7ce-f12b-4595-8d17-201810030df0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b5d9c71416465cbf710d574e82ea6c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:36.065256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677d7ce-f12b-4595-8d17-201810030df0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:36.121723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f677d7ce-f12b-4595-8d17-201810030df0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b5d9c71416465cbf710d574e82ea6c', 'request_id': 'req-f677d7ce-f12b-4595-8d17-201810030df0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:36.140734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f677d7ce-f12b-4595-8d17-201810030df0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b5d9c71416465cbf710d574e82ea6c', 'request_id': 'req-f677d7ce-f12b-4595-8d17-201810030df0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:36.150030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f677d7ce-f12b-4595-8d17-201810030df0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b5d9c71416465cbf710d574e82ea6c', 'request_id': 'req-f677d7ce-f12b-4595-8d17-201810030df0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:36.165703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f677d7ce-f12b-4595-8d17-201810030df0 admin admin] The token cache is being invalidate because project 07b5d9c71416465cbf710d574e82ea6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:14:36.166511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f677d7ce-f12b-4595-8d17-201810030df0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:36.167976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f677d7ce-f12b-4595-8d17-201810030df0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:14:36.170509 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3555/7025] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:14:36 2022] DELETE /identity/v3/projects/07b5d9c71416465cbf710d574e82ea6c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:36.175477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a228cbe-24f8-4180-bee0-4cc8a3d6fce0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:36.177368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a228cbe-24f8-4180-bee0-4cc8a3d6fce0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:36.177945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a228cbe-24f8-4180-bee0-4cc8a3d6fce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:36.203006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1a228cbe-24f8-4180-bee0-4cc8a3d6fce0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:36.204594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1a228cbe-24f8-4180-bee0-4cc8a3d6fce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:36.285849 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3471/7026] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:14:36.289929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4839f7-8964-4545-8911-ec2c6f41344a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:36.290804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4839f7-8964-4545-8911-ec2c6f41344a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:36.291336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4839f7-8964-4545-8911-ec2c6f41344a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:36.316161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cc4839f7-8964-4545-8911-ec2c6f41344a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:36.317663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cc4839f7-8964-4545-8911-ec2c6f41344a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:36.330732 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3556/7027] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:36.331647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a228cbe-24f8-4180-bee0-4cc8a3d6fce0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:36.346710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a228cbe-24f8-4180-bee0-4cc8a3d6fce0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:36.354294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49508950-5b84-4de2-b2a4-446f484bec0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:36.356032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49508950-5b84-4de2-b2a4-446f484bec0f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:36.356543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49508950-5b84-4de2-b2a4-446f484bec0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:36.357159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49508950-5b84-4de2-b2a4-446f484bec0f admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:36.358069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49508950-5b84-4de2-b2a4-446f484bec0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:14:36.358642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49508950-5b84-4de2-b2a4-446f484bec0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:36.360791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49508950-5b84-4de2-b2a4-446f484bec0f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:36.377266 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3472/7028] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:14:36.382353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4839f7-8964-4545-8911-ec2c6f41344a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:36.394861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4839f7-8964-4545-8911-ec2c6f41344a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:36.405267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93b089-f549-46b5-8015-eb1c640a0b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:36.406995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93b089-f549-46b5-8015-eb1c640a0b95 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:36.407501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93b089-f549-46b5-8015-eb1c640a0b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:36.407986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93b089-f549-46b5-8015-eb1c640a0b95 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:36.409599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93b089-f549-46b5-8015-eb1c640a0b95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:36.412178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93b089-f549-46b5-8015-eb1c640a0b95 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:36.450713 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3557/7029] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:14:36 2022] POST /identity/v3/projects => generated 357 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:14:36.455098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49508950-5b84-4de2-b2a4-446f484bec0f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:36.469940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49508950-5b84-4de2-b2a4-446f484bec0f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:36.484931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:36.533663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcea04-286f-4cdd-a184-c04aee0ddd28 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:36.533962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcea04-286f-4cdd-a184-c04aee0ddd28 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:36.534251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcea04-286f-4cdd-a184-c04aee0ddd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:36.559241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f6bcea04-286f-4cdd-a184-c04aee0ddd28 None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:36.560085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b319e0-8fee-46d3-9914-dbc640538046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:36.560470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f6bcea04-286f-4cdd-a184-c04aee0ddd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:36.560710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b319e0-8fee-46d3-9914-dbc640538046 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:36.561145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b319e0-8fee-46d3-9914-dbc640538046 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:36.561433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b319e0-8fee-46d3-9914-dbc640538046 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:36.562270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b319e0-8fee-46d3-9914-dbc640538046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:36.564385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b319e0-8fee-46d3-9914-dbc640538046 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:36.594279 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3473/7030] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:14:36 2022] POST /identity/v3/users => generated 279 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:14:36.597969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b319e0-8fee-46d3-9914-dbc640538046 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:36.609677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b319e0-8fee-46d3-9914-dbc640538046 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:36.614861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb35291-e812-4274-bd81-34b9ecf729d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:36.615486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb35291-e812-4274-bd81-34b9ecf729d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:36.615772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb35291-e812-4274-bd81-34b9ecf729d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:36.616050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb35291-e812-4274-bd81-34b9ecf729d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:36.616837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb35291-e812-4274-bd81-34b9ecf729d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:36.618545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb35291-e812-4274-bd81-34b9ecf729d0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:36.631235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:36.636801 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3474/7031] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:14:36.643000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb35291-e812-4274-bd81-34b9ecf729d0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:36.659466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb35291-e812-4274-bd81-34b9ecf729d0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:36.664620 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3558/7032] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:36.672223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e40c80-6ebb-4e6b-8069-f2784a4d4799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:36.673217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e40c80-6ebb-4e6b-8069-f2784a4d4799 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:36.673595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e40c80-6ebb-4e6b-8069-f2784a4d4799 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:36.674072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e40c80-6ebb-4e6b-8069-f2784a4d4799 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:36.674807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e40c80-6ebb-4e6b-8069-f2784a4d4799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:36.676490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e40c80-6ebb-4e6b-8069-f2784a4d4799 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:36.692810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3475/7033] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:14:36.697154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcea04-286f-4cdd-a184-c04aee0ddd28 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:36.709281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcea04-286f-4cdd-a184-c04aee0ddd28 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:36.714139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b188ea5-fd77-4636-8747-7755f82ca96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:36.715378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b188ea5-fd77-4636-8747-7755f82ca96d admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:36.715666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b188ea5-fd77-4636-8747-7755f82ca96d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:36.715949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b188ea5-fd77-4636-8747-7755f82ca96d admin admin] PATH_INFO: `/v3/projects/0afa18b8fafb40c5bbfbc279462da6e2/users/5d669705cb9e494eb8c135ad8a01b249/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:36.744217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b188ea5-fd77-4636-8747-7755f82ca96d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0afa18b8fafb40c5bbfbc279462da6e2, user_id=5d669705cb9e494eb8c135ad8a01b249, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:36.747343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b188ea5-fd77-4636-8747-7755f82ca96d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:36.762310 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3559/7034] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:14:36 2022] PUT /identity/v3/projects/0afa18b8fafb40c5bbfbc279462da6e2/users/5d669705cb9e494eb8c135ad8a01b249/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:14:36.772954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8e43a-ae1b-4c23-ad66-88980c9f826f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:36.773253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8e43a-ae1b-4c23-ad66-88980c9f826f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:36.773542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8e43a-ae1b-4c23-ad66-88980c9f826f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:36.816006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-51d8e43a-ae1b-4c23-ad66-88980c9f826f None None] MFA Rules not processed for user `5d669705cb9e494eb8c135ad8a01b249`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:36.817329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-51d8e43a-ae1b-4c23-ad66-88980c9f826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:36.932096 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3476/7035] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:36.961640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b188ea5-fd77-4636-8747-7755f82ca96d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:36.966607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8b188ea5-fd77-4636-8747-7755f82ca96d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:37.050854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b188ea5-fd77-4636-8747-7755f82ca96d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:37.054714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f228761-6b1e-43c9-9943-45c58d791a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:37.055632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f228761-6b1e-43c9-9943-45c58d791a0a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:37.055923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f228761-6b1e-43c9-9943-45c58d791a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:37.056205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f228761-6b1e-43c9-9943-45c58d791a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:37.068105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f228761-6b1e-43c9-9943-45c58d791a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:37.069677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f228761-6b1e-43c9-9943-45c58d791a0a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:37.080611 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3560/7036] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:14:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:14:42.163645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686a1a9-aa98-4307-b2cd-9d185d982ca1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:42.164448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686a1a9-aa98-4307-b2cd-9d185d982ca1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:42.164566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686a1a9-aa98-4307-b2cd-9d185d982ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:42.199896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f686a1a9-aa98-4307-b2cd-9d185d982ca1 None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:42.201407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f686a1a9-aa98-4307-b2cd-9d185d982ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:42.268590 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3477/7037] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:43.176240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a76c5-dff4-4bfa-b4ea-d95a3847bd65 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:43.176802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a76c5-dff4-4bfa-b4ea-d95a3847bd65 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:43.176802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a76c5-dff4-4bfa-b4ea-d95a3847bd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:43.209396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-eb2a76c5-dff4-4bfa-b4ea-d95a3847bd65 None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:43.216385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-eb2a76c5-dff4-4bfa-b4ea-d95a3847bd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:43.224769 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3561/7038] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:52.259097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:53.200298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:14:54.238528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e30908-079b-4036-a854-549d95bfa2c4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:54.239840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e30908-079b-4036-a854-549d95bfa2c4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:54.240493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e30908-079b-4036-a854-549d95bfa2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:54.271610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-89e30908-079b-4036-a854-549d95bfa2c4 None None] MFA Rules not processed for user `ec4d4dfdda4f4f86a6cffadc35d9e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:14:54.272959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-89e30908-079b-4036-a854-549d95bfa2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:54.285911 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3478/7039] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:14:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:14:58.958763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a76c5-dff4-4bfa-b4ea-d95a3847bd65 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:14:58.963310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-eb2a76c5-dff4-4bfa-b4ea-d95a3847bd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:58.980777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a76c5-dff4-4bfa-b4ea-d95a3847bd65 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:14:58.988107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ba0f6-2422-4507-9ac6-d68d0562c491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:14:58.988996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba0f6-2422-4507-9ac6-d68d0562c491 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:14:58.989433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba0f6-2422-4507-9ac6-d68d0562c491 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:14:58.989860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba0f6-2422-4507-9ac6-d68d0562c491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:14:58.992043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-df3ba0f6-2422-4507-9ac6-d68d0562c491 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:14:59.008216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ba0f6-2422-4507-9ac6-d68d0562c491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:14:59.010035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ba0f6-2422-4507-9ac6-d68d0562c491 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:14:59.022287 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3562/7040] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:14:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:00.344383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e30908-079b-4036-a854-549d95bfa2c4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:00.346602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-89e30908-079b-4036-a854-549d95bfa2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:00.427922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e30908-079b-4036-a854-549d95bfa2c4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:00.432353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:00.433648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:00.433941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:00.434237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] PATH_INFO: `/v3/users/a02def928c42450b9bca99d965a5c1c7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:00.435374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a02def928c42450b9bca99d965a5c1c7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:00.437329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:00.483814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a02def928c42450b9bca99d965a5c1c7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:00.496322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a02def928c42450b9bca99d965a5c1c7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:00.497047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a02def928c42450b9bca99d965a5c1c7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:00.505132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a02def928c42450b9bca99d965a5c1c7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:00.514012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a02def928c42450b9bca99d965a5c1c7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:00.520256 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3479/7041] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:00 2022] DELETE /identity/v3/users/a02def928c42450b9bca99d965a5c1c7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:00.524408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377583d-0e23-4110-af32-f9ce35df9fa5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:00.524868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377583d-0e23-4110-af32-f9ce35df9fa5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:00.525015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377583d-0e23-4110-af32-f9ce35df9fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:00.544815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3377583d-0e23-4110-af32-f9ce35df9fa5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:00.546066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3377583d-0e23-4110-af32-f9ce35df9fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:00.605941 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3563/7042] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:00.615515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:00.617060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:00.709310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5f995-d9cd-46f2-86b9-867ae370c70b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:00.713521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f21000-d2a6-4e7d-8200-54b09735a73e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:00.714730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f21000-d2a6-4e7d-8200-54b09735a73e service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:00.715017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f21000-d2a6-4e7d-8200-54b09735a73e service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:00.715290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f21000-d2a6-4e7d-8200-54b09735a73e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:00.725456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f21000-d2a6-4e7d-8200-54b09735a73e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:00.727044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f21000-d2a6-4e7d-8200-54b09735a73e service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:00.738274 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3480/7043] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:00.924029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377583d-0e23-4110-af32-f9ce35df9fa5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:00.925754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3377583d-0e23-4110-af32-f9ce35df9fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:00.941591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377583d-0e23-4110-af32-f9ce35df9fa5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:00.947239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de1dda-4b69-4052-acf3-8474c96c722e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:00.948276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de1dda-4b69-4052-acf3-8474c96c722e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:00.948759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de1dda-4b69-4052-acf3-8474c96c722e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:00.949319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de1dda-4b69-4052-acf3-8474c96c722e admin admin] PATH_INFO: `/v3/projects/374831c019614361a782e1edb7ea5324` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:00.950614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de1dda-4b69-4052-acf3-8474c96c722e admin admin] RBAC: Authorizing `identity:delete_project(project_id=374831c019614361a782e1edb7ea5324)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:00.952713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de1dda-4b69-4052-acf3-8474c96c722e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:01.029917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-36de1dda-4b69-4052-acf3-8474c96c722e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '374831c019614361a782e1edb7ea5324', 'request_id': 'req-36de1dda-4b69-4052-acf3-8474c96c722e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:01.039634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-36de1dda-4b69-4052-acf3-8474c96c722e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '374831c019614361a782e1edb7ea5324', 'request_id': 'req-36de1dda-4b69-4052-acf3-8474c96c722e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:01.047770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-36de1dda-4b69-4052-acf3-8474c96c722e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '374831c019614361a782e1edb7ea5324', 'request_id': 'req-36de1dda-4b69-4052-acf3-8474c96c722e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:01.062973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-36de1dda-4b69-4052-acf3-8474c96c722e admin admin] The token cache is being invalidate because project 374831c019614361a782e1edb7ea5324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:01.063236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-36de1dda-4b69-4052-acf3-8474c96c722e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:01.063913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-36de1dda-4b69-4052-acf3-8474c96c722e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:01.066018 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3564/7044] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:00 2022] DELETE /identity/v3/projects/374831c019614361a782e1edb7ea5324 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:01.071046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304cbec-8eec-4650-83c7-a8dcef87c797 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:01.071560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304cbec-8eec-4650-83c7-a8dcef87c797 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:01.071966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304cbec-8eec-4650-83c7-a8dcef87c797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:01.097959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1304cbec-8eec-4650-83c7-a8dcef87c797 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:01.099387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1304cbec-8eec-4650-83c7-a8dcef87c797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:01.113837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:01.193102 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3481/7045] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:01.194618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a1ef9-3831-4efe-bb56-99be6549db8e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:01.195057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a1ef9-3831-4efe-bb56-99be6549db8e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:01.195443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a1ef9-3831-4efe-bb56-99be6549db8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:01.218550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-407a1ef9-3831-4efe-bb56-99be6549db8e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:01.220411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-407a1ef9-3831-4efe-bb56-99be6549db8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:01.233828 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3565/7046] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:01.235289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304cbec-8eec-4650-83c7-a8dcef87c797 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:01.247404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304cbec-8eec-4650-83c7-a8dcef87c797 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:01.252526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9268d67-12fd-4b77-ab7a-7d2de23985e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:01.253194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9268d67-12fd-4b77-ab7a-7d2de23985e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:01.253481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9268d67-12fd-4b77-ab7a-7d2de23985e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:01.253757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9268d67-12fd-4b77-ab7a-7d2de23985e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:01.254452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9268d67-12fd-4b77-ab7a-7d2de23985e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:01.254760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9268d67-12fd-4b77-ab7a-7d2de23985e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:01.256389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9268d67-12fd-4b77-ab7a-7d2de23985e4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:01.273274 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3482/7047] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:01.275022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a1ef9-3831-4efe-bb56-99be6549db8e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:01.286326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a1ef9-3831-4efe-bb56-99be6549db8e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:01.293081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bae77b-5e7c-4eb4-a94a-a3c280611639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:01.293838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bae77b-5e7c-4eb4-a94a-a3c280611639 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:01.294379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bae77b-5e7c-4eb4-a94a-a3c280611639 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:01.294763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bae77b-5e7c-4eb4-a94a-a3c280611639 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:01.295791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bae77b-5e7c-4eb4-a94a-a3c280611639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:01.298516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bae77b-5e7c-4eb4-a94a-a3c280611639 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:01.341421 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3566/7048] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:01 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:01.346086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9268d67-12fd-4b77-ab7a-7d2de23985e4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:01.362692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9268d67-12fd-4b77-ab7a-7d2de23985e4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:01.443462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412d230-5641-4b61-84f1-5ea955884a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:01.444214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d230-5641-4b61-84f1-5ea955884a45 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:01.444783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d230-5641-4b61-84f1-5ea955884a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:01.445129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d230-5641-4b61-84f1-5ea955884a45 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:01.445779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412d230-5641-4b61-84f1-5ea955884a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:01.447737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412d230-5641-4b61-84f1-5ea955884a45 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:01.484037 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3483/7049] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:01 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:15:01.485331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bae77b-5e7c-4eb4-a94a-a3c280611639 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:01.497349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bae77b-5e7c-4eb4-a94a-a3c280611639 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:01.502279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561c03a-f210-461a-aa64-9772ca71af6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:01.502897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561c03a-f210-461a-aa64-9772ca71af6d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:01.503186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561c03a-f210-461a-aa64-9772ca71af6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:01.503465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561c03a-f210-461a-aa64-9772ca71af6d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:01.504119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561c03a-f210-461a-aa64-9772ca71af6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:01.505773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561c03a-f210-461a-aa64-9772ca71af6d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:01.525146 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3567/7050] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:15:01.525842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412d230-5641-4b61-84f1-5ea955884a45 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:01.536080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412d230-5641-4b61-84f1-5ea955884a45 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:01.541078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee8b5e-4288-4285-93d4-df4c4261dfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:01.541657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee8b5e-4288-4285-93d4-df4c4261dfca admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:01.542019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee8b5e-4288-4285-93d4-df4c4261dfca admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:01.542853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee8b5e-4288-4285-93d4-df4c4261dfca admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:01.543539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee8b5e-4288-4285-93d4-df4c4261dfca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:01.545681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee8b5e-4288-4285-93d4-df4c4261dfca admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:01.558453 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3484/7051] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:01.562908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561c03a-f210-461a-aa64-9772ca71af6d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:01.575148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561c03a-f210-461a-aa64-9772ca71af6d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:01.580841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c819f58-7e0c-4ad5-ba12-a9d4dd499ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:01.581746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c819f58-7e0c-4ad5-ba12-a9d4dd499ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:01.582228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c819f58-7e0c-4ad5-ba12-a9d4dd499ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:01.582519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c819f58-7e0c-4ad5-ba12-a9d4dd499ccb admin admin] PATH_INFO: `/v3/projects/ccdf783fce74452d8881cb32ccc56bc4/users/6a1e5e68dc3942a59dfb300b1db16043/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:01.599850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c819f58-7e0c-4ad5-ba12-a9d4dd499ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccdf783fce74452d8881cb32ccc56bc4, user_id=6a1e5e68dc3942a59dfb300b1db16043, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:01.602704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c819f58-7e0c-4ad5-ba12-a9d4dd499ccb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:01.618456 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3568/7052] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:01 2022] PUT /identity/v3/projects/ccdf783fce74452d8881cb32ccc56bc4/users/6a1e5e68dc3942a59dfb300b1db16043/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:01.629063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05724dd-4e60-4d8d-a3f2-53013f1885a2 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:01.629174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05724dd-4e60-4d8d-a3f2-53013f1885a2 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:01.629430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05724dd-4e60-4d8d-a3f2-53013f1885a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:01.660238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a05724dd-4e60-4d8d-a3f2-53013f1885a2 None None] MFA Rules not processed for user `6a1e5e68dc3942a59dfb300b1db16043`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:01.661501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a05724dd-4e60-4d8d-a3f2-53013f1885a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:01.729987 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3485/7053] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:01.740345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c819f58-7e0c-4ad5-ba12-a9d4dd499ccb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:01.742112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0c819f58-7e0c-4ad5-ba12-a9d4dd499ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:01.752209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:01.804411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c819f58-7e0c-4ad5-ba12-a9d4dd499ccb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:01.807956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433c604-fbb7-4a8c-ab68-c284b71032d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:01.808568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433c604-fbb7-4a8c-ab68-c284b71032d8 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:01.808856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433c604-fbb7-4a8c-ab68-c284b71032d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:01.809168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433c604-fbb7-4a8c-ab68-c284b71032d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:01.817941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433c604-fbb7-4a8c-ab68-c284b71032d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:01.819390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433c604-fbb7-4a8c-ab68-c284b71032d8 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:01.829286 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3569/7054] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:15:02.159026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05724dd-4e60-4d8d-a3f2-53013f1885a2 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:02.171688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:02.235330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05724dd-4e60-4d8d-a3f2-53013f1885a2 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:02.240468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:02.241154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:02.241567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:02.241854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] PATH_INFO: `/v3/users/6a1e5e68dc3942a59dfb300b1db16043` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:02.243029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a1e5e68dc3942a59dfb300b1db16043)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:02.245012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:02.308176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a1e5e68dc3942a59dfb300b1db16043'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:02.322027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a1e5e68dc3942a59dfb300b1db16043'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:02.324662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a1e5e68dc3942a59dfb300b1db16043'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:02.333328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a1e5e68dc3942a59dfb300b1db16043'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:02.343157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a1e5e68dc3942a59dfb300b1db16043'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:02.349980 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3486/7055] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:02 2022] DELETE /identity/v3/users/6a1e5e68dc3942a59dfb300b1db16043 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:02.354923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96ce9e-419b-41a5-afa3-4bfde75a3f5a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:02.355234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96ce9e-419b-41a5-afa3-4bfde75a3f5a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:02.355496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96ce9e-419b-41a5-afa3-4bfde75a3f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:02.375336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0f96ce9e-419b-41a5-afa3-4bfde75a3f5a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:02.376550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0f96ce9e-419b-41a5-afa3-4bfde75a3f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:02.447252 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3570/7056] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:02.457211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:02.458722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:02.520258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067653-78e2-41c6-bbe1-9b46f9c39af7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:02.523888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd435c2-1e55-42ba-9b6b-d8fd5cced025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:02.524499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd435c2-1e55-42ba-9b6b-d8fd5cced025 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:02.524784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd435c2-1e55-42ba-9b6b-d8fd5cced025 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:02.525092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd435c2-1e55-42ba-9b6b-d8fd5cced025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:02.538540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd435c2-1e55-42ba-9b6b-d8fd5cced025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:02.540499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd435c2-1e55-42ba-9b6b-d8fd5cced025 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:02.556928 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3487/7057] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:03.154008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96ce9e-419b-41a5-afa3-4bfde75a3f5a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:03.155744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0f96ce9e-419b-41a5-afa3-4bfde75a3f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:03.167989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96ce9e-419b-41a5-afa3-4bfde75a3f5a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:03.173146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d56ed7-a387-41eb-99f8-8a995f915b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:03.174015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d56ed7-a387-41eb-99f8-8a995f915b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:03.174294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d56ed7-a387-41eb-99f8-8a995f915b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:03.174566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d56ed7-a387-41eb-99f8-8a995f915b64 admin admin] PATH_INFO: `/v3/projects/ccdf783fce74452d8881cb32ccc56bc4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:03.175728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d56ed7-a387-41eb-99f8-8a995f915b64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccdf783fce74452d8881cb32ccc56bc4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:03.177884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d56ed7-a387-41eb-99f8-8a995f915b64 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:03.244259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e5d56ed7-a387-41eb-99f8-8a995f915b64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccdf783fce74452d8881cb32ccc56bc4', 'request_id': 'req-e5d56ed7-a387-41eb-99f8-8a995f915b64'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:03.254810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e5d56ed7-a387-41eb-99f8-8a995f915b64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccdf783fce74452d8881cb32ccc56bc4', 'request_id': 'req-e5d56ed7-a387-41eb-99f8-8a995f915b64'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:03.268588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e5d56ed7-a387-41eb-99f8-8a995f915b64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccdf783fce74452d8881cb32ccc56bc4', 'request_id': 'req-e5d56ed7-a387-41eb-99f8-8a995f915b64'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:03.285076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e5d56ed7-a387-41eb-99f8-8a995f915b64 admin admin] The token cache is being invalidate because project ccdf783fce74452d8881cb32ccc56bc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:03.285470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e5d56ed7-a387-41eb-99f8-8a995f915b64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:03.286367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e5d56ed7-a387-41eb-99f8-8a995f915b64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:03.291450 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3571/7058] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:03 2022] DELETE /identity/v3/projects/ccdf783fce74452d8881cb32ccc56bc4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:03.296813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020a4a4-47e5-4ea3-82f2-b7bfecabd7c7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:03.297151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020a4a4-47e5-4ea3-82f2-b7bfecabd7c7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:03.297466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020a4a4-47e5-4ea3-82f2-b7bfecabd7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:03.320793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7020a4a4-47e5-4ea3-82f2-b7bfecabd7c7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:03.322196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7020a4a4-47e5-4ea3-82f2-b7bfecabd7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:03.363478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:03.406420 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3488/7059] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:03.416101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd285941-7790-41b3-aafa-96e962a5ec33 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:03.416396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd285941-7790-41b3-aafa-96e962a5ec33 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:03.416682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd285941-7790-41b3-aafa-96e962a5ec33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:03.438992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-dd285941-7790-41b3-aafa-96e962a5ec33 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:03.440827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dd285941-7790-41b3-aafa-96e962a5ec33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:03.453176 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3572/7060] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:03.457601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020a4a4-47e5-4ea3-82f2-b7bfecabd7c7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:03.469816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020a4a4-47e5-4ea3-82f2-b7bfecabd7c7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:03.474501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da7dbf-89e6-457c-99c3-a27a3f68c211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:03.475197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da7dbf-89e6-457c-99c3-a27a3f68c211 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:03.475608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da7dbf-89e6-457c-99c3-a27a3f68c211 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:03.475896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da7dbf-89e6-457c-99c3-a27a3f68c211 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:03.476553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da7dbf-89e6-457c-99c3-a27a3f68c211 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:03.476858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da7dbf-89e6-457c-99c3-a27a3f68c211 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:03.479012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da7dbf-89e6-457c-99c3-a27a3f68c211 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:03.497020 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3489/7061] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:03.498172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd285941-7790-41b3-aafa-96e962a5ec33 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:03.507892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd285941-7790-41b3-aafa-96e962a5ec33 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:03.515659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff6f13-a702-4757-b8a6-65ac9a6353b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:03.516410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff6f13-a702-4757-b8a6-65ac9a6353b3 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:03.516778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff6f13-a702-4757-b8a6-65ac9a6353b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:03.517741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff6f13-a702-4757-b8a6-65ac9a6353b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:03.518481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff6f13-a702-4757-b8a6-65ac9a6353b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:03.520693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff6f13-a702-4757-b8a6-65ac9a6353b3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:03.557481 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3573/7062] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:03 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:03.562056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da7dbf-89e6-457c-99c3-a27a3f68c211 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:03.576715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da7dbf-89e6-457c-99c3-a27a3f68c211 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:03.645650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5fe01-3743-4df2-a7cf-a947cd779b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:03.646664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5fe01-3743-4df2-a7cf-a947cd779b25 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:03.647221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5fe01-3743-4df2-a7cf-a947cd779b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:03.647675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5fe01-3743-4df2-a7cf-a947cd779b25 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:03.648313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5fe01-3743-4df2-a7cf-a947cd779b25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:03.650420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5fe01-3743-4df2-a7cf-a947cd779b25 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:03.677481 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3490/7063] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:03 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:03.681787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff6f13-a702-4757-b8a6-65ac9a6353b3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:03.698115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:03.704262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff6f13-a702-4757-b8a6-65ac9a6353b3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:03.715970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e0ba9-5bfe-46dd-85ca-7d8cc029ef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:03.716619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e0ba9-5bfe-46dd-85ca-7d8cc029ef9c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:03.719985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e0ba9-5bfe-46dd-85ca-7d8cc029ef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:03.720328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e0ba9-5bfe-46dd-85ca-7d8cc029ef9c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:03.721154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e0ba9-5bfe-46dd-85ca-7d8cc029ef9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:03.725658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e0ba9-5bfe-46dd-85ca-7d8cc029ef9c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:03.760029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5fe01-3743-4df2-a7cf-a947cd779b25 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:03.760293 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3574/7064] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:03.775373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5fe01-3743-4df2-a7cf-a947cd779b25 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:03.791322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672897af-3f98-4c5f-9a8d-4a13092eea43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:03.791989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672897af-3f98-4c5f-9a8d-4a13092eea43 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:03.792282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672897af-3f98-4c5f-9a8d-4a13092eea43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:03.792550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672897af-3f98-4c5f-9a8d-4a13092eea43 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:03.793232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672897af-3f98-4c5f-9a8d-4a13092eea43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:03.794858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672897af-3f98-4c5f-9a8d-4a13092eea43 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:03.812773 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3491/7065] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:03.817074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e0ba9-5bfe-46dd-85ca-7d8cc029ef9c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:03.848964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e0ba9-5bfe-46dd-85ca-7d8cc029ef9c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:03.865501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6716035-798b-4fda-8ade-e3f6c6f82077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:03.866199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6716035-798b-4fda-8ade-e3f6c6f82077 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:03.866546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6716035-798b-4fda-8ade-e3f6c6f82077 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:03.866952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6716035-798b-4fda-8ade-e3f6c6f82077 admin admin] PATH_INFO: `/v3/projects/0df3d748ef3b431e9f9362012f499ca8/users/cac56c99a9c540cf8645e3d64c278c58/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:03.887833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6716035-798b-4fda-8ade-e3f6c6f82077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df3d748ef3b431e9f9362012f499ca8, user_id=cac56c99a9c540cf8645e3d64c278c58, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:03.890720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6716035-798b-4fda-8ade-e3f6c6f82077 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:03.912431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672897af-3f98-4c5f-9a8d-4a13092eea43 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:03.914849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-672897af-3f98-4c5f-9a8d-4a13092eea43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:03.939451 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3575/7066] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:03 2022] PUT /identity/v3/projects/0df3d748ef3b431e9f9362012f499ca8/users/cac56c99a9c540cf8645e3d64c278c58/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:03.959164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbbe49-368e-4f32-80d0-688a94334fb0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:03.959463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbbe49-368e-4f32-80d0-688a94334fb0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:03.959743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbbe49-368e-4f32-80d0-688a94334fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:04.008622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4bfbbe49-368e-4f32-80d0-688a94334fb0 None None] MFA Rules not processed for user `cac56c99a9c540cf8645e3d64c278c58`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:04.009902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4bfbbe49-368e-4f32-80d0-688a94334fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:04.091420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672897af-3f98-4c5f-9a8d-4a13092eea43 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:04.171285 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3576/7067] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:04.186611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbbe49-368e-4f32-80d0-688a94334fb0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:04.188109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4bfbbe49-368e-4f32-80d0-688a94334fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:04.221720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4281c26-af8e-46b1-98ee-0fb0444d44bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:04.222715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4281c26-af8e-46b1-98ee-0fb0444d44bf service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:04.223002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4281c26-af8e-46b1-98ee-0fb0444d44bf service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:04.223279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4281c26-af8e-46b1-98ee-0fb0444d44bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:04.225263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b4281c26-af8e-46b1-98ee-0fb0444d44bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:04.246311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4281c26-af8e-46b1-98ee-0fb0444d44bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:04.247767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4281c26-af8e-46b1-98ee-0fb0444d44bf service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:04.255619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:04.273200 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3492/7068] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:04.295317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbbe49-368e-4f32-80d0-688a94334fb0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:04.300224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16f44a-cf36-492c-8b95-7800a8e1b136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:04.300974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16f44a-cf36-492c-8b95-7800a8e1b136 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:04.301415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16f44a-cf36-492c-8b95-7800a8e1b136 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:04.301700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16f44a-cf36-492c-8b95-7800a8e1b136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:04.312502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16f44a-cf36-492c-8b95-7800a8e1b136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:04.314131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16f44a-cf36-492c-8b95-7800a8e1b136 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:04.331543 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3577/7069] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:15:04.349141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4281c26-af8e-46b1-98ee-0fb0444d44bf service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:04.456932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4281c26-af8e-46b1-98ee-0fb0444d44bf service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:04.464932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:04.465729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:04.466234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:04.467122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] PATH_INFO: `/v3/users/cac56c99a9c540cf8645e3d64c278c58` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:04.467935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cac56c99a9c540cf8645e3d64c278c58)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:04.470900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:04.542687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cac56c99a9c540cf8645e3d64c278c58'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:04.561035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cac56c99a9c540cf8645e3d64c278c58'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:04.561814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cac56c99a9c540cf8645e3d64c278c58'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:04.571618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cac56c99a9c540cf8645e3d64c278c58'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:04.583262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cac56c99a9c540cf8645e3d64c278c58'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:04.593146 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3493/7070] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:04 2022] DELETE /identity/v3/users/cac56c99a9c540cf8645e3d64c278c58 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:04.598873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a6305-246c-4d24-bf41-1bd86ebe69fb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:04.599900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a6305-246c-4d24-bf41-1bd86ebe69fb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:04.600312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a6305-246c-4d24-bf41-1bd86ebe69fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:04.631583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a93a6305-246c-4d24-bf41-1bd86ebe69fb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:04.633134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a93a6305-246c-4d24-bf41-1bd86ebe69fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:04.713890 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3578/7071] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:04.722457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:04.724057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:04.826547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21201fbd-2453-4266-b08e-7b5fdeb61d5f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:04.831769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676de19-fa7b-4f1f-98c2-be20b37398a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:04.832531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676de19-fa7b-4f1f-98c2-be20b37398a2 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:04.833507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676de19-fa7b-4f1f-98c2-be20b37398a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:04.833652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676de19-fa7b-4f1f-98c2-be20b37398a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:04.855207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676de19-fa7b-4f1f-98c2-be20b37398a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:04.857786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676de19-fa7b-4f1f-98c2-be20b37398a2 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:04.873093 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3494/7072] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:05.540439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a6305-246c-4d24-bf41-1bd86ebe69fb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:05.542621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a93a6305-246c-4d24-bf41-1bd86ebe69fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:05.551423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:05.557486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a6305-246c-4d24-bf41-1bd86ebe69fb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:05.568174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a08e45-4df6-4fba-97a3-3b72b1603a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:05.577925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a08e45-4df6-4fba-97a3-3b72b1603a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:05.580530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a08e45-4df6-4fba-97a3-3b72b1603a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:05.580819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a08e45-4df6-4fba-97a3-3b72b1603a70 admin admin] PATH_INFO: `/v3/projects/0df3d748ef3b431e9f9362012f499ca8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:05.582537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a08e45-4df6-4fba-97a3-3b72b1603a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df3d748ef3b431e9f9362012f499ca8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:05.585083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a08e45-4df6-4fba-97a3-3b72b1603a70 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:05.665000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f8a08e45-4df6-4fba-97a3-3b72b1603a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df3d748ef3b431e9f9362012f499ca8', 'request_id': 'req-f8a08e45-4df6-4fba-97a3-3b72b1603a70'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:05.684922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f8a08e45-4df6-4fba-97a3-3b72b1603a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df3d748ef3b431e9f9362012f499ca8', 'request_id': 'req-f8a08e45-4df6-4fba-97a3-3b72b1603a70'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:05.686712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f8a08e45-4df6-4fba-97a3-3b72b1603a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df3d748ef3b431e9f9362012f499ca8', 'request_id': 'req-f8a08e45-4df6-4fba-97a3-3b72b1603a70'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:05.703460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f8a08e45-4df6-4fba-97a3-3b72b1603a70 admin admin] The token cache is being invalidate because project 0df3d748ef3b431e9f9362012f499ca8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:05.703988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f8a08e45-4df6-4fba-97a3-3b72b1603a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:05.704803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f8a08e45-4df6-4fba-97a3-3b72b1603a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:05.706804 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3579/7073] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:05 2022] DELETE /identity/v3/projects/0df3d748ef3b431e9f9362012f499ca8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:05.711726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c2bf4-ba64-406e-a00e-1ffa8ad1d41f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:05.712701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c2bf4-ba64-406e-a00e-1ffa8ad1d41f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:05.713818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c2bf4-ba64-406e-a00e-1ffa8ad1d41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:05.737996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-286c2bf4-ba64-406e-a00e-1ffa8ad1d41f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:05.739537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-286c2bf4-ba64-406e-a00e-1ffa8ad1d41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:05.757460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:05.841993 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3495/7074] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:05.849843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085945cf-e8f5-48e4-8abc-5945d13a8bce None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:05.851475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085945cf-e8f5-48e4-8abc-5945d13a8bce None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:05.852523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085945cf-e8f5-48e4-8abc-5945d13a8bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:05.883231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-085945cf-e8f5-48e4-8abc-5945d13a8bce None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:05.885133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-085945cf-e8f5-48e4-8abc-5945d13a8bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:05.899919 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3580/7075] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:05.902378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c2bf4-ba64-406e-a00e-1ffa8ad1d41f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:05.912837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c2bf4-ba64-406e-a00e-1ffa8ad1d41f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:05.918255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a69f2-7df8-4150-b588-3b3b58ca6045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:05.919476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a69f2-7df8-4150-b588-3b3b58ca6045 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:05.919476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a69f2-7df8-4150-b588-3b3b58ca6045 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:05.919476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a69f2-7df8-4150-b588-3b3b58ca6045 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:05.920092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a69f2-7df8-4150-b588-3b3b58ca6045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:05.921425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a69f2-7df8-4150-b588-3b3b58ca6045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:05.922087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a69f2-7df8-4150-b588-3b3b58ca6045 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:05.939423 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3496/7076] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:05.941856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085945cf-e8f5-48e4-8abc-5945d13a8bce None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:05.957152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085945cf-e8f5-48e4-8abc-5945d13a8bce None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:05.968238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1217172-2f53-465c-90fe-7621ff4bb518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:05.968883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1217172-2f53-465c-90fe-7621ff4bb518 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:05.969516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1217172-2f53-465c-90fe-7621ff4bb518 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:05.969949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1217172-2f53-465c-90fe-7621ff4bb518 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:05.970753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1217172-2f53-465c-90fe-7621ff4bb518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:05.972721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1217172-2f53-465c-90fe-7621ff4bb518 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:06.007426 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3581/7077] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:05 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:06.011919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a69f2-7df8-4150-b588-3b3b58ca6045 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:06.024810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a69f2-7df8-4150-b588-3b3b58ca6045 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:06.100542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722a7a0-2053-4674-b2a7-3e2aad20bce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:06.101296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722a7a0-2053-4674-b2a7-3e2aad20bce4 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:06.101653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722a7a0-2053-4674-b2a7-3e2aad20bce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:06.102024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722a7a0-2053-4674-b2a7-3e2aad20bce4 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:06.102806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5722a7a0-2053-4674-b2a7-3e2aad20bce4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:06.106017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5722a7a0-2053-4674-b2a7-3e2aad20bce4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:06.133708 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3497/7078] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:06 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:06.138675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1217172-2f53-465c-90fe-7621ff4bb518 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:06.152561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1217172-2f53-465c-90fe-7621ff4bb518 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:06.157905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603214f-74dd-45fc-96ff-b511a910bea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:06.158784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603214f-74dd-45fc-96ff-b511a910bea1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:06.159072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603214f-74dd-45fc-96ff-b511a910bea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:06.159354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603214f-74dd-45fc-96ff-b511a910bea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:06.160008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603214f-74dd-45fc-96ff-b511a910bea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:06.161762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603214f-74dd-45fc-96ff-b511a910bea1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:06.176977 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3582/7079] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:15:06.178838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722a7a0-2053-4674-b2a7-3e2aad20bce4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:06.190089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722a7a0-2053-4674-b2a7-3e2aad20bce4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:06.197729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da887d-7be9-4f80-acec-06076f1d90b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:06.199161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da887d-7be9-4f80-acec-06076f1d90b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:06.199895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da887d-7be9-4f80-acec-06076f1d90b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:06.200570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da887d-7be9-4f80-acec-06076f1d90b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:06.201625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da887d-7be9-4f80-acec-06076f1d90b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:06.203372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da887d-7be9-4f80-acec-06076f1d90b1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:06.218927 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3498/7080] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:06.223079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603214f-74dd-45fc-96ff-b511a910bea1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:06.236981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603214f-74dd-45fc-96ff-b511a910bea1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:06.241805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a15606-8d1b-4e06-b352-f13de35143ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:06.242608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a15606-8d1b-4e06-b352-f13de35143ab admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:06.243289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a15606-8d1b-4e06-b352-f13de35143ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:06.243289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a15606-8d1b-4e06-b352-f13de35143ab admin admin] PATH_INFO: `/v3/projects/0e2c3167910941209b8f3bc29dc0502a/users/2d1c34feca0c4dd48f6115159166ce56/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:06.258983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a15606-8d1b-4e06-b352-f13de35143ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2c3167910941209b8f3bc29dc0502a, user_id=2d1c34feca0c4dd48f6115159166ce56, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:06.262283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a15606-8d1b-4e06-b352-f13de35143ab admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:06.278856 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3583/7081] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:06 2022] PUT /identity/v3/projects/0e2c3167910941209b8f3bc29dc0502a/users/2d1c34feca0c4dd48f6115159166ce56/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:06.290760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb754cc3-9e6e-491d-abca-5b55f134eafa None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:06.291055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb754cc3-9e6e-491d-abca-5b55f134eafa None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:06.291334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb754cc3-9e6e-491d-abca-5b55f134eafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:06.327747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bb754cc3-9e6e-491d-abca-5b55f134eafa None None] MFA Rules not processed for user `2d1c34feca0c4dd48f6115159166ce56`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:06.329008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bb754cc3-9e6e-491d-abca-5b55f134eafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:06.428965 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3499/7082] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:06.444785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a15606-8d1b-4e06-b352-f13de35143ab admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:06.446768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-93a15606-8d1b-4e06-b352-f13de35143ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:06.534589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a15606-8d1b-4e06-b352-f13de35143ab admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:06.539033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389174f2-9be0-496b-8971-ae8a6063f63f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:06.539770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389174f2-9be0-496b-8971-ae8a6063f63f service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:06.539965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389174f2-9be0-496b-8971-ae8a6063f63f service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:06.540359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389174f2-9be0-496b-8971-ae8a6063f63f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:06.554519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389174f2-9be0-496b-8971-ae8a6063f63f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:06.556158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389174f2-9be0-496b-8971-ae8a6063f63f service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:06.558753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb754cc3-9e6e-491d-abca-5b55f134eafa None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:06.561246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bb754cc3-9e6e-491d-abca-5b55f134eafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:06.574769 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3584/7083] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:06.585482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:06.594329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389174f2-9be0-496b-8971-ae8a6063f63f service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:06.630423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:06.690905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389174f2-9be0-496b-8971-ae8a6063f63f service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:06.698777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:06.699888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:06.700349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:06.700805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] PATH_INFO: `/v3/users/2d1c34feca0c4dd48f6115159166ce56` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:06.702228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d1c34feca0c4dd48f6115159166ce56)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:06.705130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:06.730406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb754cc3-9e6e-491d-abca-5b55f134eafa None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:06.738743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f5cd6-5da9-455f-b087-b1084c88a886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:06.739581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f5cd6-5da9-455f-b087-b1084c88a886 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:06.739971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f5cd6-5da9-455f-b087-b1084c88a886 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:06.740257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f5cd6-5da9-455f-b087-b1084c88a886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:06.742490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-615f5cd6-5da9-455f-b087-b1084c88a886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:06.766789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f5cd6-5da9-455f-b087-b1084c88a886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:06.768940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f5cd6-5da9-455f-b087-b1084c88a886 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:06.803269 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3500/7084] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:15:06.805802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d1c34feca0c4dd48f6115159166ce56'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:06.846828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d1c34feca0c4dd48f6115159166ce56'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:06.872022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d1c34feca0c4dd48f6115159166ce56'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:06.872978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d1c34feca0c4dd48f6115159166ce56'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:06.873806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d1c34feca0c4dd48f6115159166ce56'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:06.891782 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3585/7085] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:06 2022] DELETE /identity/v3/users/2d1c34feca0c4dd48f6115159166ce56 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:06.893255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac711e-5437-4a75-beec-f8d9c5b2b43a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:06.893567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac711e-5437-4a75-beec-f8d9c5b2b43a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:06.893872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac711e-5437-4a75-beec-f8d9c5b2b43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:06.926310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-82ac711e-5437-4a75-beec-f8d9c5b2b43a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:06.927557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-82ac711e-5437-4a75-beec-f8d9c5b2b43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:07.006981 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3501/7086] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:07.025352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:07.027143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:07.102676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c512e70-ea78-468e-9f56-a419ed42e739 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:07.107469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384160ce-a22a-472e-ab32-ee201ade1267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:07.108115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384160ce-a22a-472e-ab32-ee201ade1267 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:07.108672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384160ce-a22a-472e-ab32-ee201ade1267 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:07.108769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384160ce-a22a-472e-ab32-ee201ade1267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:07.132507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384160ce-a22a-472e-ab32-ee201ade1267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:07.134044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384160ce-a22a-472e-ab32-ee201ade1267 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:07.159040 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3586/7087] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:07.547895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac711e-5437-4a75-beec-f8d9c5b2b43a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:07.549418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-82ac711e-5437-4a75-beec-f8d9c5b2b43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:07.561814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac711e-5437-4a75-beec-f8d9c5b2b43a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:07.566497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fc0b2-17c4-4db7-b436-e696df40bb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:07.567145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fc0b2-17c4-4db7-b436-e696df40bb19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:07.567428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fc0b2-17c4-4db7-b436-e696df40bb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:07.567708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fc0b2-17c4-4db7-b436-e696df40bb19 admin admin] PATH_INFO: `/v3/projects/0e2c3167910941209b8f3bc29dc0502a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:07.568851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fc0b2-17c4-4db7-b436-e696df40bb19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e2c3167910941209b8f3bc29dc0502a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:07.570807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fc0b2-17c4-4db7-b436-e696df40bb19 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:07.622013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-606fc0b2-17c4-4db7-b436-e696df40bb19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e2c3167910941209b8f3bc29dc0502a', 'request_id': 'req-606fc0b2-17c4-4db7-b436-e696df40bb19'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:07.630506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-606fc0b2-17c4-4db7-b436-e696df40bb19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e2c3167910941209b8f3bc29dc0502a', 'request_id': 'req-606fc0b2-17c4-4db7-b436-e696df40bb19'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:07.637852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-606fc0b2-17c4-4db7-b436-e696df40bb19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e2c3167910941209b8f3bc29dc0502a', 'request_id': 'req-606fc0b2-17c4-4db7-b436-e696df40bb19'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:07.648521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-606fc0b2-17c4-4db7-b436-e696df40bb19 admin admin] The token cache is being invalidate because project 0e2c3167910941209b8f3bc29dc0502a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:07.648821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-606fc0b2-17c4-4db7-b436-e696df40bb19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:07.649547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-606fc0b2-17c4-4db7-b436-e696df40bb19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:07.651110 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3502/7088] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:07 2022] DELETE /identity/v3/projects/0e2c3167910941209b8f3bc29dc0502a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:07.656671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b155de-7e81-40e0-84e1-73f3ba257ee1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:07.657173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b155de-7e81-40e0-84e1-73f3ba257ee1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:07.657298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b155de-7e81-40e0-84e1-73f3ba257ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:07.678236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a5b155de-7e81-40e0-84e1-73f3ba257ee1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:07.679464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a5b155de-7e81-40e0-84e1-73f3ba257ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:07.723108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:07.743907 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3587/7089] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:07.751599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7bba5-9c1b-4bd8-bad4-218cd0b93a0e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:07.752298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7bba5-9c1b-4bd8-bad4-218cd0b93a0e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:07.752298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7bba5-9c1b-4bd8-bad4-218cd0b93a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:07.773430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-dba7bba5-9c1b-4bd8-bad4-218cd0b93a0e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:07.774647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dba7bba5-9c1b-4bd8-bad4-218cd0b93a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:07.783570 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3503/7090] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:07.788610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b155de-7e81-40e0-84e1-73f3ba257ee1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:07.798529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b155de-7e81-40e0-84e1-73f3ba257ee1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:07.803227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df46ba-ac69-4b89-8db6-283b67792be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:07.803851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df46ba-ac69-4b89-8db6-283b67792be9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:07.804141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df46ba-ac69-4b89-8db6-283b67792be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:07.804585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df46ba-ac69-4b89-8db6-283b67792be9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:07.805136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df46ba-ac69-4b89-8db6-283b67792be9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:07.805450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df46ba-ac69-4b89-8db6-283b67792be9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:07.807088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df46ba-ac69-4b89-8db6-283b67792be9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:07.837390 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3588/7091] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:07.842811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7bba5-9c1b-4bd8-bad4-218cd0b93a0e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:07.864741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7bba5-9c1b-4bd8-bad4-218cd0b93a0e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:07.869839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95075bf-35d3-465b-8d52-57b8f9476724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:07.870567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95075bf-35d3-465b-8d52-57b8f9476724 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:07.870949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95075bf-35d3-465b-8d52-57b8f9476724 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:07.871328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95075bf-35d3-465b-8d52-57b8f9476724 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:07.872174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95075bf-35d3-465b-8d52-57b8f9476724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:07.874215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95075bf-35d3-465b-8d52-57b8f9476724 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:07.900460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:07.920303 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3504/7092] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:07 2022] POST /identity/v3/projects => generated 371 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:07.924800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df46ba-ac69-4b89-8db6-283b67792be9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:07.936119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df46ba-ac69-4b89-8db6-283b67792be9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:08.024188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943d2c8-6feb-47cc-bca7-969119b72135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:08.024188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943d2c8-6feb-47cc-bca7-969119b72135 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:08.024188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943d2c8-6feb-47cc-bca7-969119b72135 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:08.024188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943d2c8-6feb-47cc-bca7-969119b72135 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:08.024852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943d2c8-6feb-47cc-bca7-969119b72135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:08.027149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943d2c8-6feb-47cc-bca7-969119b72135 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:08.066167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95075bf-35d3-465b-8d52-57b8f9476724 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:08.067125 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3589/7093] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:07 2022] POST /identity/v3/users => generated 286 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:15:08.080617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95075bf-35d3-465b-8d52-57b8f9476724 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:08.085483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1de383-8df5-4db9-8bc8-eec2a784a690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:08.086142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1de383-8df5-4db9-8bc8-eec2a784a690 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:08.086399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1de383-8df5-4db9-8bc8-eec2a784a690 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:08.086729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1de383-8df5-4db9-8bc8-eec2a784a690 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:08.087417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1de383-8df5-4db9-8bc8-eec2a784a690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:08.089113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1de383-8df5-4db9-8bc8-eec2a784a690 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:08.105170 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3505/7094] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:08.106437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943d2c8-6feb-47cc-bca7-969119b72135 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:08.119424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943d2c8-6feb-47cc-bca7-969119b72135 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:08.137498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba74c1-f5c7-4990-8a12-098f2ee341be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:08.139653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba74c1-f5c7-4990-8a12-098f2ee341be admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:08.156926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba74c1-f5c7-4990-8a12-098f2ee341be admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:08.156926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba74c1-f5c7-4990-8a12-098f2ee341be admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:08.156926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba74c1-f5c7-4990-8a12-098f2ee341be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:08.156926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba74c1-f5c7-4990-8a12-098f2ee341be admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:08.172944 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3590/7095] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:08.174991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1de383-8df5-4db9-8bc8-eec2a784a690 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:08.194531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1de383-8df5-4db9-8bc8-eec2a784a690 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:08.204642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a95fa5-40d4-4e49-a24f-fbd3e29e648e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:08.206387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a95fa5-40d4-4e49-a24f-fbd3e29e648e admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:08.207299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a95fa5-40d4-4e49-a24f-fbd3e29e648e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:08.207533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a95fa5-40d4-4e49-a24f-fbd3e29e648e admin admin] PATH_INFO: `/v3/projects/25f222e844114110820cff009fe9939a/users/aca655013c9a4f209f3838bb61d4b581/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:08.247059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a95fa5-40d4-4e49-a24f-fbd3e29e648e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f222e844114110820cff009fe9939a, user_id=aca655013c9a4f209f3838bb61d4b581, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:08.249933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a95fa5-40d4-4e49-a24f-fbd3e29e648e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:08.278278 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3506/7096] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:08 2022] PUT /identity/v3/projects/25f222e844114110820cff009fe9939a/users/aca655013c9a4f209f3838bb61d4b581/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:08.289670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3263d62-1141-4f59-b74f-057591378855 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:08.290012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3263d62-1141-4f59-b74f-057591378855 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:08.290453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3263d62-1141-4f59-b74f-057591378855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:08.359392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f3263d62-1141-4f59-b74f-057591378855 None None] MFA Rules not processed for user `aca655013c9a4f209f3838bb61d4b581`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:08.364278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f3263d62-1141-4f59-b74f-057591378855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:08.464944 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3591/7097] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:08.467825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a95fa5-40d4-4e49-a24f-fbd3e29e648e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:08.539727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a95fa5-40d4-4e49-a24f-fbd3e29e648e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:08.545553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676efd8-9a31-4200-9c9f-aada6f44c4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:08.546172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676efd8-9a31-4200-9c9f-aada6f44c4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:08.546595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676efd8-9a31-4200-9c9f-aada6f44c4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:08.546878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676efd8-9a31-4200-9c9f-aada6f44c4e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:08.547519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676efd8-9a31-4200-9c9f-aada6f44c4e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:08.550217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676efd8-9a31-4200-9c9f-aada6f44c4e8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:08.588534 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3507/7098] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:08 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:08.589782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3263d62-1141-4f59-b74f-057591378855 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:08.603465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3263d62-1141-4f59-b74f-057591378855 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:08.642293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676efd8-9a31-4200-9c9f-aada6f44c4e8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:08.644796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8676efd8-9a31-4200-9c9f-aada6f44c4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:08.668171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:08.687747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb730602-b9c2-44f8-8b08-9d456fd9b09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:08.688374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb730602-b9c2-44f8-8b08-9d456fd9b09f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:08.688668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb730602-b9c2-44f8-8b08-9d456fd9b09f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:08.688978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb730602-b9c2-44f8-8b08-9d456fd9b09f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:08.689616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb730602-b9c2-44f8-8b08-9d456fd9b09f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:08.691722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb730602-b9c2-44f8-8b08-9d456fd9b09f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:08.717583 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3592/7099] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:08 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:08.720497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb730602-b9c2-44f8-8b08-9d456fd9b09f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:08.730429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676efd8-9a31-4200-9c9f-aada6f44c4e8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:08.731628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb730602-b9c2-44f8-8b08-9d456fd9b09f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:08.736963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b212004-8efd-48c0-8bc4-9d3bd09bd433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:08.737655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b212004-8efd-48c0-8bc4-9d3bd09bd433 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:08.738013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b212004-8efd-48c0-8bc4-9d3bd09bd433 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:08.738387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b212004-8efd-48c0-8bc4-9d3bd09bd433 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:08.739147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b212004-8efd-48c0-8bc4-9d3bd09bd433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:08.740868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b212004-8efd-48c0-8bc4-9d3bd09bd433 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:08.750954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a8f5e-c694-4616-8319-8689232e6392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:08.751759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a8f5e-c694-4616-8319-8689232e6392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:08.752304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a8f5e-c694-4616-8319-8689232e6392 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:08.752572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a8f5e-c694-4616-8319-8689232e6392 admin admin] PATH_INFO: `/v3/users/5d669705cb9e494eb8c135ad8a01b249` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:08.756804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a8f5e-c694-4616-8319-8689232e6392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d669705cb9e494eb8c135ad8a01b249)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:08.756963 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3593/7100] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:15:08.757080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a8f5e-c694-4616-8319-8689232e6392 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:08.757982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b212004-8efd-48c0-8bc4-9d3bd09bd433 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:08.769013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b212004-8efd-48c0-8bc4-9d3bd09bd433 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:08.774647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9ffe6-3269-447f-888a-11cd108aabd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:08.775637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9ffe6-3269-447f-888a-11cd108aabd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:08.776114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9ffe6-3269-447f-888a-11cd108aabd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:08.777163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9ffe6-3269-447f-888a-11cd108aabd1 admin admin] PATH_INFO: `/v3/projects/fa174a652f634a578aae372e6a0385af/users/1664b56570d04ade8a8c792dd71ad5f8/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:08.794422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9ffe6-3269-447f-888a-11cd108aabd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa174a652f634a578aae372e6a0385af, user_id=1664b56570d04ade8a8c792dd71ad5f8, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:08.797599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9ffe6-3269-447f-888a-11cd108aabd1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:08.818463 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3594/7101] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:08 2022] PUT /identity/v3/projects/fa174a652f634a578aae372e6a0385af/users/1664b56570d04ade8a8c792dd71ad5f8/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:08.822652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9ffe6-3269-447f-888a-11cd108aabd1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:08.836753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ec1a8f5e-c694-4616-8319-8689232e6392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d669705cb9e494eb8c135ad8a01b249'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:08.838225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:08.857539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ec1a8f5e-c694-4616-8319-8689232e6392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d669705cb9e494eb8c135ad8a01b249'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:08.858388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ec1a8f5e-c694-4616-8319-8689232e6392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d669705cb9e494eb8c135ad8a01b249'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:08.866576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ec1a8f5e-c694-4616-8319-8689232e6392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d669705cb9e494eb8c135ad8a01b249'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:08.885473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ec1a8f5e-c694-4616-8319-8689232e6392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d669705cb9e494eb8c135ad8a01b249'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:08.899031 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3508/7102] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:08 2022] DELETE /identity/v3/users/5d669705cb9e494eb8c135ad8a01b249 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:08.906954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db4d9f-94f6-4f71-b443-dfb5b7b18dc6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:08.907472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db4d9f-94f6-4f71-b443-dfb5b7b18dc6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:08.907978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db4d9f-94f6-4f71-b443-dfb5b7b18dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:08.929940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9ffe6-3269-447f-888a-11cd108aabd1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:08.947534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d6db4d9f-94f6-4f71-b443-dfb5b7b18dc6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:08.951363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d6db4d9f-94f6-4f71-b443-dfb5b7b18dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:09.009193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713ae5a-2a12-46e8-bcd4-ad074dc163ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:09.009819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713ae5a-2a12-46e8-bcd4-ad074dc163ee admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:09.010649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713ae5a-2a12-46e8-bcd4-ad074dc163ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:09.010649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713ae5a-2a12-46e8-bcd4-ad074dc163ee admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:09.011054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713ae5a-2a12-46e8-bcd4-ad074dc163ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:09.012703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713ae5a-2a12-46e8-bcd4-ad074dc163ee admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:09.024937 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3595/7103] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:09.029421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713ae5a-2a12-46e8-bcd4-ad074dc163ee admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:09.031909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b713ae5a-2a12-46e8-bcd4-ad074dc163ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:09.047134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713ae5a-2a12-46e8-bcd4-ad074dc163ee admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:09.056404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0242f-5b2b-4e0f-a1c3-2c96b6db9e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:09.057529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0242f-5b2b-4e0f-a1c3-2c96b6db9e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:09.057991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0242f-5b2b-4e0f-a1c3-2c96b6db9e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:09.058277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0242f-5b2b-4e0f-a1c3-2c96b6db9e9f admin admin] PATH_INFO: `/v3/projects/fa174a652f634a578aae372e6a0385af/users/1664b56570d04ade8a8c792dd71ad5f8/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:09.059975 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3509/7104] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:09.060284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0242f-5b2b-4e0f-a1c3-2c96b6db9e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa174a652f634a578aae372e6a0385af, user_id=1664b56570d04ade8a8c792dd71ad5f8, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:09.063336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0242f-5b2b-4e0f-a1c3-2c96b6db9e9f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:09.069717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db4d9f-94f6-4f71-b443-dfb5b7b18dc6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:09.071731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d6db4d9f-94f6-4f71-b443-dfb5b7b18dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:09.074496 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3596/7105] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:09 2022] PUT /identity/v3/projects/fa174a652f634a578aae372e6a0385af/users/1664b56570d04ade8a8c792dd71ad5f8/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:09.085767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2f6d2-a02d-4bfc-94b8-b9bc8b41bc3e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:09.086523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2f6d2-a02d-4bfc-94b8-b9bc8b41bc3e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:09.087336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2f6d2-a02d-4bfc-94b8-b9bc8b41bc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:09.121182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bea2f6d2-a02d-4bfc-94b8-b9bc8b41bc3e None None] MFA Rules not processed for user `1664b56570d04ade8a8c792dd71ad5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:09.125210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bea2f6d2-a02d-4bfc-94b8-b9bc8b41bc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:09.144241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db4d9f-94f6-4f71-b443-dfb5b7b18dc6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:09.150095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:09.212075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216ed21-54f1-4da7-95bc-0e14c346dc61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:09.212694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216ed21-54f1-4da7-95bc-0e14c346dc61 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:09.213020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216ed21-54f1-4da7-95bc-0e14c346dc61 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:09.213310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216ed21-54f1-4da7-95bc-0e14c346dc61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:09.258149 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3597/7106] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:09.262306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:09.268559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2f6d2-a02d-4bfc-94b8-b9bc8b41bc3e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:09.270109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bea2f6d2-a02d-4bfc-94b8-b9bc8b41bc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:09.331795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216ed21-54f1-4da7-95bc-0e14c346dc61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:09.333378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216ed21-54f1-4da7-95bc-0e14c346dc61 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:09.345421 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3510/7107] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:09.407377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2f6d2-a02d-4bfc-94b8-b9bc8b41bc3e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:09.411542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3abf30-adff-43ec-8f47-291d47fdf967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:09.412359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3abf30-adff-43ec-8f47-291d47fdf967 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:09.413085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3abf30-adff-43ec-8f47-291d47fdf967 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:09.413372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3abf30-adff-43ec-8f47-291d47fdf967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:09.425198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3abf30-adff-43ec-8f47-291d47fdf967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:09.426733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3abf30-adff-43ec-8f47-291d47fdf967 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:09.438962 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3598/7108] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:15:09.555551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216ed21-54f1-4da7-95bc-0e14c346dc61 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:09.559129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7216ed21-54f1-4da7-95bc-0e14c346dc61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:09.578838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216ed21-54f1-4da7-95bc-0e14c346dc61 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:09.589494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaecd9-0abb-4b98-88b7-4cde91246f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:09.589494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaecd9-0abb-4b98-88b7-4cde91246f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:09.589494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaecd9-0abb-4b98-88b7-4cde91246f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:09.589494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaecd9-0abb-4b98-88b7-4cde91246f19 admin admin] PATH_INFO: `/v3/projects/0afa18b8fafb40c5bbfbc279462da6e2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:09.589852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaecd9-0abb-4b98-88b7-4cde91246f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0afa18b8fafb40c5bbfbc279462da6e2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:09.589852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaecd9-0abb-4b98-88b7-4cde91246f19 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:09.651297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bbbaecd9-0abb-4b98-88b7-4cde91246f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0afa18b8fafb40c5bbfbc279462da6e2', 'request_id': 'req-bbbaecd9-0abb-4b98-88b7-4cde91246f19'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:09.662370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bbbaecd9-0abb-4b98-88b7-4cde91246f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0afa18b8fafb40c5bbfbc279462da6e2', 'request_id': 'req-bbbaecd9-0abb-4b98-88b7-4cde91246f19'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:09.671778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bbbaecd9-0abb-4b98-88b7-4cde91246f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0afa18b8fafb40c5bbfbc279462da6e2', 'request_id': 'req-bbbaecd9-0abb-4b98-88b7-4cde91246f19'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:09.685341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bbbaecd9-0abb-4b98-88b7-4cde91246f19 admin admin] The token cache is being invalidate because project 0afa18b8fafb40c5bbfbc279462da6e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:09.685635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bbbaecd9-0abb-4b98-88b7-4cde91246f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:09.686316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bbbaecd9-0abb-4b98-88b7-4cde91246f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:09.687835 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3511/7109] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:09 2022] DELETE /identity/v3/projects/0afa18b8fafb40c5bbfbc279462da6e2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:09.692996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce7ea8-1435-4305-becc-08165649328f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:09.693900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce7ea8-1435-4305-becc-08165649328f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:09.694507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce7ea8-1435-4305-becc-08165649328f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:09.717574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-41ce7ea8-1435-4305-becc-08165649328f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:09.719014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-41ce7ea8-1435-4305-becc-08165649328f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:09.802300 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3599/7110] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:09.811006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aad517-b464-402d-9a68-0d514c77220b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:09.812060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aad517-b464-402d-9a68-0d514c77220b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:09.812547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aad517-b464-402d-9a68-0d514c77220b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:09.839662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-50aad517-b464-402d-9a68-0d514c77220b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:09.841211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-50aad517-b464-402d-9a68-0d514c77220b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:09.851754 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3512/7111] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:09.857425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce7ea8-1435-4305-becc-08165649328f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:09.871771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce7ea8-1435-4305-becc-08165649328f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:09.877775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d821a4-e0e5-44a0-8d38-33bd67a95d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:09.879600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d821a4-e0e5-44a0-8d38-33bd67a95d1a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:09.879978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d821a4-e0e5-44a0-8d38-33bd67a95d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:09.880273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d821a4-e0e5-44a0-8d38-33bd67a95d1a admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:09.881098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d821a4-e0e5-44a0-8d38-33bd67a95d1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:09.881421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d821a4-e0e5-44a0-8d38-33bd67a95d1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:09.883541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d821a4-e0e5-44a0-8d38-33bd67a95d1a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:09.902977 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3600/7112] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:09.906914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aad517-b464-402d-9a68-0d514c77220b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:09.919069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aad517-b464-402d-9a68-0d514c77220b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:09.926123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e0a4f-cf6f-4ec2-b8ad-dc28496a5243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:09.927001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e0a4f-cf6f-4ec2-b8ad-dc28496a5243 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:09.927378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e0a4f-cf6f-4ec2-b8ad-dc28496a5243 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:09.927736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e0a4f-cf6f-4ec2-b8ad-dc28496a5243 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:09.928612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e0a4f-cf6f-4ec2-b8ad-dc28496a5243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:09.930974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e0a4f-cf6f-4ec2-b8ad-dc28496a5243 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:09.988955 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3513/7113] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:09 2022] POST /identity/v3/projects => generated 369 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:09.991838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d821a4-e0e5-44a0-8d38-33bd67a95d1a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:09.996077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e0a4f-cf6f-4ec2-b8ad-dc28496a5243 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:09.999700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c32e0a4f-cf6f-4ec2-b8ad-dc28496a5243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:10.006263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d821a4-e0e5-44a0-8d38-33bd67a95d1a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:10.044456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:10.085305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1764d-b83c-4c6b-ad85-8081c7060b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:10.085964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1764d-b83c-4c6b-ad85-8081c7060b0f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:10.086260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1764d-b83c-4c6b-ad85-8081c7060b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:10.086564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1764d-b83c-4c6b-ad85-8081c7060b0f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:10.087205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1764d-b83c-4c6b-ad85-8081c7060b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:10.089811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1764d-b83c-4c6b-ad85-8081c7060b0f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:10.105731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e0a4f-cf6f-4ec2-b8ad-dc28496a5243 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:10.113127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239d1b2-0adf-4005-ba45-73572376d5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:10.114651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239d1b2-0adf-4005-ba45-73572376d5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:10.114651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239d1b2-0adf-4005-ba45-73572376d5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:10.114651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239d1b2-0adf-4005-ba45-73572376d5e6 admin admin] PATH_INFO: `/v3/users/aca655013c9a4f209f3838bb61d4b581` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:10.114933 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3601/7114] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:09 2022] POST /identity/v3/users => generated 285 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:10.119016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239d1b2-0adf-4005-ba45-73572376d5e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca655013c9a4f209f3838bb61d4b581)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:10.119933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1764d-b83c-4c6b-ad85-8081c7060b0f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:10.121605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239d1b2-0adf-4005-ba45-73572376d5e6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:10.140947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1764d-b83c-4c6b-ad85-8081c7060b0f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:10.147231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f91204-f375-4871-a70c-21186d32682c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:10.147859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f91204-f375-4871-a70c-21186d32682c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:10.148159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f91204-f375-4871-a70c-21186d32682c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:10.148446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f91204-f375-4871-a70c-21186d32682c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:10.149164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f91204-f375-4871-a70c-21186d32682c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:10.151183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f91204-f375-4871-a70c-21186d32682c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:10.159406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:10.164086 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3602/7115] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:15:10.169429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f91204-f375-4871-a70c-21186d32682c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:10.181734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f91204-f375-4871-a70c-21186d32682c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:10.186415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6239d1b2-0adf-4005-ba45-73572376d5e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca655013c9a4f209f3838bb61d4b581'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:10.187662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdceaf-85ba-4ec4-830a-dd5edab80b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:10.188278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdceaf-85ba-4ec4-830a-dd5edab80b05 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:10.188568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdceaf-85ba-4ec4-830a-dd5edab80b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:10.188852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdceaf-85ba-4ec4-830a-dd5edab80b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:10.189651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdceaf-85ba-4ec4-830a-dd5edab80b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:10.191268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdceaf-85ba-4ec4-830a-dd5edab80b05 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:10.202057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6239d1b2-0adf-4005-ba45-73572376d5e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca655013c9a4f209f3838bb61d4b581'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:10.202657 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3603/7116] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:10.205541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6239d1b2-0adf-4005-ba45-73572376d5e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca655013c9a4f209f3838bb61d4b581'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:10.207503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdceaf-85ba-4ec4-830a-dd5edab80b05 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:10.209346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-74bdceaf-85ba-4ec4-830a-dd5edab80b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:10.214361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6239d1b2-0adf-4005-ba45-73572376d5e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca655013c9a4f209f3838bb61d4b581'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:10.223281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdceaf-85ba-4ec4-830a-dd5edab80b05 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:10.224169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6239d1b2-0adf-4005-ba45-73572376d5e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca655013c9a4f209f3838bb61d4b581'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:10.230190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4786498-4d49-4580-a910-06afb456c783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:10.231104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4786498-4d49-4580-a910-06afb456c783 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:10.231104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4786498-4d49-4580-a910-06afb456c783 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:10.231410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4786498-4d49-4580-a910-06afb456c783 admin admin] PATH_INFO: `/v3/projects/a4d6a5eda87840a0ac70b059cdfa5339/users/e6d30843c5c54a1fb41d4d5dff46740c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:10.232925 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3514/7117] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:09 2022] DELETE /identity/v3/users/aca655013c9a4f209f3838bb61d4b581 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:10.238201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71e780-caf7-438e-b862-21995a245f45 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:10.238904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71e780-caf7-438e-b862-21995a245f45 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:10.238904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71e780-caf7-438e-b862-21995a245f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:10.249955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4786498-4d49-4580-a910-06afb456c783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d6a5eda87840a0ac70b059cdfa5339, user_id=e6d30843c5c54a1fb41d4d5dff46740c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:10.253058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4786498-4d49-4580-a910-06afb456c783 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:10.261225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ba71e780-caf7-438e-b862-21995a245f45 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:10.262512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ba71e780-caf7-438e-b862-21995a245f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:10.269312 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3604/7118] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:10 2022] PUT /identity/v3/projects/a4d6a5eda87840a0ac70b059cdfa5339/users/e6d30843c5c54a1fb41d4d5dff46740c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:10.280617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90049329-3600-477b-91bd-05095a3c35bf None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:10.281099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90049329-3600-477b-91bd-05095a3c35bf None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:10.281387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90049329-3600-477b-91bd-05095a3c35bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:10.314774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-90049329-3600-477b-91bd-05095a3c35bf None None] MFA Rules not processed for user `e6d30843c5c54a1fb41d4d5dff46740c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:10.316391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-90049329-3600-477b-91bd-05095a3c35bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:10.341397 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3515/7119] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:10.350624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71e780-caf7-438e-b862-21995a245f45 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:10.352111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ba71e780-caf7-438e-b862-21995a245f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:10.405107 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3605/7120] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:10.417713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90049329-3600-477b-91bd-05095a3c35bf None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:10.422235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-90049329-3600-477b-91bd-05095a3c35bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:10.489852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71e780-caf7-438e-b862-21995a245f45 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:10.499731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd37c7-acbd-416f-b048-eb274a1d48a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:10.500367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd37c7-acbd-416f-b048-eb274a1d48a2 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:10.500795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd37c7-acbd-416f-b048-eb274a1d48a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:10.501161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd37c7-acbd-416f-b048-eb274a1d48a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:10.513441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90049329-3600-477b-91bd-05095a3c35bf None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:10.517818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87ba11-842f-431d-91fc-cf8b0377c581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:10.518732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87ba11-842f-431d-91fc-cf8b0377c581 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:10.519040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87ba11-842f-431d-91fc-cf8b0377c581 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:10.519323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87ba11-842f-431d-91fc-cf8b0377c581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:10.522730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd37c7-acbd-416f-b048-eb274a1d48a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:10.524237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd37c7-acbd-416f-b048-eb274a1d48a2 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:10.531164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87ba11-842f-431d-91fc-cf8b0377c581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:10.532641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87ba11-842f-431d-91fc-cf8b0377c581 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:10.541335 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3516/7121] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:15:10.544653 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3606/7122] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:11.559435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd37c7-acbd-416f-b048-eb274a1d48a2 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:11.560608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-53bd37c7-acbd-416f-b048-eb274a1d48a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:11.575658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd37c7-acbd-416f-b048-eb274a1d48a2 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:11.580970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:11.581632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:11.581920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:11.582204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] PATH_INFO: `/v3/projects/25f222e844114110820cff009fe9939a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:11.583631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f222e844114110820cff009fe9939a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:11.585662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:11.645961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:11.653778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f222e844114110820cff009fe9939a', 'request_id': 'req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:11.665423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f222e844114110820cff009fe9939a', 'request_id': 'req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:11.672936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f222e844114110820cff009fe9939a', 'request_id': 'req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:11.687148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] The token cache is being invalidate because project 25f222e844114110820cff009fe9939a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:11.687856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:11.688845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:11.694967 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3517/7123] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:11 2022] DELETE /identity/v3/projects/25f222e844114110820cff009fe9939a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:11.696465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87ba11-842f-431d-91fc-cf8b0377c581 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:11.726468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87ba11-842f-431d-91fc-cf8b0377c581 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:11.734993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:11.819191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5ad59-4753-4d7d-8f91-698eced0616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:11.819889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5ad59-4753-4d7d-8f91-698eced0616c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:11.820122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5ad59-4753-4d7d-8f91-698eced0616c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:11.820423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5ad59-4753-4d7d-8f91-698eced0616c admin admin] PATH_INFO: `/v3/users/1664b56570d04ade8a8c792dd71ad5f8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:11.821766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5ad59-4753-4d7d-8f91-698eced0616c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1664b56570d04ade8a8c792dd71ad5f8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:11.824236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5ad59-4753-4d7d-8f91-698eced0616c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:11.917728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bed5ad59-4753-4d7d-8f91-698eced0616c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1664b56570d04ade8a8c792dd71ad5f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:11.937878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bed5ad59-4753-4d7d-8f91-698eced0616c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1664b56570d04ade8a8c792dd71ad5f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:11.947842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bed5ad59-4753-4d7d-8f91-698eced0616c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1664b56570d04ade8a8c792dd71ad5f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:11.949191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bed5ad59-4753-4d7d-8f91-698eced0616c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1664b56570d04ade8a8c792dd71ad5f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:11.949932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bed5ad59-4753-4d7d-8f91-698eced0616c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1664b56570d04ade8a8c792dd71ad5f8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:11.980268 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3607/7124] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:11 2022] DELETE /identity/v3/users/1664b56570d04ade8a8c792dd71ad5f8 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:12.694645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:12.696429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:12.717899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7b46b-c7b8-40b5-8893-8f361f9829e4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:12.796298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236525bb-7617-4df6-80a9-8782e144146b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:12.797062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236525bb-7617-4df6-80a9-8782e144146b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:12.797426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236525bb-7617-4df6-80a9-8782e144146b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:12.797756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236525bb-7617-4df6-80a9-8782e144146b admin admin] PATH_INFO: `/v3/projects/fa174a652f634a578aae372e6a0385af` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:12.798985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236525bb-7617-4df6-80a9-8782e144146b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa174a652f634a578aae372e6a0385af)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:12.801018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236525bb-7617-4df6-80a9-8782e144146b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:12.860942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-236525bb-7617-4df6-80a9-8782e144146b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa174a652f634a578aae372e6a0385af', 'request_id': 'req-236525bb-7617-4df6-80a9-8782e144146b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:12.869772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-236525bb-7617-4df6-80a9-8782e144146b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa174a652f634a578aae372e6a0385af', 'request_id': 'req-236525bb-7617-4df6-80a9-8782e144146b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:12.876923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-236525bb-7617-4df6-80a9-8782e144146b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa174a652f634a578aae372e6a0385af', 'request_id': 'req-236525bb-7617-4df6-80a9-8782e144146b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:12.895015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-236525bb-7617-4df6-80a9-8782e144146b admin admin] The token cache is being invalidate because project fa174a652f634a578aae372e6a0385af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:12.895237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-236525bb-7617-4df6-80a9-8782e144146b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:12.895903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-236525bb-7617-4df6-80a9-8782e144146b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:12.897651 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3518/7125] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:12 2022] DELETE /identity/v3/projects/fa174a652f634a578aae372e6a0385af => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:12.903007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecc0fb-25c0-4185-bd52-540f4f023d29 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:12.903505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecc0fb-25c0-4185-bd52-540f4f023d29 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:12.903590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecc0fb-25c0-4185-bd52-540f4f023d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:12.923933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5aecc0fb-25c0-4185-bd52-540f4f023d29 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:12.925257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5aecc0fb-25c0-4185-bd52-540f4f023d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:12.999522 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3608/7126] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:13.007246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903114b-2af3-4f69-8740-3ba8e0e8ea98 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:13.007871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903114b-2af3-4f69-8740-3ba8e0e8ea98 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:13.008270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903114b-2af3-4f69-8740-3ba8e0e8ea98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:13.037033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5903114b-2af3-4f69-8740-3ba8e0e8ea98 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:13.038283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5903114b-2af3-4f69-8740-3ba8e0e8ea98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:13.049551 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3519/7127] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:13.053609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecc0fb-25c0-4185-bd52-540f4f023d29 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:13.069453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecc0fb-25c0-4185-bd52-540f4f023d29 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:13.077205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad0e81-0f9c-4585-87f7-14a59ed4b7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:13.078073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad0e81-0f9c-4585-87f7-14a59ed4b7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:13.078483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad0e81-0f9c-4585-87f7-14a59ed4b7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:13.078770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad0e81-0f9c-4585-87f7-14a59ed4b7b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:13.079434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad0e81-0f9c-4585-87f7-14a59ed4b7b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:13.079810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad0e81-0f9c-4585-87f7-14a59ed4b7b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:13.082240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad0e81-0f9c-4585-87f7-14a59ed4b7b3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:13.102869 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3609/7128] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:13.104271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903114b-2af3-4f69-8740-3ba8e0e8ea98 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:13.127667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903114b-2af3-4f69-8740-3ba8e0e8ea98 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:13.145463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffda249-a094-40d9-97f0-377d6c305ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:13.146220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffda249-a094-40d9-97f0-377d6c305ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:13.146509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffda249-a094-40d9-97f0-377d6c305ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:13.147090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffda249-a094-40d9-97f0-377d6c305ff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:13.148225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffda249-a094-40d9-97f0-377d6c305ff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:13.150775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffda249-a094-40d9-97f0-377d6c305ff7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:13.219174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad0e81-0f9c-4585-87f7-14a59ed4b7b3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:13.223105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6fad0e81-0f9c-4585-87f7-14a59ed4b7b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:13.226144 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3520/7129] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:13 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:15:13.231677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffda249-a094-40d9-97f0-377d6c305ff7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:13.265740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad0e81-0f9c-4585-87f7-14a59ed4b7b3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:13.280664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:13.289619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:13.357300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffda249-a094-40d9-97f0-377d6c305ff7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:13.362689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61d6d4-168d-4b3a-86b2-0df70d0cfbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:13.363314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61d6d4-168d-4b3a-86b2-0df70d0cfbc4 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:13.363597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61d6d4-168d-4b3a-86b2-0df70d0cfbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:13.363872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61d6d4-168d-4b3a-86b2-0df70d0cfbc4 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:13.364688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61d6d4-168d-4b3a-86b2-0df70d0cfbc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:13.367552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61d6d4-168d-4b3a-86b2-0df70d0cfbc4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:13.374141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416b63d-2626-404b-bdd8-8ac69a657e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:13.375800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416b63d-2626-404b-bdd8-8ac69a657e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:13.376719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416b63d-2626-404b-bdd8-8ac69a657e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:13.377278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416b63d-2626-404b-bdd8-8ac69a657e7f admin admin] PATH_INFO: `/v3/users/9f792275f53f445ea72b6de441314a49` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:13.378988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416b63d-2626-404b-bdd8-8ac69a657e7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f792275f53f445ea72b6de441314a49)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:13.382271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416b63d-2626-404b-bdd8-8ac69a657e7f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:13.408928 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3521/7130] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:13 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:13.412156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61d6d4-168d-4b3a-86b2-0df70d0cfbc4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:13.430006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61d6d4-168d-4b3a-86b2-0df70d0cfbc4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:13.441664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310265d-2583-46db-94c9-a36f8a832abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:13.443013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310265d-2583-46db-94c9-a36f8a832abc admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:13.444518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310265d-2583-46db-94c9-a36f8a832abc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:13.445123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310265d-2583-46db-94c9-a36f8a832abc admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:13.446658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310265d-2583-46db-94c9-a36f8a832abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:13.451167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310265d-2583-46db-94c9-a36f8a832abc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:13.485257 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3522/7131] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:15:13.489865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310265d-2583-46db-94c9-a36f8a832abc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:13.526988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310265d-2583-46db-94c9-a36f8a832abc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:13.530817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2416b63d-2626-404b-bdd8-8ac69a657e7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f792275f53f445ea72b6de441314a49'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:13.539526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a4c7e-d7bc-43fd-b78a-ef8dec5c6e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:13.540491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a4c7e-d7bc-43fd-b78a-ef8dec5c6e69 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:13.540784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a4c7e-d7bc-43fd-b78a-ef8dec5c6e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:13.541590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a4c7e-d7bc-43fd-b78a-ef8dec5c6e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:13.541806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a4c7e-d7bc-43fd-b78a-ef8dec5c6e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:13.543983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a4c7e-d7bc-43fd-b78a-ef8dec5c6e69 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:13.547128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2416b63d-2626-404b-bdd8-8ac69a657e7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f792275f53f445ea72b6de441314a49'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:13.562126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2416b63d-2626-404b-bdd8-8ac69a657e7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f792275f53f445ea72b6de441314a49'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:13.562481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2416b63d-2626-404b-bdd8-8ac69a657e7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f792275f53f445ea72b6de441314a49'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:13.563250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2416b63d-2626-404b-bdd8-8ac69a657e7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f792275f53f445ea72b6de441314a49'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:13.564810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3523/7132] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:13.568775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a4c7e-d7bc-43fd-b78a-ef8dec5c6e69 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:13.570624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-827a4c7e-d7bc-43fd-b78a-ef8dec5c6e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:13.586524 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3610/7133] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:13 2022] DELETE /identity/v3/users/9f792275f53f445ea72b6de441314a49 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:13.586686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a4c7e-d7bc-43fd-b78a-ef8dec5c6e69 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:13.592419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864a58f-9181-43fb-85f7-6b5fbebd3d90 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:13.592955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864a58f-9181-43fb-85f7-6b5fbebd3d90 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:13.593343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864a58f-9181-43fb-85f7-6b5fbebd3d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:13.619258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0864a58f-9181-43fb-85f7-6b5fbebd3d90 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:13.621518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0864a58f-9181-43fb-85f7-6b5fbebd3d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:13.676987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f9ac0-261e-4558-8640-3ff3802d9c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:13.677680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f9ac0-261e-4558-8640-3ff3802d9c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:13.678017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f9ac0-261e-4558-8640-3ff3802d9c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:13.678788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f9ac0-261e-4558-8640-3ff3802d9c0f admin admin] PATH_INFO: `/v3/projects/130b90efb32f47e2ad1ece6145a5eb91/users/ad2782d0d6594bf295b8dcc945498c4c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:13.696857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f9ac0-261e-4558-8640-3ff3802d9c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=130b90efb32f47e2ad1ece6145a5eb91, user_id=ad2782d0d6594bf295b8dcc945498c4c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:13.699981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f9ac0-261e-4558-8640-3ff3802d9c0f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:13.718668 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3524/7134] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:13 2022] PUT /identity/v3/projects/130b90efb32f47e2ad1ece6145a5eb91/users/ad2782d0d6594bf295b8dcc945498c4c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:13.730113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa505b34-0be2-473b-aa2b-c5bf1e88cc02 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:13.730765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa505b34-0be2-473b-aa2b-c5bf1e88cc02 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:13.731244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa505b34-0be2-473b-aa2b-c5bf1e88cc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:13.735860 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3611/7135] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:13.747521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864a58f-9181-43fb-85f7-6b5fbebd3d90 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:13.750382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0864a58f-9181-43fb-85f7-6b5fbebd3d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:13.764747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fa505b34-0be2-473b-aa2b-c5bf1e88cc02 None None] MFA Rules not processed for user `ad2782d0d6594bf295b8dcc945498c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:13.766001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fa505b34-0be2-473b-aa2b-c5bf1e88cc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:13.814922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:13.842934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864a58f-9181-43fb-85f7-6b5fbebd3d90 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:13.856800 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3525/7136] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:13.862765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa505b34-0be2-473b-aa2b-c5bf1e88cc02 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:13.878329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:13.900749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9203b-8668-4d89-bbc7-cda40d558fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:13.901451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9203b-8668-4d89-bbc7-cda40d558fda service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:13.901737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9203b-8668-4d89-bbc7-cda40d558fda service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:13.902011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9203b-8668-4d89-bbc7-cda40d558fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:13.935826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa505b34-0be2-473b-aa2b-c5bf1e88cc02 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:13.943159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8df71-a170-4dd7-8b39-6e6a72e86516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:13.944477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8df71-a170-4dd7-8b39-6e6a72e86516 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:13.944772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8df71-a170-4dd7-8b39-6e6a72e86516 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:13.945088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8df71-a170-4dd7-8b39-6e6a72e86516 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:13.945879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8df71-a170-4dd7-8b39-6e6a72e86516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:13.947833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8df71-a170-4dd7-8b39-6e6a72e86516 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:13.972363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9203b-8668-4d89-bbc7-cda40d558fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:13.973859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9203b-8668-4d89-bbc7-cda40d558fda service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:13.981319 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3526/7137] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:13 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:13.985392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8df71-a170-4dd7-8b39-6e6a72e86516 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:13.985527 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3612/7138] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:14.072785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8df71-a170-4dd7-8b39-6e6a72e86516 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:14.078595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c6bd5-84bd-4101-825f-77298d3964a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:14.079867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c6bd5-84bd-4101-825f-77298d3964a3 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:14.080352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c6bd5-84bd-4101-825f-77298d3964a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:14.080826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c6bd5-84bd-4101-825f-77298d3964a3 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:14.081691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c6bd5-84bd-4101-825f-77298d3964a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:14.083994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c6bd5-84bd-4101-825f-77298d3964a3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:14.116961 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3527/7139] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:13 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:15:14.120425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9203b-8668-4d89-bbc7-cda40d558fda service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:14.137177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9203b-8668-4d89-bbc7-cda40d558fda service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:14.144389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69b5fa-a603-4627-a1cc-9dfda04435b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:14.145415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69b5fa-a603-4627-a1cc-9dfda04435b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:14.145415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69b5fa-a603-4627-a1cc-9dfda04435b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:14.145744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69b5fa-a603-4627-a1cc-9dfda04435b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:14.146593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69b5fa-a603-4627-a1cc-9dfda04435b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:14.148063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69b5fa-a603-4627-a1cc-9dfda04435b5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:14.159979 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3613/7140] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:14.164053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c6bd5-84bd-4101-825f-77298d3964a3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:14.176208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c6bd5-84bd-4101-825f-77298d3964a3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:14.198104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea367b1-1e20-44e1-9436-8bf960ae2a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:14.199344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea367b1-1e20-44e1-9436-8bf960ae2a32 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:14.199788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea367b1-1e20-44e1-9436-8bf960ae2a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:14.200223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea367b1-1e20-44e1-9436-8bf960ae2a32 admin admin] PATH_INFO: `/v3/projects/ae9e39b382a34f96bc8d16c01b0da7ec/users/0010fc5e9c954b2384c7c8eb989676ae/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:14.222869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69b5fa-a603-4627-a1cc-9dfda04435b5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:14.224943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7b69b5fa-a603-4627-a1cc-9dfda04435b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:14.238187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea367b1-1e20-44e1-9436-8bf960ae2a32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae9e39b382a34f96bc8d16c01b0da7ec, user_id=0010fc5e9c954b2384c7c8eb989676ae, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:14.241315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69b5fa-a603-4627-a1cc-9dfda04435b5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:14.247080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:14.247796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:14.248099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:14.248373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] PATH_INFO: `/v3/projects/d389730e46ff4f35bf0fec366e2be537` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:14.249734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea367b1-1e20-44e1-9436-8bf960ae2a32 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:14.250560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=d389730e46ff4f35bf0fec366e2be537)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:14.256826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:14.273169 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3528/7141] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:14 2022] PUT /identity/v3/projects/ae9e39b382a34f96bc8d16c01b0da7ec/users/0010fc5e9c954b2384c7c8eb989676ae/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:14.276603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea367b1-1e20-44e1-9436-8bf960ae2a32 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:14.343596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd389730e46ff4f35bf0fec366e2be537', 'request_id': 'req-2914c3ca-f449-43b4-9878-3d33c4fa72ad'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:14.355871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd389730e46ff4f35bf0fec366e2be537', 'request_id': 'req-2914c3ca-f449-43b4-9878-3d33c4fa72ad'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:14.361680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea367b1-1e20-44e1-9436-8bf960ae2a32 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:14.368467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd389730e46ff4f35bf0fec366e2be537', 'request_id': 'req-2914c3ca-f449-43b4-9878-3d33c4fa72ad'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:14.377739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:14.383587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] The token cache is being invalidate because project d389730e46ff4f35bf0fec366e2be537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:14.383944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:14.387326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:14.389543 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3614/7142] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:14 2022] DELETE /identity/v3/projects/d389730e46ff4f35bf0fec366e2be537 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:14.393660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:14.395434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:14.412162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:14.464416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e39bb6-9dee-4b7f-a91e-b4de8ec14455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:14.465562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e39bb6-9dee-4b7f-a91e-b4de8ec14455 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:14.466003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e39bb6-9dee-4b7f-a91e-b4de8ec14455 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:14.466649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e39bb6-9dee-4b7f-a91e-b4de8ec14455 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:14.467455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e39bb6-9dee-4b7f-a91e-b4de8ec14455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:14.469385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e39bb6-9dee-4b7f-a91e-b4de8ec14455 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:14.489945 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3529/7143] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:14.491015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e39bb6-9dee-4b7f-a91e-b4de8ec14455 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:14.494595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-84e39bb6-9dee-4b7f-a91e-b4de8ec14455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:14.497005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914c3ca-f449-43b4-9878-3d33c4fa72ad admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:14.502599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626d76a-4478-41f2-9070-2a5831659ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:14.503568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626d76a-4478-41f2-9070-2a5831659ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:14.504456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626d76a-4478-41f2-9070-2a5831659ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:14.504861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626d76a-4478-41f2-9070-2a5831659ce6 admin admin] PATH_INFO: `/v3/users/9a11dffdf5834c0bb605a50574e926c1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:14.507425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626d76a-4478-41f2-9070-2a5831659ce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a11dffdf5834c0bb605a50574e926c1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:14.509606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626d76a-4478-41f2-9070-2a5831659ce6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:14.520284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e39bb6-9dee-4b7f-a91e-b4de8ec14455 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:14.527310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63645101-a858-4628-aebb-1136f12bb077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:14.528587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645101-a858-4628-aebb-1136f12bb077 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:14.529676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645101-a858-4628-aebb-1136f12bb077 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:14.529676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645101-a858-4628-aebb-1136f12bb077 admin admin] PATH_INFO: `/v3/projects/ae9e39b382a34f96bc8d16c01b0da7ec/users/0010fc5e9c954b2384c7c8eb989676ae/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:14.531434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63645101-a858-4628-aebb-1136f12bb077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae9e39b382a34f96bc8d16c01b0da7ec, user_id=0010fc5e9c954b2384c7c8eb989676ae, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:14.534549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63645101-a858-4628-aebb-1136f12bb077 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:14.550962 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3530/7144] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:14 2022] PUT /identity/v3/projects/ae9e39b382a34f96bc8d16c01b0da7ec/users/0010fc5e9c954b2384c7c8eb989676ae/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:14.553059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045123f9-64e4-4f49-a89c-cda2bc7ffde7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:14.553796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045123f9-64e4-4f49-a89c-cda2bc7ffde7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:14.554268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045123f9-64e4-4f49-a89c-cda2bc7ffde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:14.583749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-045123f9-64e4-4f49-a89c-cda2bc7ffde7 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:14.585451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-045123f9-64e4-4f49-a89c-cda2bc7ffde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:14.597293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d626d76a-4478-41f2-9070-2a5831659ce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a11dffdf5834c0bb605a50574e926c1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:14.617678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d626d76a-4478-41f2-9070-2a5831659ce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a11dffdf5834c0bb605a50574e926c1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:14.631150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d626d76a-4478-41f2-9070-2a5831659ce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a11dffdf5834c0bb605a50574e926c1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:14.632615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d626d76a-4478-41f2-9070-2a5831659ce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a11dffdf5834c0bb605a50574e926c1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:14.633273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d626d76a-4478-41f2-9070-2a5831659ce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a11dffdf5834c0bb605a50574e926c1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:14.653483 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3615/7145] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:14 2022] DELETE /identity/v3/users/9a11dffdf5834c0bb605a50574e926c1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:14.657645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dca891-0f22-44c5-b8df-3157176fc2aa None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:14.657939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dca891-0f22-44c5-b8df-3157176fc2aa None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:14.658427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dca891-0f22-44c5-b8df-3157176fc2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:14.676999 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3531/7146] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:14.712448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-93dca891-0f22-44c5-b8df-3157176fc2aa None None] MFA Rules not processed for user `0010fc5e9c954b2384c7c8eb989676ae`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:14.713331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-93dca891-0f22-44c5-b8df-3157176fc2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:14.766453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045123f9-64e4-4f49-a89c-cda2bc7ffde7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:14.767985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-045123f9-64e4-4f49-a89c-cda2bc7ffde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:14.812304 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3616/7147] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:14.827382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dca891-0f22-44c5-b8df-3157176fc2aa None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:14.828823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-93dca891-0f22-44c5-b8df-3157176fc2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:14.874874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:14.891975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045123f9-64e4-4f49-a89c-cda2bc7ffde7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:14.927434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dca891-0f22-44c5-b8df-3157176fc2aa None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:14.931946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d4dbd-cf95-45e5-8dc5-ba1427cc629e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:14.932622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d4dbd-cf95-45e5-8dc5-ba1427cc629e service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:14.933020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d4dbd-cf95-45e5-8dc5-ba1427cc629e service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:14.933225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d4dbd-cf95-45e5-8dc5-ba1427cc629e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:14.935003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-849d4dbd-cf95-45e5-8dc5-ba1427cc629e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:14.980219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:15.003601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d4dbd-cf95-45e5-8dc5-ba1427cc629e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:15.005096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d4dbd-cf95-45e5-8dc5-ba1427cc629e service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:15.010568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fa82b-428b-43a7-a65f-cf508a8558da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:15.011204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fa82b-428b-43a7-a65f-cf508a8558da service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:15.011489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fa82b-428b-43a7-a65f-cf508a8558da service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:15.011776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fa82b-428b-43a7-a65f-cf508a8558da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:15.017264 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3617/7148] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:15.122949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fa82b-428b-43a7-a65f-cf508a8558da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:15.122949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fa82b-428b-43a7-a65f-cf508a8558da service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:15.129247 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3532/7149] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:15:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 21:15:15.397239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d4dbd-cf95-45e5-8dc5-ba1427cc629e service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:15.398753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-849d4dbd-cf95-45e5-8dc5-ba1427cc629e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:15.537755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d4dbd-cf95-45e5-8dc5-ba1427cc629e service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:15.544458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05bc29-e729-4ed9-a101-beab9a1af801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:15.545222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05bc29-e729-4ed9-a101-beab9a1af801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:15.545602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05bc29-e729-4ed9-a101-beab9a1af801 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:15.545899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05bc29-e729-4ed9-a101-beab9a1af801 admin admin] PATH_INFO: `/v3/projects/8b76aac08280467fafda70600597ae29` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:15.547108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05bc29-e729-4ed9-a101-beab9a1af801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b76aac08280467fafda70600597ae29)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:15.556615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05bc29-e729-4ed9-a101-beab9a1af801 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:15.604831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fa82b-428b-43a7-a65f-cf508a8558da service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:15.606297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6d9fa82b-428b-43a7-a65f-cf508a8558da service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:15.687842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e05bc29-e729-4ed9-a101-beab9a1af801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b76aac08280467fafda70600597ae29', 'request_id': 'req-0e05bc29-e729-4ed9-a101-beab9a1af801'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:15.721071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fa82b-428b-43a7-a65f-cf508a8558da service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:15.721930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e05bc29-e729-4ed9-a101-beab9a1af801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b76aac08280467fafda70600597ae29', 'request_id': 'req-0e05bc29-e729-4ed9-a101-beab9a1af801'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:15.738118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e05bc29-e729-4ed9-a101-beab9a1af801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b76aac08280467fafda70600597ae29', 'request_id': 'req-0e05bc29-e729-4ed9-a101-beab9a1af801'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:15.769909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e05bc29-e729-4ed9-a101-beab9a1af801 admin admin] The token cache is being invalidate because project 8b76aac08280467fafda70600597ae29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:15.770205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e05bc29-e729-4ed9-a101-beab9a1af801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:15.770989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0e05bc29-e729-4ed9-a101-beab9a1af801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:15.777022 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3618/7150] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:15 2022] DELETE /identity/v3/projects/8b76aac08280467fafda70600597ae29 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:15.779653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1a693-57ae-4c30-95a1-23d743229f05 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:15.780380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1a693-57ae-4c30-95a1-23d743229f05 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:15.780671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1a693-57ae-4c30-95a1-23d743229f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:15.789791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:15.807209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-90f1a693-57ae-4c30-95a1-23d743229f05 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:15.810588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-90f1a693-57ae-4c30-95a1-23d743229f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:15.824870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7dc1f-10fd-4e1e-a474-32b1d675766d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:15.826149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7dc1f-10fd-4e1e-a474-32b1d675766d service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:15.826741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7dc1f-10fd-4e1e-a474-32b1d675766d service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:15.827041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7dc1f-10fd-4e1e-a474-32b1d675766d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:15.829138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-92a7dc1f-10fd-4e1e-a474-32b1d675766d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:15.850485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7dc1f-10fd-4e1e-a474-32b1d675766d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:15.852545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7dc1f-10fd-4e1e-a474-32b1d675766d service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:15.868152 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3533/7151] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:15:15.972218 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3619/7152] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:15.981770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae006f-aad8-4037-bee0-a6ea965ca282 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:15.982301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae006f-aad8-4037-bee0-a6ea965ca282 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:15.982592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae006f-aad8-4037-bee0-a6ea965ca282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:16.009224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:16.023787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3bae006f-aad8-4037-bee0-a6ea965ca282 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:16.025065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3bae006f-aad8-4037-bee0-a6ea965ca282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:16.054095 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3534/7153] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:16.055747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1a693-57ae-4c30-95a1-23d743229f05 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:16.067635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1a693-57ae-4c30-95a1-23d743229f05 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:16.074188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c66696-3602-48ae-9637-229940034b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:16.075142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c66696-3602-48ae-9637-229940034b9a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:16.075434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c66696-3602-48ae-9637-229940034b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:16.075816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c66696-3602-48ae-9637-229940034b9a admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:16.076489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c66696-3602-48ae-9637-229940034b9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:16.076963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c66696-3602-48ae-9637-229940034b9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:16.078943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c66696-3602-48ae-9637-229940034b9a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:16.099116 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3620/7154] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:16.105001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae006f-aad8-4037-bee0-a6ea965ca282 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:16.115326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae006f-aad8-4037-bee0-a6ea965ca282 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:16.121388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1a6a3-7634-4297-bbe2-34f0e4d07488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:16.122021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1a6a3-7634-4297-bbe2-34f0e4d07488 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:16.122310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1a6a3-7634-4297-bbe2-34f0e4d07488 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:16.122591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1a6a3-7634-4297-bbe2-34f0e4d07488 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:16.123279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1a6a3-7634-4297-bbe2-34f0e4d07488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:16.128663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1a6a3-7634-4297-bbe2-34f0e4d07488 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:16.176029 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3535/7155] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:16 2022] POST /identity/v3/projects => generated 363 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:16.181200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c66696-3602-48ae-9637-229940034b9a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:16.194735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c66696-3602-48ae-9637-229940034b9a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:16.280734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3e096-4034-4046-9f8a-9983dad17ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:16.281472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3e096-4034-4046-9f8a-9983dad17ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:16.281769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3e096-4034-4046-9f8a-9983dad17ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:16.282318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3e096-4034-4046-9f8a-9983dad17ad3 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:16.283123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3e096-4034-4046-9f8a-9983dad17ad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:16.285157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3e096-4034-4046-9f8a-9983dad17ad3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:16.317267 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3621/7156] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:16 2022] POST /identity/v3/users => generated 282 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:16.322026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1a6a3-7634-4297-bbe2-34f0e4d07488 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:16.335293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1a6a3-7634-4297-bbe2-34f0e4d07488 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:16.341842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758cf5e-f83b-43b5-ac30-fb8ba58ddbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:16.342481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758cf5e-f83b-43b5-ac30-fb8ba58ddbff admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:16.342755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758cf5e-f83b-43b5-ac30-fb8ba58ddbff admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:16.343089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758cf5e-f83b-43b5-ac30-fb8ba58ddbff admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:16.345285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758cf5e-f83b-43b5-ac30-fb8ba58ddbff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:16.347267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758cf5e-f83b-43b5-ac30-fb8ba58ddbff admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:16.362629 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3536/7157] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:16.368604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3e096-4034-4046-9f8a-9983dad17ad3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:16.388174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3e096-4034-4046-9f8a-9983dad17ad3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:16.398967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9a814-02fb-42c3-8d9e-571370799f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:16.399606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9a814-02fb-42c3-8d9e-571370799f30 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:16.399895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9a814-02fb-42c3-8d9e-571370799f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:16.400179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9a814-02fb-42c3-8d9e-571370799f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:16.400843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9a814-02fb-42c3-8d9e-571370799f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:16.403597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9a814-02fb-42c3-8d9e-571370799f30 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:16.417807 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3622/7158] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:16.422554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758cf5e-f83b-43b5-ac30-fb8ba58ddbff admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:16.436912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758cf5e-f83b-43b5-ac30-fb8ba58ddbff admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:16.445472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30a89b-12a4-4766-9ecc-5efe2c01e205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:16.449638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30a89b-12a4-4766-9ecc-5efe2c01e205 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:16.449943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30a89b-12a4-4766-9ecc-5efe2c01e205 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:16.450749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30a89b-12a4-4766-9ecc-5efe2c01e205 admin admin] PATH_INFO: `/v3/projects/975e4b9f68ce4eb2b51ad6d351dda218/users/badb52e735874d5885042859c535840a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:16.473998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30a89b-12a4-4766-9ecc-5efe2c01e205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=975e4b9f68ce4eb2b51ad6d351dda218, user_id=badb52e735874d5885042859c535840a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:16.477376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30a89b-12a4-4766-9ecc-5efe2c01e205 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:16.522059 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3537/7159] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:16 2022] PUT /identity/v3/projects/975e4b9f68ce4eb2b51ad6d351dda218/users/badb52e735874d5885042859c535840a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:16.528304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cfc81-b76e-4d09-b49e-0ccae74ecd03 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:16.528590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cfc81-b76e-4d09-b49e-0ccae74ecd03 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:16.528985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cfc81-b76e-4d09-b49e-0ccae74ecd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:16.559038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7e1cfc81-b76e-4d09-b49e-0ccae74ecd03 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:16.560399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7e1cfc81-b76e-4d09-b49e-0ccae74ecd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:16.678293 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3623/7160] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:16.688996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30a89b-12a4-4766-9ecc-5efe2c01e205 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:16.691084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7b30a89b-12a4-4766-9ecc-5efe2c01e205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:16.839503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30a89b-12a4-4766-9ecc-5efe2c01e205 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:16.846766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ec2a0-b83d-449c-876f-a0fcc9f61ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:16.847704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ec2a0-b83d-449c-876f-a0fcc9f61ddc service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:16.848085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ec2a0-b83d-449c-876f-a0fcc9f61ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:16.848552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ec2a0-b83d-449c-876f-a0fcc9f61ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:16.897746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ec2a0-b83d-449c-876f-a0fcc9f61ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:16.899387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ec2a0-b83d-449c-876f-a0fcc9f61ddc service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:16.922919 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3538/7161] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:15:17.143151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1cfc81-b76e-4d09-b49e-0ccae74ecd03 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:17.145828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7e1cfc81-b76e-4d09-b49e-0ccae74ecd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:17.166221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1cfc81-b76e-4d09-b49e-0ccae74ecd03 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:17.174850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:17.176112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:17.176112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:17.176351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] PATH_INFO: `/v3/users/ad2782d0d6594bf295b8dcc945498c4c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:17.178687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad2782d0d6594bf295b8dcc945498c4c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:17.181099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:17.190226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:17.310288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad2782d0d6594bf295b8dcc945498c4c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:17.329735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad2782d0d6594bf295b8dcc945498c4c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:17.345354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad2782d0d6594bf295b8dcc945498c4c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:17.346130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad2782d0d6594bf295b8dcc945498c4c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:17.348431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad2782d0d6594bf295b8dcc945498c4c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:17.375492 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3624/7162] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:17 2022] DELETE /identity/v3/users/ad2782d0d6594bf295b8dcc945498c4c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:17.379760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c208c61-5e41-45ea-8014-ea6076588ee7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:17.380283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c208c61-5e41-45ea-8014-ea6076588ee7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:17.380768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c208c61-5e41-45ea-8014-ea6076588ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:17.412979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9c208c61-5e41-45ea-8014-ea6076588ee7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:17.414549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9c208c61-5e41-45ea-8014-ea6076588ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:17.558375 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3539/7163] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:17.580145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:17.581876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:17.686037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42547835-749c-4a6a-bd9f-abcf6bcd48bd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:17.692331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c77d42-79f2-4747-84dc-84b2c316aa62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:17.693679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c77d42-79f2-4747-84dc-84b2c316aa62 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:17.694130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c77d42-79f2-4747-84dc-84b2c316aa62 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:17.695861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c77d42-79f2-4747-84dc-84b2c316aa62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:17.710539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c77d42-79f2-4747-84dc-84b2c316aa62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:17.713622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c77d42-79f2-4747-84dc-84b2c316aa62 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:17.730558 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3625/7164] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:18.209704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c208c61-5e41-45ea-8014-ea6076588ee7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:18.212308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9c208c61-5e41-45ea-8014-ea6076588ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:18.252004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c208c61-5e41-45ea-8014-ea6076588ee7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:18.263742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:18.265739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:18.266265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:18.266645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] PATH_INFO: `/v3/projects/130b90efb32f47e2ad1ece6145a5eb91` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:18.278845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] RBAC: Authorizing `identity:delete_project(project_id=130b90efb32f47e2ad1ece6145a5eb91)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:18.285249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:18.367925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:18.388876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '130b90efb32f47e2ad1ece6145a5eb91', 'request_id': 'req-c62a94ca-a441-479e-9b17-13a3d99da59a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:18.399551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '130b90efb32f47e2ad1ece6145a5eb91', 'request_id': 'req-c62a94ca-a441-479e-9b17-13a3d99da59a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:18.411572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '130b90efb32f47e2ad1ece6145a5eb91', 'request_id': 'req-c62a94ca-a441-479e-9b17-13a3d99da59a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:18.429265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] The token cache is being invalidate because project 130b90efb32f47e2ad1ece6145a5eb91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:18.429569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:18.430265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:18.432087 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3540/7165] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:18 2022] DELETE /identity/v3/projects/130b90efb32f47e2ad1ece6145a5eb91 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:18.437373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c77d42-79f2-4747-84dc-84b2c316aa62 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:18.455951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c77d42-79f2-4747-84dc-84b2c316aa62 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:18.540623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ad3bc-915d-41c8-90c8-12ffbf586b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:18.541729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ad3bc-915d-41c8-90c8-12ffbf586b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:18.542255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ad3bc-915d-41c8-90c8-12ffbf586b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:18.542751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ad3bc-915d-41c8-90c8-12ffbf586b48 admin admin] PATH_INFO: `/v3/users/0010fc5e9c954b2384c7c8eb989676ae` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:18.549781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ad3bc-915d-41c8-90c8-12ffbf586b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0010fc5e9c954b2384c7c8eb989676ae)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:18.552640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ad3bc-915d-41c8-90c8-12ffbf586b48 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:18.670977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:18.707751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4d1ad3bc-915d-41c8-90c8-12ffbf586b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0010fc5e9c954b2384c7c8eb989676ae'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:18.732110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4d1ad3bc-915d-41c8-90c8-12ffbf586b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0010fc5e9c954b2384c7c8eb989676ae'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:18.748436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4d1ad3bc-915d-41c8-90c8-12ffbf586b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0010fc5e9c954b2384c7c8eb989676ae'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:18.749258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4d1ad3bc-915d-41c8-90c8-12ffbf586b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0010fc5e9c954b2384c7c8eb989676ae'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:18.749897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4d1ad3bc-915d-41c8-90c8-12ffbf586b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0010fc5e9c954b2384c7c8eb989676ae'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:18.811929 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3626/7166] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:18 2022] DELETE /identity/v3/users/0010fc5e9c954b2384c7c8eb989676ae => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:19.544298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:19.558790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:19.593543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a94ca-a441-479e-9b17-13a3d99da59a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:19.695577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:19.696570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:19.697065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:19.697518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1 admin admin] PATH_INFO: `/v3/projects/ae9e39b382a34f96bc8d16c01b0da7ec` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:19.699669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae9e39b382a34f96bc8d16c01b0da7ec)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:19.701835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:19.804398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae9e39b382a34f96bc8d16c01b0da7ec', 'request_id': 'req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:19.830448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae9e39b382a34f96bc8d16c01b0da7ec', 'request_id': 'req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:19.850659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae9e39b382a34f96bc8d16c01b0da7ec', 'request_id': 'req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:19.869360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1 admin admin] The token cache is being invalidate because project ae9e39b382a34f96bc8d16c01b0da7ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:19.869789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:19.871233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-baf1dfc3-19e1-45bc-85bc-3baa07a32ef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:19.873339 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3541/7167] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:19 2022] DELETE /identity/v3/projects/ae9e39b382a34f96bc8d16c01b0da7ec => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:19.888674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c8e6c-0c66-49f6-909b-e9a84f5b6fa7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:19.889262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c8e6c-0c66-49f6-909b-e9a84f5b6fa7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:19.889262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c8e6c-0c66-49f6-909b-e9a84f5b6fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:19.960155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-725c8e6c-0c66-49f6-909b-e9a84f5b6fa7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:19.965360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-725c8e6c-0c66-49f6-909b-e9a84f5b6fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:20.143078 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3627/7168] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:20.154655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185936ba-4ebb-446d-b479-accc7dd47a1c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:20.155402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185936ba-4ebb-446d-b479-accc7dd47a1c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:20.155948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185936ba-4ebb-446d-b479-accc7dd47a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:20.190414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-185936ba-4ebb-446d-b479-accc7dd47a1c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:20.192451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-185936ba-4ebb-446d-b479-accc7dd47a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:20.205851 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3542/7169] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:20.209890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c8e6c-0c66-49f6-909b-e9a84f5b6fa7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:20.232107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c8e6c-0c66-49f6-909b-e9a84f5b6fa7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:20.246060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a6be6-7323-420f-b4f4-b4d6c16fc855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:20.246931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a6be6-7323-420f-b4f4-b4d6c16fc855 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:20.247651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a6be6-7323-420f-b4f4-b4d6c16fc855 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:20.247651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a6be6-7323-420f-b4f4-b4d6c16fc855 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:20.248184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a6be6-7323-420f-b4f4-b4d6c16fc855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:20.248454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a6be6-7323-420f-b4f4-b4d6c16fc855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:20.250263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a6be6-7323-420f-b4f4-b4d6c16fc855 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:20.272526 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3628/7170] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:20.274096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185936ba-4ebb-446d-b479-accc7dd47a1c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:20.286635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185936ba-4ebb-446d-b479-accc7dd47a1c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:20.300075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87504853-02b6-4b47-9c63-5ac6b8497c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:20.300856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87504853-02b6-4b47-9c63-5ac6b8497c6a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:20.301254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87504853-02b6-4b47-9c63-5ac6b8497c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:20.301605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87504853-02b6-4b47-9c63-5ac6b8497c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:20.302416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87504853-02b6-4b47-9c63-5ac6b8497c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:20.304457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87504853-02b6-4b47-9c63-5ac6b8497c6a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:20.383139 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3543/7171] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:20 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:15:20.387564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a6be6-7323-420f-b4f4-b4d6c16fc855 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:20.419085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a6be6-7323-420f-b4f4-b4d6c16fc855 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:20.553197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60537592-2a8c-45a9-a599-19f7890ef2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:20.554105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60537592-2a8c-45a9-a599-19f7890ef2ba admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:20.554464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60537592-2a8c-45a9-a599-19f7890ef2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:20.554810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60537592-2a8c-45a9-a599-19f7890ef2ba admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:20.555745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60537592-2a8c-45a9-a599-19f7890ef2ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:20.558951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60537592-2a8c-45a9-a599-19f7890ef2ba admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:20.588807 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3629/7172] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:20 2022] POST /identity/v3/users => generated 294 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:20.590925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87504853-02b6-4b47-9c63-5ac6b8497c6a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:20.610274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87504853-02b6-4b47-9c63-5ac6b8497c6a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:20.622482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed011d1-ff86-46f8-bb08-b71c211dc340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:20.623487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed011d1-ff86-46f8-bb08-b71c211dc340 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:20.623992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed011d1-ff86-46f8-bb08-b71c211dc340 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:20.624484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed011d1-ff86-46f8-bb08-b71c211dc340 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:20.628101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed011d1-ff86-46f8-bb08-b71c211dc340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:20.630706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed011d1-ff86-46f8-bb08-b71c211dc340 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:20.643501 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3544/7173] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:20.647675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60537592-2a8c-45a9-a599-19f7890ef2ba admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:20.678658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60537592-2a8c-45a9-a599-19f7890ef2ba admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:20.702369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2f3f5-ea01-4211-b8d7-eb936ee99cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:20.703106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2f3f5-ea01-4211-b8d7-eb936ee99cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:20.703344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2f3f5-ea01-4211-b8d7-eb936ee99cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:20.703642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2f3f5-ea01-4211-b8d7-eb936ee99cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:20.707506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2f3f5-ea01-4211-b8d7-eb936ee99cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:20.712979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2f3f5-ea01-4211-b8d7-eb936ee99cd4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:20.753876 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3630/7174] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:20.756153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed011d1-ff86-46f8-bb08-b71c211dc340 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:20.793057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed011d1-ff86-46f8-bb08-b71c211dc340 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:20.804460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8af83-231d-4e4b-b772-f525bc406687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:20.813531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8af83-231d-4e4b-b772-f525bc406687 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:20.814316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8af83-231d-4e4b-b772-f525bc406687 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:20.815032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8af83-231d-4e4b-b772-f525bc406687 admin admin] PATH_INFO: `/v3/projects/5f42608558174193b1310a53518f530c/users/7270c81453d04b3b8644e38ed0aacd6d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:20.839798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8af83-231d-4e4b-b772-f525bc406687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f42608558174193b1310a53518f530c, user_id=7270c81453d04b3b8644e38ed0aacd6d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:20.853104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8af83-231d-4e4b-b772-f525bc406687 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:20.867594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:20.907006 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3545/7175] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:20 2022] PUT /identity/v3/projects/5f42608558174193b1310a53518f530c/users/7270c81453d04b3b8644e38ed0aacd6d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:20.917781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55535ba-39ba-49da-92fe-87bb43f03e84 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:20.921713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55535ba-39ba-49da-92fe-87bb43f03e84 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:20.922249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55535ba-39ba-49da-92fe-87bb43f03e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:20.995329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:21.028481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d55535ba-39ba-49da-92fe-87bb43f03e84 None None] MFA Rules not processed for user `7270c81453d04b3b8644e38ed0aacd6d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:21.036319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d55535ba-39ba-49da-92fe-87bb43f03e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:21.179748 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3631/7176] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:21.180973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8af83-231d-4e4b-b772-f525bc406687 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:21.387844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8af83-231d-4e4b-b772-f525bc406687 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:21.396123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f1b5d-0ed4-49b8-8060-59c4190bf346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:21.397741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f1b5d-0ed4-49b8-8060-59c4190bf346 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:21.398479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f1b5d-0ed4-49b8-8060-59c4190bf346 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:21.399501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f1b5d-0ed4-49b8-8060-59c4190bf346 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:21.401052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f1b5d-0ed4-49b8-8060-59c4190bf346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:21.403901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f1b5d-0ed4-49b8-8060-59c4190bf346 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:21.481131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55535ba-39ba-49da-92fe-87bb43f03e84 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:21.483556 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3546/7177] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:21 2022] POST /identity/v3/projects => generated 385 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:21.496210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55535ba-39ba-49da-92fe-87bb43f03e84 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:21.642688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbbce7-65f6-4e16-b86b-fc584f6ebf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:21.645343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbbce7-65f6-4e16-b86b-fc584f6ebf02 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:21.645950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbbce7-65f6-4e16-b86b-fc584f6ebf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:21.646348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbbce7-65f6-4e16-b86b-fc584f6ebf02 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:21.647358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbbce7-65f6-4e16-b86b-fc584f6ebf02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:21.649520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbbce7-65f6-4e16-b86b-fc584f6ebf02 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:21.727316 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3632/7178] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:21 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:21.734160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f1b5d-0ed4-49b8-8060-59c4190bf346 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:21.751586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f1b5d-0ed4-49b8-8060-59c4190bf346 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:21.777285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35bfef-b854-4898-a7d5-fd2e1b2e706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:21.777934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35bfef-b854-4898-a7d5-fd2e1b2e706c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:21.778438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35bfef-b854-4898-a7d5-fd2e1b2e706c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:21.778724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35bfef-b854-4898-a7d5-fd2e1b2e706c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:21.779392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35bfef-b854-4898-a7d5-fd2e1b2e706c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:21.781067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35bfef-b854-4898-a7d5-fd2e1b2e706c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:21.800209 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3547/7179] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:21.803866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbbce7-65f6-4e16-b86b-fc584f6ebf02 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:21.816058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbbce7-65f6-4e16-b86b-fc584f6ebf02 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:21.837934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0493d4-0a42-4755-a75b-0c76f909d7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:21.839436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0493d4-0a42-4755-a75b-0c76f909d7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:21.840174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0493d4-0a42-4755-a75b-0c76f909d7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:21.840976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0493d4-0a42-4755-a75b-0c76f909d7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:21.842612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0493d4-0a42-4755-a75b-0c76f909d7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:21.844278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0493d4-0a42-4755-a75b-0c76f909d7a5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:21.871987 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3633/7180] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:15:21.873651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35bfef-b854-4898-a7d5-fd2e1b2e706c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:21.893679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35bfef-b854-4898-a7d5-fd2e1b2e706c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:21.909749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf2a64-1c07-4283-9afe-72cc5f893332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:21.910760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf2a64-1c07-4283-9afe-72cc5f893332 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:21.911285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf2a64-1c07-4283-9afe-72cc5f893332 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:21.911672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf2a64-1c07-4283-9afe-72cc5f893332 admin admin] PATH_INFO: `/v3/projects/d127b89d61fe4348821958b32c42199b/users/57f2ba2a87234bd6b4e63e34474eeddc/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:21.952498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf2a64-1c07-4283-9afe-72cc5f893332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d127b89d61fe4348821958b32c42199b, user_id=57f2ba2a87234bd6b4e63e34474eeddc, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:21.959121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf2a64-1c07-4283-9afe-72cc5f893332 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:22.012136 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3548/7181] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:21 2022] PUT /identity/v3/projects/d127b89d61fe4348821958b32c42199b/users/57f2ba2a87234bd6b4e63e34474eeddc/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:22.017218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7180ad8-b11a-4df0-9a6e-d09e3425e2fe None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:22.017757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7180ad8-b11a-4df0-9a6e-d09e3425e2fe None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:22.017871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7180ad8-b11a-4df0-9a6e-d09e3425e2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:22.098012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a7180ad8-b11a-4df0-9a6e-d09e3425e2fe None None] MFA Rules not processed for user `57f2ba2a87234bd6b4e63e34474eeddc`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:22.098012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a7180ad8-b11a-4df0-9a6e-d09e3425e2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:22.235453 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3634/7182] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:22.240142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf2a64-1c07-4283-9afe-72cc5f893332 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:22.400619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf2a64-1c07-4283-9afe-72cc5f893332 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:22.409511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f3e4a-1282-4a95-8a47-572217a7bd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:22.410149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f3e4a-1282-4a95-8a47-572217a7bd10 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:22.410441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f3e4a-1282-4a95-8a47-572217a7bd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:22.410724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f3e4a-1282-4a95-8a47-572217a7bd10 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:22.411372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f3e4a-1282-4a95-8a47-572217a7bd10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:22.413468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f3e4a-1282-4a95-8a47-572217a7bd10 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:22.475321 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3549/7183] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:22 2022] POST /identity/v3/projects => generated 387 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:22.480488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7180ad8-b11a-4df0-9a6e-d09e3425e2fe None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:22.498676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7180ad8-b11a-4df0-9a6e-d09e3425e2fe None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:22.511731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:22.586282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac7b07-345f-4aaf-b5ad-aad9219df69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:22.587020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac7b07-345f-4aaf-b5ad-aad9219df69f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:22.587374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac7b07-345f-4aaf-b5ad-aad9219df69f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:22.587743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac7b07-345f-4aaf-b5ad-aad9219df69f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:22.590288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac7b07-345f-4aaf-b5ad-aad9219df69f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:22.594106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac7b07-345f-4aaf-b5ad-aad9219df69f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:22.647709 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3635/7184] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:22 2022] POST /identity/v3/users => generated 300 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:22.649553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f3e4a-1282-4a95-8a47-572217a7bd10 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:22.672250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f3e4a-1282-4a95-8a47-572217a7bd10 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:22.684747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca523b9b-3380-4289-9d84-1e936425ef60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:22.685689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca523b9b-3380-4289-9d84-1e936425ef60 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:22.686131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca523b9b-3380-4289-9d84-1e936425ef60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:22.686562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca523b9b-3380-4289-9d84-1e936425ef60 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:22.687372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca523b9b-3380-4289-9d84-1e936425ef60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:22.692876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca523b9b-3380-4289-9d84-1e936425ef60 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:22.731220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac7b07-345f-4aaf-b5ad-aad9219df69f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:22.742277 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3550/7185] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:22.743260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac7b07-345f-4aaf-b5ad-aad9219df69f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:22.762953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6482c-0b7f-41bf-9079-0e0d2250b852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:22.763718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6482c-0b7f-41bf-9079-0e0d2250b852 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:22.764011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6482c-0b7f-41bf-9079-0e0d2250b852 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:22.764294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6482c-0b7f-41bf-9079-0e0d2250b852 admin admin] PATH_INFO: `/v3/projects/5e652aa875cb4713adfd074083300bca/users/0bb7f2569e6a4c0e9e98052cf545dd3e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:22.788812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6482c-0b7f-41bf-9079-0e0d2250b852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e652aa875cb4713adfd074083300bca, user_id=0bb7f2569e6a4c0e9e98052cf545dd3e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:22.792956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6482c-0b7f-41bf-9079-0e0d2250b852 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:22.838154 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3636/7186] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:22 2022] PUT /identity/v3/projects/5e652aa875cb4713adfd074083300bca/users/0bb7f2569e6a4c0e9e98052cf545dd3e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:22.842746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca523b9b-3380-4289-9d84-1e936425ef60 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:22.858317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca523b9b-3380-4289-9d84-1e936425ef60 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:22.928740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:23.104124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8dddcd-4d24-4e64-b9fe-66f9da72fa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:23.105357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8dddcd-4d24-4e64-b9fe-66f9da72fa3d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:23.105661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8dddcd-4d24-4e64-b9fe-66f9da72fa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:23.105945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8dddcd-4d24-4e64-b9fe-66f9da72fa3d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:23.106605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8dddcd-4d24-4e64-b9fe-66f9da72fa3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:23.108316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8dddcd-4d24-4e64-b9fe-66f9da72fa3d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:23.134556 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3551/7187] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:23.139399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6482c-0b7f-41bf-9079-0e0d2250b852 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:23.160314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6482c-0b7f-41bf-9079-0e0d2250b852 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:23.170675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9b477-cc34-4aaa-9fa2-9732aac9dbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:23.172302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9b477-cc34-4aaa-9fa2-9732aac9dbc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:23.172760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9b477-cc34-4aaa-9fa2-9732aac9dbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:23.173796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9b477-cc34-4aaa-9fa2-9732aac9dbc0 admin admin] PATH_INFO: `/v3/projects/5e652aa875cb4713adfd074083300bca/users/0bb7f2569e6a4c0e9e98052cf545dd3e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:23.177068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9b477-cc34-4aaa-9fa2-9732aac9dbc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e652aa875cb4713adfd074083300bca, user_id=0bb7f2569e6a4c0e9e98052cf545dd3e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:23.180059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9b477-cc34-4aaa-9fa2-9732aac9dbc0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:23.207995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca6fa5-4769-421e-b5d5-1a97e295280e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:23.208556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca6fa5-4769-421e-b5d5-1a97e295280e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:23.208995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca6fa5-4769-421e-b5d5-1a97e295280e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:23.212440 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3637/7188] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:23 2022] PUT /identity/v3/projects/5e652aa875cb4713adfd074083300bca/users/0bb7f2569e6a4c0e9e98052cf545dd3e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:23.266282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-04ca6fa5-4769-421e-b5d5-1a97e295280e None None] MFA Rules not processed for user `0bb7f2569e6a4c0e9e98052cf545dd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:23.268236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-04ca6fa5-4769-421e-b5d5-1a97e295280e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:23.439016 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3552/7189] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:23.448704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9b477-cc34-4aaa-9fa2-9732aac9dbc0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:23.451377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3dd9b477-cc34-4aaa-9fa2-9732aac9dbc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:23.600945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9b477-cc34-4aaa-9fa2-9732aac9dbc0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:23.613052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15831fa5-d21c-4b3d-9356-c28be8c54d29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:23.625429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15831fa5-d21c-4b3d-9356-c28be8c54d29 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:23.627479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15831fa5-d21c-4b3d-9356-c28be8c54d29 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:23.627671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15831fa5-d21c-4b3d-9356-c28be8c54d29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:23.673555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15831fa5-d21c-4b3d-9356-c28be8c54d29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:23.675051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15831fa5-d21c-4b3d-9356-c28be8c54d29 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:23.720462 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3638/7190] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:24.717197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca6fa5-4769-421e-b5d5-1a97e295280e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:24.743287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca6fa5-4769-421e-b5d5-1a97e295280e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:24.749237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb6f0d-001c-4726-bebf-53d19359893c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:24.749887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb6f0d-001c-4726-bebf-53d19359893c service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:24.750175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb6f0d-001c-4726-bebf-53d19359893c service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:24.750457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb6f0d-001c-4726-bebf-53d19359893c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:24.895860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb6f0d-001c-4726-bebf-53d19359893c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:24.897366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb6f0d-001c-4726-bebf-53d19359893c service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:24.921130 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3553/7191] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:25.739859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15831fa5-d21c-4b3d-9356-c28be8c54d29 service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:25.757223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15831fa5-d21c-4b3d-9356-c28be8c54d29 service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:25.774156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045e2c8-7573-40fb-b2cb-2300a179da3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:25.775203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045e2c8-7573-40fb-b2cb-2300a179da3b service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:25.775697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045e2c8-7573-40fb-b2cb-2300a179da3b service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:25.776868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045e2c8-7573-40fb-b2cb-2300a179da3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:25.782158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f045e2c8-7573-40fb-b2cb-2300a179da3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:25.915505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:25.965741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045e2c8-7573-40fb-b2cb-2300a179da3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:25.967251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045e2c8-7573-40fb-b2cb-2300a179da3b service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:26.088869 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3639/7192] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:27.267586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce19b5-c35b-46ca-89e1-f66850930606 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:27.268423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce19b5-c35b-46ca-89e1-f66850930606 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:27.268854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce19b5-c35b-46ca-89e1-f66850930606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:27.340178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2cce19b5-c35b-46ca-89e1-f66850930606 None None] MFA Rules not processed for user `badb52e735874d5885042859c535840a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:27.341494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2cce19b5-c35b-46ca-89e1-f66850930606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:27.403942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:27.499937 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3554/7193] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:27.504198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045e2c8-7573-40fb-b2cb-2300a179da3b service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:27.506809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f045e2c8-7573-40fb-b2cb-2300a179da3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:27.601085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045e2c8-7573-40fb-b2cb-2300a179da3b service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:27.616430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf939d-3b1d-4a83-abc9-2886ecc8b257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:27.617254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf939d-3b1d-4a83-abc9-2886ecc8b257 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:27.617665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf939d-3b1d-4a83-abc9-2886ecc8b257 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:27.617806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf939d-3b1d-4a83-abc9-2886ecc8b257 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:27.618785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf939d-3b1d-4a83-abc9-2886ecc8b257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:27.621609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf939d-3b1d-4a83-abc9-2886ecc8b257 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:27.654932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce19b5-c35b-46ca-89e1-f66850930606 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:27.668039 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3640/7194] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:27 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:27.673848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce19b5-c35b-46ca-89e1-f66850930606 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:27.680877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c18ae-f10a-4db5-87e9-e66f3c1b03a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:27.681936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c18ae-f10a-4db5-87e9-e66f3c1b03a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:27.683005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c18ae-f10a-4db5-87e9-e66f3c1b03a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:27.683511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c18ae-f10a-4db5-87e9-e66f3c1b03a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:27.684392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c18ae-f10a-4db5-87e9-e66f3c1b03a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:27.686806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c18ae-f10a-4db5-87e9-e66f3c1b03a1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:27.706517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf939d-3b1d-4a83-abc9-2886ecc8b257 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:27.707779 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3555/7195] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:27.725216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf939d-3b1d-4a83-abc9-2886ecc8b257 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:27.742232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a91fd-7c7e-4d4f-8620-2cfdaf41e5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:27.743200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a91fd-7c7e-4d4f-8620-2cfdaf41e5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:27.743660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a91fd-7c7e-4d4f-8620-2cfdaf41e5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:27.744099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a91fd-7c7e-4d4f-8620-2cfdaf41e5a4 admin admin] PATH_INFO: `/v3/system/users/47a6d5fe42654e4e99eceaa507508ae2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:27.791357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a91fd-7c7e-4d4f-8620-2cfdaf41e5a4 admin admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=47a6d5fe42654e4e99eceaa507508ae2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:27.792461 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-0c9a91fd-7c7e-4d4f-8620-2cfdaf41e5a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:15:27.792461 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:15:27.792461 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:15:27.793412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a91fd-7c7e-4d4f-8620-2cfdaf41e5a4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:27.820389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a08dd2-32c9-46ad-b80f-b4139e3d9637 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:27.820683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a08dd2-32c9-46ad-b80f-b4139e3d9637 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:27.820990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a08dd2-32c9-46ad-b80f-b4139e3d9637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:27.824326 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3641/7196] 10.222.0.16 () {60 vars in 1378 bytes} [Sat Sep 3 21:15:27 2022] PUT /identity/v3/system/users/47a6d5fe42654e4e99eceaa507508ae2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:27.893104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-36a08dd2-32c9-46ad-b80f-b4139e3d9637 None None] MFA Rules not processed for user `47a6d5fe42654e4e99eceaa507508ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:27.896961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-36a08dd2-32c9-46ad-b80f-b4139e3d9637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:28.053160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a91fd-7c7e-4d4f-8620-2cfdaf41e5a4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:28.064820 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3556/7197] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:27 2022] POST /identity/v3/auth/tokens => generated 2415 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 21:15:28.222927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:28.249602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a91fd-7c7e-4d4f-8620-2cfdaf41e5a4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:28.315640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedf549-ca6b-42e4-88aa-53fcbec9fafb None tempest-ImageQuotaTest-1298920715-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47a6d5fe42654e4e99eceaa507508ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:28.316420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedf549-ca6b-42e4-88aa-53fcbec9fafb None tempest-ImageQuotaTest-1298920715-system-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:28.316718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedf549-ca6b-42e4-88aa-53fcbec9fafb None tempest-ImageQuotaTest-1298920715-system-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:28.317038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedf549-ca6b-42e4-88aa-53fcbec9fafb None tempest-ImageQuotaTest-1298920715-system-admin] PATH_INFO: `/v3/services` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:28.317664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beedf549-ca6b-42e4-88aa-53fcbec9fafb None tempest-ImageQuotaTest-1298920715-system-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:28.319115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beedf549-ca6b-42e4-88aa-53fcbec9fafb None tempest-ImageQuotaTest-1298920715-system-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:28.329845 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3642/7198] 10.222.0.16 () {58 vars in 1110 bytes} [Sat Sep 3 21:15:28 2022] GET /identity/v3/services => generated 2191 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:28.334073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a08dd2-32c9-46ad-b80f-b4139e3d9637 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:28.484081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:28.525280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a08dd2-32c9-46ad-b80f-b4139e3d9637 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:28.574605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1682f9-fcac-41e4-a8f7-fe326f00ec47 None tempest-ImageQuotaTest-1298920715-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47a6d5fe42654e4e99eceaa507508ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:28.575249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1682f9-fcac-41e4-a8f7-fe326f00ec47 None tempest-ImageQuotaTest-1298920715-system-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:28.575539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1682f9-fcac-41e4-a8f7-fe326f00ec47 None tempest-ImageQuotaTest-1298920715-system-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:28.575836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1682f9-fcac-41e4-a8f7-fe326f00ec47 None tempest-ImageQuotaTest-1298920715-system-admin] PATH_INFO: `/v3/limits` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:28.577380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1682f9-fcac-41e4-a8f7-fe326f00ec47 None tempest-ImageQuotaTest-1298920715-system-admin] RBAC: Authorizing `identity:create_limits()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:28.578716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1682f9-fcac-41e4-a8f7-fe326f00ec47 None tempest-ImageQuotaTest-1298920715-system-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-7b1682f9-fcac-41e4-a8f7-fe326f00ec47 None tempest-ImageQuotaTest-1298920715-system-admin] Unable to create a limit that has no corresponding registered limit.: keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/limits.py", line 110, in post Sep 03 21:15:28.637440 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application refs = PROVIDERS.unified_limit_api.create_limits(limits) Sep 03 21:15:28.638390 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:15:28.638390 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:15:28.638390 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/core.py", line 116, in create_limits Sep 03 21:15:28.638390 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.create_limits(limits) Sep 03 21:15:28.638390 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 03 21:15:28.638390 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 03 21:15:28.638390 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 260, in create_limits Sep 03 21:15:28.638390 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application target = self._check_and_fill_registered_limit_id(limit) Sep 03 21:15:28.638390 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 249, in _check_and_fill_registered_limit_id Sep 03 21:15:28.638390 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.NoLimitReference Sep 03 21:15:28.638390 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Sep 03 21:15:28.638390 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:15:28.639182 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3557/7199] 10.222.0.16 () {60 vars in 1126 bytes} [Sat Sep 3 21:15:28 2022] POST /identity/v3/limits => generated 124 bytes in 305 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Sep 03 21:15:28.650384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedf549-ca6b-42e4-88aa-53fcbec9fafb None tempest-ImageQuotaTest-1298920715-system-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:28.782473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedf549-ca6b-42e4-88aa-53fcbec9fafb None tempest-ImageQuotaTest-1298920715-system-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:28.834323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c765ed1-33f6-4ec6-a8a1-5e46bbda1f1b None tempest-ImageQuotaTest-1298920715-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47a6d5fe42654e4e99eceaa507508ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:28.834957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c765ed1-33f6-4ec6-a8a1-5e46bbda1f1b None tempest-ImageQuotaTest-1298920715-system-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:28.835246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c765ed1-33f6-4ec6-a8a1-5e46bbda1f1b None tempest-ImageQuotaTest-1298920715-system-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:28.836959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c765ed1-33f6-4ec6-a8a1-5e46bbda1f1b None tempest-ImageQuotaTest-1298920715-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:28.836959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c765ed1-33f6-4ec6-a8a1-5e46bbda1f1b None tempest-ImageQuotaTest-1298920715-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:28.840544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c765ed1-33f6-4ec6-a8a1-5e46bbda1f1b None tempest-ImageQuotaTest-1298920715-system-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:28.856795 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3643/7200] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Sep 3 21:15:28 2022] GET /identity/v3/registered_limits => generated 129 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:28.860357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1682f9-fcac-41e4-a8f7-fe326f00ec47 None tempest-ImageQuotaTest-1298920715-system-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:29.078809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1682f9-fcac-41e4-a8f7-fe326f00ec47 None tempest-ImageQuotaTest-1298920715-system-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:29.162946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5bbdc-431b-4a31-b481-c2b1c4957aa9 None tempest-ImageQuotaTest-1298920715-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47a6d5fe42654e4e99eceaa507508ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:29.163789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5bbdc-431b-4a31-b481-c2b1c4957aa9 None tempest-ImageQuotaTest-1298920715-system-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:29.165064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5bbdc-431b-4a31-b481-c2b1c4957aa9 None tempest-ImageQuotaTest-1298920715-system-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:29.165378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5bbdc-431b-4a31-b481-c2b1c4957aa9 None tempest-ImageQuotaTest-1298920715-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:29.166488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5bbdc-431b-4a31-b481-c2b1c4957aa9 None tempest-ImageQuotaTest-1298920715-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:29.167793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5bbdc-431b-4a31-b481-c2b1c4957aa9 None tempest-ImageQuotaTest-1298920715-system-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:29.192306 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3558/7201] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Sep 3 21:15:28 2022] GET /identity/v3/registered_limits => generated 129 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:30.107368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c765ed1-33f6-4ec6-a8a1-5e46bbda1f1b None tempest-ImageQuotaTest-1298920715-system-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:30.123354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c765ed1-33f6-4ec6-a8a1-5e46bbda1f1b None tempest-ImageQuotaTest-1298920715-system-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:30.129407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:30.130100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:30.130577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:30.130962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] PATH_INFO: `/v3/users/7270c81453d04b3b8644e38ed0aacd6d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:30.132375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7270c81453d04b3b8644e38ed0aacd6d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:30.134867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:30.164844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:30.227075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7270c81453d04b3b8644e38ed0aacd6d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:30.249030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7270c81453d04b3b8644e38ed0aacd6d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:30.267883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7270c81453d04b3b8644e38ed0aacd6d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:30.268761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7270c81453d04b3b8644e38ed0aacd6d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:30.269889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7270c81453d04b3b8644e38ed0aacd6d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:30.292852 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3644/7202] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:30 2022] DELETE /identity/v3/users/7270c81453d04b3b8644e38ed0aacd6d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:30.297080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a358e4-0aae-482a-ac6d-1e053d06db07 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:30.298027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a358e4-0aae-482a-ac6d-1e053d06db07 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:30.298470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a358e4-0aae-482a-ac6d-1e053d06db07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:30.306018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:30.333838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-39a358e4-0aae-482a-ac6d-1e053d06db07 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:30.335141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-39a358e4-0aae-482a-ac6d-1e053d06db07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:30.438426 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3559/7203] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:30.462074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:30.464448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:30.559620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5dcea3-5b64-4e16-9c58-006a05a0cb0d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:30.563772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a9131-62ad-4978-a010-4ae3eabb010d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:30.564408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a9131-62ad-4978-a010-4ae3eabb010d service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:30.564700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a9131-62ad-4978-a010-4ae3eabb010d service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:30.565083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a9131-62ad-4978-a010-4ae3eabb010d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:30.580282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a9131-62ad-4978-a010-4ae3eabb010d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:30.586853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a9131-62ad-4978-a010-4ae3eabb010d service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:30.602491 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3645/7204] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:31.717088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a358e4-0aae-482a-ac6d-1e053d06db07 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:31.719258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-39a358e4-0aae-482a-ac6d-1e053d06db07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:31.737901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a358e4-0aae-482a-ac6d-1e053d06db07 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:31.743472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:31.744128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:31.744420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:31.744750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] PATH_INFO: `/v3/projects/5f42608558174193b1310a53518f530c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:31.746741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f42608558174193b1310a53518f530c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:31.749113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:31.861630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f42608558174193b1310a53518f530c', 'request_id': 'req-76c1d1a5-dfa8-4e4f-961e-e897455ed730'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:31.872669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f42608558174193b1310a53518f530c', 'request_id': 'req-76c1d1a5-dfa8-4e4f-961e-e897455ed730'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:31.898064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f42608558174193b1310a53518f530c', 'request_id': 'req-76c1d1a5-dfa8-4e4f-961e-e897455ed730'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:31.931233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] The token cache is being invalidate because project 5f42608558174193b1310a53518f530c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:31.931550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:31.933391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:31.935215 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3560/7205] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:31 2022] DELETE /identity/v3/projects/5f42608558174193b1310a53518f530c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:31.941528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a9131-62ad-4978-a010-4ae3eabb010d service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:31.952506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a9131-62ad-4978-a010-4ae3eabb010d service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:32.077330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:32.078046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:32.078406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:32.078663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] PATH_INFO: `/v3/users/57f2ba2a87234bd6b4e63e34474eeddc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:32.080083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57f2ba2a87234bd6b4e63e34474eeddc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:32.084079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:32.160596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:32.176865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57f2ba2a87234bd6b4e63e34474eeddc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:32.207825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57f2ba2a87234bd6b4e63e34474eeddc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:32.229492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57f2ba2a87234bd6b4e63e34474eeddc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:32.231749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57f2ba2a87234bd6b4e63e34474eeddc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:32.233528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57f2ba2a87234bd6b4e63e34474eeddc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:32.274549 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3646/7206] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:31 2022] DELETE /identity/v3/users/57f2ba2a87234bd6b4e63e34474eeddc => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:33.265771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:33.267763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:33.290923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1d1a5-dfa8-4e4f-961e-e897455ed730 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:33.459407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:33.461341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:33.461341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:33.461341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] PATH_INFO: `/v3/projects/d127b89d61fe4348821958b32c42199b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:33.461935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d127b89d61fe4348821958b32c42199b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:33.464096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:33.537428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:33.571328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd127b89d61fe4348821958b32c42199b', 'request_id': 'req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:33.589913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd127b89d61fe4348821958b32c42199b', 'request_id': 'req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:33.615836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd127b89d61fe4348821958b32c42199b', 'request_id': 'req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:33.647324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] The token cache is being invalidate because project d127b89d61fe4348821958b32c42199b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:33.649021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:33.650162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:33.656547 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3561/7207] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:33 2022] DELETE /identity/v3/projects/d127b89d61fe4348821958b32c42199b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:33.660781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:33.663240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:33.696281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eeac9c-419e-4374-acc6-7cd2b39e6692 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:33.859105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb75ca-046b-438b-938d-fa4976ea5994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:33.860042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb75ca-046b-438b-938d-fa4976ea5994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:33.860511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb75ca-046b-438b-938d-fa4976ea5994 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:33.861036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb75ca-046b-438b-938d-fa4976ea5994 admin admin] PATH_INFO: `/v3/users/0bb7f2569e6a4c0e9e98052cf545dd3e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:33.864315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb75ca-046b-438b-938d-fa4976ea5994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb7f2569e6a4c0e9e98052cf545dd3e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:33.866739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb75ca-046b-438b-938d-fa4976ea5994 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:33.960596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a3fb75ca-046b-438b-938d-fa4976ea5994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb7f2569e6a4c0e9e98052cf545dd3e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:33.988767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a3fb75ca-046b-438b-938d-fa4976ea5994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb7f2569e6a4c0e9e98052cf545dd3e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:33.999023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a3fb75ca-046b-438b-938d-fa4976ea5994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb7f2569e6a4c0e9e98052cf545dd3e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:33.999758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a3fb75ca-046b-438b-938d-fa4976ea5994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb7f2569e6a4c0e9e98052cf545dd3e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:34.000537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a3fb75ca-046b-438b-938d-fa4976ea5994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb7f2569e6a4c0e9e98052cf545dd3e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:34.033736 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3647/7208] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:33 2022] DELETE /identity/v3/users/0bb7f2569e6a4c0e9e98052cf545dd3e => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:35.071325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:35.075093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:35.096239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de83cc2-a41f-4d21-8dfe-2af9d567dd98 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:35.176250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:35.177212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:35.177588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:35.177964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] PATH_INFO: `/v3/projects/5e652aa875cb4713adfd074083300bca` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:35.179449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e652aa875cb4713adfd074083300bca)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:35.181786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:35.243503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e652aa875cb4713adfd074083300bca', 'request_id': 'req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:35.259058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e652aa875cb4713adfd074083300bca', 'request_id': 'req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:35.268747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e652aa875cb4713adfd074083300bca', 'request_id': 'req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:35.280802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] The token cache is being invalidate because project 5e652aa875cb4713adfd074083300bca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:35.281247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:35.281920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:35.283674 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3562/7209] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:35 2022] DELETE /identity/v3/projects/5e652aa875cb4713adfd074083300bca => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:35.288238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634ff69-40d2-4c1b-957f-60488c2b9456 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:35.288714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634ff69-40d2-4c1b-957f-60488c2b9456 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:35.288957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634ff69-40d2-4c1b-957f-60488c2b9456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:35.311579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a634ff69-40d2-4c1b-957f-60488c2b9456 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:35.312984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a634ff69-40d2-4c1b-957f-60488c2b9456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:35.346202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:35.365423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:35.367316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:35.388103 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3648/7210] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:35.397402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f12ce-7dbe-4237-9d1f-003cef86aa3f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:35.398080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f12ce-7dbe-4237-9d1f-003cef86aa3f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:35.398632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f12ce-7dbe-4237-9d1f-003cef86aa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:35.422725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1a9f12ce-7dbe-4237-9d1f-003cef86aa3f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:35.424051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1a9f12ce-7dbe-4237-9d1f-003cef86aa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:35.432279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:35.436937 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3649/7211] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:35.439059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f12ce-7dbe-4237-9d1f-003cef86aa3f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:35.450664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f12ce-7dbe-4237-9d1f-003cef86aa3f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:35.454421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f93a1-4b95-4eb2-8d46-81842d7c31c0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:35.456415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286f74e-d723-497a-846a-2d72778b27bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:35.457409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286f74e-d723-497a-846a-2d72778b27bb admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:35.457844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286f74e-d723-497a-846a-2d72778b27bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:35.458283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286f74e-d723-497a-846a-2d72778b27bb admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:35.459536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286f74e-d723-497a-846a-2d72778b27bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:35.460007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286f74e-d723-497a-846a-2d72778b27bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:35.460133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:35.460687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:35.461007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:35.461290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] PATH_INFO: `/v3/users/badb52e735874d5885042859c535840a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:35.462491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] RBAC: Authorizing `identity:delete_user(user_id=badb52e735874d5885042859c535840a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:35.462899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286f74e-d723-497a-846a-2d72778b27bb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:35.465027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:35.479839 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3650/7212] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:15:35.482562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286f74e-d723-497a-846a-2d72778b27bb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:35.493520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286f74e-d723-497a-846a-2d72778b27bb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:35.499612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe33be5-d116-49d1-9b77-0b1d22c89f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:35.500219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe33be5-d116-49d1-9b77-0b1d22c89f98 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:35.500507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe33be5-d116-49d1-9b77-0b1d22c89f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:35.501279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe33be5-d116-49d1-9b77-0b1d22c89f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:35.501452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe33be5-d116-49d1-9b77-0b1d22c89f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:35.503460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe33be5-d116-49d1-9b77-0b1d22c89f98 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:35.524812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'badb52e735874d5885042859c535840a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:35.536446 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3651/7213] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:35 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:15:35.540526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe33be5-d116-49d1-9b77-0b1d22c89f98 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:35.545418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'badb52e735874d5885042859c535840a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:35.546335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'badb52e735874d5885042859c535840a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:35.554594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'badb52e735874d5885042859c535840a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:35.569925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'badb52e735874d5885042859c535840a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:35.578879 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3563/7214] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:35 2022] DELETE /identity/v3/users/badb52e735874d5885042859c535840a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:35.659759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe33be5-d116-49d1-9b77-0b1d22c89f98 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:35.782985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95c07c-65fa-4084-8c59-c31e81dac4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:35.784350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95c07c-65fa-4084-8c59-c31e81dac4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:35.784803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95c07c-65fa-4084-8c59-c31e81dac4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:35.785719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95c07c-65fa-4084-8c59-c31e81dac4e3 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:35.787012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95c07c-65fa-4084-8c59-c31e81dac4e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:35.790447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95c07c-65fa-4084-8c59-c31e81dac4e3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:35.827736 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3652/7215] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:35 2022] POST /identity/v3/users => generated 289 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:35.832283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:35.833718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:35.852117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff5941-56bb-47fb-8ee6-3ef3db0e052b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:35.858786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50b97d-4905-4725-8daa-4b360689b0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:35.859420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50b97d-4905-4725-8daa-4b360689b0bd admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:35.859716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50b97d-4905-4725-8daa-4b360689b0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:35.860112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50b97d-4905-4725-8daa-4b360689b0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:35.860781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50b97d-4905-4725-8daa-4b360689b0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:35.862593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50b97d-4905-4725-8daa-4b360689b0bd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:35.884763 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3564/7216] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:35.890835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95c07c-65fa-4084-8c59-c31e81dac4e3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:35.906132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50b97d-4905-4725-8daa-4b360689b0bd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:35.906880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95c07c-65fa-4084-8c59-c31e81dac4e3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:35.909194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ec50b97d-4905-4725-8daa-4b360689b0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:35.921613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436c044-441b-47c2-af22-3733f83180f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:35.922868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436c044-441b-47c2-af22-3733f83180f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:35.926190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436c044-441b-47c2-af22-3733f83180f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:35.926565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436c044-441b-47c2-af22-3733f83180f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:35.927538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436c044-441b-47c2-af22-3733f83180f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:35.929297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436c044-441b-47c2-af22-3733f83180f7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:35.937440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50b97d-4905-4725-8daa-4b360689b0bd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:35.942631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:35.943737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:35.944095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:35.944613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] PATH_INFO: `/v3/projects/975e4b9f68ce4eb2b51ad6d351dda218` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:35.945908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=975e4b9f68ce4eb2b51ad6d351dda218)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:35.947211 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3653/7217] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:35.954086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436c044-441b-47c2-af22-3733f83180f7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:35.954605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:35.971077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436c044-441b-47c2-af22-3733f83180f7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:35.976885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526f1c0-74ff-4e4d-96d3-38a661b430be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:35.978296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526f1c0-74ff-4e4d-96d3-38a661b430be admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:35.979331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526f1c0-74ff-4e4d-96d3-38a661b430be admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:35.988644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526f1c0-74ff-4e4d-96d3-38a661b430be admin admin] PATH_INFO: `/v3/projects/2584ba5951f948519ef50611c60e0f37/users/626a5aa5b6344d36966d0e9f745d0671/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:36.019037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526f1c0-74ff-4e4d-96d3-38a661b430be admin admin] RBAC: Authorizing `identity:create_grant(project_id=2584ba5951f948519ef50611c60e0f37, user_id=626a5aa5b6344d36966d0e9f745d0671, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:36.024599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526f1c0-74ff-4e4d-96d3-38a661b430be admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:36.031647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:36.046097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '975e4b9f68ce4eb2b51ad6d351dda218', 'request_id': 'req-ec4dbf92-94df-424e-baaa-ca75a42bea6d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:36.062148 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3654/7218] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:35 2022] PUT /identity/v3/projects/2584ba5951f948519ef50611c60e0f37/users/626a5aa5b6344d36966d0e9f745d0671/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:36.066238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05752d76-ec1a-4353-b4d6-5f7014be23ba None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:36.067234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05752d76-ec1a-4353-b4d6-5f7014be23ba None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:36.067877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05752d76-ec1a-4353-b4d6-5f7014be23ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:36.077580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '975e4b9f68ce4eb2b51ad6d351dda218', 'request_id': 'req-ec4dbf92-94df-424e-baaa-ca75a42bea6d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:36.107494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-05752d76-ec1a-4353-b4d6-5f7014be23ba None None] MFA Rules not processed for user `626a5aa5b6344d36966d0e9f745d0671`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:36.109230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-05752d76-ec1a-4353-b4d6-5f7014be23ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:36.116578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '975e4b9f68ce4eb2b51ad6d351dda218', 'request_id': 'req-ec4dbf92-94df-424e-baaa-ca75a42bea6d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:36.149585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] The token cache is being invalidate because project 975e4b9f68ce4eb2b51ad6d351dda218 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:36.149585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:36.149585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:36.149585 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3565/7219] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:35 2022] DELETE /identity/v3/projects/975e4b9f68ce4eb2b51ad6d351dda218 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:36.152374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:36.155996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:36.230254 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3655/7220] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:36.231222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05752d76-ec1a-4353-b4d6-5f7014be23ba None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:36.233194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-05752d76-ec1a-4353-b4d6-5f7014be23ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:36.270581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:36.322955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dbf92-94df-424e-baaa-ca75a42bea6d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:36.339633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05752d76-ec1a-4353-b4d6-5f7014be23ba None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:36.344811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d7e99-b6d6-48db-9d61-8acf88d6d49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:36.345317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d7e99-b6d6-48db-9d61-8acf88d6d49c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:36.345857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d7e99-b6d6-48db-9d61-8acf88d6d49c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:36.346149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d7e99-b6d6-48db-9d61-8acf88d6d49c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:36.346809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d7e99-b6d6-48db-9d61-8acf88d6d49c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:36.349036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d7e99-b6d6-48db-9d61-8acf88d6d49c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:36.391294 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3656/7221] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:36 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:36.395370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d7e99-b6d6-48db-9d61-8acf88d6d49c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:36.429369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd591df8-838d-4d87-9874-3f642fc61c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:36.431499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd591df8-838d-4d87-9874-3f642fc61c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:36.432635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd591df8-838d-4d87-9874-3f642fc61c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:36.433144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd591df8-838d-4d87-9874-3f642fc61c19 admin admin] PATH_INFO: `/v3/users/47a6d5fe42654e4e99eceaa507508ae2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:36.434645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd591df8-838d-4d87-9874-3f642fc61c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47a6d5fe42654e4e99eceaa507508ae2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:36.437283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd591df8-838d-4d87-9874-3f642fc61c19 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:36.512270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d7e99-b6d6-48db-9d61-8acf88d6d49c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:36.518151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991e985-d3f0-49cb-a177-a658ba0670b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:36.518807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991e985-d3f0-49cb-a177-a658ba0670b6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:36.519261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991e985-d3f0-49cb-a177-a658ba0670b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:36.519548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991e985-d3f0-49cb-a177-a658ba0670b6 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:36.520194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991e985-d3f0-49cb-a177-a658ba0670b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:36.522457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991e985-d3f0-49cb-a177-a658ba0670b6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:36.539800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bd591df8-838d-4d87-9874-3f642fc61c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47a6d5fe42654e4e99eceaa507508ae2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:36.552661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:36.561686 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3657/7222] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:36 2022] POST /identity/v3/users => generated 296 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:15:36.565005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991e985-d3f0-49cb-a177-a658ba0670b6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:36.571883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bd591df8-838d-4d87-9874-3f642fc61c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47a6d5fe42654e4e99eceaa507508ae2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:36.573297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bd591df8-838d-4d87-9874-3f642fc61c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47a6d5fe42654e4e99eceaa507508ae2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:36.580480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991e985-d3f0-49cb-a177-a658ba0670b6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:36.586912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bd591df8-838d-4d87-9874-3f642fc61c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47a6d5fe42654e4e99eceaa507508ae2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:36.599257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9cd68-e9a1-456f-af38-e2633a9771f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:36.600941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9cd68-e9a1-456f-af38-e2633a9771f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:36.601306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9cd68-e9a1-456f-af38-e2633a9771f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:36.605199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9cd68-e9a1-456f-af38-e2633a9771f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:36.606255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9cd68-e9a1-456f-af38-e2633a9771f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:36.607440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bd591df8-838d-4d87-9874-3f642fc61c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47a6d5fe42654e4e99eceaa507508ae2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:36.609119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9cd68-e9a1-456f-af38-e2633a9771f2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:36.618455 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3566/7223] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:36 2022] DELETE /identity/v3/users/47a6d5fe42654e4e99eceaa507508ae2 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:36.625278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee8748-5c66-4974-a374-b2d2bbbc1f81 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:36.625690 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3658/7224] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:36.628800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee8748-5c66-4974-a374-b2d2bbbc1f81 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:36.629228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee8748-5c66-4974-a374-b2d2bbbc1f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:36.630838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9cd68-e9a1-456f-af38-e2633a9771f2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:36.633030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4fb9cd68-e9a1-456f-af38-e2633a9771f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:36.655074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9cd68-e9a1-456f-af38-e2633a9771f2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:36.655825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-67ee8748-5c66-4974-a374-b2d2bbbc1f81 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:36.657826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-67ee8748-5c66-4974-a374-b2d2bbbc1f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:36.740618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:36.756923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98044a59-74bd-4d67-a50f-ec89e4ff68da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:36.757262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98044a59-74bd-4d67-a50f-ec89e4ff68da admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:36.758047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98044a59-74bd-4d67-a50f-ec89e4ff68da admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:36.758047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98044a59-74bd-4d67-a50f-ec89e4ff68da admin admin] PATH_INFO: `/v3/projects/02eb4d069635439aa3f5af0de672b889/users/c0258b53445e4f99a70a8218812105ee/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:36.767223 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3567/7225] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:36.777631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b4947-b51c-4d5c-966d-9a1b7a772fe1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:36.778156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b4947-b51c-4d5c-966d-9a1b7a772fe1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:36.778509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b4947-b51c-4d5c-966d-9a1b7a772fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:36.783653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98044a59-74bd-4d67-a50f-ec89e4ff68da admin admin] RBAC: Authorizing `identity:create_grant(project_id=02eb4d069635439aa3f5af0de672b889, user_id=c0258b53445e4f99a70a8218812105ee, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:36.786963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98044a59-74bd-4d67-a50f-ec89e4ff68da admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:36.807647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-706b4947-b51c-4d5c-966d-9a1b7a772fe1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:36.809507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-706b4947-b51c-4d5c-966d-9a1b7a772fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:36.815466 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3659/7226] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:36 2022] PUT /identity/v3/projects/02eb4d069635439aa3f5af0de672b889/users/c0258b53445e4f99a70a8218812105ee/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:36.820098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98044a59-74bd-4d67-a50f-ec89e4ff68da admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:36.918926 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3568/7227] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:36.924130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98044a59-74bd-4d67-a50f-ec89e4ff68da admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:36.926231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b4947-b51c-4d5c-966d-9a1b7a772fe1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:36.929678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afe9e0-17d3-44b0-b46b-9eb81b89916d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:36.930821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afe9e0-17d3-44b0-b46b-9eb81b89916d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:36.931463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afe9e0-17d3-44b0-b46b-9eb81b89916d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:36.931751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afe9e0-17d3-44b0-b46b-9eb81b89916d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:36.932467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afe9e0-17d3-44b0-b46b-9eb81b89916d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:36.934174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afe9e0-17d3-44b0-b46b-9eb81b89916d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:36.938282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b4947-b51c-4d5c-966d-9a1b7a772fe1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:36.944398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbdd30-c70c-484f-b6db-15938afe5948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:36.948513 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3660/7228] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:36.948767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbdd30-c70c-484f-b6db-15938afe5948 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:36.949088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbdd30-c70c-484f-b6db-15938afe5948 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:36.949438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbdd30-c70c-484f-b6db-15938afe5948 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:36.950350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbdd30-c70c-484f-b6db-15938afe5948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:36.950475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbdd30-c70c-484f-b6db-15938afe5948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:36.952596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afe9e0-17d3-44b0-b46b-9eb81b89916d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:36.953767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbdd30-c70c-484f-b6db-15938afe5948 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:36.977545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afe9e0-17d3-44b0-b46b-9eb81b89916d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:36.981110 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3569/7229] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:36.986286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbdd30-c70c-484f-b6db-15938afe5948 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:36.991904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba643e33-c650-4254-b594-1323063b0247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:36.992591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba643e33-c650-4254-b594-1323063b0247 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:36.992877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba643e33-c650-4254-b594-1323063b0247 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:36.993194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba643e33-c650-4254-b594-1323063b0247 admin admin] PATH_INFO: `/v3/projects/02eb4d069635439aa3f5af0de672b889/users/c0258b53445e4f99a70a8218812105ee/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:36.995319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba643e33-c650-4254-b594-1323063b0247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02eb4d069635439aa3f5af0de672b889, user_id=c0258b53445e4f99a70a8218812105ee, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:36.997236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbdd30-c70c-484f-b6db-15938afe5948 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:36.998087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba643e33-c650-4254-b594-1323063b0247 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:37.002393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd0f84-37e0-4c17-963d-1ae77d575c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:37.004562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd0f84-37e0-4c17-963d-1ae77d575c3d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:37.004974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd0f84-37e0-4c17-963d-1ae77d575c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:37.005329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd0f84-37e0-4c17-963d-1ae77d575c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:37.005977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd0f84-37e0-4c17-963d-1ae77d575c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:37.007928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd0f84-37e0-4c17-963d-1ae77d575c3d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:37.014639 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3661/7230] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:36 2022] PUT /identity/v3/projects/02eb4d069635439aa3f5af0de672b889/users/c0258b53445e4f99a70a8218812105ee/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:37.024419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1285c-1303-44bd-bdef-56cad9ed205d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:37.024823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1285c-1303-44bd-bdef-56cad9ed205d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:37.025052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1285c-1303-44bd-bdef-56cad9ed205d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:37.085175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bca1285c-1303-44bd-bdef-56cad9ed205d None None] MFA Rules not processed for user `c0258b53445e4f99a70a8218812105ee`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:37.086569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bca1285c-1303-44bd-bdef-56cad9ed205d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:37.092250 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3570/7231] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:36 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:37.096098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd0f84-37e0-4c17-963d-1ae77d575c3d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:37.156393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:37.200482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd0f84-37e0-4c17-963d-1ae77d575c3d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:37.201219 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3662/7232] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:37.210245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11f06-013e-4662-895d-a4fd80b01a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:37.211068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11f06-013e-4662-895d-a4fd80b01a9f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:37.211516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11f06-013e-4662-895d-a4fd80b01a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:37.211995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11f06-013e-4662-895d-a4fd80b01a9f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:37.212848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e11f06-013e-4662-895d-a4fd80b01a9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:37.216498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e11f06-013e-4662-895d-a4fd80b01a9f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:37.217338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1285c-1303-44bd-bdef-56cad9ed205d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:37.224047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bca1285c-1303-44bd-bdef-56cad9ed205d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:37.254034 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3571/7233] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:37 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:37.258246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11f06-013e-4662-895d-a4fd80b01a9f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:37.285373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11f06-013e-4662-895d-a4fd80b01a9f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:37.291552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252bdaf-da91-401a-af11-217b5cd13310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:37.292204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252bdaf-da91-401a-af11-217b5cd13310 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:37.292588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252bdaf-da91-401a-af11-217b5cd13310 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:37.292847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252bdaf-da91-401a-af11-217b5cd13310 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:37.293542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252bdaf-da91-401a-af11-217b5cd13310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:37.295145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252bdaf-da91-401a-af11-217b5cd13310 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:37.315708 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3572/7234] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:37.316641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252bdaf-da91-401a-af11-217b5cd13310 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:37.335144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252bdaf-da91-401a-af11-217b5cd13310 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:37.339867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1285c-1303-44bd-bdef-56cad9ed205d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:37.341666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336ecff-7d1a-4e71-95e9-68b63da306d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:37.342771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336ecff-7d1a-4e71-95e9-68b63da306d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:37.343248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336ecff-7d1a-4e71-95e9-68b63da306d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:37.343672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336ecff-7d1a-4e71-95e9-68b63da306d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:37.344316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105153f-ab65-4b66-83ee-8ebb02fc9d8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:37.344646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336ecff-7d1a-4e71-95e9-68b63da306d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:37.344961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105153f-ab65-4b66-83ee-8ebb02fc9d8f service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:37.345255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105153f-ab65-4b66-83ee-8ebb02fc9d8f service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:37.345539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105153f-ab65-4b66-83ee-8ebb02fc9d8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:37.346959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336ecff-7d1a-4e71-95e9-68b63da306d7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:37.358503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105153f-ab65-4b66-83ee-8ebb02fc9d8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:37.360243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105153f-ab65-4b66-83ee-8ebb02fc9d8f service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:37.364347 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3573/7235] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:37.365157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336ecff-7d1a-4e71-95e9-68b63da306d7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:37.373007 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3663/7236] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:15:37.385258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336ecff-7d1a-4e71-95e9-68b63da306d7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:37.390550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d401ce7-145a-4690-9807-bdd1802ba464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:37.391219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d401ce7-145a-4690-9807-bdd1802ba464 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:37.391505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d401ce7-145a-4690-9807-bdd1802ba464 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:37.391779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d401ce7-145a-4690-9807-bdd1802ba464 admin admin] PATH_INFO: `/v3/projects/5b8d1a9501b84b76a9ef13d51243cef3/users/c91b7026d968442eb0db3fd4b5ddc542/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:37.412431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d401ce7-145a-4690-9807-bdd1802ba464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8d1a9501b84b76a9ef13d51243cef3, user_id=c91b7026d968442eb0db3fd4b5ddc542, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:37.415473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d401ce7-145a-4690-9807-bdd1802ba464 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:37.447558 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3574/7237] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:37 2022] PUT /identity/v3/projects/5b8d1a9501b84b76a9ef13d51243cef3/users/c91b7026d968442eb0db3fd4b5ddc542/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:37.452545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee3a0d-be94-4f2c-8c5f-0e71f8b07ebb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:37.453382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee3a0d-be94-4f2c-8c5f-0e71f8b07ebb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:37.454071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee3a0d-be94-4f2c-8c5f-0e71f8b07ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:37.494344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-aaee3a0d-be94-4f2c-8c5f-0e71f8b07ebb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:37.495937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-aaee3a0d-be94-4f2c-8c5f-0e71f8b07ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:37.573548 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3664/7238] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:37.581960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d401ce7-145a-4690-9807-bdd1802ba464 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:37.583739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8d401ce7-145a-4690-9807-bdd1802ba464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:37.590074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:37.668115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d401ce7-145a-4690-9807-bdd1802ba464 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:37.672415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e1412-be54-493d-8018-56a98d27e8b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:37.673639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e1412-be54-493d-8018-56a98d27e8b3 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:37.673929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e1412-be54-493d-8018-56a98d27e8b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:37.674206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e1412-be54-493d-8018-56a98d27e8b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:37.691964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e1412-be54-493d-8018-56a98d27e8b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:37.694377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e1412-be54-493d-8018-56a98d27e8b3 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:37.709338 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3575/7239] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:37.864660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee3a0d-be94-4f2c-8c5f-0e71f8b07ebb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:37.876678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee3a0d-be94-4f2c-8c5f-0e71f8b07ebb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:37.885685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:37.886475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:37.886829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:37.887228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] PATH_INFO: `/v3/users/626a5aa5b6344d36966d0e9f745d0671` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:37.888568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=626a5aa5b6344d36966d0e9f745d0671)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:37.891324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:37.958947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '626a5aa5b6344d36966d0e9f745d0671'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:37.991236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '626a5aa5b6344d36966d0e9f745d0671'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:38.006452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '626a5aa5b6344d36966d0e9f745d0671'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:38.007718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '626a5aa5b6344d36966d0e9f745d0671'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:38.008491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '626a5aa5b6344d36966d0e9f745d0671'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:38.019622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:38.034998 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3665/7240] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:37 2022] DELETE /identity/v3/users/626a5aa5b6344d36966d0e9f745d0671 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:38.039450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271a515-6ff6-49f0-9751-984448028792 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:38.039714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271a515-6ff6-49f0-9751-984448028792 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:38.039980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271a515-6ff6-49f0-9751-984448028792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:38.079894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f271a515-6ff6-49f0-9751-984448028792 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:38.081221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f271a515-6ff6-49f0-9751-984448028792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:38.210872 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3576/7241] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:38.241818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:38.244057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:38.367101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20f9b2-46af-41c6-970b-fc33c0ba5f63 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:38.378820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8419b59-3d33-4336-a050-568f8fe23a91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:38.381689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8419b59-3d33-4336-a050-568f8fe23a91 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:38.382075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8419b59-3d33-4336-a050-568f8fe23a91 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:38.382470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8419b59-3d33-4336-a050-568f8fe23a91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:38.402725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8419b59-3d33-4336-a050-568f8fe23a91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:38.406260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8419b59-3d33-4336-a050-568f8fe23a91 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:38.427516 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3666/7242] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:39.362110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271a515-6ff6-49f0-9751-984448028792 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:39.363619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f271a515-6ff6-49f0-9751-984448028792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:39.387246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271a515-6ff6-49f0-9751-984448028792 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:39.401869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:39.402530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:39.402824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:39.403151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] PATH_INFO: `/v3/projects/2584ba5951f948519ef50611c60e0f37` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:39.405404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2584ba5951f948519ef50611c60e0f37)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:39.409745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:39.540251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2584ba5951f948519ef50611c60e0f37', 'request_id': 'req-caaca85c-8b6c-4223-8bac-865f4ad22447'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:39.554357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2584ba5951f948519ef50611c60e0f37', 'request_id': 'req-caaca85c-8b6c-4223-8bac-865f4ad22447'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:39.565604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2584ba5951f948519ef50611c60e0f37', 'request_id': 'req-caaca85c-8b6c-4223-8bac-865f4ad22447'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:39.593573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] The token cache is being invalidate because project 2584ba5951f948519ef50611c60e0f37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:39.594264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:39.594553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:39.596218 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3577/7243] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:39 2022] DELETE /identity/v3/projects/2584ba5951f948519ef50611c60e0f37 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:39.600225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8419b59-3d33-4336-a050-568f8fe23a91 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:39.615460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8419b59-3d33-4336-a050-568f8fe23a91 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:39.673857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:39.675728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:39.725833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:39.743137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:39.744571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:39.745059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:39.745514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] PATH_INFO: `/v3/users/c0258b53445e4f99a70a8218812105ee` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:39.747593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0258b53445e4f99a70a8218812105ee)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:39.750403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:39.802080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaca85c-8b6c-4223-8bac-865f4ad22447 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:39.808404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c97cd-49b1-4e64-a16d-34a4b57a59e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:39.817848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c97cd-49b1-4e64-a16d-34a4b57a59e8 service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:39.818813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c97cd-49b1-4e64-a16d-34a4b57a59e8 service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:39.819634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c97cd-49b1-4e64-a16d-34a4b57a59e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:39.823187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-110c97cd-49b1-4e64-a16d-34a4b57a59e8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:39.861646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0258b53445e4f99a70a8218812105ee'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:39.893115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0258b53445e4f99a70a8218812105ee'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:39.913355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0258b53445e4f99a70a8218812105ee'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:39.914425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0258b53445e4f99a70a8218812105ee'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:39.915195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0258b53445e4f99a70a8218812105ee'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:39.942372 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3667/7244] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:39 2022] DELETE /identity/v3/users/c0258b53445e4f99a70a8218812105ee => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:39.989978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c97cd-49b1-4e64-a16d-34a4b57a59e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:39.991479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c97cd-49b1-4e64-a16d-34a4b57a59e8 service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:40.012050 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3578/7245] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:40.146463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:40.149296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:40.212296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:40.286667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c03c87-75ca-41b6-978e-defaa664e207 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:40.294724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fff88-a1b8-43f2-ab7d-b29366d1218d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:40.295691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fff88-a1b8-43f2-ab7d-b29366d1218d service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:40.298069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fff88-a1b8-43f2-ab7d-b29366d1218d service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:40.298514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fff88-a1b8-43f2-ab7d-b29366d1218d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:40.303783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-af6fff88-a1b8-43f2-ab7d-b29366d1218d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:40.322197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fff88-a1b8-43f2-ab7d-b29366d1218d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:40.324624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fff88-a1b8-43f2-ab7d-b29366d1218d service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:40.350766 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3668/7246] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:40.422909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c97cd-49b1-4e64-a16d-34a4b57a59e8 service glance] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:40.424445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-110c97cd-49b1-4e64-a16d-34a4b57a59e8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:40.456623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c97cd-49b1-4e64-a16d-34a4b57a59e8 service glance] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:40.493773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fff88-a1b8-43f2-ab7d-b29366d1218d service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:40.508214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fff88-a1b8-43f2-ab7d-b29366d1218d service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:40.519867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c639af-2d64-4138-bfda-a6e584fbf8a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:40.520629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c639af-2d64-4138-bfda-a6e584fbf8a3 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:40.521071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c639af-2d64-4138-bfda-a6e584fbf8a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:40.521495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c639af-2d64-4138-bfda-a6e584fbf8a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:40.525535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-03c639af-2d64-4138-bfda-a6e584fbf8a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:40.563166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:40.563832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:40.564121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:40.564420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d admin admin] PATH_INFO: `/v3/projects/02eb4d069635439aa3f5af0de672b889` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:40.566684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d admin admin] RBAC: Authorizing `identity:delete_project(project_id=02eb4d069635439aa3f5af0de672b889)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:40.573198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:40.670070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02eb4d069635439aa3f5af0de672b889', 'request_id': 'req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:40.692685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02eb4d069635439aa3f5af0de672b889', 'request_id': 'req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:40.703002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02eb4d069635439aa3f5af0de672b889', 'request_id': 'req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:40.705065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c639af-2d64-4138-bfda-a6e584fbf8a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:40.707580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c639af-2d64-4138-bfda-a6e584fbf8a3 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:40.724483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:40.729528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d admin admin] The token cache is being invalidate because project 02eb4d069635439aa3f5af0de672b889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:40.730109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:40.731425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7a18b878-30b1-47d3-9b2a-b19cdf4a335d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:40.733252 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3579/7247] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:40 2022] DELETE /identity/v3/projects/02eb4d069635439aa3f5af0de672b889 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:40.739266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f30f79-d5b0-4dff-9565-97c5f589223f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:40.739896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f30f79-d5b0-4dff-9565-97c5f589223f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:40.740284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f30f79-d5b0-4dff-9565-97c5f589223f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:40.751110 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3669/7248] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:40.778761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-52f30f79-d5b0-4dff-9565-97c5f589223f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:40.780299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-52f30f79-d5b0-4dff-9565-97c5f589223f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:40.879481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c639af-2d64-4138-bfda-a6e584fbf8a3 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:40.881916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-03c639af-2d64-4138-bfda-a6e584fbf8a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:40.898392 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3580/7249] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:40.907637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48d86b-864e-4387-88ae-94d59f2dcb34 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:40.911621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48d86b-864e-4387-88ae-94d59f2dcb34 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:40.913044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48d86b-864e-4387-88ae-94d59f2dcb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:40.948369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6e48d86b-864e-4387-88ae-94d59f2dcb34 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:40.949622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6e48d86b-864e-4387-88ae-94d59f2dcb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:40.969368 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3581/7250] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:40.976520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48d86b-864e-4387-88ae-94d59f2dcb34 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:40.992151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48d86b-864e-4387-88ae-94d59f2dcb34 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:40.998787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d2e0a-7306-4c43-8dfd-b42ba9df825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:40.999497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d2e0a-7306-4c43-8dfd-b42ba9df825e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:41.000334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d2e0a-7306-4c43-8dfd-b42ba9df825e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:41.000702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d2e0a-7306-4c43-8dfd-b42ba9df825e admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:41.001430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d2e0a-7306-4c43-8dfd-b42ba9df825e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:41.001742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d2e0a-7306-4c43-8dfd-b42ba9df825e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:41.003358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d2e0a-7306-4c43-8dfd-b42ba9df825e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:41.017825 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3582/7251] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:41.022274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d2e0a-7306-4c43-8dfd-b42ba9df825e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:41.035586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d2e0a-7306-4c43-8dfd-b42ba9df825e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:41.042061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5610f68-20f6-4893-b168-e245a860d5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:41.043097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5610f68-20f6-4893-b168-e245a860d5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:41.043876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5610f68-20f6-4893-b168-e245a860d5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:41.044356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5610f68-20f6-4893-b168-e245a860d5b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:41.045291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5610f68-20f6-4893-b168-e245a860d5b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:41.047793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5610f68-20f6-4893-b168-e245a860d5b6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:41.113884 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3583/7252] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:41 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:41.123651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5610f68-20f6-4893-b168-e245a860d5b6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:41.172120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c639af-2d64-4138-bfda-a6e584fbf8a3 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:41.252003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5610f68-20f6-4893-b168-e245a860d5b6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:41.264732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:41.265471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9532602-414e-4bb3-945b-cb4367cc1f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:41.266104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9532602-414e-4bb3-945b-cb4367cc1f73 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:41.266565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9532602-414e-4bb3-945b-cb4367cc1f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:41.266753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9532602-414e-4bb3-945b-cb4367cc1f73 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:41.267307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9532602-414e-4bb3-945b-cb4367cc1f73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:41.269334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9532602-414e-4bb3-945b-cb4367cc1f73 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:41.301711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24201df-e345-4202-9e69-466b408d9166 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:41.302867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24201df-e345-4202-9e69-466b408d9166 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:41.303309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24201df-e345-4202-9e69-466b408d9166 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:41.303759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24201df-e345-4202-9e69-466b408d9166 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:41.306064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a24201df-e345-4202-9e69-466b408d9166 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:41.314103 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3584/7253] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:41 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:41.315615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9532602-414e-4bb3-945b-cb4367cc1f73 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:41.334306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24201df-e345-4202-9e69-466b408d9166 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:41.336001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9532602-414e-4bb3-945b-cb4367cc1f73 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:41.336212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24201df-e345-4202-9e69-466b408d9166 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:41.352792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157361a-e4fc-4451-81b8-346545c1eb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:41.353158 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3670/7254] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:15:41.356355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157361a-e4fc-4451-81b8-346545c1eb11 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:41.356849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157361a-e4fc-4451-81b8-346545c1eb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:41.357516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157361a-e4fc-4451-81b8-346545c1eb11 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:41.358378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157361a-e4fc-4451-81b8-346545c1eb11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:41.360803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157361a-e4fc-4451-81b8-346545c1eb11 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:41.383024 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3585/7255] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:15:41.388111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24201df-e345-4202-9e69-466b408d9166 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:41.408307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24201df-e345-4202-9e69-466b408d9166 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:41.423957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791da01-46f8-48ca-b7ed-94923a777a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:41.424587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791da01-46f8-48ca-b7ed-94923a777a85 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:41.424991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791da01-46f8-48ca-b7ed-94923a777a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:41.425180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791da01-46f8-48ca-b7ed-94923a777a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:41.426425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791da01-46f8-48ca-b7ed-94923a777a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:41.431158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791da01-46f8-48ca-b7ed-94923a777a85 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:41.467539 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3671/7256] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:41.468808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157361a-e4fc-4451-81b8-346545c1eb11 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:41.480064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157361a-e4fc-4451-81b8-346545c1eb11 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:41.497701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a22b2-dc9b-4b95-8c21-5fed145e9026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:41.498532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a22b2-dc9b-4b95-8c21-5fed145e9026 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:41.498926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a22b2-dc9b-4b95-8c21-5fed145e9026 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:41.499313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a22b2-dc9b-4b95-8c21-5fed145e9026 admin admin] PATH_INFO: `/v3/projects/914adf5b338d42518fb7d4319c54041a/users/4ff686d6ba18411698b7bcb340e21adf/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:41.538821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a22b2-dc9b-4b95-8c21-5fed145e9026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=914adf5b338d42518fb7d4319c54041a, user_id=4ff686d6ba18411698b7bcb340e21adf, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:41.541694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a22b2-dc9b-4b95-8c21-5fed145e9026 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:41.567328 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3586/7257] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:41 2022] PUT /identity/v3/projects/914adf5b338d42518fb7d4319c54041a/users/4ff686d6ba18411698b7bcb340e21adf/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:41.578609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f2dac-301f-4de6-8a3f-10cb3d1e309d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:41.578908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f2dac-301f-4de6-8a3f-10cb3d1e309d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:41.579191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f2dac-301f-4de6-8a3f-10cb3d1e309d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:41.622507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-902f2dac-301f-4de6-8a3f-10cb3d1e309d None None] MFA Rules not processed for user `4ff686d6ba18411698b7bcb340e21adf`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:41.624104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-902f2dac-301f-4de6-8a3f-10cb3d1e309d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:41.755394 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3672/7258] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:41.767266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a22b2-dc9b-4b95-8c21-5fed145e9026 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:41.769322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e53a22b2-dc9b-4b95-8c21-5fed145e9026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:41.797504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:41.899373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a22b2-dc9b-4b95-8c21-5fed145e9026 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:41.911843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f0342-b131-4fda-a393-74948610f40c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:41.912865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f0342-b131-4fda-a393-74948610f40c service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:41.917539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f0342-b131-4fda-a393-74948610f40c service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:41.918109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f0342-b131-4fda-a393-74948610f40c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:41.942198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f0342-b131-4fda-a393-74948610f40c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:41.945053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f0342-b131-4fda-a393-74948610f40c service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:41.974435 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3587/7259] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:44.376216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f2dac-301f-4de6-8a3f-10cb3d1e309d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:44.644666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f2dac-301f-4de6-8a3f-10cb3d1e309d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:44.660409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:44.662547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:44.663235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:44.663708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] PATH_INFO: `/v3/users/4ff686d6ba18411698b7bcb340e21adf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:44.665848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ff686d6ba18411698b7bcb340e21adf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:44.668332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:44.730341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ff686d6ba18411698b7bcb340e21adf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:44.734364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:44.749176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ff686d6ba18411698b7bcb340e21adf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:44.759310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ff686d6ba18411698b7bcb340e21adf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:44.759530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ff686d6ba18411698b7bcb340e21adf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:44.760926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ff686d6ba18411698b7bcb340e21adf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:44.777536 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3673/7260] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:44 2022] DELETE /identity/v3/users/4ff686d6ba18411698b7bcb340e21adf => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:44.782395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89795edd-2e05-4108-a7e7-6e9b175c284e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:44.783546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89795edd-2e05-4108-a7e7-6e9b175c284e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:44.783830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89795edd-2e05-4108-a7e7-6e9b175c284e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:44.806225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-89795edd-2e05-4108-a7e7-6e9b175c284e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:44.808349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-89795edd-2e05-4108-a7e7-6e9b175c284e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:44.899053 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3588/7261] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:44.910571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:44.912413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:45.060463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446e9d2-1f01-41ed-8a0f-58ecb0cf8c2d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:45.065075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798179f6-2c1d-403f-9b07-e096eb1a61df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:45.065824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798179f6-2c1d-403f-9b07-e096eb1a61df service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:45.066207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798179f6-2c1d-403f-9b07-e096eb1a61df service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:45.066578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798179f6-2c1d-403f-9b07-e096eb1a61df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:45.078818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798179f6-2c1d-403f-9b07-e096eb1a61df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:45.080724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798179f6-2c1d-403f-9b07-e096eb1a61df service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:45.092508 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3674/7262] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:15:45.512361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89795edd-2e05-4108-a7e7-6e9b175c284e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:45.515326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-89795edd-2e05-4108-a7e7-6e9b175c284e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:45.531656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89795edd-2e05-4108-a7e7-6e9b175c284e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:45.537867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d0e68-cc5e-4616-99ce-3275374b24d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:45.539726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0e68-cc5e-4616-99ce-3275374b24d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:45.540057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0e68-cc5e-4616-99ce-3275374b24d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:45.540580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0e68-cc5e-4616-99ce-3275374b24d7 admin admin] PATH_INFO: `/v3/projects/914adf5b338d42518fb7d4319c54041a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:45.544292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d0e68-cc5e-4616-99ce-3275374b24d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=914adf5b338d42518fb7d4319c54041a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:45.546613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d0e68-cc5e-4616-99ce-3275374b24d7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:45.623433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-da8d0e68-cc5e-4616-99ce-3275374b24d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '914adf5b338d42518fb7d4319c54041a', 'request_id': 'req-da8d0e68-cc5e-4616-99ce-3275374b24d7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:45.638282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-da8d0e68-cc5e-4616-99ce-3275374b24d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '914adf5b338d42518fb7d4319c54041a', 'request_id': 'req-da8d0e68-cc5e-4616-99ce-3275374b24d7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:45.649952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-da8d0e68-cc5e-4616-99ce-3275374b24d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '914adf5b338d42518fb7d4319c54041a', 'request_id': 'req-da8d0e68-cc5e-4616-99ce-3275374b24d7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:45.676729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-da8d0e68-cc5e-4616-99ce-3275374b24d7 admin admin] The token cache is being invalidate because project 914adf5b338d42518fb7d4319c54041a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:45.677449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-da8d0e68-cc5e-4616-99ce-3275374b24d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:45.679349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-da8d0e68-cc5e-4616-99ce-3275374b24d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:45.681237 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3589/7263] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:45 2022] DELETE /identity/v3/projects/914adf5b338d42518fb7d4319c54041a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:45.696560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d661497-b7fa-4a97-b5a2-ee2d1e272f7f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:45.697144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d661497-b7fa-4a97-b5a2-ee2d1e272f7f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:45.697243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d661497-b7fa-4a97-b5a2-ee2d1e272f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:45.734907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9d661497-b7fa-4a97-b5a2-ee2d1e272f7f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:45.741051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9d661497-b7fa-4a97-b5a2-ee2d1e272f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:45.864011 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3675/7264] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:45.873190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51e887-ad4e-4f99-b86a-c166bb4c0e28 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:45.874496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51e887-ad4e-4f99-b86a-c166bb4c0e28 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:45.875202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51e887-ad4e-4f99-b86a-c166bb4c0e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:45.898597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9d51e887-ad4e-4f99-b86a-c166bb4c0e28 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:45.899887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9d51e887-ad4e-4f99-b86a-c166bb4c0e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:45.912117 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3590/7265] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:45.912925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d661497-b7fa-4a97-b5a2-ee2d1e272f7f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:45.925467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d661497-b7fa-4a97-b5a2-ee2d1e272f7f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:45.932622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd67a7d-aa69-4e38-8e6d-ca6b3da126e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:45.933899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd67a7d-aa69-4e38-8e6d-ca6b3da126e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:45.933899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd67a7d-aa69-4e38-8e6d-ca6b3da126e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:45.934228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd67a7d-aa69-4e38-8e6d-ca6b3da126e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:45.934879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd67a7d-aa69-4e38-8e6d-ca6b3da126e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:45.935571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd67a7d-aa69-4e38-8e6d-ca6b3da126e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:45.937651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd67a7d-aa69-4e38-8e6d-ca6b3da126e8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:45.955074 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3676/7266] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:15:45.956217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51e887-ad4e-4f99-b86a-c166bb4c0e28 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:45.967041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51e887-ad4e-4f99-b86a-c166bb4c0e28 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:45.972466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4f810-766c-4c8b-82ff-ad81a959464d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:45.973450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4f810-766c-4c8b-82ff-ad81a959464d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:45.973740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4f810-766c-4c8b-82ff-ad81a959464d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:45.974185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4f810-766c-4c8b-82ff-ad81a959464d admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:45.974842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4f810-766c-4c8b-82ff-ad81a959464d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:45.976752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4f810-766c-4c8b-82ff-ad81a959464d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:46.008942 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3591/7267] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:45 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:46.013457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd67a7d-aa69-4e38-8e6d-ca6b3da126e8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:46.025313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd67a7d-aa69-4e38-8e6d-ca6b3da126e8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:46.099797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:46.113573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76919321-f6c4-454b-9f5f-8f9d1ae7d93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:46.116351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76919321-f6c4-454b-9f5f-8f9d1ae7d93e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:46.116631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76919321-f6c4-454b-9f5f-8f9d1ae7d93e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:46.117052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76919321-f6c4-454b-9f5f-8f9d1ae7d93e admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:46.117814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76919321-f6c4-454b-9f5f-8f9d1ae7d93e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:46.120830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76919321-f6c4-454b-9f5f-8f9d1ae7d93e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:46.156885 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3677/7268] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:46 2022] POST /identity/v3/users => generated 285 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:15:46.158143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4f810-766c-4c8b-82ff-ad81a959464d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:46.183019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4f810-766c-4c8b-82ff-ad81a959464d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:46.188885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f0224-61cd-4d0e-a284-b9804a3ff86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:46.189549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f0224-61cd-4d0e-a284-b9804a3ff86a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:46.189839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f0224-61cd-4d0e-a284-b9804a3ff86a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:46.190164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f0224-61cd-4d0e-a284-b9804a3ff86a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:46.190779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f0224-61cd-4d0e-a284-b9804a3ff86a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:46.193006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f0224-61cd-4d0e-a284-b9804a3ff86a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:46.209291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:46.217372 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3592/7269] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:46.219745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76919321-f6c4-454b-9f5f-8f9d1ae7d93e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:46.246999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76919321-f6c4-454b-9f5f-8f9d1ae7d93e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:46.261640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e5f6c-92aa-474d-a6a6-bfd1eaa59ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:46.262625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e5f6c-92aa-474d-a6a6-bfd1eaa59ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:46.262625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e5f6c-92aa-474d-a6a6-bfd1eaa59ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:46.262822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e5f6c-92aa-474d-a6a6-bfd1eaa59ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:46.263634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e5f6c-92aa-474d-a6a6-bfd1eaa59ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:46.265290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e5f6c-92aa-474d-a6a6-bfd1eaa59ed6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:46.299416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f0224-61cd-4d0e-a284-b9804a3ff86a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:46.302136 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3678/7270] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:46.315813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f0224-61cd-4d0e-a284-b9804a3ff86a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:46.336944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663d785-5ede-4989-84bb-9fd89cee6701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:46.337644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663d785-5ede-4989-84bb-9fd89cee6701 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:46.338446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663d785-5ede-4989-84bb-9fd89cee6701 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:46.338738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663d785-5ede-4989-84bb-9fd89cee6701 admin admin] PATH_INFO: `/v3/projects/a9ff0b4070df441cbc16db1f727e08c2/users/3a99ff0c64f94f729675b3fbe19b9286/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:46.379965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994e133-999d-4489-ad5b-1eece742fe49 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:46.381463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994e133-999d-4489-ad5b-1eece742fe49 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:46.381902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994e133-999d-4489-ad5b-1eece742fe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:46.389615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663d785-5ede-4989-84bb-9fd89cee6701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9ff0b4070df441cbc16db1f727e08c2, user_id=3a99ff0c64f94f729675b3fbe19b9286, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:46.393849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663d785-5ede-4989-84bb-9fd89cee6701 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:46.428124 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3593/7271] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:46 2022] PUT /identity/v3/projects/a9ff0b4070df441cbc16db1f727e08c2/users/3a99ff0c64f94f729675b3fbe19b9286/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:46.435409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6994e133-999d-4489-ad5b-1eece742fe49 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:46.438025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6994e133-999d-4489-ad5b-1eece742fe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:46.438554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f86f00-ae58-49e7-98ef-582c1e83cb08 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:46.438886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f86f00-ae58-49e7-98ef-582c1e83cb08 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:46.439663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f86f00-ae58-49e7-98ef-582c1e83cb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:46.526877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-21f86f00-ae58-49e7-98ef-582c1e83cb08 None None] MFA Rules not processed for user `3a99ff0c64f94f729675b3fbe19b9286`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:46.531720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-21f86f00-ae58-49e7-98ef-582c1e83cb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:46.567501 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3679/7272] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:15:46.582076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994e133-999d-4489-ad5b-1eece742fe49 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:46.584885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6994e133-999d-4489-ad5b-1eece742fe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:46.706719 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3594/7273] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:46.720391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f86f00-ae58-49e7-98ef-582c1e83cb08 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:46.722633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-21f86f00-ae58-49e7-98ef-582c1e83cb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:46.734389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994e133-999d-4489-ad5b-1eece742fe49 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:46.745306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8908c-3c2a-4737-a01d-0b1f35dbe65a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:46.746312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8908c-3c2a-4737-a01d-0b1f35dbe65a service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:46.746612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8908c-3c2a-4737-a01d-0b1f35dbe65a service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:46.746894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8908c-3c2a-4737-a01d-0b1f35dbe65a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:46.768349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8908c-3c2a-4737-a01d-0b1f35dbe65a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:46.771777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8908c-3c2a-4737-a01d-0b1f35dbe65a service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:46.799308 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3680/7274] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:15:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:15:46.896596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f86f00-ae58-49e7-98ef-582c1e83cb08 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:46.910758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb486d-21c2-4b36-bafa-66f8321dfe2e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:46.911744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb486d-21c2-4b36-bafa-66f8321dfe2e service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:46.912305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb486d-21c2-4b36-bafa-66f8321dfe2e service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:46.913021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb486d-21c2-4b36-bafa-66f8321dfe2e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:46.948535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb486d-21c2-4b36-bafa-66f8321dfe2e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:46.951077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb486d-21c2-4b36-bafa-66f8321dfe2e service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:47.005880 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3595/7275] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:15:47.138437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed2c50-e42b-455d-b3f0-7c076d4e7c9e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:47.138724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed2c50-e42b-455d-b3f0-7c076d4e7c9e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:47.139116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed2c50-e42b-455d-b3f0-7c076d4e7c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:47.203422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-82ed2c50-e42b-455d-b3f0-7c076d4e7c9e None None] MFA Rules not processed for user `c91b7026d968442eb0db3fd4b5ddc542`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:47.204817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-82ed2c50-e42b-455d-b3f0-7c076d4e7c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:47.326244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:47.343959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb486d-21c2-4b36-bafa-66f8321dfe2e service glance] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:47.346837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-76fb486d-21c2-4b36-bafa-66f8321dfe2e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:47.347086 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3681/7276] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:47.530751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb486d-21c2-4b36-bafa-66f8321dfe2e service glance] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:47.544150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6281407-08a9-424c-8d57-26392c155d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:47.544949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6281407-08a9-424c-8d57-26392c155d21 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:47.545354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6281407-08a9-424c-8d57-26392c155d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:47.545745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6281407-08a9-424c-8d57-26392c155d21 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:47.546981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6281407-08a9-424c-8d57-26392c155d21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:47.548563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6281407-08a9-424c-8d57-26392c155d21 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:47.586805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:47.612864 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3596/7277] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:47 2022] POST /identity/v3/projects => generated 379 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:15:47.616996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed2c50-e42b-455d-b3f0-7c076d4e7c9e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:47.644487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed2c50-e42b-455d-b3f0-7c076d4e7c9e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:47.758387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd93e6-217c-4f8d-acb1-9cf3f5fc2172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:47.759050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd93e6-217c-4f8d-acb1-9cf3f5fc2172 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:47.759300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd93e6-217c-4f8d-acb1-9cf3f5fc2172 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:47.759632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd93e6-217c-4f8d-acb1-9cf3f5fc2172 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:47.760400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd93e6-217c-4f8d-acb1-9cf3f5fc2172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:47.763268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd93e6-217c-4f8d-acb1-9cf3f5fc2172 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:47.799578 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3682/7278] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:47 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:47.800248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6281407-08a9-424c-8d57-26392c155d21 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:47.814480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6281407-08a9-424c-8d57-26392c155d21 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:47.822864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df21896-0af3-4292-9987-532932d3c1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:47.824744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df21896-0af3-4292-9987-532932d3c1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:47.826640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df21896-0af3-4292-9987-532932d3c1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:47.827023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df21896-0af3-4292-9987-532932d3c1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:47.827831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df21896-0af3-4292-9987-532932d3c1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:47.832769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df21896-0af3-4292-9987-532932d3c1f3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:47.859634 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3597/7279] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:47.864112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd93e6-217c-4f8d-acb1-9cf3f5fc2172 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:47.886582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd93e6-217c-4f8d-acb1-9cf3f5fc2172 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:47.902615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7b717-052e-41ff-819b-83c35062b18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:47.903478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7b717-052e-41ff-819b-83c35062b18e admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:47.904103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7b717-052e-41ff-819b-83c35062b18e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:47.904451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7b717-052e-41ff-819b-83c35062b18e admin admin] PATH_INFO: `/v3/projects/c2c59c2daa264ef4b671363b5a820051/users/ec39f02676dd4be9b12d17b463fe5971/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:47.956715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7b717-052e-41ff-819b-83c35062b18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2c59c2daa264ef4b671363b5a820051, user_id=ec39f02676dd4be9b12d17b463fe5971, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:47.959703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7b717-052e-41ff-819b-83c35062b18e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:47.994317 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3683/7280] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:47 2022] PUT /identity/v3/projects/c2c59c2daa264ef4b671363b5a820051/users/ec39f02676dd4be9b12d17b463fe5971/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:47.998696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df21896-0af3-4292-9987-532932d3c1f3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:48.011722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df21896-0af3-4292-9987-532932d3c1f3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:48.143376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e049e6-e739-4491-9959-2be6e9d33818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:48.145111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e049e6-e739-4491-9959-2be6e9d33818 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:48.145573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e049e6-e739-4491-9959-2be6e9d33818 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:48.145962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e049e6-e739-4491-9959-2be6e9d33818 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:48.146714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e049e6-e739-4491-9959-2be6e9d33818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:48.148797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e049e6-e739-4491-9959-2be6e9d33818 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:48.165809 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3598/7281] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:15:48.170199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7b717-052e-41ff-819b-83c35062b18e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:48.186230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7b717-052e-41ff-819b-83c35062b18e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:48.194489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03020d-0fef-42bd-ab41-8af0eafc1cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:48.195736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03020d-0fef-42bd-ab41-8af0eafc1cad admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:48.196206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03020d-0fef-42bd-ab41-8af0eafc1cad admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:48.196590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03020d-0fef-42bd-ab41-8af0eafc1cad admin admin] PATH_INFO: `/v3/projects/c2c59c2daa264ef4b671363b5a820051/users/ec39f02676dd4be9b12d17b463fe5971/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:48.199941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03020d-0fef-42bd-ab41-8af0eafc1cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2c59c2daa264ef4b671363b5a820051, user_id=ec39f02676dd4be9b12d17b463fe5971, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:48.204063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03020d-0fef-42bd-ab41-8af0eafc1cad admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:48.257100 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3684/7282] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:48 2022] PUT /identity/v3/projects/c2c59c2daa264ef4b671363b5a820051/users/ec39f02676dd4be9b12d17b463fe5971/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:51.503992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e049e6-e739-4491-9959-2be6e9d33818 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:51.519886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e049e6-e739-4491-9959-2be6e9d33818 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:51.638509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:51.639571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:51.639975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:51.640359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] PATH_INFO: `/v3/users/3a99ff0c64f94f729675b3fbe19b9286` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:51.643334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a99ff0c64f94f729675b3fbe19b9286)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:51.645943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:51.737817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a99ff0c64f94f729675b3fbe19b9286'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:51.762533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a99ff0c64f94f729675b3fbe19b9286'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:51.763438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a99ff0c64f94f729675b3fbe19b9286'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:51.772950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a99ff0c64f94f729675b3fbe19b9286'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:51.789746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a99ff0c64f94f729675b3fbe19b9286'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:51.793633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:51.811669 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3599/7283] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:15:51 2022] DELETE /identity/v3/users/3a99ff0c64f94f729675b3fbe19b9286 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:51.813822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3312cc3-67f8-49c3-b4a3-d1e09d27eacb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:51.814357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3312cc3-67f8-49c3-b4a3-d1e09d27eacb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:51.814357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3312cc3-67f8-49c3-b4a3-d1e09d27eacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:51.881195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a3312cc3-67f8-49c3-b4a3-d1e09d27eacb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:51.890624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a3312cc3-67f8-49c3-b4a3-d1e09d27eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:52.043672 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3685/7284] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:52.064873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:52.067040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:52.376445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62bd45e-faef-4fe9-b405-53fc4e0a2897 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:52.387587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726937e-3ad1-4b65-baf8-71385854bc4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:52.389235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726937e-3ad1-4b65-baf8-71385854bc4e service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:52.389875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726937e-3ad1-4b65-baf8-71385854bc4e service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:52.390168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726937e-3ad1-4b65-baf8-71385854bc4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:52.405811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726937e-3ad1-4b65-baf8-71385854bc4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:52.407400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726937e-3ad1-4b65-baf8-71385854bc4e service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:52.440931 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3600/7285] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:15:55.386883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3312cc3-67f8-49c3-b4a3-d1e09d27eacb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:55.388486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a3312cc3-67f8-49c3-b4a3-d1e09d27eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:55.415229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3312cc3-67f8-49c3-b4a3-d1e09d27eacb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:55.434114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:55.434708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:55.434997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:55.435279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05 admin admin] PATH_INFO: `/v3/projects/a9ff0b4070df441cbc16db1f727e08c2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:55.439651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9ff0b4070df441cbc16db1f727e08c2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:55.444353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:55.567397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:55.569818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9ff0b4070df441cbc16db1f727e08c2', 'request_id': 'req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:55.600322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9ff0b4070df441cbc16db1f727e08c2', 'request_id': 'req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:55.627473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9ff0b4070df441cbc16db1f727e08c2', 'request_id': 'req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:55.672726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05 admin admin] The token cache is being invalidate because project a9ff0b4070df441cbc16db1f727e08c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:15:55.679122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:55.681283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-19283b8b-3e04-4702-a9dc-11cbdcf0ad05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:15:55.688597 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3686/7286] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:15:55 2022] DELETE /identity/v3/projects/a9ff0b4070df441cbc16db1f727e08c2 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:15:55.697840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e098f10-bac6-479d-8db0-02c7711bc3cc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:55.698518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e098f10-bac6-479d-8db0-02c7711bc3cc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:55.698518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e098f10-bac6-479d-8db0-02c7711bc3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:55.766282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2e098f10-bac6-479d-8db0-02c7711bc3cc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:55.768295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2e098f10-bac6-479d-8db0-02c7711bc3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:55.860330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2662-aa91-4ca8-9bfc-1aa828466c45 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:55.860754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2662-aa91-4ca8-9bfc-1aa828466c45 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:55.861074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2662-aa91-4ca8-9bfc-1aa828466c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:55.884275 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3601/7287] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:55.914220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5f7c2662-aa91-4ca8-9bfc-1aa828466c45 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:55.915498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5f7c2662-aa91-4ca8-9bfc-1aa828466c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:55.936422 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3687/7288] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:55.940755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e098f10-bac6-479d-8db0-02c7711bc3cc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:55.960552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e098f10-bac6-479d-8db0-02c7711bc3cc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:55.972373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891d0cf-4e21-4040-a32d-43651a89cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:55.973354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891d0cf-4e21-4040-a32d-43651a89cfce admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:55.981402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891d0cf-4e21-4040-a32d-43651a89cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:55.982122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891d0cf-4e21-4040-a32d-43651a89cfce admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:55.983901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891d0cf-4e21-4040-a32d-43651a89cfce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:15:55.984040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891d0cf-4e21-4040-a32d-43651a89cfce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:55.988195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891d0cf-4e21-4040-a32d-43651a89cfce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:56.036043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c2662-aa91-4ca8-9bfc-1aa828466c45 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:56.047019 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3602/7289] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:15:56.055427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c2662-aa91-4ca8-9bfc-1aa828466c45 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:56.062467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9716e4-e9a8-4c74-80dc-dc1cf484d7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:56.063118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9716e4-e9a8-4c74-80dc-dc1cf484d7e5 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:56.063421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9716e4-e9a8-4c74-80dc-dc1cf484d7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:56.063697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9716e4-e9a8-4c74-80dc-dc1cf484d7e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:56.064349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9716e4-e9a8-4c74-80dc-dc1cf484d7e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:56.074262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9716e4-e9a8-4c74-80dc-dc1cf484d7e5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:56.173768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891d0cf-4e21-4040-a32d-43651a89cfce admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:56.189968 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3688/7290] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:15:56 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:56.192949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891d0cf-4e21-4040-a32d-43651a89cfce admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:56.283155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09812c-0a7b-4756-855c-877dd8fdcb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:56.284088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09812c-0a7b-4756-855c-877dd8fdcb32 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:56.284528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09812c-0a7b-4756-855c-877dd8fdcb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:56.284977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09812c-0a7b-4756-855c-877dd8fdcb32 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:56.285982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09812c-0a7b-4756-855c-877dd8fdcb32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:56.290774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09812c-0a7b-4756-855c-877dd8fdcb32 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:56.346866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9716e4-e9a8-4c74-80dc-dc1cf484d7e5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:56.353875 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3603/7291] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:15:56 2022] POST /identity/v3/users => generated 283 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:15:56.360777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9716e4-e9a8-4c74-80dc-dc1cf484d7e5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:56.366927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761765e9-a0fa-4960-a080-d9056f005ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:56.367673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761765e9-a0fa-4960-a080-d9056f005ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:56.368073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761765e9-a0fa-4960-a080-d9056f005ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:56.368563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761765e9-a0fa-4960-a080-d9056f005ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:56.370054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761765e9-a0fa-4960-a080-d9056f005ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:56.373614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761765e9-a0fa-4960-a080-d9056f005ff2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:56.423566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09812c-0a7b-4756-855c-877dd8fdcb32 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:56.447175 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3689/7292] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:15:56.454145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09812c-0a7b-4756-855c-877dd8fdcb32 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:56.470227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e4b37-b705-45cd-81cd-69e6e69e753c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:56.470882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e4b37-b705-45cd-81cd-69e6e69e753c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:56.471174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e4b37-b705-45cd-81cd-69e6e69e753c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:56.471546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e4b37-b705-45cd-81cd-69e6e69e753c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:56.472285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e4b37-b705-45cd-81cd-69e6e69e753c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:56.473931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e4b37-b705-45cd-81cd-69e6e69e753c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:56.504533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761765e9-a0fa-4960-a080-d9056f005ff2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:56.521381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761765e9-a0fa-4960-a080-d9056f005ff2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:56.522921 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3604/7293] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:15:56.533729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4cada-9a5b-474b-83ac-11a94953e09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:56.534433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4cada-9a5b-474b-83ac-11a94953e09b admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:56.534733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4cada-9a5b-474b-83ac-11a94953e09b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:56.535019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4cada-9a5b-474b-83ac-11a94953e09b admin admin] PATH_INFO: `/v3/projects/4da76c2ca781403c900bdf8bd2679ea6/users/a0ee604fc6674a6385595715254c5f89/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:56.599524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4cada-9a5b-474b-83ac-11a94953e09b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4da76c2ca781403c900bdf8bd2679ea6, user_id=a0ee604fc6674a6385595715254c5f89, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:56.602310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4cada-9a5b-474b-83ac-11a94953e09b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:56.638951 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3690/7294] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:15:56 2022] PUT /identity/v3/projects/4da76c2ca781403c900bdf8bd2679ea6/users/a0ee604fc6674a6385595715254c5f89/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:15:56.655698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b4bf9-9d7b-49d3-8641-7f839b8d68ef None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:56.656375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b4bf9-9d7b-49d3-8641-7f839b8d68ef None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:56.656942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b4bf9-9d7b-49d3-8641-7f839b8d68ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:56.677173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:15:56.704764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-032b4bf9-9d7b-49d3-8641-7f839b8d68ef None None] MFA Rules not processed for user `a0ee604fc6674a6385595715254c5f89`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:15:56.706908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-032b4bf9-9d7b-49d3-8641-7f839b8d68ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:56.846580 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3605/7295] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:15:56.854839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4cada-9a5b-474b-83ac-11a94953e09b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:15:56.856929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-06e4cada-9a5b-474b-83ac-11a94953e09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:15:56.951145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4cada-9a5b-474b-83ac-11a94953e09b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:15:56.955407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17782de5-4016-40e7-8efb-5c6baef9bdda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:15:56.956386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17782de5-4016-40e7-8efb-5c6baef9bdda service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:15:56.956939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17782de5-4016-40e7-8efb-5c6baef9bdda service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:15:56.957469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17782de5-4016-40e7-8efb-5c6baef9bdda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:15:56.968512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17782de5-4016-40e7-8efb-5c6baef9bdda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:15:56.970578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17782de5-4016-40e7-8efb-5c6baef9bdda service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:15:56.982792 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3691/7296] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:15:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:00.441570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003760f-a184-4b2a-baef-1c4395c15e71 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:00.442189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003760f-a184-4b2a-baef-1c4395c15e71 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:00.442297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003760f-a184-4b2a-baef-1c4395c15e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:00.486985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8003760f-a184-4b2a-baef-1c4395c15e71 None None] MFA Rules not processed for user `ec39f02676dd4be9b12d17b463fe5971`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:00.488673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8003760f-a184-4b2a-baef-1c4395c15e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:00.571419 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3606/7297] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:00.581448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17782de5-4016-40e7-8efb-5c6baef9bdda service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:00.583046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-17782de5-4016-40e7-8efb-5c6baef9bdda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:00.646844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17782de5-4016-40e7-8efb-5c6baef9bdda service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:00.650791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37126089-5dd3-4318-bae1-8cb90a0ad2d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:00.652169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37126089-5dd3-4318-bae1-8cb90a0ad2d0 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:00.652742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37126089-5dd3-4318-bae1-8cb90a0ad2d0 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:00.653719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37126089-5dd3-4318-bae1-8cb90a0ad2d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:00.656776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-37126089-5dd3-4318-bae1-8cb90a0ad2d0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:00.699436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:00.739612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37126089-5dd3-4318-bae1-8cb90a0ad2d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:00.741940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37126089-5dd3-4318-bae1-8cb90a0ad2d0 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:00.757993 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3692/7298] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:03.589090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003760f-a184-4b2a-baef-1c4395c15e71 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:03.677085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003760f-a184-4b2a-baef-1c4395c15e71 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:03.689485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:03.690437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:03.690883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:03.691168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] PATH_INFO: `/v3/users/a0ee604fc6674a6385595715254c5f89` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:03.693771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0ee604fc6674a6385595715254c5f89)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:03.696674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:03.752746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:03.810719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0ee604fc6674a6385595715254c5f89'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:03.840331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0ee604fc6674a6385595715254c5f89'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:03.841374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0ee604fc6674a6385595715254c5f89'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:03.863509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0ee604fc6674a6385595715254c5f89'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:03.875898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0ee604fc6674a6385595715254c5f89'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:03.887029 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3607/7299] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:03 2022] DELETE /identity/v3/users/a0ee604fc6674a6385595715254c5f89 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:03.891679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08526db6-d19a-469c-967e-312499c2726e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:03.892174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08526db6-d19a-469c-967e-312499c2726e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:03.892286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08526db6-d19a-469c-967e-312499c2726e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:03.939050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-08526db6-d19a-469c-967e-312499c2726e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:03.940676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-08526db6-d19a-469c-967e-312499c2726e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:04.069757 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3693/7300] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:04.085013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:04.086561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:04.225092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d09624-e361-467b-ba57-50012a161208 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:04.230012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483bb69c-6856-4e60-8f39-af2c7c073f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:04.230814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483bb69c-6856-4e60-8f39-af2c7c073f30 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:04.231105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483bb69c-6856-4e60-8f39-af2c7c073f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:04.231382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483bb69c-6856-4e60-8f39-af2c7c073f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:04.248479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483bb69c-6856-4e60-8f39-af2c7c073f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:04.249880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483bb69c-6856-4e60-8f39-af2c7c073f30 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:04.267736 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3608/7301] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:04.722818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08526db6-d19a-469c-967e-312499c2726e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:04.725639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-08526db6-d19a-469c-967e-312499c2726e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:04.750398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08526db6-d19a-469c-967e-312499c2726e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:04.761829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:04.762502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:04.762783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:04.766476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2 admin admin] PATH_INFO: `/v3/projects/4da76c2ca781403c900bdf8bd2679ea6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:04.770863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4da76c2ca781403c900bdf8bd2679ea6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:04.791983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:04.892129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4da76c2ca781403c900bdf8bd2679ea6', 'request_id': 'req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:04.904100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4da76c2ca781403c900bdf8bd2679ea6', 'request_id': 'req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:04.913254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4da76c2ca781403c900bdf8bd2679ea6', 'request_id': 'req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:04.927285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2 admin admin] The token cache is being invalidate because project 4da76c2ca781403c900bdf8bd2679ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:04.927659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:04.928623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0b32bd96-814d-46ca-ba6e-81a516cf9ce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:04.935378 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3694/7302] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:04 2022] DELETE /identity/v3/projects/4da76c2ca781403c900bdf8bd2679ea6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:04.936538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78d057-4dc8-4207-83f1-99f322d72a8d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:04.937104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78d057-4dc8-4207-83f1-99f322d72a8d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:04.937200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78d057-4dc8-4207-83f1-99f322d72a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:04.970867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9c78d057-4dc8-4207-83f1-99f322d72a8d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:04.972808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9c78d057-4dc8-4207-83f1-99f322d72a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:05.086492 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3609/7303] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:05.095255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cc525-0232-4fa4-83bb-9a0ca186b79b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:05.095606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cc525-0232-4fa4-83bb-9a0ca186b79b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:05.095901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cc525-0232-4fa4-83bb-9a0ca186b79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:05.126008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-803cc525-0232-4fa4-83bb-9a0ca186b79b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:05.127675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-803cc525-0232-4fa4-83bb-9a0ca186b79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:05.137575 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3695/7304] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:05.141495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78d057-4dc8-4207-83f1-99f322d72a8d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:05.157470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78d057-4dc8-4207-83f1-99f322d72a8d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:05.172632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaaa70f-0d9f-4bbc-adba-edb4af3a5e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:05.174225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaaa70f-0d9f-4bbc-adba-edb4af3a5e56 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:05.174969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaaa70f-0d9f-4bbc-adba-edb4af3a5e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:05.175542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaaa70f-0d9f-4bbc-adba-edb4af3a5e56 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:05.176367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaaa70f-0d9f-4bbc-adba-edb4af3a5e56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:16:05.176829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaaa70f-0d9f-4bbc-adba-edb4af3a5e56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:05.179186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaaa70f-0d9f-4bbc-adba-edb4af3a5e56 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:05.196314 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3610/7305] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:05.200173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cc525-0232-4fa4-83bb-9a0ca186b79b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:05.211124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cc525-0232-4fa4-83bb-9a0ca186b79b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:05.217941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2dac89-e176-4c6f-b954-8024a8b30783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:05.218556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2dac89-e176-4c6f-b954-8024a8b30783 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:05.219169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2dac89-e176-4c6f-b954-8024a8b30783 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:05.219379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2dac89-e176-4c6f-b954-8024a8b30783 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:05.220032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2dac89-e176-4c6f-b954-8024a8b30783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:05.222200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2dac89-e176-4c6f-b954-8024a8b30783 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:05.256938 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3696/7306] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:05 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:05.260647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaaa70f-0d9f-4bbc-adba-edb4af3a5e56 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:05.278858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaaa70f-0d9f-4bbc-adba-edb4af3a5e56 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:05.333034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:05.393218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8fd5-e495-45c3-b8f2-44c59a1b0923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:05.393218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8fd5-e495-45c3-b8f2-44c59a1b0923 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:05.393218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8fd5-e495-45c3-b8f2-44c59a1b0923 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:05.393218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8fd5-e495-45c3-b8f2-44c59a1b0923 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:05.405565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f8fd5-e495-45c3-b8f2-44c59a1b0923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:05.407801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f8fd5-e495-45c3-b8f2-44c59a1b0923 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:05.484599 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3611/7307] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:05 2022] POST /identity/v3/users => generated 291 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:05.485475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2dac89-e176-4c6f-b954-8024a8b30783 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:05.497461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2dac89-e176-4c6f-b954-8024a8b30783 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:05.503800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f86380-fe84-471f-8107-5a2122db1671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:05.504817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f86380-fe84-471f-8107-5a2122db1671 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:05.505292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f86380-fe84-471f-8107-5a2122db1671 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:05.506935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f86380-fe84-471f-8107-5a2122db1671 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:05.507688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f86380-fe84-471f-8107-5a2122db1671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:05.509364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f86380-fe84-471f-8107-5a2122db1671 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:05.534411 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3697/7308] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:05.542333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8fd5-e495-45c3-b8f2-44c59a1b0923 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:05.571458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8fd5-e495-45c3-b8f2-44c59a1b0923 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:05.578554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af70d63-8218-468c-a6ff-6c7123bca331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:05.579335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af70d63-8218-468c-a6ff-6c7123bca331 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:05.579691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af70d63-8218-468c-a6ff-6c7123bca331 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:05.580031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af70d63-8218-468c-a6ff-6c7123bca331 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:05.580830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af70d63-8218-468c-a6ff-6c7123bca331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:05.582668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af70d63-8218-468c-a6ff-6c7123bca331 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:05.598467 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3612/7309] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:05.602680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f86380-fe84-471f-8107-5a2122db1671 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:05.613670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f86380-fe84-471f-8107-5a2122db1671 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:05.631700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683f4c7-a1b4-41ed-9a01-d45f93708780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:05.632291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683f4c7-a1b4-41ed-9a01-d45f93708780 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:05.632575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683f4c7-a1b4-41ed-9a01-d45f93708780 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:05.632879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683f4c7-a1b4-41ed-9a01-d45f93708780 admin admin] PATH_INFO: `/v3/projects/fb6931dda93843e69b69129cdc30d0c8/users/07b3ac8e2f3f48448a2470ab148f4fd2/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:05.650700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:05.659111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683f4c7-a1b4-41ed-9a01-d45f93708780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6931dda93843e69b69129cdc30d0c8, user_id=07b3ac8e2f3f48448a2470ab148f4fd2, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:05.664417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683f4c7-a1b4-41ed-9a01-d45f93708780 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:05.685516 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3698/7310] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:05 2022] PUT /identity/v3/projects/fb6931dda93843e69b69129cdc30d0c8/users/07b3ac8e2f3f48448a2470ab148f4fd2/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:05.690626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a2b04-c95c-4d39-b113-f96467fbbf96 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:05.691217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a2b04-c95c-4d39-b113-f96467fbbf96 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:05.691579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a2b04-c95c-4d39-b113-f96467fbbf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:05.714376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-023a2b04-c95c-4d39-b113-f96467fbbf96 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:05.715725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-023a2b04-c95c-4d39-b113-f96467fbbf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:05.816309 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3613/7311] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:05.826859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683f4c7-a1b4-41ed-9a01-d45f93708780 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:05.828637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a683f4c7-a1b4-41ed-9a01-d45f93708780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:05.918042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683f4c7-a1b4-41ed-9a01-d45f93708780 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:05.922731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b5d6c-36ff-45c0-88e7-616adec82b20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:05.923538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b5d6c-36ff-45c0-88e7-616adec82b20 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:05.924111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b5d6c-36ff-45c0-88e7-616adec82b20 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:05.924611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b5d6c-36ff-45c0-88e7-616adec82b20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:05.937170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b5d6c-36ff-45c0-88e7-616adec82b20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:05.939579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b5d6c-36ff-45c0-88e7-616adec82b20 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:05.952827 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3699/7312] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:12.674053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a71e62-0c07-4ea3-b45d-9bce2cefbba0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:12.674686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a71e62-0c07-4ea3-b45d-9bce2cefbba0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:12.674686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a71e62-0c07-4ea3-b45d-9bce2cefbba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:12.713290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c9a71e62-0c07-4ea3-b45d-9bce2cefbba0 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:12.714541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c9a71e62-0c07-4ea3-b45d-9bce2cefbba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:12.817199 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3614/7313] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:16:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:12.831324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b5d6c-36ff-45c0-88e7-616adec82b20 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:12.833348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7e9b5d6c-36ff-45c0-88e7-616adec82b20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:12.972418 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b5d6c-36ff-45c0-88e7-616adec82b20 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:12.982360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c881731-decf-477b-bf2a-cf2e300c8ba8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:12.982995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c881731-decf-477b-bf2a-cf2e300c8ba8 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:12.983286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c881731-decf-477b-bf2a-cf2e300c8ba8 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:12.983570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c881731-decf-477b-bf2a-cf2e300c8ba8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:13.009901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c881731-decf-477b-bf2a-cf2e300c8ba8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:13.011999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c881731-decf-477b-bf2a-cf2e300c8ba8 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:13.037383 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3700/7314] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:16:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 21:16:14.242193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a71e62-0c07-4ea3-b45d-9bce2cefbba0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:14.244231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c9a71e62-0c07-4ea3-b45d-9bce2cefbba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:14.337573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:14.365787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a71e62-0c07-4ea3-b45d-9bce2cefbba0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:14.375057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c9daa-3c3e-4f8f-b878-4972bba45fd4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:14.376508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c9daa-3c3e-4f8f-b878-4972bba45fd4 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:14.377004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c9daa-3c3e-4f8f-b878-4972bba45fd4 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:14.377434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c9daa-3c3e-4f8f-b878-4972bba45fd4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:14.379999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8d9c9daa-3c3e-4f8f-b878-4972bba45fd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:14.412838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c9daa-3c3e-4f8f-b878-4972bba45fd4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:14.414498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c9daa-3c3e-4f8f-b878-4972bba45fd4 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:14.436422 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3615/7315] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:15.810386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbee950-6513-4288-bc1e-87f8d2d47035 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:15.812252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbee950-6513-4288-bc1e-87f8d2d47035 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:15.812746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbee950-6513-4288-bc1e-87f8d2d47035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:15.893942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9fbee950-6513-4288-bc1e-87f8d2d47035 None None] MFA Rules not processed for user `07b3ac8e2f3f48448a2470ab148f4fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:15.901820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9fbee950-6513-4288-bc1e-87f8d2d47035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:16.116619 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3701/7316] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:16.121710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c9daa-3c3e-4f8f-b878-4972bba45fd4 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:16.138193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c9daa-3c3e-4f8f-b878-4972bba45fd4 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:16.152619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff484479-3ac6-4ae0-b3b8-0f583522f578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:16.153509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff484479-3ac6-4ae0-b3b8-0f583522f578 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:16.153779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff484479-3ac6-4ae0-b3b8-0f583522f578 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:16.154058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff484479-3ac6-4ae0-b3b8-0f583522f578 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:16.154760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff484479-3ac6-4ae0-b3b8-0f583522f578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:16.158706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff484479-3ac6-4ae0-b3b8-0f583522f578 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:16.217909 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3616/7317] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:16 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:16.221964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbee950-6513-4288-bc1e-87f8d2d47035 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:16.238865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbee950-6513-4288-bc1e-87f8d2d47035 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:16.254362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:16.334169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4209ddb-bbc6-4b14-848a-5011bf0cdbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:16.335863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4209ddb-bbc6-4b14-848a-5011bf0cdbe3 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:16.336273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4209ddb-bbc6-4b14-848a-5011bf0cdbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:16.336662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4209ddb-bbc6-4b14-848a-5011bf0cdbe3 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:16.337535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4209ddb-bbc6-4b14-848a-5011bf0cdbe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:16.350151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4209ddb-bbc6-4b14-848a-5011bf0cdbe3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:16.378587 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3702/7318] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:16 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:16.382163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff484479-3ac6-4ae0-b3b8-0f583522f578 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:16.399300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff484479-3ac6-4ae0-b3b8-0f583522f578 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:16.409834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8300c6-9d8b-40c8-9ef4-ce2e7003ec94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:16.410819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8300c6-9d8b-40c8-9ef4-ce2e7003ec94 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:16.411497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8300c6-9d8b-40c8-9ef4-ce2e7003ec94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:16.411988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8300c6-9d8b-40c8-9ef4-ce2e7003ec94 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:16.413001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8300c6-9d8b-40c8-9ef4-ce2e7003ec94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:16.415041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8300c6-9d8b-40c8-9ef4-ce2e7003ec94 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:16.451051 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3617/7319] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:16.455410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4209ddb-bbc6-4b14-848a-5011bf0cdbe3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:16.469906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4209ddb-bbc6-4b14-848a-5011bf0cdbe3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:16.484625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc461e7-fad2-44bb-83d6-44788fa63a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:16.485503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc461e7-fad2-44bb-83d6-44788fa63a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:16.485772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc461e7-fad2-44bb-83d6-44788fa63a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:16.486042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc461e7-fad2-44bb-83d6-44788fa63a3a admin admin] PATH_INFO: `/v3/projects/3829db05ccd3411e84c951d8d79a1a12/users/6c812c1d223c4c9f9c753169e11c4c58/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:16.523610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc461e7-fad2-44bb-83d6-44788fa63a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3829db05ccd3411e84c951d8d79a1a12, user_id=6c812c1d223c4c9f9c753169e11c4c58, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:16.538802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc461e7-fad2-44bb-83d6-44788fa63a3a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:16.578460 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3703/7320] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:16 2022] PUT /identity/v3/projects/3829db05ccd3411e84c951d8d79a1a12/users/6c812c1d223c4c9f9c753169e11c4c58/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:16.579645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8300c6-9d8b-40c8-9ef4-ce2e7003ec94 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:16.594544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8300c6-9d8b-40c8-9ef4-ce2e7003ec94 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:16.771957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bcf4d-9006-45f4-85f4-3c3108b499c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:16.773117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bcf4d-9006-45f4-85f4-3c3108b499c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:16.773557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bcf4d-9006-45f4-85f4-3c3108b499c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:16.774063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bcf4d-9006-45f4-85f4-3c3108b499c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:16.774913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bcf4d-9006-45f4-85f4-3c3108b499c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:16.776763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bcf4d-9006-45f4-85f4-3c3108b499c7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:16.793896 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3618/7321] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:16.795267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc461e7-fad2-44bb-83d6-44788fa63a3a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:16.809281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc461e7-fad2-44bb-83d6-44788fa63a3a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:16.838086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467b3c5-ac32-44e7-9521-178585b694de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:16.839433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467b3c5-ac32-44e7-9521-178585b694de admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:16.841120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467b3c5-ac32-44e7-9521-178585b694de admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:16.841635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467b3c5-ac32-44e7-9521-178585b694de admin admin] PATH_INFO: `/v3/projects/3829db05ccd3411e84c951d8d79a1a12/users/6c812c1d223c4c9f9c753169e11c4c58/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:16.847157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467b3c5-ac32-44e7-9521-178585b694de admin admin] RBAC: Authorizing `identity:create_grant(project_id=3829db05ccd3411e84c951d8d79a1a12, user_id=6c812c1d223c4c9f9c753169e11c4c58, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:16.859843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467b3c5-ac32-44e7-9521-178585b694de admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:16.891884 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3704/7322] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:16 2022] PUT /identity/v3/projects/3829db05ccd3411e84c951d8d79a1a12/users/6c812c1d223c4c9f9c753169e11c4c58/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:25.761169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bcf4d-9006-45f4-85f4-3c3108b499c7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:25.763771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-567bcf4d-9006-45f4-85f4-3c3108b499c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:25.848770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bcf4d-9006-45f4-85f4-3c3108b499c7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:26.070221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:26.070892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:26.071188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:26.071474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] PATH_INFO: `/v3/users/c91b7026d968442eb0db3fd4b5ddc542` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:26.072736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c91b7026d968442eb0db3fd4b5ddc542)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:26.074749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:26.200592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c91b7026d968442eb0db3fd4b5ddc542'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:26.207400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:26.229916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c91b7026d968442eb0db3fd4b5ddc542'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:26.231322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c91b7026d968442eb0db3fd4b5ddc542'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:26.243039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c91b7026d968442eb0db3fd4b5ddc542'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:26.253828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c91b7026d968442eb0db3fd4b5ddc542'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:26.278225 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3619/7323] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:25 2022] DELETE /identity/v3/users/c91b7026d968442eb0db3fd4b5ddc542 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:26.875615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=39,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:27.110662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467b3c5-ac32-44e7-9521-178585b694de admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:27.112932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4467b3c5-ac32-44e7-9521-178585b694de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:27.131769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467b3c5-ac32-44e7-9521-178585b694de admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:27.218514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:27.220157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:27.220755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:27.221591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] PATH_INFO: `/v3/projects/5b8d1a9501b84b76a9ef13d51243cef3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:27.223549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8d1a9501b84b76a9ef13d51243cef3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:27.230389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:27.328036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8d1a9501b84b76a9ef13d51243cef3', 'request_id': 'req-5ce10950-ea62-4707-91c7-60975988e496'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:27.346139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8d1a9501b84b76a9ef13d51243cef3', 'request_id': 'req-5ce10950-ea62-4707-91c7-60975988e496'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:27.363669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8d1a9501b84b76a9ef13d51243cef3', 'request_id': 'req-5ce10950-ea62-4707-91c7-60975988e496'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:27.398405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] The token cache is being invalidate because project 5b8d1a9501b84b76a9ef13d51243cef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:27.398707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:27.399421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:27.401464 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3705/7324] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:27 2022] DELETE /identity/v3/projects/5b8d1a9501b84b76a9ef13d51243cef3 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:27.405943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:27.450394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368558cb-a8ed-460a-9624-8d5dc849c28d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:27.601242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fd9c6-670d-4e92-a171-6706f1af70c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:27.605138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fd9c6-670d-4e92-a171-6706f1af70c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:27.605746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fd9c6-670d-4e92-a171-6706f1af70c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:27.606223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fd9c6-670d-4e92-a171-6706f1af70c7 admin admin] PATH_INFO: `/v3/users/ec39f02676dd4be9b12d17b463fe5971` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:27.609392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fd9c6-670d-4e92-a171-6706f1af70c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec39f02676dd4be9b12d17b463fe5971)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:27.612697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fd9c6-670d-4e92-a171-6706f1af70c7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:27.724400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-794fd9c6-670d-4e92-a171-6706f1af70c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec39f02676dd4be9b12d17b463fe5971'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:27.762600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-794fd9c6-670d-4e92-a171-6706f1af70c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec39f02676dd4be9b12d17b463fe5971'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:27.766682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-794fd9c6-670d-4e92-a171-6706f1af70c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec39f02676dd4be9b12d17b463fe5971'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:27.788496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-794fd9c6-670d-4e92-a171-6706f1af70c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec39f02676dd4be9b12d17b463fe5971'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:27.808726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-794fd9c6-670d-4e92-a171-6706f1af70c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec39f02676dd4be9b12d17b463fe5971'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:27.818387 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3620/7325] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:27 2022] DELETE /identity/v3/users/ec39f02676dd4be9b12d17b463fe5971 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:28.187166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:28.188681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:28.218914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce10950-ea62-4707-91c7-60975988e496 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:28.270234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59900b-718a-4e7a-a732-893de264a43f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:28.270889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59900b-718a-4e7a-a732-893de264a43f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:28.271330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59900b-718a-4e7a-a732-893de264a43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:28.309370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2a59900b-718a-4e7a-a732-893de264a43f None None] MFA Rules not processed for user `6c812c1d223c4c9f9c753169e11c4c58`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:28.310896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2a59900b-718a-4e7a-a732-893de264a43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:28.349827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:28.351931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:28.352588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:28.353092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] PATH_INFO: `/v3/projects/c2c59c2daa264ef4b671363b5a820051` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:28.356212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2c59c2daa264ef4b671363b5a820051)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:28.359470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:28.390885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:28.440368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:28.479348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2c59c2daa264ef4b671363b5a820051', 'request_id': 'req-7deabb21-389c-4139-bc87-7d095e7e66f1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:28.480674 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3621/7326] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:28.492342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59900b-718a-4e7a-a732-893de264a43f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:28.496585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2a59900b-718a-4e7a-a732-893de264a43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:28.509635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2c59c2daa264ef4b671363b5a820051', 'request_id': 'req-7deabb21-389c-4139-bc87-7d095e7e66f1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:28.521272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2c59c2daa264ef4b671363b5a820051', 'request_id': 'req-7deabb21-389c-4139-bc87-7d095e7e66f1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:28.544921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] The token cache is being invalidate because project c2c59c2daa264ef4b671363b5a820051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:28.545141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:28.546953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:28.548979 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3706/7327] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:28 2022] DELETE /identity/v3/projects/c2c59c2daa264ef4b671363b5a820051 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:28.783044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59900b-718a-4e7a-a732-893de264a43f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:28.927160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf07a73-13b3-418e-bb2e-7b74f68dafb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:28.927963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf07a73-13b3-418e-bb2e-7b74f68dafb2 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:28.929217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf07a73-13b3-418e-bb2e-7b74f68dafb2 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:28.929609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf07a73-13b3-418e-bb2e-7b74f68dafb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:28.936608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0bf07a73-13b3-418e-bb2e-7b74f68dafb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:29.097144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf07a73-13b3-418e-bb2e-7b74f68dafb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:29.099082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf07a73-13b3-418e-bb2e-7b74f68dafb2 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:29.138855 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3622/7328] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:16:29.483831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:29.485472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:29.640764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deabb21-389c-4139-bc87-7d095e7e66f1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:29.651221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5977b15-230b-4cae-b542-7d178a51ff6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:29.652352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5977b15-230b-4cae-b542-7d178a51ff6b service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:29.652817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5977b15-230b-4cae-b542-7d178a51ff6b service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:29.653285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5977b15-230b-4cae-b542-7d178a51ff6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:29.655398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e5977b15-230b-4cae-b542-7d178a51ff6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:29.677421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5977b15-230b-4cae-b542-7d178a51ff6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:29.679307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5977b15-230b-4cae-b542-7d178a51ff6b service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:29.701456 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3707/7329] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:35.070311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf07a73-13b3-418e-bb2e-7b74f68dafb2 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:35.074317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0bf07a73-13b3-418e-bb2e-7b74f68dafb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:35.089029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:35.101520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf07a73-13b3-418e-bb2e-7b74f68dafb2 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:35.115030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef25a2-5d26-4401-ab58-51ffeacdf124 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:35.115992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef25a2-5d26-4401-ab58-51ffeacdf124 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:35.116355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef25a2-5d26-4401-ab58-51ffeacdf124 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:35.121468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef25a2-5d26-4401-ab58-51ffeacdf124 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:35.123960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2bef25a2-5d26-4401-ab58-51ffeacdf124 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:35.141645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef25a2-5d26-4401-ab58-51ffeacdf124 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:35.144079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef25a2-5d26-4401-ab58-51ffeacdf124 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:35.182929 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3623/7330] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:37.370290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5977b15-230b-4cae-b542-7d178a51ff6b service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:37.372248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e5977b15-230b-4cae-b542-7d178a51ff6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:37.441160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5977b15-230b-4cae-b542-7d178a51ff6b service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:37.447177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:37.447846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:37.448127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:37.448411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] PATH_INFO: `/v3/users/e6d30843c5c54a1fb41d4d5dff46740c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:37.449609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6d30843c5c54a1fb41d4d5dff46740c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:37.451635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:37.518082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6d30843c5c54a1fb41d4d5dff46740c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:37.525229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:37.536357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6d30843c5c54a1fb41d4d5dff46740c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:37.547629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6d30843c5c54a1fb41d4d5dff46740c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:37.548379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6d30843c5c54a1fb41d4d5dff46740c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:37.549177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6d30843c5c54a1fb41d4d5dff46740c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:37.566902 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3708/7331] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:37 2022] DELETE /identity/v3/users/e6d30843c5c54a1fb41d4d5dff46740c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:37.572192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b0bfe-1dfb-47f6-a1ae-133fc5fae261 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:37.572683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b0bfe-1dfb-47f6-a1ae-133fc5fae261 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:37.572790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b0bfe-1dfb-47f6-a1ae-133fc5fae261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:37.604089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-dc5b0bfe-1dfb-47f6-a1ae-133fc5fae261 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:37.605965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dc5b0bfe-1dfb-47f6-a1ae-133fc5fae261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:37.679587 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3624/7332] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:37.690503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:37.692145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:37.772516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627345e5-a58a-48ed-960f-96793996bb8c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:37.777609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c2dc6-d4a8-4d6d-9487-9f0e0a9b8352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:37.778303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c2dc6-d4a8-4d6d-9487-9f0e0a9b8352 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:37.778653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c2dc6-d4a8-4d6d-9487-9f0e0a9b8352 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:37.779034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c2dc6-d4a8-4d6d-9487-9f0e0a9b8352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:37.795225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c2dc6-d4a8-4d6d-9487-9f0e0a9b8352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:37.796746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c2dc6-d4a8-4d6d-9487-9f0e0a9b8352 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:37.808640 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3709/7333] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:38.015263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b0bfe-1dfb-47f6-a1ae-133fc5fae261 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:38.016984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dc5b0bfe-1dfb-47f6-a1ae-133fc5fae261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:38.050134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b0bfe-1dfb-47f6-a1ae-133fc5fae261 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:38.055909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc6293-a881-4549-a83a-40afa06e739f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:38.056660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc6293-a881-4549-a83a-40afa06e739f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:38.057063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc6293-a881-4549-a83a-40afa06e739f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:38.057413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc6293-a881-4549-a83a-40afa06e739f admin admin] PATH_INFO: `/v3/projects/a4d6a5eda87840a0ac70b059cdfa5339` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:38.058744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc6293-a881-4549-a83a-40afa06e739f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4d6a5eda87840a0ac70b059cdfa5339)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:38.060805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc6293-a881-4549-a83a-40afa06e739f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:38.138825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1bc6293-a881-4549-a83a-40afa06e739f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4d6a5eda87840a0ac70b059cdfa5339', 'request_id': 'req-b1bc6293-a881-4549-a83a-40afa06e739f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:38.148333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1bc6293-a881-4549-a83a-40afa06e739f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4d6a5eda87840a0ac70b059cdfa5339', 'request_id': 'req-b1bc6293-a881-4549-a83a-40afa06e739f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:38.162646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1bc6293-a881-4549-a83a-40afa06e739f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4d6a5eda87840a0ac70b059cdfa5339', 'request_id': 'req-b1bc6293-a881-4549-a83a-40afa06e739f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:38.179209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1bc6293-a881-4549-a83a-40afa06e739f admin admin] The token cache is being invalidate because project a4d6a5eda87840a0ac70b059cdfa5339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:38.179607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1bc6293-a881-4549-a83a-40afa06e739f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:38.180375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b1bc6293-a881-4549-a83a-40afa06e739f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:38.185725 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3625/7334] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:38 2022] DELETE /identity/v3/projects/a4d6a5eda87840a0ac70b059cdfa5339 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:38.187596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d31868-8082-416c-91e5-15be80a33e99 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:38.188097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d31868-8082-416c-91e5-15be80a33e99 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:38.190096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d31868-8082-416c-91e5-15be80a33e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:38.213157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-01d31868-8082-416c-91e5-15be80a33e99 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:38.215416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-01d31868-8082-416c-91e5-15be80a33e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:38.316985 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3710/7335] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:38.317991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc4508-fd6c-4de9-9e81-7a36fa7ab17a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:38.318306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc4508-fd6c-4de9-9e81-7a36fa7ab17a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:38.318591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc4508-fd6c-4de9-9e81-7a36fa7ab17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:38.340815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-63fc4508-fd6c-4de9-9e81-7a36fa7ab17a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:38.342281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-63fc4508-fd6c-4de9-9e81-7a36fa7ab17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:38.357169 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3626/7336] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:38.361478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d31868-8082-416c-91e5-15be80a33e99 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:38.372316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d31868-8082-416c-91e5-15be80a33e99 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:38.380315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aef45-2e1d-48de-8709-f5d1788fea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:38.381061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aef45-2e1d-48de-8709-f5d1788fea45 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:38.381354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aef45-2e1d-48de-8709-f5d1788fea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:38.381637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aef45-2e1d-48de-8709-f5d1788fea45 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:38.382319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96aef45-2e1d-48de-8709-f5d1788fea45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:16:38.382634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96aef45-2e1d-48de-8709-f5d1788fea45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:38.385645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96aef45-2e1d-48de-8709-f5d1788fea45 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:38.403106 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3711/7337] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:38.404317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc4508-fd6c-4de9-9e81-7a36fa7ab17a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:38.415403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc4508-fd6c-4de9-9e81-7a36fa7ab17a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:38.422055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf070d-8e8e-42f9-a544-8f0df03ed265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:38.422685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf070d-8e8e-42f9-a544-8f0df03ed265 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:38.422973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf070d-8e8e-42f9-a544-8f0df03ed265 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:38.423254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf070d-8e8e-42f9-a544-8f0df03ed265 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:38.423900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf070d-8e8e-42f9-a544-8f0df03ed265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:38.425899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf070d-8e8e-42f9-a544-8f0df03ed265 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:38.463174 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3627/7338] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:38 2022] POST /identity/v3/projects => generated 357 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:38.464584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aef45-2e1d-48de-8709-f5d1788fea45 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:38.475302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aef45-2e1d-48de-8709-f5d1788fea45 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:38.541967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42650e56-a113-47ac-a969-4c09e489ec68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:38.542902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42650e56-a113-47ac-a969-4c09e489ec68 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:38.543355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42650e56-a113-47ac-a969-4c09e489ec68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:38.543799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42650e56-a113-47ac-a969-4c09e489ec68 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:38.544825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42650e56-a113-47ac-a969-4c09e489ec68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:38.546781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42650e56-a113-47ac-a969-4c09e489ec68 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:38.555777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:38.580548 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3712/7339] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:38 2022] POST /identity/v3/users => generated 279 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:38.584697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf070d-8e8e-42f9-a544-8f0df03ed265 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:38.596283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf070d-8e8e-42f9-a544-8f0df03ed265 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:38.601013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce19c8-92ed-41c8-82be-995ab135305b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:38.601631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce19c8-92ed-41c8-82be-995ab135305b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:38.601922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce19c8-92ed-41c8-82be-995ab135305b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:38.602210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce19c8-92ed-41c8-82be-995ab135305b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:38.603043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce19c8-92ed-41c8-82be-995ab135305b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:38.604937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce19c8-92ed-41c8-82be-995ab135305b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:38.616172 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3628/7340] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:38.619372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42650e56-a113-47ac-a969-4c09e489ec68 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:38.629709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42650e56-a113-47ac-a969-4c09e489ec68 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:38.634860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3276d-34c7-48c8-aca8-a4f25e4cce6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:38.635459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3276d-34c7-48c8-aca8-a4f25e4cce6b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:38.635747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3276d-34c7-48c8-aca8-a4f25e4cce6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:38.636029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3276d-34c7-48c8-aca8-a4f25e4cce6b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:38.636687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3276d-34c7-48c8-aca8-a4f25e4cce6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:38.638334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3276d-34c7-48c8-aca8-a4f25e4cce6b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:38.653021 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3713/7341] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:38.654613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce19c8-92ed-41c8-82be-995ab135305b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:38.665477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce19c8-92ed-41c8-82be-995ab135305b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:38.670607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa8d29-cca0-48e2-9441-95090fe0565f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:38.671349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa8d29-cca0-48e2-9441-95090fe0565f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:38.671705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa8d29-cca0-48e2-9441-95090fe0565f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:38.672052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa8d29-cca0-48e2-9441-95090fe0565f admin admin] PATH_INFO: `/v3/projects/e9fcc50f4f7543ecb59858bac371e9c6/users/f59d36fafb414e4682aa281690462869/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:38.687465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa8d29-cca0-48e2-9441-95090fe0565f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9fcc50f4f7543ecb59858bac371e9c6, user_id=f59d36fafb414e4682aa281690462869, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:38.690298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa8d29-cca0-48e2-9441-95090fe0565f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:38.700120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:38.706772 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3629/7342] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:38 2022] PUT /identity/v3/projects/e9fcc50f4f7543ecb59858bac371e9c6/users/f59d36fafb414e4682aa281690462869/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:38.718656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b0ba0-23ec-45d4-a46f-38016d27ac33 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:38.718999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b0ba0-23ec-45d4-a46f-38016d27ac33 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:38.719281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b0ba0-23ec-45d4-a46f-38016d27ac33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:38.748637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f72b0ba0-23ec-45d4-a46f-38016d27ac33 None None] MFA Rules not processed for user `f59d36fafb414e4682aa281690462869`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:38.749896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f72b0ba0-23ec-45d4-a46f-38016d27ac33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:38.836019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa8d29-cca0-48e2-9441-95090fe0565f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:38.836437 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3714/7343] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:38.952951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa8d29-cca0-48e2-9441-95090fe0565f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:38.963254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ec87a-8e0f-43d2-ab27-782ee034f82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:38.963902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ec87a-8e0f-43d2-ab27-782ee034f82f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:38.964194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ec87a-8e0f-43d2-ab27-782ee034f82f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:38.964477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ec87a-8e0f-43d2-ab27-782ee034f82f admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:38.965187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ec87a-8e0f-43d2-ab27-782ee034f82f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:38.967250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ec87a-8e0f-43d2-ab27-782ee034f82f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:39.031098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b0ba0-23ec-45d4-a46f-38016d27ac33 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:39.033402 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3630/7344] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:38 2022] POST /identity/v3/projects => generated 357 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:39.086636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b0ba0-23ec-45d4-a46f-38016d27ac33 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:39.207316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9f2bb-6e94-44c5-b787-f2416c46874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:39.207727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9f2bb-6e94-44c5-b787-f2416c46874a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:39.207948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9f2bb-6e94-44c5-b787-f2416c46874a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:39.208237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9f2bb-6e94-44c5-b787-f2416c46874a admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:39.208906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9f2bb-6e94-44c5-b787-f2416c46874a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:39.210986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9f2bb-6e94-44c5-b787-f2416c46874a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:39.260244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ec87a-8e0f-43d2-ab27-782ee034f82f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:39.266036 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3715/7345] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:39 2022] POST /identity/v3/users => generated 285 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:39.280261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ec87a-8e0f-43d2-ab27-782ee034f82f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:39.286857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc16e1d-50ca-46eb-bbad-6c49724602d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:39.287527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc16e1d-50ca-46eb-bbad-6c49724602d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:39.287938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc16e1d-50ca-46eb-bbad-6c49724602d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:39.288230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc16e1d-50ca-46eb-bbad-6c49724602d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:39.288924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc16e1d-50ca-46eb-bbad-6c49724602d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:39.293190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc16e1d-50ca-46eb-bbad-6c49724602d3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:39.321419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9f2bb-6e94-44c5-b787-f2416c46874a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:39.328097 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3631/7346] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:39.333155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9f2bb-6e94-44c5-b787-f2416c46874a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:39.339877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa858f0-b2a8-4629-a867-c55a02b6add4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:39.340992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa858f0-b2a8-4629-a867-c55a02b6add4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:39.341389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa858f0-b2a8-4629-a867-c55a02b6add4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:39.342666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa858f0-b2a8-4629-a867-c55a02b6add4 admin admin] PATH_INFO: `/v3/projects/ff6e50afb3774ad0942be133f24a4af2/users/86b2d0fca835428e9200f70f2dcb467f/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:39.361822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa858f0-b2a8-4629-a867-c55a02b6add4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff6e50afb3774ad0942be133f24a4af2, user_id=86b2d0fca835428e9200f70f2dcb467f, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:39.364826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa858f0-b2a8-4629-a867-c55a02b6add4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:39.388935 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3716/7347] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:39 2022] PUT /identity/v3/projects/ff6e50afb3774ad0942be133f24a4af2/users/86b2d0fca835428e9200f70f2dcb467f/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:39.393531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc16e1d-50ca-46eb-bbad-6c49724602d3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:39.409815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc16e1d-50ca-46eb-bbad-6c49724602d3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:39.512517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8839e2-0232-46a8-b708-5c938af964a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:39.513179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8839e2-0232-46a8-b708-5c938af964a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:39.513474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8839e2-0232-46a8-b708-5c938af964a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:39.513757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8839e2-0232-46a8-b708-5c938af964a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:39.514562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8839e2-0232-46a8-b708-5c938af964a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:39.516207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8839e2-0232-46a8-b708-5c938af964a9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:39.532636 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3632/7348] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:39.533896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa858f0-b2a8-4629-a867-c55a02b6add4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:39.554959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa858f0-b2a8-4629-a867-c55a02b6add4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:39.576071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae7c85-b0ee-4221-8912-43fbc32c71b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:39.576798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae7c85-b0ee-4221-8912-43fbc32c71b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:39.577117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae7c85-b0ee-4221-8912-43fbc32c71b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:39.577817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae7c85-b0ee-4221-8912-43fbc32c71b7 admin admin] PATH_INFO: `/v3/projects/ff6e50afb3774ad0942be133f24a4af2/users/86b2d0fca835428e9200f70f2dcb467f/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:39.602339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae7c85-b0ee-4221-8912-43fbc32c71b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff6e50afb3774ad0942be133f24a4af2, user_id=86b2d0fca835428e9200f70f2dcb467f, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:39.611920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae7c85-b0ee-4221-8912-43fbc32c71b7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:39.635439 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3717/7349] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:39 2022] PUT /identity/v3/projects/ff6e50afb3774ad0942be133f24a4af2/users/86b2d0fca835428e9200f70f2dcb467f/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:39.647547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff2d98-edbc-48b7-8089-07ee67265bce None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:39.648065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff2d98-edbc-48b7-8089-07ee67265bce None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:39.648180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff2d98-edbc-48b7-8089-07ee67265bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:39.689604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e7ff2d98-edbc-48b7-8089-07ee67265bce None None] MFA Rules not processed for user `86b2d0fca835428e9200f70f2dcb467f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:39.690861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e7ff2d98-edbc-48b7-8089-07ee67265bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:39.741349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:39.780588 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3633/7350] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:39 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:39.802804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae7c85-b0ee-4221-8912-43fbc32c71b7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:39.804184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0fae7c85-b0ee-4221-8912-43fbc32c71b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:39.810473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:39.921899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae7c85-b0ee-4221-8912-43fbc32c71b7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:39.931006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bfedd-4121-40a7-bd68-23735dc9e107 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:39.931901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bfedd-4121-40a7-bd68-23735dc9e107 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:39.932190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bfedd-4121-40a7-bd68-23735dc9e107 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:39.932588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bfedd-4121-40a7-bd68-23735dc9e107 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:39.946173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bfedd-4121-40a7-bd68-23735dc9e107 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:39.948442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bfedd-4121-40a7-bd68-23735dc9e107 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:39.961298 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3718/7351] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:39 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:40.062616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff2d98-edbc-48b7-8089-07ee67265bce None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:40.076454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff2d98-edbc-48b7-8089-07ee67265bce None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:40.081428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f53922-ef82-488f-ac95-69cd003f8160 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:40.082706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f53922-ef82-488f-ac95-69cd003f8160 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:40.083083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f53922-ef82-488f-ac95-69cd003f8160 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:40.083436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f53922-ef82-488f-ac95-69cd003f8160 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:40.187357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f53922-ef82-488f-ac95-69cd003f8160 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:40.189220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f53922-ef82-488f-ac95-69cd003f8160 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:40.203409 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3634/7352] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:40 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:40.942184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bfedd-4121-40a7-bd68-23735dc9e107 service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:40.943884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-237bfedd-4121-40a7-bd68-23735dc9e107 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:41.058663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bfedd-4121-40a7-bd68-23735dc9e107 service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:41.063508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:41.064514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:41.065015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:41.065631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] PATH_INFO: `/v3/users/ec4d4dfdda4f4f86a6cffadc35d9e68e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:41.067058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4d4dfdda4f4f86a6cffadc35d9e68e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:41.069731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:41.184253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4d4dfdda4f4f86a6cffadc35d9e68e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.204347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4d4dfdda4f4f86a6cffadc35d9e68e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.218364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4d4dfdda4f4f86a6cffadc35d9e68e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.219016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4d4dfdda4f4f86a6cffadc35d9e68e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.222261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4d4dfdda4f4f86a6cffadc35d9e68e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.223340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f53922-ef82-488f-ac95-69cd003f8160 service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:41.224995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-96f53922-ef82-488f-ac95-69cd003f8160 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:41.239664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f53922-ef82-488f-ac95-69cd003f8160 service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:41.240277 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3719/7353] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:40 2022] DELETE /identity/v3/users/ec4d4dfdda4f4f86a6cffadc35d9e68e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:41.315477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:41.317414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:41.319508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:41.319875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] PATH_INFO: `/v3/users/07b3ac8e2f3f48448a2470ab148f4fd2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:41.321938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07b3ac8e2f3f48448a2470ab148f4fd2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:41.323944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:41.382771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07b3ac8e2f3f48448a2470ab148f4fd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.401321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07b3ac8e2f3f48448a2470ab148f4fd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.402250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07b3ac8e2f3f48448a2470ab148f4fd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.410828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07b3ac8e2f3f48448a2470ab148f4fd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.416476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:41.422396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07b3ac8e2f3f48448a2470ab148f4fd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.434294 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3635/7354] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:41 2022] DELETE /identity/v3/users/07b3ac8e2f3f48448a2470ab148f4fd2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:41.446109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:41.447758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:41.486598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503f1ad-d09b-47e4-8466-bb0a11ef5e30 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:41.515695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:41.571752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:41.572415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:41.572713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:41.573063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a admin admin] PATH_INFO: `/v3/projects/42698a69a75f4d6583e8dd7df3926aa6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:41.574264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=42698a69a75f4d6583e8dd7df3926aa6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:41.577373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:41.638593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42698a69a75f4d6583e8dd7df3926aa6', 'request_id': 'req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.648159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42698a69a75f4d6583e8dd7df3926aa6', 'request_id': 'req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.654677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:41.657604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:41.657747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42698a69a75f4d6583e8dd7df3926aa6', 'request_id': 'req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.676188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a admin admin] The token cache is being invalidate because project 42698a69a75f4d6583e8dd7df3926aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:41.676515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.677719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9771a13c-4c05-48cf-9dbb-c4834bfbca3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.680161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4471f-40d5-45b1-8b36-8c9a645e17e6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:41.684981 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3720/7355] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:41 2022] DELETE /identity/v3/projects/42698a69a75f4d6583e8dd7df3926aa6 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:41.689778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17e16d-eb07-4a23-b951-743f162d6330 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:41.690464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17e16d-eb07-4a23-b951-743f162d6330 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:41.692176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17e16d-eb07-4a23-b951-743f162d6330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:41.738473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3d17e16d-eb07-4a23-b951-743f162d6330 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:41.740301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3d17e16d-eb07-4a23-b951-743f162d6330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:41.783177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:41.783811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:41.784072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:41.784414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] PATH_INFO: `/v3/projects/fb6931dda93843e69b69129cdc30d0c8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:41.785582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6931dda93843e69b69129cdc30d0c8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:41.787579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:41.835850 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3721/7356] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:41.846729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245db6c-85eb-4b59-a95e-ab350c67fc93 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:41.847101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245db6c-85eb-4b59-a95e-ab350c67fc93 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:41.847597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245db6c-85eb-4b59-a95e-ab350c67fc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:41.871863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6931dda93843e69b69129cdc30d0c8', 'request_id': 'req-758152f8-09b1-466d-ad32-2d3e19755af5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.878318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d245db6c-85eb-4b59-a95e-ab350c67fc93 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:41.880230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d245db6c-85eb-4b59-a95e-ab350c67fc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:41.880363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6931dda93843e69b69129cdc30d0c8', 'request_id': 'req-758152f8-09b1-466d-ad32-2d3e19755af5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.889050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6931dda93843e69b69129cdc30d0c8', 'request_id': 'req-758152f8-09b1-466d-ad32-2d3e19755af5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.904642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] The token cache is being invalidate because project fb6931dda93843e69b69129cdc30d0c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:41.905058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.905792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:41.907620 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3636/7357] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:41 2022] DELETE /identity/v3/projects/fb6931dda93843e69b69129cdc30d0c8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:41.911262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:41.912669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:41.977267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:41.992120 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3722/7358] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:41.994845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758152f8-09b1-466d-ad32-2d3e19755af5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:41.996187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245db6c-85eb-4b59-a95e-ab350c67fc93 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.000404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.001086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.001390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.001646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] PATH_INFO: `/v3/users/6c812c1d223c4c9f9c753169e11c4c58` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.003218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c812c1d223c4c9f9c753169e11c4c58)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.005249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.007039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245db6c-85eb-4b59-a95e-ab350c67fc93 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.011823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546c97e-456f-4a60-9deb-7b87e3a7a029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.012436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546c97e-456f-4a60-9deb-7b87e3a7a029 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.012722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546c97e-456f-4a60-9deb-7b87e3a7a029 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.013038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546c97e-456f-4a60-9deb-7b87e3a7a029 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.013683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546c97e-456f-4a60-9deb-7b87e3a7a029 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:16:42.013990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546c97e-456f-4a60-9deb-7b87e3a7a029 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.015612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546c97e-456f-4a60-9deb-7b87e3a7a029 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.027752 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3723/7359] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:16:42.031958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546c97e-456f-4a60-9deb-7b87e3a7a029 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.038426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:42.042681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546c97e-456f-4a60-9deb-7b87e3a7a029 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.048022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29f90d-3c92-4c40-8ab9-b12e29ae450b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.048628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29f90d-3c92-4c40-8ab9-b12e29ae450b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.049031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29f90d-3c92-4c40-8ab9-b12e29ae450b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.049283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29f90d-3c92-4c40-8ab9-b12e29ae450b admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.049838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29f90d-3c92-4c40-8ab9-b12e29ae450b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.051741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29f90d-3c92-4c40-8ab9-b12e29ae450b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.058458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c812c1d223c4c9f9c753169e11c4c58'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:42.071416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c812c1d223c4c9f9c753169e11c4c58'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:42.072221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c812c1d223c4c9f9c753169e11c4c58'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:42.079263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c812c1d223c4c9f9c753169e11c4c58'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:42.083474 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3724/7360] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:42 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:42.084739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29f90d-3c92-4c40-8ab9-b12e29ae450b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.087644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ec29f90d-3c92-4c40-8ab9-b12e29ae450b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:42.091791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c812c1d223c4c9f9c753169e11c4c58'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:42.099241 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3637/7361] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:41 2022] DELETE /identity/v3/users/6c812c1d223c4c9f9c753169e11c4c58 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:42.172152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29f90d-3c92-4c40-8ab9-b12e29ae450b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.231366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc5e2f-3245-4fd6-9056-196ebaa13dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.231989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc5e2f-3245-4fd6-9056-196ebaa13dbb admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.232271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc5e2f-3245-4fd6-9056-196ebaa13dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.232544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc5e2f-3245-4fd6-9056-196ebaa13dbb admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.233208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc5e2f-3245-4fd6-9056-196ebaa13dbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.235154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc5e2f-3245-4fd6-9056-196ebaa13dbb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.257981 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3725/7362] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:42 2022] POST /identity/v3/users => generated 302 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:42.262403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.273343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c327e09-9270-4c1f-9a40-d9012bb4bd65 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.280577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc5e2f-3245-4fd6-9056-196ebaa13dbb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.281687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca551a53-9152-4da2-80d0-cd51d7f9c424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.282367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9bbc5e2f-3245-4fd6-9056-196ebaa13dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:42.282509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca551a53-9152-4da2-80d0-cd51d7f9c424 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.282808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca551a53-9152-4da2-80d0-cd51d7f9c424 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.283079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca551a53-9152-4da2-80d0-cd51d7f9c424 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.283951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca551a53-9152-4da2-80d0-cd51d7f9c424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.285627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca551a53-9152-4da2-80d0-cd51d7f9c424 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.296942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc5e2f-3245-4fd6-9056-196ebaa13dbb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.302434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.303295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.303660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.303778 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3638/7363] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:42.303988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28 admin admin] PATH_INFO: `/v3/projects/3829db05ccd3411e84c951d8d79a1a12` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.305189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3829db05ccd3411e84c951d8d79a1a12)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.307201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.308229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca551a53-9152-4da2-80d0-cd51d7f9c424 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.318736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca551a53-9152-4da2-80d0-cd51d7f9c424 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.323878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d63b4-48ad-4616-bfd6-7d85bfcc3e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.324478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d63b4-48ad-4616-bfd6-7d85bfcc3e68 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.324766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d63b4-48ad-4616-bfd6-7d85bfcc3e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.325078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d63b4-48ad-4616-bfd6-7d85bfcc3e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.325718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d63b4-48ad-4616-bfd6-7d85bfcc3e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.327305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d63b4-48ad-4616-bfd6-7d85bfcc3e68 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.338538 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3639/7364] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:42.342783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d63b4-48ad-4616-bfd6-7d85bfcc3e68 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.352734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d63b4-48ad-4616-bfd6-7d85bfcc3e68 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.360173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3829db05ccd3411e84c951d8d79a1a12', 'request_id': 'req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:42.362765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:42.369392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3829db05ccd3411e84c951d8d79a1a12', 'request_id': 'req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:42.377916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3829db05ccd3411e84c951d8d79a1a12', 'request_id': 'req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:42.390838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28 admin admin] The token cache is being invalidate because project 3829db05ccd3411e84c951d8d79a1a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:42.391134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:42.391809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5519be5c-dfc9-4fec-a27b-33bfa00a4b28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:42.393998 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3726/7365] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:42 2022] DELETE /identity/v3/projects/3829db05ccd3411e84c951d8d79a1a12 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:42.398458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8143fa-40ee-47be-a5d7-b6bb4ddee0bc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.398750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8143fa-40ee-47be-a5d7-b6bb4ddee0bc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.399028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8143fa-40ee-47be-a5d7-b6bb4ddee0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.413474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3e5c7-76ea-4d54-8236-dfbb8c556dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.414135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e5c7-76ea-4d54-8236-dfbb8c556dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.414427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e5c7-76ea-4d54-8236-dfbb8c556dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.414708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e5c7-76ea-4d54-8236-dfbb8c556dd9 admin admin] PATH_INFO: `/v3/projects/88d01939286b48788929cb691584cd08/users/1fc4f490eb164f79aeacaca196bb0a91/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.419241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0a8143fa-40ee-47be-a5d7-b6bb4ddee0bc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:42.420458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0a8143fa-40ee-47be-a5d7-b6bb4ddee0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:42.428469 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3727/7366] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:42.434401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3e5c7-76ea-4d54-8236-dfbb8c556dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88d01939286b48788929cb691584cd08, user_id=1fc4f490eb164f79aeacaca196bb0a91, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.436173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb7d64-0b4f-435a-8dfd-aef78f23d347 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.436464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb7d64-0b4f-435a-8dfd-aef78f23d347 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.436746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb7d64-0b4f-435a-8dfd-aef78f23d347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.437202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3e5c7-76ea-4d54-8236-dfbb8c556dd9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.452439 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3640/7367] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:42 2022] PUT /identity/v3/projects/88d01939286b48788929cb691584cd08/users/1fc4f490eb164f79aeacaca196bb0a91/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:42.457662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-31cb7d64-0b4f-435a-8dfd-aef78f23d347 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:42.458862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-31cb7d64-0b4f-435a-8dfd-aef78f23d347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:42.463334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef5cd-e5cd-44ff-9015-b95d2a79098f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.463751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef5cd-e5cd-44ff-9015-b95d2a79098f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.464035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef5cd-e5cd-44ff-9015-b95d2a79098f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.493635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-403ef5cd-e5cd-44ff-9015-b95d2a79098f None None] MFA Rules not processed for user `1fc4f490eb164f79aeacaca196bb0a91`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:42.494943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-403ef5cd-e5cd-44ff-9015-b95d2a79098f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:42.520993 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3728/7368] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:42.525157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb7d64-0b4f-435a-8dfd-aef78f23d347 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.534035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb7d64-0b4f-435a-8dfd-aef78f23d347 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.538743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea46df-8b3f-4101-aa33-e2cdc0e5988d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.539396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea46df-8b3f-4101-aa33-e2cdc0e5988d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.539625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea46df-8b3f-4101-aa33-e2cdc0e5988d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.539912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea46df-8b3f-4101-aa33-e2cdc0e5988d admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.540577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea46df-8b3f-4101-aa33-e2cdc0e5988d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:16:42.540887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea46df-8b3f-4101-aa33-e2cdc0e5988d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.542532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea46df-8b3f-4101-aa33-e2cdc0e5988d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.551376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:42.554223 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3729/7369] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:16:42.558897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea46df-8b3f-4101-aa33-e2cdc0e5988d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.565195 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3641/7370] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:42 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:42.566383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef5cd-e5cd-44ff-9015-b95d2a79098f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.567722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-403ef5cd-e5cd-44ff-9015-b95d2a79098f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:42.569274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea46df-8b3f-4101-aa33-e2cdc0e5988d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.574570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d74a2a-1b44-4356-8214-1d5df83aedaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.575165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d74a2a-1b44-4356-8214-1d5df83aedaa admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.575454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d74a2a-1b44-4356-8214-1d5df83aedaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.575736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d74a2a-1b44-4356-8214-1d5df83aedaa admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.576368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d74a2a-1b44-4356-8214-1d5df83aedaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.578335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d74a2a-1b44-4356-8214-1d5df83aedaa admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.581467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef5cd-e5cd-44ff-9015-b95d2a79098f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.586068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341da753-11f0-4bae-971a-79aa1517a7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.586743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341da753-11f0-4bae-971a-79aa1517a7fd admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.587057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341da753-11f0-4bae-971a-79aa1517a7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.587335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341da753-11f0-4bae-971a-79aa1517a7fd admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.587954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341da753-11f0-4bae-971a-79aa1517a7fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.589887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341da753-11f0-4bae-971a-79aa1517a7fd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.607648 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3730/7371] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:42 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:42.612052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d74a2a-1b44-4356-8214-1d5df83aedaa admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.620455 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3642/7372] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:42 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:42.624253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341da753-11f0-4bae-971a-79aa1517a7fd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.682696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d74a2a-1b44-4356-8214-1d5df83aedaa admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.687967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c0cda-0650-4631-9158-5f612c2e9c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.688587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341da753-11f0-4bae-971a-79aa1517a7fd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.688737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c0cda-0650-4631-9158-5f612c2e9c7e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.689003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c0cda-0650-4631-9158-5f612c2e9c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.689288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c0cda-0650-4631-9158-5f612c2e9c7e admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.689983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c0cda-0650-4631-9158-5f612c2e9c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.691982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c0cda-0650-4631-9158-5f612c2e9c7e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.693393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa663c7-b938-4341-9f25-e5c6ae792ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.693996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa663c7-b938-4341-9f25-e5c6ae792ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.694278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa663c7-b938-4341-9f25-e5c6ae792ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.694554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa663c7-b938-4341-9f25-e5c6ae792ec8 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.695177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa663c7-b938-4341-9f25-e5c6ae792ec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.697671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa663c7-b938-4341-9f25-e5c6ae792ec8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.715995 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3731/7373] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:42 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:42.720110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c0cda-0650-4631-9158-5f612c2e9c7e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.720689 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3643/7374] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:42 2022] POST /identity/v3/users => generated 308 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:42.724297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa663c7-b938-4341-9f25-e5c6ae792ec8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.730545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c0cda-0650-4631-9158-5f612c2e9c7e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.734776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa663c7-b938-4341-9f25-e5c6ae792ec8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.735121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7d08a-234a-416d-a2c8-af3b110e36ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.735713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7d08a-234a-416d-a2c8-af3b110e36ed admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.735998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7d08a-234a-416d-a2c8-af3b110e36ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.736273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7d08a-234a-416d-a2c8-af3b110e36ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.736933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7d08a-234a-416d-a2c8-af3b110e36ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.738515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7d08a-234a-416d-a2c8-af3b110e36ed admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.739773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187b249-6783-454d-ab5f-448b55aeb009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.740359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b249-6783-454d-ab5f-448b55aeb009 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.740642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b249-6783-454d-ab5f-448b55aeb009 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.740960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b249-6783-454d-ab5f-448b55aeb009 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.741597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187b249-6783-454d-ab5f-448b55aeb009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.743768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187b249-6783-454d-ab5f-448b55aeb009 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.749082 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3732/7375] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:42.752799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7d08a-234a-416d-a2c8-af3b110e36ed admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.754497 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3644/7376] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:42.758080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187b249-6783-454d-ab5f-448b55aeb009 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.763409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7d08a-234a-416d-a2c8-af3b110e36ed admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.768231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc53cd6-f646-44af-bb09-dd17cedc4eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.768372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187b249-6783-454d-ab5f-448b55aeb009 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.768832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc53cd6-f646-44af-bb09-dd17cedc4eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.769165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc53cd6-f646-44af-bb09-dd17cedc4eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.769448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc53cd6-f646-44af-bb09-dd17cedc4eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.770104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc53cd6-f646-44af-bb09-dd17cedc4eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.771674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc53cd6-f646-44af-bb09-dd17cedc4eb4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.773087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60fe93-9071-426e-9344-985f42b7852d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.773717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60fe93-9071-426e-9344-985f42b7852d admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.773999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60fe93-9071-426e-9344-985f42b7852d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.774325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60fe93-9071-426e-9344-985f42b7852d admin admin] PATH_INFO: `/v3/projects/4bfe1e37d86049b489c211838ea1cf2a/users/54a52f33e4024fe1bf0683fecde33717/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.782568 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3733/7377] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:42.786530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc53cd6-f646-44af-bb09-dd17cedc4eb4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.792334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60fe93-9071-426e-9344-985f42b7852d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bfe1e37d86049b489c211838ea1cf2a, user_id=54a52f33e4024fe1bf0683fecde33717, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.794987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60fe93-9071-426e-9344-985f42b7852d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.796210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc53cd6-f646-44af-bb09-dd17cedc4eb4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.801078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e461d4-1a7d-40c5-8bbb-db7d61e2402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.801717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e461d4-1a7d-40c5-8bbb-db7d61e2402d admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.802004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e461d4-1a7d-40c5-8bbb-db7d61e2402d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.802278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e461d4-1a7d-40c5-8bbb-db7d61e2402d admin admin] PATH_INFO: `/v3/projects/046a6abde8c84a498c1c063a1c908b42/users/b5c96be8fbe1436bb3e84340ceda1917/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.804541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:42.809740 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3645/7378] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:42 2022] PUT /identity/v3/projects/4bfe1e37d86049b489c211838ea1cf2a/users/54a52f33e4024fe1bf0683fecde33717/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:42.813927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60fe93-9071-426e-9344-985f42b7852d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.820229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e461d4-1a7d-40c5-8bbb-db7d61e2402d admin admin] RBAC: Authorizing `identity:create_grant(project_id=046a6abde8c84a498c1c063a1c908b42, user_id=b5c96be8fbe1436bb3e84340ceda1917, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.823546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e461d4-1a7d-40c5-8bbb-db7d61e2402d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.836224 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3734/7379] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:42 2022] PUT /identity/v3/projects/046a6abde8c84a498c1c063a1c908b42/users/b5c96be8fbe1436bb3e84340ceda1917/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:42.847747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d42da-0ff2-4e13-999a-f6ff07d9b063 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.848040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d42da-0ff2-4e13-999a-f6ff07d9b063 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.848321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d42da-0ff2-4e13-999a-f6ff07d9b063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.876747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-232d42da-0ff2-4e13-999a-f6ff07d9b063 None None] MFA Rules not processed for user `b5c96be8fbe1436bb3e84340ceda1917`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:42.878013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-232d42da-0ff2-4e13-999a-f6ff07d9b063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:42.879894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60fe93-9071-426e-9344-985f42b7852d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.914285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:42.941349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f788e37-68e4-4d5d-aabe-2cdf9ce3cb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.941963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f788e37-68e4-4d5d-aabe-2cdf9ce3cb35 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.942251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f788e37-68e4-4d5d-aabe-2cdf9ce3cb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.942527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f788e37-68e4-4d5d-aabe-2cdf9ce3cb35 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.943048 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3735/7380] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:42.943184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f788e37-68e4-4d5d-aabe-2cdf9ce3cb35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.944771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f788e37-68e4-4d5d-aabe-2cdf9ce3cb35 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.947425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d42da-0ff2-4e13-999a-f6ff07d9b063 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.958392 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3646/7381] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:42.958569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d42da-0ff2-4e13-999a-f6ff07d9b063 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.959665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f788e37-68e4-4d5d-aabe-2cdf9ce3cb35 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:42.963163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf7081-2df6-4bff-8c3b-0a0032cd3730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.963767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf7081-2df6-4bff-8c3b-0a0032cd3730 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.964059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf7081-2df6-4bff-8c3b-0a0032cd3730 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.964341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf7081-2df6-4bff-8c3b-0a0032cd3730 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.965053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf7081-2df6-4bff-8c3b-0a0032cd3730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.966945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf7081-2df6-4bff-8c3b-0a0032cd3730 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.969804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f788e37-68e4-4d5d-aabe-2cdf9ce3cb35 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:42.975757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ba6d3-8c55-4f1c-bce8-7745cffcdc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:42.975757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ba6d3-8c55-4f1c-bce8-7745cffcdc56 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:42.975757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ba6d3-8c55-4f1c-bce8-7745cffcdc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:42.975757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ba6d3-8c55-4f1c-bce8-7745cffcdc56 admin admin] PATH_INFO: `/v3/projects/4bfe1e37d86049b489c211838ea1cf2a/users/54a52f33e4024fe1bf0683fecde33717/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:42.978445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ba6d3-8c55-4f1c-bce8-7745cffcdc56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bfe1e37d86049b489c211838ea1cf2a, user_id=54a52f33e4024fe1bf0683fecde33717, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:42.981389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ba6d3-8c55-4f1c-bce8-7745cffcdc56 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:42.994031 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3647/7382] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:42 2022] PUT /identity/v3/projects/4bfe1e37d86049b489c211838ea1cf2a/users/54a52f33e4024fe1bf0683fecde33717/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:42.998565 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3736/7383] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:42 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:43.002936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ba6d3-8c55-4f1c-bce8-7745cffcdc56 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.003950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a49fa-72fe-4f96-95b9-3280f9a69a30 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.004248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a49fa-72fe-4f96-95b9-3280f9a69a30 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.004535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a49fa-72fe-4f96-95b9-3280f9a69a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.036333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1a0a49fa-72fe-4f96-95b9-3280f9a69a30 None None] MFA Rules not processed for user `54a52f33e4024fe1bf0683fecde33717`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:43.037619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1a0a49fa-72fe-4f96-95b9-3280f9a69a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:43.080641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ba6d3-8c55-4f1c-bce8-7745cffcdc56 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.137120 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3737/7384] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:43.138088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a49fa-72fe-4f96-95b9-3280f9a69a30 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.149092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a49fa-72fe-4f96-95b9-3280f9a69a30 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.154240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a717137-05eb-428e-bfb1-9a32f881b9a6 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54a52f33e4024fe1bf0683fecde33717', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bfe1e37d86049b489c211838ea1cf2a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:43.154834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a717137-05eb-428e-bfb1-9a32f881b9a6 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.155128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a717137-05eb-428e-bfb1-9a32f881b9a6 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.155413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a717137-05eb-428e-bfb1-9a32f881b9a6 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.156185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a717137-05eb-428e-bfb1-9a32f881b9a6 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:43.157223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281e45c-a956-4500-a9b1-6c76be723c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:43.157844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281e45c-a956-4500-a9b1-6c76be723c55 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.158114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a717137-05eb-428e-bfb1-9a32f881b9a6 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:43.158254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281e45c-a956-4500-a9b1-6c76be723c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.158397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281e45c-a956-4500-a9b1-6c76be723c55 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.159036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281e45c-a956-4500-a9b1-6c76be723c55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:43.160977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281e45c-a956-4500-a9b1-6c76be723c55 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:43.176227 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3738/7385] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:16:43 2022] POST /identity/v3/roles => generated 240 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:43.181395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a717137-05eb-428e-bfb1-9a32f881b9a6 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.190519 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3648/7386] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:43 2022] POST /identity/v3/users => generated 302 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:43.194024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a717137-05eb-428e-bfb1-9a32f881b9a6 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.194623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281e45c-a956-4500-a9b1-6c76be723c55 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.199228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aab5e4-4121-4517-83a1-601c92a9eed5 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54a52f33e4024fe1bf0683fecde33717', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bfe1e37d86049b489c211838ea1cf2a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:43.200535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aab5e4-4121-4517-83a1-601c92a9eed5 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.200535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aab5e4-4121-4517-83a1-601c92a9eed5 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.200535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aab5e4-4121-4517-83a1-601c92a9eed5 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] PATH_INFO: `/v3/projects/88d01939286b48788929cb691584cd08/users/1fc4f490eb164f79aeacaca196bb0a91/roles/b29edeabd2b34017b6ed9d03729a7b69` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.202830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aab5e4-4121-4517-83a1-601c92a9eed5 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] RBAC: Authorizing `identity:create_grant(project_id=88d01939286b48788929cb691584cd08, user_id=1fc4f490eb164f79aeacaca196bb0a91, role_id=b29edeabd2b34017b6ed9d03729a7b69)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:43.205631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aab5e4-4121-4517-83a1-601c92a9eed5 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:43.209107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281e45c-a956-4500-a9b1-6c76be723c55 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.212421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a74130-5306-4ba0-a523-c5d7f1449d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:43.213073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a74130-5306-4ba0-a523-c5d7f1449d27 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.213359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a74130-5306-4ba0-a523-c5d7f1449d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.213637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a74130-5306-4ba0-a523-c5d7f1449d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.214339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a74130-5306-4ba0-a523-c5d7f1449d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:43.215911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a74130-5306-4ba0-a523-c5d7f1449d27 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:43.221039 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3739/7387] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:43 2022] PUT /identity/v3/projects/88d01939286b48788929cb691584cd08/users/1fc4f490eb164f79aeacaca196bb0a91/roles/b29edeabd2b34017b6ed9d03729a7b69 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:43.225585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aab5e4-4121-4517-83a1-601c92a9eed5 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.231621 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3649/7388] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:43 2022] GET /identity/v3/roles => generated 1651 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:43.235585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a74130-5306-4ba0-a523-c5d7f1449d27 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.238887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:43.246585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a74130-5306-4ba0-a523-c5d7f1449d27 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.304489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:43.306791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aab5e4-4121-4517-83a1-601c92a9eed5 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.311993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a178118-31e7-4318-b8cc-24f8965ff302 tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc4f490eb164f79aeacaca196bb0a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '88d01939286b48788929cb691584cd08', 'project_domain_id': 'default', 'roles': ['tempest-test_role-1348404582', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:43.313228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a178118-31e7-4318-b8cc-24f8965ff302 tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.313348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a178118-31e7-4318-b8cc-24f8965ff302 tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.313602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a178118-31e7-4318-b8cc-24f8965ff302 tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] PATH_INFO: `/v3/users/1fc4f490eb164f79aeacaca196bb0a91/application_credentials` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.314370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a178118-31e7-4318-b8cc-24f8965ff302 tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] RBAC: Authorizing `identity:create_application_credential(user_id=1fc4f490eb164f79aeacaca196bb0a91)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:43.315648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a178118-31e7-4318-b8cc-24f8965ff302 tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:43.324026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58496fd-ad71-40f0-8788-29bb396ed390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:43.324589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58496fd-ad71-40f0-8788-29bb396ed390 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.328219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58496fd-ad71-40f0-8788-29bb396ed390 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.328219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58496fd-ad71-40f0-8788-29bb396ed390 admin admin] PATH_INFO: `/v3/projects/4c8013e1e6804708b2991d2dcd829154/users/01ba5e445af242a69db93175b6d44fcd/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.343629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58496fd-ad71-40f0-8788-29bb396ed390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c8013e1e6804708b2991d2dcd829154, user_id=01ba5e445af242a69db93175b6d44fcd, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:43.347337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58496fd-ad71-40f0-8788-29bb396ed390 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:43.363507 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3650/7389] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:43 2022] PUT /identity/v3/projects/4c8013e1e6804708b2991d2dcd829154/users/01ba5e445af242a69db93175b6d44fcd/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:43.367483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58496fd-ad71-40f0-8788-29bb396ed390 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.412993 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3740/7390] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Sep 3 21:16:43 2022] POST /identity/v3/users/1fc4f490eb164f79aeacaca196bb0a91/application_credentials => generated 659 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:43.418311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717d504-7243-4e6d-868b-e0889d052728 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.418603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717d504-7243-4e6d-868b-e0889d052728 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.418913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717d504-7243-4e6d-868b-e0889d052728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.442158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58496fd-ad71-40f0-8788-29bb396ed390 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.443646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8717d504-7243-4e6d-868b-e0889d052728 None None] MFA Rules not processed for user `1fc4f490eb164f79aeacaca196bb0a91`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:43.444952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8717d504-7243-4e6d-868b-e0889d052728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:43.446900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26024657-4b4f-4c5d-a242-7d2b2fda1c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:43.447503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26024657-4b4f-4c5d-a242-7d2b2fda1c0f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.447787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26024657-4b4f-4c5d-a242-7d2b2fda1c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.448068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26024657-4b4f-4c5d-a242-7d2b2fda1c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.448736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26024657-4b4f-4c5d-a242-7d2b2fda1c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:43.450366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26024657-4b4f-4c5d-a242-7d2b2fda1c0f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:43.460412 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3651/7391] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:43 2022] GET /identity/v3/roles => generated 1651 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:43.463286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26024657-4b4f-4c5d-a242-7d2b2fda1c0f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.465215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-26024657-4b4f-4c5d-a242-7d2b2fda1c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:43.478280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26024657-4b4f-4c5d-a242-7d2b2fda1c0f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.483564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:43.484189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.484477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.484756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] PATH_INFO: `/v3/users/f59d36fafb414e4682aa281690462869` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.486042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f59d36fafb414e4682aa281690462869)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:43.487976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:43.536106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f59d36fafb414e4682aa281690462869'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:43.541249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:43.550627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f59d36fafb414e4682aa281690462869'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:43.551364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f59d36fafb414e4682aa281690462869'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:43.558100 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3741/7392] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3396 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 406 bytes (2 switches on core 0) Sep 03 21:16:43.558645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717d504-7243-4e6d-868b-e0889d052728 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.561870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f59d36fafb414e4682aa281690462869'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:43.569492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717d504-7243-4e6d-868b-e0889d052728 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.569940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f59d36fafb414e4682aa281690462869'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:43.574144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a35987-09fa-47af-9e80-8bc65a595c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:43.574792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a35987-09fa-47af-9e80-8bc65a595c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.575077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a35987-09fa-47af-9e80-8bc65a595c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.575357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a35987-09fa-47af-9e80-8bc65a595c57 admin admin] PATH_INFO: `/v3/projects/4c8013e1e6804708b2991d2dcd829154/users/01ba5e445af242a69db93175b6d44fcd/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.577385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a35987-09fa-47af-9e80-8bc65a595c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c8013e1e6804708b2991d2dcd829154, user_id=01ba5e445af242a69db93175b6d44fcd, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:43.577810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3652/7393] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:43 2022] DELETE /identity/v3/users/f59d36fafb414e4682aa281690462869 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:43.579162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.580042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a35987-09fa-47af-9e80-8bc65a595c57 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:43.582111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:43.591299 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3742/7394] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:43 2022] PUT /identity/v3/projects/4c8013e1e6804708b2991d2dcd829154/users/01ba5e445af242a69db93175b6d44fcd/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:43.592506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57cc3b-69ab-4ccd-8d70-d0aadc00565f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.592799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57cc3b-69ab-4ccd-8d70-d0aadc00565f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.593113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57cc3b-69ab-4ccd-8d70-d0aadc00565f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.612812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8a57cc3b-69ab-4ccd-8d70-d0aadc00565f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:43.614062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8a57cc3b-69ab-4ccd-8d70-d0aadc00565f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:43.632318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:43.661392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886db11f-fe58-40d2-8ec0-c99948cea85a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.673184 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3743/7395] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:43.677576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a232891-aa4f-4005-8b36-14c6731d17e0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.677852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a232891-aa4f-4005-8b36-14c6731d17e0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.678141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a232891-aa4f-4005-8b36-14c6731d17e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.706149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2a232891-aa4f-4005-8b36-14c6731d17e0 None None] MFA Rules not processed for user `01ba5e445af242a69db93175b6d44fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:43.707379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2a232891-aa4f-4005-8b36-14c6731d17e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:43.723859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec632a0-fe60-46c4-83d7-c548d93febdd tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc4f490eb164f79aeacaca196bb0a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '88d01939286b48788929cb691584cd08', 'project_domain_id': 'default', 'roles': ['tempest-test_role-1348404582', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:43.724520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec632a0-fe60-46c4-83d7-c548d93febdd tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.724812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec632a0-fe60-46c4-83d7-c548d93febdd tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.725130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec632a0-fe60-46c4-83d7-c548d93febdd tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] PATH_INFO: `/v3/users/1fc4f490eb164f79aeacaca196bb0a91/application_credentials/571bd063d9f14e578dbad66ae31621b5` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.726336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec632a0-fe60-46c4-83d7-c548d93febdd tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] RBAC: Authorizing `identity:delete_application_credential(user_id=1fc4f490eb164f79aeacaca196bb0a91, application_credential_id=571bd063d9f14e578dbad66ae31621b5)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:43.727875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec632a0-fe60-46c4-83d7-c548d93febdd tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:43.752258 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3653/7396] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Sep 3 21:16:43 2022] DELETE /identity/v3/users/1fc4f490eb164f79aeacaca196bb0a91/application_credentials/571bd063d9f14e578dbad66ae31621b5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:43.753202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec632a0-fe60-46c4-83d7-c548d93febdd tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.754988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5ec632a0-fe60-46c4-83d7-c548d93febdd tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:43.780506 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3744/7397] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:43.781582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a232891-aa4f-4005-8b36-14c6731d17e0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.783155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2a232891-aa4f-4005-8b36-14c6731d17e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:43.825980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec632a0-fe60-46c4-83d7-c548d93febdd tempest-ApplicationCredentialsV3AdminTest-1579591981 tempest-ApplicationCredentialsV3AdminTest-1579591981-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.829574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50086722-06e9-41a0-94b0-6423177c11cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:43.830181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50086722-06e9-41a0-94b0-6423177c11cd service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.830477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50086722-06e9-41a0-94b0-6423177c11cd service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.830753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50086722-06e9-41a0-94b0-6423177c11cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.840477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50086722-06e9-41a0-94b0-6423177c11cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:43.841937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50086722-06e9-41a0-94b0-6423177c11cd service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:43.849830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:43.851893 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3654/7398] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:43.852793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50086722-06e9-41a0-94b0-6423177c11cd service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.854231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-50086722-06e9-41a0-94b0-6423177c11cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:43.855678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a232891-aa4f-4005-8b36-14c6731d17e0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.862005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31af1a6-6797-48d6-9811-7bfbdc3922d1 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54a52f33e4024fe1bf0683fecde33717', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bfe1e37d86049b489c211838ea1cf2a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:43.862814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31af1a6-6797-48d6-9811-7bfbdc3922d1 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:43.863111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31af1a6-6797-48d6-9811-7bfbdc3922d1 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:43.863396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31af1a6-6797-48d6-9811-7bfbdc3922d1 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] PATH_INFO: `/v3/roles/b29edeabd2b34017b6ed9d03729a7b69` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:43.864882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31af1a6-6797-48d6-9811-7bfbdc3922d1 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b29edeabd2b34017b6ed9d03729a7b69)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:43.866811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31af1a6-6797-48d6-9811-7bfbdc3922d1 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:43.930371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e31af1a6-6797-48d6-9811-7bfbdc3922d1 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] Invalidating the token cache because role b29edeabd2b34017b6ed9d03729a7b69 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:43.930715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e31af1a6-6797-48d6-9811-7bfbdc3922d1 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:43.931414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e31af1a6-6797-48d6-9811-7bfbdc3922d1 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:43.933539 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3745/7399] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:43 2022] DELETE /identity/v3/roles/b29edeabd2b34017b6ed9d03729a7b69 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:43.938986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31af1a6-6797-48d6-9811-7bfbdc3922d1 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:43.940889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e31af1a6-6797-48d6-9811-7bfbdc3922d1 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:43.966993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50086722-06e9-41a0-94b0-6423177c11cd service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:43.973135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:44.033084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31af1a6-6797-48d6-9811-7bfbdc3922d1 tempest-ApplicationCredentialsV3AdminTest-1939262908 tempest-ApplicationCredentialsV3AdminTest-1939262908-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:44.038723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:44.039363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:44.039647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:44.039961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] PATH_INFO: `/v3/users/1fc4f490eb164f79aeacaca196bb0a91` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:44.041571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc4f490eb164f79aeacaca196bb0a91)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:44.042535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ee8d1-b337-463b-b6e2-561fd16648b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:44.043162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ee8d1-b337-463b-b6e2-561fd16648b7 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:44.043452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ee8d1-b337-463b-b6e2-561fd16648b7 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:44.043624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:44.043819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ee8d1-b337-463b-b6e2-561fd16648b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:44.046475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cc6ee8d1-b337-463b-b6e2-561fd16648b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:44.106750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc4f490eb164f79aeacaca196bb0a91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.123873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc4f490eb164f79aeacaca196bb0a91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.125194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ee8d1-b337-463b-b6e2-561fd16648b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:44.127057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ee8d1-b337-463b-b6e2-561fd16648b7 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:44.138165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc4f490eb164f79aeacaca196bb0a91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.138511 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3655/7400] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:43 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:44.142512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc4f490eb164f79aeacaca196bb0a91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.142512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc4f490eb164f79aeacaca196bb0a91'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.155665 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3746/7401] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:43 2022] DELETE /identity/v3/users/1fc4f490eb164f79aeacaca196bb0a91 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:44.160405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3d340-fcf3-4660-86e4-f1f962a4e33a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:44.160700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3d340-fcf3-4660-86e4-f1f962a4e33a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:44.161023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3d340-fcf3-4660-86e4-f1f962a4e33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:44.164617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:44.167458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:44.186333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-09f3d340-fcf3-4660-86e4-f1f962a4e33a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:44.187805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-09f3d340-fcf3-4660-86e4-f1f962a4e33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:44.228821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:44.255825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f102d-401c-40eb-a64b-3042571d8ba9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:44.261588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:44.262221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:44.262508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:44.262790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] PATH_INFO: `/v3/projects/e9fcc50f4f7543ecb59858bac371e9c6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:44.263937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9fcc50f4f7543ecb59858bac371e9c6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:44.266035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:44.266645 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3656/7402] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:44.278013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3d340-fcf3-4660-86e4-f1f962a4e33a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:44.279583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-09f3d340-fcf3-4660-86e4-f1f962a4e33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:44.322857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9fcc50f4f7543ecb59858bac371e9c6', 'request_id': 'req-e97d2740-2824-4c55-a180-40c26ae0201e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.332871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9fcc50f4f7543ecb59858bac371e9c6', 'request_id': 'req-e97d2740-2824-4c55-a180-40c26ae0201e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.340816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9fcc50f4f7543ecb59858bac371e9c6', 'request_id': 'req-e97d2740-2824-4c55-a180-40c26ae0201e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.343396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:44.353955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3d340-fcf3-4660-86e4-f1f962a4e33a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:44.355743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] The token cache is being invalidate because project e9fcc50f4f7543ecb59858bac371e9c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:44.356155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.356982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.358748 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3747/7403] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:44 2022] DELETE /identity/v3/projects/e9fcc50f4f7543ecb59858bac371e9c6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:44.362353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:44.363758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:44.428091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a67670-6d0e-4742-8617-7f0ec5d8e44b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:44.428868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a67670-6d0e-4742-8617-7f0ec5d8e44b service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:44.429301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a67670-6d0e-4742-8617-7f0ec5d8e44b service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:44.429708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a67670-6d0e-4742-8617-7f0ec5d8e44b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:44.432837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-64a67670-6d0e-4742-8617-7f0ec5d8e44b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:44.449884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d2740-2824-4c55-a180-40c26ae0201e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:44.455826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:44.456467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:44.456753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:44.457226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] PATH_INFO: `/v3/users/86b2d0fca835428e9200f70f2dcb467f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:44.458812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b2d0fca835428e9200f70f2dcb467f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:44.460785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:44.523423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a67670-6d0e-4742-8617-7f0ec5d8e44b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:44.526121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a67670-6d0e-4742-8617-7f0ec5d8e44b service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:44.530051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b2d0fca835428e9200f70f2dcb467f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.536650 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3657/7404] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:44.543693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b2d0fca835428e9200f70f2dcb467f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.551757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b2d0fca835428e9200f70f2dcb467f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.552620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b2d0fca835428e9200f70f2dcb467f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.553331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b2d0fca835428e9200f70f2dcb467f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:44.555797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:44.570214 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3748/7405] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:44 2022] DELETE /identity/v3/users/86b2d0fca835428e9200f70f2dcb467f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:44.886483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a67670-6d0e-4742-8617-7f0ec5d8e44b service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:44.888461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-64a67670-6d0e-4742-8617-7f0ec5d8e44b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:44.919930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a67670-6d0e-4742-8617-7f0ec5d8e44b service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:44.971356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:44.972767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:45.030900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:45.031614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.031958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.032250 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] PATH_INFO: `/v3/projects/88d01939286b48788929cb691584cd08` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.037360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88d01939286b48788929cb691584cd08)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:45.041565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:45.050769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bb8c1-9d08-42dd-9365-6d178de9b70b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:45.056087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e84b3-6fcf-4a85-a022-59dfbac33570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:45.056828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e84b3-6fcf-4a85-a022-59dfbac33570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.057151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e84b3-6fcf-4a85-a022-59dfbac33570 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.057432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e84b3-6fcf-4a85-a022-59dfbac33570 admin admin] PATH_INFO: `/v3/projects/ff6e50afb3774ad0942be133f24a4af2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.058656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e84b3-6fcf-4a85-a022-59dfbac33570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff6e50afb3774ad0942be133f24a4af2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:45.060587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e84b3-6fcf-4a85-a022-59dfbac33570 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:45.111109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88d01939286b48788929cb691584cd08', 'request_id': 'req-477d507a-bb91-42bf-b752-4c9dbd6a5b22'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.116076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:45.119843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88d01939286b48788929cb691584cd08', 'request_id': 'req-477d507a-bb91-42bf-b752-4c9dbd6a5b22'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.128811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-379e84b3-6fcf-4a85-a022-59dfbac33570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff6e50afb3774ad0942be133f24a4af2', 'request_id': 'req-379e84b3-6fcf-4a85-a022-59dfbac33570'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.136736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88d01939286b48788929cb691584cd08', 'request_id': 'req-477d507a-bb91-42bf-b752-4c9dbd6a5b22'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.137685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-379e84b3-6fcf-4a85-a022-59dfbac33570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff6e50afb3774ad0942be133f24a4af2', 'request_id': 'req-379e84b3-6fcf-4a85-a022-59dfbac33570'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.145552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-379e84b3-6fcf-4a85-a022-59dfbac33570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff6e50afb3774ad0942be133f24a4af2', 'request_id': 'req-379e84b3-6fcf-4a85-a022-59dfbac33570'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.149225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] The token cache is being invalidate because project 88d01939286b48788929cb691584cd08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:45.150144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.150960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.152982 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3658/7406] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:44 2022] DELETE /identity/v3/projects/88d01939286b48788929cb691584cd08 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:45.156733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:45.158438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:45.158893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-379e84b3-6fcf-4a85-a022-59dfbac33570 admin admin] The token cache is being invalidate because project ff6e50afb3774ad0942be133f24a4af2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:45.159321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-379e84b3-6fcf-4a85-a022-59dfbac33570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.159986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-379e84b3-6fcf-4a85-a022-59dfbac33570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.161524 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3749/7407] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:44 2022] DELETE /identity/v3/projects/ff6e50afb3774ad0942be133f24a4af2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:45.166887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36f408-0496-4707-a275-94c66948d23b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.167183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36f408-0496-4707-a275-94c66948d23b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.167449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36f408-0496-4707-a275-94c66948d23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.192378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9d36f408-0496-4707-a275-94c66948d23b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:45.192378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9d36f408-0496-4707-a275-94c66948d23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:45.241004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d507a-bb91-42bf-b752-4c9dbd6a5b22 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:45.267275 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3750/7408] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:45.276119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17579a0-9f77-4b53-bf88-5ab582d20839 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.276422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17579a0-9f77-4b53-bf88-5ab582d20839 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.276698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17579a0-9f77-4b53-bf88-5ab582d20839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.297730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e17579a0-9f77-4b53-bf88-5ab582d20839 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:45.298960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e17579a0-9f77-4b53-bf88-5ab582d20839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:45.310642 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3751/7409] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:45.315061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:45.315218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17579a0-9f77-4b53-bf88-5ab582d20839 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:45.315717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.315997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.316801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] PATH_INFO: `/v3/users/54a52f33e4024fe1bf0683fecde33717` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.318019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a52f33e4024fe1bf0683fecde33717)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:45.320189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:45.326432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17579a0-9f77-4b53-bf88-5ab582d20839 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:45.332434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b122475-cf4b-4fa9-817f-c7af807a3379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:45.333151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b122475-cf4b-4fa9-817f-c7af807a3379 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.333496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b122475-cf4b-4fa9-817f-c7af807a3379 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.334212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b122475-cf4b-4fa9-817f-c7af807a3379 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.334608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b122475-cf4b-4fa9-817f-c7af807a3379 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:16:45.335014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b122475-cf4b-4fa9-817f-c7af807a3379 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:45.336619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b122475-cf4b-4fa9-817f-c7af807a3379 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:45.352262 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3752/7410] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:16:45.357310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b122475-cf4b-4fa9-817f-c7af807a3379 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:45.364428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:45.370348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b122475-cf4b-4fa9-817f-c7af807a3379 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:45.377466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae99b51-690e-4cd2-ab71-65367934f35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:45.378640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae99b51-690e-4cd2-ab71-65367934f35e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.378931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae99b51-690e-4cd2-ab71-65367934f35e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.379247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae99b51-690e-4cd2-ab71-65367934f35e admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.379907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae99b51-690e-4cd2-ab71-65367934f35e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:45.382130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae99b51-690e-4cd2-ab71-65367934f35e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:45.390525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a52f33e4024fe1bf0683fecde33717'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.409605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a52f33e4024fe1bf0683fecde33717'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.410535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a52f33e4024fe1bf0683fecde33717'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.422965 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3753/7411] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:45 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:45.423158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a52f33e4024fe1bf0683fecde33717'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.424937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae99b51-690e-4cd2-ab71-65367934f35e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:45.427784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0ae99b51-690e-4cd2-ab71-65367934f35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:45.430015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a52f33e4024fe1bf0683fecde33717'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:45.436507 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3659/7412] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:45 2022] DELETE /identity/v3/users/54a52f33e4024fe1bf0683fecde33717 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:45.507110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae99b51-690e-4cd2-ab71-65367934f35e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:45.586880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13093a8-aad1-4e79-b9e6-b8d8f3445092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:45.587155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13093a8-aad1-4e79-b9e6-b8d8f3445092 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.587453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13093a8-aad1-4e79-b9e6-b8d8f3445092 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.587737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13093a8-aad1-4e79-b9e6-b8d8f3445092 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.588375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13093a8-aad1-4e79-b9e6-b8d8f3445092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:45.590504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13093a8-aad1-4e79-b9e6-b8d8f3445092 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:45.615461 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3754/7413] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:45 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:45.619955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:45.631123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7654541-2fc0-4b14-8a0d-c2a774f8d3b6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:45.637083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dabcc8-3ae5-4f40-a513-a4098ded2003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:45.637702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dabcc8-3ae5-4f40-a513-a4098ded2003 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.637992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dabcc8-3ae5-4f40-a513-a4098ded2003 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.638271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dabcc8-3ae5-4f40-a513-a4098ded2003 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.638910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dabcc8-3ae5-4f40-a513-a4098ded2003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:45.640544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dabcc8-3ae5-4f40-a513-a4098ded2003 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:45.643006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:45.653036 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3660/7414] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:45.656812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13093a8-aad1-4e79-b9e6-b8d8f3445092 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:45.667642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13093a8-aad1-4e79-b9e6-b8d8f3445092 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:45.672767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c384e7-4ef6-4c04-9a92-c01925de3b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:45.673531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c384e7-4ef6-4c04-9a92-c01925de3b9b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.673820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c384e7-4ef6-4c04-9a92-c01925de3b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.674131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c384e7-4ef6-4c04-9a92-c01925de3b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.674778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c384e7-4ef6-4c04-9a92-c01925de3b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:45.676500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c384e7-4ef6-4c04-9a92-c01925de3b9b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:45.687678 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3755/7415] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:45.691609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dabcc8-3ae5-4f40-a513-a4098ded2003 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:45.703078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dabcc8-3ae5-4f40-a513-a4098ded2003 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:45.708772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d252d5f-ad70-42f8-9e5e-bce77e1b2dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:45.709499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d252d5f-ad70-42f8-9e5e-bce77e1b2dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.710089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d252d5f-ad70-42f8-9e5e-bce77e1b2dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.710089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d252d5f-ad70-42f8-9e5e-bce77e1b2dd9 admin admin] PATH_INFO: `/v3/projects/b256cc4a0dde4506bca1b7446ccc22e9/users/71605ba9c17648b7a3e0911e68c05922/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.725672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d252d5f-ad70-42f8-9e5e-bce77e1b2dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b256cc4a0dde4506bca1b7446ccc22e9, user_id=71605ba9c17648b7a3e0911e68c05922, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:45.729089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d252d5f-ad70-42f8-9e5e-bce77e1b2dd9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:45.745499 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3661/7416] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:45 2022] PUT /identity/v3/projects/b256cc4a0dde4506bca1b7446ccc22e9/users/71605ba9c17648b7a3e0911e68c05922/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:45.757552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b7de-57f0-4c88-8820-90d024f610b7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.757873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b7de-57f0-4c88-8820-90d024f610b7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.758156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b7de-57f0-4c88-8820-90d024f610b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.788199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-38e0b7de-57f0-4c88-8820-90d024f610b7 None None] MFA Rules not processed for user `71605ba9c17648b7a3e0911e68c05922`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:45.789518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-38e0b7de-57f0-4c88-8820-90d024f610b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:45.856114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:45.870621 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3756/7417] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:45.875503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d252d5f-ad70-42f8-9e5e-bce77e1b2dd9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:45.901101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b7de-57f0-4c88-8820-90d024f610b7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:45.903039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-38e0b7de-57f0-4c88-8820-90d024f610b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:45.955318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d252d5f-ad70-42f8-9e5e-bce77e1b2dd9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:45.960827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ff525-1f1c-442b-86e7-6e75faf887b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:45.961482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ff525-1f1c-442b-86e7-6e75faf887b7 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.961794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ff525-1f1c-442b-86e7-6e75faf887b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.962079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ff525-1f1c-442b-86e7-6e75faf887b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.962722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ff525-1f1c-442b-86e7-6e75faf887b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:45.964977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ff525-1f1c-442b-86e7-6e75faf887b7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:45.991142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b7de-57f0-4c88-8820-90d024f610b7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:45.996963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:45.997612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:45.997901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:45.998192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348 admin admin] PATH_INFO: `/v3/projects/4bfe1e37d86049b489c211838ea1cf2a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:45.999366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bfe1e37d86049b489c211838ea1cf2a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.000447 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3662/7418] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:45 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:46.001349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.001459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ff525-1f1c-442b-86e7-6e75faf887b7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:46.062224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bfe1e37d86049b489c211838ea1cf2a', 'request_id': 'req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:46.079421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bfe1e37d86049b489c211838ea1cf2a', 'request_id': 'req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:46.085388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ff525-1f1c-442b-86e7-6e75faf887b7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:46.087421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bfe1e37d86049b489c211838ea1cf2a', 'request_id': 'req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:46.103440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348 admin admin] The token cache is being invalidate because project 4bfe1e37d86049b489c211838ea1cf2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:46.104059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:46.105550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a113f7d-0a8b-4cd1-807e-5e57c72a1348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:46.107340 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3757/7419] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:45 2022] DELETE /identity/v3/projects/4bfe1e37d86049b489c211838ea1cf2a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:46.112462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ba171-a6f9-4789-89da-0a29389aa35e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.112840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ba171-a6f9-4789-89da-0a29389aa35e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.113223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ba171-a6f9-4789-89da-0a29389aa35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.144952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-643ba171-a6f9-4789-89da-0a29389aa35e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:46.146206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-643ba171-a6f9-4789-89da-0a29389aa35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:46.157682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1433d5-4b15-4c8e-b980-51d40a5d14aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:46.158391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1433d5-4b15-4c8e-b980-51d40a5d14aa admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.158770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1433d5-4b15-4c8e-b980-51d40a5d14aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.159124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1433d5-4b15-4c8e-b980-51d40a5d14aa admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.159842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1433d5-4b15-4c8e-b980-51d40a5d14aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.161906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1433d5-4b15-4c8e-b980-51d40a5d14aa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.175265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:46.189688 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3663/7420] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:46 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:46.193488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1433d5-4b15-4c8e-b980-51d40a5d14aa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:46.195121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-be1433d5-4b15-4c8e-b980-51d40a5d14aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:46.210015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1433d5-4b15-4c8e-b980-51d40a5d14aa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:46.215988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1b126-6cc4-4644-b5aa-87625febcfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:46.216969 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3758/7421] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:46.217090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1b126-6cc4-4644-b5aa-87625febcfdd admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.217388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1b126-6cc4-4644-b5aa-87625febcfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.217882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1b126-6cc4-4644-b5aa-87625febcfdd admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.218710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1b126-6cc4-4644-b5aa-87625febcfdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.220351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1b126-6cc4-4644-b5aa-87625febcfdd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.234426 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3664/7422] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:46.239125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ba171-a6f9-4789-89da-0a29389aa35e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:46.250457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ba171-a6f9-4789-89da-0a29389aa35e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:46.255692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2ad47-7052-48bb-9fc8-8754f9d1257d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:46.256393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2ad47-7052-48bb-9fc8-8754f9d1257d admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.256679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2ad47-7052-48bb-9fc8-8754f9d1257d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.257067 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2ad47-7052-48bb-9fc8-8754f9d1257d admin admin] PATH_INFO: `/v3/projects/c8aad9cc05674b1d908416d147eccfe7/users/6bd8c745293c42f9900f59d4e72e0d15/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.275605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2ad47-7052-48bb-9fc8-8754f9d1257d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8aad9cc05674b1d908416d147eccfe7, user_id=6bd8c745293c42f9900f59d4e72e0d15, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.278478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2ad47-7052-48bb-9fc8-8754f9d1257d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.298306 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3759/7423] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:46 2022] PUT /identity/v3/projects/c8aad9cc05674b1d908416d147eccfe7/users/6bd8c745293c42f9900f59d4e72e0d15/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:46.299694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1b126-6cc4-4644-b5aa-87625febcfdd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:46.307264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c4ca7-4831-46fa-87b3-303965d53e76 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.307561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c4ca7-4831-46fa-87b3-303965d53e76 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.307845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c4ca7-4831-46fa-87b3-303965d53e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.311980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1b126-6cc4-4644-b5aa-87625febcfdd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:46.315810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:46.330869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-508c4ca7-4831-46fa-87b3-303965d53e76 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:46.332751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-508c4ca7-4831-46fa-87b3-303965d53e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:46.383550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850d398-e0d6-4f43-b746-c1e4a681c1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:46.384579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850d398-e0d6-4f43-b746-c1e4a681c1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.385112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850d398-e0d6-4f43-b746-c1e4a681c1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.385458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850d398-e0d6-4f43-b746-c1e4a681c1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.386249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850d398-e0d6-4f43-b746-c1e4a681c1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.388170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850d398-e0d6-4f43-b746-c1e4a681c1f1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.401120 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3665/7424] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:46.401894 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3760/7425] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:46.408450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c4ca7-4831-46fa-87b3-303965d53e76 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:46.408648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850d398-e0d6-4f43-b746-c1e4a681c1f1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:46.420505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c4ca7-4831-46fa-87b3-303965d53e76 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:46.420737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850d398-e0d6-4f43-b746-c1e4a681c1f1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:46.425467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa2264-c6b9-4c2d-b20b-458e457ea47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:46.426123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa2264-c6b9-4c2d-b20b-458e457ea47e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.426236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b2c2b-715a-48bd-a5c2-17be3d0cb58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:46.426446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa2264-c6b9-4c2d-b20b-458e457ea47e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.426680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa2264-c6b9-4c2d-b20b-458e457ea47e admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.426867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b2c2b-715a-48bd-a5c2-17be3d0cb58f admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.427159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b2c2b-715a-48bd-a5c2-17be3d0cb58f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.427330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa2264-c6b9-4c2d-b20b-458e457ea47e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:16:46.427437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b2c2b-715a-48bd-a5c2-17be3d0cb58f admin admin] PATH_INFO: `/v3/projects/c8aad9cc05674b1d908416d147eccfe7/users/6bd8c745293c42f9900f59d4e72e0d15/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.427629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa2264-c6b9-4c2d-b20b-458e457ea47e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.429279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa2264-c6b9-4c2d-b20b-458e457ea47e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.429566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b2c2b-715a-48bd-a5c2-17be3d0cb58f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8aad9cc05674b1d908416d147eccfe7, user_id=6bd8c745293c42f9900f59d4e72e0d15, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.432510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b2c2b-715a-48bd-a5c2-17be3d0cb58f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.443899 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3761/7426] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:46 2022] PUT /identity/v3/projects/c8aad9cc05674b1d908416d147eccfe7/users/6bd8c745293c42f9900f59d4e72e0d15/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:46.444396 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3666/7427] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:46.448761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b2c2b-715a-48bd-a5c2-17be3d0cb58f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:46.454485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf4d2d-d420-4bbc-b8d7-18fffdd7173c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.454781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf4d2d-d420-4bbc-b8d7-18fffdd7173c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.455063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf4d2d-d420-4bbc-b8d7-18fffdd7173c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.487813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-96cf4d2d-d420-4bbc-b8d7-18fffdd7173c None None] MFA Rules not processed for user `6bd8c745293c42f9900f59d4e72e0d15`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:46.489145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-96cf4d2d-d420-4bbc-b8d7-18fffdd7173c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:46.527333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b2c2b-715a-48bd-a5c2-17be3d0cb58f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:46.532202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac107932-4839-481d-bf97-76a597db2e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:46.532844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac107932-4839-481d-bf97-76a597db2e53 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.533157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac107932-4839-481d-bf97-76a597db2e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.533569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac107932-4839-481d-bf97-76a597db2e53 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.534237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac107932-4839-481d-bf97-76a597db2e53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.536160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac107932-4839-481d-bf97-76a597db2e53 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.564940 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3762/7428] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:46 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:46.568797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac107932-4839-481d-bf97-76a597db2e53 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:46.573043 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3667/7429] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:46.581386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf4d2d-d420-4bbc-b8d7-18fffdd7173c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:46.583175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-96cf4d2d-d420-4bbc-b8d7-18fffdd7173c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:46.643646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac107932-4839-481d-bf97-76a597db2e53 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:46.648452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18db8f-fb96-4fb3-9fa8-5c130db9aba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:46.649967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18db8f-fb96-4fb3-9fa8-5c130db9aba8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.650409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18db8f-fb96-4fb3-9fa8-5c130db9aba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.650693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18db8f-fb96-4fb3-9fa8-5c130db9aba8 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.651331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18db8f-fb96-4fb3-9fa8-5c130db9aba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.655630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18db8f-fb96-4fb3-9fa8-5c130db9aba8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.658792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf4d2d-d420-4bbc-b8d7-18fffdd7173c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:46.663872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:46.681247 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3763/7430] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:46 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:46.685096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18db8f-fb96-4fb3-9fa8-5c130db9aba8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:46.692605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:46.697278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18db8f-fb96-4fb3-9fa8-5c130db9aba8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:46.704841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b60e7b-769b-49f0-b1c9-657d35e1ec00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:46.706129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b60e7b-769b-49f0-b1c9-657d35e1ec00 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.706129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b60e7b-769b-49f0-b1c9-657d35e1ec00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.706344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b60e7b-769b-49f0-b1c9-657d35e1ec00 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.706809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b60e7b-769b-49f0-b1c9-657d35e1ec00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.708435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b60e7b-769b-49f0-b1c9-657d35e1ec00 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.720523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8da98a-ef2a-44e2-a5ec-a48c60b42197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:46.720857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da98a-ef2a-44e2-a5ec-a48c60b42197 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.721174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da98a-ef2a-44e2-a5ec-a48c60b42197 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.721509 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3764/7431] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:46.724319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da98a-ef2a-44e2-a5ec-a48c60b42197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.725124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b60e7b-769b-49f0-b1c9-657d35e1ec00 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:46.735710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b60e7b-769b-49f0-b1c9-657d35e1ec00 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:46.740494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092e42c-3ba4-4899-af4f-df0f578d36a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:46.741137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092e42c-3ba4-4899-af4f-df0f578d36a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.741428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092e42c-3ba4-4899-af4f-df0f578d36a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.741709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092e42c-3ba4-4899-af4f-df0f578d36a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.742360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092e42c-3ba4-4899-af4f-df0f578d36a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.744078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092e42c-3ba4-4899-af4f-df0f578d36a1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.757749 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3765/7432] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:46.760156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092e42c-3ba4-4899-af4f-df0f578d36a1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:46.774015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092e42c-3ba4-4899-af4f-df0f578d36a1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:46.782560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c6446-bf87-4158-8110-21ecf7619575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:46.783417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c6446-bf87-4158-8110-21ecf7619575 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.783962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c6446-bf87-4158-8110-21ecf7619575 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.784445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c6446-bf87-4158-8110-21ecf7619575 admin admin] PATH_INFO: `/v3/projects/571f9f0354e54ee8bfc620031a8d4209/users/99982492c2444c6e944aae9198d9b4e6/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.802515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8da98a-ef2a-44e2-a5ec-a48c60b42197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.803729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c6446-bf87-4158-8110-21ecf7619575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=571f9f0354e54ee8bfc620031a8d4209, user_id=99982492c2444c6e944aae9198d9b4e6, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:46.804058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8da98a-ef2a-44e2-a5ec-a48c60b42197 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.807100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c6446-bf87-4158-8110-21ecf7619575 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:46.824852 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3668/7433] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:16:46.829244 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3766/7434] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:46 2022] PUT /identity/v3/projects/571f9f0354e54ee8bfc620031a8d4209/users/99982492c2444c6e944aae9198d9b4e6/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:46.839824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb781d-ff40-4a67-9c79-c1c4621dec44 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:46.840122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb781d-ff40-4a67-9c79-c1c4621dec44 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:46.840425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb781d-ff40-4a67-9c79-c1c4621dec44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:46.876491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-aecb781d-ff40-4a67-9c79-c1c4621dec44 None None] MFA Rules not processed for user `99982492c2444c6e944aae9198d9b4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:46.878397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-aecb781d-ff40-4a67-9c79-c1c4621dec44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:46.964323 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3669/7435] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:46.968754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c6446-bf87-4158-8110-21ecf7619575 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:47.050192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c6446-bf87-4158-8110-21ecf7619575 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:47.057464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b375d-4baa-42d3-8ff0-468525ebe6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:47.058082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b375d-4baa-42d3-8ff0-468525ebe6dc admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:47.061073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b375d-4baa-42d3-8ff0-468525ebe6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:47.061373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b375d-4baa-42d3-8ff0-468525ebe6dc admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:47.062038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b375d-4baa-42d3-8ff0-468525ebe6dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:47.064048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b375d-4baa-42d3-8ff0-468525ebe6dc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:47.099908 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3767/7436] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:46 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:47.104233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb781d-ff40-4a67-9c79-c1c4621dec44 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:47.124456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb781d-ff40-4a67-9c79-c1c4621dec44 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:47.195879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:47.230961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072dafaa-905f-4902-ad4c-8977cfda1f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:47.231686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072dafaa-905f-4902-ad4c-8977cfda1f42 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:47.231983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072dafaa-905f-4902-ad4c-8977cfda1f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:47.232267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072dafaa-905f-4902-ad4c-8977cfda1f42 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:47.232958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072dafaa-905f-4902-ad4c-8977cfda1f42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:47.234909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072dafaa-905f-4902-ad4c-8977cfda1f42 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:47.263530 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3670/7437] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:47 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:47.267536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b375d-4baa-42d3-8ff0-468525ebe6dc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:47.286903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b375d-4baa-42d3-8ff0-468525ebe6dc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:47.293442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658e956-09c4-4683-9a6b-23f0b990ad71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:47.294443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658e956-09c4-4683-9a6b-23f0b990ad71 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:47.294990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658e956-09c4-4683-9a6b-23f0b990ad71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:47.295352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658e956-09c4-4683-9a6b-23f0b990ad71 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:47.296236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7658e956-09c4-4683-9a6b-23f0b990ad71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:47.298527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7658e956-09c4-4683-9a6b-23f0b990ad71 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:47.312319 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3768/7438] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:47.316958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072dafaa-905f-4902-ad4c-8977cfda1f42 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:47.329453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072dafaa-905f-4902-ad4c-8977cfda1f42 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:47.334766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3df991-29af-4701-b2e3-5febd0e6435c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:47.335554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3df991-29af-4701-b2e3-5febd0e6435c admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:47.335756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3df991-29af-4701-b2e3-5febd0e6435c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:47.336038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3df991-29af-4701-b2e3-5febd0e6435c admin admin] PATH_INFO: `/v3/projects/2aa6b5734a214c1480de474d8008c59c/users/ef4b2059baf6449ca8538cca89151667/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:47.358201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3df991-29af-4701-b2e3-5febd0e6435c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa6b5734a214c1480de474d8008c59c, user_id=ef4b2059baf6449ca8538cca89151667, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:47.360865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3df991-29af-4701-b2e3-5febd0e6435c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:47.384396 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3671/7439] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:47 2022] PUT /identity/v3/projects/2aa6b5734a214c1480de474d8008c59c/users/ef4b2059baf6449ca8538cca89151667/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:47.388494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658e956-09c4-4683-9a6b-23f0b990ad71 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:47.400584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658e956-09c4-4683-9a6b-23f0b990ad71 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:47.470669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a158f2-e144-435f-b8b3-9f2f2b3971c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:47.471326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a158f2-e144-435f-b8b3-9f2f2b3971c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:47.471608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a158f2-e144-435f-b8b3-9f2f2b3971c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:47.471890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a158f2-e144-435f-b8b3-9f2f2b3971c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:47.472542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a158f2-e144-435f-b8b3-9f2f2b3971c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:47.474524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a158f2-e144-435f-b8b3-9f2f2b3971c4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:47.490641 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3769/7440] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:47.495357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3df991-29af-4701-b2e3-5febd0e6435c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:47.506818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3df991-29af-4701-b2e3-5febd0e6435c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:47.512704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ebd09-2dd7-44c5-8091-32a299260690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:47.513603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ebd09-2dd7-44c5-8091-32a299260690 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:47.513971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ebd09-2dd7-44c5-8091-32a299260690 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:47.514514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ebd09-2dd7-44c5-8091-32a299260690 admin admin] PATH_INFO: `/v3/projects/2aa6b5734a214c1480de474d8008c59c/users/ef4b2059baf6449ca8538cca89151667/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:47.521356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ebd09-2dd7-44c5-8091-32a299260690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa6b5734a214c1480de474d8008c59c, user_id=ef4b2059baf6449ca8538cca89151667, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:47.521356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ebd09-2dd7-44c5-8091-32a299260690 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:47.547514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75fe05-6a4b-4976-b596-73d113aca611 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:47.547852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75fe05-6a4b-4976-b596-73d113aca611 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:47.548169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75fe05-6a4b-4976-b596-73d113aca611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:47.548969 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3672/7441] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:47 2022] PUT /identity/v3/projects/2aa6b5734a214c1480de474d8008c59c/users/ef4b2059baf6449ca8538cca89151667/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:47.585130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2f75fe05-6a4b-4976-b596-73d113aca611 None None] MFA Rules not processed for user `ef4b2059baf6449ca8538cca89151667`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:47.585932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2f75fe05-6a4b-4976-b596-73d113aca611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:47.597902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:47.674371 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3770/7442] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:47.679037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ebd09-2dd7-44c5-8091-32a299260690 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:47.694700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ebd09-2dd7-44c5-8091-32a299260690 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:47.699943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e19539-93bf-4fbd-b9e9-acf5daf90b54 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:47.700846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e19539-93bf-4fbd-b9e9-acf5daf90b54 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:47.701169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e19539-93bf-4fbd-b9e9-acf5daf90b54 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:47.701350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e19539-93bf-4fbd-b9e9-acf5daf90b54 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:47.702689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e19539-93bf-4fbd-b9e9-acf5daf90b54 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:47.704180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e19539-93bf-4fbd-b9e9-acf5daf90b54 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:47.747998 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3673/7443] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:47 2022] POST /identity/v3/projects => generated 357 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:47.752233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75fe05-6a4b-4976-b596-73d113aca611 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:47.767879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75fe05-6a4b-4976-b596-73d113aca611 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:47.831327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e19539-93bf-4fbd-b9e9-acf5daf90b54 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:47.834198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-86e19539-93bf-4fbd-b9e9-acf5daf90b54 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:47.849925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db990d-acee-42dc-8f5d-75aca399c2f3 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:47.850704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db990d-acee-42dc-8f5d-75aca399c2f3 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:47.851062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db990d-acee-42dc-8f5d-75aca399c2f3 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:47.851414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db990d-acee-42dc-8f5d-75aca399c2f3 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:47.852268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db990d-acee-42dc-8f5d-75aca399c2f3 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:47.855928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db990d-acee-42dc-8f5d-75aca399c2f3 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:47.891970 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3771/7444] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:47 2022] POST /identity/v3/users => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:47.895342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db990d-acee-42dc-8f5d-75aca399c2f3 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:47.909351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db990d-acee-42dc-8f5d-75aca399c2f3 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:47.916069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caeee5-d67b-4400-aaa6-932121df123a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:47.916773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caeee5-d67b-4400-aaa6-932121df123a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:47.917375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caeee5-d67b-4400-aaa6-932121df123a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:47.917666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caeee5-d67b-4400-aaa6-932121df123a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/6b464a301c234c45843a8df924d1ee5a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:47.936330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e19539-93bf-4fbd-b9e9-acf5daf90b54 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:47.937399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96caeee5-d67b-4400-aaa6-932121df123a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:get_user(user_id=6b464a301c234c45843a8df924d1ee5a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:47.939740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96caeee5-d67b-4400-aaa6-932121df123a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:47.941091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ada39f-66f6-4d33-8fb4-2c2c0cb89c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:47.941678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ada39f-66f6-4d33-8fb4-2c2c0cb89c31 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:47.942218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ada39f-66f6-4d33-8fb4-2c2c0cb89c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:47.942622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ada39f-66f6-4d33-8fb4-2c2c0cb89c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:47.944685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e0ada39f-66f6-4d33-8fb4-2c2c0cb89c31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:47.945055 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3772/7445] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Sep 3 21:16:47 2022] GET /identity/v3/users/6b464a301c234c45843a8df924d1ee5a => generated 373 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:47.945940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caeee5-d67b-4400-aaa6-932121df123a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:47.963864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caeee5-d67b-4400-aaa6-932121df123a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:47.970546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:47.971136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:47.971428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:47.971713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/6b464a301c234c45843a8df924d1ee5a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:47.972880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:delete_user(user_id=6b464a301c234c45843a8df924d1ee5a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:47.975006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:48.036404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b464a301c234c45843a8df924d1ee5a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.052177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b464a301c234c45843a8df924d1ee5a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.063651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ada39f-66f6-4d33-8fb4-2c2c0cb89c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:48.066019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ada39f-66f6-4d33-8fb4-2c2c0cb89c31 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:48.071743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b464a301c234c45843a8df924d1ee5a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.072793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b464a301c234c45843a8df924d1ee5a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.073834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b464a301c234c45843a8df924d1ee5a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.079803 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3674/7446] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:47 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:16:48.090933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:48.108768 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3773/7447] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:47 2022] DELETE /identity/v3/users/6b464a301c234c45843a8df924d1ee5a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:48.114580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ada39f-66f6-4d33-8fb4-2c2c0cb89c31 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:48.141352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ada39f-66f6-4d33-8fb4-2c2c0cb89c31 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:48.146041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:48.234165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:48.235639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:48.235945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:48.236363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects/5543c4f758dc480b9327d468919c3403` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:48.239178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5543c4f758dc480b9327d468919c3403)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:48.241337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:48.330228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5543c4f758dc480b9327d468919c3403', 'request_id': 'req-55402d03-8c79-4ea3-9e37-946183d31400'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.341444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5543c4f758dc480b9327d468919c3403', 'request_id': 'req-55402d03-8c79-4ea3-9e37-946183d31400'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.351409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5543c4f758dc480b9327d468919c3403', 'request_id': 'req-55402d03-8c79-4ea3-9e37-946183d31400'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.379544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] The token cache is being invalidate because project 5543c4f758dc480b9327d468919c3403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:48.379851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.390634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.394129 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3675/7448] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:48 2022] DELETE /identity/v3/projects/5543c4f758dc480b9327d468919c3403 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:48.400179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:48.405757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:48.428504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eded4e9-b643-45c4-ad2f-8a6a719ae014 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:48.504369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49c896-1d86-4679-bbeb-0a3d7c88d7b6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:48.505261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49c896-1d86-4679-bbeb-0a3d7c88d7b6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:48.505753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49c896-1d86-4679-bbeb-0a3d7c88d7b6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:48.506170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49c896-1d86-4679-bbeb-0a3d7c88d7b6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:48.506770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49c896-1d86-4679-bbeb-0a3d7c88d7b6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:48.509503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49c896-1d86-4679-bbeb-0a3d7c88d7b6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:48.558035 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3774/7449] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:48 2022] POST /identity/v3/projects => generated 357 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:48.559204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:48.561008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:48.569384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49c896-1d86-4679-bbeb-0a3d7c88d7b6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:48.573219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8c49c896-1d86-4679-bbeb-0a3d7c88d7b6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:48.578218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55402d03-8c79-4ea3-9e37-946183d31400 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:48.650969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d09264-b138-4760-b18f-4421e8fff29f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:48.651440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d09264-b138-4760-b18f-4421e8fff29f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:48.651745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d09264-b138-4760-b18f-4421e8fff29f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:48.652033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d09264-b138-4760-b18f-4421e8fff29f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:48.652720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d09264-b138-4760-b18f-4421e8fff29f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:48.655207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d09264-b138-4760-b18f-4421e8fff29f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:48.687627 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3676/7450] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:48 2022] POST /identity/v3/users => generated 417 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:48.691622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49c896-1d86-4679-bbeb-0a3d7c88d7b6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:48.692136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d09264-b138-4760-b18f-4421e8fff29f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:48.701055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4b5fa-6b31-4822-95db-6353f4fd5854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:48.701857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4b5fa-6b31-4822-95db-6353f4fd5854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:48.702158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4b5fa-6b31-4822-95db-6353f4fd5854 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:48.702456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4b5fa-6b31-4822-95db-6353f4fd5854 admin admin] PATH_INFO: `/v3/users/71605ba9c17648b7a3e0911e68c05922` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:48.707558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d09264-b138-4760-b18f-4421e8fff29f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:48.709531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4b5fa-6b31-4822-95db-6353f4fd5854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71605ba9c17648b7a3e0911e68c05922)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:48.711793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4b5fa-6b31-4822-95db-6353f4fd5854 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:48.714069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfe650-9bcd-4aa6-a5c0-1a838eba1cd8 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:48.714772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfe650-9bcd-4aa6-a5c0-1a838eba1cd8 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:48.715219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfe650-9bcd-4aa6-a5c0-1a838eba1cd8 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:48.715897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfe650-9bcd-4aa6-a5c0-1a838eba1cd8 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:48.716936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfe650-9bcd-4aa6-a5c0-1a838eba1cd8 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:48.719564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfe650-9bcd-4aa6-a5c0-1a838eba1cd8 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:48.745002 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3677/7451] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:16:48 2022] POST /identity/v3/roles => generated 240 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:48.750765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfe650-9bcd-4aa6-a5c0-1a838eba1cd8 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:48.767089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfe650-9bcd-4aa6-a5c0-1a838eba1cd8 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:48.773400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29232a-075c-4729-ae33-6c85f5e777c6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:48.774648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29232a-075c-4729-ae33-6c85f5e777c6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:48.775151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29232a-075c-4729-ae33-6c85f5e777c6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:48.775456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29232a-075c-4729-ae33-6c85f5e777c6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/384b4016baea45a187b06d044658d9e1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:48.778414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93a4b5fa-6b31-4822-95db-6353f4fd5854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71605ba9c17648b7a3e0911e68c05922'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.794943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93a4b5fa-6b31-4822-95db-6353f4fd5854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71605ba9c17648b7a3e0911e68c05922'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.800442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:48.804373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29232a-075c-4729-ae33-6c85f5e777c6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:get_user(user_id=384b4016baea45a187b06d044658d9e1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:48.808095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29232a-075c-4729-ae33-6c85f5e777c6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:48.808547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93a4b5fa-6b31-4822-95db-6353f4fd5854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71605ba9c17648b7a3e0911e68c05922'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.812303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93a4b5fa-6b31-4822-95db-6353f4fd5854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71605ba9c17648b7a3e0911e68c05922'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.813475 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3678/7452] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Sep 3 21:16:48 2022] GET /identity/v3/users/384b4016baea45a187b06d044658d9e1 => generated 417 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:48.814545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-93a4b5fa-6b31-4822-95db-6353f4fd5854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71605ba9c17648b7a3e0911e68c05922'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:48.815675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29232a-075c-4729-ae33-6c85f5e777c6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:48.819593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0d29232a-075c-4729-ae33-6c85f5e777c6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:48.828434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:48.839195 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3775/7453] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:48 2022] DELETE /identity/v3/users/71605ba9c17648b7a3e0911e68c05922 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:48.839391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29232a-075c-4729-ae33-6c85f5e777c6 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:48.841787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c28eb-7872-4e23-ba07-5ad32b955009 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:48.843964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c28eb-7872-4e23-ba07-5ad32b955009 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:48.844377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c28eb-7872-4e23-ba07-5ad32b955009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:48.869180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f24c28eb-7872-4e23-ba07-5ad32b955009 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:48.870495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f24c28eb-7872-4e23-ba07-5ad32b955009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:48.953715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb0720-089b-482c-a2cc-8c27ef760b16 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:48.959126 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3776/7454] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:48.960773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb0720-089b-482c-a2cc-8c27ef760b16 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:48.961272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb0720-089b-482c-a2cc-8c27ef760b16 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:48.961882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb0720-089b-482c-a2cc-8c27ef760b16 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/roles/8087761cbe1148a996b39260dd5425dc` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:48.964134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb0720-089b-482c-a2cc-8c27ef760b16 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:get_role(role_id=8087761cbe1148a996b39260dd5425dc)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:48.965216 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-2ddb0720-089b-482c-a2cc-8c27ef760b16 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:16:48.965216 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:16:48.965216 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:16:48.966127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb0720-089b-482c-a2cc-8c27ef760b16 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:48.967707 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3679/7455] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Sep 3 21:16:48 2022] GET /identity/v3/roles/8087761cbe1148a996b39260dd5425dc => generated 240 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:48.968694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c28eb-7872-4e23-ba07-5ad32b955009 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:48.970327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f24c28eb-7872-4e23-ba07-5ad32b955009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:48.971389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb0720-089b-482c-a2cc-8c27ef760b16 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:48.986129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb0720-089b-482c-a2cc-8c27ef760b16 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:48.991745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f4929-490d-4302-be6a-21a498bfd217 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:48.992619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f4929-490d-4302-be6a-21a498bfd217 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:48.994244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f4929-490d-4302-be6a-21a498bfd217 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:48.994716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f4929-490d-4302-be6a-21a498bfd217 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:48.995641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f4929-490d-4302-be6a-21a498bfd217 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:48.998748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f4929-490d-4302-be6a-21a498bfd217 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:49.049847 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3680/7456] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:48 2022] POST /identity/v3/projects => generated 359 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:49.050762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c28eb-7872-4e23-ba07-5ad32b955009 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:49.050973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f4929-490d-4302-be6a-21a498bfd217 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:49.133336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16519a50-723c-49fa-8aec-1b4fee7c1b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:49.133336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16519a50-723c-49fa-8aec-1b4fee7c1b5e service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:49.133336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16519a50-723c-49fa-8aec-1b4fee7c1b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:49.133336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16519a50-723c-49fa-8aec-1b4fee7c1b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:49.171937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f4929-490d-4302-be6a-21a498bfd217 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:49.177380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0131fe3-57dd-42a3-b8cd-61e3b7d93e21 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:49.178034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0131fe3-57dd-42a3-b8cd-61e3b7d93e21 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:49.178326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0131fe3-57dd-42a3-b8cd-61e3b7d93e21 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:49.178614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0131fe3-57dd-42a3-b8cd-61e3b7d93e21 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects/98b08df2b230496fa8b4d6d99861aa4f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:49.180033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0131fe3-57dd-42a3-b8cd-61e3b7d93e21 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:get_project(project_id=98b08df2b230496fa8b4d6d99861aa4f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:49.183507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0131fe3-57dd-42a3-b8cd-61e3b7d93e21 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:49.185347 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3681/7457] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:16:49 2022] GET /identity/v3/projects/98b08df2b230496fa8b4d6d99861aa4f => generated 359 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:49.190560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0131fe3-57dd-42a3-b8cd-61e3b7d93e21 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:49.204026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0131fe3-57dd-42a3-b8cd-61e3b7d93e21 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:49.213590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfe042-afd2-4b07-a267-ea5d9644226d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:49.215111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfe042-afd2-4b07-a267-ea5d9644226d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:49.215447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfe042-afd2-4b07-a267-ea5d9644226d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:49.215739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfe042-afd2-4b07-a267-ea5d9644226d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects/98b08df2b230496fa8b4d6d99861aa4f/users/384b4016baea45a187b06d044658d9e1/roles/8087761cbe1148a996b39260dd5425dc` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:49.218184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfe042-afd2-4b07-a267-ea5d9644226d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:create_grant(project_id=98b08df2b230496fa8b4d6d99861aa4f, user_id=384b4016baea45a187b06d044658d9e1, role_id=8087761cbe1148a996b39260dd5425dc)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:49.221993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfe042-afd2-4b07-a267-ea5d9644226d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:49.232131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16519a50-723c-49fa-8aec-1b4fee7c1b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:49.234691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16519a50-723c-49fa-8aec-1b4fee7c1b5e service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:49.249959 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3777/7459] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:16:49.250168 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3682/7459] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:49 2022] PUT /identity/v3/projects/98b08df2b230496fa8b4d6d99861aa4f/users/384b4016baea45a187b06d044658d9e1/roles/8087761cbe1148a996b39260dd5425dc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:49.251521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16519a50-723c-49fa-8aec-1b4fee7c1b5e service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:49.259579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:49.264968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16519a50-723c-49fa-8aec-1b4fee7c1b5e service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:49.346659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f384cd-7c40-4e0e-8c19-294fbfcb8e2d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:49.347641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f384cd-7c40-4e0e-8c19-294fbfcb8e2d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:49.348181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f384cd-7c40-4e0e-8c19-294fbfcb8e2d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:49.348568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f384cd-7c40-4e0e-8c19-294fbfcb8e2d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:49.349498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f384cd-7c40-4e0e-8c19-294fbfcb8e2d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:49.353624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f384cd-7c40-4e0e-8c19-294fbfcb8e2d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:49.401510 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3778/7460] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:49 2022] POST /identity/v3/projects => generated 359 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:49.403510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfe042-afd2-4b07-a267-ea5d9644226d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:49.418675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfe042-afd2-4b07-a267-ea5d9644226d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:49.489128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be58986-33d7-41fd-829c-6bf615856277 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:49.490130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be58986-33d7-41fd-829c-6bf615856277 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:49.490407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be58986-33d7-41fd-829c-6bf615856277 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:49.490836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be58986-33d7-41fd-829c-6bf615856277 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects/8880221538484f54b1786afb7a2266c7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:49.492219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be58986-33d7-41fd-829c-6bf615856277 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:get_project(project_id=8880221538484f54b1786afb7a2266c7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:49.494849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be58986-33d7-41fd-829c-6bf615856277 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:49.500314 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3683/7461] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:16:49 2022] GET /identity/v3/projects/8880221538484f54b1786afb7a2266c7 => generated 359 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:49.502163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f384cd-7c40-4e0e-8c19-294fbfcb8e2d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:49.525427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f384cd-7c40-4e0e-8c19-294fbfcb8e2d tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:49.532023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3a248-c121-4eb1-9c21-95ef7c5d6265 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:49.532325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3a248-c121-4eb1-9c21-95ef7c5d6265 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:49.533226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3a248-c121-4eb1-9c21-95ef7c5d6265 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:49.533525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3a248-c121-4eb1-9c21-95ef7c5d6265 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects/8880221538484f54b1786afb7a2266c7/users/384b4016baea45a187b06d044658d9e1/roles/8087761cbe1148a996b39260dd5425dc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:49.537267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3a248-c121-4eb1-9c21-95ef7c5d6265 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:create_grant(project_id=8880221538484f54b1786afb7a2266c7, user_id=384b4016baea45a187b06d044658d9e1, role_id=8087761cbe1148a996b39260dd5425dc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:49.542935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3a248-c121-4eb1-9c21-95ef7c5d6265 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:49.562054 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3779/7462] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:49 2022] PUT /identity/v3/projects/8880221538484f54b1786afb7a2266c7/users/384b4016baea45a187b06d044658d9e1/roles/8087761cbe1148a996b39260dd5425dc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:49.565701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be58986-33d7-41fd-829c-6bf615856277 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:49.576998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be58986-33d7-41fd-829c-6bf615856277 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:49.587731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:49.636350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3a248-c121-4eb1-9c21-95ef7c5d6265 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:49.638867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f4b3a248-c121-4eb1-9c21-95ef7c5d6265 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:49.670038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb953922-b3c1-4e92-99ec-fff1650a7734 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:49.670776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb953922-b3c1-4e92-99ec-fff1650a7734 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:49.671301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb953922-b3c1-4e92-99ec-fff1650a7734 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:49.671600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb953922-b3c1-4e92-99ec-fff1650a7734 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/384b4016baea45a187b06d044658d9e1/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:49.673404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb953922-b3c1-4e92-99ec-fff1650a7734 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:list_user_projects(user_id=384b4016baea45a187b06d044658d9e1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:49.676125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb953922-b3c1-4e92-99ec-fff1650a7734 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:49.735918 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3684/7463] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Sep 3 21:16:49 2022] GET /identity/v3/users/384b4016baea45a187b06d044658d9e1/projects => generated 842 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:49.737092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3a248-c121-4eb1-9c21-95ef7c5d6265 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:49.740338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb953922-b3c1-4e92-99ec-fff1650a7734 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:49.749466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:49.750190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:49.750485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:49.750766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] PATH_INFO: `/v3/projects/b256cc4a0dde4506bca1b7446ccc22e9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:49.753014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b256cc4a0dde4506bca1b7446ccc22e9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:49.755098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:49.756856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb953922-b3c1-4e92-99ec-fff1650a7734 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:49.766623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:49.767510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:49.767941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:49.768324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects/8880221538484f54b1786afb7a2266c7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:49.769850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8880221538484f54b1786afb7a2266c7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:49.776992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:49.822247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b256cc4a0dde4506bca1b7446ccc22e9', 'request_id': 'req-18beca26-fbc3-455f-a10d-9269dda9b392'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:49.834140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b256cc4a0dde4506bca1b7446ccc22e9', 'request_id': 'req-18beca26-fbc3-455f-a10d-9269dda9b392'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:49.844781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b256cc4a0dde4506bca1b7446ccc22e9', 'request_id': 'req-18beca26-fbc3-455f-a10d-9269dda9b392'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:49.859552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] The token cache is being invalidate because project b256cc4a0dde4506bca1b7446ccc22e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:49.860663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:49.861603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:49.863688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8880221538484f54b1786afb7a2266c7', 'request_id': 'req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:49.865336 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3780/7464] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:49 2022] DELETE /identity/v3/projects/b256cc4a0dde4506bca1b7446ccc22e9 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:49.869770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:49.871944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:49.882392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8880221538484f54b1786afb7a2266c7', 'request_id': 'req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:49.889299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:49.892997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8880221538484f54b1786afb7a2266c7', 'request_id': 'req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:49.917542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] The token cache is being invalidate because project 8880221538484f54b1786afb7a2266c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:49.918088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:49.920717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:49.925736 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3685/7465] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:49 2022] DELETE /identity/v3/projects/8880221538484f54b1786afb7a2266c7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:49.927039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:49.928942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:49.975034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18beca26-fbc3-455f-a10d-9269dda9b392 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:50.021547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3cf08-e2b3-489d-af95-cbd7a8a3831a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:50.027370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:50.028317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:50.028675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:50.029070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects/98b08df2b230496fa8b4d6d99861aa4f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:50.030671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:delete_project(project_id=98b08df2b230496fa8b4d6d99861aa4f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:50.032773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:50.041485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:50.094240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:50.095404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:50.095846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:50.096275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] PATH_INFO: `/v3/users/6bd8c745293c42f9900f59d4e72e0d15` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:50.100987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bd8c745293c42f9900f59d4e72e0d15)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:50.103933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:50.124960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b08df2b230496fa8b4d6d99861aa4f', 'request_id': 'req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.135893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b08df2b230496fa8b4d6d99861aa4f', 'request_id': 'req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.148522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b08df2b230496fa8b4d6d99861aa4f', 'request_id': 'req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.166108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] The token cache is being invalidate because project 98b08df2b230496fa8b4d6d99861aa4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:50.166384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.168932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.171055 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3686/7466] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:49 2022] DELETE /identity/v3/projects/98b08df2b230496fa8b4d6d99861aa4f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:50.176710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:50.178930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:50.218301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bd8c745293c42f9900f59d4e72e0d15'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.252518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bd8c745293c42f9900f59d4e72e0d15'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.266929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bd8c745293c42f9900f59d4e72e0d15'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.267952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bd8c745293c42f9900f59d4e72e0d15'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.269607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bd8c745293c42f9900f59d4e72e0d15'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.295124 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3781/7467] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:49 2022] DELETE /identity/v3/users/6bd8c745293c42f9900f59d4e72e0d15 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:50.318044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fbdc7-04e4-42df-bb7f-6db306c575a7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:50.480667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f12b51-f25d-4adb-9616-9d99a3bb5542 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:50.481848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f12b51-f25d-4adb-9616-9d99a3bb5542 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:50.482253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f12b51-f25d-4adb-9616-9d99a3bb5542 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:50.482534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f12b51-f25d-4adb-9616-9d99a3bb5542 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/roles/8087761cbe1148a996b39260dd5425dc` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:50.487465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f12b51-f25d-4adb-9616-9d99a3bb5542 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:delete_role(role_id=8087761cbe1148a996b39260dd5425dc)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:50.495898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f12b51-f25d-4adb-9616-9d99a3bb5542 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:50.566739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-44f12b51-f25d-4adb-9616-9d99a3bb5542 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invalidating the token cache because role 8087761cbe1148a996b39260dd5425dc has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:50.567865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-44f12b51-f25d-4adb-9616-9d99a3bb5542 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.568791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-44f12b51-f25d-4adb-9616-9d99a3bb5542 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.571838 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3687/7468] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:50 2022] DELETE /identity/v3/roles/8087761cbe1148a996b39260dd5425dc => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:50.576778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:50.580127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:50.599394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a81bc-04ef-4428-b78c-c0794d09f2b6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:50.678094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:50.721344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:50.721949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:50.722826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:50.722826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/384b4016baea45a187b06d044658d9e1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:50.724369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:delete_user(user_id=384b4016baea45a187b06d044658d9e1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:50.726542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:50.788273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '384b4016baea45a187b06d044658d9e1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.799572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f12b51-f25d-4adb-9616-9d99a3bb5542 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:50.801356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-44f12b51-f25d-4adb-9616-9d99a3bb5542 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:50.805561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '384b4016baea45a187b06d044658d9e1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.814544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:50.818331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '384b4016baea45a187b06d044658d9e1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.820059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '384b4016baea45a187b06d044658d9e1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.821199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '384b4016baea45a187b06d044658d9e1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:50.844536 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3782/7469] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:50 2022] DELETE /identity/v3/users/384b4016baea45a187b06d044658d9e1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:50.845640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:50.848366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:50.918139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f12b51-f25d-4adb-9616-9d99a3bb5542 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:50.953734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf3ff1-d7bc-4b8c-8bcd-c1859259fbd9 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:50.962211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:50.963028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:50.963317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:50.963652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects/dcebef9bf50a415b9aa30c33aca6542d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:50.966106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:delete_project(project_id=dcebef9bf50a415b9aa30c33aca6542d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:50.968555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:51.004204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588921ed-798a-4c50-8ab7-7159332d13be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:51.005341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588921ed-798a-4c50-8ab7-7159332d13be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.005888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588921ed-798a-4c50-8ab7-7159332d13be admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.006367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588921ed-798a-4c50-8ab7-7159332d13be admin admin] PATH_INFO: `/v3/projects/c8aad9cc05674b1d908416d147eccfe7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.014685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588921ed-798a-4c50-8ab7-7159332d13be admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8aad9cc05674b1d908416d147eccfe7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:51.016736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588921ed-798a-4c50-8ab7-7159332d13be admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:51.036468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcebef9bf50a415b9aa30c33aca6542d', 'request_id': 'req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.041540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:51.046864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcebef9bf50a415b9aa30c33aca6542d', 'request_id': 'req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.062029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcebef9bf50a415b9aa30c33aca6542d', 'request_id': 'req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.075650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] The token cache is being invalidate because project dcebef9bf50a415b9aa30c33aca6542d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:51.076022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.076776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.079816 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3783/7470] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:50 2022] DELETE /identity/v3/projects/dcebef9bf50a415b9aa30c33aca6542d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:51.085360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:51.087719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:51.104956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-588921ed-798a-4c50-8ab7-7159332d13be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8aad9cc05674b1d908416d147eccfe7', 'request_id': 'req-588921ed-798a-4c50-8ab7-7159332d13be'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.115508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-588921ed-798a-4c50-8ab7-7159332d13be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8aad9cc05674b1d908416d147eccfe7', 'request_id': 'req-588921ed-798a-4c50-8ab7-7159332d13be'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.134742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-588921ed-798a-4c50-8ab7-7159332d13be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8aad9cc05674b1d908416d147eccfe7', 'request_id': 'req-588921ed-798a-4c50-8ab7-7159332d13be'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.150358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-588921ed-798a-4c50-8ab7-7159332d13be admin admin] The token cache is being invalidate because project c8aad9cc05674b1d908416d147eccfe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:51.150847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-588921ed-798a-4c50-8ab7-7159332d13be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.153194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-588921ed-798a-4c50-8ab7-7159332d13be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.161269 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3688/7471] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:50 2022] DELETE /identity/v3/projects/c8aad9cc05674b1d908416d147eccfe7 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:51.163037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f88c8f-cc1d-4d3e-b664-d981cb33e5e1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.163394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f88c8f-cc1d-4d3e-b664-d981cb33e5e1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.163737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f88c8f-cc1d-4d3e-b664-d981cb33e5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.205210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b438c-cdcc-46ba-8fda-6c5bc56af27a tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:51.213660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d6f88c8f-cc1d-4d3e-b664-d981cb33e5e1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:51.215116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d6f88c8f-cc1d-4d3e-b664-d981cb33e5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:51.279150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d614c85-0ce3-46b2-a8f7-3a1cb4cf260f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:51.279816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d614c85-0ce3-46b2-a8f7-3a1cb4cf260f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.280114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d614c85-0ce3-46b2-a8f7-3a1cb4cf260f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.280403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d614c85-0ce3-46b2-a8f7-3a1cb4cf260f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/99982492c2444c6e944aae9198d9b4e6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.283445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:51.286355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d614c85-0ce3-46b2-a8f7-3a1cb4cf260f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:update_user(user_id=99982492c2444c6e944aae9198d9b4e6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:51.288433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d614c85-0ce3-46b2-a8f7-3a1cb4cf260f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:51.330056 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3689/7472] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:51.339228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce02277-5aa0-4f26-8c55-99fa0bcff478 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.339586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce02277-5aa0-4f26-8c55-99fa0bcff478 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.347319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce02277-5aa0-4f26-8c55-99fa0bcff478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.354080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4d614c85-0ce3-46b2-a8f7-3a1cb4cf260f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '99982492c2444c6e944aae9198d9b4e6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.376495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4d614c85-0ce3-46b2-a8f7-3a1cb4cf260f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invalidating the token cache because user 99982492c2444c6e944aae9198d9b4e6 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:51.376855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4d614c85-0ce3-46b2-a8f7-3a1cb4cf260f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.377732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4d614c85-0ce3-46b2-a8f7-3a1cb4cf260f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.378940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cce02277-5aa0-4f26-8c55-99fa0bcff478 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:51.379814 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3784/7473] 10.222.0.16 () {60 vars in 1243 bytes} [Sat Sep 3 21:16:51 2022] PATCH /identity/v3/users/99982492c2444c6e944aae9198d9b4e6 => generated 296 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:16:51.386356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cce02277-5aa0-4f26-8c55-99fa0bcff478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:51.387091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfad43-a151-4a23-8c9d-cf9d16fca28b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.387526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfad43-a151-4a23-8c9d-cf9d16fca28b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.387812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfad43-a151-4a23-8c9d-cf9d16fca28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.403454 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3690/7474] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:51.407783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce02277-5aa0-4f26-8c55-99fa0bcff478 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:51.419936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce02277-5aa0-4f26-8c55-99fa0bcff478 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:51.426889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573d23-577d-4cf7-9a50-60a437cb12b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:51.427770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573d23-577d-4cf7-9a50-60a437cb12b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.428220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573d23-577d-4cf7-9a50-60a437cb12b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.428839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573d23-577d-4cf7-9a50-60a437cb12b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.429699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f573d23-577d-4cf7-9a50-60a437cb12b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:16:51.431473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f573d23-577d-4cf7-9a50-60a437cb12b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:51.433936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f573d23-577d-4cf7-9a50-60a437cb12b4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:51.442161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b9cfad43-a151-4a23-8c9d-cf9d16fca28b None None] MFA Rules not processed for user `99982492c2444c6e944aae9198d9b4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:51.443953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b9cfad43-a151-4a23-8c9d-cf9d16fca28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:51.447753 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3785/7475] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:51 2022] POST /identity/v3/auth/tokens => generated 348 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:16:51.450867 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3691/7476] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:16:51.454560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfad43-a151-4a23-8c9d-cf9d16fca28b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:51.456100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573d23-577d-4cf7-9a50-60a437cb12b4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:51.460144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b9cfad43-a151-4a23-8c9d-cf9d16fca28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:51.477240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573d23-577d-4cf7-9a50-60a437cb12b4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:51.487957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a54f9-cb92-4433-a3be-7f7ff4f62f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:51.488628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a54f9-cb92-4433-a3be-7f7ff4f62f6f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.489119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfad43-a151-4a23-8c9d-cf9d16fca28b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:51.490504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a54f9-cb92-4433-a3be-7f7ff4f62f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.490806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a54f9-cb92-4433-a3be-7f7ff4f62f6f admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.491563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a54f9-cb92-4433-a3be-7f7ff4f62f6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:51.494416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a54f9-cb92-4433-a3be-7f7ff4f62f6f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:51.498071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:51.498834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.499181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.499543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/99982492c2444c6e944aae9198d9b4e6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.500852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:update_user(user_id=99982492c2444c6e944aae9198d9b4e6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:51.503150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:51.544127 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3692/7477] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:51 2022] POST /identity/v3/projects => generated 399 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:51.550347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a54f9-cb92-4433-a3be-7f7ff4f62f6f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:51.556590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '99982492c2444c6e944aae9198d9b4e6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.568240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invalidating the token cache because user 99982492c2444c6e944aae9198d9b4e6 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:51.568938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.569829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.577210 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3786/7478] 10.222.0.16 () {60 vars in 1243 bytes} [Sat Sep 3 21:16:51 2022] PATCH /identity/v3/users/99982492c2444c6e944aae9198d9b4e6 => generated 296 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:51.579313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:51.585746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:51.608754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b817e-8b33-4541-bac4-1b413a037c62 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:51.627137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:UPDATE=2,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:51.664020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a54f9-cb92-4433-a3be-7f7ff4f62f6f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:51.671612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffd01f-2736-4ff4-a606-71219c4a2f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:51.672374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffd01f-2736-4ff4-a606-71219c4a2f29 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.673083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffd01f-2736-4ff4-a606-71219c4a2f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.673431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffd01f-2736-4ff4-a606-71219c4a2f29 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.674079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffd01f-2736-4ff4-a606-71219c4a2f29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:51.676091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffd01f-2736-4ff4-a606-71219c4a2f29 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:51.698743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23388bf5-57e2-4a9c-bdc5-7e93430e2354 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:51.699613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23388bf5-57e2-4a9c-bdc5-7e93430e2354 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.700013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23388bf5-57e2-4a9c-bdc5-7e93430e2354 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.700409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23388bf5-57e2-4a9c-bdc5-7e93430e2354 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/99982492c2444c6e944aae9198d9b4e6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.710394 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3693/7479] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:51 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:51.714708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffd01f-2736-4ff4-a606-71219c4a2f29 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:51.720390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-61ffd01f-2736-4ff4-a606-71219c4a2f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:51.731725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23388bf5-57e2-4a9c-bdc5-7e93430e2354 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:update_user(user_id=99982492c2444c6e944aae9198d9b4e6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:51.734394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23388bf5-57e2-4a9c-bdc5-7e93430e2354 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:51.734629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffd01f-2736-4ff4-a606-71219c4a2f29 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:51.739442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb322304-ff9a-47eb-b136-b09cfbed2cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:51.740608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb322304-ff9a-47eb-b136-b09cfbed2cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.740925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb322304-ff9a-47eb-b136-b09cfbed2cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.741246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb322304-ff9a-47eb-b136-b09cfbed2cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.741912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb322304-ff9a-47eb-b136-b09cfbed2cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:51.744163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb322304-ff9a-47eb-b136-b09cfbed2cb2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:51.768637 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3694/7480] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:51.770253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb322304-ff9a-47eb-b136-b09cfbed2cb2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:51.787205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-23388bf5-57e2-4a9c-bdc5-7e93430e2354 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '99982492c2444c6e944aae9198d9b4e6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.796051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb322304-ff9a-47eb-b136-b09cfbed2cb2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:51.802367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ed495-93e6-402a-b7a7-3579f08b3b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:51.802731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ed495-93e6-402a-b7a7-3579f08b3b58 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.803263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ed495-93e6-402a-b7a7-3579f08b3b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.803639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ed495-93e6-402a-b7a7-3579f08b3b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.804437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ed495-93e6-402a-b7a7-3579f08b3b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:51.806353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-23388bf5-57e2-4a9c-bdc5-7e93430e2354 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invalidating the token cache because user 99982492c2444c6e944aae9198d9b4e6 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:51.806725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-23388bf5-57e2-4a9c-bdc5-7e93430e2354 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.807850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-23388bf5-57e2-4a9c-bdc5-7e93430e2354 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:51.808115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ed495-93e6-402a-b7a7-3579f08b3b58 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:51.810335 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3787/7481] 10.222.0.16 () {60 vars in 1243 bytes} [Sat Sep 3 21:16:51 2022] PATCH /identity/v3/users/99982492c2444c6e944aae9198d9b4e6 => generated 296 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:51.816518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cac3e-e9a6-4718-9e26-c0e15f59cddc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.817235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cac3e-e9a6-4718-9e26-c0e15f59cddc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.817521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cac3e-e9a6-4718-9e26-c0e15f59cddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.827059 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3695/7482] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:51.829925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ed495-93e6-402a-b7a7-3579f08b3b58 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:51.833943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7d4ed495-93e6-402a-b7a7-3579f08b3b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:51.852053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ed495-93e6-402a-b7a7-3579f08b3b58 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:51.859537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4d982-1c6d-4fcd-9c5d-3f1d99975384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:51.860333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4d982-1c6d-4fcd-9c5d-3f1d99975384 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.860693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4d982-1c6d-4fcd-9c5d-3f1d99975384 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.861134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4d982-1c6d-4fcd-9c5d-3f1d99975384 admin admin] PATH_INFO: `/v3/projects/63c3dbf35caa4bbb8460d5f3d2911136/users/e8b73599e8dd45798e7290d68d0b9ba1/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.875444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9e0cac3e-e9a6-4718-9e26-c0e15f59cddc None None] MFA Rules not processed for user `99982492c2444c6e944aae9198d9b4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:51.877109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9e0cac3e-e9a6-4718-9e26-c0e15f59cddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:51.879382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4d982-1c6d-4fcd-9c5d-3f1d99975384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c3dbf35caa4bbb8460d5f3d2911136, user_id=e8b73599e8dd45798e7290d68d0b9ba1, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:51.881662 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3788/7483] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:51 2022] POST /identity/v3/auth/tokens => generated 348 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:16:51.886305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cac3e-e9a6-4718-9e26-c0e15f59cddc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:51.886479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4d982-1c6d-4fcd-9c5d-3f1d99975384 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:51.888318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9e0cac3e-e9a6-4718-9e26-c0e15f59cddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:51.907834 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3696/7484] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:51 2022] PUT /identity/v3/projects/63c3dbf35caa4bbb8460d5f3d2911136/users/e8b73599e8dd45798e7290d68d0b9ba1/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:51.920830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcb929-7692-41eb-912d-ec52ee3c5a46 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:51.921555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcb929-7692-41eb-912d-ec52ee3c5a46 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:51.921922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcb929-7692-41eb-912d-ec52ee3c5a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:51.924314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cac3e-e9a6-4718-9e26-c0e15f59cddc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:51.961018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-debcb929-7692-41eb-912d-ec52ee3c5a46 None None] MFA Rules not processed for user `e8b73599e8dd45798e7290d68d0b9ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:51.962588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-debcb929-7692-41eb-912d-ec52ee3c5a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:52.024009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:52.024485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:52.025517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:52.026156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:52.026722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/99982492c2444c6e944aae9198d9b4e6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:52.028188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:update_user(user_id=99982492c2444c6e944aae9198d9b4e6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:52.030325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:52.069013 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3697/7485] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:51 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:52.071489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6f9a1-ea5f-4c83-8ed9-a4b1d6f6651c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:52.071965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6f9a1-ea5f-4c83-8ed9-a4b1d6f6651c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:52.072399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6f9a1-ea5f-4c83-8ed9-a4b1d6f6651c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:52.078640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '99982492c2444c6e944aae9198d9b4e6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:52.087810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invalidating the token cache because user 99982492c2444c6e944aae9198d9b4e6 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:52.088222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:52.089200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:52.095391 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3789/7486] 10.222.0.16 () {60 vars in 1243 bytes} [Sat Sep 3 21:16:51 2022] PATCH /identity/v3/users/99982492c2444c6e944aae9198d9b4e6 => generated 296 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:16:52.095391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:52.097920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6ff6f9a1-ea5f-4c83-8ed9-a4b1d6f6651c None None] MFA Rules not processed for user `b5c96be8fbe1436bb3e84340ceda1917`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:52.098047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:52.099143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6ff6f9a1-ea5f-4c83-8ed9-a4b1d6f6651c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:52.165540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:UPDATE=2,keystone:INSERT=4,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:52.188001 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3698/7487] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:52.189253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6f9a1-ea5f-4c83-8ed9-a4b1d6f6651c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:52.191052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6ff6f9a1-ea5f-4c83-8ed9-a4b1d6f6651c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:52.209162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758aa77-deb9-47e3-92a3-290530eee49c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:52.212088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6f9a1-ea5f-4c83-8ed9-a4b1d6f6651c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:52.214515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd01ad5-4195-41cb-aee9-8ca21bf93cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:52.215349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd01ad5-4195-41cb-aee9-8ca21bf93cce admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:52.215837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd01ad5-4195-41cb-aee9-8ca21bf93cce admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:52.216125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd01ad5-4195-41cb-aee9-8ca21bf93cce admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:52.216888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd01ad5-4195-41cb-aee9-8ca21bf93cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:52.219102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd01ad5-4195-41cb-aee9-8ca21bf93cce admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:52.219327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5a170-238a-46d1-8483-b207eb0c013c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:52.220144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5a170-238a-46d1-8483-b207eb0c013c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:52.220726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5a170-238a-46d1-8483-b207eb0c013c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:52.221341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5a170-238a-46d1-8483-b207eb0c013c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:52.222129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5a170-238a-46d1-8483-b207eb0c013c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:52.226553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5a170-238a-46d1-8483-b207eb0c013c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:52.277002 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3790/7488] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:52 2022] POST /identity/v3/projects => generated 399 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:52.278052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd01ad5-4195-41cb-aee9-8ca21bf93cce admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:52.279714 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3699/7489] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Sep 3 21:16:52 2022] POST /identity/v3/users => generated 265 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:52.288936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5a170-238a-46d1-8483-b207eb0c013c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:52.320334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5a170-238a-46d1-8483-b207eb0c013c tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:52.385122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd01ad5-4195-41cb-aee9-8ca21bf93cce admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:52.393368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d2b00-8e86-4bf4-b766-5285a2732635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:52.396066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d2b00-8e86-4bf4-b766-5285a2732635 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:52.396589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d2b00-8e86-4bf4-b766-5285a2732635 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:52.397039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d2b00-8e86-4bf4-b766-5285a2732635 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:52.397795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d2b00-8e86-4bf4-b766-5285a2732635 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:52.400769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d2b00-8e86-4bf4-b766-5285a2732635 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:52.442034 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3791/7490] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:52 2022] POST /identity/v3/users => generated 306 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:52.451273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d2b00-8e86-4bf4-b766-5285a2732635 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:52.465858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d2b00-8e86-4bf4-b766-5285a2732635 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:52.473212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fd48c-7b4e-48bd-a91e-ac39dd46dfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:52.473920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fd48c-7b4e-48bd-a91e-ac39dd46dfc5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:52.474417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fd48c-7b4e-48bd-a91e-ac39dd46dfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:52.474773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fd48c-7b4e-48bd-a91e-ac39dd46dfc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:52.475612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fd48c-7b4e-48bd-a91e-ac39dd46dfc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:52.477916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fd48c-7b4e-48bd-a91e-ac39dd46dfc5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:52.479220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa34cc-6097-4981-90ae-e967e808cbbf tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:52.480481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa34cc-6097-4981-90ae-e967e808cbbf tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:52.481089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa34cc-6097-4981-90ae-e967e808cbbf tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:52.481582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa34cc-6097-4981-90ae-e967e808cbbf tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/716c96804647477fb525fd2205d97d04/password` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:52.503290 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3792/7491] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:52.513228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fd48c-7b4e-48bd-a91e-ac39dd46dfc5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:52.564632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fd48c-7b4e-48bd-a91e-ac39dd46dfc5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:52.576881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbb251-79f2-4059-966b-cd2825dce24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:52.577950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbb251-79f2-4059-966b-cd2825dce24e admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:52.578326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbb251-79f2-4059-966b-cd2825dce24e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:52.578974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbb251-79f2-4059-966b-cd2825dce24e admin admin] PATH_INFO: `/v3/projects/fbd93b94d2364664961a7e2acabc07d1/users/c21b5c02eb19467bbf8f6f0161a7717e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:52.591431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-edaa34cc-6097-4981-90ae-e967e808cbbf tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '716c96804647477fb525fd2205d97d04'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:52.608745 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3700/7492] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:16:52 2022] POST /identity/v3/users/716c96804647477fb525fd2205d97d04/password => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:52.617371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbb251-79f2-4059-966b-cd2825dce24e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd93b94d2364664961a7e2acabc07d1, user_id=c21b5c02eb19467bbf8f6f0161a7717e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:52.620155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbb251-79f2-4059-966b-cd2825dce24e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:52.646047 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3793/7493] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:52 2022] PUT /identity/v3/projects/fbd93b94d2364664961a7e2acabc07d1/users/c21b5c02eb19467bbf8f6f0161a7717e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:52.651497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa34cc-6097-4981-90ae-e967e808cbbf tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:52.667737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa34cc-6097-4981-90ae-e967e808cbbf tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:52.699585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:52.771980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb31284-8b0e-4e2c-8efe-f0167fdc2488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:52.773913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb31284-8b0e-4e2c-8efe-f0167fdc2488 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:52.774274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb31284-8b0e-4e2c-8efe-f0167fdc2488 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:52.775460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb31284-8b0e-4e2c-8efe-f0167fdc2488 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:52.776135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb31284-8b0e-4e2c-8efe-f0167fdc2488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:52.778037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb31284-8b0e-4e2c-8efe-f0167fdc2488 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:52.797696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbb251-79f2-4059-966b-cd2825dce24e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:52.812091 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3701/7494] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:52.813769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbb251-79f2-4059-966b-cd2825dce24e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:52.825258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cea4d-dfe2-4796-a8a0-10427d45ff3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:52.826543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cea4d-dfe2-4796-a8a0-10427d45ff3a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:52.826543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cea4d-dfe2-4796-a8a0-10427d45ff3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:52.828388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cea4d-dfe2-4796-a8a0-10427d45ff3a admin admin] PATH_INFO: `/v3/projects/fbd93b94d2364664961a7e2acabc07d1/users/c21b5c02eb19467bbf8f6f0161a7717e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:52.837882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cea4d-dfe2-4796-a8a0-10427d45ff3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd93b94d2364664961a7e2acabc07d1, user_id=c21b5c02eb19467bbf8f6f0161a7717e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:52.840973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cea4d-dfe2-4796-a8a0-10427d45ff3a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:52.879198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68f03f-f915-45d8-b8af-8472e086c593 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:52.879496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68f03f-f915-45d8-b8af-8472e086c593 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:52.880240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68f03f-f915-45d8-b8af-8472e086c593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:52.884656 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3794/7495] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:52 2022] PUT /identity/v3/projects/fbd93b94d2364664961a7e2acabc07d1/users/c21b5c02eb19467bbf8f6f0161a7717e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:52.933686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2c68f03f-f915-45d8-b8af-8472e086c593 None None] MFA Rules not processed for user `c21b5c02eb19467bbf8f6f0161a7717e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:52.935245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2c68f03f-f915-45d8-b8af-8472e086c593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:53.079647 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3702/7496] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:52 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:53.093295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cea4d-dfe2-4796-a8a0-10427d45ff3a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:53.095578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ca9cea4d-dfe2-4796-a8a0-10427d45ff3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:53.205945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cea4d-dfe2-4796-a8a0-10427d45ff3a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:53.219304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29097864-a3a1-48ef-969c-5734e692e98e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:53.221757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29097864-a3a1-48ef-969c-5734e692e98e service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:53.222468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29097864-a3a1-48ef-969c-5734e692e98e service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:53.222876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29097864-a3a1-48ef-969c-5734e692e98e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:53.238547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29097864-a3a1-48ef-969c-5734e692e98e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:53.240441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29097864-a3a1-48ef-969c-5734e692e98e service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:53.275593 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3795/7497] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:53 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:53.368224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68f03f-f915-45d8-b8af-8472e086c593 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:53.380347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68f03f-f915-45d8-b8af-8472e086c593 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:53.384836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b89f82-add6-4cbc-99d6-641a867c3e18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:53.385569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b89f82-add6-4cbc-99d6-641a867c3e18 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:53.385958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b89f82-add6-4cbc-99d6-641a867c3e18 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:53.386337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b89f82-add6-4cbc-99d6-641a867c3e18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:53.388539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-68b89f82-add6-4cbc-99d6-641a867c3e18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:53.470035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b89f82-add6-4cbc-99d6-641a867c3e18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:53.471976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b89f82-add6-4cbc-99d6-641a867c3e18 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:53.482082 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3703/7498] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:53 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:53.614604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec30d18-9872-41d3-9653-58eaec27891c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:53.615206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec30d18-9872-41d3-9653-58eaec27891c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:53.615206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec30d18-9872-41d3-9653-58eaec27891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:53.628094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:53.636795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7ec30d18-9872-41d3-9653-58eaec27891c None None] MFA Rules not processed for user `716c96804647477fb525fd2205d97d04`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:53.639471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7ec30d18-9872-41d3-9653-58eaec27891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:53.644393 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3796/7499] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:53 2022] POST /identity/v3/auth/tokens => generated 330 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:16:53.649271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b89f82-add6-4cbc-99d6-641a867c3e18 service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:53.716424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:53.744789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b89f82-add6-4cbc-99d6-641a867c3e18 service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:53.768925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5603e-5e3c-44f7-b9e3-ef83e285789b tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:53.770239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5603e-5e3c-44f7-b9e3-ef83e285789b tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:53.770641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5603e-5e3c-44f7-b9e3-ef83e285789b tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:53.771491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5603e-5e3c-44f7-b9e3-ef83e285789b tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:53.800034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5603e-5e3c-44f7-b9e3-ef83e285789b tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:53.803871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5603e-5e3c-44f7-b9e3-ef83e285789b tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:53.817424 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3704/7500] 10.222.0.16 () {56 vars in 1263 bytes} [Sat Sep 3 21:16:53 2022] GET /identity/v3/auth/tokens => generated 330 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 21:16:53.822680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec30d18-9872-41d3-9653-58eaec27891c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:53.840278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec30d18-9872-41d3-9653-58eaec27891c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:53.846594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:53.847475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:53.848754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:53.849227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/716c96804647477fb525fd2205d97d04` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:53.851109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:delete_user(user_id=716c96804647477fb525fd2205d97d04)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:53.854391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:53.940480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '716c96804647477fb525fd2205d97d04'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:53.954565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '716c96804647477fb525fd2205d97d04'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:53.963773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '716c96804647477fb525fd2205d97d04'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:53.964521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '716c96804647477fb525fd2205d97d04'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:53.965370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '716c96804647477fb525fd2205d97d04'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:53.982754 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3797/7501] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:53 2022] DELETE /identity/v3/users/716c96804647477fb525fd2205d97d04 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:53.988147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5603e-5e3c-44f7-b9e3-ef83e285789b tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:53.999486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5603e-5e3c-44f7-b9e3-ef83e285789b tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:54.073116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213a4af-e210-4fe8-b4ee-77b038e495c7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:54.075220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a4af-e210-4fe8-b4ee-77b038e495c7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:54.075761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a4af-e210-4fe8-b4ee-77b038e495c7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:54.076118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a4af-e210-4fe8-b4ee-77b038e495c7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:54.076855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213a4af-e210-4fe8-b4ee-77b038e495c7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:54.078824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213a4af-e210-4fe8-b4ee-77b038e495c7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:54.111334 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3705/7502] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:53 2022] POST /identity/v3/users => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:54.115653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:54.117295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:54.136419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71b367-a0cf-4abe-8419-38d8056de12f tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:54.142384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633575a8-1208-4afa-97f2-df6781071df4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:54.143156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633575a8-1208-4afa-97f2-df6781071df4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:54.143453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633575a8-1208-4afa-97f2-df6781071df4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:54.143740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633575a8-1208-4afa-97f2-df6781071df4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:54.144393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633575a8-1208-4afa-97f2-df6781071df4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:54.146815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633575a8-1208-4afa-97f2-df6781071df4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:54.189971 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3798/7503] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:54 2022] POST /identity/v3/projects => generated 358 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:54.193620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213a4af-e210-4fe8-b4ee-77b038e495c7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:54.222789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213a4af-e210-4fe8-b4ee-77b038e495c7 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:54.293927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afba54-3d98-4535-bcf5-fc41d2ecc2f4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:54.294694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afba54-3d98-4535-bcf5-fc41d2ecc2f4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:54.295062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afba54-3d98-4535-bcf5-fc41d2ecc2f4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:54.295428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afba54-3d98-4535-bcf5-fc41d2ecc2f4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/fe3e135cbc1a47f9bfde940b1eea5cb1` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:54.312524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afba54-3d98-4535-bcf5-fc41d2ecc2f4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:update_user(user_id=fe3e135cbc1a47f9bfde940b1eea5cb1)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:54.313888 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-36afba54-3d98-4535-bcf5-fc41d2ecc2f4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:16:54.313888 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:16:54.313888 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:16:54.315306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afba54-3d98-4535-bcf5-fc41d2ecc2f4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:54.355400 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3706/7504] 10.222.0.16 () {60 vars in 1244 bytes} [Sat Sep 3 21:16:54 2022] PATCH /identity/v3/users/fe3e135cbc1a47f9bfde940b1eea5cb1 => generated 524 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:16:54.362477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633575a8-1208-4afa-97f2-df6781071df4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:54.377491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633575a8-1208-4afa-97f2-df6781071df4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:54.382689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45ff64-7c3a-49bb-8f53-3bce00cff454 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:54.383426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45ff64-7c3a-49bb-8f53-3bce00cff454 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:54.383831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45ff64-7c3a-49bb-8f53-3bce00cff454 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:54.384194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45ff64-7c3a-49bb-8f53-3bce00cff454 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/fe3e135cbc1a47f9bfde940b1eea5cb1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:54.401176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45ff64-7c3a-49bb-8f53-3bce00cff454 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:get_user(user_id=fe3e135cbc1a47f9bfde940b1eea5cb1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:54.404155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45ff64-7c3a-49bb-8f53-3bce00cff454 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:54.406349 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3799/7505] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Sep 3 21:16:54 2022] GET /identity/v3/users/fe3e135cbc1a47f9bfde940b1eea5cb1 => generated 390 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:54.412850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afba54-3d98-4535-bcf5-fc41d2ecc2f4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:54.426439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afba54-3d98-4535-bcf5-fc41d2ecc2f4 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:54.432524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:54.433301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:54.433692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:54.434032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/projects/463cff1b31a346309350c165eb12894d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:54.435275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:delete_project(project_id=463cff1b31a346309350c165eb12894d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:54.437578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:54.474107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:UPDATE=2,keystone:DELETE=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:54.491792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '463cff1b31a346309350c165eb12894d', 'request_id': 'req-992329c8-7892-4f13-923a-cd1d0a00eb47'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.501187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '463cff1b31a346309350c165eb12894d', 'request_id': 'req-992329c8-7892-4f13-923a-cd1d0a00eb47'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.508327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '463cff1b31a346309350c165eb12894d', 'request_id': 'req-992329c8-7892-4f13-923a-cd1d0a00eb47'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.519637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] The token cache is being invalidate because project 463cff1b31a346309350c165eb12894d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:54.520089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.520719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.522326 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3707/7506] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:54 2022] DELETE /identity/v3/projects/463cff1b31a346309350c165eb12894d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:54.527086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45ff64-7c3a-49bb-8f53-3bce00cff454 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:54.529184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2c45ff64-7c3a-49bb-8f53-3bce00cff454 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:54.543451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45ff64-7c3a-49bb-8f53-3bce00cff454 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:54.603867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b0b82-0de9-4f03-a393-de868468f240 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef4b2059baf6449ca8538cca89151667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2aa6b5734a214c1480de474d8008c59c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:54.604523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b0b82-0de9-4f03-a393-de868468f240 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:54.605115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b0b82-0de9-4f03-a393-de868468f240 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:54.605399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b0b82-0de9-4f03-a393-de868468f240 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] PATH_INFO: `/v3/users/fe3e135cbc1a47f9bfde940b1eea5cb1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:54.606626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b0b82-0de9-4f03-a393-de868468f240 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorizing `identity:delete_user(user_id=fe3e135cbc1a47f9bfde940b1eea5cb1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:54.608692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b0b82-0de9-4f03-a393-de868468f240 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:54.646977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:54.658607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-285b0b82-0de9-4f03-a393-de868468f240 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3e135cbc1a47f9bfde940b1eea5cb1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.675097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-285b0b82-0de9-4f03-a393-de868468f240 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3e135cbc1a47f9bfde940b1eea5cb1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.684351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-285b0b82-0de9-4f03-a393-de868468f240 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3e135cbc1a47f9bfde940b1eea5cb1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.685363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-285b0b82-0de9-4f03-a393-de868468f240 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3e135cbc1a47f9bfde940b1eea5cb1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.686208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-285b0b82-0de9-4f03-a393-de868468f240 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3e135cbc1a47f9bfde940b1eea5cb1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.701013 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3800/7507] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:54 2022] DELETE /identity/v3/users/fe3e135cbc1a47f9bfde940b1eea5cb1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:54.705866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:54.707581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:54.789014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992329c8-7892-4f13-923a-cd1d0a00eb47 tempest-UsersV3TestJSON-129066099 tempest-UsersV3TestJSON-129066099-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:54.854160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:54.855085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:54.855375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:54.855659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] PATH_INFO: `/v3/users/99982492c2444c6e944aae9198d9b4e6` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:54.870835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99982492c2444c6e944aae9198d9b4e6)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:54.872733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:54.937220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99982492c2444c6e944aae9198d9b4e6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.937220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:54.943016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99982492c2444c6e944aae9198d9b4e6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.943825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99982492c2444c6e944aae9198d9b4e6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.959047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99982492c2444c6e944aae9198d9b4e6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.970740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99982492c2444c6e944aae9198d9b4e6'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:54.980767 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3708/7508] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:54 2022] DELETE /identity/v3/users/99982492c2444c6e944aae9198d9b4e6 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:54.986118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a4134-7148-46e1-8047-4ef00c2e5da3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:54.986592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a4134-7148-46e1-8047-4ef00c2e5da3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:54.986867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a4134-7148-46e1-8047-4ef00c2e5da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:55.011174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f86a4134-7148-46e1-8047-4ef00c2e5da3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:55.012847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f86a4134-7148-46e1-8047-4ef00c2e5da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:55.088069 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3801/7509] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:55.098046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:55.099571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:55.181113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416c29c-450e-4943-a9aa-2cca0edac6e3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:55.192361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45ead0-69af-4c3a-bb9e-3afffa4da7d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:55.193287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45ead0-69af-4c3a-bb9e-3afffa4da7d4 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:55.193589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45ead0-69af-4c3a-bb9e-3afffa4da7d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:55.193941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45ead0-69af-4c3a-bb9e-3afffa4da7d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:55.204696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45ead0-69af-4c3a-bb9e-3afffa4da7d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:55.206228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45ead0-69af-4c3a-bb9e-3afffa4da7d4 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:55.217863 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3709/7510] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:55.284206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a4134-7148-46e1-8047-4ef00c2e5da3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:55.285889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f86a4134-7148-46e1-8047-4ef00c2e5da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:55.298510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a4134-7148-46e1-8047-4ef00c2e5da3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:55.303740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:55.304176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:55.304506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:55.304935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] PATH_INFO: `/v3/users/e8b73599e8dd45798e7290d68d0b9ba1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:55.306575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8b73599e8dd45798e7290d68d0b9ba1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:55.308673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:55.371330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8b73599e8dd45798e7290d68d0b9ba1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:55.402534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8b73599e8dd45798e7290d68d0b9ba1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:55.435042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8b73599e8dd45798e7290d68d0b9ba1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:55.435891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8b73599e8dd45798e7290d68d0b9ba1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:55.436640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8b73599e8dd45798e7290d68d0b9ba1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:55.451431 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3802/7511] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:55 2022] DELETE /identity/v3/users/e8b73599e8dd45798e7290d68d0b9ba1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:55.456146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecfdfd-b58f-4076-85a0-cbeef5a4b38d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:55.456444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecfdfd-b58f-4076-85a0-cbeef5a4b38d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:55.456730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecfdfd-b58f-4076-85a0-cbeef5a4b38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:55.516648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e8ecfdfd-b58f-4076-85a0-cbeef5a4b38d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:55.518759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e8ecfdfd-b58f-4076-85a0-cbeef5a4b38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:55.603334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:55.605142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:55.625679 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3710/7512] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:55.635790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecfdfd-b58f-4076-85a0-cbeef5a4b38d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:55.637315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e8ecfdfd-b58f-4076-85a0-cbeef5a4b38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:55.659958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:55.699628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf138b-5b72-4fa9-abce-6e66e69b9ae0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:55.702092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecfdfd-b58f-4076-85a0-cbeef5a4b38d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:55.704725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:55.705475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:55.705833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:55.706120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] PATH_INFO: `/v3/projects/571f9f0354e54ee8bfc620031a8d4209` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:55.706427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbf020-09ca-4f68-9979-e1014c0cf4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:55.707055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbf020-09ca-4f68-9979-e1014c0cf4cf service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:55.707354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbf020-09ca-4f68-9979-e1014c0cf4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:55.707632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbf020-09ca-4f68-9979-e1014c0cf4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:55.707907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=571f9f0354e54ee8bfc620031a8d4209)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:55.711624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:55.717807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbf020-09ca-4f68-9979-e1014c0cf4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:55.719299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbf020-09ca-4f68-9979-e1014c0cf4cf service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:55.733910 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3711/7513] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:16:55.782152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '571f9f0354e54ee8bfc620031a8d4209', 'request_id': 'req-406986ad-3b30-41b4-8b58-f9a915b6da0d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:55.794032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '571f9f0354e54ee8bfc620031a8d4209', 'request_id': 'req-406986ad-3b30-41b4-8b58-f9a915b6da0d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:55.803237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '571f9f0354e54ee8bfc620031a8d4209', 'request_id': 'req-406986ad-3b30-41b4-8b58-f9a915b6da0d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:55.821352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] The token cache is being invalidate because project 571f9f0354e54ee8bfc620031a8d4209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:55.822313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:55.823805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:55.826452 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3803/7514] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:55 2022] DELETE /identity/v3/projects/571f9f0354e54ee8bfc620031a8d4209 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:55.830564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbf020-09ca-4f68-9979-e1014c0cf4cf service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:55.856008 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbf020-09ca-4f68-9979-e1014c0cf4cf service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:55.883676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:55.940424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:55.941106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:55.941556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:55.941842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] PATH_INFO: `/v3/users/ef4b2059baf6449ca8538cca89151667` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:55.943299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef4b2059baf6449ca8538cca89151667)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:55.945413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:56.006912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef4b2059baf6449ca8538cca89151667'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.020950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef4b2059baf6449ca8538cca89151667'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.021669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef4b2059baf6449ca8538cca89151667'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.033498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef4b2059baf6449ca8538cca89151667'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.044661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef4b2059baf6449ca8538cca89151667'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.057234 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3712/7515] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:55 2022] DELETE /identity/v3/users/ef4b2059baf6449ca8538cca89151667 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:56.064211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:56.066113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:56.091962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406986ad-3b30-41b4-8b58-f9a915b6da0d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:56.190181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:56.190826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:56.191149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:56.191512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] PATH_INFO: `/v3/projects/63c3dbf35caa4bbb8460d5f3d2911136` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:56.192983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63c3dbf35caa4bbb8460d5f3d2911136)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:56.194922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:56.245897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63c3dbf35caa4bbb8460d5f3d2911136', 'request_id': 'req-f6420cd9-f081-4b35-a1a4-efc951c0f044'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.254505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63c3dbf35caa4bbb8460d5f3d2911136', 'request_id': 'req-f6420cd9-f081-4b35-a1a4-efc951c0f044'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.258066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:56.264244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63c3dbf35caa4bbb8460d5f3d2911136', 'request_id': 'req-f6420cd9-f081-4b35-a1a4-efc951c0f044'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.277190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] The token cache is being invalidate because project 63c3dbf35caa4bbb8460d5f3d2911136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:56.277448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.278127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.279762 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3804/7516] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:56 2022] DELETE /identity/v3/projects/63c3dbf35caa4bbb8460d5f3d2911136 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:56.283654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:56.285133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:56.298729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb3d6a-c099-484a-a2c1-cdd52fda872a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:56.353045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:56.354536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:56.364295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabe4be-729d-4215-a18a-b2ebc34931cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:56.365996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabe4be-729d-4215-a18a-b2ebc34931cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:56.366286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabe4be-729d-4215-a18a-b2ebc34931cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:56.366875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabe4be-729d-4215-a18a-b2ebc34931cc admin admin] PATH_INFO: `/v3/users/c21b5c02eb19467bbf8f6f0161a7717e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:56.368082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabe4be-729d-4215-a18a-b2ebc34931cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c21b5c02eb19467bbf8f6f0161a7717e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:56.370114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabe4be-729d-4215-a18a-b2ebc34931cc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:56.425062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:56.428701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6420cd9-f081-4b35-a1a4-efc951c0f044 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:56.432535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6eabe4be-729d-4215-a18a-b2ebc34931cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c21b5c02eb19467bbf8f6f0161a7717e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.434000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af1297-47ed-494a-a44b-4c740b517f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:56.434648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af1297-47ed-494a-a44b-4c740b517f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:56.434942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af1297-47ed-494a-a44b-4c740b517f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:56.435261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af1297-47ed-494a-a44b-4c740b517f85 admin admin] PATH_INFO: `/v3/projects/2aa6b5734a214c1480de474d8008c59c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:56.436535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af1297-47ed-494a-a44b-4c740b517f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa6b5734a214c1480de474d8008c59c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:56.438696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af1297-47ed-494a-a44b-4c740b517f85 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:56.448531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6eabe4be-729d-4215-a18a-b2ebc34931cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c21b5c02eb19467bbf8f6f0161a7717e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.449279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6eabe4be-729d-4215-a18a-b2ebc34931cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c21b5c02eb19467bbf8f6f0161a7717e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.459866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6eabe4be-729d-4215-a18a-b2ebc34931cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c21b5c02eb19467bbf8f6f0161a7717e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.469712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6eabe4be-729d-4215-a18a-b2ebc34931cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c21b5c02eb19467bbf8f6f0161a7717e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.476596 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3713/7517] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:56 2022] DELETE /identity/v3/users/c21b5c02eb19467bbf8f6f0161a7717e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:56.498199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f0af1297-47ed-494a-a44b-4c740b517f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa6b5734a214c1480de474d8008c59c', 'request_id': 'req-f0af1297-47ed-494a-a44b-4c740b517f85'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.507070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f0af1297-47ed-494a-a44b-4c740b517f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa6b5734a214c1480de474d8008c59c', 'request_id': 'req-f0af1297-47ed-494a-a44b-4c740b517f85'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.514486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f0af1297-47ed-494a-a44b-4c740b517f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa6b5734a214c1480de474d8008c59c', 'request_id': 'req-f0af1297-47ed-494a-a44b-4c740b517f85'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.529493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f0af1297-47ed-494a-a44b-4c740b517f85 admin admin] The token cache is being invalidate because project 2aa6b5734a214c1480de474d8008c59c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:56.529975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f0af1297-47ed-494a-a44b-4c740b517f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.531305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f0af1297-47ed-494a-a44b-4c740b517f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:56.532864 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3805/7518] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:56 2022] DELETE /identity/v3/projects/2aa6b5734a214c1480de474d8008c59c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:56.539583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c26b5e-ea18-437f-a630-54e808c046e8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:56.539943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c26b5e-ea18-437f-a630-54e808c046e8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:56.540252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c26b5e-ea18-437f-a630-54e808c046e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:56.578065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-45c26b5e-ea18-437f-a630-54e808c046e8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:56.579457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-45c26b5e-ea18-437f-a630-54e808c046e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:56.652948 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3714/7519] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:56.658349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b7477-b89b-444c-bffb-cd2fbb8752b2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:56.658647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b7477-b89b-444c-bffb-cd2fbb8752b2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:56.658926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b7477-b89b-444c-bffb-cd2fbb8752b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:56.682391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-733b7477-b89b-444c-bffb-cd2fbb8752b2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:56.684105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-733b7477-b89b-444c-bffb-cd2fbb8752b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:56.694446 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3806/7520] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:56.709291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c26b5e-ea18-437f-a630-54e808c046e8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:56.717685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c26b5e-ea18-437f-a630-54e808c046e8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:56.723634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f820f69-199f-4109-9e1f-e18861205871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:56.724469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f820f69-199f-4109-9e1f-e18861205871 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:56.724762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f820f69-199f-4109-9e1f-e18861205871 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:56.725322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f820f69-199f-4109-9e1f-e18861205871 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:56.725776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f820f69-199f-4109-9e1f-e18861205871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:16:56.726119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f820f69-199f-4109-9e1f-e18861205871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:56.727899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f820f69-199f-4109-9e1f-e18861205871 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:56.742709 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3715/7521] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:56.752461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b7477-b89b-444c-bffb-cd2fbb8752b2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:56.763478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b7477-b89b-444c-bffb-cd2fbb8752b2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:56.769658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6bbb6-4a1a-4636-b246-e61569ee5e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:56.770381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6bbb6-4a1a-4636-b246-e61569ee5e4d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:56.770734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6bbb6-4a1a-4636-b246-e61569ee5e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:56.771081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6bbb6-4a1a-4636-b246-e61569ee5e4d admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:56.771833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6bbb6-4a1a-4636-b246-e61569ee5e4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:56.774236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6bbb6-4a1a-4636-b246-e61569ee5e4d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:56.818543 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3807/7522] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:56 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:56.824616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f820f69-199f-4109-9e1f-e18861205871 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:56.840765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f820f69-199f-4109-9e1f-e18861205871 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:56.841816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6bbb6-4a1a-4636-b246-e61569ee5e4d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:56.844578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-25f6bbb6-4a1a-4636-b246-e61569ee5e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:56.935033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:56.939862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777e5f9-f6e0-4062-8d5d-0b39c543bd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:56.940478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777e5f9-f6e0-4062-8d5d-0b39c543bd69 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:56.940839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777e5f9-f6e0-4062-8d5d-0b39c543bd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:56.941188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777e5f9-f6e0-4062-8d5d-0b39c543bd69 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:56.941694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777e5f9-f6e0-4062-8d5d-0b39c543bd69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:56.943780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777e5f9-f6e0-4062-8d5d-0b39c543bd69 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:56.953712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6bbb6-4a1a-4636-b246-e61569ee5e4d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:56.960303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05dc18-4a03-4682-ae96-3353fc599435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:56.961016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05dc18-4a03-4682-ae96-3353fc599435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:56.961265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05dc18-4a03-4682-ae96-3353fc599435 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:56.961564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05dc18-4a03-4682-ae96-3353fc599435 admin admin] PATH_INFO: `/v3/projects/fbd93b94d2364664961a7e2acabc07d1` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:56.962706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05dc18-4a03-4682-ae96-3353fc599435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbd93b94d2364664961a7e2acabc07d1)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:56.964676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05dc18-4a03-4682-ae96-3353fc599435 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:56.970609 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3716/7523] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:56 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:56.972352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777e5f9-f6e0-4062-8d5d-0b39c543bd69 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:56.983475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777e5f9-f6e0-4062-8d5d-0b39c543bd69 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:56.988225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145cf1a-23f8-4510-afce-80fbfec2838f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:56.988823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145cf1a-23f8-4510-afce-80fbfec2838f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:56.989153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145cf1a-23f8-4510-afce-80fbfec2838f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:56.989435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145cf1a-23f8-4510-afce-80fbfec2838f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:56.990298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145cf1a-23f8-4510-afce-80fbfec2838f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:56.992456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145cf1a-23f8-4510-afce-80fbfec2838f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.005745 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3717/7524] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:57.010085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145cf1a-23f8-4510-afce-80fbfec2838f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.015702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef05dc18-4a03-4682-ae96-3353fc599435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbd93b94d2364664961a7e2acabc07d1', 'request_id': 'req-ef05dc18-4a03-4682-ae96-3353fc599435'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:57.020095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145cf1a-23f8-4510-afce-80fbfec2838f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.023631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef05dc18-4a03-4682-ae96-3353fc599435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbd93b94d2364664961a7e2acabc07d1', 'request_id': 'req-ef05dc18-4a03-4682-ae96-3353fc599435'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:57.031346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef05dc18-4a03-4682-ae96-3353fc599435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbd93b94d2364664961a7e2acabc07d1', 'request_id': 'req-ef05dc18-4a03-4682-ae96-3353fc599435'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:57.031964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:57.044242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef05dc18-4a03-4682-ae96-3353fc599435 admin admin] The token cache is being invalidate because project fbd93b94d2364664961a7e2acabc07d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:57.044542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef05dc18-4a03-4682-ae96-3353fc599435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:57.045241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef05dc18-4a03-4682-ae96-3353fc599435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:57.046769 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3808/7525] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:56 2022] DELETE /identity/v3/projects/fbd93b94d2364664961a7e2acabc07d1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:57.051552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51007b-34cd-428a-8b48-d65b97598c20 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.051847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51007b-34cd-428a-8b48-d65b97598c20 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.052140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51007b-34cd-428a-8b48-d65b97598c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.072573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-be51007b-34cd-428a-8b48-d65b97598c20 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:57.074666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-be51007b-34cd-428a-8b48-d65b97598c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:57.086400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748b809-ed02-43ec-9dfd-580aa37e2e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.087292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748b809-ed02-43ec-9dfd-580aa37e2e3f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.087535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748b809-ed02-43ec-9dfd-580aa37e2e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.087815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748b809-ed02-43ec-9dfd-580aa37e2e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.088495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748b809-ed02-43ec-9dfd-580aa37e2e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.090140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748b809-ed02-43ec-9dfd-580aa37e2e3f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.103590 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3718/7526] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:57.104975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748b809-ed02-43ec-9dfd-580aa37e2e3f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.106656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0748b809-ed02-43ec-9dfd-580aa37e2e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:57.120190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748b809-ed02-43ec-9dfd-580aa37e2e3f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.125859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f11eb-a5bb-4639-826a-a65d83ee2ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.126533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f11eb-a5bb-4639-826a-a65d83ee2ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.126822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f11eb-a5bb-4639-826a-a65d83ee2ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.127104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f11eb-a5bb-4639-826a-a65d83ee2ac4 admin admin] PATH_INFO: `/v3/projects/1eb6ec9b452e45f5a22907ac7de9e1f4/users/86c03017bec445168753d973e2b6c59d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.143565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f11eb-a5bb-4639-826a-a65d83ee2ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eb6ec9b452e45f5a22907ac7de9e1f4, user_id=86c03017bec445168753d973e2b6c59d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.144136 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3809/7527] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:57.146347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f11eb-a5bb-4639-826a-a65d83ee2ac4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.152444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c6665-8cbd-4e0a-9ea6-27e995b44044 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.152879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c6665-8cbd-4e0a-9ea6-27e995b44044 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.153321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c6665-8cbd-4e0a-9ea6-27e995b44044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.160360 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3719/7528] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:57 2022] PUT /identity/v3/projects/1eb6ec9b452e45f5a22907ac7de9e1f4/users/86c03017bec445168753d973e2b6c59d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:57.170817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8859c-d8d8-40e5-80e0-0263c72b9573 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.171138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8859c-d8d8-40e5-80e0-0263c72b9573 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.171421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8859c-d8d8-40e5-80e0-0263c72b9573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.175361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f71c6665-8cbd-4e0a-9ea6-27e995b44044 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:57.176600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f71c6665-8cbd-4e0a-9ea6-27e995b44044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:57.199997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5ba8859c-d8d8-40e5-80e0-0263c72b9573 None None] MFA Rules not processed for user `86c03017bec445168753d973e2b6c59d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:57.201270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5ba8859c-d8d8-40e5-80e0-0263c72b9573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:57.238483 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3810/7529] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:57.242733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c6665-8cbd-4e0a-9ea6-27e995b44044 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.252467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c6665-8cbd-4e0a-9ea6-27e995b44044 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.257320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499844f3-5e22-4730-806e-7de368d6ceaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.257921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499844f3-5e22-4730-806e-7de368d6ceaa admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.258312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499844f3-5e22-4730-806e-7de368d6ceaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.258510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499844f3-5e22-4730-806e-7de368d6ceaa admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.259275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499844f3-5e22-4730-806e-7de368d6ceaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:16:57.259582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499844f3-5e22-4730-806e-7de368d6ceaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.261217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499844f3-5e22-4730-806e-7de368d6ceaa admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.269477 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3720/7530] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:57.276068 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3811/7531] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:16:57.276214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8859c-d8d8-40e5-80e0-0263c72b9573 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.277355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499844f3-5e22-4730-806e-7de368d6ceaa admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.282737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8859c-d8d8-40e5-80e0-0263c72b9573 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.287019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28107aa-962e-4df2-925d-019c814e22ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c03017bec445168753d973e2b6c59d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.287182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499844f3-5e22-4730-806e-7de368d6ceaa admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.287655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28107aa-962e-4df2-925d-019c814e22ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.287993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28107aa-962e-4df2-925d-019c814e22ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.288229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28107aa-962e-4df2-925d-019c814e22ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] PATH_INFO: `/v3/users/86c03017bec445168753d973e2b6c59d/application_credentials` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.288932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28107aa-962e-4df2-925d-019c814e22ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorizing `identity:create_application_credential(user_id=86c03017bec445168753d973e2b6c59d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.290660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28107aa-962e-4df2-925d-019c814e22ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.292698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac470e-1718-4cc1-ac78-0ee2456a9af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.293325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac470e-1718-4cc1-ac78-0ee2456a9af0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.293614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac470e-1718-4cc1-ac78-0ee2456a9af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.293894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac470e-1718-4cc1-ac78-0ee2456a9af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.294579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac470e-1718-4cc1-ac78-0ee2456a9af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.297281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac470e-1718-4cc1-ac78-0ee2456a9af0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.301848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:57.327358 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3812/7532] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:57 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:57.331524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac470e-1718-4cc1-ac78-0ee2456a9af0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.375359 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3721/7533] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Sep 3 21:16:57 2022] POST /identity/v3/users/86c03017bec445168753d973e2b6c59d/application_credentials => generated 921 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:57.379707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28107aa-962e-4df2-925d-019c814e22ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.384490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:57.390023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28107aa-962e-4df2-925d-019c814e22ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.395772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac470e-1718-4cc1-ac78-0ee2456a9af0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.400648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c3aca-52f2-4789-ae80-cd786cb67351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.401302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c3aca-52f2-4789-ae80-cd786cb67351 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.401591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c3aca-52f2-4789-ae80-cd786cb67351 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.401867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c3aca-52f2-4789-ae80-cd786cb67351 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.402535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c3aca-52f2-4789-ae80-cd786cb67351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.404505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c3aca-52f2-4789-ae80-cd786cb67351 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.427104 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3813/7534] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:57 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:57.431634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c3aca-52f2-4789-ae80-cd786cb67351 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.442732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c3aca-52f2-4789-ae80-cd786cb67351 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.447323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e324e9-a921-48b1-8fe8-ac9abeeab20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.447629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb38d37-7553-4e1c-8e7c-47b41b0b5f6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c03017bec445168753d973e2b6c59d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.447908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e324e9-a921-48b1-8fe8-ac9abeeab20a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.448189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e324e9-a921-48b1-8fe8-ac9abeeab20a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.448404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb38d37-7553-4e1c-8e7c-47b41b0b5f6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.448512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e324e9-a921-48b1-8fe8-ac9abeeab20a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.448731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb38d37-7553-4e1c-8e7c-47b41b0b5f6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.449057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb38d37-7553-4e1c-8e7c-47b41b0b5f6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] PATH_INFO: `/v3/users/86c03017bec445168753d973e2b6c59d/access_rules/0ff2280c46db4f18b8492ef196e101e2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.449179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e324e9-a921-48b1-8fe8-ac9abeeab20a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.450310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb38d37-7553-4e1c-8e7c-47b41b0b5f6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorizing `identity:delete_access_rule(user_id=86c03017bec445168753d973e2b6c59d, access_rule_id=0ff2280c46db4f18b8492ef196e101e2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.450684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e324e9-a921-48b1-8fe8-ac9abeeab20a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.451642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb38d37-7553-4e1c-8e7c-47b41b0b5f6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.460560 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3814/7535] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:57.464405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e324e9-a921-48b1-8fe8-ac9abeeab20a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.474441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e324e9-a921-48b1-8fe8-ac9abeeab20a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.480041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e51397-0b45-49e9-b7d1-8db072b05a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.480622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e51397-0b45-49e9-b7d1-8db072b05a20 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.480926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e51397-0b45-49e9-b7d1-8db072b05a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.481212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e51397-0b45-49e9-b7d1-8db072b05a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.481843 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e51397-0b45-49e9-b7d1-8db072b05a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.483425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e51397-0b45-49e9-b7d1-8db072b05a20 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.494386 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3815/7536] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:16:57.498718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e51397-0b45-49e9-b7d1-8db072b05a20 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-ebb38d37-7553-4e1c-8e7c-47b41b0b5f6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] May not delete access rule in use: keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 256, in delete_access_rule Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application session.delete(ref) Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application next(self.gen) Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application yield resource Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application next(self.gen) Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.session.rollback() Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.force_reraise() Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 03 21:16:57.504767 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise self.value Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application session.commit() Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._prepare_impl() Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.session.flush() Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._flush(objects) Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 03 21:16:57.505844 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application compat.raise_( Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application flush_context.execute() Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rec.execute(self) Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 577, in execute Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.dependency_processor.process_deletes(uow, states) Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 552, in process_deletes Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._synchronize( Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 610, in _synchronize Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application sync.clear(dest, self.mapper, self.prop.synchronize_pairs) Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/sync.py", line 86, in clear Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise AssertionError( Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application AssertionError: Dependency rule tried to blank-out primary key column 'application_credential_access_rule.access_rule_id' on instance '' Sep 03 21:16:57.506810 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 763, in delete Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application PROVIDERS.application_credential_api.delete_access_rule( Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:16:57.507764 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:16:57.508800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/core.py", line 252, in delete_access_rule Sep 03 21:16:57.508800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.driver.delete_access_rule(access_rule_id) Sep 03 21:16:57.508800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 258, in delete_access_rule Sep 03 21:16:57.508800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Sep 03 21:16:57.508800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Sep 03 21:16:57.508800 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:16:57.508800 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3722/7537] 10.222.0.16 () {58 vars in 1362 bytes} [Sat Sep 3 21:16:57 2022] DELETE /identity/v3/users/86c03017bec445168753d973e2b6c59d/access_rules/0ff2280c46db4f18b8492ef196e101e2 => generated 89 bytes in 127 msecs (HTTP/1.1 403) 5 headers in 183 bytes (2 switches on core 0) Sep 03 21:16:57.509288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e51397-0b45-49e9-b7d1-8db072b05a20 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.510224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb38d37-7553-4e1c-8e7c-47b41b0b5f6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.513940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff690e-b6f2-4258-a680-e7280170d9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.514589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff690e-b6f2-4258-a680-e7280170d9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.514901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff690e-b6f2-4258-a680-e7280170d9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.515176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff690e-b6f2-4258-a680-e7280170d9a2 admin admin] PATH_INFO: `/v3/projects/fd74791289004afbbdb10b805e0b9cc6/users/0e89b58490c84e64bc1d0bbd51aed5a9/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.519473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb38d37-7553-4e1c-8e7c-47b41b0b5f6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.523704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d339d-96ce-4b73-a1bc-808270c5e68c tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c03017bec445168753d973e2b6c59d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.524826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d339d-96ce-4b73-a1bc-808270c5e68c tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.525144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d339d-96ce-4b73-a1bc-808270c5e68c tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.525417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d339d-96ce-4b73-a1bc-808270c5e68c tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] PATH_INFO: `/v3/users/86c03017bec445168753d973e2b6c59d/application_credentials/f018f4d94645488aa40d9e92004ef998` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.530382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff690e-b6f2-4258-a680-e7280170d9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd74791289004afbbdb10b805e0b9cc6, user_id=0e89b58490c84e64bc1d0bbd51aed5a9, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.533194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff690e-b6f2-4258-a680-e7280170d9a2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.537288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d339d-96ce-4b73-a1bc-808270c5e68c tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorizing `identity:delete_application_credential(user_id=86c03017bec445168753d973e2b6c59d, application_credential_id=f018f4d94645488aa40d9e92004ef998)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.538754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d339d-96ce-4b73-a1bc-808270c5e68c tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.548298 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3816/7538] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:57 2022] PUT /identity/v3/projects/fd74791289004afbbdb10b805e0b9cc6/users/0e89b58490c84e64bc1d0bbd51aed5a9/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:57.561002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73ac0-b74f-40c7-b2f9-5904797e8a21 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.561358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73ac0-b74f-40c7-b2f9-5904797e8a21 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.561651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73ac0-b74f-40c7-b2f9-5904797e8a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.562049 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3723/7539] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Sep 3 21:16:57 2022] DELETE /identity/v3/users/86c03017bec445168753d973e2b6c59d/application_credentials/f018f4d94645488aa40d9e92004ef998 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:57.565906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d339d-96ce-4b73-a1bc-808270c5e68c tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.576512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d339d-96ce-4b73-a1bc-808270c5e68c tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.589944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9ef73ac0-b74f-40c7-b2f9-5904797e8a21 None None] MFA Rules not processed for user `0e89b58490c84e64bc1d0bbd51aed5a9`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:57.591197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9ef73ac0-b74f-40c7-b2f9-5904797e8a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:57.631195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceacae7-1d11-4c53-8c1e-751205c23116 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c03017bec445168753d973e2b6c59d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.631851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceacae7-1d11-4c53-8c1e-751205c23116 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.632147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceacae7-1d11-4c53-8c1e-751205c23116 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.632437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceacae7-1d11-4c53-8c1e-751205c23116 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] PATH_INFO: `/v3/users/86c03017bec445168753d973e2b6c59d/access_rules` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.633696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceacae7-1d11-4c53-8c1e-751205c23116 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorizing `identity:list_access_rules(user_id=86c03017bec445168753d973e2b6c59d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.635187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceacae7-1d11-4c53-8c1e-751205c23116 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.640234 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3724/7540] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Sep 3 21:16:57 2022] GET /identity/v3/users/86c03017bec445168753d973e2b6c59d/access_rules => generated 369 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:16:57.644210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceacae7-1d11-4c53-8c1e-751205c23116 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.653710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceacae7-1d11-4c53-8c1e-751205c23116 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.658064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa2eaf-8a28-4ac4-b455-eab341675feb tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c03017bec445168753d973e2b6c59d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.658460 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3817/7541] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:16:57.658694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa2eaf-8a28-4ac4-b455-eab341675feb tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.658987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa2eaf-8a28-4ac4-b455-eab341675feb tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.659267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa2eaf-8a28-4ac4-b455-eab341675feb tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] PATH_INFO: `/v3/users/86c03017bec445168753d973e2b6c59d/access_rules/0ff2280c46db4f18b8492ef196e101e2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.660396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa2eaf-8a28-4ac4-b455-eab341675feb tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorizing `identity:delete_access_rule(user_id=86c03017bec445168753d973e2b6c59d, access_rule_id=0ff2280c46db4f18b8492ef196e101e2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.661923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa2eaf-8a28-4ac4-b455-eab341675feb tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.670434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef73ac0-b74f-40c7-b2f9-5904797e8a21 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.672554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9ef73ac0-b74f-40c7-b2f9-5904797e8a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:57.679270 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3725/7542] 10.222.0.16 () {58 vars in 1362 bytes} [Sat Sep 3 21:16:57 2022] DELETE /identity/v3/users/86c03017bec445168753d973e2b6c59d/access_rules/0ff2280c46db4f18b8492ef196e101e2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:57.683370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa2eaf-8a28-4ac4-b455-eab341675feb tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.693275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa2eaf-8a28-4ac4-b455-eab341675feb tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.694590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:57.699762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc83f5-e435-4668-8379-30a6df75a2fe tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c03017bec445168753d973e2b6c59d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.700508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc83f5-e435-4668-8379-30a6df75a2fe tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.700734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc83f5-e435-4668-8379-30a6df75a2fe tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.701050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc83f5-e435-4668-8379-30a6df75a2fe tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] PATH_INFO: `/v3/users/86c03017bec445168753d973e2b6c59d/access_rules` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.702431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc83f5-e435-4668-8379-30a6df75a2fe tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorizing `identity:list_access_rules(user_id=86c03017bec445168753d973e2b6c59d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.704005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc83f5-e435-4668-8379-30a6df75a2fe tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.709492 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3726/7543] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Sep 3 21:16:57 2022] GET /identity/v3/users/86c03017bec445168753d973e2b6c59d/access_rules => generated 158 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:57.714998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc83f5-e435-4668-8379-30a6df75a2fe tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.724584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc83f5-e435-4668-8379-30a6df75a2fe tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.731377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8035f-c8d5-455b-a7b4-1677cfd6da6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c03017bec445168753d973e2b6c59d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.732046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8035f-c8d5-455b-a7b4-1677cfd6da6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.732340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8035f-c8d5-455b-a7b4-1677cfd6da6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.732622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8035f-c8d5-455b-a7b4-1677cfd6da6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] PATH_INFO: `/v3/users/86c03017bec445168753d973e2b6c59d/application_credentials` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.733682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8035f-c8d5-455b-a7b4-1677cfd6da6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorizing `identity:create_application_credential(user_id=86c03017bec445168753d973e2b6c59d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.734915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8035f-c8d5-455b-a7b4-1677cfd6da6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.755858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef73ac0-b74f-40c7-b2f9-5904797e8a21 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.759592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae08c2c-af87-48cc-86e7-d57ed6003d3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.760322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae08c2c-af87-48cc-86e7-d57ed6003d3b service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.760476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae08c2c-af87-48cc-86e7-d57ed6003d3b service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.760773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae08c2c-af87-48cc-86e7-d57ed6003d3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.769951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae08c2c-af87-48cc-86e7-d57ed6003d3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.772003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae08c2c-af87-48cc-86e7-d57ed6003d3b service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.783377 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3818/7544] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:16:57.785640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:57.848944 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3727/7545] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Sep 3 21:16:57 2022] POST /identity/v3/users/86c03017bec445168753d973e2b6c59d/application_credentials => generated 921 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:57.850803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae08c2c-af87-48cc-86e7-d57ed6003d3b service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.862831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae08c2c-af87-48cc-86e7-d57ed6003d3b service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.867194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929858a-50d9-4244-8a51-4183804d43ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c03017bec445168753d973e2b6c59d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.868020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929858a-50d9-4244-8a51-4183804d43ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.868312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929858a-50d9-4244-8a51-4183804d43ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.868590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929858a-50d9-4244-8a51-4183804d43ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] PATH_INFO: `/v3/users/86c03017bec445168753d973e2b6c59d/access_rules` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.869818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929858a-50d9-4244-8a51-4183804d43ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorizing `identity:list_access_rules(user_id=86c03017bec445168753d973e2b6c59d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.871310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929858a-50d9-4244-8a51-4183804d43ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:57.876793 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3819/7546] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Sep 3 21:16:57 2022] GET /identity/v3/users/86c03017bec445168753d973e2b6c59d/access_rules => generated 369 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:57.882175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8035f-c8d5-455b-a7b4-1677cfd6da6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:57.893441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8035f-c8d5-455b-a7b4-1677cfd6da6d tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:57.900290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6ea97-1dcc-4b35-a013-a26546cbbdef tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c03017bec445168753d973e2b6c59d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:57.901539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6ea97-1dcc-4b35-a013-a26546cbbdef tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:57.901840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6ea97-1dcc-4b35-a013-a26546cbbdef tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:57.902125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6ea97-1dcc-4b35-a013-a26546cbbdef tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] PATH_INFO: `/v3/users/86c03017bec445168753d973e2b6c59d/application_credentials` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:57.902844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6ea97-1dcc-4b35-a013-a26546cbbdef tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorizing `identity:create_application_credential(user_id=86c03017bec445168753d973e2b6c59d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:57.904012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6ea97-1dcc-4b35-a013-a26546cbbdef tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:58.020796 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3728/7547] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Sep 3 21:16:57 2022] POST /identity/v3/users/86c03017bec445168753d973e2b6c59d/application_credentials => generated 921 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:58.025826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929858a-50d9-4244-8a51-4183804d43ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:58.036194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929858a-50d9-4244-8a51-4183804d43ff tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:58.044245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f08b2-0af6-4454-be10-70e89bfe1898 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c03017bec445168753d973e2b6c59d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:58.045440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f08b2-0af6-4454-be10-70e89bfe1898 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:58.045440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f08b2-0af6-4454-be10-70e89bfe1898 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:58.045659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f08b2-0af6-4454-be10-70e89bfe1898 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] PATH_INFO: `/v3/users/86c03017bec445168753d973e2b6c59d/access_rules/b2a4f5a62b124860be271a196bb02168` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:58.047108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f08b2-0af6-4454-be10-70e89bfe1898 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorizing `identity:get_access_rule(user_id=86c03017bec445168753d973e2b6c59d, access_rule_id=b2a4f5a62b124860be271a196bb02168)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:58.048979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f08b2-0af6-4454-be10-70e89bfe1898 tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:58.057914 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3820/7548] 10.222.0.16 () {58 vars in 1359 bytes} [Sat Sep 3 21:16:58 2022] GET /identity/v3/users/86c03017bec445168753d973e2b6c59d/access_rules/b2a4f5a62b124860be271a196bb02168 => generated 229 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:16:58.062976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6ea97-1dcc-4b35-a013-a26546cbbdef tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:58.167756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6ea97-1dcc-4b35-a013-a26546cbbdef tempest-AccessRulesV3Test-1829558311 tempest-AccessRulesV3Test-1829558311-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:58.173985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:58.174641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:58.174931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:58.175214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] PATH_INFO: `/v3/users/86c03017bec445168753d973e2b6c59d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:58.176393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86c03017bec445168753d973e2b6c59d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:58.178419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:58.252790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86c03017bec445168753d973e2b6c59d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:58.262140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:58.298007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86c03017bec445168753d973e2b6c59d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:58.298729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86c03017bec445168753d973e2b6c59d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:58.306461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86c03017bec445168753d973e2b6c59d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:58.313951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86c03017bec445168753d973e2b6c59d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:58.319992 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3729/7549] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:58 2022] DELETE /identity/v3/users/86c03017bec445168753d973e2b6c59d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:58.325075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f935f-1eec-4d21-af93-22e0320a88a9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:58.325461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f935f-1eec-4d21-af93-22e0320a88a9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:58.325744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f935f-1eec-4d21-af93-22e0320a88a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:58.344881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8c1f935f-1eec-4d21-af93-22e0320a88a9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:58.346171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8c1f935f-1eec-4d21-af93-22e0320a88a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:58.408216 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3821/7550] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:58.417956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:58.419373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:58.479592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829644b3-f150-42a5-b430-0218c58123e6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:58.483817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0ade8-3419-427d-8c24-e5e8c8749571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:58.484436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0ade8-3419-427d-8c24-e5e8c8749571 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:58.484725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0ade8-3419-427d-8c24-e5e8c8749571 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:58.485044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0ade8-3419-427d-8c24-e5e8c8749571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:58.495010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0ade8-3419-427d-8c24-e5e8c8749571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:58.496541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0ade8-3419-427d-8c24-e5e8c8749571 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:58.506414 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3730/7551] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:16:58.926641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f935f-1eec-4d21-af93-22e0320a88a9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:58.928888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8c1f935f-1eec-4d21-af93-22e0320a88a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:58.943712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f935f-1eec-4d21-af93-22e0320a88a9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:58.948672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2136736-6257-4198-a895-6e9408b09fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:58.949352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2136736-6257-4198-a895-6e9408b09fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:58.949637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2136736-6257-4198-a895-6e9408b09fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:58.949922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2136736-6257-4198-a895-6e9408b09fa4 admin admin] PATH_INFO: `/v3/projects/1eb6ec9b452e45f5a22907ac7de9e1f4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:58.951057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2136736-6257-4198-a895-6e9408b09fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eb6ec9b452e45f5a22907ac7de9e1f4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:58.953202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2136736-6257-4198-a895-6e9408b09fa4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:59.074536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2136736-6257-4198-a895-6e9408b09fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'request_id': 'req-d2136736-6257-4198-a895-6e9408b09fa4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:59.085522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2136736-6257-4198-a895-6e9408b09fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'request_id': 'req-d2136736-6257-4198-a895-6e9408b09fa4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:59.099158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2136736-6257-4198-a895-6e9408b09fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eb6ec9b452e45f5a22907ac7de9e1f4', 'request_id': 'req-d2136736-6257-4198-a895-6e9408b09fa4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:59.115364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2136736-6257-4198-a895-6e9408b09fa4 admin admin] The token cache is being invalidate because project 1eb6ec9b452e45f5a22907ac7de9e1f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:16:59.115901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2136736-6257-4198-a895-6e9408b09fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:59.116731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2136736-6257-4198-a895-6e9408b09fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:59.118556 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3822/7552] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:16:58 2022] DELETE /identity/v3/projects/1eb6ec9b452e45f5a22907ac7de9e1f4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:59.124721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3eb45-4ca1-4628-96d5-10fe3c694ac5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.125288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3eb45-4ca1-4628-96d5-10fe3c694ac5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.125540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3eb45-4ca1-4628-96d5-10fe3c694ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.150417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-20b3eb45-4ca1-4628-96d5-10fe3c694ac5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:59.151670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-20b3eb45-4ca1-4628-96d5-10fe3c694ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:59.223333 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3731/7553] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:59.225903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2ae20-e54c-4804-b773-004fb121aff4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.226197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2ae20-e54c-4804-b773-004fb121aff4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.226477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2ae20-e54c-4804-b773-004fb121aff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.254755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-38d2ae20-e54c-4804-b773-004fb121aff4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:59.256501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-38d2ae20-e54c-4804-b773-004fb121aff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:59.269818 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3823/7554] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:59.270650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3eb45-4ca1-4628-96d5-10fe3c694ac5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:59.282431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3eb45-4ca1-4628-96d5-10fe3c694ac5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:59.297783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d88a90-974c-4399-802b-3de6b0fda3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:59.298890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d88a90-974c-4399-802b-3de6b0fda3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.299183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d88a90-974c-4399-802b-3de6b0fda3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.299462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d88a90-974c-4399-802b-3de6b0fda3a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.300445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d88a90-974c-4399-802b-3de6b0fda3a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:16:59.301347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d88a90-974c-4399-802b-3de6b0fda3a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:59.304137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d88a90-974c-4399-802b-3de6b0fda3a7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:59.324882 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3732/7555] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:16:59.325199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2ae20-e54c-4804-b773-004fb121aff4 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:59.341681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2ae20-e54c-4804-b773-004fb121aff4 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:59.351743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d934c2a-b40b-4281-b1f3-48df3db779e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:59.353546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d934c2a-b40b-4281-b1f3-48df3db779e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.353546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d934c2a-b40b-4281-b1f3-48df3db779e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.353865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d934c2a-b40b-4281-b1f3-48df3db779e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.354493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d934c2a-b40b-4281-b1f3-48df3db779e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:59.357035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d934c2a-b40b-4281-b1f3-48df3db779e0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:59.385689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:59.408485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d88a90-974c-4399-802b-3de6b0fda3a7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:59.416269 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3824/7556] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:16:59 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:16:59.421286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d88a90-974c-4399-802b-3de6b0fda3a7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:59.489297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:16:59.517511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0358c2-23b1-4c19-b4b9-c6be89c9a48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:59.518479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d934c2a-b40b-4281-b1f3-48df3db779e0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:59.520287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9d934c2a-b40b-4281-b1f3-48df3db779e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:59.521222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0358c2-23b1-4c19-b4b9-c6be89c9a48e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.521701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0358c2-23b1-4c19-b4b9-c6be89c9a48e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.521805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0358c2-23b1-4c19-b4b9-c6be89c9a48e admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.522581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0358c2-23b1-4c19-b4b9-c6be89c9a48e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:59.525195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0358c2-23b1-4c19-b4b9-c6be89c9a48e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:59.538481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d934c2a-b40b-4281-b1f3-48df3db779e0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:59.545053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54627faf-d468-4fe4-8ed0-8aa75de80a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:59.546344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627faf-d468-4fe4-8ed0-8aa75de80a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.546641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627faf-d468-4fe4-8ed0-8aa75de80a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.546923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627faf-d468-4fe4-8ed0-8aa75de80a00 admin admin] PATH_INFO: `/v3/users/0e89b58490c84e64bc1d0bbd51aed5a9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.548137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54627faf-d468-4fe4-8ed0-8aa75de80a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e89b58490c84e64bc1d0bbd51aed5a9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:59.550248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54627faf-d468-4fe4-8ed0-8aa75de80a00 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:59.552060 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3733/7557] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:16:59 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:59.557175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0358c2-23b1-4c19-b4b9-c6be89c9a48e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:59.572936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0358c2-23b1-4c19-b4b9-c6be89c9a48e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:59.579487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fd8c5-0f55-49bb-8755-d917835bfadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:59.580437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fd8c5-0f55-49bb-8755-d917835bfadc admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.580933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fd8c5-0f55-49bb-8755-d917835bfadc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.581396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fd8c5-0f55-49bb-8755-d917835bfadc admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.582234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6fd8c5-0f55-49bb-8755-d917835bfadc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:59.584647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6fd8c5-0f55-49bb-8755-d917835bfadc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:59.599231 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3734/7558] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:59.605430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fd8c5-0f55-49bb-8755-d917835bfadc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:59.622533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fd8c5-0f55-49bb-8755-d917835bfadc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:59.628844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe9ea4-21a8-48c9-86b9-df608a14e0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:59.629744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe9ea4-21a8-48c9-86b9-df608a14e0ce admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.630253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe9ea4-21a8-48c9-86b9-df608a14e0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.630440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe9ea4-21a8-48c9-86b9-df608a14e0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.631173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe9ea4-21a8-48c9-86b9-df608a14e0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:59.640794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe9ea4-21a8-48c9-86b9-df608a14e0ce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:59.657880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-54627faf-d468-4fe4-8ed0-8aa75de80a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e89b58490c84e64bc1d0bbd51aed5a9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:59.669264 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3735/7559] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:16:59.670213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe9ea4-21a8-48c9-86b9-df608a14e0ce admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:59.679392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-54627faf-d468-4fe4-8ed0-8aa75de80a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e89b58490c84e64bc1d0bbd51aed5a9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:59.685058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe9ea4-21a8-48c9-86b9-df608a14e0ce admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:59.691119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-54627faf-d468-4fe4-8ed0-8aa75de80a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e89b58490c84e64bc1d0bbd51aed5a9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:59.692441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08392a0-fa38-4f55-98ee-0d9c137223e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:59.693033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-54627faf-d468-4fe4-8ed0-8aa75de80a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e89b58490c84e64bc1d0bbd51aed5a9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:59.693331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08392a0-fa38-4f55-98ee-0d9c137223e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.693814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08392a0-fa38-4f55-98ee-0d9c137223e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.694174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08392a0-fa38-4f55-98ee-0d9c137223e8 admin admin] PATH_INFO: `/v3/projects/d022688b439748e3a289af5b98d37f20/users/ca0ef8c187d04478bb02c2217f8d894d/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.694396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-54627faf-d468-4fe4-8ed0-8aa75de80a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e89b58490c84e64bc1d0bbd51aed5a9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:16:59.714408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08392a0-fa38-4f55-98ee-0d9c137223e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d022688b439748e3a289af5b98d37f20, user_id=ca0ef8c187d04478bb02c2217f8d894d, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:59.715005 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3825/7560] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:16:59 2022] DELETE /identity/v3/users/0e89b58490c84e64bc1d0bbd51aed5a9 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:16:59.718056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08392a0-fa38-4f55-98ee-0d9c137223e8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:59.719927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c7342-4320-4879-94e2-0c6659c5cc00 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.720392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c7342-4320-4879-94e2-0c6659c5cc00 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.720852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c7342-4320-4879-94e2-0c6659c5cc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.744970 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3736/7561] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:16:59 2022] PUT /identity/v3/projects/d022688b439748e3a289af5b98d37f20/users/ca0ef8c187d04478bb02c2217f8d894d/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:16:59.755475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1d4cb-f04a-4d5f-a1df-3935d3206083 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.758065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1d4cb-f04a-4d5f-a1df-3935d3206083 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.760442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2f9c7342-4320-4879-94e2-0c6659c5cc00 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:59.761820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2f9c7342-4320-4879-94e2-0c6659c5cc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:59.766544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1d4cb-f04a-4d5f-a1df-3935d3206083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.809119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7ab1d4cb-f04a-4d5f-a1df-3935d3206083 None None] MFA Rules not processed for user `ca0ef8c187d04478bb02c2217f8d894d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:16:59.810413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7ab1d4cb-f04a-4d5f-a1df-3935d3206083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:59.871900 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3826/7562] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:59.893204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c7342-4320-4879-94e2-0c6659c5cc00 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:59.897072 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3737/7563] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:16:59.897210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2f9c7342-4320-4879-94e2-0c6659c5cc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:16:59.898384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1d4cb-f04a-4d5f-a1df-3935d3206083 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:59.917004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1d4cb-f04a-4d5f-a1df-3935d3206083 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:59.923065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9c5e1-e61c-4bc3-9db3-a396d3777198 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:59.923881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9c5e1-e61c-4bc3-9db3-a396d3777198 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.924177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9c5e1-e61c-4bc3-9db3-a396d3777198 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.924603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9c5e1-e61c-4bc3-9db3-a396d3777198 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.925311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9c5e1-e61c-4bc3-9db3-a396d3777198 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=ca0ef8c187d04478bb02c2217f8d894d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:59.926834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9c5e1-e61c-4bc3-9db3-a396d3777198 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:59.950260 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3738/7564] 10.222.0.16 () {60 vars in 1299 bytes} [Sat Sep 3 21:16:59 2022] POST /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2 => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:16:59.954246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9c5e1-e61c-4bc3-9db3-a396d3777198 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:16:59.966330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9c5e1-e61c-4bc3-9db3-a396d3777198 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:59.971296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a968e-1d1e-4cba-a87d-8aa674fbdd3f tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:59.972373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a968e-1d1e-4cba-a87d-8aa674fbdd3f tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.972667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a968e-1d1e-4cba-a87d-8aa674fbdd3f tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.973044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a968e-1d1e-4cba-a87d-8aa674fbdd3f tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/f8eb2e1eb91a4ed7b4827a0f9170e140` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:16:59.975609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a968e-1d1e-4cba-a87d-8aa674fbdd3f tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=ca0ef8c187d04478bb02c2217f8d894d, credential_id=f8eb2e1eb91a4ed7b4827a0f9170e140)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:16:59.977144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a968e-1d1e-4cba-a87d-8aa674fbdd3f tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:16:59.989084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c7342-4320-4879-94e2-0c6659c5cc00 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:16:59.996857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e49ffb-4032-4185-ab25-bc6faaf36508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:16:59.997711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e49ffb-4032-4185-ab25-bc6faaf36508 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:16:59.998632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e49ffb-4032-4185-ab25-bc6faaf36508 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:16:59.999021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e49ffb-4032-4185-ab25-bc6faaf36508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.005210 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3739/7565] 10.222.0.16 () {58 vars in 1380 bytes} [Sat Sep 3 21:16:59 2022] DELETE /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/f8eb2e1eb91a4ed7b4827a0f9170e140 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:00.010493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a968e-1d1e-4cba-a87d-8aa674fbdd3f tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.015041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e49ffb-4032-4185-ab25-bc6faaf36508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.016825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e49ffb-4032-4185-ab25-bc6faaf36508 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.027890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a968e-1d1e-4cba-a87d-8aa674fbdd3f tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.030067 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3827/7566] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:00.034878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5c62a-d613-4884-9835-7f9f5add642a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.035998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5c62a-d613-4884-9835-7f9f5add642a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.036294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5c62a-d613-4884-9835-7f9f5add642a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.036639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5c62a-d613-4884-9835-7f9f5add642a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.037367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5c62a-d613-4884-9835-7f9f5add642a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=ca0ef8c187d04478bb02c2217f8d894d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.038903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5c62a-d613-4884-9835-7f9f5add642a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.060653 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3740/7567] 10.222.0.16 () {60 vars in 1299 bytes} [Sat Sep 3 21:17:00 2022] POST /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2 => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:00.065039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e49ffb-4032-4185-ab25-bc6faaf36508 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.085784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e49ffb-4032-4185-ab25-bc6faaf36508 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.092499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f94da6-83c2-4247-8747-d5d12da49776 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.093760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f94da6-83c2-4247-8747-d5d12da49776 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.093760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f94da6-83c2-4247-8747-d5d12da49776 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.094293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f94da6-83c2-4247-8747-d5d12da49776 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/9f6765583c9c43ab9b19149354af2007` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.097685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f94da6-83c2-4247-8747-d5d12da49776 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=ca0ef8c187d04478bb02c2217f8d894d, credential_id=9f6765583c9c43ab9b19149354af2007)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.099571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f94da6-83c2-4247-8747-d5d12da49776 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.120531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:00.125522 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3828/7568] 10.222.0.16 () {58 vars in 1380 bytes} [Sat Sep 3 21:17:00 2022] DELETE /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/9f6765583c9c43ab9b19149354af2007 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:00.129806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5c62a-d613-4884-9835-7f9f5add642a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.151344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5c62a-d613-4884-9835-7f9f5add642a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.155704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604c20b-283c-4305-9dab-f3a0d3f0b525 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.156928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604c20b-283c-4305-9dab-f3a0d3f0b525 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.156928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604c20b-283c-4305-9dab-f3a0d3f0b525 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.157126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604c20b-283c-4305-9dab-f3a0d3f0b525 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/9f6765583c9c43ab9b19149354af2007` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-8604c20b-283c-4305-9dab-f3a0d3f0b525 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Could not find credential: fb527f098338e903710c5e63bc975b3c858b19963f5276dbedf4122bce604b77.: keystone.exception.CredentialNotFound: Could not find credential: fb527f098338e903710c5e63bc975b3c858b19963f5276dbedf4122bce604b77. Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 426, in get Sep 03 21:17:00.184782 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 119, in _build_enforcer_target_data_owner_and_user_id_match Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref['credential'] = PROVIDERS.credential_api.get_credential( Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 119, in get_credential Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application credential = self._get_credential(credential_id) Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:17:00.185856 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 124, in _get_credential Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.driver.get_credential(credential_id) Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 88, in get_credential Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_credential(session, credential_id).to_dict() Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 83, in _get_credential Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.CredentialNotFound(credential_id=credential_id) Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.CredentialNotFound: Could not find credential: fb527f098338e903710c5e63bc975b3c858b19963f5276dbedf4122bce604b77. Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:00.187790 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3741/7569] 10.222.0.16 () {58 vars in 1377 bytes} [Sat Sep 3 21:17:00 2022] GET /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/9f6765583c9c43ab9b19149354af2007 => generated 148 bytes in 57 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:17:00.191730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f94da6-83c2-4247-8747-d5d12da49776 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.202004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f94da6-83c2-4247-8747-d5d12da49776 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.206614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09365a3-3897-407a-a2ec-53a3a877c17a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.207297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09365a3-3897-407a-a2ec-53a3a877c17a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.207587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09365a3-3897-407a-a2ec-53a3a877c17a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.207869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09365a3-3897-407a-a2ec-53a3a877c17a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.208503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09365a3-3897-407a-a2ec-53a3a877c17a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=ca0ef8c187d04478bb02c2217f8d894d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.210191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09365a3-3897-407a-a2ec-53a3a877c17a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.227718 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3829/7570] 10.222.0.16 () {60 vars in 1299 bytes} [Sat Sep 3 21:17:00 2022] POST /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2 => generated 367 bytes in 37 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:00.232100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604c20b-283c-4305-9dab-f3a0d3f0b525 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.243608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604c20b-283c-4305-9dab-f3a0d3f0b525 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.248392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27385f-43d3-43e0-88b0-f2c4fc0113c3 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.249082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27385f-43d3-43e0-88b0-f2c4fc0113c3 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.249372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27385f-43d3-43e0-88b0-f2c4fc0113c3 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.249658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27385f-43d3-43e0-88b0-f2c4fc0113c3 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.250287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27385f-43d3-43e0-88b0-f2c4fc0113c3 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=ca0ef8c187d04478bb02c2217f8d894d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.251867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27385f-43d3-43e0-88b0-f2c4fc0113c3 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.268718 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3742/7571] 10.222.0.16 () {60 vars in 1299 bytes} [Sat Sep 3 21:17:00 2022] POST /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2 => generated 367 bytes in 37 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:00.273052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09365a3-3897-407a-a2ec-53a3a877c17a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.283424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09365a3-3897-407a-a2ec-53a3a877c17a tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.287802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019eec0b-ab1f-4d95-8ef1-2db8d0eac35b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.288448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019eec0b-ab1f-4d95-8ef1-2db8d0eac35b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.288742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019eec0b-ab1f-4d95-8ef1-2db8d0eac35b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.289063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019eec0b-ab1f-4d95-8ef1-2db8d0eac35b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.290202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019eec0b-ab1f-4d95-8ef1-2db8d0eac35b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorizing `identity:ec2_list_credentials(user_id=ca0ef8c187d04478bb02c2217f8d894d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.291871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019eec0b-ab1f-4d95-8ef1-2db8d0eac35b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.307318 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3830/7572] 10.222.0.16 () {58 vars in 1278 bytes} [Sat Sep 3 21:17:00 2022] GET /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2 => generated 865 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:17:00.308440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27385f-43d3-43e0-88b0-f2c4fc0113c3 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.318122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27385f-43d3-43e0-88b0-f2c4fc0113c3 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.322545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34073945-571d-477d-acc8-3aa158bfdb85 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.323201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34073945-571d-477d-acc8-3aa158bfdb85 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.323496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34073945-571d-477d-acc8-3aa158bfdb85 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.323779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34073945-571d-477d-acc8-3aa158bfdb85 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/822e7de1186c470f92dcbc248fcdba4e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.325803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34073945-571d-477d-acc8-3aa158bfdb85 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=ca0ef8c187d04478bb02c2217f8d894d, credential_id=822e7de1186c470f92dcbc248fcdba4e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.327387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34073945-571d-477d-acc8-3aa158bfdb85 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.333594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=7,keystone:DELETE=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:00.346941 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3743/7573] 10.222.0.16 () {58 vars in 1380 bytes} [Sat Sep 3 21:17:00 2022] DELETE /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/822e7de1186c470f92dcbc248fcdba4e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:00.351350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019eec0b-ab1f-4d95-8ef1-2db8d0eac35b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.364556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019eec0b-ab1f-4d95-8ef1-2db8d0eac35b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.372313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae49a91-9184-4e76-a69e-d440ed8badd1 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.373642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae49a91-9184-4e76-a69e-d440ed8badd1 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.374151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae49a91-9184-4e76-a69e-d440ed8badd1 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.374888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae49a91-9184-4e76-a69e-d440ed8badd1 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/8ce16bf41d0b40999ff6c25e8c48cfd6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.388955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae49a91-9184-4e76-a69e-d440ed8badd1 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=ca0ef8c187d04478bb02c2217f8d894d, credential_id=8ce16bf41d0b40999ff6c25e8c48cfd6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.388955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae49a91-9184-4e76-a69e-d440ed8badd1 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.402040 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3831/7574] 10.222.0.16 () {58 vars in 1380 bytes} [Sat Sep 3 21:17:00 2022] DELETE /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/8ce16bf41d0b40999ff6c25e8c48cfd6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:00.407491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34073945-571d-477d-acc8-3aa158bfdb85 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.417760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34073945-571d-477d-acc8-3aa158bfdb85 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.422069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dc3ae-e8c0-439f-bffe-a147a74ca487 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.422712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dc3ae-e8c0-439f-bffe-a147a74ca487 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.423004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dc3ae-e8c0-439f-bffe-a147a74ca487 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.423302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dc3ae-e8c0-439f-bffe-a147a74ca487 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.423913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584dc3ae-e8c0-439f-bffe-a147a74ca487 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=ca0ef8c187d04478bb02c2217f8d894d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.425603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584dc3ae-e8c0-439f-bffe-a147a74ca487 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.445212 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3744/7575] 10.222.0.16 () {60 vars in 1299 bytes} [Sat Sep 3 21:17:00 2022] POST /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2 => generated 367 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:00.449382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae49a91-9184-4e76-a69e-d440ed8badd1 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.459413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae49a91-9184-4e76-a69e-d440ed8badd1 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.463851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bbc09-f093-4092-8f7e-9be0b2941b07 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.464486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bbc09-f093-4092-8f7e-9be0b2941b07 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.464781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bbc09-f093-4092-8f7e-9be0b2941b07 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.465186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bbc09-f093-4092-8f7e-9be0b2941b07 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/743b297b43e440eebf18271a5d6da9de` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.467081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bbc09-f093-4092-8f7e-9be0b2941b07 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorizing `identity:ec2_get_credential(user_id=ca0ef8c187d04478bb02c2217f8d894d, credential_id=743b297b43e440eebf18271a5d6da9de)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.468639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bbc09-f093-4092-8f7e-9be0b2941b07 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.473537 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3832/7576] 10.222.0.16 () {58 vars in 1377 bytes} [Sat Sep 3 21:17:00 2022] GET /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/743b297b43e440eebf18271a5d6da9de => generated 367 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:00.475595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dc3ae-e8c0-439f-bffe-a147a74ca487 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.495311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dc3ae-e8c0-439f-bffe-a147a74ca487 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.499800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffa03b-c0a2-4fd3-8d5c-6bb88c70f83b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca0ef8c187d04478bb02c2217f8d894d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd022688b439748e3a289af5b98d37f20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.500476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffa03b-c0a2-4fd3-8d5c-6bb88c70f83b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.500770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffa03b-c0a2-4fd3-8d5c-6bb88c70f83b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.501144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffa03b-c0a2-4fd3-8d5c-6bb88c70f83b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/743b297b43e440eebf18271a5d6da9de` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.503156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffa03b-c0a2-4fd3-8d5c-6bb88c70f83b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=ca0ef8c187d04478bb02c2217f8d894d, credential_id=743b297b43e440eebf18271a5d6da9de)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.504602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffa03b-c0a2-4fd3-8d5c-6bb88c70f83b tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.521738 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3745/7577] 10.222.0.16 () {58 vars in 1380 bytes} [Sat Sep 3 21:17:00 2022] DELETE /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d/credentials/OS-EC2/743b297b43e440eebf18271a5d6da9de => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:00.527520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bbc09-f093-4092-8f7e-9be0b2941b07 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.529389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0b9bbc09-f093-4092-8f7e-9be0b2941b07 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:00.544502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bbc09-f093-4092-8f7e-9be0b2941b07 tempest-EC2CredentialsTest-2034630319 tempest-EC2CredentialsTest-2034630319-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.549746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.550381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.550667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.550950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] PATH_INFO: `/v3/users/ca0ef8c187d04478bb02c2217f8d894d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.552147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0ef8c187d04478bb02c2217f8d894d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.554379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.605865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0ef8c187d04478bb02c2217f8d894d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:00.619530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0ef8c187d04478bb02c2217f8d894d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:00.628163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0ef8c187d04478bb02c2217f8d894d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:00.629160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0ef8c187d04478bb02c2217f8d894d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:00.629853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0ef8c187d04478bb02c2217f8d894d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:00.642898 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3833/7578] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:00 2022] DELETE /identity/v3/users/ca0ef8c187d04478bb02c2217f8d894d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:00.647654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8ebdb-f774-49d2-96f4-b70f384db00d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.648034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8ebdb-f774-49d2-96f4-b70f384db00d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.648330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8ebdb-f774-49d2-96f4-b70f384db00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.670002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d7d8ebdb-f774-49d2-96f4-b70f384db00d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:00.671241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d7d8ebdb-f774-49d2-96f4-b70f384db00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:00.757232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.759340 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3746/7579] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:00.769282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:00.772563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8ebdb-f774-49d2-96f4-b70f384db00d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:00.775942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d7d8ebdb-f774-49d2-96f4-b70f384db00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:00.793392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afeaa2-77c8-4fc2-aab4-f50057216a71 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.798560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1792d2f-37e2-4c0b-b7df-2a5101031e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.800766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1792d2f-37e2-4c0b-b7df-2a5101031e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.801158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1792d2f-37e2-4c0b-b7df-2a5101031e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.801412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1792d2f-37e2-4c0b-b7df-2a5101031e79 admin admin] PATH_INFO: `/v3/projects/fd74791289004afbbdb10b805e0b9cc6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.802725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1792d2f-37e2-4c0b-b7df-2a5101031e79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd74791289004afbbdb10b805e0b9cc6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:00.809646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1792d2f-37e2-4c0b-b7df-2a5101031e79 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:00.858508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8ebdb-f774-49d2-96f4-b70f384db00d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:00.876659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b1792d2f-37e2-4c0b-b7df-2a5101031e79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd74791289004afbbdb10b805e0b9cc6', 'request_id': 'req-b1792d2f-37e2-4c0b-b7df-2a5101031e79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:00.886446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b1792d2f-37e2-4c0b-b7df-2a5101031e79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd74791289004afbbdb10b805e0b9cc6', 'request_id': 'req-b1792d2f-37e2-4c0b-b7df-2a5101031e79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:00.893612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b1792d2f-37e2-4c0b-b7df-2a5101031e79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd74791289004afbbdb10b805e0b9cc6', 'request_id': 'req-b1792d2f-37e2-4c0b-b7df-2a5101031e79'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:00.912317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b1792d2f-37e2-4c0b-b7df-2a5101031e79 admin admin] The token cache is being invalidate because project fd74791289004afbbdb10b805e0b9cc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:00.912642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b1792d2f-37e2-4c0b-b7df-2a5101031e79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:00.913361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b1792d2f-37e2-4c0b-b7df-2a5101031e79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:00.913481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dd366-3717-4a1b-83d0-9009b74541bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:00.914144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dd366-3717-4a1b-83d0-9009b74541bb service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.914498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dd366-3717-4a1b-83d0-9009b74541bb service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.914881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dd366-3717-4a1b-83d0-9009b74541bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.917892 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3834/7580] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:00 2022] DELETE /identity/v3/projects/fd74791289004afbbdb10b805e0b9cc6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:00.918030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4a3dd366-3717-4a1b-83d0-9009b74541bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:00.926330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24651d7d-1a72-4e90-b4ad-f6b659deb437 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:00.926624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24651d7d-1a72-4e90-b4ad-f6b659deb437 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:00.926907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24651d7d-1a72-4e90-b4ad-f6b659deb437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:00.948020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-24651d7d-1a72-4e90-b4ad-f6b659deb437 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:00.949583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-24651d7d-1a72-4e90-b4ad-f6b659deb437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:00.956104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=79,keystone:INSERT=3,keystone:DELETE=18 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:00.969683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:01.004579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dd366-3717-4a1b-83d0-9009b74541bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.006227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dd366-3717-4a1b-83d0-9009b74541bb service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.017872 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3747/7581] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:01.025109 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3835/7582] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:01.032782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca43ad3-2f68-4a2d-9836-e04605d49d9c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.033187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca43ad3-2f68-4a2d-9836-e04605d49d9c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.033572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca43ad3-2f68-4a2d-9836-e04605d49d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.055307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2ca43ad3-2f68-4a2d-9836-e04605d49d9c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:01.056765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2ca43ad3-2f68-4a2d-9836-e04605d49d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:01.068943 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3748/7583] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:01.072552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24651d7d-1a72-4e90-b4ad-f6b659deb437 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.085615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24651d7d-1a72-4e90-b4ad-f6b659deb437 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.094934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca402c30-a93b-4528-8da8-9b2847a9b1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.095689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca402c30-a93b-4528-8da8-9b2847a9b1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.095988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca402c30-a93b-4528-8da8-9b2847a9b1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.096264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca402c30-a93b-4528-8da8-9b2847a9b1c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.097690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca402c30-a93b-4528-8da8-9b2847a9b1c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:01.097690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca402c30-a93b-4528-8da8-9b2847a9b1c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.099647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca402c30-a93b-4528-8da8-9b2847a9b1c9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.120196 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3836/7584] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:01.121396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca43ad3-2f68-4a2d-9836-e04605d49d9c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.132699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca43ad3-2f68-4a2d-9836-e04605d49d9c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.144075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49281e-794c-48ff-b742-97e5cefbb374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.144288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49281e-794c-48ff-b742-97e5cefbb374 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.144941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49281e-794c-48ff-b742-97e5cefbb374 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.144941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49281e-794c-48ff-b742-97e5cefbb374 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.145688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49281e-794c-48ff-b742-97e5cefbb374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.147871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49281e-794c-48ff-b742-97e5cefbb374 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.191774 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3749/7585] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:01 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:01.195535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca402c30-a93b-4528-8da8-9b2847a9b1c9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.213207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca402c30-a93b-4528-8da8-9b2847a9b1c9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.279128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedf4a0-63b4-402b-8a39-443ea6f7620f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.279796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedf4a0-63b4-402b-8a39-443ea6f7620f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.280045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedf4a0-63b4-402b-8a39-443ea6f7620f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.280323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedf4a0-63b4-402b-8a39-443ea6f7620f admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.281023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedf4a0-63b4-402b-8a39-443ea6f7620f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.283016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedf4a0-63b4-402b-8a39-443ea6f7620f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.308175 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3837/7586] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:01 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:01.312164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49281e-794c-48ff-b742-97e5cefbb374 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.323874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49281e-794c-48ff-b742-97e5cefbb374 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.328613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1957e-3295-4ac5-8c34-79cd32ddee04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.329339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1957e-3295-4ac5-8c34-79cd32ddee04 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.329620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1957e-3295-4ac5-8c34-79cd32ddee04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.329894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1957e-3295-4ac5-8c34-79cd32ddee04 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.330539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1957e-3295-4ac5-8c34-79cd32ddee04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.332167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1957e-3295-4ac5-8c34-79cd32ddee04 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.343358 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3750/7587] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:01.347644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedf4a0-63b4-402b-8a39-443ea6f7620f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.359322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedf4a0-63b4-402b-8a39-443ea6f7620f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.364628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e005633-7372-484c-b144-cc9fbb532a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.365287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e005633-7372-484c-b144-cc9fbb532a70 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.365576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e005633-7372-484c-b144-cc9fbb532a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.365858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e005633-7372-484c-b144-cc9fbb532a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.366503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e005633-7372-484c-b144-cc9fbb532a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.367036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1957e-3295-4ac5-8c34-79cd32ddee04 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.368694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e005633-7372-484c-b144-cc9fbb532a70 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.369940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-06f1957e-3295-4ac5-8c34-79cd32ddee04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:01.383219 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3838/7588] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:01.385127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1957e-3295-4ac5-8c34-79cd32ddee04 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.388563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e005633-7372-484c-b144-cc9fbb532a70 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.390571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5165d9d-b58c-42db-b162-054e6742b88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.391219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5165d9d-b58c-42db-b162-054e6742b88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.391508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5165d9d-b58c-42db-b162-054e6742b88b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.391784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5165d9d-b58c-42db-b162-054e6742b88b admin admin] PATH_INFO: `/v3/projects/d022688b439748e3a289af5b98d37f20` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.393051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5165d9d-b58c-42db-b162-054e6742b88b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d022688b439748e3a289af5b98d37f20)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.395158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5165d9d-b58c-42db-b162-054e6742b88b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.402810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e005633-7372-484c-b144-cc9fbb532a70 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.408601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930fc3f-cbb1-4f06-bef7-e3d38ba92a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.409333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930fc3f-cbb1-4f06-bef7-e3d38ba92a98 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.409625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930fc3f-cbb1-4f06-bef7-e3d38ba92a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.409914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930fc3f-cbb1-4f06-bef7-e3d38ba92a98 admin admin] PATH_INFO: `/v3/projects/d1cd4a55bad043adb69747bae9562c81/users/9075ee2c1112443e8c1ab534e3e913ed/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.432000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930fc3f-cbb1-4f06-bef7-e3d38ba92a98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1cd4a55bad043adb69747bae9562c81, user_id=9075ee2c1112443e8c1ab534e3e913ed, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.435756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930fc3f-cbb1-4f06-bef7-e3d38ba92a98 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.451327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a5165d9d-b58c-42db-b162-054e6742b88b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd022688b439748e3a289af5b98d37f20', 'request_id': 'req-a5165d9d-b58c-42db-b162-054e6742b88b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:01.451573 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3839/7589] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:01 2022] PUT /identity/v3/projects/d1cd4a55bad043adb69747bae9562c81/users/9075ee2c1112443e8c1ab534e3e913ed/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:01.462714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a8178-9d8a-4a93-84f0-030bd0ae3e68 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.463004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a8178-9d8a-4a93-84f0-030bd0ae3e68 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.463281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a8178-9d8a-4a93-84f0-030bd0ae3e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.464344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a5165d9d-b58c-42db-b162-054e6742b88b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd022688b439748e3a289af5b98d37f20', 'request_id': 'req-a5165d9d-b58c-42db-b162-054e6742b88b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:01.472532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a5165d9d-b58c-42db-b162-054e6742b88b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd022688b439748e3a289af5b98d37f20', 'request_id': 'req-a5165d9d-b58c-42db-b162-054e6742b88b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:01.484288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a5165d9d-b58c-42db-b162-054e6742b88b admin admin] The token cache is being invalidate because project d022688b439748e3a289af5b98d37f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:01.484765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a5165d9d-b58c-42db-b162-054e6742b88b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:01.485611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a5165d9d-b58c-42db-b162-054e6742b88b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:01.487475 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3751/7590] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:01 2022] DELETE /identity/v3/projects/d022688b439748e3a289af5b98d37f20 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:01.492207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c974b-d7c6-497f-8009-461d503aebd6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.492598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c974b-d7c6-497f-8009-461d503aebd6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.493515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c974b-d7c6-497f-8009-461d503aebd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.494784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-332a8178-9d8a-4a93-84f0-030bd0ae3e68 None None] MFA Rules not processed for user `9075ee2c1112443e8c1ab534e3e913ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:01.496002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-332a8178-9d8a-4a93-84f0-030bd0ae3e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:01.526350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-de3c974b-d7c6-497f-8009-461d503aebd6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:01.528559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-de3c974b-d7c6-497f-8009-461d503aebd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:01.546205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:01.578672 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3840/7591] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:01.583456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a8178-9d8a-4a93-84f0-030bd0ae3e68 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.585554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-332a8178-9d8a-4a93-84f0-030bd0ae3e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:01.599518 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3752/7592] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:01.607547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5bdd3-861a-4955-b4aa-d350bdd4bab4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.607846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5bdd3-861a-4955-b4aa-d350bdd4bab4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.608117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5bdd3-861a-4955-b4aa-d350bdd4bab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.630109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bed5bdd3-861a-4955-b4aa-d350bdd4bab4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:01.631928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bed5bdd3-861a-4955-b4aa-d350bdd4bab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:01.640610 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3753/7593] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:01.644684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5bdd3-861a-4955-b4aa-d350bdd4bab4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.653679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5bdd3-861a-4955-b4aa-d350bdd4bab4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.661751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a8178-9d8a-4a93-84f0-030bd0ae3e68 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.662377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab3d07-5cef-4a87-b8cc-22063a556a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.663181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab3d07-5cef-4a87-b8cc-22063a556a1a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.663645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab3d07-5cef-4a87-b8cc-22063a556a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.663932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab3d07-5cef-4a87-b8cc-22063a556a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.665306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab3d07-5cef-4a87-b8cc-22063a556a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:01.665624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab3d07-5cef-4a87-b8cc-22063a556a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.667483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab3d07-5cef-4a87-b8cc-22063a556a1a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.669705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f405a-0ace-4bb7-85c7-72464fe156cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.671089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f405a-0ace-4bb7-85c7-72464fe156cd admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.671375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f405a-0ace-4bb7-85c7-72464fe156cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.671661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f405a-0ace-4bb7-85c7-72464fe156cd admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.672727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f405a-0ace-4bb7-85c7-72464fe156cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.675665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f405a-0ace-4bb7-85c7-72464fe156cd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.677278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:01.685896 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3754/7594] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:17:01.689966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab3d07-5cef-4a87-b8cc-22063a556a1a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.702065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab3d07-5cef-4a87-b8cc-22063a556a1a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.706700 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3841/7595] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:01 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:01.709017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d1208-e265-40d9-83f7-0eff4faf7e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.709735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d1208-e265-40d9-83f7-0eff4faf7e7d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.710054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d1208-e265-40d9-83f7-0eff4faf7e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.710348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d1208-e265-40d9-83f7-0eff4faf7e7d admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.710973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d1208-e265-40d9-83f7-0eff4faf7e7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.712989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d1208-e265-40d9-83f7-0eff4faf7e7d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.713898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f405a-0ace-4bb7-85c7-72464fe156cd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.741563 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3755/7596] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:01 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:01.746141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d1208-e265-40d9-83f7-0eff4faf7e7d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.778850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f405a-0ace-4bb7-85c7-72464fe156cd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.810613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d1208-e265-40d9-83f7-0eff4faf7e7d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.815563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dbf13-37ec-4f43-ba90-cb79a918f1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.816190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dbf13-37ec-4f43-ba90-cb79a918f1cf admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.816487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dbf13-37ec-4f43-ba90-cb79a918f1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.816766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dbf13-37ec-4f43-ba90-cb79a918f1cf admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.817462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4dbf13-37ec-4f43-ba90-cb79a918f1cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.819603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4dbf13-37ec-4f43-ba90-cb79a918f1cf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.836741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33883c-5e0f-4638-91bc-1de75acb65d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.837397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33883c-5e0f-4638-91bc-1de75acb65d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.837683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33883c-5e0f-4638-91bc-1de75acb65d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.837959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33883c-5e0f-4638-91bc-1de75acb65d0 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.838614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33883c-5e0f-4638-91bc-1de75acb65d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.843793 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3756/7597] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:01 2022] POST /identity/v3/users => generated 305 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:01.843957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33883c-5e0f-4638-91bc-1de75acb65d0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.844856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dbf13-37ec-4f43-ba90-cb79a918f1cf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.855552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dbf13-37ec-4f43-ba90-cb79a918f1cf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.860861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb21fc-af66-475b-af1d-135d10f2502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.861498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb21fc-af66-475b-af1d-135d10f2502a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.861785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb21fc-af66-475b-af1d-135d10f2502a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.862187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb21fc-af66-475b-af1d-135d10f2502a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.862837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb21fc-af66-475b-af1d-135d10f2502a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.866043 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3842/7598] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:01 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:01.866186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb21fc-af66-475b-af1d-135d10f2502a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.866970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33883c-5e0f-4638-91bc-1de75acb65d0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.873482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:01.875387 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3757/7599] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:01.879283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb21fc-af66-475b-af1d-135d10f2502a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.880729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33883c-5e0f-4638-91bc-1de75acb65d0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.885651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bada29a-f645-4bdc-9b7c-f03d26928e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.886257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bada29a-f645-4bdc-9b7c-f03d26928e58 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.886542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bada29a-f645-4bdc-9b7c-f03d26928e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.886818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bada29a-f645-4bdc-9b7c-f03d26928e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.887455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bada29a-f645-4bdc-9b7c-f03d26928e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.889120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bada29a-f645-4bdc-9b7c-f03d26928e58 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.889321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb21fc-af66-475b-af1d-135d10f2502a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.894499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d528f-0de2-43be-80bf-994f5ca2b253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.895162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d528f-0de2-43be-80bf-994f5ca2b253 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.895452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d528f-0de2-43be-80bf-994f5ca2b253 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.895732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d528f-0de2-43be-80bf-994f5ca2b253 admin admin] PATH_INFO: `/v3/projects/600b9ff123174962a7f5610f16156646/users/17fd70925a9d41c8acc8d069eea067cb/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.902462 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3843/7600] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:01.903746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bada29a-f645-4bdc-9b7c-f03d26928e58 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.910700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d528f-0de2-43be-80bf-994f5ca2b253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=600b9ff123174962a7f5610f16156646, user_id=17fd70925a9d41c8acc8d069eea067cb, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.913417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d528f-0de2-43be-80bf-994f5ca2b253 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.913585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bada29a-f645-4bdc-9b7c-f03d26928e58 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:01.918937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1551d35-ebce-4e56-8015-ac2eace2a3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:01.919729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1551d35-ebce-4e56-8015-ac2eace2a3df admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:01.920059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1551d35-ebce-4e56-8015-ac2eace2a3df admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:01.920381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1551d35-ebce-4e56-8015-ac2eace2a3df admin admin] PATH_INFO: `/v3/projects/6565741e5f8a4d84a97ab6cac6358258/users/4931128af122461e9dc071e921263352/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:01.927178 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3758/7601] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:01 2022] PUT /identity/v3/projects/600b9ff123174962a7f5610f16156646/users/17fd70925a9d41c8acc8d069eea067cb/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:01.931607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d528f-0de2-43be-80bf-994f5ca2b253 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.936742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1551d35-ebce-4e56-8015-ac2eace2a3df admin admin] RBAC: Authorizing `identity:create_grant(project_id=6565741e5f8a4d84a97ab6cac6358258, user_id=4931128af122461e9dc071e921263352, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:01.939829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1551d35-ebce-4e56-8015-ac2eace2a3df admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:01.953062 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3844/7602] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:01 2022] PUT /identity/v3/projects/6565741e5f8a4d84a97ab6cac6358258/users/4931128af122461e9dc071e921263352/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:01.956753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1551d35-ebce-4e56-8015-ac2eace2a3df admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:01.995305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d528f-0de2-43be-80bf-994f5ca2b253 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:02.022034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1551d35-ebce-4e56-8015-ac2eace2a3df admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:02.027322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae38aa-e52c-4630-bc7a-0cc86ef019eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:02.027931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae38aa-e52c-4630-bc7a-0cc86ef019eb admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.028201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae38aa-e52c-4630-bc7a-0cc86ef019eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.028477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae38aa-e52c-4630-bc7a-0cc86ef019eb admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.029570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae38aa-e52c-4630-bc7a-0cc86ef019eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:02.030745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae38aa-e52c-4630-bc7a-0cc86ef019eb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:02.033050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:02.041399 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3845/7603] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:17:02.046764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae38aa-e52c-4630-bc7a-0cc86ef019eb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:02.056789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae38aa-e52c-4630-bc7a-0cc86ef019eb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:02.058448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd3c76-653c-4c72-acb1-f41e5c092e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:02.059071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd3c76-653c-4c72-acb1-f41e5c092e99 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.059357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd3c76-653c-4c72-acb1-f41e5c092e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.059634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd3c76-653c-4c72-acb1-f41e5c092e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.060278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd3c76-653c-4c72-acb1-f41e5c092e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:02.061729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df992ac-d4b9-4688-92fa-63dc802fcb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:02.061902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd3c76-653c-4c72-acb1-f41e5c092e99 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:02.062508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df992ac-d4b9-4688-92fa-63dc802fcb70 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.062798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df992ac-d4b9-4688-92fa-63dc802fcb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.063081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df992ac-d4b9-4688-92fa-63dc802fcb70 admin admin] PATH_INFO: `/v3/projects/6565741e5f8a4d84a97ab6cac6358258/users/4931128af122461e9dc071e921263352/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.065080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df992ac-d4b9-4688-92fa-63dc802fcb70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6565741e5f8a4d84a97ab6cac6358258, user_id=4931128af122461e9dc071e921263352, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:02.067769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df992ac-d4b9-4688-92fa-63dc802fcb70 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:02.072216 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3759/7604] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:17:02.078114 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3846/7605] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:02 2022] PUT /identity/v3/projects/6565741e5f8a4d84a97ab6cac6358258/users/4931128af122461e9dc071e921263352/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:02.080723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd3c76-653c-4c72-acb1-f41e5c092e99 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:02.089840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56180c-6cad-4709-905c-997a72802eeb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.090138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56180c-6cad-4709-905c-997a72802eeb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.090427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56180c-6cad-4709-905c-997a72802eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.092134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd3c76-653c-4c72-acb1-f41e5c092e99 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:02.126128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6f56180c-6cad-4709-905c-997a72802eeb None None] MFA Rules not processed for user `4931128af122461e9dc071e921263352`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:02.128174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6f56180c-6cad-4709-905c-997a72802eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:02.170139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26205a-413a-4ed6-a42b-bc6edae43ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:02.170822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26205a-413a-4ed6-a42b-bc6edae43ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.171115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26205a-413a-4ed6-a42b-bc6edae43ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.171404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26205a-413a-4ed6-a42b-bc6edae43ac0 admin admin] PATH_INFO: `/v3/projects/600b9ff123174962a7f5610f16156646/users/17fd70925a9d41c8acc8d069eea067cb/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.174103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26205a-413a-4ed6-a42b-bc6edae43ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=600b9ff123174962a7f5610f16156646, user_id=17fd70925a9d41c8acc8d069eea067cb, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:02.176922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26205a-413a-4ed6-a42b-bc6edae43ac0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:02.189039 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3760/7606] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:02 2022] PUT /identity/v3/projects/600b9ff123174962a7f5610f16156646/users/17fd70925a9d41c8acc8d069eea067cb/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:02.200350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc445756-a413-48d8-999a-71d3c842361b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.200645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc445756-a413-48d8-999a-71d3c842361b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.200957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc445756-a413-48d8-999a-71d3c842361b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.213280 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3847/7607] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:02.224303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56180c-6cad-4709-905c-997a72802eeb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:02.226030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6f56180c-6cad-4709-905c-997a72802eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:02.245427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bc445756-a413-48d8-999a-71d3c842361b None None] MFA Rules not processed for user `17fd70925a9d41c8acc8d069eea067cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:02.246681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bc445756-a413-48d8-999a-71d3c842361b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:02.276294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:02.300378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56180c-6cad-4709-905c-997a72802eeb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:02.330222 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3761/7608] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:02 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:02.334807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc445756-a413-48d8-999a-71d3c842361b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:02.357382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b0a66-8243-4c76-9f67-02f13d07a531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:02.358009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b0a66-8243-4c76-9f67-02f13d07a531 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.358300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b0a66-8243-4c76-9f67-02f13d07a531 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.358593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b0a66-8243-4c76-9f67-02f13d07a531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.392977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:02.406863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc445756-a413-48d8-999a-71d3c842361b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:02.411617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb26bf-1454-4c14-8c8b-1c8d156d26c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:02.412237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb26bf-1454-4c14-8c8b-1c8d156d26c9 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.412530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb26bf-1454-4c14-8c8b-1c8d156d26c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.412815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb26bf-1454-4c14-8c8b-1c8d156d26c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.413487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb26bf-1454-4c14-8c8b-1c8d156d26c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:02.415460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb26bf-1454-4c14-8c8b-1c8d156d26c9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:02.430527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b0a66-8243-4c76-9f67-02f13d07a531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:02.432620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b0a66-8243-4c76-9f67-02f13d07a531 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:02.444596 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3848/7609] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:02.450644 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3762/7610] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:02 2022] POST /identity/v3/projects => generated 397 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:02.451983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b0a66-8243-4c76-9f67-02f13d07a531 service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:02.464918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b0a66-8243-4c76-9f67-02f13d07a531 service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:02.530289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb3524-52e6-48f7-a880-779c4b4525a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:02.531582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb3524-52e6-48f7-a880-779c4b4525a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.531975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb3524-52e6-48f7-a880-779c4b4525a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.532403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb3524-52e6-48f7-a880-779c4b4525a7 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.533171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb3524-52e6-48f7-a880-779c4b4525a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:02.535193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb3524-52e6-48f7-a880-779c4b4525a7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:02.564710 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3849/7611] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:02 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:02.566284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb26bf-1454-4c14-8c8b-1c8d156d26c9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:02.578733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb26bf-1454-4c14-8c8b-1c8d156d26c9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:02.584801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a689ec9-7a91-4594-a36f-0900f9ca9167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:02.586393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a689ec9-7a91-4594-a36f-0900f9ca9167 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.587244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a689ec9-7a91-4594-a36f-0900f9ca9167 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.587528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a689ec9-7a91-4594-a36f-0900f9ca9167 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.588183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a689ec9-7a91-4594-a36f-0900f9ca9167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:02.590616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a689ec9-7a91-4594-a36f-0900f9ca9167 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:02.602682 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3763/7612] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:02.607192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb3524-52e6-48f7-a880-779c4b4525a7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:02.630272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb3524-52e6-48f7-a880-779c4b4525a7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:02.639208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe49f81-b67a-4db9-8ae0-942ccbe04590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:02.640390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe49f81-b67a-4db9-8ae0-942ccbe04590 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.640811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe49f81-b67a-4db9-8ae0-942ccbe04590 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.641301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe49f81-b67a-4db9-8ae0-942ccbe04590 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.642064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe49f81-b67a-4db9-8ae0-942ccbe04590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:02.643917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe49f81-b67a-4db9-8ae0-942ccbe04590 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:02.658992 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3850/7613] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:02.660700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a689ec9-7a91-4594-a36f-0900f9ca9167 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:02.671434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a689ec9-7a91-4594-a36f-0900f9ca9167 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:02.678223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de967d7-2c0c-4380-a8ce-5ca09753a352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:02.679182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de967d7-2c0c-4380-a8ce-5ca09753a352 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.679713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de967d7-2c0c-4380-a8ce-5ca09753a352 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.680164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de967d7-2c0c-4380-a8ce-5ca09753a352 admin admin] PATH_INFO: `/v3/projects/daa792652df244649a49b95bf3809f3c/users/93ed72b972f2457b9633f4fcc06aa789/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.699312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de967d7-2c0c-4380-a8ce-5ca09753a352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa792652df244649a49b95bf3809f3c, user_id=93ed72b972f2457b9633f4fcc06aa789, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:02.702441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de967d7-2c0c-4380-a8ce-5ca09753a352 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:02.724088 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3764/7614] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:02 2022] PUT /identity/v3/projects/daa792652df244649a49b95bf3809f3c/users/93ed72b972f2457b9633f4fcc06aa789/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:02.739831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af04847-e100-4bf2-a443-4453cc9c27ab None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.740107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af04847-e100-4bf2-a443-4453cc9c27ab None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.740381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af04847-e100-4bf2-a443-4453cc9c27ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:02.790579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4af04847-e100-4bf2-a443-4453cc9c27ab None None] MFA Rules not processed for user `93ed72b972f2457b9633f4fcc06aa789`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:02.791836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4af04847-e100-4bf2-a443-4453cc9c27ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:02.812682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:02.864507 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3851/7615] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:02 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:02.876351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de967d7-2c0c-4380-a8ce-5ca09753a352 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:02.878192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2de967d7-2c0c-4380-a8ce-5ca09753a352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:02.937976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de967d7-2c0c-4380-a8ce-5ca09753a352 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:02.941694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae975af-7e8a-4d8c-946c-12c64c65bd7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:02.942413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae975af-7e8a-4d8c-946c-12c64c65bd7a service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:02.942708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae975af-7e8a-4d8c-946c-12c64c65bd7a service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:02.942984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae975af-7e8a-4d8c-946c-12c64c65bd7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:03.002139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae975af-7e8a-4d8c-946c-12c64c65bd7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:03.003580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae975af-7e8a-4d8c-946c-12c64c65bd7a service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:03.015414 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3765/7616] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:02 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:03.433108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af04847-e100-4bf2-a443-4453cc9c27ab None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:03.496130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af04847-e100-4bf2-a443-4453cc9c27ab None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:03.500718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:03.501392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:03.501689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:03.501942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] PATH_INFO: `/v3/users/17fd70925a9d41c8acc8d069eea067cb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:03.503072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=17fd70925a9d41c8acc8d069eea067cb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:03.505034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:03.551245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17fd70925a9d41c8acc8d069eea067cb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:03.563080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17fd70925a9d41c8acc8d069eea067cb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:03.571007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17fd70925a9d41c8acc8d069eea067cb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:03.571711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17fd70925a9d41c8acc8d069eea067cb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:03.572322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17fd70925a9d41c8acc8d069eea067cb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:03.585007 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3852/7617] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:03 2022] DELETE /identity/v3/users/17fd70925a9d41c8acc8d069eea067cb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:03.590054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea73e3e-4168-4d2e-b23e-62277d1049a1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:03.590572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea73e3e-4168-4d2e-b23e-62277d1049a1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:03.590751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea73e3e-4168-4d2e-b23e-62277d1049a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:03.611306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0ea73e3e-4168-4d2e-b23e-62277d1049a1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:03.612447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0ea73e3e-4168-4d2e-b23e-62277d1049a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:03.662668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:03.677922 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3766/7618] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:03.687702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:03.689245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:03.732419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:03.753406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194a6ad-5011-410d-b52c-b045bb734a8f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:03.757012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dca7f-af11-45a2-9057-22076a5bbdd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:03.757621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dca7f-af11-45a2-9057-22076a5bbdd8 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:03.757905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dca7f-af11-45a2-9057-22076a5bbdd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:03.758188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dca7f-af11-45a2-9057-22076a5bbdd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:03.767593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7dca7f-af11-45a2-9057-22076a5bbdd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:03.769071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7dca7f-af11-45a2-9057-22076a5bbdd8 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:03.778973 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3853/7619] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:17:04.057390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea73e3e-4168-4d2e-b23e-62277d1049a1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:04.059056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0ea73e3e-4168-4d2e-b23e-62277d1049a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:04.089773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea73e3e-4168-4d2e-b23e-62277d1049a1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:04.103837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dca7f-af11-45a2-9057-22076a5bbdd8 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:04.105679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bd7dca7f-af11-45a2-9057-22076a5bbdd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:04.111963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d92ce8-1516-4847-8f07-cd8299c3f7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:04.113245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d92ce8-1516-4847-8f07-cd8299c3f7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:04.113540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d92ce8-1516-4847-8f07-cd8299c3f7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:04.114622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d92ce8-1516-4847-8f07-cd8299c3f7b3 admin admin] PATH_INFO: `/v3/users/9075ee2c1112443e8c1ab534e3e913ed` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:04.117865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d92ce8-1516-4847-8f07-cd8299c3f7b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9075ee2c1112443e8c1ab534e3e913ed)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:04.120599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d92ce8-1516-4847-8f07-cd8299c3f7b3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:04.121970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dca7f-af11-45a2-9057-22076a5bbdd8 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:04.135147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:04.136187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:04.137327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:04.137879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] PATH_INFO: `/v3/projects/600b9ff123174962a7f5610f16156646` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:04.139380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] RBAC: Authorizing `identity:delete_project(project_id=600b9ff123174962a7f5610f16156646)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:04.141583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:04.195407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '600b9ff123174962a7f5610f16156646', 'request_id': 'req-cf9c012c-f75b-4364-92f3-4ec6156d185a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.200818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-14d92ce8-1516-4847-8f07-cd8299c3f7b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9075ee2c1112443e8c1ab534e3e913ed'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.206297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '600b9ff123174962a7f5610f16156646', 'request_id': 'req-cf9c012c-f75b-4364-92f3-4ec6156d185a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.215934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-14d92ce8-1516-4847-8f07-cd8299c3f7b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9075ee2c1112443e8c1ab534e3e913ed'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.216921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-14d92ce8-1516-4847-8f07-cd8299c3f7b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9075ee2c1112443e8c1ab534e3e913ed'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.217331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '600b9ff123174962a7f5610f16156646', 'request_id': 'req-cf9c012c-f75b-4364-92f3-4ec6156d185a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.224313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-14d92ce8-1516-4847-8f07-cd8299c3f7b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9075ee2c1112443e8c1ab534e3e913ed'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.232485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] The token cache is being invalidate because project 600b9ff123174962a7f5610f16156646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:04.232889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.233191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-14d92ce8-1516-4847-8f07-cd8299c3f7b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9075ee2c1112443e8c1ab534e3e913ed'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.234072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.235685 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3854/7620] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:04 2022] DELETE /identity/v3/projects/600b9ff123174962a7f5610f16156646 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:04.239348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:04.240684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:04.241429 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3767/7621] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:04 2022] DELETE /identity/v3/users/9075ee2c1112443e8c1ab534e3e913ed => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:04.247025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55168006-7a6c-4ac3-810f-487ab5dd6eca None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:04.247957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55168006-7a6c-4ac3-810f-487ab5dd6eca None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:04.248259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55168006-7a6c-4ac3-810f-487ab5dd6eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:04.269145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-55168006-7a6c-4ac3-810f-487ab5dd6eca None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:04.270492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-55168006-7a6c-4ac3-810f-487ab5dd6eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:04.306944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c012c-f75b-4364-92f3-4ec6156d185a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:04.330698 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3768/7622] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:04.340647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55168006-7a6c-4ac3-810f-487ab5dd6eca None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:04.343132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-55168006-7a6c-4ac3-810f-487ab5dd6eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:04.367492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:04.368636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:04.369091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:04.369382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] PATH_INFO: `/v3/users/93ed72b972f2457b9633f4fcc06aa789` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:04.370573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ed72b972f2457b9633f4fcc06aa789)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:04.372577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:04.385607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:04.421517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55168006-7a6c-4ac3-810f-487ab5dd6eca None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:04.425266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a0178-51cc-4475-90ba-17d5a0d9601e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:04.425940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a0178-51cc-4475-90ba-17d5a0d9601e service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:04.426164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a0178-51cc-4475-90ba-17d5a0d9601e service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:04.426440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a0178-51cc-4475-90ba-17d5a0d9601e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:04.426729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ed72b972f2457b9633f4fcc06aa789'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.436154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a0178-51cc-4475-90ba-17d5a0d9601e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:04.437627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a0178-51cc-4475-90ba-17d5a0d9601e service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:04.439222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ed72b972f2457b9633f4fcc06aa789'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.448300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ed72b972f2457b9633f4fcc06aa789'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.449281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ed72b972f2457b9633f4fcc06aa789'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.449554 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3769/7623] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:17:04.449893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ed72b972f2457b9633f4fcc06aa789'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.464720 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3855/7624] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:04 2022] DELETE /identity/v3/users/93ed72b972f2457b9633f4fcc06aa789 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:04.741974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a0178-51cc-4475-90ba-17d5a0d9601e service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:04.744422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:04.745313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b66a0178-51cc-4475-90ba-17d5a0d9601e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:04.745998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:04.778087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a0178-51cc-4475-90ba-17d5a0d9601e service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:04.789091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:04.825959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3121d5-d69f-4d16-8d54-db33e6343baf admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:04.830782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:04.831928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:04.832245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:04.832532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c admin admin] PATH_INFO: `/v3/projects/daa792652df244649a49b95bf3809f3c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:04.833721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa792652df244649a49b95bf3809f3c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:04.835650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:04.851377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:04.852527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:04.852825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:04.853141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] PATH_INFO: `/v3/projects/d1cd4a55bad043adb69747bae9562c81` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:04.854293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1cd4a55bad043adb69747bae9562c81)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:04.856508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:04.889294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa792652df244649a49b95bf3809f3c', 'request_id': 'req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.898359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa792652df244649a49b95bf3809f3c', 'request_id': 'req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.905737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa792652df244649a49b95bf3809f3c', 'request_id': 'req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.906942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1cd4a55bad043adb69747bae9562c81', 'request_id': 'req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.914231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1cd4a55bad043adb69747bae9562c81', 'request_id': 'req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.918247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c admin admin] The token cache is being invalidate because project daa792652df244649a49b95bf3809f3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:04.918542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.919194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bd7f0c2a-ff38-4528-8420-d85b6d0a9e3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.920779 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3856/7625] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:04 2022] DELETE /identity/v3/projects/daa792652df244649a49b95bf3809f3c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:04.921705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1cd4a55bad043adb69747bae9562c81', 'request_id': 'req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.925531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113b690-dbc1-4445-b3ef-9f618c343ddd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:04.926010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113b690-dbc1-4445-b3ef-9f618c343ddd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:04.926293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113b690-dbc1-4445-b3ef-9f618c343ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:04.933300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] The token cache is being invalidate because project d1cd4a55bad043adb69747bae9562c81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:04.933596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.934263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:04.935866 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3770/7626] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:04 2022] DELETE /identity/v3/projects/d1cd4a55bad043adb69747bae9562c81 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:04.939611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:04.940918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:04.949642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4113b690-dbc1-4445-b3ef-9f618c343ddd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:04.951380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4113b690-dbc1-4445-b3ef-9f618c343ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:04.963209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:05.006333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd0979-dbd7-4eb0-a0c3-beb73f10c9ba admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.011272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.011915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.012205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.012491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] PATH_INFO: `/v3/users/4931128af122461e9dc071e921263352` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.013670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4931128af122461e9dc071e921263352)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.015559 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3857/7627] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:05.015679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.023318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f7941-28b6-41a0-8c32-fe6835990c6e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.023626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f7941-28b6-41a0-8c32-fe6835990c6e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.024458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f7941-28b6-41a0-8c32-fe6835990c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.044382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0b1f7941-28b6-41a0-8c32-fe6835990c6e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:05.045629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0b1f7941-28b6-41a0-8c32-fe6835990c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:05.053869 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3858/7628] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:05.058116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f7941-28b6-41a0-8c32-fe6835990c6e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.063143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:05.067586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f7941-28b6-41a0-8c32-fe6835990c6e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.071414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4931128af122461e9dc071e921263352'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:05.073260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9b6df-4830-43ca-aeeb-0c4f3ef23371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.073870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b6df-4830-43ca-aeeb-0c4f3ef23371 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.074161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b6df-4830-43ca-aeeb-0c4f3ef23371 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.074435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b6df-4830-43ca-aeeb-0c4f3ef23371 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.075083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9b6df-4830-43ca-aeeb-0c4f3ef23371 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:05.075396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9b6df-4830-43ca-aeeb-0c4f3ef23371 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.077034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9b6df-4830-43ca-aeeb-0c4f3ef23371 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.086278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4931128af122461e9dc071e921263352'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:05.087994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4931128af122461e9dc071e921263352'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:05.090755 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3859/7629] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:17:05.094779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9b6df-4830-43ca-aeeb-0c4f3ef23371 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.096325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4931128af122461e9dc071e921263352'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:05.096469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c5c9b6df-4830-43ca-aeeb-0c4f3ef23371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:05.105276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4931128af122461e9dc071e921263352'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:05.110576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9b6df-4830-43ca-aeeb-0c4f3ef23371 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.113072 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3771/7630] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:04 2022] DELETE /identity/v3/users/4931128af122461e9dc071e921263352 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:05.193854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16694d62-c908-4d69-a5be-4de9af0d1e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.194504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16694d62-c908-4d69-a5be-4de9af0d1e0a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.194846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16694d62-c908-4d69-a5be-4de9af0d1e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.195130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16694d62-c908-4d69-a5be-4de9af0d1e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.195794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16694d62-c908-4d69-a5be-4de9af0d1e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.197822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16694d62-c908-4d69-a5be-4de9af0d1e0a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.228232 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3860/7631] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:05 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:05.232939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.245049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a3298-1e53-4e7e-acd4-c867a74628b6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.307306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fda894-232e-4655-90da-9995bc87bced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.308101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fda894-232e-4655-90da-9995bc87bced admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.308398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fda894-232e-4655-90da-9995bc87bced admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.308675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fda894-232e-4655-90da-9995bc87bced admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.309479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fda894-232e-4655-90da-9995bc87bced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.312251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fda894-232e-4655-90da-9995bc87bced admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.340725 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3772/7632] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:05 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:05.341767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16694d62-c908-4d69-a5be-4de9af0d1e0a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.358894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16694d62-c908-4d69-a5be-4de9af0d1e0a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.376294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242c756-7ec9-4b2b-97fc-ef62ffe8c73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.376967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242c756-7ec9-4b2b-97fc-ef62ffe8c73d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.377287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242c756-7ec9-4b2b-97fc-ef62ffe8c73d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.377577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242c756-7ec9-4b2b-97fc-ef62ffe8c73d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.378244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242c756-7ec9-4b2b-97fc-ef62ffe8c73d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.379881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242c756-7ec9-4b2b-97fc-ef62ffe8c73d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.391748 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3861/7633] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:05.395976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fda894-232e-4655-90da-9995bc87bced admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.410812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fda894-232e-4655-90da-9995bc87bced admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.416135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829f6d3-b959-4cf1-b79b-1da5cb4c573b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.416750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829f6d3-b959-4cf1-b79b-1da5cb4c573b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.417087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829f6d3-b959-4cf1-b79b-1da5cb4c573b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.417368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829f6d3-b959-4cf1-b79b-1da5cb4c573b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.418015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829f6d3-b959-4cf1-b79b-1da5cb4c573b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.419631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829f6d3-b959-4cf1-b79b-1da5cb4c573b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.431322 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3773/7634] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:05.435484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242c756-7ec9-4b2b-97fc-ef62ffe8c73d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.446818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242c756-7ec9-4b2b-97fc-ef62ffe8c73d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.451778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dad93-cbd0-4669-a39c-bdfbab771d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.452526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dad93-cbd0-4669-a39c-bdfbab771d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.452926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dad93-cbd0-4669-a39c-bdfbab771d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.453214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dad93-cbd0-4669-a39c-bdfbab771d5a admin admin] PATH_INFO: `/v3/projects/1e03a2cdfa444a92ba6eff58619e1ac9/users/35f9e2978eec461a9a7efc33ac270cdc/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.470182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314dad93-cbd0-4669-a39c-bdfbab771d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e03a2cdfa444a92ba6eff58619e1ac9, user_id=35f9e2978eec461a9a7efc33ac270cdc, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.472992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314dad93-cbd0-4669-a39c-bdfbab771d5a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.487084 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3862/7635] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:05 2022] PUT /identity/v3/projects/1e03a2cdfa444a92ba6eff58619e1ac9/users/35f9e2978eec461a9a7efc33ac270cdc/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:05.500791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f0ea4-6960-4875-858c-3e6768544ff7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.501490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f0ea4-6960-4875-858c-3e6768544ff7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.501661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f0ea4-6960-4875-858c-3e6768544ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.532830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dad93-cbd0-4669-a39c-bdfbab771d5a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.535122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-314dad93-cbd0-4669-a39c-bdfbab771d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:05.536062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-976f0ea4-6960-4875-858c-3e6768544ff7 None None] MFA Rules not processed for user `35f9e2978eec461a9a7efc33ac270cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:05.537695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-976f0ea4-6960-4875-858c-3e6768544ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:05.594022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:05.614863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dad93-cbd0-4669-a39c-bdfbab771d5a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.620168 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3774/7636] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:05.621674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83093af6-bf86-4e57-8a99-9e64dedbccc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.622332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83093af6-bf86-4e57-8a99-9e64dedbccc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.622772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83093af6-bf86-4e57-8a99-9e64dedbccc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.623069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83093af6-bf86-4e57-8a99-9e64dedbccc4 admin admin] PATH_INFO: `/v3/projects/6565741e5f8a4d84a97ab6cac6358258` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.624399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83093af6-bf86-4e57-8a99-9e64dedbccc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6565741e5f8a4d84a97ab6cac6358258)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.624689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f0ea4-6960-4875-858c-3e6768544ff7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.626534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83093af6-bf86-4e57-8a99-9e64dedbccc4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.636619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f0ea4-6960-4875-858c-3e6768544ff7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.641807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b42a0b-28b9-4f05-93a8-e7e2f8aa3e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.642761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b42a0b-28b9-4f05-93a8-e7e2f8aa3e9c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.643366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b42a0b-28b9-4f05-93a8-e7e2f8aa3e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.643818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b42a0b-28b9-4f05-93a8-e7e2f8aa3e9c admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.644598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b42a0b-28b9-4f05-93a8-e7e2f8aa3e9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.646916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b42a0b-28b9-4f05-93a8-e7e2f8aa3e9c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.678970 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3775/7637] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:05 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:05.686503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-83093af6-bf86-4e57-8a99-9e64dedbccc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6565741e5f8a4d84a97ab6cac6358258', 'request_id': 'req-83093af6-bf86-4e57-8a99-9e64dedbccc4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:05.686702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b42a0b-28b9-4f05-93a8-e7e2f8aa3e9c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.695628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:05.698421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-83093af6-bf86-4e57-8a99-9e64dedbccc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6565741e5f8a4d84a97ab6cac6358258', 'request_id': 'req-83093af6-bf86-4e57-8a99-9e64dedbccc4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:05.710679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-83093af6-bf86-4e57-8a99-9e64dedbccc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6565741e5f8a4d84a97ab6cac6358258', 'request_id': 'req-83093af6-bf86-4e57-8a99-9e64dedbccc4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:05.725058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-83093af6-bf86-4e57-8a99-9e64dedbccc4 admin admin] The token cache is being invalidate because project 6565741e5f8a4d84a97ab6cac6358258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:05.725604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-83093af6-bf86-4e57-8a99-9e64dedbccc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:05.726422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-83093af6-bf86-4e57-8a99-9e64dedbccc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:05.728452 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3863/7638] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:05 2022] DELETE /identity/v3/projects/6565741e5f8a4d84a97ab6cac6358258 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:05.737265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c028d-1e34-443d-8b8f-28ebf76b0bb2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.737570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c028d-1e34-443d-8b8f-28ebf76b0bb2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.737859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c028d-1e34-443d-8b8f-28ebf76b0bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.767240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ca0c028d-1e34-443d-8b8f-28ebf76b0bb2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:05.769127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ca0c028d-1e34-443d-8b8f-28ebf76b0bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:05.778655 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3864/7639] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:05.779001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b42a0b-28b9-4f05-93a8-e7e2f8aa3e9c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.786114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d438f-184f-464f-8ff0-656a59733df8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.786480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d438f-184f-464f-8ff0-656a59733df8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.786902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d438f-184f-464f-8ff0-656a59733df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.788675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9a47e-2aec-4885-bd4d-ec33dff9a879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.790103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9a47e-2aec-4885-bd4d-ec33dff9a879 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.790397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9a47e-2aec-4885-bd4d-ec33dff9a879 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.790681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9a47e-2aec-4885-bd4d-ec33dff9a879 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.791328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9a47e-2aec-4885-bd4d-ec33dff9a879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.793515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9a47e-2aec-4885-bd4d-ec33dff9a879 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.807952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8d9d438f-184f-464f-8ff0-656a59733df8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:05.809305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8d9d438f-184f-464f-8ff0-656a59733df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:05.820611 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3865/7640] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:05.821386 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3776/7641] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:05 2022] POST /identity/v3/users => generated 285 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:05.822766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d438f-184f-464f-8ff0-656a59733df8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.825897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9a47e-2aec-4885-bd4d-ec33dff9a879 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.827737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-26e9a47e-2aec-4885-bd4d-ec33dff9a879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:05.832911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d438f-184f-464f-8ff0-656a59733df8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.838641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb4a8c-8a42-497c-8cce-2873a50ce8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.839331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb4a8c-8a42-497c-8cce-2873a50ce8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.839679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb4a8c-8a42-497c-8cce-2873a50ce8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.840023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb4a8c-8a42-497c-8cce-2873a50ce8e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.840744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb4a8c-8a42-497c-8cce-2873a50ce8e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:05.841165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb4a8c-8a42-497c-8cce-2873a50ce8e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.842148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9a47e-2aec-4885-bd4d-ec33dff9a879 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.842998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb4a8c-8a42-497c-8cce-2873a50ce8e1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.849558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261f1ca-9097-4341-82b2-7708d927a792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.850265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261f1ca-9097-4341-82b2-7708d927a792 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.850757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261f1ca-9097-4341-82b2-7708d927a792 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.851537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261f1ca-9097-4341-82b2-7708d927a792 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.852245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261f1ca-9097-4341-82b2-7708d927a792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.854045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261f1ca-9097-4341-82b2-7708d927a792 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.856254 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3866/7642] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:17:05.860455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb4a8c-8a42-497c-8cce-2873a50ce8e1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.866219 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3777/7643] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:05.872421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb4a8c-8a42-497c-8cce-2873a50ce8e1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.874031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261f1ca-9097-4341-82b2-7708d927a792 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.877355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6540d2-aef2-42e0-a7be-a657f870cfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.877957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6540d2-aef2-42e0-a7be-a657f870cfda admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.878925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6540d2-aef2-42e0-a7be-a657f870cfda admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.879257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6540d2-aef2-42e0-a7be-a657f870cfda admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.879975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6540d2-aef2-42e0-a7be-a657f870cfda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.881948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6540d2-aef2-42e0-a7be-a657f870cfda admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.884949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261f1ca-9097-4341-82b2-7708d927a792 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.889710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2b936-1fb5-4a35-b060-a006db63c8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.890306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2b936-1fb5-4a35-b060-a006db63c8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.890599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2b936-1fb5-4a35-b060-a006db63c8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.890877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2b936-1fb5-4a35-b060-a006db63c8a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.891535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2b936-1fb5-4a35-b060-a006db63c8a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.893171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2b936-1fb5-4a35-b060-a006db63c8a0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.906682 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3778/7644] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:05.910917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2b936-1fb5-4a35-b060-a006db63c8a0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.916639 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3867/7645] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:05 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:05.920647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6540d2-aef2-42e0-a7be-a657f870cfda admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:05.921431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2b936-1fb5-4a35-b060-a006db63c8a0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.980087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e108a7f-2d79-4d83-93d8-ff352d1623c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.980665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e108a7f-2d79-4d83-93d8-ff352d1623c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.981070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e108a7f-2d79-4d83-93d8-ff352d1623c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.981352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e108a7f-2d79-4d83-93d8-ff352d1623c1 admin admin] PATH_INFO: `/v3/projects/c843e59a12634cb0be89913074d9c52a/users/07e85f70511147a28434f6e5275425fe/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.985882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6540d2-aef2-42e0-a7be-a657f870cfda admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:05.990669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178800e6-e1d5-43b4-a474-aec8f588a5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:05.991277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178800e6-e1d5-43b4-a474-aec8f588a5f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:05.991569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178800e6-e1d5-43b4-a474-aec8f588a5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:05.991850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178800e6-e1d5-43b4-a474-aec8f588a5f8 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:05.992488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178800e6-e1d5-43b4-a474-aec8f588a5f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.994998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178800e6-e1d5-43b4-a474-aec8f588a5f8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:05.996200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e108a7f-2d79-4d83-93d8-ff352d1623c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c843e59a12634cb0be89913074d9c52a, user_id=07e85f70511147a28434f6e5275425fe, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:05.998892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e108a7f-2d79-4d83-93d8-ff352d1623c1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:06.015158 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3779/7646] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:05 2022] PUT /identity/v3/projects/c843e59a12634cb0be89913074d9c52a/users/07e85f70511147a28434f6e5275425fe/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:06.017782 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3868/7647] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:05 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:06.021815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e108a7f-2d79-4d83-93d8-ff352d1623c1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:06.025166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4dcfe-789f-4309-b705-6bf0dfa9354a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:06.025478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4dcfe-789f-4309-b705-6bf0dfa9354a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:06.025763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4dcfe-789f-4309-b705-6bf0dfa9354a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:06.054518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-78a4dcfe-789f-4309-b705-6bf0dfa9354a None None] MFA Rules not processed for user `07e85f70511147a28434f6e5275425fe`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:06.056271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-78a4dcfe-789f-4309-b705-6bf0dfa9354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:06.064608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:06.081405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:06.087186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e108a7f-2d79-4d83-93d8-ff352d1623c1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:06.093320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2d0cb-2f18-42f7-8e5a-c77bb7dc92df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:06.093938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2d0cb-2f18-42f7-8e5a-c77bb7dc92df admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:06.094259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2d0cb-2f18-42f7-8e5a-c77bb7dc92df admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:06.094542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2d0cb-2f18-42f7-8e5a-c77bb7dc92df admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:06.095189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2d0cb-2f18-42f7-8e5a-c77bb7dc92df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:06.096801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2d0cb-2f18-42f7-8e5a-c77bb7dc92df admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:06.107603 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3780/7648] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:17:06.111486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2d0cb-2f18-42f7-8e5a-c77bb7dc92df admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:06.122193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2d0cb-2f18-42f7-8e5a-c77bb7dc92df admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:06.126963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff84e57-7bb6-428d-9aa9-96a7557f88b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:06.127569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff84e57-7bb6-428d-9aa9-96a7557f88b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:06.127852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff84e57-7bb6-428d-9aa9-96a7557f88b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:06.128130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff84e57-7bb6-428d-9aa9-96a7557f88b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:06.128770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff84e57-7bb6-428d-9aa9-96a7557f88b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:06.130542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff84e57-7bb6-428d-9aa9-96a7557f88b8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:06.138143 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3869/7649] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:06.142392 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3781/7650] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:17:06.146809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4dcfe-789f-4309-b705-6bf0dfa9354a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:06.149880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff84e57-7bb6-428d-9aa9-96a7557f88b8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:06.151638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3ff84e57-7bb6-428d-9aa9-96a7557f88b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:06.158565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4dcfe-789f-4309-b705-6bf0dfa9354a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:06.162780 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5e747-0a01-4651-b182-0ff114530c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:06.163438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5e747-0a01-4651-b182-0ff114530c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:06.163728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5e747-0a01-4651-b182-0ff114530c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:06.164004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5e747-0a01-4651-b182-0ff114530c3d admin admin] PATH_INFO: `/v3/projects/38e7137b2c0442a9867fce0aee3969af/users/687f1660219a4420b1623009fb387012/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:06.179729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5e747-0a01-4651-b182-0ff114530c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e7137b2c0442a9867fce0aee3969af, user_id=687f1660219a4420b1623009fb387012, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:06.183091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5e747-0a01-4651-b182-0ff114530c3d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:06.196726 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3870/7651] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:06 2022] PUT /identity/v3/projects/38e7137b2c0442a9867fce0aee3969af/users/687f1660219a4420b1623009fb387012/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:06.201560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4c178-d07d-4717-9ed1-8de36c7bb5e2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:06.201852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4c178-d07d-4717-9ed1-8de36c7bb5e2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:06.202131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4c178-d07d-4717-9ed1-8de36c7bb5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:06.215138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff84e57-7bb6-428d-9aa9-96a7557f88b8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:06.220531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f9a4c178-d07d-4717-9ed1-8de36c7bb5e2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:06.221788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f9a4c178-d07d-4717-9ed1-8de36c7bb5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:06.263775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c6d20-7e77-43ae-8d6d-5d076e50f078 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:06.264311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c6d20-7e77-43ae-8d6d-5d076e50f078 service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:06.264593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c6d20-7e77-43ae-8d6d-5d076e50f078 service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:06.264870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c6d20-7e77-43ae-8d6d-5d076e50f078 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:06.266632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-050c6d20-7e77-43ae-8d6d-5d076e50f078 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:06.280953 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3871/7652] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:06.291348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4c178-d07d-4717-9ed1-8de36c7bb5e2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:06.292802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f9a4c178-d07d-4717-9ed1-8de36c7bb5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:06.326405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c6d20-7e77-43ae-8d6d-5d076e50f078 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:06.327856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c6d20-7e77-43ae-8d6d-5d076e50f078 service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:06.337147 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3782/7653] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:06.354145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4c178-d07d-4717-9ed1-8de36c7bb5e2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:06.358316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab846a7b-446c-46aa-8d0c-929c342d86ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:06.359047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab846a7b-446c-46aa-8d0c-929c342d86ec service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:06.359334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab846a7b-446c-46aa-8d0c-929c342d86ec service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:06.359616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab846a7b-446c-46aa-8d0c-929c342d86ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:06.370959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab846a7b-446c-46aa-8d0c-929c342d86ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:06.372415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab846a7b-446c-46aa-8d0c-929c342d86ec service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:06.383893 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3872/7654] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:17:06.461633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c6d20-7e77-43ae-8d6d-5d076e50f078 service glance] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:06.473262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c6d20-7e77-43ae-8d6d-5d076e50f078 service glance] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:06.476878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017d4db-6f8c-4aac-aafb-02fdc78c8b6f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:06.478094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017d4db-6f8c-4aac-aafb-02fdc78c8b6f service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:06.478383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017d4db-6f8c-4aac-aafb-02fdc78c8b6f service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:06.478661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017d4db-6f8c-4aac-aafb-02fdc78c8b6f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:06.541846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:06.552590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017d4db-6f8c-4aac-aafb-02fdc78c8b6f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:06.554422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017d4db-6f8c-4aac-aafb-02fdc78c8b6f service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:06.564587 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3783/7655] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:17:07.168381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6530c9-af16-4485-bd89-ad4b2c7eb702 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:07.168931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6530c9-af16-4485-bd89-ad4b2c7eb702 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:07.169034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6530c9-af16-4485-bd89-ad4b2c7eb702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:07.198720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4d6530c9-af16-4485-bd89-ad4b2c7eb702 None None] MFA Rules not processed for user `687f1660219a4420b1623009fb387012`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:07.201349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4d6530c9-af16-4485-bd89-ad4b2c7eb702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:07.225899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:07.293758 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3873/7656] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:07.301076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017d4db-6f8c-4aac-aafb-02fdc78c8b6f service glance] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:07.302799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8017d4db-6f8c-4aac-aafb-02fdc78c8b6f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:07.376786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017d4db-6f8c-4aac-aafb-02fdc78c8b6f service glance] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:07.380595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d675b79-b2f0-4508-a141-7c7d56506bcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:07.381312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d675b79-b2f0-4508-a141-7c7d56506bcb service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:07.381696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d675b79-b2f0-4508-a141-7c7d56506bcb service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:07.382062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d675b79-b2f0-4508-a141-7c7d56506bcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:07.391071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d675b79-b2f0-4508-a141-7c7d56506bcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:07.392580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d675b79-b2f0-4508-a141-7c7d56506bcb service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:07.412954 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3784/7657] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:07.977231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6530c9-af16-4485-bd89-ad4b2c7eb702 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:07.988984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6530c9-af16-4485-bd89-ad4b2c7eb702 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:07.994576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:07.995093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:07.995378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:07.995655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] PATH_INFO: `/v3/users/35f9e2978eec461a9a7efc33ac270cdc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:07.996769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f9e2978eec461a9a7efc33ac270cdc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:07.998748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:08.047114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f9e2978eec461a9a7efc33ac270cdc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:08.062806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f9e2978eec461a9a7efc33ac270cdc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:08.070968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f9e2978eec461a9a7efc33ac270cdc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:08.071774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f9e2978eec461a9a7efc33ac270cdc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:08.072401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f9e2978eec461a9a7efc33ac270cdc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:08.087646 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3874/7658] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:07 2022] DELETE /identity/v3/users/35f9e2978eec461a9a7efc33ac270cdc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:08.093050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a956bd-3ef8-488a-b941-90e479485f4f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:08.093341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a956bd-3ef8-488a-b941-90e479485f4f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:08.093621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a956bd-3ef8-488a-b941-90e479485f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:08.113355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-25a956bd-3ef8-488a-b941-90e479485f4f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:08.114568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-25a956bd-3ef8-488a-b941-90e479485f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:08.197166 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3785/7659] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:08.207260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:08.209848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:08.274355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a9909-f980-4923-adc7-6f1232ee94e2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:08.278045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4c599-5613-4bbd-9a7d-562f026eb781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:08.278660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4c599-5613-4bbd-9a7d-562f026eb781 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:08.278950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4c599-5613-4bbd-9a7d-562f026eb781 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:08.279230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4c599-5613-4bbd-9a7d-562f026eb781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:08.289741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4c599-5613-4bbd-9a7d-562f026eb781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:08.291555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4c599-5613-4bbd-9a7d-562f026eb781 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:08.302338 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3875/7660] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:08.762671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a956bd-3ef8-488a-b941-90e479485f4f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:08.765559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-25a956bd-3ef8-488a-b941-90e479485f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:08.809956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a956bd-3ef8-488a-b941-90e479485f4f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:08.819686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:08.820345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:08.820635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:08.821541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] PATH_INFO: `/v3/projects/1e03a2cdfa444a92ba6eff58619e1ac9` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:08.824843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e03a2cdfa444a92ba6eff58619e1ac9)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:08.826961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:08.896448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e03a2cdfa444a92ba6eff58619e1ac9', 'request_id': 'req-673faaba-aa64-4873-92d3-7b4d370a945f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:08.907893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e03a2cdfa444a92ba6eff58619e1ac9', 'request_id': 'req-673faaba-aa64-4873-92d3-7b4d370a945f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:08.916276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e03a2cdfa444a92ba6eff58619e1ac9', 'request_id': 'req-673faaba-aa64-4873-92d3-7b4d370a945f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:08.929400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] The token cache is being invalidate because project 1e03a2cdfa444a92ba6eff58619e1ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:08.929981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:08.930882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:08.932419 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3786/7661] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:08 2022] DELETE /identity/v3/projects/1e03a2cdfa444a92ba6eff58619e1ac9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:08.936516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4c599-5613-4bbd-9a7d-562f026eb781 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:08.947500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4c599-5613-4bbd-9a7d-562f026eb781 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:08.986845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:09.027759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:09.028514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:09.028861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:09.029241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] PATH_INFO: `/v3/users/07e85f70511147a28434f6e5275425fe` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:09.030469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e85f70511147a28434f6e5275425fe)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:09.032560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:09.104704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e85f70511147a28434f6e5275425fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:09.131687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e85f70511147a28434f6e5275425fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:09.139739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e85f70511147a28434f6e5275425fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:09.141147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e85f70511147a28434f6e5275425fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:09.142309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e85f70511147a28434f6e5275425fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:09.166629 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3876/7662] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:08 2022] DELETE /identity/v3/users/07e85f70511147a28434f6e5275425fe => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:09.807385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:09.810185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:09.823160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:09.826838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:09.833892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673faaba-aa64-4873-92d3-7b4d370a945f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:09.852186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:09.909594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16d921-2437-4c0a-aab4-bf4d05e31d49 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:09.913375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46e8a6-12a7-42f3-9227-806290de6e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:09.914031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46e8a6-12a7-42f3-9227-806290de6e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:09.915412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46e8a6-12a7-42f3-9227-806290de6e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:09.915868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:09.916160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46e8a6-12a7-42f3-9227-806290de6e80 admin admin] PATH_INFO: `/v3/users/687f1660219a4420b1623009fb387012` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:09.916514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:09.916837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:09.917152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a admin admin] PATH_INFO: `/v3/projects/c843e59a12634cb0be89913074d9c52a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:09.918344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c843e59a12634cb0be89913074d9c52a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:09.919259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46e8a6-12a7-42f3-9227-806290de6e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=687f1660219a4420b1623009fb387012)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:09.920275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:09.921437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46e8a6-12a7-42f3-9227-806290de6e80 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:09.982027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb46e8a6-12a7-42f3-9227-806290de6e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '687f1660219a4420b1623009fb387012'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:09.989572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c843e59a12634cb0be89913074d9c52a', 'request_id': 'req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:09.994203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:09.998933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb46e8a6-12a7-42f3-9227-806290de6e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '687f1660219a4420b1623009fb387012'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.000047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb46e8a6-12a7-42f3-9227-806290de6e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '687f1660219a4420b1623009fb387012'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.000698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c843e59a12634cb0be89913074d9c52a', 'request_id': 'req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.011336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb46e8a6-12a7-42f3-9227-806290de6e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '687f1660219a4420b1623009fb387012'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.012494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c843e59a12634cb0be89913074d9c52a', 'request_id': 'req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.022027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bb46e8a6-12a7-42f3-9227-806290de6e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '687f1660219a4420b1623009fb387012'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.026757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a admin admin] The token cache is being invalidate because project c843e59a12634cb0be89913074d9c52a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:10.027416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.027982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c970b89e-25c6-4d2f-81e2-10b31bfbc39a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.028868 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3787/7663] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:09 2022] DELETE /identity/v3/users/687f1660219a4420b1623009fb387012 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:10.033219 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3877/7664] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:09 2022] DELETE /identity/v3/projects/c843e59a12634cb0be89913074d9c52a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:10.039009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db600df-cdba-44f1-9ed6-1f02c10d10c9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.040058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db600df-cdba-44f1-9ed6-1f02c10d10c9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.040407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db600df-cdba-44f1-9ed6-1f02c10d10c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.063516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3db600df-cdba-44f1-9ed6-1f02c10d10c9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:10.065286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3db600df-cdba-44f1-9ed6-1f02c10d10c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:10.141689 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3788/7665] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:10.150576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109ffc3-d201-45e4-917b-26723938f278 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.150880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109ffc3-d201-45e4-917b-26723938f278 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.151201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109ffc3-d201-45e4-917b-26723938f278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.174858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7109ffc3-d201-45e4-917b-26723938f278 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:10.176115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7109ffc3-d201-45e4-917b-26723938f278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:10.186856 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3878/7666] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:10.191255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db600df-cdba-44f1-9ed6-1f02c10d10c9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:10.201914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db600df-cdba-44f1-9ed6-1f02c10d10c9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:10.207263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840df2c5-2cd5-46e3-87d0-fe4fdb3fe7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:10.207979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840df2c5-2cd5-46e3-87d0-fe4fdb3fe7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.208332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840df2c5-2cd5-46e3-87d0-fe4fdb3fe7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.208615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840df2c5-2cd5-46e3-87d0-fe4fdb3fe7f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.209444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840df2c5-2cd5-46e3-87d0-fe4fdb3fe7f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:10.209756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840df2c5-2cd5-46e3-87d0-fe4fdb3fe7f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:10.211532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840df2c5-2cd5-46e3-87d0-fe4fdb3fe7f3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:10.232336 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3789/7667] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:10.233944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109ffc3-d201-45e4-917b-26723938f278 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:10.253568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840df2c5-2cd5-46e3-87d0-fe4fdb3fe7f3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:10.255464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-840df2c5-2cd5-46e3-87d0-fe4fdb3fe7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:10.265845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109ffc3-d201-45e4-917b-26723938f278 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:10.270983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc4a96-5923-4b4b-96ad-94f21982bb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:10.271609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc4a96-5923-4b4b-96ad-94f21982bb15 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.271899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc4a96-5923-4b4b-96ad-94f21982bb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.272271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc4a96-5923-4b4b-96ad-94f21982bb15 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.272822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc4a96-5923-4b4b-96ad-94f21982bb15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:10.275502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc4a96-5923-4b4b-96ad-94f21982bb15 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:10.282782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840df2c5-2cd5-46e3-87d0-fe4fdb3fe7f3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:10.288058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94284f2-294f-458c-9102-70d8d8f5c1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:10.288737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94284f2-294f-458c-9102-70d8d8f5c1ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.289131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94284f2-294f-458c-9102-70d8d8f5c1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.289489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94284f2-294f-458c-9102-70d8d8f5c1ce admin admin] PATH_INFO: `/v3/projects/38e7137b2c0442a9867fce0aee3969af` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.290713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94284f2-294f-458c-9102-70d8d8f5c1ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e7137b2c0442a9867fce0aee3969af)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:10.293100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94284f2-294f-458c-9102-70d8d8f5c1ce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:10.313356 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3879/7668] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:10 2022] POST /identity/v3/projects => generated 367 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:10.315374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc4a96-5923-4b4b-96ad-94f21982bb15 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:10.316042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:10.399399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f94284f2-294f-458c-9102-70d8d8f5c1ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e7137b2c0442a9867fce0aee3969af', 'request_id': 'req-f94284f2-294f-458c-9102-70d8d8f5c1ce'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.415612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f94284f2-294f-458c-9102-70d8d8f5c1ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e7137b2c0442a9867fce0aee3969af', 'request_id': 'req-f94284f2-294f-458c-9102-70d8d8f5c1ce'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.424106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f94284f2-294f-458c-9102-70d8d8f5c1ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e7137b2c0442a9867fce0aee3969af', 'request_id': 'req-f94284f2-294f-458c-9102-70d8d8f5c1ce'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.437533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f94284f2-294f-458c-9102-70d8d8f5c1ce admin admin] The token cache is being invalidate because project 38e7137b2c0442a9867fce0aee3969af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:10.438024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f94284f2-294f-458c-9102-70d8d8f5c1ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.438723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f94284f2-294f-458c-9102-70d8d8f5c1ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:10.442417 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3790/7669] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:10 2022] DELETE /identity/v3/projects/38e7137b2c0442a9867fce0aee3969af => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:10.447535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d46096-6cbc-4371-b8ee-1f1b2fdffcfc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.448027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d46096-6cbc-4371-b8ee-1f1b2fdffcfc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.448392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d46096-6cbc-4371-b8ee-1f1b2fdffcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.458504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc4a96-5923-4b4b-96ad-94f21982bb15 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:10.475431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b6d46096-6cbc-4371-b8ee-1f1b2fdffcfc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:10.476735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b6d46096-6cbc-4371-b8ee-1f1b2fdffcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:10.559177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a9c1a-502c-4648-95ac-041b90803df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:10.561245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a9c1a-502c-4648-95ac-041b90803df5 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.561661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a9c1a-502c-4648-95ac-041b90803df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.562124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a9c1a-502c-4648-95ac-041b90803df5 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.563531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a9c1a-502c-4648-95ac-041b90803df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:10.565930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a9c1a-502c-4648-95ac-041b90803df5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:10.589228 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3791/7670] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:10.599351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3e98e-3d7e-4512-a271-e0a02b4b192d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.599804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3e98e-3d7e-4512-a271-e0a02b4b192d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.600299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3e98e-3d7e-4512-a271-e0a02b4b192d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.616916 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3880/7671] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:10 2022] POST /identity/v3/users => generated 284 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:10.620953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a9c1a-502c-4648-95ac-041b90803df5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:10.623068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-433a9c1a-502c-4648-95ac-041b90803df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:10.636722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a9c1a-502c-4648-95ac-041b90803df5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:10.641825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e9a44-a6bb-478f-82fb-dee3b362dfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:10.642439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e9a44-a6bb-478f-82fb-dee3b362dfb9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.642720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e9a44-a6bb-478f-82fb-dee3b362dfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.643000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e9a44-a6bb-478f-82fb-dee3b362dfb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.643650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e9a44-a6bb-478f-82fb-dee3b362dfb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:10.645475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e9a44-a6bb-478f-82fb-dee3b362dfb9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:10.649689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e5f3e98e-3d7e-4512-a271-e0a02b4b192d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:10.651633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e5f3e98e-3d7e-4512-a271-e0a02b4b192d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:10.656968 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3881/7672] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:17:10.660574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e9a44-a6bb-478f-82fb-dee3b362dfb9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:10.662439 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3792/7673] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:10.667746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3e98e-3d7e-4512-a271-e0a02b4b192d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:10.671029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e9a44-a6bb-478f-82fb-dee3b362dfb9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:10.677073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc129685-462c-4795-872c-f1250b2795d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:10.678098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc129685-462c-4795-872c-f1250b2795d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.678098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc129685-462c-4795-872c-f1250b2795d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.678325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc129685-462c-4795-872c-f1250b2795d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.678852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc129685-462c-4795-872c-f1250b2795d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:10.680546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc129685-462c-4795-872c-f1250b2795d0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:10.688209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3e98e-3d7e-4512-a271-e0a02b4b192d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:10.692299 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3882/7674] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:10.694658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649d0b0-cc78-47f6-96a1-349dea876a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:10.696460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649d0b0-cc78-47f6-96a1-349dea876a28 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.696888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc129685-462c-4795-872c-f1250b2795d0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:10.698276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649d0b0-cc78-47f6-96a1-349dea876a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.698808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649d0b0-cc78-47f6-96a1-349dea876a28 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.700090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649d0b0-cc78-47f6-96a1-349dea876a28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:10.700707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649d0b0-cc78-47f6-96a1-349dea876a28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:10.703200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649d0b0-cc78-47f6-96a1-349dea876a28 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:10.709884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc129685-462c-4795-872c-f1250b2795d0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:10.723367 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3793/7675] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:10.727401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649d0b0-cc78-47f6-96a1-349dea876a28 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:10.734193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907353da-a8fb-4a8b-8078-dc544f5b30db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:10.734877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907353da-a8fb-4a8b-8078-dc544f5b30db admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.735206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907353da-a8fb-4a8b-8078-dc544f5b30db admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.735454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907353da-a8fb-4a8b-8078-dc544f5b30db admin admin] PATH_INFO: `/v3/projects/b8ef8e80542e462b91cbda1654e024e4/users/c322885b6ead449a80498ba6b865566b/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.747483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649d0b0-cc78-47f6-96a1-349dea876a28 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:10.757782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370680b-f985-4dbf-a478-c0fa97c00851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:10.759340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370680b-f985-4dbf-a478-c0fa97c00851 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.759816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370680b-f985-4dbf-a478-c0fa97c00851 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.760279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370680b-f985-4dbf-a478-c0fa97c00851 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.761251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370680b-f985-4dbf-a478-c0fa97c00851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:10.764134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370680b-f985-4dbf-a478-c0fa97c00851 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:10.768131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907353da-a8fb-4a8b-8078-dc544f5b30db admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ef8e80542e462b91cbda1654e024e4, user_id=c322885b6ead449a80498ba6b865566b, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:10.771496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907353da-a8fb-4a8b-8078-dc544f5b30db admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:10.801775 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3883/7676] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:10 2022] PUT /identity/v3/projects/b8ef8e80542e462b91cbda1654e024e4/users/c322885b6ead449a80498ba6b865566b/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:10.812266 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3794/7677] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:10 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:10.818394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370680b-f985-4dbf-a478-c0fa97c00851 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:10.819433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c01b10-8022-4b2c-8567-78f8c44629c6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.819791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c01b10-8022-4b2c-8567-78f8c44629c6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.820180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c01b10-8022-4b2c-8567-78f8c44629c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.830053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:10.867207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-51c01b10-8022-4b2c-8567-78f8c44629c6 None None] MFA Rules not processed for user `c322885b6ead449a80498ba6b865566b`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:10.868867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-51c01b10-8022-4b2c-8567-78f8c44629c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:10.911905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370680b-f985-4dbf-a478-c0fa97c00851 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:10.920968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158a21-d452-4ea2-ab00-eba740232341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:10.922288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158a21-d452-4ea2-ab00-eba740232341 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:10.923059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158a21-d452-4ea2-ab00-eba740232341 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:10.923556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158a21-d452-4ea2-ab00-eba740232341 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:10.924531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29158a21-d452-4ea2-ab00-eba740232341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:10.928237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29158a21-d452-4ea2-ab00-eba740232341 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:10.958129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:10.997896 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3795/7679] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:10 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:10.997896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158a21-d452-4ea2-ab00-eba740232341 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:11.009783 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3884/7678] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:11.012325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c01b10-8022-4b2c-8567-78f8c44629c6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:11.015729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-51c01b10-8022-4b2c-8567-78f8c44629c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:11.016650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158a21-d452-4ea2-ab00-eba740232341 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:11.022676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db507703-ba0e-47e7-b04e-2e76ec1bd751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:11.023867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db507703-ba0e-47e7-b04e-2e76ec1bd751 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:11.024182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db507703-ba0e-47e7-b04e-2e76ec1bd751 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:11.024531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db507703-ba0e-47e7-b04e-2e76ec1bd751 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:11.025364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db507703-ba0e-47e7-b04e-2e76ec1bd751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:11.027655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db507703-ba0e-47e7-b04e-2e76ec1bd751 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:11.062386 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3796/7680] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:11.066192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db507703-ba0e-47e7-b04e-2e76ec1bd751 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:11.091528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db507703-ba0e-47e7-b04e-2e76ec1bd751 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:11.103810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f414be4-1e1f-4976-b14e-1705a75a2527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:11.104523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f414be4-1e1f-4976-b14e-1705a75a2527 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:11.105673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f414be4-1e1f-4976-b14e-1705a75a2527 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:11.105966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f414be4-1e1f-4976-b14e-1705a75a2527 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:11.106768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f414be4-1e1f-4976-b14e-1705a75a2527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:11.108397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f414be4-1e1f-4976-b14e-1705a75a2527 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:11.167827 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3797/7681] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:11.172053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f414be4-1e1f-4976-b14e-1705a75a2527 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:11.174558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c01b10-8022-4b2c-8567-78f8c44629c6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:11.184849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af56f5-8609-4c69-b7c6-2977b4efbe97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:11.186242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af56f5-8609-4c69-b7c6-2977b4efbe97 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:11.186909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af56f5-8609-4c69-b7c6-2977b4efbe97 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:11.187411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af56f5-8609-4c69-b7c6-2977b4efbe97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:11.208039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af56f5-8609-4c69-b7c6-2977b4efbe97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:11.215430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af56f5-8609-4c69-b7c6-2977b4efbe97 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:11.220608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f414be4-1e1f-4976-b14e-1705a75a2527 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:11.233485 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3885/7682] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:11.252985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc08ed-d1cd-437e-8c12-aa490c06fe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:11.254923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc08ed-d1cd-437e-8c12-aa490c06fe8f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:11.257677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc08ed-d1cd-437e-8c12-aa490c06fe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:11.258793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc08ed-d1cd-437e-8c12-aa490c06fe8f admin admin] PATH_INFO: `/v3/projects/f0e0a7bbfbdf46bebae44db90c248290/users/d2cfd53bda54427e9c2a337824fa4176/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:11.292192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc08ed-d1cd-437e-8c12-aa490c06fe8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e0a7bbfbdf46bebae44db90c248290, user_id=d2cfd53bda54427e9c2a337824fa4176, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:11.295670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc08ed-d1cd-437e-8c12-aa490c06fe8f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:11.318197 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3798/7683] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:11 2022] PUT /identity/v3/projects/f0e0a7bbfbdf46bebae44db90c248290/users/d2cfd53bda54427e9c2a337824fa4176/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:11.328952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7c807-73f5-4fe5-8c67-68167d898fed None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:11.329198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7c807-73f5-4fe5-8c67-68167d898fed None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:11.329372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7c807-73f5-4fe5-8c67-68167d898fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:11.386213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3df7c807-73f5-4fe5-8c67-68167d898fed None None] MFA Rules not processed for user `d2cfd53bda54427e9c2a337824fa4176`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:11.388250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3df7c807-73f5-4fe5-8c67-68167d898fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:11.492085 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3886/7684] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:11.504855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc08ed-d1cd-437e-8c12-aa490c06fe8f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:11.506780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8fbc08ed-d1cd-437e-8c12-aa490c06fe8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:11.585695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc08ed-d1cd-437e-8c12-aa490c06fe8f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:11.589671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b9ab0-e4e6-4b19-a0d0-6e47292cac48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:11.590315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b9ab0-e4e6-4b19-a0d0-6e47292cac48 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:11.590596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b9ab0-e4e6-4b19-a0d0-6e47292cac48 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:11.590883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b9ab0-e4e6-4b19-a0d0-6e47292cac48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:11.601275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b9ab0-e4e6-4b19-a0d0-6e47292cac48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:11.603551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b9ab0-e4e6-4b19-a0d0-6e47292cac48 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:11.614360 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3799/7685] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:14.884086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7c807-73f5-4fe5-8c67-68167d898fed None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:14.911068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b9ab0-e4e6-4b19-a0d0-6e47292cac48 service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:14.913029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-784b9ab0-e4e6-4b19-a0d0-6e47292cac48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:14.917389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:14.966110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7c807-73f5-4fe5-8c67-68167d898fed None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:14.971288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:14.971939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:14.972213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:14.972499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] PATH_INFO: `/v3/users/d2cfd53bda54427e9c2a337824fa4176` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:14.973688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2cfd53bda54427e9c2a337824fa4176)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:14.975662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:15.020477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b9ab0-e4e6-4b19-a0d0-6e47292cac48 service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:15.027395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9222c-3d7d-4b03-acbe-754a95956a81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:15.028019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9222c-3d7d-4b03-acbe-754a95956a81 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:15.028304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9222c-3d7d-4b03-acbe-754a95956a81 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:15.028645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9222c-3d7d-4b03-acbe-754a95956a81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:15.030831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-70f9222c-3d7d-4b03-acbe-754a95956a81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:15.033973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2cfd53bda54427e9c2a337824fa4176'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:15.046098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9222c-3d7d-4b03-acbe-754a95956a81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:15.047557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9222c-3d7d-4b03-acbe-754a95956a81 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:15.050516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2cfd53bda54427e9c2a337824fa4176'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:15.063513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2cfd53bda54427e9c2a337824fa4176'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:15.064733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2cfd53bda54427e9c2a337824fa4176'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:15.065383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2cfd53bda54427e9c2a337824fa4176'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:15.067077 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3800/7686] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:17:15.081218 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3887/7687] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:14 2022] DELETE /identity/v3/users/d2cfd53bda54427e9c2a337824fa4176 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:15.087709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ecb6d-8419-4357-a701-2774a8c8e35a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:15.088023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ecb6d-8419-4357-a701-2774a8c8e35a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:15.088337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ecb6d-8419-4357-a701-2774a8c8e35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:15.135659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d19ecb6d-8419-4357-a701-2774a8c8e35a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:15.140486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d19ecb6d-8419-4357-a701-2774a8c8e35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:15.156161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:15.217706 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3801/7688] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:15.230400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:15.232599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:15.299407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeec7c1-89ac-4bc7-94c3-30ce224f3efa admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:15.303116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a708-6291-4c3b-8975-a1db84c25bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:15.303731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a708-6291-4c3b-8975-a1db84c25bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:15.304016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a708-6291-4c3b-8975-a1db84c25bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:15.304292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a708-6291-4c3b-8975-a1db84c25bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:15.316398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942a708-6291-4c3b-8975-a1db84c25bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:15.317935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942a708-6291-4c3b-8975-a1db84c25bf5 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:15.329159 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3888/7689] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:15.828232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ecb6d-8419-4357-a701-2774a8c8e35a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:15.829956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d19ecb6d-8419-4357-a701-2774a8c8e35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:15.845853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ecb6d-8419-4357-a701-2774a8c8e35a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:15.854358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:15.855923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:15.856312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:15.856729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d admin admin] PATH_INFO: `/v3/projects/f0e0a7bbfbdf46bebae44db90c248290` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:15.858749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e0a7bbfbdf46bebae44db90c248290)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:15.861622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:15.934270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e0a7bbfbdf46bebae44db90c248290', 'request_id': 'req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:15.943438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e0a7bbfbdf46bebae44db90c248290', 'request_id': 'req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:15.952293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e0a7bbfbdf46bebae44db90c248290', 'request_id': 'req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:15.966006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d admin admin] The token cache is being invalidate because project f0e0a7bbfbdf46bebae44db90c248290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:15.966293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:15.967243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1bca9e1b-c1f8-4d2b-ab7f-bbe4f3959f7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:15.972577 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3802/7690] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:15 2022] DELETE /identity/v3/projects/f0e0a7bbfbdf46bebae44db90c248290 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:15.974314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c093c-452d-4437-aa0f-c1eb2f738817 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:15.974765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c093c-452d-4437-aa0f-c1eb2f738817 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:15.974765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c093c-452d-4437-aa0f-c1eb2f738817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:16.006338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ed5c093c-452d-4437-aa0f-c1eb2f738817 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:16.007580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ed5c093c-452d-4437-aa0f-c1eb2f738817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:16.046497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:16.099266 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3889/7691] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:16.207613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212483e-d538-4002-b457-c3d9b2a9f383 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:16.207939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212483e-d538-4002-b457-c3d9b2a9f383 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:16.208180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212483e-d538-4002-b457-c3d9b2a9f383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:16.235220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f212483e-d538-4002-b457-c3d9b2a9f383 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:16.237356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f212483e-d538-4002-b457-c3d9b2a9f383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:16.270931 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3803/7692] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:16.272603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c093c-452d-4437-aa0f-c1eb2f738817 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:16.294770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c093c-452d-4437-aa0f-c1eb2f738817 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:16.304806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080fd77-a47b-45ef-a4ac-c86a182aa78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:16.305669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080fd77-a47b-45ef-a4ac-c86a182aa78a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:16.306019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080fd77-a47b-45ef-a4ac-c86a182aa78a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:16.306533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080fd77-a47b-45ef-a4ac-c86a182aa78a admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:16.307197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080fd77-a47b-45ef-a4ac-c86a182aa78a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:16.307582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080fd77-a47b-45ef-a4ac-c86a182aa78a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:16.309723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080fd77-a47b-45ef-a4ac-c86a182aa78a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:16.325690 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3890/7693] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:17:16.327564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212483e-d538-4002-b457-c3d9b2a9f383 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:16.339981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212483e-d538-4002-b457-c3d9b2a9f383 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:16.348987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821a9dc-0be6-4fb9-bbe1-3c7fa822b4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:16.349779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821a9dc-0be6-4fb9-bbe1-3c7fa822b4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:16.350141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821a9dc-0be6-4fb9-bbe1-3c7fa822b4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:16.350498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821a9dc-0be6-4fb9-bbe1-3c7fa822b4c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:16.351288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821a9dc-0be6-4fb9-bbe1-3c7fa822b4c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:16.355538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821a9dc-0be6-4fb9-bbe1-3c7fa822b4c8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:16.392943 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3804/7694] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:16 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:16.396616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080fd77-a47b-45ef-a4ac-c86a182aa78a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:16.408703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080fd77-a47b-45ef-a4ac-c86a182aa78a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:16.516781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d04ae-13c7-4556-880b-774abe7e7fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:16.517849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d04ae-13c7-4556-880b-774abe7e7fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:16.518483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d04ae-13c7-4556-880b-774abe7e7fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:16.519142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d04ae-13c7-4556-880b-774abe7e7fe8 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:16.522490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d04ae-13c7-4556-880b-774abe7e7fe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:16.528215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d04ae-13c7-4556-880b-774abe7e7fe8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:16.555995 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3891/7695] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:16 2022] POST /identity/v3/users => generated 284 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:16.560760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821a9dc-0be6-4fb9-bbe1-3c7fa822b4c8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:16.588739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821a9dc-0be6-4fb9-bbe1-3c7fa822b4c8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:16.594745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77142db-6636-43f5-9b5f-476f80df3b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:16.595442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77142db-6636-43f5-9b5f-476f80df3b66 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:16.595755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77142db-6636-43f5-9b5f-476f80df3b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:16.596431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77142db-6636-43f5-9b5f-476f80df3b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:16.597154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77142db-6636-43f5-9b5f-476f80df3b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:16.598869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77142db-6636-43f5-9b5f-476f80df3b66 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:16.611490 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3805/7696] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:16.616655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d04ae-13c7-4556-880b-774abe7e7fe8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:16.637749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d04ae-13c7-4556-880b-774abe7e7fe8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:16.653197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210d37b-cfdb-4e8c-a7bb-4e68ded23650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:16.654236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210d37b-cfdb-4e8c-a7bb-4e68ded23650 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:16.655482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210d37b-cfdb-4e8c-a7bb-4e68ded23650 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:16.656123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210d37b-cfdb-4e8c-a7bb-4e68ded23650 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:16.657485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210d37b-cfdb-4e8c-a7bb-4e68ded23650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:16.659136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210d37b-cfdb-4e8c-a7bb-4e68ded23650 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:16.698388 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3892/7697] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:16.699529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77142db-6636-43f5-9b5f-476f80df3b66 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:16.716982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77142db-6636-43f5-9b5f-476f80df3b66 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:16.723137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f6971-7ee8-42c8-8b69-926c190862a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:16.723823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f6971-7ee8-42c8-8b69-926c190862a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:16.724112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f6971-7ee8-42c8-8b69-926c190862a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:16.724395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f6971-7ee8-42c8-8b69-926c190862a8 admin admin] PATH_INFO: `/v3/projects/af6ca48ce44a4fef8267b34481ae24ef/users/351de7acdd544d159c9f0d9d7f6b05f0/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:16.747867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f6971-7ee8-42c8-8b69-926c190862a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af6ca48ce44a4fef8267b34481ae24ef, user_id=351de7acdd544d159c9f0d9d7f6b05f0, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:16.751656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f6971-7ee8-42c8-8b69-926c190862a8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:16.768465 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3806/7698] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:16 2022] PUT /identity/v3/projects/af6ca48ce44a4fef8267b34481ae24ef/users/351de7acdd544d159c9f0d9d7f6b05f0/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:16.779630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b1cb9-9892-4f74-9ffb-8cab9657155e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:16.780063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b1cb9-9892-4f74-9ffb-8cab9657155e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:16.780440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b1cb9-9892-4f74-9ffb-8cab9657155e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:16.811025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4e8b1cb9-9892-4f74-9ffb-8cab9657155e None None] MFA Rules not processed for user `351de7acdd544d159c9f0d9d7f6b05f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:16.812586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4e8b1cb9-9892-4f74-9ffb-8cab9657155e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:16.903803 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3893/7699] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:16.918751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f6971-7ee8-42c8-8b69-926c190862a8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:16.921604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-454f6971-7ee8-42c8-8b69-926c190862a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:16.981509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:17.003482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f6971-7ee8-42c8-8b69-926c190862a8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:17.007320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e6122-c3a3-45ba-ae98-758b86bdcbea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:17.007977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e6122-c3a3-45ba-ae98-758b86bdcbea service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:17.008886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e6122-c3a3-45ba-ae98-758b86bdcbea service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:17.009196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e6122-c3a3-45ba-ae98-758b86bdcbea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:17.018816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e6122-c3a3-45ba-ae98-758b86bdcbea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:17.020314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e6122-c3a3-45ba-ae98-758b86bdcbea service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:17.030743 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3807/7700] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:20.601932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b1cb9-9892-4f74-9ffb-8cab9657155e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:20.669759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:20.716968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b1cb9-9892-4f74-9ffb-8cab9657155e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:20.722789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:20.723466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:20.723902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:20.724195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] PATH_INFO: `/v3/users/351de7acdd544d159c9f0d9d7f6b05f0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:20.731571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=351de7acdd544d159c9f0d9d7f6b05f0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:20.735863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:20.812726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '351de7acdd544d159c9f0d9d7f6b05f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:20.828594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '351de7acdd544d159c9f0d9d7f6b05f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:20.840496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '351de7acdd544d159c9f0d9d7f6b05f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:20.841497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '351de7acdd544d159c9f0d9d7f6b05f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:20.842342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '351de7acdd544d159c9f0d9d7f6b05f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:20.859451 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3894/7701] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:20 2022] DELETE /identity/v3/users/351de7acdd544d159c9f0d9d7f6b05f0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:20.865059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb32d4-3134-459b-abf2-aeb52651c694 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:20.865726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb32d4-3134-459b-abf2-aeb52651c694 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:20.866017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb32d4-3134-459b-abf2-aeb52651c694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:20.893265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-90bb32d4-3134-459b-abf2-aeb52651c694 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:20.894765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-90bb32d4-3134-459b-abf2-aeb52651c694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:20.991298 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3808/7702] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:21.003020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:21.005119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:21.096185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb77531-466c-4d19-9233-44614ba192b7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:21.100517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2b7a6-2fce-474d-a1da-dbf3b5a5d135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:21.101407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2b7a6-2fce-474d-a1da-dbf3b5a5d135 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:21.102420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2b7a6-2fce-474d-a1da-dbf3b5a5d135 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:21.102889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2b7a6-2fce-474d-a1da-dbf3b5a5d135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:21.115263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2b7a6-2fce-474d-a1da-dbf3b5a5d135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:21.116921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2b7a6-2fce-474d-a1da-dbf3b5a5d135 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:21.127974 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3895/7703] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:21.551167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb32d4-3134-459b-abf2-aeb52651c694 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:21.552786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-90bb32d4-3134-459b-abf2-aeb52651c694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:21.571578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb32d4-3134-459b-abf2-aeb52651c694 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:21.588597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477d2cc-87f7-4171-9fa9-5ead503e8b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:21.589673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477d2cc-87f7-4171-9fa9-5ead503e8b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:21.589964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477d2cc-87f7-4171-9fa9-5ead503e8b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:21.590247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477d2cc-87f7-4171-9fa9-5ead503e8b44 admin admin] PATH_INFO: `/v3/projects/af6ca48ce44a4fef8267b34481ae24ef` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:21.595402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477d2cc-87f7-4171-9fa9-5ead503e8b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af6ca48ce44a4fef8267b34481ae24ef)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:21.597403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477d2cc-87f7-4171-9fa9-5ead503e8b44 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:21.671163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0477d2cc-87f7-4171-9fa9-5ead503e8b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af6ca48ce44a4fef8267b34481ae24ef', 'request_id': 'req-0477d2cc-87f7-4171-9fa9-5ead503e8b44'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:21.684784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0477d2cc-87f7-4171-9fa9-5ead503e8b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af6ca48ce44a4fef8267b34481ae24ef', 'request_id': 'req-0477d2cc-87f7-4171-9fa9-5ead503e8b44'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:21.697573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0477d2cc-87f7-4171-9fa9-5ead503e8b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af6ca48ce44a4fef8267b34481ae24ef', 'request_id': 'req-0477d2cc-87f7-4171-9fa9-5ead503e8b44'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:21.754253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0477d2cc-87f7-4171-9fa9-5ead503e8b44 admin admin] The token cache is being invalidate because project af6ca48ce44a4fef8267b34481ae24ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:21.754558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0477d2cc-87f7-4171-9fa9-5ead503e8b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:21.755758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0477d2cc-87f7-4171-9fa9-5ead503e8b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:21.758857 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3809/7704] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:21 2022] DELETE /identity/v3/projects/af6ca48ce44a4fef8267b34481ae24ef => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:21.763560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242d08-4133-48d3-b25f-81987f6ea439 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:21.764044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242d08-4133-48d3-b25f-81987f6ea439 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:21.764134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242d08-4133-48d3-b25f-81987f6ea439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:21.816388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-48242d08-4133-48d3-b25f-81987f6ea439 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:21.817695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-48242d08-4133-48d3-b25f-81987f6ea439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:21.858670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:21.904553 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3896/7705] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:21.912663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe482f-3142-4724-8a4a-65c50b60df6f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:21.913163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe482f-3142-4724-8a4a-65c50b60df6f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:21.913163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe482f-3142-4724-8a4a-65c50b60df6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:21.941950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1ebe482f-3142-4724-8a4a-65c50b60df6f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:21.943985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1ebe482f-3142-4724-8a4a-65c50b60df6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:21.954402 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3810/7706] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:21.958723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242d08-4133-48d3-b25f-81987f6ea439 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:21.986457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242d08-4133-48d3-b25f-81987f6ea439 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:21.992056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67637a9-24f7-4867-9619-5bda6fd48423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:21.992737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67637a9-24f7-4867-9619-5bda6fd48423 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:21.993061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67637a9-24f7-4867-9619-5bda6fd48423 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:21.993344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67637a9-24f7-4867-9619-5bda6fd48423 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:21.994014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67637a9-24f7-4867-9619-5bda6fd48423 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:21.994322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67637a9-24f7-4867-9619-5bda6fd48423 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:21.996012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67637a9-24f7-4867-9619-5bda6fd48423 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:22.014809 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3897/7707] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:22.015809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe482f-3142-4724-8a4a-65c50b60df6f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:22.026106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe482f-3142-4724-8a4a-65c50b60df6f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:22.032254 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba30e10-18c5-4387-8d44-55ecfb282e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:22.032889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba30e10-18c5-4387-8d44-55ecfb282e00 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:22.033205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba30e10-18c5-4387-8d44-55ecfb282e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:22.033486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba30e10-18c5-4387-8d44-55ecfb282e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:22.034138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba30e10-18c5-4387-8d44-55ecfb282e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:22.036129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba30e10-18c5-4387-8d44-55ecfb282e00 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:22.071783 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3811/7708] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:22 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:22.075884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67637a9-24f7-4867-9619-5bda6fd48423 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:22.090215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67637a9-24f7-4867-9619-5bda6fd48423 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:22.180600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf66314-c697-4419-8825-aba2115e03b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:22.180888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf66314-c697-4419-8825-aba2115e03b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:22.181164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf66314-c697-4419-8825-aba2115e03b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:22.181607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf66314-c697-4419-8825-aba2115e03b2 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:22.182655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf66314-c697-4419-8825-aba2115e03b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:22.184281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf66314-c697-4419-8825-aba2115e03b2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:22.227198 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3898/7709] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:22 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:22.231732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba30e10-18c5-4387-8d44-55ecfb282e00 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:22.246772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba30e10-18c5-4387-8d44-55ecfb282e00 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:22.252918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11225324-5205-4932-9bcf-e9374229ec2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:22.253571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11225324-5205-4932-9bcf-e9374229ec2e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:22.253943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11225324-5205-4932-9bcf-e9374229ec2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:22.254120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11225324-5205-4932-9bcf-e9374229ec2e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:22.254778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11225324-5205-4932-9bcf-e9374229ec2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:22.257074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11225324-5205-4932-9bcf-e9374229ec2e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:22.272235 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3812/7710] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:17:22.273441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf66314-c697-4419-8825-aba2115e03b2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:22.283997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf66314-c697-4419-8825-aba2115e03b2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:22.288847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f159a-1b99-456a-bbeb-8e0633b8bc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:22.289595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f159a-1b99-456a-bbeb-8e0633b8bc30 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:22.290056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f159a-1b99-456a-bbeb-8e0633b8bc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:22.290412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f159a-1b99-456a-bbeb-8e0633b8bc30 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:22.291238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f159a-1b99-456a-bbeb-8e0633b8bc30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:22.293192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f159a-1b99-456a-bbeb-8e0633b8bc30 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:22.306962 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3899/7711] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:17:22.311451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11225324-5205-4932-9bcf-e9374229ec2e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:22.322482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11225324-5205-4932-9bcf-e9374229ec2e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:22.327357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9444767-9c93-4af0-8f1a-de4325fd7412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:22.328130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9444767-9c93-4af0-8f1a-de4325fd7412 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:22.328357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9444767-9c93-4af0-8f1a-de4325fd7412 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:22.328938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9444767-9c93-4af0-8f1a-de4325fd7412 admin admin] PATH_INFO: `/v3/projects/08011156dc184f1cb1691565fbef9221/users/872f91cddd294d64b92eca1c76ad9435/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:22.344199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9444767-9c93-4af0-8f1a-de4325fd7412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08011156dc184f1cb1691565fbef9221, user_id=872f91cddd294d64b92eca1c76ad9435, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:22.346929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9444767-9c93-4af0-8f1a-de4325fd7412 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:22.357024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:22.363356 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3813/7712] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:22 2022] PUT /identity/v3/projects/08011156dc184f1cb1691565fbef9221/users/872f91cddd294d64b92eca1c76ad9435/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:22.375672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970df82-02b2-4444-9d94-d33658d82f28 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:22.376462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970df82-02b2-4444-9d94-d33658d82f28 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:22.376821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970df82-02b2-4444-9d94-d33658d82f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:22.405684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a970df82-02b2-4444-9d94-d33658d82f28 None None] MFA Rules not processed for user `872f91cddd294d64b92eca1c76ad9435`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:22.406934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a970df82-02b2-4444-9d94-d33658d82f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:22.474217 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3900/7713] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:22.485697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9444767-9c93-4af0-8f1a-de4325fd7412 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:22.487478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a9444767-9c93-4af0-8f1a-de4325fd7412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:22.552060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9444767-9c93-4af0-8f1a-de4325fd7412 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:22.556274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5c409-6cf2-42af-bcb2-ff6149089985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:22.556884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5c409-6cf2-42af-bcb2-ff6149089985 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:22.557199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5c409-6cf2-42af-bcb2-ff6149089985 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:22.557603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5c409-6cf2-42af-bcb2-ff6149089985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:22.566102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5c409-6cf2-42af-bcb2-ff6149089985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:22.567554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5c409-6cf2-42af-bcb2-ff6149089985 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:22.577304 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3814/7714] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:22.603657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970df82-02b2-4444-9d94-d33658d82f28 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:22.640806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:22.675025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970df82-02b2-4444-9d94-d33658d82f28 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:22.679963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:22.680601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:22.680940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:22.681224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] PATH_INFO: `/v3/users/872f91cddd294d64b92eca1c76ad9435` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:22.682365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=872f91cddd294d64b92eca1c76ad9435)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:22.684396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:22.744594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '872f91cddd294d64b92eca1c76ad9435'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:22.760922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '872f91cddd294d64b92eca1c76ad9435'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:22.769164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '872f91cddd294d64b92eca1c76ad9435'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:22.770185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '872f91cddd294d64b92eca1c76ad9435'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:22.770878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '872f91cddd294d64b92eca1c76ad9435'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:22.786844 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3901/7715] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:22 2022] DELETE /identity/v3/users/872f91cddd294d64b92eca1c76ad9435 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:22.791159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6053bc-b586-4e41-b932-af72873e4090 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:22.791516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6053bc-b586-4e41-b932-af72873e4090 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:22.791870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6053bc-b586-4e41-b932-af72873e4090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:22.814932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ff6053bc-b586-4e41-b932-af72873e4090 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:22.816303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ff6053bc-b586-4e41-b932-af72873e4090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:22.889854 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3815/7716] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:22.901944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:22.903549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:23.005409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73ff97-1cb9-41aa-b3ff-c0c9a12fdad6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:23.022264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3db8d1-bd13-4085-804a-d768be28736a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:23.022996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3db8d1-bd13-4085-804a-d768be28736a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:23.023366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3db8d1-bd13-4085-804a-d768be28736a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:23.023565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3db8d1-bd13-4085-804a-d768be28736a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:23.034874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3db8d1-bd13-4085-804a-d768be28736a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:23.036245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3db8d1-bd13-4085-804a-d768be28736a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:23.047017 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3902/7717] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:23.445056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6053bc-b586-4e41-b932-af72873e4090 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:23.446764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ff6053bc-b586-4e41-b932-af72873e4090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:23.464082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6053bc-b586-4e41-b932-af72873e4090 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:23.469771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907d9ef-aa2c-42aa-8924-70b4001c1943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:23.470575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907d9ef-aa2c-42aa-8924-70b4001c1943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:23.470957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907d9ef-aa2c-42aa-8924-70b4001c1943 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:23.471336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907d9ef-aa2c-42aa-8924-70b4001c1943 admin admin] PATH_INFO: `/v3/projects/08011156dc184f1cb1691565fbef9221` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:23.472591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907d9ef-aa2c-42aa-8924-70b4001c1943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08011156dc184f1cb1691565fbef9221)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:23.475462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907d9ef-aa2c-42aa-8924-70b4001c1943 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:23.556146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f907d9ef-aa2c-42aa-8924-70b4001c1943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08011156dc184f1cb1691565fbef9221', 'request_id': 'req-f907d9ef-aa2c-42aa-8924-70b4001c1943'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:23.567961 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f907d9ef-aa2c-42aa-8924-70b4001c1943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08011156dc184f1cb1691565fbef9221', 'request_id': 'req-f907d9ef-aa2c-42aa-8924-70b4001c1943'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:23.576243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f907d9ef-aa2c-42aa-8924-70b4001c1943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08011156dc184f1cb1691565fbef9221', 'request_id': 'req-f907d9ef-aa2c-42aa-8924-70b4001c1943'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:23.588685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f907d9ef-aa2c-42aa-8924-70b4001c1943 admin admin] The token cache is being invalidate because project 08011156dc184f1cb1691565fbef9221 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:23.589016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f907d9ef-aa2c-42aa-8924-70b4001c1943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:23.589774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f907d9ef-aa2c-42aa-8924-70b4001c1943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:23.591584 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3816/7718] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:23 2022] DELETE /identity/v3/projects/08011156dc184f1cb1691565fbef9221 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:23.596455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acbcf7-9da9-4f6f-8146-dce990fab550 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:23.596746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acbcf7-9da9-4f6f-8146-dce990fab550 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:23.597078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acbcf7-9da9-4f6f-8146-dce990fab550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:23.620322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-23acbcf7-9da9-4f6f-8146-dce990fab550 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:23.621697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-23acbcf7-9da9-4f6f-8146-dce990fab550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:23.684333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:23.726561 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3903/7719] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:23.729405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffddcd8-f4b2-4018-a727-8e2e3358861b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:23.730369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffddcd8-f4b2-4018-a727-8e2e3358861b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:23.730906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffddcd8-f4b2-4018-a727-8e2e3358861b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:23.754076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2ffddcd8-f4b2-4018-a727-8e2e3358861b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:23.755308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2ffddcd8-f4b2-4018-a727-8e2e3358861b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:23.764666 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3817/7720] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:23.768839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acbcf7-9da9-4f6f-8146-dce990fab550 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:23.779792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acbcf7-9da9-4f6f-8146-dce990fab550 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:23.785102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90dd59-8621-4858-bcef-58e82c4435bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:23.785736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90dd59-8621-4858-bcef-58e82c4435bf admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:23.786022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90dd59-8621-4858-bcef-58e82c4435bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:23.786301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90dd59-8621-4858-bcef-58e82c4435bf admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:23.786964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90dd59-8621-4858-bcef-58e82c4435bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:23.787313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90dd59-8621-4858-bcef-58e82c4435bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:23.789892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90dd59-8621-4858-bcef-58e82c4435bf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:23.808964 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3904/7721] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:23.810921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffddcd8-f4b2-4018-a727-8e2e3358861b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:23.822642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffddcd8-f4b2-4018-a727-8e2e3358861b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:23.830210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4df01-757b-42af-a16f-2716c58494a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:23.830830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4df01-757b-42af-a16f-2716c58494a5 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:23.831351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4df01-757b-42af-a16f-2716c58494a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:23.831674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4df01-757b-42af-a16f-2716c58494a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:23.832330 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4df01-757b-42af-a16f-2716c58494a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:23.834417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4df01-757b-42af-a16f-2716c58494a5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:23.867570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:23.881312 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3818/7722] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:23 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:23.882753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90dd59-8621-4858-bcef-58e82c4435bf admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:23.904830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90dd59-8621-4858-bcef-58e82c4435bf admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:23.978703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0ffc0-e208-4099-ab9a-a8ab25ae4062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:23.979595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0ffc0-e208-4099-ab9a-a8ab25ae4062 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:23.980719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0ffc0-e208-4099-ab9a-a8ab25ae4062 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:23.981317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0ffc0-e208-4099-ab9a-a8ab25ae4062 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:23.982182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0ffc0-e208-4099-ab9a-a8ab25ae4062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:23.984584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0ffc0-e208-4099-ab9a-a8ab25ae4062 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:24.016050 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3905/7723] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:23 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:24.020312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4df01-757b-42af-a16f-2716c58494a5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:24.032742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4df01-757b-42af-a16f-2716c58494a5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:24.038778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a61536-f53c-4a23-bebb-2f8ef3897b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:24.040080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a61536-f53c-4a23-bebb-2f8ef3897b7b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:24.040521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a61536-f53c-4a23-bebb-2f8ef3897b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:24.040839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a61536-f53c-4a23-bebb-2f8ef3897b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:24.041607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a61536-f53c-4a23-bebb-2f8ef3897b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:24.043933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a61536-f53c-4a23-bebb-2f8ef3897b7b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:24.055961 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3819/7724] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:24.059978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0ffc0-e208-4099-ab9a-a8ab25ae4062 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:24.070741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0ffc0-e208-4099-ab9a-a8ab25ae4062 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:24.090253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b198e31-c934-4af9-8a50-ec4be68641e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:24.091452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b198e31-c934-4af9-8a50-ec4be68641e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:24.091903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b198e31-c934-4af9-8a50-ec4be68641e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:24.093150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b198e31-c934-4af9-8a50-ec4be68641e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:24.094698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b198e31-c934-4af9-8a50-ec4be68641e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:24.096334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b198e31-c934-4af9-8a50-ec4be68641e1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:24.109675 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3906/7725] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:24.115791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a61536-f53c-4a23-bebb-2f8ef3897b7b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:24.132368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a61536-f53c-4a23-bebb-2f8ef3897b7b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:24.138649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bc898-14f3-435c-b2d6-c4e66296bb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:24.139508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bc898-14f3-435c-b2d6-c4e66296bb8e admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:24.140534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bc898-14f3-435c-b2d6-c4e66296bb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:24.140534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bc898-14f3-435c-b2d6-c4e66296bb8e admin admin] PATH_INFO: `/v3/projects/7f326412b264406080ff421d1397d1db/users/4e0be0d9e8484d22b661ab0064f5d182/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:24.158256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bc898-14f3-435c-b2d6-c4e66296bb8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f326412b264406080ff421d1397d1db, user_id=4e0be0d9e8484d22b661ab0064f5d182, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:24.161217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bc898-14f3-435c-b2d6-c4e66296bb8e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:24.175942 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3820/7726] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:24 2022] PUT /identity/v3/projects/7f326412b264406080ff421d1397d1db/users/4e0be0d9e8484d22b661ab0064f5d182/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:24.180986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d893a4-3a47-49e7-b435-92413444cdb7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:24.181305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d893a4-3a47-49e7-b435-92413444cdb7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:24.181592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d893a4-3a47-49e7-b435-92413444cdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:24.202448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-18d893a4-3a47-49e7-b435-92413444cdb7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:24.203692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-18d893a4-3a47-49e7-b435-92413444cdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:24.281365 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3907/7727] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:24.287536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bc898-14f3-435c-b2d6-c4e66296bb8e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:24.290140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a20bc898-14f3-435c-b2d6-c4e66296bb8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:24.382470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bc898-14f3-435c-b2d6-c4e66296bb8e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:24.388551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502f02a-cf37-4d83-858a-2ee61ec5c8cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:24.389463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502f02a-cf37-4d83-858a-2ee61ec5c8cb service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:24.390056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502f02a-cf37-4d83-858a-2ee61ec5c8cb service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:24.390510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502f02a-cf37-4d83-858a-2ee61ec5c8cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:24.413871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502f02a-cf37-4d83-858a-2ee61ec5c8cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:24.415414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502f02a-cf37-4d83-858a-2ee61ec5c8cb service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:24.428437 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3821/7728] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:17:26.170888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30dbe5-5a15-4598-84ad-000587c48536 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:26.174524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30dbe5-5a15-4598-84ad-000587c48536 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:26.175032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30dbe5-5a15-4598-84ad-000587c48536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:26.255963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2b30dbe5-5a15-4598-84ad-000587c48536 None None] MFA Rules not processed for user `4e0be0d9e8484d22b661ab0064f5d182`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:26.264725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b30dbe5-5a15-4598-84ad-000587c48536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:26.341275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:26.419003 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3908/7729] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:26.426346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502f02a-cf37-4d83-858a-2ee61ec5c8cb service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:26.429412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1502f02a-cf37-4d83-858a-2ee61ec5c8cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:26.592767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502f02a-cf37-4d83-858a-2ee61ec5c8cb service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:26.599561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f2945-2838-41ef-9b34-e06f1797736f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:26.600593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f2945-2838-41ef-9b34-e06f1797736f service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:26.601128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f2945-2838-41ef-9b34-e06f1797736f service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:26.602108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f2945-2838-41ef-9b34-e06f1797736f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:26.614703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f2945-2838-41ef-9b34-e06f1797736f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:26.616276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f2945-2838-41ef-9b34-e06f1797736f service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:26.628196 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3822/7730] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:27.167863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30dbe5-5a15-4598-84ad-000587c48536 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:27.169876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b30dbe5-5a15-4598-84ad-000587c48536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:27.192064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30dbe5-5a15-4598-84ad-000587c48536 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:27.200956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070baad6-bf3d-453f-8e8f-fa36911edc92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:27.201559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070baad6-bf3d-453f-8e8f-fa36911edc92 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:27.201854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070baad6-bf3d-453f-8e8f-fa36911edc92 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:27.202130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070baad6-bf3d-453f-8e8f-fa36911edc92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:27.204434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-070baad6-bf3d-453f-8e8f-fa36911edc92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:27.223731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070baad6-bf3d-453f-8e8f-fa36911edc92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:27.225238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070baad6-bf3d-453f-8e8f-fa36911edc92 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:27.246220 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3909/7731] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:29.174622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f2945-2838-41ef-9b34-e06f1797736f service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:29.194199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f2945-2838-41ef-9b34-e06f1797736f service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:29.199952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:29.200675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:29.201311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:29.201825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] PATH_INFO: `/v3/users/4e0be0d9e8484d22b661ab0064f5d182` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:29.203069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e0be0d9e8484d22b661ab0064f5d182)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:29.205496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:29.276590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e0be0d9e8484d22b661ab0064f5d182'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:29.300833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:29.302235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e0be0d9e8484d22b661ab0064f5d182'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:29.303061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e0be0d9e8484d22b661ab0064f5d182'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:29.314546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e0be0d9e8484d22b661ab0064f5d182'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:29.324079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e0be0d9e8484d22b661ab0064f5d182'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:29.332992 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3823/7732] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:29 2022] DELETE /identity/v3/users/4e0be0d9e8484d22b661ab0064f5d182 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:29.546416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070baad6-bf3d-453f-8e8f-fa36911edc92 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:29.560877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070baad6-bf3d-453f-8e8f-fa36911edc92 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:29.642430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:29.643571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:29.643929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:29.644339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] PATH_INFO: `/v3/projects/7f326412b264406080ff421d1397d1db` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:29.645990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f326412b264406080ff421d1397d1db)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:29.648361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:29.704842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f326412b264406080ff421d1397d1db', 'request_id': 'req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:29.715713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f326412b264406080ff421d1397d1db', 'request_id': 'req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:29.722974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f326412b264406080ff421d1397d1db', 'request_id': 'req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:29.735817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] The token cache is being invalidate because project 7f326412b264406080ff421d1397d1db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:29.736130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:29.738259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:29.741759 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3910/7733] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:29 2022] DELETE /identity/v3/projects/7f326412b264406080ff421d1397d1db => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:31.728242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:31.731786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:31.754977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ad0b8-98a8-4034-8295-9038933f3d5e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:31.822928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621d056-44e2-4917-8aa4-ed295fa2d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:31.823596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621d056-44e2-4917-8aa4-ed295fa2d6df admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:31.823883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621d056-44e2-4917-8aa4-ed295fa2d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:31.824166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621d056-44e2-4917-8aa4-ed295fa2d6df admin admin] PATH_INFO: `/v3/users/db307623f73f4532a96342f551ba011c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:31.825476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621d056-44e2-4917-8aa4-ed295fa2d6df admin admin] RBAC: Authorizing `identity:delete_user(user_id=db307623f73f4532a96342f551ba011c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:31.827470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621d056-44e2-4917-8aa4-ed295fa2d6df admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:31.884021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3621d056-44e2-4917-8aa4-ed295fa2d6df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db307623f73f4532a96342f551ba011c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:31.899857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3621d056-44e2-4917-8aa4-ed295fa2d6df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db307623f73f4532a96342f551ba011c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:31.900613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3621d056-44e2-4917-8aa4-ed295fa2d6df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db307623f73f4532a96342f551ba011c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:31.908360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3621d056-44e2-4917-8aa4-ed295fa2d6df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db307623f73f4532a96342f551ba011c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:31.920630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3621d056-44e2-4917-8aa4-ed295fa2d6df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db307623f73f4532a96342f551ba011c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:31.930263 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3824/7734] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:31 2022] DELETE /identity/v3/users/db307623f73f4532a96342f551ba011c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:32.233495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:32.235077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:32.253692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e342c5-26a0-4aeb-a3da-9301fa5250e7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:32.312281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:32.320615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3809c-d77c-491b-ac85-5c7b1075f875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:32.321414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3809c-d77c-491b-ac85-5c7b1075f875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:32.321711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3809c-d77c-491b-ac85-5c7b1075f875 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:32.322173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3809c-d77c-491b-ac85-5c7b1075f875 admin admin] PATH_INFO: `/v3/projects/a7715a4d463a44168bc78b9fa7a7718f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:32.323569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3809c-d77c-491b-ac85-5c7b1075f875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7715a4d463a44168bc78b9fa7a7718f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:32.325579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3809c-d77c-491b-ac85-5c7b1075f875 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:32.392132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d0d3809c-d77c-491b-ac85-5c7b1075f875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7715a4d463a44168bc78b9fa7a7718f', 'request_id': 'req-d0d3809c-d77c-491b-ac85-5c7b1075f875'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:32.402626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d0d3809c-d77c-491b-ac85-5c7b1075f875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7715a4d463a44168bc78b9fa7a7718f', 'request_id': 'req-d0d3809c-d77c-491b-ac85-5c7b1075f875'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:32.409373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d0d3809c-d77c-491b-ac85-5c7b1075f875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7715a4d463a44168bc78b9fa7a7718f', 'request_id': 'req-d0d3809c-d77c-491b-ac85-5c7b1075f875'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:32.420567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d0d3809c-d77c-491b-ac85-5c7b1075f875 admin admin] The token cache is being invalidate because project a7715a4d463a44168bc78b9fa7a7718f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:32.420863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d0d3809c-d77c-491b-ac85-5c7b1075f875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:32.421545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d0d3809c-d77c-491b-ac85-5c7b1075f875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:32.423172 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3911/7735] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:32 2022] DELETE /identity/v3/projects/a7715a4d463a44168bc78b9fa7a7718f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:32.429728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f35266-072b-4ceb-a83d-af8e8444f0a5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:32.430023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f35266-072b-4ceb-a83d-af8e8444f0a5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:32.430303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f35266-072b-4ceb-a83d-af8e8444f0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:32.452735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d5f35266-072b-4ceb-a83d-af8e8444f0a5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:32.454755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d5f35266-072b-4ceb-a83d-af8e8444f0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:32.509665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:32.521454 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3825/7736] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:32.531879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be8156-fbb6-4699-87fc-6f8ab2ba2241 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:32.532584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be8156-fbb6-4699-87fc-6f8ab2ba2241 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:32.532918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be8156-fbb6-4699-87fc-6f8ab2ba2241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:32.555953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e8be8156-fbb6-4699-87fc-6f8ab2ba2241 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:32.557215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e8be8156-fbb6-4699-87fc-6f8ab2ba2241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:32.571726 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3912/7737] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:32.572336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f35266-072b-4ceb-a83d-af8e8444f0a5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:32.583760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f35266-072b-4ceb-a83d-af8e8444f0a5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:32.589492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca033d21-4762-4c3f-9ee1-a2232962e6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:32.589774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca033d21-4762-4c3f-9ee1-a2232962e6bf admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:32.590064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca033d21-4762-4c3f-9ee1-a2232962e6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:32.590353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca033d21-4762-4c3f-9ee1-a2232962e6bf admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:32.591003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca033d21-4762-4c3f-9ee1-a2232962e6bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:32.591410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca033d21-4762-4c3f-9ee1-a2232962e6bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:32.593188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca033d21-4762-4c3f-9ee1-a2232962e6bf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:32.607004 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3826/7738] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:32.611069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be8156-fbb6-4699-87fc-6f8ab2ba2241 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:32.620814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be8156-fbb6-4699-87fc-6f8ab2ba2241 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:32.626286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcb4fc-06cb-478c-beda-c5cb10e0aef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:32.627284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcb4fc-06cb-478c-beda-c5cb10e0aef4 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:32.627495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcb4fc-06cb-478c-beda-c5cb10e0aef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:32.627819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcb4fc-06cb-478c-beda-c5cb10e0aef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:32.628407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dcb4fc-06cb-478c-beda-c5cb10e0aef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:32.630459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dcb4fc-06cb-478c-beda-c5cb10e0aef4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:32.676397 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3913/7739] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:32 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:32.680486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca033d21-4762-4c3f-9ee1-a2232962e6bf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:32.692277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca033d21-4762-4c3f-9ee1-a2232962e6bf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:32.753615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89864321-2fc1-4980-9804-bf2ea0b9efac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:32.754298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89864321-2fc1-4980-9804-bf2ea0b9efac admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:32.754406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89864321-2fc1-4980-9804-bf2ea0b9efac admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:32.754685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89864321-2fc1-4980-9804-bf2ea0b9efac admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:32.755316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89864321-2fc1-4980-9804-bf2ea0b9efac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:32.757264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89864321-2fc1-4980-9804-bf2ea0b9efac admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:32.778835 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3827/7740] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:32 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:32.783006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcb4fc-06cb-478c-beda-c5cb10e0aef4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:32.794332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcb4fc-06cb-478c-beda-c5cb10e0aef4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:32.799188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73d458-212a-4d30-b0d0-e6bd0d600ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:32.799853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73d458-212a-4d30-b0d0-e6bd0d600ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:32.800150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73d458-212a-4d30-b0d0-e6bd0d600ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:32.800453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73d458-212a-4d30-b0d0-e6bd0d600ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:32.801445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73d458-212a-4d30-b0d0-e6bd0d600ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:32.802927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73d458-212a-4d30-b0d0-e6bd0d600ca6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:32.814744 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3914/7741] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:32.818586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89864321-2fc1-4980-9804-bf2ea0b9efac admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:32.829794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89864321-2fc1-4980-9804-bf2ea0b9efac admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:32.834618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43807ce-8d63-4f1b-aa1c-4d9aaf56da25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:32.835179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43807ce-8d63-4f1b-aa1c-4d9aaf56da25 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:32.835463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43807ce-8d63-4f1b-aa1c-4d9aaf56da25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:32.835738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43807ce-8d63-4f1b-aa1c-4d9aaf56da25 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:32.836450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43807ce-8d63-4f1b-aa1c-4d9aaf56da25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:32.838093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43807ce-8d63-4f1b-aa1c-4d9aaf56da25 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:32.851144 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3828/7742] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:32.855652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73d458-212a-4d30-b0d0-e6bd0d600ca6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:32.865709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73d458-212a-4d30-b0d0-e6bd0d600ca6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:32.871004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc97e1-ba1b-473b-b2ab-fe6a35565b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:32.871662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc97e1-ba1b-473b-b2ab-fe6a35565b02 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:32.871948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc97e1-ba1b-473b-b2ab-fe6a35565b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:32.872254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc97e1-ba1b-473b-b2ab-fe6a35565b02 admin admin] PATH_INFO: `/v3/projects/84fe2fe110814e34b56ba13afa3a2920/users/4e25ee9b0f5e434f99152f93dd651496/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:32.889318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc97e1-ba1b-473b-b2ab-fe6a35565b02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84fe2fe110814e34b56ba13afa3a2920, user_id=4e25ee9b0f5e434f99152f93dd651496, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:32.892030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc97e1-ba1b-473b-b2ab-fe6a35565b02 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:32.908424 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3915/7743] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:32 2022] PUT /identity/v3/projects/84fe2fe110814e34b56ba13afa3a2920/users/4e25ee9b0f5e434f99152f93dd651496/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:32.913716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c81e9-ca88-4902-a4c0-3f682cb67e94 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:32.914463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c81e9-ca88-4902-a4c0-3f682cb67e94 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:32.914463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c81e9-ca88-4902-a4c0-3f682cb67e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:32.934713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-276c81e9-ca88-4902-a4c0-3f682cb67e94 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:32.935958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-276c81e9-ca88-4902-a4c0-3f682cb67e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:33.009431 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3829/7744] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:33.016046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc97e1-ba1b-473b-b2ab-fe6a35565b02 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:33.018663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-64bc97e1-ba1b-473b-b2ab-fe6a35565b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:33.113197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc97e1-ba1b-473b-b2ab-fe6a35565b02 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:33.119600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7cf36-ae53-40d6-815b-55ac5e2d00ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:33.120252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7cf36-ae53-40d6-815b-55ac5e2d00ba service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:33.120541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7cf36-ae53-40d6-815b-55ac5e2d00ba service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:33.120819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7cf36-ae53-40d6-815b-55ac5e2d00ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:33.138661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7cf36-ae53-40d6-815b-55ac5e2d00ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:33.140642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7cf36-ae53-40d6-815b-55ac5e2d00ba service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:33.154660 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3916/7745] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:38.287909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfc1ea-31cf-4a79-816a-826ae569c115 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:38.289960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfc1ea-31cf-4a79-816a-826ae569c115 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:38.290722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfc1ea-31cf-4a79-816a-826ae569c115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:38.365213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c1cfc1ea-31cf-4a79-816a-826ae569c115 None None] MFA Rules not processed for user `4e25ee9b0f5e434f99152f93dd651496`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:38.369340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c1cfc1ea-31cf-4a79-816a-826ae569c115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:38.479180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:38.493080 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3830/7746] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:40.501756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7cf36-ae53-40d6-815b-55ac5e2d00ba service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:40.503660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c7a7cf36-ae53-40d6-815b-55ac5e2d00ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:40.521708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7cf36-ae53-40d6-815b-55ac5e2d00ba service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:40.530331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:40.530733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:40.530917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:40.531200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] PATH_INFO: `/v3/users/c322885b6ead449a80498ba6b865566b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:40.533226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c322885b6ead449a80498ba6b865566b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:40.537113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:40.561782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:40.596810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c322885b6ead449a80498ba6b865566b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:40.612252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c322885b6ead449a80498ba6b865566b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:40.624317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c322885b6ead449a80498ba6b865566b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:40.625595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c322885b6ead449a80498ba6b865566b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:40.626467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c322885b6ead449a80498ba6b865566b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:40.642317 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3917/7747] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:40 2022] DELETE /identity/v3/users/c322885b6ead449a80498ba6b865566b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:40.646698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad2b27-4d81-442d-b2c7-e86407b56b0c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:40.647187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad2b27-4d81-442d-b2c7-e86407b56b0c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:40.647273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad2b27-4d81-442d-b2c7-e86407b56b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:40.668864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-76ad2b27-4d81-442d-b2c7-e86407b56b0c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:40.671060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-76ad2b27-4d81-442d-b2c7-e86407b56b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:40.745627 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3831/7748] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:40.755050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:40.756517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:40.825704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c1099-9f0d-4395-b691-ac4f400ae834 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:40.832721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67494bc-f2c5-480e-a41d-aabc0f34194b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:40.833454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67494bc-f2c5-480e-a41d-aabc0f34194b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:40.833818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67494bc-f2c5-480e-a41d-aabc0f34194b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:40.834477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67494bc-f2c5-480e-a41d-aabc0f34194b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:40.846787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67494bc-f2c5-480e-a41d-aabc0f34194b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:40.848309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67494bc-f2c5-480e-a41d-aabc0f34194b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:40.859912 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3918/7749] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:41.060093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad2b27-4d81-442d-b2c7-e86407b56b0c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:41.062043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-76ad2b27-4d81-442d-b2c7-e86407b56b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:41.078981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad2b27-4d81-442d-b2c7-e86407b56b0c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:41.084209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5366452-5854-4491-b097-46df66f2ddab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:41.085283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5366452-5854-4491-b097-46df66f2ddab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:41.085570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5366452-5854-4491-b097-46df66f2ddab admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:41.086025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5366452-5854-4491-b097-46df66f2ddab admin admin] PATH_INFO: `/v3/projects/b8ef8e80542e462b91cbda1654e024e4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:41.087681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5366452-5854-4491-b097-46df66f2ddab admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8ef8e80542e462b91cbda1654e024e4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:41.092346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5366452-5854-4491-b097-46df66f2ddab admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:41.156308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5366452-5854-4491-b097-46df66f2ddab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8ef8e80542e462b91cbda1654e024e4', 'request_id': 'req-d5366452-5854-4491-b097-46df66f2ddab'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:41.169088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5366452-5854-4491-b097-46df66f2ddab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8ef8e80542e462b91cbda1654e024e4', 'request_id': 'req-d5366452-5854-4491-b097-46df66f2ddab'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:41.179306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5366452-5854-4491-b097-46df66f2ddab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8ef8e80542e462b91cbda1654e024e4', 'request_id': 'req-d5366452-5854-4491-b097-46df66f2ddab'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:41.198435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5366452-5854-4491-b097-46df66f2ddab admin admin] The token cache is being invalidate because project b8ef8e80542e462b91cbda1654e024e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:41.198977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5366452-5854-4491-b097-46df66f2ddab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:41.199769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d5366452-5854-4491-b097-46df66f2ddab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:41.201414 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3832/7750] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:41 2022] DELETE /identity/v3/projects/b8ef8e80542e462b91cbda1654e024e4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:41.207163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd8db7-d3d5-48ed-b72c-f39aa688af66 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:41.207457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd8db7-d3d5-48ed-b72c-f39aa688af66 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:41.207740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd8db7-d3d5-48ed-b72c-f39aa688af66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:41.228257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ddcd8db7-d3d5-48ed-b72c-f39aa688af66 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:41.229540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ddcd8db7-d3d5-48ed-b72c-f39aa688af66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:41.300825 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3919/7751] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:41.309049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdde89-2b10-40a7-9f7a-f582ba6370b8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:41.309383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdde89-2b10-40a7-9f7a-f582ba6370b8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:41.309643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdde89-2b10-40a7-9f7a-f582ba6370b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:41.335442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c4cdde89-2b10-40a7-9f7a-f582ba6370b8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:41.336885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c4cdde89-2b10-40a7-9f7a-f582ba6370b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:41.348770 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3833/7752] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:41.349705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd8db7-d3d5-48ed-b72c-f39aa688af66 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:41.360528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd8db7-d3d5-48ed-b72c-f39aa688af66 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:41.367627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e1d51-399d-4031-bf58-3e418ddad6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:41.368517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e1d51-399d-4031-bf58-3e418ddad6be admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:41.368881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e1d51-399d-4031-bf58-3e418ddad6be admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:41.369267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e1d51-399d-4031-bf58-3e418ddad6be admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:41.370008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e1d51-399d-4031-bf58-3e418ddad6be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:41.370619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e1d51-399d-4031-bf58-3e418ddad6be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:41.372115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e1d51-399d-4031-bf58-3e418ddad6be admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:41.389096 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3920/7753] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:41.390657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdde89-2b10-40a7-9f7a-f582ba6370b8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:41.400727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdde89-2b10-40a7-9f7a-f582ba6370b8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:41.405954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3acbc-c2ea-408b-aca1-a0884fc25aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:41.406586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3acbc-c2ea-408b-aca1-a0884fc25aab admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:41.407323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3acbc-c2ea-408b-aca1-a0884fc25aab admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:41.407323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3acbc-c2ea-408b-aca1-a0884fc25aab admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:41.407829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3acbc-c2ea-408b-aca1-a0884fc25aab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:41.409891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3acbc-c2ea-408b-aca1-a0884fc25aab admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:41.448567 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3834/7754] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:41 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:41.449825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e1d51-399d-4031-bf58-3e418ddad6be admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:41.461790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e1d51-399d-4031-bf58-3e418ddad6be admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:41.501834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:41.547849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca956e0-dc1f-4d33-b340-f1df13eda239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:41.548889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca956e0-dc1f-4d33-b340-f1df13eda239 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:41.549482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca956e0-dc1f-4d33-b340-f1df13eda239 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:41.549995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca956e0-dc1f-4d33-b340-f1df13eda239 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:41.550856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca956e0-dc1f-4d33-b340-f1df13eda239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:41.553467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca956e0-dc1f-4d33-b340-f1df13eda239 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:41.593741 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3921/7755] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:41 2022] POST /identity/v3/users => generated 282 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:41.594972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3acbc-c2ea-408b-aca1-a0884fc25aab admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:41.617044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3acbc-c2ea-408b-aca1-a0884fc25aab admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:41.623753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57b1b8-c7f8-4653-9bc2-38ca754d4555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:41.624452 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57b1b8-c7f8-4653-9bc2-38ca754d4555 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:41.624733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57b1b8-c7f8-4653-9bc2-38ca754d4555 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:41.625043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57b1b8-c7f8-4653-9bc2-38ca754d4555 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:41.625890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57b1b8-c7f8-4653-9bc2-38ca754d4555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:41.627556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57b1b8-c7f8-4653-9bc2-38ca754d4555 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:41.643888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca956e0-dc1f-4d33-b340-f1df13eda239 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:41.644184 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3835/7756] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:41.655760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca956e0-dc1f-4d33-b340-f1df13eda239 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:41.661493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453de42-bb66-4783-88fd-8a12206be11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:41.662142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453de42-bb66-4783-88fd-8a12206be11e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:41.662681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453de42-bb66-4783-88fd-8a12206be11e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:41.662874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453de42-bb66-4783-88fd-8a12206be11e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:41.663529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453de42-bb66-4783-88fd-8a12206be11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:41.665248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453de42-bb66-4783-88fd-8a12206be11e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:41.676838 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3922/7757] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:17:41.681539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57b1b8-c7f8-4653-9bc2-38ca754d4555 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:41.694870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57b1b8-c7f8-4653-9bc2-38ca754d4555 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:41.701208 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c2d99-1b11-40bf-896c-bbce5aa816e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:41.702575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c2d99-1b11-40bf-896c-bbce5aa816e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:41.702750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c2d99-1b11-40bf-896c-bbce5aa816e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:41.704015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c2d99-1b11-40bf-896c-bbce5aa816e0 admin admin] PATH_INFO: `/v3/projects/d39308a9753f4ee297ea40e4faf33923/users/41d806d2ab5b469faa8f5c619b7b33a3/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:41.729196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c2d99-1b11-40bf-896c-bbce5aa816e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39308a9753f4ee297ea40e4faf33923, user_id=41d806d2ab5b469faa8f5c619b7b33a3, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:41.732453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c2d99-1b11-40bf-896c-bbce5aa816e0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:41.758568 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3836/7758] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:41 2022] PUT /identity/v3/projects/d39308a9753f4ee297ea40e4faf33923/users/41d806d2ab5b469faa8f5c619b7b33a3/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:41.759142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777db50-3593-47e8-87f4-41430b7aad8b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:41.759442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777db50-3593-47e8-87f4-41430b7aad8b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:41.759740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777db50-3593-47e8-87f4-41430b7aad8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:41.797576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e777db50-3593-47e8-87f4-41430b7aad8b None None] MFA Rules not processed for user `41d806d2ab5b469faa8f5c619b7b33a3`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:41.799038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e777db50-3593-47e8-87f4-41430b7aad8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:41.892639 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3923/7759] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:41.903093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c2d99-1b11-40bf-896c-bbce5aa816e0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:41.907672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1c9c2d99-1b11-40bf-896c-bbce5aa816e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:41.930950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:41.989693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c2d99-1b11-40bf-896c-bbce5aa816e0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:41.994459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f16682-0964-4c57-919d-dfaa5b19dbc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:41.995086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f16682-0964-4c57-919d-dfaa5b19dbc7 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:41.995371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f16682-0964-4c57-919d-dfaa5b19dbc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:41.995647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f16682-0964-4c57-919d-dfaa5b19dbc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:42.004957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f16682-0964-4c57-919d-dfaa5b19dbc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:42.006397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f16682-0964-4c57-919d-dfaa5b19dbc7 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:42.019737 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3837/7760] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:42.097177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777db50-3593-47e8-87f4-41430b7aad8b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:42.098820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e777db50-3593-47e8-87f4-41430b7aad8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:42.187655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777db50-3593-47e8-87f4-41430b7aad8b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:42.196340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a10ef-b906-4863-b55e-cdafa0979823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:42.196340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a10ef-b906-4863-b55e-cdafa0979823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:42.196771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a10ef-b906-4863-b55e-cdafa0979823 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:42.197334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a10ef-b906-4863-b55e-cdafa0979823 admin admin] PATH_INFO: `/v3/users/4e25ee9b0f5e434f99152f93dd651496` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:42.198683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a10ef-b906-4863-b55e-cdafa0979823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e25ee9b0f5e434f99152f93dd651496)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:42.200629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a10ef-b906-4863-b55e-cdafa0979823 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:42.288371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:42.292857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-000a10ef-b906-4863-b55e-cdafa0979823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e25ee9b0f5e434f99152f93dd651496'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:42.321461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-000a10ef-b906-4863-b55e-cdafa0979823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e25ee9b0f5e434f99152f93dd651496'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:42.329673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-000a10ef-b906-4863-b55e-cdafa0979823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e25ee9b0f5e434f99152f93dd651496'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:42.330445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-000a10ef-b906-4863-b55e-cdafa0979823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e25ee9b0f5e434f99152f93dd651496'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:42.331045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-000a10ef-b906-4863-b55e-cdafa0979823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e25ee9b0f5e434f99152f93dd651496'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:42.349480 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3924/7761] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:42 2022] DELETE /identity/v3/users/4e25ee9b0f5e434f99152f93dd651496 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:42.797744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f16682-0964-4c57-919d-dfaa5b19dbc7 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:42.809403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f16682-0964-4c57-919d-dfaa5b19dbc7 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:42.889641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbaf7a-3dca-44dc-ae1a-0203fe678839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:42.890430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbaf7a-3dca-44dc-ae1a-0203fe678839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:42.890731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbaf7a-3dca-44dc-ae1a-0203fe678839 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:42.891010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbaf7a-3dca-44dc-ae1a-0203fe678839 admin admin] PATH_INFO: `/v3/projects/84fe2fe110814e34b56ba13afa3a2920` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:42.892170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbaf7a-3dca-44dc-ae1a-0203fe678839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84fe2fe110814e34b56ba13afa3a2920)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:42.894132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbaf7a-3dca-44dc-ae1a-0203fe678839 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:42.953083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fabbaf7a-3dca-44dc-ae1a-0203fe678839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84fe2fe110814e34b56ba13afa3a2920', 'request_id': 'req-fabbaf7a-3dca-44dc-ae1a-0203fe678839'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:42.963532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fabbaf7a-3dca-44dc-ae1a-0203fe678839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84fe2fe110814e34b56ba13afa3a2920', 'request_id': 'req-fabbaf7a-3dca-44dc-ae1a-0203fe678839'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:42.971510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fabbaf7a-3dca-44dc-ae1a-0203fe678839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84fe2fe110814e34b56ba13afa3a2920', 'request_id': 'req-fabbaf7a-3dca-44dc-ae1a-0203fe678839'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:42.993255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fabbaf7a-3dca-44dc-ae1a-0203fe678839 admin admin] The token cache is being invalidate because project 84fe2fe110814e34b56ba13afa3a2920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:42.993609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fabbaf7a-3dca-44dc-ae1a-0203fe678839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:42.994381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fabbaf7a-3dca-44dc-ae1a-0203fe678839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:42.996222 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3838/7762] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:42 2022] DELETE /identity/v3/projects/84fe2fe110814e34b56ba13afa3a2920 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:43.005470 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165afd1a-bc37-43ce-bd27-a712ffebdb8e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:43.005989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165afd1a-bc37-43ce-bd27-a712ffebdb8e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:43.006076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165afd1a-bc37-43ce-bd27-a712ffebdb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:43.034524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-165afd1a-bc37-43ce-bd27-a712ffebdb8e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:43.035929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-165afd1a-bc37-43ce-bd27-a712ffebdb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:43.129058 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3925/7763] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:43.136301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567ac78-81bd-4f7d-8d50-5063366eacc6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:43.136595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567ac78-81bd-4f7d-8d50-5063366eacc6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:43.136888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567ac78-81bd-4f7d-8d50-5063366eacc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:43.161341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6567ac78-81bd-4f7d-8d50-5063366eacc6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:43.162980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6567ac78-81bd-4f7d-8d50-5063366eacc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:43.178728 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3839/7764] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:43.183798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165afd1a-bc37-43ce-bd27-a712ffebdb8e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:43.195310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165afd1a-bc37-43ce-bd27-a712ffebdb8e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:43.199961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0a79a-2218-49e0-9325-3ff7447a9452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:43.200605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0a79a-2218-49e0-9325-3ff7447a9452 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:43.200887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0a79a-2218-49e0-9325-3ff7447a9452 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:43.201189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0a79a-2218-49e0-9325-3ff7447a9452 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:43.201833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0a79a-2218-49e0-9325-3ff7447a9452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:43.202149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0a79a-2218-49e0-9325-3ff7447a9452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:43.203733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0a79a-2218-49e0-9325-3ff7447a9452 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:43.217013 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3926/7765] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:43.221816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567ac78-81bd-4f7d-8d50-5063366eacc6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:43.232869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567ac78-81bd-4f7d-8d50-5063366eacc6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:43.237755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e50404-234a-4717-8fba-017639328ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:43.238459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e50404-234a-4717-8fba-017639328ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:43.238757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e50404-234a-4717-8fba-017639328ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:43.239039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e50404-234a-4717-8fba-017639328ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:43.239678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e50404-234a-4717-8fba-017639328ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:43.241653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e50404-234a-4717-8fba-017639328ef7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:43.271052 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3840/7766] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:43 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:43.276262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0a79a-2218-49e0-9325-3ff7447a9452 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:43.288356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0a79a-2218-49e0-9325-3ff7447a9452 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:43.343428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e50404-234a-4717-8fba-017639328ef7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:43.345912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a8e50404-234a-4717-8fba-017639328ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:43.358792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3ad1c-0201-4006-b1e0-6a5ead15d932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:43.370260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3ad1c-0201-4006-b1e0-6a5ead15d932 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:43.374461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3ad1c-0201-4006-b1e0-6a5ead15d932 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:43.374751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3ad1c-0201-4006-b1e0-6a5ead15d932 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:43.375589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3ad1c-0201-4006-b1e0-6a5ead15d932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:43.377861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3ad1c-0201-4006-b1e0-6a5ead15d932 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:43.394242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:43.407410 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3927/7767] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:43 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:43.412224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3ad1c-0201-4006-b1e0-6a5ead15d932 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:43.426803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3ad1c-0201-4006-b1e0-6a5ead15d932 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:43.432297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51c550-8cbc-4d73-80ca-da8644294535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:43.433221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51c550-8cbc-4d73-80ca-da8644294535 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:43.433751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51c550-8cbc-4d73-80ca-da8644294535 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:43.434246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51c550-8cbc-4d73-80ca-da8644294535 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:43.435194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51c550-8cbc-4d73-80ca-da8644294535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:43.437105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51c550-8cbc-4d73-80ca-da8644294535 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:43.450813 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3928/7768] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:43.454727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51c550-8cbc-4d73-80ca-da8644294535 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:43.462385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e50404-234a-4717-8fba-017639328ef7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:43.470542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ac2c0-0fd4-45eb-a8be-aea467e1ecbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:43.470951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51c550-8cbc-4d73-80ca-da8644294535 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:43.471175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ac2c0-0fd4-45eb-a8be-aea467e1ecbd service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:43.471464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ac2c0-0fd4-45eb-a8be-aea467e1ecbd service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:43.471774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ac2c0-0fd4-45eb-a8be-aea467e1ecbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:43.473596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dc6ac2c0-0fd4-45eb-a8be-aea467e1ecbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:43.477394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76655d8-74b3-4b32-b99e-15794a6a1ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:43.478084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76655d8-74b3-4b32-b99e-15794a6a1ffe admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:43.478443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76655d8-74b3-4b32-b99e-15794a6a1ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:43.478833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76655d8-74b3-4b32-b99e-15794a6a1ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:43.479646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76655d8-74b3-4b32-b99e-15794a6a1ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:43.481802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76655d8-74b3-4b32-b99e-15794a6a1ffe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:43.487725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ac2c0-0fd4-45eb-a8be-aea467e1ecbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:43.489193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ac2c0-0fd4-45eb-a8be-aea467e1ecbd service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:43.498638 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3929/7769] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:17:43.503535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76655d8-74b3-4b32-b99e-15794a6a1ffe admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:43.505168 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3841/7770] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:17:43.513803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76655d8-74b3-4b32-b99e-15794a6a1ffe admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:43.518571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fcaa1-3558-4c47-9da7-987661b943ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:43.519235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fcaa1-3558-4c47-9da7-987661b943ea admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:43.519522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fcaa1-3558-4c47-9da7-987661b943ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:43.519804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fcaa1-3558-4c47-9da7-987661b943ea admin admin] PATH_INFO: `/v3/projects/db906426fb8a437cb03072319b2d92db/users/3aa7ddc203f848faaf8746e5b09a3c76/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:43.525334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:43.545544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fcaa1-3558-4c47-9da7-987661b943ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=db906426fb8a437cb03072319b2d92db, user_id=3aa7ddc203f848faaf8746e5b09a3c76, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:43.548716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fcaa1-3558-4c47-9da7-987661b943ea admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:43.569056 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3930/7771] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:43 2022] PUT /identity/v3/projects/db906426fb8a437cb03072319b2d92db/users/3aa7ddc203f848faaf8746e5b09a3c76/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:43.579779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c5007-bbe7-4a39-9d11-8dd98f256cd0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:43.580074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c5007-bbe7-4a39-9d11-8dd98f256cd0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:43.580359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c5007-bbe7-4a39-9d11-8dd98f256cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:43.613257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-253c5007-bbe7-4a39-9d11-8dd98f256cd0 None None] MFA Rules not processed for user `3aa7ddc203f848faaf8746e5b09a3c76`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:43.614827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-253c5007-bbe7-4a39-9d11-8dd98f256cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:43.692410 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3842/7772] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:43.704639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fcaa1-3558-4c47-9da7-987661b943ea admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:43.706569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e95fcaa1-3558-4c47-9da7-987661b943ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:43.820905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fcaa1-3558-4c47-9da7-987661b943ea admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:43.826832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a2b3e-731b-47ae-97c5-851776fe6981 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:43.827443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a2b3e-731b-47ae-97c5-851776fe6981 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:43.827737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a2b3e-731b-47ae-97c5-851776fe6981 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:43.828012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a2b3e-731b-47ae-97c5-851776fe6981 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:43.836659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a2b3e-731b-47ae-97c5-851776fe6981 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:43.838298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a2b3e-731b-47ae-97c5-851776fe6981 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:43.847727 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3931/7773] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:17:43.915580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c5007-bbe7-4a39-9d11-8dd98f256cd0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:43.917670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-253c5007-bbe7-4a39-9d11-8dd98f256cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:44.034736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c5007-bbe7-4a39-9d11-8dd98f256cd0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:44.040223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c9283-7648-4d9c-a3b3-fbf536555a7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:44.041021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c9283-7648-4d9c-a3b3-fbf536555a7c service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:44.041340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c9283-7648-4d9c-a3b3-fbf536555a7c service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:44.041667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c9283-7648-4d9c-a3b3-fbf536555a7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:44.043622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-410c9283-7648-4d9c-a3b3-fbf536555a7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:44.062927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c9283-7648-4d9c-a3b3-fbf536555a7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:44.064632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c9283-7648-4d9c-a3b3-fbf536555a7c service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:44.084185 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3843/7774] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:45.616475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a2b3e-731b-47ae-97c5-851776fe6981 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:45.693095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a2b3e-731b-47ae-97c5-851776fe6981 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:45.699031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:45.700113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:45.700113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:45.700320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] PATH_INFO: `/v3/users/3aa7ddc203f848faaf8746e5b09a3c76` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:45.701575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa7ddc203f848faaf8746e5b09a3c76)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:45.703756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:45.765455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa7ddc203f848faaf8746e5b09a3c76'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:45.773676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:45.780412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa7ddc203f848faaf8746e5b09a3c76'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:45.790959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa7ddc203f848faaf8746e5b09a3c76'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:45.791610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa7ddc203f848faaf8746e5b09a3c76'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:45.792372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa7ddc203f848faaf8746e5b09a3c76'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:45.807605 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3932/7775] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:45 2022] DELETE /identity/v3/users/3aa7ddc203f848faaf8746e5b09a3c76 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:45.812285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65e966-2a50-4e8d-a404-b2d45e8d9fcc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:45.812738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65e966-2a50-4e8d-a404-b2d45e8d9fcc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:45.812842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65e966-2a50-4e8d-a404-b2d45e8d9fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:45.863076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1f65e966-2a50-4e8d-a404-b2d45e8d9fcc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:45.864471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1f65e966-2a50-4e8d-a404-b2d45e8d9fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:45.921444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:45.955448 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3844/7776] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:45.971919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:45.973739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:46.057450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7dbbe-4423-4476-b45f-435da1731b7e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:46.061325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6941b-c0ec-46ac-976f-042c624dc51f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:46.062000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6941b-c0ec-46ac-976f-042c624dc51f service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:46.062255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6941b-c0ec-46ac-976f-042c624dc51f service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:46.062532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6941b-c0ec-46ac-976f-042c624dc51f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:46.073330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6941b-c0ec-46ac-976f-042c624dc51f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:46.075491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6941b-c0ec-46ac-976f-042c624dc51f service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:46.086735 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3933/7777] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:46.582724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65e966-2a50-4e8d-a404-b2d45e8d9fcc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:46.584301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1f65e966-2a50-4e8d-a404-b2d45e8d9fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:46.598986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65e966-2a50-4e8d-a404-b2d45e8d9fcc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:46.604297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02184e19-689b-401a-a13f-3aaf8ff90a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:46.604977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02184e19-689b-401a-a13f-3aaf8ff90a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:46.605531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02184e19-689b-401a-a13f-3aaf8ff90a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:46.605812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02184e19-689b-401a-a13f-3aaf8ff90a13 admin admin] PATH_INFO: `/v3/projects/db906426fb8a437cb03072319b2d92db` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:46.607153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02184e19-689b-401a-a13f-3aaf8ff90a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db906426fb8a437cb03072319b2d92db)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:46.609274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02184e19-689b-401a-a13f-3aaf8ff90a13 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:46.665635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-02184e19-689b-401a-a13f-3aaf8ff90a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db906426fb8a437cb03072319b2d92db', 'request_id': 'req-02184e19-689b-401a-a13f-3aaf8ff90a13'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:46.673340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-02184e19-689b-401a-a13f-3aaf8ff90a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db906426fb8a437cb03072319b2d92db', 'request_id': 'req-02184e19-689b-401a-a13f-3aaf8ff90a13'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:46.681043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-02184e19-689b-401a-a13f-3aaf8ff90a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db906426fb8a437cb03072319b2d92db', 'request_id': 'req-02184e19-689b-401a-a13f-3aaf8ff90a13'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:46.698931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-02184e19-689b-401a-a13f-3aaf8ff90a13 admin admin] The token cache is being invalidate because project db906426fb8a437cb03072319b2d92db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:46.699588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-02184e19-689b-401a-a13f-3aaf8ff90a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:46.700525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-02184e19-689b-401a-a13f-3aaf8ff90a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:46.702856 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3845/7778] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:46 2022] DELETE /identity/v3/projects/db906426fb8a437cb03072319b2d92db => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:46.708468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a52ce-5ca8-43c6-a7b0-a5365c509f84 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:46.708995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a52ce-5ca8-43c6-a7b0-a5365c509f84 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:46.709094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a52ce-5ca8-43c6-a7b0-a5365c509f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:46.729833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-184a52ce-5ca8-43c6-a7b0-a5365c509f84 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:46.731333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-184a52ce-5ca8-43c6-a7b0-a5365c509f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:46.793861 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3934/7779] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:46.802703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a295a1-9ba3-4f30-a98a-0f834406147b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:46.803077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a295a1-9ba3-4f30-a98a-0f834406147b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:46.803432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a295a1-9ba3-4f30-a98a-0f834406147b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:46.826094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-47a295a1-9ba3-4f30-a98a-0f834406147b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:46.827699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-47a295a1-9ba3-4f30-a98a-0f834406147b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:46.837063 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3846/7780] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:46.841069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a52ce-5ca8-43c6-a7b0-a5365c509f84 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:46.851446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a52ce-5ca8-43c6-a7b0-a5365c509f84 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:46.857696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9df904-d474-4596-9959-ab8b683579a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:46.858475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9df904-d474-4596-9959-ab8b683579a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:46.858691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9df904-d474-4596-9959-ab8b683579a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:46.859243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9df904-d474-4596-9959-ab8b683579a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:46.859942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9df904-d474-4596-9959-ab8b683579a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:46.860373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9df904-d474-4596-9959-ab8b683579a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:46.862252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9df904-d474-4596-9959-ab8b683579a9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:46.879524 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3935/7781] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:46.884598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a295a1-9ba3-4f30-a98a-0f834406147b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:46.894574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a295a1-9ba3-4f30-a98a-0f834406147b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:46.900342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1e335-8903-4ea6-8c9b-8fe259bcf71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:46.901002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1e335-8903-4ea6-8c9b-8fe259bcf71d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:46.901290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1e335-8903-4ea6-8c9b-8fe259bcf71d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:46.901571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1e335-8903-4ea6-8c9b-8fe259bcf71d admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:46.902407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1e335-8903-4ea6-8c9b-8fe259bcf71d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:46.904343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1e335-8903-4ea6-8c9b-8fe259bcf71d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:46.932855 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3847/7782] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:46 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:46.936785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9df904-d474-4596-9959-ab8b683579a9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:46.947888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9df904-d474-4596-9959-ab8b683579a9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:47.009271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4333f2-d9f2-4565-89ab-0d08badf3793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:47.009892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4333f2-d9f2-4565-89ab-0d08badf3793 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.010187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4333f2-d9f2-4565-89ab-0d08badf3793 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.010474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4333f2-d9f2-4565-89ab-0d08badf3793 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.011118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4333f2-d9f2-4565-89ab-0d08badf3793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:47.013108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4333f2-d9f2-4565-89ab-0d08badf3793 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:47.023309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:47.043484 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3936/7783] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:46 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:47.045567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1e335-8903-4ea6-8c9b-8fe259bcf71d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:47.057943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1e335-8903-4ea6-8c9b-8fe259bcf71d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:47.062932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa52d62-92dd-4c27-afa0-71fb1d7ea42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:47.063560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa52d62-92dd-4c27-afa0-71fb1d7ea42a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.063842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa52d62-92dd-4c27-afa0-71fb1d7ea42a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.064140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa52d62-92dd-4c27-afa0-71fb1d7ea42a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.064770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa52d62-92dd-4c27-afa0-71fb1d7ea42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:47.066557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa52d62-92dd-4c27-afa0-71fb1d7ea42a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:47.078594 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3848/7784] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:47.082371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4333f2-d9f2-4565-89ab-0d08badf3793 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:47.096350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4333f2-d9f2-4565-89ab-0d08badf3793 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:47.102515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49006f5c-868a-4384-954a-4c83accb4202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:47.103136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49006f5c-868a-4384-954a-4c83accb4202 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.103420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49006f5c-868a-4384-954a-4c83accb4202 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.103696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49006f5c-868a-4384-954a-4c83accb4202 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.104342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49006f5c-868a-4384-954a-4c83accb4202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:47.106251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49006f5c-868a-4384-954a-4c83accb4202 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:47.121570 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3937/7785] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:17:47.122795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa52d62-92dd-4c27-afa0-71fb1d7ea42a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:47.134129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa52d62-92dd-4c27-afa0-71fb1d7ea42a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:47.139988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab36af-b4dd-463d-96ca-5089024f208f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:47.140671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab36af-b4dd-463d-96ca-5089024f208f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.140991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab36af-b4dd-463d-96ca-5089024f208f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.141274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab36af-b4dd-463d-96ca-5089024f208f admin admin] PATH_INFO: `/v3/projects/71494b232cff4b769d2f28f3188f6afc/users/d64b029d8f6341d68a6f397a05b92af3/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.156742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab36af-b4dd-463d-96ca-5089024f208f admin admin] RBAC: Authorizing `identity:create_grant(project_id=71494b232cff4b769d2f28f3188f6afc, user_id=d64b029d8f6341d68a6f397a05b92af3, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:47.159480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab36af-b4dd-463d-96ca-5089024f208f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:47.174401 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3849/7786] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:47 2022] PUT /identity/v3/projects/71494b232cff4b769d2f28f3188f6afc/users/d64b029d8f6341d68a6f397a05b92af3/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:47.185873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce75fd9-36ec-4a84-82e1-9e0e38de9b25 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.186305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce75fd9-36ec-4a84-82e1-9e0e38de9b25 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.186717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce75fd9-36ec-4a84-82e1-9e0e38de9b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.221888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9ce75fd9-36ec-4a84-82e1-9e0e38de9b25 None None] MFA Rules not processed for user `d64b029d8f6341d68a6f397a05b92af3`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:47.223309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9ce75fd9-36ec-4a84-82e1-9e0e38de9b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:47.305561 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3938/7787] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:47.310373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab36af-b4dd-463d-96ca-5089024f208f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:47.376800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab36af-b4dd-463d-96ca-5089024f208f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:47.381598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7c5f2-7a2e-413e-bf21-331baf07b008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:47.382210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7c5f2-7a2e-413e-bf21-331baf07b008 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.383054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7c5f2-7a2e-413e-bf21-331baf07b008 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.383345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7c5f2-7a2e-413e-bf21-331baf07b008 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.383981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7c5f2-7a2e-413e-bf21-331baf07b008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:47.385940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7c5f2-7a2e-413e-bf21-331baf07b008 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:47.391325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:47.418961 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3850/7788] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:47 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:47.420213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce75fd9-36ec-4a84-82e1-9e0e38de9b25 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:47.431883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce75fd9-36ec-4a84-82e1-9e0e38de9b25 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:47.491605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483a787-d0b7-4cf3-b855-a94084cd5d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:47.492877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483a787-d0b7-4cf3-b855-a94084cd5d30 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.493197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483a787-d0b7-4cf3-b855-a94084cd5d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.493497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483a787-d0b7-4cf3-b855-a94084cd5d30 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.495735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483a787-d0b7-4cf3-b855-a94084cd5d30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:47.498000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483a787-d0b7-4cf3-b855-a94084cd5d30 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:47.524942 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3939/7789] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:47 2022] POST /identity/v3/users => generated 301 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:47.527829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7c5f2-7a2e-413e-bf21-331baf07b008 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:47.538446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7c5f2-7a2e-413e-bf21-331baf07b008 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:47.544150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5336aa9-42ee-4fd0-9dbd-b623fd32485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:47.544871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5336aa9-42ee-4fd0-9dbd-b623fd32485f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.545262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5336aa9-42ee-4fd0-9dbd-b623fd32485f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.545615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5336aa9-42ee-4fd0-9dbd-b623fd32485f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.546397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5336aa9-42ee-4fd0-9dbd-b623fd32485f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:47.548280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5336aa9-42ee-4fd0-9dbd-b623fd32485f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:47.562499 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3851/7790] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:47.568462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483a787-d0b7-4cf3-b855-a94084cd5d30 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:47.579670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483a787-d0b7-4cf3-b855-a94084cd5d30 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:47.584564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a426445-9b00-4cfa-9a1a-8d4aa6f6625b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:47.585333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a426445-9b00-4cfa-9a1a-8d4aa6f6625b admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.585808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a426445-9b00-4cfa-9a1a-8d4aa6f6625b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.586247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a426445-9b00-4cfa-9a1a-8d4aa6f6625b admin admin] PATH_INFO: `/v3/projects/240e5ce0a8ef40518b2c2ff59b64ca41/users/3a6fa32c5ae14add86e30fbf370e5291/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.606048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a426445-9b00-4cfa-9a1a-8d4aa6f6625b admin admin] RBAC: Authorizing `identity:create_grant(project_id=240e5ce0a8ef40518b2c2ff59b64ca41, user_id=3a6fa32c5ae14add86e30fbf370e5291, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:47.609613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a426445-9b00-4cfa-9a1a-8d4aa6f6625b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:47.626270 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3940/7791] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:47 2022] PUT /identity/v3/projects/240e5ce0a8ef40518b2c2ff59b64ca41/users/3a6fa32c5ae14add86e30fbf370e5291/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:47.631332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5336aa9-42ee-4fd0-9dbd-b623fd32485f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:47.642883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5336aa9-42ee-4fd0-9dbd-b623fd32485f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:47.730163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cda12-93f0-43df-a169-2f54356245e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:47.731893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cda12-93f0-43df-a169-2f54356245e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.732251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cda12-93f0-43df-a169-2f54356245e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.732713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cda12-93f0-43df-a169-2f54356245e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.733587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cda12-93f0-43df-a169-2f54356245e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:47.735621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cda12-93f0-43df-a169-2f54356245e1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:47.753103 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3852/7792] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:47.757324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a426445-9b00-4cfa-9a1a-8d4aa6f6625b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:47.768551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a426445-9b00-4cfa-9a1a-8d4aa6f6625b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:47.775326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa6453-a516-4b45-a0e8-579af46e0940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:47.775934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa6453-a516-4b45-a0e8-579af46e0940 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.776219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa6453-a516-4b45-a0e8-579af46e0940 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.776558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa6453-a516-4b45-a0e8-579af46e0940 admin admin] PATH_INFO: `/v3/projects/240e5ce0a8ef40518b2c2ff59b64ca41/users/3a6fa32c5ae14add86e30fbf370e5291/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.779251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa6453-a516-4b45-a0e8-579af46e0940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240e5ce0a8ef40518b2c2ff59b64ca41, user_id=3a6fa32c5ae14add86e30fbf370e5291, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:47.782151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa6453-a516-4b45-a0e8-579af46e0940 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:47.804942 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3941/7793] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:47 2022] PUT /identity/v3/projects/240e5ce0a8ef40518b2c2ff59b64ca41/users/3a6fa32c5ae14add86e30fbf370e5291/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:47.806471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe120c4-a822-4358-b6f2-620d23db84fa None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.806766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe120c4-a822-4358-b6f2-620d23db84fa None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.807041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe120c4-a822-4358-b6f2-620d23db84fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.837727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ebe120c4-a822-4358-b6f2-620d23db84fa None None] MFA Rules not processed for user `3a6fa32c5ae14add86e30fbf370e5291`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:47.838855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ebe120c4-a822-4358-b6f2-620d23db84fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:47.929963 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3853/7794] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:47.931301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa6453-a516-4b45-a0e8-579af46e0940 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:47.947600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa6453-a516-4b45-a0e8-579af46e0940 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:47.955583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c49bd-187c-45ed-acaf-47fd93809274 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:47.956964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c49bd-187c-45ed-acaf-47fd93809274 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:47.957337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c49bd-187c-45ed-acaf-47fd93809274 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:47.957627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c49bd-187c-45ed-acaf-47fd93809274 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:47.958325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c49bd-187c-45ed-acaf-47fd93809274 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:47.960150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c49bd-187c-45ed-acaf-47fd93809274 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:47.985853 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3942/7795] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:17:47 2022] POST /identity/v3/domains => generated 277 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:47.990685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe120c4-a822-4358-b6f2-620d23db84fa None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:48.013445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe120c4-a822-4358-b6f2-620d23db84fa None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:48.099210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:48.110710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a325f0-d33a-45e6-9dd3-0187be4cd9e8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:48.111959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a325f0-d33a-45e6-9dd3-0187be4cd9e8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:48.112765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a325f0-d33a-45e6-9dd3-0187be4cd9e8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:48.113257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a325f0-d33a-45e6-9dd3-0187be4cd9e8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/67b879590c0e4e458a9bbf138cf43193/config` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:48.114187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a325f0-d33a-45e6-9dd3-0187be4cd9e8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=67b879590c0e4e458a9bbf138cf43193)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:48.115358 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-06a325f0-d33a-45e6-9dd3-0187be4cd9e8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:17:48.115358 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:17:48.115358 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:17:48.116830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a325f0-d33a-45e6-9dd3-0187be4cd9e8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:48.178328 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3854/7796] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:17:47 2022] PUT /identity/v3/domains/67b879590c0e4e458a9bbf138cf43193/config => generated 138 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:48.182578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c49bd-187c-45ed-acaf-47fd93809274 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:48.196970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c49bd-187c-45ed-acaf-47fd93809274 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:48.203176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898b345-98b0-4dec-9358-36597fd785e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:48.203856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898b345-98b0-4dec-9358-36597fd785e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:48.204142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898b345-98b0-4dec-9358-36597fd785e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:48.204419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898b345-98b0-4dec-9358-36597fd785e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/67b879590c0e4e458a9bbf138cf43193/config/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:48.205735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898b345-98b0-4dec-9358-36597fd785e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=67b879590c0e4e458a9bbf138cf43193, group=identity)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:48.206809 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-4898b345-98b0-4dec-9358-36597fd785e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:17:48.206809 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:17:48.206809 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:17:48.207696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898b345-98b0-4dec-9358-36597fd785e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:48.213129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:48.220180 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3943/7797] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Sep 3 21:17:48 2022] GET /identity/v3/domains/67b879590c0e4e458a9bbf138cf43193/config/identity => generated 45 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:48.221366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a325f0-d33a-45e6-9dd3-0187be4cd9e8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:48.234436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a325f0-d33a-45e6-9dd3-0187be4cd9e8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:48.248350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10e996-6d4c-48f0-a755-86aa6f114eca tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:48.249176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10e996-6d4c-48f0-a755-86aa6f114eca tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:48.250846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10e996-6d4c-48f0-a755-86aa6f114eca tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:48.251157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10e996-6d4c-48f0-a755-86aa6f114eca tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/67b879590c0e4e458a9bbf138cf43193/config/identity/driver` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:48.252503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10e996-6d4c-48f0-a755-86aa6f114eca tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=67b879590c0e4e458a9bbf138cf43193, group=identity, option=driver)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:48.253870 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-af10e996-6d4c-48f0-a755-86aa6f114eca tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:17:48.253870 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:17:48.253870 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:17:48.254779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10e996-6d4c-48f0-a755-86aa6f114eca tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:48.263050 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3855/7798] 10.222.0.16 () {58 vars in 1296 bytes} [Sat Sep 3 21:17:48 2022] GET /identity/v3/domains/67b879590c0e4e458a9bbf138cf43193/config/identity/driver => generated 31 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:48.266759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898b345-98b0-4dec-9358-36597fd785e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:48.278056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898b345-98b0-4dec-9358-36597fd785e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:48.283128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05670e-a701-4755-a84e-8bcc6914ea35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:48.283794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05670e-a701-4755-a84e-8bcc6914ea35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:48.284084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05670e-a701-4755-a84e-8bcc6914ea35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:48.284365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05670e-a701-4755-a84e-8bcc6914ea35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/67b879590c0e4e458a9bbf138cf43193/config/ldap` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:48.287920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05670e-a701-4755-a84e-8bcc6914ea35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=67b879590c0e4e458a9bbf138cf43193, group=ldap)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:48.287920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05670e-a701-4755-a84e-8bcc6914ea35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:48.292941 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3944/7799] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Sep 3 21:17:48 2022] GET /identity/v3/domains/67b879590c0e4e458a9bbf138cf43193/config/ldap => generated 106 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:48.298973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10e996-6d4c-48f0-a755-86aa6f114eca tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:48.308516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10e996-6d4c-48f0-a755-86aa6f114eca tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:48.316296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08c8bf-15da-4604-ada0-1d593567bf66 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:48.318019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08c8bf-15da-4604-ada0-1d593567bf66 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:48.318019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08c8bf-15da-4604-ada0-1d593567bf66 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:48.318019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08c8bf-15da-4604-ada0-1d593567bf66 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/67b879590c0e4e458a9bbf138cf43193/config/ldap/url` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:48.318806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08c8bf-15da-4604-ada0-1d593567bf66 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=67b879590c0e4e458a9bbf138cf43193, group=ldap, option=url)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:48.321340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08c8bf-15da-4604-ada0-1d593567bf66 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:48.327595 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3856/7800] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Sep 3 21:17:48 2022] GET /identity/v3/domains/67b879590c0e4e458a9bbf138cf43193/config/ldap/url => generated 46 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:48.331804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05670e-a701-4755-a84e-8bcc6914ea35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:48.345253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05670e-a701-4755-a84e-8bcc6914ea35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:48.350618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b83cd-0ba4-42e1-a798-a0bcfe160dfc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:48.351340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b83cd-0ba4-42e1-a798-a0bcfe160dfc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:48.351630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b83cd-0ba4-42e1-a798-a0bcfe160dfc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:48.351910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b83cd-0ba4-42e1-a798-a0bcfe160dfc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/67b879590c0e4e458a9bbf138cf43193/config/ldap/user_tree_dn` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:48.353127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b83cd-0ba4-42e1-a798-a0bcfe160dfc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=67b879590c0e4e458a9bbf138cf43193, group=ldap, option=user_tree_dn)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:48.354760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b83cd-0ba4-42e1-a798-a0bcfe160dfc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:48.364767 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3945/7801] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Sep 3 21:17:48 2022] GET /identity/v3/domains/67b879590c0e4e458a9bbf138cf43193/config/ldap/user_tree_dn => generated 63 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:48.366131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08c8bf-15da-4604-ada0-1d593567bf66 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:48.379792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08c8bf-15da-4604-ada0-1d593567bf66 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:48.385427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324797a-45fa-4df8-9d1b-e583231261e9 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:48.386093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324797a-45fa-4df8-9d1b-e583231261e9 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:48.386571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324797a-45fa-4df8-9d1b-e583231261e9 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:48.386686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324797a-45fa-4df8-9d1b-e583231261e9 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/67b879590c0e4e458a9bbf138cf43193/config` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:48.387291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324797a-45fa-4df8-9d1b-e583231261e9 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=67b879590c0e4e458a9bbf138cf43193)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:48.388267 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-6324797a-45fa-4df8-9d1b-e583231261e9 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:17:48.388267 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:17:48.388267 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:17:48.389205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324797a-45fa-4df8-9d1b-e583231261e9 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:48.406598 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3857/7802] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:17:48 2022] DELETE /identity/v3/domains/67b879590c0e4e458a9bbf138cf43193/config => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:48.407499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b83cd-0ba4-42e1-a798-a0bcfe160dfc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:48.419251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b83cd-0ba4-42e1-a798-a0bcfe160dfc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:48.424244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70818b-c846-4a89-8de7-37a58e16cdfd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:48.425055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70818b-c846-4a89-8de7-37a58e16cdfd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:48.425421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70818b-c846-4a89-8de7-37a58e16cdfd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:48.425788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70818b-c846-4a89-8de7-37a58e16cdfd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/67b879590c0e4e458a9bbf138cf43193` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:48.427138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70818b-c846-4a89-8de7-37a58e16cdfd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=67b879590c0e4e458a9bbf138cf43193)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:48.428875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70818b-c846-4a89-8de7-37a58e16cdfd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:48.456131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e70818b-c846-4a89-8de7-37a58e16cdfd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '67b879590c0e4e458a9bbf138cf43193'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.457226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e70818b-c846-4a89-8de7-37a58e16cdfd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '67b879590c0e4e458a9bbf138cf43193'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.486991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e70818b-c846-4a89-8de7-37a58e16cdfd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '67b879590c0e4e458a9bbf138cf43193'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.487882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5e70818b-c846-4a89-8de7-37a58e16cdfd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '67b879590c0e4e458a9bbf138cf43193'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.496862 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3946/7803] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:17:48 2022] PATCH /identity/v3/domains/67b879590c0e4e458a9bbf138cf43193 => generated 278 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:17:48.497978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324797a-45fa-4df8-9d1b-e583231261e9 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:48.500100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6324797a-45fa-4df8-9d1b-e583231261e9 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:48.515937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324797a-45fa-4df8-9d1b-e583231261e9 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:48.589592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:48.590396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:48.590721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:48.591214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/67b879590c0e4e458a9bbf138cf43193` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:48.607693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=67b879590c0e4e458a9bbf138cf43193)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:48.609441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:48.640175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '67b879590c0e4e458a9bbf138cf43193'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.644581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '67b879590c0e4e458a9bbf138cf43193'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.709176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b879590c0e4e458a9bbf138cf43193', 'request_id': 'req-5887255c-ef79-468e-a33c-4b24806d196c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.721643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b879590c0e4e458a9bbf138cf43193', 'request_id': 'req-5887255c-ef79-468e-a33c-4b24806d196c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.732275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b879590c0e4e458a9bbf138cf43193', 'request_id': 'req-5887255c-ef79-468e-a33c-4b24806d196c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.741407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:48.753953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] The token cache is being invalidate because project 67b879590c0e4e458a9bbf138cf43193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:48.754532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.755412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.771692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '67b879590c0e4e458a9bbf138cf43193', 'request_id': 'req-5887255c-ef79-468e-a33c-4b24806d196c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.777006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '67b879590c0e4e458a9bbf138cf43193', 'request_id': 'req-5887255c-ef79-468e-a33c-4b24806d196c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.777887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '67b879590c0e4e458a9bbf138cf43193', 'request_id': 'req-5887255c-ef79-468e-a33c-4b24806d196c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:48.785464 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3858/7804] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:17:48 2022] DELETE /identity/v3/domains/67b879590c0e4e458a9bbf138cf43193 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:48.790354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70818b-c846-4a89-8de7-37a58e16cdfd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:48.800949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70818b-c846-4a89-8de7-37a58e16cdfd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:48.879816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf2701-5795-4f21-bd7b-657aef3cd127 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:48.880683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf2701-5795-4f21-bd7b-657aef3cd127 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:48.881112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf2701-5795-4f21-bd7b-657aef3cd127 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:48.881571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf2701-5795-4f21-bd7b-657aef3cd127 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:48.883519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf2701-5795-4f21-bd7b-657aef3cd127 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:48.885902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf2701-5795-4f21-bd7b-657aef3cd127 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:48.914265 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3947/7805] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:17:48 2022] POST /identity/v3/domains => generated 274 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:48.915478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:48.916985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:48.932713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887255c-ef79-468e-a33c-4b24806d196c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:49.006792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256351b2-f587-408c-9dbc-45620983bd35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:49.007874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256351b2-f587-408c-9dbc-45620983bd35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:49.008374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256351b2-f587-408c-9dbc-45620983bd35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:49.008728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256351b2-f587-408c-9dbc-45620983bd35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/9ec22cc71fe54f5493b01033fa06cae7/config` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:49.009593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256351b2-f587-408c-9dbc-45620983bd35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=9ec22cc71fe54f5493b01033fa06cae7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:49.011442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256351b2-f587-408c-9dbc-45620983bd35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:49.053175 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3859/7806] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:17:48 2022] PUT /identity/v3/domains/9ec22cc71fe54f5493b01033fa06cae7/config => generated 138 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:49.057672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf2701-5795-4f21-bd7b-657aef3cd127 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:49.069640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf2701-5795-4f21-bd7b-657aef3cd127 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:49.074511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e4543-4d54-4973-b653-0cd69c624190 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:49.075153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e4543-4d54-4973-b653-0cd69c624190 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:49.075439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e4543-4d54-4973-b653-0cd69c624190 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:49.075733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e4543-4d54-4973-b653-0cd69c624190 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/9ec22cc71fe54f5493b01033fa06cae7/config` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:49.076406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e4543-4d54-4973-b653-0cd69c624190 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=9ec22cc71fe54f5493b01033fa06cae7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:49.077463 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-e37e4543-4d54-4973-b653-0cd69c624190 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:17:49.077463 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:17:49.077463 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:17:49.078359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e4543-4d54-4973-b653-0cd69c624190 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:49.120637 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3948/7807] 10.222.0.16 () {60 vars in 1271 bytes} [Sat Sep 3 21:17:49 2022] PATCH /identity/v3/domains/9ec22cc71fe54f5493b01033fa06cae7/config => generated 141 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:49.124543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256351b2-f587-408c-9dbc-45620983bd35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:49.135920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256351b2-f587-408c-9dbc-45620983bd35 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:49.142284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d7986-0b84-46f0-ade3-980706c0b741 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:49.142953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d7986-0b84-46f0-ade3-980706c0b741 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:49.143270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d7986-0b84-46f0-ade3-980706c0b741 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:49.143627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d7986-0b84-46f0-ade3-980706c0b741 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/9ec22cc71fe54f5493b01033fa06cae7/config` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:49.144862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d7986-0b84-46f0-ade3-980706c0b741 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=9ec22cc71fe54f5493b01033fa06cae7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:49.146540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d7986-0b84-46f0-ade3-980706c0b741 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:49.156096 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3860/7808] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Sep 3 21:17:49 2022] GET /identity/v3/domains/9ec22cc71fe54f5493b01033fa06cae7/config => generated 141 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:17:49.159947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e4543-4d54-4973-b653-0cd69c624190 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:49.172867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e4543-4d54-4973-b653-0cd69c624190 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:49.178309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cc9f5-6726-4ff8-89f2-a7c9227a731c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:49.180018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cc9f5-6726-4ff8-89f2-a7c9227a731c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:49.180451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cc9f5-6726-4ff8-89f2-a7c9227a731c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:49.180849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cc9f5-6726-4ff8-89f2-a7c9227a731c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/9ec22cc71fe54f5493b01033fa06cae7/config` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:49.181570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cc9f5-6726-4ff8-89f2-a7c9227a731c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=9ec22cc71fe54f5493b01033fa06cae7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:49.185889 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-8a2cc9f5-6726-4ff8-89f2-a7c9227a731c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:17:49.185889 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:17:49.185889 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:17:49.187199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cc9f5-6726-4ff8-89f2-a7c9227a731c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:49.199826 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3949/7809] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:17:49 2022] DELETE /identity/v3/domains/9ec22cc71fe54f5493b01033fa06cae7/config => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:49.203954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d7986-0b84-46f0-ade3-980706c0b741 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:49.215579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d7986-0b84-46f0-ade3-980706c0b741 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:49.224245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01dea3-9f90-4da6-846e-832f449259fc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:49.225038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01dea3-9f90-4da6-846e-832f449259fc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:49.225410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01dea3-9f90-4da6-846e-832f449259fc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:49.225758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01dea3-9f90-4da6-846e-832f449259fc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/9ec22cc71fe54f5493b01033fa06cae7/config` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:49.227529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c01dea3-9f90-4da6-846e-832f449259fc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=9ec22cc71fe54f5493b01033fa06cae7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:49.229278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c01dea3-9f90-4da6-846e-832f449259fc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-9c01dea3-9f90-4da6-846e-832f449259fc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Could not find any options in domain configuration for domain 9ec22cc71fe54f5493b01033fa06cae7.: keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain 9ec22cc71fe54f5493b01033fa06cae7. Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 03 21:17:49.238534 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 03 21:17:49.239754 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 03 21:17:49.239754 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 03 21:17:49.239754 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:17:49.239754 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:17:49.239754 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 03 21:17:49.239754 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 03 21:17:49.239754 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain 9ec22cc71fe54f5493b01033fa06cae7. Sep 03 21:17:49.239754 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:49.241131 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3861/7810] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Sep 3 21:17:49 2022] GET /identity/v3/domains/9ec22cc71fe54f5493b01033fa06cae7/config => generated 151 bytes in 38 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:17:49.245839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cc9f5-6726-4ff8-89f2-a7c9227a731c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:49.256509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cc9f5-6726-4ff8-89f2-a7c9227a731c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:49.261453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374d7b6-9c77-4e83-bd64-241ce4618b6c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:49.262734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374d7b6-9c77-4e83-bd64-241ce4618b6c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:49.263036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374d7b6-9c77-4e83-bd64-241ce4618b6c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:49.263834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374d7b6-9c77-4e83-bd64-241ce4618b6c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/9ec22cc71fe54f5493b01033fa06cae7/config` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:49.263984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374d7b6-9c77-4e83-bd64-241ce4618b6c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=9ec22cc71fe54f5493b01033fa06cae7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:49.266275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374d7b6-9c77-4e83-bd64-241ce4618b6c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:49.273723 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3950/7811] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:17:49 2022] DELETE /identity/v3/domains/9ec22cc71fe54f5493b01033fa06cae7/config => generated 0 bytes in 28 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:49.278316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01dea3-9f90-4da6-846e-832f449259fc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:49.290212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01dea3-9f90-4da6-846e-832f449259fc tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:49.295148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dadc8d-fa16-4fdc-8e95-d0adf6f67154 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:49.295774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dadc8d-fa16-4fdc-8e95-d0adf6f67154 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:49.296063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dadc8d-fa16-4fdc-8e95-d0adf6f67154 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:49.296341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dadc8d-fa16-4fdc-8e95-d0adf6f67154 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/9ec22cc71fe54f5493b01033fa06cae7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:49.297734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dadc8d-fa16-4fdc-8e95-d0adf6f67154 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=9ec22cc71fe54f5493b01033fa06cae7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:49.299338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dadc8d-fa16-4fdc-8e95-d0adf6f67154 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:49.334115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-92dadc8d-fa16-4fdc-8e95-d0adf6f67154 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '9ec22cc71fe54f5493b01033fa06cae7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.335457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-92dadc8d-fa16-4fdc-8e95-d0adf6f67154 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '9ec22cc71fe54f5493b01033fa06cae7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.363284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-92dadc8d-fa16-4fdc-8e95-d0adf6f67154 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '9ec22cc71fe54f5493b01033fa06cae7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.363948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-92dadc8d-fa16-4fdc-8e95-d0adf6f67154 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '9ec22cc71fe54f5493b01033fa06cae7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.369549 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3862/7812] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:17:49 2022] PATCH /identity/v3/domains/9ec22cc71fe54f5493b01033fa06cae7 => generated 275 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:49.370561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374d7b6-9c77-4e83-bd64-241ce4618b6c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:49.372443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f374d7b6-9c77-4e83-bd64-241ce4618b6c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:49.386175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374d7b6-9c77-4e83-bd64-241ce4618b6c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:49.428960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=85,keystone:UPDATE=1,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:49.456125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:49.456304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:49.456553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:49.457000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/9ec22cc71fe54f5493b01033fa06cae7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:49.470286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=9ec22cc71fe54f5493b01033fa06cae7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:49.472185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:49.500920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '9ec22cc71fe54f5493b01033fa06cae7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.503922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '9ec22cc71fe54f5493b01033fa06cae7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.571486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec22cc71fe54f5493b01033fa06cae7', 'request_id': 'req-5465c711-ec12-4687-bf87-f9db17b3f1c0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.582911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec22cc71fe54f5493b01033fa06cae7', 'request_id': 'req-5465c711-ec12-4687-bf87-f9db17b3f1c0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.592273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec22cc71fe54f5493b01033fa06cae7', 'request_id': 'req-5465c711-ec12-4687-bf87-f9db17b3f1c0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.604374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] The token cache is being invalidate because project 9ec22cc71fe54f5493b01033fa06cae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:49.604983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.608197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.616890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '9ec22cc71fe54f5493b01033fa06cae7', 'request_id': 'req-5465c711-ec12-4687-bf87-f9db17b3f1c0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.620643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '9ec22cc71fe54f5493b01033fa06cae7', 'request_id': 'req-5465c711-ec12-4687-bf87-f9db17b3f1c0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.621468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '9ec22cc71fe54f5493b01033fa06cae7', 'request_id': 'req-5465c711-ec12-4687-bf87-f9db17b3f1c0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:49.628298 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3951/7813] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:17:49 2022] DELETE /identity/v3/domains/9ec22cc71fe54f5493b01033fa06cae7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:49.634151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dadc8d-fa16-4fdc-8e95-d0adf6f67154 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:49.646003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dadc8d-fa16-4fdc-8e95-d0adf6f67154 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:49.721065 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:49.735630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78697d2-c046-41f9-82df-467e2218ffcf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:49.737099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78697d2-c046-41f9-82df-467e2218ffcf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:49.737644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78697d2-c046-41f9-82df-467e2218ffcf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:49.738557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78697d2-c046-41f9-82df-467e2218ffcf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:49.739533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78697d2-c046-41f9-82df-467e2218ffcf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:49.742455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78697d2-c046-41f9-82df-467e2218ffcf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:49.768233 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3863/7814] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Sep 3 21:17:49 2022] POST /identity/v3/domains => generated 275 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:49.770690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:49.772670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:49.789293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465c711-ec12-4687-bf87-f9db17b3f1c0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:49.865209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8c5b2-fcd3-4669-9145-438aa783987f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:49.866014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8c5b2-fcd3-4669-9145-438aa783987f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:49.867513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8c5b2-fcd3-4669-9145-438aa783987f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:49.867914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8c5b2-fcd3-4669-9145-438aa783987f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:49.869223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8c5b2-fcd3-4669-9145-438aa783987f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=f339c8d0ab2343d3ac90fd0d76f13b60)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:49.871096 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-70c8c5b2-fcd3-4669-9145-438aa783987f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:17:49.871096 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:17:49.871096 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:17:49.872674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8c5b2-fcd3-4669-9145-438aa783987f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:49.903588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:49.922104 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3952/7815] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Sep 3 21:17:49 2022] PUT /identity/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config => generated 138 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:49.926362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78697d2-c046-41f9-82df-467e2218ffcf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:49.939396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78697d2-c046-41f9-82df-467e2218ffcf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:49.946315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7940c-dee1-41c4-bb62-18f184990881 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:49.947103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7940c-dee1-41c4-bb62-18f184990881 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:49.947462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7940c-dee1-41c4-bb62-18f184990881 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:49.947819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7940c-dee1-41c4-bb62-18f184990881 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:49.948740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7940c-dee1-41c4-bb62-18f184990881 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=f339c8d0ab2343d3ac90fd0d76f13b60, group=identity)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:49.950118 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-bdb7940c-dee1-41c4-bb62-18f184990881 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:17:49.950118 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:17:49.950118 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:17:49.951891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7940c-dee1-41c4-bb62-18f184990881 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.006457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8c5b2-fcd3-4669-9145-438aa783987f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.006922 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3864/7816] 10.222.0.16 () {60 vars in 1297 bytes} [Sat Sep 3 21:17:49 2022] PATCH /identity/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config/identity => generated 175 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:17:50.024828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8c5b2-fcd3-4669-9145-438aa783987f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.047848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280458a-aef9-4919-be1d-818c9f8026a7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.050256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280458a-aef9-4919-be1d-818c9f8026a7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.050810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280458a-aef9-4919-be1d-818c9f8026a7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.051286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280458a-aef9-4919-be1d-818c9f8026a7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config/identity/driver` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.052727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280458a-aef9-4919-be1d-818c9f8026a7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=f339c8d0ab2343d3ac90fd0d76f13b60, group=identity, option=driver)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.055234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280458a-aef9-4919-be1d-818c9f8026a7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.089658 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3953/7817] 10.222.0.16 () {60 vars in 1318 bytes} [Sat Sep 3 21:17:50 2022] PATCH /identity/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config/identity/driver => generated 175 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:17:50.092478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7940c-dee1-41c4-bb62-18f184990881 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.104159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7940c-dee1-41c4-bb62-18f184990881 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.109888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cae3cd-5430-4d23-a69f-6e49bda5f752 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.110450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cae3cd-5430-4d23-a69f-6e49bda5f752 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.110812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cae3cd-5430-4d23-a69f-6e49bda5f752 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.111159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cae3cd-5430-4d23-a69f-6e49bda5f752 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config/identity/driver` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.111844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cae3cd-5430-4d23-a69f-6e49bda5f752 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=f339c8d0ab2343d3ac90fd0d76f13b60, group=identity, option=driver)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.113587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cae3cd-5430-4d23-a69f-6e49bda5f752 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.133249 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3865/7818] 10.222.0.16 () {58 vars in 1299 bytes} [Sat Sep 3 21:17:50 2022] DELETE /identity/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config/identity/driver => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:50.137510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280458a-aef9-4919-be1d-818c9f8026a7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.150561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280458a-aef9-4919-be1d-818c9f8026a7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.156830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a7d10-398f-4d62-8362-afed8a174481 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.157528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a7d10-398f-4d62-8362-afed8a174481 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.157816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a7d10-398f-4d62-8362-afed8a174481 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.158100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a7d10-398f-4d62-8362-afed8a174481 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config/identity/driver` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.159392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a7d10-398f-4d62-8362-afed8a174481 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=f339c8d0ab2343d3ac90fd0d76f13b60, group=identity, option=driver)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.168057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a7d10-398f-4d62-8362-afed8a174481 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-a50a7d10-398f-4d62-8362-afed8a174481 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Could not find option driver in group identity in domain configuration for domain f339c8d0ab2343d3ac90fd0d76f13b60.: keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain f339c8d0ab2343d3ac90fd0d76f13b60. Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:50.172944 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 03 21:17:50.178042 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 03 21:17:50.178042 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 03 21:17:50.178042 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 03 21:17:50.178042 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:17:50.178042 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:17:50.178042 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 03 21:17:50.178042 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 03 21:17:50.178042 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain f339c8d0ab2343d3ac90fd0d76f13b60. Sep 03 21:17:50.178042 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:50.178042 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3954/7819] 10.222.0.16 () {58 vars in 1296 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config/identity/driver => generated 171 bytes in 37 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 03 21:17:50.180192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cae3cd-5430-4d23-a69f-6e49bda5f752 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.190570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cae3cd-5430-4d23-a69f-6e49bda5f752 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.196462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85ceab-322f-4c9f-b78d-cdea38fde79b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.197654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ceab-322f-4c9f-b78d-cdea38fde79b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.197654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ceab-322f-4c9f-b78d-cdea38fde79b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.197867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ceab-322f-4c9f-b78d-cdea38fde79b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.198673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85ceab-322f-4c9f-b78d-cdea38fde79b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=f339c8d0ab2343d3ac90fd0d76f13b60, group=identity)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.205393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85ceab-322f-4c9f-b78d-cdea38fde79b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.236590 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3866/7820] 10.222.0.16 () {58 vars in 1278 bytes} [Sat Sep 3 21:17:50 2022] DELETE /identity/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config/identity => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:50.241528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a7d10-398f-4d62-8362-afed8a174481 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.255944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a7d10-398f-4d62-8362-afed8a174481 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.270567 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf20383-ddc3-4048-8df4-fd4cc355917e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.271313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf20383-ddc3-4048-8df4-fd4cc355917e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.271775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf20383-ddc3-4048-8df4-fd4cc355917e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.272131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf20383-ddc3-4048-8df4-fd4cc355917e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.273446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf20383-ddc3-4048-8df4-fd4cc355917e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=f339c8d0ab2343d3ac90fd0d76f13b60, group=identity)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.275141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf20383-ddc3-4048-8df4-fd4cc355917e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-caf20383-ddc3-4048-8df4-fd4cc355917e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Could not find group identity in domain configuration for domain f339c8d0ab2343d3ac90fd0d76f13b60.: keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain f339c8d0ab2343d3ac90fd0d76f13b60. Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:50.284336 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 03 21:17:50.285406 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 03 21:17:50.285406 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 03 21:17:50.285406 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 03 21:17:50.285406 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:17:50.285406 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:17:50.285406 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 03 21:17:50.285406 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 03 21:17:50.285406 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain f339c8d0ab2343d3ac90fd0d76f13b60. Sep 03 21:17:50.285406 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:50.285983 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3955/7821] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config/identity => generated 154 bytes in 45 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 03 21:17:50.289814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85ceab-322f-4c9f-b78d-cdea38fde79b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.300857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85ceab-322f-4c9f-b78d-cdea38fde79b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.306539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16bfd4-2382-4f44-94d1-7a3b8f38c44f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.307186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16bfd4-2382-4f44-94d1-7a3b8f38c44f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.307474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16bfd4-2382-4f44-94d1-7a3b8f38c44f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.307761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16bfd4-2382-4f44-94d1-7a3b8f38c44f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.308359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16bfd4-2382-4f44-94d1-7a3b8f38c44f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=f339c8d0ab2343d3ac90fd0d76f13b60)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.310135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16bfd4-2382-4f44-94d1-7a3b8f38c44f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.319149 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3867/7822] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:17:50 2022] DELETE /identity/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60/config => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:50.324148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf20383-ddc3-4048-8df4-fd4cc355917e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.335115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf20383-ddc3-4048-8df4-fd4cc355917e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.339824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb55ea-ebc4-4bb1-bc44-d3c59c3444d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.340454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb55ea-ebc4-4bb1-bc44-d3c59c3444d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.340744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb55ea-ebc4-4bb1-bc44-d3c59c3444d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.341161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb55ea-ebc4-4bb1-bc44-d3c59c3444d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.342441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb55ea-ebc4-4bb1-bc44-d3c59c3444d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f339c8d0ab2343d3ac90fd0d76f13b60)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.344045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb55ea-ebc4-4bb1-bc44-d3c59c3444d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.366129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c8cb55ea-ebc4-4bb1-bc44-d3c59c3444d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f339c8d0ab2343d3ac90fd0d76f13b60'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.367466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c8cb55ea-ebc4-4bb1-bc44-d3c59c3444d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f339c8d0ab2343d3ac90fd0d76f13b60'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.396668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c8cb55ea-ebc4-4bb1-bc44-d3c59c3444d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f339c8d0ab2343d3ac90fd0d76f13b60'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.397416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c8cb55ea-ebc4-4bb1-bc44-d3c59c3444d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f339c8d0ab2343d3ac90fd0d76f13b60'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.400184 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3956/7823] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:17:50 2022] PATCH /identity/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60 => generated 276 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:17:50.404447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16bfd4-2382-4f44-94d1-7a3b8f38c44f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.406563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4d16bfd4-2382-4f44-94d1-7a3b8f38c44f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:50.420862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16bfd4-2382-4f44-94d1-7a3b8f38c44f tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.498648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.499420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.499780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.500136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.515171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f339c8d0ab2343d3ac90fd0d76f13b60)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.517006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.542944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f339c8d0ab2343d3ac90fd0d76f13b60'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.547241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f339c8d0ab2343d3ac90fd0d76f13b60'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.583668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:50.599680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f339c8d0ab2343d3ac90fd0d76f13b60', 'request_id': 'req-2b65b4c5-2180-4532-9a4f-e22935d1d05c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.608336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f339c8d0ab2343d3ac90fd0d76f13b60', 'request_id': 'req-2b65b4c5-2180-4532-9a4f-e22935d1d05c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.616317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f339c8d0ab2343d3ac90fd0d76f13b60', 'request_id': 'req-2b65b4c5-2180-4532-9a4f-e22935d1d05c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.627952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] The token cache is being invalidate because project f339c8d0ab2343d3ac90fd0d76f13b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:50.628367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.629183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.638387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f339c8d0ab2343d3ac90fd0d76f13b60', 'request_id': 'req-2b65b4c5-2180-4532-9a4f-e22935d1d05c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.641791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f339c8d0ab2343d3ac90fd0d76f13b60', 'request_id': 'req-2b65b4c5-2180-4532-9a4f-e22935d1d05c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.642594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f339c8d0ab2343d3ac90fd0d76f13b60', 'request_id': 'req-2b65b4c5-2180-4532-9a4f-e22935d1d05c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:50.648382 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3868/7824] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:17:50 2022] DELETE /identity/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:50.653817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb55ea-ebc4-4bb1-bc44-d3c59c3444d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.664827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb55ea-ebc4-4bb1-bc44-d3c59c3444d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.723282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c9185-61be-415f-9001-e604415b8de1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.723913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c9185-61be-415f-9001-e604415b8de1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.724190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c9185-61be-415f-9001-e604415b8de1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.724469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c9185-61be-415f-9001-e604415b8de1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.725083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c9185-61be-415f-9001-e604415b8de1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.726687 ncdv-master-1 devstack@keystone.service[99210]: WARNING py.warnings [None req-a30c9185-61be-415f-9001-e604415b8de1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:17:50.726687 ncdv-master-1 devstack@keystone.service[99210]: warnings.warn(msg) Sep 03 21:17:50.726687 ncdv-master-1 devstack@keystone.service[99210]: Sep 03 21:17:50.727585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c9185-61be-415f-9001-e604415b8de1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.734954 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3957/7825] 10.222.0.16 () {58 vars in 1173 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/config/default => generated 1527 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:50.736157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.737746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:50.751148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65b4c5-2180-4532-9a4f-e22935d1d05c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.756402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe728b2-ec93-44c5-886f-7b02dda9a72e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.757085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe728b2-ec93-44c5-886f-7b02dda9a72e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.757398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe728b2-ec93-44c5-886f-7b02dda9a72e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.757682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe728b2-ec93-44c5-886f-7b02dda9a72e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/identity/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.758269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe728b2-ec93-44c5-886f-7b02dda9a72e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.759289 ncdv-master-1 devstack@keystone.service[99211]: WARNING py.warnings [None req-ffe728b2-ec93-44c5-886f-7b02dda9a72e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 21:17:50.759289 ncdv-master-1 devstack@keystone.service[99211]: warnings.warn(msg) Sep 03 21:17:50.759289 ncdv-master-1 devstack@keystone.service[99211]: Sep 03 21:17:50.760191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe728b2-ec93-44c5-886f-7b02dda9a72e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.761616 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3869/7826] 10.222.0.16 () {58 vars in 1200 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/config/identity/default => generated 64 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:50.765465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c9185-61be-415f-9001-e604415b8de1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.776574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c9185-61be-415f-9001-e604415b8de1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.781330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6189e0a-43b0-43ee-8f23-986842302335 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.782070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6189e0a-43b0-43ee-8f23-986842302335 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.782262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6189e0a-43b0-43ee-8f23-986842302335 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.782553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6189e0a-43b0-43ee-8f23-986842302335 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/identity/driver/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.783139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6189e0a-43b0-43ee-8f23-986842302335 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=driver)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.784739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6189e0a-43b0-43ee-8f23-986842302335 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.789715 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3958/7827] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/config/identity/driver/default => generated 30 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:50.791161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe728b2-ec93-44c5-886f-7b02dda9a72e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.803276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe728b2-ec93-44c5-886f-7b02dda9a72e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.808341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1c6b0-36cb-41df-a3e3-a02e5ce58b91 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.809039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c6b0-36cb-41df-a3e3-a02e5ce58b91 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.809365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c6b0-36cb-41df-a3e3-a02e5ce58b91 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.809612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c6b0-36cb-41df-a3e3-a02e5ce58b91 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/identity/list_limit/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.810339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1c6b0-36cb-41df-a3e3-a02e5ce58b91 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=list_limit)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.811947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1c6b0-36cb-41df-a3e3-a02e5ce58b91 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.816083 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3870/7828] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/config/identity/list_limit/default => generated 33 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:50.817115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6189e0a-43b0-43ee-8f23-986842302335 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.830097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6189e0a-43b0-43ee-8f23-986842302335 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.839240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d65ea-6915-4bac-87c9-6b55bf7e92a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.839240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d65ea-6915-4bac-87c9-6b55bf7e92a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.839240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d65ea-6915-4bac-87c9-6b55bf7e92a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.839622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d65ea-6915-4bac-87c9-6b55bf7e92a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.839622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d65ea-6915-4bac-87c9-6b55bf7e92a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.840209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d65ea-6915-4bac-87c9-6b55bf7e92a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.842052 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3959/7829] 10.222.0.16 () {58 vars in 1188 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/config/ldap/default => generated 1476 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:50.846171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1c6b0-36cb-41df-a3e3-a02e5ce58b91 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.857598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1c6b0-36cb-41df-a3e3-a02e5ce58b91 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.862446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3403f-ed75-414b-8e39-b55c394a1934 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.863104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3403f-ed75-414b-8e39-b55c394a1934 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.863391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3403f-ed75-414b-8e39-b55c394a1934 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.863673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3403f-ed75-414b-8e39-b55c394a1934 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/url/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.864267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3403f-ed75-414b-8e39-b55c394a1934 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=url)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.865895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3403f-ed75-414b-8e39-b55c394a1934 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.867280 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3871/7830] 10.222.0.16 () {58 vars in 1200 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/config/ldap/url/default => generated 40 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:50.872501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d65ea-6915-4bac-87c9-6b55bf7e92a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.884494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d65ea-6915-4bac-87c9-6b55bf7e92a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.890034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaadde-cbe8-4679-acba-cd5eb08e2456 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.891330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaadde-cbe8-4679-acba-cd5eb08e2456 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.891330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaadde-cbe8-4679-acba-cd5eb08e2456 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.891330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaadde-cbe8-4679-acba-cd5eb08e2456 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.892135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaadde-cbe8-4679-acba-cd5eb08e2456 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.893794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaadde-cbe8-4679-acba-cd5eb08e2456 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.898539 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3960/7831] 10.222.0.16 () {58 vars in 1203 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/config/ldap/user/default => generated 27 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 03 21:17:50.899413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3403f-ed75-414b-8e39-b55c394a1934 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.910311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3403f-ed75-414b-8e39-b55c394a1934 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.915722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72c089-96af-428a-9536-4b4390ca88ec tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.916376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72c089-96af-428a-9536-4b4390ca88ec tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.916669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72c089-96af-428a-9536-4b4390ca88ec tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.916989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72c089-96af-428a-9536-4b4390ca88ec tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/suffix/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.917589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72c089-96af-428a-9536-4b4390ca88ec tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=suffix)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.919194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72c089-96af-428a-9536-4b4390ca88ec tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.923087 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3872/7832] 10.222.0.16 () {58 vars in 1209 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/config/ldap/suffix/default => generated 44 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:50.924241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaadde-cbe8-4679-acba-cd5eb08e2456 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.936253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaadde-cbe8-4679-acba-cd5eb08e2456 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.943256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc57bf-2b8b-4dce-a291-68f67f363962 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.944136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc57bf-2b8b-4dce-a291-68f67f363962 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.944616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc57bf-2b8b-4dce-a291-68f67f363962 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.944968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc57bf-2b8b-4dce-a291-68f67f363962 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/query_scope/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.945573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc57bf-2b8b-4dce-a291-68f67f363962 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=query_scope)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.947216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc57bf-2b8b-4dce-a291-68f67f363962 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.948326 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3961/7833] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/config/ldap/query_scope/default => generated 35 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:50.952799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72c089-96af-428a-9536-4b4390ca88ec tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.966461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72c089-96af-428a-9536-4b4390ca88ec tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.971430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabf495-807f-42f0-82e5-2872de3472f6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.972090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabf495-807f-42f0-82e5-2872de3472f6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:50.972380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabf495-807f-42f0-82e5-2872de3472f6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:50.972661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabf495-807f-42f0-82e5-2872de3472f6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/page_size/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:50.973291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabf495-807f-42f0-82e5-2872de3472f6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=page_size)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:50.974916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabf495-807f-42f0-82e5-2872de3472f6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:50.976159 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3873/7834] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/config/ldap/page_size/default => generated 29 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:50.981818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc57bf-2b8b-4dce-a291-68f67f363962 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:50.994161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc57bf-2b8b-4dce-a291-68f67f363962 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:50.999815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bfeca-c35f-4105-9b6a-bd74ae333d2b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:50.999988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bfeca-c35f-4105-9b6a-bd74ae333d2b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.000255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bfeca-c35f-4105-9b6a-bd74ae333d2b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.000804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bfeca-c35f-4105-9b6a-bd74ae333d2b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/alias_dereferencing/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.001545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bfeca-c35f-4105-9b6a-bd74ae333d2b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=alias_dereferencing)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.003590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bfeca-c35f-4105-9b6a-bd74ae333d2b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.008092 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3962/7835] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Sep 3 21:17:50 2022] GET /identity/v3/domains/config/ldap/alias_dereferencing/default => generated 47 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.009323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabf495-807f-42f0-82e5-2872de3472f6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.020759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabf495-807f-42f0-82e5-2872de3472f6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.026249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3856983-54d6-4fd4-b3c2-4d19aa968ad3 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.027234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3856983-54d6-4fd4-b3c2-4d19aa968ad3 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.027716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3856983-54d6-4fd4-b3c2-4d19aa968ad3 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.028155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3856983-54d6-4fd4-b3c2-4d19aa968ad3 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/debug_level/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.028757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3856983-54d6-4fd4-b3c2-4d19aa968ad3 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=debug_level)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.030404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3856983-54d6-4fd4-b3c2-4d19aa968ad3 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.031768 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3874/7836] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/debug_level/default => generated 34 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.035936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bfeca-c35f-4105-9b6a-bd74ae333d2b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.047604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bfeca-c35f-4105-9b6a-bd74ae333d2b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.052317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735dcd9a-7c35-45e4-9607-d88dd95e2137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.052990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dcd9a-7c35-45e4-9607-d88dd95e2137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.053275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dcd9a-7c35-45e4-9607-d88dd95e2137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.053564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dcd9a-7c35-45e4-9607-d88dd95e2137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/chase_referrals/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.054156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735dcd9a-7c35-45e4-9607-d88dd95e2137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=chase_referrals)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.056776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735dcd9a-7c35-45e4-9607-d88dd95e2137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.057290 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3963/7837] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/chase_referrals/default => generated 38 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.061186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3856983-54d6-4fd4-b3c2-4d19aa968ad3 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.072378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3856983-54d6-4fd4-b3c2-4d19aa968ad3 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.077351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f900d-46bd-490d-90ef-58ae7cde7fef tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.078111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f900d-46bd-490d-90ef-58ae7cde7fef tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.078415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f900d-46bd-490d-90ef-58ae7cde7fef tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.078785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f900d-46bd-490d-90ef-58ae7cde7fef tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_tree_dn/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.079449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f900d-46bd-490d-90ef-58ae7cde7fef tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_tree_dn)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.081393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f900d-46bd-490d-90ef-58ae7cde7fef tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.082734 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3875/7838] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_tree_dn/default => generated 35 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.087043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735dcd9a-7c35-45e4-9607-d88dd95e2137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.098143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735dcd9a-7c35-45e4-9607-d88dd95e2137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.103617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31542fe0-2a12-45e2-9e30-28a4c09ee004 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.104274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31542fe0-2a12-45e2-9e30-28a4c09ee004 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.104563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31542fe0-2a12-45e2-9e30-28a4c09ee004 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.104847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31542fe0-2a12-45e2-9e30-28a4c09ee004 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_filter/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.105753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31542fe0-2a12-45e2-9e30-28a4c09ee004 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_filter)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.107338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31542fe0-2a12-45e2-9e30-28a4c09ee004 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.108491 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3964/7839] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_filter/default => generated 34 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 03 21:17:51.112383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f900d-46bd-490d-90ef-58ae7cde7fef tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.131093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f900d-46bd-490d-90ef-58ae7cde7fef tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.137354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023834f-51ba-4277-a656-3ea26c9f35a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.138016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023834f-51ba-4277-a656-3ea26c9f35a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.138302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023834f-51ba-4277-a656-3ea26c9f35a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.138694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023834f-51ba-4277-a656-3ea26c9f35a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_objectclass/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.139164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023834f-51ba-4277-a656-3ea26c9f35a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_objectclass)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.140855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023834f-51ba-4277-a656-3ea26c9f35a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.142304 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3876/7840] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_objectclass/default => generated 50 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.146084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31542fe0-2a12-45e2-9e30-28a4c09ee004 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.157167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31542fe0-2a12-45e2-9e30-28a4c09ee004 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.161857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01148476-6641-42e8-a4b4-a38659cddb56 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.162493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01148476-6641-42e8-a4b4-a38659cddb56 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.162781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01148476-6641-42e8-a4b4-a38659cddb56 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.163066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01148476-6641-42e8-a4b4-a38659cddb56 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_id_attribute/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.163683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01148476-6641-42e8-a4b4-a38659cddb56 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_id_attribute)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.165339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01148476-6641-42e8-a4b4-a38659cddb56 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.166408 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3965/7841] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_id_attribute/default => generated 40 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.170206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023834f-51ba-4277-a656-3ea26c9f35a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.181042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023834f-51ba-4277-a656-3ea26c9f35a4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.185856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45673d5c-1711-4189-86d4-e034cbb49b8c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.186629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45673d5c-1711-4189-86d4-e034cbb49b8c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.186932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45673d5c-1711-4189-86d4-e034cbb49b8c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.187221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45673d5c-1711-4189-86d4-e034cbb49b8c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_name_attribute/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.187814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45673d5c-1711-4189-86d4-e034cbb49b8c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_name_attribute)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.190725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45673d5c-1711-4189-86d4-e034cbb49b8c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.191971 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3877/7842] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_name_attribute/default => generated 42 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.197106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01148476-6641-42e8-a4b4-a38659cddb56 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.207277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01148476-6641-42e8-a4b4-a38659cddb56 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.213732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897eab21-417f-448b-b884-a8461d548cf6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.214477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eab21-417f-448b-b884-a8461d548cf6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.214773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eab21-417f-448b-b884-a8461d548cf6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.215202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eab21-417f-448b-b884-a8461d548cf6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_mail_attribute/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.216297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897eab21-417f-448b-b884-a8461d548cf6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_mail_attribute)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.217568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897eab21-417f-448b-b884-a8461d548cf6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.220945 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3966/7843] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_mail_attribute/default => generated 44 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.222876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45673d5c-1711-4189-86d4-e034cbb49b8c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.233364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45673d5c-1711-4189-86d4-e034cbb49b8c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.238036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faf53a-2ff4-40f0-ae8c-fe7c219d1f3b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.238680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faf53a-2ff4-40f0-ae8c-fe7c219d1f3b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.238969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faf53a-2ff4-40f0-ae8c-fe7c219d1f3b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.239252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faf53a-2ff4-40f0-ae8c-fe7c219d1f3b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_description_attribute/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.239845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88faf53a-2ff4-40f0-ae8c-fe7c219d1f3b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_description_attribute)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.241468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88faf53a-2ff4-40f0-ae8c-fe7c219d1f3b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.242648 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3878/7844] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_description_attribute/default => generated 58 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.246932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897eab21-417f-448b-b884-a8461d548cf6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.258326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897eab21-417f-448b-b884-a8461d548cf6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.263877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91dd5ff-ce5c-4194-bd97-7971a16d3893 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.264528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dd5ff-ce5c-4194-bd97-7971a16d3893 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.264818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dd5ff-ce5c-4194-bd97-7971a16d3893 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.265134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dd5ff-ce5c-4194-bd97-7971a16d3893 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_pass_attribute/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.265723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91dd5ff-ce5c-4194-bd97-7971a16d3893 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_pass_attribute)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.267859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91dd5ff-ce5c-4194-bd97-7971a16d3893 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.272669 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3967/7845] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_pass_attribute/default => generated 52 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.274303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faf53a-2ff4-40f0-ae8c-fe7c219d1f3b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.310399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faf53a-2ff4-40f0-ae8c-fe7c219d1f3b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.318907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8907278-0dc4-4ed7-b6aa-9a9a706d424a tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.321169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8907278-0dc4-4ed7-b6aa-9a9a706d424a tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.321640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8907278-0dc4-4ed7-b6aa-9a9a706d424a tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.321939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8907278-0dc4-4ed7-b6aa-9a9a706d424a tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_attribute/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.324371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8907278-0dc4-4ed7-b6aa-9a9a706d424a tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_attribute)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.328320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8907278-0dc4-4ed7-b6aa-9a9a706d424a tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.337240 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3879/7846] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_enabled_attribute/default => generated 50 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.339676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91dd5ff-ce5c-4194-bd97-7971a16d3893 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.351825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91dd5ff-ce5c-4194-bd97-7971a16d3893 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.357564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fca7a-b47f-40d6-994a-ad2df4fe9393 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.358210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fca7a-b47f-40d6-994a-ad2df4fe9393 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.358499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fca7a-b47f-40d6-994a-ad2df4fe9393 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.358945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fca7a-b47f-40d6-994a-ad2df4fe9393 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_invert/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.359699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fca7a-b47f-40d6-994a-ad2df4fe9393 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_invert)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.361457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fca7a-b47f-40d6-994a-ad2df4fe9393 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.366739 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3968/7847] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_enabled_invert/default => generated 43 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.367180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8907278-0dc4-4ed7-b6aa-9a9a706d424a tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.378346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8907278-0dc4-4ed7-b6aa-9a9a706d424a tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.384286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868f029-f700-476d-b879-5432c7e2818d tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.385261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868f029-f700-476d-b879-5432c7e2818d tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.385803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868f029-f700-476d-b879-5432c7e2818d tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.386167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868f029-f700-476d-b879-5432c7e2818d tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_mask/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.387405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868f029-f700-476d-b879-5432c7e2818d tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_mask)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.389279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868f029-f700-476d-b879-5432c7e2818d tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.390918 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3880/7848] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_enabled_mask/default => generated 37 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.395725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fca7a-b47f-40d6-994a-ad2df4fe9393 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.412056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fca7a-b47f-40d6-994a-ad2df4fe9393 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.417833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70950c30-d36e-4f5e-97c1-c7cf96f3d1eb tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.418604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70950c30-d36e-4f5e-97c1-c7cf96f3d1eb tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.419019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70950c30-d36e-4f5e-97c1-c7cf96f3d1eb tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.419402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70950c30-d36e-4f5e-97c1-c7cf96f3d1eb tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_default/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.420232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70950c30-d36e-4f5e-97c1-c7cf96f3d1eb tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_default)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.421985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70950c30-d36e-4f5e-97c1-c7cf96f3d1eb tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.426393 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3969/7849] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_enabled_default/default => generated 45 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.427255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868f029-f700-476d-b879-5432c7e2818d tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.439554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868f029-f700-476d-b879-5432c7e2818d tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.445882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d4fc2-b10b-4524-afff-39633deba348 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.446733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d4fc2-b10b-4524-afff-39633deba348 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.447285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d4fc2-b10b-4524-afff-39633deba348 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.447643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d4fc2-b10b-4524-afff-39633deba348 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_attribute_ignore/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.448341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d4fc2-b10b-4524-afff-39633deba348 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_attribute_ignore)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.450078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d4fc2-b10b-4524-afff-39633deba348 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.451600 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3881/7850] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_attribute_ignore/default => generated 62 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.456131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70950c30-d36e-4f5e-97c1-c7cf96f3d1eb tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.473367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70950c30-d36e-4f5e-97c1-c7cf96f3d1eb tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.479111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3471a6-42b4-44df-81a0-d63467961209 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.480147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3471a6-42b4-44df-81a0-d63467961209 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.480830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3471a6-42b4-44df-81a0-d63467961209 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.481241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3471a6-42b4-44df-81a0-d63467961209 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_default_project_id_attribute/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.482348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3471a6-42b4-44df-81a0-d63467961209 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_default_project_id_attribute)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.484267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3471a6-42b4-44df-81a0-d63467961209 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.485639 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3970/7851] 10.222.0.16 () {58 vars in 1290 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_default_project_id_attribute/default => generated 56 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.490037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d4fc2-b10b-4524-afff-39633deba348 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.506072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d4fc2-b10b-4524-afff-39633deba348 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.519830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b086db4-8d6e-4466-9ee6-058289e63cfe tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.520665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b086db4-8d6e-4466-9ee6-058289e63cfe tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.520995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b086db4-8d6e-4466-9ee6-058289e63cfe tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.521284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b086db4-8d6e-4466-9ee6-058289e63cfe tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.521884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b086db4-8d6e-4466-9ee6-058289e63cfe tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.523707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b086db4-8d6e-4466-9ee6-058289e63cfe tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.525426 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3882/7852] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation/default => generated 46 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.529835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3471a6-42b4-44df-81a0-d63467961209 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.545102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3471a6-42b4-44df-81a0-d63467961209 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.550284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb775b0-4676-4d86-9144-8d504672b2f5 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.550949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb775b0-4676-4d86-9144-8d504672b2f5 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.551242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb775b0-4676-4d86-9144-8d504672b2f5 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.551531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb775b0-4676-4d86-9144-8d504672b2f5 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_dn/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.552137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb775b0-4676-4d86-9144-8d504672b2f5 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_dn)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.553807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb775b0-4676-4d86-9144-8d504672b2f5 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.555733 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3971/7853] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_dn/default => generated 48 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.559743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b086db4-8d6e-4466-9ee6-058289e63cfe tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.583768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b086db4-8d6e-4466-9ee6-058289e63cfe tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.593059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df88fd5-d8eb-4c9f-a042-cb74121def72 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.593937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88fd5-d8eb-4c9f-a042-cb74121def72 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.594368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88fd5-d8eb-4c9f-a042-cb74121def72 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.594753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88fd5-d8eb-4c9f-a042-cb74121def72 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.595625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df88fd5-d8eb-4c9f-a042-cb74121def72 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_use_group_config)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.597398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df88fd5-d8eb-4c9f-a042-cb74121def72 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.600945 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3883/7854] 10.222.0.16 () {58 vars in 1308 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default => generated 63 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.603096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb775b0-4676-4d86-9144-8d504672b2f5 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.617327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb775b0-4676-4d86-9144-8d504672b2f5 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.624480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4234d7-25ff-4ba3-888c-8909e5a24512 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.625385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4234d7-25ff-4ba3-888c-8909e5a24512 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.625698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4234d7-25ff-4ba3-888c-8909e5a24512 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.626084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4234d7-25ff-4ba3-888c-8909e5a24512 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_additional_attribute_mapping/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.626865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4234d7-25ff-4ba3-888c-8909e5a24512 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_additional_attribute_mapping)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.628675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4234d7-25ff-4ba3-888c-8909e5a24512 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.629751 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3972/7855] 10.222.0.16 () {58 vars in 1290 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/user_additional_attribute_mapping/default => generated 54 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.636510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df88fd5-d8eb-4c9f-a042-cb74121def72 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.654420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df88fd5-d8eb-4c9f-a042-cb74121def72 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.660622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98e3b0-f87b-43ef-9c9d-510b6e3f1f4c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.661336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98e3b0-f87b-43ef-9c9d-510b6e3f1f4c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.661606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98e3b0-f87b-43ef-9c9d-510b6e3f1f4c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.661903 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98e3b0-f87b-43ef-9c9d-510b6e3f1f4c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_tree_dn/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.662492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98e3b0-f87b-43ef-9c9d-510b6e3f1f4c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_tree_dn)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.664241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98e3b0-f87b-43ef-9c9d-510b6e3f1f4c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.668526 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3884/7856] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/group_tree_dn/default => generated 36 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.670360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4234d7-25ff-4ba3-888c-8909e5a24512 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.675747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=3,keystone:INSERT=2,keystone:SELECT=94,keystone:UPDATE=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:51.683281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4234d7-25ff-4ba3-888c-8909e5a24512 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.690635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf45b9-70a1-4c24-962a-a7566bc3caf0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.691585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf45b9-70a1-4c24-962a-a7566bc3caf0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.692148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf45b9-70a1-4c24-962a-a7566bc3caf0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.692658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf45b9-70a1-4c24-962a-a7566bc3caf0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_filter/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.693945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf45b9-70a1-4c24-962a-a7566bc3caf0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_filter)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.696426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf45b9-70a1-4c24-962a-a7566bc3caf0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.698301 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3973/7857] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/group_filter/default => generated 35 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.702158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98e3b0-f87b-43ef-9c9d-510b6e3f1f4c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.715694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98e3b0-f87b-43ef-9c9d-510b6e3f1f4c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.730523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6488-9b02-4086-8e91-c1719da6a7e4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.731307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6488-9b02-4086-8e91-c1719da6a7e4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.731745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6488-9b02-4086-8e91-c1719da6a7e4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.732009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6488-9b02-4086-8e91-c1719da6a7e4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_objectclass/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.732686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae6488-9b02-4086-8e91-c1719da6a7e4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_objectclass)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.738163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae6488-9b02-4086-8e91-c1719da6a7e4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.745016 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3885/7858] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/group_objectclass/default => generated 50 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 03 21:17:51.746201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf45b9-70a1-4c24-962a-a7566bc3caf0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.764151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf45b9-70a1-4c24-962a-a7566bc3caf0 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.771059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c27e1-cc8e-49bb-91a4-f86df4e7a23e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.771822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c27e1-cc8e-49bb-91a4-f86df4e7a23e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.772109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c27e1-cc8e-49bb-91a4-f86df4e7a23e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.772641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c27e1-cc8e-49bb-91a4-f86df4e7a23e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_id_attribute/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.773478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c27e1-cc8e-49bb-91a4-f86df4e7a23e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_id_attribute)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.775183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c27e1-cc8e-49bb-91a4-f86df4e7a23e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.776359 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3974/7859] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/group_id_attribute/default => generated 41 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.781319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6488-9b02-4086-8e91-c1719da6a7e4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.794955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6488-9b02-4086-8e91-c1719da6a7e4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.805696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c178c9-b8a4-4559-b51a-7b0fab4706dd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.806520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c178c9-b8a4-4559-b51a-7b0fab4706dd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.806910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c178c9-b8a4-4559-b51a-7b0fab4706dd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.807288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c178c9-b8a4-4559-b51a-7b0fab4706dd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_name_attribute/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.807988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c178c9-b8a4-4559-b51a-7b0fab4706dd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_name_attribute)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.809762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c178c9-b8a4-4559-b51a-7b0fab4706dd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.811355 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3886/7860] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/group_name_attribute/default => generated 43 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.815480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c27e1-cc8e-49bb-91a4-f86df4e7a23e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.826637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c27e1-cc8e-49bb-91a4-f86df4e7a23e tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.831953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ab81d-fc9b-4171-8f8d-e6cf8b3366e1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.832745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ab81d-fc9b-4171-8f8d-e6cf8b3366e1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.833163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ab81d-fc9b-4171-8f8d-e6cf8b3366e1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.833548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ab81d-fc9b-4171-8f8d-e6cf8b3366e1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_members_are_ids/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.834263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ab81d-fc9b-4171-8f8d-e6cf8b3366e1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_members_are_ids)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.836120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ab81d-fc9b-4171-8f8d-e6cf8b3366e1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.837264 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3975/7861] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/group_members_are_ids/default => generated 45 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.842000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c178c9-b8a4-4559-b51a-7b0fab4706dd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.853542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c178c9-b8a4-4559-b51a-7b0fab4706dd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.858889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38abbe-e43b-480b-be0e-4969847a87e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.859705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38abbe-e43b-480b-be0e-4969847a87e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.860066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38abbe-e43b-480b-be0e-4969847a87e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.860426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38abbe-e43b-480b-be0e-4969847a87e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_member_attribute/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.861192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38abbe-e43b-480b-be0e-4969847a87e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_member_attribute)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.863119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38abbe-e43b-480b-be0e-4969847a87e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.864944 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3887/7862] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/group_member_attribute/default => generated 49 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.869112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ab81d-fc9b-4171-8f8d-e6cf8b3366e1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.882975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ab81d-fc9b-4171-8f8d-e6cf8b3366e1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.888003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61fee6-d829-4de4-bb5d-b7bde7fd63ba tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.888949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61fee6-d829-4de4-bb5d-b7bde7fd63ba tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.889246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61fee6-d829-4de4-bb5d-b7bde7fd63ba tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.889529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61fee6-d829-4de4-bb5d-b7bde7fd63ba tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_desc_attribute/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.890126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61fee6-d829-4de4-bb5d-b7bde7fd63ba tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_desc_attribute)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.891759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61fee6-d829-4de4-bb5d-b7bde7fd63ba tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.892911 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3976/7863] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/group_desc_attribute/default => generated 52 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 03 21:17:51.897291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38abbe-e43b-480b-be0e-4969847a87e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.907602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38abbe-e43b-480b-be0e-4969847a87e6 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.912415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffb39b-3bbc-4a26-9ebb-dc04339fcff4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.913150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb39b-3bbc-4a26-9ebb-dc04339fcff4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.913439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb39b-3bbc-4a26-9ebb-dc04339fcff4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.914310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb39b-3bbc-4a26-9ebb-dc04339fcff4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_attribute_ignore/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.914912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffb39b-3bbc-4a26-9ebb-dc04339fcff4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_attribute_ignore)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.916603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffb39b-3bbc-4a26-9ebb-dc04339fcff4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.918125 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3888/7864] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/group_attribute_ignore/default => generated 43 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.921776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61fee6-d829-4de4-bb5d-b7bde7fd63ba tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.932555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61fee6-d829-4de4-bb5d-b7bde7fd63ba tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.937972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82ff1a-7131-44e7-98ca-40d4a9267c93 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.938906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82ff1a-7131-44e7-98ca-40d4a9267c93 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.939259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82ff1a-7131-44e7-98ca-40d4a9267c93 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.939611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82ff1a-7131-44e7-98ca-40d4a9267c93 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_additional_attribute_mapping/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.940332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82ff1a-7131-44e7-98ca-40d4a9267c93 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_additional_attribute_mapping)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.942145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82ff1a-7131-44e7-98ca-40d4a9267c93 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.943398 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3977/7865] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/group_additional_attribute_mapping/default => generated 55 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 03 21:17:51.947343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffb39b-3bbc-4a26-9ebb-dc04339fcff4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.960135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffb39b-3bbc-4a26-9ebb-dc04339fcff4 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:51.968543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d1b09-cebb-46ba-9c38-17d7f6f2566c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:51.969525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d1b09-cebb-46ba-9c38-17d7f6f2566c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:51.969819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d1b09-cebb-46ba-9c38-17d7f6f2566c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:51.970101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d1b09-cebb-46ba-9c38-17d7f6f2566c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertfile/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:51.970701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d1b09-cebb-46ba-9c38-17d7f6f2566c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertfile)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:51.973946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d1b09-cebb-46ba-9c38-17d7f6f2566c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:51.976873 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3889/7866] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/tls_cacertfile/default => generated 37 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:51.981357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82ff1a-7131-44e7-98ca-40d4a9267c93 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:51.994464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82ff1a-7131-44e7-98ca-40d4a9267c93 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.000190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507f369-8e3e-456a-b584-7cd2d95d0637 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.000972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507f369-8e3e-456a-b584-7cd2d95d0637 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.001329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507f369-8e3e-456a-b584-7cd2d95d0637 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.002050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507f369-8e3e-456a-b584-7cd2d95d0637 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertdir/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.002728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507f369-8e3e-456a-b584-7cd2d95d0637 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertdir)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.004601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507f369-8e3e-456a-b584-7cd2d95d0637 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.005918 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3978/7867] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:17:51 2022] GET /identity/v3/domains/config/ldap/tls_cacertdir/default => generated 36 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 03 21:17:52.010938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d1b09-cebb-46ba-9c38-17d7f6f2566c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.023566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d1b09-cebb-46ba-9c38-17d7f6f2566c tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.029046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fa177-6d97-4b4f-ade2-928eae3fb137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.029690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fa177-6d97-4b4f-ade2-928eae3fb137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.029975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fa177-6d97-4b4f-ade2-928eae3fb137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.030255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fa177-6d97-4b4f-ade2-928eae3fb137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_tls/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.030845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6fa177-6d97-4b4f-ade2-928eae3fb137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_tls)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.032616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6fa177-6d97-4b4f-ade2-928eae3fb137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.033810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3890/7868] 10.222.0.16 () {58 vars in 1212 bytes} [Sat Sep 3 21:17:52 2022] GET /identity/v3/domains/config/ldap/use_tls/default => generated 31 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 03 21:17:52.038081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507f369-8e3e-456a-b584-7cd2d95d0637 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.051440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507f369-8e3e-456a-b584-7cd2d95d0637 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.057691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf37b7-5fd3-4f2b-9ec0-9884bb4812ed tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.058467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf37b7-5fd3-4f2b-9ec0-9884bb4812ed tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.058860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf37b7-5fd3-4f2b-9ec0-9884bb4812ed tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.059245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf37b7-5fd3-4f2b-9ec0-9884bb4812ed tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_req_cert/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.059949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf37b7-5fd3-4f2b-9ec0-9884bb4812ed tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_req_cert)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.061706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf37b7-5fd3-4f2b-9ec0-9884bb4812ed tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.065754 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3979/7869] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Sep 3 21:17:52 2022] GET /identity/v3/domains/config/ldap/tls_req_cert/default => generated 39 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:52.066955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fa177-6d97-4b4f-ade2-928eae3fb137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.079371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fa177-6d97-4b4f-ade2-928eae3fb137 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.085242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78512ec-a2f5-4134-a692-cea23967c974 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.085982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78512ec-a2f5-4134-a692-cea23967c974 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.086278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78512ec-a2f5-4134-a692-cea23967c974 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.086557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78512ec-a2f5-4134-a692-cea23967c974 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_pool/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.091123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78512ec-a2f5-4134-a692-cea23967c974 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_pool)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.093116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78512ec-a2f5-4134-a692-cea23967c974 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.095028 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3891/7870] 10.222.0.16 () {58 vars in 1215 bytes} [Sat Sep 3 21:17:52 2022] GET /identity/v3/domains/config/ldap/use_pool/default => generated 31 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:52.099140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf37b7-5fd3-4f2b-9ec0-9884bb4812ed tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.114378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf37b7-5fd3-4f2b-9ec0-9884bb4812ed tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.120547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d737b45-fdbb-4d6a-8b2d-b37d29e80e76 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.121246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d737b45-fdbb-4d6a-8b2d-b37d29e80e76 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.121537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d737b45-fdbb-4d6a-8b2d-b37d29e80e76 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.121821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d737b45-fdbb-4d6a-8b2d-b37d29e80e76 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_size/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.122592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d737b45-fdbb-4d6a-8b2d-b37d29e80e76 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_size)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.124222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d737b45-fdbb-4d6a-8b2d-b37d29e80e76 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.128205 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3980/7871] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Sep 3 21:17:52 2022] GET /identity/v3/domains/config/ldap/pool_size/default => generated 30 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:52.134831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78512ec-a2f5-4134-a692-cea23967c974 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.147492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78512ec-a2f5-4134-a692-cea23967c974 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.152326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf6361-c6c0-480b-96aa-d84d258f4407 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.153043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf6361-c6c0-480b-96aa-d84d258f4407 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.153336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf6361-c6c0-480b-96aa-d84d258f4407 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.153618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf6361-c6c0-480b-96aa-d84d258f4407 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_max/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.154211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf6361-c6c0-480b-96aa-d84d258f4407 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_max)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.155824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf6361-c6c0-480b-96aa-d84d258f4407 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.157645 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3892/7872] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:52 2022] GET /identity/v3/domains/config/ldap/pool_retry_max/default => generated 34 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:52.161493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d737b45-fdbb-4d6a-8b2d-b37d29e80e76 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.174052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d737b45-fdbb-4d6a-8b2d-b37d29e80e76 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.181354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae3add-71e0-4b54-bf5b-7c4118a898c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.183619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae3add-71e0-4b54-bf5b-7c4118a898c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.183905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae3add-71e0-4b54-bf5b-7c4118a898c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.185453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae3add-71e0-4b54-bf5b-7c4118a898c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_delay/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.186072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae3add-71e0-4b54-bf5b-7c4118a898c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_delay)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.187697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae3add-71e0-4b54-bf5b-7c4118a898c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.188847 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3981/7873] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Sep 3 21:17:52 2022] GET /identity/v3/domains/config/ldap/pool_retry_delay/default => generated 38 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:52.193191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf6361-c6c0-480b-96aa-d84d258f4407 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.203550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf6361-c6c0-480b-96aa-d84d258f4407 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.210214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562c705-d375-4aa1-8038-e301e16029d1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.210933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562c705-d375-4aa1-8038-e301e16029d1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.211223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562c705-d375-4aa1-8038-e301e16029d1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.211555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562c705-d375-4aa1-8038-e301e16029d1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_timeout/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.212128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562c705-d375-4aa1-8038-e301e16029d1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_timeout)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.213769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562c705-d375-4aa1-8038-e301e16029d1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.216274 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3893/7874] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Sep 3 21:17:52 2022] GET /identity/v3/domains/config/ldap/pool_connection_timeout/default => generated 44 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:52.220348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae3add-71e0-4b54-bf5b-7c4118a898c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.232571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae3add-71e0-4b54-bf5b-7c4118a898c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.237517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18a4db-49f7-43e0-8cb0-0fd8345fcdd8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.238275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18a4db-49f7-43e0-8cb0-0fd8345fcdd8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.238656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18a4db-49f7-43e0-8cb0-0fd8345fcdd8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.239004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18a4db-49f7-43e0-8cb0-0fd8345fcdd8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_lifetime/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.240114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18a4db-49f7-43e0-8cb0-0fd8345fcdd8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_lifetime)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.243423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18a4db-49f7-43e0-8cb0-0fd8345fcdd8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.246406 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3982/7875] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Sep 3 21:17:52 2022] GET /identity/v3/domains/config/ldap/pool_connection_lifetime/default => generated 46 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:52.249694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562c705-d375-4aa1-8038-e301e16029d1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.262803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562c705-d375-4aa1-8038-e301e16029d1 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.271660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab0698-e8a5-4c2b-9e1e-7793380770d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.272425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab0698-e8a5-4c2b-9e1e-7793380770d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.272826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab0698-e8a5-4c2b-9e1e-7793380770d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.273902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab0698-e8a5-4c2b-9e1e-7793380770d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_auth_pool/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.274573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab0698-e8a5-4c2b-9e1e-7793380770d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_auth_pool)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.276329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab0698-e8a5-4c2b-9e1e-7793380770d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.277668 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3894/7876] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Sep 3 21:17:52 2022] GET /identity/v3/domains/config/ldap/use_auth_pool/default => generated 36 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:52.281668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18a4db-49f7-43e0-8cb0-0fd8345fcdd8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.293317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18a4db-49f7-43e0-8cb0-0fd8345fcdd8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.298511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a708e8-23d5-4e73-911c-ad2915f6f63b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.299451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a708e8-23d5-4e73-911c-ad2915f6f63b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.299950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a708e8-23d5-4e73-911c-ad2915f6f63b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.300420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a708e8-23d5-4e73-911c-ad2915f6f63b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_size/default` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.301265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a708e8-23d5-4e73-911c-ad2915f6f63b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_size)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.303101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a708e8-23d5-4e73-911c-ad2915f6f63b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.305303 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3983/7877] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:52 2022] GET /identity/v3/domains/config/ldap/auth_pool_size/default => generated 36 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 03 21:17:52.308832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab0698-e8a5-4c2b-9e1e-7793380770d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.321539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab0698-e8a5-4c2b-9e1e-7793380770d7 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.328078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07229a-3e73-4f24-8c29-4fb3893d06c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.328823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07229a-3e73-4f24-8c29-4fb3893d06c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.329222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07229a-3e73-4f24-8c29-4fb3893d06c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.329514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07229a-3e73-4f24-8c29-4fb3893d06c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_connection_lifetime/default` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.330117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07229a-3e73-4f24-8c29-4fb3893d06c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_connection_lifetime)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.331850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07229a-3e73-4f24-8c29-4fb3893d06c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.333252 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3895/7878] 10.222.0.16 () {58 vars in 1278 bytes} [Sat Sep 3 21:17:52 2022] GET /identity/v3/domains/config/ldap/auth_pool_connection_lifetime/default => generated 50 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 03 21:17:52.342538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a708e8-23d5-4e73-911c-ad2915f6f63b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.356378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a708e8-23d5-4e73-911c-ad2915f6f63b tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.361290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffb532-468a-4c3d-bcf2-34825766a8fd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.362023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffb532-468a-4c3d-bcf2-34825766a8fd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.362383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffb532-468a-4c3d-bcf2-34825766a8fd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.362906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffb532-468a-4c3d-bcf2-34825766a8fd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.370666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ffb532-468a-4c3d-bcf2-34825766a8fd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-e7ffb532-468a-4c3d-bcf2-34825766a8fd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Could not find domain: f339c8d0ab2343d3ac90fd0d76f13b60.: keystone.exception.DomainNotFound: Could not find domain: f339c8d0ab2343d3ac90fd0d76f13b60. Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f339c8d0ab2343d3ac90fd0d76f13b60. Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:52.376031 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:17:52.377232 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:17:52.378688 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: f339c8d0ab2343d3ac90fd0d76f13b60. Sep 03 21:17:52.380779 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:52.380779 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3984/7879] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:17:52 2022] PATCH /identity/v3/domains/f339c8d0ab2343d3ac90fd0d76f13b60 => generated 112 bytes in 38 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:17:52.381994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07229a-3e73-4f24-8c29-4fb3893d06c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.393878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07229a-3e73-4f24-8c29-4fb3893d06c8 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.402651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87253d5-d4ce-482f-96ab-cb3fc5757baf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.403721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87253d5-d4ce-482f-96ab-cb3fc5757baf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.403721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87253d5-d4ce-482f-96ab-cb3fc5757baf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.404304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87253d5-d4ce-482f-96ab-cb3fc5757baf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/9ec22cc71fe54f5493b01033fa06cae7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.417514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87253d5-d4ce-482f-96ab-cb3fc5757baf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-f87253d5-d4ce-482f-96ab-cb3fc5757baf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Could not find domain: 9ec22cc71fe54f5493b01033fa06cae7.: keystone.exception.DomainNotFound: Could not find domain: 9ec22cc71fe54f5493b01033fa06cae7. Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 9ec22cc71fe54f5493b01033fa06cae7. Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:52.423628 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:17:52.425531 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application with Lock( Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:17:52.426586 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 9ec22cc71fe54f5493b01033fa06cae7. Sep 03 21:17:52.427812 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:52.427812 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3896/7880] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:17:52 2022] PATCH /identity/v3/domains/9ec22cc71fe54f5493b01033fa06cae7 => generated 112 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:17:52.429983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffb532-468a-4c3d-bcf2-34825766a8fd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.441346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffb532-468a-4c3d-bcf2-34825766a8fd tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.446464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec24ac-2e73-47f8-a3a9-f0cb299bfdc2 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a6fa32c5ae14add86e30fbf370e5291', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '240e5ce0a8ef40518b2c2ff59b64ca41', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.447183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec24ac-2e73-47f8-a3a9-f0cb299bfdc2 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.447538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec24ac-2e73-47f8-a3a9-f0cb299bfdc2 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.447885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec24ac-2e73-47f8-a3a9-f0cb299bfdc2 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] PATH_INFO: `/v3/domains/67b879590c0e4e458a9bbf138cf43193` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.455625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec24ac-2e73-47f8-a3a9-f0cb299bfdc2 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-8bec24ac-2e73-47f8-a3a9-f0cb299bfdc2 tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Could not find domain: 67b879590c0e4e458a9bbf138cf43193.: keystone.exception.DomainNotFound: Could not find domain: 67b879590c0e4e458a9bbf138cf43193. Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 67b879590c0e4e458a9bbf138cf43193. Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:52.461387 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:17:52.462460 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application with Lock( Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._enter() Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.creator() Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application created_value = creator( Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 21:17:52.463422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 67b879590c0e4e458a9bbf138cf43193. Sep 03 21:17:52.464422 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:52.464422 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3985/7881] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Sep 3 21:17:52 2022] PATCH /identity/v3/domains/67b879590c0e4e458a9bbf138cf43193 => generated 112 bytes in 34 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 03 21:17:52.466605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87253d5-d4ce-482f-96ab-cb3fc5757baf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.468447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f87253d5-d4ce-482f-96ab-cb3fc5757baf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:52.510227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:52.558475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87253d5-d4ce-482f-96ab-cb3fc5757baf tempest-DomainConfigurationTestJSON-102305757 tempest-DomainConfigurationTestJSON-102305757-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.564192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.564856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.565337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.565639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] PATH_INFO: `/v3/users/d64b029d8f6341d68a6f397a05b92af3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.566880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d64b029d8f6341d68a6f397a05b92af3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.569039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.629716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd64b029d8f6341d68a6f397a05b92af3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:52.650056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd64b029d8f6341d68a6f397a05b92af3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:52.650970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd64b029d8f6341d68a6f397a05b92af3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:52.673840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd64b029d8f6341d68a6f397a05b92af3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:52.691906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd64b029d8f6341d68a6f397a05b92af3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:52.703200 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3897/7882] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:52 2022] DELETE /identity/v3/users/d64b029d8f6341d68a6f397a05b92af3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:52.707534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edeae8e-6bd0-459e-8bff-01e1271af8a7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.707956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edeae8e-6bd0-459e-8bff-01e1271af8a7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.708322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edeae8e-6bd0-459e-8bff-01e1271af8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.737242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4edeae8e-6bd0-459e-8bff-01e1271af8a7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:52.738610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4edeae8e-6bd0-459e-8bff-01e1271af8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:52.809733 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3986/7883] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:52.820622 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:52.822189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:52.900206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31468108-27d6-4060-a4f8-bf11e93414a5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:52.904366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcea3a-ffc5-4a0e-a341-6ff1f9c7a739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:52.905210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcea3a-ffc5-4a0e-a341-6ff1f9c7a739 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:52.905507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcea3a-ffc5-4a0e-a341-6ff1f9c7a739 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:52.905785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcea3a-ffc5-4a0e-a341-6ff1f9c7a739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:52.916384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bcea3a-ffc5-4a0e-a341-6ff1f9c7a739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:52.918476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bcea3a-ffc5-4a0e-a341-6ff1f9c7a739 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:52.930714 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3898/7884] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:53.229979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edeae8e-6bd0-459e-8bff-01e1271af8a7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:53.232214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4edeae8e-6bd0-459e-8bff-01e1271af8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:53.247931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edeae8e-6bd0-459e-8bff-01e1271af8a7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:53.269485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:53.270405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:53.271373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:53.271665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] PATH_INFO: `/v3/projects/71494b232cff4b769d2f28f3188f6afc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:53.273686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] RBAC: Authorizing `identity:delete_project(project_id=71494b232cff4b769d2f28f3188f6afc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:53.275720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:53.332876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71494b232cff4b769d2f28f3188f6afc', 'request_id': 'req-f1752545-058f-4172-9983-11bcf789f79f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:53.340484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71494b232cff4b769d2f28f3188f6afc', 'request_id': 'req-f1752545-058f-4172-9983-11bcf789f79f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:53.347331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71494b232cff4b769d2f28f3188f6afc', 'request_id': 'req-f1752545-058f-4172-9983-11bcf789f79f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:53.358495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] The token cache is being invalidate because project 71494b232cff4b769d2f28f3188f6afc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:53.358790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:53.359452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:53.361001 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3987/7885] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:53 2022] DELETE /identity/v3/projects/71494b232cff4b769d2f28f3188f6afc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:53.365019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcea3a-ffc5-4a0e-a341-6ff1f9c7a739 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:53.375326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcea3a-ffc5-4a0e-a341-6ff1f9c7a739 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:53.428507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:53.440521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d315c-07f3-4cd8-8750-023de07a23cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:53.441321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d315c-07f3-4cd8-8750-023de07a23cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:53.441610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d315c-07f3-4cd8-8750-023de07a23cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:53.441890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d315c-07f3-4cd8-8750-023de07a23cb admin admin] PATH_INFO: `/v3/users/3a6fa32c5ae14add86e30fbf370e5291` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:53.443150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d315c-07f3-4cd8-8750-023de07a23cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a6fa32c5ae14add86e30fbf370e5291)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:53.445183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d315c-07f3-4cd8-8750-023de07a23cb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:53.497076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6c0d315c-07f3-4cd8-8750-023de07a23cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a6fa32c5ae14add86e30fbf370e5291'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:53.511552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6c0d315c-07f3-4cd8-8750-023de07a23cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a6fa32c5ae14add86e30fbf370e5291'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:53.512332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6c0d315c-07f3-4cd8-8750-023de07a23cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a6fa32c5ae14add86e30fbf370e5291'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:53.520779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6c0d315c-07f3-4cd8-8750-023de07a23cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a6fa32c5ae14add86e30fbf370e5291'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:53.529616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-6c0d315c-07f3-4cd8-8750-023de07a23cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a6fa32c5ae14add86e30fbf370e5291'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:53.535974 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3899/7886] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:53 2022] DELETE /identity/v3/users/3a6fa32c5ae14add86e30fbf370e5291 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:53.838877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:53.840305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:53.855739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1752545-058f-4172-9983-11bcf789f79f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:53.886193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:53.956307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:53.957056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:53.959025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:53.959792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce admin admin] PATH_INFO: `/v3/projects/240e5ce0a8ef40518b2c2ff59b64ca41` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:53.962929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=240e5ce0a8ef40518b2c2ff59b64ca41)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:53.964923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:54.032862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240e5ce0a8ef40518b2c2ff59b64ca41', 'request_id': 'req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:54.042280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240e5ce0a8ef40518b2c2ff59b64ca41', 'request_id': 'req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:54.049104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240e5ce0a8ef40518b2c2ff59b64ca41', 'request_id': 'req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:54.060360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce admin admin] The token cache is being invalidate because project 240e5ce0a8ef40518b2c2ff59b64ca41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:54.060734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:54.061773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8d5a61ae-019d-4d11-9cca-05ac80ca5bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:54.063412 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3988/7887] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:53 2022] DELETE /identity/v3/projects/240e5ce0a8ef40518b2c2ff59b64ca41 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:54.068391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bba454-73c2-405b-bb78-6c025a0f4b4e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.068691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bba454-73c2-405b-bb78-6c025a0f4b4e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.069003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bba454-73c2-405b-bb78-6c025a0f4b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.089888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-35bba454-73c2-405b-bb78-6c025a0f4b4e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:54.091152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-35bba454-73c2-405b-bb78-6c025a0f4b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:54.160853 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3900/7888] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:54.169797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b40314-a529-440a-8c55-44ba95ef37cc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.170097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b40314-a529-440a-8c55-44ba95ef37cc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.170379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b40314-a529-440a-8c55-44ba95ef37cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.190788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d2b40314-a529-440a-8c55-44ba95ef37cc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:54.192015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d2b40314-a529-440a-8c55-44ba95ef37cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:54.200597 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3989/7889] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:54.204853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bba454-73c2-405b-bb78-6c025a0f4b4e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:54.215658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bba454-73c2-405b-bb78-6c025a0f4b4e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:54.220465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4721d-3d6c-47d3-88c2-dccbbc2c19e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:54.221115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4721d-3d6c-47d3-88c2-dccbbc2c19e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.221403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4721d-3d6c-47d3-88c2-dccbbc2c19e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.221681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4721d-3d6c-47d3-88c2-dccbbc2c19e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.222336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4721d-3d6c-47d3-88c2-dccbbc2c19e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:54.222651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4721d-3d6c-47d3-88c2-dccbbc2c19e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:54.224270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4721d-3d6c-47d3-88c2-dccbbc2c19e1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:54.237616 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3901/7890] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:54.242022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b40314-a529-440a-8c55-44ba95ef37cc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:54.251792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b40314-a529-440a-8c55-44ba95ef37cc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:54.256493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f0a8d-8447-43fb-b9aa-f1d2d2af8a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:54.257130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f0a8d-8447-43fb-b9aa-f1d2d2af8a74 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.257916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f0a8d-8447-43fb-b9aa-f1d2d2af8a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.258191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f0a8d-8447-43fb-b9aa-f1d2d2af8a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.258868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f0a8d-8447-43fb-b9aa-f1d2d2af8a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:54.260821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f0a8d-8447-43fb-b9aa-f1d2d2af8a74 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:54.293596 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3990/7891] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:54 2022] POST /identity/v3/projects => generated 395 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:54.294949 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4721d-3d6c-47d3-88c2-dccbbc2c19e1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:54.305206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4721d-3d6c-47d3-88c2-dccbbc2c19e1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:54.364325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3077e18-4e96-47e0-a692-ddef2e940693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:54.364969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3077e18-4e96-47e0-a692-ddef2e940693 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.365258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3077e18-4e96-47e0-a692-ddef2e940693 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.365815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3077e18-4e96-47e0-a692-ddef2e940693 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.366160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3077e18-4e96-47e0-a692-ddef2e940693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:54.368123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3077e18-4e96-47e0-a692-ddef2e940693 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:54.376148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:54.393235 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3902/7892] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:54 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:54.397356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f0a8d-8447-43fb-b9aa-f1d2d2af8a74 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:54.409833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f0a8d-8447-43fb-b9aa-f1d2d2af8a74 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:54.414747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535eefbb-a973-4e22-a2f0-93f6d26e2e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:54.415357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535eefbb-a973-4e22-a2f0-93f6d26e2e52 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.415766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535eefbb-a973-4e22-a2f0-93f6d26e2e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.416063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535eefbb-a973-4e22-a2f0-93f6d26e2e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.416717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535eefbb-a973-4e22-a2f0-93f6d26e2e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:54.418388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535eefbb-a973-4e22-a2f0-93f6d26e2e52 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:54.431522 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3991/7893] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:17:54.432791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3077e18-4e96-47e0-a692-ddef2e940693 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:54.443289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3077e18-4e96-47e0-a692-ddef2e940693 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:54.448493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99151191-9611-4625-b26e-703088dc2cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:54.449264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99151191-9611-4625-b26e-703088dc2cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.449547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99151191-9611-4625-b26e-703088dc2cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.449823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99151191-9611-4625-b26e-703088dc2cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.450464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99151191-9611-4625-b26e-703088dc2cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:54.452056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99151191-9611-4625-b26e-703088dc2cc9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:54.463598 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3903/7894] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:54.467775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535eefbb-a973-4e22-a2f0-93f6d26e2e52 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:54.478396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535eefbb-a973-4e22-a2f0-93f6d26e2e52 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:54.483403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ab349-646c-4473-b5c9-cc1383530d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:54.484070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ab349-646c-4473-b5c9-cc1383530d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.484357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ab349-646c-4473-b5c9-cc1383530d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.484637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ab349-646c-4473-b5c9-cc1383530d1d admin admin] PATH_INFO: `/v3/projects/1343671579db42d889cd90f044bdc85f/users/80146a9207e24411b073a1ec1aee6ee2/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.500678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ab349-646c-4473-b5c9-cc1383530d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1343671579db42d889cd90f044bdc85f, user_id=80146a9207e24411b073a1ec1aee6ee2, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:54.503407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ab349-646c-4473-b5c9-cc1383530d1d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:54.518579 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3992/7895] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:54 2022] PUT /identity/v3/projects/1343671579db42d889cd90f044bdc85f/users/80146a9207e24411b073a1ec1aee6ee2/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:17:54.530347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2395dc2-bc0f-4b74-97c7-87d7c977509f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.533401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2395dc2-bc0f-4b74-97c7-87d7c977509f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.533644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2395dc2-bc0f-4b74-97c7-87d7c977509f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.563255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a2395dc2-bc0f-4b74-97c7-87d7c977509f None None] MFA Rules not processed for user `80146a9207e24411b073a1ec1aee6ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:54.564627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a2395dc2-bc0f-4b74-97c7-87d7c977509f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:54.636878 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3904/7896] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:54.641378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ab349-646c-4473-b5c9-cc1383530d1d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:54.691090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:54.709472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ab349-646c-4473-b5c9-cc1383530d1d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:54.714328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866f875-d8ae-478b-b0e6-781ec1ba6927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:54.714940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866f875-d8ae-478b-b0e6-781ec1ba6927 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.715228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866f875-d8ae-478b-b0e6-781ec1ba6927 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.715533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866f875-d8ae-478b-b0e6-781ec1ba6927 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.716169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866f875-d8ae-478b-b0e6-781ec1ba6927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:54.718154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866f875-d8ae-478b-b0e6-781ec1ba6927 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:54.746869 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3993/7897] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:54 2022] POST /identity/v3/projects => generated 397 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:54.750884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2395dc2-bc0f-4b74-97c7-87d7c977509f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:54.762645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2395dc2-bc0f-4b74-97c7-87d7c977509f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:54.823552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641b1e0-58f4-4149-9f0c-fd7ea4483c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:54.824074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641b1e0-58f4-4149-9f0c-fd7ea4483c86 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.824363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641b1e0-58f4-4149-9f0c-fd7ea4483c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.824644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641b1e0-58f4-4149-9f0c-fd7ea4483c86 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.825321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641b1e0-58f4-4149-9f0c-fd7ea4483c86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:54.827426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641b1e0-58f4-4149-9f0c-fd7ea4483c86 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:54.849765 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3905/7898] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:54 2022] POST /identity/v3/users => generated 305 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:54.853876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866f875-d8ae-478b-b0e6-781ec1ba6927 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:54.864834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866f875-d8ae-478b-b0e6-781ec1ba6927 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:54.876986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f68b2-b2a0-4313-9c4c-441108e6afd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:54.876986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f68b2-b2a0-4313-9c4c-441108e6afd8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.876986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f68b2-b2a0-4313-9c4c-441108e6afd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.876986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f68b2-b2a0-4313-9c4c-441108e6afd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.877603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f68b2-b2a0-4313-9c4c-441108e6afd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:54.877974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f68b2-b2a0-4313-9c4c-441108e6afd8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:54.892939 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3994/7899] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:17:54.894791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641b1e0-58f4-4149-9f0c-fd7ea4483c86 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:54.905387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641b1e0-58f4-4149-9f0c-fd7ea4483c86 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:54.912450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74484c5d-06d5-4484-8bda-f0a655e59675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:54.913180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74484c5d-06d5-4484-8bda-f0a655e59675 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:54.913474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74484c5d-06d5-4484-8bda-f0a655e59675 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:54.914405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74484c5d-06d5-4484-8bda-f0a655e59675 admin admin] PATH_INFO: `/v3/projects/8d0f085b66d84891b9fe00e76f113c4f/users/8805154cac844420aa2e47f0569ae2bf/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:54.930573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74484c5d-06d5-4484-8bda-f0a655e59675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0f085b66d84891b9fe00e76f113c4f, user_id=8805154cac844420aa2e47f0569ae2bf, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:54.933303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74484c5d-06d5-4484-8bda-f0a655e59675 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:54.947103 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3906/7900] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:54 2022] PUT /identity/v3/projects/8d0f085b66d84891b9fe00e76f113c4f/users/8805154cac844420aa2e47f0569ae2bf/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:54.951068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f68b2-b2a0-4313-9c4c-441108e6afd8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:54.961506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f68b2-b2a0-4313-9c4c-441108e6afd8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:55.029197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99568f9-b316-4b57-8f54-c1c5bcf409ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:55.029987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99568f9-b316-4b57-8f54-c1c5bcf409ba admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:55.030539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99568f9-b316-4b57-8f54-c1c5bcf409ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:55.030926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99568f9-b316-4b57-8f54-c1c5bcf409ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:55.031683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99568f9-b316-4b57-8f54-c1c5bcf409ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:55.033582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99568f9-b316-4b57-8f54-c1c5bcf409ba admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:55.049048 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3995/7901] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:55.050704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74484c5d-06d5-4484-8bda-f0a655e59675 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:55.062094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74484c5d-06d5-4484-8bda-f0a655e59675 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:55.066808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f0362-98f5-4840-9e01-f5cbee0cf517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:55.067493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f0362-98f5-4840-9e01-f5cbee0cf517 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:55.067784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f0362-98f5-4840-9e01-f5cbee0cf517 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:55.068650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f0362-98f5-4840-9e01-f5cbee0cf517 admin admin] PATH_INFO: `/v3/projects/8d0f085b66d84891b9fe00e76f113c4f/users/8805154cac844420aa2e47f0569ae2bf/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:55.070146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f0362-98f5-4840-9e01-f5cbee0cf517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0f085b66d84891b9fe00e76f113c4f, user_id=8805154cac844420aa2e47f0569ae2bf, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:55.073100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f0362-98f5-4840-9e01-f5cbee0cf517 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:55.084463 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3907/7902] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:55 2022] PUT /identity/v3/projects/8d0f085b66d84891b9fe00e76f113c4f/users/8805154cac844420aa2e47f0569ae2bf/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:55.096221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbe5cd-9027-48ec-97c3-b336c74731cc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:55.096568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbe5cd-9027-48ec-97c3-b336c74731cc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:55.097003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbe5cd-9027-48ec-97c3-b336c74731cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:55.133308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-91bbe5cd-9027-48ec-97c3-b336c74731cc None None] MFA Rules not processed for user `8805154cac844420aa2e47f0569ae2bf`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:55.134575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-91bbe5cd-9027-48ec-97c3-b336c74731cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:55.209153 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3996/7903] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:55.213929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f0362-98f5-4840-9e01-f5cbee0cf517 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:55.236973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:55.298290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f0362-98f5-4840-9e01-f5cbee0cf517 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:55.302761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80b7f4-4ef7-4d48-8e35-8e496a268c3a tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80146a9207e24411b073a1ec1aee6ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1343671579db42d889cd90f044bdc85f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:55.303389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80b7f4-4ef7-4d48-8e35-8e496a268c3a tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:55.303740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80b7f4-4ef7-4d48-8e35-8e496a268c3a tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:55.303938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80b7f4-4ef7-4d48-8e35-8e496a268c3a tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:55.304669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80b7f4-4ef7-4d48-8e35-8e496a268c3a tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-ed80b7f4-4ef7-4d48-8e35-8e496a268c3a tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] You are not authorized to perform the requested action: identity:create_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 162, in post Sep 03 21:17:55.317151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 03 21:17:55.318588 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 03 21:17:55.318588 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 03 21:17:55.318588 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 03 21:17:55.318588 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 03 21:17:55.318588 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 03 21:17:55.318588 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 03 21:17:55.318588 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Sep 03 21:17:55.318588 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.318588 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3908/7904] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:17:55 2022] POST /identity/v3/projects => generated 136 bytes in 105 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:17:55.324063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbe5cd-9027-48ec-97c3-b336c74731cc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:55.333836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbe5cd-9027-48ec-97c3-b336c74731cc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:55.339220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b397-23dc-4975-89b7-17f3722a0e41 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8805154cac844420aa2e47f0569ae2bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d0f085b66d84891b9fe00e76f113c4f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:55.339777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b397-23dc-4975-89b7-17f3722a0e41 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:55.340069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b397-23dc-4975-89b7-17f3722a0e41 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:55.340383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b397-23dc-4975-89b7-17f3722a0e41 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:55.341051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447b397-23dc-4975-89b7-17f3722a0e41 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:55.342947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447b397-23dc-4975-89b7-17f3722a0e41 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-1447b397-23dc-4975-89b7-17f3722a0e41 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Invalid input for field 'name': '' is too short Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: Failed validating 'minLength' in schema['properties']['name']: Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: On instance['name']: Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: Failed validating 'minLength' in schema['properties']['name']: Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: On instance['name']: Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: '' Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise error Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application On instance['name']: Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application '' Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:55.349650 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 03 21:17:55.356679 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 03 21:17:55.357948 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 03 21:17:55.357948 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 03 21:17:55.357948 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:55.357948 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 03 21:17:55.357948 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:17:55.357948 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:55.357948 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application On instance['name']: Sep 03 21:17:55.357948 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application '' Sep 03 21:17:55.357948 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:55.357948 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3997/7905] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:17:55 2022] POST /identity/v3/projects => generated 283 bytes in 28 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 03 21:17:55.358735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80b7f4-4ef7-4d48-8e35-8e496a268c3a tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:55.369784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80b7f4-4ef7-4d48-8e35-8e496a268c3a tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:55.374871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7513004-75d1-41a4-90be-e7ec0e627530 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8805154cac844420aa2e47f0569ae2bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d0f085b66d84891b9fe00e76f113c4f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:55.375609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7513004-75d1-41a4-90be-e7ec0e627530 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:55.376011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7513004-75d1-41a4-90be-e7ec0e627530 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:55.376353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7513004-75d1-41a4-90be-e7ec0e627530 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:55.377096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7513004-75d1-41a4-90be-e7ec0e627530 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:55.379027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7513004-75d1-41a4-90be-e7ec0e627530 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-e7513004-75d1-41a4-90be-e7ec0e627530 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: Failed validating 'maxLength' in schema['properties']['name']: Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: On instance['name']: Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: Failed validating 'maxLength' in schema['properties']['name']: Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: On instance['name']: Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise error Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 03 21:17:55.386651 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application On instance['name']: Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:55.387873 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application On instance['name']: Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.389079 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3909/7906] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:17:55 2022] POST /identity/v3/projects => generated 412 bytes in 32 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 03 21:17:55.393461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b397-23dc-4975-89b7-17f3722a0e41 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:55.413166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b397-23dc-4975-89b7-17f3722a0e41 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:55.418718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad334e3-f626-4c10-8f19-d58ba30619c3 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8805154cac844420aa2e47f0569ae2bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d0f085b66d84891b9fe00e76f113c4f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:55.419425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad334e3-f626-4c10-8f19-d58ba30619c3 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:55.419795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad334e3-f626-4c10-8f19-d58ba30619c3 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:55.420168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad334e3-f626-4c10-8f19-d58ba30619c3 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] PATH_INFO: `/v3/projects/46338ca36ce242328b2172b8ebfa201d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:55.427373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad334e3-f626-4c10-8f19-d58ba30619c3 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: Authorizing `identity:delete_project(project_id=46338ca36ce242328b2172b8ebfa201d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:55.429304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad334e3-f626-4c10-8f19-d58ba30619c3 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application [None req-bad334e3-f626-4c10-8f19-d58ba30619c3 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Could not find project: 46338ca36ce242328b2172b8ebfa201d.: keystone.exception.ProjectNotFound: Could not find project: 46338ca36ce242328b2172b8ebfa201d. Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Sep 03 21:17:55.439704 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Sep 03 21:17:55.440815 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:17:55.440815 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:17:55.440815 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Sep 03 21:17:55.440815 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Sep 03 21:17:55.440815 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 21:17:55.440815 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 21:17:55.440815 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 21:17:55.440815 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 21:17:55.440815 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 46338ca36ce242328b2172b8ebfa201d. Sep 03 21:17:55.440815 ncdv-master-1 devstack@keystone.service[99210]: ERROR keystone.server.flask.application Sep 03 21:17:55.441728 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3998/7907] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:55 2022] DELETE /identity/v3/projects/46338ca36ce242328b2172b8ebfa201d => generated 113 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:17:55.446682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7513004-75d1-41a4-90be-e7ec0e627530 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:55.456814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7513004-75d1-41a4-90be-e7ec0e627530 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:55.461612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0472823-69c7-423a-b13e-295982a92bd5 tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80146a9207e24411b073a1ec1aee6ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1343671579db42d889cd90f044bdc85f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:55.462287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0472823-69c7-423a-b13e-295982a92bd5 tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:55.462571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0472823-69c7-423a-b13e-295982a92bd5 tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:55.462847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0472823-69c7-423a-b13e-295982a92bd5 tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:55.463508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0472823-69c7-423a-b13e-295982a92bd5 tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] RBAC: Authorizing `identity:list_projects()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-f0472823-69c7-423a-b13e-295982a92bd5 tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] You are not authorized to perform the requested action: identity:list_projects.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 153, in get Sep 03 21:17:55.469402 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self._list_projects() Sep 03 21:17:55.470527 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 120, in _list_projects Sep 03 21:17:55.470527 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:list_projects', Sep 03 21:17:55.470527 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 03 21:17:55.470527 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 03 21:17:55.470527 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 03 21:17:55.470527 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 03 21:17:55.470527 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 03 21:17:55.470527 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 03 21:17:55.470527 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Sep 03 21:17:55.470527 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.470527 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3910/7908] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Sep 3 21:17:55 2022] GET /identity/v3/projects => generated 135 bytes in 24 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 03 21:17:55.475871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad334e3-f626-4c10-8f19-d58ba30619c3 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:55.486364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad334e3-f626-4c10-8f19-d58ba30619c3 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:55.492413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce941088-8cd6-4450-bdf6-79a13c07a0f6 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8805154cac844420aa2e47f0569ae2bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d0f085b66d84891b9fe00e76f113c4f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:55.493103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce941088-8cd6-4450-bdf6-79a13c07a0f6 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:55.493884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce941088-8cd6-4450-bdf6-79a13c07a0f6 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:55.493884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce941088-8cd6-4450-bdf6-79a13c07a0f6 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:55.494301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce941088-8cd6-4450-bdf6-79a13c07a0f6 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:55.496213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce941088-8cd6-4450-bdf6-79a13c07a0f6 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:55.529076 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 3999/7909] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:55 2022] POST /identity/v3/projects => generated 359 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:55.533367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0472823-69c7-423a-b13e-295982a92bd5 tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:55.545525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0472823-69c7-423a-b13e-295982a92bd5 tempest-ProjectsNegativeStaticTestJSON-569345000 tempest-ProjectsNegativeStaticTestJSON-569345000-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:55.613253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee7bca-9940-477b-a591-4092f666f697 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8805154cac844420aa2e47f0569ae2bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d0f085b66d84891b9fe00e76f113c4f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:55.613869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee7bca-9940-477b-a591-4092f666f697 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:55.614161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee7bca-9940-477b-a591-4092f666f697 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:55.614446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee7bca-9940-477b-a591-4092f666f697 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:55.615069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee7bca-9940-477b-a591-4092f666f697 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:55.617134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee7bca-9940-477b-a591-4092f666f697 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:55.655410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.sql.core [None req-85ee7bca-9940-477b-a591-4092f666f697 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-1920785293' for key 'project.ixu_project_name_domain_id'") Sep 03 21:17:55.655410 ncdv-master-1 devstack@keystone.service[99211]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 03 21:17:55.655410 ncdv-master-1 devstack@keystone.service[99211]: [parameters: {'id': 'de31b27745174eb093e270d5fb50a8a1', 'name': 'tempest-project-dup-1920785293', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 03 21:17:55.655410 ncdv-master-1 devstack@keystone.service[99211]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99211) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 03 21:17:55.662617 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-85ee7bca-9940-477b-a591-4092f666f697 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-1920785293, project id de31b27745174eb093e270d5fb50a8a1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-1920785293, project id de31b27745174eb093e270d5fb50a8a1. Sep 03 21:17:55.662617 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:55.662617 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 21:17:55.662617 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 21:17:55.662617 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 21:17:55.662617 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 21:17:55.662617 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 21:17:55.662617 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 21:17:55.662617 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 21:17:55.662617 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application conn.query(q) Sep 03 21:17:55.662617 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 21:17:55.662617 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application result.read() Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-tempest-project-dup-1920785293' for key 'project.ixu_project_name_domain_id'") Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 03 21:17:55.663456 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application next(self.gen) Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application yield resource Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application next(self.gen) Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.session.rollback() Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.force_reraise() Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise self.value Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 03 21:17:55.667887 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application session.commit() Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._prepare_impl() Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.session.flush() Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._flush(objects) Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application compat.raise_( Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application flush_context.execute() Sep 03 21:17:55.669115 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rec.execute(self) Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 03 21:17:55.670598 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application conn.query(q) Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application result.read() Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 21:17:55.671590 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-1920785293' for key 'project.ixu_project_name_domain_id'") Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [parameters: {'id': 'de31b27745174eb093e270d5fb50a8a1', 'name': 'tempest-project-dup-1920785293', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:55.672702 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-project-dup-1920785293 at domain ID default. Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:17:55.675889 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:17:55.681624 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:17:55.681624 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 03 21:17:55.681624 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 03 21:17:55.681624 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:17:55.681624 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:17:55.681624 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 03 21:17:55.681624 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 03 21:17:55.681624 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-1920785293, project id de31b27745174eb093e270d5fb50a8a1. Sep 03 21:17:55.681624 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:17:55.681624 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3911/7910] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Sep 3 21:17:55 2022] POST /identity/v3/projects => generated 285 bytes in 131 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 03 21:17:55.682885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce941088-8cd6-4450-bdf6-79a13c07a0f6 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:55.682885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:55.687044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce941088-8cd6-4450-bdf6-79a13c07a0f6 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:55.692784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8805154cac844420aa2e47f0569ae2bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d0f085b66d84891b9fe00e76f113c4f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:55.693565 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:55.693826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:55.694112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] PATH_INFO: `/v3/projects/ce04eb1cd1914c768639fdd4867c53a7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:55.695852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ce04eb1cd1914c768639fdd4867c53a7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:55.698038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:55.752990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce04eb1cd1914c768639fdd4867c53a7', 'request_id': 'req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:55.761658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce04eb1cd1914c768639fdd4867c53a7', 'request_id': 'req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:55.768452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce04eb1cd1914c768639fdd4867c53a7', 'request_id': 'req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:55.780255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] The token cache is being invalidate because project ce04eb1cd1914c768639fdd4867c53a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:55.780546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:55.781367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d5ca5b58-f8f3-4c73-a71a-1c1ea8db7a30 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:55.783168 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4000/7911] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:55 2022] DELETE /identity/v3/projects/ce04eb1cd1914c768639fdd4867c53a7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:55.788178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee7bca-9940-477b-a591-4092f666f697 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:55.790509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-85ee7bca-9940-477b-a591-4092f666f697 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:55.859199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee7bca-9940-477b-a591-4092f666f697 tempest-ProjectsNegativeStaticTestJSON-1333232590 tempest-ProjectsNegativeStaticTestJSON-1333232590-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:55.909229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:55.921573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:55.922672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:55.923110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:55.923529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] PATH_INFO: `/v3/users/80146a9207e24411b073a1ec1aee6ee2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:55.925366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80146a9207e24411b073a1ec1aee6ee2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:55.928808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:55.992944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80146a9207e24411b073a1ec1aee6ee2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.006072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80146a9207e24411b073a1ec1aee6ee2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.007171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80146a9207e24411b073a1ec1aee6ee2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.014919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80146a9207e24411b073a1ec1aee6ee2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.025607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80146a9207e24411b073a1ec1aee6ee2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.032660 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3912/7912] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:55 2022] DELETE /identity/v3/users/80146a9207e24411b073a1ec1aee6ee2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:56.037545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b5173-6948-4bbb-a361-075a0342ea1a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:56.037838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b5173-6948-4bbb-a361-075a0342ea1a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:56.038113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b5173-6948-4bbb-a361-075a0342ea1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:56.058807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-317b5173-6948-4bbb-a361-075a0342ea1a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:56.060288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-317b5173-6948-4bbb-a361-075a0342ea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:56.125807 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4001/7913] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:17:56.136643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:56.138463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:56.201568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208b9ad-48ea-42e7-ab2c-0e14ac649c68 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:56.205285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8eb842-097b-4058-946f-ebc79b842d3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:56.205953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8eb842-097b-4058-946f-ebc79b842d3f service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:56.206245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8eb842-097b-4058-946f-ebc79b842d3f service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:56.206520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8eb842-097b-4058-946f-ebc79b842d3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:56.215814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8eb842-097b-4058-946f-ebc79b842d3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:56.217297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8eb842-097b-4058-946f-ebc79b842d3f service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:56.227614 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3913/7914] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:56.526602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b5173-6948-4bbb-a361-075a0342ea1a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:56.528133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-317b5173-6948-4bbb-a361-075a0342ea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:56.541691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b5173-6948-4bbb-a361-075a0342ea1a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:56.546748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:56.547402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:56.547781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:56.548315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] PATH_INFO: `/v3/projects/1343671579db42d889cd90f044bdc85f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:56.549260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1343671579db42d889cd90f044bdc85f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:56.551202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:56.618881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1343671579db42d889cd90f044bdc85f', 'request_id': 'req-ee806003-b1f4-444b-b838-900284615863'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.626400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1343671579db42d889cd90f044bdc85f', 'request_id': 'req-ee806003-b1f4-444b-b838-900284615863'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.632853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1343671579db42d889cd90f044bdc85f', 'request_id': 'req-ee806003-b1f4-444b-b838-900284615863'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.643400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] The token cache is being invalidate because project 1343671579db42d889cd90f044bdc85f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:56.643696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.644360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.646037 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4002/7915] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:56 2022] DELETE /identity/v3/projects/1343671579db42d889cd90f044bdc85f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:56.649831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8eb842-097b-4058-946f-ebc79b842d3f service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:56.659768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8eb842-097b-4058-946f-ebc79b842d3f service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:56.721819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb6bd6-4729-4799-b1fd-2863f1bce8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:56.722461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb6bd6-4729-4799-b1fd-2863f1bce8f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:56.722747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb6bd6-4729-4799-b1fd-2863f1bce8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:56.723029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb6bd6-4729-4799-b1fd-2863f1bce8f7 admin admin] PATH_INFO: `/v3/users/8805154cac844420aa2e47f0569ae2bf` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:56.724163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb6bd6-4729-4799-b1fd-2863f1bce8f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8805154cac844420aa2e47f0569ae2bf)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:56.726648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb6bd6-4729-4799-b1fd-2863f1bce8f7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:56.750104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:56.779239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b9eb6bd6-4729-4799-b1fd-2863f1bce8f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8805154cac844420aa2e47f0569ae2bf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.804670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b9eb6bd6-4729-4799-b1fd-2863f1bce8f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8805154cac844420aa2e47f0569ae2bf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.805534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b9eb6bd6-4729-4799-b1fd-2863f1bce8f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8805154cac844420aa2e47f0569ae2bf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.813811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b9eb6bd6-4729-4799-b1fd-2863f1bce8f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8805154cac844420aa2e47f0569ae2bf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.823155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b9eb6bd6-4729-4799-b1fd-2863f1bce8f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8805154cac844420aa2e47f0569ae2bf'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:56.830362 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3914/7916] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:56 2022] DELETE /identity/v3/users/8805154cac844420aa2e47f0569ae2bf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:57.160761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:57.162711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:57.181335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee806003-b1f4-444b-b838-900284615863 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:57.186462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:57.280733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b2170-18a6-4204-8771-0cfffc62dc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:57.281465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2170-18a6-4204-8771-0cfffc62dc49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:57.281753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2170-18a6-4204-8771-0cfffc62dc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:57.282048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2170-18a6-4204-8771-0cfffc62dc49 admin admin] PATH_INFO: `/v3/projects/8d0f085b66d84891b9fe00e76f113c4f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:57.283983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b2170-18a6-4204-8771-0cfffc62dc49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d0f085b66d84891b9fe00e76f113c4f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:57.287428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b2170-18a6-4204-8771-0cfffc62dc49 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:57.347728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a0b2170-18a6-4204-8771-0cfffc62dc49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d0f085b66d84891b9fe00e76f113c4f', 'request_id': 'req-4a0b2170-18a6-4204-8771-0cfffc62dc49'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:57.358271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a0b2170-18a6-4204-8771-0cfffc62dc49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d0f085b66d84891b9fe00e76f113c4f', 'request_id': 'req-4a0b2170-18a6-4204-8771-0cfffc62dc49'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:57.367953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a0b2170-18a6-4204-8771-0cfffc62dc49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d0f085b66d84891b9fe00e76f113c4f', 'request_id': 'req-4a0b2170-18a6-4204-8771-0cfffc62dc49'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:57.382315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a0b2170-18a6-4204-8771-0cfffc62dc49 admin admin] The token cache is being invalidate because project 8d0f085b66d84891b9fe00e76f113c4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:57.382670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a0b2170-18a6-4204-8771-0cfffc62dc49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:57.383497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a0b2170-18a6-4204-8771-0cfffc62dc49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:57.385197 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4003/7917] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:57 2022] DELETE /identity/v3/projects/8d0f085b66d84891b9fe00e76f113c4f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:57.392561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78135ec6-26e8-41c0-b394-055d3346d179 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:57.392864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78135ec6-26e8-41c0-b394-055d3346d179 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:57.393201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78135ec6-26e8-41c0-b394-055d3346d179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:57.417834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-78135ec6-26e8-41c0-b394-055d3346d179 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:57.419155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-78135ec6-26e8-41c0-b394-055d3346d179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:57.491106 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3915/7918] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:57.500302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c9c88-877f-4223-9c0b-877ea72af060 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:57.500604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c9c88-877f-4223-9c0b-877ea72af060 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:57.500885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c9c88-877f-4223-9c0b-877ea72af060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:57.533051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-d80c9c88-877f-4223-9c0b-877ea72af060 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:57.535046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d80c9c88-877f-4223-9c0b-877ea72af060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:57.559034 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4004/7919] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:57.563189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78135ec6-26e8-41c0-b394-055d3346d179 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:57.574258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78135ec6-26e8-41c0-b394-055d3346d179 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:57.579584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02786e8-fc84-434c-aea7-40719038ed35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:57.579948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02786e8-fc84-434c-aea7-40719038ed35 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:57.580556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02786e8-fc84-434c-aea7-40719038ed35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:57.580990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02786e8-fc84-434c-aea7-40719038ed35 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:57.582045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02786e8-fc84-434c-aea7-40719038ed35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:57.582438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02786e8-fc84-434c-aea7-40719038ed35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:57.585825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02786e8-fc84-434c-aea7-40719038ed35 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:57.609533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c9c88-877f-4223-9c0b-877ea72af060 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:57.609894 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3916/7920] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:57.620748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c9c88-877f-4223-9c0b-877ea72af060 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:57.627175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4de2f1-1214-4aaa-ad7a-67456347d679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:57.627995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4de2f1-1214-4aaa-ad7a-67456347d679 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:57.628869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4de2f1-1214-4aaa-ad7a-67456347d679 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:57.629461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4de2f1-1214-4aaa-ad7a-67456347d679 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:57.630528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4de2f1-1214-4aaa-ad7a-67456347d679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:57.633824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4de2f1-1214-4aaa-ad7a-67456347d679 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:57.681101 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4005/7921] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:57 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:17:57.685906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02786e8-fc84-434c-aea7-40719038ed35 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:57.702913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02786e8-fc84-434c-aea7-40719038ed35 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:57.818465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d0d62-330a-47f3-8950-484ced618506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:57.819078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d0d62-330a-47f3-8950-484ced618506 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:57.819362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d0d62-330a-47f3-8950-484ced618506 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:57.819642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d0d62-330a-47f3-8950-484ced618506 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:57.820316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d0d62-330a-47f3-8950-484ced618506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:57.823822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d0d62-330a-47f3-8950-484ced618506 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:57.864729 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3917/7922] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:57 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:57.865964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4de2f1-1214-4aaa-ad7a-67456347d679 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:57.878335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4de2f1-1214-4aaa-ad7a-67456347d679 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:57.884614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c663c-afa5-4f50-a3ee-9a0f895339c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:57.885374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c663c-afa5-4f50-a3ee-9a0f895339c9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:57.885993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c663c-afa5-4f50-a3ee-9a0f895339c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:57.886351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c663c-afa5-4f50-a3ee-9a0f895339c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:57.887468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c663c-afa5-4f50-a3ee-9a0f895339c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:57.889687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c663c-afa5-4f50-a3ee-9a0f895339c9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:57.906695 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4006/7923] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:57.907953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d0d62-330a-47f3-8950-484ced618506 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:57.919363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d0d62-330a-47f3-8950-484ced618506 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:57.927052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163b764-4e09-41a0-b95f-2d8c9f5f5470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:57.928027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163b764-4e09-41a0-b95f-2d8c9f5f5470 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:57.928348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163b764-4e09-41a0-b95f-2d8c9f5f5470 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:57.928629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163b764-4e09-41a0-b95f-2d8c9f5f5470 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:57.929372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163b764-4e09-41a0-b95f-2d8c9f5f5470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:57.931174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163b764-4e09-41a0-b95f-2d8c9f5f5470 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:57.949017 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3918/7924] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:57.952430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c663c-afa5-4f50-a3ee-9a0f895339c9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:57.964479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c663c-afa5-4f50-a3ee-9a0f895339c9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:57.970508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1803c56-0556-452e-9d2a-3c38f1b5cceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:57.971229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1803c56-0556-452e-9d2a-3c38f1b5cceb admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:57.971462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1803c56-0556-452e-9d2a-3c38f1b5cceb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:57.971748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1803c56-0556-452e-9d2a-3c38f1b5cceb admin admin] PATH_INFO: `/v3/projects/ff752ebc372946f7ae5ab48e286118eb/users/51cb09cfc93d45d582a5fc67c2a43ac0/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:57.993379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1803c56-0556-452e-9d2a-3c38f1b5cceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff752ebc372946f7ae5ab48e286118eb, user_id=51cb09cfc93d45d582a5fc67c2a43ac0, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:57.996072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1803c56-0556-452e-9d2a-3c38f1b5cceb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:58.014096 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4007/7925] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:57 2022] PUT /identity/v3/projects/ff752ebc372946f7ae5ab48e286118eb/users/51cb09cfc93d45d582a5fc67c2a43ac0/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:58.025295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d49109-5feb-4333-8068-6beae4442fac None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:58.025582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d49109-5feb-4333-8068-6beae4442fac None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:58.025862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d49109-5feb-4333-8068-6beae4442fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:58.033165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:58.056760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-65d49109-5feb-4333-8068-6beae4442fac None None] MFA Rules not processed for user `51cb09cfc93d45d582a5fc67c2a43ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:58.058138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-65d49109-5feb-4333-8068-6beae4442fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:58.152505 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3919/7926] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:58.154132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1803c56-0556-452e-9d2a-3c38f1b5cceb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:58.170190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1803c56-0556-452e-9d2a-3c38f1b5cceb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:58.179460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad15b1-a00e-40ce-9a79-d43b142abf74 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51cb09cfc93d45d582a5fc67c2a43ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff752ebc372946f7ae5ab48e286118eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:58.180103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad15b1-a00e-40ce-9a79-d43b142abf74 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:58.180401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad15b1-a00e-40ce-9a79-d43b142abf74 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:58.180718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad15b1-a00e-40ce-9a79-d43b142abf74 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] PATH_INFO: `/v3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:58.186085 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4008/7927] 10.222.0.16 () {58 vars in 1104 bytes} [Sat Sep 3 21:17:58 2022] GET /identity/v3 => generated 255 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:58.187616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d49109-5feb-4333-8068-6beae4442fac None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:58.202744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d49109-5feb-4333-8068-6beae4442fac None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:58.207662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a99a3-0198-4f74-bf9c-6963cd21d976 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51cb09cfc93d45d582a5fc67c2a43ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff752ebc372946f7ae5ab48e286118eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:58.208617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a99a3-0198-4f74-bf9c-6963cd21d976 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:58.208617 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a99a3-0198-4f74-bf9c-6963cd21d976 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:58.208943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a99a3-0198-4f74-bf9c-6963cd21d976 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] PATH_INFO: `/v3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:58.210251 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3920/7928] 10.222.0.16 () {58 vars in 1104 bytes} [Sat Sep 3 21:17:58 2022] GET /identity/v3 => generated 255 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:58.215661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad15b1-a00e-40ce-9a79-d43b142abf74 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:58.226252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad15b1-a00e-40ce-9a79-d43b142abf74 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:58.231881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a091d-d855-46b6-8569-a195817a57d8 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51cb09cfc93d45d582a5fc67c2a43ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff752ebc372946f7ae5ab48e286118eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:58.232381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a091d-d855-46b6-8569-a195817a57d8 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:58.232686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a091d-d855-46b6-8569-a195817a57d8 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:58.233074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a091d-d855-46b6-8569-a195817a57d8 tempest-TestApiDiscovery-910848535 tempest-TestApiDiscovery-910848535-project] PATH_INFO: `/v3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:58.234595 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4009/7929] 10.222.0.16 () {58 vars in 1104 bytes} [Sat Sep 3 21:17:58 2022] GET /identity/v3 => generated 255 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:58.241544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316cbaa-2da8-4fb1-8130-564e1e776382 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:58.242541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316cbaa-2da8-4fb1-8130-564e1e776382 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:58.242820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316cbaa-2da8-4fb1-8130-564e1e776382 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:58.245686 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3921/7930] 10.222.0.16 () {56 vars in 895 bytes} [Sat Sep 3 21:17:58 2022] GET /identity/ => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 21:17:58.251935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bda25c-0a26-400a-ac36-c0dc3ef9a7cf None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:58.252122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bda25c-0a26-400a-ac36-c0dc3ef9a7cf None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:58.252548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bda25c-0a26-400a-ac36-c0dc3ef9a7cf None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:58.255620 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4010/7931] 10.222.0.16 () {56 vars in 895 bytes} [Sat Sep 3 21:17:58 2022] GET /identity/ => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 21:17:58.260890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316cbaa-2da8-4fb1-8130-564e1e776382 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:58.338183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316cbaa-2da8-4fb1-8130-564e1e776382 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:58.343734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82154719-c6d3-4ca9-abc5-f02e5c705ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:58.344722 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82154719-c6d3-4ca9-abc5-f02e5c705ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:58.345244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82154719-c6d3-4ca9-abc5-f02e5c705ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:58.345720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82154719-c6d3-4ca9-abc5-f02e5c705ca1 admin admin] PATH_INFO: `/v3/users/51cb09cfc93d45d582a5fc67c2a43ac0` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:58.347223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82154719-c6d3-4ca9-abc5-f02e5c705ca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51cb09cfc93d45d582a5fc67c2a43ac0)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:58.349591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82154719-c6d3-4ca9-abc5-f02e5c705ca1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:58.407650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-82154719-c6d3-4ca9-abc5-f02e5c705ca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51cb09cfc93d45d582a5fc67c2a43ac0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:58.420828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-82154719-c6d3-4ca9-abc5-f02e5c705ca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51cb09cfc93d45d582a5fc67c2a43ac0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:58.422168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-82154719-c6d3-4ca9-abc5-f02e5c705ca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51cb09cfc93d45d582a5fc67c2a43ac0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:58.432116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bda25c-0a26-400a-ac36-c0dc3ef9a7cf None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:58.434089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d6bda25c-0a26-400a-ac36-c0dc3ef9a7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:58.436266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-82154719-c6d3-4ca9-abc5-f02e5c705ca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51cb09cfc93d45d582a5fc67c2a43ac0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:58.446294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-82154719-c6d3-4ca9-abc5-f02e5c705ca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51cb09cfc93d45d582a5fc67c2a43ac0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:58.454060 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3922/7932] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:17:58 2022] DELETE /identity/v3/users/51cb09cfc93d45d582a5fc67c2a43ac0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:58.458771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04184393-6f46-47c1-8c4e-b664c7bd6ee5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:58.459277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04184393-6f46-47c1-8c4e-b664c7bd6ee5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:58.460286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04184393-6f46-47c1-8c4e-b664c7bd6ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:58.487728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:58.491020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:58.496990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-04184393-6f46-47c1-8c4e-b664c7bd6ee5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:58.498484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-04184393-6f46-47c1-8c4e-b664c7bd6ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:58.557898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bda25c-0a26-400a-ac36-c0dc3ef9a7cf None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:58.582020 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3923/7933] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:58.593040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04184393-6f46-47c1-8c4e-b664c7bd6ee5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:58.594771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-04184393-6f46-47c1-8c4e-b664c7bd6ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:58.636260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b51675-6336-48d8-8ef0-b912f6dde561 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:58.637873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b51675-6336-48d8-8ef0-b912f6dde561 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:58.638253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b51675-6336-48d8-8ef0-b912f6dde561 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:58.638444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b51675-6336-48d8-8ef0-b912f6dde561 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:58.642421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-55b51675-6336-48d8-8ef0-b912f6dde561 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:58.662607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b51675-6336-48d8-8ef0-b912f6dde561 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:58.664748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b51675-6336-48d8-8ef0-b912f6dde561 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:58.679275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04184393-6f46-47c1-8c4e-b664c7bd6ee5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:58.685821 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4011/7934] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:58.688003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b90988-c0ce-4790-9dc8-a1da113e4999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:58.688642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b90988-c0ce-4790-9dc8-a1da113e4999 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:58.688952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b90988-c0ce-4790-9dc8-a1da113e4999 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:58.689318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b90988-c0ce-4790-9dc8-a1da113e4999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:58.703887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b90988-c0ce-4790-9dc8-a1da113e4999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:58.708141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b90988-c0ce-4790-9dc8-a1da113e4999 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:58.727284 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3924/7935] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:17:59.180082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b51675-6336-48d8-8ef0-b912f6dde561 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:59.182120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-55b51675-6336-48d8-8ef0-b912f6dde561 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:59.231042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b51675-6336-48d8-8ef0-b912f6dde561 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:59.238229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:59.239647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:59.239946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:59.240237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5 admin admin] PATH_INFO: `/v3/projects/ff752ebc372946f7ae5ab48e286118eb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:59.242235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff752ebc372946f7ae5ab48e286118eb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:59.244664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:59.314031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff752ebc372946f7ae5ab48e286118eb', 'request_id': 'req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:59.324056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff752ebc372946f7ae5ab48e286118eb', 'request_id': 'req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:59.337767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff752ebc372946f7ae5ab48e286118eb', 'request_id': 'req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:59.351792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5 admin admin] The token cache is being invalidate because project ff752ebc372946f7ae5ab48e286118eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:17:59.352089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:59.352759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd1fc900-33d6-4455-8a4d-dc8f92c86de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:17:59.356940 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4012/7936] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:17:59 2022] DELETE /identity/v3/projects/ff752ebc372946f7ae5ab48e286118eb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:59.359899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da921b7-4211-4c5f-875d-484b17fcaa20 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:59.361745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da921b7-4211-4c5f-875d-484b17fcaa20 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:59.362102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da921b7-4211-4c5f-875d-484b17fcaa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:59.386435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1da921b7-4211-4c5f-875d-484b17fcaa20 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:59.388441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1da921b7-4211-4c5f-875d-484b17fcaa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:59.461293 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3925/7937] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:59.470629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c428f-6e36-4797-97e4-b9e643b48f3f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:59.473057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c428f-6e36-4797-97e4-b9e643b48f3f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:59.473500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c428f-6e36-4797-97e4-b9e643b48f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:59.501696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-018c428f-6e36-4797-97e4-b9e643b48f3f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:59.502951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-018c428f-6e36-4797-97e4-b9e643b48f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:17:59.511783 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4013/7938] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:17:59.516221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da921b7-4211-4c5f-875d-484b17fcaa20 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:59.527817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da921b7-4211-4c5f-875d-484b17fcaa20 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:59.532605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5158e3c-8a77-4397-9631-89c2b88ee44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:59.533287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5158e3c-8a77-4397-9631-89c2b88ee44e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:59.533876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5158e3c-8a77-4397-9631-89c2b88ee44e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:59.533876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5158e3c-8a77-4397-9631-89c2b88ee44e admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:59.534457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5158e3c-8a77-4397-9631-89c2b88ee44e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:17:59.534790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5158e3c-8a77-4397-9631-89c2b88ee44e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:59.536359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5158e3c-8a77-4397-9631-89c2b88ee44e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:59.553105 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3926/7939] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:17:59.558347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c428f-6e36-4797-97e4-b9e643b48f3f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:59.569938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c428f-6e36-4797-97e4-b9e643b48f3f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:59.575492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83cbc6-3ea6-4472-af0c-0791672de144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:59.577748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83cbc6-3ea6-4472-af0c-0791672de144 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:59.577748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83cbc6-3ea6-4472-af0c-0791672de144 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:59.577748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83cbc6-3ea6-4472-af0c-0791672de144 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:59.577748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83cbc6-3ea6-4472-af0c-0791672de144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:59.579799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83cbc6-3ea6-4472-af0c-0791672de144 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:59.613194 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4014/7940] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:17:59 2022] POST /identity/v3/projects => generated 359 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:59.618534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5158e3c-8a77-4397-9631-89c2b88ee44e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:59.630052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5158e3c-8a77-4397-9631-89c2b88ee44e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:59.675301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=100 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:59.701540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f845ad1-f9c5-475e-996b-5f91d7a171bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:59.702225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f845ad1-f9c5-475e-996b-5f91d7a171bb admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:59.702566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f845ad1-f9c5-475e-996b-5f91d7a171bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:59.702851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f845ad1-f9c5-475e-996b-5f91d7a171bb admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:59.703503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f845ad1-f9c5-475e-996b-5f91d7a171bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:59.705504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f845ad1-f9c5-475e-996b-5f91d7a171bb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:59.733206 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3927/7941] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:17:59 2022] POST /identity/v3/users => generated 280 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:17:59.737108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83cbc6-3ea6-4472-af0c-0791672de144 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:59.753244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83cbc6-3ea6-4472-af0c-0791672de144 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:59.758970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714e1cd-7589-4ed7-8026-5a27d7586cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:59.759904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714e1cd-7589-4ed7-8026-5a27d7586cdc admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:59.760428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714e1cd-7589-4ed7-8026-5a27d7586cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:59.760860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714e1cd-7589-4ed7-8026-5a27d7586cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:59.761754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714e1cd-7589-4ed7-8026-5a27d7586cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:59.763548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714e1cd-7589-4ed7-8026-5a27d7586cdc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:59.777935 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4015/7942] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:59.781728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f845ad1-f9c5-475e-996b-5f91d7a171bb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:59.797206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f845ad1-f9c5-475e-996b-5f91d7a171bb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:59.803957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39333d74-805e-45a1-a8f2-787c88cc5a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:59.804590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39333d74-805e-45a1-a8f2-787c88cc5a38 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:59.804875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39333d74-805e-45a1-a8f2-787c88cc5a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:59.805186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39333d74-805e-45a1-a8f2-787c88cc5a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:59.805836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39333d74-805e-45a1-a8f2-787c88cc5a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:59.807463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39333d74-805e-45a1-a8f2-787c88cc5a38 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:59.819818 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3928/7943] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:17:59.824078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714e1cd-7589-4ed7-8026-5a27d7586cdc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:17:59.829477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:17:59.836522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714e1cd-7589-4ed7-8026-5a27d7586cdc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:17:59.842964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19e116-be38-42a0-a342-f084874b8312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:17:59.843746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19e116-be38-42a0-a342-f084874b8312 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:59.844103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19e116-be38-42a0-a342-f084874b8312 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:59.844452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19e116-be38-42a0-a342-f084874b8312 admin admin] PATH_INFO: `/v3/projects/f7949c2aab6e41b98707ce29f9a4233f/users/7f40985b2e904392a0adacd7b12db3c4/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:59.863495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19e116-be38-42a0-a342-f084874b8312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7949c2aab6e41b98707ce29f9a4233f, user_id=7f40985b2e904392a0adacd7b12db3c4, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:17:59.866548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19e116-be38-42a0-a342-f084874b8312 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:17:59.884097 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4016/7944] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:17:59 2022] PUT /identity/v3/projects/f7949c2aab6e41b98707ce29f9a4233f/users/7f40985b2e904392a0adacd7b12db3c4/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:17:59.899267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846e26-cef7-444f-b9ba-4657ab7e3caa None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:17:59.899634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846e26-cef7-444f-b9ba-4657ab7e3caa None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:17:59.900014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846e26-cef7-444f-b9ba-4657ab7e3caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:17:59.932678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7f846e26-cef7-444f-b9ba-4657ab7e3caa None None] MFA Rules not processed for user `7f40985b2e904392a0adacd7b12db3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:17:59.933770 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7f846e26-cef7-444f-b9ba-4657ab7e3caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:00.020521 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3929/7945] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:17:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:00.026444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3906cbd-4041-4fdf-8d9f-6eee8dc702a0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.026761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3906cbd-4041-4fdf-8d9f-6eee8dc702a0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.027045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3906cbd-4041-4fdf-8d9f-6eee8dc702a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.048979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f3906cbd-4041-4fdf-8d9f-6eee8dc702a0 None None] MFA Rules not processed for user `7f40985b2e904392a0adacd7b12db3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:00.050517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f3906cbd-4041-4fdf-8d9f-6eee8dc702a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:00.059587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caeded7-dd03-4d3e-9546-6f648311273f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.060248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caeded7-dd03-4d3e-9546-6f648311273f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.060746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caeded7-dd03-4d3e-9546-6f648311273f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.064550 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4017/7946] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:00 2022] POST /identity/v3/auth/tokens => generated 345 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:18:00.090877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6caeded7-dd03-4d3e-9546-6f648311273f None None] MFA Rules not processed for user `7f40985b2e904392a0adacd7b12db3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:00.092718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6caeded7-dd03-4d3e-9546-6f648311273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:00.100778 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3930/7947] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:00 2022] POST /identity/v3/auth/tokens => generated 345 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:18:00.102250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3906cbd-4041-4fdf-8d9f-6eee8dc702a0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:00.130632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3906cbd-4041-4fdf-8d9f-6eee8dc702a0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:00.141439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4f494-cebe-4b26-8695-4f0eadc9fbcf tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f40985b2e904392a0adacd7b12db3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7949c2aab6e41b98707ce29f9a4233f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:00.142399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4f494-cebe-4b26-8695-4f0eadc9fbcf tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] REQUEST_METHOD: `HEAD` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.142920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4f494-cebe-4b26-8695-4f0eadc9fbcf tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.143361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4f494-cebe-4b26-8695-4f0eadc9fbcf tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.152535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4f494-cebe-4b26-8695-4f0eadc9fbcf tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: Authorizing `identity:check_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:00.154771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4f494-cebe-4b26-8695-4f0eadc9fbcf tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:00.163893 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4018/7948] 10.222.0.16 () {56 vars in 1264 bytes} [Sat Sep 3 21:18:00 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 358 bytes (0 switches on core 0) Sep 03 21:18:00.168218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caeded7-dd03-4d3e-9546-6f648311273f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:00.180977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caeded7-dd03-4d3e-9546-6f648311273f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:00.186405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f946038-e10e-4899-a8e2-6fd78770fa49 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f40985b2e904392a0adacd7b12db3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7949c2aab6e41b98707ce29f9a4233f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:00.187320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f946038-e10e-4899-a8e2-6fd78770fa49 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.187828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f946038-e10e-4899-a8e2-6fd78770fa49 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.188327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f946038-e10e-4899-a8e2-6fd78770fa49 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.197475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f946038-e10e-4899-a8e2-6fd78770fa49 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:00.199935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f946038-e10e-4899-a8e2-6fd78770fa49 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:00.208317 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3931/7949] 10.222.0.16 () {56 vars in 1263 bytes} [Sat Sep 3 21:18:00 2022] GET /identity/v3/auth/tokens => generated 345 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 21:18:00.213199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4f494-cebe-4b26-8695-4f0eadc9fbcf tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:00.235945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4f494-cebe-4b26-8695-4f0eadc9fbcf tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:00.245830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7db55-00e1-4958-84a0-e6c1f1c87661 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f40985b2e904392a0adacd7b12db3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7949c2aab6e41b98707ce29f9a4233f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:00.246815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7db55-00e1-4958-84a0-e6c1f1c87661 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.247110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7db55-00e1-4958-84a0-e6c1f1c87661 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.247399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7db55-00e1-4958-84a0-e6c1f1c87661 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.256357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7db55-00e1-4958-84a0-e6c1f1c87661 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:00.275034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7db55-00e1-4958-84a0-e6c1f1c87661 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:00.297488 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4019/7950] 10.222.0.16 () {56 vars in 1266 bytes} [Sat Sep 3 21:18:00 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:00.302203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f946038-e10e-4899-a8e2-6fd78770fa49 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:00.316384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f946038-e10e-4899-a8e2-6fd78770fa49 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:00.325605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb278b44-0858-4171-98e7-897f3ab4b616 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f40985b2e904392a0adacd7b12db3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7949c2aab6e41b98707ce29f9a4233f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:00.327224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb278b44-0858-4171-98e7-897f3ab4b616 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] REQUEST_METHOD: `HEAD` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.327785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb278b44-0858-4171-98e7-897f3ab4b616 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.328499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb278b44-0858-4171-98e7-897f3ab4b616 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.331348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fb278b44-0858-4171-98e7-897f3ab4b616 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-fb278b44-0858-4171-98e7-897f3ab4b616 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 283, in get Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:check_token') Sep 03 21:18:00.355151 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 03 21:18:00.356498 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 03 21:18:00.357566 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:18:00.357566 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:18:00.357566 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 03 21:18:00.357566 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 03 21:18:00.357566 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 03 21:18:00.357566 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:18:00.360889 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3932/7951] 10.222.0.16 () {56 vars in 1264 bytes} [Sat Sep 3 21:18:00 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 59 msecs (HTTP/1.1 404) 5 headers in 183 bytes (0 switches on core 0) Sep 03 21:18:00.366372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e4246-158f-4c17-b30d-27b51473abbd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.366709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e4246-158f-4c17-b30d-27b51473abbd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.367602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e4246-158f-4c17-b30d-27b51473abbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.395253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7f8e4246-158f-4c17-b30d-27b51473abbd None None] MFA Rules not processed for user `7f40985b2e904392a0adacd7b12db3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:00.397319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7f8e4246-158f-4c17-b30d-27b51473abbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:00.411299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb278b44-0858-4171-98e7-897f3ab4b616 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:00.435856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb278b44-0858-4171-98e7-897f3ab4b616 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:00.436534 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4020/7952] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:00 2022] POST /identity/v3/auth/tokens => generated 345 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 21:18:00.443938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0969776-8bf4-483f-984c-0399688ea340 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f40985b2e904392a0adacd7b12db3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7949c2aab6e41b98707ce29f9a4233f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:00.445503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0969776-8bf4-483f-984c-0399688ea340 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.446015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0969776-8bf4-483f-984c-0399688ea340 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.446542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0969776-8bf4-483f-984c-0399688ea340 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.455892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0969776-8bf4-483f-984c-0399688ea340 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:00.459179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0969776-8bf4-483f-984c-0399688ea340 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:00.474198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e4246-158f-4c17-b30d-27b51473abbd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:00.485226 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3933/7953] 10.222.0.16 () {56 vars in 1263 bytes} [Sat Sep 3 21:18:00 2022] GET /identity/v3/auth/tokens => generated 345 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 21:18:00.490853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e4246-158f-4c17-b30d-27b51473abbd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:00.497263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793874bf-2bbc-4307-b305-690fdbdad3f9 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f40985b2e904392a0adacd7b12db3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7949c2aab6e41b98707ce29f9a4233f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:00.498187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793874bf-2bbc-4307-b305-690fdbdad3f9 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.498693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793874bf-2bbc-4307-b305-690fdbdad3f9 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.499219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793874bf-2bbc-4307-b305-690fdbdad3f9 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.507959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793874bf-2bbc-4307-b305-690fdbdad3f9 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:00.510695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793874bf-2bbc-4307-b305-690fdbdad3f9 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:00.533319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0969776-8bf4-483f-984c-0399688ea340 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:00.539101 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4021/7954] 10.222.0.16 () {56 vars in 1266 bytes} [Sat Sep 3 21:18:00 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:00.547030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0969776-8bf4-483f-984c-0399688ea340 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:00.552984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ddeced-71eb-45e4-917b-f962a8822d69 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f40985b2e904392a0adacd7b12db3c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7949c2aab6e41b98707ce29f9a4233f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:00.553972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddeced-71eb-45e4-917b-f962a8822d69 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.555216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddeced-71eb-45e4-917b-f962a8822d69 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.555657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddeced-71eb-45e4-917b-f962a8822d69 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.558041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-02ddeced-71eb-45e4-917b-f962a8822d69 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application [None req-02ddeced-71eb-45e4-917b-f962a8822d69 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 03 21:18:00.572160 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 03 21:18:00.573416 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 03 21:18:00.574620 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 21:18:00.574620 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 21:18:00.574620 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 03 21:18:00.574620 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 03 21:18:00.574620 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 03 21:18:00.574620 ncdv-master-1 devstack@keystone.service[99211]: ERROR keystone.server.flask.application Sep 03 21:18:00.577461 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3934/7955] 10.222.0.16 () {56 vars in 1263 bytes} [Sat Sep 3 21:18:00 2022] GET /identity/v3/auth/tokens => generated 80 bytes in 45 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 21:18:00.578905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793874bf-2bbc-4307-b305-690fdbdad3f9 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:00.670035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793874bf-2bbc-4307-b305-690fdbdad3f9 tempest-TokensV3Test-149144692 tempest-TokensV3Test-149144692-project] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:00.674959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:00.675608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.675903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.676187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] PATH_INFO: `/v3/users/7f40985b2e904392a0adacd7b12db3c4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.677546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f40985b2e904392a0adacd7b12db3c4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:00.680382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:00.735923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f40985b2e904392a0adacd7b12db3c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:00.750225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f40985b2e904392a0adacd7b12db3c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:00.759446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f40985b2e904392a0adacd7b12db3c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:00.760201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f40985b2e904392a0adacd7b12db3c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:00.761153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f40985b2e904392a0adacd7b12db3c4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:00.763686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:00.779622 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4022/7956] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:00 2022] DELETE /identity/v3/users/7f40985b2e904392a0adacd7b12db3c4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:00.783908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84c2d0-1d27-4df3-a5e3-4b402081ba17 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.784439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84c2d0-1d27-4df3-a5e3-4b402081ba17 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.784555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84c2d0-1d27-4df3-a5e3-4b402081ba17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.806007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bc84c2d0-1d27-4df3-a5e3-4b402081ba17 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:00.807247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bc84c2d0-1d27-4df3-a5e3-4b402081ba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:00.863098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:00.889074 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3935/7957] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:00.899795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:00.901516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:00.982081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc4c3a-4095-4bb7-bd52-eb70a0de6277 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:00.986487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79a00d-27fe-4b10-aaee-d9464af36c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:00.987334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79a00d-27fe-4b10-aaee-d9464af36c2b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:00.987625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79a00d-27fe-4b10-aaee-d9464af36c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:00.987910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79a00d-27fe-4b10-aaee-d9464af36c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:00.999315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79a00d-27fe-4b10-aaee-d9464af36c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:01.001180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79a00d-27fe-4b10-aaee-d9464af36c2b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:01.012719 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4023/7958] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:01.925824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84c2d0-1d27-4df3-a5e3-4b402081ba17 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:01.927451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bc84c2d0-1d27-4df3-a5e3-4b402081ba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:01.941870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84c2d0-1d27-4df3-a5e3-4b402081ba17 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:01.946989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6844e-3475-4782-8ead-d855b47f263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:01.947838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6844e-3475-4782-8ead-d855b47f263f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:01.948487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6844e-3475-4782-8ead-d855b47f263f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:01.949121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6844e-3475-4782-8ead-d855b47f263f admin admin] PATH_INFO: `/v3/projects/f7949c2aab6e41b98707ce29f9a4233f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:01.950546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6844e-3475-4782-8ead-d855b47f263f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7949c2aab6e41b98707ce29f9a4233f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:01.953235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6844e-3475-4782-8ead-d855b47f263f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:02.006969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2ef6844e-3475-4782-8ead-d855b47f263f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7949c2aab6e41b98707ce29f9a4233f', 'request_id': 'req-2ef6844e-3475-4782-8ead-d855b47f263f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:02.017918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2ef6844e-3475-4782-8ead-d855b47f263f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7949c2aab6e41b98707ce29f9a4233f', 'request_id': 'req-2ef6844e-3475-4782-8ead-d855b47f263f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:02.026005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2ef6844e-3475-4782-8ead-d855b47f263f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7949c2aab6e41b98707ce29f9a4233f', 'request_id': 'req-2ef6844e-3475-4782-8ead-d855b47f263f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:02.037737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2ef6844e-3475-4782-8ead-d855b47f263f admin admin] The token cache is being invalidate because project f7949c2aab6e41b98707ce29f9a4233f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:18:02.038089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2ef6844e-3475-4782-8ead-d855b47f263f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:02.039487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-2ef6844e-3475-4782-8ead-d855b47f263f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:02.041294 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3936/7959] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:18:01 2022] DELETE /identity/v3/projects/f7949c2aab6e41b98707ce29f9a4233f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:02.047374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213ef5d-9dad-48e6-9db7-5d67b8e046e2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:02.047829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213ef5d-9dad-48e6-9db7-5d67b8e046e2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:02.048051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213ef5d-9dad-48e6-9db7-5d67b8e046e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:02.075521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9213ef5d-9dad-48e6-9db7-5d67b8e046e2 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:02.076888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9213ef5d-9dad-48e6-9db7-5d67b8e046e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:02.159398 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4024/7960] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:02.167599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76240031-4949-4635-af9b-cdf1b8dfa1ee None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:02.167926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76240031-4949-4635-af9b-cdf1b8dfa1ee None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:02.168273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76240031-4949-4635-af9b-cdf1b8dfa1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:02.194025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-76240031-4949-4635-af9b-cdf1b8dfa1ee None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:02.195466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-76240031-4949-4635-af9b-cdf1b8dfa1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:02.204713 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3937/7961] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:18:02.208796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213ef5d-9dad-48e6-9db7-5d67b8e046e2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:02.220344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213ef5d-9dad-48e6-9db7-5d67b8e046e2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:02.225429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76fe4c-b4d7-4d91-8584-e9a8e5b387d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:02.226747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76fe4c-b4d7-4d91-8584-e9a8e5b387d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:02.227041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76fe4c-b4d7-4d91-8584-e9a8e5b387d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:02.227322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76fe4c-b4d7-4d91-8584-e9a8e5b387d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:02.228008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76fe4c-b4d7-4d91-8584-e9a8e5b387d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:18:02.228318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76fe4c-b4d7-4d91-8584-e9a8e5b387d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:02.230003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76fe4c-b4d7-4d91-8584-e9a8e5b387d5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:02.243180 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4025/7962] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:18:02.247345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76240031-4949-4635-af9b-cdf1b8dfa1ee None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:02.258731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76240031-4949-4635-af9b-cdf1b8dfa1ee None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:02.263855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27029734-e99b-4b30-b383-4df130bceecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:02.264473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27029734-e99b-4b30-b383-4df130bceecc admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:02.264762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27029734-e99b-4b30-b383-4df130bceecc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:02.265079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27029734-e99b-4b30-b383-4df130bceecc admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:02.266158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27029734-e99b-4b30-b383-4df130bceecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:02.267804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27029734-e99b-4b30-b383-4df130bceecc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:02.300280 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3938/7963] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:02 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:02.304847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76fe4c-b4d7-4d91-8584-e9a8e5b387d5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:02.318742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76fe4c-b4d7-4d91-8584-e9a8e5b387d5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:02.398561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502eddb-f650-43af-89ff-b593ce1caf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:02.399187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502eddb-f650-43af-89ff-b593ce1caf65 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:02.399508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502eddb-f650-43af-89ff-b593ce1caf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:02.399788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502eddb-f650-43af-89ff-b593ce1caf65 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:02.400542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502eddb-f650-43af-89ff-b593ce1caf65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:02.402434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502eddb-f650-43af-89ff-b593ce1caf65 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:02.430374 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4026/7964] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:02 2022] POST /identity/v3/users => generated 303 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:02.434461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27029734-e99b-4b30-b383-4df130bceecc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:02.446975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27029734-e99b-4b30-b383-4df130bceecc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:02.451978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3edee0-137f-4fe3-b8c0-de901a5c154b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:02.452597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3edee0-137f-4fe3-b8c0-de901a5c154b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:02.452881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3edee0-137f-4fe3-b8c0-de901a5c154b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:02.453195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3edee0-137f-4fe3-b8c0-de901a5c154b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:02.453840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3edee0-137f-4fe3-b8c0-de901a5c154b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:02.455448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3edee0-137f-4fe3-b8c0-de901a5c154b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:02.470466 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3939/7965] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:02.471669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502eddb-f650-43af-89ff-b593ce1caf65 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:02.482754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502eddb-f650-43af-89ff-b593ce1caf65 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:02.487477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfb59d-7d6f-413b-aec9-8fb7444a872a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:02.488145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfb59d-7d6f-413b-aec9-8fb7444a872a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:02.488437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfb59d-7d6f-413b-aec9-8fb7444a872a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:02.488716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfb59d-7d6f-413b-aec9-8fb7444a872a admin admin] PATH_INFO: `/v3/projects/0e2f56b56f274e08a5a4f7d34fe0a491/users/b49c8d00906d4ce0897b679cf09e4d9d/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:02.500789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:02.509419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfb59d-7d6f-413b-aec9-8fb7444a872a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2f56b56f274e08a5a4f7d34fe0a491, user_id=b49c8d00906d4ce0897b679cf09e4d9d, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:02.512682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfb59d-7d6f-413b-aec9-8fb7444a872a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:02.527095 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4027/7966] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:02 2022] PUT /identity/v3/projects/0e2f56b56f274e08a5a4f7d34fe0a491/users/b49c8d00906d4ce0897b679cf09e4d9d/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:02.530889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3edee0-137f-4fe3-b8c0-de901a5c154b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:02.542062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3edee0-137f-4fe3-b8c0-de901a5c154b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:02.610577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114f04d-00dc-4397-8d0e-84375e6daa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:02.611205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114f04d-00dc-4397-8d0e-84375e6daa2c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:02.611492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114f04d-00dc-4397-8d0e-84375e6daa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:02.611771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114f04d-00dc-4397-8d0e-84375e6daa2c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:02.612421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114f04d-00dc-4397-8d0e-84375e6daa2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:02.614046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114f04d-00dc-4397-8d0e-84375e6daa2c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:02.627447 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3940/7967] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:18:02.629518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfb59d-7d6f-413b-aec9-8fb7444a872a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:02.640572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfb59d-7d6f-413b-aec9-8fb7444a872a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:02.645427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778223e-8108-4578-8d4f-c34104dcfb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:02.646145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778223e-8108-4578-8d4f-c34104dcfb0e admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:02.646433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778223e-8108-4578-8d4f-c34104dcfb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:02.646718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778223e-8108-4578-8d4f-c34104dcfb0e admin admin] PATH_INFO: `/v3/projects/0e2f56b56f274e08a5a4f7d34fe0a491/users/b49c8d00906d4ce0897b679cf09e4d9d/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:02.649349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778223e-8108-4578-8d4f-c34104dcfb0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2f56b56f274e08a5a4f7d34fe0a491, user_id=b49c8d00906d4ce0897b679cf09e4d9d, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:02.652011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778223e-8108-4578-8d4f-c34104dcfb0e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:02.666064 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4028/7968] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:02 2022] PUT /identity/v3/projects/0e2f56b56f274e08a5a4f7d34fe0a491/users/b49c8d00906d4ce0897b679cf09e4d9d/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:02.677425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570ef9e-2057-40d9-ad0a-c4d253dfc872 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:02.677654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570ef9e-2057-40d9-ad0a-c4d253dfc872 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:02.677942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570ef9e-2057-40d9-ad0a-c4d253dfc872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:02.714190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7570ef9e-2057-40d9-ad0a-c4d253dfc872 None None] MFA Rules not processed for user `b49c8d00906d4ce0897b679cf09e4d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:02.715525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7570ef9e-2057-40d9-ad0a-c4d253dfc872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:02.742990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:02.795186 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3941/7969] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:02.800252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778223e-8108-4578-8d4f-c34104dcfb0e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:02.874290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778223e-8108-4578-8d4f-c34104dcfb0e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:02.879073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb9ba8-e78f-433d-88e3-7980fc5fd067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:02.879681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb9ba8-e78f-433d-88e3-7980fc5fd067 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:02.879966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb9ba8-e78f-433d-88e3-7980fc5fd067 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:02.880242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb9ba8-e78f-433d-88e3-7980fc5fd067 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:02.881094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb9ba8-e78f-433d-88e3-7980fc5fd067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:02.883637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb9ba8-e78f-433d-88e3-7980fc5fd067 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:02.919124 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4029/7970] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:02 2022] POST /identity/v3/projects => generated 393 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:02.920659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570ef9e-2057-40d9-ad0a-c4d253dfc872 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:02.935238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570ef9e-2057-40d9-ad0a-c4d253dfc872 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:03.041878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89530d1b-1011-49f3-ac26-627cf0d90ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:03.042619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89530d1b-1011-49f3-ac26-627cf0d90ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:03.042916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89530d1b-1011-49f3-ac26-627cf0d90ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:03.043350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89530d1b-1011-49f3-ac26-627cf0d90ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:03.044028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89530d1b-1011-49f3-ac26-627cf0d90ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:03.046945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89530d1b-1011-49f3-ac26-627cf0d90ea8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:03.082200 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3942/7971] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:02 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:03.086817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb9ba8-e78f-433d-88e3-7980fc5fd067 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:03.124499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb9ba8-e78f-433d-88e3-7980fc5fd067 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:03.140245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52783b3e-df4f-4f8c-8ae9-a8cb83e4f84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:03.141268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52783b3e-df4f-4f8c-8ae9-a8cb83e4f84b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:03.141791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52783b3e-df4f-4f8c-8ae9-a8cb83e4f84b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:03.142245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52783b3e-df4f-4f8c-8ae9-a8cb83e4f84b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:03.144804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52783b3e-df4f-4f8c-8ae9-a8cb83e4f84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:03.147395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52783b3e-df4f-4f8c-8ae9-a8cb83e4f84b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:03.162654 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4030/7972] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:03.168025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89530d1b-1011-49f3-ac26-627cf0d90ea8 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:03.184274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89530d1b-1011-49f3-ac26-627cf0d90ea8 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:03.203426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b634d-feb9-4e66-b3fe-4db6db696d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:03.204473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b634d-feb9-4e66-b3fe-4db6db696d95 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:03.204968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b634d-feb9-4e66-b3fe-4db6db696d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:03.206172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b634d-feb9-4e66-b3fe-4db6db696d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:03.207511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b634d-feb9-4e66-b3fe-4db6db696d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:03.209506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b634d-feb9-4e66-b3fe-4db6db696d95 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:03.228141 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3943/7973] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:03.231973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52783b3e-df4f-4f8c-8ae9-a8cb83e4f84b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:03.244521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52783b3e-df4f-4f8c-8ae9-a8cb83e4f84b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:03.249661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b9cbf-d88f-4aec-845e-f5d817afacae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:03.250457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9cbf-d88f-4aec-845e-f5d817afacae admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:03.250819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9cbf-d88f-4aec-845e-f5d817afacae admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:03.251178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9cbf-d88f-4aec-845e-f5d817afacae admin admin] PATH_INFO: `/v3/projects/3dabab049da940a681df79d692dcbcf9/users/12e36e431cd94e7a908890b5b771ae15/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:03.271539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b9cbf-d88f-4aec-845e-f5d817afacae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dabab049da940a681df79d692dcbcf9, user_id=12e36e431cd94e7a908890b5b771ae15, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:03.274464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b9cbf-d88f-4aec-845e-f5d817afacae admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:03.292338 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4031/7974] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:03 2022] PUT /identity/v3/projects/3dabab049da940a681df79d692dcbcf9/users/12e36e431cd94e7a908890b5b771ae15/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:03.303402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b12bd-585d-4419-ad53-55cb6f1fff18 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:03.303813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b12bd-585d-4419-ad53-55cb6f1fff18 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:03.304817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b12bd-585d-4419-ad53-55cb6f1fff18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:03.335632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2d0b12bd-585d-4419-ad53-55cb6f1fff18 None None] MFA Rules not processed for user `12e36e431cd94e7a908890b5b771ae15`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:03.336890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2d0b12bd-585d-4419-ad53-55cb6f1fff18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:03.415857 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3944/7975] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:03.432024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b9cbf-d88f-4aec-845e-f5d817afacae admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:03.434118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cd1b9cbf-d88f-4aec-845e-f5d817afacae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:03.509607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b9cbf-d88f-4aec-845e-f5d817afacae admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:03.514321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c512c6-e6ff-4ff2-8df1-8409678c677e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:03.514892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c512c6-e6ff-4ff2-8df1-8409678c677e service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:03.515224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c512c6-e6ff-4ff2-8df1-8409678c677e service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:03.515449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c512c6-e6ff-4ff2-8df1-8409678c677e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:03.568879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:03.614084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c512c6-e6ff-4ff2-8df1-8409678c677e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:03.615590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c512c6-e6ff-4ff2-8df1-8409678c677e service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:03.626678 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4032/7976] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:03 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:04.808821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b12bd-585d-4419-ad53-55cb6f1fff18 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:04.859476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:04.907197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b12bd-585d-4419-ad53-55cb6f1fff18 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:04.912610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:04.913625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:04.913950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:04.914205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] PATH_INFO: `/v3/users/b49c8d00906d4ce0897b679cf09e4d9d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:04.915419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b49c8d00906d4ce0897b679cf09e4d9d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:04.917437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:04.984071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b49c8d00906d4ce0897b679cf09e4d9d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.015199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b49c8d00906d4ce0897b679cf09e4d9d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.016015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b49c8d00906d4ce0897b679cf09e4d9d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.026373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b49c8d00906d4ce0897b679cf09e4d9d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.042974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b49c8d00906d4ce0897b679cf09e4d9d'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.065919 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3945/7977] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:04 2022] DELETE /identity/v3/users/b49c8d00906d4ce0897b679cf09e4d9d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:05.071015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e84a14-5441-4e5e-b2e9-f02bf3dfda11 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:05.071516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e84a14-5441-4e5e-b2e9-f02bf3dfda11 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:05.071659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e84a14-5441-4e5e-b2e9-f02bf3dfda11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:05.095828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e3e84a14-5441-4e5e-b2e9-f02bf3dfda11 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:05.097364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e3e84a14-5441-4e5e-b2e9-f02bf3dfda11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:05.199802 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4033/7978] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:05.212726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:05.227876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:05.316871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69b0b5-c30d-4e60-b21e-2ec0e235a348 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:05.321340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173813a-28f6-46d4-81c5-3729ff04746c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:05.322835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173813a-28f6-46d4-81c5-3729ff04746c service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:05.323489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173813a-28f6-46d4-81c5-3729ff04746c service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:05.323778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173813a-28f6-46d4-81c5-3729ff04746c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:05.334359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173813a-28f6-46d4-81c5-3729ff04746c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:05.336111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173813a-28f6-46d4-81c5-3729ff04746c service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:05.348371 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3946/7979] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:05.652346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e84a14-5441-4e5e-b2e9-f02bf3dfda11 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:05.654292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e3e84a14-5441-4e5e-b2e9-f02bf3dfda11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:05.672859 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e84a14-5441-4e5e-b2e9-f02bf3dfda11 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:05.697301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:05.701157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:05.701574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:05.701867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] PATH_INFO: `/v3/projects/0e2f56b56f274e08a5a4f7d34fe0a491` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:05.704867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e2f56b56f274e08a5a4f7d34fe0a491)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:05.706995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:05.762010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e2f56b56f274e08a5a4f7d34fe0a491', 'request_id': 'req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.775410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e2f56b56f274e08a5a4f7d34fe0a491', 'request_id': 'req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.783658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e2f56b56f274e08a5a4f7d34fe0a491', 'request_id': 'req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.799356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] The token cache is being invalidate because project 0e2f56b56f274e08a5a4f7d34fe0a491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:18:05.799949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.800815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.802841 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4034/7980] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:18:05 2022] DELETE /identity/v3/projects/0e2f56b56f274e08a5a4f7d34fe0a491 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:05.807290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173813a-28f6-46d4-81c5-3729ff04746c service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:05.818931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173813a-28f6-46d4-81c5-3729ff04746c service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:05.878960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:05.896060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de6ef4-0f74-4a4c-a491-5db0d94c5ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:05.896652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de6ef4-0f74-4a4c-a491-5db0d94c5ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:05.897025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de6ef4-0f74-4a4c-a491-5db0d94c5ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:05.897327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de6ef4-0f74-4a4c-a491-5db0d94c5ac8 admin admin] PATH_INFO: `/v3/users/12e36e431cd94e7a908890b5b771ae15` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:05.899680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de6ef4-0f74-4a4c-a491-5db0d94c5ac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e36e431cd94e7a908890b5b771ae15)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:05.901806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de6ef4-0f74-4a4c-a491-5db0d94c5ac8 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:05.956335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48de6ef4-0f74-4a4c-a491-5db0d94c5ac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e36e431cd94e7a908890b5b771ae15'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.973707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48de6ef4-0f74-4a4c-a491-5db0d94c5ac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e36e431cd94e7a908890b5b771ae15'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.975045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48de6ef4-0f74-4a4c-a491-5db0d94c5ac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e36e431cd94e7a908890b5b771ae15'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:05.994986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48de6ef4-0f74-4a4c-a491-5db0d94c5ac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e36e431cd94e7a908890b5b771ae15'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:06.008308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48de6ef4-0f74-4a4c-a491-5db0d94c5ac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e36e431cd94e7a908890b5b771ae15'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:06.016300 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3947/7981] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:05 2022] DELETE /identity/v3/users/12e36e431cd94e7a908890b5b771ae15 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:06.630249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:06.631968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:06.652528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b93bd-8c65-441c-b0e8-f4ff88e6ab9b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:06.734566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:06.744815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea35510-4d0b-42a2-89df-c02063fa7dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:06.745652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea35510-4d0b-42a2-89df-c02063fa7dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:06.745964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea35510-4d0b-42a2-89df-c02063fa7dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:06.746543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea35510-4d0b-42a2-89df-c02063fa7dd4 admin admin] PATH_INFO: `/v3/projects/3dabab049da940a681df79d692dcbcf9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:06.747943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea35510-4d0b-42a2-89df-c02063fa7dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dabab049da940a681df79d692dcbcf9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:06.749952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea35510-4d0b-42a2-89df-c02063fa7dd4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:06.827262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ea35510-4d0b-42a2-89df-c02063fa7dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dabab049da940a681df79d692dcbcf9', 'request_id': 'req-9ea35510-4d0b-42a2-89df-c02063fa7dd4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:06.847028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ea35510-4d0b-42a2-89df-c02063fa7dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dabab049da940a681df79d692dcbcf9', 'request_id': 'req-9ea35510-4d0b-42a2-89df-c02063fa7dd4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:06.858032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ea35510-4d0b-42a2-89df-c02063fa7dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dabab049da940a681df79d692dcbcf9', 'request_id': 'req-9ea35510-4d0b-42a2-89df-c02063fa7dd4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:06.872762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ea35510-4d0b-42a2-89df-c02063fa7dd4 admin admin] The token cache is being invalidate because project 3dabab049da940a681df79d692dcbcf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:18:06.873148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ea35510-4d0b-42a2-89df-c02063fa7dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:06.873974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-9ea35510-4d0b-42a2-89df-c02063fa7dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:06.877253 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4035/7982] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:18:06 2022] DELETE /identity/v3/projects/3dabab049da940a681df79d692dcbcf9 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:06.884721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4c7d3-f66c-44cb-91ec-e47a652f19f7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:06.885252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4c7d3-f66c-44cb-91ec-e47a652f19f7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:06.885349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4c7d3-f66c-44cb-91ec-e47a652f19f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:06.920821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f3a4c7d3-f66c-44cb-91ec-e47a652f19f7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:06.923607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f3a4c7d3-f66c-44cb-91ec-e47a652f19f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:07.034957 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3948/7983] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:07.043634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f929ced-d508-406d-8cad-0ff7e5c872a6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:07.045311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f929ced-d508-406d-8cad-0ff7e5c872a6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:07.045897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f929ced-d508-406d-8cad-0ff7e5c872a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:07.100407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3f929ced-d508-406d-8cad-0ff7e5c872a6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:07.103432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3f929ced-d508-406d-8cad-0ff7e5c872a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:07.124437 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4036/7984] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:07.128883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4c7d3-f66c-44cb-91ec-e47a652f19f7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:07.146507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4c7d3-f66c-44cb-91ec-e47a652f19f7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:07.153884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7c558-ebc9-42ec-b215-987b47431196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:07.154944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7c558-ebc9-42ec-b215-987b47431196 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:07.155333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7c558-ebc9-42ec-b215-987b47431196 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:07.155706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7c558-ebc9-42ec-b215-987b47431196 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:07.156433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7c558-ebc9-42ec-b215-987b47431196 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:18:07.156819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7c558-ebc9-42ec-b215-987b47431196 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:07.159034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7c558-ebc9-42ec-b215-987b47431196 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:07.178704 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3949/7985] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:18:07.183992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f929ced-d508-406d-8cad-0ff7e5c872a6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:07.195675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f929ced-d508-406d-8cad-0ff7e5c872a6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:07.200875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265c277-fc96-415e-b5f4-ec3afa02979d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:07.201739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265c277-fc96-415e-b5f4-ec3afa02979d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:07.202321 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265c277-fc96-415e-b5f4-ec3afa02979d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:07.202812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265c277-fc96-415e-b5f4-ec3afa02979d admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:07.204068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265c277-fc96-415e-b5f4-ec3afa02979d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:07.208250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265c277-fc96-415e-b5f4-ec3afa02979d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:07.243478 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4037/7986] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:07 2022] POST /identity/v3/projects => generated 399 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:07.248243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7c558-ebc9-42ec-b215-987b47431196 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:07.260773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7c558-ebc9-42ec-b215-987b47431196 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:07.325901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:07.331633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9197a-5ede-45f4-af78-7add88fe42df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:07.334074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9197a-5ede-45f4-af78-7add88fe42df admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:07.334561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9197a-5ede-45f4-af78-7add88fe42df admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:07.335748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9197a-5ede-45f4-af78-7add88fe42df admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:07.335748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9197a-5ede-45f4-af78-7add88fe42df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:07.338418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9197a-5ede-45f4-af78-7add88fe42df admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:07.369258 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3950/7987] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:07 2022] POST /identity/v3/users => generated 306 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:07.370500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265c277-fc96-415e-b5f4-ec3afa02979d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:07.382747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265c277-fc96-415e-b5f4-ec3afa02979d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:07.388166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18073d9f-5b20-4c8c-9916-70e8c1aceaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:07.388886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18073d9f-5b20-4c8c-9916-70e8c1aceaa9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:07.389292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18073d9f-5b20-4c8c-9916-70e8c1aceaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:07.390087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18073d9f-5b20-4c8c-9916-70e8c1aceaa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:07.390419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18073d9f-5b20-4c8c-9916-70e8c1aceaa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:07.392128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18073d9f-5b20-4c8c-9916-70e8c1aceaa9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:07.413308 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4038/7988] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:18:07.417443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9197a-5ede-45f4-af78-7add88fe42df admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:07.428362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9197a-5ede-45f4-af78-7add88fe42df admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:07.434255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07bae0-82d9-4ffa-b590-a0140695b37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:07.434938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07bae0-82d9-4ffa-b590-a0140695b37e admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:07.435230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07bae0-82d9-4ffa-b590-a0140695b37e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:07.435516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07bae0-82d9-4ffa-b590-a0140695b37e admin admin] PATH_INFO: `/v3/projects/56d11580750d4b22b77fcedd4cbc196e/users/643c66361e934cd79ffa1251acd40fe2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:07.451763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07bae0-82d9-4ffa-b590-a0140695b37e admin admin] RBAC: Authorizing `identity:create_grant(project_id=56d11580750d4b22b77fcedd4cbc196e, user_id=643c66361e934cd79ffa1251acd40fe2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:07.454507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07bae0-82d9-4ffa-b590-a0140695b37e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:07.469029 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3951/7989] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:07 2022] PUT /identity/v3/projects/56d11580750d4b22b77fcedd4cbc196e/users/643c66361e934cd79ffa1251acd40fe2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:18:07.473037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18073d9f-5b20-4c8c-9916-70e8c1aceaa9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:07.483961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18073d9f-5b20-4c8c-9916-70e8c1aceaa9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:07.563545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89edae-f5ad-4174-8998-552120131185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:07.564329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89edae-f5ad-4174-8998-552120131185 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:07.564619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89edae-f5ad-4174-8998-552120131185 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:07.564919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89edae-f5ad-4174-8998-552120131185 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:07.565576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89edae-f5ad-4174-8998-552120131185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:07.567260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89edae-f5ad-4174-8998-552120131185 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:07.582479 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4039/7990] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:18:07.583729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07bae0-82d9-4ffa-b590-a0140695b37e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:07.597189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07bae0-82d9-4ffa-b590-a0140695b37e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:07.602750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cef935-d989-41fe-9886-699ea20a73d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:07.603455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cef935-d989-41fe-9886-699ea20a73d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:07.603744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cef935-d989-41fe-9886-699ea20a73d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:07.604025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cef935-d989-41fe-9886-699ea20a73d5 admin admin] PATH_INFO: `/v3/projects/56d11580750d4b22b77fcedd4cbc196e/users/643c66361e934cd79ffa1251acd40fe2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:07.606376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cef935-d989-41fe-9886-699ea20a73d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56d11580750d4b22b77fcedd4cbc196e, user_id=643c66361e934cd79ffa1251acd40fe2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:07.609232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cef935-d989-41fe-9886-699ea20a73d5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:07.633063 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3952/7991] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:07 2022] PUT /identity/v3/projects/56d11580750d4b22b77fcedd4cbc196e/users/643c66361e934cd79ffa1251acd40fe2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:07.636137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbadc4-bafc-4604-ab3a-8933274da9ba None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:07.636646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbadc4-bafc-4604-ab3a-8933274da9ba None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:07.636732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbadc4-bafc-4604-ab3a-8933274da9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:07.671085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-91cbadc4-bafc-4604-ab3a-8933274da9ba None None] MFA Rules not processed for user `643c66361e934cd79ffa1251acd40fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:07.672424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-91cbadc4-bafc-4604-ab3a-8933274da9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:07.721284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:07.780936 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4040/7992] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:07 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:07.782723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cef935-d989-41fe-9886-699ea20a73d5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:07.857274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cef935-d989-41fe-9886-699ea20a73d5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:07.862063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b2f54-97c8-4f3f-b63f-7875f378ee4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:07.862746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b2f54-97c8-4f3f-b63f-7875f378ee4a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:07.863041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b2f54-97c8-4f3f-b63f-7875f378ee4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:07.863325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b2f54-97c8-4f3f-b63f-7875f378ee4a admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:07.863969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b2f54-97c8-4f3f-b63f-7875f378ee4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:07.865916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b2f54-97c8-4f3f-b63f-7875f378ee4a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:07.906574 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3953/7993] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:07 2022] POST /identity/v3/projects => generated 399 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:07.910566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbadc4-bafc-4604-ab3a-8933274da9ba None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:07.922118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbadc4-bafc-4604-ab3a-8933274da9ba None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:07.996148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdba4f-4edc-423b-a059-5d283c2e30c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:07.996882 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdba4f-4edc-423b-a059-5d283c2e30c2 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:07.997300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdba4f-4edc-423b-a059-5d283c2e30c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:07.997635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdba4f-4edc-423b-a059-5d283c2e30c2 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:07.998350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdba4f-4edc-423b-a059-5d283c2e30c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:08.000378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdba4f-4edc-423b-a059-5d283c2e30c2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:08.029215 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4041/7994] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:07 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:08.030491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b2f54-97c8-4f3f-b63f-7875f378ee4a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:08.041618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b2f54-97c8-4f3f-b63f-7875f378ee4a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:08.046755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfde36-3b49-49e5-a893-7753a9d88293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:08.047326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfde36-3b49-49e5-a893-7753a9d88293 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:08.047611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfde36-3b49-49e5-a893-7753a9d88293 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:08.047885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfde36-3b49-49e5-a893-7753a9d88293 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:08.048752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfde36-3b49-49e5-a893-7753a9d88293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:08.050392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfde36-3b49-49e5-a893-7753a9d88293 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:08.061571 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3954/7995] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:08.065777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdba4f-4edc-423b-a059-5d283c2e30c2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:08.077433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdba4f-4edc-423b-a059-5d283c2e30c2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:08.082338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885f782-09ab-4294-a2a6-498eaf494d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:08.082949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885f782-09ab-4294-a2a6-498eaf494d3e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:08.083239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885f782-09ab-4294-a2a6-498eaf494d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:08.083671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885f782-09ab-4294-a2a6-498eaf494d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:08.084392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885f782-09ab-4294-a2a6-498eaf494d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:08.086866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885f782-09ab-4294-a2a6-498eaf494d3e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:08.098558 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4042/7996] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:08.106450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfde36-3b49-49e5-a893-7753a9d88293 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:08.126102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfde36-3b49-49e5-a893-7753a9d88293 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:08.142380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbeb235-87be-4951-b283-3fbda00e8f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:08.143084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbeb235-87be-4951-b283-3fbda00e8f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:08.143368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbeb235-87be-4951-b283-3fbda00e8f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:08.143652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbeb235-87be-4951-b283-3fbda00e8f12 admin admin] PATH_INFO: `/v3/projects/6f4a90e7d4704bb39f29cf16bd6812d5/users/dc1c91bf32764afe9542fca815f2d2d6/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:08.160170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbeb235-87be-4951-b283-3fbda00e8f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f4a90e7d4704bb39f29cf16bd6812d5, user_id=dc1c91bf32764afe9542fca815f2d2d6, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:08.163058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbeb235-87be-4951-b283-3fbda00e8f12 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:08.180124 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3955/7997] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:08 2022] PUT /identity/v3/projects/6f4a90e7d4704bb39f29cf16bd6812d5/users/dc1c91bf32764afe9542fca815f2d2d6/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:08.190370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c599af3-663b-4590-8992-5064beadc1f6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:08.190812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c599af3-663b-4590-8992-5064beadc1f6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:08.191204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c599af3-663b-4590-8992-5064beadc1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:08.232058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-5c599af3-663b-4590-8992-5064beadc1f6 None None] MFA Rules not processed for user `dc1c91bf32764afe9542fca815f2d2d6`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:08.233437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5c599af3-663b-4590-8992-5064beadc1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:08.325794 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4043/7998] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:18:08.340253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbeb235-87be-4951-b283-3fbda00e8f12 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:08.341972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-afbeb235-87be-4951-b283-3fbda00e8f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:08.408386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbeb235-87be-4951-b283-3fbda00e8f12 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:08.412419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50f08a-d612-417e-9f21-4c49cbb59ffa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:08.413010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50f08a-d612-417e-9f21-4c49cbb59ffa service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:08.413344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50f08a-d612-417e-9f21-4c49cbb59ffa service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:08.413626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50f08a-d612-417e-9f21-4c49cbb59ffa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:08.426995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:08.486715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50f08a-d612-417e-9f21-4c49cbb59ffa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:08.488273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50f08a-d612-417e-9f21-4c49cbb59ffa service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:08.498935 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3956/7999] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:08 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:08.994478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c599af3-663b-4590-8992-5064beadc1f6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:09.073714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:09.108115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c599af3-663b-4590-8992-5064beadc1f6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:09.122985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:09.123723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:09.124020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:09.124304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] PATH_INFO: `/v3/users/643c66361e934cd79ffa1251acd40fe2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:09.126099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=643c66361e934cd79ffa1251acd40fe2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:09.128317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:09.192635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643c66361e934cd79ffa1251acd40fe2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:09.208397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643c66361e934cd79ffa1251acd40fe2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:09.217655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643c66361e934cd79ffa1251acd40fe2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:09.218487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643c66361e934cd79ffa1251acd40fe2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:09.218923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643c66361e934cd79ffa1251acd40fe2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:09.237009 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4044/8000] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:08 2022] DELETE /identity/v3/users/643c66361e934cd79ffa1251acd40fe2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:09.240320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023b561-c939-4742-ad27-bece31ff143b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:09.240852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023b561-c939-4742-ad27-bece31ff143b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:09.241001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023b561-c939-4742-ad27-bece31ff143b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:09.263361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2023b561-c939-4742-ad27-bece31ff143b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:09.264598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2023b561-c939-4742-ad27-bece31ff143b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:09.343186 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3957/8001] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:18:09.356055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:09.357770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:09.438841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda54495-28a8-4a7f-ba3e-14137df35369 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:09.442734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58156124-3ca9-47e7-96d2-fa570babb5f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:09.443289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58156124-3ca9-47e7-96d2-fa570babb5f4 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:09.443578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58156124-3ca9-47e7-96d2-fa570babb5f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:09.443857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58156124-3ca9-47e7-96d2-fa570babb5f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:09.453652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58156124-3ca9-47e7-96d2-fa570babb5f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:09.455169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58156124-3ca9-47e7-96d2-fa570babb5f4 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:09.465348 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4045/8002] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:18:10.014660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023b561-c939-4742-ad27-bece31ff143b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:10.017341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2023b561-c939-4742-ad27-bece31ff143b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:10.043516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023b561-c939-4742-ad27-bece31ff143b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:10.051814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:10.052475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:10.052995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:10.053471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] PATH_INFO: `/v3/projects/56d11580750d4b22b77fcedd4cbc196e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:10.055545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=56d11580750d4b22b77fcedd4cbc196e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:10.058863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:10.157302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56d11580750d4b22b77fcedd4cbc196e', 'request_id': 'req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:10.171068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56d11580750d4b22b77fcedd4cbc196e', 'request_id': 'req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:10.183187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56d11580750d4b22b77fcedd4cbc196e', 'request_id': 'req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:10.200654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] The token cache is being invalidate because project 56d11580750d4b22b77fcedd4cbc196e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:18:10.201153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:10.213757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:10.219136 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3958/8003] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:18:10 2022] DELETE /identity/v3/projects/56d11580750d4b22b77fcedd4cbc196e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:10.222834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58156124-3ca9-47e7-96d2-fa570babb5f4 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:10.233876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58156124-3ca9-47e7-96d2-fa570babb5f4 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:10.332601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9d4f2-859c-4f45-b32c-69ae74cabd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:10.333360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9d4f2-859c-4f45-b32c-69ae74cabd5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:10.333645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9d4f2-859c-4f45-b32c-69ae74cabd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:10.333929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9d4f2-859c-4f45-b32c-69ae74cabd5a admin admin] PATH_INFO: `/v3/users/dc1c91bf32764afe9542fca815f2d2d6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:10.335153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9d4f2-859c-4f45-b32c-69ae74cabd5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc1c91bf32764afe9542fca815f2d2d6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:10.337160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9d4f2-859c-4f45-b32c-69ae74cabd5a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:10.338947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:10.405331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5ea9d4f2-859c-4f45-b32c-69ae74cabd5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc1c91bf32764afe9542fca815f2d2d6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:10.420054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5ea9d4f2-859c-4f45-b32c-69ae74cabd5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc1c91bf32764afe9542fca815f2d2d6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:10.430124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5ea9d4f2-859c-4f45-b32c-69ae74cabd5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc1c91bf32764afe9542fca815f2d2d6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:10.431191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5ea9d4f2-859c-4f45-b32c-69ae74cabd5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc1c91bf32764afe9542fca815f2d2d6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:10.432277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5ea9d4f2-859c-4f45-b32c-69ae74cabd5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc1c91bf32764afe9542fca815f2d2d6'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:10.450195 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4046/8004] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:10 2022] DELETE /identity/v3/users/dc1c91bf32764afe9542fca815f2d2d6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:10.777123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:10.778607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:10.807054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b63c50-d4b0-4e34-a3dc-8412c9ffc4ab admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:10.867423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:10.921333 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:10.922128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:10.926822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:10.927258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2 admin admin] PATH_INFO: `/v3/projects/6f4a90e7d4704bb39f29cf16bd6812d5` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:10.930027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f4a90e7d4704bb39f29cf16bd6812d5)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:10.932382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:11.018350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f4a90e7d4704bb39f29cf16bd6812d5', 'request_id': 'req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:11.030716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f4a90e7d4704bb39f29cf16bd6812d5', 'request_id': 'req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:11.039771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f4a90e7d4704bb39f29cf16bd6812d5', 'request_id': 'req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:11.053178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2 admin admin] The token cache is being invalidate because project 6f4a90e7d4704bb39f29cf16bd6812d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:18:11.053521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:11.054262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-1c051c82-fb13-46ab-969d-a7ac8d6e5eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:11.056112 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3959/8005] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:18:10 2022] DELETE /identity/v3/projects/6f4a90e7d4704bb39f29cf16bd6812d5 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:11.062965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16057d6-2720-42bc-b10e-48d6e488d08e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:11.064079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16057d6-2720-42bc-b10e-48d6e488d08e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:11.064079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16057d6-2720-42bc-b10e-48d6e488d08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:11.085983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c16057d6-2720-42bc-b10e-48d6e488d08e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:11.087909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c16057d6-2720-42bc-b10e-48d6e488d08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:11.183744 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4047/8006] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:11.192032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc7731-2aca-4ff9-9a3b-079646a564b7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:11.192324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc7731-2aca-4ff9-9a3b-079646a564b7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:11.192605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc7731-2aca-4ff9-9a3b-079646a564b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:11.215268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3bdc7731-2aca-4ff9-9a3b-079646a564b7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:11.216642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3bdc7731-2aca-4ff9-9a3b-079646a564b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:11.226492 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3960/8007] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:11.232102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16057d6-2720-42bc-b10e-48d6e488d08e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:11.242382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16057d6-2720-42bc-b10e-48d6e488d08e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:11.247384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402f34d-7743-4c22-946d-c48f505334fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:11.248134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402f34d-7743-4c22-946d-c48f505334fa admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:11.248431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402f34d-7743-4c22-946d-c48f505334fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:11.248717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402f34d-7743-4c22-946d-c48f505334fa admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:11.249493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402f34d-7743-4c22-946d-c48f505334fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:18:11.249814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402f34d-7743-4c22-946d-c48f505334fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:11.251744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402f34d-7743-4c22-946d-c48f505334fa admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:11.271110 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4048/8008] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:18:11.271956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc7731-2aca-4ff9-9a3b-079646a564b7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:11.282591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc7731-2aca-4ff9-9a3b-079646a564b7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:11.289304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bb95a-4c52-43f0-9025-630e58801aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:11.290182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bb95a-4c52-43f0-9025-630e58801aec admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:11.290468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bb95a-4c52-43f0-9025-630e58801aec admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:11.290755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bb95a-4c52-43f0-9025-630e58801aec admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:11.291395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17bb95a-4c52-43f0-9025-630e58801aec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:11.293351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17bb95a-4c52-43f0-9025-630e58801aec admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:11.326900 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3961/8009] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:11 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:11.330795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402f34d-7743-4c22-946d-c48f505334fa admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:11.342353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402f34d-7743-4c22-946d-c48f505334fa admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:11.412037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d4ac6-461e-41a4-b23b-e52def6e5260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:11.412262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d4ac6-461e-41a4-b23b-e52def6e5260 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:11.413211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d4ac6-461e-41a4-b23b-e52def6e5260 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:11.413674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d4ac6-461e-41a4-b23b-e52def6e5260 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:11.414485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d4ac6-461e-41a4-b23b-e52def6e5260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:11.416885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d4ac6-461e-41a4-b23b-e52def6e5260 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:11.428657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:11.444331 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4049/8010] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:11 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:11.448605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bb95a-4c52-43f0-9025-630e58801aec admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:11.464864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bb95a-4c52-43f0-9025-630e58801aec admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:11.473419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec772c-a785-497f-81d3-da577088222a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:11.474526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec772c-a785-497f-81d3-da577088222a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:11.475129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec772c-a785-497f-81d3-da577088222a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:11.475757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec772c-a785-497f-81d3-da577088222a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:11.479049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec772c-a785-497f-81d3-da577088222a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:11.484232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec772c-a785-497f-81d3-da577088222a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:11.496958 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3962/8011] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:11.500811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d4ac6-461e-41a4-b23b-e52def6e5260 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:11.515291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d4ac6-461e-41a4-b23b-e52def6e5260 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:11.525320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1f719-fc0f-4ad3-9627-7bd74ca2c8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:11.526199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1f719-fc0f-4ad3-9627-7bd74ca2c8c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:11.526592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1f719-fc0f-4ad3-9627-7bd74ca2c8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:11.527306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1f719-fc0f-4ad3-9627-7bd74ca2c8c2 admin admin] PATH_INFO: `/v3/projects/5fa285f2dc4649f3b6a3bca59610f93e/users/eeaf239a2f44416fba2ab20e668105b2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:11.556696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1f719-fc0f-4ad3-9627-7bd74ca2c8c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa285f2dc4649f3b6a3bca59610f93e, user_id=eeaf239a2f44416fba2ab20e668105b2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:11.559636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1f719-fc0f-4ad3-9627-7bd74ca2c8c2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:11.580466 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4050/8012] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:11 2022] PUT /identity/v3/projects/5fa285f2dc4649f3b6a3bca59610f93e/users/eeaf239a2f44416fba2ab20e668105b2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:11.583837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec772c-a785-497f-81d3-da577088222a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:11.599822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec772c-a785-497f-81d3-da577088222a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:11.703205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2744ab-30c5-49ee-a2ba-c9ae3859000d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:11.703941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2744ab-30c5-49ee-a2ba-c9ae3859000d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:11.705115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2744ab-30c5-49ee-a2ba-c9ae3859000d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:11.705115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2744ab-30c5-49ee-a2ba-c9ae3859000d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:11.705344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2744ab-30c5-49ee-a2ba-c9ae3859000d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:11.707066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2744ab-30c5-49ee-a2ba-c9ae3859000d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:11.719438 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3963/8013] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:18:11.723246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1f719-fc0f-4ad3-9627-7bd74ca2c8c2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:11.735159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1f719-fc0f-4ad3-9627-7bd74ca2c8c2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:11.740269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082eadc9-22df-4ab1-b1c1-e5ea81a16500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:11.741268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082eadc9-22df-4ab1-b1c1-e5ea81a16500 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:11.742271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082eadc9-22df-4ab1-b1c1-e5ea81a16500 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:11.742558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082eadc9-22df-4ab1-b1c1-e5ea81a16500 admin admin] PATH_INFO: `/v3/projects/5fa285f2dc4649f3b6a3bca59610f93e/users/eeaf239a2f44416fba2ab20e668105b2/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:11.744682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082eadc9-22df-4ab1-b1c1-e5ea81a16500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa285f2dc4649f3b6a3bca59610f93e, user_id=eeaf239a2f44416fba2ab20e668105b2, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:11.748313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082eadc9-22df-4ab1-b1c1-e5ea81a16500 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:11.767595 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4051/8014] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:11 2022] PUT /identity/v3/projects/5fa285f2dc4649f3b6a3bca59610f93e/users/eeaf239a2f44416fba2ab20e668105b2/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:18:11.779142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda8b95-0f60-4357-8ee8-42ab8c899938 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:11.779142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda8b95-0f60-4357-8ee8-42ab8c899938 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:11.779142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda8b95-0f60-4357-8ee8-42ab8c899938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:11.816096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0eda8b95-0f60-4357-8ee8-42ab8c899938 None None] MFA Rules not processed for user `eeaf239a2f44416fba2ab20e668105b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:11.817287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0eda8b95-0f60-4357-8ee8-42ab8c899938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:11.829347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:11.941264 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3964/8015] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:11.946878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082eadc9-22df-4ab1-b1c1-e5ea81a16500 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:12.107237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082eadc9-22df-4ab1-b1c1-e5ea81a16500 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:12.126790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f9203-b970-4f60-8347-e1df75ebc912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:12.136890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f9203-b970-4f60-8347-e1df75ebc912 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:12.136890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f9203-b970-4f60-8347-e1df75ebc912 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:12.138208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f9203-b970-4f60-8347-e1df75ebc912 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:12.138208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f9203-b970-4f60-8347-e1df75ebc912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:12.139842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f9203-b970-4f60-8347-e1df75ebc912 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:12.177570 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4052/8016] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:11 2022] POST /identity/v3/projects => generated 387 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:12.183101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda8b95-0f60-4357-8ee8-42ab8c899938 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:12.195506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda8b95-0f60-4357-8ee8-42ab8c899938 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:12.293365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031debe-ac34-4f36-9130-9ae6102746dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:12.294096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031debe-ac34-4f36-9130-9ae6102746dc admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:12.294394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031debe-ac34-4f36-9130-9ae6102746dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:12.294677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031debe-ac34-4f36-9130-9ae6102746dc admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:12.295334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031debe-ac34-4f36-9130-9ae6102746dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:12.297847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031debe-ac34-4f36-9130-9ae6102746dc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:12.338655 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3965/8017] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:12 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:12.342406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f9203-b970-4f60-8347-e1df75ebc912 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:12.358290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f9203-b970-4f60-8347-e1df75ebc912 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:12.363919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb57da4-88e7-45df-b1c8-8dfae7583d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:12.364661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb57da4-88e7-45df-b1c8-8dfae7583d0d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:12.365039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb57da4-88e7-45df-b1c8-8dfae7583d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:12.365387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb57da4-88e7-45df-b1c8-8dfae7583d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:12.366119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb57da4-88e7-45df-b1c8-8dfae7583d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:12.368378 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb57da4-88e7-45df-b1c8-8dfae7583d0d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:12.388482 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4053/8018] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:12.389886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031debe-ac34-4f36-9130-9ae6102746dc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:12.402504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031debe-ac34-4f36-9130-9ae6102746dc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:12.410151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4835c8-3765-4a12-945c-11d25bf3bd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:12.411134 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4835c8-3765-4a12-945c-11d25bf3bd9f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:12.411555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4835c8-3765-4a12-945c-11d25bf3bd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:12.411823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4835c8-3765-4a12-945c-11d25bf3bd9f admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:12.412479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4835c8-3765-4a12-945c-11d25bf3bd9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:12.415214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4835c8-3765-4a12-945c-11d25bf3bd9f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:12.430382 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3966/8019] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:12.431607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb57da4-88e7-45df-b1c8-8dfae7583d0d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:12.444489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb57da4-88e7-45df-b1c8-8dfae7583d0d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:12.450057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07c60c-ab1d-42c4-bf1a-f572030e35e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:12.451150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07c60c-ab1d-42c4-bf1a-f572030e35e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:12.451150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07c60c-ab1d-42c4-bf1a-f572030e35e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:12.451654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07c60c-ab1d-42c4-bf1a-f572030e35e1 admin admin] PATH_INFO: `/v3/projects/cf5605a5ef8b43b88ff93d655d2cc165/users/465c820ab731446ea6dd8c64a01cd31a/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:12.470906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07c60c-ab1d-42c4-bf1a-f572030e35e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5605a5ef8b43b88ff93d655d2cc165, user_id=465c820ab731446ea6dd8c64a01cd31a, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:12.473680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07c60c-ab1d-42c4-bf1a-f572030e35e1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:12.492936 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4054/8020] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:12 2022] PUT /identity/v3/projects/cf5605a5ef8b43b88ff93d655d2cc165/users/465c820ab731446ea6dd8c64a01cd31a/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:12.504964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4df34-9462-4890-a8c7-5ad9e4088b71 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:12.505704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4df34-9462-4890-a8c7-5ad9e4088b71 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:12.506135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4df34-9462-4890-a8c7-5ad9e4088b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:12.547470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-dbe4df34-9462-4890-a8c7-5ad9e4088b71 None None] MFA Rules not processed for user `465c820ab731446ea6dd8c64a01cd31a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:12.549397 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-dbe4df34-9462-4890-a8c7-5ad9e4088b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:12.640995 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3967/8021] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:12.653166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07c60c-ab1d-42c4-bf1a-f572030e35e1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:12.655086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-db07c60c-ab1d-42c4-bf1a-f572030e35e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:12.725354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07c60c-ab1d-42c4-bf1a-f572030e35e1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:12.729791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29675d4-ad26-4622-804d-06843b4deeb2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:12.730726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29675d4-ad26-4622-804d-06843b4deeb2 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:12.731016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29675d4-ad26-4622-804d-06843b4deeb2 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:12.731298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29675d4-ad26-4622-804d-06843b4deeb2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:12.754441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:12.803341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29675d4-ad26-4622-804d-06843b4deeb2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:12.804825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29675d4-ad26-4622-804d-06843b4deeb2 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:12.827604 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4055/8022] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:13.151246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4df34-9462-4890-a8c7-5ad9e4088b71 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:13.165830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:13.228179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4df34-9462-4890-a8c7-5ad9e4088b71 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:13.233256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:13.233910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:13.234200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:13.234479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] PATH_INFO: `/v3/users/eeaf239a2f44416fba2ab20e668105b2` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:13.235631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeaf239a2f44416fba2ab20e668105b2)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:13.237636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:13.293971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeaf239a2f44416fba2ab20e668105b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:13.307068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeaf239a2f44416fba2ab20e668105b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:13.307827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeaf239a2f44416fba2ab20e668105b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:13.315212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeaf239a2f44416fba2ab20e668105b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:13.322826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeaf239a2f44416fba2ab20e668105b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:13.329252 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3968/8023] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:13 2022] DELETE /identity/v3/users/eeaf239a2f44416fba2ab20e668105b2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:13.334085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a31243-a8b1-4807-8cb6-ea01bfad418a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:13.334398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a31243-a8b1-4807-8cb6-ea01bfad418a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:13.334657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a31243-a8b1-4807-8cb6-ea01bfad418a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:13.358658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-83a31243-a8b1-4807-8cb6-ea01bfad418a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:13.359901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-83a31243-a8b1-4807-8cb6-ea01bfad418a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:13.422927 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4056/8024] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:13.433023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:13.434620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:13.500607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134e6a-eb76-4c07-b541-a436082e9df2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:13.504487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c51463-0812-4245-b14c-00ac94e4075c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:13.505162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c51463-0812-4245-b14c-00ac94e4075c service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:13.505552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c51463-0812-4245-b14c-00ac94e4075c service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:13.505838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c51463-0812-4245-b14c-00ac94e4075c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:13.516163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c51463-0812-4245-b14c-00ac94e4075c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:13.517666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c51463-0812-4245-b14c-00ac94e4075c service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:13.528883 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3969/8025] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:13.858953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a31243-a8b1-4807-8cb6-ea01bfad418a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:13.861161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-83a31243-a8b1-4807-8cb6-ea01bfad418a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:13.875738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a31243-a8b1-4807-8cb6-ea01bfad418a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:13.885805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:13.886394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:13.886689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:13.886972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] PATH_INFO: `/v3/projects/5fa285f2dc4649f3b6a3bca59610f93e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:13.888350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa285f2dc4649f3b6a3bca59610f93e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:13.890371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:13.967169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa285f2dc4649f3b6a3bca59610f93e', 'request_id': 'req-a4cacf09-590e-4823-98bd-d402998af6f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:13.975544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa285f2dc4649f3b6a3bca59610f93e', 'request_id': 'req-a4cacf09-590e-4823-98bd-d402998af6f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:13.982637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa285f2dc4649f3b6a3bca59610f93e', 'request_id': 'req-a4cacf09-590e-4823-98bd-d402998af6f0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:13.994212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] The token cache is being invalidate because project 5fa285f2dc4649f3b6a3bca59610f93e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:18:13.994450 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:13.995198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:13.996766 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4057/8026] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:18:13 2022] DELETE /identity/v3/projects/5fa285f2dc4649f3b6a3bca59610f93e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:14.000684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c51463-0812-4245-b14c-00ac94e4075c service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:14.011200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c51463-0812-4245-b14c-00ac94e4075c service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:14.046360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:14.074184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:14.074861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:14.075155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:14.075421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] PATH_INFO: `/v3/users/465c820ab731446ea6dd8c64a01cd31a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:14.076634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=465c820ab731446ea6dd8c64a01cd31a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:14.078797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:14.135899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '465c820ab731446ea6dd8c64a01cd31a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:14.151717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '465c820ab731446ea6dd8c64a01cd31a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:14.152504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '465c820ab731446ea6dd8c64a01cd31a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:14.160861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '465c820ab731446ea6dd8c64a01cd31a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:14.171963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '465c820ab731446ea6dd8c64a01cd31a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:14.180920 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3970/8027] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:13 2022] DELETE /identity/v3/users/465c820ab731446ea6dd8c64a01cd31a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:14.637697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:14.639234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:14.653901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cacf09-590e-4823-98bd-d402998af6f0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:14.699802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:14.721801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416603e9-7f85-45fc-82c8-6a24dfe659c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:14.722442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416603e9-7f85-45fc-82c8-6a24dfe659c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:14.722730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416603e9-7f85-45fc-82c8-6a24dfe659c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:14.723035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416603e9-7f85-45fc-82c8-6a24dfe659c7 admin admin] PATH_INFO: `/v3/projects/cf5605a5ef8b43b88ff93d655d2cc165` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:14.724242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416603e9-7f85-45fc-82c8-6a24dfe659c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf5605a5ef8b43b88ff93d655d2cc165)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:14.726864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416603e9-7f85-45fc-82c8-6a24dfe659c7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:14.777689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-416603e9-7f85-45fc-82c8-6a24dfe659c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf5605a5ef8b43b88ff93d655d2cc165', 'request_id': 'req-416603e9-7f85-45fc-82c8-6a24dfe659c7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:14.779012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:14.781609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:14.785921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-416603e9-7f85-45fc-82c8-6a24dfe659c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf5605a5ef8b43b88ff93d655d2cc165', 'request_id': 'req-416603e9-7f85-45fc-82c8-6a24dfe659c7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:14.793304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-416603e9-7f85-45fc-82c8-6a24dfe659c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf5605a5ef8b43b88ff93d655d2cc165', 'request_id': 'req-416603e9-7f85-45fc-82c8-6a24dfe659c7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:14.805399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-416603e9-7f85-45fc-82c8-6a24dfe659c7 admin admin] The token cache is being invalidate because project cf5605a5ef8b43b88ff93d655d2cc165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:18:14.805695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-416603e9-7f85-45fc-82c8-6a24dfe659c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:14.806462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-416603e9-7f85-45fc-82c8-6a24dfe659c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:14.808354 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4058/8028] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:18:14 2022] DELETE /identity/v3/projects/cf5605a5ef8b43b88ff93d655d2cc165 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:14.813109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6be21d-ae48-4d1b-be69-37084c2a8f8c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:14.813786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6be21d-ae48-4d1b-be69-37084c2a8f8c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:14.814234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6be21d-ae48-4d1b-be69-37084c2a8f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:14.839258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-aa6be21d-ae48-4d1b-be69-37084c2a8f8c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:14.840659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-aa6be21d-ae48-4d1b-be69-37084c2a8f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:14.881153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce93b27-63f1-484d-afae-6e140398f2e0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:14.918367 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4059/8029] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:18:14.927960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875f2d6-8a2a-43af-9fb8-a9db80b4d764 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:14.928258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875f2d6-8a2a-43af-9fb8-a9db80b4d764 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:14.928545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875f2d6-8a2a-43af-9fb8-a9db80b4d764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:14.951373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b875f2d6-8a2a-43af-9fb8-a9db80b4d764 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:14.952730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b875f2d6-8a2a-43af-9fb8-a9db80b4d764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:14.955916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:14.964667 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4060/8030] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:18:14.965771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875f2d6-8a2a-43af-9fb8-a9db80b4d764 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:14.975003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b954d-cc88-47d9-9847-877f6afdda6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:14.975452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875f2d6-8a2a-43af-9fb8-a9db80b4d764 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:14.975796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b954d-cc88-47d9-9847-877f6afdda6f service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:14.976327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b954d-cc88-47d9-9847-877f6afdda6f service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:14.976609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b954d-cc88-47d9-9847-877f6afdda6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:14.978725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-705b954d-cc88-47d9-9847-877f6afdda6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:14.982826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa3943-8b2b-4350-9c93-1a21a9b597d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:14.983867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa3943-8b2b-4350-9c93-1a21a9b597d4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:14.984441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa3943-8b2b-4350-9c93-1a21a9b597d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:14.985015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa3943-8b2b-4350-9c93-1a21a9b597d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:14.985938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa3943-8b2b-4350-9c93-1a21a9b597d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:18:14.986531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa3943-8b2b-4350-9c93-1a21a9b597d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:14.989979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa3943-8b2b-4350-9c93-1a21a9b597d4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:14.995651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b954d-cc88-47d9-9847-877f6afdda6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:14.997424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b954d-cc88-47d9-9847-877f6afdda6f service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:15.010721 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4061/8031] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:18:15.011995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa3943-8b2b-4350-9c93-1a21a9b597d4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:15.016049 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3971/8032] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:18:15.025112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa3943-8b2b-4350-9c93-1a21a9b597d4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:15.030133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da933588-bdeb-4e45-a2ff-b2f334f6b2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:15.030756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da933588-bdeb-4e45-a2ff-b2f334f6b2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.031174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da933588-bdeb-4e45-a2ff-b2f334f6b2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.031359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da933588-bdeb-4e45-a2ff-b2f334f6b2b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.032284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da933588-bdeb-4e45-a2ff-b2f334f6b2b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:15.034385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da933588-bdeb-4e45-a2ff-b2f334f6b2b9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:15.067012 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4062/8033] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:15 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:15.071733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b954d-cc88-47d9-9847-877f6afdda6f service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:15.082566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b954d-cc88-47d9-9847-877f6afdda6f service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:15.145018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b042b-2ebc-46be-9085-81216e681291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:15.145651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b042b-2ebc-46be-9085-81216e681291 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.145950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b042b-2ebc-46be-9085-81216e681291 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.146216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b042b-2ebc-46be-9085-81216e681291 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.146871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b042b-2ebc-46be-9085-81216e681291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:15.150209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b042b-2ebc-46be-9085-81216e681291 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:15.175234 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3972/8034] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:15 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:15.179912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da933588-bdeb-4e45-a2ff-b2f334f6b2b9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:15.191451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da933588-bdeb-4e45-a2ff-b2f334f6b2b9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:15.196415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f69847-5ec5-4297-bc38-63a13e1e7e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:15.197065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f69847-5ec5-4297-bc38-63a13e1e7e21 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.197635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f69847-5ec5-4297-bc38-63a13e1e7e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.197980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f69847-5ec5-4297-bc38-63a13e1e7e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.198640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f69847-5ec5-4297-bc38-63a13e1e7e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:15.200926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f69847-5ec5-4297-bc38-63a13e1e7e21 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:15.214946 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4063/8035] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:15.216314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b042b-2ebc-46be-9085-81216e681291 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:15.228142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b042b-2ebc-46be-9085-81216e681291 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:15.233486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75ff39-65d0-4fb2-9323-8a08325b12b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:15.234099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ff39-65d0-4fb2-9323-8a08325b12b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.234383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ff39-65d0-4fb2-9323-8a08325b12b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.234656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ff39-65d0-4fb2-9323-8a08325b12b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.235302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75ff39-65d0-4fb2-9323-8a08325b12b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:15.237135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75ff39-65d0-4fb2-9323-8a08325b12b6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:15.248912 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3973/8036] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:18:15.253149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f69847-5ec5-4297-bc38-63a13e1e7e21 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:15.263967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f69847-5ec5-4297-bc38-63a13e1e7e21 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:15.268718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cdc11-e57d-4ac5-b718-aa78a7c04635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:15.269407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cdc11-e57d-4ac5-b718-aa78a7c04635 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.269706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cdc11-e57d-4ac5-b718-aa78a7c04635 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.269991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cdc11-e57d-4ac5-b718-aa78a7c04635 admin admin] PATH_INFO: `/v3/projects/d9e491009ac74f10af26afbfe0e87181/users/7db236cd120a4b75a83292e99a6f2e33/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.286028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cdc11-e57d-4ac5-b718-aa78a7c04635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e491009ac74f10af26afbfe0e87181, user_id=7db236cd120a4b75a83292e99a6f2e33, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:15.288784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cdc11-e57d-4ac5-b718-aa78a7c04635 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:15.291994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:15.308993 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4064/8037] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:15 2022] PUT /identity/v3/projects/d9e491009ac74f10af26afbfe0e87181/users/7db236cd120a4b75a83292e99a6f2e33/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:15.314932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f567c65-bde3-4eb5-90aa-81e68335d918 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.315222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f567c65-bde3-4eb5-90aa-81e68335d918 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.315500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f567c65-bde3-4eb5-90aa-81e68335d918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.347401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9f567c65-bde3-4eb5-90aa-81e68335d918 None None] MFA Rules not processed for user `7db236cd120a4b75a83292e99a6f2e33`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:15.349458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9f567c65-bde3-4eb5-90aa-81e68335d918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:15.419412 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3974/8038] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:15.424087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cdc11-e57d-4ac5-b718-aa78a7c04635 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:15.488466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cdc11-e57d-4ac5-b718-aa78a7c04635 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:15.494304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daed00f-6bcc-4fa8-a961-475bfc77574f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:15.494921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daed00f-6bcc-4fa8-a961-475bfc77574f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.495204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daed00f-6bcc-4fa8-a961-475bfc77574f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.495484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daed00f-6bcc-4fa8-a961-475bfc77574f admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.496119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daed00f-6bcc-4fa8-a961-475bfc77574f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:15.498460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daed00f-6bcc-4fa8-a961-475bfc77574f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:15.527035 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4065/8039] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:15 2022] POST /identity/v3/projects => generated 391 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:15.531212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f567c65-bde3-4eb5-90aa-81e68335d918 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:15.542258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f567c65-bde3-4eb5-90aa-81e68335d918 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:15.602697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:15.614212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd4fcf-aed2-40f9-b170-223fa670e4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:15.615279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd4fcf-aed2-40f9-b170-223fa670e4f0 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.615279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd4fcf-aed2-40f9-b170-223fa670e4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.615652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd4fcf-aed2-40f9-b170-223fa670e4f0 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.616921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd4fcf-aed2-40f9-b170-223fa670e4f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:15.619285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd4fcf-aed2-40f9-b170-223fa670e4f0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:15.641879 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3975/8040] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:15 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:15.646597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daed00f-6bcc-4fa8-a961-475bfc77574f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:15.658765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daed00f-6bcc-4fa8-a961-475bfc77574f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:15.663476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b384e8-8893-44a7-b0f7-4da11dea479b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:15.664168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b384e8-8893-44a7-b0f7-4da11dea479b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.664585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b384e8-8893-44a7-b0f7-4da11dea479b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.664813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b384e8-8893-44a7-b0f7-4da11dea479b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.665481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b384e8-8893-44a7-b0f7-4da11dea479b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:15.667245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b384e8-8893-44a7-b0f7-4da11dea479b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:15.678680 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4066/8041] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:15.685200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd4fcf-aed2-40f9-b170-223fa670e4f0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:15.701665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd4fcf-aed2-40f9-b170-223fa670e4f0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:15.708423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed61e05-e2f2-4141-aabf-9b79c9e46423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:15.709231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed61e05-e2f2-4141-aabf-9b79c9e46423 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.709460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed61e05-e2f2-4141-aabf-9b79c9e46423 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.709752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed61e05-e2f2-4141-aabf-9b79c9e46423 admin admin] PATH_INFO: `/v3/projects/04b8b86563ff4b6197e2a704895f4721/users/c90e18da83904024b05a69cc2356fa85/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.732215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed61e05-e2f2-4141-aabf-9b79c9e46423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b8b86563ff4b6197e2a704895f4721, user_id=c90e18da83904024b05a69cc2356fa85, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:15.735001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed61e05-e2f2-4141-aabf-9b79c9e46423 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:15.755881 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3976/8042] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:15 2022] PUT /identity/v3/projects/04b8b86563ff4b6197e2a704895f4721/users/c90e18da83904024b05a69cc2356fa85/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:15.760329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b384e8-8893-44a7-b0f7-4da11dea479b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:15.772161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b384e8-8893-44a7-b0f7-4da11dea479b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:15.839831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf156eb7-f64d-42c3-9e98-e666dc76a943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:15.840461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf156eb7-f64d-42c3-9e98-e666dc76a943 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.840751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf156eb7-f64d-42c3-9e98-e666dc76a943 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.841448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf156eb7-f64d-42c3-9e98-e666dc76a943 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.842239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf156eb7-f64d-42c3-9e98-e666dc76a943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:15.843861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf156eb7-f64d-42c3-9e98-e666dc76a943 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:15.859155 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4067/8043] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:15.860202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed61e05-e2f2-4141-aabf-9b79c9e46423 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:15.871792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed61e05-e2f2-4141-aabf-9b79c9e46423 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:15.876569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c57e9-cef1-4d0e-91b7-de871940adb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:15.877297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c57e9-cef1-4d0e-91b7-de871940adb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.877647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c57e9-cef1-4d0e-91b7-de871940adb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.878780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c57e9-cef1-4d0e-91b7-de871940adb5 admin admin] PATH_INFO: `/v3/projects/04b8b86563ff4b6197e2a704895f4721/users/c90e18da83904024b05a69cc2356fa85/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.879999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c57e9-cef1-4d0e-91b7-de871940adb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b8b86563ff4b6197e2a704895f4721, user_id=c90e18da83904024b05a69cc2356fa85, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:15.883374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c57e9-cef1-4d0e-91b7-de871940adb5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:15.904417 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3977/8044] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:15 2022] PUT /identity/v3/projects/04b8b86563ff4b6197e2a704895f4721/users/c90e18da83904024b05a69cc2356fa85/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:18:15.906125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9c817-c720-433e-814a-2d3b23585b40 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:15.906595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9c817-c720-433e-814a-2d3b23585b40 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:15.906881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9c817-c720-433e-814a-2d3b23585b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:15.940962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f6d9c817-c720-433e-814a-2d3b23585b40 None None] MFA Rules not processed for user `c90e18da83904024b05a69cc2356fa85`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:15.942229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f6d9c817-c720-433e-814a-2d3b23585b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:16.015137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:16.037204 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4068/8045] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:15 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:18:16.048054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c57e9-cef1-4d0e-91b7-de871940adb5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:16.049944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6a9c57e9-cef1-4d0e-91b7-de871940adb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:16.113101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c57e9-cef1-4d0e-91b7-de871940adb5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:16.117449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f57603-4703-463a-b923-f9f0a397d7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:16.118078 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f57603-4703-463a-b923-f9f0a397d7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:16.118366 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f57603-4703-463a-b923-f9f0a397d7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:16.118646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f57603-4703-463a-b923-f9f0a397d7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:16.177014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f57603-4703-463a-b923-f9f0a397d7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:16.178477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f57603-4703-463a-b923-f9f0a397d7d1 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:16.188335 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3978/8046] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:19.904776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9c817-c720-433e-814a-2d3b23585b40 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:19.917689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9c817-c720-433e-814a-2d3b23585b40 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:19.923212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe45769-72bc-48d3-80cb-7d432810f36c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:19.923884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe45769-72bc-48d3-80cb-7d432810f36c service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:19.924272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe45769-72bc-48d3-80cb-7d432810f36c service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:19.924870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe45769-72bc-48d3-80cb-7d432810f36c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:19.943962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe45769-72bc-48d3-80cb-7d432810f36c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:19.946556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe45769-72bc-48d3-80cb-7d432810f36c service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:19.961937 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4069/8047] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:19 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:23.136095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f57603-4703-463a-b923-f9f0a397d7d1 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:23.222713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f57603-4703-463a-b923-f9f0a397d7d1 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:23.227498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:23.228141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:23.228428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:23.228707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] PATH_INFO: `/v3/users/7db236cd120a4b75a83292e99a6f2e33` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:23.229896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db236cd120a4b75a83292e99a6f2e33)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:23.232113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:23.234808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:23.282400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db236cd120a4b75a83292e99a6f2e33'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.296384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db236cd120a4b75a83292e99a6f2e33'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.297175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db236cd120a4b75a83292e99a6f2e33'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.305015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db236cd120a4b75a83292e99a6f2e33'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.313233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db236cd120a4b75a83292e99a6f2e33'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.320055 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3979/8048] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:23 2022] DELETE /identity/v3/users/7db236cd120a4b75a83292e99a6f2e33 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:23.324564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eda5fc-8fe0-4273-83d8-895936aadf17 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:23.325193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eda5fc-8fe0-4273-83d8-895936aadf17 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:23.325193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eda5fc-8fe0-4273-83d8-895936aadf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:23.344977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-91eda5fc-8fe0-4273-83d8-895936aadf17 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:23.346252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-91eda5fc-8fe0-4273-83d8-895936aadf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:23.412674 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4070/8049] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:23.423213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:23.424726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:23.487294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be83d8-f726-4327-9d48-14abdf2a6691 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:23.491815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50fa49-f213-47c1-be37-f4a4464f476f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:23.492448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50fa49-f213-47c1-be37-f4a4464f476f service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:23.492753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50fa49-f213-47c1-be37-f4a4464f476f service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:23.493066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50fa49-f213-47c1-be37-f4a4464f476f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:23.503075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50fa49-f213-47c1-be37-f4a4464f476f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:23.504582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50fa49-f213-47c1-be37-f4a4464f476f service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:23.514414 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3980/8050] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:23.693241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eda5fc-8fe0-4273-83d8-895936aadf17 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:23.694991 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-91eda5fc-8fe0-4273-83d8-895936aadf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:23.708431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eda5fc-8fe0-4273-83d8-895936aadf17 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:23.714157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:23.715283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:23.715283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:23.715523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] PATH_INFO: `/v3/projects/d9e491009ac74f10af26afbfe0e87181` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:23.716553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e491009ac74f10af26afbfe0e87181)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:23.719131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:23.769409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e491009ac74f10af26afbfe0e87181', 'request_id': 'req-d1e6088e-0387-40bd-bfa2-013004c67404'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.777774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e491009ac74f10af26afbfe0e87181', 'request_id': 'req-d1e6088e-0387-40bd-bfa2-013004c67404'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.787749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e491009ac74f10af26afbfe0e87181', 'request_id': 'req-d1e6088e-0387-40bd-bfa2-013004c67404'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.799589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] The token cache is being invalidate because project d9e491009ac74f10af26afbfe0e87181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:18:23.799937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.800624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.802147 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4071/8051] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:18:23 2022] DELETE /identity/v3/projects/d9e491009ac74f10af26afbfe0e87181 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:23.806428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50fa49-f213-47c1-be37-f4a4464f476f service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:23.816276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50fa49-f213-47c1-be37-f4a4464f476f service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:23.882826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f35d4-2f6b-42f2-9cc3-84604b05a3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:23.883554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f35d4-2f6b-42f2-9cc3-84604b05a3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:23.883911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f35d4-2f6b-42f2-9cc3-84604b05a3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:23.884261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f35d4-2f6b-42f2-9cc3-84604b05a3b3 admin admin] PATH_INFO: `/v3/users/c90e18da83904024b05a69cc2356fa85` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:23.885919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f35d4-2f6b-42f2-9cc3-84604b05a3b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c90e18da83904024b05a69cc2356fa85)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:23.887532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f35d4-2f6b-42f2-9cc3-84604b05a3b3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:23.921702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:23.950313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-251f35d4-2f6b-42f2-9cc3-84604b05a3b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c90e18da83904024b05a69cc2356fa85'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.972829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-251f35d4-2f6b-42f2-9cc3-84604b05a3b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c90e18da83904024b05a69cc2356fa85'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.974326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-251f35d4-2f6b-42f2-9cc3-84604b05a3b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c90e18da83904024b05a69cc2356fa85'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:23.991776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-251f35d4-2f6b-42f2-9cc3-84604b05a3b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c90e18da83904024b05a69cc2356fa85'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:24.013817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-251f35d4-2f6b-42f2-9cc3-84604b05a3b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c90e18da83904024b05a69cc2356fa85'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:24.023039 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3981/8052] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:23 2022] DELETE /identity/v3/users/c90e18da83904024b05a69cc2356fa85 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:24.358637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:24.361085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:24.378447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6088e-0387-40bd-bfa2-013004c67404 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:24.451472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ead0e-1878-464a-98d6-dbe3a7e08630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:24.452119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ead0e-1878-464a-98d6-dbe3a7e08630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:24.452457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ead0e-1878-464a-98d6-dbe3a7e08630 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:24.452943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ead0e-1878-464a-98d6-dbe3a7e08630 admin admin] PATH_INFO: `/v3/projects/04b8b86563ff4b6197e2a704895f4721` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:24.454552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ead0e-1878-464a-98d6-dbe3a7e08630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04b8b86563ff4b6197e2a704895f4721)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:24.456726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ead0e-1878-464a-98d6-dbe3a7e08630 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:24.465032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:24.522711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-623ead0e-1878-464a-98d6-dbe3a7e08630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04b8b86563ff4b6197e2a704895f4721', 'request_id': 'req-623ead0e-1878-464a-98d6-dbe3a7e08630'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:24.533186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-623ead0e-1878-464a-98d6-dbe3a7e08630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04b8b86563ff4b6197e2a704895f4721', 'request_id': 'req-623ead0e-1878-464a-98d6-dbe3a7e08630'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:24.541824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-623ead0e-1878-464a-98d6-dbe3a7e08630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04b8b86563ff4b6197e2a704895f4721', 'request_id': 'req-623ead0e-1878-464a-98d6-dbe3a7e08630'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:24.560053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-623ead0e-1878-464a-98d6-dbe3a7e08630 admin admin] The token cache is being invalidate because project 04b8b86563ff4b6197e2a704895f4721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:18:24.560503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-623ead0e-1878-464a-98d6-dbe3a7e08630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:24.561611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-623ead0e-1878-464a-98d6-dbe3a7e08630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:24.563948 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4072/8053] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:18:24 2022] DELETE /identity/v3/projects/04b8b86563ff4b6197e2a704895f4721 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:24.568705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966af804-ee16-4a32-a266-eefc74f1622d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:24.569838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966af804-ee16-4a32-a266-eefc74f1622d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:24.571003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966af804-ee16-4a32-a266-eefc74f1622d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:24.593189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-966af804-ee16-4a32-a266-eefc74f1622d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:24.595547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-966af804-ee16-4a32-a266-eefc74f1622d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:24.697119 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3982/8054] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:24.792524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd41d3-9281-41e9-bd1a-a335e5555153 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:24.792815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd41d3-9281-41e9-bd1a-a335e5555153 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:24.793129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd41d3-9281-41e9-bd1a-a335e5555153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:24.818209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-01fd41d3-9281-41e9-bd1a-a335e5555153 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:24.820016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-01fd41d3-9281-41e9-bd1a-a335e5555153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:24.854231 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4073/8055] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:24.859092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966af804-ee16-4a32-a266-eefc74f1622d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:24.869455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966af804-ee16-4a32-a266-eefc74f1622d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:24.874042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305172e1-9872-493d-92d0-dd18956c2291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:24.874643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305172e1-9872-493d-92d0-dd18956c2291 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:24.874929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305172e1-9872-493d-92d0-dd18956c2291 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:24.875212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305172e1-9872-493d-92d0-dd18956c2291 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:24.875874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305172e1-9872-493d-92d0-dd18956c2291 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:18:24.876178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305172e1-9872-493d-92d0-dd18956c2291 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:24.879735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305172e1-9872-493d-92d0-dd18956c2291 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:24.903072 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3983/8056] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:18:24.907194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd41d3-9281-41e9-bd1a-a335e5555153 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:24.919665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd41d3-9281-41e9-bd1a-a335e5555153 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:24.925165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ed92d-5aa5-46c7-89bb-24d29decea43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:24.925862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ed92d-5aa5-46c7-89bb-24d29decea43 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:24.926277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ed92d-5aa5-46c7-89bb-24d29decea43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:24.926630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ed92d-5aa5-46c7-89bb-24d29decea43 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:24.927324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ed92d-5aa5-46c7-89bb-24d29decea43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:24.930640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ed92d-5aa5-46c7-89bb-24d29decea43 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:24.971405 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4074/8057] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:24 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:24.976135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305172e1-9872-493d-92d0-dd18956c2291 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:24.989225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305172e1-9872-493d-92d0-dd18956c2291 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:25.053331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3da4c9-831b-4a76-bc6a-627b4885805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:25.054540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3da4c9-831b-4a76-bc6a-627b4885805d admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:25.054938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3da4c9-831b-4a76-bc6a-627b4885805d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:25.055287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3da4c9-831b-4a76-bc6a-627b4885805d admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:25.056061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3da4c9-831b-4a76-bc6a-627b4885805d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:25.058035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3da4c9-831b-4a76-bc6a-627b4885805d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:25.082468 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3984/8058] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:24 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:25.086334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ed92d-5aa5-46c7-89bb-24d29decea43 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:25.097310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ed92d-5aa5-46c7-89bb-24d29decea43 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:25.103270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321cc7f-5dcb-4906-bd57-99e8d3ef8f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:25.103980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321cc7f-5dcb-4906-bd57-99e8d3ef8f32 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:25.104345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321cc7f-5dcb-4906-bd57-99e8d3ef8f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:25.104746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321cc7f-5dcb-4906-bd57-99e8d3ef8f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:25.105552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321cc7f-5dcb-4906-bd57-99e8d3ef8f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:25.107504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321cc7f-5dcb-4906-bd57-99e8d3ef8f32 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:25.119454 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4075/8059] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:25.124852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3da4c9-831b-4a76-bc6a-627b4885805d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:25.134942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3da4c9-831b-4a76-bc6a-627b4885805d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:25.141234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9858e-3040-44a8-b86b-bb3017d1a72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:25.141879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9858e-3040-44a8-b86b-bb3017d1a72b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:25.142165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9858e-3040-44a8-b86b-bb3017d1a72b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:25.142446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9858e-3040-44a8-b86b-bb3017d1a72b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:25.143130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9858e-3040-44a8-b86b-bb3017d1a72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:25.144831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9858e-3040-44a8-b86b-bb3017d1a72b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:25.156762 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3985/8060] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:25.161061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321cc7f-5dcb-4906-bd57-99e8d3ef8f32 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:25.178449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321cc7f-5dcb-4906-bd57-99e8d3ef8f32 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:25.184252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baadf53-dbaf-4ad2-b6c1-483a4271ee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:25.184865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baadf53-dbaf-4ad2-b6c1-483a4271ee74 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:25.185249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baadf53-dbaf-4ad2-b6c1-483a4271ee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:25.185606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baadf53-dbaf-4ad2-b6c1-483a4271ee74 admin admin] PATH_INFO: `/v3/projects/59a58f5ad18647c18b7d875ff0af8c30/users/bdec6b5929794265b7a2d1ba6c44b44e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:25.201725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baadf53-dbaf-4ad2-b6c1-483a4271ee74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a58f5ad18647c18b7d875ff0af8c30, user_id=bdec6b5929794265b7a2d1ba6c44b44e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:25.204395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baadf53-dbaf-4ad2-b6c1-483a4271ee74 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:25.218169 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4076/8061] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:25 2022] PUT /identity/v3/projects/59a58f5ad18647c18b7d875ff0af8c30/users/bdec6b5929794265b7a2d1ba6c44b44e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:25.228695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4feffc-2116-42f2-8098-6bef1e5976f5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:25.229181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4feffc-2116-42f2-8098-6bef1e5976f5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:25.229464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4feffc-2116-42f2-8098-6bef1e5976f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:25.250614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:25.261714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3d4feffc-2116-42f2-8098-6bef1e5976f5 None None] MFA Rules not processed for user `bdec6b5929794265b7a2d1ba6c44b44e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:25.263446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3d4feffc-2116-42f2-8098-6bef1e5976f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:25.346254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baadf53-dbaf-4ad2-b6c1-483a4271ee74 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:25.346665 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3986/8062] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:25.421021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baadf53-dbaf-4ad2-b6c1-483a4271ee74 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:25.426178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18def581-f93f-45a1-baaa-cd259ac38b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:25.426804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18def581-f93f-45a1-baaa-cd259ac38b6a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:25.427089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18def581-f93f-45a1-baaa-cd259ac38b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:25.427367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18def581-f93f-45a1-baaa-cd259ac38b6a admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:25.428026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18def581-f93f-45a1-baaa-cd259ac38b6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:25.430250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18def581-f93f-45a1-baaa-cd259ac38b6a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:25.464559 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4077/8063] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:25 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:25.465806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4feffc-2116-42f2-8098-6bef1e5976f5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:25.477911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4feffc-2116-42f2-8098-6bef1e5976f5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:25.556012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba7c59-6c06-4a00-b6c0-fc861f5d0f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:25.556782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba7c59-6c06-4a00-b6c0-fc861f5d0f35 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:25.557170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba7c59-6c06-4a00-b6c0-fc861f5d0f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:25.557540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba7c59-6c06-4a00-b6c0-fc861f5d0f35 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:25.558279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba7c59-6c06-4a00-b6c0-fc861f5d0f35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:25.560415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba7c59-6c06-4a00-b6c0-fc861f5d0f35 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:25.584244 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3987/8064] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:25 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:25.589630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18def581-f93f-45a1-baaa-cd259ac38b6a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:25.602960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18def581-f93f-45a1-baaa-cd259ac38b6a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:25.608208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bc7f9-1433-4604-a75f-d663c343c1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:25.608855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bc7f9-1433-4604-a75f-d663c343c1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:25.609166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bc7f9-1433-4604-a75f-d663c343c1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:25.609445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bc7f9-1433-4604-a75f-d663c343c1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:25.610092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bc7f9-1433-4604-a75f-d663c343c1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:25.611708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bc7f9-1433-4604-a75f-d663c343c1e6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:25.630179 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4078/8065] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:18:25.634987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba7c59-6c06-4a00-b6c0-fc861f5d0f35 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:25.646006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba7c59-6c06-4a00-b6c0-fc861f5d0f35 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:25.651996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176ba5e-b1a2-4883-a663-6e8db4050fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:25.652761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ba5e-b1a2-4883-a663-6e8db4050fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:25.653153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ba5e-b1a2-4883-a663-6e8db4050fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:25.653508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ba5e-b1a2-4883-a663-6e8db4050fc4 admin admin] PATH_INFO: `/v3/projects/18b10fffd3e24fcdb8f029c6e88c7c9a/users/dd21454527dd4525b69f0d6ee8a4104a/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:25.671882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176ba5e-b1a2-4883-a663-6e8db4050fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b10fffd3e24fcdb8f029c6e88c7c9a, user_id=dd21454527dd4525b69f0d6ee8a4104a, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:25.674669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176ba5e-b1a2-4883-a663-6e8db4050fc4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:25.703958 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3988/8066] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:25 2022] PUT /identity/v3/projects/18b10fffd3e24fcdb8f029c6e88c7c9a/users/dd21454527dd4525b69f0d6ee8a4104a/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:25.705221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bc7f9-1433-4604-a75f-d663c343c1e6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:25.711637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:25.718847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bc7f9-1433-4604-a75f-d663c343c1e6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:25.803849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ab12d-749d-43bc-8c73-16e7bbfede0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:25.809131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ab12d-749d-43bc-8c73-16e7bbfede0c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:25.809342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ab12d-749d-43bc-8c73-16e7bbfede0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:25.809740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ab12d-749d-43bc-8c73-16e7bbfede0c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:25.810603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ab12d-749d-43bc-8c73-16e7bbfede0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:25.814001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ab12d-749d-43bc-8c73-16e7bbfede0c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:25.827980 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4079/8067] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:25.833892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176ba5e-b1a2-4883-a663-6e8db4050fc4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:25.849752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176ba5e-b1a2-4883-a663-6e8db4050fc4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:25.855047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea2aa9-b710-49f5-b211-274636c0feb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:25.856062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea2aa9-b710-49f5-b211-274636c0feb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:25.856437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea2aa9-b710-49f5-b211-274636c0feb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:25.856796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea2aa9-b710-49f5-b211-274636c0feb2 admin admin] PATH_INFO: `/v3/projects/18b10fffd3e24fcdb8f029c6e88c7c9a/users/dd21454527dd4525b69f0d6ee8a4104a/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:25.858937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea2aa9-b710-49f5-b211-274636c0feb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b10fffd3e24fcdb8f029c6e88c7c9a, user_id=dd21454527dd4525b69f0d6ee8a4104a, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:25.861776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea2aa9-b710-49f5-b211-274636c0feb2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:25.886885 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3989/8068] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:25 2022] PUT /identity/v3/projects/18b10fffd3e24fcdb8f029c6e88c7c9a/users/dd21454527dd4525b69f0d6ee8a4104a/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:25.888944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f031dd-a8a7-4f48-a9c6-1070fbfe4591 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:25.889240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f031dd-a8a7-4f48-a9c6-1070fbfe4591 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:25.889522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f031dd-a8a7-4f48-a9c6-1070fbfe4591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:25.925148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-32f031dd-a8a7-4f48-a9c6-1070fbfe4591 None None] MFA Rules not processed for user `dd21454527dd4525b69f0d6ee8a4104a`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:25.926765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-32f031dd-a8a7-4f48-a9c6-1070fbfe4591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:26.038823 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4080/8069] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:26.080466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea2aa9-b710-49f5-b211-274636c0feb2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:26.083040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-72ea2aa9-b710-49f5-b211-274636c0feb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:26.180428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea2aa9-b710-49f5-b211-274636c0feb2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:26.185479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d5dbe-448f-4c9f-b291-970583492860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:26.186105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d5dbe-448f-4c9f-b291-970583492860 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:26.186391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d5dbe-448f-4c9f-b291-970583492860 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:26.186790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d5dbe-448f-4c9f-b291-970583492860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:26.193303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:26.264271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d5dbe-448f-4c9f-b291-970583492860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:26.265812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d5dbe-448f-4c9f-b291-970583492860 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:26.275878 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3990/8070] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:26.933367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f031dd-a8a7-4f48-a9c6-1070fbfe4591 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:26.937243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-32f031dd-a8a7-4f48-a9c6-1070fbfe4591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:26.964601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f031dd-a8a7-4f48-a9c6-1070fbfe4591 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:26.969544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dd9c8-67cd-4ad3-b116-b48883edb4af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:26.970191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dd9c8-67cd-4ad3-b116-b48883edb4af service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:26.970630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dd9c8-67cd-4ad3-b116-b48883edb4af service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:26.970924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dd9c8-67cd-4ad3-b116-b48883edb4af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:26.985088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dd9c8-67cd-4ad3-b116-b48883edb4af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:26.987343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dd9c8-67cd-4ad3-b116-b48883edb4af service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:27.014231 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4081/8071] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:35.438817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d5dbe-448f-4c9f-b291-970583492860 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:35.533037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d5dbe-448f-4c9f-b291-970583492860 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:35.538343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:35.539091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:35.539407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:35.539700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] PATH_INFO: `/v3/users/bdec6b5929794265b7a2d1ba6c44b44e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:35.540858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdec6b5929794265b7a2d1ba6c44b44e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:35.542831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:35.605343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdec6b5929794265b7a2d1ba6c44b44e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:35.620285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdec6b5929794265b7a2d1ba6c44b44e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:35.621252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdec6b5929794265b7a2d1ba6c44b44e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:35.631327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdec6b5929794265b7a2d1ba6c44b44e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:35.641792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdec6b5929794265b7a2d1ba6c44b44e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:35.649261 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3991/8072] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:35 2022] DELETE /identity/v3/users/bdec6b5929794265b7a2d1ba6c44b44e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:35.653893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bc807-cfde-41f4-b69f-03bf2a8774ec None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:35.654386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bc807-cfde-41f4-b69f-03bf2a8774ec None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:35.654550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bc807-cfde-41f4-b69f-03bf2a8774ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:35.675756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a79bc807-cfde-41f4-b69f-03bf2a8774ec None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:35.677659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a79bc807-cfde-41f4-b69f-03bf2a8774ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:35.764043 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4082/8073] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:18:35.774776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:35.779157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:35.818546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:35.865610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2245b4-d16d-4e4b-aaa4-6857ab12f8d2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:35.870076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df01c8-f5e9-41ab-8d87-82dfe66b6f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:35.870797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df01c8-f5e9-41ab-8d87-82dfe66b6f82 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:35.871152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df01c8-f5e9-41ab-8d87-82dfe66b6f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:35.871511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df01c8-f5e9-41ab-8d87-82dfe66b6f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:35.882530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df01c8-f5e9-41ab-8d87-82dfe66b6f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:35.884284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df01c8-f5e9-41ab-8d87-82dfe66b6f82 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:35.896187 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3992/8074] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:36.136095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bc807-cfde-41f4-b69f-03bf2a8774ec None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:36.137570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a79bc807-cfde-41f4-b69f-03bf2a8774ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:36.167386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bc807-cfde-41f4-b69f-03bf2a8774ec None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:36.172598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:36.173306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:36.173664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:36.176631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] PATH_INFO: `/v3/projects/59a58f5ad18647c18b7d875ff0af8c30` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:36.176631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a58f5ad18647c18b7d875ff0af8c30)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:36.180656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:36.189241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:36.259143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a58f5ad18647c18b7d875ff0af8c30', 'request_id': 'req-75d37d46-fcc3-4dcd-8998-e5a6273382ee'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.266787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a58f5ad18647c18b7d875ff0af8c30', 'request_id': 'req-75d37d46-fcc3-4dcd-8998-e5a6273382ee'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.278517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a58f5ad18647c18b7d875ff0af8c30', 'request_id': 'req-75d37d46-fcc3-4dcd-8998-e5a6273382ee'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.306060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] The token cache is being invalidate because project 59a58f5ad18647c18b7d875ff0af8c30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:18:36.306369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.307119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.309489 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4083/8075] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:18:36 2022] DELETE /identity/v3/projects/59a58f5ad18647c18b7d875ff0af8c30 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:36.313089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df01c8-f5e9-41ab-8d87-82dfe66b6f82 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:36.323609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df01c8-f5e9-41ab-8d87-82dfe66b6f82 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:36.403797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37bbcd-f276-478d-91d7-75cc963d17d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:36.404765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37bbcd-f276-478d-91d7-75cc963d17d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:36.405088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37bbcd-f276-478d-91d7-75cc963d17d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:36.405371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37bbcd-f276-478d-91d7-75cc963d17d5 admin admin] PATH_INFO: `/v3/users/dd21454527dd4525b69f0d6ee8a4104a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:36.406680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37bbcd-f276-478d-91d7-75cc963d17d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd21454527dd4525b69f0d6ee8a4104a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:36.408754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37bbcd-f276-478d-91d7-75cc963d17d5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:36.468138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-de37bbcd-f276-478d-91d7-75cc963d17d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd21454527dd4525b69f0d6ee8a4104a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.484597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-de37bbcd-f276-478d-91d7-75cc963d17d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd21454527dd4525b69f0d6ee8a4104a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.485348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-de37bbcd-f276-478d-91d7-75cc963d17d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd21454527dd4525b69f0d6ee8a4104a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.493721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-de37bbcd-f276-478d-91d7-75cc963d17d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd21454527dd4525b69f0d6ee8a4104a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.504942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-de37bbcd-f276-478d-91d7-75cc963d17d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd21454527dd4525b69f0d6ee8a4104a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.519123 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3993/8076] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:36 2022] DELETE /identity/v3/users/dd21454527dd4525b69f0d6ee8a4104a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:36.718760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:36.720459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:36.740467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d37d46-fcc3-4dcd-8998-e5a6273382ee admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:36.805241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a82991-8b4b-4ea8-8037-f609578e9c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:36.805998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a82991-8b4b-4ea8-8037-f609578e9c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:36.806350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a82991-8b4b-4ea8-8037-f609578e9c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:36.806704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a82991-8b4b-4ea8-8037-f609578e9c37 admin admin] PATH_INFO: `/v3/projects/18b10fffd3e24fcdb8f029c6e88c7c9a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:36.808149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a82991-8b4b-4ea8-8037-f609578e9c37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b10fffd3e24fcdb8f029c6e88c7c9a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:36.810295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a82991-8b4b-4ea8-8037-f609578e9c37 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:36.866805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-97a82991-8b4b-4ea8-8037-f609578e9c37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b10fffd3e24fcdb8f029c6e88c7c9a', 'request_id': 'req-97a82991-8b4b-4ea8-8037-f609578e9c37'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.875059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-97a82991-8b4b-4ea8-8037-f609578e9c37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b10fffd3e24fcdb8f029c6e88c7c9a', 'request_id': 'req-97a82991-8b4b-4ea8-8037-f609578e9c37'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.883063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-97a82991-8b4b-4ea8-8037-f609578e9c37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b10fffd3e24fcdb8f029c6e88c7c9a', 'request_id': 'req-97a82991-8b4b-4ea8-8037-f609578e9c37'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.894940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-97a82991-8b4b-4ea8-8037-f609578e9c37 admin admin] The token cache is being invalidate because project 18b10fffd3e24fcdb8f029c6e88c7c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:18:36.895243 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-97a82991-8b4b-4ea8-8037-f609578e9c37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.895913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-97a82991-8b4b-4ea8-8037-f609578e9c37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:36.898040 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4084/8077] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:18:36 2022] DELETE /identity/v3/projects/18b10fffd3e24fcdb8f029c6e88c7c9a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:36.902799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757a23c-ec2d-41de-86f3-b0aea763ce85 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:36.903103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757a23c-ec2d-41de-86f3-b0aea763ce85 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:36.903389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757a23c-ec2d-41de-86f3-b0aea763ce85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:36.923041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6757a23c-ec2d-41de-86f3-b0aea763ce85 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:36.924251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6757a23c-ec2d-41de-86f3-b0aea763ce85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:36.968831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:36.990540 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3994/8078] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:36.998281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d822a6-5aa3-44f1-bbdf-c6ff04fba42f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:36.998578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d822a6-5aa3-44f1-bbdf-c6ff04fba42f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:36.998864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d822a6-5aa3-44f1-bbdf-c6ff04fba42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.019751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-25d822a6-5aa3-44f1-bbdf-c6ff04fba42f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:37.021017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-25d822a6-5aa3-44f1-bbdf-c6ff04fba42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:37.032107 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4085/8079] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:37.036574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757a23c-ec2d-41de-86f3-b0aea763ce85 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.046845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757a23c-ec2d-41de-86f3-b0aea763ce85 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.052011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22715b0b-36ac-45af-b13f-9cf5b522950d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.052700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22715b0b-36ac-45af-b13f-9cf5b522950d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.053093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22715b0b-36ac-45af-b13f-9cf5b522950d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.053719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22715b0b-36ac-45af-b13f-9cf5b522950d admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.054352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22715b0b-36ac-45af-b13f-9cf5b522950d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:18:37.054669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22715b0b-36ac-45af-b13f-9cf5b522950d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.056281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22715b0b-36ac-45af-b13f-9cf5b522950d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.068731 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3995/8080] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:18:37.073528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d822a6-5aa3-44f1-bbdf-c6ff04fba42f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.082813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d822a6-5aa3-44f1-bbdf-c6ff04fba42f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.088135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d1e6f-a9c6-4339-ad74-ccbd0e577b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.088738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d1e6f-a9c6-4339-ad74-ccbd0e577b4e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.089064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d1e6f-a9c6-4339-ad74-ccbd0e577b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.089344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d1e6f-a9c6-4339-ad74-ccbd0e577b4e admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.089994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d1e6f-a9c6-4339-ad74-ccbd0e577b4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.091916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d1e6f-a9c6-4339-ad74-ccbd0e577b4e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.113179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:37.132449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22715b0b-36ac-45af-b13f-9cf5b522950d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.136958 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4086/8081] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:37 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:37.144241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22715b0b-36ac-45af-b13f-9cf5b522950d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.203920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787e691-4f39-4a7b-bfbc-fe883e05bd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.204556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787e691-4f39-4a7b-bfbc-fe883e05bd1c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.204829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787e691-4f39-4a7b-bfbc-fe883e05bd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.205137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787e691-4f39-4a7b-bfbc-fe883e05bd1c admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.205782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787e691-4f39-4a7b-bfbc-fe883e05bd1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.207699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787e691-4f39-4a7b-bfbc-fe883e05bd1c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.232459 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3996/8082] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:37 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:37.236699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d1e6f-a9c6-4339-ad74-ccbd0e577b4e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.249000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d1e6f-a9c6-4339-ad74-ccbd0e577b4e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.253855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea86d41-873e-4ff7-8333-7363fd7c4000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.255154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea86d41-873e-4ff7-8333-7363fd7c4000 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.255446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea86d41-873e-4ff7-8333-7363fd7c4000 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.255728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea86d41-873e-4ff7-8333-7363fd7c4000 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.256400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea86d41-873e-4ff7-8333-7363fd7c4000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.258723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea86d41-873e-4ff7-8333-7363fd7c4000 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.272575 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4087/8083] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:18:37.273862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787e691-4f39-4a7b-bfbc-fe883e05bd1c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.284693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787e691-4f39-4a7b-bfbc-fe883e05bd1c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.290970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c345d-8af3-4552-8161-a58eb59ae0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.291614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c345d-8af3-4552-8161-a58eb59ae0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.291909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c345d-8af3-4552-8161-a58eb59ae0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.292192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c345d-8af3-4552-8161-a58eb59ae0e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.292853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c345d-8af3-4552-8161-a58eb59ae0e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.294878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c345d-8af3-4552-8161-a58eb59ae0e3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.308501 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3997/8084] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:18:37.312237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea86d41-873e-4ff7-8333-7363fd7c4000 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.323442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea86d41-873e-4ff7-8333-7363fd7c4000 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.328440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec6366-f969-4de4-b139-41214b394f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.329227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec6366-f969-4de4-b139-41214b394f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.329514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec6366-f969-4de4-b139-41214b394f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.329791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec6366-f969-4de4-b139-41214b394f84 admin admin] PATH_INFO: `/v3/projects/4fa173ce8042421a9dd704632c1a12ae/users/2c941fe0dbc64880b3c4a421cfdae0ee/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.345925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec6366-f969-4de4-b139-41214b394f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa173ce8042421a9dd704632c1a12ae, user_id=2c941fe0dbc64880b3c4a421cfdae0ee, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.348652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec6366-f969-4de4-b139-41214b394f84 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.363592 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4088/8085] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:37 2022] PUT /identity/v3/projects/4fa173ce8042421a9dd704632c1a12ae/users/2c941fe0dbc64880b3c4a421cfdae0ee/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:18:37.374716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e100ab3-aff6-41df-87e9-315317c1e870 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.375104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e100ab3-aff6-41df-87e9-315317c1e870 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.375460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e100ab3-aff6-41df-87e9-315317c1e870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.405151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0e100ab3-aff6-41df-87e9-315317c1e870 None None] MFA Rules not processed for user `2c941fe0dbc64880b3c4a421cfdae0ee`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:37.406511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0e100ab3-aff6-41df-87e9-315317c1e870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:37.469900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec6366-f969-4de4-b139-41214b394f84 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.472655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-09ec6366-f969-4de4-b139-41214b394f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:37.476475 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3998/8086] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:37.481861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e100ab3-aff6-41df-87e9-315317c1e870 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.527294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:37.541503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec6366-f969-4de4-b139-41214b394f84 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.546941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bac749-f7a1-4fbd-9987-264ebf0ad0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.547146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e100ab3-aff6-41df-87e9-315317c1e870 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.548022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bac749-f7a1-4fbd-9987-264ebf0ad0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.548465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bac749-f7a1-4fbd-9987-264ebf0ad0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.548755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bac749-f7a1-4fbd-9987-264ebf0ad0b6 admin admin] PATH_INFO: `/v3/users/41d806d2ab5b469faa8f5c619b7b33a3` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.549925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bac749-f7a1-4fbd-9987-264ebf0ad0b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d806d2ab5b469faa8f5c619b7b33a3)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.551909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bac749-f7a1-4fbd-9987-264ebf0ad0b6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.552291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dc122-6211-4a49-9b6e-af79eb6b2998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.552948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dc122-6211-4a49-9b6e-af79eb6b2998 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.553218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dc122-6211-4a49-9b6e-af79eb6b2998 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.554009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dc122-6211-4a49-9b6e-af79eb6b2998 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.554647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dc122-6211-4a49-9b6e-af79eb6b2998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.556549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dc122-6211-4a49-9b6e-af79eb6b2998 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.586678 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 3999/8087] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:37 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:37.590912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dc122-6211-4a49-9b6e-af79eb6b2998 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.601828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2bac749-f7a1-4fbd-9987-264ebf0ad0b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d806d2ab5b469faa8f5c619b7b33a3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:37.614826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2bac749-f7a1-4fbd-9987-264ebf0ad0b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d806d2ab5b469faa8f5c619b7b33a3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:37.624129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2bac749-f7a1-4fbd-9987-264ebf0ad0b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d806d2ab5b469faa8f5c619b7b33a3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:37.624789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2bac749-f7a1-4fbd-9987-264ebf0ad0b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d806d2ab5b469faa8f5c619b7b33a3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:37.625542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d2bac749-f7a1-4fbd-9987-264ebf0ad0b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d806d2ab5b469faa8f5c619b7b33a3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:37.638153 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4089/8088] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:37 2022] DELETE /identity/v3/users/41d806d2ab5b469faa8f5c619b7b33a3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:37.642509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115aa96-98fc-4f78-b207-8768b1fedaee None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.642806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115aa96-98fc-4f78-b207-8768b1fedaee None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.643089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115aa96-98fc-4f78-b207-8768b1fedaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.655959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dc122-6211-4a49-9b6e-af79eb6b2998 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.662357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e115aa96-98fc-4f78-b207-8768b1fedaee None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:37.664065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e115aa96-98fc-4f78-b207-8768b1fedaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:37.697058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:37.714244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9109aa4-58fb-4d24-aa10-4c061f1e423a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.714787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9109aa4-58fb-4d24-aa10-4c061f1e423a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.715075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9109aa4-58fb-4d24-aa10-4c061f1e423a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.715347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9109aa4-58fb-4d24-aa10-4c061f1e423a admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.715970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9109aa4-58fb-4d24-aa10-4c061f1e423a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.717923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9109aa4-58fb-4d24-aa10-4c061f1e423a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.725127 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4090/8089] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:18:37.735312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115aa96-98fc-4f78-b207-8768b1fedaee None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.737013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e115aa96-98fc-4f78-b207-8768b1fedaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:37.742073 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4000/8090] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:37 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:37.745813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9109aa4-58fb-4d24-aa10-4c061f1e423a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.747908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f9109aa4-58fb-4d24-aa10-4c061f1e423a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:37.760364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9109aa4-58fb-4d24-aa10-4c061f1e423a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.764749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9a202-2a6a-4bd9-b3bc-a4d691031695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.765384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9a202-2a6a-4bd9-b3bc-a4d691031695 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.765674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9a202-2a6a-4bd9-b3bc-a4d691031695 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.765956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9a202-2a6a-4bd9-b3bc-a4d691031695 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.766593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9a202-2a6a-4bd9-b3bc-a4d691031695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.768178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9a202-2a6a-4bd9-b3bc-a4d691031695 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.778315 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4001/8091] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:37.782179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9a202-2a6a-4bd9-b3bc-a4d691031695 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.792353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9a202-2a6a-4bd9-b3bc-a4d691031695 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.797069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701bc49-1e7f-4949-8d00-4c193cf06b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.797684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701bc49-1e7f-4949-8d00-4c193cf06b8d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.797969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701bc49-1e7f-4949-8d00-4c193cf06b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.798267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701bc49-1e7f-4949-8d00-4c193cf06b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.798928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701bc49-1e7f-4949-8d00-4c193cf06b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.800501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701bc49-1e7f-4949-8d00-4c193cf06b8d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.801594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115aa96-98fc-4f78-b207-8768b1fedaee None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.806070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56743202-3129-4da9-adb1-9448997adf9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.806708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56743202-3129-4da9-adb1-9448997adf9a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.806994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56743202-3129-4da9-adb1-9448997adf9a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.807272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56743202-3129-4da9-adb1-9448997adf9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.811498 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4002/8092] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:18:37.818830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701bc49-1e7f-4949-8d00-4c193cf06b8d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:37.822093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56743202-3129-4da9-adb1-9448997adf9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.823559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56743202-3129-4da9-adb1-9448997adf9a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.828648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701bc49-1e7f-4949-8d00-4c193cf06b8d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:37.833456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4977805-5145-457d-8c4e-b84cb44ea849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:37.833623 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4091/8093] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:18:37.834105 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4977805-5145-457d-8c4e-b84cb44ea849 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.834389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4977805-5145-457d-8c4e-b84cb44ea849 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.834706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4977805-5145-457d-8c4e-b84cb44ea849 admin admin] PATH_INFO: `/v3/projects/dafaee1fc8e84454ba30c1f73b4a562e/users/9aae9a894187433da81d68eaf483df3e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.849415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4977805-5145-457d-8c4e-b84cb44ea849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dafaee1fc8e84454ba30c1f73b4a562e, user_id=9aae9a894187433da81d68eaf483df3e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:37.852602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4977805-5145-457d-8c4e-b84cb44ea849 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:37.868314 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4003/8094] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:37 2022] PUT /identity/v3/projects/dafaee1fc8e84454ba30c1f73b4a562e/users/9aae9a894187433da81d68eaf483df3e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:18:37.882536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3eb4a-87db-4809-b62c-3e5354d46f9d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:37.882834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3eb4a-87db-4809-b62c-3e5354d46f9d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:37.883145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3eb4a-87db-4809-b62c-3e5354d46f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:37.911063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-07c3eb4a-87db-4809-b62c-3e5354d46f9d None None] MFA Rules not processed for user `9aae9a894187433da81d68eaf483df3e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:37.912344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-07c3eb4a-87db-4809-b62c-3e5354d46f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:37.983986 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4092/8095] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:37.989072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4977805-5145-457d-8c4e-b84cb44ea849 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.014433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:38.021814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3eb4a-87db-4809-b62c-3e5354d46f9d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.024156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-07c3eb4a-87db-4809-b62c-3e5354d46f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:38.080602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4977805-5145-457d-8c4e-b84cb44ea849 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.085927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d86496-390d-491a-b24f-e445f4c4aa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.086539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d86496-390d-491a-b24f-e445f4c4aa13 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.086825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d86496-390d-491a-b24f-e445f4c4aa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.087102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d86496-390d-491a-b24f-e445f4c4aa13 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.087728 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d86496-390d-491a-b24f-e445f4c4aa13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.089675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d86496-390d-491a-b24f-e445f4c4aa13 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.105980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3eb4a-87db-4809-b62c-3e5354d46f9d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.112404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058c385-e850-46ac-a052-7688b964c70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.113332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058c385-e850-46ac-a052-7688b964c70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.113807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058c385-e850-46ac-a052-7688b964c70d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.114463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058c385-e850-46ac-a052-7688b964c70d admin admin] PATH_INFO: `/v3/projects/d39308a9753f4ee297ea40e4faf33923` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.115693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058c385-e850-46ac-a052-7688b964c70d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d39308a9753f4ee297ea40e4faf33923)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.122841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058c385-e850-46ac-a052-7688b964c70d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.131532 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4004/8096] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:37 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:38.135604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d86496-390d-491a-b24f-e445f4c4aa13 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.138149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:38.173615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3058c385-e850-46ac-a052-7688b964c70d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd39308a9753f4ee297ea40e4faf33923', 'request_id': 'req-3058c385-e850-46ac-a052-7688b964c70d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:38.181969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3058c385-e850-46ac-a052-7688b964c70d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd39308a9753f4ee297ea40e4faf33923', 'request_id': 'req-3058c385-e850-46ac-a052-7688b964c70d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:38.190198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3058c385-e850-46ac-a052-7688b964c70d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd39308a9753f4ee297ea40e4faf33923', 'request_id': 'req-3058c385-e850-46ac-a052-7688b964c70d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:38.201744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3058c385-e850-46ac-a052-7688b964c70d admin admin] The token cache is being invalidate because project d39308a9753f4ee297ea40e4faf33923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:18:38.202171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3058c385-e850-46ac-a052-7688b964c70d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:38.202944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3058c385-e850-46ac-a052-7688b964c70d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:38.204533 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4093/8097] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:18:38 2022] DELETE /identity/v3/projects/d39308a9753f4ee297ea40e4faf33923 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:18:38.207421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d86496-390d-491a-b24f-e445f4c4aa13 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.209232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4821bb-e6f0-458e-bca1-8c2fe71bd0e3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.210156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4821bb-e6f0-458e-bca1-8c2fe71bd0e3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.210509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4821bb-e6f0-458e-bca1-8c2fe71bd0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.232136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-dd4821bb-e6f0-458e-bca1-8c2fe71bd0e3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:38.233324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dd4821bb-e6f0-458e-bca1-8c2fe71bd0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:38.266261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af9885-f913-493e-aad7-6291abd6a2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.267018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af9885-f913-493e-aad7-6291abd6a2ce admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.267307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af9885-f913-493e-aad7-6291abd6a2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.267586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af9885-f913-493e-aad7-6291abd6a2ce admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.268212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af9885-f913-493e-aad7-6291abd6a2ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.270210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af9885-f913-493e-aad7-6291abd6a2ce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.292697 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4005/8098] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:38 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:38.293493 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4094/8099] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:38.296971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af9885-f913-493e-aad7-6291abd6a2ce admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.298664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-77af9885-f913-493e-aad7-6291abd6a2ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:38.300760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea9b77-81f9-49c6-99e5-5824bbee14ea None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.301091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea9b77-81f9-49c6-99e5-5824bbee14ea None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.301377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea9b77-81f9-49c6-99e5-5824bbee14ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.311014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af9885-f913-493e-aad7-6291abd6a2ce admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.315912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b7fa9-7ae4-4c38-81a4-95046feb144b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.316520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b7fa9-7ae4-4c38-81a4-95046feb144b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.316807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b7fa9-7ae4-4c38-81a4-95046feb144b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.317251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b7fa9-7ae4-4c38-81a4-95046feb144b admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.317924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b7fa9-7ae4-4c38-81a4-95046feb144b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.319546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b7fa9-7ae4-4c38-81a4-95046feb144b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.320534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-beea9b77-81f9-49c6-99e5-5824bbee14ea None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:38.321768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-beea9b77-81f9-49c6-99e5-5824bbee14ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:38.329661 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4006/8100] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:18:38.329797 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4095/8101] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:18:38.333871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b7fa9-7ae4-4c38-81a4-95046feb144b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.334014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea9b77-81f9-49c6-99e5-5824bbee14ea None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.344339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b7fa9-7ae4-4c38-81a4-95046feb144b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.344490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea9b77-81f9-49c6-99e5-5824bbee14ea None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.349680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073e518-9d99-42cb-aa1c-5e4fdf48c9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.349821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88332819-2733-4f7d-a6c1-43947f367f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.350893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073e518-9d99-42cb-aa1c-5e4fdf48c9fe admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.351025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88332819-2733-4f7d-a6c1-43947f367f03 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.351190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88332819-2733-4f7d-a6c1-43947f367f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.351316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073e518-9d99-42cb-aa1c-5e4fdf48c9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.351465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88332819-2733-4f7d-a6c1-43947f367f03 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.351570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073e518-9d99-42cb-aa1c-5e4fdf48c9fe admin admin] PATH_INFO: `/v3/projects/b40a1aa488a343c8be4cf2e839c02ed4/users/53cc0d61301b429b9b3d25d266322968/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.352129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88332819-2733-4f7d-a6c1-43947f367f03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:18:38.352439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88332819-2733-4f7d-a6c1-43947f367f03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.354290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88332819-2733-4f7d-a6c1-43947f367f03 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.366584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073e518-9d99-42cb-aa1c-5e4fdf48c9fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b40a1aa488a343c8be4cf2e839c02ed4, user_id=53cc0d61301b429b9b3d25d266322968, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.367466 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4007/8102] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:18:38.369305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073e518-9d99-42cb-aa1c-5e4fdf48c9fe admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.371638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88332819-2733-4f7d-a6c1-43947f367f03 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.382909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88332819-2733-4f7d-a6c1-43947f367f03 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.383669 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4096/8103] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:38 2022] PUT /identity/v3/projects/b40a1aa488a343c8be4cf2e839c02ed4/users/53cc0d61301b429b9b3d25d266322968/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:38.387809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073e518-9d99-42cb-aa1c-5e4fdf48c9fe admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.395876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:38.443251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098bca1-c2e1-4921-9b95-edc1c576f8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.443867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098bca1-c2e1-4921-9b95-edc1c576f8b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.444683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098bca1-c2e1-4921-9b95-edc1c576f8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.445003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098bca1-c2e1-4921-9b95-edc1c576f8b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.445667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098bca1-c2e1-4921-9b95-edc1c576f8b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.447623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098bca1-c2e1-4921-9b95-edc1c576f8b2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.456187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073e518-9d99-42cb-aa1c-5e4fdf48c9fe admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.461591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d34e7-b249-4490-b384-8d2ac39c4165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.462197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d34e7-b249-4490-b384-8d2ac39c4165 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.462488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d34e7-b249-4490-b384-8d2ac39c4165 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.462772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d34e7-b249-4490-b384-8d2ac39c4165 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.463437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d34e7-b249-4490-b384-8d2ac39c4165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.465313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d34e7-b249-4490-b384-8d2ac39c4165 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.476383 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4097/8104] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:18:38.481041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d34e7-b249-4490-b384-8d2ac39c4165 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.482436 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4008/8105] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:18:38 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:18:38.483175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098bca1-c2e1-4921-9b95-edc1c576f8b2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.491218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d34e7-b249-4490-b384-8d2ac39c4165 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.496361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:38.547573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098bca1-c2e1-4921-9b95-edc1c576f8b2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.550832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3218b9-7713-4d96-bef4-1bcafd0b6693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.551497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3218b9-7713-4d96-bef4-1bcafd0b6693 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.551787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3218b9-7713-4d96-bef4-1bcafd0b6693 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.552076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3218b9-7713-4d96-bef4-1bcafd0b6693 admin admin] PATH_INFO: `/v3/projects/b40a1aa488a343c8be4cf2e839c02ed4/users/53cc0d61301b429b9b3d25d266322968/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.552969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb1826-8be2-4cf4-8ba1-e5bf60da6c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.553657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb1826-8be2-4cf4-8ba1-e5bf60da6c4c admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.553944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb1826-8be2-4cf4-8ba1-e5bf60da6c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.554079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3218b9-7713-4d96-bef4-1bcafd0b6693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b40a1aa488a343c8be4cf2e839c02ed4, user_id=53cc0d61301b429b9b3d25d266322968, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.554327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb1826-8be2-4cf4-8ba1-e5bf60da6c4c admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.554995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb1826-8be2-4cf4-8ba1-e5bf60da6c4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.557006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb1826-8be2-4cf4-8ba1-e5bf60da6c4c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.557455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3218b9-7713-4d96-bef4-1bcafd0b6693 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.567745 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4098/8106] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:38 2022] PUT /identity/v3/projects/b40a1aa488a343c8be4cf2e839c02ed4/users/53cc0d61301b429b9b3d25d266322968/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:18:38.579121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e51ea-8397-4aef-a3fa-e2fbe47c1f14 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.579516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e51ea-8397-4aef-a3fa-e2fbe47c1f14 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.579825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e51ea-8397-4aef-a3fa-e2fbe47c1f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.588012 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4009/8107] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:18:38 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:18:38.589070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb1826-8be2-4cf4-8ba1-e5bf60da6c4c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.600170 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb1826-8be2-4cf4-8ba1-e5bf60da6c4c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.608856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-522e51ea-8397-4aef-a3fa-e2fbe47c1f14 None None] MFA Rules not processed for user `53cc0d61301b429b9b3d25d266322968`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:38.610131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-522e51ea-8397-4aef-a3fa-e2fbe47c1f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:38.659786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d9fa5-3ecb-4dde-b4f1-f876e972c0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.660424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d9fa5-3ecb-4dde-b4f1-f876e972c0cc admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.660741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d9fa5-3ecb-4dde-b4f1-f876e972c0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.661052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d9fa5-3ecb-4dde-b4f1-f876e972c0cc admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.661698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d9fa5-3ecb-4dde-b4f1-f876e972c0cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.663443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d9fa5-3ecb-4dde-b4f1-f876e972c0cc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.673372 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4010/8108] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:18:38.677239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d9fa5-3ecb-4dde-b4f1-f876e972c0cc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.679034 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4099/8109] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:18:38.688918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e51ea-8397-4aef-a3fa-e2fbe47c1f14 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.690404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-522e51ea-8397-4aef-a3fa-e2fbe47c1f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:38.691392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d9fa5-3ecb-4dde-b4f1-f876e972c0cc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.696585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238ed1-343b-4d03-9cad-ae76ae46bd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.697231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238ed1-343b-4d03-9cad-ae76ae46bd73 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.697521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238ed1-343b-4d03-9cad-ae76ae46bd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.697798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238ed1-343b-4d03-9cad-ae76ae46bd73 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.698431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2238ed1-343b-4d03-9cad-ae76ae46bd73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.700027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2238ed1-343b-4d03-9cad-ae76ae46bd73 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.710809 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4011/8110] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:18:38.714794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238ed1-343b-4d03-9cad-ae76ae46bd73 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.724519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238ed1-343b-4d03-9cad-ae76ae46bd73 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.729329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bdc34-c156-4fc4-9e81-c46e96b96d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.729969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bdc34-c156-4fc4-9e81-c46e96b96d87 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.730259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bdc34-c156-4fc4-9e81-c46e96b96d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.730542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bdc34-c156-4fc4-9e81-c46e96b96d87 admin admin] PATH_INFO: `/v3/projects/c29640aa4d4643e2bffa43bab367012b/users/2a3339181fa7493db149e408d85e37f2/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.743428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:38.746855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bdc34-c156-4fc4-9e81-c46e96b96d87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29640aa4d4643e2bffa43bab367012b, user_id=2a3339181fa7493db149e408d85e37f2, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.750343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bdc34-c156-4fc4-9e81-c46e96b96d87 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.753341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e51ea-8397-4aef-a3fa-e2fbe47c1f14 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.758327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0bf5bf-7dfe-499b-bf34-f31108f97b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.759104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0bf5bf-7dfe-499b-bf34-f31108f97b5b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.759422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0bf5bf-7dfe-499b-bf34-f31108f97b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.759713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0bf5bf-7dfe-499b-bf34-f31108f97b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.761678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6a0bf5bf-7dfe-499b-bf34-f31108f97b5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:38.764737 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4012/8111] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:18:38 2022] PUT /identity/v3/projects/c29640aa4d4643e2bffa43bab367012b/users/2a3339181fa7493db149e408d85e37f2/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:18:38.774495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60424b37-33f7-4727-95b1-34cd955c4c21 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.774802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60424b37-33f7-4727-95b1-34cd955c4c21 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.775067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60424b37-33f7-4727-95b1-34cd955c4c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.801957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-60424b37-33f7-4727-95b1-34cd955c4c21 None None] MFA Rules not processed for user `2a3339181fa7493db149e408d85e37f2`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:38.803184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-60424b37-33f7-4727-95b1-34cd955c4c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:38.806190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:38.825511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0bf5bf-7dfe-499b-bf34-f31108f97b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.826978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0bf5bf-7dfe-499b-bf34-f31108f97b5b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.836247 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4100/8112] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:18:38.872408 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4013/8113] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:18:38.885028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0bf5bf-7dfe-499b-bf34-f31108f97b5b service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:38.886512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6a0bf5bf-7dfe-499b-bf34-f31108f97b5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:38.900126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0bf5bf-7dfe-499b-bf34-f31108f97b5b service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:38.952219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553993a9-6e26-459c-94e4-065a405254d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:38.952841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553993a9-6e26-459c-94e4-065a405254d0 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:38.953185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553993a9-6e26-459c-94e4-065a405254d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:38.953467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553993a9-6e26-459c-94e4-065a405254d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:38.962910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553993a9-6e26-459c-94e4-065a405254d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:38.964535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553993a9-6e26-459c-94e4-065a405254d0 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:38.973757 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4101/8114] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:18:46.693644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60424b37-33f7-4727-95b1-34cd955c4c21 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:46.708125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60424b37-33f7-4727-95b1-34cd955c4c21 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:46.712561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b2653-e446-4562-ac33-ea36622330be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:46.713222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b2653-e446-4562-ac33-ea36622330be service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:46.713517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b2653-e446-4562-ac33-ea36622330be service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:46.713803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b2653-e446-4562-ac33-ea36622330be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:46.793045 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b2653-e446-4562-ac33-ea36622330be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:46.795265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b2653-e446-4562-ac33-ea36622330be service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:46.821935 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4014/8115] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:46 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:48.971238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=34 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:51.137861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553993a9-6e26-459c-94e4-065a405254d0 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:51.149825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553993a9-6e26-459c-94e4-065a405254d0 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:51.153635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1bab5-0cbd-4114-976e-ef673e7f80d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:51.154312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1bab5-0cbd-4114-976e-ef673e7f80d6 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:51.154662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1bab5-0cbd-4114-976e-ef673e7f80d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:51.155005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1bab5-0cbd-4114-976e-ef673e7f80d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:51.156787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2cf1bab5-0cbd-4114-976e-ef673e7f80d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:51.217685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1bab5-0cbd-4114-976e-ef673e7f80d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:51.219061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1bab5-0cbd-4114-976e-ef673e7f80d6 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:51.229062 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4102/8116] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:18:56.808641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:18:59.365268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b2653-e446-4562-ac33-ea36622330be service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:59.492477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b2653-e446-4562-ac33-ea36622330be service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:59.502319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:59.503257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:59.503727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:59.504011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] PATH_INFO: `/v3/users/2c941fe0dbc64880b3c4a421cfdae0ee` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:59.505210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c941fe0dbc64880b3c4a421cfdae0ee)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:59.507258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:59.570126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c941fe0dbc64880b3c4a421cfdae0ee'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:59.588515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c941fe0dbc64880b3c4a421cfdae0ee'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:59.589751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c941fe0dbc64880b3c4a421cfdae0ee'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:59.604791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c941fe0dbc64880b3c4a421cfdae0ee'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:59.614731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c941fe0dbc64880b3c4a421cfdae0ee'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:18:59.622303 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4015/8117] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:18:59 2022] DELETE /identity/v3/users/2c941fe0dbc64880b3c4a421cfdae0ee => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:18:59.626872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112ee79-94ec-4da0-912d-eb4cef469739 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:59.627359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112ee79-94ec-4da0-912d-eb4cef469739 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:59.627554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112ee79-94ec-4da0-912d-eb4cef469739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:59.650710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c112ee79-94ec-4da0-912d-eb4cef469739 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:18:59.651854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c112ee79-94ec-4da0-912d-eb4cef469739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:59.755473 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4103/8118] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:18:59.765166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:18:59.766688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:18:59.841225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49c79c-6dce-426b-b84c-32d6e7d758ab admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:18:59.844838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a485-3a38-4e49-924b-525a79c1f65d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:18:59.845489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a485-3a38-4e49-924b-525a79c1f65d service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:18:59.845774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a485-3a38-4e49-924b-525a79c1f65d service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:18:59.846052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a485-3a38-4e49-924b-525a79c1f65d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:18:59.856440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a485-3a38-4e49-924b-525a79c1f65d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:18:59.858138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a485-3a38-4e49-924b-525a79c1f65d service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:18:59.869386 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4016/8119] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:00.055390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112ee79-94ec-4da0-912d-eb4cef469739 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:00.057236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c112ee79-94ec-4da0-912d-eb4cef469739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:00.072242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112ee79-94ec-4da0-912d-eb4cef469739 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:00.078121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:00.079204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:00.079715 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:00.080193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] PATH_INFO: `/v3/projects/4fa173ce8042421a9dd704632c1a12ae` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:00.081858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa173ce8042421a9dd704632c1a12ae)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:00.084703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:00.155084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa173ce8042421a9dd704632c1a12ae', 'request_id': 'req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:00.167760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa173ce8042421a9dd704632c1a12ae', 'request_id': 'req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:00.177805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa173ce8042421a9dd704632c1a12ae', 'request_id': 'req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:00.195463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] The token cache is being invalidate because project 4fa173ce8042421a9dd704632c1a12ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:19:00.195775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:00.197372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:00.207806 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4104/8120] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:19:00 2022] DELETE /identity/v3/projects/4fa173ce8042421a9dd704632c1a12ae => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:00.209245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a485-3a38-4e49-924b-525a79c1f65d service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:00.227148 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a485-3a38-4e49-924b-525a79c1f65d service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:00.255020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:00.321387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:00.325241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:00.325924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:00.326533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] PATH_INFO: `/v3/users/9aae9a894187433da81d68eaf483df3e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:00.328182 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aae9a894187433da81d68eaf483df3e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:00.330563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:00.425666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aae9a894187433da81d68eaf483df3e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:00.444106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aae9a894187433da81d68eaf483df3e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:00.444983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aae9a894187433da81d68eaf483df3e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:00.453018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aae9a894187433da81d68eaf483df3e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:00.464260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aae9a894187433da81d68eaf483df3e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:00.471428 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4017/8121] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:19:00 2022] DELETE /identity/v3/users/9aae9a894187433da81d68eaf483df3e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:19:00.794443 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:00.797523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:00.816483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf38431-4a95-4fa5-a0ef-34fc45ae4d48 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:00.839300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:00.897402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:00.898317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:00.898605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:00.898937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] PATH_INFO: `/v3/projects/dafaee1fc8e84454ba30c1f73b4a562e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:00.900138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dafaee1fc8e84454ba30c1f73b4a562e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:00.902298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:00.965109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dafaee1fc8e84454ba30c1f73b4a562e', 'request_id': 'req-28464ec7-7f31-4445-85ca-d87e5b772ec8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:00.987552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dafaee1fc8e84454ba30c1f73b4a562e', 'request_id': 'req-28464ec7-7f31-4445-85ca-d87e5b772ec8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.000755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dafaee1fc8e84454ba30c1f73b4a562e', 'request_id': 'req-28464ec7-7f31-4445-85ca-d87e5b772ec8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.020745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] The token cache is being invalidate because project dafaee1fc8e84454ba30c1f73b4a562e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:19:01.021137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.022421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.028480 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4105/8122] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:19:00 2022] DELETE /identity/v3/projects/dafaee1fc8e84454ba30c1f73b4a562e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:01.029621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:01.031179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:01.047630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdfd37-c67b-43b5-8362-a26f84a6826f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:01.125055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7aa208-6bba-4555-aa37-70f79b9bf625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:01.125714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7aa208-6bba-4555-aa37-70f79b9bf625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:01.125998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7aa208-6bba-4555-aa37-70f79b9bf625 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:01.126280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7aa208-6bba-4555-aa37-70f79b9bf625 admin admin] PATH_INFO: `/v3/users/53cc0d61301b429b9b3d25d266322968` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:01.127583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7aa208-6bba-4555-aa37-70f79b9bf625 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53cc0d61301b429b9b3d25d266322968)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:01.133054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7aa208-6bba-4555-aa37-70f79b9bf625 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:01.199481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:01.203266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0a7aa208-6bba-4555-aa37-70f79b9bf625 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53cc0d61301b429b9b3d25d266322968'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.223395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0a7aa208-6bba-4555-aa37-70f79b9bf625 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53cc0d61301b429b9b3d25d266322968'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.224151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0a7aa208-6bba-4555-aa37-70f79b9bf625 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53cc0d61301b429b9b3d25d266322968'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.233007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0a7aa208-6bba-4555-aa37-70f79b9bf625 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53cc0d61301b429b9b3d25d266322968'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.244124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0a7aa208-6bba-4555-aa37-70f79b9bf625 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53cc0d61301b429b9b3d25d266322968'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.251629 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4018/8123] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:19:01 2022] DELETE /identity/v3/users/53cc0d61301b429b9b3d25d266322968 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:01.581591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:01.583519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:01.608165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28464ec7-7f31-4445-85ca-d87e5b772ec8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:01.700962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea467c-f06e-4b85-bfbd-e76f95a877d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:01.701913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea467c-f06e-4b85-bfbd-e76f95a877d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:01.702281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea467c-f06e-4b85-bfbd-e76f95a877d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:01.702658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea467c-f06e-4b85-bfbd-e76f95a877d5 admin admin] PATH_INFO: `/v3/projects/b40a1aa488a343c8be4cf2e839c02ed4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:01.703898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea467c-f06e-4b85-bfbd-e76f95a877d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b40a1aa488a343c8be4cf2e839c02ed4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:01.706175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea467c-f06e-4b85-bfbd-e76f95a877d5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:01.767068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2fea467c-f06e-4b85-bfbd-e76f95a877d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b40a1aa488a343c8be4cf2e839c02ed4', 'request_id': 'req-2fea467c-f06e-4b85-bfbd-e76f95a877d5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.772252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:01.776547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2fea467c-f06e-4b85-bfbd-e76f95a877d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b40a1aa488a343c8be4cf2e839c02ed4', 'request_id': 'req-2fea467c-f06e-4b85-bfbd-e76f95a877d5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.784404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2fea467c-f06e-4b85-bfbd-e76f95a877d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b40a1aa488a343c8be4cf2e839c02ed4', 'request_id': 'req-2fea467c-f06e-4b85-bfbd-e76f95a877d5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.799262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2fea467c-f06e-4b85-bfbd-e76f95a877d5 admin admin] The token cache is being invalidate because project b40a1aa488a343c8be4cf2e839c02ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:19:01.800266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2fea467c-f06e-4b85-bfbd-e76f95a877d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.800969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2fea467c-f06e-4b85-bfbd-e76f95a877d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:01.802608 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4106/8124] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:19:01 2022] DELETE /identity/v3/projects/b40a1aa488a343c8be4cf2e839c02ed4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:01.808238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78bcad-03d0-4509-a14a-21d77cc5922a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:01.808796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78bcad-03d0-4509-a14a-21d77cc5922a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:01.809221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78bcad-03d0-4509-a14a-21d77cc5922a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:01.829690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5c78bcad-03d0-4509-a14a-21d77cc5922a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:01.830955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5c78bcad-03d0-4509-a14a-21d77cc5922a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:01.892053 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4019/8125] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:19:01.899879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651e488-a7dc-4511-b1cc-ddb4018c11dc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:01.900259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651e488-a7dc-4511-b1cc-ddb4018c11dc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:01.900657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651e488-a7dc-4511-b1cc-ddb4018c11dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:01.927264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8651e488-a7dc-4511-b1cc-ddb4018c11dc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:01.929974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8651e488-a7dc-4511-b1cc-ddb4018c11dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:01.942710 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4107/8126] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:01.948011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78bcad-03d0-4509-a14a-21d77cc5922a None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:01.959706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78bcad-03d0-4509-a14a-21d77cc5922a None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:01.964489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83f081-67dc-4e16-85eb-07b9a427790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:01.965139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83f081-67dc-4e16-85eb-07b9a427790f admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:01.965425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83f081-67dc-4e16-85eb-07b9a427790f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:01.965705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83f081-67dc-4e16-85eb-07b9a427790f admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:01.966351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83f081-67dc-4e16-85eb-07b9a427790f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:19:01.966666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83f081-67dc-4e16-85eb-07b9a427790f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:01.968280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83f081-67dc-4e16-85eb-07b9a427790f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:01.980673 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4020/8127] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:19:01.985325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651e488-a7dc-4511-b1cc-ddb4018c11dc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:01.996163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651e488-a7dc-4511-b1cc-ddb4018c11dc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:02.002451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb6796-39a1-4df9-9f55-7cb513f1f6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:02.003150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb6796-39a1-4df9-9f55-7cb513f1f6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.003497 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb6796-39a1-4df9-9f55-7cb513f1f6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.003938 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb6796-39a1-4df9-9f55-7cb513f1f6e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.004582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb6796-39a1-4df9-9f55-7cb513f1f6e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:02.006967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb6796-39a1-4df9-9f55-7cb513f1f6e1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:02.060644 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4108/8128] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:19:01 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:02.065677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83f081-67dc-4e16-85eb-07b9a427790f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:02.078222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83f081-67dc-4e16-85eb-07b9a427790f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:02.140109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1fa84-16ea-4507-9d37-9a9d16a1dc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:02.142405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1fa84-16ea-4507-9d37-9a9d16a1dc3f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.142808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1fa84-16ea-4507-9d37-9a9d16a1dc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.143212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1fa84-16ea-4507-9d37-9a9d16a1dc3f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.143955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1fa84-16ea-4507-9d37-9a9d16a1dc3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:02.146213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1fa84-16ea-4507-9d37-9a9d16a1dc3f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:02.185914 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4021/8129] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:19:02 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:02.187447 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb6796-39a1-4df9-9f55-7cb513f1f6e1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:02.200333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb6796-39a1-4df9-9f55-7cb513f1f6e1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:02.213653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e228645-f4b6-4e55-9637-c5f0d8f73778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:02.214545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e228645-f4b6-4e55-9637-c5f0d8f73778 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.214825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e228645-f4b6-4e55-9637-c5f0d8f73778 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.215112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e228645-f4b6-4e55-9637-c5f0d8f73778 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.215778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e228645-f4b6-4e55-9637-c5f0d8f73778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:02.217621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e228645-f4b6-4e55-9637-c5f0d8f73778 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:02.233621 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4109/8130] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:02.234828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1fa84-16ea-4507-9d37-9a9d16a1dc3f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:02.248102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1fa84-16ea-4507-9d37-9a9d16a1dc3f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:02.260512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828e5f5-a6e8-402d-a012-7f5a52fdc5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:02.261193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828e5f5-a6e8-402d-a012-7f5a52fdc5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.261505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828e5f5-a6e8-402d-a012-7f5a52fdc5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.261786 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828e5f5-a6e8-402d-a012-7f5a52fdc5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.262439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828e5f5-a6e8-402d-a012-7f5a52fdc5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:02.264052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828e5f5-a6e8-402d-a012-7f5a52fdc5a2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:02.276923 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4022/8131] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:19:02.281142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e228645-f4b6-4e55-9637-c5f0d8f73778 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:02.293315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e228645-f4b6-4e55-9637-c5f0d8f73778 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:02.298496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee10b4-66fd-4a9d-868d-8e73b59276f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:02.299198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee10b4-66fd-4a9d-868d-8e73b59276f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.299514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee10b4-66fd-4a9d-868d-8e73b59276f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.299793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee10b4-66fd-4a9d-868d-8e73b59276f2 admin admin] PATH_INFO: `/v3/projects/78ff96b1ef384c1da1a2a3c268860150/users/7c4fb4ec62a144d38ffaa100d0a45f4f/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.318193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee10b4-66fd-4a9d-868d-8e73b59276f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ff96b1ef384c1da1a2a3c268860150, user_id=7c4fb4ec62a144d38ffaa100d0a45f4f, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:02.321142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee10b4-66fd-4a9d-868d-8e73b59276f2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:02.338043 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4110/8132] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:02 2022] PUT /identity/v3/projects/78ff96b1ef384c1da1a2a3c268860150/users/7c4fb4ec62a144d38ffaa100d0a45f4f/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:02.349360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38a2d3-ad9e-4423-a7e3-bcf29072c878 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.350593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38a2d3-ad9e-4423-a7e3-bcf29072c878 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.351402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38a2d3-ad9e-4423-a7e3-bcf29072c878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.381715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3e38a2d3-ad9e-4423-a7e3-bcf29072c878 None None] MFA Rules not processed for user `7c4fb4ec62a144d38ffaa100d0a45f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:02.382902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3e38a2d3-ad9e-4423-a7e3-bcf29072c878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:02.396249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:02.451919 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4023/8133] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:02.456201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee10b4-66fd-4a9d-868d-8e73b59276f2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:02.526428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee10b4-66fd-4a9d-868d-8e73b59276f2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:02.531549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b87460-a99c-4a21-9a29-373fdd205160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:02.532281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87460-a99c-4a21-9a29-373fdd205160 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.532667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87460-a99c-4a21-9a29-373fdd205160 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.533078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87460-a99c-4a21-9a29-373fdd205160 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.533846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b87460-a99c-4a21-9a29-373fdd205160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:02.535884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b87460-a99c-4a21-9a29-373fdd205160 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:02.572252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38a2d3-ad9e-4423-a7e3-bcf29072c878 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:02.578759 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4111/8134] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:19:02 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:02.585360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38a2d3-ad9e-4423-a7e3-bcf29072c878 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:02.649129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c0834-7313-4080-b104-99c626e9ccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:02.649754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c0834-7313-4080-b104-99c626e9ccc2 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.650042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c0834-7313-4080-b104-99c626e9ccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.650326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c0834-7313-4080-b104-99c626e9ccc2 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.650982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c0834-7313-4080-b104-99c626e9ccc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:02.652926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c0834-7313-4080-b104-99c626e9ccc2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:02.674703 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4024/8135] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:19:02 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:19:02.679835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b87460-a99c-4a21-9a29-373fdd205160 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:02.691133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b87460-a99c-4a21-9a29-373fdd205160 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:02.696615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac5f20-6c1a-4bb1-9870-6f145382b763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:02.697256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac5f20-6c1a-4bb1-9870-6f145382b763 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.697546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac5f20-6c1a-4bb1-9870-6f145382b763 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.697824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac5f20-6c1a-4bb1-9870-6f145382b763 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.698477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac5f20-6c1a-4bb1-9870-6f145382b763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:02.700095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac5f20-6c1a-4bb1-9870-6f145382b763 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:02.712937 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4112/8136] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:02.715899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c0834-7313-4080-b104-99c626e9ccc2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:02.726231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c0834-7313-4080-b104-99c626e9ccc2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:02.730907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478853-9e6e-433b-be30-6186475efebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:02.731814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478853-9e6e-433b-be30-6186475efebe admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.732099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478853-9e6e-433b-be30-6186475efebe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.732377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478853-9e6e-433b-be30-6186475efebe admin admin] PATH_INFO: `/v3/projects/9a73759161594a0d9ba314b01dbf5b6c/users/2db0f62c4f5649dfa68efb553f23dac7/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.749018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41478853-9e6e-433b-be30-6186475efebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a73759161594a0d9ba314b01dbf5b6c, user_id=2db0f62c4f5649dfa68efb553f23dac7, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:02.751673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41478853-9e6e-433b-be30-6186475efebe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:02.765871 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4025/8137] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:02 2022] PUT /identity/v3/projects/9a73759161594a0d9ba314b01dbf5b6c/users/2db0f62c4f5649dfa68efb553f23dac7/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:02.769801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac5f20-6c1a-4bb1-9870-6f145382b763 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:02.780274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac5f20-6c1a-4bb1-9870-6f145382b763 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:02.823014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:02.840932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660cf6d2-79af-455f-ad30-42f035ab08bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:02.842056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cf6d2-79af-455f-ad30-42f035ab08bd admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.842395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cf6d2-79af-455f-ad30-42f035ab08bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.842904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cf6d2-79af-455f-ad30-42f035ab08bd admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.843576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660cf6d2-79af-455f-ad30-42f035ab08bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:02.845284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660cf6d2-79af-455f-ad30-42f035ab08bd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:02.856954 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4113/8138] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:02.861265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478853-9e6e-433b-be30-6186475efebe admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:02.873173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478853-9e6e-433b-be30-6186475efebe admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:02.878006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8738c-5dd8-4ad3-b4e6-34d4bdcb599d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:02.878667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8738c-5dd8-4ad3-b4e6-34d4bdcb599d admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.878955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8738c-5dd8-4ad3-b4e6-34d4bdcb599d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.879233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8738c-5dd8-4ad3-b4e6-34d4bdcb599d admin admin] PATH_INFO: `/v3/projects/9a73759161594a0d9ba314b01dbf5b6c/users/2db0f62c4f5649dfa68efb553f23dac7/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.881744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae8738c-5dd8-4ad3-b4e6-34d4bdcb599d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a73759161594a0d9ba314b01dbf5b6c, user_id=2db0f62c4f5649dfa68efb553f23dac7, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:02.884451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae8738c-5dd8-4ad3-b4e6-34d4bdcb599d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:02.894991 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4026/8139] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:02 2022] PUT /identity/v3/projects/9a73759161594a0d9ba314b01dbf5b6c/users/2db0f62c4f5649dfa68efb553f23dac7/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:02.905220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be501f7-eab1-46f9-b5b3-0d7fe6a1b757 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:02.905522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be501f7-eab1-46f9-b5b3-0d7fe6a1b757 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:02.905807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be501f7-eab1-46f9-b5b3-0d7fe6a1b757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:02.933687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6be501f7-eab1-46f9-b5b3-0d7fe6a1b757 None None] MFA Rules not processed for user `2db0f62c4f5649dfa68efb553f23dac7`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:02.935026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6be501f7-eab1-46f9-b5b3-0d7fe6a1b757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:03.005921 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4114/8140] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:03.011531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8738c-5dd8-4ad3-b4e6-34d4bdcb599d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:03.022794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8738c-5dd8-4ad3-b4e6-34d4bdcb599d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:03.028357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ace07-13a9-4967-97b1-586672fc19b1 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db0f62c4f5649dfa68efb553f23dac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a73759161594a0d9ba314b01dbf5b6c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:03.029125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ace07-13a9-4967-97b1-586672fc19b1 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:03.029479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ace07-13a9-4967-97b1-586672fc19b1 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:03.029906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ace07-13a9-4967-97b1-586672fc19b1 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:03.030579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ace07-13a9-4967-97b1-586672fc19b1 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:19:03.030973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ace07-13a9-4967-97b1-586672fc19b1 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:03.033264 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ace07-13a9-4967-97b1-586672fc19b1 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:03.047432 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4027/8141] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:19:03.051476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be501f7-eab1-46f9-b5b3-0d7fe6a1b757 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:03.061681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be501f7-eab1-46f9-b5b3-0d7fe6a1b757 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:03.066489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8763ede-2c87-4f18-9280-16a835a151e5 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db0f62c4f5649dfa68efb553f23dac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a73759161594a0d9ba314b01dbf5b6c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:03.067103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8763ede-2c87-4f18-9280-16a835a151e5 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:03.067392 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8763ede-2c87-4f18-9280-16a835a151e5 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:03.067674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8763ede-2c87-4f18-9280-16a835a151e5 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:03.068355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8763ede-2c87-4f18-9280-16a835a151e5 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:03.070915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8763ede-2c87-4f18-9280-16a835a151e5 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:03.100237 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4115/8142] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:19:03 2022] POST /identity/v3/projects => generated 347 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:19:03.104168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ace07-13a9-4967-97b1-586672fc19b1 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:03.116167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ace07-13a9-4967-97b1-586672fc19b1 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:03.187756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:03.202093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba53ae7-b8a6-47cb-b237-e9fa3f89d9a7 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db0f62c4f5649dfa68efb553f23dac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a73759161594a0d9ba314b01dbf5b6c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:03.203273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba53ae7-b8a6-47cb-b237-e9fa3f89d9a7 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:03.203703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba53ae7-b8a6-47cb-b237-e9fa3f89d9a7 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:03.204073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba53ae7-b8a6-47cb-b237-e9fa3f89d9a7 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:03.207155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba53ae7-b8a6-47cb-b237-e9fa3f89d9a7 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:19:03.207155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba53ae7-b8a6-47cb-b237-e9fa3f89d9a7 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:03.207155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba53ae7-b8a6-47cb-b237-e9fa3f89d9a7 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:03.224092 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4028/8143] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:19:03.233799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8763ede-2c87-4f18-9280-16a835a151e5 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:03.236258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f8763ede-2c87-4f18-9280-16a835a151e5 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:03.320957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8763ede-2c87-4f18-9280-16a835a151e5 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:03.324790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36258474-a2bb-46c2-b854-5277bf3ca07b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:03.325538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36258474-a2bb-46c2-b854-5277bf3ca07b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:03.325900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36258474-a2bb-46c2-b854-5277bf3ca07b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:03.326248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36258474-a2bb-46c2-b854-5277bf3ca07b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:03.336276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36258474-a2bb-46c2-b854-5277bf3ca07b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:03.337821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36258474-a2bb-46c2-b854-5277bf3ca07b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:03.349489 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4116/8144] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:04.833421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba53ae7-b8a6-47cb-b237-e9fa3f89d9a7 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:04.852811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba53ae7-b8a6-47cb-b237-e9fa3f89d9a7 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:04.857875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982193e-d582-458b-8c78-17a10f3e1d76 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db0f62c4f5649dfa68efb553f23dac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a73759161594a0d9ba314b01dbf5b6c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:04.858518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982193e-d582-458b-8c78-17a10f3e1d76 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:04.858817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982193e-d582-458b-8c78-17a10f3e1d76 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:04.859102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982193e-d582-458b-8c78-17a10f3e1d76 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] PATH_INFO: `/v3/projects/f1b6070b2f8f4c8aacbd62ac9364f032` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:04.860238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982193e-d582-458b-8c78-17a10f3e1d76 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f1b6070b2f8f4c8aacbd62ac9364f032)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:04.862225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982193e-d582-458b-8c78-17a10f3e1d76 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:04.911483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8982193e-d582-458b-8c78-17a10f3e1d76 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b6070b2f8f4c8aacbd62ac9364f032', 'request_id': 'req-8982193e-d582-458b-8c78-17a10f3e1d76'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:04.920648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8982193e-d582-458b-8c78-17a10f3e1d76 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b6070b2f8f4c8aacbd62ac9364f032', 'request_id': 'req-8982193e-d582-458b-8c78-17a10f3e1d76'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:04.929030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8982193e-d582-458b-8c78-17a10f3e1d76 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b6070b2f8f4c8aacbd62ac9364f032', 'request_id': 'req-8982193e-d582-458b-8c78-17a10f3e1d76'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:04.940524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8982193e-d582-458b-8c78-17a10f3e1d76 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] The token cache is being invalidate because project f1b6070b2f8f4c8aacbd62ac9364f032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:19:04.941042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8982193e-d582-458b-8c78-17a10f3e1d76 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:04.941740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8982193e-d582-458b-8c78-17a10f3e1d76 tempest-QuotasNegativeTest-1201935849 tempest-QuotasNegativeTest-1201935849-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:04.943386 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4029/8145] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:19:04 2022] DELETE /identity/v3/projects/f1b6070b2f8f4c8aacbd62ac9364f032 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:04.948552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36258474-a2bb-46c2-b854-5277bf3ca07b service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:05.001529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:05.019446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36258474-a2bb-46c2-b854-5277bf3ca07b service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:05.080058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:05.080703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:05.081024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:05.081310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] PATH_INFO: `/v3/users/7c4fb4ec62a144d38ffaa100d0a45f4f` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:05.082456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c4fb4ec62a144d38ffaa100d0a45f4f)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:05.085010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:05.137017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c4fb4ec62a144d38ffaa100d0a45f4f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.150157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c4fb4ec62a144d38ffaa100d0a45f4f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.159558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c4fb4ec62a144d38ffaa100d0a45f4f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.160201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c4fb4ec62a144d38ffaa100d0a45f4f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.160937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c4fb4ec62a144d38ffaa100d0a45f4f'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.175225 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4117/8146] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:19:04 2022] DELETE /identity/v3/users/7c4fb4ec62a144d38ffaa100d0a45f4f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:05.179796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c96db9-a33a-4399-b5cd-c8e5bdc114b0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:05.180063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c96db9-a33a-4399-b5cd-c8e5bdc114b0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:05.180380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c96db9-a33a-4399-b5cd-c8e5bdc114b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:05.200042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-36c96db9-a33a-4399-b5cd-c8e5bdc114b0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:05.201595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-36c96db9-a33a-4399-b5cd-c8e5bdc114b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:05.265258 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4030/8147] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:19:05.275610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:05.277084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:05.338187 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f25cb-5e96-4f8b-9888-0e6d1737e809 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:05.342209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd99db-d2e6-4f55-962a-027f2fc00c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:05.342822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd99db-d2e6-4f55-962a-027f2fc00c74 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:05.343221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd99db-d2e6-4f55-962a-027f2fc00c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:05.343507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd99db-d2e6-4f55-962a-027f2fc00c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:05.353500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd99db-d2e6-4f55-962a-027f2fc00c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:05.354956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd99db-d2e6-4f55-962a-027f2fc00c74 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:05.365138 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4118/8148] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:05.643551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c96db9-a33a-4399-b5cd-c8e5bdc114b0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:05.645301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-36c96db9-a33a-4399-b5cd-c8e5bdc114b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:05.663227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c96db9-a33a-4399-b5cd-c8e5bdc114b0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:05.668455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:05.669248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:05.669534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:05.669809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] PATH_INFO: `/v3/projects/78ff96b1ef384c1da1a2a3c268860150` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:05.671837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ff96b1ef384c1da1a2a3c268860150)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:05.673951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:05.738557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ff96b1ef384c1da1a2a3c268860150', 'request_id': 'req-58ee8eb6-e3e5-45be-802d-863b72edf12c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.748056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ff96b1ef384c1da1a2a3c268860150', 'request_id': 'req-58ee8eb6-e3e5-45be-802d-863b72edf12c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.755935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ff96b1ef384c1da1a2a3c268860150', 'request_id': 'req-58ee8eb6-e3e5-45be-802d-863b72edf12c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.763795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:05.771911 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] The token cache is being invalidate because project 78ff96b1ef384c1da1a2a3c268860150 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:19:05.772275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.773062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.777922 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4031/8149] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:19:05 2022] DELETE /identity/v3/projects/78ff96b1ef384c1da1a2a3c268860150 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:05.779702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd99db-d2e6-4f55-962a-027f2fc00c74 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:05.790890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd99db-d2e6-4f55-962a-027f2fc00c74 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:05.805521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:05.857113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f026f48-4e85-45c3-8804-713a9462058d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:05.857877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f026f48-4e85-45c3-8804-713a9462058d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:05.858229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f026f48-4e85-45c3-8804-713a9462058d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:05.858586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f026f48-4e85-45c3-8804-713a9462058d admin admin] PATH_INFO: `/v3/users/2db0f62c4f5649dfa68efb553f23dac7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:05.859804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f026f48-4e85-45c3-8804-713a9462058d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2db0f62c4f5649dfa68efb553f23dac7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:05.861883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f026f48-4e85-45c3-8804-713a9462058d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:05.908545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6f026f48-4e85-45c3-8804-713a9462058d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2db0f62c4f5649dfa68efb553f23dac7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.920791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6f026f48-4e85-45c3-8804-713a9462058d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2db0f62c4f5649dfa68efb553f23dac7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.929461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6f026f48-4e85-45c3-8804-713a9462058d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2db0f62c4f5649dfa68efb553f23dac7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.930124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6f026f48-4e85-45c3-8804-713a9462058d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2db0f62c4f5649dfa68efb553f23dac7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.930796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6f026f48-4e85-45c3-8804-713a9462058d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2db0f62c4f5649dfa68efb553f23dac7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:05.943195 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4119/8150] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:19:05 2022] DELETE /identity/v3/users/2db0f62c4f5649dfa68efb553f23dac7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:06.221653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:06.223143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:06.247851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee8eb6-e3e5-45be-802d-863b72edf12c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:06.308231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784c744-35ec-4dc2-a667-5567a6262fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:06.309209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784c744-35ec-4dc2-a667-5567a6262fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:06.309666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784c744-35ec-4dc2-a667-5567a6262fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:06.309946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784c744-35ec-4dc2-a667-5567a6262fd2 admin admin] PATH_INFO: `/v3/projects/9a73759161594a0d9ba314b01dbf5b6c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:06.311104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784c744-35ec-4dc2-a667-5567a6262fd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a73759161594a0d9ba314b01dbf5b6c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:06.313036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784c744-35ec-4dc2-a667-5567a6262fd2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:06.359947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f784c744-35ec-4dc2-a667-5567a6262fd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a73759161594a0d9ba314b01dbf5b6c', 'request_id': 'req-f784c744-35ec-4dc2-a667-5567a6262fd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:06.369368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f784c744-35ec-4dc2-a667-5567a6262fd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a73759161594a0d9ba314b01dbf5b6c', 'request_id': 'req-f784c744-35ec-4dc2-a667-5567a6262fd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:06.377410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f784c744-35ec-4dc2-a667-5567a6262fd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a73759161594a0d9ba314b01dbf5b6c', 'request_id': 'req-f784c744-35ec-4dc2-a667-5567a6262fd2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:06.388500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f784c744-35ec-4dc2-a667-5567a6262fd2 admin admin] The token cache is being invalidate because project 9a73759161594a0d9ba314b01dbf5b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:19:06.388793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f784c744-35ec-4dc2-a667-5567a6262fd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:06.389500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f784c744-35ec-4dc2-a667-5567a6262fd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:06.391015 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4032/8151] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:19:06 2022] DELETE /identity/v3/projects/9a73759161594a0d9ba314b01dbf5b6c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:06.396144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9210c6-a5b8-4432-aa11-bfd7c430e3d8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:06.396639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9210c6-a5b8-4432-aa11-bfd7c430e3d8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:06.396745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9210c6-a5b8-4432-aa11-bfd7c430e3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:06.417110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3d9210c6-a5b8-4432-aa11-bfd7c430e3d8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:06.418325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3d9210c6-a5b8-4432-aa11-bfd7c430e3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:06.483979 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4120/8152] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:06.492839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9113cca-bcb3-4fec-84b3-4c0d09e212cf None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:06.493428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9113cca-bcb3-4fec-84b3-4c0d09e212cf None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:06.493756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9113cca-bcb3-4fec-84b3-4c0d09e212cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:06.514446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f9113cca-bcb3-4fec-84b3-4c0d09e212cf None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:06.515672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f9113cca-bcb3-4fec-84b3-4c0d09e212cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:06.525959 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4033/8153] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:06.530279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9210c6-a5b8-4432-aa11-bfd7c430e3d8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:06.540863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9210c6-a5b8-4432-aa11-bfd7c430e3d8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:06.546062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed0579-4abf-4c22-814c-072a721d8027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:06.546683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed0579-4abf-4c22-814c-072a721d8027 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:06.546974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed0579-4abf-4c22-814c-072a721d8027 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:06.547252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed0579-4abf-4c22-814c-072a721d8027 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:06.547911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed0579-4abf-4c22-814c-072a721d8027 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:19:06.548222 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed0579-4abf-4c22-814c-072a721d8027 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:06.549874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed0579-4abf-4c22-814c-072a721d8027 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:06.562259 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4121/8154] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:19:06.567912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9113cca-bcb3-4fec-84b3-4c0d09e212cf None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:06.581081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9113cca-bcb3-4fec-84b3-4c0d09e212cf None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:06.586745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336b57c-9c2e-4e6b-9645-b276d875e195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:06.587360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336b57c-9c2e-4e6b-9645-b276d875e195 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:06.587647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336b57c-9c2e-4e6b-9645-b276d875e195 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:06.587925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336b57c-9c2e-4e6b-9645-b276d875e195 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:06.588563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336b57c-9c2e-4e6b-9645-b276d875e195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:06.592511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336b57c-9c2e-4e6b-9645-b276d875e195 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:06.620937 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4034/8155] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:19:06 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:06.625064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed0579-4abf-4c22-814c-072a721d8027 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:06.635631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed0579-4abf-4c22-814c-072a721d8027 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:06.655411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:06.697254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3cca1-9a82-46f1-9c8b-66d00a0596bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:06.697870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3cca1-9a82-46f1-9c8b-66d00a0596bc admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:06.698159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3cca1-9a82-46f1-9c8b-66d00a0596bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:06.698440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3cca1-9a82-46f1-9c8b-66d00a0596bc admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:06.699077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3cca1-9a82-46f1-9c8b-66d00a0596bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:06.701042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3cca1-9a82-46f1-9c8b-66d00a0596bc admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:06.727178 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4122/8156] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:19:06 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:06.731699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336b57c-9c2e-4e6b-9645-b276d875e195 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:06.742265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336b57c-9c2e-4e6b-9645-b276d875e195 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:06.747411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2987-025e-4b9d-a068-d6ec019b94c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:06.748023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2987-025e-4b9d-a068-d6ec019b94c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:06.748324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2987-025e-4b9d-a068-d6ec019b94c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:06.748580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2987-025e-4b9d-a068-d6ec019b94c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:06.749259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a2987-025e-4b9d-a068-d6ec019b94c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:06.750872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a2987-025e-4b9d-a068-d6ec019b94c1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:06.767337 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4035/8157] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:06.768645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3cca1-9a82-46f1-9c8b-66d00a0596bc admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:06.780059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3cca1-9a82-46f1-9c8b-66d00a0596bc admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:06.784662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4cecb-9b50-4130-a9ef-49d9f4905264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:06.785299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4cecb-9b50-4130-a9ef-49d9f4905264 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:06.785581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4cecb-9b50-4130-a9ef-49d9f4905264 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:06.785855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4cecb-9b50-4130-a9ef-49d9f4905264 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:06.786492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4cecb-9b50-4130-a9ef-49d9f4905264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:06.788704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4cecb-9b50-4130-a9ef-49d9f4905264 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:06.802682 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4123/8158] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:06.804074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2987-025e-4b9d-a068-d6ec019b94c1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:06.814443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2987-025e-4b9d-a068-d6ec019b94c1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:06.819629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cacb9-02e2-44f0-bf4f-fea5dee30853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:06.820288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cacb9-02e2-44f0-bf4f-fea5dee30853 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:06.820598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cacb9-02e2-44f0-bf4f-fea5dee30853 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:06.821007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cacb9-02e2-44f0-bf4f-fea5dee30853 admin admin] PATH_INFO: `/v3/projects/bc037098ffb44ad9838f05f8ecc19907/users/444d5198a5db41acbd37299a8697f99e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:06.838648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cacb9-02e2-44f0-bf4f-fea5dee30853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc037098ffb44ad9838f05f8ecc19907, user_id=444d5198a5db41acbd37299a8697f99e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:06.841720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cacb9-02e2-44f0-bf4f-fea5dee30853 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:06.855679 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4036/8159] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:06 2022] PUT /identity/v3/projects/bc037098ffb44ad9838f05f8ecc19907/users/444d5198a5db41acbd37299a8697f99e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:06.867078 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb53408-c118-4998-bb64-93875882cc71 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:06.867693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb53408-c118-4998-bb64-93875882cc71 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:06.867693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb53408-c118-4998-bb64-93875882cc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:06.897399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3eb53408-c118-4998-bb64-93875882cc71 None None] MFA Rules not processed for user `444d5198a5db41acbd37299a8697f99e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:06.898641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3eb53408-c118-4998-bb64-93875882cc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:06.968201 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4124/8160] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:06.972924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cacb9-02e2-44f0-bf4f-fea5dee30853 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:06.998308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:07.039630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cacb9-02e2-44f0-bf4f-fea5dee30853 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:07.044399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacde9ab-2821-4359-b1a7-e7ee7704a328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:07.045041 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacde9ab-2821-4359-b1a7-e7ee7704a328 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:07.045328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacde9ab-2821-4359-b1a7-e7ee7704a328 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:07.045605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacde9ab-2821-4359-b1a7-e7ee7704a328 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:07.046246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacde9ab-2821-4359-b1a7-e7ee7704a328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:07.048191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacde9ab-2821-4359-b1a7-e7ee7704a328 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:07.076227 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4037/8161] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:19:06 2022] POST /identity/v3/projects => generated 397 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:07.080818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb53408-c118-4998-bb64-93875882cc71 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:07.093868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb53408-c118-4998-bb64-93875882cc71 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:07.159080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f62f-7124-4385-b337-193c230b8be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:07.159711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f62f-7124-4385-b337-193c230b8be3 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:07.160007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f62f-7124-4385-b337-193c230b8be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:07.160291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f62f-7124-4385-b337-193c230b8be3 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:07.160989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594f62f-7124-4385-b337-193c230b8be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:07.162967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594f62f-7124-4385-b337-193c230b8be3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:07.192093 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4125/8162] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:19:07 2022] POST /identity/v3/users => generated 305 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:07.193933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacde9ab-2821-4359-b1a7-e7ee7704a328 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:07.204467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacde9ab-2821-4359-b1a7-e7ee7704a328 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:07.209809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e3c79-952b-42dc-9797-ed79730e4ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:07.210420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3c79-952b-42dc-9797-ed79730e4ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:07.210710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3c79-952b-42dc-9797-ed79730e4ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:07.210984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3c79-952b-42dc-9797-ed79730e4ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:07.211657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e3c79-952b-42dc-9797-ed79730e4ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:07.213334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e3c79-952b-42dc-9797-ed79730e4ca4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:07.232122 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4038/8163] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:07.233336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f62f-7124-4385-b337-193c230b8be3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:07.244659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f62f-7124-4385-b337-193c230b8be3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:07.249563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e7d41-7a3e-4d41-9bb4-8d3838dea4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:07.250238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e7d41-7a3e-4d41-9bb4-8d3838dea4f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:07.250546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e7d41-7a3e-4d41-9bb4-8d3838dea4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:07.250823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e7d41-7a3e-4d41-9bb4-8d3838dea4f2 admin admin] PATH_INFO: `/v3/projects/b31a3e81e6ff48639b9eb4191c8fd683/users/a4a8674e22cf45ee8f225d7610ba881b/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:07.266783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e7d41-7a3e-4d41-9bb4-8d3838dea4f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31a3e81e6ff48639b9eb4191c8fd683, user_id=a4a8674e22cf45ee8f225d7610ba881b, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:07.269540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e7d41-7a3e-4d41-9bb4-8d3838dea4f2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:07.273093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:07.286095 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4126/8164] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:07 2022] PUT /identity/v3/projects/b31a3e81e6ff48639b9eb4191c8fd683/users/a4a8674e22cf45ee8f225d7610ba881b/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:07.290127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e3c79-952b-42dc-9797-ed79730e4ca4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:07.300527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e3c79-952b-42dc-9797-ed79730e4ca4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:07.357637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279b352-3d89-4357-b581-a56ad6521d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:07.358248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279b352-3d89-4357-b581-a56ad6521d9c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:07.358545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279b352-3d89-4357-b581-a56ad6521d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:07.358808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279b352-3d89-4357-b581-a56ad6521d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:07.359446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279b352-3d89-4357-b581-a56ad6521d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:07.361103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279b352-3d89-4357-b581-a56ad6521d9c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:07.371616 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4039/8165] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:19:07.376028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e7d41-7a3e-4d41-9bb4-8d3838dea4f2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:07.389275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e7d41-7a3e-4d41-9bb4-8d3838dea4f2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:07.397979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21c2d-ca8d-419e-9373-7daff19613b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:07.398657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21c2d-ca8d-419e-9373-7daff19613b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:07.398949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21c2d-ca8d-419e-9373-7daff19613b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:07.399229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21c2d-ca8d-419e-9373-7daff19613b7 admin admin] PATH_INFO: `/v3/projects/b31a3e81e6ff48639b9eb4191c8fd683/users/a4a8674e22cf45ee8f225d7610ba881b/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:07.401260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff21c2d-ca8d-419e-9373-7daff19613b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31a3e81e6ff48639b9eb4191c8fd683, user_id=a4a8674e22cf45ee8f225d7610ba881b, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:07.403937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff21c2d-ca8d-419e-9373-7daff19613b7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:07.414831 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4127/8166] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:07 2022] PUT /identity/v3/projects/b31a3e81e6ff48639b9eb4191c8fd683/users/a4a8674e22cf45ee8f225d7610ba881b/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:07.425443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424a054-8c53-4c02-8ae7-3c213750b9d6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:07.425767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424a054-8c53-4c02-8ae7-3c213750b9d6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:07.426304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424a054-8c53-4c02-8ae7-3c213750b9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:07.456673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1424a054-8c53-4c02-8ae7-3c213750b9d6 None None] MFA Rules not processed for user `a4a8674e22cf45ee8f225d7610ba881b`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:07.457984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1424a054-8c53-4c02-8ae7-3c213750b9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:07.532137 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4040/8167] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:07 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:07.542204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21c2d-ca8d-419e-9373-7daff19613b7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:07.544599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3ff21c2d-ca8d-419e-9373-7daff19613b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:07.611850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21c2d-ca8d-419e-9373-7daff19613b7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:07.616178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195740cf-a6da-4367-9456-5780fc6a6c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:07.616789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195740cf-a6da-4367-9456-5780fc6a6c46 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:07.617077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195740cf-a6da-4367-9456-5780fc6a6c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:07.617356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195740cf-a6da-4367-9456-5780fc6a6c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:07.688614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195740cf-a6da-4367-9456-5780fc6a6c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:07.690476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195740cf-a6da-4367-9456-5780fc6a6c46 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:07.702168 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4128/8168] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:07 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:08.018219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424a054-8c53-4c02-8ae7-3c213750b9d6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:08.021080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1424a054-8c53-4c02-8ae7-3c213750b9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:08.056517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:08.134839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424a054-8c53-4c02-8ae7-3c213750b9d6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:08.141338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06009dcf-3304-49e1-aeed-0f4292b820ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:08.142326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06009dcf-3304-49e1-aeed-0f4292b820ee service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:08.142737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06009dcf-3304-49e1-aeed-0f4292b820ee service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:08.143112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06009dcf-3304-49e1-aeed-0f4292b820ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:08.153758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06009dcf-3304-49e1-aeed-0f4292b820ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:08.155520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06009dcf-3304-49e1-aeed-0f4292b820ee service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:08.168836 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4041/8169] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:08 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:08.267623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195740cf-a6da-4367-9456-5780fc6a6c46 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:08.288110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195740cf-a6da-4367-9456-5780fc6a6c46 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:08.293433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0753d1-1bd8-4227-aede-149f7470cf9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:08.294295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0753d1-1bd8-4227-aede-149f7470cf9a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:08.294672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0753d1-1bd8-4227-aede-149f7470cf9a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:08.295116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0753d1-1bd8-4227-aede-149f7470cf9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:08.297213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dc0753d1-1bd8-4227-aede-149f7470cf9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:08.317967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0753d1-1bd8-4227-aede-149f7470cf9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:08.320300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0753d1-1bd8-4227-aede-149f7470cf9a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:08.343772 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4129/8170] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:12.211037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06009dcf-3304-49e1-aeed-0f4292b820ee service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:12.214099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-06009dcf-3304-49e1-aeed-0f4292b820ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:12.233152 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06009dcf-3304-49e1-aeed-0f4292b820ee service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:12.237632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262bd91-d4ce-40d7-b53b-ca23909e33d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:12.238255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262bd91-d4ce-40d7-b53b-ca23909e33d3 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:12.238539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262bd91-d4ce-40d7-b53b-ca23909e33d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:12.238815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262bd91-d4ce-40d7-b53b-ca23909e33d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:12.241190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5262bd91-d4ce-40d7-b53b-ca23909e33d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:12.259321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262bd91-d4ce-40d7-b53b-ca23909e33d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:12.260736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262bd91-d4ce-40d7-b53b-ca23909e33d3 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:12.279877 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4042/8171] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:16.521079 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0753d1-1bd8-4227-aede-149f7470cf9a service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:16.608493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0753d1-1bd8-4227-aede-149f7470cf9a service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:16.614263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:16.614263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:16.614624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:16.614732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] PATH_INFO: `/v3/users/444d5198a5db41acbd37299a8697f99e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:16.616448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] RBAC: Authorizing `identity:delete_user(user_id=444d5198a5db41acbd37299a8697f99e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:16.618505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:16.651609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:16.678408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '444d5198a5db41acbd37299a8697f99e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:16.692860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '444d5198a5db41acbd37299a8697f99e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:16.702176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '444d5198a5db41acbd37299a8697f99e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:16.703004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '444d5198a5db41acbd37299a8697f99e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:16.703855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '444d5198a5db41acbd37299a8697f99e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:16.720097 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4130/8172] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:19:16 2022] DELETE /identity/v3/users/444d5198a5db41acbd37299a8697f99e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:16.725464 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee3917-40bd-47c6-894e-0f62f4b0795b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:16.726337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee3917-40bd-47c6-894e-0f62f4b0795b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:16.726631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee3917-40bd-47c6-894e-0f62f4b0795b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:16.748166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-42ee3917-40bd-47c6-894e-0f62f4b0795b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:16.749469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-42ee3917-40bd-47c6-894e-0f62f4b0795b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:16.828016 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4043/8173] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:16.881589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:16.883907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:16.963519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93618ab-f6ee-4e27-a25c-d23027c4841d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:16.971657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f22dc-16f5-466a-ac74-7d6e3dbaa006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:16.971867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f22dc-16f5-466a-ac74-7d6e3dbaa006 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:16.972144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f22dc-16f5-466a-ac74-7d6e3dbaa006 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:16.972479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f22dc-16f5-466a-ac74-7d6e3dbaa006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:16.984018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f22dc-16f5-466a-ac74-7d6e3dbaa006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:16.985519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f22dc-16f5-466a-ac74-7d6e3dbaa006 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:16.997725 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4131/8174] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:17.339335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee3917-40bd-47c6-894e-0f62f4b0795b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:17.341046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-42ee3917-40bd-47c6-894e-0f62f4b0795b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:17.358155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee3917-40bd-47c6-894e-0f62f4b0795b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:17.364767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:17.365501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:17.365787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:17.366064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] PATH_INFO: `/v3/projects/bc037098ffb44ad9838f05f8ecc19907` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:17.367271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc037098ffb44ad9838f05f8ecc19907)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:17.369471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:17.451982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc037098ffb44ad9838f05f8ecc19907', 'request_id': 'req-c32c3361-abae-48ba-9242-ddccb4cc7473'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:17.464840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc037098ffb44ad9838f05f8ecc19907', 'request_id': 'req-c32c3361-abae-48ba-9242-ddccb4cc7473'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:17.474468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc037098ffb44ad9838f05f8ecc19907', 'request_id': 'req-c32c3361-abae-48ba-9242-ddccb4cc7473'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:17.511521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] The token cache is being invalidate because project bc037098ffb44ad9838f05f8ecc19907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:19:17.511521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:17.516229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:17.524058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f22dc-16f5-466a-ac74-7d6e3dbaa006 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:17.524718 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4044/8175] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:19:17 2022] DELETE /identity/v3/projects/bc037098ffb44ad9838f05f8ecc19907 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:17.539355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f22dc-16f5-466a-ac74-7d6e3dbaa006 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:17.622261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4f57-8371-47b9-8c45-0f7ead5a2552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:17.623414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4f57-8371-47b9-8c45-0f7ead5a2552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:17.624056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4f57-8371-47b9-8c45-0f7ead5a2552 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:17.624553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4f57-8371-47b9-8c45-0f7ead5a2552 admin admin] PATH_INFO: `/v3/users/a4a8674e22cf45ee8f225d7610ba881b` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:17.626355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e4f57-8371-47b9-8c45-0f7ead5a2552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a8674e22cf45ee8f225d7610ba881b)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:17.629109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e4f57-8371-47b9-8c45-0f7ead5a2552 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:17.733847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-360e4f57-8371-47b9-8c45-0f7ead5a2552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a8674e22cf45ee8f225d7610ba881b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:17.735830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:17.763817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-360e4f57-8371-47b9-8c45-0f7ead5a2552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a8674e22cf45ee8f225d7610ba881b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:17.786286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-360e4f57-8371-47b9-8c45-0f7ead5a2552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a8674e22cf45ee8f225d7610ba881b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:17.787284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-360e4f57-8371-47b9-8c45-0f7ead5a2552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a8674e22cf45ee8f225d7610ba881b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:17.788119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-360e4f57-8371-47b9-8c45-0f7ead5a2552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a8674e22cf45ee8f225d7610ba881b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:17.813724 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4132/8176] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:19:17 2022] DELETE /identity/v3/users/a4a8674e22cf45ee8f225d7610ba881b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:18.403515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:18.406723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:18.432334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c3361-abae-48ba-9242-ddccb4cc7473 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:18.446574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:18.574720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e68f1-c330-4202-b7da-cbc88bd4e1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:18.577368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e68f1-c330-4202-b7da-cbc88bd4e1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:18.577945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e68f1-c330-4202-b7da-cbc88bd4e1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:18.579258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e68f1-c330-4202-b7da-cbc88bd4e1b2 admin admin] PATH_INFO: `/v3/projects/b31a3e81e6ff48639b9eb4191c8fd683` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:18.581009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e68f1-c330-4202-b7da-cbc88bd4e1b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b31a3e81e6ff48639b9eb4191c8fd683)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:18.585034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e68f1-c330-4202-b7da-cbc88bd4e1b2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:18.665226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-435e68f1-c330-4202-b7da-cbc88bd4e1b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31a3e81e6ff48639b9eb4191c8fd683', 'request_id': 'req-435e68f1-c330-4202-b7da-cbc88bd4e1b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:18.681573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-435e68f1-c330-4202-b7da-cbc88bd4e1b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31a3e81e6ff48639b9eb4191c8fd683', 'request_id': 'req-435e68f1-c330-4202-b7da-cbc88bd4e1b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:18.693651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-435e68f1-c330-4202-b7da-cbc88bd4e1b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31a3e81e6ff48639b9eb4191c8fd683', 'request_id': 'req-435e68f1-c330-4202-b7da-cbc88bd4e1b2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:18.714166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-435e68f1-c330-4202-b7da-cbc88bd4e1b2 admin admin] The token cache is being invalidate because project b31a3e81e6ff48639b9eb4191c8fd683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:19:18.715939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-435e68f1-c330-4202-b7da-cbc88bd4e1b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:18.716739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-435e68f1-c330-4202-b7da-cbc88bd4e1b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:18.718393 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4045/8177] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:19:18 2022] DELETE /identity/v3/projects/b31a3e81e6ff48639b9eb4191c8fd683 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:18.723446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2d3ca-34c5-499f-89c1-48ab3767e69f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:18.724412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2d3ca-34c5-499f-89c1-48ab3767e69f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:18.726523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2d3ca-34c5-499f-89c1-48ab3767e69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:18.759660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-51c2d3ca-34c5-499f-89c1-48ab3767e69f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:18.761052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-51c2d3ca-34c5-499f-89c1-48ab3767e69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:18.863568 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4133/8178] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:18.872404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58393d4-3a58-4ea9-8aa6-576958307446 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:18.872539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58393d4-3a58-4ea9-8aa6-576958307446 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:18.872818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58393d4-3a58-4ea9-8aa6-576958307446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:18.904934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f58393d4-3a58-4ea9-8aa6-576958307446 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:18.906039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f58393d4-3a58-4ea9-8aa6-576958307446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:18.917197 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4046/8179] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:18.921200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2d3ca-34c5-499f-89c1-48ab3767e69f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:18.937872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2d3ca-34c5-499f-89c1-48ab3767e69f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:18.949300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3808c-d09e-4a65-88d4-fa18f25d48b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:18.950194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3808c-d09e-4a65-88d4-fa18f25d48b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:18.950642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3808c-d09e-4a65-88d4-fa18f25d48b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:18.951101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3808c-d09e-4a65-88d4-fa18f25d48b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:18.952034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3808c-d09e-4a65-88d4-fa18f25d48b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:19:18.952646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3808c-d09e-4a65-88d4-fa18f25d48b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:18.957366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3808c-d09e-4a65-88d4-fa18f25d48b7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:18.996020 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4134/8180] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:19:19.000410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58393d4-3a58-4ea9-8aa6-576958307446 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:19.018392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58393d4-3a58-4ea9-8aa6-576958307446 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:19.030407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9d529-b934-4366-93cd-d912a42cfddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:19.031185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9d529-b934-4366-93cd-d912a42cfddb admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:19.031550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9d529-b934-4366-93cd-d912a42cfddb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:19.031902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9d529-b934-4366-93cd-d912a42cfddb admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:19.033095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9d529-b934-4366-93cd-d912a42cfddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:19.041097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9d529-b934-4366-93cd-d912a42cfddb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:19.094790 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4047/8181] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:19:18 2022] POST /identity/v3/projects => generated 393 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:19.098655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3808c-d09e-4a65-88d4-fa18f25d48b7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:19.109424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3808c-d09e-4a65-88d4-fa18f25d48b7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:19.202652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c25915-9a6d-49a7-993c-e06c1bd7d08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:19.203706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c25915-9a6d-49a7-993c-e06c1bd7d08a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:19.204155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c25915-9a6d-49a7-993c-e06c1bd7d08a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:19.204589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c25915-9a6d-49a7-993c-e06c1bd7d08a admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:19.205519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c25915-9a6d-49a7-993c-e06c1bd7d08a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:19.210124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c25915-9a6d-49a7-993c-e06c1bd7d08a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:19.243044 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4135/8182] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:19:19 2022] POST /identity/v3/users => generated 297 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:19.244656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9d529-b934-4366-93cd-d912a42cfddb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:19.258259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9d529-b934-4366-93cd-d912a42cfddb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:19.263385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c2e48-078e-4710-87d7-3a260d8f0763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:19.264359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c2e48-078e-4710-87d7-3a260d8f0763 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:19.264707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c2e48-078e-4710-87d7-3a260d8f0763 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:19.265091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c2e48-078e-4710-87d7-3a260d8f0763 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:19.265821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c2e48-078e-4710-87d7-3a260d8f0763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:19.267542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c2e48-078e-4710-87d7-3a260d8f0763 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:19.284683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c25915-9a6d-49a7-993c-e06c1bd7d08a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:19.284843 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4048/8183] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:19.303195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c25915-9a6d-49a7-993c-e06c1bd7d08a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:19.309927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b1618-abe8-43f6-8e4d-89dec463beb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:19.310789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b1618-abe8-43f6-8e4d-89dec463beb5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:19.311075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b1618-abe8-43f6-8e4d-89dec463beb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:19.311355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b1618-abe8-43f6-8e4d-89dec463beb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:19.312102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b1618-abe8-43f6-8e4d-89dec463beb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:19.313847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b1618-abe8-43f6-8e4d-89dec463beb5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:19.328602 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4136/8184] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:19.334401 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c2e48-078e-4710-87d7-3a260d8f0763 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:19.352412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c2e48-078e-4710-87d7-3a260d8f0763 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:19.381155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92d1a8-7c4c-4e19-861f-9cd2ff396b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:19.381882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92d1a8-7c4c-4e19-861f-9cd2ff396b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:19.382177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92d1a8-7c4c-4e19-861f-9cd2ff396b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:19.382550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92d1a8-7c4c-4e19-861f-9cd2ff396b92 admin admin] PATH_INFO: `/v3/projects/774e3ecab33c4271a7cb3fe040d6c5e7/users/c4dde169c2474129aca640fc6af8fbed/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:19.435081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92d1a8-7c4c-4e19-861f-9cd2ff396b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774e3ecab33c4271a7cb3fe040d6c5e7, user_id=c4dde169c2474129aca640fc6af8fbed, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:19.438131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92d1a8-7c4c-4e19-861f-9cd2ff396b92 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:19.468390 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4049/8185] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:19 2022] PUT /identity/v3/projects/774e3ecab33c4271a7cb3fe040d6c5e7/users/c4dde169c2474129aca640fc6af8fbed/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:19.482613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3edea-78db-4b2c-a8d1-e8bff4e8f32b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:19.483171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3edea-78db-4b2c-a8d1-e8bff4e8f32b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:19.483669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3edea-78db-4b2c-a8d1-e8bff4e8f32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:19.553602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-41d3edea-78db-4b2c-a8d1-e8bff4e8f32b None None] MFA Rules not processed for user `c4dde169c2474129aca640fc6af8fbed`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:19.555734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-41d3edea-78db-4b2c-a8d1-e8bff4e8f32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:19.589366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:19.683477 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4137/8186] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:19.710958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92d1a8-7c4c-4e19-861f-9cd2ff396b92 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:19.714801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bd92d1a8-7c4c-4e19-861f-9cd2ff396b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:19.806533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:19.850712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92d1a8-7c4c-4e19-861f-9cd2ff396b92 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:19.858993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948b841-5f8c-4f20-ada9-172611f3a3e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:19.858993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948b841-5f8c-4f20-ada9-172611f3a3e7 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:19.858993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948b841-5f8c-4f20-ada9-172611f3a3e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:19.859426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948b841-5f8c-4f20-ada9-172611f3a3e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:19.876976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948b841-5f8c-4f20-ada9-172611f3a3e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:19.878573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948b841-5f8c-4f20-ada9-172611f3a3e7 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:19.894337 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4050/8187] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:19 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:29.546244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3edea-78db-4b2c-a8d1-e8bff4e8f32b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:29.665403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3edea-78db-4b2c-a8d1-e8bff4e8f32b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:29.682326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:29.683807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:29.684266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:29.684733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] PATH_INFO: `/v3/users/c4dde169c2474129aca640fc6af8fbed` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:29.687147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4dde169c2474129aca640fc6af8fbed)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:29.707946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:29.785644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4dde169c2474129aca640fc6af8fbed'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:29.799387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4dde169c2474129aca640fc6af8fbed'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:29.813371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4dde169c2474129aca640fc6af8fbed'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:29.814632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4dde169c2474129aca640fc6af8fbed'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:29.815532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4dde169c2474129aca640fc6af8fbed'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:29.848772 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4138/8188] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:19:29 2022] DELETE /identity/v3/users/c4dde169c2474129aca640fc6af8fbed => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:29.850448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29ce46-91e8-479a-8597-fde013152057 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:29.850919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29ce46-91e8-479a-8597-fde013152057 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:29.851052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29ce46-91e8-479a-8597-fde013152057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:29.877155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3d29ce46-91e8-479a-8597-fde013152057 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:29.878403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3d29ce46-91e8-479a-8597-fde013152057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:29.971461 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4051/8189] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:29.982186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:29.983749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:30.109186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:30.162047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771613de-d5a5-4385-8deb-a731b8a86815 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:30.177168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642fcb2-4792-418c-92ea-b18d2f04972e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:30.178142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642fcb2-4792-418c-92ea-b18d2f04972e service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:30.178359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642fcb2-4792-418c-92ea-b18d2f04972e service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:30.178745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642fcb2-4792-418c-92ea-b18d2f04972e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:30.200481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642fcb2-4792-418c-92ea-b18d2f04972e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:30.202846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642fcb2-4792-418c-92ea-b18d2f04972e service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:30.226850 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4139/8190] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:30.511261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29ce46-91e8-479a-8597-fde013152057 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:30.512863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3d29ce46-91e8-479a-8597-fde013152057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:30.533835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29ce46-91e8-479a-8597-fde013152057 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:30.540822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916497c-618f-4acf-8203-bb2e0e30b453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:30.541944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916497c-618f-4acf-8203-bb2e0e30b453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:30.542964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916497c-618f-4acf-8203-bb2e0e30b453 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:30.544118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916497c-618f-4acf-8203-bb2e0e30b453 admin admin] PATH_INFO: `/v3/projects/774e3ecab33c4271a7cb3fe040d6c5e7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:30.548504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916497c-618f-4acf-8203-bb2e0e30b453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=774e3ecab33c4271a7cb3fe040d6c5e7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:30.551778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916497c-618f-4acf-8203-bb2e0e30b453 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:30.677209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0916497c-618f-4acf-8203-bb2e0e30b453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774e3ecab33c4271a7cb3fe040d6c5e7', 'request_id': 'req-0916497c-618f-4acf-8203-bb2e0e30b453'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:30.699138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0916497c-618f-4acf-8203-bb2e0e30b453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774e3ecab33c4271a7cb3fe040d6c5e7', 'request_id': 'req-0916497c-618f-4acf-8203-bb2e0e30b453'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:30.721957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0916497c-618f-4acf-8203-bb2e0e30b453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774e3ecab33c4271a7cb3fe040d6c5e7', 'request_id': 'req-0916497c-618f-4acf-8203-bb2e0e30b453'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:30.741883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0916497c-618f-4acf-8203-bb2e0e30b453 admin admin] The token cache is being invalidate because project 774e3ecab33c4271a7cb3fe040d6c5e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:19:30.742186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0916497c-618f-4acf-8203-bb2e0e30b453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:30.743053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0916497c-618f-4acf-8203-bb2e0e30b453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:30.745748 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4052/8191] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:19:30 2022] DELETE /identity/v3/projects/774e3ecab33c4271a7cb3fe040d6c5e7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:30.750769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a8920-8368-427a-a640-56b2935aa093 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:30.751238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a8920-8368-427a-a640-56b2935aa093 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:30.751351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a8920-8368-427a-a640-56b2935aa093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:30.794660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8f5a8920-8368-427a-a640-56b2935aa093 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:30.796660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8f5a8920-8368-427a-a640-56b2935aa093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:30.877024 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4140/8192] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:30.878361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40b1fd-8624-4e73-b0e5-6d5a8fadba32 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:30.879172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40b1fd-8624-4e73-b0e5-6d5a8fadba32 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:30.879665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40b1fd-8624-4e73-b0e5-6d5a8fadba32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:30.922093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ff40b1fd-8624-4e73-b0e5-6d5a8fadba32 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:30.923363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ff40b1fd-8624-4e73-b0e5-6d5a8fadba32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:30.960417 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4053/8193] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:19:30.964396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a8920-8368-427a-a640-56b2935aa093 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:30.975912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a8920-8368-427a-a640-56b2935aa093 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:30.980711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d90e72-5419-47da-a242-0511f19ebe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:30.981630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d90e72-5419-47da-a242-0511f19ebe90 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:30.981915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d90e72-5419-47da-a242-0511f19ebe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:30.982195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d90e72-5419-47da-a242-0511f19ebe90 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:30.982864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d90e72-5419-47da-a242-0511f19ebe90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:19:30.983176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d90e72-5419-47da-a242-0511f19ebe90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:30.984813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d90e72-5419-47da-a242-0511f19ebe90 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:31.011101 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4141/8194] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:19:31.012649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40b1fd-8624-4e73-b0e5-6d5a8fadba32 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:31.024033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40b1fd-8624-4e73-b0e5-6d5a8fadba32 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:31.029701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35e266-bc0d-4bbc-90b2-81be7a3fd612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:31.031062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35e266-bc0d-4bbc-90b2-81be7a3fd612 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:31.031587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35e266-bc0d-4bbc-90b2-81be7a3fd612 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:31.032028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35e266-bc0d-4bbc-90b2-81be7a3fd612 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:31.032932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35e266-bc0d-4bbc-90b2-81be7a3fd612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:31.035210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35e266-bc0d-4bbc-90b2-81be7a3fd612 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:31.087926 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4054/8195] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:19:31 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:31.091870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d90e72-5419-47da-a242-0511f19ebe90 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:31.107910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d90e72-5419-47da-a242-0511f19ebe90 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:31.214500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30784d69-2a7f-49cc-bd12-2090a0028466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:31.215479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30784d69-2a7f-49cc-bd12-2090a0028466 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:31.215998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30784d69-2a7f-49cc-bd12-2090a0028466 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:31.216306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30784d69-2a7f-49cc-bd12-2090a0028466 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:31.217004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30784d69-2a7f-49cc-bd12-2090a0028466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:31.221551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30784d69-2a7f-49cc-bd12-2090a0028466 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:31.256862 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4142/8196] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:19:31 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:31.264434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35e266-bc0d-4bbc-90b2-81be7a3fd612 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:31.281053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35e266-bc0d-4bbc-90b2-81be7a3fd612 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:31.288362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab418d-989b-4d1d-b2ff-ff8b8e0dca36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:31.289427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab418d-989b-4d1d-b2ff-ff8b8e0dca36 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:31.289958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab418d-989b-4d1d-b2ff-ff8b8e0dca36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:31.290389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab418d-989b-4d1d-b2ff-ff8b8e0dca36 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:31.291294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab418d-989b-4d1d-b2ff-ff8b8e0dca36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:31.294342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab418d-989b-4d1d-b2ff-ff8b8e0dca36 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:31.315427 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4055/8197] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:31.319266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30784d69-2a7f-49cc-bd12-2090a0028466 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:31.336942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30784d69-2a7f-49cc-bd12-2090a0028466 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:31.348658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964d9df-3bca-4533-bd25-dac63614a429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:31.349904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964d9df-3bca-4533-bd25-dac63614a429 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:31.350349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964d9df-3bca-4533-bd25-dac63614a429 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:31.350722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964d9df-3bca-4533-bd25-dac63614a429 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:31.351276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964d9df-3bca-4533-bd25-dac63614a429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:31.352926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964d9df-3bca-4533-bd25-dac63614a429 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:31.396988 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4143/8198] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:31.398411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab418d-989b-4d1d-b2ff-ff8b8e0dca36 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:31.423315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab418d-989b-4d1d-b2ff-ff8b8e0dca36 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:31.441563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f021573-0920-42d8-ba42-31cd688dfe52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:31.448206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f021573-0920-42d8-ba42-31cd688dfe52 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:31.449194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f021573-0920-42d8-ba42-31cd688dfe52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:31.449496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f021573-0920-42d8-ba42-31cd688dfe52 admin admin] PATH_INFO: `/v3/projects/590ad7390672421c811fbb9f973799f4/users/cb31238c2b054c629a6bb2de95cc913f/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:31.480935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f021573-0920-42d8-ba42-31cd688dfe52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=590ad7390672421c811fbb9f973799f4, user_id=cb31238c2b054c629a6bb2de95cc913f, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:31.484177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f021573-0920-42d8-ba42-31cd688dfe52 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:31.503312 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4056/8199] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:31 2022] PUT /identity/v3/projects/590ad7390672421c811fbb9f973799f4/users/cb31238c2b054c629a6bb2de95cc913f/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:31.514163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81072fa-7b88-4292-baac-cada3b3ec9d4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:31.514446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81072fa-7b88-4292-baac-cada3b3ec9d4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:31.514885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81072fa-7b88-4292-baac-cada3b3ec9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:31.557346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b81072fa-7b88-4292-baac-cada3b3ec9d4 None None] MFA Rules not processed for user `cb31238c2b054c629a6bb2de95cc913f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:31.558627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b81072fa-7b88-4292-baac-cada3b3ec9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:31.609407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:31.656357 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4144/8200] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:31.664241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f021573-0920-42d8-ba42-31cd688dfe52 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:31.701395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7f021573-0920-42d8-ba42-31cd688dfe52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:31.707999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:31.850219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f021573-0920-42d8-ba42-31cd688dfe52 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:31.863319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f48e1-846c-407e-80c7-2f4944349c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:31.871775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f48e1-846c-407e-80c7-2f4944349c6c service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:31.872131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f48e1-846c-407e-80c7-2f4944349c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:31.873810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f48e1-846c-407e-80c7-2f4944349c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:31.886745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f48e1-846c-407e-80c7-2f4944349c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:31.890403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f48e1-846c-407e-80c7-2f4944349c6c service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:31.906570 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4057/8201] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:32.944153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884919b8-fe63-4f0c-81e3-a823e498e55a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:32.944788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884919b8-fe63-4f0c-81e3-a823e498e55a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:32.944788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884919b8-fe63-4f0c-81e3-a823e498e55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:32.987008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-884919b8-fe63-4f0c-81e3-a823e498e55a None None] MFA Rules not processed for user `b5c96be8fbe1436bb3e84340ceda1917`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:32.988437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-884919b8-fe63-4f0c-81e3-a823e498e55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:33.112278 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4145/8202] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:41.898821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=25 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:43.100536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=35 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:53.435082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f48e1-846c-407e-80c7-2f4944349c6c service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:53.540038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f48e1-846c-407e-80c7-2f4944349c6c service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:53.545533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:53.546385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:53.547025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:53.547497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] PATH_INFO: `/v3/users/cb31238c2b054c629a6bb2de95cc913f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:53.548926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb31238c2b054c629a6bb2de95cc913f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:53.551233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:53.621605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb31238c2b054c629a6bb2de95cc913f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:53.635099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb31238c2b054c629a6bb2de95cc913f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:53.635854 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb31238c2b054c629a6bb2de95cc913f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:53.643971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb31238c2b054c629a6bb2de95cc913f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:53.655551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb31238c2b054c629a6bb2de95cc913f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:53.669551 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4058/8203] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:19:53 2022] DELETE /identity/v3/users/cb31238c2b054c629a6bb2de95cc913f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:53.671872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81721b3-f652-41bc-9ef1-8f789df669c1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:53.673264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81721b3-f652-41bc-9ef1-8f789df669c1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:53.673730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81721b3-f652-41bc-9ef1-8f789df669c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:53.728976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f81721b3-f652-41bc-9ef1-8f789df669c1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:53.731477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f81721b3-f652-41bc-9ef1-8f789df669c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:53.897065 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4146/8204] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:53.907804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:53.909860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:54.050785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133a7b8-b1e7-43fb-b8a6-54995278afdf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:54.055234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08ac93-a077-4923-be90-93121bae9854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:54.056171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08ac93-a077-4923-be90-93121bae9854 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:54.056473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08ac93-a077-4923-be90-93121bae9854 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:54.056754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08ac93-a077-4923-be90-93121bae9854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:54.071712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08ac93-a077-4923-be90-93121bae9854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:54.074062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08ac93-a077-4923-be90-93121bae9854 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:54.096217 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4059/8205] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:54.381681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81721b3-f652-41bc-9ef1-8f789df669c1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:54.383238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f81721b3-f652-41bc-9ef1-8f789df669c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:54.400669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81721b3-f652-41bc-9ef1-8f789df669c1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:54.414827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d35e30-5638-476f-a3c5-c82740b029c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:54.415508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d35e30-5638-476f-a3c5-c82740b029c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:54.415800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d35e30-5638-476f-a3c5-c82740b029c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:54.416090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d35e30-5638-476f-a3c5-c82740b029c3 admin admin] PATH_INFO: `/v3/projects/590ad7390672421c811fbb9f973799f4` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:54.417435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d35e30-5638-476f-a3c5-c82740b029c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=590ad7390672421c811fbb9f973799f4)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:54.419423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d35e30-5638-476f-a3c5-c82740b029c3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:54.530529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8d35e30-5638-476f-a3c5-c82740b029c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '590ad7390672421c811fbb9f973799f4', 'request_id': 'req-a8d35e30-5638-476f-a3c5-c82740b029c3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:54.539428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8d35e30-5638-476f-a3c5-c82740b029c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '590ad7390672421c811fbb9f973799f4', 'request_id': 'req-a8d35e30-5638-476f-a3c5-c82740b029c3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:54.548099 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8d35e30-5638-476f-a3c5-c82740b029c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '590ad7390672421c811fbb9f973799f4', 'request_id': 'req-a8d35e30-5638-476f-a3c5-c82740b029c3'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:54.568654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8d35e30-5638-476f-a3c5-c82740b029c3 admin admin] The token cache is being invalidate because project 590ad7390672421c811fbb9f973799f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:19:54.568928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8d35e30-5638-476f-a3c5-c82740b029c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:54.569950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a8d35e30-5638-476f-a3c5-c82740b029c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:54.571134 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4147/8206] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:19:54 2022] DELETE /identity/v3/projects/590ad7390672421c811fbb9f973799f4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:54.576233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a67539-0c57-444e-96be-4f3b9df61368 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:54.576733 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a67539-0c57-444e-96be-4f3b9df61368 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:54.576853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a67539-0c57-444e-96be-4f3b9df61368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:54.603508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-53a67539-0c57-444e-96be-4f3b9df61368 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:54.605705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-53a67539-0c57-444e-96be-4f3b9df61368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:54.629834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:54.701273 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4060/8207] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:54.709535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8690fe-797f-421c-b916-a309b71b51bc None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:54.710081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8690fe-797f-421c-b916-a309b71b51bc None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:54.710204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8690fe-797f-421c-b916-a309b71b51bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:54.737995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fa8690fe-797f-421c-b916-a309b71b51bc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:54.742316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fa8690fe-797f-421c-b916-a309b71b51bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:54.773270 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4148/8208] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:54.774104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a67539-0c57-444e-96be-4f3b9df61368 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:54.784432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a67539-0c57-444e-96be-4f3b9df61368 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:54.790183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e97fc-bc97-4981-8af1-01e49a2d26ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:54.790934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e97fc-bc97-4981-8af1-01e49a2d26ad admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:54.791313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e97fc-bc97-4981-8af1-01e49a2d26ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:54.791680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e97fc-bc97-4981-8af1-01e49a2d26ad admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:54.792435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e97fc-bc97-4981-8af1-01e49a2d26ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:19:54.792845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e97fc-bc97-4981-8af1-01e49a2d26ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:54.795054 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e97fc-bc97-4981-8af1-01e49a2d26ad admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:54.810314 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4061/8209] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:19:54.814550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8690fe-797f-421c-b916-a309b71b51bc None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:54.827158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8690fe-797f-421c-b916-a309b71b51bc None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:54.833995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd4b39-d93e-4470-bbd7-a30674d4bf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:54.834737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd4b39-d93e-4470-bbd7-a30674d4bf62 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:54.835122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd4b39-d93e-4470-bbd7-a30674d4bf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:54.836026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd4b39-d93e-4470-bbd7-a30674d4bf62 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:54.836776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd4b39-d93e-4470-bbd7-a30674d4bf62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:54.839024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd4b39-d93e-4470-bbd7-a30674d4bf62 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:54.889368 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4149/8210] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:19:54 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:54.893732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e97fc-bc97-4981-8af1-01e49a2d26ad admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:54.919136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e97fc-bc97-4981-8af1-01e49a2d26ad admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:55.015512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c4549-794f-4ea8-963e-259fcf0360b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:55.016131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c4549-794f-4ea8-963e-259fcf0360b3 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:55.016416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c4549-794f-4ea8-963e-259fcf0360b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:55.016690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c4549-794f-4ea8-963e-259fcf0360b3 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:55.017396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c4549-794f-4ea8-963e-259fcf0360b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:55.019387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c4549-794f-4ea8-963e-259fcf0360b3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:55.049992 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4062/8211] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:19:54 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:19:55.051409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd4b39-d93e-4470-bbd7-a30674d4bf62 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:55.063165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd4b39-d93e-4470-bbd7-a30674d4bf62 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:55.069979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88abbc3-6810-4ee4-bb3d-4c9bbda0f12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:55.069979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88abbc3-6810-4ee4-bb3d-4c9bbda0f12e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:55.069979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88abbc3-6810-4ee4-bb3d-4c9bbda0f12e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:55.070372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88abbc3-6810-4ee4-bb3d-4c9bbda0f12e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:55.071422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88abbc3-6810-4ee4-bb3d-4c9bbda0f12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:55.073362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88abbc3-6810-4ee4-bb3d-4c9bbda0f12e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:55.091647 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4150/8212] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:55.094695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c4549-794f-4ea8-963e-259fcf0360b3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:55.106762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c4549-794f-4ea8-963e-259fcf0360b3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:55.112199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6685b2d-8d63-482c-8a36-8587d6b2c507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:55.112831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6685b2d-8d63-482c-8a36-8587d6b2c507 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:55.113138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6685b2d-8d63-482c-8a36-8587d6b2c507 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:55.113462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6685b2d-8d63-482c-8a36-8587d6b2c507 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:55.114132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6685b2d-8d63-482c-8a36-8587d6b2c507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:55.116025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6685b2d-8d63-482c-8a36-8587d6b2c507 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:55.138236 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4063/8213] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:55.142422 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88abbc3-6810-4ee4-bb3d-4c9bbda0f12e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:55.170343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88abbc3-6810-4ee4-bb3d-4c9bbda0f12e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:55.175634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267a83f-1350-413a-b0bc-55598e3d60d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:55.176310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267a83f-1350-413a-b0bc-55598e3d60d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:55.176603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267a83f-1350-413a-b0bc-55598e3d60d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:55.177838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267a83f-1350-413a-b0bc-55598e3d60d2 admin admin] PATH_INFO: `/v3/projects/a875a4e25ecc4ec584cd44b4d5ea8517/users/a09ca6b2c604437b826a652a1d39ab8c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:55.195371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267a83f-1350-413a-b0bc-55598e3d60d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a875a4e25ecc4ec584cd44b4d5ea8517, user_id=a09ca6b2c604437b826a652a1d39ab8c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:55.198603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267a83f-1350-413a-b0bc-55598e3d60d2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:55.215680 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4151/8214] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:55 2022] PUT /identity/v3/projects/a875a4e25ecc4ec584cd44b4d5ea8517/users/a09ca6b2c604437b826a652a1d39ab8c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:19:55.226526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670109d-50f0-4050-bdca-dcaea873be08 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:55.226815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670109d-50f0-4050-bdca-dcaea873be08 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:55.227093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670109d-50f0-4050-bdca-dcaea873be08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:55.263675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1670109d-50f0-4050-bdca-dcaea873be08 None None] MFA Rules not processed for user `a09ca6b2c604437b826a652a1d39ab8c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:55.265516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1670109d-50f0-4050-bdca-dcaea873be08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:55.355936 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4064/8215] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:55.361280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267a83f-1350-413a-b0bc-55598e3d60d2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:55.383436 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:55.429680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267a83f-1350-413a-b0bc-55598e3d60d2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:55.434522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42d335-8006-444c-8893-1aa38d36f23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:55.435136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d335-8006-444c-8893-1aa38d36f23b admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:55.435430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d335-8006-444c-8893-1aa38d36f23b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:55.435712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d335-8006-444c-8893-1aa38d36f23b admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:55.436352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42d335-8006-444c-8893-1aa38d36f23b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:55.438308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42d335-8006-444c-8893-1aa38d36f23b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:55.465711 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4152/8216] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:19:55 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:19:55.469901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670109d-50f0-4050-bdca-dcaea873be08 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:55.481376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670109d-50f0-4050-bdca-dcaea873be08 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:55.509478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:55.564583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d6c49-dcb3-40fa-8800-49301ebb2137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:55.565247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d6c49-dcb3-40fa-8800-49301ebb2137 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:55.565538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d6c49-dcb3-40fa-8800-49301ebb2137 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:55.566010 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d6c49-dcb3-40fa-8800-49301ebb2137 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:55.566675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d6c49-dcb3-40fa-8800-49301ebb2137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:55.568599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d6c49-dcb3-40fa-8800-49301ebb2137 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:55.592944 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4065/8217] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:19:55 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:55.594855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42d335-8006-444c-8893-1aa38d36f23b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:55.605628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42d335-8006-444c-8893-1aa38d36f23b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:55.611208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9bc5d-1178-4910-a074-ba6e19157a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:55.611918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9bc5d-1178-4910-a074-ba6e19157a09 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:55.612302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9bc5d-1178-4910-a074-ba6e19157a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:55.612697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9bc5d-1178-4910-a074-ba6e19157a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:55.613483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9bc5d-1178-4910-a074-ba6e19157a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:55.615733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9bc5d-1178-4910-a074-ba6e19157a09 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:55.626844 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4153/8218] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:55.631562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d6c49-dcb3-40fa-8800-49301ebb2137 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:55.642062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d6c49-dcb3-40fa-8800-49301ebb2137 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:55.647346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b347bdb-43f0-46e4-acc4-af27b5207e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:55.649741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b347bdb-43f0-46e4-acc4-af27b5207e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:55.650058 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b347bdb-43f0-46e4-acc4-af27b5207e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:55.650348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b347bdb-43f0-46e4-acc4-af27b5207e2b admin admin] PATH_INFO: `/v3/projects/f848ff2870b44dcc8bf91a9803cd4326/users/93f060a23ad64f7090c260659256510d/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:55.669650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b347bdb-43f0-46e4-acc4-af27b5207e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f848ff2870b44dcc8bf91a9803cd4326, user_id=93f060a23ad64f7090c260659256510d, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:55.672471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b347bdb-43f0-46e4-acc4-af27b5207e2b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:55.690252 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4066/8219] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:55 2022] PUT /identity/v3/projects/f848ff2870b44dcc8bf91a9803cd4326/users/93f060a23ad64f7090c260659256510d/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:19:55.694397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9bc5d-1178-4910-a074-ba6e19157a09 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:55.705763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9bc5d-1178-4910-a074-ba6e19157a09 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:55.775472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf65c3-1d95-459e-aa07-abcc7e7b447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:55.776331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf65c3-1d95-459e-aa07-abcc7e7b447a admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:55.776681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf65c3-1d95-459e-aa07-abcc7e7b447a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:55.777061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf65c3-1d95-459e-aa07-abcc7e7b447a admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:55.777813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf65c3-1d95-459e-aa07-abcc7e7b447a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:55.779486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf65c3-1d95-459e-aa07-abcc7e7b447a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:55.794078 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4154/8220] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:55.795287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b347bdb-43f0-46e4-acc4-af27b5207e2b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:55.806528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b347bdb-43f0-46e4-acc4-af27b5207e2b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:55.812113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df0aec-186f-4dec-8bd7-11cb4121f673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:55.812948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df0aec-186f-4dec-8bd7-11cb4121f673 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:55.813327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df0aec-186f-4dec-8bd7-11cb4121f673 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:55.813723 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df0aec-186f-4dec-8bd7-11cb4121f673 admin admin] PATH_INFO: `/v3/projects/f848ff2870b44dcc8bf91a9803cd4326/users/93f060a23ad64f7090c260659256510d/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:55.815963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df0aec-186f-4dec-8bd7-11cb4121f673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f848ff2870b44dcc8bf91a9803cd4326, user_id=93f060a23ad64f7090c260659256510d, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:55.819731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df0aec-186f-4dec-8bd7-11cb4121f673 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:55.831796 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4067/8221] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:55 2022] PUT /identity/v3/projects/f848ff2870b44dcc8bf91a9803cd4326/users/93f060a23ad64f7090c260659256510d/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:55.843390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22acdc7-f7fc-4700-832f-16e55edb6142 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:55.843680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22acdc7-f7fc-4700-832f-16e55edb6142 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:55.843959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22acdc7-f7fc-4700-832f-16e55edb6142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:55.873985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a22acdc7-f7fc-4700-832f-16e55edb6142 None None] MFA Rules not processed for user `93f060a23ad64f7090c260659256510d`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:55.875223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a22acdc7-f7fc-4700-832f-16e55edb6142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:55.944488 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4155/8222] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:19:55.956142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df0aec-186f-4dec-8bd7-11cb4121f673 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:55.957946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-68df0aec-186f-4dec-8bd7-11cb4121f673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:56.039850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df0aec-186f-4dec-8bd7-11cb4121f673 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:56.045760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04db2e-53d6-4eb9-845a-ba9bb0257e4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:56.046517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04db2e-53d6-4eb9-845a-ba9bb0257e4b service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:56.046811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04db2e-53d6-4eb9-845a-ba9bb0257e4b service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:56.047092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04db2e-53d6-4eb9-845a-ba9bb0257e4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:56.057886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04db2e-53d6-4eb9-845a-ba9bb0257e4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:56.059484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04db2e-53d6-4eb9-845a-ba9bb0257e4b service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:56.070220 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4068/8223] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:56.223282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22acdc7-f7fc-4700-832f-16e55edb6142 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:56.243111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:56.298492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22acdc7-f7fc-4700-832f-16e55edb6142 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:56.304453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:56.305249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:56.305643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:56.306023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] PATH_INFO: `/v3/users/a09ca6b2c604437b826a652a1d39ab8c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:56.307271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a09ca6b2c604437b826a652a1d39ab8c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:56.309488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:56.387778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a09ca6b2c604437b826a652a1d39ab8c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:56.401589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a09ca6b2c604437b826a652a1d39ab8c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:56.413459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a09ca6b2c604437b826a652a1d39ab8c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:56.414855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a09ca6b2c604437b826a652a1d39ab8c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:56.415603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a09ca6b2c604437b826a652a1d39ab8c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:56.429245 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4156/8224] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:19:56 2022] DELETE /identity/v3/users/a09ca6b2c604437b826a652a1d39ab8c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:56.433824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12967b-67ce-4b1a-8129-05c653443592 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:56.434124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12967b-67ce-4b1a-8129-05c653443592 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:56.434410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12967b-67ce-4b1a-8129-05c653443592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:56.459276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4a12967b-67ce-4b1a-8129-05c653443592 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:56.460515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4a12967b-67ce-4b1a-8129-05c653443592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:56.527633 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4069/8225] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:56.538148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:56.539601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:56.602553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f015d52-1a12-44d6-93e4-eadc42cbe413 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:56.606247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e10f5-25bd-4fc6-8353-606884aeb75c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:56.607107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e10f5-25bd-4fc6-8353-606884aeb75c service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:56.607388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e10f5-25bd-4fc6-8353-606884aeb75c service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:56.607663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e10f5-25bd-4fc6-8353-606884aeb75c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:56.617380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e10f5-25bd-4fc6-8353-606884aeb75c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:56.618865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e10f5-25bd-4fc6-8353-606884aeb75c service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:56.629298 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4157/8226] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:57.240876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12967b-67ce-4b1a-8129-05c653443592 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:57.242511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4a12967b-67ce-4b1a-8129-05c653443592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:57.251348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:57.259243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12967b-67ce-4b1a-8129-05c653443592 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:57.264509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:57.266017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:57.266378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:57.266978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] PATH_INFO: `/v3/projects/a875a4e25ecc4ec584cd44b4d5ea8517` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:57.268155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a875a4e25ecc4ec584cd44b4d5ea8517)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:57.270181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:57.319221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a875a4e25ecc4ec584cd44b4d5ea8517', 'request_id': 'req-28a9b077-e8bd-490b-975a-47c337ac832b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:57.326705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a875a4e25ecc4ec584cd44b4d5ea8517', 'request_id': 'req-28a9b077-e8bd-490b-975a-47c337ac832b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:57.335225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a875a4e25ecc4ec584cd44b4d5ea8517', 'request_id': 'req-28a9b077-e8bd-490b-975a-47c337ac832b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:57.345944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] The token cache is being invalidate because project a875a4e25ecc4ec584cd44b4d5ea8517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:19:57.346456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:57.347130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:57.349290 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4070/8227] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:19:57 2022] DELETE /identity/v3/projects/a875a4e25ecc4ec584cd44b4d5ea8517 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:57.353216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e10f5-25bd-4fc6-8353-606884aeb75c service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:57.363296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e10f5-25bd-4fc6-8353-606884aeb75c service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:57.399455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:57.424790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46fcd0-0c5b-4c77-97c2-b13ac4b3783a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:57.425462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46fcd0-0c5b-4c77-97c2-b13ac4b3783a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:57.425751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46fcd0-0c5b-4c77-97c2-b13ac4b3783a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:57.426030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46fcd0-0c5b-4c77-97c2-b13ac4b3783a admin admin] PATH_INFO: `/v3/users/93f060a23ad64f7090c260659256510d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:57.427206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46fcd0-0c5b-4c77-97c2-b13ac4b3783a admin admin] RBAC: Authorizing `identity:delete_user(user_id=93f060a23ad64f7090c260659256510d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:57.429167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46fcd0-0c5b-4c77-97c2-b13ac4b3783a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:57.484720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd46fcd0-0c5b-4c77-97c2-b13ac4b3783a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93f060a23ad64f7090c260659256510d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:57.504127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd46fcd0-0c5b-4c77-97c2-b13ac4b3783a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93f060a23ad64f7090c260659256510d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:57.514060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd46fcd0-0c5b-4c77-97c2-b13ac4b3783a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93f060a23ad64f7090c260659256510d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:57.515120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd46fcd0-0c5b-4c77-97c2-b13ac4b3783a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93f060a23ad64f7090c260659256510d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:57.516095 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-fd46fcd0-0c5b-4c77-97c2-b13ac4b3783a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93f060a23ad64f7090c260659256510d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:57.533853 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4158/8228] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:19:57 2022] DELETE /identity/v3/users/93f060a23ad64f7090c260659256510d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:57.864652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:57.869840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:57.893800 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9b077-e8bd-490b-975a-47c337ac832b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:57.971924 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5818a-e619-4162-a252-90fb1866e4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:57.972618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5818a-e619-4162-a252-90fb1866e4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:57.972936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5818a-e619-4162-a252-90fb1866e4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:57.973253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5818a-e619-4162-a252-90fb1866e4e4 admin admin] PATH_INFO: `/v3/projects/f848ff2870b44dcc8bf91a9803cd4326` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:57.974929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5818a-e619-4162-a252-90fb1866e4e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f848ff2870b44dcc8bf91a9803cd4326)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:57.976865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5818a-e619-4162-a252-90fb1866e4e4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:58.026966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3be5818a-e619-4162-a252-90fb1866e4e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f848ff2870b44dcc8bf91a9803cd4326', 'request_id': 'req-3be5818a-e619-4162-a252-90fb1866e4e4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:58.034625 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3be5818a-e619-4162-a252-90fb1866e4e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f848ff2870b44dcc8bf91a9803cd4326', 'request_id': 'req-3be5818a-e619-4162-a252-90fb1866e4e4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:58.041276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3be5818a-e619-4162-a252-90fb1866e4e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f848ff2870b44dcc8bf91a9803cd4326', 'request_id': 'req-3be5818a-e619-4162-a252-90fb1866e4e4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:58.052494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3be5818a-e619-4162-a252-90fb1866e4e4 admin admin] The token cache is being invalidate because project f848ff2870b44dcc8bf91a9803cd4326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:19:58.052718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3be5818a-e619-4162-a252-90fb1866e4e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:58.053421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3be5818a-e619-4162-a252-90fb1866e4e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:19:58.054985 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4071/8229] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:19:57 2022] DELETE /identity/v3/projects/f848ff2870b44dcc8bf91a9803cd4326 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:58.059849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bbd17-69ce-4c8a-96d5-a7267567b5a3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.060238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bbd17-69ce-4c8a-96d5-a7267567b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.060583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bbd17-69ce-4c8a-96d5-a7267567b5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.081005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-da4bbd17-69ce-4c8a-96d5-a7267567b5a3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:58.082317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-da4bbd17-69ce-4c8a-96d5-a7267567b5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:58.146319 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4159/8230] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:58.154094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a3155-62e4-48f7-b936-ea1c7353aee8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.154391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a3155-62e4-48f7-b936-ea1c7353aee8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.154670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a3155-62e4-48f7-b936-ea1c7353aee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.175218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-995a3155-62e4-48f7-b936-ea1c7353aee8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:58.176440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-995a3155-62e4-48f7-b936-ea1c7353aee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:58.185058 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4072/8231] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:58.189462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bbd17-69ce-4c8a-96d5-a7267567b5a3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:58.200520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bbd17-69ce-4c8a-96d5-a7267567b5a3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:58.209475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62082c2a-b5e1-43cf-8479-62f72ed31974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:58.210036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62082c2a-b5e1-43cf-8479-62f72ed31974 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.210316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62082c2a-b5e1-43cf-8479-62f72ed31974 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.210591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62082c2a-b5e1-43cf-8479-62f72ed31974 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.211255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62082c2a-b5e1-43cf-8479-62f72ed31974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:19:58.211562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62082c2a-b5e1-43cf-8479-62f72ed31974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:58.213259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62082c2a-b5e1-43cf-8479-62f72ed31974 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:58.230867 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4160/8232] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:19:58.232637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a3155-62e4-48f7-b936-ea1c7353aee8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:58.251385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a3155-62e4-48f7-b936-ea1c7353aee8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:58.256500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8d169-b178-4bfd-839f-737114bdd9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:58.257266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8d169-b178-4bfd-839f-737114bdd9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.257628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8d169-b178-4bfd-839f-737114bdd9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.257983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8d169-b178-4bfd-839f-737114bdd9b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.258705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8d169-b178-4bfd-839f-737114bdd9b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:58.260851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8d169-b178-4bfd-839f-737114bdd9b3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:58.266971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:58.294894 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4073/8233] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:19:58 2022] POST /identity/v3/projects => generated 409 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:58.298766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62082c2a-b5e1-43cf-8479-62f72ed31974 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:58.310062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62082c2a-b5e1-43cf-8479-62f72ed31974 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:58.357769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:58.375202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ae296-eccf-40ee-9242-8e47bb4f0150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:58.375835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ae296-eccf-40ee-9242-8e47bb4f0150 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.376122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ae296-eccf-40ee-9242-8e47bb4f0150 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.376398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ae296-eccf-40ee-9242-8e47bb4f0150 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.377075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ae296-eccf-40ee-9242-8e47bb4f0150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:58.379032 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ae296-eccf-40ee-9242-8e47bb4f0150 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:58.405518 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4161/8234] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:19:58 2022] POST /identity/v3/users => generated 305 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:58.406875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8d169-b178-4bfd-839f-737114bdd9b3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:58.418553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8d169-b178-4bfd-839f-737114bdd9b3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:58.423420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77264b20-8f47-4054-b70b-2faa25a8e5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:58.424053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77264b20-8f47-4054-b70b-2faa25a8e5dc admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.424384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77264b20-8f47-4054-b70b-2faa25a8e5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.424626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77264b20-8f47-4054-b70b-2faa25a8e5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.425310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77264b20-8f47-4054-b70b-2faa25a8e5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:58.426970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77264b20-8f47-4054-b70b-2faa25a8e5dc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:58.440890 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4074/8235] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:58.442342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ae296-eccf-40ee-9242-8e47bb4f0150 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:58.452856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ae296-eccf-40ee-9242-8e47bb4f0150 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:58.457587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7615ba0-7c89-4673-a642-dea3fce38b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:58.458709 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7615ba0-7c89-4673-a642-dea3fce38b7b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.458994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7615ba0-7c89-4673-a642-dea3fce38b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.459272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7615ba0-7c89-4673-a642-dea3fce38b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.459920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7615ba0-7c89-4673-a642-dea3fce38b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:58.461574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7615ba0-7c89-4673-a642-dea3fce38b7b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:58.472902 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4162/8236] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:58.477806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77264b20-8f47-4054-b70b-2faa25a8e5dc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:58.488259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77264b20-8f47-4054-b70b-2faa25a8e5dc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:58.493388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196683c8-2642-4550-8ee6-c9a6c12e3542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:58.494165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196683c8-2642-4550-8ee6-c9a6c12e3542 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.494455 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196683c8-2642-4550-8ee6-c9a6c12e3542 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.494736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196683c8-2642-4550-8ee6-c9a6c12e3542 admin admin] PATH_INFO: `/v3/projects/266eb337a0f241e2a3a6bb0f3b874cfa/users/2671fbffe48546f29ae69a2205e650fe/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.509979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196683c8-2642-4550-8ee6-c9a6c12e3542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=266eb337a0f241e2a3a6bb0f3b874cfa, user_id=2671fbffe48546f29ae69a2205e650fe, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:58.512710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196683c8-2642-4550-8ee6-c9a6c12e3542 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:58.526980 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4075/8237] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:58 2022] PUT /identity/v3/projects/266eb337a0f241e2a3a6bb0f3b874cfa/users/2671fbffe48546f29ae69a2205e650fe/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:19:58.539152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea68801-749b-4062-b166-e50ce775cb76 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.539540 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea68801-749b-4062-b166-e50ce775cb76 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.539959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea68801-749b-4062-b166-e50ce775cb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.574319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-dea68801-749b-4062-b166-e50ce775cb76 None None] MFA Rules not processed for user `2671fbffe48546f29ae69a2205e650fe`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:58.575494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dea68801-749b-4062-b166-e50ce775cb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:58.645261 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4163/8238] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:58 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:19:58.649916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196683c8-2642-4550-8ee6-c9a6c12e3542 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:58.717205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196683c8-2642-4550-8ee6-c9a6c12e3542 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:58.722022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d9e7a-4269-43b2-8e5d-6cbafc0e5125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:58.722660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d9e7a-4269-43b2-8e5d-6cbafc0e5125 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.722920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d9e7a-4269-43b2-8e5d-6cbafc0e5125 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.723202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d9e7a-4269-43b2-8e5d-6cbafc0e5125 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.724370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d9e7a-4269-43b2-8e5d-6cbafc0e5125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:58.726358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d9e7a-4269-43b2-8e5d-6cbafc0e5125 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:58.764413 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4076/8239] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:19:58 2022] POST /identity/v3/projects => generated 409 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:19:58.765822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea68801-749b-4062-b166-e50ce775cb76 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:58.777270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea68801-749b-4062-b166-e50ce775cb76 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:58.845327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9048d3c-7448-4dac-a2f9-f85eaa0455cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:58.845952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9048d3c-7448-4dac-a2f9-f85eaa0455cb admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.846237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9048d3c-7448-4dac-a2f9-f85eaa0455cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.846516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9048d3c-7448-4dac-a2f9-f85eaa0455cb admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.847786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9048d3c-7448-4dac-a2f9-f85eaa0455cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:58.850329 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9048d3c-7448-4dac-a2f9-f85eaa0455cb admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:58.878453 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4164/8240] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:19:58 2022] POST /identity/v3/users => generated 311 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:19:58.882999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d9e7a-4269-43b2-8e5d-6cbafc0e5125 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:58.894985 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d9e7a-4269-43b2-8e5d-6cbafc0e5125 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:58.900659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a26971-a9f6-451b-839c-d1619c5bb031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:58.901507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a26971-a9f6-451b-839c-d1619c5bb031 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.901765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a26971-a9f6-451b-839c-d1619c5bb031 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.902241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a26971-a9f6-451b-839c-d1619c5bb031 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.902977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a26971-a9f6-451b-839c-d1619c5bb031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:58.904669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a26971-a9f6-451b-839c-d1619c5bb031 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:58.919082 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4077/8241] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:58.923262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9048d3c-7448-4dac-a2f9-f85eaa0455cb admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:58.934369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9048d3c-7448-4dac-a2f9-f85eaa0455cb admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:58.939175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711f760-bc5d-4cb4-82c0-94f6340a211c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:58.939930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711f760-bc5d-4cb4-82c0-94f6340a211c admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:58.940323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711f760-bc5d-4cb4-82c0-94f6340a211c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:58.941083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711f760-bc5d-4cb4-82c0-94f6340a211c admin admin] PATH_INFO: `/v3/projects/2f0826098cee408ead3ec19041cfdc6a/users/b202e158639649f79fcbf72184a04490/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:58.957249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711f760-bc5d-4cb4-82c0-94f6340a211c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0826098cee408ead3ec19041cfdc6a, user_id=b202e158639649f79fcbf72184a04490, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:58.959871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711f760-bc5d-4cb4-82c0-94f6340a211c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:58.977758 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4165/8242] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:58 2022] PUT /identity/v3/projects/2f0826098cee408ead3ec19041cfdc6a/users/b202e158639649f79fcbf72184a04490/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:19:58.978995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a26971-a9f6-451b-839c-d1619c5bb031 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:58.992376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a26971-a9f6-451b-839c-d1619c5bb031 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:59.062676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230d93c-3dee-49fc-8fc3-669b287f876a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:59.063384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d93c-3dee-49fc-8fc3-669b287f876a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:59.063743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d93c-3dee-49fc-8fc3-669b287f876a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:59.064103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d93c-3dee-49fc-8fc3-669b287f876a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:59.064846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230d93c-3dee-49fc-8fc3-669b287f876a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:59.066783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230d93c-3dee-49fc-8fc3-669b287f876a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:59.078145 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4078/8243] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:19:59.082400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711f760-bc5d-4cb4-82c0-94f6340a211c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:59.094166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711f760-bc5d-4cb4-82c0-94f6340a211c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:59.099858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1883d7-009b-4660-9a65-1f453974204e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:59.100543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1883d7-009b-4660-9a65-1f453974204e admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:59.101098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1883d7-009b-4660-9a65-1f453974204e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:59.101451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1883d7-009b-4660-9a65-1f453974204e admin admin] PATH_INFO: `/v3/projects/2f0826098cee408ead3ec19041cfdc6a/users/b202e158639649f79fcbf72184a04490/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:59.103528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1883d7-009b-4660-9a65-1f453974204e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0826098cee408ead3ec19041cfdc6a, user_id=b202e158639649f79fcbf72184a04490, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:59.106340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1883d7-009b-4660-9a65-1f453974204e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:59.125402 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4166/8244] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:19:59 2022] PUT /identity/v3/projects/2f0826098cee408ead3ec19041cfdc6a/users/b202e158639649f79fcbf72184a04490/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:19:59.139090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1e4d7-bb64-4353-bcb1-e35b5a4376c4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:59.139405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1e4d7-bb64-4353-bcb1-e35b5a4376c4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:59.139684 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1e4d7-bb64-4353-bcb1-e35b5a4376c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:59.179920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0cc1e4d7-bb64-4353-bcb1-e35b5a4376c4 None None] MFA Rules not processed for user `b202e158639649f79fcbf72184a04490`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:19:59.181202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0cc1e4d7-bb64-4353-bcb1-e35b5a4376c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:59.192171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:59.267872 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4079/8245] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:19:59 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:19:59.279311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1883d7-009b-4660-9a65-1f453974204e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:59.281085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0a1883d7-009b-4660-9a65-1f453974204e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:19:59.288858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:19:59.355765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1883d7-009b-4660-9a65-1f453974204e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:59.359740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b8526-261b-4971-a8fa-d18791e8dd7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:59.360349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b8526-261b-4971-a8fa-d18791e8dd7b service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:59.360633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b8526-261b-4971-a8fa-d18791e8dd7b service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:59.360932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b8526-261b-4971-a8fa-d18791e8dd7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:59.373086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b8526-261b-4971-a8fa-d18791e8dd7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:59.374690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b8526-261b-4971-a8fa-d18791e8dd7b service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:59.387462 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4167/8246] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:59 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:19:59.550681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1e4d7-bb64-4353-bcb1-e35b5a4376c4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:19:59.563077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1e4d7-bb64-4353-bcb1-e35b5a4376c4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:19:59.570018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26226bc-a9d7-402c-b1e9-4e7832bc689b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:19:59.570735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26226bc-a9d7-402c-b1e9-4e7832bc689b service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:19:59.571020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26226bc-a9d7-402c-b1e9-4e7832bc689b service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:19:59.571294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26226bc-a9d7-402c-b1e9-4e7832bc689b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:19:59.635470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26226bc-a9d7-402c-b1e9-4e7832bc689b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:19:59.636763 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26226bc-a9d7-402c-b1e9-4e7832bc689b service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:19:59.648208 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4080/8247] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:19:59 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:20:00.025705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b8526-261b-4971-a8fa-d18791e8dd7b service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:00.025705 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a68b8526-261b-4971-a8fa-d18791e8dd7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:00.098737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b8526-261b-4971-a8fa-d18791e8dd7b service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:00.103495 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:00.104122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:00.104551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:00.104842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] PATH_INFO: `/v3/users/2a3339181fa7493db149e408d85e37f2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:00.108092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a3339181fa7493db149e408d85e37f2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:00.110182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:00.162333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a3339181fa7493db149e408d85e37f2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:00.174338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a3339181fa7493db149e408d85e37f2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:00.183555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a3339181fa7493db149e408d85e37f2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:00.184181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a3339181fa7493db149e408d85e37f2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:00.184790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a3339181fa7493db149e408d85e37f2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:00.197645 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4168/8248] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:20:00 2022] DELETE /identity/v3/users/2a3339181fa7493db149e408d85e37f2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:00.202412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802a1c1-0ee7-4d2a-9d67-a806cc6063a5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:00.202712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802a1c1-0ee7-4d2a-9d67-a806cc6063a5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:00.202999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802a1c1-0ee7-4d2a-9d67-a806cc6063a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:00.222311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a802a1c1-0ee7-4d2a-9d67-a806cc6063a5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:00.224095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a802a1c1-0ee7-4d2a-9d67-a806cc6063a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:00.283542 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4081/8249] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:00.294023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:00.295444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:00.327108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:00.359373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca97ac2-7b71-4cda-8dc1-273374a9ac96 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:00.363205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed73c77-38df-4e25-a68b-52cc4336cc36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:00.363805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed73c77-38df-4e25-a68b-52cc4336cc36 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:00.364093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed73c77-38df-4e25-a68b-52cc4336cc36 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:00.364943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed73c77-38df-4e25-a68b-52cc4336cc36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:00.374198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed73c77-38df-4e25-a68b-52cc4336cc36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:00.375650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed73c77-38df-4e25-a68b-52cc4336cc36 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:00.385112 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4169/8250] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:20:00.571663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802a1c1-0ee7-4d2a-9d67-a806cc6063a5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:00.573828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a802a1c1-0ee7-4d2a-9d67-a806cc6063a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:00.593697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802a1c1-0ee7-4d2a-9d67-a806cc6063a5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:00.600314 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:00.600970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:00.601261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:00.601584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248 admin admin] PATH_INFO: `/v3/projects/c29640aa4d4643e2bffa43bab367012b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:00.602650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29640aa4d4643e2bffa43bab367012b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:00.604551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:00.645239 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:00.653106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29640aa4d4643e2bffa43bab367012b', 'request_id': 'req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:00.662378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29640aa4d4643e2bffa43bab367012b', 'request_id': 'req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:00.669526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29640aa4d4643e2bffa43bab367012b', 'request_id': 'req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:00.679637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248 admin admin] The token cache is being invalidate because project c29640aa4d4643e2bffa43bab367012b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:20:00.679929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:00.680579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-dc307b81-9fcb-4e6f-bdcc-9f59a0e66248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:00.682089 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4082/8251] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:20:00 2022] DELETE /identity/v3/projects/c29640aa4d4643e2bffa43bab367012b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:00.689036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b786a6d-96bf-4ba1-a99d-2b77ee397bf3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:00.689266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b786a6d-96bf-4ba1-a99d-2b77ee397bf3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:00.689551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b786a6d-96bf-4ba1-a99d-2b77ee397bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:00.710410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2b786a6d-96bf-4ba1-a99d-2b77ee397bf3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:00.711647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b786a6d-96bf-4ba1-a99d-2b77ee397bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:00.772410 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4170/8252] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:00.861942 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dd825-67e5-4db2-92b7-cff632517c7f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:00.862230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dd825-67e5-4db2-92b7-cff632517c7f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:00.862864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dd825-67e5-4db2-92b7-cff632517c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:00.882963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-530dd825-67e5-4db2-92b7-cff632517c7f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:00.884164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-530dd825-67e5-4db2-92b7-cff632517c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:00.891983 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4083/8253] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:00.896806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b786a6d-96bf-4ba1-a99d-2b77ee397bf3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:00.906604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b786a6d-96bf-4ba1-a99d-2b77ee397bf3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:00.911136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a6cb9-a811-4783-84a9-56a602bc18ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:00.911736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a6cb9-a811-4783-84a9-56a602bc18ef admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:00.912024 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a6cb9-a811-4783-84a9-56a602bc18ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:00.912306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a6cb9-a811-4783-84a9-56a602bc18ef admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:00.912987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a6cb9-a811-4783-84a9-56a602bc18ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:20:00.913307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a6cb9-a811-4783-84a9-56a602bc18ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:00.914941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a6cb9-a811-4783-84a9-56a602bc18ef admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:00.926336 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4171/8254] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:20:00.930386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dd825-67e5-4db2-92b7-cff632517c7f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:00.939517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dd825-67e5-4db2-92b7-cff632517c7f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:00.944608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcecf1-94a1-4088-b832-d89f12944fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:00.945228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcecf1-94a1-4088-b832-d89f12944fda admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:00.945510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcecf1-94a1-4088-b832-d89f12944fda admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:00.945784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcecf1-94a1-4088-b832-d89f12944fda admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:00.946410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dcecf1-94a1-4088-b832-d89f12944fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:00.948315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dcecf1-94a1-4088-b832-d89f12944fda admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:00.976406 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4084/8255] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:20:00 2022] POST /identity/v3/projects => generated 357 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:00.980733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a6cb9-a811-4783-84a9-56a602bc18ef admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:00.991337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a6cb9-a811-4783-84a9-56a602bc18ef admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:01.063507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147f53b-ec47-4a90-a491-2a326f9a48a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:01.064419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f53b-ec47-4a90-a491-2a326f9a48a5 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:01.064860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f53b-ec47-4a90-a491-2a326f9a48a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:01.065326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f53b-ec47-4a90-a491-2a326f9a48a5 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:01.066327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147f53b-ec47-4a90-a491-2a326f9a48a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:01.070811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147f53b-ec47-4a90-a491-2a326f9a48a5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:01.099271 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4172/8256] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:20:00 2022] POST /identity/v3/users => generated 279 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:20:01.103252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcecf1-94a1-4088-b832-d89f12944fda admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:01.118183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcecf1-94a1-4088-b832-d89f12944fda admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:01.125320 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1db3b-91ef-4396-bea9-5682b6b2603e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:01.125953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1db3b-91ef-4396-bea9-5682b6b2603e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:01.126244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1db3b-91ef-4396-bea9-5682b6b2603e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:01.126537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1db3b-91ef-4396-bea9-5682b6b2603e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:01.127439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1db3b-91ef-4396-bea9-5682b6b2603e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:01.130560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1db3b-91ef-4396-bea9-5682b6b2603e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:01.144092 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4085/8257] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:01.149002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147f53b-ec47-4a90-a491-2a326f9a48a5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:01.160270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147f53b-ec47-4a90-a491-2a326f9a48a5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:01.165364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9c1d1-461c-4bee-971c-712ec04afdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:01.166246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9c1d1-461c-4bee-971c-712ec04afdcd admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:01.166529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9c1d1-461c-4bee-971c-712ec04afdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:01.166815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9c1d1-461c-4bee-971c-712ec04afdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:01.167442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9c1d1-461c-4bee-971c-712ec04afdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:01.169088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9c1d1-461c-4bee-971c-712ec04afdcd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:01.182050 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4173/8258] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:20:01.186291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1db3b-91ef-4396-bea9-5682b6b2603e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:01.200451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1db3b-91ef-4396-bea9-5682b6b2603e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:01.205298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd81387-c11d-4e21-af2a-f6c459293d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:01.205959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd81387-c11d-4e21-af2a-f6c459293d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:01.206248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd81387-c11d-4e21-af2a-f6c459293d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:01.206530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd81387-c11d-4e21-af2a-f6c459293d89 admin admin] PATH_INFO: `/v3/projects/7b1dbe8cc2a144c68f7b85b47c26a302/users/e3b33ad8cf6f4543b6cf451f504eb2cf/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:01.221888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd81387-c11d-4e21-af2a-f6c459293d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1dbe8cc2a144c68f7b85b47c26a302, user_id=e3b33ad8cf6f4543b6cf451f504eb2cf, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:01.224644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd81387-c11d-4e21-af2a-f6c459293d89 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:01.239737 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4086/8259] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:01 2022] PUT /identity/v3/projects/7b1dbe8cc2a144c68f7b85b47c26a302/users/e3b33ad8cf6f4543b6cf451f504eb2cf/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:20:01.249502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35bf09-ba75-476a-88af-602bf7b1f4b6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:01.249797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35bf09-ba75-476a-88af-602bf7b1f4b6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:01.250083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35bf09-ba75-476a-88af-602bf7b1f4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:01.279272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7b35bf09-ba75-476a-88af-602bf7b1f4b6 None None] MFA Rules not processed for user `e3b33ad8cf6f4543b6cf451f504eb2cf`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:01.280504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7b35bf09-ba75-476a-88af-602bf7b1f4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:01.307563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:01.352397 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4174/8260] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:01.365092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd81387-c11d-4e21-af2a-f6c459293d89 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:01.366987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1bd81387-c11d-4e21-af2a-f6c459293d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:01.367679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35bf09-ba75-476a-88af-602bf7b1f4b6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:01.369092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7b35bf09-ba75-476a-88af-602bf7b1f4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:01.434389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35bf09-ba75-476a-88af-602bf7b1f4b6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:01.439995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:01.440611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:01.440911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:01.441218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] PATH_INFO: `/v3/users/2671fbffe48546f29ae69a2205e650fe` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:01.441793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd81387-c11d-4e21-af2a-f6c459293d89 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:01.443164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2671fbffe48546f29ae69a2205e650fe)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:01.445412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:01.446318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1baa42-8b43-4ff1-8e8a-b42406f37755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:01.446946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1baa42-8b43-4ff1-8e8a-b42406f37755 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:01.447233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1baa42-8b43-4ff1-8e8a-b42406f37755 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:01.447511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1baa42-8b43-4ff1-8e8a-b42406f37755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:01.456649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1baa42-8b43-4ff1-8e8a-b42406f37755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:01.458130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1baa42-8b43-4ff1-8e8a-b42406f37755 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:01.467387 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4087/8261] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:20:01.495634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2671fbffe48546f29ae69a2205e650fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:01.508825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2671fbffe48546f29ae69a2205e650fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:01.517001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2671fbffe48546f29ae69a2205e650fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:01.517807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2671fbffe48546f29ae69a2205e650fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:01.518622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2671fbffe48546f29ae69a2205e650fe'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:01.533101 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4175/8262] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:20:01 2022] DELETE /identity/v3/users/2671fbffe48546f29ae69a2205e650fe => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:20:01.537977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1a08-0560-4234-8bcc-d66d2d5e067b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:01.538436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1a08-0560-4234-8bcc-d66d2d5e067b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:01.538866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1a08-0560-4234-8bcc-d66d2d5e067b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:01.568829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d14a1a08-0560-4234-8bcc-d66d2d5e067b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:01.570393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d14a1a08-0560-4234-8bcc-d66d2d5e067b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:01.635444 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4088/8263] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:01.653863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:01.655648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:01.706000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:01.722969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fe16a-a12a-435e-bb6f-f59b1f8f311b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:01.727371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd37af5-5a2b-4889-8413-83f2edf61413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:01.727992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd37af5-5a2b-4889-8413-83f2edf61413 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:01.728278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd37af5-5a2b-4889-8413-83f2edf61413 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:01.728557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd37af5-5a2b-4889-8413-83f2edf61413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:01.741864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd37af5-5a2b-4889-8413-83f2edf61413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:01.744616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd37af5-5a2b-4889-8413-83f2edf61413 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:01.756124 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4176/8264] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:20:02.153196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1a08-0560-4234-8bcc-d66d2d5e067b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:02.154683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d14a1a08-0560-4234-8bcc-d66d2d5e067b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:02.171309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1a08-0560-4234-8bcc-d66d2d5e067b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:02.177732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd37af5-5a2b-4889-8413-83f2edf61413 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:02.178629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:02.179291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:02.179582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:02.179868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] PATH_INFO: `/v3/projects/266eb337a0f241e2a3a6bb0f3b874cfa` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:02.181081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=266eb337a0f241e2a3a6bb0f3b874cfa)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:02.183163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:02.186983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:02.193861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd37af5-5a2b-4889-8413-83f2edf61413 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:02.204502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fec81-955b-46c9-8e0d-d4270a93e0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:02.205338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fec81-955b-46c9-8e0d-d4270a93e0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:02.205699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fec81-955b-46c9-8e0d-d4270a93e0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:02.206299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fec81-955b-46c9-8e0d-d4270a93e0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:02.208981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ae7fec81-955b-46c9-8e0d-d4270a93e0c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:02.222798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fec81-955b-46c9-8e0d-d4270a93e0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:02.225283 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fec81-955b-46c9-8e0d-d4270a93e0c8 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:02.239349 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4177/8265] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:20:02.247338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '266eb337a0f241e2a3a6bb0f3b874cfa', 'request_id': 'req-a8a9c052-6a13-4ff5-8102-5659501d1e27'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:02.255682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '266eb337a0f241e2a3a6bb0f3b874cfa', 'request_id': 'req-a8a9c052-6a13-4ff5-8102-5659501d1e27'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:02.262641 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '266eb337a0f241e2a3a6bb0f3b874cfa', 'request_id': 'req-a8a9c052-6a13-4ff5-8102-5659501d1e27'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:02.273790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] The token cache is being invalidate because project 266eb337a0f241e2a3a6bb0f3b874cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:20:02.274085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:02.274756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:02.277247 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4089/8266] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:20:02 2022] DELETE /identity/v3/projects/266eb337a0f241e2a3a6bb0f3b874cfa => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:02.281508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fec81-955b-46c9-8e0d-d4270a93e0c8 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:02.294111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fec81-955b-46c9-8e0d-d4270a93e0c8 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:02.366185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e18f71-aa4b-466d-8ebd-ad26ea42b819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:02.366829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e18f71-aa4b-466d-8ebd-ad26ea42b819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:02.367118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e18f71-aa4b-466d-8ebd-ad26ea42b819 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:02.367393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e18f71-aa4b-466d-8ebd-ad26ea42b819 admin admin] PATH_INFO: `/v3/users/b202e158639649f79fcbf72184a04490` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:02.368612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e18f71-aa4b-466d-8ebd-ad26ea42b819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b202e158639649f79fcbf72184a04490)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:02.370625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e18f71-aa4b-466d-8ebd-ad26ea42b819 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:02.419873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-69e18f71-aa4b-466d-8ebd-ad26ea42b819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b202e158639649f79fcbf72184a04490'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:02.435590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-69e18f71-aa4b-466d-8ebd-ad26ea42b819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b202e158639649f79fcbf72184a04490'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:02.443960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-69e18f71-aa4b-466d-8ebd-ad26ea42b819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b202e158639649f79fcbf72184a04490'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:02.444606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-69e18f71-aa4b-466d-8ebd-ad26ea42b819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b202e158639649f79fcbf72184a04490'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:02.445377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-69e18f71-aa4b-466d-8ebd-ad26ea42b819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b202e158639649f79fcbf72184a04490'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:02.458599 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4178/8267] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:20:02 2022] DELETE /identity/v3/users/b202e158639649f79fcbf72184a04490 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:02.859593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:02.862665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:02.883364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9c052-6a13-4ff5-8102-5659501d1e27 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:02.969493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:02.970581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:02.971125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:02.971517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c admin admin] PATH_INFO: `/v3/projects/2f0826098cee408ead3ec19041cfdc6a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:02.973486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f0826098cee408ead3ec19041cfdc6a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:02.976868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:03.064194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f0826098cee408ead3ec19041cfdc6a', 'request_id': 'req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:03.086969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f0826098cee408ead3ec19041cfdc6a', 'request_id': 'req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:03.094996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f0826098cee408ead3ec19041cfdc6a', 'request_id': 'req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:03.110840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c admin admin] The token cache is being invalidate because project 2f0826098cee408ead3ec19041cfdc6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:20:03.111287 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:03.112202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a9253afe-4227-47a3-8f3d-b5ae02dedd1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:03.115046 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4090/8268] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:20:02 2022] DELETE /identity/v3/projects/2f0826098cee408ead3ec19041cfdc6a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:03.132877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc74a2f-f001-410a-a717-7016f8c6e32a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:03.133391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc74a2f-f001-410a-a717-7016f8c6e32a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:03.133814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc74a2f-f001-410a-a717-7016f8c6e32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:03.166669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9cc74a2f-f001-410a-a717-7016f8c6e32a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:03.167931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9cc74a2f-f001-410a-a717-7016f8c6e32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:03.250035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:03.268998 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4179/8269] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:03.277378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fc711-02a6-4ee3-b1e7-954783e1af9e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:03.277677 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fc711-02a6-4ee3-b1e7-954783e1af9e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:03.277963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fc711-02a6-4ee3-b1e7-954783e1af9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:03.300106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-976fc711-02a6-4ee3-b1e7-954783e1af9e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:03.301541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-976fc711-02a6-4ee3-b1e7-954783e1af9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:03.316150 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4091/8270] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:03.322230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc74a2f-f001-410a-a717-7016f8c6e32a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:03.342990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc74a2f-f001-410a-a717-7016f8c6e32a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:03.351158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68323317-7440-4a7a-8691-1511e661a4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:03.351797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68323317-7440-4a7a-8691-1511e661a4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:03.352290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68323317-7440-4a7a-8691-1511e661a4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:03.352548 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68323317-7440-4a7a-8691-1511e661a4a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:03.353208 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68323317-7440-4a7a-8691-1511e661a4a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:20:03.353654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68323317-7440-4a7a-8691-1511e661a4a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:03.355345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68323317-7440-4a7a-8691-1511e661a4a0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:03.372765 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4180/8271] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:20:03.377292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fc711-02a6-4ee3-b1e7-954783e1af9e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:03.392522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fc711-02a6-4ee3-b1e7-954783e1af9e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:03.398704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc816a0-4828-43a1-bab5-603e1981d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:03.399355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc816a0-4828-43a1-bab5-603e1981d931 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:03.399642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc816a0-4828-43a1-bab5-603e1981d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:03.399921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc816a0-4828-43a1-bab5-603e1981d931 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:03.400705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc816a0-4828-43a1-bab5-603e1981d931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:03.404780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc816a0-4828-43a1-bab5-603e1981d931 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:03.422714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:03.441529 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4092/8272] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:20:03 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:03.445601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68323317-7440-4a7a-8691-1511e661a4a0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:03.461461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68323317-7440-4a7a-8691-1511e661a4a0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:03.529424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa7e6b-89d7-49f9-be03-97c44b0f0b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:03.529424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa7e6b-89d7-49f9-be03-97c44b0f0b29 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:03.529742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa7e6b-89d7-49f9-be03-97c44b0f0b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:03.530006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa7e6b-89d7-49f9-be03-97c44b0f0b29 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:03.530908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa7e6b-89d7-49f9-be03-97c44b0f0b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:03.532578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa7e6b-89d7-49f9-be03-97c44b0f0b29 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:03.559134 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4181/8273] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:20:03 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:20:03.560403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc816a0-4828-43a1-bab5-603e1981d931 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:03.572417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc816a0-4828-43a1-bab5-603e1981d931 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:03.577382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90ec70-2d62-4189-a672-c8e096cbd553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:03.578006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90ec70-2d62-4189-a672-c8e096cbd553 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:03.578357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90ec70-2d62-4189-a672-c8e096cbd553 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:03.578645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90ec70-2d62-4189-a672-c8e096cbd553 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:03.579290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90ec70-2d62-4189-a672-c8e096cbd553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:03.580913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90ec70-2d62-4189-a672-c8e096cbd553 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:03.597282 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4093/8274] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:03.601634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa7e6b-89d7-49f9-be03-97c44b0f0b29 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:03.613738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa7e6b-89d7-49f9-be03-97c44b0f0b29 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:03.619638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953d7ec-f826-4a0a-84ad-a442fb8ec36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:03.619921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953d7ec-f826-4a0a-84ad-a442fb8ec36e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:03.620213 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953d7ec-f826-4a0a-84ad-a442fb8ec36e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:03.620491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953d7ec-f826-4a0a-84ad-a442fb8ec36e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:03.621445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953d7ec-f826-4a0a-84ad-a442fb8ec36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:03.623266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953d7ec-f826-4a0a-84ad-a442fb8ec36e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:03.635824 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4182/8275] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:03.640030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90ec70-2d62-4189-a672-c8e096cbd553 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:03.651589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90ec70-2d62-4189-a672-c8e096cbd553 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:03.669759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe75514-a36b-404c-ab82-080a7852441d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:03.673175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe75514-a36b-404c-ab82-080a7852441d admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:03.673742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe75514-a36b-404c-ab82-080a7852441d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:03.674249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe75514-a36b-404c-ab82-080a7852441d admin admin] PATH_INFO: `/v3/projects/b561be6100d045319b06162ced6d6485/users/052e6eedf19647ff9cf5530691f1ec40/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:03.703630 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe75514-a36b-404c-ab82-080a7852441d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b561be6100d045319b06162ced6d6485, user_id=052e6eedf19647ff9cf5530691f1ec40, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:03.708403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe75514-a36b-404c-ab82-080a7852441d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:03.726183 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4094/8276] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:03 2022] PUT /identity/v3/projects/b561be6100d045319b06162ced6d6485/users/052e6eedf19647ff9cf5530691f1ec40/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:03.738644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16111c3e-0384-4ec7-b9a8-5df08b488599 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:03.739028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16111c3e-0384-4ec7-b9a8-5df08b488599 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:03.740117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16111c3e-0384-4ec7-b9a8-5df08b488599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:03.781296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-16111c3e-0384-4ec7-b9a8-5df08b488599 None None] MFA Rules not processed for user `052e6eedf19647ff9cf5530691f1ec40`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:03.783464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-16111c3e-0384-4ec7-b9a8-5df08b488599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:03.885560 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4183/8277] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:20:03.890897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe75514-a36b-404c-ab82-080a7852441d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:03.996506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe75514-a36b-404c-ab82-080a7852441d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:04.004389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d426efb-74eb-4b26-a071-273edb0a79ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:04.005953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d426efb-74eb-4b26-a071-273edb0a79ce admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:04.006778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d426efb-74eb-4b26-a071-273edb0a79ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:04.007125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d426efb-74eb-4b26-a071-273edb0a79ce admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:04.007889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d426efb-74eb-4b26-a071-273edb0a79ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:04.010969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d426efb-74eb-4b26-a071-273edb0a79ce admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:04.045026 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4095/8278] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:20:03 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:04.048782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16111c3e-0384-4ec7-b9a8-5df08b488599 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:04.060584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16111c3e-0384-4ec7-b9a8-5df08b488599 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:04.128066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:04.154683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309c15d-879e-42e4-b398-d4097f73ff9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:04.155356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309c15d-879e-42e4-b398-d4097f73ff9c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:04.155647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309c15d-879e-42e4-b398-d4097f73ff9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:04.155930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309c15d-879e-42e4-b398-d4097f73ff9c admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:04.156597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309c15d-879e-42e4-b398-d4097f73ff9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:04.159340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309c15d-879e-42e4-b398-d4097f73ff9c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:04.190091 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4184/8279] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:20:04 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:04.194430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d426efb-74eb-4b26-a071-273edb0a79ce admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:04.222776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d426efb-74eb-4b26-a071-273edb0a79ce admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:04.229113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a342617-d7c5-4abf-ac24-2bae5245cb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:04.229562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a342617-d7c5-4abf-ac24-2bae5245cb1e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:04.229858 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a342617-d7c5-4abf-ac24-2bae5245cb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:04.230145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a342617-d7c5-4abf-ac24-2bae5245cb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:04.230838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a342617-d7c5-4abf-ac24-2bae5245cb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:04.232690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a342617-d7c5-4abf-ac24-2bae5245cb1e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:04.245932 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4096/8280] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:04.250179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309c15d-879e-42e4-b398-d4097f73ff9c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:04.263071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309c15d-879e-42e4-b398-d4097f73ff9c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:04.268239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6ba6b-9565-431b-81fe-8e2ba06b5083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:04.268989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6ba6b-9565-431b-81fe-8e2ba06b5083 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:04.269308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6ba6b-9565-431b-81fe-8e2ba06b5083 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:04.269592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6ba6b-9565-431b-81fe-8e2ba06b5083 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:04.270238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6ba6b-9565-431b-81fe-8e2ba06b5083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:04.272380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6ba6b-9565-431b-81fe-8e2ba06b5083 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:04.284371 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4185/8281] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:20:04.289341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a342617-d7c5-4abf-ac24-2bae5245cb1e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:04.301502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a342617-d7c5-4abf-ac24-2bae5245cb1e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:04.308012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ded4d7-49ae-44ee-8a64-0f774c1b4c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:04.308978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ded4d7-49ae-44ee-8a64-0f774c1b4c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:04.309276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ded4d7-49ae-44ee-8a64-0f774c1b4c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:04.310586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ded4d7-49ae-44ee-8a64-0f774c1b4c10 admin admin] PATH_INFO: `/v3/projects/fd48cd8cbaa74276abed3d03ece0fa64/users/a9b702718cf94dada7a6094bee12e489/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:04.331679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ded4d7-49ae-44ee-8a64-0f774c1b4c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd48cd8cbaa74276abed3d03ece0fa64, user_id=a9b702718cf94dada7a6094bee12e489, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:04.334908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ded4d7-49ae-44ee-8a64-0f774c1b4c10 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:04.376619 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4097/8282] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:04 2022] PUT /identity/v3/projects/fd48cd8cbaa74276abed3d03ece0fa64/users/a9b702718cf94dada7a6094bee12e489/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:04.378804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29181f4-f155-48ee-a8d7-f81ea95ab1d2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:04.379256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29181f4-f155-48ee-a8d7-f81ea95ab1d2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:04.379419 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29181f4-f155-48ee-a8d7-f81ea95ab1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:04.445347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c29181f4-f155-48ee-a8d7-f81ea95ab1d2 None None] MFA Rules not processed for user `a9b702718cf94dada7a6094bee12e489`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:04.446679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c29181f4-f155-48ee-a8d7-f81ea95ab1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:04.537165 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4186/8283] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:04.538440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ded4d7-49ae-44ee-8a64-0f774c1b4c10 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:04.618842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ded4d7-49ae-44ee-8a64-0f774c1b4c10 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:04.623782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92480b46-2656-4ba2-8ac9-65b4d650a31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:04.624479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92480b46-2656-4ba2-8ac9-65b4d650a31e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:04.624843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92480b46-2656-4ba2-8ac9-65b4d650a31e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:04.625241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92480b46-2656-4ba2-8ac9-65b4d650a31e admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:04.625963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92480b46-2656-4ba2-8ac9-65b4d650a31e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:04.628067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92480b46-2656-4ba2-8ac9-65b4d650a31e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:04.644399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:04.660079 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4098/8284] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:20:04 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:04.664549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29181f4-f155-48ee-a8d7-f81ea95ab1d2 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:04.676667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29181f4-f155-48ee-a8d7-f81ea95ab1d2 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:04.756916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487e2e2-b77d-4e4b-b922-9f92c653454a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:04.757605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487e2e2-b77d-4e4b-b922-9f92c653454a admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:04.757900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487e2e2-b77d-4e4b-b922-9f92c653454a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:04.758182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487e2e2-b77d-4e4b-b922-9f92c653454a admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:04.759019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487e2e2-b77d-4e4b-b922-9f92c653454a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:04.761296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487e2e2-b77d-4e4b-b922-9f92c653454a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:04.789619 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4187/8285] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:20:04 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:04.791106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92480b46-2656-4ba2-8ac9-65b4d650a31e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:04.802441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92480b46-2656-4ba2-8ac9-65b4d650a31e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:04.808509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee36f20-ec48-48bd-9ee3-e1bf6490dc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:04.809276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee36f20-ec48-48bd-9ee3-e1bf6490dc37 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:04.809572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee36f20-ec48-48bd-9ee3-e1bf6490dc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:04.809952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee36f20-ec48-48bd-9ee3-e1bf6490dc37 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:04.810715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee36f20-ec48-48bd-9ee3-e1bf6490dc37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:04.812890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee36f20-ec48-48bd-9ee3-e1bf6490dc37 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:04.831880 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4099/8286] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:04.833687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487e2e2-b77d-4e4b-b922-9f92c653454a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:04.845700 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487e2e2-b77d-4e4b-b922-9f92c653454a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:04.851017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992152c1-41c0-43e8-8c32-353e07d7f1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:04.851630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992152c1-41c0-43e8-8c32-353e07d7f1db admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:04.852126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992152c1-41c0-43e8-8c32-353e07d7f1db admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:04.852369 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992152c1-41c0-43e8-8c32-353e07d7f1db admin admin] PATH_INFO: `/v3/projects/041cac07f86a44218583643382fbb5d5/users/080bdd5ec8de4a00b9e26498eb99fe4a/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:04.871440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992152c1-41c0-43e8-8c32-353e07d7f1db admin admin] RBAC: Authorizing `identity:create_grant(project_id=041cac07f86a44218583643382fbb5d5, user_id=080bdd5ec8de4a00b9e26498eb99fe4a, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:04.874241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992152c1-41c0-43e8-8c32-353e07d7f1db admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:04.899440 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4188/8287] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:04 2022] PUT /identity/v3/projects/041cac07f86a44218583643382fbb5d5/users/080bdd5ec8de4a00b9e26498eb99fe4a/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:20:04.903222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee36f20-ec48-48bd-9ee3-e1bf6490dc37 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:04.922167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee36f20-ec48-48bd-9ee3-e1bf6490dc37 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:04.998843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec408e4f-b3dd-43d7-98e2-622b3f85570e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:04.999515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec408e4f-b3dd-43d7-98e2-622b3f85570e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:04.999782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec408e4f-b3dd-43d7-98e2-622b3f85570e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:05.000063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec408e4f-b3dd-43d7-98e2-622b3f85570e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:05.000715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec408e4f-b3dd-43d7-98e2-622b3f85570e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:05.004477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec408e4f-b3dd-43d7-98e2-622b3f85570e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:05.022623 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4100/8288] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:05.025093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992152c1-41c0-43e8-8c32-353e07d7f1db admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:05.042741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992152c1-41c0-43e8-8c32-353e07d7f1db admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:05.054950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0ddc2-7e3d-4312-8935-e035e4e20039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:05.056438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0ddc2-7e3d-4312-8935-e035e4e20039 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:05.057027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0ddc2-7e3d-4312-8935-e035e4e20039 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:05.057386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0ddc2-7e3d-4312-8935-e035e4e20039 admin admin] PATH_INFO: `/v3/projects/041cac07f86a44218583643382fbb5d5/users/080bdd5ec8de4a00b9e26498eb99fe4a/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:05.059657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0ddc2-7e3d-4312-8935-e035e4e20039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=041cac07f86a44218583643382fbb5d5, user_id=080bdd5ec8de4a00b9e26498eb99fe4a, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:05.075033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0ddc2-7e3d-4312-8935-e035e4e20039 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:05.094798 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4189/8289] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:05 2022] PUT /identity/v3/projects/041cac07f86a44218583643382fbb5d5/users/080bdd5ec8de4a00b9e26498eb99fe4a/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:05.105586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991df72-e257-405c-9fb1-8494a8a0bea1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:05.107004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991df72-e257-405c-9fb1-8494a8a0bea1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:05.107410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991df72-e257-405c-9fb1-8494a8a0bea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:05.162129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e991df72-e257-405c-9fb1-8494a8a0bea1 None None] MFA Rules not processed for user `080bdd5ec8de4a00b9e26498eb99fe4a`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:05.163542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e991df72-e257-405c-9fb1-8494a8a0bea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:05.262821 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4101/8290] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:05.274517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0ddc2-7e3d-4312-8935-e035e4e20039 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:05.276663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b5a0ddc2-7e3d-4312-8935-e035e4e20039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:05.289570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:05.384676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0ddc2-7e3d-4312-8935-e035e4e20039 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:05.390278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25037a-7df6-43bc-9b01-b2be4a7a41ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:05.391212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25037a-7df6-43bc-9b01-b2be4a7a41ae service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:05.391581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25037a-7df6-43bc-9b01-b2be4a7a41ae service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:05.391955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25037a-7df6-43bc-9b01-b2be4a7a41ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:05.483543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25037a-7df6-43bc-9b01-b2be4a7a41ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:05.485478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25037a-7df6-43bc-9b01-b2be4a7a41ae service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:05.500079 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4190/8291] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:20:08.697982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991df72-e257-405c-9fb1-8494a8a0bea1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:08.767626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991df72-e257-405c-9fb1-8494a8a0bea1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:08.772710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:08.773393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:08.773651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:08.773932 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] PATH_INFO: `/v3/users/052e6eedf19647ff9cf5530691f1ec40` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:08.775067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=052e6eedf19647ff9cf5530691f1ec40)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:08.777024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:08.782108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:08.826076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '052e6eedf19647ff9cf5530691f1ec40'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:08.837968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '052e6eedf19647ff9cf5530691f1ec40'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:08.838708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '052e6eedf19647ff9cf5530691f1ec40'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:08.845520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '052e6eedf19647ff9cf5530691f1ec40'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:08.854025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '052e6eedf19647ff9cf5530691f1ec40'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:08.860314 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4102/8292] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:20:08 2022] DELETE /identity/v3/users/052e6eedf19647ff9cf5530691f1ec40 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:08.865410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489053b0-ba8f-44f3-803c-e4f2ba102c03 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:08.865950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489053b0-ba8f-44f3-803c-e4f2ba102c03 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:08.866076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489053b0-ba8f-44f3-803c-e4f2ba102c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:08.886246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-489053b0-ba8f-44f3-803c-e4f2ba102c03 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:08.887511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-489053b0-ba8f-44f3-803c-e4f2ba102c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:08.948312 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4191/8293] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:20:08.958169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:08.959777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:09.030803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e7b37-c73a-4f9e-b772-8268c9623e81 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:09.035460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcde39-f850-44c8-b874-54384f26dd81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:09.036381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcde39-f850-44c8-b874-54384f26dd81 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:09.036768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcde39-f850-44c8-b874-54384f26dd81 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:09.037352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcde39-f850-44c8-b874-54384f26dd81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:09.048675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcde39-f850-44c8-b874-54384f26dd81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:09.049881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcde39-f850-44c8-b874-54384f26dd81 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:09.061382 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4103/8294] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:20:09.519346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489053b0-ba8f-44f3-803c-e4f2ba102c03 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:09.521530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-489053b0-ba8f-44f3-803c-e4f2ba102c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:09.534923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489053b0-ba8f-44f3-803c-e4f2ba102c03 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:09.541127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:09.541778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:09.542097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:09.542380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] PATH_INFO: `/v3/projects/b561be6100d045319b06162ced6d6485` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:09.543685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b561be6100d045319b06162ced6d6485)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:09.545671 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:09.598452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b561be6100d045319b06162ced6d6485', 'request_id': 'req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:09.607252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b561be6100d045319b06162ced6d6485', 'request_id': 'req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:09.609353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:09.615152 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b561be6100d045319b06162ced6d6485', 'request_id': 'req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:09.632606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] The token cache is being invalidate because project b561be6100d045319b06162ced6d6485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:20:09.632953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:09.633625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:09.635161 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4192/8295] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:20:09 2022] DELETE /identity/v3/projects/b561be6100d045319b06162ced6d6485 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:20:09.639162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcde39-f850-44c8-b874-54384f26dd81 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:09.649499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcde39-f850-44c8-b874-54384f26dd81 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:09.707388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:09.708573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:09.708811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:09.709122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] PATH_INFO: `/v3/users/a9b702718cf94dada7a6094bee12e489` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:09.710244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9b702718cf94dada7a6094bee12e489)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:09.712793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:09.743489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:09.761823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9b702718cf94dada7a6094bee12e489'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:09.774731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9b702718cf94dada7a6094bee12e489'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:09.775635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9b702718cf94dada7a6094bee12e489'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:09.783169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9b702718cf94dada7a6094bee12e489'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:09.791602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9b702718cf94dada7a6094bee12e489'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:09.797802 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4104/8296] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:20:09 2022] DELETE /identity/v3/users/a9b702718cf94dada7a6094bee12e489 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:10.126731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:10.128813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:10.150417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e46200-6ff3-4200-8ed7-541f8ee8bf6b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:10.213527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:10.214424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:10.215041 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:10.215554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] PATH_INFO: `/v3/projects/fd48cd8cbaa74276abed3d03ece0fa64` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:10.216872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd48cd8cbaa74276abed3d03ece0fa64)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:10.218871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:10.272674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd48cd8cbaa74276abed3d03ece0fa64', 'request_id': 'req-f7e212fb-ad6c-476d-94b8-d5423808e44d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.281401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd48cd8cbaa74276abed3d03ece0fa64', 'request_id': 'req-f7e212fb-ad6c-476d-94b8-d5423808e44d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.288578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd48cd8cbaa74276abed3d03ece0fa64', 'request_id': 'req-f7e212fb-ad6c-476d-94b8-d5423808e44d'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.299840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] The token cache is being invalidate because project fd48cd8cbaa74276abed3d03ece0fa64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:20:10.300130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.300803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.302944 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4193/8297] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:20:10 2022] DELETE /identity/v3/projects/fd48cd8cbaa74276abed3d03ece0fa64 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:10.307160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:10.308816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:10.324628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560187c8-1bef-45fa-a379-f356d3caa23b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:10.394156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51dc42-7c21-4a13-a041-be8a6b9c4af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:10.395113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51dc42-7c21-4a13-a041-be8a6b9c4af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:10.395478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51dc42-7c21-4a13-a041-be8a6b9c4af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:10.395999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51dc42-7c21-4a13-a041-be8a6b9c4af7 admin admin] PATH_INFO: `/v3/users/080bdd5ec8de4a00b9e26498eb99fe4a` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:10.397406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51dc42-7c21-4a13-a041-be8a6b9c4af7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=080bdd5ec8de4a00b9e26498eb99fe4a)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:10.399441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51dc42-7c21-4a13-a041-be8a6b9c4af7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:10.453331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8e51dc42-7c21-4a13-a041-be8a6b9c4af7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '080bdd5ec8de4a00b9e26498eb99fe4a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.466504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8e51dc42-7c21-4a13-a041-be8a6b9c4af7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '080bdd5ec8de4a00b9e26498eb99fe4a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.467344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8e51dc42-7c21-4a13-a041-be8a6b9c4af7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '080bdd5ec8de4a00b9e26498eb99fe4a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.474757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8e51dc42-7c21-4a13-a041-be8a6b9c4af7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '080bdd5ec8de4a00b9e26498eb99fe4a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.483578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8e51dc42-7c21-4a13-a041-be8a6b9c4af7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '080bdd5ec8de4a00b9e26498eb99fe4a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.490939 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4105/8298] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:20:10 2022] DELETE /identity/v3/users/080bdd5ec8de4a00b9e26498eb99fe4a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:10.789487 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:10.790984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:10.814477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e212fb-ad6c-476d-94b8-d5423808e44d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:10.887519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:10.888350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:10.888673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:10.888987 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1 admin admin] PATH_INFO: `/v3/projects/041cac07f86a44218583643382fbb5d5` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:10.890489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=041cac07f86a44218583643382fbb5d5)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:10.892554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:10.914599 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:10.942679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '041cac07f86a44218583643382fbb5d5', 'request_id': 'req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.950699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '041cac07f86a44218583643382fbb5d5', 'request_id': 'req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.958180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '041cac07f86a44218583643382fbb5d5', 'request_id': 'req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.969188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1 admin admin] The token cache is being invalidate because project 041cac07f86a44218583643382fbb5d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:20:10.969737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.970171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c4efb9a5-c0c5-43d9-9978-7eabb7a8afd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:10.971799 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4194/8299] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:20:10 2022] DELETE /identity/v3/projects/041cac07f86a44218583643382fbb5d5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:10.979910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec58c8-ae2e-4ef5-8efe-8818845446f0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:10.980374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec58c8-ae2e-4ef5-8efe-8818845446f0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:10.980476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec58c8-ae2e-4ef5-8efe-8818845446f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.001363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-07ec58c8-ae2e-4ef5-8efe-8818845446f0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:11.002612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-07ec58c8-ae2e-4ef5-8efe-8818845446f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:11.022537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:11.065942 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4106/8300] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:11.073949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad763ecb-27cf-4a7d-92b5-707670442cef None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.074274 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad763ecb-27cf-4a7d-92b5-707670442cef None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.074582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad763ecb-27cf-4a7d-92b5-707670442cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.094970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ad763ecb-27cf-4a7d-92b5-707670442cef None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:11.096205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ad763ecb-27cf-4a7d-92b5-707670442cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:11.105202 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4195/8301] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:11.109341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec58c8-ae2e-4ef5-8efe-8818845446f0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:11.120683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec58c8-ae2e-4ef5-8efe-8818845446f0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:11.128249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57851ef7-7cc7-43b4-8cd8-806fce3bccbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:11.128914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57851ef7-7cc7-43b4-8cd8-806fce3bccbb admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.133317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57851ef7-7cc7-43b4-8cd8-806fce3bccbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.134159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57851ef7-7cc7-43b4-8cd8-806fce3bccbb admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.135329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57851ef7-7cc7-43b4-8cd8-806fce3bccbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:20:11.135974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57851ef7-7cc7-43b4-8cd8-806fce3bccbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:11.138538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57851ef7-7cc7-43b4-8cd8-806fce3bccbb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:11.156929 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4107/8302] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:20:11.158311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad763ecb-27cf-4a7d-92b5-707670442cef None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:11.169764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad763ecb-27cf-4a7d-92b5-707670442cef None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:11.174553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7d20b-02de-4bda-bcb3-02a40df8d1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:11.175331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7d20b-02de-4bda-bcb3-02a40df8d1ad admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.175665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7d20b-02de-4bda-bcb3-02a40df8d1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.176085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7d20b-02de-4bda-bcb3-02a40df8d1ad admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.177527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7d20b-02de-4bda-bcb3-02a40df8d1ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:11.180278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7d20b-02de-4bda-bcb3-02a40df8d1ad admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:11.214513 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4196/8303] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:20:11 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:11.218773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57851ef7-7cc7-43b4-8cd8-806fce3bccbb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:11.230241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57851ef7-7cc7-43b4-8cd8-806fce3bccbb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:11.299419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c248b-08ae-4b79-b9c2-b276940103ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:11.300584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c248b-08ae-4b79-b9c2-b276940103ef admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.300881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c248b-08ae-4b79-b9c2-b276940103ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.301192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c248b-08ae-4b79-b9c2-b276940103ef admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.301867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c248b-08ae-4b79-b9c2-b276940103ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:11.304565 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c248b-08ae-4b79-b9c2-b276940103ef admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:11.335110 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4108/8304] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:20:11 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:11.337344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7d20b-02de-4bda-bcb3-02a40df8d1ad admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:11.349793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7d20b-02de-4bda-bcb3-02a40df8d1ad admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:11.355239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa01d46-2812-4671-99a8-c32259ae762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:11.355856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa01d46-2812-4671-99a8-c32259ae762c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.356142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa01d46-2812-4671-99a8-c32259ae762c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.356420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa01d46-2812-4671-99a8-c32259ae762c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.357373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa01d46-2812-4671-99a8-c32259ae762c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:11.359119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa01d46-2812-4671-99a8-c32259ae762c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:11.372940 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4197/8305] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:11.375851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c248b-08ae-4b79-b9c2-b276940103ef admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:11.388189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c248b-08ae-4b79-b9c2-b276940103ef admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:11.393351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46266a-ec07-4bf8-8f7f-6c2d913e163e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:11.394070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46266a-ec07-4bf8-8f7f-6c2d913e163e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.394441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46266a-ec07-4bf8-8f7f-6c2d913e163e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.394782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46266a-ec07-4bf8-8f7f-6c2d913e163e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.396688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46266a-ec07-4bf8-8f7f-6c2d913e163e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:11.398429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46266a-ec07-4bf8-8f7f-6c2d913e163e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:11.414417 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4109/8306] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:11.415485 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa01d46-2812-4671-99a8-c32259ae762c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:11.426365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa01d46-2812-4671-99a8-c32259ae762c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:11.431523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33aeca-e0ee-4e23-ba0e-4fe8d55ebb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:11.432198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33aeca-e0ee-4e23-ba0e-4fe8d55ebb37 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.432486 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33aeca-e0ee-4e23-ba0e-4fe8d55ebb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.432769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33aeca-e0ee-4e23-ba0e-4fe8d55ebb37 admin admin] PATH_INFO: `/v3/projects/ac65433f9bf84f9c8b4771d54c3c16b4/users/e31059313fd543bebad86a698ed08587/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.455962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33aeca-e0ee-4e23-ba0e-4fe8d55ebb37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac65433f9bf84f9c8b4771d54c3c16b4, user_id=e31059313fd543bebad86a698ed08587, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:11.460558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33aeca-e0ee-4e23-ba0e-4fe8d55ebb37 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:11.475357 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4198/8307] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:11 2022] PUT /identity/v3/projects/ac65433f9bf84f9c8b4771d54c3c16b4/users/e31059313fd543bebad86a698ed08587/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:20:11.485740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe2984-7cac-4ac3-93bf-fc6747a1c469 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.486064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe2984-7cac-4ac3-93bf-fc6747a1c469 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.486369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe2984-7cac-4ac3-93bf-fc6747a1c469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.519724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cdbe2984-7cac-4ac3-93bf-fc6747a1c469 None None] MFA Rules not processed for user `e31059313fd543bebad86a698ed08587`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:11.521986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cdbe2984-7cac-4ac3-93bf-fc6747a1c469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:11.604775 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4110/8308] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:20:11.605735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33aeca-e0ee-4e23-ba0e-4fe8d55ebb37 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:11.686077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33aeca-e0ee-4e23-ba0e-4fe8d55ebb37 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:11.691794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bdef2-d418-4f94-b985-a9984005f3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:11.692529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bdef2-d418-4f94-b985-a9984005f3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.692924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bdef2-d418-4f94-b985-a9984005f3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.693280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bdef2-d418-4f94-b985-a9984005f3f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.693972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bdef2-d418-4f94-b985-a9984005f3f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:11.695968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bdef2-d418-4f94-b985-a9984005f3f5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:11.729375 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4199/8309] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:20:11 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:11.733316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe2984-7cac-4ac3-93bf-fc6747a1c469 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:11.750217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe2984-7cac-4ac3-93bf-fc6747a1c469 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:11.775596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:11.832495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85fac5-7f8a-4146-b722-cf18a16b312f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:11.833893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85fac5-7f8a-4146-b722-cf18a16b312f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.834188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85fac5-7f8a-4146-b722-cf18a16b312f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.834470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85fac5-7f8a-4146-b722-cf18a16b312f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.835183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85fac5-7f8a-4146-b722-cf18a16b312f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:11.837210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85fac5-7f8a-4146-b722-cf18a16b312f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:11.861833 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4111/8310] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:20:11 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:11.865815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bdef2-d418-4f94-b985-a9984005f3f5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:11.876975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bdef2-d418-4f94-b985-a9984005f3f5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:11.882781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13ae00-9961-42b0-a126-18c8a44b6059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:11.883569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13ae00-9961-42b0-a126-18c8a44b6059 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.883951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13ae00-9961-42b0-a126-18c8a44b6059 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.884235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13ae00-9961-42b0-a126-18c8a44b6059 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.884990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13ae00-9961-42b0-a126-18c8a44b6059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:11.886629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13ae00-9961-42b0-a126-18c8a44b6059 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:11.900940 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4200/8311] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:11.902746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85fac5-7f8a-4146-b722-cf18a16b312f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:11.914150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85fac5-7f8a-4146-b722-cf18a16b312f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:11.919494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdf4da-b2e8-4cd6-9394-7eebe10165bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:11.920191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdf4da-b2e8-4cd6-9394-7eebe10165bd admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.920538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdf4da-b2e8-4cd6-9394-7eebe10165bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.920886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdf4da-b2e8-4cd6-9394-7eebe10165bd admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.921668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafdf4da-b2e8-4cd6-9394-7eebe10165bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:11.923896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafdf4da-b2e8-4cd6-9394-7eebe10165bd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:11.937579 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4112/8312] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:11.941689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13ae00-9961-42b0-a126-18c8a44b6059 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:11.964290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13ae00-9961-42b0-a126-18c8a44b6059 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:11.969779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2031d-f201-4fdb-b22d-3d6afe3f4159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:11.970211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2031d-f201-4fdb-b22d-3d6afe3f4159 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:11.970439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2031d-f201-4fdb-b22d-3d6afe3f4159 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:11.970842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2031d-f201-4fdb-b22d-3d6afe3f4159 admin admin] PATH_INFO: `/v3/projects/fc41fdb3eb9b4895abf88f180001c1ef/users/c4071140f98e410586576763dd6c9191/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:11.990579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2031d-f201-4fdb-b22d-3d6afe3f4159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc41fdb3eb9b4895abf88f180001c1ef, user_id=c4071140f98e410586576763dd6c9191, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:11.994173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2031d-f201-4fdb-b22d-3d6afe3f4159 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:11.997970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:12.029289 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4201/8313] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:11 2022] PUT /identity/v3/projects/fc41fdb3eb9b4895abf88f180001c1ef/users/c4071140f98e410586576763dd6c9191/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:20:12.120772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d0c82-fe4c-4f1d-ae99-8e24b08d5b55 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:12.121760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d0c82-fe4c-4f1d-ae99-8e24b08d5b55 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:12.122304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d0c82-fe4c-4f1d-ae99-8e24b08d5b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:12.156747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fc7d0c82-fe4c-4f1d-ae99-8e24b08d5b55 None None] MFA Rules not processed for user `c4071140f98e410586576763dd6c9191`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:12.158305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fc7d0c82-fe4c-4f1d-ae99-8e24b08d5b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:12.230369 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4113/8314] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:12.235129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2031d-f201-4fdb-b22d-3d6afe3f4159 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:12.303480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2031d-f201-4fdb-b22d-3d6afe3f4159 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:12.308135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38f4a7-0a53-4db9-9225-1af7c0c2f7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:12.308755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38f4a7-0a53-4db9-9225-1af7c0c2f7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:12.309077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38f4a7-0a53-4db9-9225-1af7c0c2f7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:12.309358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38f4a7-0a53-4db9-9225-1af7c0c2f7d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:12.309989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38f4a7-0a53-4db9-9225-1af7c0c2f7d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:12.311909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38f4a7-0a53-4db9-9225-1af7c0c2f7d9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:12.338839 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4202/8315] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:20:12 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:12.342956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d0c82-fe4c-4f1d-ae99-8e24b08d5b55 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:12.354614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d0c82-fe4c-4f1d-ae99-8e24b08d5b55 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:12.413755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74673699-9359-46ca-9203-9d94f38e946f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:12.414300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74673699-9359-46ca-9203-9d94f38e946f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:12.414586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74673699-9359-46ca-9203-9d94f38e946f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:12.414862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74673699-9359-46ca-9203-9d94f38e946f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:12.415569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74673699-9359-46ca-9203-9d94f38e946f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:12.417570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74673699-9359-46ca-9203-9d94f38e946f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:12.442603 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4114/8316] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:20:12 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:12.443754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38f4a7-0a53-4db9-9225-1af7c0c2f7d9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:12.454395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38f4a7-0a53-4db9-9225-1af7c0c2f7d9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:12.460101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf2686-1573-45dc-a839-0d7cd0559029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:12.460827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf2686-1573-45dc-a839-0d7cd0559029 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:12.461158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf2686-1573-45dc-a839-0d7cd0559029 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:12.461667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf2686-1573-45dc-a839-0d7cd0559029 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:12.462191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf2686-1573-45dc-a839-0d7cd0559029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:12.463978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf2686-1573-45dc-a839-0d7cd0559029 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:12.477295 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4203/8317] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:12.481605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74673699-9359-46ca-9203-9d94f38e946f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:12.493295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74673699-9359-46ca-9203-9d94f38e946f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:12.498012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164d6b1-9bf7-4dba-8532-cd80555c66a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:12.498674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164d6b1-9bf7-4dba-8532-cd80555c66a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:12.498963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164d6b1-9bf7-4dba-8532-cd80555c66a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:12.499247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164d6b1-9bf7-4dba-8532-cd80555c66a4 admin admin] PATH_INFO: `/v3/projects/0780ad941adc4fc3a94a3a7dc5ebb0e7/users/6a793389409b445d807584d0ee0263e8/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:12.515597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164d6b1-9bf7-4dba-8532-cd80555c66a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0780ad941adc4fc3a94a3a7dc5ebb0e7, user_id=6a793389409b445d807584d0ee0263e8, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:12.518487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164d6b1-9bf7-4dba-8532-cd80555c66a4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:12.521603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:12.533577 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4115/8318] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:12 2022] PUT /identity/v3/projects/0780ad941adc4fc3a94a3a7dc5ebb0e7/users/6a793389409b445d807584d0ee0263e8/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:12.538114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf2686-1573-45dc-a839-0d7cd0559029 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:12.548287 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf2686-1573-45dc-a839-0d7cd0559029 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:12.614489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc92f5-b28e-47cf-a617-36a320dfd383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:12.615119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc92f5-b28e-47cf-a617-36a320dfd383 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:12.615547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc92f5-b28e-47cf-a617-36a320dfd383 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:12.615888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc92f5-b28e-47cf-a617-36a320dfd383 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:12.616634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc92f5-b28e-47cf-a617-36a320dfd383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:12.618368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc92f5-b28e-47cf-a617-36a320dfd383 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:12.631917 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4204/8319] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:20:12.633106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164d6b1-9bf7-4dba-8532-cd80555c66a4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:12.651126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164d6b1-9bf7-4dba-8532-cd80555c66a4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:12.656607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44003316-29cf-4615-b4de-b517d2c4eca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:12.657370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44003316-29cf-4615-b4de-b517d2c4eca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:12.657656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44003316-29cf-4615-b4de-b517d2c4eca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:12.657935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44003316-29cf-4615-b4de-b517d2c4eca4 admin admin] PATH_INFO: `/v3/projects/0780ad941adc4fc3a94a3a7dc5ebb0e7/users/6a793389409b445d807584d0ee0263e8/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:12.659881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44003316-29cf-4615-b4de-b517d2c4eca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0780ad941adc4fc3a94a3a7dc5ebb0e7, user_id=6a793389409b445d807584d0ee0263e8, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:12.663191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44003316-29cf-4615-b4de-b517d2c4eca4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:12.674337 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4116/8320] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:12 2022] PUT /identity/v3/projects/0780ad941adc4fc3a94a3a7dc5ebb0e7/users/6a793389409b445d807584d0ee0263e8/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:20:12.684721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09a530-b3a7-43c0-b8e0-3665806f0355 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:12.685380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09a530-b3a7-43c0-b8e0-3665806f0355 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:12.685667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09a530-b3a7-43c0-b8e0-3665806f0355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:12.715530 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-be09a530-b3a7-43c0-b8e0-3665806f0355 None None] MFA Rules not processed for user `6a793389409b445d807584d0ee0263e8`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:12.716773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-be09a530-b3a7-43c0-b8e0-3665806f0355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:12.768480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:12.790695 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4205/8321] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:12.803070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44003316-29cf-4615-b4de-b517d2c4eca4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:12.805668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-44003316-29cf-4615-b4de-b517d2c4eca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:12.887918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44003316-29cf-4615-b4de-b517d2c4eca4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:12.893061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362bcec-f0d7-457b-8708-201dbaaf4b03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:12.893687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362bcec-f0d7-457b-8708-201dbaaf4b03 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:12.893996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362bcec-f0d7-457b-8708-201dbaaf4b03 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:12.894285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362bcec-f0d7-457b-8708-201dbaaf4b03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:12.958062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362bcec-f0d7-457b-8708-201dbaaf4b03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:12.959445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362bcec-f0d7-457b-8708-201dbaaf4b03 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:12.969884 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4117/8322] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:20:16.249173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09a530-b3a7-43c0-b8e0-3665806f0355 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:16.320874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09a530-b3a7-43c0-b8e0-3665806f0355 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:16.325499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:16.326137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:16.326433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:16.326712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] PATH_INFO: `/v3/users/e31059313fd543bebad86a698ed08587` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:16.327844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e31059313fd543bebad86a698ed08587)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:16.329815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:16.386760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e31059313fd543bebad86a698ed08587'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:16.399347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e31059313fd543bebad86a698ed08587'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:16.407053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e31059313fd543bebad86a698ed08587'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:16.407733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e31059313fd543bebad86a698ed08587'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:16.408349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e31059313fd543bebad86a698ed08587'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:16.421018 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4206/8323] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:20:16 2022] DELETE /identity/v3/users/e31059313fd543bebad86a698ed08587 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:16.425669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02807a-c1d6-4141-9ef6-6230ac74760c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:16.426169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02807a-c1d6-4141-9ef6-6230ac74760c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:16.426832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02807a-c1d6-4141-9ef6-6230ac74760c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:16.448348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8f02807a-c1d6-4141-9ef6-6230ac74760c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:16.449689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8f02807a-c1d6-4141-9ef6-6230ac74760c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:16.511002 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4118/8324] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:16.521063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:16.522784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:16.597570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5908bc-6aef-4e95-ba77-8588cd66170d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:16.601576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2a1c2-9e96-4ece-a5d7-6ce0c1a30fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:16.602365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2a1c2-9e96-4ece-a5d7-6ce0c1a30fdf service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:16.602685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2a1c2-9e96-4ece-a5d7-6ce0c1a30fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:16.603046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2a1c2-9e96-4ece-a5d7-6ce0c1a30fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:16.613233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2a1c2-9e96-4ece-a5d7-6ce0c1a30fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:16.615279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2a1c2-9e96-4ece-a5d7-6ce0c1a30fdf service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:16.625831 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4207/8325] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:20:17.048759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02807a-c1d6-4141-9ef6-6230ac74760c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:17.050340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8f02807a-c1d6-4141-9ef6-6230ac74760c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:17.063560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02807a-c1d6-4141-9ef6-6230ac74760c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:17.073741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:17.075140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:17.075140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:17.075140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] PATH_INFO: `/v3/projects/ac65433f9bf84f9c8b4771d54c3c16b4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:17.077499 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac65433f9bf84f9c8b4771d54c3c16b4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:17.079624 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:17.145760 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=2,keystone:SELECT=96,keystone:DELETE=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:17.165468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac65433f9bf84f9c8b4771d54c3c16b4', 'request_id': 'req-cc25022d-e403-4f6a-a6a6-6c9d146868f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.175543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac65433f9bf84f9c8b4771d54c3c16b4', 'request_id': 'req-cc25022d-e403-4f6a-a6a6-6c9d146868f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.183537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac65433f9bf84f9c8b4771d54c3c16b4', 'request_id': 'req-cc25022d-e403-4f6a-a6a6-6c9d146868f2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.198266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] The token cache is being invalidate because project ac65433f9bf84f9c8b4771d54c3c16b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:20:17.198570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.199258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.201160 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4119/8326] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:20:17 2022] DELETE /identity/v3/projects/ac65433f9bf84f9c8b4771d54c3c16b4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:17.205167 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2a1c2-9e96-4ece-a5d7-6ce0c1a30fdf service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:17.215943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2a1c2-9e96-4ece-a5d7-6ce0c1a30fdf service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:17.225466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:17.287189 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:17.288592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:17.289685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:17.290132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] PATH_INFO: `/v3/users/c4071140f98e410586576763dd6c9191` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:17.291543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4071140f98e410586576763dd6c9191)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:17.293892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:17.349636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4071140f98e410586576763dd6c9191'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.364043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4071140f98e410586576763dd6c9191'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.372554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4071140f98e410586576763dd6c9191'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.373291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4071140f98e410586576763dd6c9191'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.373924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4071140f98e410586576763dd6c9191'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.387725 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4208/8327] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:20:17 2022] DELETE /identity/v3/users/c4071140f98e410586576763dd6c9191 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:17.684688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:17.686291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:17.706325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25022d-e403-4f6a-a6a6-6c9d146868f2 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:17.769033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:17.769609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:17.769895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:17.770734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] PATH_INFO: `/v3/projects/fc41fdb3eb9b4895abf88f180001c1ef` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:17.771890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc41fdb3eb9b4895abf88f180001c1ef)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:17.773917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:17.824599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc41fdb3eb9b4895abf88f180001c1ef', 'request_id': 'req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.833811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc41fdb3eb9b4895abf88f180001c1ef', 'request_id': 'req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.841305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc41fdb3eb9b4895abf88f180001c1ef', 'request_id': 'req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.853122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] The token cache is being invalidate because project fc41fdb3eb9b4895abf88f180001c1ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:20:17.853425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.854110 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:17.855920 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4120/8328] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:20:17 2022] DELETE /identity/v3/projects/fc41fdb3eb9b4895abf88f180001c1ef => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:17.860425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:17.861919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:17.875160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2aff58-2e94-49c6-b0cd-1b9f58f1e9d1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:17.936180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d71e8-4275-4ca8-a3b7-8ce1703cbcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:17.936826 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d71e8-4275-4ca8-a3b7-8ce1703cbcbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:17.937154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d71e8-4275-4ca8-a3b7-8ce1703cbcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:17.937431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d71e8-4275-4ca8-a3b7-8ce1703cbcbf admin admin] PATH_INFO: `/v3/users/6a793389409b445d807584d0ee0263e8` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:17.938589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d71e8-4275-4ca8-a3b7-8ce1703cbcbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a793389409b445d807584d0ee0263e8)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:17.940596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d71e8-4275-4ca8-a3b7-8ce1703cbcbf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:17.978091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:17.994553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-112d71e8-4275-4ca8-a3b7-8ce1703cbcbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a793389409b445d807584d0ee0263e8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:18.009092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-112d71e8-4275-4ca8-a3b7-8ce1703cbcbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a793389409b445d807584d0ee0263e8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:18.017791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-112d71e8-4275-4ca8-a3b7-8ce1703cbcbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a793389409b445d807584d0ee0263e8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:18.018432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-112d71e8-4275-4ca8-a3b7-8ce1703cbcbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a793389409b445d807584d0ee0263e8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:18.019101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-112d71e8-4275-4ca8-a3b7-8ce1703cbcbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a793389409b445d807584d0ee0263e8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:18.032407 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4209/8329] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:20:17 2022] DELETE /identity/v3/users/6a793389409b445d807584d0ee0263e8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:18.323449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:18.325581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:18.349443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436fc0a3-4c88-4ac9-a2a0-d04e58dd4a8a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:18.404075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:18.418133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87116a0-1f3e-4c44-9d52-a0133a95043c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:18.418915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87116a0-1f3e-4c44-9d52-a0133a95043c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:18.419252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87116a0-1f3e-4c44-9d52-a0133a95043c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:18.419533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87116a0-1f3e-4c44-9d52-a0133a95043c admin admin] PATH_INFO: `/v3/projects/0780ad941adc4fc3a94a3a7dc5ebb0e7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:18.420693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87116a0-1f3e-4c44-9d52-a0133a95043c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0780ad941adc4fc3a94a3a7dc5ebb0e7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:18.422678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87116a0-1f3e-4c44-9d52-a0133a95043c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:18.470560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e87116a0-1f3e-4c44-9d52-a0133a95043c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0780ad941adc4fc3a94a3a7dc5ebb0e7', 'request_id': 'req-e87116a0-1f3e-4c44-9d52-a0133a95043c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:18.478390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e87116a0-1f3e-4c44-9d52-a0133a95043c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0780ad941adc4fc3a94a3a7dc5ebb0e7', 'request_id': 'req-e87116a0-1f3e-4c44-9d52-a0133a95043c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:18.485124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e87116a0-1f3e-4c44-9d52-a0133a95043c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0780ad941adc4fc3a94a3a7dc5ebb0e7', 'request_id': 'req-e87116a0-1f3e-4c44-9d52-a0133a95043c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:18.497362 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e87116a0-1f3e-4c44-9d52-a0133a95043c admin admin] The token cache is being invalidate because project 0780ad941adc4fc3a94a3a7dc5ebb0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:20:18.497685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e87116a0-1f3e-4c44-9d52-a0133a95043c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:18.498376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e87116a0-1f3e-4c44-9d52-a0133a95043c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:20:18.500152 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4121/8330] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:20:18 2022] DELETE /identity/v3/projects/0780ad941adc4fc3a94a3a7dc5ebb0e7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:18.505433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3a1fd-db6d-4833-add2-0e71d7261f6b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:18.505923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3a1fd-db6d-4833-add2-0e71d7261f6b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:18.506036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3a1fd-db6d-4833-add2-0e71d7261f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:18.526692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-03d3a1fd-db6d-4833-add2-0e71d7261f6b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:18.528011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-03d3a1fd-db6d-4833-add2-0e71d7261f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:18.589479 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4210/8331] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:18.597374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a2e88-ef36-4a9b-8855-0f3e1d72ce1c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:18.597700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a2e88-ef36-4a9b-8855-0f3e1d72ce1c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:18.597994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a2e88-ef36-4a9b-8855-0f3e1d72ce1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:18.622233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-da6a2e88-ef36-4a9b-8855-0f3e1d72ce1c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:18.623566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-da6a2e88-ef36-4a9b-8855-0f3e1d72ce1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:18.631713 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4122/8332] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:18.636489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3a1fd-db6d-4833-add2-0e71d7261f6b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:18.648704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3a1fd-db6d-4833-add2-0e71d7261f6b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:18.653777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a36852-f5b3-44f2-a07b-921e12f78346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:18.654387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a36852-f5b3-44f2-a07b-921e12f78346 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:18.654675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a36852-f5b3-44f2-a07b-921e12f78346 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:18.654952 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a36852-f5b3-44f2-a07b-921e12f78346 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:18.655785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a36852-f5b3-44f2-a07b-921e12f78346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:20:18.656100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a36852-f5b3-44f2-a07b-921e12f78346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:18.657749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a36852-f5b3-44f2-a07b-921e12f78346 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:18.674136 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4211/8333] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:20:18.675345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a2e88-ef36-4a9b-8855-0f3e1d72ce1c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:18.686036 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a2e88-ef36-4a9b-8855-0f3e1d72ce1c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:18.693682 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d63789-9568-49f9-b8e2-45fd62081b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:18.694318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d63789-9568-49f9-b8e2-45fd62081b9e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:18.694812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d63789-9568-49f9-b8e2-45fd62081b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:18.695092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d63789-9568-49f9-b8e2-45fd62081b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:18.696018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d63789-9568-49f9-b8e2-45fd62081b9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:18.697703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d63789-9568-49f9-b8e2-45fd62081b9e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:18.735243 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4123/8334] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:20:18 2022] POST /identity/v3/projects => generated 389 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:20:18.739090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a36852-f5b3-44f2-a07b-921e12f78346 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:18.750680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a36852-f5b3-44f2-a07b-921e12f78346 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:18.829935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cb7e8-81b2-4cfd-96df-b8a3f8be40e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:18.830559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cb7e8-81b2-4cfd-96df-b8a3f8be40e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:18.830850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cb7e8-81b2-4cfd-96df-b8a3f8be40e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:18.831135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cb7e8-81b2-4cfd-96df-b8a3f8be40e0 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:18.831779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cb7e8-81b2-4cfd-96df-b8a3f8be40e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:18.834340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cb7e8-81b2-4cfd-96df-b8a3f8be40e0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:18.859227 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4212/8335] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:20:18 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:18.863479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d63789-9568-49f9-b8e2-45fd62081b9e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:18.874300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d63789-9568-49f9-b8e2-45fd62081b9e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:18.879365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a040f-81de-4af5-b2c3-154f6d8914a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:18.885026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a040f-81de-4af5-b2c3-154f6d8914a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:18.885485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a040f-81de-4af5-b2c3-154f6d8914a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:18.885923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a040f-81de-4af5-b2c3-154f6d8914a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:18.886586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a040f-81de-4af5-b2c3-154f6d8914a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:18.888209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a040f-81de-4af5-b2c3-154f6d8914a9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:18.899708 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4124/8336] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:18.903701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cb7e8-81b2-4cfd-96df-b8a3f8be40e0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:18.919957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cb7e8-81b2-4cfd-96df-b8a3f8be40e0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:18.926388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3c4e5-3560-4db6-8a58-b85bb912c7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:18.927504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3c4e5-3560-4db6-8a58-b85bb912c7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:18.927800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3c4e5-3560-4db6-8a58-b85bb912c7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:18.928096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3c4e5-3560-4db6-8a58-b85bb912c7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:18.928860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3c4e5-3560-4db6-8a58-b85bb912c7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:18.930706 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3c4e5-3560-4db6-8a58-b85bb912c7b4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:18.942463 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4213/8337] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:18.946664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a040f-81de-4af5-b2c3-154f6d8914a9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:18.957255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a040f-81de-4af5-b2c3-154f6d8914a9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:18.964726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325667a-3cf7-4c3f-b9a7-7e7f14b35b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:18.965819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325667a-3cf7-4c3f-b9a7-7e7f14b35b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:18.965819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325667a-3cf7-4c3f-b9a7-7e7f14b35b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:18.966420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325667a-3cf7-4c3f-b9a7-7e7f14b35b8f admin admin] PATH_INFO: `/v3/projects/800b0894aabe47369a800b88dae3d10e/users/7a40d55acc0d452dae9ffadfa35fae88/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:18.988094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325667a-3cf7-4c3f-b9a7-7e7f14b35b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=800b0894aabe47369a800b88dae3d10e, user_id=7a40d55acc0d452dae9ffadfa35fae88, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:18.991070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325667a-3cf7-4c3f-b9a7-7e7f14b35b8f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:19.016437 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4125/8338] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:18 2022] PUT /identity/v3/projects/800b0894aabe47369a800b88dae3d10e/users/7a40d55acc0d452dae9ffadfa35fae88/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:20:19.028673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ec187-1d32-45e1-a3dd-691f68273101 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:19.029004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ec187-1d32-45e1-a3dd-691f68273101 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:19.029293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ec187-1d32-45e1-a3dd-691f68273101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:19.063075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a79ec187-1d32-45e1-a3dd-691f68273101 None None] MFA Rules not processed for user `7a40d55acc0d452dae9ffadfa35fae88`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:19.064371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a79ec187-1d32-45e1-a3dd-691f68273101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:19.070852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:19.163906 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4214/8339] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:19.168750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325667a-3cf7-4c3f-b9a7-7e7f14b35b8f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:19.248107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325667a-3cf7-4c3f-b9a7-7e7f14b35b8f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:19.253025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46804f61-b351-48ce-8fb4-78d2067f5fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:19.253640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46804f61-b351-48ce-8fb4-78d2067f5fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:19.253926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46804f61-b351-48ce-8fb4-78d2067f5fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:19.254206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46804f61-b351-48ce-8fb4-78d2067f5fb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:19.255531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46804f61-b351-48ce-8fb4-78d2067f5fb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:19.257481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46804f61-b351-48ce-8fb4-78d2067f5fb6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:19.279132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:19.286477 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4126/8340] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:20:19 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:19.290669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ec187-1d32-45e1-a3dd-691f68273101 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:19.305302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ec187-1d32-45e1-a3dd-691f68273101 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:19.387763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0127b-325a-4216-a611-5bbba99fc769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:19.388385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0127b-325a-4216-a611-5bbba99fc769 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:19.388675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0127b-325a-4216-a611-5bbba99fc769 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:19.388998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0127b-325a-4216-a611-5bbba99fc769 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:19.389644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0127b-325a-4216-a611-5bbba99fc769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:19.391595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0127b-325a-4216-a611-5bbba99fc769 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:19.413516 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4215/8341] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:20:19 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:19.417044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46804f61-b351-48ce-8fb4-78d2067f5fb6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:19.428349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46804f61-b351-48ce-8fb4-78d2067f5fb6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:19.433179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c3f3f-8b1e-41f5-af33-e6638a4dcaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:19.433734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c3f3f-8b1e-41f5-af33-e6638a4dcaf1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:19.434023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c3f3f-8b1e-41f5-af33-e6638a4dcaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:19.434306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c3f3f-8b1e-41f5-af33-e6638a4dcaf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:19.434959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c3f3f-8b1e-41f5-af33-e6638a4dcaf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:19.436582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c3f3f-8b1e-41f5-af33-e6638a4dcaf1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:19.453920 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4127/8342] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:19.455175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0127b-325a-4216-a611-5bbba99fc769 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:19.465750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0127b-325a-4216-a611-5bbba99fc769 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:19.470468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23e18b-577c-4c91-a254-dbf1878771f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:19.471073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23e18b-577c-4c91-a254-dbf1878771f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:19.471352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23e18b-577c-4c91-a254-dbf1878771f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:19.471626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23e18b-577c-4c91-a254-dbf1878771f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:19.472266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23e18b-577c-4c91-a254-dbf1878771f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:19.473892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23e18b-577c-4c91-a254-dbf1878771f4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:19.487839 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4216/8343] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:19.489692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c3f3f-8b1e-41f5-af33-e6638a4dcaf1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:19.500953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c3f3f-8b1e-41f5-af33-e6638a4dcaf1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:19.506518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f9a67-5262-4f3e-bd29-4866e525b9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:19.507283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f9a67-5262-4f3e-bd29-4866e525b9ff admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:19.507573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f9a67-5262-4f3e-bd29-4866e525b9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:19.507853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f9a67-5262-4f3e-bd29-4866e525b9ff admin admin] PATH_INFO: `/v3/projects/95fe7cf631a648e6a49712582f0d64bf/users/bb2753efc0794ab1a69fc22cd6d6e410/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:19.523531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f9a67-5262-4f3e-bd29-4866e525b9ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=95fe7cf631a648e6a49712582f0d64bf, user_id=bb2753efc0794ab1a69fc22cd6d6e410, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:19.526229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f9a67-5262-4f3e-bd29-4866e525b9ff admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:19.542976 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4128/8344] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:19 2022] PUT /identity/v3/projects/95fe7cf631a648e6a49712582f0d64bf/users/bb2753efc0794ab1a69fc22cd6d6e410/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:19.554679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de685448-b3cc-4490-9c1c-081983a19bb7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:19.555228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de685448-b3cc-4490-9c1c-081983a19bb7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:19.555228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de685448-b3cc-4490-9c1c-081983a19bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:19.587449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-de685448-b3cc-4490-9c1c-081983a19bb7 None None] MFA Rules not processed for user `bb2753efc0794ab1a69fc22cd6d6e410`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:19.589149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-de685448-b3cc-4490-9c1c-081983a19bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:19.662894 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4217/8345] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:19.667411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f9a67-5262-4f3e-bd29-4866e525b9ff admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:19.730228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f9a67-5262-4f3e-bd29-4866e525b9ff admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:19.735243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ec181-521f-49eb-914d-8e5bdbd50d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:19.735855 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ec181-521f-49eb-914d-8e5bdbd50d12 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:19.736138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ec181-521f-49eb-914d-8e5bdbd50d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:19.736411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ec181-521f-49eb-914d-8e5bdbd50d12 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:19.737084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ec181-521f-49eb-914d-8e5bdbd50d12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:19.739582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ec181-521f-49eb-914d-8e5bdbd50d12 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:19.769129 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4129/8346] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:20:19 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:19.773253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de685448-b3cc-4490-9c1c-081983a19bb7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:19.785564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de685448-b3cc-4490-9c1c-081983a19bb7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:19.795215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:19.847663 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b05430-bf53-4426-b957-564071793f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:19.848786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b05430-bf53-4426-b957-564071793f4c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:19.849106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b05430-bf53-4426-b957-564071793f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:19.849386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b05430-bf53-4426-b957-564071793f4c admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:19.850142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b05430-bf53-4426-b957-564071793f4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:19.852091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b05430-bf53-4426-b957-564071793f4c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:19.878326 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4218/8347] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:20:19 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:20:19.882193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ec181-521f-49eb-914d-8e5bdbd50d12 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:19.893047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ec181-521f-49eb-914d-8e5bdbd50d12 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:19.898324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0827b-f3a2-4952-ba3a-7d2c765be107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:19.898981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0827b-f3a2-4952-ba3a-7d2c765be107 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:19.899356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0827b-f3a2-4952-ba3a-7d2c765be107 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:19.899623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0827b-f3a2-4952-ba3a-7d2c765be107 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:19.900324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0827b-f3a2-4952-ba3a-7d2c765be107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:19.902328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0827b-f3a2-4952-ba3a-7d2c765be107 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:19.916346 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4130/8348] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:19.920228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b05430-bf53-4426-b957-564071793f4c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:19.934539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b05430-bf53-4426-b957-564071793f4c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:19.941085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35564c-b9e5-4a89-b50b-5dc005740736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:19.941827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35564c-b9e5-4a89-b50b-5dc005740736 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:19.942185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35564c-b9e5-4a89-b50b-5dc005740736 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:19.942541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35564c-b9e5-4a89-b50b-5dc005740736 admin admin] PATH_INFO: `/v3/projects/98726b2139ed475da0b0482cd0a23d38/users/828c9cd414c24a2b9e2f1de713020d21/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:19.960350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35564c-b9e5-4a89-b50b-5dc005740736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98726b2139ed475da0b0482cd0a23d38, user_id=828c9cd414c24a2b9e2f1de713020d21, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:19.963319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35564c-b9e5-4a89-b50b-5dc005740736 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:19.982787 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4219/8349] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:19 2022] PUT /identity/v3/projects/98726b2139ed475da0b0482cd0a23d38/users/828c9cd414c24a2b9e2f1de713020d21/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:19.986821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0827b-f3a2-4952-ba3a-7d2c765be107 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:19.998174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0827b-f3a2-4952-ba3a-7d2c765be107 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:20.067212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dad8b-c0bf-4ab1-914c-f688db107cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:20.067802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dad8b-c0bf-4ab1-914c-f688db107cdb admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:20.068094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dad8b-c0bf-4ab1-914c-f688db107cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:20.068375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dad8b-c0bf-4ab1-914c-f688db107cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:20.069079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4dad8b-c0bf-4ab1-914c-f688db107cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:20.070698 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4dad8b-c0bf-4ab1-914c-f688db107cdb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:20.081220 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4131/8350] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:20:20.086004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35564c-b9e5-4a89-b50b-5dc005740736 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:20.097057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35564c-b9e5-4a89-b50b-5dc005740736 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:20.102529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9376d3-9bae-49ea-b6e3-cfd5361d6404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:20.103252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9376d3-9bae-49ea-b6e3-cfd5361d6404 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:20.104523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9376d3-9bae-49ea-b6e3-cfd5361d6404 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:20.104523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9376d3-9bae-49ea-b6e3-cfd5361d6404 admin admin] PATH_INFO: `/v3/projects/98726b2139ed475da0b0482cd0a23d38/users/828c9cd414c24a2b9e2f1de713020d21/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:20.105930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9376d3-9bae-49ea-b6e3-cfd5361d6404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98726b2139ed475da0b0482cd0a23d38, user_id=828c9cd414c24a2b9e2f1de713020d21, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:20.108850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9376d3-9bae-49ea-b6e3-cfd5361d6404 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:20.131276 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4220/8351] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:20:20 2022] PUT /identity/v3/projects/98726b2139ed475da0b0482cd0a23d38/users/828c9cd414c24a2b9e2f1de713020d21/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:20:20.141510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c537155-7fdb-4995-975e-c442e40e2748 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:20.141817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c537155-7fdb-4995-975e-c442e40e2748 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:20.142096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c537155-7fdb-4995-975e-c442e40e2748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:20.179944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1c537155-7fdb-4995-975e-c442e40e2748 None None] MFA Rules not processed for user `828c9cd414c24a2b9e2f1de713020d21`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:20.181227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1c537155-7fdb-4995-975e-c442e40e2748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:20.212500 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:20.261418 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4132/8352] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:20.278525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9376d3-9bae-49ea-b6e3-cfd5361d6404 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:20.281442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bd9376d3-9bae-49ea-b6e3-cfd5361d6404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:20.372496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9376d3-9bae-49ea-b6e3-cfd5361d6404 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:20.383231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30392d-eef6-46a6-8c20-a9ad6baee3ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:20.384075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30392d-eef6-46a6-8c20-a9ad6baee3ed service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:20.384367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30392d-eef6-46a6-8c20-a9ad6baee3ed service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:20.384640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30392d-eef6-46a6-8c20-a9ad6baee3ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:20.456245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30392d-eef6-46a6-8c20-a9ad6baee3ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:20.459940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30392d-eef6-46a6-8c20-a9ad6baee3ed service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:20.469187 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4221/8353] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:20:23.561226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c537155-7fdb-4995-975e-c442e40e2748 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:23.563751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1c537155-7fdb-4995-975e-c442e40e2748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:23.654719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c537155-7fdb-4995-975e-c442e40e2748 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:23.659180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b91b4-98b4-427e-b2b5-6fd33332fb46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:23.659966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b91b4-98b4-427e-b2b5-6fd33332fb46 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:23.661083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b91b4-98b4-427e-b2b5-6fd33332fb46 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:23.661409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b91b4-98b4-427e-b2b5-6fd33332fb46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:23.673750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b91b4-98b4-427e-b2b5-6fd33332fb46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:23.675223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b91b4-98b4-427e-b2b5-6fd33332fb46 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:23.686768 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4133/8354] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:20:29.100929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728a7f2-5def-4c31-a17d-a164fa7e3ba6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:29.101515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728a7f2-5def-4c31-a17d-a164fa7e3ba6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:29.101515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728a7f2-5def-4c31-a17d-a164fa7e3ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:29.134914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:29.154641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8728a7f2-5def-4c31-a17d-a164fa7e3ba6 None None] MFA Rules not processed for user `7a40d55acc0d452dae9ffadfa35fae88`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:20:29.156259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8728a7f2-5def-4c31-a17d-a164fa7e3ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:29.170782 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4222/8355] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:20:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:20:33.681920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=44 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:38.615046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b91b4-98b4-427e-b2b5-6fd33332fb46 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:38.617099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e18b91b4-98b4-427e-b2b5-6fd33332fb46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:38.644429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b91b4-98b4-427e-b2b5-6fd33332fb46 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:38.651199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07279c-46b4-4856-99a1-4ec52cdd9920 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:38.651849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07279c-46b4-4856-99a1-4ec52cdd9920 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:38.652212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07279c-46b4-4856-99a1-4ec52cdd9920 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:38.652822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07279c-46b4-4856-99a1-4ec52cdd9920 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:38.654813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9c07279c-46b4-4856-99a1-4ec52cdd9920 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:38.675467 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07279c-46b4-4856-99a1-4ec52cdd9920 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:38.678109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07279c-46b4-4856-99a1-4ec52cdd9920 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:38.696173 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4134/8356] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:20:39.155481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:44.829973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728a7f2-5def-4c31-a17d-a164fa7e3ba6 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:44.850647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728a7f2-5def-4c31-a17d-a164fa7e3ba6 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:44.856881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d6f8a-c931-4f38-9bdf-f679e5ef9667 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:44.857833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d6f8a-c931-4f38-9bdf-f679e5ef9667 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:44.858306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d6f8a-c931-4f38-9bdf-f679e5ef9667 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:44.858592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d6f8a-c931-4f38-9bdf-f679e5ef9667 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:44.860528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a26d6f8a-c931-4f38-9bdf-f679e5ef9667 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:44.877821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d6f8a-c931-4f38-9bdf-f679e5ef9667 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:44.880279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d6f8a-c931-4f38-9bdf-f679e5ef9667 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:44.894581 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4223/8357] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:20:47.592094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07279c-46b4-4856-99a1-4ec52cdd9920 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:20:47.617493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07279c-46b4-4856-99a1-4ec52cdd9920 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:20:47.622704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e06cd-83ce-4560-8bda-225d2612aba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:20:47.623589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e06cd-83ce-4560-8bda-225d2612aba8 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:20:47.624210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e06cd-83ce-4560-8bda-225d2612aba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:20:47.624688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e06cd-83ce-4560-8bda-225d2612aba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:20:47.627868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b45e06cd-83ce-4560-8bda-225d2612aba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:20:47.642514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e06cd-83ce-4560-8bda-225d2612aba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:20:47.644415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e06cd-83ce-4560-8bda-225d2612aba8 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:20:47.656824 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4135/8358] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:20:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:20:54.887307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:20:57.649807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=12 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:10.574815 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d6f8a-c931-4f38-9bdf-f679e5ef9667 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:10.576252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a26d6f8a-c931-4f38-9bdf-f679e5ef9667 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:10.639827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d6f8a-c931-4f38-9bdf-f679e5ef9667 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:10.644524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:10.645138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:10.645424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:10.645702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] PATH_INFO: `/v3/users/e3b33ad8cf6f4543b6cf451f504eb2cf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:10.646818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b33ad8cf6f4543b6cf451f504eb2cf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:10.648743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:10.695328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b33ad8cf6f4543b6cf451f504eb2cf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:10.707610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b33ad8cf6f4543b6cf451f504eb2cf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:10.714995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b33ad8cf6f4543b6cf451f504eb2cf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:10.715614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b33ad8cf6f4543b6cf451f504eb2cf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:10.716234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b33ad8cf6f4543b6cf451f504eb2cf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:10.728744 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4224/8359] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:21:10 2022] DELETE /identity/v3/users/e3b33ad8cf6f4543b6cf451f504eb2cf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:10.733534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58f8aa-e04c-45f4-a4ec-2168c202fb84 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:10.734044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58f8aa-e04c-45f4-a4ec-2168c202fb84 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:10.734164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58f8aa-e04c-45f4-a4ec-2168c202fb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:10.753404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fe58f8aa-e04c-45f4-a4ec-2168c202fb84 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:10.754620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fe58f8aa-e04c-45f4-a4ec-2168c202fb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:10.812313 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4136/8360] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:10.821932 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:10.823344 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:10.882306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccd545-603e-46bc-9d41-95a5d21de2ef admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:10.885944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93600110-a0ac-479a-bc6d-65d415f1d00a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:10.886569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93600110-a0ac-479a-bc6d-65d415f1d00a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:10.886855 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93600110-a0ac-479a-bc6d-65d415f1d00a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:10.887128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93600110-a0ac-479a-bc6d-65d415f1d00a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:10.895992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93600110-a0ac-479a-bc6d-65d415f1d00a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:10.897475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93600110-a0ac-479a-bc6d-65d415f1d00a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:10.906773 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4225/8361] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:11.073147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58f8aa-e04c-45f4-a4ec-2168c202fb84 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:11.074611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fe58f8aa-e04c-45f4-a4ec-2168c202fb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:11.087460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58f8aa-e04c-45f4-a4ec-2168c202fb84 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:11.092508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:11.093185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.093474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.093755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64 admin admin] PATH_INFO: `/v3/projects/7b1dbe8cc2a144c68f7b85b47c26a302` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.094986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1dbe8cc2a144c68f7b85b47c26a302)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:11.097015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:11.147782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1dbe8cc2a144c68f7b85b47c26a302', 'request_id': 'req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:11.154966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1dbe8cc2a144c68f7b85b47c26a302', 'request_id': 'req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:11.161787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1dbe8cc2a144c68f7b85b47c26a302', 'request_id': 'req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:11.171895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64 admin admin] The token cache is being invalidate because project 7b1dbe8cc2a144c68f7b85b47c26a302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:21:11.172183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:11.172820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-00584fdf-4dbe-4b7a-bd8e-b34528e51e64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:11.174327 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4137/8362] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:21:11 2022] DELETE /identity/v3/projects/7b1dbe8cc2a144c68f7b85b47c26a302 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:11.179192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41b922-ec57-4a41-a356-c62d35b83bd0 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.179488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41b922-ec57-4a41-a356-c62d35b83bd0 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.179769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41b922-ec57-4a41-a356-c62d35b83bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.199384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ae41b922-ec57-4a41-a356-c62d35b83bd0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:11.200598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ae41b922-ec57-4a41-a356-c62d35b83bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:11.217746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:11.260735 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4226/8363] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:11.269712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838b6dc-7deb-43fa-aa51-2d5935948d51 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.270069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838b6dc-7deb-43fa-aa51-2d5935948d51 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.270351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838b6dc-7deb-43fa-aa51-2d5935948d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.290399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e838b6dc-7deb-43fa-aa51-2d5935948d51 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:11.291598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e838b6dc-7deb-43fa-aa51-2d5935948d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:11.299644 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4138/8364] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:11.303664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41b922-ec57-4a41-a356-c62d35b83bd0 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:11.313311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41b922-ec57-4a41-a356-c62d35b83bd0 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:11.318133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dafaf-7d9a-466b-8bb8-2964e30e4856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:11.318729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dafaf-7d9a-466b-8bb8-2964e30e4856 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.319013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dafaf-7d9a-466b-8bb8-2964e30e4856 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.319302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dafaf-7d9a-466b-8bb8-2964e30e4856 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.319925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dafaf-7d9a-466b-8bb8-2964e30e4856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:21:11.320235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dafaf-7d9a-466b-8bb8-2964e30e4856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:11.321862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dafaf-7d9a-466b-8bb8-2964e30e4856 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:11.333631 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4227/8365] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:21:11.337727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838b6dc-7deb-43fa-aa51-2d5935948d51 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:11.347068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838b6dc-7deb-43fa-aa51-2d5935948d51 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:11.352285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d2145-407a-4491-a51f-12f9d4a2b926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:11.352876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d2145-407a-4491-a51f-12f9d4a2b926 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.353191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d2145-407a-4491-a51f-12f9d4a2b926 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.353471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d2145-407a-4491-a51f-12f9d4a2b926 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.354101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d2145-407a-4491-a51f-12f9d4a2b926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:11.355982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d2145-407a-4491-a51f-12f9d4a2b926 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:11.382800 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4139/8366] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:11 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:11.386849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dafaf-7d9a-466b-8bb8-2964e30e4856 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:11.397550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dafaf-7d9a-466b-8bb8-2964e30e4856 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:11.452544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dad09a-7040-4c17-8ff0-e4ed828e2834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:11.453181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dad09a-7040-4c17-8ff0-e4ed828e2834 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.453472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dad09a-7040-4c17-8ff0-e4ed828e2834 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.453749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dad09a-7040-4c17-8ff0-e4ed828e2834 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.454384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dad09a-7040-4c17-8ff0-e4ed828e2834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:11.456322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dad09a-7040-4c17-8ff0-e4ed828e2834 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:11.480938 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4228/8367] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:11 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:21:11.482276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d2145-407a-4491-a51f-12f9d4a2b926 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:11.492663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d2145-407a-4491-a51f-12f9d4a2b926 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:11.497244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f6cc8-24e3-4f34-9b4d-215922794619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:11.497835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f6cc8-24e3-4f34-9b4d-215922794619 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.498123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f6cc8-24e3-4f34-9b4d-215922794619 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.498398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f6cc8-24e3-4f34-9b4d-215922794619 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.499030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f6cc8-24e3-4f34-9b4d-215922794619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:11.500615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f6cc8-24e3-4f34-9b4d-215922794619 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:11.511029 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4140/8368] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:11.514978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dad09a-7040-4c17-8ff0-e4ed828e2834 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:11.526002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dad09a-7040-4c17-8ff0-e4ed828e2834 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:11.530620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904936df-7ae8-40c8-9021-8bba4e964d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:11.531412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904936df-7ae8-40c8-9021-8bba4e964d71 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.531550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904936df-7ae8-40c8-9021-8bba4e964d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.531757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904936df-7ae8-40c8-9021-8bba4e964d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.532414 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904936df-7ae8-40c8-9021-8bba4e964d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:11.534144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904936df-7ae8-40c8-9021-8bba4e964d71 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:11.544987 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4229/8369] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:11.549074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f6cc8-24e3-4f34-9b4d-215922794619 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:11.559003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f6cc8-24e3-4f34-9b4d-215922794619 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:11.563645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936e22b-7897-4fe4-ae1d-b90743f0aeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:11.564296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936e22b-7897-4fe4-ae1d-b90743f0aeba admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.564581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936e22b-7897-4fe4-ae1d-b90743f0aeba admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.564861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936e22b-7897-4fe4-ae1d-b90743f0aeba admin admin] PATH_INFO: `/v3/projects/07359173ea184163ae7475f48853a2e4/users/7dc9a5cf13bc4592ae4aa85e38e0b8bd/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.579983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936e22b-7897-4fe4-ae1d-b90743f0aeba admin admin] RBAC: Authorizing `identity:create_grant(project_id=07359173ea184163ae7475f48853a2e4, user_id=7dc9a5cf13bc4592ae4aa85e38e0b8bd, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:11.583442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936e22b-7897-4fe4-ae1d-b90743f0aeba admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:11.596173 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4141/8370] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:11 2022] PUT /identity/v3/projects/07359173ea184163ae7475f48853a2e4/users/7dc9a5cf13bc4592ae4aa85e38e0b8bd/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:11.606211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367fe92-1615-437c-b1ff-15f0d556b4f5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.606683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367fe92-1615-437c-b1ff-15f0d556b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.606885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367fe92-1615-437c-b1ff-15f0d556b4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.635139 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-c367fe92-1615-437c-b1ff-15f0d556b4f5 None None] MFA Rules not processed for user `7dc9a5cf13bc4592ae4aa85e38e0b8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:11.636353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c367fe92-1615-437c-b1ff-15f0d556b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:11.698753 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4230/8371] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:11.709825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936e22b-7897-4fe4-ae1d-b90743f0aeba admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:11.711510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8936e22b-7897-4fe4-ae1d-b90743f0aeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:11.733948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:11.773963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936e22b-7897-4fe4-ae1d-b90743f0aeba admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:11.777675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dedfb2-24d1-4b6c-ba0b-41637676756a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:11.778284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dedfb2-24d1-4b6c-ba0b-41637676756a service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.778569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dedfb2-24d1-4b6c-ba0b-41637676756a service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.778846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dedfb2-24d1-4b6c-ba0b-41637676756a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.787775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dedfb2-24d1-4b6c-ba0b-41637676756a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:11.789175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dedfb2-24d1-4b6c-ba0b-41637676756a service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:11.797859 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4142/8372] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:11.813483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367fe92-1615-437c-b1ff-15f0d556b4f5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:11.832723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:11.877104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367fe92-1615-437c-b1ff-15f0d556b4f5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:11.881718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:11.882342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.882626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.882901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] PATH_INFO: `/v3/users/7dc9a5cf13bc4592ae4aa85e38e0b8bd` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.884015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc9a5cf13bc4592ae4aa85e38e0b8bd)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:11.885988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:11.933008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc9a5cf13bc4592ae4aa85e38e0b8bd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:11.945164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc9a5cf13bc4592ae4aa85e38e0b8bd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:11.953508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc9a5cf13bc4592ae4aa85e38e0b8bd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:11.954114 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc9a5cf13bc4592ae4aa85e38e0b8bd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:11.954745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc9a5cf13bc4592ae4aa85e38e0b8bd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:11.967477 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4231/8373] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:21:11 2022] DELETE /identity/v3/users/7dc9a5cf13bc4592ae4aa85e38e0b8bd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:11.971887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572adf0-3a0a-4788-b622-80f77b5e5b9f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:11.972837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572adf0-3a0a-4788-b622-80f77b5e5b9f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:11.973150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572adf0-3a0a-4788-b622-80f77b5e5b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:11.993619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9572adf0-3a0a-4788-b622-80f77b5e5b9f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:11.994825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9572adf0-3a0a-4788-b622-80f77b5e5b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:12.055588 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4143/8374] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:12.065248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:12.066690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:12.130013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a017343-0113-4ba4-b1ac-f096e1700c42 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:12.136822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf9f569-db3f-4216-971f-6c5396d07019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:12.137407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9f569-db3f-4216-971f-6c5396d07019 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:12.137698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9f569-db3f-4216-971f-6c5396d07019 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:12.137978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9f569-db3f-4216-971f-6c5396d07019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:12.148808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf9f569-db3f-4216-971f-6c5396d07019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:12.150286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf9f569-db3f-4216-971f-6c5396d07019 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:12.159845 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4232/8375] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:12.420927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572adf0-3a0a-4788-b622-80f77b5e5b9f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:12.422412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9572adf0-3a0a-4788-b622-80f77b5e5b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:12.435354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572adf0-3a0a-4788-b622-80f77b5e5b9f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:12.440072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae17f4-7b43-429f-b70c-a80fb455ceec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:12.440708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae17f4-7b43-429f-b70c-a80fb455ceec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:12.441575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae17f4-7b43-429f-b70c-a80fb455ceec admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:12.441824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae17f4-7b43-429f-b70c-a80fb455ceec admin admin] PATH_INFO: `/v3/projects/07359173ea184163ae7475f48853a2e4` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:12.442964 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae17f4-7b43-429f-b70c-a80fb455ceec admin admin] RBAC: Authorizing `identity:delete_project(project_id=07359173ea184163ae7475f48853a2e4)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:12.444870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae17f4-7b43-429f-b70c-a80fb455ceec admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:12.493710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-87ae17f4-7b43-429f-b70c-a80fb455ceec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07359173ea184163ae7475f48853a2e4', 'request_id': 'req-87ae17f4-7b43-429f-b70c-a80fb455ceec'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:12.501347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-87ae17f4-7b43-429f-b70c-a80fb455ceec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07359173ea184163ae7475f48853a2e4', 'request_id': 'req-87ae17f4-7b43-429f-b70c-a80fb455ceec'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:12.508559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-87ae17f4-7b43-429f-b70c-a80fb455ceec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07359173ea184163ae7475f48853a2e4', 'request_id': 'req-87ae17f4-7b43-429f-b70c-a80fb455ceec'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:12.519828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-87ae17f4-7b43-429f-b70c-a80fb455ceec admin admin] The token cache is being invalidate because project 07359173ea184163ae7475f48853a2e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:21:12.520133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-87ae17f4-7b43-429f-b70c-a80fb455ceec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:12.520801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-87ae17f4-7b43-429f-b70c-a80fb455ceec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:12.522328 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4144/8376] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:21:12 2022] DELETE /identity/v3/projects/07359173ea184163ae7475f48853a2e4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:12.527576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243db20-5437-49fe-bc50-478bf93c2ccd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:12.527869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243db20-5437-49fe-bc50-478bf93c2ccd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:12.528148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243db20-5437-49fe-bc50-478bf93c2ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:12.547439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b243db20-5437-49fe-bc50-478bf93c2ccd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:12.548650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b243db20-5437-49fe-bc50-478bf93c2ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:12.582703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:12.609603 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4233/8377] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:12.617669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c0d21-3fa0-4e11-8093-ecf2b66413c0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:12.617959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c0d21-3fa0-4e11-8093-ecf2b66413c0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:12.618233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c0d21-3fa0-4e11-8093-ecf2b66413c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:12.638168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a40c0d21-3fa0-4e11-8093-ecf2b66413c0 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:12.639364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a40c0d21-3fa0-4e11-8093-ecf2b66413c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:12.647222 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4145/8378] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:12.651327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243db20-5437-49fe-bc50-478bf93c2ccd None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:12.660970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243db20-5437-49fe-bc50-478bf93c2ccd None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:12.665523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233e1e2-7b4d-4257-91de-d5886edee510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:12.666120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233e1e2-7b4d-4257-91de-d5886edee510 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:12.666399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233e1e2-7b4d-4257-91de-d5886edee510 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:12.666682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233e1e2-7b4d-4257-91de-d5886edee510 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:12.667327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233e1e2-7b4d-4257-91de-d5886edee510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:21:12.667630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233e1e2-7b4d-4257-91de-d5886edee510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:12.669282 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233e1e2-7b4d-4257-91de-d5886edee510 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:12.681102 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4234/8379] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:21:12.685241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c0d21-3fa0-4e11-8093-ecf2b66413c0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:12.695232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c0d21-3fa0-4e11-8093-ecf2b66413c0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:12.700017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf5279-937c-4cf5-85fb-5bf7692d9ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:12.700672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf5279-937c-4cf5-85fb-5bf7692d9ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:12.701053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf5279-937c-4cf5-85fb-5bf7692d9ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:12.701332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf5279-937c-4cf5-85fb-5bf7692d9ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:12.702052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf5279-937c-4cf5-85fb-5bf7692d9ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:12.703945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf5279-937c-4cf5-85fb-5bf7692d9ec7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:12.733734 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4146/8380] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:12 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:12.734934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233e1e2-7b4d-4257-91de-d5886edee510 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:12.745615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233e1e2-7b4d-4257-91de-d5886edee510 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:12.800919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85438bd5-1f28-4faf-a451-dfbb25a96770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:12.801478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85438bd5-1f28-4faf-a451-dfbb25a96770 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:12.801766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85438bd5-1f28-4faf-a451-dfbb25a96770 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:12.802046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85438bd5-1f28-4faf-a451-dfbb25a96770 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:12.802678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85438bd5-1f28-4faf-a451-dfbb25a96770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:12.804640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85438bd5-1f28-4faf-a451-dfbb25a96770 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:12.826079 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4235/8381] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:12 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:21:12.830074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf5279-937c-4cf5-85fb-5bf7692d9ec7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:12.840419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf5279-937c-4cf5-85fb-5bf7692d9ec7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:12.845146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f33a43-39ee-439b-98fd-7de0743644e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:12.845743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f33a43-39ee-439b-98fd-7de0743644e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:12.846031 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f33a43-39ee-439b-98fd-7de0743644e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:12.846312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f33a43-39ee-439b-98fd-7de0743644e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:12.846981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f33a43-39ee-439b-98fd-7de0743644e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:12.848570 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f33a43-39ee-439b-98fd-7de0743644e9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:12.858910 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4147/8382] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:12.862679 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85438bd5-1f28-4faf-a451-dfbb25a96770 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:12.873016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85438bd5-1f28-4faf-a451-dfbb25a96770 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:12.877617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdd0c5-ce4d-40a3-a717-379ea29a1f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:12.878207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdd0c5-ce4d-40a3-a717-379ea29a1f28 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:12.879046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdd0c5-ce4d-40a3-a717-379ea29a1f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:12.879327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdd0c5-ce4d-40a3-a717-379ea29a1f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:12.879975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdd0c5-ce4d-40a3-a717-379ea29a1f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:12.881600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdd0c5-ce4d-40a3-a717-379ea29a1f28 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:12.891941 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4236/8383] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:12.895736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f33a43-39ee-439b-98fd-7de0743644e9 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:12.905808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f33a43-39ee-439b-98fd-7de0743644e9 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:12.910952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f68d9-20ec-4efb-9c60-4913bc838f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:12.911601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f68d9-20ec-4efb-9c60-4913bc838f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:12.911891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f68d9-20ec-4efb-9c60-4913bc838f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:12.912175 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f68d9-20ec-4efb-9c60-4913bc838f0b admin admin] PATH_INFO: `/v3/projects/451afaf83aed42ac814d855651a766ed/users/58dc43d720a64ca6ba3b812da2864e6c/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:12.914995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:12.930736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f68d9-20ec-4efb-9c60-4913bc838f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=451afaf83aed42ac814d855651a766ed, user_id=58dc43d720a64ca6ba3b812da2864e6c, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:12.933481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f68d9-20ec-4efb-9c60-4913bc838f0b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:12.945924 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4148/8384] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:12 2022] PUT /identity/v3/projects/451afaf83aed42ac814d855651a766ed/users/58dc43d720a64ca6ba3b812da2864e6c/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:12.956823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6cbaf-bb88-4e90-bd2f-7e603edf1620 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:12.957149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6cbaf-bb88-4e90-bd2f-7e603edf1620 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:12.957432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6cbaf-bb88-4e90-bd2f-7e603edf1620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:12.985358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e3b6cbaf-bb88-4e90-bd2f-7e603edf1620 None None] MFA Rules not processed for user `58dc43d720a64ca6ba3b812da2864e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:12.986576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e3b6cbaf-bb88-4e90-bd2f-7e603edf1620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:13.051696 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4237/8385] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:13.056224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f68d9-20ec-4efb-9c60-4913bc838f0b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:13.118729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f68d9-20ec-4efb-9c60-4913bc838f0b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:13.124501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6c374-01eb-4317-b5ab-ccbe92776f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:13.125189 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6c374-01eb-4317-b5ab-ccbe92776f00 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:13.125485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6c374-01eb-4317-b5ab-ccbe92776f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:13.125797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6c374-01eb-4317-b5ab-ccbe92776f00 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:13.126437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6c374-01eb-4317-b5ab-ccbe92776f00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:13.128456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6c374-01eb-4317-b5ab-ccbe92776f00 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:13.155803 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4149/8386] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:13 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:21:13.160442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6cbaf-bb88-4e90-bd2f-7e603edf1620 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:13.171327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6cbaf-bb88-4e90-bd2f-7e603edf1620 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:13.203446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:13.228918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89ae7f-4db0-4684-b35f-583b374cce3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:13.229617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89ae7f-4db0-4684-b35f-583b374cce3e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:13.229933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89ae7f-4db0-4684-b35f-583b374cce3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:13.230209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89ae7f-4db0-4684-b35f-583b374cce3e admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:13.230846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89ae7f-4db0-4684-b35f-583b374cce3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:13.233118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89ae7f-4db0-4684-b35f-583b374cce3e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:13.254141 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4238/8387] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:13 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:13.258034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6c374-01eb-4317-b5ab-ccbe92776f00 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:13.268477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6c374-01eb-4317-b5ab-ccbe92776f00 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:13.273776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9c3ce-6735-4ed8-8dc8-a28ca292563a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:13.274387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9c3ce-6735-4ed8-8dc8-a28ca292563a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:13.274676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9c3ce-6735-4ed8-8dc8-a28ca292563a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:13.274954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9c3ce-6735-4ed8-8dc8-a28ca292563a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:13.275599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9c3ce-6735-4ed8-8dc8-a28ca292563a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:13.277377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9c3ce-6735-4ed8-8dc8-a28ca292563a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:13.289649 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4150/8388] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:13.293743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89ae7f-4db0-4684-b35f-583b374cce3e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:13.303838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89ae7f-4db0-4684-b35f-583b374cce3e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:13.308691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5e6ce-4ccb-4427-8ab3-9a71b0a076c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:13.309389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5e6ce-4ccb-4427-8ab3-9a71b0a076c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:13.309676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5e6ce-4ccb-4427-8ab3-9a71b0a076c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:13.309953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5e6ce-4ccb-4427-8ab3-9a71b0a076c2 admin admin] PATH_INFO: `/v3/projects/854d5ebaa20a4c399ef086db48777e4e/users/f573d4c25bb04ab2a80836a7c8b10332/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:13.324753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5e6ce-4ccb-4427-8ab3-9a71b0a076c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=854d5ebaa20a4c399ef086db48777e4e, user_id=f573d4c25bb04ab2a80836a7c8b10332, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:13.327445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5e6ce-4ccb-4427-8ab3-9a71b0a076c2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:13.340601 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4239/8389] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:13 2022] PUT /identity/v3/projects/854d5ebaa20a4c399ef086db48777e4e/users/f573d4c25bb04ab2a80836a7c8b10332/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:13.344443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9c3ce-6735-4ed8-8dc8-a28ca292563a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:13.355113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9c3ce-6735-4ed8-8dc8-a28ca292563a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:13.412440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725307f-9007-4ad0-a7e9-05121313a1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:13.413087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725307f-9007-4ad0-a7e9-05121313a1ad admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:13.413376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725307f-9007-4ad0-a7e9-05121313a1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:13.413658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725307f-9007-4ad0-a7e9-05121313a1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:13.414305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725307f-9007-4ad0-a7e9-05121313a1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:13.415900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725307f-9007-4ad0-a7e9-05121313a1ad admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:13.425646 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4151/8390] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:13.429737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5e6ce-4ccb-4427-8ab3-9a71b0a076c2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:13.440135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5e6ce-4ccb-4427-8ab3-9a71b0a076c2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:13.444790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc213255-0c37-41f7-8ec3-2c3d92203e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:13.445479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc213255-0c37-41f7-8ec3-2c3d92203e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:13.445766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc213255-0c37-41f7-8ec3-2c3d92203e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:13.446046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc213255-0c37-41f7-8ec3-2c3d92203e0d admin admin] PATH_INFO: `/v3/projects/854d5ebaa20a4c399ef086db48777e4e/users/f573d4c25bb04ab2a80836a7c8b10332/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:13.447965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc213255-0c37-41f7-8ec3-2c3d92203e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=854d5ebaa20a4c399ef086db48777e4e, user_id=f573d4c25bb04ab2a80836a7c8b10332, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:13.450697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc213255-0c37-41f7-8ec3-2c3d92203e0d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:13.461172 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4240/8391] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:13 2022] PUT /identity/v3/projects/854d5ebaa20a4c399ef086db48777e4e/users/f573d4c25bb04ab2a80836a7c8b10332/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:13.470788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b8470-16d9-4312-9f3f-93b685f71a5e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:13.471080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b8470-16d9-4312-9f3f-93b685f71a5e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:13.471357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b8470-16d9-4312-9f3f-93b685f71a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:13.499584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-617b8470-16d9-4312-9f3f-93b685f71a5e None None] MFA Rules not processed for user `f573d4c25bb04ab2a80836a7c8b10332`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:13.500785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-617b8470-16d9-4312-9f3f-93b685f71a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:13.543409 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:13.570127 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4152/8392] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:13.582608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc213255-0c37-41f7-8ec3-2c3d92203e0d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:13.584323 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bc213255-0c37-41f7-8ec3-2c3d92203e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:13.644683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc213255-0c37-41f7-8ec3-2c3d92203e0d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:13.648396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5fd94-6a73-4ccb-bf5a-cb19d0830c5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:13.649098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5fd94-6a73-4ccb-bf5a-cb19d0830c5f service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:13.649387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5fd94-6a73-4ccb-bf5a-cb19d0830c5f service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:13.649691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5fd94-6a73-4ccb-bf5a-cb19d0830c5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:13.658614 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5fd94-6a73-4ccb-bf5a-cb19d0830c5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:13.660044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5fd94-6a73-4ccb-bf5a-cb19d0830c5f service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:13.669438 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4241/8393] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:13.744094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b8470-16d9-4312-9f3f-93b685f71a5e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:13.757072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b8470-16d9-4312-9f3f-93b685f71a5e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:13.760730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbef11c-0ff1-4780-9987-039fa7c8a269 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:13.761369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbef11c-0ff1-4780-9987-039fa7c8a269 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:13.761657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbef11c-0ff1-4780-9987-039fa7c8a269 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:13.761933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbef11c-0ff1-4780-9987-039fa7c8a269 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:13.818311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbef11c-0ff1-4780-9987-039fa7c8a269 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:13.819751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbef11c-0ff1-4780-9987-039fa7c8a269 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:13.828740 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4153/8394] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:18.263350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5fd94-6a73-4ccb-bf5a-cb19d0830c5f service cinder] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:18.334861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5fd94-6a73-4ccb-bf5a-cb19d0830c5f service cinder] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:18.339342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:18.339968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:18.340259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:18.340524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] PATH_INFO: `/v3/users/58dc43d720a64ca6ba3b812da2864e6c` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:18.341677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58dc43d720a64ca6ba3b812da2864e6c)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:18.343610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:18.380469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:18.395124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58dc43d720a64ca6ba3b812da2864e6c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:18.411002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58dc43d720a64ca6ba3b812da2864e6c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:18.417971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58dc43d720a64ca6ba3b812da2864e6c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:18.418689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58dc43d720a64ca6ba3b812da2864e6c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:18.419306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58dc43d720a64ca6ba3b812da2864e6c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:18.431267 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4242/8395] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:21:18 2022] DELETE /identity/v3/users/58dc43d720a64ca6ba3b812da2864e6c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:18.435651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d976a-bc11-4c5d-b213-7414578ad7ef None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:18.436158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d976a-bc11-4c5d-b213-7414578ad7ef None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:18.436279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d976a-bc11-4c5d-b213-7414578ad7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:18.457304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d85d976a-bc11-4c5d-b213-7414578ad7ef None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:18.458512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d85d976a-bc11-4c5d-b213-7414578ad7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:18.517939 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4154/8396] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:18.527905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:18.529420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:18.589431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5dd28-eee0-4957-b710-2c139c3d1436 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:18.593134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b33d23-213d-4cd0-af49-0acd759abb61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:18.593743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b33d23-213d-4cd0-af49-0acd759abb61 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:18.594029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b33d23-213d-4cd0-af49-0acd759abb61 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:18.594307 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b33d23-213d-4cd0-af49-0acd759abb61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:18.603311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b33d23-213d-4cd0-af49-0acd759abb61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:18.604753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b33d23-213d-4cd0-af49-0acd759abb61 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:18.615009 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4243/8397] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:18.888779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d976a-bc11-4c5d-b213-7414578ad7ef None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:18.890497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d85d976a-bc11-4c5d-b213-7414578ad7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:18.903798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d976a-bc11-4c5d-b213-7414578ad7ef None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:18.908997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:18.911106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:18.911493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:18.911871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] PATH_INFO: `/v3/projects/451afaf83aed42ac814d855651a766ed` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:18.913177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=451afaf83aed42ac814d855651a766ed)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:18.915210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:18.969861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '451afaf83aed42ac814d855651a766ed', 'request_id': 'req-25185990-4e2f-409c-a0c7-db763279cc06'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:18.979232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '451afaf83aed42ac814d855651a766ed', 'request_id': 'req-25185990-4e2f-409c-a0c7-db763279cc06'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:18.986069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '451afaf83aed42ac814d855651a766ed', 'request_id': 'req-25185990-4e2f-409c-a0c7-db763279cc06'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:18.995701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:19.000869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] The token cache is being invalidate because project 451afaf83aed42ac814d855651a766ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:21:19.001456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:19.002639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:19.008186 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4155/8398] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:21:18 2022] DELETE /identity/v3/projects/451afaf83aed42ac814d855651a766ed => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:19.008913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b33d23-213d-4cd0-af49-0acd759abb61 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:19.019504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b33d23-213d-4cd0-af49-0acd759abb61 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:19.079498 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3d8e1-4b2d-4be6-952b-396df7b30519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:19.080398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3d8e1-4b2d-4be6-952b-396df7b30519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:19.080692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3d8e1-4b2d-4be6-952b-396df7b30519 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:19.081006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3d8e1-4b2d-4be6-952b-396df7b30519 admin admin] PATH_INFO: `/v3/users/f573d4c25bb04ab2a80836a7c8b10332` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:19.082209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3d8e1-4b2d-4be6-952b-396df7b30519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f573d4c25bb04ab2a80836a7c8b10332)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:19.084162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3d8e1-4b2d-4be6-952b-396df7b30519 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:19.148184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3b3d8e1-4b2d-4be6-952b-396df7b30519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f573d4c25bb04ab2a80836a7c8b10332'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:19.154128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:19.163854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3b3d8e1-4b2d-4be6-952b-396df7b30519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f573d4c25bb04ab2a80836a7c8b10332'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:19.174481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3b3d8e1-4b2d-4be6-952b-396df7b30519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f573d4c25bb04ab2a80836a7c8b10332'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:19.175732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3b3d8e1-4b2d-4be6-952b-396df7b30519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f573d4c25bb04ab2a80836a7c8b10332'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:19.176648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-b3b3d8e1-4b2d-4be6-952b-396df7b30519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f573d4c25bb04ab2a80836a7c8b10332'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:19.201103 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4244/8399] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:21:19 2022] DELETE /identity/v3/users/f573d4c25bb04ab2a80836a7c8b10332 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:19.670893 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:19.672349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:19.685462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25185990-4e2f-409c-a0c7-db763279cc06 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:19.753038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22762223-16df-46e4-b8a1-947dfa333df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:19.754127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22762223-16df-46e4-b8a1-947dfa333df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:19.754415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22762223-16df-46e4-b8a1-947dfa333df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:19.754694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22762223-16df-46e4-b8a1-947dfa333df3 admin admin] PATH_INFO: `/v3/projects/854d5ebaa20a4c399ef086db48777e4e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:19.755840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22762223-16df-46e4-b8a1-947dfa333df3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=854d5ebaa20a4c399ef086db48777e4e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:19.758416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22762223-16df-46e4-b8a1-947dfa333df3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:19.806053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-22762223-16df-46e4-b8a1-947dfa333df3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854d5ebaa20a4c399ef086db48777e4e', 'request_id': 'req-22762223-16df-46e4-b8a1-947dfa333df3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:19.813231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-22762223-16df-46e4-b8a1-947dfa333df3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854d5ebaa20a4c399ef086db48777e4e', 'request_id': 'req-22762223-16df-46e4-b8a1-947dfa333df3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:19.820882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-22762223-16df-46e4-b8a1-947dfa333df3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854d5ebaa20a4c399ef086db48777e4e', 'request_id': 'req-22762223-16df-46e4-b8a1-947dfa333df3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:19.831391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-22762223-16df-46e4-b8a1-947dfa333df3 admin admin] The token cache is being invalidate because project 854d5ebaa20a4c399ef086db48777e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:21:19.832187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-22762223-16df-46e4-b8a1-947dfa333df3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:19.832850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-22762223-16df-46e4-b8a1-947dfa333df3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:19.834398 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4156/8400] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:21:19 2022] DELETE /identity/v3/projects/854d5ebaa20a4c399ef086db48777e4e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:19.841154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c6cdf-6050-49cf-91e5-707f336a7118 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:19.841672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c6cdf-6050-49cf-91e5-707f336a7118 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:19.841790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c6cdf-6050-49cf-91e5-707f336a7118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:19.861572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9e6c6cdf-6050-49cf-91e5-707f336a7118 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:19.862835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9e6c6cdf-6050-49cf-91e5-707f336a7118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:19.924665 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4245/8401] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:21:19.932850 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf116ba-d816-41bc-b2a5-9f2591daebe1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:19.933187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf116ba-d816-41bc-b2a5-9f2591daebe1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:19.933466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf116ba-d816-41bc-b2a5-9f2591daebe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:19.954918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1bf116ba-d816-41bc-b2a5-9f2591daebe1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:19.956137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1bf116ba-d816-41bc-b2a5-9f2591daebe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:19.965237 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4157/8402] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:19.969685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c6cdf-6050-49cf-91e5-707f336a7118 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:19.980434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c6cdf-6050-49cf-91e5-707f336a7118 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:19.987133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9713cc-c36b-4f21-b062-22c7cc30a85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:19.988438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9713cc-c36b-4f21-b062-22c7cc30a85b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:19.988729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9713cc-c36b-4f21-b062-22c7cc30a85b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:19.989046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9713cc-c36b-4f21-b062-22c7cc30a85b admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:19.989708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9713cc-c36b-4f21-b062-22c7cc30a85b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:21:19.990015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9713cc-c36b-4f21-b062-22c7cc30a85b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:19.991626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9713cc-c36b-4f21-b062-22c7cc30a85b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:20.005115 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4246/8403] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:21:20.009465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf116ba-d816-41bc-b2a5-9f2591daebe1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:20.019355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf116ba-d816-41bc-b2a5-9f2591daebe1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:20.024142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bf5d6-b06a-4286-8635-e2ac0d7a3cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:20.024748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bf5d6-b06a-4286-8635-e2ac0d7a3cfe admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.025079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bf5d6-b06a-4286-8635-e2ac0d7a3cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.025541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bf5d6-b06a-4286-8635-e2ac0d7a3cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.026177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bf5d6-b06a-4286-8635-e2ac0d7a3cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:20.028082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bf5d6-b06a-4286-8635-e2ac0d7a3cfe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:20.058974 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4158/8404] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:20 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:21:20.060240 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9713cc-c36b-4f21-b062-22c7cc30a85b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:20.071302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9713cc-c36b-4f21-b062-22c7cc30a85b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:20.132931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad6920-f084-4090-8aa3-1837eb193e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:20.133877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad6920-f084-4090-8aa3-1837eb193e24 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.134014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad6920-f084-4090-8aa3-1837eb193e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.134215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad6920-f084-4090-8aa3-1837eb193e24 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.134878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad6920-f084-4090-8aa3-1837eb193e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:20.136825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad6920-f084-4090-8aa3-1837eb193e24 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:20.160942 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4247/8405] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:20 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:21:20.164064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bf5d6-b06a-4286-8635-e2ac0d7a3cfe admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:20.174745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bf5d6-b06a-4286-8635-e2ac0d7a3cfe admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:20.179973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd07cf5-5021-4adb-9061-09351301da83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:20.180583 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd07cf5-5021-4adb-9061-09351301da83 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.180869 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd07cf5-5021-4adb-9061-09351301da83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.181419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd07cf5-5021-4adb-9061-09351301da83 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.182068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd07cf5-5021-4adb-9061-09351301da83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:20.183775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd07cf5-5021-4adb-9061-09351301da83 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:20.196929 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4159/8406] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:20.198847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad6920-f084-4090-8aa3-1837eb193e24 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:20.209281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad6920-f084-4090-8aa3-1837eb193e24 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:20.214312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00df1d-4293-4fbc-8f0c-e971d4c95b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:20.214917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00df1d-4293-4fbc-8f0c-e971d4c95b80 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.215194 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00df1d-4293-4fbc-8f0c-e971d4c95b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.215429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00df1d-4293-4fbc-8f0c-e971d4c95b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.216092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00df1d-4293-4fbc-8f0c-e971d4c95b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:20.217737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00df1d-4293-4fbc-8f0c-e971d4c95b80 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:20.229140 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4248/8407] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:21:20.233628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd07cf5-5021-4adb-9061-09351301da83 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:20.244811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd07cf5-5021-4adb-9061-09351301da83 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:20.249537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3d5b8-9cd2-471a-9f52-73280891ab12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:20.250221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3d5b8-9cd2-471a-9f52-73280891ab12 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.250489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3d5b8-9cd2-471a-9f52-73280891ab12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.250767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3d5b8-9cd2-471a-9f52-73280891ab12 admin admin] PATH_INFO: `/v3/projects/0747e48fbdbb4b3480c217e651153b3c/users/a6144d802d9b4e5cbc0f72b7ec281506/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.266488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3d5b8-9cd2-471a-9f52-73280891ab12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0747e48fbdbb4b3480c217e651153b3c, user_id=a6144d802d9b4e5cbc0f72b7ec281506, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:20.269139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3d5b8-9cd2-471a-9f52-73280891ab12 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:20.283038 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4160/8408] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:20 2022] PUT /identity/v3/projects/0747e48fbdbb4b3480c217e651153b3c/users/a6144d802d9b4e5cbc0f72b7ec281506/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:20.294312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674df28-4f64-435b-bff8-b06c7ba9e055 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.294602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674df28-4f64-435b-bff8-b06c7ba9e055 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.294878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674df28-4f64-435b-bff8-b06c7ba9e055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.324098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0674df28-4f64-435b-bff8-b06c7ba9e055 None None] MFA Rules not processed for user `a6144d802d9b4e5cbc0f72b7ec281506`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:20.325381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0674df28-4f64-435b-bff8-b06c7ba9e055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:20.352508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:20.397013 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4249/8409] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:20.401658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3d5b8-9cd2-471a-9f52-73280891ab12 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:20.432871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:20.467891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3d5b8-9cd2-471a-9f52-73280891ab12 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:20.473496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d9620-0692-4957-bb1a-b3bd96478e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:20.474109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d9620-0692-4957-bb1a-b3bd96478e68 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.474394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d9620-0692-4957-bb1a-b3bd96478e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.474669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d9620-0692-4957-bb1a-b3bd96478e68 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.475301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d9620-0692-4957-bb1a-b3bd96478e68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:20.477234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d9620-0692-4957-bb1a-b3bd96478e68 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:20.504830 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4161/8410] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:20 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:20.509880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674df28-4f64-435b-bff8-b06c7ba9e055 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:20.522026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674df28-4f64-435b-bff8-b06c7ba9e055 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:20.580527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e062c1d-155d-419a-bb21-959e73073837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:20.581183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e062c1d-155d-419a-bb21-959e73073837 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.581505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e062c1d-155d-419a-bb21-959e73073837 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.581784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e062c1d-155d-419a-bb21-959e73073837 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.582449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e062c1d-155d-419a-bb21-959e73073837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:20.584452 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e062c1d-155d-419a-bb21-959e73073837 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:20.608975 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4250/8411] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:20 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:20.613586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d9620-0692-4957-bb1a-b3bd96478e68 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:20.624506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d9620-0692-4957-bb1a-b3bd96478e68 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:20.629273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc2695-914f-435b-951f-7176a2d70b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:20.629883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc2695-914f-435b-951f-7176a2d70b9c admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.630169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc2695-914f-435b-951f-7176a2d70b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.630446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc2695-914f-435b-951f-7176a2d70b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.631090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc2695-914f-435b-951f-7176a2d70b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:20.632689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc2695-914f-435b-951f-7176a2d70b9c admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:20.643756 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4162/8412] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:20.647643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e062c1d-155d-419a-bb21-959e73073837 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:20.657839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e062c1d-155d-419a-bb21-959e73073837 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:20.663228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90b3ea-1a71-481e-b34f-01866e5325f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:20.664339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90b3ea-1a71-481e-b34f-01866e5325f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.664339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90b3ea-1a71-481e-b34f-01866e5325f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.664547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90b3ea-1a71-481e-b34f-01866e5325f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.665113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90b3ea-1a71-481e-b34f-01866e5325f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:20.667165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90b3ea-1a71-481e-b34f-01866e5325f8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:20.682426 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4251/8413] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:20.683864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc2695-914f-435b-951f-7176a2d70b9c admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:20.694108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc2695-914f-435b-951f-7176a2d70b9c admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:20.698807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4d93f-7d10-40c5-ace8-4bc24e82dfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:20.699475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4d93f-7d10-40c5-ace8-4bc24e82dfb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.699759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4d93f-7d10-40c5-ace8-4bc24e82dfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.700038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4d93f-7d10-40c5-ace8-4bc24e82dfb6 admin admin] PATH_INFO: `/v3/projects/de71c5b08813446eb9ba5fa5fe6a7938/users/d84f4574b6184467840dced0d5d21989/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.717768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4d93f-7d10-40c5-ace8-4bc24e82dfb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de71c5b08813446eb9ba5fa5fe6a7938, user_id=d84f4574b6184467840dced0d5d21989, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:20.720532 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4d93f-7d10-40c5-ace8-4bc24e82dfb6 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:20.736737 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4163/8414] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:20 2022] PUT /identity/v3/projects/de71c5b08813446eb9ba5fa5fe6a7938/users/d84f4574b6184467840dced0d5d21989/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:20.747438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71216a55-96fe-4d4c-92fb-a9a06e6eb490 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.747769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71216a55-96fe-4d4c-92fb-a9a06e6eb490 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.748009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71216a55-96fe-4d4c-92fb-a9a06e6eb490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.784026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-71216a55-96fe-4d4c-92fb-a9a06e6eb490 None None] MFA Rules not processed for user `d84f4574b6184467840dced0d5d21989`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:20.785871 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-71216a55-96fe-4d4c-92fb-a9a06e6eb490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:20.860372 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4252/8415] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:20.861593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4d93f-7d10-40c5-ace8-4bc24e82dfb6 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:20.929122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4d93f-7d10-40c5-ace8-4bc24e82dfb6 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:20.934350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc0c85-4b13-46c2-a6ca-3b2d24e1bd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:20.935000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc0c85-4b13-46c2-a6ca-3b2d24e1bd22 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:20.935290 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc0c85-4b13-46c2-a6ca-3b2d24e1bd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:20.935571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc0c85-4b13-46c2-a6ca-3b2d24e1bd22 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:20.936339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc0c85-4b13-46c2-a6ca-3b2d24e1bd22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:20.938311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc0c85-4b13-46c2-a6ca-3b2d24e1bd22 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:20.976644 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4164/8416] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:20 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:21:20.977787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71216a55-96fe-4d4c-92fb-a9a06e6eb490 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:20.991015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71216a55-96fe-4d4c-92fb-a9a06e6eb490 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:21.047647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:21.081180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa5e82-5b5a-4f4b-a690-cee6aaf91496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:21.082536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa5e82-5b5a-4f4b-a690-cee6aaf91496 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:21.082840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa5e82-5b5a-4f4b-a690-cee6aaf91496 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:21.083173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa5e82-5b5a-4f4b-a690-cee6aaf91496 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:21.083847 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa5e82-5b5a-4f4b-a690-cee6aaf91496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:21.085836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa5e82-5b5a-4f4b-a690-cee6aaf91496 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:21.118456 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4253/8417] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:20 2022] POST /identity/v3/users => generated 295 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:21.124127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc0c85-4b13-46c2-a6ca-3b2d24e1bd22 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:21.143874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc0c85-4b13-46c2-a6ca-3b2d24e1bd22 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:21.150070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c15c0-55a3-42d6-912a-430645c51300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:21.150822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c15c0-55a3-42d6-912a-430645c51300 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:21.151217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c15c0-55a3-42d6-912a-430645c51300 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:21.151661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c15c0-55a3-42d6-912a-430645c51300 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:21.152426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c15c0-55a3-42d6-912a-430645c51300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:21.154204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c15c0-55a3-42d6-912a-430645c51300 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:21.166653 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4165/8418] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:21.170931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa5e82-5b5a-4f4b-a690-cee6aaf91496 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:21.184467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa5e82-5b5a-4f4b-a690-cee6aaf91496 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:21.190337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437a757-0a40-41f8-9f89-fb1f0d7ccede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:21.190337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437a757-0a40-41f8-9f89-fb1f0d7ccede admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:21.191387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437a757-0a40-41f8-9f89-fb1f0d7ccede admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:21.191772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437a757-0a40-41f8-9f89-fb1f0d7ccede admin admin] PATH_INFO: `/v3/projects/23c6fbad100f4c988526fc4eb753ec02/users/53e0556021f94f348098183bf58fba5e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:21.210488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437a757-0a40-41f8-9f89-fb1f0d7ccede admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c6fbad100f4c988526fc4eb753ec02, user_id=53e0556021f94f348098183bf58fba5e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:21.213579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437a757-0a40-41f8-9f89-fb1f0d7ccede admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:21.229285 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4254/8419] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:21 2022] PUT /identity/v3/projects/23c6fbad100f4c988526fc4eb753ec02/users/53e0556021f94f348098183bf58fba5e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:21.233689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c15c0-55a3-42d6-912a-430645c51300 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:21.244940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c15c0-55a3-42d6-912a-430645c51300 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:21.451493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:21.466214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62044c-9524-45c2-813e-8e3511b1e240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:21.466830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62044c-9524-45c2-813e-8e3511b1e240 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:21.467112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62044c-9524-45c2-813e-8e3511b1e240 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:21.467383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62044c-9524-45c2-813e-8e3511b1e240 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:21.468114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62044c-9524-45c2-813e-8e3511b1e240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:21.469767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62044c-9524-45c2-813e-8e3511b1e240 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:21.485193 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4166/8420] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:21.486685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437a757-0a40-41f8-9f89-fb1f0d7ccede admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:21.498504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437a757-0a40-41f8-9f89-fb1f0d7ccede admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:21.503885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596adfb9-387a-4d2c-b809-0d7a92122141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:21.504964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596adfb9-387a-4d2c-b809-0d7a92122141 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:21.504964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596adfb9-387a-4d2c-b809-0d7a92122141 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:21.505196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596adfb9-387a-4d2c-b809-0d7a92122141 admin admin] PATH_INFO: `/v3/projects/23c6fbad100f4c988526fc4eb753ec02/users/53e0556021f94f348098183bf58fba5e/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:21.507209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596adfb9-387a-4d2c-b809-0d7a92122141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c6fbad100f4c988526fc4eb753ec02, user_id=53e0556021f94f348098183bf58fba5e, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:21.510004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596adfb9-387a-4d2c-b809-0d7a92122141 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:21.527889 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4255/8421] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:21 2022] PUT /identity/v3/projects/23c6fbad100f4c988526fc4eb753ec02/users/53e0556021f94f348098183bf58fba5e/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:21:21.540804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9b148-1a85-4960-950d-5c4b886bd4a0 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:21.541258 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9b148-1a85-4960-950d-5c4b886bd4a0 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:21.542051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9b148-1a85-4960-950d-5c4b886bd4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:21.578451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-28e9b148-1a85-4960-950d-5c4b886bd4a0 None None] MFA Rules not processed for user `53e0556021f94f348098183bf58fba5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:21.580324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-28e9b148-1a85-4960-950d-5c4b886bd4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:21.687246 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4167/8422] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:21.700103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596adfb9-387a-4d2c-b809-0d7a92122141 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:21.701959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-596adfb9-387a-4d2c-b809-0d7a92122141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:21.776402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596adfb9-387a-4d2c-b809-0d7a92122141 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:21.782046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed7f81-0dd4-471d-9935-73886f5c7c46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:21.782046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed7f81-0dd4-471d-9935-73886f5c7c46 service cinder] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:21.782563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed7f81-0dd4-471d-9935-73886f5c7c46 service cinder] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:21.782936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed7f81-0dd4-471d-9935-73886f5c7c46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:21.793666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed7f81-0dd4-471d-9935-73886f5c7c46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:21.795465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed7f81-0dd4-471d-9935-73886f5c7c46 service cinder] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:21.805877 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4256/8423] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:21.869492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e9b148-1a85-4960-950d-5c4b886bd4a0 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:21.870995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-28e9b148-1a85-4960-950d-5c4b886bd4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:21.888825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e9b148-1a85-4960-950d-5c4b886bd4a0 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:21.898493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9abab-2565-4bc5-9a64-479e1d9a0f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:21.899125 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9abab-2565-4bc5-9a64-479e1d9a0f4a service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:21.899571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9abab-2565-4bc5-9a64-479e1d9a0f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:21.899862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9abab-2565-4bc5-9a64-479e1d9a0f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:21.975012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9abab-2565-4bc5-9a64-479e1d9a0f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:21.976496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9abab-2565-4bc5-9a64-479e1d9a0f4a service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:21.989945 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4168/8424] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:24.092399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3f67d-1971-4bee-a56b-85ecb5bf0533 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:24.092963 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3f67d-1971-4bee-a56b-85ecb5bf0533 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:24.093101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3f67d-1971-4bee-a56b-85ecb5bf0533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:24.120029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-33d3f67d-1971-4bee-a56b-85ecb5bf0533 None None] MFA Rules not processed for user `7a40d55acc0d452dae9ffadfa35fae88`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:24.122003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-33d3f67d-1971-4bee-a56b-85ecb5bf0533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:24.199508 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4257/8425] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:24.204248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9abab-2565-4bc5-9a64-479e1d9a0f4a service cinder] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:24.298759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9abab-2565-4bc5-9a64-479e1d9a0f4a service cinder] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:24.303393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:24.304035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:24.304316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:24.304592 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] PATH_INFO: `/v3/users/a6144d802d9b4e5cbc0f72b7ec281506` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:24.305797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6144d802d9b4e5cbc0f72b7ec281506)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:24.308061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:24.322811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:24.372644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6144d802d9b4e5cbc0f72b7ec281506'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:24.394769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6144d802d9b4e5cbc0f72b7ec281506'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:24.395808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6144d802d9b4e5cbc0f72b7ec281506'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:24.403891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6144d802d9b4e5cbc0f72b7ec281506'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:24.413543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6144d802d9b4e5cbc0f72b7ec281506'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:24.420604 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4169/8426] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:21:24 2022] DELETE /identity/v3/users/a6144d802d9b4e5cbc0f72b7ec281506 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:24.425704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e0aad-5662-4aff-a6e0-892b6a2711b7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:24.426209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e0aad-5662-4aff-a6e0-892b6a2711b7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:24.426651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e0aad-5662-4aff-a6e0-892b6a2711b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:24.447252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6b3e0aad-5662-4aff-a6e0-892b6a2711b7 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:24.448792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6b3e0aad-5662-4aff-a6e0-892b6a2711b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:24.498584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:24.534294 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4258/8427] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:24.544687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:24.546185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:24.618076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d7fff-ec36-4b2e-be16-94b931522d3b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:24.621842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab4a18-4469-488a-bb17-2b14d3c53b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:24.622493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab4a18-4469-488a-bb17-2b14d3c53b07 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:24.622944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab4a18-4469-488a-bb17-2b14d3c53b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:24.623229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab4a18-4469-488a-bb17-2b14d3c53b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:24.633859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab4a18-4469-488a-bb17-2b14d3c53b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:24.635419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab4a18-4469-488a-bb17-2b14d3c53b07 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:24.646678 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4170/8428] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:24.923976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e0aad-5662-4aff-a6e0-892b6a2711b7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:24.926848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6b3e0aad-5662-4aff-a6e0-892b6a2711b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:25.017282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab4a18-4469-488a-bb17-2b14d3c53b07 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:25.019328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-faab4a18-4469-488a-bb17-2b14d3c53b07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:25.041707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab4a18-4469-488a-bb17-2b14d3c53b07 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:25.048944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:25.049690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:25.050103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:25.050445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] PATH_INFO: `/v3/projects/0747e48fbdbb4b3480c217e651153b3c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:25.051978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0747e48fbdbb4b3480c217e651153b3c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:25.053851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e0aad-5662-4aff-a6e0-892b6a2711b7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:25.054202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:25.067822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d46df-69ea-4fd9-b644-b21cd77d561e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:25.070860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d46df-69ea-4fd9-b644-b21cd77d561e service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:25.073276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d46df-69ea-4fd9-b644-b21cd77d561e service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:25.073754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d46df-69ea-4fd9-b644-b21cd77d561e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:25.078176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-518d46df-69ea-4fd9-b644-b21cd77d561e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:25.109911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d46df-69ea-4fd9-b644-b21cd77d561e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:25.111779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d46df-69ea-4fd9-b644-b21cd77d561e service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:25.150782 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4259/8429] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:25.157618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0747e48fbdbb4b3480c217e651153b3c', 'request_id': 'req-db734546-7af7-4f72-a87e-bd902d1bc46e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.168197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0747e48fbdbb4b3480c217e651153b3c', 'request_id': 'req-db734546-7af7-4f72-a87e-bd902d1bc46e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.178575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0747e48fbdbb4b3480c217e651153b3c', 'request_id': 'req-db734546-7af7-4f72-a87e-bd902d1bc46e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.199358 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] The token cache is being invalidate because project 0747e48fbdbb4b3480c217e651153b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:21:25.199681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.200347 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.203472 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4171/8430] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:21:25 2022] DELETE /identity/v3/projects/0747e48fbdbb4b3480c217e651153b3c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:25.207502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d46df-69ea-4fd9-b644-b21cd77d561e service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:25.226646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d46df-69ea-4fd9-b644-b21cd77d561e service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:25.309439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:25.311198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:25.311722 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:25.312235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] PATH_INFO: `/v3/users/d84f4574b6184467840dced0d5d21989` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:25.313751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84f4574b6184467840dced0d5d21989)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:25.316133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:25.380398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84f4574b6184467840dced0d5d21989'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.397224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84f4574b6184467840dced0d5d21989'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.400394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:25.408003 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84f4574b6184467840dced0d5d21989'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.409200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84f4574b6184467840dced0d5d21989'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.410803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84f4574b6184467840dced0d5d21989'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.431655 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4260/8431] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:21:25 2022] DELETE /identity/v3/users/d84f4574b6184467840dced0d5d21989 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:25.792206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:25.794240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:25.812518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db734546-7af7-4f72-a87e-bd902d1bc46e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:25.828496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:25.885248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:25.886068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:25.886458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:25.886749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] PATH_INFO: `/v3/projects/de71c5b08813446eb9ba5fa5fe6a7938` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:25.888759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de71c5b08813446eb9ba5fa5fe6a7938)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:25.889905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:25.953286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de71c5b08813446eb9ba5fa5fe6a7938', 'request_id': 'req-f73a33a9-f95b-43ec-9146-9d77ac1b0025'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.962332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de71c5b08813446eb9ba5fa5fe6a7938', 'request_id': 'req-f73a33a9-f95b-43ec-9146-9d77ac1b0025'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.969923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de71c5b08813446eb9ba5fa5fe6a7938', 'request_id': 'req-f73a33a9-f95b-43ec-9146-9d77ac1b0025'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.985063 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] The token cache is being invalidate because project de71c5b08813446eb9ba5fa5fe6a7938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:21:25.985545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.986554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:25.989621 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4172/8432] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:21:25 2022] DELETE /identity/v3/projects/de71c5b08813446eb9ba5fa5fe6a7938 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:25.993727 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:25.995186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:26.010505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abb381-8cab-4c94-8dde-27507e03834e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:26.090061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88ba42-a40f-4e3f-8bad-cf602282e352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:26.090707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88ba42-a40f-4e3f-8bad-cf602282e352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:26.091223 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88ba42-a40f-4e3f-8bad-cf602282e352 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:26.091400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88ba42-a40f-4e3f-8bad-cf602282e352 admin admin] PATH_INFO: `/v3/users/53e0556021f94f348098183bf58fba5e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:26.092533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88ba42-a40f-4e3f-8bad-cf602282e352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e0556021f94f348098183bf58fba5e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:26.094894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88ba42-a40f-4e3f-8bad-cf602282e352 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:26.157295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f88ba42-a40f-4e3f-8bad-cf602282e352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e0556021f94f348098183bf58fba5e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:26.171061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f88ba42-a40f-4e3f-8bad-cf602282e352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e0556021f94f348098183bf58fba5e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:26.181917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f88ba42-a40f-4e3f-8bad-cf602282e352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e0556021f94f348098183bf58fba5e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:26.182616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f88ba42-a40f-4e3f-8bad-cf602282e352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e0556021f94f348098183bf58fba5e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:26.183230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2f88ba42-a40f-4e3f-8bad-cf602282e352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e0556021f94f348098183bf58fba5e'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:26.198058 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4261/8433] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:21:25 2022] DELETE /identity/v3/users/53e0556021f94f348098183bf58fba5e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:26.511898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:26.514106 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:26.540282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a33a9-f95b-43ec-9146-9d77ac1b0025 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:26.622790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d7750-2d8a-422b-a62b-49979b4a19ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:26.623453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d7750-2d8a-422b-a62b-49979b4a19ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:26.623740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d7750-2d8a-422b-a62b-49979b4a19ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:26.624127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d7750-2d8a-422b-a62b-49979b4a19ae admin admin] PATH_INFO: `/v3/projects/23c6fbad100f4c988526fc4eb753ec02` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:26.625930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d7750-2d8a-422b-a62b-49979b4a19ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c6fbad100f4c988526fc4eb753ec02)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:26.629100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d7750-2d8a-422b-a62b-49979b4a19ae admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:26.695331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:26.697219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e0d7750-2d8a-422b-a62b-49979b4a19ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c6fbad100f4c988526fc4eb753ec02', 'request_id': 'req-3e0d7750-2d8a-422b-a62b-49979b4a19ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:26.706771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e0d7750-2d8a-422b-a62b-49979b4a19ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c6fbad100f4c988526fc4eb753ec02', 'request_id': 'req-3e0d7750-2d8a-422b-a62b-49979b4a19ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:26.714781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e0d7750-2d8a-422b-a62b-49979b4a19ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c6fbad100f4c988526fc4eb753ec02', 'request_id': 'req-3e0d7750-2d8a-422b-a62b-49979b4a19ae'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:26.727670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e0d7750-2d8a-422b-a62b-49979b4a19ae admin admin] The token cache is being invalidate because project 23c6fbad100f4c988526fc4eb753ec02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:21:26.728037 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e0d7750-2d8a-422b-a62b-49979b4a19ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:26.729373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-3e0d7750-2d8a-422b-a62b-49979b4a19ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:26.731259 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4173/8434] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:21:26 2022] DELETE /identity/v3/projects/23c6fbad100f4c988526fc4eb753ec02 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:26.737413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3cc47-6f19-474e-96ae-3ac542676c51 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:26.738300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3cc47-6f19-474e-96ae-3ac542676c51 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:26.738585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3cc47-6f19-474e-96ae-3ac542676c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:26.762591 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a1e3cc47-6f19-474e-96ae-3ac542676c51 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:26.763917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a1e3cc47-6f19-474e-96ae-3ac542676c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:26.875339 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4262/8435] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:21:26.883941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f42aa-9147-4e74-b5e4-bfb3dd38b06b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:26.884473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f42aa-9147-4e74-b5e4-bfb3dd38b06b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:26.884576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f42aa-9147-4e74-b5e4-bfb3dd38b06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:26.915276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8e7f42aa-9147-4e74-b5e4-bfb3dd38b06b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:26.916953 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8e7f42aa-9147-4e74-b5e4-bfb3dd38b06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:26.926396 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4174/8436] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:26.930735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3cc47-6f19-474e-96ae-3ac542676c51 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:26.937116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:26.947640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3cc47-6f19-474e-96ae-3ac542676c51 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:26.954442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e4aa0-a8af-4698-9e64-a81f0996dea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:26.955177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4aa0-a8af-4698-9e64-a81f0996dea3 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:26.956092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4aa0-a8af-4698-9e64-a81f0996dea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:26.956472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4aa0-a8af-4698-9e64-a81f0996dea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:26.957953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e4aa0-a8af-4698-9e64-a81f0996dea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:21:26.958361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e4aa0-a8af-4698-9e64-a81f0996dea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:26.960155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e4aa0-a8af-4698-9e64-a81f0996dea3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:26.974755 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4263/8437] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:21:26.978956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f42aa-9147-4e74-b5e4-bfb3dd38b06b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:26.989788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f42aa-9147-4e74-b5e4-bfb3dd38b06b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:26.997709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a69a23-2515-494b-aaf3-e052345afb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:26.998348 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a69a23-2515-494b-aaf3-e052345afb64 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:26.998638 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a69a23-2515-494b-aaf3-e052345afb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:26.998912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a69a23-2515-494b-aaf3-e052345afb64 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:26.999561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a69a23-2515-494b-aaf3-e052345afb64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:27.001673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a69a23-2515-494b-aaf3-e052345afb64 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:27.044954 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4175/8438] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:26 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:27.047921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e4aa0-a8af-4698-9e64-a81f0996dea3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:27.063398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e4aa0-a8af-4698-9e64-a81f0996dea3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:27.149228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364bdf5-5754-4750-83c9-e98c4e7bb5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:27.149998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364bdf5-5754-4750-83c9-e98c4e7bb5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:27.150514 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364bdf5-5754-4750-83c9-e98c4e7bb5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:27.151045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364bdf5-5754-4750-83c9-e98c4e7bb5b1 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:27.151807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364bdf5-5754-4750-83c9-e98c4e7bb5b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:27.153891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364bdf5-5754-4750-83c9-e98c4e7bb5b1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:27.198401 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4264/8439] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:27 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:27.199759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a69a23-2515-494b-aaf3-e052345afb64 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:27.223611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a69a23-2515-494b-aaf3-e052345afb64 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:27.231321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea246b-755d-4ac1-b362-d32bceb039fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:27.232741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea246b-755d-4ac1-b362-d32bceb039fa admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:27.233131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea246b-755d-4ac1-b362-d32bceb039fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:27.233508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea246b-755d-4ac1-b362-d32bceb039fa admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:27.234281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea246b-755d-4ac1-b362-d32bceb039fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:27.253533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea246b-755d-4ac1-b362-d32bceb039fa admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:27.271078 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4176/8440] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:27.271887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364bdf5-5754-4750-83c9-e98c4e7bb5b1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:27.282930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364bdf5-5754-4750-83c9-e98c4e7bb5b1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:27.289653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911f177-5172-4649-9e9f-7c92d2d75983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:27.290538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911f177-5172-4649-9e9f-7c92d2d75983 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:27.291022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911f177-5172-4649-9e9f-7c92d2d75983 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:27.291511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911f177-5172-4649-9e9f-7c92d2d75983 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:27.292350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911f177-5172-4649-9e9f-7c92d2d75983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:27.294960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911f177-5172-4649-9e9f-7c92d2d75983 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:27.308413 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4265/8441] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:27.312461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea246b-755d-4ac1-b362-d32bceb039fa admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:27.325795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea246b-755d-4ac1-b362-d32bceb039fa admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:27.337645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284aa5a5-4477-48a1-8f2d-4cdb9abed3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:27.338255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284aa5a5-4477-48a1-8f2d-4cdb9abed3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:27.338553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284aa5a5-4477-48a1-8f2d-4cdb9abed3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:27.338920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284aa5a5-4477-48a1-8f2d-4cdb9abed3bf admin admin] PATH_INFO: `/v3/projects/3d87947af5414e8aab9bee8fe74f08d8/users/8671c5dfa28c4dca872b66c011662ff2/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:27.356450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284aa5a5-4477-48a1-8f2d-4cdb9abed3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d87947af5414e8aab9bee8fe74f08d8, user_id=8671c5dfa28c4dca872b66c011662ff2, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:27.359445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284aa5a5-4477-48a1-8f2d-4cdb9abed3bf admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:27.387098 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4177/8442] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:27 2022] PUT /identity/v3/projects/3d87947af5414e8aab9bee8fe74f08d8/users/8671c5dfa28c4dca872b66c011662ff2/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:27.388649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2a8bb-76c9-4bcf-a55b-cb6a4c9d4543 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:27.388998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2a8bb-76c9-4bcf-a55b-cb6a4c9d4543 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:27.389941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2a8bb-76c9-4bcf-a55b-cb6a4c9d4543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:27.431102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fdd2a8bb-76c9-4bcf-a55b-cb6a4c9d4543 None None] MFA Rules not processed for user `8671c5dfa28c4dca872b66c011662ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:27.432416 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fdd2a8bb-76c9-4bcf-a55b-cb6a4c9d4543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:27.522337 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4266/8443] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:21:27.531552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284aa5a5-4477-48a1-8f2d-4cdb9abed3bf admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:27.533396 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-284aa5a5-4477-48a1-8f2d-4cdb9abed3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:27.606883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284aa5a5-4477-48a1-8f2d-4cdb9abed3bf admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:27.610872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c244a1-2c6d-4e29-b39c-0a91afd42bee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:27.611545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c244a1-2c6d-4e29-b39c-0a91afd42bee service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:27.611835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c244a1-2c6d-4e29-b39c-0a91afd42bee service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:27.612257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c244a1-2c6d-4e29-b39c-0a91afd42bee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:27.622385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c244a1-2c6d-4e29-b39c-0a91afd42bee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:27.623984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c244a1-2c6d-4e29-b39c-0a91afd42bee service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:27.635126 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4178/8444] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:21:27.655563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2a8bb-76c9-4bcf-a55b-cb6a4c9d4543 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:27.730537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2a8bb-76c9-4bcf-a55b-cb6a4c9d4543 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:27.735728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:27.736458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:27.736749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:27.737120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] PATH_INFO: `/v3/users/8671c5dfa28c4dca872b66c011662ff2` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:27.739083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8671c5dfa28c4dca872b66c011662ff2)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:27.741077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:27.742910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:27.807290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8671c5dfa28c4dca872b66c011662ff2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:27.830566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8671c5dfa28c4dca872b66c011662ff2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:27.846909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8671c5dfa28c4dca872b66c011662ff2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:27.848014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8671c5dfa28c4dca872b66c011662ff2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:27.848951 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8671c5dfa28c4dca872b66c011662ff2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:27.870053 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4267/8445] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:21:27 2022] DELETE /identity/v3/users/8671c5dfa28c4dca872b66c011662ff2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:27.875107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ada0e-0178-4198-88b4-8838efdd7fbb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:27.875107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ada0e-0178-4198-88b4-8838efdd7fbb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:27.875379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ada0e-0178-4198-88b4-8838efdd7fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:27.913461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-004ada0e-0178-4198-88b4-8838efdd7fbb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:27.914792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-004ada0e-0178-4198-88b4-8838efdd7fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:28.024469 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4179/8446] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:28.035659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:28.037131 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:28.158246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdafa22-984a-4968-80f4-2dc2a445aef6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:28.162583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c5d62-7628-47b0-ba46-a03eb35e1750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:28.163205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c5d62-7628-47b0-ba46-a03eb35e1750 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:28.163493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c5d62-7628-47b0-ba46-a03eb35e1750 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:28.163771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c5d62-7628-47b0-ba46-a03eb35e1750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:28.176464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c5d62-7628-47b0-ba46-a03eb35e1750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:28.182520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c5d62-7628-47b0-ba46-a03eb35e1750 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:28.195701 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4268/8447] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:28.717656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ada0e-0178-4198-88b4-8838efdd7fbb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:28.719604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-004ada0e-0178-4198-88b4-8838efdd7fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:28.735844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ada0e-0178-4198-88b4-8838efdd7fbb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:28.741686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c47dc8-b579-4ff9-a9f9-698202f32ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:28.742372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c47dc8-b579-4ff9-a9f9-698202f32ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:28.742687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c47dc8-b579-4ff9-a9f9-698202f32ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:28.742994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c47dc8-b579-4ff9-a9f9-698202f32ad1 admin admin] PATH_INFO: `/v3/projects/3d87947af5414e8aab9bee8fe74f08d8` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:28.744205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c47dc8-b579-4ff9-a9f9-698202f32ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d87947af5414e8aab9bee8fe74f08d8)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:28.746377 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c47dc8-b579-4ff9-a9f9-698202f32ad1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:28.761392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:28.811611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-76c47dc8-b579-4ff9-a9f9-698202f32ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d87947af5414e8aab9bee8fe74f08d8', 'request_id': 'req-76c47dc8-b579-4ff9-a9f9-698202f32ad1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:28.825059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-76c47dc8-b579-4ff9-a9f9-698202f32ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d87947af5414e8aab9bee8fe74f08d8', 'request_id': 'req-76c47dc8-b579-4ff9-a9f9-698202f32ad1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:28.835111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-76c47dc8-b579-4ff9-a9f9-698202f32ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d87947af5414e8aab9bee8fe74f08d8', 'request_id': 'req-76c47dc8-b579-4ff9-a9f9-698202f32ad1'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:28.847590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-76c47dc8-b579-4ff9-a9f9-698202f32ad1 admin admin] The token cache is being invalidate because project 3d87947af5414e8aab9bee8fe74f08d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:21:28.847871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-76c47dc8-b579-4ff9-a9f9-698202f32ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:28.848539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-76c47dc8-b579-4ff9-a9f9-698202f32ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:28.850082 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4180/8448] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:21:28 2022] DELETE /identity/v3/projects/3d87947af5414e8aab9bee8fe74f08d8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:28.854967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e970eb9-a531-42fa-be0c-bff20c8fdfa9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:28.855465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e970eb9-a531-42fa-be0c-bff20c8fdfa9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:28.855658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e970eb9-a531-42fa-be0c-bff20c8fdfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:28.877083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1e970eb9-a531-42fa-be0c-bff20c8fdfa9 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:28.878922 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1e970eb9-a531-42fa-be0c-bff20c8fdfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:28.939943 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4269/8449] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:21:28.949085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277632db-8858-48b0-8219-31b116400a1d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:28.949571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277632db-8858-48b0-8219-31b116400a1d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:28.949696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277632db-8858-48b0-8219-31b116400a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:28.970084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-277632db-8858-48b0-8219-31b116400a1d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:28.971309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-277632db-8858-48b0-8219-31b116400a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:28.979799 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4181/8450] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:28.984364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e970eb9-a531-42fa-be0c-bff20c8fdfa9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:28.994676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e970eb9-a531-42fa-be0c-bff20c8fdfa9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:28.999288 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b643a74-75e1-40cb-9654-ff8a94fc2be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:28.999891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b643a74-75e1-40cb-9654-ff8a94fc2be9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:29.000177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b643a74-75e1-40cb-9654-ff8a94fc2be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:29.000454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b643a74-75e1-40cb-9654-ff8a94fc2be9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:29.001135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b643a74-75e1-40cb-9654-ff8a94fc2be9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:21:29.001442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b643a74-75e1-40cb-9654-ff8a94fc2be9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:29.003038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b643a74-75e1-40cb-9654-ff8a94fc2be9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:29.015978 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4270/8451] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:21:29.021007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277632db-8858-48b0-8219-31b116400a1d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:29.033404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277632db-8858-48b0-8219-31b116400a1d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:29.042919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b79c36-15e1-452d-b517-dc8cbddd9b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:29.043632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b79c36-15e1-452d-b517-dc8cbddd9b64 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:29.043989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b79c36-15e1-452d-b517-dc8cbddd9b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:29.044279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b79c36-15e1-452d-b517-dc8cbddd9b64 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:29.044951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b79c36-15e1-452d-b517-dc8cbddd9b64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:29.046886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b79c36-15e1-452d-b517-dc8cbddd9b64 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:29.081016 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4182/8452] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:29 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:29.085212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b643a74-75e1-40cb-9654-ff8a94fc2be9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:29.095660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b643a74-75e1-40cb-9654-ff8a94fc2be9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:29.101690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:29.164143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8ee12-b3c7-4de5-bc21-f24187fb4c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:29.164767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8ee12-b3c7-4de5-bc21-f24187fb4c17 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:29.165841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8ee12-b3c7-4de5-bc21-f24187fb4c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:29.166150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8ee12-b3c7-4de5-bc21-f24187fb4c17 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:29.166811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8ee12-b3c7-4de5-bc21-f24187fb4c17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:29.168836 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8ee12-b3c7-4de5-bc21-f24187fb4c17 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:29.196303 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4271/8453] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:29 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:29.200584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b79c36-15e1-452d-b517-dc8cbddd9b64 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:29.211277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b79c36-15e1-452d-b517-dc8cbddd9b64 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:29.216411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc9074-2982-4e83-91bb-320340e5c015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:29.217171 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc9074-2982-4e83-91bb-320340e5c015 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:29.217459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc9074-2982-4e83-91bb-320340e5c015 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:29.217805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc9074-2982-4e83-91bb-320340e5c015 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:29.218387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc9074-2982-4e83-91bb-320340e5c015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:29.220756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc9074-2982-4e83-91bb-320340e5c015 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:29.236553 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4183/8454] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:29.241276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8ee12-b3c7-4de5-bc21-f24187fb4c17 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:29.252575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8ee12-b3c7-4de5-bc21-f24187fb4c17 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:29.258198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a314a22-8f11-4aca-9dec-7e326597ba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:29.258833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a314a22-8f11-4aca-9dec-7e326597ba68 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:29.259170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a314a22-8f11-4aca-9dec-7e326597ba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:29.259478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a314a22-8f11-4aca-9dec-7e326597ba68 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:29.260164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a314a22-8f11-4aca-9dec-7e326597ba68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:29.261976 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a314a22-8f11-4aca-9dec-7e326597ba68 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:29.276129 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4272/8455] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:21:29.277334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc9074-2982-4e83-91bb-320340e5c015 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:29.287529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc9074-2982-4e83-91bb-320340e5c015 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:29.292368 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f041d-3208-4629-a431-0a97219ff271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:29.293057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f041d-3208-4629-a431-0a97219ff271 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:29.293340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f041d-3208-4629-a431-0a97219ff271 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:29.293620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f041d-3208-4629-a431-0a97219ff271 admin admin] PATH_INFO: `/v3/projects/cf4adcecb0d446519a737553dc757420/users/0c918bea4bf443faa3bbd168f9b1a0c7/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:29.308410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f041d-3208-4629-a431-0a97219ff271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf4adcecb0d446519a737553dc757420, user_id=0c918bea4bf443faa3bbd168f9b1a0c7, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:29.311075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f041d-3208-4629-a431-0a97219ff271 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:29.324565 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4184/8456] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:29 2022] PUT /identity/v3/projects/cf4adcecb0d446519a737553dc757420/users/0c918bea4bf443faa3bbd168f9b1a0c7/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:29.334973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b021af-f752-4ca4-9ced-58ddd1d8d4e7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:29.335292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b021af-f752-4ca4-9ced-58ddd1d8d4e7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:29.335583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b021af-f752-4ca4-9ced-58ddd1d8d4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:29.365029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-27b021af-f752-4ca4-9ced-58ddd1d8d4e7 None None] MFA Rules not processed for user `0c918bea4bf443faa3bbd168f9b1a0c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:29.366253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-27b021af-f752-4ca4-9ced-58ddd1d8d4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:29.431323 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4273/8457] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:21:29.443096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f041d-3208-4629-a431-0a97219ff271 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:29.444785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-022f041d-3208-4629-a431-0a97219ff271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:29.513516 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f041d-3208-4629-a431-0a97219ff271 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:29.517190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa780b7-d869-40f8-9cef-253b492c0770 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:29.517803 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa780b7-d869-40f8-9cef-253b492c0770 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:29.518088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa780b7-d869-40f8-9cef-253b492c0770 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:29.518367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa780b7-d869-40f8-9cef-253b492c0770 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:29.527133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa780b7-d869-40f8-9cef-253b492c0770 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:29.528563 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa780b7-d869-40f8-9cef-253b492c0770 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:29.538119 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4185/8458] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:31.037863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b021af-f752-4ca4-9ced-58ddd1d8d4e7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:31.099868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b021af-f752-4ca4-9ced-58ddd1d8d4e7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:31.104409 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:31.105074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:31.105363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:31.105644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] PATH_INFO: `/v3/users/0c918bea4bf443faa3bbd168f9b1a0c7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:31.106774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c918bea4bf443faa3bbd168f9b1a0c7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:31.109034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:31.130752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:31.157582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c918bea4bf443faa3bbd168f9b1a0c7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:31.169408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c918bea4bf443faa3bbd168f9b1a0c7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:31.177597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c918bea4bf443faa3bbd168f9b1a0c7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:31.178298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c918bea4bf443faa3bbd168f9b1a0c7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:31.178931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c918bea4bf443faa3bbd168f9b1a0c7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:31.197684 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4274/8459] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:21:31 2022] DELETE /identity/v3/users/0c918bea4bf443faa3bbd168f9b1a0c7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:31.202234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce73e0-9870-4acd-9833-c8e2681c6493 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:31.202701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce73e0-9870-4acd-9833-c8e2681c6493 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:31.202836 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce73e0-9870-4acd-9833-c8e2681c6493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:31.221588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6cce73e0-9870-4acd-9833-c8e2681c6493 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:31.223273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6cce73e0-9870-4acd-9833-c8e2681c6493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:31.260056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:31.287832 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4186/8460] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:31.297589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:31.299071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:31.360672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee65ed-301d-46dd-b3ee-8b8d14a4c76e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:31.364394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcbcf0-ded2-456d-938e-406542f7062d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:31.365059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcbcf0-ded2-456d-938e-406542f7062d service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:31.365345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcbcf0-ded2-456d-938e-406542f7062d service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:31.365622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcbcf0-ded2-456d-938e-406542f7062d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:31.374865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bcbcf0-ded2-456d-938e-406542f7062d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:31.376299 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bcbcf0-ded2-456d-938e-406542f7062d service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:31.386153 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4275/8461] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:31.833321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce73e0-9870-4acd-9833-c8e2681c6493 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:31.834870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6cce73e0-9870-4acd-9833-c8e2681c6493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:31.847835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce73e0-9870-4acd-9833-c8e2681c6493 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:31.854966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07483703-ee48-4dab-a574-4680c6cfcc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:31.855611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07483703-ee48-4dab-a574-4680c6cfcc37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:31.856663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07483703-ee48-4dab-a574-4680c6cfcc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:31.856663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07483703-ee48-4dab-a574-4680c6cfcc37 admin admin] PATH_INFO: `/v3/projects/cf4adcecb0d446519a737553dc757420` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:31.857896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07483703-ee48-4dab-a574-4680c6cfcc37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf4adcecb0d446519a737553dc757420)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:31.859999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07483703-ee48-4dab-a574-4680c6cfcc37 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:31.911992 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07483703-ee48-4dab-a574-4680c6cfcc37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf4adcecb0d446519a737553dc757420', 'request_id': 'req-07483703-ee48-4dab-a574-4680c6cfcc37'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:31.920494 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07483703-ee48-4dab-a574-4680c6cfcc37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf4adcecb0d446519a737553dc757420', 'request_id': 'req-07483703-ee48-4dab-a574-4680c6cfcc37'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:31.927543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07483703-ee48-4dab-a574-4680c6cfcc37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf4adcecb0d446519a737553dc757420', 'request_id': 'req-07483703-ee48-4dab-a574-4680c6cfcc37'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:31.941272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07483703-ee48-4dab-a574-4680c6cfcc37 admin admin] The token cache is being invalidate because project cf4adcecb0d446519a737553dc757420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:21:31.941577 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07483703-ee48-4dab-a574-4680c6cfcc37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:31.942406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07483703-ee48-4dab-a574-4680c6cfcc37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:31.944547 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4187/8462] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:21:31 2022] DELETE /identity/v3/projects/cf4adcecb0d446519a737553dc757420 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:21:31.949500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e751578-0cc4-4009-a2a5-b3ecf1447196 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:31.949792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e751578-0cc4-4009-a2a5-b3ecf1447196 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:31.950071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e751578-0cc4-4009-a2a5-b3ecf1447196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:31.970130 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4e751578-0cc4-4009-a2a5-b3ecf1447196 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:31.971363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4e751578-0cc4-4009-a2a5-b3ecf1447196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:32.036390 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4276/8463] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:32.044029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc46c6-9e10-427a-bbd5-4d20d1629ff4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:32.044336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc46c6-9e10-427a-bbd5-4d20d1629ff4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:32.044603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc46c6-9e10-427a-bbd5-4d20d1629ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:32.064325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-51dc46c6-9e10-427a-bbd5-4d20d1629ff4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:32.065607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-51dc46c6-9e10-427a-bbd5-4d20d1629ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:32.073488 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4188/8464] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:32.077680 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e751578-0cc4-4009-a2a5-b3ecf1447196 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:32.089232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e751578-0cc4-4009-a2a5-b3ecf1447196 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:32.093986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3150a-01ea-4e9c-9c0e-9ee2ae29985f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:32.094595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3150a-01ea-4e9c-9c0e-9ee2ae29985f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:32.094921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3150a-01ea-4e9c-9c0e-9ee2ae29985f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:32.095186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3150a-01ea-4e9c-9c0e-9ee2ae29985f admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:32.095841 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3150a-01ea-4e9c-9c0e-9ee2ae29985f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:21:32.096175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3150a-01ea-4e9c-9c0e-9ee2ae29985f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:32.098479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3150a-01ea-4e9c-9c0e-9ee2ae29985f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:32.110249 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4277/8465] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:21:32.114495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc46c6-9e10-427a-bbd5-4d20d1629ff4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:32.123914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc46c6-9e10-427a-bbd5-4d20d1629ff4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:32.129323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb787278-e8b8-4305-b469-0ea3d4684701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:32.129999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb787278-e8b8-4305-b469-0ea3d4684701 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:32.130371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb787278-e8b8-4305-b469-0ea3d4684701 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:32.130721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb787278-e8b8-4305-b469-0ea3d4684701 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:32.131471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb787278-e8b8-4305-b469-0ea3d4684701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:32.133482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb787278-e8b8-4305-b469-0ea3d4684701 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:32.161412 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4189/8466] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:32 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:21:32.165978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3150a-01ea-4e9c-9c0e-9ee2ae29985f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:32.176104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3150a-01ea-4e9c-9c0e-9ee2ae29985f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:32.199947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:32.233978 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9f2d9-94da-4b84-9ce4-4e7859af0565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:32.234578 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9f2d9-94da-4b84-9ce4-4e7859af0565 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:32.234864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9f2d9-94da-4b84-9ce4-4e7859af0565 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:32.235141 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9f2d9-94da-4b84-9ce4-4e7859af0565 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:32.235771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9f2d9-94da-4b84-9ce4-4e7859af0565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:32.238266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9f2d9-94da-4b84-9ce4-4e7859af0565 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:32.259369 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4278/8467] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:32 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:21:32.263629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb787278-e8b8-4305-b469-0ea3d4684701 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:32.274817 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb787278-e8b8-4305-b469-0ea3d4684701 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:32.279717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9bd03d-1907-44ef-bed4-cd389832d74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:32.280323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bd03d-1907-44ef-bed4-cd389832d74e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:32.280609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bd03d-1907-44ef-bed4-cd389832d74e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:32.280888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bd03d-1907-44ef-bed4-cd389832d74e admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:32.281554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9bd03d-1907-44ef-bed4-cd389832d74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:32.283153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9bd03d-1907-44ef-bed4-cd389832d74e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:32.293251 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4190/8468] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:32.297394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9f2d9-94da-4b84-9ce4-4e7859af0565 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:32.308113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9f2d9-94da-4b84-9ce4-4e7859af0565 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:32.312713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021af50-79ec-4465-a407-595c72e2139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:32.313347 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021af50-79ec-4465-a407-595c72e2139d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:32.313633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021af50-79ec-4465-a407-595c72e2139d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:32.313909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021af50-79ec-4465-a407-595c72e2139d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:32.314566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021af50-79ec-4465-a407-595c72e2139d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:32.316760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021af50-79ec-4465-a407-595c72e2139d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:32.328226 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4279/8469] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:32.333429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9bd03d-1907-44ef-bed4-cd389832d74e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:32.343662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9bd03d-1907-44ef-bed4-cd389832d74e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:32.348389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aaf88f-cf86-4d58-9549-055a1bcdacbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:32.349091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aaf88f-cf86-4d58-9549-055a1bcdacbd admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:32.349381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aaf88f-cf86-4d58-9549-055a1bcdacbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:32.349662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aaf88f-cf86-4d58-9549-055a1bcdacbd admin admin] PATH_INFO: `/v3/projects/b12b9e642f6946c79152e986c1dad753/users/80e4e5672f9147cea5ba0433b6c2b002/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:32.364645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aaf88f-cf86-4d58-9549-055a1bcdacbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b12b9e642f6946c79152e986c1dad753, user_id=80e4e5672f9147cea5ba0433b6c2b002, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:32.367352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aaf88f-cf86-4d58-9549-055a1bcdacbd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:32.380810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4191/8470] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:32 2022] PUT /identity/v3/projects/b12b9e642f6946c79152e986c1dad753/users/80e4e5672f9147cea5ba0433b6c2b002/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:21:32.391656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0767b-df2d-4757-8f73-ca642ae8c170 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:32.391886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0767b-df2d-4757-8f73-ca642ae8c170 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:32.392168 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0767b-df2d-4757-8f73-ca642ae8c170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:32.419879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-56f0767b-df2d-4757-8f73-ca642ae8c170 None None] MFA Rules not processed for user `80e4e5672f9147cea5ba0433b6c2b002`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:32.421166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-56f0767b-df2d-4757-8f73-ca642ae8c170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:32.484229 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4280/8471] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:32.496034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aaf88f-cf86-4d58-9549-055a1bcdacbd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:32.497782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c3aaf88f-cf86-4d58-9549-055a1bcdacbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:32.559004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aaf88f-cf86-4d58-9549-055a1bcdacbd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:32.562791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b60b6d-5d50-4ddd-b050-5f1dd9467bdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:32.563454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b60b6d-5d50-4ddd-b050-5f1dd9467bdc service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:32.563742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b60b6d-5d50-4ddd-b050-5f1dd9467bdc service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:32.564025 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b60b6d-5d50-4ddd-b050-5f1dd9467bdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:32.572711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b60b6d-5d50-4ddd-b050-5f1dd9467bdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:32.574524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b60b6d-5d50-4ddd-b050-5f1dd9467bdc service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:32.579424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:32.584974 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4192/8472] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:34.072997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0767b-df2d-4757-8f73-ca642ae8c170 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:34.143397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0767b-df2d-4757-8f73-ca642ae8c170 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:34.148861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:34.149582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:34.149894 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:34.150177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] PATH_INFO: `/v3/users/80e4e5672f9147cea5ba0433b6c2b002` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:34.151536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80e4e5672f9147cea5ba0433b6c2b002)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:34.153458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:34.197417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:34.208333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80e4e5672f9147cea5ba0433b6c2b002'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:34.220106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80e4e5672f9147cea5ba0433b6c2b002'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:34.227218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80e4e5672f9147cea5ba0433b6c2b002'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:34.227899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80e4e5672f9147cea5ba0433b6c2b002'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:34.228539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80e4e5672f9147cea5ba0433b6c2b002'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:34.241674 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4281/8473] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:21:34 2022] DELETE /identity/v3/users/80e4e5672f9147cea5ba0433b6c2b002 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:34.246243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd9e36-f10a-457f-9d06-28657192a951 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:34.246759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd9e36-f10a-457f-9d06-28657192a951 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:34.246867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd9e36-f10a-457f-9d06-28657192a951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:34.266435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b6dd9e36-f10a-457f-9d06-28657192a951 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:34.267650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b6dd9e36-f10a-457f-9d06-28657192a951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:34.325377 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4193/8474] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:34.335285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:34.336685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:34.396326 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2096597-7696-4e97-884c-cac1d20b4632 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:34.399927 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aeb136-72c2-40d7-a56f-f05d2c83f526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:34.400527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aeb136-72c2-40d7-a56f-f05d2c83f526 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:34.400812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aeb136-72c2-40d7-a56f-f05d2c83f526 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:34.401118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aeb136-72c2-40d7-a56f-f05d2c83f526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:34.410580 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aeb136-72c2-40d7-a56f-f05d2c83f526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:34.412011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aeb136-72c2-40d7-a56f-f05d2c83f526 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:34.421343 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4282/8475] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:34.693876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd9e36-f10a-457f-9d06-28657192a951 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:34.695342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b6dd9e36-f10a-457f-9d06-28657192a951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:34.710407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd9e36-f10a-457f-9d06-28657192a951 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:34.715989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8178a39-0b5a-400f-b175-9ed4220c5404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:34.716640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8178a39-0b5a-400f-b175-9ed4220c5404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:34.716962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8178a39-0b5a-400f-b175-9ed4220c5404 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:34.717428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8178a39-0b5a-400f-b175-9ed4220c5404 admin admin] PATH_INFO: `/v3/projects/b12b9e642f6946c79152e986c1dad753` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:34.718749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8178a39-0b5a-400f-b175-9ed4220c5404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b12b9e642f6946c79152e986c1dad753)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:34.720831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8178a39-0b5a-400f-b175-9ed4220c5404 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:34.773179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d8178a39-0b5a-400f-b175-9ed4220c5404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b12b9e642f6946c79152e986c1dad753', 'request_id': 'req-d8178a39-0b5a-400f-b175-9ed4220c5404'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:34.780990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d8178a39-0b5a-400f-b175-9ed4220c5404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b12b9e642f6946c79152e986c1dad753', 'request_id': 'req-d8178a39-0b5a-400f-b175-9ed4220c5404'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:34.788668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d8178a39-0b5a-400f-b175-9ed4220c5404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b12b9e642f6946c79152e986c1dad753', 'request_id': 'req-d8178a39-0b5a-400f-b175-9ed4220c5404'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:34.799155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d8178a39-0b5a-400f-b175-9ed4220c5404 admin admin] The token cache is being invalidate because project b12b9e642f6946c79152e986c1dad753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:21:34.799451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d8178a39-0b5a-400f-b175-9ed4220c5404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:34.804205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d8178a39-0b5a-400f-b175-9ed4220c5404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:34.805764 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4194/8476] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:21:34 2022] DELETE /identity/v3/projects/b12b9e642f6946c79152e986c1dad753 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:34.813417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a7312-be8a-4103-b9ba-a4801175f840 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:34.813784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a7312-be8a-4103-b9ba-a4801175f840 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:34.814126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a7312-be8a-4103-b9ba-a4801175f840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:34.835777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-aa9a7312-be8a-4103-b9ba-a4801175f840 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:34.837453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-aa9a7312-be8a-4103-b9ba-a4801175f840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:34.898357 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4283/8477] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:34.906212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823337d-b8c3-4325-ac3e-066c1858c9db None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:34.906508 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823337d-b8c3-4325-ac3e-066c1858c9db None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:34.906792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823337d-b8c3-4325-ac3e-066c1858c9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:34.926860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6823337d-b8c3-4325-ac3e-066c1858c9db None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:34.928067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6823337d-b8c3-4325-ac3e-066c1858c9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:34.936266 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4195/8478] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:34.941033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a7312-be8a-4103-b9ba-a4801175f840 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:34.951153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a7312-be8a-4103-b9ba-a4801175f840 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:34.955741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e86b0a-7808-45b0-8503-c9dd6dafb78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:34.956887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e86b0a-7808-45b0-8503-c9dd6dafb78d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:34.957205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e86b0a-7808-45b0-8503-c9dd6dafb78d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:34.957480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e86b0a-7808-45b0-8503-c9dd6dafb78d admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:34.958124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e86b0a-7808-45b0-8503-c9dd6dafb78d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:21:34.958429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e86b0a-7808-45b0-8503-c9dd6dafb78d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:34.960031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e86b0a-7808-45b0-8503-c9dd6dafb78d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:34.973025 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4284/8479] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:21:34.977292 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823337d-b8c3-4325-ac3e-066c1858c9db None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:34.986914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823337d-b8c3-4325-ac3e-066c1858c9db None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:34.992227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae46e6-0e27-4e38-9b1a-8af26453ce69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:34.992829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae46e6-0e27-4e38-9b1a-8af26453ce69 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:34.993158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae46e6-0e27-4e38-9b1a-8af26453ce69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:34.993435 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae46e6-0e27-4e38-9b1a-8af26453ce69 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:34.994073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae46e6-0e27-4e38-9b1a-8af26453ce69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:34.995976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae46e6-0e27-4e38-9b1a-8af26453ce69 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:35.026277 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4196/8480] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:34 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:21:35.030401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e86b0a-7808-45b0-8503-c9dd6dafb78d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:35.042291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e86b0a-7808-45b0-8503-c9dd6dafb78d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:35.086320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:35.103379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753cf68-0a2a-409f-b6c8-f582e505db14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:35.103994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753cf68-0a2a-409f-b6c8-f582e505db14 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:35.104280 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753cf68-0a2a-409f-b6c8-f582e505db14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:35.104560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753cf68-0a2a-409f-b6c8-f582e505db14 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:35.105239 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753cf68-0a2a-409f-b6c8-f582e505db14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:35.107251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753cf68-0a2a-409f-b6c8-f582e505db14 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:35.133350 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4285/8481] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:35 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:35.137976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae46e6-0e27-4e38-9b1a-8af26453ce69 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:35.149636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae46e6-0e27-4e38-9b1a-8af26453ce69 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:35.154382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8ce95-34a5-4f71-8e5f-916b6c1ccf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:35.154975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8ce95-34a5-4f71-8e5f-916b6c1ccf47 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:35.155263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8ce95-34a5-4f71-8e5f-916b6c1ccf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:35.155539 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8ce95-34a5-4f71-8e5f-916b6c1ccf47 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:35.156177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8ce95-34a5-4f71-8e5f-916b6c1ccf47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:35.157812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8ce95-34a5-4f71-8e5f-916b6c1ccf47 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:35.167810 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4197/8482] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:35.171943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753cf68-0a2a-409f-b6c8-f582e505db14 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:35.182635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753cf68-0a2a-409f-b6c8-f582e505db14 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:35.187595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d98886-bcd9-4b14-8ac0-fc9364a61b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:35.188153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d98886-bcd9-4b14-8ac0-fc9364a61b56 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:35.188442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d98886-bcd9-4b14-8ac0-fc9364a61b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:35.189147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d98886-bcd9-4b14-8ac0-fc9364a61b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:35.189400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d98886-bcd9-4b14-8ac0-fc9364a61b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:35.191658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d98886-bcd9-4b14-8ac0-fc9364a61b56 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:35.209035 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4286/8483] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:21:35.213133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8ce95-34a5-4f71-8e5f-916b6c1ccf47 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:35.224181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8ce95-34a5-4f71-8e5f-916b6c1ccf47 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:35.228967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273034fd-edae-47f6-b8b5-fd704fb34d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:35.229635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273034fd-edae-47f6-b8b5-fd704fb34d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:35.229920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273034fd-edae-47f6-b8b5-fd704fb34d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:35.230201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273034fd-edae-47f6-b8b5-fd704fb34d54 admin admin] PATH_INFO: `/v3/projects/bacdca8fbe7e40eca4bafd2161319d10/users/02022e74588e47eba9e9506ef78161cc/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:35.247835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273034fd-edae-47f6-b8b5-fd704fb34d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bacdca8fbe7e40eca4bafd2161319d10, user_id=02022e74588e47eba9e9506ef78161cc, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:35.250589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273034fd-edae-47f6-b8b5-fd704fb34d54 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:35.264669 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4198/8484] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:35 2022] PUT /identity/v3/projects/bacdca8fbe7e40eca4bafd2161319d10/users/02022e74588e47eba9e9506ef78161cc/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:21:35.274979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746a38d-4a1a-47f8-967f-ebadd133337f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:35.275275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746a38d-4a1a-47f8-967f-ebadd133337f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:35.275662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746a38d-4a1a-47f8-967f-ebadd133337f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:35.305788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3746a38d-4a1a-47f8-967f-ebadd133337f None None] MFA Rules not processed for user `02022e74588e47eba9e9506ef78161cc`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:35.307038 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3746a38d-4a1a-47f8-967f-ebadd133337f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:35.383055 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4287/8485] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:35.394620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273034fd-edae-47f6-b8b5-fd704fb34d54 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:35.396346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-273034fd-edae-47f6-b8b5-fd704fb34d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:35.419136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:35.459242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273034fd-edae-47f6-b8b5-fd704fb34d54 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:35.463015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe7884-a0e4-49d4-8bcc-6bb1075aacf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:35.463631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe7884-a0e4-49d4-8bcc-6bb1075aacf4 service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:35.463918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe7884-a0e4-49d4-8bcc-6bb1075aacf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:35.464196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe7884-a0e4-49d4-8bcc-6bb1075aacf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:35.473301 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe7884-a0e4-49d4-8bcc-6bb1075aacf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:35.474739 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe7884-a0e4-49d4-8bcc-6bb1075aacf4 service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:35.483861 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4199/8486] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:37.035521 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746a38d-4a1a-47f8-967f-ebadd133337f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:37.104103 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746a38d-4a1a-47f8-967f-ebadd133337f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:37.108878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:37.109541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:37.109822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:37.110097 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] PATH_INFO: `/v3/users/02022e74588e47eba9e9506ef78161cc` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:37.111217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02022e74588e47eba9e9506ef78161cc)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:37.113221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:37.164264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02022e74588e47eba9e9506ef78161cc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:37.170459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:37.178077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02022e74588e47eba9e9506ef78161cc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:37.189346 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02022e74588e47eba9e9506ef78161cc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:37.190021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02022e74588e47eba9e9506ef78161cc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:37.191185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02022e74588e47eba9e9506ef78161cc'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:37.205236 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4288/8487] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:21:37 2022] DELETE /identity/v3/users/02022e74588e47eba9e9506ef78161cc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:37.209940 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da738662-8c2a-49f1-a755-bf4cb8eb0cd6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:37.210387 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da738662-8c2a-49f1-a755-bf4cb8eb0cd6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:37.210519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da738662-8c2a-49f1-a755-bf4cb8eb0cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:37.229892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-da738662-8c2a-49f1-a755-bf4cb8eb0cd6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:37.231097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-da738662-8c2a-49f1-a755-bf4cb8eb0cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:37.293821 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4200/8488] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:21:37.304318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:37.305837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:37.370125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a92e-17f0-4558-95e8-3af626520e79 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:37.374006 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c89895-1ff7-45e7-afc6-837db364cf03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:37.374619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c89895-1ff7-45e7-afc6-837db364cf03 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:37.374905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c89895-1ff7-45e7-afc6-837db364cf03 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:37.375181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c89895-1ff7-45e7-afc6-837db364cf03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:37.386319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c89895-1ff7-45e7-afc6-837db364cf03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:37.387708 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c89895-1ff7-45e7-afc6-837db364cf03 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:37.399250 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4289/8489] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:37.995415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da738662-8c2a-49f1-a755-bf4cb8eb0cd6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:37.998851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-da738662-8c2a-49f1-a755-bf4cb8eb0cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:38.029350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da738662-8c2a-49f1-a755-bf4cb8eb0cd6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:38.035385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:38.036353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:38.036781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:38.037112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3 admin admin] PATH_INFO: `/v3/projects/bacdca8fbe7e40eca4bafd2161319d10` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:38.038271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bacdca8fbe7e40eca4bafd2161319d10)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:38.040205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:38.088335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bacdca8fbe7e40eca4bafd2161319d10', 'request_id': 'req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:38.096473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bacdca8fbe7e40eca4bafd2161319d10', 'request_id': 'req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:38.104055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bacdca8fbe7e40eca4bafd2161319d10', 'request_id': 'req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:38.121376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3 admin admin] The token cache is being invalidate because project bacdca8fbe7e40eca4bafd2161319d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:21:38.121892 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:38.123001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f5ce17e7-e6b0-4d9a-a941-e9b86fbb2fa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:38.124834 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4201/8490] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:21:37 2022] DELETE /identity/v3/projects/bacdca8fbe7e40eca4bafd2161319d10 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:38.146769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4fcc6-bfe2-424a-928d-776187687b09 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:38.148145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4fcc6-bfe2-424a-928d-776187687b09 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:38.148145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4fcc6-bfe2-424a-928d-776187687b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:38.171446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-dff4fcc6-bfe2-424a-928d-776187687b09 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:38.172795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dff4fcc6-bfe2-424a-928d-776187687b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:38.238691 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4290/8491] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:21:38.248071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9148e-d24e-444b-898a-823a3fb8b904 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:38.248579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9148e-d24e-444b-898a-823a3fb8b904 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:38.248882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9148e-d24e-444b-898a-823a3fb8b904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:38.293823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-48c9148e-d24e-444b-898a-823a3fb8b904 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:38.296433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-48c9148e-d24e-444b-898a-823a3fb8b904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:38.305704 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4202/8492] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:38.309798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4fcc6-bfe2-424a-928d-776187687b09 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:38.322630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4fcc6-bfe2-424a-928d-776187687b09 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:38.330901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947545e8-6160-495c-93dc-1f245abf24c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:38.332094 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947545e8-6160-495c-93dc-1f245abf24c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:38.332494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947545e8-6160-495c-93dc-1f245abf24c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:38.332655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947545e8-6160-495c-93dc-1f245abf24c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:38.333373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947545e8-6160-495c-93dc-1f245abf24c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:21:38.333683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947545e8-6160-495c-93dc-1f245abf24c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:38.335335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947545e8-6160-495c-93dc-1f245abf24c1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:38.350693 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4291/8493] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:21:38.353860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9148e-d24e-444b-898a-823a3fb8b904 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:38.373334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9148e-d24e-444b-898a-823a3fb8b904 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:38.373334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3669dc-1f86-4578-b677-0f2d3a082710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:38.373334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3669dc-1f86-4578-b677-0f2d3a082710 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:38.373334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3669dc-1f86-4578-b677-0f2d3a082710 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:38.373768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3669dc-1f86-4578-b677-0f2d3a082710 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:38.373768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3669dc-1f86-4578-b677-0f2d3a082710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:38.375645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3669dc-1f86-4578-b677-0f2d3a082710 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:38.414279 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4203/8494] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:38 2022] POST /identity/v3/projects => generated 391 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:38.420180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947545e8-6160-495c-93dc-1f245abf24c1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:38.432969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947545e8-6160-495c-93dc-1f245abf24c1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:38.500751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827d485-a07b-4439-afc7-1abba4ca9536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:38.501603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827d485-a07b-4439-afc7-1abba4ca9536 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:38.501956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827d485-a07b-4439-afc7-1abba4ca9536 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:38.502298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827d485-a07b-4439-afc7-1abba4ca9536 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:38.503509 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827d485-a07b-4439-afc7-1abba4ca9536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:38.505808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827d485-a07b-4439-afc7-1abba4ca9536 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:38.516183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:38.536190 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4292/8495] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:38 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:38.537247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3669dc-1f86-4578-b677-0f2d3a082710 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:38.557736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3669dc-1f86-4578-b677-0f2d3a082710 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:38.566478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83e94d-b7ee-433c-b281-7ce4343d3a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:38.567442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83e94d-b7ee-433c-b281-7ce4343d3a6d admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:38.567735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83e94d-b7ee-433c-b281-7ce4343d3a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:38.568012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83e94d-b7ee-433c-b281-7ce4343d3a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:38.568665 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f83e94d-b7ee-433c-b281-7ce4343d3a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:38.570701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f83e94d-b7ee-433c-b281-7ce4343d3a6d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:38.582068 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4204/8496] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:38.586162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827d485-a07b-4439-afc7-1abba4ca9536 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:38.597893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827d485-a07b-4439-afc7-1abba4ca9536 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:38.604559 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818156a-4759-4876-80c2-f7139437f946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:38.605273 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818156a-4759-4876-80c2-f7139437f946 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:38.605729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818156a-4759-4876-80c2-f7139437f946 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:38.605989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818156a-4759-4876-80c2-f7139437f946 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:38.606672 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818156a-4759-4876-80c2-f7139437f946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:38.608639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818156a-4759-4876-80c2-f7139437f946 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:38.621554 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4293/8497] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:21:38.625581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83e94d-b7ee-433c-b281-7ce4343d3a6d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:38.637145 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83e94d-b7ee-433c-b281-7ce4343d3a6d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:38.643349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005a438-9f7c-4ef3-b241-81c3989e4ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:38.644046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005a438-9f7c-4ef3-b241-81c3989e4ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:38.644325 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005a438-9f7c-4ef3-b241-81c3989e4ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:38.644604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005a438-9f7c-4ef3-b241-81c3989e4ba5 admin admin] PATH_INFO: `/v3/projects/5181e2d5ea864275b54dc982cb907709/users/cb23efd57c684bbf9f1974d4482a2cdb/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:38.647205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:38.665493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005a438-9f7c-4ef3-b241-81c3989e4ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5181e2d5ea864275b54dc982cb907709, user_id=cb23efd57c684bbf9f1974d4482a2cdb, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:38.668364 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005a438-9f7c-4ef3-b241-81c3989e4ba5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:38.685401 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4205/8498] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:38 2022] PUT /identity/v3/projects/5181e2d5ea864275b54dc982cb907709/users/cb23efd57c684bbf9f1974d4482a2cdb/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:21:38.696200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb56f0-ffb4-49b7-a7b9-9df73789057d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:38.697170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb56f0-ffb4-49b7-a7b9-9df73789057d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:38.697482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb56f0-ffb4-49b7-a7b9-9df73789057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:38.727990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f8eb56f0-ffb4-49b7-a7b9-9df73789057d None None] MFA Rules not processed for user `cb23efd57c684bbf9f1974d4482a2cdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:38.729180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f8eb56f0-ffb4-49b7-a7b9-9df73789057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:38.817254 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4294/8499] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:38.828242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005a438-9f7c-4ef3-b241-81c3989e4ba5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:38.830116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7005a438-9f7c-4ef3-b241-81c3989e4ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:38.894681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005a438-9f7c-4ef3-b241-81c3989e4ba5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:38.898513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3f0e9-21d1-4276-b69e-97bf58f7d58a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '133a1ff64e224593be467eb7c27cbb09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:38.899157 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3f0e9-21d1-4276-b69e-97bf58f7d58a service cinder] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:38.899439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3f0e9-21d1-4276-b69e-97bf58f7d58a service cinder] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:38.899714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3f0e9-21d1-4276-b69e-97bf58f7d58a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:38.908737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb3f0e9-21d1-4276-b69e-97bf58f7d58a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:38.911104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb3f0e9-21d1-4276-b69e-97bf58f7d58a service cinder] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:38.920108 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4206/8500] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:40.436716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb56f0-ffb4-49b7-a7b9-9df73789057d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:40.501387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb56f0-ffb4-49b7-a7b9-9df73789057d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:40.506020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:40.506662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:40.506950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:40.507231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] PATH_INFO: `/v3/users/cb23efd57c684bbf9f1974d4482a2cdb` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:40.508361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb23efd57c684bbf9f1974d4482a2cdb)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:40.510355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:40.559117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb23efd57c684bbf9f1974d4482a2cdb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:40.570936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb23efd57c684bbf9f1974d4482a2cdb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:40.577610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb23efd57c684bbf9f1974d4482a2cdb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:40.578355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb23efd57c684bbf9f1974d4482a2cdb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:40.579000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb23efd57c684bbf9f1974d4482a2cdb'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:40.580244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:40.594888 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4295/8501] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:21:40 2022] DELETE /identity/v3/users/cb23efd57c684bbf9f1974d4482a2cdb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:21:40.600091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bedbe0-cc67-4a90-b974-e69c1c21eb5d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:40.600568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bedbe0-cc67-4a90-b974-e69c1c21eb5d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:40.600699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bedbe0-cc67-4a90-b974-e69c1c21eb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:40.621043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d5bedbe0-cc67-4a90-b974-e69c1c21eb5d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:40.622416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d5bedbe0-cc67-4a90-b974-e69c1c21eb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:40.686197 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4207/8502] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:40.696692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:40.698454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:40.760298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337412ae-69ce-434e-99cf-27dcc6d9bbed admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:40.763945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e61cfb-f010-4a93-96c0-c3e25960a4be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:40.764561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e61cfb-f010-4a93-96c0-c3e25960a4be service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:40.764845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e61cfb-f010-4a93-96c0-c3e25960a4be service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:40.765155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e61cfb-f010-4a93-96c0-c3e25960a4be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:40.775018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e61cfb-f010-4a93-96c0-c3e25960a4be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:40.776456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e61cfb-f010-4a93-96c0-c3e25960a4be service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:40.787393 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4296/8503] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:41.302828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bedbe0-cc67-4a90-b974-e69c1c21eb5d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:41.305906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d5bedbe0-cc67-4a90-b974-e69c1c21eb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:41.323830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bedbe0-cc67-4a90-b974-e69c1c21eb5d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:41.330343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:41.330715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:41.331051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:41.331324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9 admin admin] PATH_INFO: `/v3/projects/5181e2d5ea864275b54dc982cb907709` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:41.332459 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5181e2d5ea864275b54dc982cb907709)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:41.334652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:41.392987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5181e2d5ea864275b54dc982cb907709', 'request_id': 'req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:41.406584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5181e2d5ea864275b54dc982cb907709', 'request_id': 'req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:41.416764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5181e2d5ea864275b54dc982cb907709', 'request_id': 'req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:41.433206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9 admin admin] The token cache is being invalidate because project 5181e2d5ea864275b54dc982cb907709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:21:41.433754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:41.434881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42e6d47d-cf89-4436-a9d8-5afe221e5fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:21:41.437203 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4208/8504] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:21:41 2022] DELETE /identity/v3/projects/5181e2d5ea864275b54dc982cb907709 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:41.448606 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e8861-4324-4c5d-80b7-5ae9a1971bba None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:41.449291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e8861-4324-4c5d-80b7-5ae9a1971bba None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:41.449501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e8861-4324-4c5d-80b7-5ae9a1971bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:41.476075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-486e8861-4324-4c5d-80b7-5ae9a1971bba None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:41.477690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-486e8861-4324-4c5d-80b7-5ae9a1971bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:41.602555 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4297/8505] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:41.612796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01a336-3a62-4b52-916e-ae4496e664ab None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:41.613777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01a336-3a62-4b52-916e-ae4496e664ab None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:41.614402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01a336-3a62-4b52-916e-ae4496e664ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:41.654920 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0b01a336-3a62-4b52-916e-ae4496e664ab None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:41.660001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0b01a336-3a62-4b52-916e-ae4496e664ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:41.671394 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4209/8506] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:21:41.676518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e8861-4324-4c5d-80b7-5ae9a1971bba None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:41.689080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e8861-4324-4c5d-80b7-5ae9a1971bba None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:41.705762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28c26e-f64f-478b-958f-5638edb68475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:41.706892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28c26e-f64f-478b-958f-5638edb68475 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:41.707619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28c26e-f64f-478b-958f-5638edb68475 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:41.707907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28c26e-f64f-478b-958f-5638edb68475 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:41.710235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28c26e-f64f-478b-958f-5638edb68475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:21:41.710773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28c26e-f64f-478b-958f-5638edb68475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:41.714050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28c26e-f64f-478b-958f-5638edb68475 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:41.728793 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4298/8507] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:21:41.734900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01a336-3a62-4b52-916e-ae4496e664ab None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:41.741650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:41.751845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01a336-3a62-4b52-916e-ae4496e664ab None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:41.761410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d0451-e14c-4240-ace9-9b1d9f7c828e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:41.762766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d0451-e14c-4240-ace9-9b1d9f7c828e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:41.763238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d0451-e14c-4240-ace9-9b1d9f7c828e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:41.763708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d0451-e14c-4240-ace9-9b1d9f7c828e admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:41.765089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d0451-e14c-4240-ace9-9b1d9f7c828e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:41.767901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d0451-e14c-4240-ace9-9b1d9f7c828e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:41.820631 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4210/8508] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:41 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:41.825737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28c26e-f64f-478b-958f-5638edb68475 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:41.838171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28c26e-f64f-478b-958f-5638edb68475 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:41.905072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1abcd-e633-46a1-8083-0874fcdb5a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:41.905770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1abcd-e633-46a1-8083-0874fcdb5a09 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:41.906125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1abcd-e633-46a1-8083-0874fcdb5a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:41.906407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1abcd-e633-46a1-8083-0874fcdb5a09 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:41.907035 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1abcd-e633-46a1-8083-0874fcdb5a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:41.909087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1abcd-e633-46a1-8083-0874fcdb5a09 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:41.934046 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4299/8509] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:41 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:41.935328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d0451-e14c-4240-ace9-9b1d9f7c828e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:41.945825 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d0451-e14c-4240-ace9-9b1d9f7c828e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:41.950744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb6c71-cc99-4807-be7e-856f93e0f944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:41.951478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb6c71-cc99-4807-be7e-856f93e0f944 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:41.951856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb6c71-cc99-4807-be7e-856f93e0f944 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:41.952139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb6c71-cc99-4807-be7e-856f93e0f944 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:41.952782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb6c71-cc99-4807-be7e-856f93e0f944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:41.954437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb6c71-cc99-4807-be7e-856f93e0f944 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:41.965531 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4211/8510] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:21:41.971310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1abcd-e633-46a1-8083-0874fcdb5a09 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:41.982560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1abcd-e633-46a1-8083-0874fcdb5a09 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:41.987585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c768789-3bf0-4e0b-99c3-b2f890c61092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:41.988214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c768789-3bf0-4e0b-99c3-b2f890c61092 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:41.988500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c768789-3bf0-4e0b-99c3-b2f890c61092 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:41.988773 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c768789-3bf0-4e0b-99c3-b2f890c61092 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:41.989474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c768789-3bf0-4e0b-99c3-b2f890c61092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:41.991073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c768789-3bf0-4e0b-99c3-b2f890c61092 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:41.993331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:42.006647 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4300/8511] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:21:42.007894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb6c71-cc99-4807-be7e-856f93e0f944 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:42.019310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb6c71-cc99-4807-be7e-856f93e0f944 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:42.024549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346632ac-67f4-447a-8bc6-fdd142060e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:42.025252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346632ac-67f4-447a-8bc6-fdd142060e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:42.025540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346632ac-67f4-447a-8bc6-fdd142060e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:42.025822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346632ac-67f4-447a-8bc6-fdd142060e96 admin admin] PATH_INFO: `/v3/projects/6d0bae9fc4a443f1bdbd9d136784516a/users/98434520e99e42099265bd81e88f1862/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:42.048877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346632ac-67f4-447a-8bc6-fdd142060e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0bae9fc4a443f1bdbd9d136784516a, user_id=98434520e99e42099265bd81e88f1862, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:42.051558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346632ac-67f4-447a-8bc6-fdd142060e96 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:42.065797 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4212/8512] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:42 2022] PUT /identity/v3/projects/6d0bae9fc4a443f1bdbd9d136784516a/users/98434520e99e42099265bd81e88f1862/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:42.070726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf61c0-3ab3-4458-8ed1-949bed4cc3ee None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:42.071018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf61c0-3ab3-4458-8ed1-949bed4cc3ee None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:42.071297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf61c0-3ab3-4458-8ed1-949bed4cc3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:42.093051 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2fdf61c0-3ab3-4458-8ed1-949bed4cc3ee None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:42.094285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2fdf61c0-3ab3-4458-8ed1-949bed4cc3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:42.171809 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4301/8513] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:42.186703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346632ac-67f4-447a-8bc6-fdd142060e96 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:42.189039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-346632ac-67f4-447a-8bc6-fdd142060e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:42.269064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346632ac-67f4-447a-8bc6-fdd142060e96 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:42.269064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269bacb-1dfd-4b7e-824a-9fd806850620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:42.269064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269bacb-1dfd-4b7e-824a-9fd806850620 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:42.269064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269bacb-1dfd-4b7e-824a-9fd806850620 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:42.269594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269bacb-1dfd-4b7e-824a-9fd806850620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:42.278090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269bacb-1dfd-4b7e-824a-9fd806850620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:42.279686 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269bacb-1dfd-4b7e-824a-9fd806850620 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:42.292495 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4213/8514] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:42.741330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf61c0-3ab3-4458-8ed1-949bed4cc3ee None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:42.743759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2fdf61c0-3ab3-4458-8ed1-949bed4cc3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:42.851054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf61c0-3ab3-4458-8ed1-949bed4cc3ee None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:42.856603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff4fa3-652f-4c78-a726-80a359ac2eb6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:42.857384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff4fa3-652f-4c78-a726-80a359ac2eb6 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:42.858001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff4fa3-652f-4c78-a726-80a359ac2eb6 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:42.858354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff4fa3-652f-4c78-a726-80a359ac2eb6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:42.860105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-97ff4fa3-652f-4c78-a726-80a359ac2eb6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:42.874930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff4fa3-652f-4c78-a726-80a359ac2eb6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:42.876529 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff4fa3-652f-4c78-a726-80a359ac2eb6 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:42.890108 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4302/8515] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:44.234215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269bacb-1dfd-4b7e-824a-9fd806850620 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:44.238860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7269bacb-1dfd-4b7e-824a-9fd806850620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:44.263244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269bacb-1dfd-4b7e-824a-9fd806850620 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:44.270663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67817d5e-71f2-4eae-8482-d927c961970c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:44.271648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67817d5e-71f2-4eae-8482-d927c961970c service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:44.272791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67817d5e-71f2-4eae-8482-d927c961970c service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:44.273491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67817d5e-71f2-4eae-8482-d927c961970c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:44.277316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-67817d5e-71f2-4eae-8482-d927c961970c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:44.386838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67817d5e-71f2-4eae-8482-d927c961970c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:44.388506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67817d5e-71f2-4eae-8482-d927c961970c service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:44.399520 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4214/8516] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:44 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:48.817177 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff4fa3-652f-4c78-a726-80a359ac2eb6 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:48.818621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-97ff4fa3-652f-4c78-a726-80a359ac2eb6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:48.926704 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff4fa3-652f-4c78-a726-80a359ac2eb6 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:48.935803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450aabfc-fef4-4bdc-aa3b-3bf06465c0c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:48.936710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450aabfc-fef4-4bdc-aa3b-3bf06465c0c7 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:48.937181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450aabfc-fef4-4bdc-aa3b-3bf06465c0c7 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:48.937620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450aabfc-fef4-4bdc-aa3b-3bf06465c0c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:48.939897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-450aabfc-fef4-4bdc-aa3b-3bf06465c0c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:48.967371 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450aabfc-fef4-4bdc-aa3b-3bf06465c0c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:48.972301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450aabfc-fef4-4bdc-aa3b-3bf06465c0c7 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:48.999214 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4303/8517] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:21:50.144713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6c06-a575-4d79-a269-716924aceae9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:50.145612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6c06-a575-4d79-a269-716924aceae9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:50.146064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6c06-a575-4d79-a269-716924aceae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:50.181296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-779b6c06-a575-4d79-a269-716924aceae9 None None] MFA Rules not processed for user `98434520e99e42099265bd81e88f1862`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:50.182988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-779b6c06-a575-4d79-a269-716924aceae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:50.241004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:50.298450 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4215/8518] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:50.303408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450aabfc-fef4-4bdc-aa3b-3bf06465c0c7 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:50.317990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450aabfc-fef4-4bdc-aa3b-3bf06465c0c7 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:50.323994 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee845411-4835-40bf-a72c-bc6e1e93d3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:50.325535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee845411-4835-40bf-a72c-bc6e1e93d3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:50.327251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee845411-4835-40bf-a72c-bc6e1e93d3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:50.327554 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee845411-4835-40bf-a72c-bc6e1e93d3a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:50.328217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee845411-4835-40bf-a72c-bc6e1e93d3a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:50.330285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee845411-4835-40bf-a72c-bc6e1e93d3a8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:50.351630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:50.383830 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4304/8519] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:21:50 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:21:50.389863 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b6c06-a575-4d79-a269-716924aceae9 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:50.406043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b6c06-a575-4d79-a269-716924aceae9 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:50.500082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d6bf5-bf9e-406a-a2c0-72b754e9bac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:50.500878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d6bf5-bf9e-406a-a2c0-72b754e9bac3 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:50.501275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d6bf5-bf9e-406a-a2c0-72b754e9bac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:50.501634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d6bf5-bf9e-406a-a2c0-72b754e9bac3 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:50.502352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d6bf5-bf9e-406a-a2c0-72b754e9bac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:50.505644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d6bf5-bf9e-406a-a2c0-72b754e9bac3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:50.553483 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4216/8520] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:21:50 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:21:50.554173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee845411-4835-40bf-a72c-bc6e1e93d3a8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:50.569598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee845411-4835-40bf-a72c-bc6e1e93d3a8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:50.581573 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d74c70-095c-499b-b561-b193b30a7f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:50.582246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d74c70-095c-499b-b561-b193b30a7f28 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:50.582725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d74c70-095c-499b-b561-b193b30a7f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:50.583173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d74c70-095c-499b-b561-b193b30a7f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:50.583986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d74c70-095c-499b-b561-b193b30a7f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:50.589424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d74c70-095c-499b-b561-b193b30a7f28 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:50.625367 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4305/8521] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:50.627138 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d6bf5-bf9e-406a-a2c0-72b754e9bac3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:50.641871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d6bf5-bf9e-406a-a2c0-72b754e9bac3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:50.652465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d402bbd-d9f1-4992-9815-390dd2c0345d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:50.652465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d402bbd-d9f1-4992-9815-390dd2c0345d admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:50.652465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d402bbd-d9f1-4992-9815-390dd2c0345d admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:50.652465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d402bbd-d9f1-4992-9815-390dd2c0345d admin admin] PATH_INFO: `/v3/projects/a10f513ed83941c8a6e7ca5ef6602ce9/users/23cb2d04eeac461b870d4f6d05c26f3c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:50.679322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d402bbd-d9f1-4992-9815-390dd2c0345d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a10f513ed83941c8a6e7ca5ef6602ce9, user_id=23cb2d04eeac461b870d4f6d05c26f3c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:50.683207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d402bbd-d9f1-4992-9815-390dd2c0345d admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:50.712461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d74c70-095c-499b-b561-b193b30a7f28 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:50.717309 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4217/8522] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:50 2022] PUT /identity/v3/projects/a10f513ed83941c8a6e7ca5ef6602ce9/users/23cb2d04eeac461b870d4f6d05c26f3c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:21:50.722285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d402bbd-d9f1-4992-9815-390dd2c0345d admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:50.865587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d74c70-095c-499b-b561-b193b30a7f28 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:50.898050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d402bbd-d9f1-4992-9815-390dd2c0345d admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:50.909600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d73c-17cc-4c94-8c5c-ed3f33e9c5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:50.910237 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d73c-17cc-4c94-8c5c-ed3f33e9c5b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:50.910524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d73c-17cc-4c94-8c5c-ed3f33e9c5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:50.910805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d73c-17cc-4c94-8c5c-ed3f33e9c5b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:50.911465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25d73c-17cc-4c94-8c5c-ed3f33e9c5b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:50.916172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25d73c-17cc-4c94-8c5c-ed3f33e9c5b5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:50.933184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32657d-ac4e-4ad4-8233-a0408ce624c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:50.934138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32657d-ac4e-4ad4-8233-a0408ce624c2 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:50.935534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32657d-ac4e-4ad4-8233-a0408ce624c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:50.936745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32657d-ac4e-4ad4-8233-a0408ce624c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:50.942674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7d32657d-ac4e-4ad4-8233-a0408ce624c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:50.945479 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4218/8523] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:21:50.946618 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d73c-17cc-4c94-8c5c-ed3f33e9c5b5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:50.968438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d73c-17cc-4c94-8c5c-ed3f33e9c5b5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:50.974081 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14716e6c-cd4d-41f9-9478-0ae15a2f2562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:50.975141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14716e6c-cd4d-41f9-9478-0ae15a2f2562 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:50.975141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14716e6c-cd4d-41f9-9478-0ae15a2f2562 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:50.975621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14716e6c-cd4d-41f9-9478-0ae15a2f2562 admin admin] PATH_INFO: `/v3/projects/a10f513ed83941c8a6e7ca5ef6602ce9/users/23cb2d04eeac461b870d4f6d05c26f3c/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:50.977916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14716e6c-cd4d-41f9-9478-0ae15a2f2562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a10f513ed83941c8a6e7ca5ef6602ce9, user_id=23cb2d04eeac461b870d4f6d05c26f3c, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:50.981044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14716e6c-cd4d-41f9-9478-0ae15a2f2562 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:50.994202 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4219/8524] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:21:50 2022] PUT /identity/v3/projects/a10f513ed83941c8a6e7ca5ef6602ce9/users/23cb2d04eeac461b870d4f6d05c26f3c/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:21:51.074839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32657d-ac4e-4ad4-8233-a0408ce624c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:51.077886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32657d-ac4e-4ad4-8233-a0408ce624c2 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:51.102956 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4306/8525] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:50 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:21:55.825293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c0c4f-3bd0-4127-86fe-e73c1a77346e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:55.826460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c0c4f-3bd0-4127-86fe-e73c1a77346e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:55.827014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c0c4f-3bd0-4127-86fe-e73c1a77346e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:55.873644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-2c6c0c4f-3bd0-4127-86fe-e73c1a77346e None None] MFA Rules not processed for user `b5c96be8fbe1436bb3e84340ceda1917`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:55.875345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2c6c0c4f-3bd0-4127-86fe-e73c1a77346e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:55.903413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:55.979070 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4220/8526] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:57.055524 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32657d-ac4e-4ad4-8233-a0408ce624c2 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:21:57.073048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32657d-ac4e-4ad4-8233-a0408ce624c2 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:21:57.181048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:21:57.197915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15b813-bd9e-441d-a309-56a559b2b442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:21:57.198886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15b813-bd9e-441d-a309-56a559b2b442 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:57.199579 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15b813-bd9e-441d-a309-56a559b2b442 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:57.200092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15b813-bd9e-441d-a309-56a559b2b442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:57.206197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2b15b813-bd9e-441d-a309-56a559b2b442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:57.234482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b15b813-bd9e-441d-a309-56a559b2b442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:21:57.236386 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b15b813-bd9e-441d-a309-56a559b2b442 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:21:57.263853 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4307/8527] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:21:59.730111 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ba368-228c-4c84-af17-f4dca1f51572 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:21:59.730599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ba368-228c-4c84-af17-f4dca1f51572 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:21:59.730725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ba368-228c-4c84-af17-f4dca1f51572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:21:59.779611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-128ba368-228c-4c84-af17-f4dca1f51572 None None] MFA Rules not processed for user `23cb2d04eeac461b870d4f6d05c26f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:21:59.781109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-128ba368-228c-4c84-af17-f4dca1f51572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:21:59.909440 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4221/8528] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:21:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:21:59.923904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15b813-bd9e-441d-a309-56a559b2b442 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:22:00.069760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15b813-bd9e-441d-a309-56a559b2b442 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:22:00.085870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec693ba2-0787-4573-b108-642395545fa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:22:00.087849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec693ba2-0787-4573-b108-642395545fa0 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:00.088319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec693ba2-0787-4573-b108-642395545fa0 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:00.088761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec693ba2-0787-4573-b108-642395545fa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:00.200451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec693ba2-0787-4573-b108-642395545fa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:22:00.203135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec693ba2-0787-4573-b108-642395545fa0 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:22:00.231019 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4308/8529] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:21:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:22:00.403143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ba368-228c-4c84-af17-f4dca1f51572 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:22:00.404931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-128ba368-228c-4c84-af17-f4dca1f51572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:00.495042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ba368-228c-4c84-af17-f4dca1f51572 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:22:00.505513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:22:00.506206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:00.506511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:00.506783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] PATH_INFO: `/v3/users/7a40d55acc0d452dae9ffadfa35fae88` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:00.514558 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a40d55acc0d452dae9ffadfa35fae88)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:22:00.517230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:22:00.618026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a40d55acc0d452dae9ffadfa35fae88'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:00.628647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:22:00.641542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a40d55acc0d452dae9ffadfa35fae88'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:00.642861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a40d55acc0d452dae9ffadfa35fae88'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:00.657894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a40d55acc0d452dae9ffadfa35fae88'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:00.680075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a40d55acc0d452dae9ffadfa35fae88'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:00.702170 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4222/8530] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:22:00 2022] DELETE /identity/v3/users/7a40d55acc0d452dae9ffadfa35fae88 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:22:00.708062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7e31c-19ab-4e14-b3c4-ed584c8d723a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:00.708907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7e31c-19ab-4e14-b3c4-ed584c8d723a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:00.709808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7e31c-19ab-4e14-b3c4-ed584c8d723a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:00.757844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e9b7e31c-19ab-4e14-b3c4-ed584c8d723a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:22:00.761315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e9b7e31c-19ab-4e14-b3c4-ed584c8d723a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:00.883665 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4309/8531] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:22:00.900551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:22:00.902187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:01.020755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28a09b-4dbb-4468-a5d2-6d41e2d367c0 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:22:01.029003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ce231-ff77-4ebe-a34f-9a2a592e2053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:22:01.030776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ce231-ff77-4ebe-a34f-9a2a592e2053 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:01.031328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ce231-ff77-4ebe-a34f-9a2a592e2053 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:01.031801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ce231-ff77-4ebe-a34f-9a2a592e2053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:01.051140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ce231-ff77-4ebe-a34f-9a2a592e2053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:22:01.052801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ce231-ff77-4ebe-a34f-9a2a592e2053 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:22:01.081264 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4223/8532] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:22:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:22:01.410059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7e31c-19ab-4e14-b3c4-ed584c8d723a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:22:01.411555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e9b7e31c-19ab-4e14-b3c4-ed584c8d723a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:01.436166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7e31c-19ab-4e14-b3c4-ed584c8d723a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:22:01.483804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:22:01.485880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:01.487526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:01.489091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] PATH_INFO: `/v3/projects/800b0894aabe47369a800b88dae3d10e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:01.494618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=800b0894aabe47369a800b88dae3d10e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:22:01.498995 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:22:01.570822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:22:01.625137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '800b0894aabe47369a800b88dae3d10e', 'request_id': 'req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:01.640752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '800b0894aabe47369a800b88dae3d10e', 'request_id': 'req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:01.651936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '800b0894aabe47369a800b88dae3d10e', 'request_id': 'req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:01.676463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] The token cache is being invalidate because project 800b0894aabe47369a800b88dae3d10e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:22:01.676835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:01.677890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:01.679986 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4310/8533] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:22:01 2022] DELETE /identity/v3/projects/800b0894aabe47369a800b88dae3d10e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:22:01.683818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ce231-ff77-4ebe-a34f-9a2a592e2053 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:22:01.697072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ce231-ff77-4ebe-a34f-9a2a592e2053 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:22:01.774978 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3e034-bed0-4454-82d0-b0537d6eee27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:22:01.775772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3e034-bed0-4454-82d0-b0537d6eee27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:01.775917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3e034-bed0-4454-82d0-b0537d6eee27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:01.776149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3e034-bed0-4454-82d0-b0537d6eee27 admin admin] PATH_INFO: `/v3/users/bb2753efc0794ab1a69fc22cd6d6e410` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:01.778033 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3e034-bed0-4454-82d0-b0537d6eee27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb2753efc0794ab1a69fc22cd6d6e410)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:22:01.780253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3e034-bed0-4454-82d0-b0537d6eee27 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:22:01.874901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d1a3e034-bed0-4454-82d0-b0537d6eee27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb2753efc0794ab1a69fc22cd6d6e410'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:01.907256 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d1a3e034-bed0-4454-82d0-b0537d6eee27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb2753efc0794ab1a69fc22cd6d6e410'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:01.908566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d1a3e034-bed0-4454-82d0-b0537d6eee27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb2753efc0794ab1a69fc22cd6d6e410'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:01.922719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d1a3e034-bed0-4454-82d0-b0537d6eee27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb2753efc0794ab1a69fc22cd6d6e410'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:01.930702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:22:01.940389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-d1a3e034-bed0-4454-82d0-b0537d6eee27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb2753efc0794ab1a69fc22cd6d6e410'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:01.950457 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4224/8534] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:22:01 2022] DELETE /identity/v3/users/bb2753efc0794ab1a69fc22cd6d6e410 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:22:02.492681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:22:02.494718 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:02.512761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfaf33-d64e-4f37-af9a-1b2acfbcccf1 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:22:02.536561 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235d1cf-5a45-4c03-96ee-78f616c07c59 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:02.541253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235d1cf-5a45-4c03-96ee-78f616c07c59 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:02.541664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235d1cf-5a45-4c03-96ee-78f616c07c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:02.589272 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-6235d1cf-5a45-4c03-96ee-78f616c07c59 None None] MFA Rules not processed for user `98434520e99e42099265bd81e88f1862`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:22:02.594690 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6235d1cf-5a45-4c03-96ee-78f616c07c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:02.626279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:22:02.627052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:02.627983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:02.628427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] PATH_INFO: `/v3/projects/95fe7cf631a648e6a49712582f0d64bf` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:02.631204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95fe7cf631a648e6a49712582f0d64bf)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:22:02.633384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:22:02.695319 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4225/8535] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:22:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:22:02.728343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95fe7cf631a648e6a49712582f0d64bf', 'request_id': 'req-ce6f3736-2675-4340-ada4-984171575492'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:02.776525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95fe7cf631a648e6a49712582f0d64bf', 'request_id': 'req-ce6f3736-2675-4340-ada4-984171575492'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:02.798233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95fe7cf631a648e6a49712582f0d64bf', 'request_id': 'req-ce6f3736-2675-4340-ada4-984171575492'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:02.817322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] The token cache is being invalidate because project 95fe7cf631a648e6a49712582f0d64bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:22:02.817740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:02.818581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:02.820304 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4311/8536] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:22:02 2022] DELETE /identity/v3/projects/95fe7cf631a648e6a49712582f0d64bf => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:22:02.824211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235d1cf-5a45-4c03-96ee-78f616c07c59 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:22:02.826734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6235d1cf-5a45-4c03-96ee-78f616c07c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:02.876278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235d1cf-5a45-4c03-96ee-78f616c07c59 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:22:02.992160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:22:02.993097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:02.993465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:02.993822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] PATH_INFO: `/v3/users/828c9cd414c24a2b9e2f1de713020d21` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:02.995103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=828c9cd414c24a2b9e2f1de713020d21)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:22:02.997240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:22:03.095236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '828c9cd414c24a2b9e2f1de713020d21'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:03.149868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '828c9cd414c24a2b9e2f1de713020d21'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:03.150783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '828c9cd414c24a2b9e2f1de713020d21'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:03.164781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '828c9cd414c24a2b9e2f1de713020d21'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:03.175004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '828c9cd414c24a2b9e2f1de713020d21'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:03.188939 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4226/8537] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:22:02 2022] DELETE /identity/v3/users/828c9cd414c24a2b9e2f1de713020d21 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:22:03.629860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:22:03.634198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:03.658185 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f3736-2675-4340-ada4-984171575492 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:22:03.723315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:22:03.747472 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:22:03.748214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:03.749163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:03.749547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] PATH_INFO: `/v3/projects/98726b2139ed475da0b0482cd0a23d38` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:03.751488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98726b2139ed475da0b0482cd0a23d38)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:22:03.754343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:22:03.817731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98726b2139ed475da0b0482cd0a23d38', 'request_id': 'req-a2c314bf-bb5b-4d2e-a772-a43474411fc8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:03.827711 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98726b2139ed475da0b0482cd0a23d38', 'request_id': 'req-a2c314bf-bb5b-4d2e-a772-a43474411fc8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:03.836484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98726b2139ed475da0b0482cd0a23d38', 'request_id': 'req-a2c314bf-bb5b-4d2e-a772-a43474411fc8'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:03.851372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] The token cache is being invalidate because project 98726b2139ed475da0b0482cd0a23d38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:22:03.851985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:03.853080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:22:03.855143 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4312/8538] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:22:03 2022] DELETE /identity/v3/projects/98726b2139ed475da0b0482cd0a23d38 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:22:13.179367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=80,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:22:13.847695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:22:15.133666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:22:15.133666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:15.198503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63866348-c055-466d-856b-7532e4fe1ec3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:22:15.247548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d57536-9bd0-49e3-b296-6b5dfac9ca3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:22:15.248160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d57536-9bd0-49e3-b296-6b5dfac9ca3c service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:15.248442 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d57536-9bd0-49e3-b296-6b5dfac9ca3c service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:15.248719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d57536-9bd0-49e3-b296-6b5dfac9ca3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:15.250476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-77d57536-9bd0-49e3-b296-6b5dfac9ca3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:15.312405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d57536-9bd0-49e3-b296-6b5dfac9ca3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:22:15.313868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d57536-9bd0-49e3-b296-6b5dfac9ca3c service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:22:15.323506 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4227/8539] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:22:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:22:16.245911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:22:16.247387 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:16.322823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c314bf-bb5b-4d2e-a772-a43474411fc8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:22:16.329609 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f58056-a813-49ef-8748-dbdaba9d9963 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:22:16.330360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f58056-a813-49ef-8748-dbdaba9d9963 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:16.330654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f58056-a813-49ef-8748-dbdaba9d9963 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:16.330933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f58056-a813-49ef-8748-dbdaba9d9963 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:16.333301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-90f58056-a813-49ef-8748-dbdaba9d9963 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:16.349746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f58056-a813-49ef-8748-dbdaba9d9963 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:22:16.351601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f58056-a813-49ef-8748-dbdaba9d9963 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:22:16.365457 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4313/8540] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:22:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:22:25.319140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=70 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:22:26.357775 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=32 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:22:51.334198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f74c9-2d1c-4b61-8947-4bf981bd6d93 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:51.334664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f74c9-2d1c-4b61-8947-4bf981bd6d93 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:51.334785 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f74c9-2d1c-4b61-8947-4bf981bd6d93 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:51.336122 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4228/8541] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 21:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 21:22:51.341490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd095e8f-c91a-429c-9fb4-86be3ba292a3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:51.341945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd095e8f-c91a-429c-9fb4-86be3ba292a3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:51.342065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd095e8f-c91a-429c-9fb4-86be3ba292a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:51.389081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fd095e8f-c91a-429c-9fb4-86be3ba292a3 None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:22:51.390296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fd095e8f-c91a-429c-9fb4-86be3ba292a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:51.396776 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4314/8542] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:22:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:22:51.409469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f74c9-2d1c-4b61-8947-4bf981bd6d93 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:22:51.411179 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9e4f74c9-2d1c-4b61-8947-4bf981bd6d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:22:51.427670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f74c9-2d1c-4b61-8947-4bf981bd6d93 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:22:51.431329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67863b1-7a6b-43f2-ace0-62468632eff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:22:51.431930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67863b1-7a6b-43f2-ace0-62468632eff4 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:22:51.432215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67863b1-7a6b-43f2-ace0-62468632eff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:22:51.432497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67863b1-7a6b-43f2-ace0-62468632eff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:22:51.440691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67863b1-7a6b-43f2-ace0-62468632eff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:22:51.442184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67863b1-7a6b-43f2-ace0-62468632eff4 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:22:51.450168 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4229/8543] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:22:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:23:01.386929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=13 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:23:01.394248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd095e8f-c91a-429c-9fb4-86be3ba292a3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:23:01.395634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fd095e8f-c91a-429c-9fb4-86be3ba292a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:23:01.413946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd095e8f-c91a-429c-9fb4-86be3ba292a3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:23:01.418821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd388953-1eeb-42a2-a320-2b0f0cdd4bfc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:23:01.419420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd388953-1eeb-42a2-a320-2b0f0cdd4bfc service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:23:01.419716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd388953-1eeb-42a2-a320-2b0f0cdd4bfc service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:23:01.420015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd388953-1eeb-42a2-a320-2b0f0cdd4bfc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:23:01.421769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dd388953-1eeb-42a2-a320-2b0f0cdd4bfc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:23:01.435361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd388953-1eeb-42a2-a320-2b0f0cdd4bfc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:23:01.436981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd388953-1eeb-42a2-a320-2b0f0cdd4bfc service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:23:01.446668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:23:01.449582 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4315/8544] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:23:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:23:11.442681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:23:29.420610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67863b1-7a6b-43f2-ace0-62468632eff4 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:23:29.435191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67863b1-7a6b-43f2-ace0-62468632eff4 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:23:29.440103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa9c0d-3545-4ea6-bdeb-f7e183c07a50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:23:29.440696 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa9c0d-3545-4ea6-bdeb-f7e183c07a50 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:23:29.441018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa9c0d-3545-4ea6-bdeb-f7e183c07a50 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:23:29.441339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa9c0d-3545-4ea6-bdeb-f7e183c07a50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:23:29.443015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9afa9c0d-3545-4ea6-bdeb-f7e183c07a50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:23:29.457109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa9c0d-3545-4ea6-bdeb-f7e183c07a50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:23:29.458676 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa9c0d-3545-4ea6-bdeb-f7e183c07a50 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:23:29.471402 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4230/8545] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:23:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:23:29.630148 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd388953-1eeb-42a2-a320-2b0f0cdd4bfc service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:23:29.642912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd388953-1eeb-42a2-a320-2b0f0cdd4bfc service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:23:29.646598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6447b-7c7c-41fa-8395-5ab74bfa4ecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:23:29.647231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6447b-7c7c-41fa-8395-5ab74bfa4ecc service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:23:29.647522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6447b-7c7c-41fa-8395-5ab74bfa4ecc service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:23:29.647839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6447b-7c7c-41fa-8395-5ab74bfa4ecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:23:29.649625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7ae6447b-7c7c-41fa-8395-5ab74bfa4ecc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:23:29.661519 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6447b-7c7c-41fa-8395-5ab74bfa4ecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:23:29.662953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6447b-7c7c-41fa-8395-5ab74bfa4ecc service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:23:29.673632 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4316/8546] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:23:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:23:29.755889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa9c0d-3545-4ea6-bdeb-f7e183c07a50 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:23:29.768468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa9c0d-3545-4ea6-bdeb-f7e183c07a50 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:23:29.772018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108e6e8-d7c2-44d3-af29-aea2bd055f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:23:29.772611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108e6e8-d7c2-44d3-af29-aea2bd055f10 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:23:29.772905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108e6e8-d7c2-44d3-af29-aea2bd055f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:23:29.773200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108e6e8-d7c2-44d3-af29-aea2bd055f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:23:29.774898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e108e6e8-d7c2-44d3-af29-aea2bd055f10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:23:29.847789 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108e6e8-d7c2-44d3-af29-aea2bd055f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:23:29.849266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108e6e8-d7c2-44d3-af29-aea2bd055f10 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:23:29.863828 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4231/8547] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:23:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:23:39.667407 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:23:39.857295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=40 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:23:52.087755 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9b5c1-da0e-40eb-a5de-469d239d131a None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:23:52.088301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9b5c1-da0e-40eb-a5de-469d239d131a None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:23:52.088469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9b5c1-da0e-40eb-a5de-469d239d131a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:23:52.110275 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-def9b5c1-da0e-40eb-a5de-469d239d131a None None] MFA Rules not processed for user `b5c96be8fbe1436bb3e84340ceda1917`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:23:52.111642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-def9b5c1-da0e-40eb-a5de-469d239d131a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:23:52.170336 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4317/8548] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:23:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:24:02.163022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=25 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:25:06.732874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108e6e8-d7c2-44d3-af29-aea2bd055f10 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:25:06.754077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108e6e8-d7c2-44d3-af29-aea2bd055f10 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:25:06.757732 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15633a4f-e97d-4882-ac01-03692f434344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:25:06.758329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15633a4f-e97d-4882-ac01-03692f434344 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:25:06.758614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15633a4f-e97d-4882-ac01-03692f434344 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:25:06.758890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15633a4f-e97d-4882-ac01-03692f434344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:25:06.760572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-15633a4f-e97d-4882-ac01-03692f434344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:25:06.773514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15633a4f-e97d-4882-ac01-03692f434344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:25:06.774937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15633a4f-e97d-4882-ac01-03692f434344 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:25:06.785288 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4232/8549] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:25:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:25:16.779282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=10 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:25:17.060076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9b5c1-da0e-40eb-a5de-469d239d131a None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:25:17.061574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-def9b5c1-da0e-40eb-a5de-469d239d131a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:25:17.143461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9b5c1-da0e-40eb-a5de-469d239d131a None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:25:17.147081 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c09d4-4883-4d91-9faa-680a9443521c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:25:17.147691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c09d4-4883-4d91-9faa-680a9443521c service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:25:17.147980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c09d4-4883-4d91-9faa-680a9443521c service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:25:17.148266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c09d4-4883-4d91-9faa-680a9443521c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:25:17.150080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b54c09d4-4883-4d91-9faa-680a9443521c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:25:17.167919 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c09d4-4883-4d91-9faa-680a9443521c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:25:17.169458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c09d4-4883-4d91-9faa-680a9443521c service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:25:17.193483 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4318/8550] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:25:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:25:17.412796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15633a4f-e97d-4882-ac01-03692f434344 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:25:17.424522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15633a4f-e97d-4882-ac01-03692f434344 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:25:17.428354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c2e18-f3e4-4ada-9351-9bca0dffa215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:25:17.428995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c2e18-f3e4-4ada-9351-9bca0dffa215 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:25:17.429288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c2e18-f3e4-4ada-9351-9bca0dffa215 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:25:17.429571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c2e18-f3e4-4ada-9351-9bca0dffa215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:25:17.431551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d82c2e18-f3e4-4ada-9351-9bca0dffa215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:25:17.448745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c2e18-f3e4-4ada-9351-9bca0dffa215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:25:17.450219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c2e18-f3e4-4ada-9351-9bca0dffa215 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:25:17.464948 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4233/8551] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:25:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:25:27.190891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=39 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:25:27.457211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:26:27.955045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c09d4-4883-4d91-9faa-680a9443521c service glance] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:27.967192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c09d4-4883-4d91-9faa-680a9443521c service glance] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:27.970752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff34d5b-c4ea-45b3-afe2-9a5f518f6099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:27.971446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff34d5b-c4ea-45b3-afe2-9a5f518f6099 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:27.971733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff34d5b-c4ea-45b3-afe2-9a5f518f6099 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:27.972009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff34d5b-c4ea-45b3-afe2-9a5f518f6099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:27.973737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bff34d5b-c4ea-45b3-afe2-9a5f518f6099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:27.986209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff34d5b-c4ea-45b3-afe2-9a5f518f6099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:27.987630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff34d5b-c4ea-45b3-afe2-9a5f518f6099 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:27.997914 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4319/8552] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:26:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:26:37.991999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:26:44.051904 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c2e18-f3e4-4ada-9351-9bca0dffa215 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:44.054245 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d82c2e18-f3e4-4ada-9351-9bca0dffa215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:44.099603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c2e18-f3e4-4ada-9351-9bca0dffa215 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:44.105434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae022c5-248d-4b34-8b4b-722ee36404bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:44.106043 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae022c5-248d-4b34-8b4b-722ee36404bb service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:44.106329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae022c5-248d-4b34-8b4b-722ee36404bb service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:44.106609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae022c5-248d-4b34-8b4b-722ee36404bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:44.108383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9ae022c5-248d-4b34-8b4b-722ee36404bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:44.126708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae022c5-248d-4b34-8b4b-722ee36404bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:44.128504 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae022c5-248d-4b34-8b4b-722ee36404bb service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:44.145059 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4234/8553] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:26:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:26:44.589858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff34d5b-c4ea-45b3-afe2-9a5f518f6099 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:44.591891 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bff34d5b-c4ea-45b3-afe2-9a5f518f6099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:44.626278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff34d5b-c4ea-45b3-afe2-9a5f518f6099 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:44.632054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23686a33-a3e5-4192-9603-8db2c7f51ac2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:44.632764 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23686a33-a3e5-4192-9603-8db2c7f51ac2 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:44.633085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23686a33-a3e5-4192-9603-8db2c7f51ac2 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:44.633432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23686a33-a3e5-4192-9603-8db2c7f51ac2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:44.635161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-23686a33-a3e5-4192-9603-8db2c7f51ac2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:44.667642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23686a33-a3e5-4192-9603-8db2c7f51ac2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:44.669294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23686a33-a3e5-4192-9603-8db2c7f51ac2 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:44.688940 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4320/8554] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:26:44 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:26:47.443980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae022c5-248d-4b34-8b4b-722ee36404bb service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:47.485818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae022c5-248d-4b34-8b4b-722ee36404bb service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:47.492984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788af2ee-dd0e-4c8b-8edd-ea3157122f72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:47.493714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788af2ee-dd0e-4c8b-8edd-ea3157122f72 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:47.493995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788af2ee-dd0e-4c8b-8edd-ea3157122f72 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:47.494365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788af2ee-dd0e-4c8b-8edd-ea3157122f72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:47.496389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-788af2ee-dd0e-4c8b-8edd-ea3157122f72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:47.516834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788af2ee-dd0e-4c8b-8edd-ea3157122f72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:47.518762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788af2ee-dd0e-4c8b-8edd-ea3157122f72 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:47.537236 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4235/8555] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:26:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:26:48.772533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23686a33-a3e5-4192-9603-8db2c7f51ac2 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:48.774007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-23686a33-a3e5-4192-9603-8db2c7f51ac2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:48.855981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23686a33-a3e5-4192-9603-8db2c7f51ac2 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:48.860832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:48.861492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:48.861776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:48.862055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] PATH_INFO: `/v3/users/b5c96be8fbe1436bb3e84340ceda1917` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:48.863162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5c96be8fbe1436bb3e84340ceda1917)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:48.865125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:48.912427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5c96be8fbe1436bb3e84340ceda1917'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:48.924974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5c96be8fbe1436bb3e84340ceda1917'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:48.932220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5c96be8fbe1436bb3e84340ceda1917'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:48.932834 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5c96be8fbe1436bb3e84340ceda1917'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:48.933481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5c96be8fbe1436bb3e84340ceda1917'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:48.945899 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4321/8556] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:26:48 2022] DELETE /identity/v3/users/b5c96be8fbe1436bb3e84340ceda1917 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:26:48.950568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ab0ed-6fac-4715-b2fd-bae6247f5e98 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:48.951107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ab0ed-6fac-4715-b2fd-bae6247f5e98 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:48.951107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ab0ed-6fac-4715-b2fd-bae6247f5e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:48.971336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fc4ab0ed-6fac-4715-b2fd-bae6247f5e98 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:26:48.972875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fc4ab0ed-6fac-4715-b2fd-bae6247f5e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:49.031678 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4236/8557] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:26:49.041272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:49.042733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:49.052796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:26:49.120244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbf4f5-b154-48e6-8b37-b7ed090890a4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:49.124488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ef957-5826-42cf-b0cf-ac74a4b42d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:49.125129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ef957-5826-42cf-b0cf-ac74a4b42d9a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:49.125420 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ef957-5826-42cf-b0cf-ac74a4b42d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:49.125699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ef957-5826-42cf-b0cf-ac74a4b42d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:49.135258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ef957-5826-42cf-b0cf-ac74a4b42d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:49.136543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ef957-5826-42cf-b0cf-ac74a4b42d9a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:49.146089 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4322/8558] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:26:49.317141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ab0ed-6fac-4715-b2fd-bae6247f5e98 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:49.318640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fc4ab0ed-6fac-4715-b2fd-bae6247f5e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:49.334742 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ab0ed-6fac-4715-b2fd-bae6247f5e98 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:49.340948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:49.341610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:49.342509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:49.342844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] PATH_INFO: `/v3/projects/046a6abde8c84a498c1c063a1c908b42` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:49.343897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=046a6abde8c84a498c1c063a1c908b42)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:49.345989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:49.393061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046a6abde8c84a498c1c063a1c908b42', 'request_id': 'req-a264963f-e36f-4946-ab7b-28894458f3d4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.400440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046a6abde8c84a498c1c063a1c908b42', 'request_id': 'req-a264963f-e36f-4946-ab7b-28894458f3d4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.407085 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046a6abde8c84a498c1c063a1c908b42', 'request_id': 'req-a264963f-e36f-4946-ab7b-28894458f3d4'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.417493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] The token cache is being invalidate because project 046a6abde8c84a498c1c063a1c908b42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:26:49.417787 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.418533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.420637 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4237/8559] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:26:49 2022] DELETE /identity/v3/projects/046a6abde8c84a498c1c063a1c908b42 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:26:49.424552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ef957-5826-42cf-b0cf-ac74a4b42d9a service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:49.434833 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ef957-5826-42cf-b0cf-ac74a4b42d9a service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:49.496961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24ed48-57eb-4ee2-9c35-a1692a5c4412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:49.497613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24ed48-57eb-4ee2-9c35-a1692a5c4412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:49.497898 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24ed48-57eb-4ee2-9c35-a1692a5c4412 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:49.498182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24ed48-57eb-4ee2-9c35-a1692a5c4412 admin admin] PATH_INFO: `/v3/users/01ba5e445af242a69db93175b6d44fcd` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:49.512013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24ed48-57eb-4ee2-9c35-a1692a5c4412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01ba5e445af242a69db93175b6d44fcd)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:49.514023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24ed48-57eb-4ee2-9c35-a1692a5c4412 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:49.559819 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6a24ed48-57eb-4ee2-9c35-a1692a5c4412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01ba5e445af242a69db93175b6d44fcd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.571604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6a24ed48-57eb-4ee2-9c35-a1692a5c4412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01ba5e445af242a69db93175b6d44fcd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.579221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6a24ed48-57eb-4ee2-9c35-a1692a5c4412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01ba5e445af242a69db93175b6d44fcd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.580235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6a24ed48-57eb-4ee2-9c35-a1692a5c4412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01ba5e445af242a69db93175b6d44fcd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.580948 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6a24ed48-57eb-4ee2-9c35-a1692a5c4412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01ba5e445af242a69db93175b6d44fcd'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.594051 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4323/8560] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:26:49 2022] DELETE /identity/v3/users/01ba5e445af242a69db93175b6d44fcd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:26:49.762319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:49.764221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:49.780744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264963f-e36f-4946-ab7b-28894458f3d4 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:49.794146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:26:49.842484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73718a17-74f2-40a2-80ba-c9b9c11eddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:49.843071 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73718a17-74f2-40a2-80ba-c9b9c11eddf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:49.843356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73718a17-74f2-40a2-80ba-c9b9c11eddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:49.843633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73718a17-74f2-40a2-80ba-c9b9c11eddf5 admin admin] PATH_INFO: `/v3/projects/4c8013e1e6804708b2991d2dcd829154` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:49.854363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73718a17-74f2-40a2-80ba-c9b9c11eddf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c8013e1e6804708b2991d2dcd829154)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:49.856285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73718a17-74f2-40a2-80ba-c9b9c11eddf5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:49.901835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-73718a17-74f2-40a2-80ba-c9b9c11eddf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c8013e1e6804708b2991d2dcd829154', 'request_id': 'req-73718a17-74f2-40a2-80ba-c9b9c11eddf5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.909616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-73718a17-74f2-40a2-80ba-c9b9c11eddf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c8013e1e6804708b2991d2dcd829154', 'request_id': 'req-73718a17-74f2-40a2-80ba-c9b9c11eddf5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.917724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-73718a17-74f2-40a2-80ba-c9b9c11eddf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c8013e1e6804708b2991d2dcd829154', 'request_id': 'req-73718a17-74f2-40a2-80ba-c9b9c11eddf5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.927872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-73718a17-74f2-40a2-80ba-c9b9c11eddf5 admin admin] The token cache is being invalidate because project 4c8013e1e6804708b2991d2dcd829154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:26:49.928163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-73718a17-74f2-40a2-80ba-c9b9c11eddf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.928804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-73718a17-74f2-40a2-80ba-c9b9c11eddf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:26:49.930298 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4238/8561] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:26:49 2022] DELETE /identity/v3/projects/4c8013e1e6804708b2991d2dcd829154 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:26:49.935304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca226-91f9-4852-ac7c-46d38e9d49c3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:49.935880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca226-91f9-4852-ac7c-46d38e9d49c3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:49.935880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca226-91f9-4852-ac7c-46d38e9d49c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:49.944546 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:26:49.956507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ca2ca226-91f9-4852-ac7c-46d38e9d49c3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:26:49.957957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ca2ca226-91f9-4852-ac7c-46d38e9d49c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:50.020397 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4324/8562] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:26:50.029905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1caa75-5899-4568-a836-560595a4c4bb None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:50.030389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1caa75-5899-4568-a836-560595a4c4bb None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:50.030496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1caa75-5899-4568-a836-560595a4c4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:50.050032 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ea1caa75-5899-4568-a836-560595a4c4bb None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:26:50.051257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ea1caa75-5899-4568-a836-560595a4c4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:50.059203 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4239/8563] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:26:50.063857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ca226-91f9-4852-ac7c-46d38e9d49c3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:50.073701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ca226-91f9-4852-ac7c-46d38e9d49c3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:50.079268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c7106-b6fa-45c0-8d17-1e6887119204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:50.079867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c7106-b6fa-45c0-8d17-1e6887119204 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:50.080154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c7106-b6fa-45c0-8d17-1e6887119204 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:50.080431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c7106-b6fa-45c0-8d17-1e6887119204 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:50.081110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c7106-b6fa-45c0-8d17-1e6887119204 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:26:50.081417 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c7106-b6fa-45c0-8d17-1e6887119204 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:50.083013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c7106-b6fa-45c0-8d17-1e6887119204 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:50.094505 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4325/8564] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:26:50.098538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1caa75-5899-4568-a836-560595a4c4bb None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:50.107827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1caa75-5899-4568-a836-560595a4c4bb None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:50.112766 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431b77b-332f-4ce6-8f1e-d9f4888cc3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:50.113402 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431b77b-332f-4ce6-8f1e-d9f4888cc3c5 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:50.113691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431b77b-332f-4ce6-8f1e-d9f4888cc3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:50.113968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431b77b-332f-4ce6-8f1e-d9f4888cc3c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:50.114600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431b77b-332f-4ce6-8f1e-d9f4888cc3c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:50.116487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431b77b-332f-4ce6-8f1e-d9f4888cc3c5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:50.145698 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4240/8565] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:26:50 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:26:50.149949 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c7106-b6fa-45c0-8d17-1e6887119204 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:50.160327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c7106-b6fa-45c0-8d17-1e6887119204 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:50.215985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af22ad-a0f3-4d2a-bae5-33c357ef1c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:50.216592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af22ad-a0f3-4d2a-bae5-33c357ef1c23 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:50.216877 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af22ad-a0f3-4d2a-bae5-33c357ef1c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:50.217192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af22ad-a0f3-4d2a-bae5-33c357ef1c23 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:50.217831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af22ad-a0f3-4d2a-bae5-33c357ef1c23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:50.219759 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af22ad-a0f3-4d2a-bae5-33c357ef1c23 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:50.240390 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4326/8566] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:26:50 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:26:50.244675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431b77b-332f-4ce6-8f1e-d9f4888cc3c5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:50.255424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431b77b-332f-4ce6-8f1e-d9f4888cc3c5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:50.260015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc90d8-7704-4092-90ea-bc92c55bc9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:50.260610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc90d8-7704-4092-90ea-bc92c55bc9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:50.260915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc90d8-7704-4092-90ea-bc92c55bc9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:50.261204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc90d8-7704-4092-90ea-bc92c55bc9d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:50.261845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc90d8-7704-4092-90ea-bc92c55bc9d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:50.263426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc90d8-7704-4092-90ea-bc92c55bc9d5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:50.274434 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4241/8567] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:26:50.278235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af22ad-a0f3-4d2a-bae5-33c357ef1c23 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:50.288163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af22ad-a0f3-4d2a-bae5-33c357ef1c23 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:50.292753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5e574-e65e-4b90-a782-08cdb96135a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:50.294014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5e574-e65e-4b90-a782-08cdb96135a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:50.294301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5e574-e65e-4b90-a782-08cdb96135a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:50.294576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5e574-e65e-4b90-a782-08cdb96135a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:50.295268 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5e574-e65e-4b90-a782-08cdb96135a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:50.296911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5e574-e65e-4b90-a782-08cdb96135a3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:50.307250 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4327/8568] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:26:50.311683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc90d8-7704-4092-90ea-bc92c55bc9d5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:50.321525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc90d8-7704-4092-90ea-bc92c55bc9d5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:50.326050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645d509-49c1-4508-8a53-b422e68d38fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:50.327293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645d509-49c1-4508-8a53-b422e68d38fc admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:50.327574 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645d509-49c1-4508-8a53-b422e68d38fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:50.327851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645d509-49c1-4508-8a53-b422e68d38fc admin admin] PATH_INFO: `/v3/projects/ef9efdb2b0cd4585bb5d48040c46cdf0/users/4f8e7d53f6ce49c6ac44ade281e1309e/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:50.342473 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645d509-49c1-4508-8a53-b422e68d38fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9efdb2b0cd4585bb5d48040c46cdf0, user_id=4f8e7d53f6ce49c6ac44ade281e1309e, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:50.345218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645d509-49c1-4508-8a53-b422e68d38fc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:50.360148 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4242/8569] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:26:50 2022] PUT /identity/v3/projects/ef9efdb2b0cd4585bb5d48040c46cdf0/users/4f8e7d53f6ce49c6ac44ade281e1309e/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:26:50.365375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1fd4c-fa04-4337-9d91-43f677584ea4 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:50.365667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1fd4c-fa04-4337-9d91-43f677584ea4 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:50.365967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1fd4c-fa04-4337-9d91-43f677584ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:50.385332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3ff1fd4c-fa04-4337-9d91-43f677584ea4 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:26:50.386534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3ff1fd4c-fa04-4337-9d91-43f677584ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:50.444026 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4328/8570] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:26:50.453779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645d509-49c1-4508-8a53-b422e68d38fc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:50.455568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6645d509-49c1-4508-8a53-b422e68d38fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:50.491131 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:26:50.518042 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645d509-49c1-4508-8a53-b422e68d38fc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:50.521725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540542e-a0af-4200-bf15-ebf7d9d53710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:50.522324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540542e-a0af-4200-bf15-ebf7d9d53710 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:50.522649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540542e-a0af-4200-bf15-ebf7d9d53710 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:50.522923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540542e-a0af-4200-bf15-ebf7d9d53710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:50.531963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540542e-a0af-4200-bf15-ebf7d9d53710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:50.533413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540542e-a0af-4200-bf15-ebf7d9d53710 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:50.542795 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4243/8571] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:26:51.146248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87be4cf-724f-4518-b9b9-efd198f701a7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:51.146768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87be4cf-724f-4518-b9b9-efd198f701a7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:51.146879 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87be4cf-724f-4518-b9b9-efd198f701a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:51.168699 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:26:51.177661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e87be4cf-724f-4518-b9b9-efd198f701a7 None None] MFA Rules not processed for user `4f8e7d53f6ce49c6ac44ade281e1309e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:26:51.179151 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e87be4cf-724f-4518-b9b9-efd198f701a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:51.246177 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4329/8572] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:26:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:26:51.250994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540542e-a0af-4200-bf15-ebf7d9d53710 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:51.260662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540542e-a0af-4200-bf15-ebf7d9d53710 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:51.265848 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2dc01-e68a-4775-96e2-007a58d3224e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:51.266444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2dc01-e68a-4775-96e2-007a58d3224e admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:51.266729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2dc01-e68a-4775-96e2-007a58d3224e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:51.267005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2dc01-e68a-4775-96e2-007a58d3224e admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:51.267639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2dc01-e68a-4775-96e2-007a58d3224e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:51.269588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2dc01-e68a-4775-96e2-007a58d3224e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:51.297834 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4244/8573] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:26:51 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:26:51.301732 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87be4cf-724f-4518-b9b9-efd198f701a7 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:51.313005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87be4cf-724f-4518-b9b9-efd198f701a7 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:51.372358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6f4a9-e43b-4aa6-900b-f4c8c42c765e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:51.373040 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6f4a9-e43b-4aa6-900b-f4c8c42c765e admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:51.373336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6f4a9-e43b-4aa6-900b-f4c8c42c765e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:51.373620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6f4a9-e43b-4aa6-900b-f4c8c42c765e admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:51.374250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6f4a9-e43b-4aa6-900b-f4c8c42c765e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:51.376201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6f4a9-e43b-4aa6-900b-f4c8c42c765e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:51.397624 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4330/8574] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:26:51 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:26:51.401607 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2dc01-e68a-4775-96e2-007a58d3224e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:51.412505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2dc01-e68a-4775-96e2-007a58d3224e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:51.417120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21601d1a-2703-430a-9be8-b419372ca299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:51.417735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21601d1a-2703-430a-9be8-b419372ca299 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:51.418004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21601d1a-2703-430a-9be8-b419372ca299 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:51.418282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21601d1a-2703-430a-9be8-b419372ca299 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:51.418972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21601d1a-2703-430a-9be8-b419372ca299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:51.420552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21601d1a-2703-430a-9be8-b419372ca299 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:51.430949 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4245/8575] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:26:51.434944 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6f4a9-e43b-4aa6-900b-f4c8c42c765e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:51.445172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6f4a9-e43b-4aa6-900b-f4c8c42c765e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:51.449831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad0214-af3d-4147-a051-9c06fdb630bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:51.450516 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad0214-af3d-4147-a051-9c06fdb630bd admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:51.450805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad0214-af3d-4147-a051-9c06fdb630bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:51.451088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad0214-af3d-4147-a051-9c06fdb630bd admin admin] PATH_INFO: `/v3/projects/8aafdd7e20f04d85b7d0134d8bb00942/users/c16bde17ce494d04be9309030582270b/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:51.467510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad0214-af3d-4147-a051-9c06fdb630bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aafdd7e20f04d85b7d0134d8bb00942, user_id=c16bde17ce494d04be9309030582270b, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:51.470267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad0214-af3d-4147-a051-9c06fdb630bd admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:51.483278 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4331/8576] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:26:51 2022] PUT /identity/v3/projects/8aafdd7e20f04d85b7d0134d8bb00942/users/c16bde17ce494d04be9309030582270b/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:26:51.487491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21601d1a-2703-430a-9be8-b419372ca299 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:51.497427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21601d1a-2703-430a-9be8-b419372ca299 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:51.553252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2409d1-a615-41ef-a245-615d897232f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:51.553913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2409d1-a615-41ef-a245-615d897232f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:51.554288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2409d1-a615-41ef-a245-615d897232f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:51.554441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2409d1-a615-41ef-a245-615d897232f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:51.555080 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2409d1-a615-41ef-a245-615d897232f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:51.556683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2409d1-a615-41ef-a245-615d897232f1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:51.567034 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4246/8577] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:26:51.571254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad0214-af3d-4147-a051-9c06fdb630bd admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:51.582137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad0214-af3d-4147-a051-9c06fdb630bd admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:51.586808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41692f5-6fe0-40ea-b195-29eb3e4dac0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:51.587489 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41692f5-6fe0-40ea-b195-29eb3e4dac0a admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:51.587783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41692f5-6fe0-40ea-b195-29eb3e4dac0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:51.588091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41692f5-6fe0-40ea-b195-29eb3e4dac0a admin admin] PATH_INFO: `/v3/projects/8aafdd7e20f04d85b7d0134d8bb00942/users/c16bde17ce494d04be9309030582270b/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:51.590171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41692f5-6fe0-40ea-b195-29eb3e4dac0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aafdd7e20f04d85b7d0134d8bb00942, user_id=c16bde17ce494d04be9309030582270b, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:51.593096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41692f5-6fe0-40ea-b195-29eb3e4dac0a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:51.603413 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4332/8578] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:26:51 2022] PUT /identity/v3/projects/8aafdd7e20f04d85b7d0134d8bb00942/users/c16bde17ce494d04be9309030582270b/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:26:52.204023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06145a-13b6-40e0-9983-51de4e84ba77 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:52.204318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06145a-13b6-40e0-9983-51de4e84ba77 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:52.204616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06145a-13b6-40e0-9983-51de4e84ba77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:52.234174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8a06145a-13b6-40e0-9983-51de4e84ba77 None None] MFA Rules not processed for user `c16bde17ce494d04be9309030582270b`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:26:52.235623 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8a06145a-13b6-40e0-9983-51de4e84ba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:52.306811 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4247/8579] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:26:52.318448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929f946-68a7-4941-aab4-3e8adf4c5acf None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:52.318965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929f946-68a7-4941-aab4-3e8adf4c5acf None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:52.319101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929f946-68a7-4941-aab4-3e8adf4c5acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:52.337966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2929f946-68a7-4941-aab4-3e8adf4c5acf None None] MFA Rules not processed for user `4f8e7d53f6ce49c6ac44ade281e1309e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:26:52.339203 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2929f946-68a7-4941-aab4-3e8adf4c5acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:52.390827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:26:52.401084 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4333/8580] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:26:52.413398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06145a-13b6-40e0-9983-51de4e84ba77 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:52.414788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8a06145a-13b6-40e0-9983-51de4e84ba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:52.442796 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:26:52.492569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06145a-13b6-40e0-9983-51de4e84ba77 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:52.497982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1141f-5698-4ef3-9f6c-f77d374db8ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:52.498581 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1141f-5698-4ef3-9f6c-f77d374db8ba service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:52.498864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1141f-5698-4ef3-9f6c-f77d374db8ba service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:52.499137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1141f-5698-4ef3-9f6c-f77d374db8ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:52.507147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1141f-5698-4ef3-9f6c-f77d374db8ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:52.508725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1141f-5698-4ef3-9f6c-f77d374db8ba service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:52.517419 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4248/8581] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:26:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:26:54.072538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929f946-68a7-4941-aab4-3e8adf4c5acf None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:26:54.073980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2929f946-68a7-4941-aab4-3e8adf4c5acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:54.148484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929f946-68a7-4941-aab4-3e8adf4c5acf None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:26:54.153493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f433ea-46d4-4f97-9a9a-e226b471a49d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:26:54.154104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f433ea-46d4-4f97-9a9a-e226b471a49d service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:26:54.155047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f433ea-46d4-4f97-9a9a-e226b471a49d service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:26:54.155334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f433ea-46d4-4f97-9a9a-e226b471a49d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:26:54.157153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-37f433ea-46d4-4f97-9a9a-e226b471a49d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:26:54.172087 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f433ea-46d4-4f97-9a9a-e226b471a49d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:26:54.173721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f433ea-46d4-4f97-9a9a-e226b471a49d service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:26:54.186352 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4334/8582] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:26:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:27:00.296928 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1141f-5698-4ef3-9f6c-f77d374db8ba service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:00.298344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9be1141f-5698-4ef3-9f6c-f77d374db8ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:00.316886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1141f-5698-4ef3-9f6c-f77d374db8ba service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:00.322591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d340b-5073-4b1b-bd50-bb246593db9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:00.323195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d340b-5073-4b1b-bd50-bb246593db9e service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:00.323478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d340b-5073-4b1b-bd50-bb246593db9e service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:00.323752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d340b-5073-4b1b-bd50-bb246593db9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:00.325510 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-237d340b-5073-4b1b-bd50-bb246593db9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:00.385089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d340b-5073-4b1b-bd50-bb246593db9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:00.386701 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d340b-5073-4b1b-bd50-bb246593db9e service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:00.396521 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4249/8583] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:27:00.667810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f433ea-46d4-4f97-9a9a-e226b471a49d service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:00.669315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-37f433ea-46d4-4f97-9a9a-e226b471a49d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:00.728225 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f433ea-46d4-4f97-9a9a-e226b471a49d service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:00.731804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c4b51-fdd3-458b-8e7c-376a7d98b028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:00.732988 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c4b51-fdd3-458b-8e7c-376a7d98b028 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:00.733271 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c4b51-fdd3-458b-8e7c-376a7d98b028 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:00.733545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c4b51-fdd3-458b-8e7c-376a7d98b028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:00.735481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7a4c4b51-fdd3-458b-8e7c-376a7d98b028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:00.799073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c4b51-fdd3-458b-8e7c-376a7d98b028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:00.800550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c4b51-fdd3-458b-8e7c-376a7d98b028 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:00.810055 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4335/8584] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:27:03.804837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d340b-5073-4b1b-bd50-bb246593db9e service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:03.830312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d340b-5073-4b1b-bd50-bb246593db9e service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:03.855281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725b252-5836-40ed-9000-b3c8748facbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:03.855926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725b252-5836-40ed-9000-b3c8748facbf service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:03.856430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725b252-5836-40ed-9000-b3c8748facbf service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:03.857343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725b252-5836-40ed-9000-b3c8748facbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:03.859439 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5725b252-5836-40ed-9000-b3c8748facbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:03.879793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725b252-5836-40ed-9000-b3c8748facbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:03.882871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725b252-5836-40ed-9000-b3c8748facbf service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:03.912035 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4250/8585] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:27:04.216221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c4b51-fdd3-458b-8e7c-376a7d98b028 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:04.218381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7a4c4b51-fdd3-458b-8e7c-376a7d98b028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:04.239626 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c4b51-fdd3-458b-8e7c-376a7d98b028 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:04.245979 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33239ded-c034-4f70-9ec4-4cb1c4bc2bd0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:04.246625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33239ded-c034-4f70-9ec4-4cb1c4bc2bd0 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:04.246914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33239ded-c034-4f70-9ec4-4cb1c4bc2bd0 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:04.247193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33239ded-c034-4f70-9ec4-4cb1c4bc2bd0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:04.249928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-33239ded-c034-4f70-9ec4-4cb1c4bc2bd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:04.265771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33239ded-c034-4f70-9ec4-4cb1c4bc2bd0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:04.267413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33239ded-c034-4f70-9ec4-4cb1c4bc2bd0 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:04.294460 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4336/8586] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:27:07.843512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725b252-5836-40ed-9000-b3c8748facbf service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:07.847035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5725b252-5836-40ed-9000-b3c8748facbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:07.864417 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725b252-5836-40ed-9000-b3c8748facbf service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:07.869337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:07.869968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:07.870255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:07.870564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] PATH_INFO: `/v3/users/98434520e99e42099265bd81e88f1862` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:07.871688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] RBAC: Authorizing `identity:delete_user(user_id=98434520e99e42099265bd81e88f1862)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:07.873651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:07.919915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98434520e99e42099265bd81e88f1862'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:07.931866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98434520e99e42099265bd81e88f1862'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:07.932537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98434520e99e42099265bd81e88f1862'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:07.939560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98434520e99e42099265bd81e88f1862'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:07.946777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98434520e99e42099265bd81e88f1862'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:07.953335 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4251/8587] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:27:07 2022] DELETE /identity/v3/users/98434520e99e42099265bd81e88f1862 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:08.130568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33239ded-c034-4f70-9ec4-4cb1c4bc2bd0 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:08.143608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33239ded-c034-4f70-9ec4-4cb1c4bc2bd0 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:08.164117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:08.201358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:08.202031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:08.202320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:08.202601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] PATH_INFO: `/v3/projects/6d0bae9fc4a443f1bdbd9d136784516a` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:08.203756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d0bae9fc4a443f1bdbd9d136784516a)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:08.205746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:08.250758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d0bae9fc4a443f1bdbd9d136784516a', 'request_id': 'req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.257806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d0bae9fc4a443f1bdbd9d136784516a', 'request_id': 'req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.264438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d0bae9fc4a443f1bdbd9d136784516a', 'request_id': 'req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.275180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] The token cache is being invalidate because project 6d0bae9fc4a443f1bdbd9d136784516a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:27:08.275474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.276146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.277669 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4337/8588] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:27:08 2022] DELETE /identity/v3/projects/6d0bae9fc4a443f1bdbd9d136784516a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:08.281901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:08.283376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:08.296642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1fee1-aa36-4a66-9fd3-26c9b21ad33f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:08.300211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:08.355737 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de55565-d745-4a59-b0af-e10d15d19c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:08.356427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de55565-d745-4a59-b0af-e10d15d19c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:08.356725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de55565-d745-4a59-b0af-e10d15d19c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:08.357537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de55565-d745-4a59-b0af-e10d15d19c69 admin admin] PATH_INFO: `/v3/users/23cb2d04eeac461b870d4f6d05c26f3c` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:08.358683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de55565-d745-4a59-b0af-e10d15d19c69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23cb2d04eeac461b870d4f6d05c26f3c)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:08.360861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de55565-d745-4a59-b0af-e10d15d19c69 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:08.408955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4de55565-d745-4a59-b0af-e10d15d19c69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23cb2d04eeac461b870d4f6d05c26f3c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.422186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4de55565-d745-4a59-b0af-e10d15d19c69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23cb2d04eeac461b870d4f6d05c26f3c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.423006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4de55565-d745-4a59-b0af-e10d15d19c69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23cb2d04eeac461b870d4f6d05c26f3c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.431702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4de55565-d745-4a59-b0af-e10d15d19c69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23cb2d04eeac461b870d4f6d05c26f3c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.439715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-4de55565-d745-4a59-b0af-e10d15d19c69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23cb2d04eeac461b870d4f6d05c26f3c'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.445577 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4252/8589] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:27:08 2022] DELETE /identity/v3/users/23cb2d04eeac461b870d4f6d05c26f3c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:27:08.614920 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:08.616352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:08.630281 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba7699-ef18-478d-a3dc-458cbdf7ce9a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:08.689134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd578172-7e7e-4963-bd7c-717ee3385ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:08.689758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd578172-7e7e-4963-bd7c-717ee3385ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:08.690047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd578172-7e7e-4963-bd7c-717ee3385ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:08.690324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd578172-7e7e-4963-bd7c-717ee3385ab5 admin admin] PATH_INFO: `/v3/projects/a10f513ed83941c8a6e7ca5ef6602ce9` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:08.691433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd578172-7e7e-4963-bd7c-717ee3385ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a10f513ed83941c8a6e7ca5ef6602ce9)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:08.693401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd578172-7e7e-4963-bd7c-717ee3385ab5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:08.744337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bd578172-7e7e-4963-bd7c-717ee3385ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a10f513ed83941c8a6e7ca5ef6602ce9', 'request_id': 'req-bd578172-7e7e-4963-bd7c-717ee3385ab5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.751865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bd578172-7e7e-4963-bd7c-717ee3385ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a10f513ed83941c8a6e7ca5ef6602ce9', 'request_id': 'req-bd578172-7e7e-4963-bd7c-717ee3385ab5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.753368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:08.759270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bd578172-7e7e-4963-bd7c-717ee3385ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a10f513ed83941c8a6e7ca5ef6602ce9', 'request_id': 'req-bd578172-7e7e-4963-bd7c-717ee3385ab5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.772568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bd578172-7e7e-4963-bd7c-717ee3385ab5 admin admin] The token cache is being invalidate because project a10f513ed83941c8a6e7ca5ef6602ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:27:08.772860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bd578172-7e7e-4963-bd7c-717ee3385ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.773583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-bd578172-7e7e-4963-bd7c-717ee3385ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:08.775121 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4338/8590] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:27:08 2022] DELETE /identity/v3/projects/a10f513ed83941c8a6e7ca5ef6602ce9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:08.780503 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77974470-2971-47ec-8432-31c3f3693413 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:08.780791 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77974470-2971-47ec-8432-31c3f3693413 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:08.781107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77974470-2971-47ec-8432-31c3f3693413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:08.810795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-77974470-2971-47ec-8432-31c3f3693413 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:08.812007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-77974470-2971-47ec-8432-31c3f3693413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:08.877411 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4253/8591] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:27:08.885284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ddd0b-bdbd-45a4-89ef-b106e30bb115 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:08.885645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ddd0b-bdbd-45a4-89ef-b106e30bb115 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:08.885997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ddd0b-bdbd-45a4-89ef-b106e30bb115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:08.906547 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6f1ddd0b-bdbd-45a4-89ef-b106e30bb115 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:08.907795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6f1ddd0b-bdbd-45a4-89ef-b106e30bb115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:08.915984 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4339/8592] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:27:08.920860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77974470-2971-47ec-8432-31c3f3693413 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:08.930870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77974470-2971-47ec-8432-31c3f3693413 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:08.935840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d35a89-1b59-468f-8241-553c24929e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:08.936469 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d35a89-1b59-468f-8241-553c24929e7e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:08.936755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d35a89-1b59-468f-8241-553c24929e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:08.937066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d35a89-1b59-468f-8241-553c24929e7e admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:08.937712 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d35a89-1b59-468f-8241-553c24929e7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:27:08.938044 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d35a89-1b59-468f-8241-553c24929e7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:08.939645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d35a89-1b59-468f-8241-553c24929e7e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:08.951396 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4254/8593] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:27:08.955990 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ddd0b-bdbd-45a4-89ef-b106e30bb115 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:08.965746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ddd0b-bdbd-45a4-89ef-b106e30bb115 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:08.971305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56490c00-bebb-46b3-9574-d70a4296c452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:08.971921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56490c00-bebb-46b3-9574-d70a4296c452 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:08.972211 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56490c00-bebb-46b3-9574-d70a4296c452 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:08.972490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56490c00-bebb-46b3-9574-d70a4296c452 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:08.973201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56490c00-bebb-46b3-9574-d70a4296c452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:08.975134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56490c00-bebb-46b3-9574-d70a4296c452 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:09.008601 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4340/8594] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:27:08 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:27:09.015714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d35a89-1b59-468f-8241-553c24929e7e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:09.022302 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:09.026730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d35a89-1b59-468f-8241-553c24929e7e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:09.087564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad8848-aa4a-4c30-8452-991a8ff2aa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:09.088185 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad8848-aa4a-4c30-8452-991a8ff2aa26 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:09.088453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad8848-aa4a-4c30-8452-991a8ff2aa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:09.088741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad8848-aa4a-4c30-8452-991a8ff2aa26 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:09.089533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad8848-aa4a-4c30-8452-991a8ff2aa26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:09.092297 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad8848-aa4a-4c30-8452-991a8ff2aa26 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:09.120930 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4255/8595] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:27:09 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:27:09.123182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56490c00-bebb-46b3-9574-d70a4296c452 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:09.147520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56490c00-bebb-46b3-9574-d70a4296c452 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:09.154865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739811d7-390a-4a92-8188-2f0173d2f5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:09.155506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739811d7-390a-4a92-8188-2f0173d2f5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:09.155862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739811d7-390a-4a92-8188-2f0173d2f5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:09.156359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739811d7-390a-4a92-8188-2f0173d2f5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:09.157145 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739811d7-390a-4a92-8188-2f0173d2f5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:09.158887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739811d7-390a-4a92-8188-2f0173d2f5a2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:09.170303 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4341/8596] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:27:09.174894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad8848-aa4a-4c30-8452-991a8ff2aa26 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:09.186477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad8848-aa4a-4c30-8452-991a8ff2aa26 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:09.191726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027552d4-5c38-4782-9fbb-348c0ca586a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:09.192449 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027552d4-5c38-4782-9fbb-348c0ca586a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:09.192823 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027552d4-5c38-4782-9fbb-348c0ca586a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:09.193220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027552d4-5c38-4782-9fbb-348c0ca586a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:09.194001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027552d4-5c38-4782-9fbb-348c0ca586a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:09.195731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027552d4-5c38-4782-9fbb-348c0ca586a4 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:09.211872 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4256/8597] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:27:09.213837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739811d7-390a-4a92-8188-2f0173d2f5a2 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:09.225036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739811d7-390a-4a92-8188-2f0173d2f5a2 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:09.231427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4560fd-6b26-4b20-b5fa-eaeeee2d2702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:09.232249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4560fd-6b26-4b20-b5fa-eaeeee2d2702 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:09.232604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4560fd-6b26-4b20-b5fa-eaeeee2d2702 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:09.233004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4560fd-6b26-4b20-b5fa-eaeeee2d2702 admin admin] PATH_INFO: `/v3/projects/3a0a70f488e14e5ba98031eb96bbd856/users/24ba8dbc85894110acc63a099908d433/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:09.250110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4560fd-6b26-4b20-b5fa-eaeeee2d2702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a70f488e14e5ba98031eb96bbd856, user_id=24ba8dbc85894110acc63a099908d433, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:09.252942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4560fd-6b26-4b20-b5fa-eaeeee2d2702 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:09.273552 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4342/8598] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:27:09 2022] PUT /identity/v3/projects/3a0a70f488e14e5ba98031eb96bbd856/users/24ba8dbc85894110acc63a099908d433/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:09.279709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d5f3a-6cf7-459b-8727-442dba3d6f7f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:09.280013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d5f3a-6cf7-459b-8727-442dba3d6f7f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:09.280277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d5f3a-6cf7-459b-8727-442dba3d6f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:09.302461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bd5d5f3a-6cf7-459b-8727-442dba3d6f7f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:09.303793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bd5d5f3a-6cf7-459b-8727-442dba3d6f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:09.371518 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4257/8599] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:27:09.382241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4560fd-6b26-4b20-b5fa-eaeeee2d2702 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:09.383980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4f4560fd-6b26-4b20-b5fa-eaeeee2d2702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:09.445127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4560fd-6b26-4b20-b5fa-eaeeee2d2702 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:09.449022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657b964-5687-40d4-ba5d-b5663f57c76b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:09.449647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657b964-5687-40d4-ba5d-b5663f57c76b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:09.449941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657b964-5687-40d4-ba5d-b5663f57c76b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:09.450229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657b964-5687-40d4-ba5d-b5663f57c76b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:09.461868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657b964-5687-40d4-ba5d-b5663f57c76b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:09.463333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657b964-5687-40d4-ba5d-b5663f57c76b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:09.474644 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4343/8600] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:27:14.018736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dac796-289d-4ca2-84a4-90a71c8fc13b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:14.019324 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dac796-289d-4ca2-84a4-90a71c8fc13b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:14.019544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dac796-289d-4ca2-84a4-90a71c8fc13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:14.084937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-71dac796-289d-4ca2-84a4-90a71c8fc13b None None] MFA Rules not processed for user `24ba8dbc85894110acc63a099908d433`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:14.086057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-71dac796-289d-4ca2-84a4-90a71c8fc13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:14.182164 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4258/8601] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:27:14.187237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657b964-5687-40d4-ba5d-b5663f57c76b service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:14.197333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657b964-5687-40d4-ba5d-b5663f57c76b service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:14.203277 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61c8b4-c2a1-40fc-be64-f88a9ac4d1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:14.203896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61c8b4-c2a1-40fc-be64-f88a9ac4d1ba admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:14.204183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61c8b4-c2a1-40fc-be64-f88a9ac4d1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:14.204461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61c8b4-c2a1-40fc-be64-f88a9ac4d1ba admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:14.205175 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61c8b4-c2a1-40fc-be64-f88a9ac4d1ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:14.207120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61c8b4-c2a1-40fc-be64-f88a9ac4d1ba admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:14.243163 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4344/8602] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:27:14 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:27:14.247708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dac796-289d-4ca2-84a4-90a71c8fc13b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:14.259872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dac796-289d-4ca2-84a4-90a71c8fc13b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:14.270436 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:14.329253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276aaa1-f388-4689-aadd-cfb0a48f2d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:14.329901 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276aaa1-f388-4689-aadd-cfb0a48f2d74 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:14.330190 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276aaa1-f388-4689-aadd-cfb0a48f2d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:14.330492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276aaa1-f388-4689-aadd-cfb0a48f2d74 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:14.331136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276aaa1-f388-4689-aadd-cfb0a48f2d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:14.333246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276aaa1-f388-4689-aadd-cfb0a48f2d74 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:14.357499 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4259/8603] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:27:14 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:27:14.362266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61c8b4-c2a1-40fc-be64-f88a9ac4d1ba admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:14.374217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61c8b4-c2a1-40fc-be64-f88a9ac4d1ba admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:14.379351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42c5f9-a5bd-49c3-a197-9c8daa450a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:14.380012 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42c5f9-a5bd-49c3-a197-9c8daa450a19 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:14.380335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42c5f9-a5bd-49c3-a197-9c8daa450a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:14.380619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42c5f9-a5bd-49c3-a197-9c8daa450a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:14.381317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f42c5f9-a5bd-49c3-a197-9c8daa450a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:14.382945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f42c5f9-a5bd-49c3-a197-9c8daa450a19 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:14.394789 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4345/8604] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:27:14.399328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276aaa1-f388-4689-aadd-cfb0a48f2d74 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:14.410481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276aaa1-f388-4689-aadd-cfb0a48f2d74 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:14.416261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99590a9-3de1-4d8e-86c8-d2d908636d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:14.417046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99590a9-3de1-4d8e-86c8-d2d908636d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:14.417331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99590a9-3de1-4d8e-86c8-d2d908636d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:14.417611 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99590a9-3de1-4d8e-86c8-d2d908636d76 admin admin] PATH_INFO: `/v3/projects/88d3ddf7567c43d5b8e1f3d05e0c762e/users/9b50e90e8079411fa8a80de7d061441f/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:14.434018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99590a9-3de1-4d8e-86c8-d2d908636d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88d3ddf7567c43d5b8e1f3d05e0c762e, user_id=9b50e90e8079411fa8a80de7d061441f, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:14.437360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99590a9-3de1-4d8e-86c8-d2d908636d76 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:14.451517 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4260/8605] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:27:14 2022] PUT /identity/v3/projects/88d3ddf7567c43d5b8e1f3d05e0c762e/users/9b50e90e8079411fa8a80de7d061441f/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:27:14.455742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42c5f9-a5bd-49c3-a197-9c8daa450a19 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:14.466445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42c5f9-a5bd-49c3-a197-9c8daa450a19 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:14.517336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:14.528513 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2179ab4-84a3-41e7-a422-c20d59e9845c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:14.529206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179ab4-84a3-41e7-a422-c20d59e9845c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:14.529677 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179ab4-84a3-41e7-a422-c20d59e9845c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:14.530136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179ab4-84a3-41e7-a422-c20d59e9845c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:14.530901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2179ab4-84a3-41e7-a422-c20d59e9845c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:14.532793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2179ab4-84a3-41e7-a422-c20d59e9845c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:14.543340 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4346/8606] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:27:14.548014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99590a9-3de1-4d8e-86c8-d2d908636d76 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:14.558829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99590a9-3de1-4d8e-86c8-d2d908636d76 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:14.564101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0053c9b-21de-4eb1-b059-6649513675bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:14.564774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0053c9b-21de-4eb1-b059-6649513675bd admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:14.565095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0053c9b-21de-4eb1-b059-6649513675bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:14.565374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0053c9b-21de-4eb1-b059-6649513675bd admin admin] PATH_INFO: `/v3/projects/88d3ddf7567c43d5b8e1f3d05e0c762e/users/9b50e90e8079411fa8a80de7d061441f/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:14.567355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0053c9b-21de-4eb1-b059-6649513675bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=88d3ddf7567c43d5b8e1f3d05e0c762e, user_id=9b50e90e8079411fa8a80de7d061441f, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:14.570029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0053c9b-21de-4eb1-b059-6649513675bd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:14.581320 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4261/8607] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:27:14 2022] PUT /identity/v3/projects/88d3ddf7567c43d5b8e1f3d05e0c762e/users/9b50e90e8079411fa8a80de7d061441f/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:19.295082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207b80-7569-4b33-add2-7e3f038b1b23 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:19.295961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207b80-7569-4b33-add2-7e3f038b1b23 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:19.296248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207b80-7569-4b33-add2-7e3f038b1b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:19.330054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-91207b80-7569-4b33-add2-7e3f038b1b23 None None] MFA Rules not processed for user `9b50e90e8079411fa8a80de7d061441f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:19.331338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-91207b80-7569-4b33-add2-7e3f038b1b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:19.412176 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4347/8608] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:27:19.425374 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0053c9b-21de-4eb1-b059-6649513675bd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:19.427883 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e0053c9b-21de-4eb1-b059-6649513675bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:19.522215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0053c9b-21de-4eb1-b059-6649513675bd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:19.526811 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ff590-a03c-4224-bfe2-4ffb4faadbe6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:19.527526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ff590-a03c-4224-bfe2-4ffb4faadbe6 service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:19.528087 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ff590-a03c-4224-bfe2-4ffb4faadbe6 service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:19.530275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ff590-a03c-4224-bfe2-4ffb4faadbe6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:19.634582 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ff590-a03c-4224-bfe2-4ffb4faadbe6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:19.636119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ff590-a03c-4224-bfe2-4ffb4faadbe6 service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:19.646478 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4262/8609] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:27:20.020413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91207b80-7569-4b33-add2-7e3f038b1b23 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:20.022028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-91207b80-7569-4b33-add2-7e3f038b1b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:20.172876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91207b80-7569-4b33-add2-7e3f038b1b23 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:20.178433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1281050-c85f-41cd-ae71-7c62edcd2992 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:20.179072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1281050-c85f-41cd-ae71-7c62edcd2992 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:20.179366 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1281050-c85f-41cd-ae71-7c62edcd2992 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:20.179641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1281050-c85f-41cd-ae71-7c62edcd2992 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:20.182940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f1281050-c85f-41cd-ae71-7c62edcd2992 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:20.200842 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1281050-c85f-41cd-ae71-7c62edcd2992 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:20.202491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1281050-c85f-41cd-ae71-7c62edcd2992 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:20.221160 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4348/8610] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:27:23.086746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ff590-a03c-4224-bfe2-4ffb4faadbe6 service glance] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:23.089410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fa6ff590-a03c-4224-bfe2-4ffb4faadbe6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:23.148373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:23.227833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ff590-a03c-4224-bfe2-4ffb4faadbe6 service glance] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:23.232720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:23.233444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:23.234235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:23.234235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] PATH_INFO: `/v3/users/4f8e7d53f6ce49c6ac44ade281e1309e` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:23.236066 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8e7d53f6ce49c6ac44ade281e1309e)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:23.239167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:23.299648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8e7d53f6ce49c6ac44ade281e1309e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:23.325830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8e7d53f6ce49c6ac44ade281e1309e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:23.326540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8e7d53f6ce49c6ac44ade281e1309e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:23.336845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8e7d53f6ce49c6ac44ade281e1309e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:23.350173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8e7d53f6ce49c6ac44ade281e1309e'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:23.358567 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4263/8611] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:27:23 2022] DELETE /identity/v3/users/4f8e7d53f6ce49c6ac44ade281e1309e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:27:23.773467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1281050-c85f-41cd-ae71-7c62edcd2992 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:23.786327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1281050-c85f-41cd-ae71-7c62edcd2992 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:23.853165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:23.866966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:23.867681 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:23.868076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:23.868455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] PATH_INFO: `/v3/projects/ef9efdb2b0cd4585bb5d48040c46cdf0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:23.870164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef9efdb2b0cd4585bb5d48040c46cdf0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:23.872441 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:23.940646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef9efdb2b0cd4585bb5d48040c46cdf0', 'request_id': 'req-10471c59-ac92-416b-bcf3-82c6391cc6d0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:23.954846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef9efdb2b0cd4585bb5d48040c46cdf0', 'request_id': 'req-10471c59-ac92-416b-bcf3-82c6391cc6d0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:23.962930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef9efdb2b0cd4585bb5d48040c46cdf0', 'request_id': 'req-10471c59-ac92-416b-bcf3-82c6391cc6d0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:23.976291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] The token cache is being invalidate because project ef9efdb2b0cd4585bb5d48040c46cdf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:27:23.976801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:23.977576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:23.979529 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4349/8612] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:27:23 2022] DELETE /identity/v3/projects/ef9efdb2b0cd4585bb5d48040c46cdf0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:23.984885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:23.986939 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:24.009702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39639f-0bf9-4412-a244-0d4852fd466f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:24.095810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:24.097028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:24.097339 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:24.098974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] PATH_INFO: `/v3/users/c16bde17ce494d04be9309030582270b` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:24.100128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c16bde17ce494d04be9309030582270b)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:24.102156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:24.159450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:24.209946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c16bde17ce494d04be9309030582270b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:24.227421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c16bde17ce494d04be9309030582270b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:24.228970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c16bde17ce494d04be9309030582270b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:24.242688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c16bde17ce494d04be9309030582270b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:24.255610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c16bde17ce494d04be9309030582270b'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:24.264640 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4264/8613] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:27:23 2022] DELETE /identity/v3/users/c16bde17ce494d04be9309030582270b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:24.500499 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:24.502431 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:24.523613 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10471c59-ac92-416b-bcf3-82c6391cc6d0 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:24.631985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:24.632954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:24.633571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:24.633961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] PATH_INFO: `/v3/projects/8aafdd7e20f04d85b7d0134d8bb00942` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:24.635475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aafdd7e20f04d85b7d0134d8bb00942)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:24.638637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:24.707380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aafdd7e20f04d85b7d0134d8bb00942', 'request_id': 'req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:24.716375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aafdd7e20f04d85b7d0134d8bb00942', 'request_id': 'req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:24.727986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aafdd7e20f04d85b7d0134d8bb00942', 'request_id': 'req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:24.745201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] The token cache is being invalidate because project 8aafdd7e20f04d85b7d0134d8bb00942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:27:24.745658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:24.746496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:24.750623 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4350/8614] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:27:24 2022] DELETE /identity/v3/projects/8aafdd7e20f04d85b7d0134d8bb00942 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:28.629017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:28.630614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:28.648425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6896bc6-068d-49e5-9f3c-c8f7a48b8846 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:28.709635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:28.710278 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:28.710566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:28.710851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] PATH_INFO: `/v3/users/24ba8dbc85894110acc63a099908d433` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:28.711990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24ba8dbc85894110acc63a099908d433)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:28.713976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:28.760470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24ba8dbc85894110acc63a099908d433'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:28.773428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24ba8dbc85894110acc63a099908d433'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:28.774146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24ba8dbc85894110acc63a099908d433'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:28.781968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24ba8dbc85894110acc63a099908d433'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:28.793826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24ba8dbc85894110acc63a099908d433'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:28.799016 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4265/8615] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:27:28 2022] DELETE /identity/v3/users/24ba8dbc85894110acc63a099908d433 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:28.974669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:28.976205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:28.992670 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1d94a-8367-4ec9-9cd8-c7de415f82d9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:29.002557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:29.052640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:29.053316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:29.053608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:29.053890 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] PATH_INFO: `/v3/projects/3a0a70f488e14e5ba98031eb96bbd856` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:29.055017 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a0a70f488e14e5ba98031eb96bbd856)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:29.056965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:29.102106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a0a70f488e14e5ba98031eb96bbd856', 'request_id': 'req-6cf82ae0-8f8f-4a4a-a85a-595423284c75'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.109071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a0a70f488e14e5ba98031eb96bbd856', 'request_id': 'req-6cf82ae0-8f8f-4a4a-a85a-595423284c75'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.115330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a0a70f488e14e5ba98031eb96bbd856', 'request_id': 'req-6cf82ae0-8f8f-4a4a-a85a-595423284c75'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.126345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] The token cache is being invalidate because project 3a0a70f488e14e5ba98031eb96bbd856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:27:29.126641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.127302 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.128824 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4351/8616] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:27:28 2022] DELETE /identity/v3/projects/3a0a70f488e14e5ba98031eb96bbd856 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:29.132841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:29.134357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:29.148298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fd4db-15f8-4730-9b14-b4ae557eb5b3 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:29.161299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:29.207124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3153d4e-e9ac-417c-beb5-066ba17c2c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:29.207756 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3153d4e-e9ac-417c-beb5-066ba17c2c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:29.208040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3153d4e-e9ac-417c-beb5-066ba17c2c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:29.208315 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3153d4e-e9ac-417c-beb5-066ba17c2c14 admin admin] PATH_INFO: `/v3/users/9b50e90e8079411fa8a80de7d061441f` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:29.209495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3153d4e-e9ac-417c-beb5-066ba17c2c14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b50e90e8079411fa8a80de7d061441f)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:29.211413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3153d4e-e9ac-417c-beb5-066ba17c2c14 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:29.255866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e3153d4e-e9ac-417c-beb5-066ba17c2c14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b50e90e8079411fa8a80de7d061441f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.268719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e3153d4e-e9ac-417c-beb5-066ba17c2c14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b50e90e8079411fa8a80de7d061441f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.269419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e3153d4e-e9ac-417c-beb5-066ba17c2c14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b50e90e8079411fa8a80de7d061441f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.276431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e3153d4e-e9ac-417c-beb5-066ba17c2c14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b50e90e8079411fa8a80de7d061441f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.284603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-e3153d4e-e9ac-417c-beb5-066ba17c2c14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b50e90e8079411fa8a80de7d061441f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.291002 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4266/8617] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:27:29 2022] DELETE /identity/v3/users/9b50e90e8079411fa8a80de7d061441f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:29.469793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:29.471278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:29.484669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf82ae0-8f8f-4a4a-a85a-595423284c75 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:29.553353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:29.553999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:29.554314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:29.554595 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2 admin admin] PATH_INFO: `/v3/projects/88d3ddf7567c43d5b8e1f3d05e0c762e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:29.555730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88d3ddf7567c43d5b8e1f3d05e0c762e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:29.557751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:29.602415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:29.607144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88d3ddf7567c43d5b8e1f3d05e0c762e', 'request_id': 'req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.615354 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88d3ddf7567c43d5b8e1f3d05e0c762e', 'request_id': 'req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.622777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88d3ddf7567c43d5b8e1f3d05e0c762e', 'request_id': 'req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.636760 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2 admin admin] The token cache is being invalidate because project 88d3ddf7567c43d5b8e1f3d05e0c762e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:27:29.637108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.637783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-3bd1c3b8-3d42-49d7-a6f9-9ba7ef82e1d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:29.639302 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4352/8618] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:27:29 2022] DELETE /identity/v3/projects/88d3ddf7567c43d5b8e1f3d05e0c762e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:39.287271 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:39.633542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=8,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:39.642370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab3e21-69c0-4752-b8fc-854cdea99f05 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:39.642658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab3e21-69c0-4752-b8fc-854cdea99f05 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:39.642933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab3e21-69c0-4752-b8fc-854cdea99f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:39.693711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-52ab3e21-69c0-4752-b8fc-854cdea99f05 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:39.694937 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-52ab3e21-69c0-4752-b8fc-854cdea99f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:39.753530 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4267/8619] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:27:39.762556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf483f1-de63-4704-bb1e-01bf4da26168 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:39.762846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf483f1-de63-4704-bb1e-01bf4da26168 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:39.763124 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf483f1-de63-4704-bb1e-01bf4da26168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:39.784106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4bf483f1-de63-4704-bb1e-01bf4da26168 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:39.785345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4bf483f1-de63-4704-bb1e-01bf4da26168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:39.793208 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4353/8620] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:27:39.797900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab3e21-69c0-4752-b8fc-854cdea99f05 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:39.807667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab3e21-69c0-4752-b8fc-854cdea99f05 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:39.812253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb836a7c-d5a5-4f95-9551-32c5ee136b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:39.812847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb836a7c-d5a5-4f95-9551-32c5ee136b57 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:39.813160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb836a7c-d5a5-4f95-9551-32c5ee136b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:39.813925 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb836a7c-d5a5-4f95-9551-32c5ee136b57 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:39.814613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb836a7c-d5a5-4f95-9551-32c5ee136b57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:27:39.814919 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb836a7c-d5a5-4f95-9551-32c5ee136b57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:39.816512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb836a7c-d5a5-4f95-9551-32c5ee136b57 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:39.827850 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4268/8621] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:27:39.836621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf483f1-de63-4704-bb1e-01bf4da26168 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:39.845857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf483f1-de63-4704-bb1e-01bf4da26168 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:39.850433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a43358-e822-4c1a-8f4a-306f94a6ce7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:39.851028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a43358-e822-4c1a-8f4a-306f94a6ce7f admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:39.851314 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a43358-e822-4c1a-8f4a-306f94a6ce7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:39.851592 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a43358-e822-4c1a-8f4a-306f94a6ce7f admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:39.852217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a43358-e822-4c1a-8f4a-306f94a6ce7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:39.854149 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a43358-e822-4c1a-8f4a-306f94a6ce7f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:39.880776 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4354/8622] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:27:39 2022] POST /identity/v3/projects => generated 393 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:27:39.884809 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb836a7c-d5a5-4f95-9551-32c5ee136b57 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:39.895229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb836a7c-d5a5-4f95-9551-32c5ee136b57 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:39.953012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c639cc1-82d5-424a-98ec-4fc41748e432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:39.953626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c639cc1-82d5-424a-98ec-4fc41748e432 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:39.954555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c639cc1-82d5-424a-98ec-4fc41748e432 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:39.954844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c639cc1-82d5-424a-98ec-4fc41748e432 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:39.955484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c639cc1-82d5-424a-98ec-4fc41748e432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:39.957440 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c639cc1-82d5-424a-98ec-4fc41748e432 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:39.981061 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4269/8623] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:27:39 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:27:39.982342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a43358-e822-4c1a-8f4a-306f94a6ce7f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:39.992731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a43358-e822-4c1a-8f4a-306f94a6ce7f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:39.997359 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ab720-d75d-4bd3-81ba-6075c33f2ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:39.997977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ab720-d75d-4bd3-81ba-6075c33f2ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:39.998249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ab720-d75d-4bd3-81ba-6075c33f2ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:39.998528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ab720-d75d-4bd3-81ba-6075c33f2ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:39.999170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ab720-d75d-4bd3-81ba-6075c33f2ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:40.001325 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ab720-d75d-4bd3-81ba-6075c33f2ec5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:40.011794 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4355/8624] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:27:40.015705 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c639cc1-82d5-424a-98ec-4fc41748e432 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:40.027051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c639cc1-82d5-424a-98ec-4fc41748e432 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:40.031687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdddd01-f21f-49b8-8961-562045ab0329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:40.032336 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdddd01-f21f-49b8-8961-562045ab0329 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:40.032643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdddd01-f21f-49b8-8961-562045ab0329 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:40.032959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdddd01-f21f-49b8-8961-562045ab0329 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:40.033610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdddd01-f21f-49b8-8961-562045ab0329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:40.035211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdddd01-f21f-49b8-8961-562045ab0329 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:40.045769 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4270/8625] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:27:40.049765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ab720-d75d-4bd3-81ba-6075c33f2ec5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:40.059620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ab720-d75d-4bd3-81ba-6075c33f2ec5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:40.064697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc696bf4-7bab-4594-b7aa-f8c568749e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:40.065376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc696bf4-7bab-4594-b7aa-f8c568749e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:40.065667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc696bf4-7bab-4594-b7aa-f8c568749e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:40.065945 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc696bf4-7bab-4594-b7aa-f8c568749e30 admin admin] PATH_INFO: `/v3/projects/621154db31a744688409d362a5a51d2d/users/75a909287112419fab8c2bb8735d6913/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:40.080983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc696bf4-7bab-4594-b7aa-f8c568749e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=621154db31a744688409d362a5a51d2d, user_id=75a909287112419fab8c2bb8735d6913, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:40.083660 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc696bf4-7bab-4594-b7aa-f8c568749e30 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:40.096325 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4356/8626] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:27:40 2022] PUT /identity/v3/projects/621154db31a744688409d362a5a51d2d/users/75a909287112419fab8c2bb8735d6913/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:40.109211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f03bc-b7f4-4c90-8507-ef616a44aa75 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:40.109507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f03bc-b7f4-4c90-8507-ef616a44aa75 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:40.110035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f03bc-b7f4-4c90-8507-ef616a44aa75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:40.138987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d79f03bc-b7f4-4c90-8507-ef616a44aa75 None None] MFA Rules not processed for user `75a909287112419fab8c2bb8735d6913`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:40.140232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d79f03bc-b7f4-4c90-8507-ef616a44aa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:40.174821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:40.209377 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4271/8627] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:27:40.214173 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc696bf4-7bab-4594-b7aa-f8c568749e30 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:40.278996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc696bf4-7bab-4594-b7aa-f8c568749e30 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:40.283714 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf33666-290f-45ab-bf9b-5bdf76790f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:40.284322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf33666-290f-45ab-bf9b-5bdf76790f40 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:40.284611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf33666-290f-45ab-bf9b-5bdf76790f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:40.284921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf33666-290f-45ab-bf9b-5bdf76790f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:40.285564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf33666-290f-45ab-bf9b-5bdf76790f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:40.287490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf33666-290f-45ab-bf9b-5bdf76790f40 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:40.316528 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4357/8628] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:27:40 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:27:40.319456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f03bc-b7f4-4c90-8507-ef616a44aa75 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:40.330233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f03bc-b7f4-4c90-8507-ef616a44aa75 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:40.386252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bfc13-0b2f-4da3-9fe0-e3f879c0becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:40.386864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bfc13-0b2f-4da3-9fe0-e3f879c0becc admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:40.387154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bfc13-0b2f-4da3-9fe0-e3f879c0becc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:40.387434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bfc13-0b2f-4da3-9fe0-e3f879c0becc admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:40.388095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bfc13-0b2f-4da3-9fe0-e3f879c0becc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:40.390034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bfc13-0b2f-4da3-9fe0-e3f879c0becc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:40.410555 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4272/8629] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:27:40 2022] POST /identity/v3/users => generated 303 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:27:40.414545 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf33666-290f-45ab-bf9b-5bdf76790f40 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:40.425703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf33666-290f-45ab-bf9b-5bdf76790f40 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:40.430342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8766829-5d62-43a3-9455-ef3d49990e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:40.430939 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8766829-5d62-43a3-9455-ef3d49990e6b admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:40.431227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8766829-5d62-43a3-9455-ef3d49990e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:40.431506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8766829-5d62-43a3-9455-ef3d49990e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:40.432178 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8766829-5d62-43a3-9455-ef3d49990e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:40.433818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8766829-5d62-43a3-9455-ef3d49990e6b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:40.445143 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4358/8630] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:27:40.448996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bfc13-0b2f-4da3-9fe0-e3f879c0becc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:40.458994 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bfc13-0b2f-4da3-9fe0-e3f879c0becc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:40.463750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7383a5-23c2-4f88-9992-407b17493b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:40.464399 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7383a5-23c2-4f88-9992-407b17493b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:40.464689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7383a5-23c2-4f88-9992-407b17493b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:40.465026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7383a5-23c2-4f88-9992-407b17493b43 admin admin] PATH_INFO: `/v3/projects/239da9b5cb54407399dc57c618206624/users/7b2d3d471ac246a18005f7388bbe3914/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:40.479518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7383a5-23c2-4f88-9992-407b17493b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=239da9b5cb54407399dc57c618206624, user_id=7b2d3d471ac246a18005f7388bbe3914, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:40.482240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7383a5-23c2-4f88-9992-407b17493b43 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:40.495678 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4273/8631] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:27:40 2022] PUT /identity/v3/projects/239da9b5cb54407399dc57c618206624/users/7b2d3d471ac246a18005f7388bbe3914/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:40.499623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8766829-5d62-43a3-9455-ef3d49990e6b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:40.510300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8766829-5d62-43a3-9455-ef3d49990e6b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:40.567442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189607f8-ee53-47ab-80fc-95f05485aa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:40.568013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189607f8-ee53-47ab-80fc-95f05485aa0e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:40.568303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189607f8-ee53-47ab-80fc-95f05485aa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:40.568589 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189607f8-ee53-47ab-80fc-95f05485aa0e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:40.569256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189607f8-ee53-47ab-80fc-95f05485aa0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:40.570864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189607f8-ee53-47ab-80fc-95f05485aa0e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:40.580433 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4359/8632] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:27:40.584691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7383a5-23c2-4f88-9992-407b17493b43 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:40.595571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7383a5-23c2-4f88-9992-407b17493b43 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:40.600280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8289e83-38db-4600-abf0-362d8439e3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:40.600957 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8289e83-38db-4600-abf0-362d8439e3bd admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:40.601247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8289e83-38db-4600-abf0-362d8439e3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:40.601529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8289e83-38db-4600-abf0-362d8439e3bd admin admin] PATH_INFO: `/v3/projects/239da9b5cb54407399dc57c618206624/users/7b2d3d471ac246a18005f7388bbe3914/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:40.603458 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8289e83-38db-4600-abf0-362d8439e3bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=239da9b5cb54407399dc57c618206624, user_id=7b2d3d471ac246a18005f7388bbe3914, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:40.606143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8289e83-38db-4600-abf0-362d8439e3bd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:40.617180 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4274/8633] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:27:40 2022] PUT /identity/v3/projects/239da9b5cb54407399dc57c618206624/users/7b2d3d471ac246a18005f7388bbe3914/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:40.627338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664298fc-d4cf-48ae-b838-4afab46483ef None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:40.627655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664298fc-d4cf-48ae-b838-4afab46483ef None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:40.627937 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664298fc-d4cf-48ae-b838-4afab46483ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:40.641662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:40.656096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-664298fc-d4cf-48ae-b838-4afab46483ef None None] MFA Rules not processed for user `7b2d3d471ac246a18005f7388bbe3914`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:40.657989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-664298fc-d4cf-48ae-b838-4afab46483ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:40.725796 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4360/8634] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:27:40.739927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0dc69-5009-435c-8825-993dbdaa7ac7 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:40.740226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0dc69-5009-435c-8825-993dbdaa7ac7 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:40.740544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0dc69-5009-435c-8825-993dbdaa7ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:40.759944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f1a0dc69-5009-435c-8825-993dbdaa7ac7 None None] MFA Rules not processed for user `75a909287112419fab8c2bb8735d6913`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:40.761202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f1a0dc69-5009-435c-8825-993dbdaa7ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:40.818258 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4275/8635] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:27:40.832256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664298fc-d4cf-48ae-b838-4afab46483ef None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:40.833912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-664298fc-d4cf-48ae-b838-4afab46483ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:40.910456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664298fc-d4cf-48ae-b838-4afab46483ef None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:40.915981 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90759a8-5c2c-441d-bd05-c4c0bd552894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:40.916584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90759a8-5c2c-441d-bd05-c4c0bd552894 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:40.916872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90759a8-5c2c-441d-bd05-c4c0bd552894 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:40.917180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90759a8-5c2c-441d-bd05-c4c0bd552894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:40.924753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90759a8-5c2c-441d-bd05-c4c0bd552894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:40.926377 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90759a8-5c2c-441d-bd05-c4c0bd552894 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:40.935023 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4361/8636] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:27:50.811557 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:50.855593 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0dc69-5009-435c-8825-993dbdaa7ac7 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:50.877820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0dc69-5009-435c-8825-993dbdaa7ac7 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:50.883241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4469f-d4e2-41d3-be7d-3dd909a3ef1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:50.883844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4469f-d4e2-41d3-be7d-3dd909a3ef1c service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:50.884130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4469f-d4e2-41d3-be7d-3dd909a3ef1c service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:50.884406 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4469f-d4e2-41d3-be7d-3dd909a3ef1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:50.893023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4469f-d4e2-41d3-be7d-3dd909a3ef1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:50.894610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4469f-d4e2-41d3-be7d-3dd909a3ef1c service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:50.904336 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4276/8637] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:50 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:27:50.919659 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd59df-9031-49de-8603-7edbaebdc6f8 None None] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:50.920068 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd59df-9031-49de-8603-7edbaebdc6f8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:50.920220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd59df-9031-49de-8603-7edbaebdc6f8 None None] PATH_INFO: `/` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:50.921580 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4362/8638] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 21:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 21:27:50.926181 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4469f-d4e2-41d3-be7d-3dd909a3ef1c service nova] Authenticating service token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 21:27:50.930953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=65 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:50.939914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4469f-d4e2-41d3-be7d-3dd909a3ef1c service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:50.948146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4469f-d4e2-41d3-be7d-3dd909a3ef1c service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:50.952702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91088113-9ab5-4feb-afc6-00599d8a0e31 tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b2d3d471ac246a18005f7388bbe3914', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '239da9b5cb54407399dc57c618206624', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:50.953353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91088113-9ab5-4feb-afc6-00599d8a0e31 tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:50.953646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91088113-9ab5-4feb-afc6-00599d8a0e31 tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:50.953930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91088113-9ab5-4feb-afc6-00599d8a0e31 tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] PATH_INFO: `/v3/projects/239da9b5cb54407399dc57c618206624` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:50.955597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91088113-9ab5-4feb-afc6-00599d8a0e31 tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] RBAC: Authorizing `identity:get_project(project_id=239da9b5cb54407399dc57c618206624)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:50.957388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91088113-9ab5-4feb-afc6-00599d8a0e31 tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:50.958990 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4277/8639] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 21:27:50 2022] GET /identity/v3/projects/239da9b5cb54407399dc57c618206624 => generated 393 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:27:51.117076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd59df-9031-49de-8603-7edbaebdc6f8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:51.118720 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-86bd59df-9031-49de-8603-7edbaebdc6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:51.190726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd59df-9031-49de-8603-7edbaebdc6f8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:51.194269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abb611-b9fb-4461-a932-50f3d6d97829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:51.194872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abb611-b9fb-4461-a932-50f3d6d97829 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:51.195157 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abb611-b9fb-4461-a932-50f3d6d97829 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:51.195432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abb611-b9fb-4461-a932-50f3d6d97829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:51.197163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f0abb611-b9fb-4461-a932-50f3d6d97829 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:51.209336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0abb611-b9fb-4461-a932-50f3d6d97829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:51.210761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0abb611-b9fb-4461-a932-50f3d6d97829 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:51.221256 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4363/8640] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:27:51.376746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cb3bf-5730-4a32-85be-fb0d7bf7dc52 None None] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:51.377027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cb3bf-5730-4a32-85be-fb0d7bf7dc52 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:51.377310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cb3bf-5730-4a32-85be-fb0d7bf7dc52 None None] PATH_INFO: `/` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:51.378699 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4278/8641] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 3 21:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 21:27:51.383527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abb611-b9fb-4461-a932-50f3d6d97829 service neutron] Authenticating service token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 03 21:27:51.385049 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f0abb611-b9fb-4461-a932-50f3d6d97829 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:51.416838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abb611-b9fb-4461-a932-50f3d6d97829 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:51.426122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abb611-b9fb-4461-a932-50f3d6d97829 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:51.431000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b5bf8-b564-4ad4-a743-75a8bd5c092b tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b2d3d471ac246a18005f7388bbe3914', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '239da9b5cb54407399dc57c618206624', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:51.431665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b5bf8-b564-4ad4-a743-75a8bd5c092b tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:51.431959 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b5bf8-b564-4ad4-a743-75a8bd5c092b tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:51.432246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b5bf8-b564-4ad4-a743-75a8bd5c092b tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] PATH_INFO: `/v3/projects/239da9b5cb54407399dc57c618206624` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:51.433453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b5bf8-b564-4ad4-a743-75a8bd5c092b tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] RBAC: Authorizing `identity:get_project(project_id=239da9b5cb54407399dc57c618206624)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:51.435292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b5bf8-b564-4ad4-a743-75a8bd5c092b tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:51.436961 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4364/8642] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Sep 3 21:27:51 2022] GET /identity/v3/projects/239da9b5cb54407399dc57c618206624 => generated 393 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:27:51.593851 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cb3bf-5730-4a32-85be-fb0d7bf7dc52 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:51.595603 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-713cb3bf-5730-4a32-85be-fb0d7bf7dc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:51.611882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cb3bf-5730-4a32-85be-fb0d7bf7dc52 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:51.615420 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3323dd-1d2c-4065-900b-a76c154e1472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:51.616014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3323dd-1d2c-4065-900b-a76c154e1472 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:51.616300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3323dd-1d2c-4065-900b-a76c154e1472 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:51.616579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3323dd-1d2c-4065-900b-a76c154e1472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:51.618265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7c3323dd-1d2c-4065-900b-a76c154e1472 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:51.630535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3323dd-1d2c-4065-900b-a76c154e1472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:51.631890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3323dd-1d2c-4065-900b-a76c154e1472 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:51.642596 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4279/8643] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:27:54.364798 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b5bf8-b564-4ad4-a743-75a8bd5c092b tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:54.426492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b5bf8-b564-4ad4-a743-75a8bd5c092b tempest-ServersAdminNegativeTestJSON-1367276198 tempest-ServersAdminNegativeTestJSON-1367276198-project-admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:54.430975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:54.431621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:54.431905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:54.432184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] PATH_INFO: `/v3/users/75a909287112419fab8c2bb8735d6913` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:54.433375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a909287112419fab8c2bb8735d6913)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:54.435319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:54.483000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a909287112419fab8c2bb8735d6913'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:54.494961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a909287112419fab8c2bb8735d6913'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:54.502254 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a909287112419fab8c2bb8735d6913'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:54.502875 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a909287112419fab8c2bb8735d6913'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:54.503504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a909287112419fab8c2bb8735d6913'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:54.505560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:54.518881 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4365/8644] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:27:54 2022] DELETE /identity/v3/users/75a909287112419fab8c2bb8735d6913 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:54.523549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821433f-0930-42cd-a519-d66c31325405 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:54.524013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821433f-0930-42cd-a519-d66c31325405 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:54.524135 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821433f-0930-42cd-a519-d66c31325405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:54.543774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f821433f-0930-42cd-a519-d66c31325405 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:54.545027 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f821433f-0930-42cd-a519-d66c31325405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:54.603180 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4280/8645] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:27:54.613086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:54.614561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:54.673170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4fafdb-c423-4b7c-a45f-09ce11ea4b62 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:54.676917 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5343eb-38c7-4ef1-9fc0-15ee521e43d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:54.677467 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5343eb-38c7-4ef1-9fc0-15ee521e43d9 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:54.677752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5343eb-38c7-4ef1-9fc0-15ee521e43d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:54.678031 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5343eb-38c7-4ef1-9fc0-15ee521e43d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:54.686803 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5343eb-38c7-4ef1-9fc0-15ee521e43d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:54.688227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5343eb-38c7-4ef1-9fc0-15ee521e43d9 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:54.697526 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4366/8646] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:27:54.864053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821433f-0930-42cd-a519-d66c31325405 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:54.865523 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f821433f-0930-42cd-a519-d66c31325405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:54.878700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821433f-0930-42cd-a519-d66c31325405 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:54.884465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:54.885159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:54.885446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:54.886322 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] PATH_INFO: `/v3/projects/621154db31a744688409d362a5a51d2d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:54.886847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=621154db31a744688409d362a5a51d2d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:54.889639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:54.939351 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '621154db31a744688409d362a5a51d2d', 'request_id': 'req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:54.946605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '621154db31a744688409d362a5a51d2d', 'request_id': 'req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:54.952886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '621154db31a744688409d362a5a51d2d', 'request_id': 'req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:54.963678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] The token cache is being invalidate because project 621154db31a744688409d362a5a51d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:27:54.963969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:54.964606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:54.966097 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4281/8647] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:27:54 2022] DELETE /identity/v3/projects/621154db31a744688409d362a5a51d2d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:27:54.970115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5343eb-38c7-4ef1-9fc0-15ee521e43d9 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:54.979741 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5343eb-38c7-4ef1-9fc0-15ee521e43d9 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:55.036669 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c6c2a-c4d6-4077-8f22-e77ccc091a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:55.037330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c6c2a-c4d6-4077-8f22-e77ccc091a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:55.037620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c6c2a-c4d6-4077-8f22-e77ccc091a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:55.037903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c6c2a-c4d6-4077-8f22-e77ccc091a3a admin admin] PATH_INFO: `/v3/users/7b2d3d471ac246a18005f7388bbe3914` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:55.039060 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c6c2a-c4d6-4077-8f22-e77ccc091a3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b2d3d471ac246a18005f7388bbe3914)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:55.041351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c6c2a-c4d6-4077-8f22-e77ccc091a3a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:55.085512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-725c6c2a-c4d6-4077-8f22-e77ccc091a3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b2d3d471ac246a18005f7388bbe3914'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:55.097109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-725c6c2a-c4d6-4077-8f22-e77ccc091a3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b2d3d471ac246a18005f7388bbe3914'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:55.108119 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-725c6c2a-c4d6-4077-8f22-e77ccc091a3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b2d3d471ac246a18005f7388bbe3914'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:55.108854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-725c6c2a-c4d6-4077-8f22-e77ccc091a3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b2d3d471ac246a18005f7388bbe3914'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:55.109628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-725c6c2a-c4d6-4077-8f22-e77ccc091a3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b2d3d471ac246a18005f7388bbe3914'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:55.124292 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4367/8648] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:27:54 2022] DELETE /identity/v3/users/7b2d3d471ac246a18005f7388bbe3914 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:55.393915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:55.395373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:55.408591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228f4f5-e9cd-4321-a4b3-2c5255d442fe admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:55.450769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:55.468948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6074c-4016-4596-9ec6-3569ed909428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:55.469534 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6074c-4016-4596-9ec6-3569ed909428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:55.469819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6074c-4016-4596-9ec6-3569ed909428 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:55.470095 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6074c-4016-4596-9ec6-3569ed909428 admin admin] PATH_INFO: `/v3/projects/239da9b5cb54407399dc57c618206624` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:55.471209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6074c-4016-4596-9ec6-3569ed909428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=239da9b5cb54407399dc57c618206624)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:55.473772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6074c-4016-4596-9ec6-3569ed909428 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:55.519710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f1e6074c-4016-4596-9ec6-3569ed909428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '239da9b5cb54407399dc57c618206624', 'request_id': 'req-f1e6074c-4016-4596-9ec6-3569ed909428'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:55.526626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f1e6074c-4016-4596-9ec6-3569ed909428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '239da9b5cb54407399dc57c618206624', 'request_id': 'req-f1e6074c-4016-4596-9ec6-3569ed909428'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:55.533151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f1e6074c-4016-4596-9ec6-3569ed909428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '239da9b5cb54407399dc57c618206624', 'request_id': 'req-f1e6074c-4016-4596-9ec6-3569ed909428'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:55.544252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f1e6074c-4016-4596-9ec6-3569ed909428 admin admin] The token cache is being invalidate because project 239da9b5cb54407399dc57c618206624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:27:55.544554 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f1e6074c-4016-4596-9ec6-3569ed909428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:55.545231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-f1e6074c-4016-4596-9ec6-3569ed909428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:27:55.546755 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4282/8649] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:27:55 2022] DELETE /identity/v3/projects/239da9b5cb54407399dc57c618206624 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:27:55.553457 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831b07-2191-4a02-8107-702490d37ed5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:55.553901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831b07-2191-4a02-8107-702490d37ed5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:55.554048 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831b07-2191-4a02-8107-702490d37ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:55.575389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-8e831b07-2191-4a02-8107-702490d37ed5 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:55.576604 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8e831b07-2191-4a02-8107-702490d37ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:55.585180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:27:55.640800 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4368/8650] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:27:55.649316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f73469-8ea2-42fc-9f37-7f979e4c2dae None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:55.649604 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f73469-8ea2-42fc-9f37-7f979e4c2dae None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:55.649881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f73469-8ea2-42fc-9f37-7f979e4c2dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:55.669647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-10f73469-8ea2-42fc-9f37-7f979e4c2dae None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:55.670847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-10f73469-8ea2-42fc-9f37-7f979e4c2dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:55.679169 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4283/8651] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:27:55.683401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e831b07-2191-4a02-8107-702490d37ed5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:55.693199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e831b07-2191-4a02-8107-702490d37ed5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:55.697743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ce13f-1748-41dd-ad8a-d25bc8b38684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:55.698345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ce13f-1748-41dd-ad8a-d25bc8b38684 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:55.698632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ce13f-1748-41dd-ad8a-d25bc8b38684 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:55.698910 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ce13f-1748-41dd-ad8a-d25bc8b38684 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:55.699555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ce13f-1748-41dd-ad8a-d25bc8b38684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:27:55.699923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ce13f-1748-41dd-ad8a-d25bc8b38684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:55.701583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ce13f-1748-41dd-ad8a-d25bc8b38684 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:55.712939 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4369/8652] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:27:55.717216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f73469-8ea2-42fc-9f37-7f979e4c2dae None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:55.726931 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f73469-8ea2-42fc-9f37-7f979e4c2dae None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:55.731637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c730a43-24b4-450e-9a96-9181ba488133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:55.732227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c730a43-24b4-450e-9a96-9181ba488133 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:55.732509 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c730a43-24b4-450e-9a96-9181ba488133 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:55.732780 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c730a43-24b4-450e-9a96-9181ba488133 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:55.733438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c730a43-24b4-450e-9a96-9181ba488133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:55.735575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c730a43-24b4-450e-9a96-9181ba488133 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:55.762710 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4284/8653] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:27:55 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:27:55.766930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ce13f-1748-41dd-ad8a-d25bc8b38684 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:55.776923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ce13f-1748-41dd-ad8a-d25bc8b38684 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:55.833821 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f2301-6698-42fe-a4b7-8828264c6889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:55.834433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f2301-6698-42fe-a4b7-8828264c6889 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:55.834723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f2301-6698-42fe-a4b7-8828264c6889 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:55.835005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f2301-6698-42fe-a4b7-8828264c6889 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:55.835648 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f2301-6698-42fe-a4b7-8828264c6889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:55.837630 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f2301-6698-42fe-a4b7-8828264c6889 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:55.859082 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4370/8654] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:27:55 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:27:55.863529 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c730a43-24b4-450e-9a96-9181ba488133 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:55.873714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c730a43-24b4-450e-9a96-9181ba488133 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:55.878243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab898d-0d9a-4c63-9f00-b7241401f079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:55.878831 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab898d-0d9a-4c63-9f00-b7241401f079 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:55.879119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab898d-0d9a-4c63-9f00-b7241401f079 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:55.879388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab898d-0d9a-4c63-9f00-b7241401f079 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:55.880019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab898d-0d9a-4c63-9f00-b7241401f079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:55.881653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab898d-0d9a-4c63-9f00-b7241401f079 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:55.891405 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4285/8655] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:27:55.895686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f2301-6698-42fe-a4b7-8828264c6889 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:55.905642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f2301-6698-42fe-a4b7-8828264c6889 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:55.910884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715e629-6b9e-4458-bb9a-c3a6cafa289d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:55.911479 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715e629-6b9e-4458-bb9a-c3a6cafa289d admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:55.911766 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715e629-6b9e-4458-bb9a-c3a6cafa289d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:55.912046 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715e629-6b9e-4458-bb9a-c3a6cafa289d admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:55.912685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715e629-6b9e-4458-bb9a-c3a6cafa289d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:55.914317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715e629-6b9e-4458-bb9a-c3a6cafa289d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:55.925089 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4371/8656] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:27:55.928616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab898d-0d9a-4c63-9f00-b7241401f079 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:55.938743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab898d-0d9a-4c63-9f00-b7241401f079 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:55.943556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65f05c-7f2a-4218-931b-2e14e87ab90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:55.944203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f05c-7f2a-4218-931b-2e14e87ab90f admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:55.944488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f05c-7f2a-4218-931b-2e14e87ab90f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:55.944790 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f05c-7f2a-4218-931b-2e14e87ab90f admin admin] PATH_INFO: `/v3/projects/896d00e971ae41bd9a93fe68df8abc5e/users/045d6e2cb47c40c3b84895b0d472ddc0/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:55.960153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65f05c-7f2a-4218-931b-2e14e87ab90f admin admin] RBAC: Authorizing `identity:create_grant(project_id=896d00e971ae41bd9a93fe68df8abc5e, user_id=045d6e2cb47c40c3b84895b0d472ddc0, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:55.962856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65f05c-7f2a-4218-931b-2e14e87ab90f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:55.975834 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4286/8657] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:27:55 2022] PUT /identity/v3/projects/896d00e971ae41bd9a93fe68df8abc5e/users/045d6e2cb47c40c3b84895b0d472ddc0/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:27:55.980953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb018ad3-5c4e-47f0-bfec-4aadb09e1453 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:55.981199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb018ad3-5c4e-47f0-bfec-4aadb09e1453 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:55.981482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb018ad3-5c4e-47f0-bfec-4aadb09e1453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:56.000427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bb018ad3-5c4e-47f0-bfec-4aadb09e1453 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:27:56.001668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bb018ad3-5c4e-47f0-bfec-4aadb09e1453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:56.060013 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4372/8658] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:27:56.069815 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65f05c-7f2a-4218-931b-2e14e87ab90f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:27:56.071493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-df65f05c-7f2a-4218-931b-2e14e87ab90f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:27:56.138329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65f05c-7f2a-4218-931b-2e14e87ab90f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:27:56.142001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fc335-99f6-4429-beab-59c16db512d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:27:56.142655 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fc335-99f6-4429-beab-59c16db512d9 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:27:56.142941 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fc335-99f6-4429-beab-59c16db512d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:27:56.143879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fc335-99f6-4429-beab-59c16db512d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:27:56.153222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fc335-99f6-4429-beab-59c16db512d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:27:56.154716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fc335-99f6-4429-beab-59c16db512d9 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:27:56.164918 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4287/8659] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:28:00.420034 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ff627-a33c-4583-803d-1238aa3c0b7d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:00.420542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ff627-a33c-4583-803d-1238aa3c0b7d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:00.420624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ff627-a33c-4583-803d-1238aa3c0b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:00.453030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2d5ff627-a33c-4583-803d-1238aa3c0b7d None None] MFA Rules not processed for user `045d6e2cb47c40c3b84895b0d472ddc0`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:28:00.454510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2d5ff627-a33c-4583-803d-1238aa3c0b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:00.464073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:28:00.530873 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4373/8660] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:28:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:28:00.542001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c8ec4-d0e8-4fe7-b3cd-512d102de123 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:00.542543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c8ec4-d0e8-4fe7-b3cd-512d102de123 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:00.542543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c8ec4-d0e8-4fe7-b3cd-512d102de123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:00.562849 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ac0c8ec4-d0e8-4fe7-b3cd-512d102de123 None None] MFA Rules not processed for user `045d6e2cb47c40c3b84895b0d472ddc0`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:28:00.564089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ac0c8ec4-d0e8-4fe7-b3cd-512d102de123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:00.572119 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4288/8661] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:28:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:28:00.584435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ff627-a33c-4583-803d-1238aa3c0b7d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:00.585969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2d5ff627-a33c-4583-803d-1238aa3c0b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:00.667269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ff627-a33c-4583-803d-1238aa3c0b7d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:00.672984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78d175-42f7-4648-a07a-6e4c258d7ba6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:00.673607 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78d175-42f7-4648-a07a-6e4c258d7ba6 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:00.673895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78d175-42f7-4648-a07a-6e4c258d7ba6 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:00.674248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78d175-42f7-4648-a07a-6e4c258d7ba6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:00.682770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78d175-42f7-4648-a07a-6e4c258d7ba6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:00.684388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78d175-42f7-4648-a07a-6e4c258d7ba6 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:00.694784 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4374/8662] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:28:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:28:01.825090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c8ec4-d0e8-4fe7-b3cd-512d102de123 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:01.826588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ac0c8ec4-d0e8-4fe7-b3cd-512d102de123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:01.834450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:28:01.905207 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c8ec4-d0e8-4fe7-b3cd-512d102de123 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:01.910865 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302e8e1-4ba7-4da1-bc59-f5bf49fbf540 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:01.911481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302e8e1-4ba7-4da1-bc59-f5bf49fbf540 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:01.911768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302e8e1-4ba7-4da1-bc59-f5bf49fbf540 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:01.912051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302e8e1-4ba7-4da1-bc59-f5bf49fbf540 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:01.913804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c302e8e1-4ba7-4da1-bc59-f5bf49fbf540 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:01.928166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302e8e1-4ba7-4da1-bc59-f5bf49fbf540 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:01.929793 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302e8e1-4ba7-4da1-bc59-f5bf49fbf540 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:01.942335 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4289/8663] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:28:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:28:10.690397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=57 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:28:11.419301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78d175-42f7-4648-a07a-6e4c258d7ba6 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:11.420745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9a78d175-42f7-4648-a07a-6e4c258d7ba6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:11.435544 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78d175-42f7-4648-a07a-6e4c258d7ba6 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:11.439174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee1dae-4d5e-41f5-a1bd-1faae9e97fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:11.440400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee1dae-4d5e-41f5-a1bd-1faae9e97fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:11.440691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee1dae-4d5e-41f5-a1bd-1faae9e97fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:11.441073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee1dae-4d5e-41f5-a1bd-1faae9e97fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:11.442772 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cfee1dae-4d5e-41f5-a1bd-1faae9e97fe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:11.458731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee1dae-4d5e-41f5-a1bd-1faae9e97fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:11.460104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee1dae-4d5e-41f5-a1bd-1faae9e97fe5 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:11.474120 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4375/8664] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:28:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:28:11.935884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=30 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:28:21.466306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:28:51.641772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302e8e1-4ba7-4da1-bc59-f5bf49fbf540 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:51.659634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302e8e1-4ba7-4da1-bc59-f5bf49fbf540 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:51.665259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0cf25-934d-408e-8dab-9695dca48fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:51.665902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0cf25-934d-408e-8dab-9695dca48fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:51.666191 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0cf25-934d-408e-8dab-9695dca48fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:51.666813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0cf25-934d-408e-8dab-9695dca48fdb admin admin] PATH_INFO: `/v3/users/045d6e2cb47c40c3b84895b0d472ddc0` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:51.667894 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0cf25-934d-408e-8dab-9695dca48fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=045d6e2cb47c40c3b84895b0d472ddc0)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:51.669880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0cf25-934d-408e-8dab-9695dca48fdb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:51.726866 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bdb0cf25-934d-408e-8dab-9695dca48fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '045d6e2cb47c40c3b84895b0d472ddc0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:28:51.739129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bdb0cf25-934d-408e-8dab-9695dca48fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '045d6e2cb47c40c3b84895b0d472ddc0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:28:51.739797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bdb0cf25-934d-408e-8dab-9695dca48fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '045d6e2cb47c40c3b84895b0d472ddc0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:28:51.746771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bdb0cf25-934d-408e-8dab-9695dca48fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '045d6e2cb47c40c3b84895b0d472ddc0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:28:51.753915 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bdb0cf25-934d-408e-8dab-9695dca48fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '045d6e2cb47c40c3b84895b0d472ddc0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:28:51.760466 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4290/8665] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:28:51 2022] DELETE /identity/v3/users/045d6e2cb47c40c3b84895b0d472ddc0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:28:51.932090 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee1dae-4d5e-41f5-a1bd-1faae9e97fe5 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:51.943448 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee1dae-4d5e-41f5-a1bd-1faae9e97fe5 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:52.000394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55600ba7-3d82-4132-91f9-266ac72684e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:52.001381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55600ba7-3d82-4132-91f9-266ac72684e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:52.001707 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55600ba7-3d82-4132-91f9-266ac72684e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:52.001992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55600ba7-3d82-4132-91f9-266ac72684e5 admin admin] PATH_INFO: `/v3/projects/896d00e971ae41bd9a93fe68df8abc5e` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:52.003137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55600ba7-3d82-4132-91f9-266ac72684e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=896d00e971ae41bd9a93fe68df8abc5e)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:52.005689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55600ba7-3d82-4132-91f9-266ac72684e5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:52.053385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-55600ba7-3d82-4132-91f9-266ac72684e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896d00e971ae41bd9a93fe68df8abc5e', 'request_id': 'req-55600ba7-3d82-4132-91f9-266ac72684e5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:28:52.060771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-55600ba7-3d82-4132-91f9-266ac72684e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896d00e971ae41bd9a93fe68df8abc5e', 'request_id': 'req-55600ba7-3d82-4132-91f9-266ac72684e5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:28:52.067666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-55600ba7-3d82-4132-91f9-266ac72684e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896d00e971ae41bd9a93fe68df8abc5e', 'request_id': 'req-55600ba7-3d82-4132-91f9-266ac72684e5'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:28:52.077863 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-55600ba7-3d82-4132-91f9-266ac72684e5 admin admin] The token cache is being invalidate because project 896d00e971ae41bd9a93fe68df8abc5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:28:52.078153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-55600ba7-3d82-4132-91f9-266ac72684e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:28:52.078808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-55600ba7-3d82-4132-91f9-266ac72684e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:28:52.080294 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4376/8666] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:28:51 2022] DELETE /identity/v3/projects/896d00e971ae41bd9a93fe68df8abc5e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:28:52.085520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884feca6-4b75-4f3d-af6a-ca2f4801bb32 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:52.085812 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884feca6-4b75-4f3d-af6a-ca2f4801bb32 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:52.086090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884feca6-4b75-4f3d-af6a-ca2f4801bb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:52.105764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-884feca6-4b75-4f3d-af6a-ca2f4801bb32 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:28:52.106900 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-884feca6-4b75-4f3d-af6a-ca2f4801bb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:52.165480 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4291/8667] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:28:52.173507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56418e-7858-459e-a9ac-4ef49d489c2f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:52.173923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56418e-7858-459e-a9ac-4ef49d489c2f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:52.174236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56418e-7858-459e-a9ac-4ef49d489c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:52.194846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0c56418e-7858-459e-a9ac-4ef49d489c2f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:28:52.196036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0c56418e-7858-459e-a9ac-4ef49d489c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:52.204129 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4377/8668] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:28:52.208289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884feca6-4b75-4f3d-af6a-ca2f4801bb32 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:52.218214 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884feca6-4b75-4f3d-af6a-ca2f4801bb32 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:52.222830 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce38a2c-6cc9-484b-8db4-99a9c915b155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:52.223423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce38a2c-6cc9-484b-8db4-99a9c915b155 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:52.223711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce38a2c-6cc9-484b-8db4-99a9c915b155 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:52.223989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce38a2c-6cc9-484b-8db4-99a9c915b155 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:52.224632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce38a2c-6cc9-484b-8db4-99a9c915b155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:28:52.224973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce38a2c-6cc9-484b-8db4-99a9c915b155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:52.226568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce38a2c-6cc9-484b-8db4-99a9c915b155 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:52.238318 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4292/8669] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:28:52.242848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56418e-7858-459e-a9ac-4ef49d489c2f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:52.252120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56418e-7858-459e-a9ac-4ef49d489c2f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:52.257427 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8574927-58a5-4ee7-86ee-3a3517e59db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:52.258020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8574927-58a5-4ee7-86ee-3a3517e59db7 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:52.258311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8574927-58a5-4ee7-86ee-3a3517e59db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:52.258594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8574927-58a5-4ee7-86ee-3a3517e59db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:52.259236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8574927-58a5-4ee7-86ee-3a3517e59db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:52.261192 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8574927-58a5-4ee7-86ee-3a3517e59db7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:52.288526 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4378/8670] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:28:52 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:28:52.292969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce38a2c-6cc9-484b-8db4-99a9c915b155 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:52.304090 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce38a2c-6cc9-484b-8db4-99a9c915b155 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:52.360853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ae830-6a2b-415e-a51a-c89acd02b71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:52.361404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ae830-6a2b-415e-a51a-c89acd02b71b admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:52.361687 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ae830-6a2b-415e-a51a-c89acd02b71b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:52.361959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ae830-6a2b-415e-a51a-c89acd02b71b admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:52.362614 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ae830-6a2b-415e-a51a-c89acd02b71b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:52.365236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ae830-6a2b-415e-a51a-c89acd02b71b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:52.373474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:28:52.387136 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4293/8671] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:28:52 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:28:52.391368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8574927-58a5-4ee7-86ee-3a3517e59db7 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:52.401598 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8574927-58a5-4ee7-86ee-3a3517e59db7 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:52.406588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27eab69-9e8a-4fe7-be55-63a1968a0ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:52.407188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eab69-9e8a-4fe7-be55-63a1968a0ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:52.407468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eab69-9e8a-4fe7-be55-63a1968a0ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:52.407743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eab69-9e8a-4fe7-be55-63a1968a0ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:52.408380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27eab69-9e8a-4fe7-be55-63a1968a0ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:52.409998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27eab69-9e8a-4fe7-be55-63a1968a0ba5 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:52.420158 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4379/8672] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:28:52.423970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ae830-6a2b-415e-a51a-c89acd02b71b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:52.434758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ae830-6a2b-415e-a51a-c89acd02b71b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:52.439437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a09682-565e-40d7-98d3-2147b708aa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:52.440073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a09682-565e-40d7-98d3-2147b708aa61 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:52.440354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a09682-565e-40d7-98d3-2147b708aa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:52.440654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a09682-565e-40d7-98d3-2147b708aa61 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:52.441354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a09682-565e-40d7-98d3-2147b708aa61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:52.442946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a09682-565e-40d7-98d3-2147b708aa61 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:52.453640 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4294/8673] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:28:52.457762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27eab69-9e8a-4fe7-be55-63a1968a0ba5 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:52.467805 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27eab69-9e8a-4fe7-be55-63a1968a0ba5 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:52.472473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d4277-8ab2-4624-b075-8a2e6aaea25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:52.473637 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d4277-8ab2-4624-b075-8a2e6aaea25b admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:52.473918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d4277-8ab2-4624-b075-8a2e6aaea25b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:52.474191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d4277-8ab2-4624-b075-8a2e6aaea25b admin admin] PATH_INFO: `/v3/projects/90775ee27e9d462aaf236e95d95d5512/users/d4a6118a9f10425c9a95842105534ea0/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:52.488723 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d4277-8ab2-4624-b075-8a2e6aaea25b admin admin] RBAC: Authorizing `identity:create_grant(project_id=90775ee27e9d462aaf236e95d95d5512, user_id=d4a6118a9f10425c9a95842105534ea0, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:52.491900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d4277-8ab2-4624-b075-8a2e6aaea25b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:52.505013 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4380/8674] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:28:52 2022] PUT /identity/v3/projects/90775ee27e9d462aaf236e95d95d5512/users/d4a6118a9f10425c9a95842105534ea0/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:28:52.509951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8d484-541d-4252-8a0c-f2015fa5efb6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:52.510248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8d484-541d-4252-8a0c-f2015fa5efb6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:52.510527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8d484-541d-4252-8a0c-f2015fa5efb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:52.529317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c5d8d484-541d-4252-8a0c-f2015fa5efb6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:28:52.530575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c5d8d484-541d-4252-8a0c-f2015fa5efb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:52.588531 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4295/8675] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:28:52.598887 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d4277-8ab2-4624-b075-8a2e6aaea25b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:52.600581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2c5d4277-8ab2-4624-b075-8a2e6aaea25b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:52.631620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:28:52.662064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d4277-8ab2-4624-b075-8a2e6aaea25b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:52.665799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547fb22-bc61-4f05-91f5-6370af5d92e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:52.666406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547fb22-bc61-4f05-91f5-6370af5d92e8 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:52.666693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547fb22-bc61-4f05-91f5-6370af5d92e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:52.666977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547fb22-bc61-4f05-91f5-6370af5d92e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:52.676070 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547fb22-bc61-4f05-91f5-6370af5d92e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:52.678115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547fb22-bc61-4f05-91f5-6370af5d92e8 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:52.687504 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4381/8676] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:28:56.449983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b615ca-cf8a-4549-bfa4-d6b1e4bf018d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:56.450491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b615ca-cf8a-4549-bfa4-d6b1e4bf018d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:56.450578 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b615ca-cf8a-4549-bfa4-d6b1e4bf018d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:56.480725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-27b615ca-cf8a-4549-bfa4-d6b1e4bf018d None None] MFA Rules not processed for user `d4a6118a9f10425c9a95842105534ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:28:56.482014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-27b615ca-cf8a-4549-bfa4-d6b1e4bf018d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:56.550205 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4296/8677] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:28:56.561375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe3a36-32dd-4843-b53e-d532e96cc3cb None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:56.561908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe3a36-32dd-4843-b53e-d532e96cc3cb None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:56.561908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe3a36-32dd-4843-b53e-d532e96cc3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:56.582812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-06fe3a36-32dd-4843-b53e-d532e96cc3cb None None] MFA Rules not processed for user `d4a6118a9f10425c9a95842105534ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:28:56.584052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-06fe3a36-32dd-4843-b53e-d532e96cc3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:56.591177 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4382/8678] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:28:56.603801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b615ca-cf8a-4549-bfa4-d6b1e4bf018d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:28:56.605249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-27b615ca-cf8a-4549-bfa4-d6b1e4bf018d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:28:56.686846 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b615ca-cf8a-4549-bfa4-d6b1e4bf018d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:28:56.693098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2748295-c63a-4455-bdde-9a3f1d13731f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:28:56.693719 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2748295-c63a-4455-bdde-9a3f1d13731f service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:28:56.694004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2748295-c63a-4455-bdde-9a3f1d13731f service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:28:56.694281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2748295-c63a-4455-bdde-9a3f1d13731f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:28:56.699342 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:28:56.703082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2748295-c63a-4455-bdde-9a3f1d13731f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:28:56.704782 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2748295-c63a-4455-bdde-9a3f1d13731f service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:28:56.715794 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4297/8679] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:28:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:29:06.576686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=17 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:29:06.712055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:29:06.748438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe3a36-32dd-4843-b53e-d532e96cc3cb None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:29:06.761399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe3a36-32dd-4843-b53e-d532e96cc3cb None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:29:06.767571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c399e-9734-434a-bbd6-0fc7d82f6799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:29:06.768190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c399e-9734-434a-bbd6-0fc7d82f6799 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:29:06.768476 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c399e-9734-434a-bbd6-0fc7d82f6799 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:29:06.768754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c399e-9734-434a-bbd6-0fc7d82f6799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:29:06.771221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f50c399e-9734-434a-bbd6-0fc7d82f6799 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:29:06.783839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c399e-9734-434a-bbd6-0fc7d82f6799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:29:06.785330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c399e-9734-434a-bbd6-0fc7d82f6799 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:29:06.796190 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4383/8680] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:29:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:29:07.179253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2748295-c63a-4455-bdde-9a3f1d13731f service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:29:07.180691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a2748295-c63a-4455-bdde-9a3f1d13731f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:29:07.252103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2748295-c63a-4455-bdde-9a3f1d13731f service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:29:07.257820 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64db94-6208-42de-af2c-de683c9f990e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:29:07.258426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64db94-6208-42de-af2c-de683c9f990e service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:29:07.258713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64db94-6208-42de-af2c-de683c9f990e service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:29:07.258990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64db94-6208-42de-af2c-de683c9f990e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:29:07.260693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3b64db94-6208-42de-af2c-de683c9f990e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:29:07.274518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64db94-6208-42de-af2c-de683c9f990e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:29:07.276103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64db94-6208-42de-af2c-de683c9f990e service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:29:07.288531 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4298/8681] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:29:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:29:16.790052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:29:17.281772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=32 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:29:44.364164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04df659-4710-4c46-9875-6c03e8551477 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:29:44.364712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04df659-4710-4c46-9875-6c03e8551477 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:29:44.364712 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04df659-4710-4c46-9875-6c03e8551477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:29:44.383928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f04df659-4710-4c46-9875-6c03e8551477 None None] MFA Rules not processed for user `d4a6118a9f10425c9a95842105534ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:29:44.385166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f04df659-4710-4c46-9875-6c03e8551477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:29:44.392014 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4384/8682] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:29:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:29:47.533415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64db94-6208-42de-af2c-de683c9f990e service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:29:47.535005 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3b64db94-6208-42de-af2c-de683c9f990e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:29:47.554372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64db94-6208-42de-af2c-de683c9f990e service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:29:47.560431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bce81-6fa2-44b6-8f39-3e0fd3576837 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:29:47.561205 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bce81-6fa2-44b6-8f39-3e0fd3576837 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:29:47.561497 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bce81-6fa2-44b6-8f39-3e0fd3576837 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:29:47.561781 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bce81-6fa2-44b6-8f39-3e0fd3576837 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:29:47.563699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-925bce81-6fa2-44b6-8f39-3e0fd3576837 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:29:47.579702 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925bce81-6fa2-44b6-8f39-3e0fd3576837 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:29:47.581316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925bce81-6fa2-44b6-8f39-3e0fd3576837 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:29:47.595450 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4299/8683] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:29:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:29:54.378961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:29:57.587965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:30:11.167618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04df659-4710-4c46-9875-6c03e8551477 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:11.169116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f04df659-4710-4c46-9875-6c03e8551477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:30:11.184358 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04df659-4710-4c46-9875-6c03e8551477 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:11.187914 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df540ff7-62db-4726-a7e2-5d96754488f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:11.188518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df540ff7-62db-4726-a7e2-5d96754488f6 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:11.188799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df540ff7-62db-4726-a7e2-5d96754488f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:11.189126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df540ff7-62db-4726-a7e2-5d96754488f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:11.190776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-df540ff7-62db-4726-a7e2-5d96754488f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:30:11.202697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df540ff7-62db-4726-a7e2-5d96754488f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:11.204123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df540ff7-62db-4726-a7e2-5d96754488f6 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:11.214435 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4385/8684] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:30:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:30:17.274051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bce81-6fa2-44b6-8f39-3e0fd3576837 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:17.286745 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bce81-6fa2-44b6-8f39-3e0fd3576837 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:17.290383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80d612-f958-4ca7-9466-9fb9d5973746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:17.290980 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80d612-f958-4ca7-9466-9fb9d5973746 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:17.291267 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80d612-f958-4ca7-9466-9fb9d5973746 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:17.291547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80d612-f958-4ca7-9466-9fb9d5973746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:17.293298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8b80d612-f958-4ca7-9466-9fb9d5973746 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:30:17.309824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80d612-f958-4ca7-9466-9fb9d5973746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:17.311262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80d612-f958-4ca7-9466-9fb9d5973746 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:17.325028 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4300/8685] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:30:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:30:18.510838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df540ff7-62db-4726-a7e2-5d96754488f6 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:18.522726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df540ff7-62db-4726-a7e2-5d96754488f6 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:18.526517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12aab6-0eb7-4094-b6f9-c2eb73947f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:18.527123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12aab6-0eb7-4094-b6f9-c2eb73947f73 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:18.527410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12aab6-0eb7-4094-b6f9-c2eb73947f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:18.527690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12aab6-0eb7-4094-b6f9-c2eb73947f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:18.529456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7e12aab6-0eb7-4094-b6f9-c2eb73947f73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:30:18.546312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12aab6-0eb7-4094-b6f9-c2eb73947f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:18.547751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12aab6-0eb7-4094-b6f9-c2eb73947f73 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:18.561247 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4386/8686] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:30:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:30:27.317531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:30:28.553940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=12 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:30:38.908108 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80d612-f958-4ca7-9466-9fb9d5973746 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:38.931186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80d612-f958-4ca7-9466-9fb9d5973746 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:38.936120 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ee15a-9b5e-489b-acb3-054cf7350916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:38.936774 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ee15a-9b5e-489b-acb3-054cf7350916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:38.937203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ee15a-9b5e-489b-acb3-054cf7350916 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:38.937488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ee15a-9b5e-489b-acb3-054cf7350916 admin admin] PATH_INFO: `/v3/users/d4a6118a9f10425c9a95842105534ea0` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:38.938651 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ee15a-9b5e-489b-acb3-054cf7350916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a6118a9f10425c9a95842105534ea0)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:38.941021 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ee15a-9b5e-489b-acb3-054cf7350916 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:38.993917 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-158ee15a-9b5e-489b-acb3-054cf7350916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a6118a9f10425c9a95842105534ea0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:30:39.007317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-158ee15a-9b5e-489b-acb3-054cf7350916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a6118a9f10425c9a95842105534ea0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:30:39.008857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-158ee15a-9b5e-489b-acb3-054cf7350916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a6118a9f10425c9a95842105534ea0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:30:39.016229 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-158ee15a-9b5e-489b-acb3-054cf7350916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a6118a9f10425c9a95842105534ea0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:30:39.025608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-158ee15a-9b5e-489b-acb3-054cf7350916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a6118a9f10425c9a95842105534ea0'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:30:39.032655 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4301/8687] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:30:38 2022] DELETE /identity/v3/users/d4a6118a9f10425c9a95842105534ea0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:30:39.215232 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12aab6-0eb7-4094-b6f9-c2eb73947f73 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:39.227127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12aab6-0eb7-4094-b6f9-c2eb73947f73 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:39.290756 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045bfd3-fae4-44e7-9713-70e299e8ebab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:39.291501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045bfd3-fae4-44e7-9713-70e299e8ebab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:39.291792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045bfd3-fae4-44e7-9713-70e299e8ebab admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:39.292074 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045bfd3-fae4-44e7-9713-70e299e8ebab admin admin] PATH_INFO: `/v3/projects/90775ee27e9d462aaf236e95d95d5512` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:39.293246 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045bfd3-fae4-44e7-9713-70e299e8ebab admin admin] RBAC: Authorizing `identity:delete_project(project_id=90775ee27e9d462aaf236e95d95d5512)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:39.295163 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045bfd3-fae4-44e7-9713-70e299e8ebab admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:39.340878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4045bfd3-fae4-44e7-9713-70e299e8ebab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90775ee27e9d462aaf236e95d95d5512', 'request_id': 'req-4045bfd3-fae4-44e7-9713-70e299e8ebab'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:30:39.349037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4045bfd3-fae4-44e7-9713-70e299e8ebab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90775ee27e9d462aaf236e95d95d5512', 'request_id': 'req-4045bfd3-fae4-44e7-9713-70e299e8ebab'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:30:39.355656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4045bfd3-fae4-44e7-9713-70e299e8ebab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90775ee27e9d462aaf236e95d95d5512', 'request_id': 'req-4045bfd3-fae4-44e7-9713-70e299e8ebab'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:30:39.366471 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4045bfd3-fae4-44e7-9713-70e299e8ebab admin admin] The token cache is being invalidate because project 90775ee27e9d462aaf236e95d95d5512 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:30:39.366768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4045bfd3-fae4-44e7-9713-70e299e8ebab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:30:39.367428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4045bfd3-fae4-44e7-9713-70e299e8ebab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:30:39.368974 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4387/8688] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:30:39 2022] DELETE /identity/v3/projects/90775ee27e9d462aaf236e95d95d5512 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:30:39.373935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b1fd7-2d67-41aa-b624-cfb04d01f128 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:39.374224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b1fd7-2d67-41aa-b624-cfb04d01f128 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:39.374501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b1fd7-2d67-41aa-b624-cfb04d01f128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:39.393959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-641b1fd7-2d67-41aa-b624-cfb04d01f128 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:30:39.395661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-641b1fd7-2d67-41aa-b624-cfb04d01f128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:30:39.454713 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4302/8689] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:30:39.464088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364727a-6cdd-4ce0-8677-1952513a339b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:39.464385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364727a-6cdd-4ce0-8677-1952513a339b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:39.464668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364727a-6cdd-4ce0-8677-1952513a339b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:39.485162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9364727a-6cdd-4ce0-8677-1952513a339b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:30:39.486389 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9364727a-6cdd-4ce0-8677-1952513a339b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:30:39.494389 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4388/8690] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:30:39.498526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b1fd7-2d67-41aa-b624-cfb04d01f128 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:39.509528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b1fd7-2d67-41aa-b624-cfb04d01f128 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:39.514798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7bdf5f-1a19-4bf9-a00e-cf70f6a38275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:39.515640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bdf5f-1a19-4bf9-a00e-cf70f6a38275 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:39.515936 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bdf5f-1a19-4bf9-a00e-cf70f6a38275 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:39.516219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bdf5f-1a19-4bf9-a00e-cf70f6a38275 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:39.516873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bdf5f-1a19-4bf9-a00e-cf70f6a38275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:30:39.517223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bdf5f-1a19-4bf9-a00e-cf70f6a38275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:39.518860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bdf5f-1a19-4bf9-a00e-cf70f6a38275 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:39.531185 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4303/8691] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:30:39.535605 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364727a-6cdd-4ce0-8677-1952513a339b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:39.545037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364727a-6cdd-4ce0-8677-1952513a339b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:39.549750 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0ec47-1378-428e-ac45-9e5d6ca4e31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:39.550349 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0ec47-1378-428e-ac45-9e5d6ca4e31d admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:39.550642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0ec47-1378-428e-ac45-9e5d6ca4e31d admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:39.550924 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0ec47-1378-428e-ac45-9e5d6ca4e31d admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:39.551809 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0ec47-1378-428e-ac45-9e5d6ca4e31d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:39.554181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0ec47-1378-428e-ac45-9e5d6ca4e31d admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:39.595357 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4389/8692] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:30:39 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:30:39.599398 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7bdf5f-1a19-4bf9-a00e-cf70f6a38275 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:39.610279 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7bdf5f-1a19-4bf9-a00e-cf70f6a38275 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:39.670195 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647c0e1-f41c-4136-830a-a8eb5ff88f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:39.670814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647c0e1-f41c-4136-830a-a8eb5ff88f65 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:39.671100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647c0e1-f41c-4136-830a-a8eb5ff88f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:39.671370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647c0e1-f41c-4136-830a-a8eb5ff88f65 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:39.672002 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647c0e1-f41c-4136-830a-a8eb5ff88f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:39.673962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647c0e1-f41c-4136-830a-a8eb5ff88f65 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:39.682303 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:30:39.701301 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4304/8693] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:30:39 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:30:39.702484 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0ec47-1378-428e-ac45-9e5d6ca4e31d admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:39.715783 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0ec47-1378-428e-ac45-9e5d6ca4e31d admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:39.720522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca4872-b75e-41b8-87cf-57e9b9ad4ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:39.721180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4872-b75e-41b8-87cf-57e9b9ad4ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:39.721474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4872-b75e-41b8-87cf-57e9b9ad4ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:39.721762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4872-b75e-41b8-87cf-57e9b9ad4ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:39.722410 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca4872-b75e-41b8-87cf-57e9b9ad4ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:39.724045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca4872-b75e-41b8-87cf-57e9b9ad4ed9 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:39.734533 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4390/8694] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:30:39.738591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647c0e1-f41c-4136-830a-a8eb5ff88f65 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:39.749060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647c0e1-f41c-4136-830a-a8eb5ff88f65 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:39.753847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27302b58-7803-4cc8-8a25-71dad24dcc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:39.754995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27302b58-7803-4cc8-8a25-71dad24dcc22 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:39.755282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27302b58-7803-4cc8-8a25-71dad24dcc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:39.755579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27302b58-7803-4cc8-8a25-71dad24dcc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:39.756240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27302b58-7803-4cc8-8a25-71dad24dcc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:39.757845 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27302b58-7803-4cc8-8a25-71dad24dcc22 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:39.767747 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4305/8695] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:30:39.772500 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca4872-b75e-41b8-87cf-57e9b9ad4ed9 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:39.782440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca4872-b75e-41b8-87cf-57e9b9ad4ed9 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:39.787137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfe47e-cbb1-437f-bdbb-3b7b9b4e8f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:39.787789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfe47e-cbb1-437f-bdbb-3b7b9b4e8f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:39.788088 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfe47e-cbb1-437f-bdbb-3b7b9b4e8f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:39.788373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfe47e-cbb1-437f-bdbb-3b7b9b4e8f06 admin admin] PATH_INFO: `/v3/projects/934cd0abe2484e06a909466637406e45/users/40161a00c7b64a359eb4124919b060ac/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:39.802921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfe47e-cbb1-437f-bdbb-3b7b9b4e8f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=934cd0abe2484e06a909466637406e45, user_id=40161a00c7b64a359eb4124919b060ac, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:39.806020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfe47e-cbb1-437f-bdbb-3b7b9b4e8f06 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:39.819872 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4391/8696] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:30:39 2022] PUT /identity/v3/projects/934cd0abe2484e06a909466637406e45/users/40161a00c7b64a359eb4124919b060ac/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:30:39.825016 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf9d32-1e15-4edb-bb4a-2f5d5f584ebf None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:39.825219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf9d32-1e15-4edb-bb4a-2f5d5f584ebf None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:39.825502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf9d32-1e15-4edb-bb4a-2f5d5f584ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:39.844643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9cdf9d32-1e15-4edb-bb4a-2f5d5f584ebf None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:30:39.845833 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9cdf9d32-1e15-4edb-bb4a-2f5d5f584ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:30:39.903961 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4306/8697] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:30:39.913612 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfe47e-cbb1-437f-bdbb-3b7b9b4e8f06 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:39.915364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1ddfe47e-cbb1-437f-bdbb-3b7b9b4e8f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:30:39.935850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:30:39.977121 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfe47e-cbb1-437f-bdbb-3b7b9b4e8f06 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:39.980797 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135248a-e4ae-4c31-83df-3264173c893a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:39.981438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135248a-e4ae-4c31-83df-3264173c893a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:39.981726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135248a-e4ae-4c31-83df-3264173c893a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:39.982007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135248a-e4ae-4c31-83df-3264173c893a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:39.991768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135248a-e4ae-4c31-83df-3264173c893a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:39.993272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135248a-e4ae-4c31-83df-3264173c893a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:40.003478 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4392/8698] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:30:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:30:43.713647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974546d0-34cb-4913-b04c-1683a5cd8275 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:43.714098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974546d0-34cb-4913-b04c-1683a5cd8275 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:43.714213 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974546d0-34cb-4913-b04c-1683a5cd8275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:43.746346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-974546d0-34cb-4913-b04c-1683a5cd8275 None None] MFA Rules not processed for user `40161a00c7b64a359eb4124919b060ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:30:43.747485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-974546d0-34cb-4913-b04c-1683a5cd8275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:30:43.819198 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4307/8699] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:30:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:30:43.834056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135248a-e4ae-4c31-83df-3264173c893a service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:30:43.835632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1135248a-e4ae-4c31-83df-3264173c893a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:30:43.945562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135248a-e4ae-4c31-83df-3264173c893a service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:30:43.954352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1fd2b-06c1-4d95-a91d-80de79c34359 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:30:43.954923 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1fd2b-06c1-4d95-a91d-80de79c34359 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:43.955215 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1fd2b-06c1-4d95-a91d-80de79c34359 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:43.955496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1fd2b-06c1-4d95-a91d-80de79c34359 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:43.964249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1fd2b-06c1-4d95-a91d-80de79c34359 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:30:43.965916 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1fd2b-06c1-4d95-a91d-80de79c34359 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:30:43.975789 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4393/8700] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:30:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:30:45.296400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc56ee-5cc6-431b-b9da-fc27d78be776 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:30:45.296860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc56ee-5cc6-431b-b9da-fc27d78be776 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:30:45.297015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc56ee-5cc6-431b-b9da-fc27d78be776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:30:45.316541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3bbc56ee-5cc6-431b-b9da-fc27d78be776 None None] MFA Rules not processed for user `40161a00c7b64a359eb4124919b060ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:30:45.318340 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3bbc56ee-5cc6-431b-b9da-fc27d78be776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:30:45.325762 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4308/8701] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:30:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:30:53.971290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=52 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:30:55.311634 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:31:35.940245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1fd2b-06c1-4d95-a91d-80de79c34359 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:31:35.941776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c7d1fd2b-06c1-4d95-a91d-80de79c34359 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:31:35.956186 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1fd2b-06c1-4d95-a91d-80de79c34359 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:31:35.959767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d9505-5327-4e0f-bf7a-98354170c514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:31:35.960384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d9505-5327-4e0f-bf7a-98354170c514 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:31:35.960668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d9505-5327-4e0f-bf7a-98354170c514 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:31:35.960973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d9505-5327-4e0f-bf7a-98354170c514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:31:35.962619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e18d9505-5327-4e0f-bf7a-98354170c514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:31:35.974736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d9505-5327-4e0f-bf7a-98354170c514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:31:35.976150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d9505-5327-4e0f-bf7a-98354170c514 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:31:35.986929 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4394/8702] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:31:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:31:45.980488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:32:05.863792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc56ee-5cc6-431b-b9da-fc27d78be776 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:32:05.865246 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3bbc56ee-5cc6-431b-b9da-fc27d78be776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:32:05.941643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc56ee-5cc6-431b-b9da-fc27d78be776 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:32:05.947217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce127b3-00e9-46a8-9508-7f7a247460ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:32:05.947824 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce127b3-00e9-46a8-9508-7f7a247460ab service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:32:05.948114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce127b3-00e9-46a8-9508-7f7a247460ab service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:32:05.948393 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce127b3-00e9-46a8-9508-7f7a247460ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:32:05.950117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3ce127b3-00e9-46a8-9508-7f7a247460ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:32:05.964628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce127b3-00e9-46a8-9508-7f7a247460ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:32:05.966244 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce127b3-00e9-46a8-9508-7f7a247460ab service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:32:05.978880 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4309/8703] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:32:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:32:15.972116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=32 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:32:25.698091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d9505-5327-4e0f-bf7a-98354170c514 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:32:25.715143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d9505-5327-4e0f-bf7a-98354170c514 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:32:25.720249 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97bccb-9a5d-45fc-a377-d045346ef335 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:32:25.720957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97bccb-9a5d-45fc-a377-d045346ef335 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:32:25.721311 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97bccb-9a5d-45fc-a377-d045346ef335 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:32:25.721676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97bccb-9a5d-45fc-a377-d045346ef335 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:32:25.723572 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6d97bccb-9a5d-45fc-a377-d045346ef335 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:32:25.739643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97bccb-9a5d-45fc-a377-d045346ef335 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:32:25.741860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97bccb-9a5d-45fc-a377-d045346ef335 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:32:25.754847 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4395/8704] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:32:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:32:27.845762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce127b3-00e9-46a8-9508-7f7a247460ab service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:32:27.848137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3ce127b3-00e9-46a8-9508-7f7a247460ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:32:27.883372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce127b3-00e9-46a8-9508-7f7a247460ab service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:32:27.889074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d10ca-1d07-432b-8d29-36ec8e86979d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:32:27.889694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d10ca-1d07-432b-8d29-36ec8e86979d service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:32:27.889984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d10ca-1d07-432b-8d29-36ec8e86979d service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:32:27.890263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d10ca-1d07-432b-8d29-36ec8e86979d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:32:27.892011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-910d10ca-1d07-432b-8d29-36ec8e86979d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:32:27.907967 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d10ca-1d07-432b-8d29-36ec8e86979d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:32:27.909636 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d10ca-1d07-432b-8d29-36ec8e86979d service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:32:27.923723 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4310/8705] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:32:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:32:31.394171 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93893c71-4f9f-4872-a959-1caebe85d7f9 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:32:31.394656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93893c71-4f9f-4872-a959-1caebe85d7f9 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:32:31.394753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93893c71-4f9f-4872-a959-1caebe85d7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:32:31.415009 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-93893c71-4f9f-4872-a959-1caebe85d7f9 None None] MFA Rules not processed for user `40161a00c7b64a359eb4124919b060ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:32:31.416200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-93893c71-4f9f-4872-a959-1caebe85d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:32:31.423014 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4396/8706] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:32:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:32:34.626149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d10ca-1d07-432b-8d29-36ec8e86979d service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:32:34.642828 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d10ca-1d07-432b-8d29-36ec8e86979d service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:32:34.649548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7a77d-f7de-4dab-a953-9205e0d7f85c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:32:34.650159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7a77d-f7de-4dab-a953-9205e0d7f85c service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:32:34.650448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7a77d-f7de-4dab-a953-9205e0d7f85c service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:32:34.650721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7a77d-f7de-4dab-a953-9205e0d7f85c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:32:34.652429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c3a7a77d-f7de-4dab-a953-9205e0d7f85c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:32:34.669451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7a77d-f7de-4dab-a953-9205e0d7f85c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:32:34.671056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7a77d-f7de-4dab-a953-9205e0d7f85c service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:32:34.685374 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4311/8707] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:32:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:32:41.409432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=9 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:32:44.678186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=12 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:33:10.384312 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93893c71-4f9f-4872-a959-1caebe85d7f9 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:33:10.385874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-93893c71-4f9f-4872-a959-1caebe85d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:33:10.403674 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93893c71-4f9f-4872-a959-1caebe85d7f9 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:33:10.408954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209baa1a-e6bc-4b3a-aba3-4a7bd1aac2cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:33:10.409568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209baa1a-e6bc-4b3a-aba3-4a7bd1aac2cb service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:33:10.409958 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209baa1a-e6bc-4b3a-aba3-4a7bd1aac2cb service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:33:10.410242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209baa1a-e6bc-4b3a-aba3-4a7bd1aac2cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:33:10.411930 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-209baa1a-e6bc-4b3a-aba3-4a7bd1aac2cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:33:10.425744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209baa1a-e6bc-4b3a-aba3-4a7bd1aac2cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:33:10.427337 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209baa1a-e6bc-4b3a-aba3-4a7bd1aac2cb service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:33:10.440034 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4397/8708] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:33:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:33:20.433341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:33:59.874434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7a77d-f7de-4dab-a953-9205e0d7f85c service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:33:59.887096 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7a77d-f7de-4dab-a953-9205e0d7f85c service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:33:59.890710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb772e3d-3b4b-48b2-8376-186cb38ba00f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:33:59.891338 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb772e3d-3b4b-48b2-8376-186cb38ba00f service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:33:59.891621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb772e3d-3b4b-48b2-8376-186cb38ba00f service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:33:59.891899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb772e3d-3b4b-48b2-8376-186cb38ba00f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:33:59.893635 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cb772e3d-3b4b-48b2-8376-186cb38ba00f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:33:59.909194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb772e3d-3b4b-48b2-8376-186cb38ba00f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:33:59.910613 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb772e3d-3b4b-48b2-8376-186cb38ba00f service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:33:59.923895 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4312/8709] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:33:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:34:09.916927 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:34:14.011143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3e119-afb2-4f59-8d89-4be37c7e80d1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:34:14.011623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3e119-afb2-4f59-8d89-4be37c7e80d1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:34:14.011744 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3e119-afb2-4f59-8d89-4be37c7e80d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:34:14.040444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3ca3e119-afb2-4f59-8d89-4be37c7e80d1 None None] MFA Rules not processed for user `40161a00c7b64a359eb4124919b060ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:34:14.042116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3ca3e119-afb2-4f59-8d89-4be37c7e80d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:34:14.049335 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4398/8710] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:34:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:34:14.366188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb772e3d-3b4b-48b2-8376-186cb38ba00f service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:34:14.377556 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb772e3d-3b4b-48b2-8376-186cb38ba00f service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:34:14.381204 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b9665-c2d6-45d9-8344-e6a930485e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:34:14.381801 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b9665-c2d6-45d9-8344-e6a930485e5a service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:34:14.382091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b9665-c2d6-45d9-8344-e6a930485e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:34:14.382372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b9665-c2d6-45d9-8344-e6a930485e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:34:14.384117 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1c2b9665-c2d6-45d9-8344-e6a930485e5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:34:14.396052 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b9665-c2d6-45d9-8344-e6a930485e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:34:14.397530 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b9665-c2d6-45d9-8344-e6a930485e5a service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:34:14.408571 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4313/8711] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:34:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:34:24.035015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:34:24.402550 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:34:58.385433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3e119-afb2-4f59-8d89-4be37c7e80d1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:34:58.398109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3e119-afb2-4f59-8d89-4be37c7e80d1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:34:58.402209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a26ca-c01b-490b-b422-175c422c7789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:34:58.402975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a26ca-c01b-490b-b422-175c422c7789 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:34:58.403404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a26ca-c01b-490b-b422-175c422c7789 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:34:58.403692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a26ca-c01b-490b-b422-175c422c7789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:34:58.406029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a67a26ca-c01b-490b-b422-175c422c7789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:34:58.419110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a26ca-c01b-490b-b422-175c422c7789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:34:58.420577 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a26ca-c01b-490b-b422-175c422c7789 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:34:58.433346 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4399/8712] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:34:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:34:58.522620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b9665-c2d6-45d9-8344-e6a930485e5a service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:34:58.538880 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b9665-c2d6-45d9-8344-e6a930485e5a service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:34:58.547586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1641b9-5e5d-4b9b-8e59-a28b69e28cd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:34:58.548194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1641b9-5e5d-4b9b-8e59-a28b69e28cd7 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:34:58.548484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1641b9-5e5d-4b9b-8e59-a28b69e28cd7 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:34:58.548762 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1641b9-5e5d-4b9b-8e59-a28b69e28cd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:34:58.550502 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6c1641b9-5e5d-4b9b-8e59-a28b69e28cd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:34:58.565039 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1641b9-5e5d-4b9b-8e59-a28b69e28cd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:34:58.566632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1641b9-5e5d-4b9b-8e59-a28b69e28cd7 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:34:58.579182 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4314/8713] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:34:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:35:08.428257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:35:08.572407 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:35:14.767044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a26ca-c01b-490b-b422-175c422c7789 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:35:14.788125 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a26ca-c01b-490b-b422-175c422c7789 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:35:14.791728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80800820-f57f-46f4-a9f8-5bf2d52b99fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:35:14.792332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80800820-f57f-46f4-a9f8-5bf2d52b99fb service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:35:14.792619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80800820-f57f-46f4-a9f8-5bf2d52b99fb service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:35:14.792918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80800820-f57f-46f4-a9f8-5bf2d52b99fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:35:14.794639 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-80800820-f57f-46f4-a9f8-5bf2d52b99fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:35:14.806934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80800820-f57f-46f4-a9f8-5bf2d52b99fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:35:14.808381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80800820-f57f-46f4-a9f8-5bf2d52b99fb service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:35:14.819299 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4400/8714] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:35:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:35:18.823835 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1641b9-5e5d-4b9b-8e59-a28b69e28cd7 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:35:18.836545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1641b9-5e5d-4b9b-8e59-a28b69e28cd7 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:35:18.840172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e215f2f-95e2-4df4-9bc9-9291d1abe581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:35:18.840797 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e215f2f-95e2-4df4-9bc9-9291d1abe581 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:35:18.841113 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e215f2f-95e2-4df4-9bc9-9291d1abe581 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:35:18.841411 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e215f2f-95e2-4df4-9bc9-9291d1abe581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:35:18.843414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3e215f2f-95e2-4df4-9bc9-9291d1abe581 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:35:18.859856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e215f2f-95e2-4df4-9bc9-9291d1abe581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:35:18.861329 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e215f2f-95e2-4df4-9bc9-9291d1abe581 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:35:18.885265 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4315/8715] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:35:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:35:21.329846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80800820-f57f-46f4-a9f8-5bf2d52b99fb service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:35:21.341233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80800820-f57f-46f4-a9f8-5bf2d52b99fb service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:35:21.344874 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c10f4-7a09-4058-a3b1-6054110cd44d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:35:21.345502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c10f4-7a09-4058-a3b1-6054110cd44d service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:35:21.345787 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c10f4-7a09-4058-a3b1-6054110cd44d service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:35:21.346062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c10f4-7a09-4058-a3b1-6054110cd44d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:35:21.347788 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-639c10f4-7a09-4058-a3b1-6054110cd44d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:35:21.364250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c10f4-7a09-4058-a3b1-6054110cd44d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:35:21.365698 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c10f4-7a09-4058-a3b1-6054110cd44d service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:35:21.380095 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4401/8716] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:35:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:35:26.234902 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e215f2f-95e2-4df4-9bc9-9291d1abe581 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:35:26.250013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e215f2f-95e2-4df4-9bc9-9291d1abe581 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:35:26.256047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7783b8e-c150-4ac6-b359-d822d6dabeba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:35:26.256678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7783b8e-c150-4ac6-b359-d822d6dabeba service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:35:26.256981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7783b8e-c150-4ac6-b359-d822d6dabeba service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:35:26.257262 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7783b8e-c150-4ac6-b359-d822d6dabeba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:35:26.259255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d7783b8e-c150-4ac6-b359-d822d6dabeba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:35:26.275091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7783b8e-c150-4ac6-b359-d822d6dabeba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:35:26.276735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7783b8e-c150-4ac6-b359-d822d6dabeba service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:35:26.290305 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4316/8717] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:35:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:35:31.371913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=16 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:35:33.171575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a41c8-3baa-429a-a47b-91feadd24d9f None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:35:33.172132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a41c8-3baa-429a-a47b-91feadd24d9f None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:35:33.172132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a41c8-3baa-429a-a47b-91feadd24d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:35:33.191100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-cc3a41c8-3baa-429a-a47b-91feadd24d9f None None] MFA Rules not processed for user `40161a00c7b64a359eb4124919b060ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:35:33.192318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-cc3a41c8-3baa-429a-a47b-91feadd24d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:35:33.200330 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4402/8718] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:35:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:35:33.504531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7783b8e-c150-4ac6-b359-d822d6dabeba service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:35:33.517861 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7783b8e-c150-4ac6-b359-d822d6dabeba service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:35:33.521568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee8d18-18fd-4c68-8182-0646206cacc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:35:33.522173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee8d18-18fd-4c68-8182-0646206cacc6 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:35:33.522461 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee8d18-18fd-4c68-8182-0646206cacc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:35:33.522757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee8d18-18fd-4c68-8182-0646206cacc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:35:33.524487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-6eee8d18-18fd-4c68-8182-0646206cacc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:35:33.536569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee8d18-18fd-4c68-8182-0646206cacc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:35:33.538047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee8d18-18fd-4c68-8182-0646206cacc6 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:35:33.548669 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4317/8719] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:35:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:35:43.186792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:35:43.209106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a41c8-3baa-429a-a47b-91feadd24d9f None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:35:43.225278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a41c8-3baa-429a-a47b-91feadd24d9f None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:35:43.230931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b4491-1360-4e34-aa1a-e6a3bbd42b7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:35:43.231537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b4491-1360-4e34-aa1a-e6a3bbd42b7e service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:35:43.231822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b4491-1360-4e34-aa1a-e6a3bbd42b7e service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:35:43.232108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b4491-1360-4e34-aa1a-e6a3bbd42b7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:35:43.241154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b4491-1360-4e34-aa1a-e6a3bbd42b7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:35:43.242768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b4491-1360-4e34-aa1a-e6a3bbd42b7e service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:35:43.252459 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4403/8720] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:35:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:35:43.542446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=22 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:35:53.247857 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:36:38.924097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baaf821-9b89-4df2-b086-66384fd88f93 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:36:38.924642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baaf821-9b89-4df2-b086-66384fd88f93 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:36:38.924642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baaf821-9b89-4df2-b086-66384fd88f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:36:38.971777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-9baaf821-9b89-4df2-b086-66384fd88f93 None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:36:38.973019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9baaf821-9b89-4df2-b086-66384fd88f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:36:38.979806 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4318/8721] 10.222.0.16 () {58 vars in 1062 bytes} [Sat Sep 3 21:36:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:36:38.986469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b4491-1360-4e34-aa1a-e6a3bbd42b7e service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:36:38.995440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b4491-1360-4e34-aa1a-e6a3bbd42b7e service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:36:38.999123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6342f2-cd3b-4988-a662-403b30de2aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:36:38.999735 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6342f2-cd3b-4988-a662-403b30de2aed service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:36:39.000022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6342f2-cd3b-4988-a662-403b30de2aed service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:36:39.000301 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6342f2-cd3b-4988-a662-403b30de2aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:36:39.010811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6342f2-cd3b-4988-a662-403b30de2aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:36:39.012237 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6342f2-cd3b-4988-a662-403b30de2aed service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:36:39.022583 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4404/8722] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:36:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:36:48.969124 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=13 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:36:49.016511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:03.454826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baaf821-9b89-4df2-b086-66384fd88f93 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:03.474206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baaf821-9b89-4df2-b086-66384fd88f93 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:03.477839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049dcdf-104d-4f4a-8392-6ee279682471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:03.479465 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049dcdf-104d-4f4a-8392-6ee279682471 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:03.479753 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049dcdf-104d-4f4a-8392-6ee279682471 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:03.480030 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049dcdf-104d-4f4a-8392-6ee279682471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:03.510486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049dcdf-104d-4f4a-8392-6ee279682471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:03.511956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049dcdf-104d-4f4a-8392-6ee279682471 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:03.530939 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4319/8723] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:37:06.353153 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6342f2-cd3b-4988-a662-403b30de2aed service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:06.436460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049dcdf-104d-4f4a-8392-6ee279682471 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:06.449392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049dcdf-104d-4f4a-8392-6ee279682471 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:06.450193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6342f2-cd3b-4988-a662-403b30de2aed service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:06.456451 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc21637-f355-453b-a386-adeb2fd5bc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:06.457549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc21637-f355-453b-a386-adeb2fd5bc53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:06.457643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276db95-6d71-47d2-b54d-b5afde279850 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:06.457740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc21637-f355-453b-a386-adeb2fd5bc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:06.457814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc21637-f355-453b-a386-adeb2fd5bc53 admin admin] PATH_INFO: `/v3/users/40161a00c7b64a359eb4124919b060ac` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:06.457964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276db95-6d71-47d2-b54d-b5afde279850 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:06.458429 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276db95-6d71-47d2-b54d-b5afde279850 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:06.458710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276db95-6d71-47d2-b54d-b5afde279850 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:06.458969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc21637-f355-453b-a386-adeb2fd5bc53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40161a00c7b64a359eb4124919b060ac)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:06.460587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5276db95-6d71-47d2-b54d-b5afde279850 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:06.461000 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc21637-f355-453b-a386-adeb2fd5bc53 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:06.486616 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276db95-6d71-47d2-b54d-b5afde279850 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:06.488384 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276db95-6d71-47d2-b54d-b5afde279850 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:06.502238 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4405/8724] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:37:06.511253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bcc21637-f355-453b-a386-adeb2fd5bc53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40161a00c7b64a359eb4124919b060ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:06.524378 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bcc21637-f355-453b-a386-adeb2fd5bc53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40161a00c7b64a359eb4124919b060ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:06.525206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bcc21637-f355-453b-a386-adeb2fd5bc53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40161a00c7b64a359eb4124919b060ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:06.534074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bcc21637-f355-453b-a386-adeb2fd5bc53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40161a00c7b64a359eb4124919b060ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:06.541734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bcc21637-f355-453b-a386-adeb2fd5bc53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40161a00c7b64a359eb4124919b060ac'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:06.548024 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4320/8725] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:37:06 2022] DELETE /identity/v3/users/40161a00c7b64a359eb4124919b060ac => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:37:06.714881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276db95-6d71-47d2-b54d-b5afde279850 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:06.728505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276db95-6d71-47d2-b54d-b5afde279850 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:06.790319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a430238b-94c4-4736-9429-13535206f3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:06.790955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430238b-94c4-4736-9429-13535206f3f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:06.791241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430238b-94c4-4736-9429-13535206f3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:06.791520 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430238b-94c4-4736-9429-13535206f3f7 admin admin] PATH_INFO: `/v3/projects/934cd0abe2484e06a909466637406e45` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:06.792658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a430238b-94c4-4736-9429-13535206f3f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=934cd0abe2484e06a909466637406e45)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:06.794673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a430238b-94c4-4736-9429-13535206f3f7 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:06.839970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a430238b-94c4-4736-9429-13535206f3f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934cd0abe2484e06a909466637406e45', 'request_id': 'req-a430238b-94c4-4736-9429-13535206f3f7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:06.847650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a430238b-94c4-4736-9429-13535206f3f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934cd0abe2484e06a909466637406e45', 'request_id': 'req-a430238b-94c4-4736-9429-13535206f3f7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:06.853653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a430238b-94c4-4736-9429-13535206f3f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934cd0abe2484e06a909466637406e45', 'request_id': 'req-a430238b-94c4-4736-9429-13535206f3f7'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:06.864279 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a430238b-94c4-4736-9429-13535206f3f7 admin admin] The token cache is being invalidate because project 934cd0abe2484e06a909466637406e45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:37:06.864574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a430238b-94c4-4736-9429-13535206f3f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:06.865278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-a430238b-94c4-4736-9429-13535206f3f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:06.866779 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4406/8726] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:37:06 2022] DELETE /identity/v3/projects/934cd0abe2484e06a909466637406e45 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:37:06.871714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eddcb4-19f4-4431-9f81-156d791d93bc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:06.872007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eddcb4-19f4-4431-9f81-156d791d93bc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:06.872286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eddcb4-19f4-4431-9f81-156d791d93bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:06.893487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b2eddcb4-19f4-4431-9f81-156d791d93bc None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:06.895668 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b2eddcb4-19f4-4431-9f81-156d791d93bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:06.955414 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4321/8727] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:06.963170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a87f9b-768d-4da3-9e2e-0925974d9e7b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:06.963463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a87f9b-768d-4da3-9e2e-0925974d9e7b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:06.963749 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a87f9b-768d-4da3-9e2e-0925974d9e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:06.983796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-30a87f9b-768d-4da3-9e2e-0925974d9e7b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:06.985370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-30a87f9b-768d-4da3-9e2e-0925974d9e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:06.993593 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4407/8728] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:06.997773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eddcb4-19f4-4431-9f81-156d791d93bc None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:07.007633 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eddcb4-19f4-4431-9f81-156d791d93bc None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:07.012243 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa4154-1cb4-4d88-b28b-0a963408d68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:07.012841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa4154-1cb4-4d88-b28b-0a963408d68b admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:07.013165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa4154-1cb4-4d88-b28b-0a963408d68b admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:07.013443 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa4154-1cb4-4d88-b28b-0a963408d68b admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:07.014084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa4154-1cb4-4d88-b28b-0a963408d68b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:37:07.014392 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa4154-1cb4-4d88-b28b-0a963408d68b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:07.015990 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa4154-1cb4-4d88-b28b-0a963408d68b admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:07.027906 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4322/8729] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:37:07.032098 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a87f9b-768d-4da3-9e2e-0925974d9e7b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:07.042029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a87f9b-768d-4da3-9e2e-0925974d9e7b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:07.047133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36490101-9b87-4c18-8276-49ec31cb2d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:07.047728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36490101-9b87-4c18-8276-49ec31cb2d27 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:07.048018 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36490101-9b87-4c18-8276-49ec31cb2d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:07.048300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36490101-9b87-4c18-8276-49ec31cb2d27 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:07.048967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36490101-9b87-4c18-8276-49ec31cb2d27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:07.050884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36490101-9b87-4c18-8276-49ec31cb2d27 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:07.081102 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4408/8730] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:37:07 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:37:07.082158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa4154-1cb4-4d88-b28b-0a963408d68b admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:07.092234 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa4154-1cb4-4d88-b28b-0a963408d68b admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:07.096599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:07.154268 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa4947-4c4d-4be6-892e-524aa3245052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:07.154829 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa4947-4c4d-4be6-892e-524aa3245052 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:07.155119 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa4947-4c4d-4be6-892e-524aa3245052 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:07.155400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa4947-4c4d-4be6-892e-524aa3245052 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:07.156049 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa4947-4c4d-4be6-892e-524aa3245052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:07.158019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa4947-4c4d-4be6-892e-524aa3245052 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:07.178737 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4323/8731] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:37:07 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:37:07.182906 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36490101-9b87-4c18-8276-49ec31cb2d27 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:07.193629 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36490101-9b87-4c18-8276-49ec31cb2d27 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:07.198184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707aabc1-7c7e-4187-b4c1-d94f1079822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:07.198785 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707aabc1-7c7e-4187-b4c1-d94f1079822f admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:07.199072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707aabc1-7c7e-4187-b4c1-d94f1079822f admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:07.199348 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707aabc1-7c7e-4187-b4c1-d94f1079822f admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:07.199986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707aabc1-7c7e-4187-b4c1-d94f1079822f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:07.201645 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707aabc1-7c7e-4187-b4c1-d94f1079822f admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:07.212209 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4409/8732] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:37:07.216047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa4947-4c4d-4be6-892e-524aa3245052 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:07.226101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa4947-4c4d-4be6-892e-524aa3245052 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:07.231544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7bc58-3fc2-420a-a9b3-f9547b23dc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:07.232143 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7bc58-3fc2-420a-a9b3-f9547b23dc8a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:07.232425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7bc58-3fc2-420a-a9b3-f9547b23dc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:07.232697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7bc58-3fc2-420a-a9b3-f9547b23dc8a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:07.233379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7bc58-3fc2-420a-a9b3-f9547b23dc8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:07.235226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7bc58-3fc2-420a-a9b3-f9547b23dc8a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:07.245860 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4324/8733] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:37:07.249878 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707aabc1-7c7e-4187-b4c1-d94f1079822f admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:07.255459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:07.261576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707aabc1-7c7e-4187-b4c1-d94f1079822f admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:07.267717 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe60eb-1fbb-44ad-b71b-06bfab18dcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:07.268390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe60eb-1fbb-44ad-b71b-06bfab18dcef admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:07.268657 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe60eb-1fbb-44ad-b71b-06bfab18dcef admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:07.268957 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe60eb-1fbb-44ad-b71b-06bfab18dcef admin admin] PATH_INFO: `/v3/projects/f173156588e346d481341b28d4594784/users/4cc1b001d6c14a4c8030d9682d7a3a95/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:07.284477 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe60eb-1fbb-44ad-b71b-06bfab18dcef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f173156588e346d481341b28d4594784, user_id=4cc1b001d6c14a4c8030d9682d7a3a95, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:07.287180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe60eb-1fbb-44ad-b71b-06bfab18dcef admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:07.299455 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4410/8734] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:37:07 2022] PUT /identity/v3/projects/f173156588e346d481341b28d4594784/users/4cc1b001d6c14a4c8030d9682d7a3a95/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:37:07.304621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355636dc-5857-47ae-b7d3-4b5fb257510a None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:07.304947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355636dc-5857-47ae-b7d3-4b5fb257510a None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:07.305233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355636dc-5857-47ae-b7d3-4b5fb257510a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:07.324242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-355636dc-5857-47ae-b7d3-4b5fb257510a None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:07.325474 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-355636dc-5857-47ae-b7d3-4b5fb257510a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:07.384125 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4325/8735] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:07.394726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe60eb-1fbb-44ad-b71b-06bfab18dcef admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:07.396412 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6abe60eb-1fbb-44ad-b71b-06bfab18dcef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:07.452451 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe60eb-1fbb-44ad-b71b-06bfab18dcef admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:07.456757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dbb83-09e7-481a-a70e-b09997620be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:07.457390 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dbb83-09e7-481a-a70e-b09997620be5 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:07.457678 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dbb83-09e7-481a-a70e-b09997620be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:07.457955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dbb83-09e7-481a-a70e-b09997620be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:07.466931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dbb83-09e7-481a-a70e-b09997620be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:07.468523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dbb83-09e7-481a-a70e-b09997620be5 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:07.478354 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4411/8736] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:37:11.564015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58794c6d-de35-4443-87bd-19098ccce0d6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:11.564525 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58794c6d-de35-4443-87bd-19098ccce0d6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:11.564662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58794c6d-de35-4443-87bd-19098ccce0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:11.594476 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-58794c6d-de35-4443-87bd-19098ccce0d6 None None] MFA Rules not processed for user `4cc1b001d6c14a4c8030d9682d7a3a95`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:11.596367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-58794c6d-de35-4443-87bd-19098ccce0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:11.665632 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4326/8737] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:11.675537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3046028-22e5-49df-b977-31c5e3d16ab1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:11.676000 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3046028-22e5-49df-b977-31c5e3d16ab1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:11.676116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3046028-22e5-49df-b977-31c5e3d16ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:11.696710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e3046028-22e5-49df-b977-31c5e3d16ab1 None None] MFA Rules not processed for user `4cc1b001d6c14a4c8030d9682d7a3a95`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:11.698056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e3046028-22e5-49df-b977-31c5e3d16ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:11.706220 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4412/8738] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:11.725253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58794c6d-de35-4443-87bd-19098ccce0d6 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:11.726764 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-58794c6d-de35-4443-87bd-19098ccce0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:11.747047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:11.827950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58794c6d-de35-4443-87bd-19098ccce0d6 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:11.834361 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3ec8e-0c56-4697-bb76-c518a2b4828d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:11.834984 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3ec8e-0c56-4697-bb76-c518a2b4828d service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:11.835273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3ec8e-0c56-4697-bb76-c518a2b4828d service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:11.835553 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3ec8e-0c56-4697-bb76-c518a2b4828d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:11.844645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3ec8e-0c56-4697-bb76-c518a2b4828d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:11.846472 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3ec8e-0c56-4697-bb76-c518a2b4828d service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:11.855813 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4327/8739] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:37:12.767195 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3046028-22e5-49df-b977-31c5e3d16ab1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:12.769292 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e3046028-22e5-49df-b977-31c5e3d16ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:12.860399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3046028-22e5-49df-b977-31c5e3d16ab1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:12.865490 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e0627-7bff-461b-b920-a59213a0f773 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:12.866113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e0627-7bff-461b-b920-a59213a0f773 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:12.866401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e0627-7bff-461b-b920-a59213a0f773 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:12.866690 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e0627-7bff-461b-b920-a59213a0f773 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:12.868434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-163e0627-7bff-461b-b920-a59213a0f773 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:12.882649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e0627-7bff-461b-b920-a59213a0f773 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:12.884596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e0627-7bff-461b-b920-a59213a0f773 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:12.897442 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4413/8740] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:37:21.851589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=27 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:22.890001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=70,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:23.507028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d3fa4-7f53-4183-88cf-994f46bb7e5d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:23.507496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d3fa4-7f53-4183-88cf-994f46bb7e5d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:23.507989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d3fa4-7f53-4183-88cf-994f46bb7e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:23.527505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e33d3fa4-7f53-4183-88cf-994f46bb7e5d None None] MFA Rules not processed for user `4cc1b001d6c14a4c8030d9682d7a3a95`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:23.528708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e33d3fa4-7f53-4183-88cf-994f46bb7e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:23.536411 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4328/8741] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:24.099045 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d29919-0a87-4b9b-929a-b606e699432c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:24.099506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d29919-0a87-4b9b-929a-b606e699432c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:24.099594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d29919-0a87-4b9b-929a-b606e699432c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:24.172691 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-26d29919-0a87-4b9b-929a-b606e699432c None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:24.173989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-26d29919-0a87-4b9b-929a-b606e699432c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:24.182721 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4414/8742] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 21:37:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:24.194280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d3fa4-7f53-4183-88cf-994f46bb7e5d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:24.195631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e33d3fa4-7f53-4183-88cf-994f46bb7e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:24.213821 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d3fa4-7f53-4183-88cf-994f46bb7e5d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:24.218748 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaf59f-9c54-4983-9a6b-7c401acee651 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:24.219345 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaf59f-9c54-4983-9a6b-7c401acee651 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:24.219632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaf59f-9c54-4983-9a6b-7c401acee651 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:24.219905 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaf59f-9c54-4983-9a6b-7c401acee651 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:24.228870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeaf59f-9c54-4983-9a6b-7c401acee651 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:24.230735 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeaf59f-9c54-4983-9a6b-7c401acee651 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:24.241357 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4329/8743] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:37:24.742460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d29919-0a87-4b9b-929a-b606e699432c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:24.756370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d29919-0a87-4b9b-929a-b606e699432c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:24.761408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52800886-90db-4b24-9077-11280e23bc90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:24.762004 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52800886-90db-4b24-9077-11280e23bc90 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:24.762290 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52800886-90db-4b24-9077-11280e23bc90 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:24.762569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52800886-90db-4b24-9077-11280e23bc90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:24.764296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-52800886-90db-4b24-9077-11280e23bc90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:24.778739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52800886-90db-4b24-9077-11280e23bc90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:24.780345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52800886-90db-4b24-9077-11280e23bc90 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:24.792870 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4415/8744] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:37:32.744960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaf59f-9c54-4983-9a6b-7c401acee651 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:32.746965 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4eeaf59f-9c54-4983-9a6b-7c401acee651 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:32.766834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaf59f-9c54-4983-9a6b-7c401acee651 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:32.772955 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8cf00-581f-4cb6-bf65-bd1b60e0a643 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:32.773572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8cf00-581f-4cb6-bf65-bd1b60e0a643 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:32.773859 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8cf00-581f-4cb6-bf65-bd1b60e0a643 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:32.774137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8cf00-581f-4cb6-bf65-bd1b60e0a643 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:32.776083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-16e8cf00-581f-4cb6-bf65-bd1b60e0a643 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:32.791969 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8cf00-581f-4cb6-bf65-bd1b60e0a643 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:32.793644 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8cf00-581f-4cb6-bf65-bd1b60e0a643 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:32.808471 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4330/8745] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:37:34.785980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=24 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:37.026523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52800886-90db-4b24-9077-11280e23bc90 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:37.057796 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52800886-90db-4b24-9077-11280e23bc90 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:37.064883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228ce30-b4f3-49fc-9d86-a396550e0386 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:37.065770 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228ce30-b4f3-49fc-9d86-a396550e0386 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:37.066059 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228ce30-b4f3-49fc-9d86-a396550e0386 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:37.067207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228ce30-b4f3-49fc-9d86-a396550e0386 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:37.070158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5228ce30-b4f3-49fc-9d86-a396550e0386 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:37.090183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228ce30-b4f3-49fc-9d86-a396550e0386 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:37.091881 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228ce30-b4f3-49fc-9d86-a396550e0386 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:37.113615 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4416/8746] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:37:42.800979 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=15 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:44.989450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8cf00-581f-4cb6-bf65-bd1b60e0a643 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:45.011628 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8cf00-581f-4cb6-bf65-bd1b60e0a643 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:45.024462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e2bd3-8ef6-4b6b-be74-0fb82ca04e2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:45.025196 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e2bd3-8ef6-4b6b-be74-0fb82ca04e2c service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:45.025487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e2bd3-8ef6-4b6b-be74-0fb82ca04e2c service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:45.025777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e2bd3-8ef6-4b6b-be74-0fb82ca04e2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:45.027652 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b32e2bd3-8ef6-4b6b-be74-0fb82ca04e2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:45.045609 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e2bd3-8ef6-4b6b-be74-0fb82ca04e2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:45.049544 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e2bd3-8ef6-4b6b-be74-0fb82ca04e2c service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:45.050212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228ce30-b4f3-49fc-9d86-a396550e0386 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:45.067076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228ce30-b4f3-49fc-9d86-a396550e0386 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:45.069783 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4331/8747] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:37:45.073066 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a106778-9e2a-406d-af8a-03733bf22052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:45.073918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a106778-9e2a-406d-af8a-03733bf22052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:45.074073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a106778-9e2a-406d-af8a-03733bf22052 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:45.074356 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a106778-9e2a-406d-af8a-03733bf22052 admin admin] PATH_INFO: `/v3/users/4cc1b001d6c14a4c8030d9682d7a3a95` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:45.075848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a106778-9e2a-406d-af8a-03733bf22052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc1b001d6c14a4c8030d9682d7a3a95)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:45.079316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a106778-9e2a-406d-af8a-03733bf22052 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:45.133317 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a106778-9e2a-406d-af8a-03733bf22052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc1b001d6c14a4c8030d9682d7a3a95'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:45.145553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a106778-9e2a-406d-af8a-03733bf22052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc1b001d6c14a4c8030d9682d7a3a95'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:45.152675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a106778-9e2a-406d-af8a-03733bf22052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc1b001d6c14a4c8030d9682d7a3a95'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:45.153396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a106778-9e2a-406d-af8a-03733bf22052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc1b001d6c14a4c8030d9682d7a3a95'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:45.154026 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-2a106778-9e2a-406d-af8a-03733bf22052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc1b001d6c14a4c8030d9682d7a3a95'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:45.166157 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4417/8748] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:37:45 2022] DELETE /identity/v3/users/4cc1b001d6c14a4c8030d9682d7a3a95 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:37:45.340423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e2bd3-8ef6-4b6b-be74-0fb82ca04e2c service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:45.352431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e2bd3-8ef6-4b6b-be74-0fb82ca04e2c service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:45.419956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123cceb-6572-4948-bd71-6df0594bf0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:45.420585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123cceb-6572-4948-bd71-6df0594bf0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:45.420870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123cceb-6572-4948-bd71-6df0594bf0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:45.421178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123cceb-6572-4948-bd71-6df0594bf0e2 admin admin] PATH_INFO: `/v3/projects/f173156588e346d481341b28d4594784` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:45.422309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123cceb-6572-4948-bd71-6df0594bf0e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f173156588e346d481341b28d4594784)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:45.424281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123cceb-6572-4948-bd71-6df0594bf0e2 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:45.471089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b123cceb-6572-4948-bd71-6df0594bf0e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f173156588e346d481341b28d4594784', 'request_id': 'req-b123cceb-6572-4948-bd71-6df0594bf0e2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:45.478270 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b123cceb-6572-4948-bd71-6df0594bf0e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f173156588e346d481341b28d4594784', 'request_id': 'req-b123cceb-6572-4948-bd71-6df0594bf0e2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:45.486798 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b123cceb-6572-4948-bd71-6df0594bf0e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f173156588e346d481341b28d4594784', 'request_id': 'req-b123cceb-6572-4948-bd71-6df0594bf0e2'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:45.509296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b123cceb-6572-4948-bd71-6df0594bf0e2 admin admin] The token cache is being invalidate because project f173156588e346d481341b28d4594784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:37:45.509589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b123cceb-6572-4948-bd71-6df0594bf0e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:45.510240 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-b123cceb-6572-4948-bd71-6df0594bf0e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:45.511700 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4332/8749] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:37:45 2022] DELETE /identity/v3/projects/f173156588e346d481341b28d4594784 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:37:45.516693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e33c3-66f1-47b1-9989-66f1aacf6f2c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:45.517574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e33c3-66f1-47b1-9989-66f1aacf6f2c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:45.517880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e33c3-66f1-47b1-9989-66f1aacf6f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:45.564454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-ab7e33c3-66f1-47b1-9989-66f1aacf6f2c None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:45.565731 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ab7e33c3-66f1-47b1-9989-66f1aacf6f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:45.624187 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4418/8750] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:45.632757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac964181-013a-4a2f-af64-e9e4dfb9d3df None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:45.633086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac964181-013a-4a2f-af64-e9e4dfb9d3df None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:45.633381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac964181-013a-4a2f-af64-e9e4dfb9d3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:45.653178 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ac964181-013a-4a2f-af64-e9e4dfb9d3df None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:45.654792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ac964181-013a-4a2f-af64-e9e4dfb9d3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:45.662512 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4333/8751] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:45.666542 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e33c3-66f1-47b1-9989-66f1aacf6f2c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:45.676382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e33c3-66f1-47b1-9989-66f1aacf6f2c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:45.681052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d1732-1054-4750-ae1c-6f154479b5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:45.681653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d1732-1054-4750-ae1c-6f154479b5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:45.681942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d1732-1054-4750-ae1c-6f154479b5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:45.682221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d1732-1054-4750-ae1c-6f154479b5f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:45.682867 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d1732-1054-4750-ae1c-6f154479b5f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:37:45.683214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d1732-1054-4750-ae1c-6f154479b5f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:45.684851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d1732-1054-4750-ae1c-6f154479b5f3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:45.696789 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4419/8752] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:37:45.700738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac964181-013a-4a2f-af64-e9e4dfb9d3df None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:45.710547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac964181-013a-4a2f-af64-e9e4dfb9d3df None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:45.715123 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adccbe-5802-4a1e-b9d3-9e0bdbcbff9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:45.715707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adccbe-5802-4a1e-b9d3-9e0bdbcbff9f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:45.715993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adccbe-5802-4a1e-b9d3-9e0bdbcbff9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:45.716266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adccbe-5802-4a1e-b9d3-9e0bdbcbff9f admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:45.716956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46adccbe-5802-4a1e-b9d3-9e0bdbcbff9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:45.718872 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46adccbe-5802-4a1e-b9d3-9e0bdbcbff9f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:45.746074 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4334/8753] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:37:45 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:37:45.750104 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d1732-1054-4750-ae1c-6f154479b5f3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:45.760608 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d1732-1054-4750-ae1c-6f154479b5f3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:45.780202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:45.820037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2540c98-25cb-4933-a1b8-86ce08f88615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:45.820661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2540c98-25cb-4933-a1b8-86ce08f88615 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:45.820980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2540c98-25cb-4933-a1b8-86ce08f88615 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:45.821265 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2540c98-25cb-4933-a1b8-86ce08f88615 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:45.821947 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2540c98-25cb-4933-a1b8-86ce08f88615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:45.823883 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2540c98-25cb-4933-a1b8-86ce08f88615 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:45.845935 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4420/8754] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:37:45 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:37:45.850552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adccbe-5802-4a1e-b9d3-9e0bdbcbff9f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:45.864806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adccbe-5802-4a1e-b9d3-9e0bdbcbff9f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:45.869938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743d29d-b279-4ed6-bca0-7e18b253ff81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:45.870555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743d29d-b279-4ed6-bca0-7e18b253ff81 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:45.870840 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743d29d-b279-4ed6-bca0-7e18b253ff81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:45.871121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743d29d-b279-4ed6-bca0-7e18b253ff81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:45.871769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743d29d-b279-4ed6-bca0-7e18b253ff81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:45.873612 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743d29d-b279-4ed6-bca0-7e18b253ff81 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:45.884729 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4335/8755] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:37:45.888972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2540c98-25cb-4933-a1b8-86ce08f88615 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:45.899771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2540c98-25cb-4933-a1b8-86ce08f88615 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:45.905251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f512e-0ef5-449f-8c2c-ca1c21836cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:45.905865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f512e-0ef5-449f-8c2c-ca1c21836cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:45.906154 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f512e-0ef5-449f-8c2c-ca1c21836cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:45.906435 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f512e-0ef5-449f-8c2c-ca1c21836cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:45.907083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f512e-0ef5-449f-8c2c-ca1c21836cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:45.908849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f512e-0ef5-449f-8c2c-ca1c21836cf3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:45.919667 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4421/8756] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:37:45.924428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743d29d-b279-4ed6-bca0-7e18b253ff81 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:45.934710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743d29d-b279-4ed6-bca0-7e18b253ff81 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:45.940019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aad0ba-3c4d-4232-bb95-c760be340059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:45.940675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aad0ba-3c4d-4232-bb95-c760be340059 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:45.940987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aad0ba-3c4d-4232-bb95-c760be340059 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:45.941269 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aad0ba-3c4d-4232-bb95-c760be340059 admin admin] PATH_INFO: `/v3/projects/343adbc1fc324643aead015bda022815/users/dab241c4033f4f65b5c5008232d2ea30/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:45.955954 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aad0ba-3c4d-4232-bb95-c760be340059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=343adbc1fc324643aead015bda022815, user_id=dab241c4033f4f65b5c5008232d2ea30, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:45.958716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aad0ba-3c4d-4232-bb95-c760be340059 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:45.960947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:45.972815 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4336/8757] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:37:45 2022] PUT /identity/v3/projects/343adbc1fc324643aead015bda022815/users/dab241c4033f4f65b5c5008232d2ea30/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:37:45.982961 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666e417-baaa-4c48-b903-a2342229a89e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:45.983255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666e417-baaa-4c48-b903-a2342229a89e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:45.983538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666e417-baaa-4c48-b903-a2342229a89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:46.012868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f666e417-baaa-4c48-b903-a2342229a89e None None] MFA Rules not processed for user `dab241c4033f4f65b5c5008232d2ea30`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:46.014111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f666e417-baaa-4c48-b903-a2342229a89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:46.078701 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4422/8758] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:37:46.088768 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aad0ba-3c4d-4232-bb95-c760be340059 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:46.090588 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-13aad0ba-3c4d-4232-bb95-c760be340059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:46.148225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aad0ba-3c4d-4232-bb95-c760be340059 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:46.152046 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595144a-0692-4451-b7cd-50d0ce4178d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:46.152662 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595144a-0692-4451-b7cd-50d0ce4178d8 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:46.152971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595144a-0692-4451-b7cd-50d0ce4178d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:46.153255 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595144a-0692-4451-b7cd-50d0ce4178d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:46.162212 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595144a-0692-4451-b7cd-50d0ce4178d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:46.163666 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595144a-0692-4451-b7cd-50d0ce4178d8 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:46.175887 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4337/8759] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:37:56.075752 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:56.168584 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:56.825357 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666e417-baaa-4c48-b903-a2342229a89e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:56.889763 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666e417-baaa-4c48-b903-a2342229a89e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:56.894854 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a01191-0a79-46e5-ac89-7d03006f3b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:56.895512 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a01191-0a79-46e5-ac89-7d03006f3b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:56.895807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a01191-0a79-46e5-ac89-7d03006f3b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:56.896089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a01191-0a79-46e5-ac89-7d03006f3b7b admin admin] PATH_INFO: `/v3/users/dab241c4033f4f65b5c5008232d2ea30` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:56.897375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a01191-0a79-46e5-ac89-7d03006f3b7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dab241c4033f4f65b5c5008232d2ea30)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:56.899972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a01191-0a79-46e5-ac89-7d03006f3b7b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:56.949138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3a01191-0a79-46e5-ac89-7d03006f3b7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dab241c4033f4f65b5c5008232d2ea30'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:56.961494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3a01191-0a79-46e5-ac89-7d03006f3b7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dab241c4033f4f65b5c5008232d2ea30'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:56.969137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3a01191-0a79-46e5-ac89-7d03006f3b7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dab241c4033f4f65b5c5008232d2ea30'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:56.970388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3a01191-0a79-46e5-ac89-7d03006f3b7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dab241c4033f4f65b5c5008232d2ea30'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:56.971029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c3a01191-0a79-46e5-ac89-7d03006f3b7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dab241c4033f4f65b5c5008232d2ea30'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:56.983251 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4423/8760] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:37:56 2022] DELETE /identity/v3/users/dab241c4033f4f65b5c5008232d2ea30 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:37:56.988164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c90f75-daa5-4aab-90bf-7ef912214c8f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:56.988460 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c90f75-daa5-4aab-90bf-7ef912214c8f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:56.988747 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c90f75-daa5-4aab-90bf-7ef912214c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.008506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a1c90f75-daa5-4aab-90bf-7ef912214c8f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:57.009947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a1c90f75-daa5-4aab-90bf-7ef912214c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:57.069283 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4338/8761] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:57.078635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01275d09-7270-4ec7-b688-d9aae741fa90 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:57.079581 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01275d09-7270-4ec7-b688-d9aae741fa90 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:57.079685 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01275d09-7270-4ec7-b688-d9aae741fa90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.111929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-01275d09-7270-4ec7-b688-d9aae741fa90 None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:57.113169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-01275d09-7270-4ec7-b688-d9aae741fa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:57.165487 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4424/8762] 10.222.0.16 () {58 vars in 1062 bytes} [Sat Sep 3 21:37:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:57.171344 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c90f75-daa5-4aab-90bf-7ef912214c8f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:57.180885 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c90f75-daa5-4aab-90bf-7ef912214c8f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:57.184468 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a500e-0ac4-4593-ba06-e4e2ddbb4286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:57.185107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a500e-0ac4-4593-ba06-e4e2ddbb4286 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:57.185395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a500e-0ac4-4593-ba06-e4e2ddbb4286 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:57.185673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a500e-0ac4-4593-ba06-e4e2ddbb4286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.194637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a500e-0ac4-4593-ba06-e4e2ddbb4286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:57.196057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a500e-0ac4-4593-ba06-e4e2ddbb4286 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:57.206189 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4339/8763] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:37:57.372116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01275d09-7270-4ec7-b688-d9aae741fa90 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:57.375220 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-01275d09-7270-4ec7-b688-d9aae741fa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:57.390984 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01275d09-7270-4ec7-b688-d9aae741fa90 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:57.400204 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a683c9-ca16-4324-aa14-3fe8f2174542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:57.400868 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a683c9-ca16-4324-aa14-3fe8f2174542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:57.401388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a683c9-ca16-4324-aa14-3fe8f2174542 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:57.401582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a683c9-ca16-4324-aa14-3fe8f2174542 admin admin] PATH_INFO: `/v3/projects/343adbc1fc324643aead015bda022815` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.402793 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a683c9-ca16-4324-aa14-3fe8f2174542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=343adbc1fc324643aead015bda022815)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:57.404729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a683c9-ca16-4324-aa14-3fe8f2174542 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:57.417324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:57.452515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-80a683c9-ca16-4324-aa14-3fe8f2174542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '343adbc1fc324643aead015bda022815', 'request_id': 'req-80a683c9-ca16-4324-aa14-3fe8f2174542'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:57.460260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-80a683c9-ca16-4324-aa14-3fe8f2174542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '343adbc1fc324643aead015bda022815', 'request_id': 'req-80a683c9-ca16-4324-aa14-3fe8f2174542'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:57.466624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-80a683c9-ca16-4324-aa14-3fe8f2174542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '343adbc1fc324643aead015bda022815', 'request_id': 'req-80a683c9-ca16-4324-aa14-3fe8f2174542'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:57.477093 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-80a683c9-ca16-4324-aa14-3fe8f2174542 admin admin] The token cache is being invalidate because project 343adbc1fc324643aead015bda022815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:37:57.477394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-80a683c9-ca16-4324-aa14-3fe8f2174542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:57.478027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-80a683c9-ca16-4324-aa14-3fe8f2174542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:37:57.479509 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4425/8764] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:37:57 2022] DELETE /identity/v3/projects/343adbc1fc324643aead015bda022815 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:37:57.484891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22b2f8-4e47-46a9-992b-88178576c03f None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:57.485201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22b2f8-4e47-46a9-992b-88178576c03f None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:57.485478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22b2f8-4e47-46a9-992b-88178576c03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.505295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ea22b2f8-4e47-46a9-992b-88178576c03f None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:57.506490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ea22b2f8-4e47-46a9-992b-88178576c03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:57.564649 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4340/8765] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:57.572336 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f151f-5f9c-4255-b983-615b4e30a6d8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:57.572632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f151f-5f9c-4255-b983-615b4e30a6d8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:57.572942 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f151f-5f9c-4255-b983-615b4e30a6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.593142 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-745f151f-5f9c-4255-b983-615b4e30a6d8 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:57.594372 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-745f151f-5f9c-4255-b983-615b4e30a6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:57.602349 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4426/8766] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:37:57.606408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22b2f8-4e47-46a9-992b-88178576c03f None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:57.618891 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22b2f8-4e47-46a9-992b-88178576c03f None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:57.623569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fd755-15cf-4f32-b04f-9b377821193e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:57.624164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fd755-15cf-4f32-b04f-9b377821193e admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:57.624448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fd755-15cf-4f32-b04f-9b377821193e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:57.624729 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fd755-15cf-4f32-b04f-9b377821193e admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.625400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fd755-15cf-4f32-b04f-9b377821193e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:37:57.625707 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fd755-15cf-4f32-b04f-9b377821193e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:57.627802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fd755-15cf-4f32-b04f-9b377821193e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:57.639135 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4341/8767] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:37:57.644008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f151f-5f9c-4255-b983-615b4e30a6d8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:57.653444 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f151f-5f9c-4255-b983-615b4e30a6d8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:57.658043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef4d40-31d6-416b-858f-7d7342affc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:57.658635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef4d40-31d6-416b-858f-7d7342affc63 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:57.658918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef4d40-31d6-416b-858f-7d7342affc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:57.659193 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef4d40-31d6-416b-858f-7d7342affc63 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.659813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef4d40-31d6-416b-858f-7d7342affc63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:57.661830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef4d40-31d6-416b-858f-7d7342affc63 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:57.688987 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4427/8768] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:37:57 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:37:57.693035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fd755-15cf-4f32-b04f-9b377821193e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:57.703323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fd755-15cf-4f32-b04f-9b377821193e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:57.760242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5efba-e2b4-489b-9b79-57f2169a3609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:57.760844 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5efba-e2b4-489b-9b79-57f2169a3609 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:57.761166 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5efba-e2b4-489b-9b79-57f2169a3609 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:57.761446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5efba-e2b4-489b-9b79-57f2169a3609 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.762086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5efba-e2b4-489b-9b79-57f2169a3609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:57.764020 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5efba-e2b4-489b-9b79-57f2169a3609 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:57.784800 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4342/8769] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:37:57 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:37:57.789089 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef4d40-31d6-416b-858f-7d7342affc63 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:57.800020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef4d40-31d6-416b-858f-7d7342affc63 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:57.804693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabf1eb-e864-4d46-8d84-b72b0d9a4d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:57.805327 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabf1eb-e864-4d46-8d84-b72b0d9a4d0e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:57.805615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabf1eb-e864-4d46-8d84-b72b0d9a4d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:57.805911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabf1eb-e864-4d46-8d84-b72b0d9a4d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.806619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabf1eb-e864-4d46-8d84-b72b0d9a4d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:57.808129 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabf1eb-e864-4d46-8d84-b72b0d9a4d0e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:57.818981 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4428/8770] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:37:57.823146 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5efba-e2b4-489b-9b79-57f2169a3609 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:57.829832 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:37:57.834416 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5efba-e2b4-489b-9b79-57f2169a3609 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:57.840414 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbd019-c981-47b6-8814-d688a9245aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:57.841103 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbd019-c981-47b6-8814-d688a9245aba admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:57.841389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbd019-c981-47b6-8814-d688a9245aba admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:57.841669 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbd019-c981-47b6-8814-d688a9245aba admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.842313 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbd019-c981-47b6-8814-d688a9245aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:57.843945 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbd019-c981-47b6-8814-d688a9245aba admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:57.858285 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4343/8771] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:37:57.859408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabf1eb-e864-4d46-8d84-b72b0d9a4d0e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:57.870385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabf1eb-e864-4d46-8d84-b72b0d9a4d0e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:57.875133 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3235f51-3352-4dec-b509-8863e18b9700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:57.875789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3235f51-3352-4dec-b509-8863e18b9700 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:57.876658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3235f51-3352-4dec-b509-8863e18b9700 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:57.876967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3235f51-3352-4dec-b509-8863e18b9700 admin admin] PATH_INFO: `/v3/projects/d567a52176494b9d916caaf95fae8c57/users/6f8108f9d2c04000aa8685a9bdd43386/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.891510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3235f51-3352-4dec-b509-8863e18b9700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d567a52176494b9d916caaf95fae8c57, user_id=6f8108f9d2c04000aa8685a9bdd43386, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:57.894236 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3235f51-3352-4dec-b509-8863e18b9700 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:57.909882 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4429/8772] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:37:57 2022] PUT /identity/v3/projects/d567a52176494b9d916caaf95fae8c57/users/6f8108f9d2c04000aa8685a9bdd43386/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:37:57.920531 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d91b0b-2aee-462a-b7e7-389a1b14d726 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:57.920819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d91b0b-2aee-462a-b7e7-389a1b14d726 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:57.921136 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d91b0b-2aee-462a-b7e7-389a1b14d726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:57.949445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-30d91b0b-2aee-462a-b7e7-389a1b14d726 None None] MFA Rules not processed for user `6f8108f9d2c04000aa8685a9bdd43386`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:37:57.950649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-30d91b0b-2aee-462a-b7e7-389a1b14d726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:58.015062 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4344/8773] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:37:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:37:58.024971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3235f51-3352-4dec-b509-8863e18b9700 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:37:58.026673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a3235f51-3352-4dec-b509-8863e18b9700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:37:58.082233 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3235f51-3352-4dec-b509-8863e18b9700 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:37:58.085975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c48c7-0dba-4120-ad9f-d71c48082981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:37:58.086625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c48c7-0dba-4120-ad9f-d71c48082981 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:37:58.086968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c48c7-0dba-4120-ad9f-d71c48082981 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:37:58.087247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c48c7-0dba-4120-ad9f-d71c48082981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:37:58.095969 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c48c7-0dba-4120-ad9f-d71c48082981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:37:58.097518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c48c7-0dba-4120-ad9f-d71c48082981 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:37:58.106269 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4430/8774] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:37:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:38:02.203012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d91b0b-2aee-462a-b7e7-389a1b14d726 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:02.266233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d91b0b-2aee-462a-b7e7-389a1b14d726 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:02.270750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:02.271381 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:02.271671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:02.271959 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] PATH_INFO: `/v3/users/6f8108f9d2c04000aa8685a9bdd43386` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:02.273127 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f8108f9d2c04000aa8685a9bdd43386)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:02.275053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:02.323300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f8108f9d2c04000aa8685a9bdd43386'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:38:02.335265 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f8108f9d2c04000aa8685a9bdd43386'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:38:02.335975 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f8108f9d2c04000aa8685a9bdd43386'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:38:02.342918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f8108f9d2c04000aa8685a9bdd43386'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:38:02.350014 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f8108f9d2c04000aa8685a9bdd43386'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:38:02.355984 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4345/8775] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:38:02 2022] DELETE /identity/v3/users/6f8108f9d2c04000aa8685a9bdd43386 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:38:02.360747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee96984-ca76-4ae2-b58a-6fcf1f66b1a3 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:02.361761 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee96984-ca76-4ae2-b58a-6fcf1f66b1a3 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:02.361858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee96984-ca76-4ae2-b58a-6fcf1f66b1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:02.381397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2ee96984-ca76-4ae2-b58a-6fcf1f66b1a3 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:02.382634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2ee96984-ca76-4ae2-b58a-6fcf1f66b1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:02.415345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:38:02.443780 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4431/8776] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:02.453568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:02.455048 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:02.460549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:38:02.515434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97389c-5118-4082-9cb7-28e98cdd6d14 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:02.519227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228f455-6069-495c-afa6-1a500530e34c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:02.519784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228f455-6069-495c-afa6-1a500530e34c service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:02.520074 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228f455-6069-495c-afa6-1a500530e34c service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:02.520354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228f455-6069-495c-afa6-1a500530e34c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:02.529370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228f455-6069-495c-afa6-1a500530e34c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:02.530804 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228f455-6069-495c-afa6-1a500530e34c service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:02.540257 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4346/8777] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:38:02.710972 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee96984-ca76-4ae2-b58a-6fcf1f66b1a3 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:02.712852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2ee96984-ca76-4ae2-b58a-6fcf1f66b1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:02.727008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee96984-ca76-4ae2-b58a-6fcf1f66b1a3 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:02.733052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d66b97-9fd3-463f-8e18-4735177a009c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:02.733593 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d66b97-9fd3-463f-8e18-4735177a009c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:02.733831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d66b97-9fd3-463f-8e18-4735177a009c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:02.734118 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d66b97-9fd3-463f-8e18-4735177a009c admin admin] PATH_INFO: `/v3/projects/d567a52176494b9d916caaf95fae8c57` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:02.735465 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d66b97-9fd3-463f-8e18-4735177a009c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d567a52176494b9d916caaf95fae8c57)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:02.738072 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d66b97-9fd3-463f-8e18-4735177a009c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:02.787777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-47d66b97-9fd3-463f-8e18-4735177a009c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd567a52176494b9d916caaf95fae8c57', 'request_id': 'req-47d66b97-9fd3-463f-8e18-4735177a009c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:38:02.795238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-47d66b97-9fd3-463f-8e18-4735177a009c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd567a52176494b9d916caaf95fae8c57', 'request_id': 'req-47d66b97-9fd3-463f-8e18-4735177a009c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:38:02.802330 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-47d66b97-9fd3-463f-8e18-4735177a009c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd567a52176494b9d916caaf95fae8c57', 'request_id': 'req-47d66b97-9fd3-463f-8e18-4735177a009c'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:38:02.813140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-47d66b97-9fd3-463f-8e18-4735177a009c admin admin] The token cache is being invalidate because project d567a52176494b9d916caaf95fae8c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:38:02.813439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-47d66b97-9fd3-463f-8e18-4735177a009c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:38:02.814105 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-47d66b97-9fd3-463f-8e18-4735177a009c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:38:02.815656 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4432/8778] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:38:02 2022] DELETE /identity/v3/projects/d567a52176494b9d916caaf95fae8c57 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:38:02.821552 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50bfea-5fc7-4c2e-9ddd-ee08b56983c1 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:02.821847 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50bfea-5fc7-4c2e-9ddd-ee08b56983c1 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:02.822161 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50bfea-5fc7-4c2e-9ddd-ee08b56983c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:02.841594 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4a50bfea-5fc7-4c2e-9ddd-ee08b56983c1 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:02.842841 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4a50bfea-5fc7-4c2e-9ddd-ee08b56983c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:02.901748 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4347/8779] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:02.910036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6279e8-a2d6-4df4-90ef-cc8051f0ec8d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:02.910335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6279e8-a2d6-4df4-90ef-cc8051f0ec8d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:02.910644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6279e8-a2d6-4df4-90ef-cc8051f0ec8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:02.931128 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-df6279e8-a2d6-4df4-90ef-cc8051f0ec8d None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:02.932334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-df6279e8-a2d6-4df4-90ef-cc8051f0ec8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:02.940231 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4433/8780] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:38:02.944572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50bfea-5fc7-4c2e-9ddd-ee08b56983c1 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:02.955228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50bfea-5fc7-4c2e-9ddd-ee08b56983c1 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:02.960230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b558cc-ae56-481f-9512-cda84ae3e408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:02.961277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b558cc-ae56-481f-9512-cda84ae3e408 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:02.961568 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b558cc-ae56-481f-9512-cda84ae3e408 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:02.961842 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b558cc-ae56-481f-9512-cda84ae3e408 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:02.962485 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b558cc-ae56-481f-9512-cda84ae3e408 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:38:02.962788 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b558cc-ae56-481f-9512-cda84ae3e408 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:02.964365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b558cc-ae56-481f-9512-cda84ae3e408 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:02.977688 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4348/8781] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:38:02.981743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6279e8-a2d6-4df4-90ef-cc8051f0ec8d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:02.991736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6279e8-a2d6-4df4-90ef-cc8051f0ec8d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:02.996533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c694b79-634c-4471-9e9e-27b386ea3a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:02.997172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c694b79-634c-4471-9e9e-27b386ea3a29 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:02.997466 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c694b79-634c-4471-9e9e-27b386ea3a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:02.997748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c694b79-634c-4471-9e9e-27b386ea3a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:02.998393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c694b79-634c-4471-9e9e-27b386ea3a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:03.000315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c694b79-634c-4471-9e9e-27b386ea3a29 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:03.027277 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4434/8782] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:38:02 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:38:03.031363 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b558cc-ae56-481f-9512-cda84ae3e408 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:03.042312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b558cc-ae56-481f-9512-cda84ae3e408 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:03.097922 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c24bf-5d5e-4a0b-9887-a2068966ae09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:03.098528 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c24bf-5d5e-4a0b-9887-a2068966ae09 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:03.098813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c24bf-5d5e-4a0b-9887-a2068966ae09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:03.099102 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c24bf-5d5e-4a0b-9887-a2068966ae09 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:03.099752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c24bf-5d5e-4a0b-9887-a2068966ae09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:03.101724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c24bf-5d5e-4a0b-9887-a2068966ae09 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:03.123381 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4349/8783] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:38:03 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:38:03.127588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c694b79-634c-4471-9e9e-27b386ea3a29 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:03.138196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c694b79-634c-4471-9e9e-27b386ea3a29 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:03.143455 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24123f32-04b7-4b3d-951e-d1d124545e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:03.144055 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24123f32-04b7-4b3d-951e-d1d124545e5e admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:03.144345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24123f32-04b7-4b3d-951e-d1d124545e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:03.144625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24123f32-04b7-4b3d-951e-d1d124545e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:03.145300 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24123f32-04b7-4b3d-951e-d1d124545e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:03.146913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24123f32-04b7-4b3d-951e-d1d124545e5e admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:03.156644 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4435/8784] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:38:03.161247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c24bf-5d5e-4a0b-9887-a2068966ae09 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:03.171304 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c24bf-5d5e-4a0b-9887-a2068966ae09 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:03.176012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175dd58b-e6c2-4bd3-8803-be4acdafbc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:03.176608 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dd58b-e6c2-4bd3-8803-be4acdafbc01 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:03.176923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dd58b-e6c2-4bd3-8803-be4acdafbc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:03.177217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dd58b-e6c2-4bd3-8803-be4acdafbc01 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:03.177852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175dd58b-e6c2-4bd3-8803-be4acdafbc01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:03.179489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175dd58b-e6c2-4bd3-8803-be4acdafbc01 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:03.190740 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4350/8785] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:38:03.194962 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24123f32-04b7-4b3d-951e-d1d124545e5e admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:03.205462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24123f32-04b7-4b3d-951e-d1d124545e5e admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:03.211115 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5793cda-1d0d-4793-8bf9-11e5b67adbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:03.212057 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5793cda-1d0d-4793-8bf9-11e5b67adbd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:03.212408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5793cda-1d0d-4793-8bf9-11e5b67adbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:03.212730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5793cda-1d0d-4793-8bf9-11e5b67adbd8 admin admin] PATH_INFO: `/v3/projects/fd7ad842395c4865bcc994a9dc9f9900/users/65b80b19aac6432f85b33c3405b26ab0/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:03.228850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5793cda-1d0d-4793-8bf9-11e5b67adbd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7ad842395c4865bcc994a9dc9f9900, user_id=65b80b19aac6432f85b33c3405b26ab0, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:03.231596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5793cda-1d0d-4793-8bf9-11e5b67adbd8 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:03.244019 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4436/8786] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:38:03 2022] PUT /identity/v3/projects/fd7ad842395c4865bcc994a9dc9f9900/users/65b80b19aac6432f85b33c3405b26ab0/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:38:03.249093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a95eb-f76a-4a9b-baf4-7fc3fe31775e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:03.249541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a95eb-f76a-4a9b-baf4-7fc3fe31775e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:03.249675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a95eb-f76a-4a9b-baf4-7fc3fe31775e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:03.269109 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-889a95eb-f76a-4a9b-baf4-7fc3fe31775e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:03.270316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-889a95eb-f76a-4a9b-baf4-7fc3fe31775e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:03.275752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:38:03.330684 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4351/8787] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:03.340355 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5793cda-1d0d-4793-8bf9-11e5b67adbd8 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:03.342102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c5793cda-1d0d-4793-8bf9-11e5b67adbd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:03.404729 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5793cda-1d0d-4793-8bf9-11e5b67adbd8 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:03.408413 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207d4b8-68b0-428b-a0f4-11888783a77d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:03.409054 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207d4b8-68b0-428b-a0f4-11888783a77d service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:03.409339 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207d4b8-68b0-428b-a0f4-11888783a77d service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:03.409617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207d4b8-68b0-428b-a0f4-11888783a77d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:03.418527 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207d4b8-68b0-428b-a0f4-11888783a77d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:03.419954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207d4b8-68b0-428b-a0f4-11888783a77d service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:03.429503 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4437/8788] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:38:07.779483 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80d325-adc1-41a1-a545-69249c0135a4 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:07.779933 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80d325-adc1-41a1-a545-69249c0135a4 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:07.780038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80d325-adc1-41a1-a545-69249c0135a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:07.809874 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3f80d325-adc1-41a1-a545-69249c0135a4 None None] MFA Rules not processed for user `65b80b19aac6432f85b33c3405b26ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:07.811659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3f80d325-adc1-41a1-a545-69249c0135a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:07.893045 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4352/8789] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:38:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:07.897701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207d4b8-68b0-428b-a0f4-11888783a77d service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:07.902802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:38:07.909039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207d4b8-68b0-428b-a0f4-11888783a77d service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:07.918025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8af34-ceda-4fee-a804-e165a1a4c49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:07.918781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8af34-ceda-4fee-a804-e165a1a4c49c admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:07.919388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8af34-ceda-4fee-a804-e165a1a4c49c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:07.919576 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8af34-ceda-4fee-a804-e165a1a4c49c admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:07.920661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8af34-ceda-4fee-a804-e165a1a4c49c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:07.923061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8af34-ceda-4fee-a804-e165a1a4c49c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:07.956176 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4438/8790] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:38:07 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:38:07.960193 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80d325-adc1-41a1-a545-69249c0135a4 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:07.972298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80d325-adc1-41a1-a545-69249c0135a4 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:08.042370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abbcb8-d07a-409d-ba41-3f999f732490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:08.043075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abbcb8-d07a-409d-ba41-3f999f732490 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:08.043365 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abbcb8-d07a-409d-ba41-3f999f732490 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:08.043673 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abbcb8-d07a-409d-ba41-3f999f732490 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:08.044734 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abbcb8-d07a-409d-ba41-3f999f732490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:08.046757 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abbcb8-d07a-409d-ba41-3f999f732490 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:08.072307 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4353/8791] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:38:07 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:38:08.076424 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8af34-ceda-4fee-a804-e165a1a4c49c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:08.088263 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8af34-ceda-4fee-a804-e165a1a4c49c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:08.093082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b242b77-bace-4bc7-a13f-87e79e6a2ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:08.093697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b242b77-bace-4bc7-a13f-87e79e6a2ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:08.093977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b242b77-bace-4bc7-a13f-87e79e6a2ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:08.094256 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b242b77-bace-4bc7-a13f-87e79e6a2ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:08.094913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b242b77-bace-4bc7-a13f-87e79e6a2ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:08.096610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b242b77-bace-4bc7-a13f-87e79e6a2ea4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:08.108832 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4439/8792] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:38:08.113683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abbcb8-d07a-409d-ba41-3f999f732490 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:08.130573 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abbcb8-d07a-409d-ba41-3f999f732490 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:08.138498 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a26375-7467-45e3-bda7-f78da44eaafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:08.139253 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a26375-7467-45e3-bda7-f78da44eaafe admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:08.139537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a26375-7467-45e3-bda7-f78da44eaafe admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:08.139816 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a26375-7467-45e3-bda7-f78da44eaafe admin admin] PATH_INFO: `/v3/projects/4cc64490bc674597a2e37ca13ded0e83/users/2ac1b8c67b194dfe97aeba76f7f08169/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:08.157837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a26375-7467-45e3-bda7-f78da44eaafe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc64490bc674597a2e37ca13ded0e83, user_id=2ac1b8c67b194dfe97aeba76f7f08169, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:08.160692 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a26375-7467-45e3-bda7-f78da44eaafe admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:08.179671 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4354/8793] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:38:08 2022] PUT /identity/v3/projects/4cc64490bc674597a2e37ca13ded0e83/users/2ac1b8c67b194dfe97aeba76f7f08169/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:38:08.183656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b242b77-bace-4bc7-a13f-87e79e6a2ea4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:08.195399 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b242b77-bace-4bc7-a13f-87e79e6a2ea4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:08.263184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4157d-f02f-4b5f-b97e-eeed01831e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:08.263736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4157d-f02f-4b5f-b97e-eeed01831e9c admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:08.264025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4157d-f02f-4b5f-b97e-eeed01831e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:08.264331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4157d-f02f-4b5f-b97e-eeed01831e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:08.265039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b4157d-f02f-4b5f-b97e-eeed01831e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:08.266666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b4157d-f02f-4b5f-b97e-eeed01831e9c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:08.278884 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4440/8794] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:38:08.283282 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a26375-7467-45e3-bda7-f78da44eaafe admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:08.295024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a26375-7467-45e3-bda7-f78da44eaafe admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:08.301694 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d0e74-686f-4e3a-a49d-c76e00213fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:08.302371 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d0e74-686f-4e3a-a49d-c76e00213fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:08.302653 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d0e74-686f-4e3a-a49d-c76e00213fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:08.302929 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d0e74-686f-4e3a-a49d-c76e00213fb1 admin admin] PATH_INFO: `/v3/projects/4cc64490bc674597a2e37ca13ded0e83/users/2ac1b8c67b194dfe97aeba76f7f08169/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:08.305097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d0e74-686f-4e3a-a49d-c76e00213fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc64490bc674597a2e37ca13ded0e83, user_id=2ac1b8c67b194dfe97aeba76f7f08169, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:08.307808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d0e74-686f-4e3a-a49d-c76e00213fb1 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:08.310299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:38:08.322811 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4355/8795] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:38:08 2022] PUT /identity/v3/projects/4cc64490bc674597a2e37ca13ded0e83/users/2ac1b8c67b194dfe97aeba76f7f08169/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:38:12.399161 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27969060-0268-4f52-951d-7033e74daad1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:12.399643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27969060-0268-4f52-951d-7033e74daad1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:12.399734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27969060-0268-4f52-951d-7033e74daad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:12.429946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-27969060-0268-4f52-951d-7033e74daad1 None None] MFA Rules not processed for user `2ac1b8c67b194dfe97aeba76f7f08169`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:12.431184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-27969060-0268-4f52-951d-7033e74daad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:12.504152 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4441/8796] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:38:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:12.531299 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d0e74-686f-4e3a-a49d-c76e00213fb1 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:12.534907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-756d0e74-686f-4e3a-a49d-c76e00213fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:12.638541 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d0e74-686f-4e3a-a49d-c76e00213fb1 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:12.644678 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4728a-a4c9-41a4-8b1e-b740023ae7b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:12.645337 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4728a-a4c9-41a4-8b1e-b740023ae7b1 service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:12.645627 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4728a-a4c9-41a4-8b1e-b740023ae7b1 service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:12.645926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4728a-a4c9-41a4-8b1e-b740023ae7b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:12.707187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4728a-a4c9-41a4-8b1e-b740023ae7b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:12.708648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4728a-a4c9-41a4-8b1e-b740023ae7b1 service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:12.718285 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4356/8797] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:38:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:38:12.901743 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bfcb2-f746-4c48-b299-b3954598506c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:12.902044 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bfcb2-f746-4c48-b299-b3954598506c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:12.902345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bfcb2-f746-4c48-b299-b3954598506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:12.959566 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-886bfcb2-f746-4c48-b299-b3954598506c None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:12.960869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-886bfcb2-f746-4c48-b299-b3954598506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:12.985062 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:38:13.043123 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4442/8798] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:38:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:13.097098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a249d5-c4b1-4bcd-8a08-b5ab221159be None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:13.097477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a249d5-c4b1-4bcd-8a08-b5ab221159be None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:13.097761 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a249d5-c4b1-4bcd-8a08-b5ab221159be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:13.163007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-01a249d5-c4b1-4bcd-8a08-b5ab221159be None None] MFA Rules not processed for user `83865463b2a84f99a136b73dffeeb378`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:13.164242 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-01a249d5-c4b1-4bcd-8a08-b5ab221159be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:13.225097 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:38:13.264054 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4357/8799] 10.222.0.16 () {58 vars in 1040 bytes} [Sat Sep 3 21:38:13 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:13.270234 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bfcb2-f746-4c48-b299-b3954598506c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:13.281954 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bfcb2-f746-4c48-b299-b3954598506c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:13.288808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776c065-5fb0-4476-b367-8b2def60f354 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:13.289453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776c065-5fb0-4476-b367-8b2def60f354 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:13.289746 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776c065-5fb0-4476-b367-8b2def60f354 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:13.290147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776c065-5fb0-4476-b367-8b2def60f354 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:13.302590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776c065-5fb0-4476-b367-8b2def60f354 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:13.304202 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776c065-5fb0-4476-b367-8b2def60f354 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:13.313110 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4443/8800] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:38:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:38:17.413477 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a249d5-c4b1-4bcd-8a08-b5ab221159be None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:17.415446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-01a249d5-c4b1-4bcd-8a08-b5ab221159be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:17.508718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a249d5-c4b1-4bcd-8a08-b5ab221159be None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:17.514418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070889e-0a09-4663-8819-7d83f958d126 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:17.515249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070889e-0a09-4663-8819-7d83f958d126 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:17.515838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070889e-0a09-4663-8819-7d83f958d126 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:17.516288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070889e-0a09-4663-8819-7d83f958d126 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:17.518257 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5070889e-0a09-4663-8819-7d83f958d126 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:17.535826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070889e-0a09-4663-8819-7d83f958d126 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:17.537750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070889e-0a09-4663-8819-7d83f958d126 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:17.558590 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4358/8801] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:38:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:38:22.077320 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e375b8d-361b-4468-bb1f-054355915d72 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:22.077895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e375b8d-361b-4468-bb1f-054355915d72 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:22.077895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e375b8d-361b-4468-bb1f-054355915d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:22.098822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0e375b8d-361b-4468-bb1f-054355915d72 None None] MFA Rules not processed for user `65b80b19aac6432f85b33c3405b26ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:22.100052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0e375b8d-361b-4468-bb1f-054355915d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:22.108750 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4444/8802] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:38:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:22.672158 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27d326-0d76-498f-aa75-4a5c5c9780db None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:22.672650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27d326-0d76-498f-aa75-4a5c5c9780db None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:22.672751 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27d326-0d76-498f-aa75-4a5c5c9780db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:22.693837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-3b27d326-0d76-498f-aa75-4a5c5c9780db None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:22.695162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-3b27d326-0d76-498f-aa75-4a5c5c9780db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:22.704270 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4359/8803] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 21:38:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:22.715986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e375b8d-361b-4468-bb1f-054355915d72 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:22.730696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e375b8d-361b-4468-bb1f-054355915d72 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:22.736146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd993079-dc06-4a18-91cc-984e9f943391 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:22.736799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd993079-dc06-4a18-91cc-984e9f943391 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:22.737694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd993079-dc06-4a18-91cc-984e9f943391 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:22.737977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd993079-dc06-4a18-91cc-984e9f943391 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:22.746538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd993079-dc06-4a18-91cc-984e9f943391 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:22.748117 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd993079-dc06-4a18-91cc-984e9f943391 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:22.758397 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4445/8804] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:38:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:38:23.909675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e24b96-ba92-4a68-8d33-223e7b6a0062 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:23.910133 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e24b96-ba92-4a68-8d33-223e7b6a0062 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:23.910227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e24b96-ba92-4a68-8d33-223e7b6a0062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:23.929776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-14e24b96-ba92-4a68-8d33-223e7b6a0062 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:23.930986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-14e24b96-ba92-4a68-8d33-223e7b6a0062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:23.938395 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4360/8805] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:38:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:23.951205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd993079-dc06-4a18-91cc-984e9f943391 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:23.963642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd993079-dc06-4a18-91cc-984e9f943391 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:23.968434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941098b2-36b3-45f8-b6f3-5def86a52966 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:23.969811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941098b2-36b3-45f8-b6f3-5def86a52966 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:23.970096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941098b2-36b3-45f8-b6f3-5def86a52966 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:23.970383 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941098b2-36b3-45f8-b6f3-5def86a52966 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:23.979140 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941098b2-36b3-45f8-b6f3-5def86a52966 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:23.980808 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941098b2-36b3-45f8-b6f3-5def86a52966 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:23.992776 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4446/8806] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:38:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:38:33.925083 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=42 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:38:33.988016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=41 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:38:44.071616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d0448-3e33-440e-9d4b-745aca29181d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:44.072094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d0448-3e33-440e-9d4b-745aca29181d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:44.072192 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d0448-3e33-440e-9d4b-745aca29181d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:44.091003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d99d0448-3e33-440e-9d4b-745aca29181d None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:44.092200 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d99d0448-3e33-440e-9d4b-745aca29181d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:44.099773 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4361/8807] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:38:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:44.113047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941098b2-36b3-45f8-b6f3-5def86a52966 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:44.123258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941098b2-36b3-45f8-b6f3-5def86a52966 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:44.128594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a243f73-8bfc-4186-a02c-81bace98e481 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:44.129253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a243f73-8bfc-4186-a02c-81bace98e481 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:44.129543 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a243f73-8bfc-4186-a02c-81bace98e481 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:44.129822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a243f73-8bfc-4186-a02c-81bace98e481 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:44.138207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a243f73-8bfc-4186-a02c-81bace98e481 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:44.139786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a243f73-8bfc-4186-a02c-81bace98e481 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:44.148751 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4447/8808] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:38:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 21:38:44.649249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf14e9-8b6f-4639-9942-27713b2e66a9 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:44.649545 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf14e9-8b6f-4639-9942-27713b2e66a9 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:44.649826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf14e9-8b6f-4639-9942-27713b2e66a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:44.670206 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-90bf14e9-8b6f-4639-9942-27713b2e66a9 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:44.671428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-90bf14e9-8b6f-4639-9942-27713b2e66a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:44.679400 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4362/8809] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:38:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:44.692251 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a243f73-8bfc-4186-a02c-81bace98e481 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:44.702653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a243f73-8bfc-4186-a02c-81bace98e481 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:44.708332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac36f7d-08ff-4447-928b-7720732f532f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:44.708960 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac36f7d-08ff-4447-928b-7720732f532f service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:44.709250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac36f7d-08ff-4447-928b-7720732f532f service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:44.709531 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac36f7d-08ff-4447-928b-7720732f532f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:44.717439 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac36f7d-08ff-4447-928b-7720732f532f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:44.719020 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac36f7d-08ff-4447-928b-7720732f532f service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:44.727750 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4448/8810] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:38:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 21:38:53.109077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a0775-bdf5-47a7-a0ac-78eda0764a4d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:53.109632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a0775-bdf5-47a7-a0ac-78eda0764a4d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:53.109632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a0775-bdf5-47a7-a0ac-78eda0764a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:53.129619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a42a0775-bdf5-47a7-a0ac-78eda0764a4d None None] MFA Rules not processed for user `65b80b19aac6432f85b33c3405b26ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:53.130839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a42a0775-bdf5-47a7-a0ac-78eda0764a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:53.138268 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4363/8811] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:38:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:54.724169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=12 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:38:54.937872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16f39-87d2-42d9-a675-7ad8dfa8ccf2 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:54.938170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16f39-87d2-42d9-a675-7ad8dfa8ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:54.938454 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16f39-87d2-42d9-a675-7ad8dfa8ccf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:54.959483 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f4d16f39-87d2-42d9-a675-7ad8dfa8ccf2 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:38:54.960908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f4d16f39-87d2-42d9-a675-7ad8dfa8ccf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:38:54.968370 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4449/8812] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:38:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:38:54.982079 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a0775-bdf5-47a7-a0ac-78eda0764a4d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:38:54.995056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a0775-bdf5-47a7-a0ac-78eda0764a4d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:38:55.000169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0570d05-3d0b-408a-ae95-a920c3682b49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:38:55.001132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0570d05-3d0b-408a-ae95-a920c3682b49 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:38:55.001421 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0570d05-3d0b-408a-ae95-a920c3682b49 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:38:55.001716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0570d05-3d0b-408a-ae95-a920c3682b49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:38:55.010837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0570d05-3d0b-408a-ae95-a920c3682b49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:38:55.012408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0570d05-3d0b-408a-ae95-a920c3682b49 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:38:55.020472 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4364/8813] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:38:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:39:04.953652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:39:05.017898 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=15 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:39:16.034110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab3073-9a64-4962-9bfc-5f1fe8d1eed1 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:16.034561 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab3073-9a64-4962-9bfc-5f1fe8d1eed1 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:16.034666 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab3073-9a64-4962-9bfc-5f1fe8d1eed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:16.053866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0bab3073-9a64-4962-9bfc-5f1fe8d1eed1 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:39:16.055050 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0bab3073-9a64-4962-9bfc-5f1fe8d1eed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:39:16.062114 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4450/8814] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:39:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:39:16.084470 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0570d05-3d0b-408a-ae95-a920c3682b49 service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:16.095169 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0570d05-3d0b-408a-ae95-a920c3682b49 service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:16.100236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f252e3d-5498-4834-ba1f-8f783913d22b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:16.100838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f252e3d-5498-4834-ba1f-8f783913d22b service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:16.101160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f252e3d-5498-4834-ba1f-8f783913d22b service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:16.101441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f252e3d-5498-4834-ba1f-8f783913d22b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:16.109610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f252e3d-5498-4834-ba1f-8f783913d22b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:16.111527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f252e3d-5498-4834-ba1f-8f783913d22b service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:16.119951 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4365/8815] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:39:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:39:22.902872 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab3073-9a64-4962-9bfc-5f1fe8d1eed1 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:22.969642 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab3073-9a64-4962-9bfc-5f1fe8d1eed1 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:22.974276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:22.974904 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:22.975188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:22.975463 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] PATH_INFO: `/v3/users/65b80b19aac6432f85b33c3405b26ab0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:22.976615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b80b19aac6432f85b33c3405b26ab0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:22.978582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:23.024848 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b80b19aac6432f85b33c3405b26ab0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.036774 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b80b19aac6432f85b33c3405b26ab0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.044303 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b80b19aac6432f85b33c3405b26ab0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.045069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b80b19aac6432f85b33c3405b26ab0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.045728 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b80b19aac6432f85b33c3405b26ab0'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.058137 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4451/8816] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:39:22 2022] DELETE /identity/v3/users/65b80b19aac6432f85b33c3405b26ab0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:39:23.239619 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f252e3d-5498-4834-ba1f-8f783913d22b service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:23.250952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f252e3d-5498-4834-ba1f-8f783913d22b service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:23.307912 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:23.308549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:23.308834 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:23.309144 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] PATH_INFO: `/v3/projects/fd7ad842395c4865bcc994a9dc9f9900` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:23.310263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7ad842395c4865bcc994a9dc9f9900)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:23.312400 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:23.358077 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7ad842395c4865bcc994a9dc9f9900', 'request_id': 'req-8b62067b-0ba6-4f37-801d-1b54843a4ce7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.366921 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7ad842395c4865bcc994a9dc9f9900', 'request_id': 'req-8b62067b-0ba6-4f37-801d-1b54843a4ce7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.373463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7ad842395c4865bcc994a9dc9f9900', 'request_id': 'req-8b62067b-0ba6-4f37-801d-1b54843a4ce7'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.385359 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] The token cache is being invalidate because project fd7ad842395c4865bcc994a9dc9f9900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:39:23.385649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.386293 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.387786 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4366/8817] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:39:23 2022] DELETE /identity/v3/projects/fd7ad842395c4865bcc994a9dc9f9900 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:39:23.391767 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:23.393198 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:39:23.408285 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de429d-a76f-4cbb-a26f-e73fdb6eca85 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:23.466261 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ee8ba-156a-4e31-9485-860cc3a7e263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:23.466895 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ee8ba-156a-4e31-9485-860cc3a7e263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:23.467180 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ee8ba-156a-4e31-9485-860cc3a7e263 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:23.467458 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ee8ba-156a-4e31-9485-860cc3a7e263 admin admin] PATH_INFO: `/v3/users/2ac1b8c67b194dfe97aeba76f7f08169` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:23.468590 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ee8ba-156a-4e31-9485-860cc3a7e263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac1b8c67b194dfe97aeba76f7f08169)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:23.470556 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ee8ba-156a-4e31-9485-860cc3a7e263 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:23.484170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:39:23.517182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-532ee8ba-156a-4e31-9485-860cc3a7e263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac1b8c67b194dfe97aeba76f7f08169'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.529628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-532ee8ba-156a-4e31-9485-860cc3a7e263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac1b8c67b194dfe97aeba76f7f08169'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.536658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-532ee8ba-156a-4e31-9485-860cc3a7e263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac1b8c67b194dfe97aeba76f7f08169'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.537322 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-532ee8ba-156a-4e31-9485-860cc3a7e263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac1b8c67b194dfe97aeba76f7f08169'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.537934 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-532ee8ba-156a-4e31-9485-860cc3a7e263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac1b8c67b194dfe97aeba76f7f08169'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.549871 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4452/8818] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:39:23 2022] DELETE /identity/v3/users/2ac1b8c67b194dfe97aeba76f7f08169 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:39:23.727232 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:23.729057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:39:23.742888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62067b-0ba6-4f37-801d-1b54843a4ce7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:23.801084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07425b38-d881-41a6-aa49-416b8a010851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:23.801741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07425b38-d881-41a6-aa49-416b8a010851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:23.802022 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07425b38-d881-41a6-aa49-416b8a010851 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:23.802294 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07425b38-d881-41a6-aa49-416b8a010851 admin admin] PATH_INFO: `/v3/projects/4cc64490bc674597a2e37ca13ded0e83` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:23.803505 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07425b38-d881-41a6-aa49-416b8a010851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cc64490bc674597a2e37ca13ded0e83)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:23.805520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07425b38-d881-41a6-aa49-416b8a010851 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:23.826448 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:39:23.853944 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07425b38-d881-41a6-aa49-416b8a010851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cc64490bc674597a2e37ca13ded0e83', 'request_id': 'req-07425b38-d881-41a6-aa49-416b8a010851'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.862034 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07425b38-d881-41a6-aa49-416b8a010851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cc64490bc674597a2e37ca13ded0e83', 'request_id': 'req-07425b38-d881-41a6-aa49-416b8a010851'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.869129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07425b38-d881-41a6-aa49-416b8a010851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cc64490bc674597a2e37ca13ded0e83', 'request_id': 'req-07425b38-d881-41a6-aa49-416b8a010851'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.880511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07425b38-d881-41a6-aa49-416b8a010851 admin admin] The token cache is being invalidate because project 4cc64490bc674597a2e37ca13ded0e83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:39:23.880810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07425b38-d881-41a6-aa49-416b8a010851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.881521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-07425b38-d881-41a6-aa49-416b8a010851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:39:23.883132 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4367/8819] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:39:23 2022] DELETE /identity/v3/projects/4cc64490bc674597a2e37ca13ded0e83 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:39:23.888286 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da862b26-90e1-410c-b3ef-360cc2af39ae None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:23.888649 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da862b26-90e1-410c-b3ef-360cc2af39ae None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:23.889037 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da862b26-90e1-410c-b3ef-360cc2af39ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:23.910940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-da862b26-90e1-410c-b3ef-360cc2af39ae None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:39:23.912964 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-da862b26-90e1-410c-b3ef-360cc2af39ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:39:23.974893 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4453/8820] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:39:23.982895 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7722ec8-33e6-4830-a963-45012addcb6b None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:23.983284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7722ec8-33e6-4830-a963-45012addcb6b None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:23.983596 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7722ec8-33e6-4830-a963-45012addcb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.005860 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a7722ec8-33e6-4830-a963-45012addcb6b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:39:24.007038 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a7722ec8-33e6-4830-a963-45012addcb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:39:24.015295 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4368/8821] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:39:24.020270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da862b26-90e1-410c-b3ef-360cc2af39ae None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.030905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da862b26-90e1-410c-b3ef-360cc2af39ae None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:24.036442 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54a52d-5442-4955-a111-0e4e01420188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:24.037109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54a52d-5442-4955-a111-0e4e01420188 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.037402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54a52d-5442-4955-a111-0e4e01420188 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.037716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54a52d-5442-4955-a111-0e4e01420188 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.038385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54a52d-5442-4955-a111-0e4e01420188 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:39:24.038721 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54a52d-5442-4955-a111-0e4e01420188 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:24.040381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54a52d-5442-4955-a111-0e4e01420188 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:24.055935 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4454/8822] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 21:39:24.060856 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7722ec8-33e6-4830-a963-45012addcb6b None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.076839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7722ec8-33e6-4830-a963-45012addcb6b None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:24.082221 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f65fbe-4aa8-496f-8869-e6299e2d849a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:24.082852 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f65fbe-4aa8-496f-8869-e6299e2d849a admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.083140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f65fbe-4aa8-496f-8869-e6299e2d849a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.083546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f65fbe-4aa8-496f-8869-e6299e2d849a admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.084203 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f65fbe-4aa8-496f-8869-e6299e2d849a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:24.086219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f65fbe-4aa8-496f-8869-e6299e2d849a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:24.121250 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4369/8823] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:39:24 2022] POST /identity/v3/projects => generated 391 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:39:24.125027 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54a52d-5442-4955-a111-0e4e01420188 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.137191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54a52d-5442-4955-a111-0e4e01420188 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:24.195101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4c0f2-f57a-45a6-8dae-06ae688d2189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:24.195710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4c0f2-f57a-45a6-8dae-06ae688d2189 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.195998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4c0f2-f57a-45a6-8dae-06ae688d2189 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.196278 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4c0f2-f57a-45a6-8dae-06ae688d2189 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.196918 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4c0f2-f57a-45a6-8dae-06ae688d2189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:24.199226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4c0f2-f57a-45a6-8dae-06ae688d2189 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:24.220004 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4455/8824] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:39:24 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:39:24.224646 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f65fbe-4aa8-496f-8869-e6299e2d849a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.235501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f65fbe-4aa8-496f-8869-e6299e2d849a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:24.240769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6cb4d-fafd-43f5-b99d-4a4cda602bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:24.241408 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6cb4d-fafd-43f5-b99d-4a4cda602bdd admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.241695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6cb4d-fafd-43f5-b99d-4a4cda602bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.241973 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6cb4d-fafd-43f5-b99d-4a4cda602bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.242615 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6cb4d-fafd-43f5-b99d-4a4cda602bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:24.244219 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6cb4d-fafd-43f5-b99d-4a4cda602bdd admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:24.254090 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4370/8825] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:39:24.258172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4c0f2-f57a-45a6-8dae-06ae688d2189 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.268364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4c0f2-f57a-45a6-8dae-06ae688d2189 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:24.273156 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9d7c1-f096-4bb6-8007-a402887ac114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:24.273765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9d7c1-f096-4bb6-8007-a402887ac114 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.274053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9d7c1-f096-4bb6-8007-a402887ac114 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.274332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9d7c1-f096-4bb6-8007-a402887ac114 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.274973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9d7c1-f096-4bb6-8007-a402887ac114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:24.276665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9d7c1-f096-4bb6-8007-a402887ac114 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:24.279036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:39:24.289842 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4456/8826] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:39:24.293187 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6cb4d-fafd-43f5-b99d-4a4cda602bdd admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.304285 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6cb4d-fafd-43f5-b99d-4a4cda602bdd admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:24.309350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280e473-df99-4953-8411-c1f10c460742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:24.310017 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e473-df99-4953-8411-c1f10c460742 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.310306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e473-df99-4953-8411-c1f10c460742 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.310589 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e473-df99-4953-8411-c1f10c460742 admin admin] PATH_INFO: `/v3/projects/9f2c6e6317c6424f83e70d52aee5b85d/users/22d9a2cf58104e3988e468a329eb6b80/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.325590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280e473-df99-4953-8411-c1f10c460742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f2c6e6317c6424f83e70d52aee5b85d, user_id=22d9a2cf58104e3988e468a329eb6b80, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:24.328277 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280e473-df99-4953-8411-c1f10c460742 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:24.340864 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4371/8827] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:39:24 2022] PUT /identity/v3/projects/9f2c6e6317c6424f83e70d52aee5b85d/users/22d9a2cf58104e3988e468a329eb6b80/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:39:24.352112 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f895d8-6a4c-4e9a-ad0e-129f7afaf4ff None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.352402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f895d8-6a4c-4e9a-ad0e-129f7afaf4ff None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.352682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f895d8-6a4c-4e9a-ad0e-129f7afaf4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.380304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-85f895d8-6a4c-4e9a-ad0e-129f7afaf4ff None None] MFA Rules not processed for user `22d9a2cf58104e3988e468a329eb6b80`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:39:24.381536 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-85f895d8-6a4c-4e9a-ad0e-129f7afaf4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:39:24.446289 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4457/8828] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:39:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:39:24.450773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280e473-df99-4953-8411-c1f10c460742 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.514086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280e473-df99-4953-8411-c1f10c460742 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:24.518771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aaa254-17ab-4df6-a960-92b4ee1f4acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:24.519380 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aaa254-17ab-4df6-a960-92b4ee1f4acb admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.519663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aaa254-17ab-4df6-a960-92b4ee1f4acb admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.519938 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aaa254-17ab-4df6-a960-92b4ee1f4acb admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.520567 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aaa254-17ab-4df6-a960-92b4ee1f4acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:24.522522 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aaa254-17ab-4df6-a960-92b4ee1f4acb admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:24.552176 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4372/8829] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:39:24 2022] POST /identity/v3/projects => generated 391 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:39:24.553425 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f895d8-6a4c-4e9a-ad0e-129f7afaf4ff None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.564159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f895d8-6a4c-4e9a-ad0e-129f7afaf4ff None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:24.619933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984406f-6d77-4f6d-931c-34bbe3355107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:24.620582 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984406f-6d77-4f6d-931c-34bbe3355107 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.620870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984406f-6d77-4f6d-931c-34bbe3355107 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.621183 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984406f-6d77-4f6d-931c-34bbe3355107 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.621823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984406f-6d77-4f6d-931c-34bbe3355107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:24.623748 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984406f-6d77-4f6d-931c-34bbe3355107 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:24.644420 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4458/8830] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:39:24 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:39:24.648495 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aaa254-17ab-4df6-a960-92b4ee1f4acb admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.658744 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aaa254-17ab-4df6-a960-92b4ee1f4acb admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:24.663355 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014f291-4731-4bc7-bc05-e1a581f79265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:24.663951 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014f291-4731-4bc7-bc05-e1a581f79265 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.664236 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014f291-4731-4bc7-bc05-e1a581f79265 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.664511 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014f291-4731-4bc7-bc05-e1a581f79265 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.665188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014f291-4731-4bc7-bc05-e1a581f79265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:24.666772 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014f291-4731-4bc7-bc05-e1a581f79265 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:24.677974 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4373/8831] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:39:24.682025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984406f-6d77-4f6d-931c-34bbe3355107 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.692505 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984406f-6d77-4f6d-931c-34bbe3355107 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:24.697176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55746f-9f8e-41da-8d89-34f44879fbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:24.697830 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55746f-9f8e-41da-8d89-34f44879fbc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.698116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55746f-9f8e-41da-8d89-34f44879fbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.698395 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55746f-9f8e-41da-8d89-34f44879fbc3 admin admin] PATH_INFO: `/v3/projects/e172f417ddc84f1386f93b20b9bd01d7/users/eda37c7fce364fec94d2e610316b3489/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.712869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55746f-9f8e-41da-8d89-34f44879fbc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e172f417ddc84f1386f93b20b9bd01d7, user_id=eda37c7fce364fec94d2e610316b3489, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:24.715549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55746f-9f8e-41da-8d89-34f44879fbc3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:24.727479 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4459/8832] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:39:24 2022] PUT /identity/v3/projects/e172f417ddc84f1386f93b20b9bd01d7/users/eda37c7fce364fec94d2e610316b3489/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:39:24.731372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014f291-4731-4bc7-bc05-e1a581f79265 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.742060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014f291-4731-4bc7-bc05-e1a581f79265 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:24.754730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:39:24.798441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2ca68-7c93-4976-85a7-06ad85d1ebbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:24.798974 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2ca68-7c93-4976-85a7-06ad85d1ebbc admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.799275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2ca68-7c93-4976-85a7-06ad85d1ebbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.799551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2ca68-7c93-4976-85a7-06ad85d1ebbc admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.800731 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2ca68-7c93-4976-85a7-06ad85d1ebbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:24.802357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2ca68-7c93-4976-85a7-06ad85d1ebbc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:24.811959 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4374/8833] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:39:24.817160 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55746f-9f8e-41da-8d89-34f44879fbc3 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.827617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55746f-9f8e-41da-8d89-34f44879fbc3 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:24.832526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13fd1c-90c3-48e9-a1c5-c437598c7764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:24.833217 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13fd1c-90c3-48e9-a1c5-c437598c7764 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.833506 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13fd1c-90c3-48e9-a1c5-c437598c7764 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.833784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13fd1c-90c3-48e9-a1c5-c437598c7764 admin admin] PATH_INFO: `/v3/projects/e172f417ddc84f1386f93b20b9bd01d7/users/eda37c7fce364fec94d2e610316b3489/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.835765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13fd1c-90c3-48e9-a1c5-c437598c7764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e172f417ddc84f1386f93b20b9bd01d7, user_id=eda37c7fce364fec94d2e610316b3489, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:24.838523 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13fd1c-90c3-48e9-a1c5-c437598c7764 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:24.848649 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4460/8834] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:39:24 2022] PUT /identity/v3/projects/e172f417ddc84f1386f93b20b9bd01d7/users/eda37c7fce364fec94d2e610316b3489/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:39:24.858513 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b4532-ab81-44c5-a579-cfc24e42de4d None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:24.858805 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b4532-ab81-44c5-a579-cfc24e42de4d None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:24.859088 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b4532-ab81-44c5-a579-cfc24e42de4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:24.889067 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-181b4532-ab81-44c5-a579-cfc24e42de4d None None] MFA Rules not processed for user `eda37c7fce364fec94d2e610316b3489`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:39:24.890312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-181b4532-ab81-44c5-a579-cfc24e42de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:39:24.957946 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4375/8835] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:39:24 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:39:24.973437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13fd1c-90c3-48e9-a1c5-c437598c7764 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:24.975341 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-8a13fd1c-90c3-48e9-a1c5-c437598c7764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:39:25.020423 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:39:25.065528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13fd1c-90c3-48e9-a1c5-c437598c7764 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:25.071179 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970844fd-86ac-432b-b8df-6518a73b8942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:25.071784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970844fd-86ac-432b-b8df-6518a73b8942 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:25.072071 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970844fd-86ac-432b-b8df-6518a73b8942 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:25.072350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970844fd-86ac-432b-b8df-6518a73b8942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:25.126402 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970844fd-86ac-432b-b8df-6518a73b8942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:25.128507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970844fd-86ac-432b-b8df-6518a73b8942 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:25.137512 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4461/8836] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:39:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:39:27.769341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b4532-ab81-44c5-a579-cfc24e42de4d None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:27.770752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-181b4532-ab81-44c5-a579-cfc24e42de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:39:27.827950 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b4532-ab81-44c5-a579-cfc24e42de4d None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:27.832385 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc5f94-045a-4cd6-bacb-56d7cdc912b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:27.833094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc5f94-045a-4cd6-bacb-56d7cdc912b0 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:27.833383 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc5f94-045a-4cd6-bacb-56d7cdc912b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:27.833663 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc5f94-045a-4cd6-bacb-56d7cdc912b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:27.843026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc5f94-045a-4cd6-bacb-56d7cdc912b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:27.844517 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc5f94-045a-4cd6-bacb-56d7cdc912b0 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:27.855172 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4376/8837] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:39:27 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:39:30.742305 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa2399-f6da-4063-aa63-3f3db89e472d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:30.742800 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa2399-f6da-4063-aa63-3f3db89e472d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:30.742889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa2399-f6da-4063-aa63-3f3db89e472d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:30.764638 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-e1aa2399-f6da-4063-aa63-3f3db89e472d None None] MFA Rules not processed for user `22d9a2cf58104e3988e468a329eb6b80`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:39:30.766002 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e1aa2399-f6da-4063-aa63-3f3db89e472d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:39:30.777925 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4462/8838] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:39:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:39:31.171226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc5f94-045a-4cd6-bacb-56d7cdc912b0 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:39:31.172708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-babc5f94-045a-4cd6-bacb-56d7cdc912b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:39:31.186524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc5f94-045a-4cd6-bacb-56d7cdc912b0 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:39:31.191792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f6865-3206-49ce-b78d-ed4d883a2b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:39:31.192433 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f6865-3206-49ce-b78d-ed4d883a2b1d service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:39:31.192720 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f6865-3206-49ce-b78d-ed4d883a2b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:39:31.193026 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f6865-3206-49ce-b78d-ed4d883a2b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:39:31.195215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ea6f6865-3206-49ce-b78d-ed4d883a2b1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:39:31.212934 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f6865-3206-49ce-b78d-ed4d883a2b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:39:31.214918 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f6865-3206-49ce-b78d-ed4d883a2b1d service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:39:31.237121 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4377/8839] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:39:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:39:40.757014 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=46 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:39:41.221642 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=87 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:40:49.178076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa2399-f6da-4063-aa63-3f3db89e472d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:40:49.187936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa2399-f6da-4063-aa63-3f3db89e472d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:40:49.191724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237e535-1e4d-4ca0-8d1d-b13581695bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:40:49.192328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237e535-1e4d-4ca0-8d1d-b13581695bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:40:49.192611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237e535-1e4d-4ca0-8d1d-b13581695bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:40:49.192885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237e535-1e4d-4ca0-8d1d-b13581695bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:40:49.194641 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d237e535-1e4d-4ca0-8d1d-b13581695bf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:40:49.211754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237e535-1e4d-4ca0-8d1d-b13581695bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:40:49.213216 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237e535-1e4d-4ca0-8d1d-b13581695bf2 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:40:49.227610 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4463/8840] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:40:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:40:50.417372 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f6865-3206-49ce-b78d-ed4d883a2b1d service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:40:50.425926 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f6865-3206-49ce-b78d-ed4d883a2b1d service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:40:50.429626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4238b-19d7-450e-9010-ecc14e51ec91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:40:50.430220 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4238b-19d7-450e-9010-ecc14e51ec91 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:40:50.430506 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4238b-19d7-450e-9010-ecc14e51ec91 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:40:50.430784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4238b-19d7-450e-9010-ecc14e51ec91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:40:50.432507 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7ab4238b-19d7-450e-9010-ecc14e51ec91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:40:50.448306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4238b-19d7-450e-9010-ecc14e51ec91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:40:50.449746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4238b-19d7-450e-9010-ecc14e51ec91 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:40:50.463660 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4378/8841] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:40:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:40:54.951218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237e535-1e4d-4ca0-8d1d-b13581695bf2 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:40:54.965110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237e535-1e4d-4ca0-8d1d-b13581695bf2 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:40:54.970794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4855b45-e89a-4bef-80b6-393973cbea58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:40:54.971400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4855b45-e89a-4bef-80b6-393973cbea58 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:40:54.971693 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4855b45-e89a-4bef-80b6-393973cbea58 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:40:54.971970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4855b45-e89a-4bef-80b6-393973cbea58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:40:54.973828 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-e4855b45-e89a-4bef-80b6-393973cbea58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:40:54.989789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4855b45-e89a-4bef-80b6-393973cbea58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:40:54.991915 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4855b45-e89a-4bef-80b6-393973cbea58 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:40:55.006130 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4464/8842] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:40:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:41:00.456626 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:41:04.998518 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=12 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:41:07.405743 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc660e60-083a-4935-821b-82c6bda0348c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:41:07.406576 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc660e60-083a-4935-821b-82c6bda0348c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:41:07.407024 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc660e60-083a-4935-821b-82c6bda0348c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:41:07.435389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cc660e60-083a-4935-821b-82c6bda0348c None None] MFA Rules not processed for user `22d9a2cf58104e3988e468a329eb6b80`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:41:07.436771 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cc660e60-083a-4935-821b-82c6bda0348c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:41:07.445083 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4379/8843] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:41:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:41:13.942654 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4855b45-e89a-4bef-80b6-393973cbea58 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:41:13.953144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4855b45-e89a-4bef-80b6-393973cbea58 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:41:13.956850 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b76ae-ba52-4ef6-8934-8484414c4445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:41:13.957535 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b76ae-ba52-4ef6-8934-8484414c4445 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:41:13.957822 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b76ae-ba52-4ef6-8934-8484414c4445 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:41:13.958102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b76ae-ba52-4ef6-8934-8484414c4445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:41:13.960474 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-5e8b76ae-ba52-4ef6-8934-8484414c4445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:41:13.973736 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b76ae-ba52-4ef6-8934-8484414c4445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:41:13.975786 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b76ae-ba52-4ef6-8934-8484414c4445 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:41:13.987710 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4465/8844] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:41:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:41:17.429524 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:41:23.980662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:42:36.224769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc660e60-083a-4935-821b-82c6bda0348c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:42:36.226346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cc660e60-083a-4935-821b-82c6bda0348c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:42:36.299343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc660e60-083a-4935-821b-82c6bda0348c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:42:36.304323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d906373-effd-4df5-9cd4-b4849172a68f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:42:36.304995 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d906373-effd-4df5-9cd4-b4849172a68f service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:42:36.305309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d906373-effd-4df5-9cd4-b4849172a68f service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:42:36.305590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d906373-effd-4df5-9cd4-b4849172a68f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:42:36.307298 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0d906373-effd-4df5-9cd4-b4849172a68f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:42:36.322209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d906373-effd-4df5-9cd4-b4849172a68f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:42:36.323792 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d906373-effd-4df5-9cd4-b4849172a68f service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:42:36.336324 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4380/8845] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:42:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:42:46.329671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=32 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:42:46.797021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b76ae-ba52-4ef6-8934-8484414c4445 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:42:46.814835 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b76ae-ba52-4ef6-8934-8484414c4445 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:42:46.820334 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ec402-8cba-4dfa-877d-114f5befc2f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:42:46.820986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ec402-8cba-4dfa-877d-114f5befc2f9 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:42:46.821398 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ec402-8cba-4dfa-877d-114f5befc2f9 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:42:46.821683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ec402-8cba-4dfa-877d-114f5befc2f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:42:46.823449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ed4ec402-8cba-4dfa-877d-114f5befc2f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:42:46.839651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ec402-8cba-4dfa-877d-114f5befc2f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:42:46.841345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ec402-8cba-4dfa-877d-114f5befc2f9 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:42:46.855402 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4466/8846] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:42:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:42:53.213670 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d906373-effd-4df5-9cd4-b4849172a68f service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:42:53.215235 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0d906373-effd-4df5-9cd4-b4849172a68f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:42:53.232259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d906373-effd-4df5-9cd4-b4849172a68f service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:42:53.238177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3243d-dc16-40d4-a49d-f2986abd6d69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:42:53.239064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3243d-dc16-40d4-a49d-f2986abd6d69 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:42:53.239419 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3243d-dc16-40d4-a49d-f2986abd6d69 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:42:53.239769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3243d-dc16-40d4-a49d-f2986abd6d69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:42:53.241672 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-68e3243d-dc16-40d4-a49d-f2986abd6d69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:42:53.258273 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3243d-dc16-40d4-a49d-f2986abd6d69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:42:53.259987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3243d-dc16-40d4-a49d-f2986abd6d69 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:42:53.274590 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4381/8847] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:42:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:42:56.848039 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:42:59.307403 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e849c-a522-4640-a203-4b09d59061b8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:42:59.308036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e849c-a522-4640-a203-4b09d59061b8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:42:59.308036 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e849c-a522-4640-a203-4b09d59061b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:42:59.331069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-820e849c-a522-4640-a203-4b09d59061b8 None None] MFA Rules not processed for user `22d9a2cf58104e3988e468a329eb6b80`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:42:59.332453 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-820e849c-a522-4640-a203-4b09d59061b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:42:59.340263 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4467/8848] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:42:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:43:02.464518 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3243d-dc16-40d4-a49d-f2986abd6d69 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:43:02.476881 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3243d-dc16-40d4-a49d-f2986abd6d69 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:43:02.480818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a18f9-85b1-460b-931e-460cc14bc894 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:43:02.481708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a18f9-85b1-460b-931e-460cc14bc894 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:43:02.482159 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a18f9-85b1-460b-931e-460cc14bc894 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:43:02.482632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a18f9-85b1-460b-931e-460cc14bc894 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:43:02.484697 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7c0a18f9-85b1-460b-931e-460cc14bc894 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:43:02.502882 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a18f9-85b1-460b-931e-460cc14bc894 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:43:02.504658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a18f9-85b1-460b-931e-460cc14bc894 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:43:02.517341 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4382/8849] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:43:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:43:07.879856 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e849c-a522-4640-a203-4b09d59061b8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:43:07.897008 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e849c-a522-4640-a203-4b09d59061b8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:43:07.902501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84644206-7de5-4603-bfb1-e7d20569c9e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:43:07.903109 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84644206-7de5-4603-bfb1-e7d20569c9e6 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:43:07.903396 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84644206-7de5-4603-bfb1-e7d20569c9e6 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:43:07.903673 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84644206-7de5-4603-bfb1-e7d20569c9e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:43:07.905428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-84644206-7de5-4603-bfb1-e7d20569c9e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:43:07.920496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84644206-7de5-4603-bfb1-e7d20569c9e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:43:07.922342 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84644206-7de5-4603-bfb1-e7d20569c9e6 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:43:07.936810 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4468/8850] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:43:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:43:12.510092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=12 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:43:17.928846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=9 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:43:25.368776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a18f9-85b1-460b-931e-460cc14bc894 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:43:25.384289 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a18f9-85b1-460b-931e-460cc14bc894 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:43:25.389887 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ed10b-068f-4b47-bbc1-ffde29d5b9eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:43:25.390444 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ed10b-068f-4b47-bbc1-ffde29d5b9eb service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:43:25.390727 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ed10b-068f-4b47-bbc1-ffde29d5b9eb service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:43:25.391018 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ed10b-068f-4b47-bbc1-ffde29d5b9eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:43:25.393011 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ac2ed10b-068f-4b47-bbc1-ffde29d5b9eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:43:25.407661 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ed10b-068f-4b47-bbc1-ffde29d5b9eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:43:25.409275 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ed10b-068f-4b47-bbc1-ffde29d5b9eb service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:43:25.422116 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4383/8851] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:43:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:43:35.414899 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:44:16.783297 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84644206-7de5-4603-bfb1-e7d20569c9e6 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:44:16.792931 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84644206-7de5-4603-bfb1-e7d20569c9e6 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:44:16.796560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abadc6f-697b-4dfe-85b9-02b9da3cb935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:44:16.797560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abadc6f-697b-4dfe-85b9-02b9da3cb935 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:44:16.797852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abadc6f-697b-4dfe-85b9-02b9da3cb935 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:44:16.798137 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abadc6f-697b-4dfe-85b9-02b9da3cb935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:44:16.799864 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6abadc6f-697b-4dfe-85b9-02b9da3cb935 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:44:16.812625 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abadc6f-697b-4dfe-85b9-02b9da3cb935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:44:16.814108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abadc6f-697b-4dfe-85b9-02b9da3cb935 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:44:16.825386 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4469/8852] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:44:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:44:16.936437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad35b7-9b92-44e1-8da3-434810e22c97 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:44:16.936948 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad35b7-9b92-44e1-8da3-434810e22c97 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:44:16.937231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad35b7-9b92-44e1-8da3-434810e22c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:44:16.958070 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e8ad35b7-9b92-44e1-8da3-434810e22c97 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:44:16.959295 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e8ad35b7-9b92-44e1-8da3-434810e22c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:44:16.968064 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4384/8853] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 21:44:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:44:16.979434 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abadc6f-697b-4dfe-85b9-02b9da3cb935 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:44:16.995206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abadc6f-697b-4dfe-85b9-02b9da3cb935 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:44:17.000765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291b7c7-c91f-4f58-8828-790c03e7f6ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:44:17.001400 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291b7c7-c91f-4f58-8828-790c03e7f6ab service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:44:17.001689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291b7c7-c91f-4f58-8828-790c03e7f6ab service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:44:17.001967 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291b7c7-c91f-4f58-8828-790c03e7f6ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:44:17.010248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291b7c7-c91f-4f58-8828-790c03e7f6ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:44:17.011852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291b7c7-c91f-4f58-8828-790c03e7f6ab service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:44:17.021788 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4470/8854] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:44:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:44:17.197693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad35b7-9b92-44e1-8da3-434810e22c97 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:44:17.207300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad35b7-9b92-44e1-8da3-434810e22c97 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:44:17.210986 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3658d-6c7e-409a-a226-9d236ccb1c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:44:17.211590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3658d-6c7e-409a-a226-9d236ccb1c14 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:44:17.211876 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3658d-6c7e-409a-a226-9d236ccb1c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:44:17.212153 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3658d-6c7e-409a-a226-9d236ccb1c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:44:17.213914 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-92a3658d-6c7e-409a-a226-9d236ccb1c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:44:17.230478 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3658d-6c7e-409a-a226-9d236ccb1c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:44:17.231913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3658d-6c7e-409a-a226-9d236ccb1c14 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:44:17.246444 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4385/8855] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:44:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:44:17.307650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291b7c7-c91f-4f58-8828-790c03e7f6ab service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:44:17.321228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291b7c7-c91f-4f58-8828-790c03e7f6ab service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:44:17.326170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6fb39-eedd-42b6-adf1-d9acffa996fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:44:17.326769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6fb39-eedd-42b6-adf1-d9acffa996fd service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:44:17.327056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6fb39-eedd-42b6-adf1-d9acffa996fd service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:44:17.327335 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6fb39-eedd-42b6-adf1-d9acffa996fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:44:17.329689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f9b6fb39-eedd-42b6-adf1-d9acffa996fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:44:17.341086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6fb39-eedd-42b6-adf1-d9acffa996fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:44:17.342661 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6fb39-eedd-42b6-adf1-d9acffa996fd service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:44:17.352284 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4471/8856] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:44:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:44:26.210675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3658d-6c7e-409a-a226-9d236ccb1c14 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:44:26.225814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3658d-6c7e-409a-a226-9d236ccb1c14 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:44:26.231606 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0cec4-7fc6-41da-adf1-28f8d041bf1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:44:26.232259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0cec4-7fc6-41da-adf1-28f8d041bf1f service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:44:26.232664 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0cec4-7fc6-41da-adf1-28f8d041bf1f service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:44:26.233035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0cec4-7fc6-41da-adf1-28f8d041bf1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:44:26.242375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0cec4-7fc6-41da-adf1-28f8d041bf1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:44:26.244360 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0cec4-7fc6-41da-adf1-28f8d041bf1f service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:44:26.254358 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4386/8857] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:44:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:44:27.348382 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=18 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:44:36.249714 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=15 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:45:08.047158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6870f4a-adaf-4c29-a9ae-3f9e712e2553 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:45:08.047652 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6870f4a-adaf-4c29-a9ae-3f9e712e2553 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:45:08.047753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6870f4a-adaf-4c29-a9ae-3f9e712e2553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:45:08.067574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-a6870f4a-adaf-4c29-a9ae-3f9e712e2553 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:45:08.068782 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a6870f4a-adaf-4c29-a9ae-3f9e712e2553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:45:08.077361 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4472/8858] 10.222.0.42 () {58 vars in 1015 bytes} [Sat Sep 3 21:45:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:45:08.089908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fd089-65aa-48d5-914e-e59d7983fe14 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:45:08.090328 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fd089-65aa-48d5-914e-e59d7983fe14 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:45:08.090838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fd089-65aa-48d5-914e-e59d7983fe14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:45:08.109579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-e27fd089-65aa-48d5-914e-e59d7983fe14 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:45:08.110795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-e27fd089-65aa-48d5-914e-e59d7983fe14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:45:08.120621 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4387/8859] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Sep 3 21:45:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:45:08.129534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6870f4a-adaf-4c29-a9ae-3f9e712e2553 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:45:08.140974 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6870f4a-adaf-4c29-a9ae-3f9e712e2553 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:45:08.146831 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fad68c-fc99-40da-9c7a-10f131076f83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:45:08.147469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fad68c-fc99-40da-9c7a-10f131076f83 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:45:08.147754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fad68c-fc99-40da-9c7a-10f131076f83 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:45:08.148030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fad68c-fc99-40da-9c7a-10f131076f83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:45:08.156406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fad68c-fc99-40da-9c7a-10f131076f83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:45:08.157940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fad68c-fc99-40da-9c7a-10f131076f83 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:45:08.167530 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4473/8860] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:45:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:45:11.678629 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228591f3-2200-42e7-bc4b-f4f5839b8768 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:45:11.679647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228591f3-2200-42e7-bc4b-f4f5839b8768 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:45:11.680122 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228591f3-2200-42e7-bc4b-f4f5839b8768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:45:11.710164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-228591f3-2200-42e7-bc4b-f4f5839b8768 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:45:11.711853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-228591f3-2200-42e7-bc4b-f4f5839b8768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:45:11.732166 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4388/8861] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Sep 3 21:45:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:45:11.741730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fad68c-fc99-40da-9c7a-10f131076f83 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:45:11.755063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fad68c-fc99-40da-9c7a-10f131076f83 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:45:11.762885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f277a-950a-4057-a277-6a5f75c46b26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:45:11.763515 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f277a-950a-4057-a277-6a5f75c46b26 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:45:11.763810 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f277a-950a-4057-a277-6a5f75c46b26 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:45:11.764096 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f277a-950a-4057-a277-6a5f75c46b26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:45:11.771643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f26f277a-950a-4057-a277-6a5f75c46b26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:45:11.789308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f277a-950a-4057-a277-6a5f75c46b26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:45:11.791150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f277a-950a-4057-a277-6a5f75c46b26 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:45:11.805600 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4474/8862] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:45:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:45:16.880587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228591f3-2200-42e7-bc4b-f4f5839b8768 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:45:16.901228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228591f3-2200-42e7-bc4b-f4f5839b8768 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:45:16.905155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667e395-3006-414a-a09a-3c1327b2eff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:45:16.906009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667e395-3006-414a-a09a-3c1327b2eff8 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:45:16.906296 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667e395-3006-414a-a09a-3c1327b2eff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:45:16.906575 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667e395-3006-414a-a09a-3c1327b2eff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:45:16.916956 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667e395-3006-414a-a09a-3c1327b2eff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:45:16.918430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667e395-3006-414a-a09a-3c1327b2eff8 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:45:16.933278 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4389/8863] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:45:16 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:45:19.489588 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97b5ac-9921-4389-a4b8-2989fedbdca5 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:45:19.490120 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97b5ac-9921-4389-a4b8-2989fedbdca5 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:45:19.490310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97b5ac-9921-4389-a4b8-2989fedbdca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:45:19.512043 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-9f97b5ac-9921-4389-a4b8-2989fedbdca5 None None] MFA Rules not processed for user `22d9a2cf58104e3988e468a329eb6b80`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:45:19.513538 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9f97b5ac-9921-4389-a4b8-2989fedbdca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:45:19.527264 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4475/8864] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:45:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:45:19.887430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667e395-3006-414a-a09a-3c1327b2eff8 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:45:19.898873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667e395-3006-414a-a09a-3c1327b2eff8 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:45:19.903230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae8857-fb51-42e4-99ae-6cfe428e0127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:45:19.904241 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae8857-fb51-42e4-99ae-6cfe428e0127 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:45:19.904688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae8857-fb51-42e4-99ae-6cfe428e0127 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:45:19.905167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae8857-fb51-42e4-99ae-6cfe428e0127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:45:19.907007 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-69ae8857-fb51-42e4-99ae-6cfe428e0127 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:45:19.922354 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae8857-fb51-42e4-99ae-6cfe428e0127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:45:19.923708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae8857-fb51-42e4-99ae-6cfe428e0127 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:45:19.937513 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4390/8865] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:45:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:45:21.345255 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97b5ac-9921-4389-a4b8-2989fedbdca5 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:45:21.357428 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97b5ac-9921-4389-a4b8-2989fedbdca5 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:45:21.362617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cc447-5dce-4482-a891-40d80b9179af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:45:21.363226 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cc447-5dce-4482-a891-40d80b9179af service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:45:21.363510 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cc447-5dce-4482-a891-40d80b9179af service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:45:21.363784 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cc447-5dce-4482-a891-40d80b9179af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:45:21.365619 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-542cc447-5dce-4482-a891-40d80b9179af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:45:21.377260 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cc447-5dce-4482-a891-40d80b9179af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:45:21.378799 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cc447-5dce-4482-a891-40d80b9179af service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:45:21.389277 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4476/8866] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:45:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:45:24.955993 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fda26-4a63-4abd-9865-8a2787f89226 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:45:24.956546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fda26-4a63-4abd-9865-8a2787f89226 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:45:24.956671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fda26-4a63-4abd-9865-8a2787f89226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:45:24.976438 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1d4fda26-4a63-4abd-9865-8a2787f89226 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:45:24.977688 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1d4fda26-4a63-4abd-9865-8a2787f89226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:45:24.986401 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4391/8867] 10.222.0.30 () {58 vars in 1015 bytes} [Sat Sep 3 21:45:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:45:24.998257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cc447-5dce-4482-a891-40d80b9179af service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:45:25.008622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cc447-5dce-4482-a891-40d80b9179af service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:45:25.013640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03456a3-5ded-47de-9e1b-f759fdf93016 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:45:25.014235 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03456a3-5ded-47de-9e1b-f759fdf93016 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:45:25.014522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03456a3-5ded-47de-9e1b-f759fdf93016 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:45:25.014802 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03456a3-5ded-47de-9e1b-f759fdf93016 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:45:25.023643 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03456a3-5ded-47de-9e1b-f759fdf93016 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:45:25.025289 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03456a3-5ded-47de-9e1b-f759fdf93016 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:45:25.035744 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4477/8868] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:45:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:45:34.971680 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=25 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:45:35.031080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=30 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:46:41.597233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fda26-4a63-4abd-9865-8a2787f89226 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:46:41.606602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fda26-4a63-4abd-9865-8a2787f89226 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:46:41.610261 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edc764-291c-4bde-a28d-8881f3d2afaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:46:41.610868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edc764-291c-4bde-a28d-8881f3d2afaa service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:46:41.611154 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edc764-291c-4bde-a28d-8881f3d2afaa service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:46:41.611432 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edc764-291c-4bde-a28d-8881f3d2afaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:46:41.625004 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edc764-291c-4bde-a28d-8881f3d2afaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:46:41.626471 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edc764-291c-4bde-a28d-8881f3d2afaa service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:46:41.650068 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4392/8869] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:46:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:46:42.841496 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03456a3-5ded-47de-9e1b-f759fdf93016 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:46:42.851350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03456a3-5ded-47de-9e1b-f759fdf93016 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:46:42.854897 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d83805-4bf2-4a9e-ab89-e8246692d0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:46:42.855492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d83805-4bf2-4a9e-ab89-e8246692d0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:46:42.855776 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d83805-4bf2-4a9e-ab89-e8246692d0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:46:42.856053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d83805-4bf2-4a9e-ab89-e8246692d0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:46:42.869739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d83805-4bf2-4a9e-ab89-e8246692d0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:46:42.871184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d83805-4bf2-4a9e-ab89-e8246692d0e6 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:46:42.884648 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4478/8870] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:46:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:46:44.062616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f9e58-255b-481c-84d3-1aca96ccdae5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:46:44.063259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f9e58-255b-481c-84d3-1aca96ccdae5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:46:44.063259 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f9e58-255b-481c-84d3-1aca96ccdae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:46:44.111569 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f55f9e58-255b-481c-84d3-1aca96ccdae5 None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:46:44.113139 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f55f9e58-255b-481c-84d3-1aca96ccdae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:46:44.122536 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4393/8871] 10.222.0.16 () {58 vars in 1059 bytes} [Sat Sep 3 21:46:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:46:46.475873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7f8d8-f265-4c18-ab74-d865bf7f4430 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:46:46.476393 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7f8d8-f265-4c18-ab74-d865bf7f4430 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:46:46.476501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7f8d8-f265-4c18-ab74-d865bf7f4430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:46:46.496258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-4ad7f8d8-f265-4c18-ab74-d865bf7f4430 None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:46:46.497491 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4ad7f8d8-f265-4c18-ab74-d865bf7f4430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:46:46.507332 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4479/8872] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:46:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:46:46.517750 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f9e58-255b-481c-84d3-1aca96ccdae5 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:46:46.526970 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f9e58-255b-481c-84d3-1aca96ccdae5 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:46:46.530706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169dff9a-97ac-4554-bed9-4efa0e6845ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:46:46.531308 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dff9a-97ac-4554-bed9-4efa0e6845ba service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:46:46.531587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dff9a-97ac-4554-bed9-4efa0e6845ba service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:46:46.531873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dff9a-97ac-4554-bed9-4efa0e6845ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:46:46.541907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169dff9a-97ac-4554-bed9-4efa0e6845ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:46:46.543334 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169dff9a-97ac-4554-bed9-4efa0e6845ba service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:46:46.554634 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4394/8873] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:46:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:46:47.735107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f77e6-abb7-4fa1-bbc5-3f22d936c5f8 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:46:47.735623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f77e6-abb7-4fa1-bbc5-3f22d936c5f8 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:46:47.735734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f77e6-abb7-4fa1-bbc5-3f22d936c5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:46:47.755644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-3b4f77e6-abb7-4fa1-bbc5-3f22d936c5f8 None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:46:47.756858 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-3b4f77e6-abb7-4fa1-bbc5-3f22d936c5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:46:47.766129 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4480/8874] 10.222.0.16 () {58 vars in 1059 bytes} [Sat Sep 3 21:46:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:46:47.772390 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824b8a6-96b9-493c-9f29-c85f1770d9e5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:46:47.772878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824b8a6-96b9-493c-9f29-c85f1770d9e5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:46:47.773006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824b8a6-96b9-493c-9f29-c85f1770d9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:46:47.791871 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-f824b8a6-96b9-493c-9f29-c85f1770d9e5 None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:46:47.793172 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f824b8a6-96b9-493c-9f29-c85f1770d9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:46:47.802615 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4395/8875] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:46:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:46:47.812996 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f77e6-abb7-4fa1-bbc5-3f22d936c5f8 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:46:47.822790 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f77e6-abb7-4fa1-bbc5-3f22d936c5f8 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:46:47.826426 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422a418-9d71-436a-8896-181f0e659d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:46:47.827021 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422a418-9d71-436a-8896-181f0e659d2d service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:46:47.827313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422a418-9d71-436a-8896-181f0e659d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:46:47.827594 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422a418-9d71-436a-8896-181f0e659d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:46:47.838147 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422a418-9d71-436a-8896-181f0e659d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:46:47.839668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422a418-9d71-436a-8896-181f0e659d2d service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:46:47.850980 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4481/8876] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:46:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:46:57.787112 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=32 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:46:57.845985 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=18 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:46:58.725660 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee674c95-1d7a-47f3-bbb1-411bba1699cc None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:46:58.726062 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee674c95-1d7a-47f3-bbb1-411bba1699cc None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:46:58.726346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee674c95-1d7a-47f3-bbb1-411bba1699cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:46:58.772466 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-ee674c95-1d7a-47f3-bbb1-411bba1699cc None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:46:58.773741 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-ee674c95-1d7a-47f3-bbb1-411bba1699cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:46:58.780428 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4396/8877] 10.222.0.30 () {58 vars in 1015 bytes} [Sat Sep 3 21:46:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:46:58.792816 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422a418-9d71-436a-8896-181f0e659d2d service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:46:58.801445 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422a418-9d71-436a-8896-181f0e659d2d service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:46:58.805730 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9c500-0b3c-4425-8734-02068b954e15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:46:58.806332 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9c500-0b3c-4425-8734-02068b954e15 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:46:58.806634 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9c500-0b3c-4425-8734-02068b954e15 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:46:58.806921 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9c500-0b3c-4425-8734-02068b954e15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:46:58.813941 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9c500-0b3c-4425-8734-02068b954e15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:46:58.815370 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9c500-0b3c-4425-8734-02068b954e15 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:46:58.823612 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4482/8878] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:46:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:47:08.769068 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=9 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:47:08.820224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:47:48.667426 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b368b15-8f43-40b4-a8b5-ef254a3d88af None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:47:48.668167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b368b15-8f43-40b4-a8b5-ef254a3d88af None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:47:48.668167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b368b15-8f43-40b4-a8b5-ef254a3d88af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:47:48.713305 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-7b368b15-8f43-40b4-a8b5-ef254a3d88af None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:47:48.714546 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7b368b15-8f43-40b4-a8b5-ef254a3d88af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:47:48.777411 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4397/8879] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 21:47:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:47:48.794999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9c500-0b3c-4425-8734-02068b954e15 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:47:48.825385 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9c500-0b3c-4425-8734-02068b954e15 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:47:48.833631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d84b25-081d-4ccd-ba5c-762ca45d34fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:47:48.834903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d84b25-081d-4ccd-ba5c-762ca45d34fa service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:47:48.835293 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d84b25-081d-4ccd-ba5c-762ca45d34fa service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:47:48.835694 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d84b25-081d-4ccd-ba5c-762ca45d34fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:47:48.845886 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d84b25-081d-4ccd-ba5c-762ca45d34fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:47:48.847966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d84b25-081d-4ccd-ba5c-762ca45d34fa service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:47:48.860877 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4483/8880] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:47:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:47:55.355013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f042a-f59b-47db-8c0c-895668571877 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:47:55.355490 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f042a-f59b-47db-8c0c-895668571877 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:47:55.355736 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f042a-f59b-47db-8c0c-895668571877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:47:55.407425 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-8b2f042a-f59b-47db-8c0c-895668571877 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:47:55.408779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-8b2f042a-f59b-47db-8c0c-895668571877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:47:55.431076 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4398/8881] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 21:47:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:47:55.444824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d84b25-081d-4ccd-ba5c-762ca45d34fa service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:47:55.457238 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d84b25-081d-4ccd-ba5c-762ca45d34fa service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:47:55.463533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ca423-b594-4a89-89e9-c2b9e2ce7b9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:47:55.464245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ca423-b594-4a89-89e9-c2b9e2ce7b9a service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:47:55.464603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ca423-b594-4a89-89e9-c2b9e2ce7b9a service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:47:55.465244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ca423-b594-4a89-89e9-c2b9e2ce7b9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:47:55.475308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ca423-b594-4a89-89e9-c2b9e2ce7b9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:47:55.477047 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ca423-b594-4a89-89e9-c2b9e2ce7b9a service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:47:55.487157 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4484/8882] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:47:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:48:01.448491 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dee1f-bda4-437d-93f9-cd6719cfb51e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:48:01.448977 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dee1f-bda4-437d-93f9-cd6719cfb51e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:48:01.449073 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dee1f-bda4-437d-93f9-cd6719cfb51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:48:01.468430 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-555dee1f-bda4-437d-93f9-cd6719cfb51e None None] MFA Rules not processed for user `22d9a2cf58104e3988e468a329eb6b80`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:48:01.469700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-555dee1f-bda4-437d-93f9-cd6719cfb51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:48:01.477446 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4399/8883] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:48:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:48:01.829804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba25a67-64ac-41d2-9a47-d08a56150a16 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:48:01.830682 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba25a67-64ac-41d2-9a47-d08a56150a16 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:48:01.830966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba25a67-64ac-41d2-9a47-d08a56150a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:48:01.868333 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7ba25a67-64ac-41d2-9a47-d08a56150a16 None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:48:01.869702 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7ba25a67-64ac-41d2-9a47-d08a56150a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:48:01.878294 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4485/8884] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:48:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:48:01.890149 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dee1f-bda4-437d-93f9-cd6719cfb51e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:48:01.910218 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dee1f-bda4-437d-93f9-cd6719cfb51e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:48:01.920318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b02019-6c3f-4d26-9e9b-6ee42f34dbd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:48:01.921403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b02019-6c3f-4d26-9e9b-6ee42f34dbd7 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:48:01.922222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b02019-6c3f-4d26-9e9b-6ee42f34dbd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:48:01.922566 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b02019-6c3f-4d26-9e9b-6ee42f34dbd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:48:01.932094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b02019-6c3f-4d26-9e9b-6ee42f34dbd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:48:01.933586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b02019-6c3f-4d26-9e9b-6ee42f34dbd7 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:48:01.944332 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4400/8885] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:48:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:48:05.174493 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc381cc-a9d0-4052-a0c2-03e989e0e3b6 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:48:05.175025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc381cc-a9d0-4052-a0c2-03e989e0e3b6 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:48:05.175123 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc381cc-a9d0-4052-a0c2-03e989e0e3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:48:05.196324 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-1dc381cc-a9d0-4052-a0c2-03e989e0e3b6 None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:48:05.197665 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1dc381cc-a9d0-4052-a0c2-03e989e0e3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:48:05.204345 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4486/8886] 10.222.0.42 () {58 vars in 1015 bytes} [Sat Sep 3 21:48:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:48:05.216645 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b02019-6c3f-4d26-9e9b-6ee42f34dbd7 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:48:05.226306 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b02019-6c3f-4d26-9e9b-6ee42f34dbd7 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:48:05.230089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28179f-0c78-45f8-aaff-6cec275f9d3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:48:05.230864 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28179f-0c78-45f8-aaff-6cec275f9d3c service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:48:05.231151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28179f-0c78-45f8-aaff-6cec275f9d3c service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:48:05.231424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28179f-0c78-45f8-aaff-6cec275f9d3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:48:05.240331 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28179f-0c78-45f8-aaff-6cec275f9d3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:48:05.241806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28179f-0c78-45f8-aaff-6cec275f9d3c service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:48:05.250380 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4401/8887] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:48:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:48:05.904056 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42074706-3e55-4e00-bcc0-e3f6dabbfa1b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:48:05.904701 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42074706-3e55-4e00-bcc0-e3f6dabbfa1b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:48:05.905228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42074706-3e55-4e00-bcc0-e3f6dabbfa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:48:05.925695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-42074706-3e55-4e00-bcc0-e3f6dabbfa1b None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:48:05.926977 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-42074706-3e55-4e00-bcc0-e3f6dabbfa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:48:05.936577 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4487/8888] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 21:48:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:48:05.959909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28179f-0c78-45f8-aaff-6cec275f9d3c service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:48:05.973997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28179f-0c78-45f8-aaff-6cec275f9d3c service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:48:05.979643 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e9a49-1119-4a11-84a6-871a413093a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:48:05.980251 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e9a49-1119-4a11-84a6-871a413093a7 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:48:05.980537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e9a49-1119-4a11-84a6-871a413093a7 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:48:05.980810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e9a49-1119-4a11-84a6-871a413093a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:48:05.991726 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e9a49-1119-4a11-84a6-871a413093a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:48:05.993410 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e9a49-1119-4a11-84a6-871a413093a7 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:48:06.005532 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4402/8889] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:48:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:48:10.376525 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d095c0e-dc3a-4c49-83f5-a32a49ce1c7b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:48:10.377042 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d095c0e-dc3a-4c49-83f5-a32a49ce1c7b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:48:10.377164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d095c0e-dc3a-4c49-83f5-a32a49ce1c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:48:10.385468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=32 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:48:10.400837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-2d095c0e-dc3a-4c49-83f5-a32a49ce1c7b None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:48:10.402136 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-2d095c0e-dc3a-4c49-83f5-a32a49ce1c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:48:10.411701 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4488/8890] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 21:48:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:48:10.438174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e9a49-1119-4a11-84a6-871a413093a7 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:48:10.445300 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=69 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:48:10.451543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e9a49-1119-4a11-84a6-871a413093a7 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:48:10.458147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12a70b-ac7c-4fb2-bf8e-bc020126b8ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:48:10.458810 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12a70b-ac7c-4fb2-bf8e-bc020126b8ad service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:48:10.459099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12a70b-ac7c-4fb2-bf8e-bc020126b8ad service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:48:10.459376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12a70b-ac7c-4fb2-bf8e-bc020126b8ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:48:10.468807 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12a70b-ac7c-4fb2-bf8e-bc020126b8ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:48:10.470437 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12a70b-ac7c-4fb2-bf8e-bc020126b8ad service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:48:10.481757 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4403/8891] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:48:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:48:20.393433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:48:20.476999 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:48:59.637011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d095c0e-dc3a-4c49-83f5-a32a49ce1c7b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:48:59.720860 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d095c0e-dc3a-4c49-83f5-a32a49ce1c7b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:48:59.725676 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:48:59.726315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:48:59.726686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:48:59.726866 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] PATH_INFO: `/v3/users/22d9a2cf58104e3988e468a329eb6b80` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:48:59.728011 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] RBAC: Authorizing `identity:delete_user(user_id=22d9a2cf58104e3988e468a329eb6b80)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:48:59.730584 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:48:59.780733 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22d9a2cf58104e3988e468a329eb6b80'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:48:59.793508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22d9a2cf58104e3988e468a329eb6b80'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:48:59.801258 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22d9a2cf58104e3988e468a329eb6b80'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:48:59.801908 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22d9a2cf58104e3988e468a329eb6b80'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:48:59.802716 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22d9a2cf58104e3988e468a329eb6b80'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:48:59.816132 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4489/8892] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:48:59 2022] DELETE /identity/v3/users/22d9a2cf58104e3988e468a329eb6b80 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:48:59.820873 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b3ae-957b-48cb-89ba-a45c8d200974 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:48:59.821382 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b3ae-957b-48cb-89ba-a45c8d200974 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:48:59.821671 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b3ae-957b-48cb-89ba-a45c8d200974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:48:59.867307 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-4ed6b3ae-957b-48cb-89ba-a45c8d200974 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:48:59.868492 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4ed6b3ae-957b-48cb-89ba-a45c8d200974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:48:59.930300 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4404/8893] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:48:59.940122 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:48:59.941558 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:48:59.998562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34df18-3290-4368-abd0-13e137a6523a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:00.002207 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b554f-37d2-4899-9bcf-4a4626140e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:00.002812 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b554f-37d2-4899-9bcf-4a4626140e0d service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:00.003100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b554f-37d2-4899-9bcf-4a4626140e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:00.003380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b554f-37d2-4899-9bcf-4a4626140e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:00.012765 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b554f-37d2-4899-9bcf-4a4626140e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:00.014248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b554f-37d2-4899-9bcf-4a4626140e0d service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:00.023614 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4490/8894] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:48:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:49:00.207658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6b3ae-957b-48cb-89ba-a45c8d200974 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:00.209373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-4ed6b3ae-957b-48cb-89ba-a45c8d200974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:49:00.238286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6b3ae-957b-48cb-89ba-a45c8d200974 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:00.243597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:00.244730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:00.245356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:00.245699 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] PATH_INFO: `/v3/projects/9f2c6e6317c6424f83e70d52aee5b85d` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:00.246981 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f2c6e6317c6424f83e70d52aee5b85d)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:00.249003 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:00.307367 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f2c6e6317c6424f83e70d52aee5b85d', 'request_id': 'req-bbff9850-59a0-4e45-bb7d-7576e854572f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.315585 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f2c6e6317c6424f83e70d52aee5b85d', 'request_id': 'req-bbff9850-59a0-4e45-bb7d-7576e854572f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.322724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f2c6e6317c6424f83e70d52aee5b85d', 'request_id': 'req-bbff9850-59a0-4e45-bb7d-7576e854572f'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.332988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] The token cache is being invalidate because project 9f2c6e6317c6424f83e70d52aee5b85d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:49:00.333281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.333923 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.335412 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4405/8895] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:49:00 2022] DELETE /identity/v3/projects/9f2c6e6317c6424f83e70d52aee5b85d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:49:00.339610 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b554f-37d2-4899-9bcf-4a4626140e0d service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:00.345801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:49:00.349560 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b554f-37d2-4899-9bcf-4a4626140e0d service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:00.407575 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785d4c7-c12a-4828-befe-ded50df03662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:00.408127 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785d4c7-c12a-4828-befe-ded50df03662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:00.408415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785d4c7-c12a-4828-befe-ded50df03662 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:00.408696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785d4c7-c12a-4828-befe-ded50df03662 admin admin] PATH_INFO: `/v3/users/eda37c7fce364fec94d2e610316b3489` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:00.409853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785d4c7-c12a-4828-befe-ded50df03662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda37c7fce364fec94d2e610316b3489)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:00.411827 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785d4c7-c12a-4828-befe-ded50df03662 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:00.459227 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d785d4c7-c12a-4828-befe-ded50df03662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda37c7fce364fec94d2e610316b3489'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.471229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d785d4c7-c12a-4828-befe-ded50df03662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda37c7fce364fec94d2e610316b3489'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.478449 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d785d4c7-c12a-4828-befe-ded50df03662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda37c7fce364fec94d2e610316b3489'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.479077 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d785d4c7-c12a-4828-befe-ded50df03662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda37c7fce364fec94d2e610316b3489'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.479688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-d785d4c7-c12a-4828-befe-ded50df03662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda37c7fce364fec94d2e610316b3489'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.491954 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4491/8896] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:49:00 2022] DELETE /identity/v3/users/eda37c7fce364fec94d2e610316b3489 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:49:00.754013 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:00.755549 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:49:00.769562 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff9850-59a0-4e45-bb7d-7576e854572f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:00.826783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c420a-1b74-496f-a658-9e76b89847f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:00.827415 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c420a-1b74-496f-a658-9e76b89847f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:00.827706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c420a-1b74-496f-a658-9e76b89847f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:00.827971 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c420a-1b74-496f-a658-9e76b89847f5 admin admin] PATH_INFO: `/v3/projects/e172f417ddc84f1386f93b20b9bd01d7` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:00.829118 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c420a-1b74-496f-a658-9e76b89847f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e172f417ddc84f1386f93b20b9bd01d7)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:00.831019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c420a-1b74-496f-a658-9e76b89847f5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:00.838769 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:49:00.878475 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d4c420a-1b74-496f-a658-9e76b89847f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e172f417ddc84f1386f93b20b9bd01d7', 'request_id': 'req-0d4c420a-1b74-496f-a658-9e76b89847f5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.885749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d4c420a-1b74-496f-a658-9e76b89847f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e172f417ddc84f1386f93b20b9bd01d7', 'request_id': 'req-0d4c420a-1b74-496f-a658-9e76b89847f5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.891909 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d4c420a-1b74-496f-a658-9e76b89847f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e172f417ddc84f1386f93b20b9bd01d7', 'request_id': 'req-0d4c420a-1b74-496f-a658-9e76b89847f5'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.904857 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d4c420a-1b74-496f-a658-9e76b89847f5 admin admin] The token cache is being invalidate because project e172f417ddc84f1386f93b20b9bd01d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:49:00.905173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d4c420a-1b74-496f-a658-9e76b89847f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.905819 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-0d4c420a-1b74-496f-a658-9e76b89847f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:49:00.907311 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4406/8897] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:49:00 2022] DELETE /identity/v3/projects/e172f417ddc84f1386f93b20b9bd01d7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:49:00.912568 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41671999-21ce-4655-ab4a-4b645a32596b None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:00.913065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41671999-21ce-4655-ab4a-4b645a32596b None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:00.913184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41671999-21ce-4655-ab4a-4b645a32596b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:00.952086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-41671999-21ce-4655-ab4a-4b645a32596b None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:49:00.953328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-41671999-21ce-4655-ab4a-4b645a32596b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:49:01.011182 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4492/8898] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:49:01.020424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3671b-9459-4b67-a54f-e1d0e9b057ee None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:01.020721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3671b-9459-4b67-a54f-e1d0e9b057ee None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:01.021051 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3671b-9459-4b67-a54f-e1d0e9b057ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:01.042767 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fcb3671b-9459-4b67-a54f-e1d0e9b057ee None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:49:01.043989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fcb3671b-9459-4b67-a54f-e1d0e9b057ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:49:01.052122 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4407/8899] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:49:01.056267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41671999-21ce-4655-ab4a-4b645a32596b None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:01.065869 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41671999-21ce-4655-ab4a-4b645a32596b None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:01.070623 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626d64a-4d67-485f-b017-52256b9e4914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:01.071228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626d64a-4d67-485f-b017-52256b9e4914 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:01.071517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626d64a-4d67-485f-b017-52256b9e4914 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:01.071795 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626d64a-4d67-485f-b017-52256b9e4914 admin admin] PATH_INFO: `/v3/domains` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:01.072446 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626d64a-4d67-485f-b017-52256b9e4914 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:49:01.072758 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626d64a-4d67-485f-b017-52256b9e4914 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:01.074405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626d64a-4d67-485f-b017-52256b9e4914 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:01.076754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:49:01.088055 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4493/8900] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:49:01.092165 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3671b-9459-4b67-a54f-e1d0e9b057ee None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:01.101586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3671b-9459-4b67-a54f-e1d0e9b057ee None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:01.106886 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0555c-7235-45b2-a4bf-6c619283e383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:01.107487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0555c-7235-45b2-a4bf-6c619283e383 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:01.107777 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0555c-7235-45b2-a4bf-6c619283e383 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:01.108059 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0555c-7235-45b2-a4bf-6c619283e383 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:01.108703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0555c-7235-45b2-a4bf-6c619283e383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:01.110683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0555c-7235-45b2-a4bf-6c619283e383 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:01.148958 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4408/8901] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:49:01 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:49:01.153726 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626d64a-4d67-485f-b017-52256b9e4914 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:01.163971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626d64a-4d67-485f-b017-52256b9e4914 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:01.219829 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f01c8-5450-44e5-8d2f-6f36ce230aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:01.220433 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f01c8-5450-44e5-8d2f-6f36ce230aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:01.220719 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f01c8-5450-44e5-8d2f-6f36ce230aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:01.221053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f01c8-5450-44e5-8d2f-6f36ce230aa6 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:01.221689 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f01c8-5450-44e5-8d2f-6f36ce230aa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:01.223597 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f01c8-5450-44e5-8d2f-6f36ce230aa6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:01.245097 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4494/8902] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:49:01 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:49:01.249174 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0555c-7235-45b2-a4bf-6c619283e383 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:01.259536 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0555c-7235-45b2-a4bf-6c619283e383 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:01.264429 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff5b82-d713-4dac-a5fc-faff0f009223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:01.265099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff5b82-d713-4dac-a5fc-faff0f009223 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:01.265384 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff5b82-d713-4dac-a5fc-faff0f009223 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:01.265725 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff5b82-d713-4dac-a5fc-faff0f009223 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:01.266389 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff5b82-d713-4dac-a5fc-faff0f009223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:01.268001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff5b82-d713-4dac-a5fc-faff0f009223 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:01.278424 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4409/8903] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:49:01.282318 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f01c8-5450-44e5-8d2f-6f36ce230aa6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:01.292824 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f01c8-5450-44e5-8d2f-6f36ce230aa6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:01.297406 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ef645-7dd1-4c0a-ab13-9298a8c7c9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:01.297998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ef645-7dd1-4c0a-ab13-9298a8c7c9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:01.298284 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ef645-7dd1-4c0a-ab13-9298a8c7c9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:01.298562 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ef645-7dd1-4c0a-ab13-9298a8c7c9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:01.299199 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ef645-7dd1-4c0a-ab13-9298a8c7c9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:01.300801 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ef645-7dd1-4c0a-ab13-9298a8c7c9f3 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:01.312300 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4495/8904] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:49:01.316284 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff5b82-d713-4dac-a5fc-faff0f009223 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:01.326693 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff5b82-d713-4dac-a5fc-faff0f009223 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:01.331316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf23256-6954-4859-907f-2e1388af97fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:01.331958 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf23256-6954-4859-907f-2e1388af97fc admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:01.332238 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf23256-6954-4859-907f-2e1388af97fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:01.332514 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf23256-6954-4859-907f-2e1388af97fc admin admin] PATH_INFO: `/v3/projects/f101b0010f234a4199e698ad0b8decd7/users/03f57e800a6d4ca8a9080aa14d875cf3/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:01.347484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf23256-6954-4859-907f-2e1388af97fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f101b0010f234a4199e698ad0b8decd7, user_id=03f57e800a6d4ca8a9080aa14d875cf3, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:01.350176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf23256-6954-4859-907f-2e1388af97fc admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:01.363299 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4410/8905] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:49:01 2022] PUT /identity/v3/projects/f101b0010f234a4199e698ad0b8decd7/users/03f57e800a6d4ca8a9080aa14d875cf3/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:49:01.368316 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b47250-aa6d-4766-be77-0ae975ade6bd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:01.368635 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b47250-aa6d-4766-be77-0ae975ade6bd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:01.368975 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b47250-aa6d-4766-be77-0ae975ade6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:01.387862 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-77b47250-aa6d-4766-be77-0ae975ade6bd None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:49:01.389092 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-77b47250-aa6d-4766-be77-0ae975ade6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:49:01.445572 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4496/8906] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:49:01.455640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf23256-6954-4859-907f-2e1388af97fc admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:01.457373 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-9cf23256-6954-4859-907f-2e1388af97fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:49:01.513802 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf23256-6954-4859-907f-2e1388af97fc admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:01.517496 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51342d49-5545-4020-a299-e46f796fa76d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:01.518092 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51342d49-5545-4020-a299-e46f796fa76d service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:01.518375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51342d49-5545-4020-a299-e46f796fa76d service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:01.518649 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51342d49-5545-4020-a299-e46f796fa76d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:01.528094 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51342d49-5545-4020-a299-e46f796fa76d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:01.529551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51342d49-5545-4020-a299-e46f796fa76d service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:01.539071 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4411/8907] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:49:05.553394 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159e61b-f72d-42c3-b63e-084a0629de7d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:05.554328 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159e61b-f72d-42c3-b63e-084a0629de7d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:05.554688 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159e61b-f72d-42c3-b63e-084a0629de7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:05.584350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0159e61b-f72d-42c3-b63e-084a0629de7d None None] MFA Rules not processed for user `03f57e800a6d4ca8a9080aa14d875cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:49:05.585631 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0159e61b-f72d-42c3-b63e-084a0629de7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:49:05.657235 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4497/8908] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:49:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:49:05.661888 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51342d49-5545-4020-a299-e46f796fa76d service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:05.672738 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51342d49-5545-4020-a299-e46f796fa76d service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:05.678173 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e37e60-340d-4b82-bc8e-7adaa33118b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:05.678795 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e37e60-340d-4b82-bc8e-7adaa33118b7 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:05.679082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e37e60-340d-4b82-bc8e-7adaa33118b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:05.679369 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e37e60-340d-4b82-bc8e-7adaa33118b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:05.680009 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e37e60-340d-4b82-bc8e-7adaa33118b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:05.682075 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e37e60-340d-4b82-bc8e-7adaa33118b7 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:05.716103 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4412/8909] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:49:05 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:49:05.720221 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159e61b-f72d-42c3-b63e-084a0629de7d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:05.727415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:49:05.733903 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159e61b-f72d-42c3-b63e-084a0629de7d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:05.795884 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003333e-9582-4ed5-8cb5-e32f888d9644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:05.796522 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003333e-9582-4ed5-8cb5-e32f888d9644 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:05.796811 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003333e-9582-4ed5-8cb5-e32f888d9644 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:05.797113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003333e-9582-4ed5-8cb5-e32f888d9644 admin admin] PATH_INFO: `/v3/users` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:05.797865 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003333e-9582-4ed5-8cb5-e32f888d9644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:05.800381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003333e-9582-4ed5-8cb5-e32f888d9644 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:05.831311 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4498/8910] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:49:05 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 21:49:05.832538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e37e60-340d-4b82-bc8e-7adaa33118b7 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:05.840177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:49:05.844332 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e37e60-340d-4b82-bc8e-7adaa33118b7 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:05.850353 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa309fc-5fbf-48f7-b982-a606d1af3c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:05.851047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa309fc-5fbf-48f7-b982-a606d1af3c79 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:05.851521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa309fc-5fbf-48f7-b982-a606d1af3c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:05.851976 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa309fc-5fbf-48f7-b982-a606d1af3c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:05.852827 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa309fc-5fbf-48f7-b982-a606d1af3c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:05.854538 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa309fc-5fbf-48f7-b982-a606d1af3c79 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:05.866413 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4413/8911] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:49:05.870899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003333e-9582-4ed5-8cb5-e32f888d9644 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:05.881587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003333e-9582-4ed5-8cb5-e32f888d9644 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:05.886999 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326611e0-55b6-4f2d-bbd6-840f4d93897b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:05.887757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326611e0-55b6-4f2d-bbd6-840f4d93897b admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:05.888113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326611e0-55b6-4f2d-bbd6-840f4d93897b admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:05.888462 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326611e0-55b6-4f2d-bbd6-840f4d93897b admin admin] PATH_INFO: `/v3/projects/306407da7eef49318e1b08089a74d3d0/users/dcf0397ee6564254827baff0b2aa73be/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:05.905113 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326611e0-55b6-4f2d-bbd6-840f4d93897b admin admin] RBAC: Authorizing `identity:create_grant(project_id=306407da7eef49318e1b08089a74d3d0, user_id=dcf0397ee6564254827baff0b2aa73be, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:05.907936 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326611e0-55b6-4f2d-bbd6-840f4d93897b admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:05.923271 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4499/8912] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:49:05 2022] PUT /identity/v3/projects/306407da7eef49318e1b08089a74d3d0/users/dcf0397ee6564254827baff0b2aa73be/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:49:05.927403 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa309fc-5fbf-48f7-b982-a606d1af3c79 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:05.938446 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa309fc-5fbf-48f7-b982-a606d1af3c79 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:05.998481 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955dc4f-a7ca-4679-ad7d-3dfd1ecdad0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:05.999121 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955dc4f-a7ca-4679-ad7d-3dfd1ecdad0a admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:05.999535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955dc4f-a7ca-4679-ad7d-3dfd1ecdad0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:05.999679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955dc4f-a7ca-4679-ad7d-3dfd1ecdad0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:06.000321 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955dc4f-a7ca-4679-ad7d-3dfd1ecdad0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:06.001946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955dc4f-a7ca-4679-ad7d-3dfd1ecdad0a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:06.012714 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4414/8913] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 21:49:06.017108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326611e0-55b6-4f2d-bbd6-840f4d93897b admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:06.028174 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326611e0-55b6-4f2d-bbd6-840f4d93897b admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:06.033023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bc74f-bab1-433e-877d-cf5ffc185e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:06.033692 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bc74f-bab1-433e-877d-cf5ffc185e9c admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:06.033980 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bc74f-bab1-433e-877d-cf5ffc185e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:06.034262 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bc74f-bab1-433e-877d-cf5ffc185e9c admin admin] PATH_INFO: `/v3/projects/306407da7eef49318e1b08089a74d3d0/users/dcf0397ee6564254827baff0b2aa73be/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:06.036201 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bc74f-bab1-433e-877d-cf5ffc185e9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=306407da7eef49318e1b08089a74d3d0, user_id=dcf0397ee6564254827baff0b2aa73be, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:06.038925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bc74f-bab1-433e-877d-cf5ffc185e9c admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:06.051438 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4500/8914] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:49:06 2022] PUT /identity/v3/projects/306407da7eef49318e1b08089a74d3d0/users/dcf0397ee6564254827baff0b2aa73be/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 21:49:10.692350 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44061-050a-4e28-a4be-d23a3e45da16 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:10.692877 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44061-050a-4e28-a4be-d23a3e45da16 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:10.693155 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44061-050a-4e28-a4be-d23a3e45da16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:10.733072 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-c0a44061-050a-4e28-a4be-d23a3e45da16 None None] MFA Rules not processed for user `dcf0397ee6564254827baff0b2aa73be`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:49:10.734310 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c0a44061-050a-4e28-a4be-d23a3e45da16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:49:10.817574 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4415/8915] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:49:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:49:10.840073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bc74f-bab1-433e-877d-cf5ffc185e9c admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:10.842533 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-517bc74f-bab1-433e-877d-cf5ffc185e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:49:10.951069 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bc74f-bab1-433e-877d-cf5ffc185e9c admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:10.956888 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c127c3-786a-4d22-b033-1a2d91ad1810 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:10.957557 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c127c3-786a-4d22-b033-1a2d91ad1810 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:10.957849 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c127c3-786a-4d22-b033-1a2d91ad1810 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:10.958134 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c127c3-786a-4d22-b033-1a2d91ad1810 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:11.013380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:49:11.031992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c127c3-786a-4d22-b033-1a2d91ad1810 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:11.035851 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c127c3-786a-4d22-b033-1a2d91ad1810 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:11.046019 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4501/8916] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:49:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:49:12.030752 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10028f51-3534-45a2-a743-8fd567ff58db None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:12.031223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10028f51-3534-45a2-a743-8fd567ff58db None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:12.031319 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10028f51-3534-45a2-a743-8fd567ff58db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:12.050281 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-10028f51-3534-45a2-a743-8fd567ff58db None None] MFA Rules not processed for user `03f57e800a6d4ca8a9080aa14d875cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:49:12.051484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-10028f51-3534-45a2-a743-8fd567ff58db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:49:12.058418 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4416/8917] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:49:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:49:21.041360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:49:22.045388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=69 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:49:23.552210 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c127c3-786a-4d22-b033-1a2d91ad1810 service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:49:23.611315 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c127c3-786a-4d22-b033-1a2d91ad1810 service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:49:23.616511 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba28af-f50c-45de-88bc-ef28e8114654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:49:23.617165 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba28af-f50c-45de-88bc-ef28e8114654 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:49:23.617456 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba28af-f50c-45de-88bc-ef28e8114654 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:49:23.617737 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba28af-f50c-45de-88bc-ef28e8114654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:49:23.627896 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba28af-f50c-45de-88bc-ef28e8114654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:49:23.629360 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba28af-f50c-45de-88bc-ef28e8114654 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:49:23.641392 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4502/8918] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:49:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:49:33.635172 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=26 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:50:35.349035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10028f51-3534-45a2-a743-8fd567ff58db None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:50:35.350462 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-10028f51-3534-45a2-a743-8fd567ff58db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:50:35.420434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10028f51-3534-45a2-a743-8fd567ff58db None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:50:35.425291 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ed64c-f316-4550-bb04-6e1d5d812e08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:50:35.425896 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ed64c-f316-4550-bb04-6e1d5d812e08 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:50:35.426180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ed64c-f316-4550-bb04-6e1d5d812e08 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:50:35.426453 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ed64c-f316-4550-bb04-6e1d5d812e08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:50:35.428156 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2f1ed64c-f316-4550-bb04-6e1d5d812e08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:50:35.439137 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ed64c-f316-4550-bb04-6e1d5d812e08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:50:35.440709 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ed64c-f316-4550-bb04-6e1d5d812e08 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:50:35.450252 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4417/8919] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:50:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:50:45.446274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=32 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:50:48.198086 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba28af-f50c-45de-88bc-ef28e8114654 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:50:48.209658 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba28af-f50c-45de-88bc-ef28e8114654 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:50:48.214905 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21d96d-6119-4487-bf04-8d93eb9609d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:50:48.215503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21d96d-6119-4487-bf04-8d93eb9609d1 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:50:48.215789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21d96d-6119-4487-bf04-8d93eb9609d1 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:50:48.216065 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21d96d-6119-4487-bf04-8d93eb9609d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:50:48.218379 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ea21d96d-6119-4487-bf04-8d93eb9609d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:50:48.229971 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21d96d-6119-4487-bf04-8d93eb9609d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:50:48.231550 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21d96d-6119-4487-bf04-8d93eb9609d1 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:50:48.241755 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4503/8920] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:50:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:50:58.237245 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:51:51.526091 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26a4ca-b2c9-43ff-926c-dc23a13879a3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:51:51.526598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26a4ca-b2c9-43ff-926c-dc23a13879a3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:51:51.526713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26a4ca-b2c9-43ff-926c-dc23a13879a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:51:51.576447 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-1e26a4ca-b2c9-43ff-926c-dc23a13879a3 None None] MFA Rules not processed for user `7a36c3c09b2d425fbf6ccbe051c293d4`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:51:51.577718 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1e26a4ca-b2c9-43ff-926c-dc23a13879a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:51:51.639539 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4418/8921] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Sep 3 21:51:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:51:51.645989 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21d96d-6119-4487-bf04-8d93eb9609d1 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:51:51.655064 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21d96d-6119-4487-bf04-8d93eb9609d1 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:51:51.658751 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20faf6e9-545f-485f-a7d8-777a93dea8de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:51:51.659361 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faf6e9-545f-485f-a7d8-777a93dea8de service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:51:51.659647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faf6e9-545f-485f-a7d8-777a93dea8de service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:51:51.659929 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faf6e9-545f-485f-a7d8-777a93dea8de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:51:51.661695 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-20faf6e9-545f-485f-a7d8-777a93dea8de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:51:51.687982 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20faf6e9-545f-485f-a7d8-777a93dea8de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:51:51.689480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20faf6e9-545f-485f-a7d8-777a93dea8de service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:51:51.700237 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4504/8922] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:51:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:51:51.902349 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26a4ca-b2c9-43ff-926c-dc23a13879a3 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:51:51.903711 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-1e26a4ca-b2c9-43ff-926c-dc23a13879a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:51:51.916463 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26a4ca-b2c9-43ff-926c-dc23a13879a3 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:51:51.920434 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a57624-6bfd-4095-a94b-2f357fdcf4d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:51:51.921100 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a57624-6bfd-4095-a94b-2f357fdcf4d5 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:51:51.921391 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a57624-6bfd-4095-a94b-2f357fdcf4d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:51:51.921675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a57624-6bfd-4095-a94b-2f357fdcf4d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:51:51.925226 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f9a57624-6bfd-4095-a94b-2f357fdcf4d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:51:51.939721 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a57624-6bfd-4095-a94b-2f357fdcf4d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:51:51.941311 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a57624-6bfd-4095-a94b-2f357fdcf4d5 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:51:51.953909 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4419/8923] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:51:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:52:01.695585 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=11 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:52:01.948405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=44 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:52:08.881266 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7903ab0-bfba-49d2-a3f4-0d3efb5c5a57 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:08.881734 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7903ab0-bfba-49d2-a3f4-0d3efb5c5a57 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:08.881839 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7903ab0-bfba-49d2-a3f4-0d3efb5c5a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:08.934460 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-f7903ab0-bfba-49d2-a3f4-0d3efb5c5a57 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:52:08.935901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-f7903ab0-bfba-49d2-a3f4-0d3efb5c5a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:52:09.002592 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4505/8924] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:52:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:52:09.049202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a57624-6bfd-4095-a94b-2f357fdcf4d5 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:52:09.050610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-f9a57624-6bfd-4095-a94b-2f357fdcf4d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:52:09.143140 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a57624-6bfd-4095-a94b-2f357fdcf4d5 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:52:09.148276 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d04b3b-f834-4251-88e1-e47bf4158a6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:52:09.148884 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d04b3b-f834-4251-88e1-e47bf4158a6e service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:09.149199 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d04b3b-f834-4251-88e1-e47bf4158a6e service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:09.149479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d04b3b-f834-4251-88e1-e47bf4158a6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:09.158180 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d04b3b-f834-4251-88e1-e47bf4158a6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:52:09.159775 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d04b3b-f834-4251-88e1-e47bf4158a6e service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:52:09.167817 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4420/8925] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:52:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:52:18.502632 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7903ab0-bfba-49d2-a3f4-0d3efb5c5a57 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:52:18.511965 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7903ab0-bfba-49d2-a3f4-0d3efb5c5a57 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:52:18.515532 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a44c85-9f86-4d7b-9b59-6c709304d263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:52:18.516138 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44c85-9f86-4d7b-9b59-6c709304d263 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:18.516421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44c85-9f86-4d7b-9b59-6c709304d263 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:18.516697 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44c85-9f86-4d7b-9b59-6c709304d263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:18.519019 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-97a44c85-9f86-4d7b-9b59-6c709304d263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:52:18.528164 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a44c85-9f86-4d7b-9b59-6c709304d263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:52:18.529615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a44c85-9f86-4d7b-9b59-6c709304d263 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:52:18.537322 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4506/8926] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:52:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:52:19.164590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=41 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:52:22.869826 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bda9f3-458f-4c8b-9a8d-8ea8c78b07c6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:22.870427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bda9f3-458f-4c8b-9a8d-8ea8c78b07c6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:22.870427 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bda9f3-458f-4c8b-9a8d-8ea8c78b07c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:22.890386 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-d9bda9f3-458f-4c8b-9a8d-8ea8c78b07c6 None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:52:22.891610 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-d9bda9f3-458f-4c8b-9a8d-8ea8c78b07c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:52:22.901674 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4421/8927] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 21:52:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:52:22.912943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a44c85-9f86-4d7b-9b59-6c709304d263 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:52:22.922143 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a44c85-9f86-4d7b-9b59-6c709304d263 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:52:22.925757 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bed19-fc1e-4d17-82ee-64ef019450c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:52:22.926362 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bed19-fc1e-4d17-82ee-64ef019450c7 service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:22.926651 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bed19-fc1e-4d17-82ee-64ef019450c7 service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:22.926925 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bed19-fc1e-4d17-82ee-64ef019450c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:22.937553 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bed19-fc1e-4d17-82ee-64ef019450c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:52:22.939517 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bed19-fc1e-4d17-82ee-64ef019450c7 service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:52:22.951164 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4507/8928] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:52:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:52:23.013998 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940e5a2-4c89-48a8-9f57-c34fc3a4e836 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:23.014318 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940e5a2-4c89-48a8-9f57-c34fc3a4e836 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:23.014599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940e5a2-4c89-48a8-9f57-c34fc3a4e836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:23.034484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-0940e5a2-4c89-48a8-9f57-c34fc3a4e836 None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:52:23.035681 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-0940e5a2-4c89-48a8-9f57-c34fc3a4e836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:52:23.042290 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4422/8929] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 21:52:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:52:23.053229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bed19-fc1e-4d17-82ee-64ef019450c7 service glance] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:52:23.062611 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bed19-fc1e-4d17-82ee-64ef019450c7 service glance] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:52:23.066781 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2ca6a-d035-411c-8762-e201969e0110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:52:23.067375 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2ca6a-d035-411c-8762-e201969e0110 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:23.067662 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2ca6a-d035-411c-8762-e201969e0110 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:23.067943 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2ca6a-d035-411c-8762-e201969e0110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:23.074880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2ca6a-d035-411c-8762-e201969e0110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:52:23.076308 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2ca6a-d035-411c-8762-e201969e0110 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:52:23.084690 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4508/8930] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:52:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:52:23.467713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13262d45-60ce-4930-868b-f064fabc33e8 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:23.468006 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13262d45-60ce-4930-868b-f064fabc33e8 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:23.468286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13262d45-60ce-4930-868b-f064fabc33e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:23.488704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-13262d45-60ce-4930-868b-f064fabc33e8 None None] MFA Rules not processed for user `a56702329eae48819121dcb58f7e3070`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:52:23.490233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-13262d45-60ce-4930-868b-f064fabc33e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:52:23.499690 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4423/8931] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 21:52:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:52:23.510713 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2ca6a-d035-411c-8762-e201969e0110 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:52:23.519219 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2ca6a-d035-411c-8762-e201969e0110 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:52:23.522997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93c289-52ca-465d-a822-c67be5b28ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:52:23.523603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93c289-52ca-465d-a822-c67be5b28ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:23.523893 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93c289-52ca-465d-a822-c67be5b28ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:23.524206 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93c289-52ca-465d-a822-c67be5b28ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:23.534950 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93c289-52ca-465d-a822-c67be5b28ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:52:23.536388 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93c289-52ca-465d-a822-c67be5b28ed1 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:52:23.547210 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4509/8932] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:52:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:52:23.607560 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13262d45-60ce-4930-868b-f064fabc33e8 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:52:23.619580 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13262d45-60ce-4930-868b-f064fabc33e8 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:52:23.624600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735598a6-2289-4e58-a54c-707d82368c1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:52:23.625233 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735598a6-2289-4e58-a54c-707d82368c1a service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:23.625540 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735598a6-2289-4e58-a54c-707d82368c1a service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:23.625862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735598a6-2289-4e58-a54c-707d82368c1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:23.627527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-735598a6-2289-4e58-a54c-707d82368c1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:52:23.638579 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735598a6-2289-4e58-a54c-707d82368c1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:52:23.640162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735598a6-2289-4e58-a54c-707d82368c1a service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:52:23.650076 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4424/8933] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:52:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:52:25.905933 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b5761-ee7d-4365-aa5e-24699b0c35d7 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:25.906814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b5761-ee7d-4365-aa5e-24699b0c35d7 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:25.907101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b5761-ee7d-4365-aa5e-24699b0c35d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:25.927656 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-186b5761-ee7d-4365-aa5e-24699b0c35d7 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:52:25.928926 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-186b5761-ee7d-4365-aa5e-24699b0c35d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:52:25.936118 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4510/8934] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:52:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:52:25.949413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735598a6-2289-4e58-a54c-707d82368c1a service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:52:25.961597 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735598a6-2289-4e58-a54c-707d82368c1a service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:52:25.967053 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12386f7-f115-458f-9362-5f94b3856e67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:52:25.967659 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12386f7-f115-458f-9362-5f94b3856e67 service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:25.967943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12386f7-f115-458f-9362-5f94b3856e67 service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:25.968228 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12386f7-f115-458f-9362-5f94b3856e67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:25.976730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12386f7-f115-458f-9362-5f94b3856e67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:52:25.978591 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12386f7-f115-458f-9362-5f94b3856e67 service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:52:25.987573 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4425/8935] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:52:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:52:35.922555 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=67 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:52:35.983960 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=21 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:52:41.091742 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea8e87-be34-4620-916b-9131f87c268c None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:41.092212 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea8e87-be34-4620-916b-9131f87c268c None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:41.092309 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea8e87-be34-4620-916b-9131f87c268c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:41.111107 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-bbea8e87-be34-4620-916b-9131f87c268c None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:52:41.112272 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-bbea8e87-be34-4620-916b-9131f87c268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:52:41.119754 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4511/8936] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:52:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:52:41.132116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12386f7-f115-458f-9362-5f94b3856e67 service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:52:41.143441 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12386f7-f115-458f-9362-5f94b3856e67 service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:52:41.148431 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7f88e-6eff-47e4-bc59-a71602338bdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:52:41.149061 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7f88e-6eff-47e4-bc59-a71602338bdf service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:52:41.149346 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7f88e-6eff-47e4-bc59-a71602338bdf service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:52:41.149620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7f88e-6eff-47e4-bc59-a71602338bdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:52:41.157658 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7f88e-6eff-47e4-bc59-a71602338bdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:52:41.159230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7f88e-6eff-47e4-bc59-a71602338bdf service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:52:41.167081 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4426/8937] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:52:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:52:51.105771 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=3 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:52:51.164029 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:53:32.095846 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea8e87-be34-4620-916b-9131f87c268c None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:53:32.105687 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea8e87-be34-4620-916b-9131f87c268c None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:53:32.109276 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3b71d-4e5a-4fcc-8708-beb11c004f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:53:32.109876 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3b71d-4e5a-4fcc-8708-beb11c004f83 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:53:32.110159 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3b71d-4e5a-4fcc-8708-beb11c004f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:53:32.110432 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3b71d-4e5a-4fcc-8708-beb11c004f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:53:32.112170 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-9db3b71d-4e5a-4fcc-8708-beb11c004f83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:53:32.124624 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3b71d-4e5a-4fcc-8708-beb11c004f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:53:32.126111 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3b71d-4e5a-4fcc-8708-beb11c004f83 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:53:32.135152 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4512/8938] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:53:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:53:42.131646 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:53:55.013312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7f88e-6eff-47e4-bc59-a71602338bdf service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:53:55.029519 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7f88e-6eff-47e4-bc59-a71602338bdf service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:53:55.035779 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febd1f1-4de5-4fe3-97b3-9cc38d153b3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:53:55.036480 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febd1f1-4de5-4fe3-97b3-9cc38d153b3e service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:53:55.036838 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febd1f1-4de5-4fe3-97b3-9cc38d153b3e service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:53:55.037223 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febd1f1-4de5-4fe3-97b3-9cc38d153b3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:53:55.039183 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-2febd1f1-4de5-4fe3-97b3-9cc38d153b3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:53:55.053555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febd1f1-4de5-4fe3-97b3-9cc38d153b3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:53:55.056247 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febd1f1-4de5-4fe3-97b3-9cc38d153b3e service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:53:55.069166 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4427/8939] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:53:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:53:58.790724 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06f922-9833-4cb0-8977-f1763db98dcd None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:53:58.791253 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06f922-9833-4cb0-8977-f1763db98dcd None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:53:58.791350 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06f922-9833-4cb0-8977-f1763db98dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:53:58.809900 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-0b06f922-9833-4cb0-8977-f1763db98dcd None None] MFA Rules not processed for user `03f57e800a6d4ca8a9080aa14d875cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:53:58.811108 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0b06f922-9833-4cb0-8977-f1763db98dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:53:58.817940 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4513/8940] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:53:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:54:00.632445 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febd1f1-4de5-4fe3-97b3-9cc38d153b3e service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:00.641783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febd1f1-4de5-4fe3-97b3-9cc38d153b3e service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:00.646343 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d57941-c0e4-45d0-9792-1f446bf0bfe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:00.646952 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d57941-c0e4-45d0-9792-1f446bf0bfe2 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:00.647288 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d57941-c0e4-45d0-9792-1f446bf0bfe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:00.647564 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d57941-c0e4-45d0-9792-1f446bf0bfe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:00.708543 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d57941-c0e4-45d0-9792-1f446bf0bfe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:00.710023 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d57941-c0e4-45d0-9792-1f446bf0bfe2 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:00.719632 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4428/8941] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:54:04.121956 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2340a-323f-4135-9167-1a06ac492353 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:04.122475 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2340a-323f-4135-9167-1a06ac492353 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:04.125016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2340a-323f-4135-9167-1a06ac492353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:04.150231 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-6fe2340a-323f-4135-9167-1a06ac492353 None None] MFA Rules not processed for user `6dfb57e006f344808f7a2ef41b246d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:54:04.151478 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6fe2340a-323f-4135-9167-1a06ac492353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:04.160138 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4514/8942] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 3 21:54:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:54:04.174418 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d57941-c0e4-45d0-9792-1f446bf0bfe2 service neutron] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:04.176759 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-80d57941-c0e4-45d0-9792-1f446bf0bfe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:04.193323 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d57941-c0e4-45d0-9792-1f446bf0bfe2 service neutron] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:04.198493 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d8b4-1175-4770-86a0-0b898c05253e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:04.199116 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d8b4-1175-4770-86a0-0b898c05253e service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:04.199405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d8b4-1175-4770-86a0-0b898c05253e service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:04.199685 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d8b4-1175-4770-86a0-0b898c05253e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:04.209019 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081d8b4-1175-4770-86a0-0b898c05253e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:04.210778 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081d8b4-1175-4770-86a0-0b898c05253e service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:04.221220 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4429/8943] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:54:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:54:07.034063 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe2340a-323f-4135-9167-1a06ac492353 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:07.044966 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe2340a-323f-4135-9167-1a06ac492353 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:07.051084 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db743b-6631-4ad6-b7da-b5286bfbf070 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:07.051813 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db743b-6631-4ad6-b7da-b5286bfbf070 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:07.052257 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db743b-6631-4ad6-b7da-b5286bfbf070 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:07.052983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db743b-6631-4ad6-b7da-b5286bfbf070 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:07.054845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-d1db743b-6631-4ad6-b7da-b5286bfbf070 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:07.068762 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db743b-6631-4ad6-b7da-b5286bfbf070 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:07.070539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db743b-6631-4ad6-b7da-b5286bfbf070 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:07.087557 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4515/8944] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:54:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:54:07.111114 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d8b4-1175-4770-86a0-0b898c05253e service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:07.130962 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d8b4-1175-4770-86a0-0b898c05253e service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:07.138015 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:07.138675 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:07.139104 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:07.139394 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] PATH_INFO: `/v3/users/03f57e800a6d4ca8a9080aa14d875cf3` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:07.140571 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f57e800a6d4ca8a9080aa14d875cf3)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:07.142713 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:07.190818 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f57e800a6d4ca8a9080aa14d875cf3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.202868 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f57e800a6d4ca8a9080aa14d875cf3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.203547 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f57e800a6d4ca8a9080aa14d875cf3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.210527 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f57e800a6d4ca8a9080aa14d875cf3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.217906 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f57e800a6d4ca8a9080aa14d875cf3'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.224146 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4430/8945] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:54:07 2022] DELETE /identity/v3/users/03f57e800a6d4ca8a9080aa14d875cf3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:54:07.398296 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db743b-6631-4ad6-b7da-b5286bfbf070 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:07.411968 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db743b-6631-4ad6-b7da-b5286bfbf070 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:07.468408 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:07.469076 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:07.469365 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:07.469644 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] PATH_INFO: `/v3/projects/f101b0010f234a4199e698ad0b8decd7` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:07.470779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f101b0010f234a4199e698ad0b8decd7)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:07.473230 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:07.518504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f101b0010f234a4199e698ad0b8decd7', 'request_id': 'req-09a817c0-ce5c-4ec9-960d-f02fdc730529'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.524806 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f101b0010f234a4199e698ad0b8decd7', 'request_id': 'req-09a817c0-ce5c-4ec9-960d-f02fdc730529'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.531058 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f101b0010f234a4199e698ad0b8decd7', 'request_id': 'req-09a817c0-ce5c-4ec9-960d-f02fdc730529'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.542247 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] The token cache is being invalidate because project f101b0010f234a4199e698ad0b8decd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:54:07.542574 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.543214 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.544721 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4516/8946] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:54:07 2022] DELETE /identity/v3/projects/f101b0010f234a4199e698ad0b8decd7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:54:07.548695 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:07.550746 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:07.563799 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42158549-c436-40f3-be1d-c6563eff7e4a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:07.601521 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:54:07.626047 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95532a3b-c934-4905-b339-0e8c1ea0a275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:07.626679 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95532a3b-c934-4905-b339-0e8c1ea0a275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:07.626968 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95532a3b-c934-4905-b339-0e8c1ea0a275 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:07.627248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95532a3b-c934-4905-b339-0e8c1ea0a275 admin admin] PATH_INFO: `/v3/users/dcf0397ee6564254827baff0b2aa73be` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:07.628405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95532a3b-c934-4905-b339-0e8c1ea0a275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf0397ee6564254827baff0b2aa73be)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:07.630424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95532a3b-c934-4905-b339-0e8c1ea0a275 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:07.677632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-95532a3b-c934-4905-b339-0e8c1ea0a275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf0397ee6564254827baff0b2aa73be'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.689605 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-95532a3b-c934-4905-b339-0e8c1ea0a275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf0397ee6564254827baff0b2aa73be'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.690316 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-95532a3b-c934-4905-b339-0e8c1ea0a275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf0397ee6564254827baff0b2aa73be'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.697283 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-95532a3b-c934-4905-b339-0e8c1ea0a275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf0397ee6564254827baff0b2aa73be'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.704551 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-95532a3b-c934-4905-b339-0e8c1ea0a275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf0397ee6564254827baff0b2aa73be'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:07.711060 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4431/8947] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:54:07 2022] DELETE /identity/v3/users/dcf0397ee6564254827baff0b2aa73be => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:54:07.884029 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:07.885586 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:07.901464 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a817c0-ce5c-4ec9-960d-f02fdc730529 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:07.957526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426251e-29f7-469c-8254-a6adb3b16aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:07.958155 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426251e-29f7-469c-8254-a6adb3b16aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:07.958438 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426251e-29f7-469c-8254-a6adb3b16aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:07.958739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426251e-29f7-469c-8254-a6adb3b16aaf admin admin] PATH_INFO: `/v3/projects/306407da7eef49318e1b08089a74d3d0` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:07.959852 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426251e-29f7-469c-8254-a6adb3b16aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=306407da7eef49318e1b08089a74d3d0)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:07.961804 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426251e-29f7-469c-8254-a6adb3b16aaf admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:07.967853 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:54:08.009080 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4426251e-29f7-469c-8254-a6adb3b16aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '306407da7eef49318e1b08089a74d3d0', 'request_id': 'req-4426251e-29f7-469c-8254-a6adb3b16aaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:08.015992 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4426251e-29f7-469c-8254-a6adb3b16aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '306407da7eef49318e1b08089a74d3d0', 'request_id': 'req-4426251e-29f7-469c-8254-a6adb3b16aaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:08.022633 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4426251e-29f7-469c-8254-a6adb3b16aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '306407da7eef49318e1b08089a74d3d0', 'request_id': 'req-4426251e-29f7-469c-8254-a6adb3b16aaf'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:08.032955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4426251e-29f7-469c-8254-a6adb3b16aaf admin admin] The token cache is being invalidate because project 306407da7eef49318e1b08089a74d3d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:54:08.033224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4426251e-29f7-469c-8254-a6adb3b16aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:08.033970 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-4426251e-29f7-469c-8254-a6adb3b16aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:54:08.035941 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4517/8948] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:54:07 2022] DELETE /identity/v3/projects/306407da7eef49318e1b08089a74d3d0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:54:08.040989 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd1c77-6ecf-45ff-be19-e41d4609e59e None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:08.041375 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd1c77-6ecf-45ff-be19-e41d4609e59e None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:08.041654 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd1c77-6ecf-45ff-be19-e41d4609e59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:08.061422 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-decd1c77-6ecf-45ff-be19-e41d4609e59e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:54:08.062640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-decd1c77-6ecf-45ff-be19-e41d4609e59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:08.121298 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4432/8949] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:54:08.129650 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3882e-0b73-4b89-a02f-2ca733e93c7e None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:08.129946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3882e-0b73-4b89-a02f-2ca733e93c7e None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:08.130224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3882e-0b73-4b89-a02f-2ca733e93c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:08.150492 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-b7e3882e-0b73-4b89-a02f-2ca733e93c7e None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:54:08.151684 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b7e3882e-0b73-4b89-a02f-2ca733e93c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:08.159470 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4518/8950] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:54:08.163765 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd1c77-6ecf-45ff-be19-e41d4609e59e None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:08.173405 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd1c77-6ecf-45ff-be19-e41d4609e59e None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:08.177935 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476246fd-e084-435f-b768-ffd927eacdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:08.178526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476246fd-e084-435f-b768-ffd927eacdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:08.178813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476246fd-e084-435f-b768-ffd927eacdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:08.179086 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476246fd-e084-435f-b768-ffd927eacdf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:08.179724 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476246fd-e084-435f-b768-ffd927eacdf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 21:54:08.180028 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476246fd-e084-435f-b768-ffd927eacdf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:08.181640 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476246fd-e084-435f-b768-ffd927eacdf5 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:08.193428 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4433/8951] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 3 21:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 21:54:08.197628 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3882e-0b73-4b89-a02f-2ca733e93c7e None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:08.207338 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3882e-0b73-4b89-a02f-2ca733e93c7e None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:08.212267 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce6dcf-1b5d-451c-8a6c-7c6ecbb32f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:08.212861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce6dcf-1b5d-451c-8a6c-7c6ecbb32f23 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:08.213176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce6dcf-1b5d-451c-8a6c-7c6ecbb32f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:08.213587 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce6dcf-1b5d-451c-8a6c-7c6ecbb32f23 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:08.214218 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce6dcf-1b5d-451c-8a6c-7c6ecbb32f23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:08.216116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce6dcf-1b5d-451c-8a6c-7c6ecbb32f23 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:08.243765 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4519/8952] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:54:08 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:54:08.247966 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476246fd-e084-435f-b768-ffd927eacdf5 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:08.258572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476246fd-e084-435f-b768-ffd927eacdf5 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:08.302101 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:54:08.317512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a53e7-8925-4645-9e2e-601b3103e10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:08.318162 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a53e7-8925-4645-9e2e-601b3103e10f admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:08.318309 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a53e7-8925-4645-9e2e-601b3103e10f admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:08.318586 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a53e7-8925-4645-9e2e-601b3103e10f admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:08.319488 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a53e7-8925-4645-9e2e-601b3103e10f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:08.322216 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a53e7-8925-4645-9e2e-601b3103e10f admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:08.347566 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4434/8953] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:54:08 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:54:08.351907 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce6dcf-1b5d-451c-8a6c-7c6ecbb32f23 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:08.365381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce6dcf-1b5d-451c-8a6c-7c6ecbb32f23 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:08.371269 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6bdfe-77c6-4fec-b384-73d797001c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:08.371892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6bdfe-77c6-4fec-b384-73d797001c98 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:08.372182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6bdfe-77c6-4fec-b384-73d797001c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:08.372459 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6bdfe-77c6-4fec-b384-73d797001c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:08.373197 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6bdfe-77c6-4fec-b384-73d797001c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:08.375601 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6bdfe-77c6-4fec-b384-73d797001c98 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:08.389272 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4520/8954] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:54:08.393987 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a53e7-8925-4645-9e2e-601b3103e10f admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:08.403997 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a53e7-8925-4645-9e2e-601b3103e10f admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:08.408683 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3de0de-78f8-4ff6-a513-44fc0c16e132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:08.409352 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3de0de-78f8-4ff6-a513-44fc0c16e132 admin admin] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:08.409639 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3de0de-78f8-4ff6-a513-44fc0c16e132 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:08.409916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3de0de-78f8-4ff6-a513-44fc0c16e132 admin admin] PATH_INFO: `/v3/roles` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:08.410559 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3de0de-78f8-4ff6-a513-44fc0c16e132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:08.412164 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3de0de-78f8-4ff6-a513-44fc0c16e132 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:08.423635 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4435/8955] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:54:08.427725 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6bdfe-77c6-4fec-b384-73d797001c98 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:08.437899 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6bdfe-77c6-4fec-b384-73d797001c98 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:08.443082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf72e6-6f9e-44ab-8552-c5c683dda614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:08.443738 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf72e6-6f9e-44ab-8552-c5c683dda614 admin admin] REQUEST_METHOD: `PUT` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:08.444023 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf72e6-6f9e-44ab-8552-c5c683dda614 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:08.444304 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf72e6-6f9e-44ab-8552-c5c683dda614 admin admin] PATH_INFO: `/v3/projects/6ceb7d1e235e467c847a20cbd4b660b6/users/229f2394f4a64bb09c05994e012e2a55/roles/d167c236cee64ede8d62bce075f83fff` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:08.459411 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf72e6-6f9e-44ab-8552-c5c683dda614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ceb7d1e235e467c847a20cbd4b660b6, user_id=229f2394f4a64bb09c05994e012e2a55, role_id=d167c236cee64ede8d62bce075f83fff)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:08.462169 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf72e6-6f9e-44ab-8552-c5c683dda614 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:08.474698 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4521/8956] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:54:08 2022] PUT /identity/v3/projects/6ceb7d1e235e467c847a20cbd4b660b6/users/229f2394f4a64bb09c05994e012e2a55/roles/d167c236cee64ede8d62bce075f83fff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:54:08.479862 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8141e2-1b1e-4683-829b-56b53046dcc6 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:08.480249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8141e2-1b1e-4683-829b-56b53046dcc6 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:08.480598 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8141e2-1b1e-4683-829b-56b53046dcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:08.500704 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-bb8141e2-1b1e-4683-829b-56b53046dcc6 None None] MFA Rules not processed for user `efa1ecf6d7f2438dad00007510b3e878`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:54:08.501943 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-bb8141e2-1b1e-4683-829b-56b53046dcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:08.560507 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4436/8957] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:54:08.570667 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf72e6-6f9e-44ab-8552-c5c683dda614 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:08.572364 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-1abf72e6-6f9e-44ab-8552-c5c683dda614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:08.628768 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf72e6-6f9e-44ab-8552-c5c683dda614 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:08.632374 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b9e7f-0f44-4fa4-88a5-a22e49e5830b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:08.633007 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b9e7f-0f44-4fa4-88a5-a22e49e5830b service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:08.633294 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b9e7f-0f44-4fa4-88a5-a22e49e5830b service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:08.633570 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b9e7f-0f44-4fa4-88a5-a22e49e5830b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:08.642376 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b9e7f-0f44-4fa4-88a5-a22e49e5830b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:08.643825 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b9e7f-0f44-4fa4-88a5-a22e49e5830b service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:08.653475 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4522/8958] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:54:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:54:12.717890 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2ec70-0d9d-4b98-a0c3-416ded3c6179 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:12.718428 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2ec70-0d9d-4b98-a0c3-416ded3c6179 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:12.718601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2ec70-0d9d-4b98-a0c3-416ded3c6179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:12.756843 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-57c2ec70-0d9d-4b98-a0c3-416ded3c6179 None None] MFA Rules not processed for user `229f2394f4a64bb09c05994e012e2a55`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:54:12.758231 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-57c2ec70-0d9d-4b98-a0c3-416ded3c6179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:12.830197 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4437/8959] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:54:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:54:12.834997 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b9e7f-0f44-4fa4-88a5-a22e49e5830b service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:12.845291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b9e7f-0f44-4fa4-88a5-a22e49e5830b service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:12.850176 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c8a19-2f2b-4a87-8adf-302edd0aa4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:12.850792 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c8a19-2f2b-4a87-8adf-302edd0aa4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:12.851075 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c8a19-2f2b-4a87-8adf-302edd0aa4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:12.851352 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c8a19-2f2b-4a87-8adf-302edd0aa4f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:12.852030 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c8a19-2f2b-4a87-8adf-302edd0aa4f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:12.854016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c8a19-2f2b-4a87-8adf-302edd0aa4f6 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:12.885579 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4523/8960] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 3 21:54:12 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:54:12.886879 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2ec70-0d9d-4b98-a0c3-416ded3c6179 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:12.899084 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2ec70-0d9d-4b98-a0c3-416ded3c6179 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:12.953572 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:54:12.963647 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e52f94-ae65-41fb-9a80-c0542c79c551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:12.964487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e52f94-ae65-41fb-9a80-c0542c79c551 admin admin] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:12.964875 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e52f94-ae65-41fb-9a80-c0542c79c551 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:12.965286 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e52f94-ae65-41fb-9a80-c0542c79c551 admin admin] PATH_INFO: `/v3/users` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:12.966050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e52f94-ae65-41fb-9a80-c0542c79c551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:12.968555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e52f94-ae65-41fb-9a80-c0542c79c551 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:12.995413 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4438/8961] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 3 21:54:12 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 21:54:12.996889 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c8a19-2f2b-4a87-8adf-302edd0aa4f6 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:13.008310 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c8a19-2f2b-4a87-8adf-302edd0aa4f6 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:13.013209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8d8f-2a50-41a3-a344-537c8d89e268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:13.013823 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8d8f-2a50-41a3-a344-537c8d89e268 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:13.014106 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8d8f-2a50-41a3-a344-537c8d89e268 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:13.014381 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8d8f-2a50-41a3-a344-537c8d89e268 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:13.015025 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d8d8f-2a50-41a3-a344-537c8d89e268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:13.016683 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d8d8f-2a50-41a3-a344-537c8d89e268 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:13.023627 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:54:13.032855 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4524/8962] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:54:13.034225 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e52f94-ae65-41fb-9a80-c0542c79c551 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:13.044356 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e52f94-ae65-41fb-9a80-c0542c79c551 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:13.049197 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6f580-98b4-4b8c-8a21-281f2c146264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:13.050457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6f580-98b4-4b8c-8a21-281f2c146264 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:13.050808 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6f580-98b4-4b8c-8a21-281f2c146264 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:13.051163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6f580-98b4-4b8c-8a21-281f2c146264 admin admin] PATH_INFO: `/v3/projects/8bf4b843f6ac45ecba12bcee2a82018d/users/d44b98df884c4cf4831654a06335bb96/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:13.067632 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6f580-98b4-4b8c-8a21-281f2c146264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf4b843f6ac45ecba12bcee2a82018d, user_id=d44b98df884c4cf4831654a06335bb96, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:13.070456 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6f580-98b4-4b8c-8a21-281f2c146264 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:13.084358 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4439/8963] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:54:13 2022] PUT /identity/v3/projects/8bf4b843f6ac45ecba12bcee2a82018d/users/d44b98df884c4cf4831654a06335bb96/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:54:13.088270 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8d8f-2a50-41a3-a344-537c8d89e268 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:13.099397 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8d8f-2a50-41a3-a344-537c8d89e268 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:13.162244 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ea3d3-5746-4511-bc75-3c76a512a712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:13.162791 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ea3d3-5746-4511-bc75-3c76a512a712 admin admin] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:13.163073 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ea3d3-5746-4511-bc75-3c76a512a712 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:13.163351 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ea3d3-5746-4511-bc75-3c76a512a712 admin admin] PATH_INFO: `/v3/roles` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:13.163986 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ea3d3-5746-4511-bc75-3c76a512a712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:13.165620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ea3d3-5746-4511-bc75-3c76a512a712 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:13.176665 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4525/8964] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 3 21:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 21:54:13.181064 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6f580-98b4-4b8c-8a21-281f2c146264 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:13.194224 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6f580-98b4-4b8c-8a21-281f2c146264 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:13.199248 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e79be-7632-459f-bc71-e55b90447169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:13.199916 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e79be-7632-459f-bc71-e55b90447169 admin admin] REQUEST_METHOD: `PUT` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:13.200202 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e79be-7632-459f-bc71-e55b90447169 admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:13.200482 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e79be-7632-459f-bc71-e55b90447169 admin admin] PATH_INFO: `/v3/projects/8bf4b843f6ac45ecba12bcee2a82018d/users/d44b98df884c4cf4831654a06335bb96/roles/bc768522dc984239be4a1b39a45ab437` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:13.202457 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e79be-7632-459f-bc71-e55b90447169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf4b843f6ac45ecba12bcee2a82018d, user_id=d44b98df884c4cf4831654a06335bb96, role_id=bc768522dc984239be4a1b39a45ab437)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:13.205151 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e79be-7632-459f-bc71-e55b90447169 admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:13.215686 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4440/8965] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 3 21:54:13 2022] PUT /identity/v3/projects/8bf4b843f6ac45ecba12bcee2a82018d/users/d44b98df884c4cf4831654a06335bb96/roles/bc768522dc984239be4a1b39a45ab437 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:54:17.840461 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bff700-d68a-43fe-98d0-c9fe79342847 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:17.841537 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bff700-d68a-43fe-98d0-c9fe79342847 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:17.841983 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bff700-d68a-43fe-98d0-c9fe79342847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:17.895001 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-77bff700-d68a-43fe-98d0-c9fe79342847 None None] MFA Rules not processed for user `d44b98df884c4cf4831654a06335bb96`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:54:17.900769 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-77bff700-d68a-43fe-98d0-c9fe79342847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:18.008109 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4526/8966] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:54:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:54:18.026379 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e79be-7632-459f-bc71-e55b90447169 admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:18.028404 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-c81e79be-7632-459f-bc71-e55b90447169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:18.113201 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e79be-7632-459f-bc71-e55b90447169 admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:18.123327 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289aefea-e483-4c0a-8d34-55e3d0c2d8b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:18.125055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289aefea-e483-4c0a-8d34-55e3d0c2d8b2 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:18.125813 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289aefea-e483-4c0a-8d34-55e3d0c2d8b2 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:18.126526 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289aefea-e483-4c0a-8d34-55e3d0c2d8b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:18.194972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289aefea-e483-4c0a-8d34-55e3d0c2d8b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:18.196599 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289aefea-e483-4c0a-8d34-55e3d0c2d8b2 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:18.207035 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4441/8967] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:54:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:54:19.212205 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0f3f2-b4a5-4a3e-8f77-9b52fcf0233d None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:19.212653 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0f3f2-b4a5-4a3e-8f77-9b52fcf0233d None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:19.212789 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0f3f2-b4a5-4a3e-8f77-9b52fcf0233d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:19.232162 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-80e0f3f2-b4a5-4a3e-8f77-9b52fcf0233d None None] MFA Rules not processed for user `229f2394f4a64bb09c05994e012e2a55`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:54:19.233404 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-80e0f3f2-b4a5-4a3e-8f77-9b52fcf0233d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:19.240235 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4527/8968] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:54:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:54:19.553217 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b8962-c261-4c05-b912-139fb4516b7c None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:19.553717 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b8962-c261-4c05-b912-139fb4516b7c None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:19.553814 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b8962-c261-4c05-b912-139fb4516b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:19.574706 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-871b8962-c261-4c05-b912-139fb4516b7c None None] MFA Rules not processed for user `c0fb67db342b477aaf547543c498434f`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:54:19.575910 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-871b8962-c261-4c05-b912-139fb4516b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:19.618215 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:54:19.630830 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4442/8969] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:54:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:54:19.642655 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0f3f2-b4a5-4a3e-8f77-9b52fcf0233d None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:19.653132 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0f3f2-b4a5-4a3e-8f77-9b52fcf0233d None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:19.657053 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b9a73-a4b8-416f-a041-8b0e04d93cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:19.659539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b9a73-a4b8-416f-a041-8b0e04d93cae service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:19.659539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b9a73-a4b8-416f-a041-8b0e04d93cae service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:19.659539 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b9a73-a4b8-416f-a041-8b0e04d93cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:19.665909 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b9a73-a4b8-416f-a041-8b0e04d93cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:19.667391 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b9a73-a4b8-416f-a041-8b0e04d93cae service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:19.675959 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4528/8970] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:54:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:54:20.030184 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b8962-c261-4c05-b912-139fb4516b7c None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:20.031587 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-871b8962-c261-4c05-b912-139fb4516b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:20.115749 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b8962-c261-4c05-b912-139fb4516b7c None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:20.121266 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76818b45-df10-471b-a158-1250c469b834 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:20.121889 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76818b45-df10-471b-a158-1250c469b834 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:20.122176 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76818b45-df10-471b-a158-1250c469b834 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:20.122537 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76818b45-df10-471b-a158-1250c469b834 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:20.124249 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-76818b45-df10-471b-a158-1250c469b834 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:20.137188 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76818b45-df10-471b-a158-1250c469b834 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:20.139126 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76818b45-df10-471b-a158-1250c469b834 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:20.150256 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4443/8971] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:54:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:54:22.770482 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b9a73-a4b8-416f-a041-8b0e04d93cae service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:22.772166 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-b35b9a73-a4b8-416f-a041-8b0e04d93cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:22.791209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b9a73-a4b8-416f-a041-8b0e04d93cae service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:22.796885 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00017d8-3f48-4fa7-9b90-29ac04a0bf3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:22.797528 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00017d8-3f48-4fa7-9b90-29ac04a0bf3e service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:22.797817 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00017d8-3f48-4fa7-9b90-29ac04a0bf3e service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:22.798102 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00017d8-3f48-4fa7-9b90-29ac04a0bf3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:22.799953 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a00017d8-3f48-4fa7-9b90-29ac04a0bf3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:22.812777 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00017d8-3f48-4fa7-9b90-29ac04a0bf3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:22.814440 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00017d8-3f48-4fa7-9b90-29ac04a0bf3e service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:22.826549 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4529/8972] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:54:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:54:29.027069 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76818b45-df10-471b-a158-1250c469b834 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:29.040489 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76818b45-df10-471b-a158-1250c469b834 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:29.046132 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9374b-701a-4605-b1c6-c85bb103f72c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:29.046740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9374b-701a-4605-b1c6-c85bb103f72c service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:29.047141 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9374b-701a-4605-b1c6-c85bb103f72c service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:29.047424 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9374b-701a-4605-b1c6-c85bb103f72c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:29.049150 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-11d9374b-701a-4605-b1c6-c85bb103f72c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:29.061312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9374b-701a-4605-b1c6-c85bb103f72c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:29.062930 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9374b-701a-4605-b1c6-c85bb103f72c service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:29.073858 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4444/8973] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:54:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:54:32.096563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00017d8-3f48-4fa7-9b90-29ac04a0bf3e service nova] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:54:32.098082 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-a00017d8-3f48-4fa7-9b90-29ac04a0bf3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:54:32.109291 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00017d8-3f48-4fa7-9b90-29ac04a0bf3e service nova] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:54:32.112912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7971ab5-37f3-40ca-8a1f-28de00a974e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:54:32.113526 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7971ab5-37f3-40ca-8a1f-28de00a974e3 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:54:32.113818 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7971ab5-37f3-40ca-8a1f-28de00a974e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:54:32.114100 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7971ab5-37f3-40ca-8a1f-28de00a974e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:54:32.124488 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7971ab5-37f3-40ca-8a1f-28de00a974e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:54:32.126013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7971ab5-37f3-40ca-8a1f-28de00a974e3 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:54:32.136437 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4530/8974] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:54:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:54:39.069263 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=41 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:54:42.132028 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=84 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:55:43.343370 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9374b-701a-4605-b1c6-c85bb103f72c service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:55:43.344870 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-11d9374b-701a-4605-b1c6-c85bb103f72c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:55:43.359128 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9374b-701a-4605-b1c6-c85bb103f72c service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:55:43.364878 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692dffd-31d1-46de-bf0d-6fc45977dd66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:55:43.365616 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692dffd-31d1-46de-bf0d-6fc45977dd66 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:55:43.365907 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692dffd-31d1-46de-bf0d-6fc45977dd66 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:55:43.366210 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692dffd-31d1-46de-bf0d-6fc45977dd66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:55:43.368129 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-7692dffd-31d1-46de-bf0d-6fc45977dd66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:55:43.379252 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692dffd-31d1-46de-bf0d-6fc45977dd66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:55:43.380806 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692dffd-31d1-46de-bf0d-6fc45977dd66 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:55:43.390545 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4445/8975] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:55:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:55:53.386590 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:55:55.176401 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7971ab5-37f3-40ca-8a1f-28de00a974e3 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:55:55.187259 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7971ab5-37f3-40ca-8a1f-28de00a974e3 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:55:55.192880 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a7335-25c7-4db0-9524-732515351812 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:55:55.193494 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a7335-25c7-4db0-9524-732515351812 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:55:55.193778 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a7335-25c7-4db0-9524-732515351812 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:55:55.194052 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a7335-25c7-4db0-9524-732515351812 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:55:55.195754 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-dd4a7335-25c7-4db0-9524-732515351812 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:55:55.206779 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a7335-25c7-4db0-9524-732515351812 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:55:55.208340 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a7335-25c7-4db0-9524-732515351812 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:55:55.217860 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4531/8976] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:55:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:56:05.213935 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:57:00.216168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa8af5-392f-4f05-8b40-203b7aa35143 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:57:00.216730 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa8af5-392f-4f05-8b40-203b7aa35143 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:57:00.216839 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa8af5-392f-4f05-8b40-203b7aa35143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:57:00.246773 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-5faa8af5-392f-4f05-8b40-203b7aa35143 None None] MFA Rules not processed for user `7a36c3c09b2d425fbf6ccbe051c293d4`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:57:00.247988 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-5faa8af5-392f-4f05-8b40-203b7aa35143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:57:00.298716 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4446/8977] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Sep 3 21:57:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:57:00.304814 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a7335-25c7-4db0-9524-732515351812 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:57:00.313902 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a7335-25c7-4db0-9524-732515351812 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:57:00.317468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c07e0-1f6b-444a-95b0-09edc319c23c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:57:00.318061 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c07e0-1f6b-444a-95b0-09edc319c23c service glance] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:57:00.318345 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c07e0-1f6b-444a-95b0-09edc319c23c service glance] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:57:00.318621 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c07e0-1f6b-444a-95b0-09edc319c23c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:57:00.320319 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0b6c07e0-1f6b-444a-95b0-09edc319c23c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:57:00.334083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c07e0-1f6b-444a-95b0-09edc319c23c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:57:00.335508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c07e0-1f6b-444a-95b0-09edc319c23c service glance] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:57:00.356060 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4532/8978] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:57:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:57:09.096512 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75ac6d-af80-4e4c-8603-f1d6089072e5 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:57:09.097160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75ac6d-af80-4e4c-8603-f1d6089072e5 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:57:09.097160 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75ac6d-af80-4e4c-8603-f1d6089072e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:57:09.117012 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-cb75ac6d-af80-4e4c-8603-f1d6089072e5 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:57:09.118548 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-cb75ac6d-af80-4e4c-8603-f1d6089072e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:57:09.173790 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4447/8979] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:57:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:57:09.228150 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c07e0-1f6b-444a-95b0-09edc319c23c service glance] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:57:09.229618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-0b6c07e0-1f6b-444a-95b0-09edc319c23c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:57:09.297534 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c07e0-1f6b-444a-95b0-09edc319c23c service glance] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:57:09.303135 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb86e3-8704-4076-b7d7-0ed50838f125 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:57:09.303747 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb86e3-8704-4076-b7d7-0ed50838f125 service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:57:09.304033 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb86e3-8704-4076-b7d7-0ed50838f125 service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:57:09.304313 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb86e3-8704-4076-b7d7-0ed50838f125 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:57:09.312191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fb86e3-8704-4076-b7d7-0ed50838f125 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:57:09.313820 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fb86e3-8704-4076-b7d7-0ed50838f125 service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:57:09.322275 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4533/8980] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:57:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:57:19.135130 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa555114-a9cf-4c9a-a1e4-9d44592b7652 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:57:19.135515 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa555114-a9cf-4c9a-a1e4-9d44592b7652 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:57:19.135703 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa555114-a9cf-4c9a-a1e4-9d44592b7652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:57:19.155274 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-fa555114-a9cf-4c9a-a1e4-9d44592b7652 None None] MFA Rules not processed for user `229f2394f4a64bb09c05994e012e2a55`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:57:19.156486 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fa555114-a9cf-4c9a-a1e4-9d44592b7652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:57:19.164207 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4448/8981] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 3 21:57:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 21:57:19.319091 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=42 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:57:19.840209 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb86e3-8704-4076-b7d7-0ed50838f125 service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:57:19.860343 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb86e3-8704-4076-b7d7-0ed50838f125 service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:57:19.864469 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee7982-3592-4d0a-8e8c-386580af36e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:57:19.865116 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee7982-3592-4d0a-8e8c-386580af36e1 service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:57:19.865405 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee7982-3592-4d0a-8e8c-386580af36e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:57:19.865675 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee7982-3592-4d0a-8e8c-386580af36e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:57:19.867668 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-adee7982-3592-4d0a-8e8c-386580af36e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:57:19.882126 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee7982-3592-4d0a-8e8c-386580af36e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:57:19.883563 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee7982-3592-4d0a-8e8c-386580af36e1 service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:57:19.894833 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4534/8982] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:57:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 21:57:20.989050 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0c42-9d01-4a58-9db7-96dd66b468f3 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:57:20.989555 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0c42-9d01-4a58-9db7-96dd66b468f3 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:57:20.989637 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0c42-9d01-4a58-9db7-96dd66b468f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:57:21.010211 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-b91a0c42-9d01-4a58-9db7-96dd66b468f3 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:57:21.011600 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-b91a0c42-9d01-4a58-9db7-96dd66b468f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:57:21.019093 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4449/8983] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:57:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:57:21.031551 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee7982-3592-4d0a-8e8c-386580af36e1 service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:57:21.042564 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee7982-3592-4d0a-8e8c-386580af36e1 service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:57:21.047703 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d48422-407b-451b-910c-e4bfc214590f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:57:21.048373 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d48422-407b-451b-910c-e4bfc214590f service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:57:21.048583 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d48422-407b-451b-910c-e4bfc214590f service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:57:21.048861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d48422-407b-451b-910c-e4bfc214590f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:57:21.057368 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d48422-407b-451b-910c-e4bfc214590f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:57:21.058955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d48422-407b-451b-910c-e4bfc214590f service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:57:21.070032 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4535/8984] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:57:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:57:31.004035 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=58 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:57:31.064503 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=16 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:57:59.704376 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0c42-9d01-4a58-9db7-96dd66b468f3 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:57:59.724620 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0c42-9d01-4a58-9db7-96dd66b468f3 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:57:59.728913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7b828-7648-443f-9764-941dd75e4791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:57:59.729533 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7b828-7648-443f-9764-941dd75e4791 service neutron] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:57:59.729822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7b828-7648-443f-9764-941dd75e4791 service neutron] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:57:59.730098 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7b828-7648-443f-9764-941dd75e4791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:57:59.731822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-19a7b828-7648-443f-9764-941dd75e4791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:57:59.762222 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7b828-7648-443f-9764-941dd75e4791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:57:59.763656 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7b828-7648-443f-9764-941dd75e4791 service neutron] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:57:59.772176 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4450/8985] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:57:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:58:09.768209 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=19 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:58:38.085415 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d48422-407b-451b-910c-e4bfc214590f service swift] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:58:38.106664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d48422-407b-451b-910c-e4bfc214590f service swift] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:58:38.110618 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d887522-53a8-44a3-b369-a81617a1563c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:58:38.111264 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d887522-53a8-44a3-b369-a81617a1563c service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:58:38.111552 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d887522-53a8-44a3-b369-a81617a1563c service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:58:38.111901 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d887522-53a8-44a3-b369-a81617a1563c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:58:38.114229 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-6d887522-53a8-44a3-b369-a81617a1563c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:58:38.137298 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d887522-53a8-44a3-b369-a81617a1563c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:58:38.138753 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d887522-53a8-44a3-b369-a81617a1563c service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:58:38.146658 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4536/8986] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:58:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:58:48.143191 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=16 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:59:10.431454 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc8fbd-15bd-41da-9799-e0a63b96da65 None None] REQUEST_METHOD: `POST` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:10.432001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc8fbd-15bd-41da-9799-e0a63b96da65 None None] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:10.432001 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc8fbd-15bd-41da-9799-e0a63b96da65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:10.451198 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.auth.core [None req-a0fc8fbd-15bd-41da-9799-e0a63b96da65 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:59:10.452388 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a0fc8fbd-15bd-41da-9799-e0a63b96da65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:59:10.459627 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4451/8987] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 3 21:59:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:59:10.499832 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d887522-53a8-44a3-b369-a81617a1563c service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:10.532837 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d887522-53a8-44a3-b369-a81617a1563c service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:10.537946 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5414b0-7af0-46f9-9bc3-63bc808dd9ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:10.539184 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5414b0-7af0-46f9-9bc3-63bc808dd9ee service swift] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:10.539468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5414b0-7af0-46f9-9bc3-63bc808dd9ee service swift] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:10.539745 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5414b0-7af0-46f9-9bc3-63bc808dd9ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:10.547596 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5414b0-7af0-46f9-9bc3-63bc808dd9ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:10.549224 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5414b0-7af0-46f9-9bc3-63bc808dd9ee service swift] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:10.557055 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4537/8988] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:59:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:59:18.520057 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc8fbd-15bd-41da-9799-e0a63b96da65 None None] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:18.521501 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-a0fc8fbd-15bd-41da-9799-e0a63b96da65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:59:18.533947 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc8fbd-15bd-41da-9799-e0a63b96da65 None None] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:18.537755 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d578813-af60-4663-a591-ec7137399e9a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a36c3c09b2d425fbf6ccbe051c293d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:18.538595 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d578813-af60-4663-a591-ec7137399e9a service glance] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:18.539082 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d578813-af60-4663-a591-ec7137399e9a service glance] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:18.539357 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d578813-af60-4663-a591-ec7137399e9a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:18.552689 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d578813-af60-4663-a591-ec7137399e9a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:18.554186 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d578813-af60-4663-a591-ec7137399e9a service glance] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:18.565457 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4452/8989] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:59:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:59:19.671241 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd659aa8-dd91-4217-85a6-06782044a389 None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:19.671739 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd659aa8-dd91-4217-85a6-06782044a389 None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:19.671838 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd659aa8-dd91-4217-85a6-06782044a389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:19.690993 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-fd659aa8-dd91-4217-85a6-06782044a389 None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:59:19.692196 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-fd659aa8-dd91-4217-85a6-06782044a389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:59:19.706346 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4538/8990] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:59:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:59:19.718167 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d578813-af60-4663-a591-ec7137399e9a service glance] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:19.729667 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d578813-af60-4663-a591-ec7137399e9a service glance] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:19.734740 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb239c-aa55-4428-acf5-e800d0841ebf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:19.735335 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb239c-aa55-4428-acf5-e800d0841ebf service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:19.735621 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb239c-aa55-4428-acf5-e800d0841ebf service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:19.735897 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb239c-aa55-4428-acf5-e800d0841ebf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:19.744794 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb239c-aa55-4428-acf5-e800d0841ebf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:19.746412 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb239c-aa55-4428-acf5-e800d0841ebf service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:19.754641 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4453/8991] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:59:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:59:20.893250 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd659aa8-dd91-4217-85a6-06782044a389 None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:20.908380 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd659aa8-dd91-4217-85a6-06782044a389 None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:20.914861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4bbac-ca9e-49ab-a075-0e698e3bd677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:20.915696 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4bbac-ca9e-49ab-a075-0e698e3bd677 service nova] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:20.916228 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4bbac-ca9e-49ab-a075-0e698e3bd677 service nova] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:20.916507 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4bbac-ca9e-49ab-a075-0e698e3bd677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:20.918306 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-4fa4bbac-ca9e-49ab-a075-0e698e3bd677 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:59:20.931686 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4bbac-ca9e-49ab-a075-0e698e3bd677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:20.933647 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4bbac-ca9e-49ab-a075-0e698e3bd677 service nova] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:20.945518 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4539/8992] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:59:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:59:22.997142 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb239c-aa55-4428-acf5-e800d0841ebf service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:23.009317 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb239c-aa55-4428-acf5-e800d0841ebf service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:23.015230 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cd9f1-1075-4b12-8e90-9edc07934fa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a56702329eae48819121dcb58f7e3070', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:23.016055 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cd9f1-1075-4b12-8e90-9edc07934fa2 service nova] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:23.016413 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cd9f1-1075-4b12-8e90-9edc07934fa2 service nova] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:23.016691 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cd9f1-1075-4b12-8e90-9edc07934fa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:23.027784 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08cd9f1-1075-4b12-8e90-9edc07934fa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:23.029963 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08cd9f1-1075-4b12-8e90-9edc07934fa2 service nova] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:23.041327 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4454/8993] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:59:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:59:24.632508 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a9067-e331-4f35-9384-7fbb8e19d7ab None None] REQUEST_METHOD: `POST` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:24.633005 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a9067-e331-4f35-9384-7fbb8e19d7ab None None] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:24.633101 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a9067-e331-4f35-9384-7fbb8e19d7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:24.653331 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.auth.core [None req-7c7a9067-e331-4f35-9384-7fbb8e19d7ab None None] MFA Rules not processed for user `9388372667f8454498b2e4480187b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=99210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 21:59:24.654541 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-7c7a9067-e331-4f35-9384-7fbb8e19d7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:59:24.661747 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4540/8994] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 3 21:59:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 21:59:24.673913 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cd9f1-1075-4b12-8e90-9edc07934fa2 service nova] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:24.685631 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cd9f1-1075-4b12-8e90-9edc07934fa2 service nova] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:24.690602 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cff9dd-652d-4529-bccd-cc44c28e6e2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83865463b2a84f99a136b73dffeeb378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:24.691194 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cff9dd-652d-4529-bccd-cc44c28e6e2c service swift] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:24.691479 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cff9dd-652d-4529-bccd-cc44c28e6e2c service swift] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:24.691754 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cff9dd-652d-4529-bccd-cc44c28e6e2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:24.699716 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cff9dd-652d-4529-bccd-cc44c28e6e2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:24.701326 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cff9dd-652d-4529-bccd-cc44c28e6e2c service swift] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:24.709408 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4455/8995] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 3 21:59:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 21:59:29.869295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a9067-e331-4f35-9384-7fbb8e19d7ab None None] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:29.878873 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a9067-e331-4f35-9384-7fbb8e19d7ab None None] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:29.882480 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd9920-256f-4791-9580-ae26fd962f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0fb67db342b477aaf547543c498434f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:29.883013 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd9920-256f-4791-9580-ae26fd962f6a service neutron] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:29.883295 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd9920-256f-4791-9580-ae26fd962f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:29.883569 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd9920-256f-4791-9580-ae26fd962f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:29.952363 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd9920-256f-4791-9580-ae26fd962f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:29.953845 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd9920-256f-4791-9580-ae26fd962f6a service neutron] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:29.972914 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4541/8996] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:59:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:59:32.796076 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cff9dd-652d-4529-bccd-cc44c28e6e2c service swift] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:32.833648 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cff9dd-652d-4529-bccd-cc44c28e6e2c service swift] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:32.841853 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2443ae-983b-4c76-931c-e6e3308466a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:32.844177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2443ae-983b-4c76-931c-e6e3308466a2 service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:32.844177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2443ae-983b-4c76-931c-e6e3308466a2 service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:32.844177 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2443ae-983b-4c76-931c-e6e3308466a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:32.846115 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-aa2443ae-983b-4c76-931c-e6e3308466a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:59:32.886089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2443ae-983b-4c76-931c-e6e3308466a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:32.887996 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2443ae-983b-4c76-931c-e6e3308466a2 service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:32.903053 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4456/8997] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:59:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:59:36.009015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd9920-256f-4791-9580-ae26fd962f6a service neutron] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:36.020182 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd9920-256f-4791-9580-ae26fd962f6a service neutron] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:36.025468 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24e23d-7ae5-4c89-ba58-de586c063b4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:36.026083 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24e23d-7ae5-4c89-ba58-de586c063b4f service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:36.026367 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24e23d-7ae5-4c89-ba58-de586c063b4f service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:36.026640 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24e23d-7ae5-4c89-ba58-de586c063b4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:36.028421 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-ab24e23d-7ae5-4c89-ba58-de586c063b4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:59:36.040242 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24e23d-7ae5-4c89-ba58-de586c063b4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:36.041870 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24e23d-7ae5-4c89-ba58-de586c063b4f service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:36.052738 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4542/8998] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 21:59:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 21:59:36.094341 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2443ae-983b-4c76-931c-e6e3308466a2 service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:36.105908 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2443ae-983b-4c76-931c-e6e3308466a2 service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:36.110837 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:36.111487 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:36.111776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:36.112060 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] PATH_INFO: `/v3/users/229f2394f4a64bb09c05994e012e2a55` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:36.113227 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] RBAC: Authorizing `identity:delete_user(user_id=229f2394f4a64bb09c05994e012e2a55)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:36.115168 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:36.164867 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '229f2394f4a64bb09c05994e012e2a55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.177758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '229f2394f4a64bb09c05994e012e2a55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.178535 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '229f2394f4a64bb09c05994e012e2a55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.186280 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '229f2394f4a64bb09c05994e012e2a55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.194450 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '229f2394f4a64bb09c05994e012e2a55'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.202192 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4457/8999] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:59:36 2022] DELETE /identity/v3/users/229f2394f4a64bb09c05994e012e2a55 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:59:36.381617 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24e23d-7ae5-4c89-ba58-de586c063b4f service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:36.392252 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24e23d-7ae5-4c89-ba58-de586c063b4f service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:36.443807 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=100 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:59:36.450248 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:36.450913 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:36.451200 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:36.451481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] PATH_INFO: `/v3/projects/6ceb7d1e235e467c847a20cbd4b660b6` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:36.452600 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ceb7d1e235e467c847a20cbd4b660b6)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:36.454549 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:36.503602 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ceb7d1e235e467c847a20cbd4b660b6', 'request_id': 'req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.511501 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ceb7d1e235e467c847a20cbd4b660b6', 'request_id': 'req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.518353 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ceb7d1e235e467c847a20cbd4b660b6', 'request_id': 'req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.542794 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] The token cache is being invalidate because project 6ceb7d1e235e467c847a20cbd4b660b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:59:36.543010 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.543636 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.545181 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4543/9000] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:59:36 2022] DELETE /identity/v3/projects/6ceb7d1e235e467c847a20cbd4b660b6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:59:36.549484 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:36.551107 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:59:36.564710 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a96592-0f65-4913-b524-9476cc7fc55e admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:36.610147 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:59:36.630783 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:36.631423 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:36.631708 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:36.631991 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] PATH_INFO: `/v3/users/d44b98df884c4cf4831654a06335bb96` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:36.633163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44b98df884c4cf4831654a06335bb96)` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:36.635099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:36.682040 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44b98df884c4cf4831654a06335bb96'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.693972 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44b98df884c4cf4831654a06335bb96'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.694650 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44b98df884c4cf4831654a06335bb96'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.701601 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44b98df884c4cf4831654a06335bb96'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.708776 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.notifications [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44b98df884c4cf4831654a06335bb96'} {{(pid=99211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:36.715151 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4458/9001] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 3 21:59:36 2022] DELETE /identity/v3/users/d44b98df884c4cf4831654a06335bb96 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:59:36.890158 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 21:59:36.891664 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 21:59:36.905973 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1a9c6-b6d4-466b-9d1e-f68ff52417f4 admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 21:59:36.963473 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa1ecf6d7f2438dad00007510b3e878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53641d46504c444d9350b7e021f6a4a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 21:59:36.964146 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 21:59:36.964430 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 21:59:36.964710 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] PATH_INFO: `/v3/projects/8bf4b843f6ac45ecba12bcee2a82018d` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 21:59:36.965892 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf4b843f6ac45ecba12bcee2a82018d)` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 21:59:36.967844 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 21:59:37.015181 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf4b843f6ac45ecba12bcee2a82018d', 'request_id': 'req-25783b4b-ad48-46ea-814b-29cba3436a4a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:37.022481 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf4b843f6ac45ecba12bcee2a82018d', 'request_id': 'req-25783b4b-ad48-46ea-814b-29cba3436a4a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:37.028740 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf4b843f6ac45ecba12bcee2a82018d', 'request_id': 'req-25783b4b-ad48-46ea-814b-29cba3436a4a'} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:37.039571 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] The token cache is being invalidate because project 8bf4b843f6ac45ecba12bcee2a82018d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 21:59:37.039861 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:37.040502 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.notifications [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 21:59:37.042049 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4544/9002] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 3 21:59:36 2022] DELETE /identity/v3/projects/8bf4b843f6ac45ecba12bcee2a82018d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 21:59:46.711700 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 21:59:47.036085 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 22:00:50.328395 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 22:00:50.329758 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 22:00:50.400715 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48344492-a51d-4a86-b54a-f90b63427b9a admin admin] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 22:00:50.460056 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc486249-daa0-4d04-afe2-bca5719032fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 22:00:50.460674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc486249-daa0-4d04-afe2-bca5719032fe service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 22:00:50.460983 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc486249-daa0-4d04-afe2-bca5719032fe service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 22:00:50.461260 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc486249-daa0-4d04-afe2-bca5719032fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 22:00:50.462982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.fernet_utils [None req-fc486249-daa0-4d04-afe2-bca5719032fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 22:00:50.474520 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc486249-daa0-4d04-afe2-bca5719032fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 22:00:50.476089 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc486249-daa0-4d04-afe2-bca5719032fe service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 22:00:50.485776 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4459/9003] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 22:00:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 22:01:00.167603 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 22:01:00.169110 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 22:01:00.183940 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25783b4b-ad48-46ea-814b-29cba3436a4a admin admin] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 22:01:00.189022 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf9163-8c70-4e3d-b328-1aed1b48fb41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 22:01:00.189620 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf9163-8c70-4e3d-b328-1aed1b48fb41 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 22:01:00.189912 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf9163-8c70-4e3d-b328-1aed1b48fb41 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 22:01:00.190190 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf9163-8c70-4e3d-b328-1aed1b48fb41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 22:01:00.191840 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.fernet_utils [None req-46cf9163-8c70-4e3d-b328-1aed1b48fb41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 22:01:00.203504 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf9163-8c70-4e3d-b328-1aed1b48fb41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 22:01:00.205144 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf9163-8c70-4e3d-b328-1aed1b48fb41 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 22:01:00.215197 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4545/9004] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 22:01:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 22:01:00.482674 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=58 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 22:01:10.210998 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 22:05:54.326093 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc486249-daa0-4d04-afe2-bca5719032fe service placement] Authenticating user token {{(pid=99211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 22:05:54.336982 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc486249-daa0-4d04-afe2-bca5719032fe service placement] Validating token access rules against request {{(pid=99211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 22:05:54.341946 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7934442-f414-4477-b2b3-58340c550cbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 22:05:54.342542 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934442-f414-4477-b2b3-58340c550cbd service placement] REQUEST_METHOD: `GET` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 22:05:54.342822 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934442-f414-4477-b2b3-58340c550cbd service placement] SCRIPT_NAME: `/identity` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 22:05:54.343099 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934442-f414-4477-b2b3-58340c550cbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 22:05:54.351657 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7934442-f414-4477-b2b3-58340c550cbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 22:05:54.353312 ncdv-master-1 devstack@keystone.service[99211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7934442-f414-4477-b2b3-58340c550cbd service placement] RBAC: Authorization granted {{(pid=99211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 22:05:54.363424 ncdv-master-1 devstack@keystone.service[99211]: [pid: 99211|app: 0|req: 4460/9005] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 22:05:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 22:06:04.359163 ncdv-master-1 devstack@keystone.service[99211]: DEBUG dbcounter [-] [99211] Writing DB stats keystone:SELECT=6 {{(pid=99211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 22:06:05.188955 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf9163-8c70-4e3d-b328-1aed1b48fb41 service placement] Authenticating user token {{(pid=99210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 22:06:05.199928 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf9163-8c70-4e3d-b328-1aed1b48fb41 service placement] Validating token access rules against request {{(pid=99210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 22:06:05.205016 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a97038-1b4f-4e4d-977b-2e66e1d35875 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dfb57e006f344808f7a2ef41b246d5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e040b2fca724c58a2660441a72cd350', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 22:06:05.205622 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a97038-1b4f-4e4d-977b-2e66e1d35875 service placement] REQUEST_METHOD: `GET` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 22:06:05.205911 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a97038-1b4f-4e4d-977b-2e66e1d35875 service placement] SCRIPT_NAME: `/identity` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 22:06:05.206188 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a97038-1b4f-4e4d-977b-2e66e1d35875 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 22:06:05.214437 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a97038-1b4f-4e4d-977b-2e66e1d35875 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 22:06:05.216015 ncdv-master-1 devstack@keystone.service[99210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a97038-1b4f-4e4d-977b-2e66e1d35875 service placement] RBAC: Authorization granted {{(pid=99210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 22:06:05.225635 ncdv-master-1 devstack@keystone.service[99210]: [pid: 99210|app: 0|req: 4546/9006] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 3 22:06:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 22:06:15.221615 ncdv-master-1 devstack@keystone.service[99210]: DEBUG dbcounter [-] [99210] Writing DB stats keystone:SELECT=6 {{(pid=99210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}